2.1 Factors leading to data being at risk

The range of documented factors that may potentially result in data being at risk is portrayed in Table 1.

The information presented in Table 1 reveals that risk factors can be grouped into categories. The identified categories include researcher behaviour, institutional factors, format issues, and random events. Research behaviour that does not adhere to best practices is the category that is linked to the most risk factors. Lack of awareness, lack of metadata, displaced data, data that is saved in one location only, perceived data value and substandard data management practices all fall into this category. Overlap between categories can also be detected. An example is the issue of data deterioration, which can form part of both format issues and researcher behaviour.

2.2 Data rescue obstacles and challenges

Studies that report on data rescue challenges are not commonplace, as published rescue studies tend to report on the success of the project. However, Downs (2015) has listed several problematic issues that research institutes need to consider when contemplating data rescue. It is important to mention the realistic rescue challenge of at-risk data that is in poor condition, incomplete, or partially curated. Moreover, Downs (2015) highlights the feasibility of a rescue project that depends on technical factors, including capabilities, resources and infrastructure. Training or the acquisition of experts is a possible data rescue project requirement. Additional data rescue challenges that need to be resolved prior to the start of a rescue project relate to decisions regarding validity and authenticity; whether the value of the data will justify the costs and whether the trade-off of rescue techniques might compromise the effort. Obstacles such as limited data quality information, incomplete data provenance, and delays in acquiring the required infrastructure or equipment should be considered and alleviated.

Published sources reporting on factors that lead to data being at risk and the challenges experienced by those considering the rescue of at-risk data sketch a bleak picture in this regard.

4.1 Research participants

This case study collected information from the institute’s RGLs. RGLs manage the institute’s various research groups. They are experts within their research disciplines, and have knowledge of the data held by the research group, as well as the data activities performed by the group members.

The study made use of the two RGL sample groups:

• Sample A consisted of 49 RGLs and comprised the entire RGL population of the institute. Participants received invitations to complete the study’s web-based questionnaire. Each of the institute’s research groups is managed by a single RGL, and an RGL may manage more than one research group. The rationale behind the selection of Sample A was to have a group of experienced researchers who would:
  • provide a broad overview of the presence of data at risk in the institute;
  • provide a broad overview of the presence of data rescue activities in the institute; and
  • enable the selection of a sub-sample (Sample B), based on their responses.
• Sample B consisted of RGLs who had revealed in the web-based questionnaire that they either had at-risk data in their possession, or they had performed data rescue activities in the past. These respondents were invited to participate in one-on-one interviews. The invited sample comprised 18 RGLs. Of the 18 invited RGLs, eight agreed to be interviewed. It was unfortunate that 10 of the 18 invited RGLs, who had indicated in the questionnaire responses that they had at-risk data, declined to be interviewed.

4.2 Web-based questionnaire

A first data collection stage involved the collection of information that would provide a good overall glimpse of the current state of data at risk and data rescue at the research institute. A web-based questionnaire presents several advantages, such as its capacity to reach remote participants (Adams & Lawrence 2015; Leedy & Ormrod 2005), its ability to reach many participants in a brief period (Menon & Muraleedharan 2020) and its capacity to export questionnaire data to different formats, depending on the requirements or preferences of the interviewer (eSurv 2019). The researcher therefore decided to use this data collection method as a survey tool.

Regmi et al. (2016) list six vital factors to consider when designing an online or web-based questionnaire. These aspects influenced the design of the online survey:

• A progress indicator was used. It was possible to page back and forth between questions, and ample space was available for long responses.
• Participant selection was done very carefully (see Section 4.1), and sampling was purposive.
• The possibility of multiple responses by the same respondent was avoided as the online questionnaire was not anonymous. Identifying details were required to select Sample B.
• Data management was considered. As a web-based tool, eSurv could export questionnaire data to PDF, CSV, and Microsoft Excel formats. In addition, data were private, as the investigator’s log-in details were required to access the data.
• The online questionnaire adhered to ethical requirements. All study components (including the questions asked, the wording of the cover letter, the wording of the consent form, and the use of data) were examined by the relevant Research Ethics Committees. Ethical clearance was obtained. An online indication of informed consent formed part of the web-based questionnaire.
• The web-based questionnaire was piloted before it was launched, and a link distributed to the RGLs.

After taking the above advantages and requirements into account, a short web-based questionnaire comprising eight questions on at-risk data and data rescue was distributed to the institute’s 49 RGLs, also referred to as Sample A. The questions pertained to the existence of each group’s at-risk data, the format/s of the data, a brief description of the data and the location of the at-risk data. Regarding data rescue, RGLs were asked whether they had previously performed any data rescue activities, whether the group had data rescue documentation or procedures in place, whether data documentation could be shared with the investigator, and whether participants were able to direct the study investigator to institutional parties who had performed data rescue activities. The concepts of ‘at-risk data’ and ‘data rescue’ were explained in the questionnaire. A copy of the web-based questionnaire’s wording is attached as Appendix 1.

4.3 One-on-one interviews

Members of Sample B were invited to participate in one-on-one interviews. This sample comprised RGLs who had indicated in the web-based questionnaire that they either had at-risk data, or had performed data rescue activities in the past. The objective of the interviews was to obtain more detailed information pertaining to each research group’s at-risk data. The interviewer also collected details about each research group’s data rescue experiences. The main advantage of personal interviews is that the interviewer is able to clarify responses, probe when additional information is required, and even ask additional questions that do not form part of the interview schedule. These factors can improve the accuracy of responses (Adams & Lawrence 2015). Additional benefits include the high response rates (Akba Yrak 2000) and permitting long and complex questionnaires (Neuman 2014). Interviews conducted in this study were in-depth in nature, and made use of probing questions, as well as follow-up questions in the case of vague or ambiguous replies, and delved into data rescue-related topics that did not form part of the interview schedule.

Virtual one-on-one interviews took place with members of Sample B as this activity took place during the South African COVID-19 lockdown period. Virtual interviews offer the additional benefits of allowing for the collection of data over wide geographical areas (Jowett 2020), and they are often less expensive and more time-efficient than in-person interviews (Krouwel, Jolly & Greenfield 2019). Participants often feel more comfortable disclosing sensitive information by not being face to face with the interviewer, and by being able to participate in the interview from a familiar space such as their own home (Hanna 2012; Sipes, Roberts & Mulan 2019).

Interviews were semi-structured and were conducted by the principal investigator of the study (also the first author of the article). Verbal informed consent was given at the start of each interview. Interviews were recorded and subsequently transcribed by the interviewer. A copy of the outline of the interview schedule is attached as Appendix 2.

Transcribed interviews were subjected to thematic analysis, thereby enabling the identification of topics, ideas and patterns that came up repeatedly, such as the performance of data rescue activities, data risk factors and data rescue obstacles.

5.1 Responses

As all of the institute’s RGLs were invited to complete the questionnaire, it was important to examine both the conveyed information linked to data at risk and data rescue activities, and also the response rates, feedback pertaining to RGL group management, and the number of research groups involved in responses.

The responses to the web-based questionnaire revealed that:

• 23 of the 49 RGLs (51%) submitted completed or partially completed web-based questionnaires;
• two of the above-mentioned 23 RGLs managed more than one research group;
• research groups were never managed by more than one RGL;
• findings provided information on 25 research groups; and
• 18 RGLs met the interview invitation requirement by indicating that their group had at-risk data.

Eight of the 18 invited RGLs, including one proxy, agreed to be interviewed.

5.2 Factors putting data at risk

In-depth interviews with eight RGLs resulted in the identification of 34 factors that have put data at risk (Patterton 2023b; Patterton 2023c). Each factor, together with its frequency, is shown in Figure 1. Factors mentioned by respondents are applicable to both analogue and digital data.

5.3 Data rescue activities

The results of the question ‘Has your research group ever performed data rescue activities?’ are portrayed in Figure 2.

Most research groups had never performed any data rescue activities, with only a quarter of the respondents stating that some data rescue activities had taken place. The number of respondents that indicated that they were unsure whether data rescue had been done is troubling. However, it is not clear from the responses whether some of the RGLs were newly appointed managers (and unfamiliar with the group’s activity history), or whether data management activities were not shared with all the members of the group, and with the managers.

These results were unexpected. Even though this researcher was not surprised by the dominance of data at risk in the institute, the overwhelming absence of data rescue projects and activities was not an anticipated outcome of the web survey. The expectation was that many of the research groups would somehow have attempted or organised data rescue initiatives, ranging between rudimentary to structured data rescue activities. The unanticipated nature of this finding is further exacerbated by the fact that most of the subsequently interviewed parties indicated that their at-risk data was of value and needed to be shared.

The interviews that followed the web-based questionnaires enabled deeper probing into data rescue activities that had been performed. Table 2 contains a summary of the nature and number of data rescue undertakings by the respective research groups.

As indicated, the two most common data rescue activities involved uploading data to a repository, and adding metadata. Other data rescue activities were mentioned by fewer than three of the eight respondents.

A single instance of a complete data rescue project was discovered. The rescue project involved early format digital audio data, with the main rescue steps resembling the typical rescue steps described in published data rescue literature. The project also involved the outsourcing of many of the rescue steps to an external party, with the resultant product being data in a modern electronic format, accompanied by metadata that was made accessible to the public via a dedicated discipline repository.

Data rescue was a rare activity at the institute and many of the interviewed RGLs had experienced data loss. This either entailed the loss of data they had collected themselves, or the loss of data belonging to the research group. Activities that were not mentioned, but were expected to have been shared with the investigator, included the identification of valuable data, the identification of data at risk, and approaching the institute’s research library for data rescue guidance. In addition, despite several RGLs having historic analogue data in their groups, reported data rescue activities only involved early and modern digital data.

5.4 Data rescue obstacles and challenges

Respondents identified 26 data rescue obstacles. The RGLs each mentioned between two and eleven obstacles (Patterton 2023b). The most common data rescue challenges were found to be cost, data quantity/volume, time required to rescue data, and the fact that a data rescue venture would require skills, insight, and experience.

Figure 3 shows the data rescue obstacles mentioned by the RGLs when they were asked to list the obstacles they experienced when rescuing data or considering the rescue of data, or the obstacles they foresaw in a hypothetical data rescue scenario. The challenges and obstacles mentioned by the respondents are applicable to both analogue and digital data.

6.1 Factors that put data at risk

In-depth one-on-one interviews with eight RGLs resulted in the identification of 34 risk factors thought to lead to data being at risk. This number of risk factors is troublesome and greater than anticipated. Risk factors reflect on aspects involving human behaviour, information technology (IT) matters and institutional policy. With many of the risk factors sharing similarities in nature and root cause, it was possible to place the listed factors into broader categories. In addition, several of the listed risk factors fit into more than one of the broader categories. A great deal of overlap was detected. These broader divisions, containing instances of the respondents’ mentioned data risk factors, are listed and expanded on below.

6.2 Data rescue obstacles and challenges

The study identified 26 data rescue challenges or obstacles. The myriad stated challenges provides an insightful glimpse into the problems envisaged by researchers when asked to list challenges experienced during past rescue efforts or to envisage challenges that would be encountered during a hypothetical data rescue project.

The number and range of listed challenges are worrying, but not unexpected. Previous research into the institute’s data management practices revealed low adherence to data management best practices, an absence of vital data management platforms or tools, and data management procedures not yet in place (Patterton 2014; Patterton 2016; Patterton, Bothma & Van Deventer 2018). In addition, many of the stated data rescue challenges show similarity with factors that lead to data being at risk. Examples are the high incidence of IT-related factors, lack of processes, and problems with the current and available institutional systems. While the findings emanate from data collected at a single research institute, it is suspected that many of the obstacles are universal in nature and will be prevalent and cause for concern at other research institutes in a similar stage of data management maturity.

The 26 data rescue obstacles and challenges listed by respondents can be clustered into four broad categories, with several obstacles fitting into more than one category.

8.1 Address and promote awareness of at-risk data

Lack of insight into the value of older data and the correct way of handling at-risk data are likely to result in the current volumes of at-risk data increasing daily. Failure to address this scenario may lead to a state where only a small percentage of at-risk data rescue is deemed feasible. It is vital for researchers to be aware of the following:

• The potential value of historic data
• The correct ways of handling older data
• The important role of metadata and data documentation
• The enormous costs associated with data rescue
• Parties to contact should at-risk data be located and identified

Awareness creation can take the form of an intranet article, posts on the blog of the institute’s Library and Information Services (LIS) division, visits of the LIS division’s representatives to research groups, and a possible institute-wide webinar.

Promotional activities that are focused on at-risk data will go hand in hand with an understanding of data rescue (see Section 8.2).

8.2 Promote awareness of data rescue

In-depth interviews with RGLs have shown that data rescue activities are not commonly performed. Knowledge of the concept of data rescue is also limited. It is doubtful whether the skills set required for the rescue of data is currently present within the research sector to which the institute belongs, as the only example of a complete data rescue project found comprised the outsourcing of the project to an external party.

To deal with these gaps in knowledge, data rescue should be promoted as both a concept and an activity, with the following concepts forming part of knowledge-sharing sessions:

• Conveyance of the benefits of data rescue
• Explanation of the typical data rescue workflow
• Demonstration of a data rescue workflow model
• Indication of the anticipated involvement of various institutional sectors

As will be the case for the advancement of institutional knowledge pertaining to at-risk data, data rescue awareness at research institutes can make use of the institutional LIS blog or website, an intraweb article, personal visits by the designated LIS rescue trainer to the research groups and an institute-wide data rescue webinar. It is anticipated that the two concepts—at-risk data and data rescue—will form part of the same training session, webinar, or personal visit.

It is important that awareness sessions also explain that not all data can be rescued, and that not all data should be rescued. The importance of data assessment to determine the value of data, and of evaluating data rescue resources should form part of the preliminary stages of each prospective research project.

8.3 Launch a data rescue pilot project

Following the activities described in Section 8.1 and Section 8.2, the launch of a data rescue pilot project is recommended. This pilot project will be the first of its kind at the selected institute and entails testing the rescue activities and the required outputs, as well as the feasibility and efficacy of the involvement of different institutional sections (e.g., researchers, members of the LIS division and the IT department). Even though the data rescue project will adhere to the data rescue workflow model created by the institutional data librarian (Patterton 2023a), the pilot project experience is anticipated to result in several changes and adaptations to the recommended model.

The main recommended stages of a single generic data rescue project, with its crucial activities, are indicated in Table 3. While this table contains the steps for a rescue project involving paper-based data or data in an early digital format, several of the rescue steps (e.g., data description or data sharing) can also be applied to digital data rescue.

Elaborative details regarding the activities linked to each stage form part of the data rescue workflow model created by the institute’s data librarian (Patterton 2023a).

8.4 Increase data management awareness and compliance

Findings have shown that the presence of sub-standard data management practices is both a risk factor for data and an obstacle during data rescue activities. Examples of non-ideal practices and their linked detrimental effects include the mention of a certain ‘research culture,’ researchers adhering to their ‘own processes,’ the absence of data management planning and the lack of metadata linked to datasets. It is interesting to note that while funder mandates drive data management in many countries (Digital Curation Centre 2023), this had not yet manifested at the research institute in question. To deal with these issues, the recommendation is made that all researchers be aware of good data management practices, with certain practices being subjected to a monitoring and compliance process.

The approval of an institutional data management procedure, coupled with institute-wide data management training, had not yet commenced during this study’s data collection stages, but followed soon afterwards. The attendance of the mandatory sessions was relatively poor. It is therefore recommended that future training sessions be complemented by online self-training tools, personal visits of the data librarian to research groups, and a slow roll-out of data management compliance monitoring.

In addition to data management adhering to best practices, researchers should also be familiar with an institute’s data management policy and data management procedure. Although following best practices regarding data management will not undo the threatened state of an institute’s historic data, it can ensure that data currently being generated, and to be generated in the future, do not succumb to the same predicament.

8.5 Broaden the scope of the risk management plan

A risk management plan currently forms part of the project registration requirements at the selected institute. It is suggested that potential risk concerns pertaining to data be added to the risk document templates used by this institute and elsewhere, and that potential threats to a project’s data accessibility, understanding and usability be anticipated and documented.

8.6 Involve the LIS sector in data rescue

The institute’s LIS division currently performs many tasks, similar to the rescue activities mentioned in Sections 8.2. and 8.3. In addition to familiarity with data rescue activities, the study findings have indicated that researchers experience data rescue constraints with regard to time, costs, human resources, and skills. Moreover, the RGLs also referred to the dangers of researchers having their own research practices, researchers’ data management activities displaying sub-standard qualities, and research groups not having a strategy when it comes to the rescue of at-risk data. To overcome many of these obstacles, it is recommended that an institute’s LIS section be involved with data rescue projects. The roles and responsibilities listed by Patterton (2023a) can be used for guidance. Based on the suggested roles, an institute’s LIS section may potentially be involved in all data rescue stages, with the section’s participation including data rescue supervisory activities and contributing towards data rescue training material.

The involvement of an institute’s LIS section is important. Suggestions for the section’s participation in data rescue include the following:

• Training of the LIS sector: For the LIS sector (both at the selected institute and wider) to be optimally involved with data rescue, it is crucial to make provision for on-the-job training, to attend data rescue and data curation workshops, to engage in self-training by means of published data rescue manuals and guidelines, and for active LIS professionals to share their data rescue knowledge, skills, and experience with the LIS community. Examples of published manuals or guidelines that are available include the paper-based climate data rescue manual of the World Meteorological Organization (WMO) (2016) and the elaborate digital marine species data rescue manual of Kennedy (2017).
• Institute-wide view: Researchers focus on the project, and perhaps on its contribution to a discipline. LIS specialists support and enable a variety of these research projects across the organisation. This makes it possible to share best practice and knowledge, and assists with the identification of assets across the organisation. LIS specialists are also regularly in contact with a variety of staff members. They have someone to talk to about problems and can express concerns regarding neglected collections. These collections could be evaluated for their value in rescuing data.
• LIS metadata expertise: The survey results reported by Thompson, Davenport Robertson, and Greenberg (2014) indicate that there is a general institutional expectation that a librarian, archivist, or information professional should be responsible for the generation of metadata. Their findings also indicate that LIS professionals are indeed the parties that add metadata to data in the overwhelming majority of instances. The current study’s RGL feedback and scholarly publications emphasise the significance of metadata. Coupling this with the metadata expertise and experience of LIS professionals indicates the importance of including the LIS sector in data rescue activities, especially during the stage concerned with metadata creation.
• LIS curriculum change: In North America, many universities have developed graduate programmes to prepare information professionals for data curation (Keralis 2021). Including data rescue and data curation modules in the LIS curriculum is an important step in equipping the LIS workforce. These curriculum inclusions, ideally occurring at postgraduate level where students already have some degree of data familiarity, will contribute to a better-equipped LIS workforce and LIS professionals who are able to participate in data rescue upon entering the workforce.
• Sharing of LIS data rescue expertise: Adding to the point about LIS data rescue training is the recommendation for sharing data rescue expertise. LIS workshops, conferences, and symposia, particularly those involving academics, researchers, and special libraries, should feature sessions on at-risk data and data rescue. Handover activities, such as the imparting of data rescue knowledge and skills preceding retirement and resignation, is another form of expertise sharing.

The involvement of library and information services in data rescue can involve all members of the sector: undergraduate technicians or assistants, semi-professional staff members, interns, research library professionals, and high-level managerial staff. Additionally, LIS professionals with data management experience can contribute to the areas of training, the creation of guidelines, the adaptation of the model, data rescue project management, and data management plan training and quality control.

Ideally, data rescue should become a dedicated LIS responsibility at research institutes, with the research sector associating assistance regarding at-risk data and data rescue with the institute’s LIS sector.

8.7 Consider collaborative research projects

RGLs have commented on the fact that data rescue can be costly, time-consuming, labour-intensive, and an activity requiring specialised skills. While a previous recommendation (see Section 8.6) has referred to the beneficial involvement of the LIS sector, it is also suggested that data rescue projects consider collaborating with additional partners, including retired research experts, citizen scientists, and postgraduate students. Moreover, data rescue partnerships with entities, including science councils and departments in tertiary institutes, are additional collaboration opportunities.

Data rescue projects ideally suited to collaborative ventures are those where the envisaged project is associated with challenges regarding rescue funding, the available human resources, the available rescue skills, available insight into the research discipline, and equipment to read the data.

8.8 Investigate dedicated institutional data repository options

Institutes without a dedicated institutional data repository are likely to find themselves in a position where research data are stored in a range of locations, on different devices, and not necessarily accompanied by metadata or data documentation. The recommendation is therefore made that these research institutes investigate the possibility of acquiring a dedicated institutional data repository to be used for the preservation and sharing of institutional research data by default.

Considerations linked to this suggestion involve the data repository meeting an institute’s unique requirements, such as varying data access levels, the ability to accommodate a range of data formats, a handle assigned to the data for sharing and citation purposes, or integration with established institutional systems.

While older at-risk data are unlikely to end up in a data repository, unless it forms part of a rescue project, the future access and use of current and future data will undoubtedly be enhanced by a dedicated data repository. The availability of a dedicated institutional data repository may coincide with beneficial outcomes, including knowing where data are stored, the creation of metadata and data documentation, the assurance of data backups, easier handover, limiting issues with the current DMS, easier compliance with the data management procedure, and alleviating the use of sub-standard and non-secure storage options.

8.9 Ensure handover considers relevant data

The research findings have shown that RGLs are concerned about data not being accessible to members of the research group, poor data management continuation, and group members being unaware that data are at risk. To deal with these factors, it is recommended that institutional handover activities consider the potentially detrimental effects any transitions will have on any valuable research data and data knowledge. Institutional handover activities and change management strategies also need to ensure that there is a record of the affected data, that several parties are aware of the data, that the data can be accessed, and that the data can be understood and used.

Changes in strategic outlook and the restructuring of research groups, coupled with resignations and retirements, form part of the research work environment. When not managed well, with detrimental outcomes not foreseen, these events can affect the future access and usability of institutional data.