Barriers to Full Participation in the Open Science Life Cycle among Early Career Researchers

Pre-Registration

Pre-registration is a practice in which scientists submit their study design to a free public registration platform (e.g. Open Science Framework, Registered Reports) in advance of conducting the research itself. This practice can help researchers to be intentional about and receive feedback on their research plans. It can improve transparency, reproducibility, and rigor and can reduce publication bias (Gonzales et. al. 2015; Kupferschmidt 2018). From 2012–2018, the number of journals that offer pre-registration, as well as the number of pre-registered studies on Open Science Framework, have grown exponentially (Kupferschmidt 2018). However, the practice is still largely restricted to certain fields (e.g. pre-clinical research; Nosek et al. 2018).

Barriers to pre-registration include the time commitment, an inability to make changes to the procedure once committed to pre-registration, and uncertainties about how to register projects involving pre-existing data (Kupferschmidt et. al. 2018; Nosek et al. 2018). For example, researchers may have to collect more data than they want because of a pre-registered commitment to a sample size, even if it seems clear that the results will be negative.

Additionally, effective pre-registration requires practice and enough experience to anticipate workflows ahead of time, which may be more difficult for ECRs who lack decades of experience in their field. Because pre-registration takes time and heightens scientific vulnerability, it may not be appealing to ECRs unless there are institutional incentives. Perhaps most importantly, there is a lack of awareness and training in pre-registration among ECRs, which may lead them to believe that this practice is not relevant to their scientific approach.

Our Perspective: Our survey respondents, who have a demonstrated interest in OS, showed low familiarity with pre-registration. Open-ended responses suggested that, in large part, the low frequency of pre-registration among the survey respondents stemmed from lack of knowledge. Comments like ‘I have never heard of it before’ (or similar sentiments) were very common in response to discussions on and questions about pre-registration. ECRs also voiced concerns about the time commitment and about being ‘scooped’. One respondent noted that, previously, the ideas they shared during pre-registration had been copied.

Open Processes

Open processes begin at the planning stage of research, with organization of documentation and research processes in an appropriate central repository, and continue throughout the life of a project. Since ‘open notebook science’, a central component of open processes, was coined (Bradley 2006), several online platforms have been established for openly tracking research projects (e.g. Openlabnotebooks, Github, Wellcome Open Research publishing platform/Open Research Fund, Zenodo).

Open processes represent a major departure from current practices and hold a mix of promises and risks. Advantages include accessibility of information prior to peer-reviewed publication, allowing for more efficient use of resources and knowledge, and the ability to reproduce detailed protocols (Powell 2016; Wallach, Boyack, & Loannidis 2018; Schapira & Harding 2019). Negative data, which may not be published beyond open notebooks, can provide important scientific insight (Carroll et al. 2017; Mlinarić, Horvat, & Smolčić et al. 2017). However, developing fully reproducible workflows presents a major challenge and it is still unclear that open notebooks or other documented processes will be read and used (Schapira & Harding 2019). Furthermore, misleading or non-rigorous science can still find its way into open platforms, so open comments are a necessity (e.g. healthy discussions at OpenSource Malaria) (Williamson et al. 2016). Other obstacles to the uptake of open processes include a fear of being ‘scooped’ (Grubb & Easterbrook 2011), disagreements among collaborators or other constraints on sharing data and ideas (Bourne et al. 2013), time constraints, and language barriers for non-native English speakers. These obstacles are likely to be exacerbated for ECRs.

Our Perspective: Similar to pre-registration, open processes were among the least well-known and least frequently implemented by the ECRs that we surveyed. This lack of knowledge about open processes was the most-frequently mentioned barrier to their use. One of the respondents stated that, ‘I know what open notebooks are but not sure how best to integrate them into my lab activities…some of the projects I have are industry-facing with…a different electronic reporting process…’ Another respondent stated that, ‘Finding the time to learn the tools necessary for this has been the major barrier for me.’ In addition to a lack of knowledge, respondents mentioned ethics, time, lack of incentives, irrelevance to the field, fear of being ‘scooped’, and protection of shared data as barriers to using open processes.

Open Hardware and Infrastructure

Open hardware is ‘hardware whose design is made publicly available so that anyone can study, modify, distribute, make, and sell the design or hardware based on that design’ (OSHWA 2016). It is based on the assumption that publishing a ‘design’ realizes the four freedoms of the open source concept; in the context of tangible products, the freedom to study, modify, make, and distribute. At the moment, researchers often use proprietary instrumentation or hardware in their research that is not freely accessible, reusable, or adaptable. While the open source hardware movement is a young phenomenon overall, it is increasingly growing beyond the domain of electronics and DIY to that of non-electronic and complex open source hardware products (Bonvoisin et al. 2017). As the inaccessibility to ventilators during the COVID crisis has highlighted (Pearce 2020), there is an ethical imperative that research infrastructure is accessible to researchers worldwide—be it a specific sensor, state-of-the-art biomedical laboratories, or particle colliders.

There is a push by the scientific community and policy makers to move towards Open Hardware (e.g. GOSH 2018; UNESCO 2021). Open Hardware should therefore also be seen in the context of open infrastructures, i.e. the sharing or giving access to the existing material culture and infrastructure of science. Even trends such as migration of ECRs need to be viewed through the lens of access to research infrastructure and the need for equal access to research infrastructure around the world (POST 2008). However, challenges remain in moving beyond the hobbyist level towards research-grade devices and outputs (Hansen & Howard 2013); this movement is being facilitated by collaborative approaches in the open hardware community (Maher, Paulini, & Murty 2011). The primary limitation is that effective development of novel, open source hardware requires a specific set of skills, tools, and infrastructure (Raasch, Herstatt, & Balka 2009).

Our Perspective: Our survey and discussions indicate that familiarity with and use of open hardware is limited among ECRs, more so than any other OS practice that we categorized under Data Collection. This lack of awareness is detrimental because ECRs specifically face problems with accessing relevant hardware. Among all of the OS practices included in the survey, respondents were most likely to select ‘unsure’ when asked if they used this OS practice in their teaching or research. One respondent stated ‘I still feel very uncertain of the meaning of this or how it might pertain to the type of research that I do.’ This sentiment was echoed by several respondents.

Open Software

To be considered open software, source code must be freely available with a license (or terms and conditions) that allows for free dissemination and adaptation. Open source software has been highlighted as ‘the forgotten pillar of open science implementation’ (Vermeir et al. 2018) and one of the top needs for scientific research (UNESCO 2020). There are risks, however, of commercial use of open-source materials without compensation of creators (UNESCO 2020).

A study by the GYA’s Global Access to Research Software Working Group showed that researchers in Bangladesh, Ghana, and Nigeria have only limited knowledge about and experience with open research software, despite the clear benefits it may yield to their research (Vermeir et al. 2018). There also seems to be little interaction between the open research software communities in high-income and low-income countries. If these results are generalizable, the lack of active promotion and training may mean that Open Research Software remains inaccessible to most researchers in low-income countries, where it is most needed.

Our Perspective: Most of our survey respondents reported at least some familiarity with open software and many reported using it in their research and/or teaching. They did, however, report some obstacles in using open software. Reported obstacles included a lack of skills and the time investment needed to build these skills, confidentiality aspects of the research, and restrictive software policies from employers. One respondent stated that ‘[Organization Name] is strict with its software policies. Employees can do [limited] things with their computers.’

Open Data

Open Data provides opportunities for collaboration and encourages further analysis and reuse of data that is often expensive and time-consuming to collect. It is also key to ensuring scientific reproducibility and transparency (Penders & Janssens 2018). The core set of principles that optimize reusability are the FAIR Data Principles: data or any digital object should be Findable, Accessible, Interoperable, and Reusable (Wilkinson et al. 2016). From the perspective of ECRs, Open Data can provide opportunities for research using pre-existing data even if they have limited funding and are unable to generate data themselves.

However, for all its promise, the Open Data movement has met with a number of challenges. Some of the challenges are tied to the Digital Divide—middle to low income countries seem to have unique challenges related to ICT infrastructure and internet connectivity, which has resulted in dis-incentivetizing data sharing (Bezuidenhout & Chakauya 2018). Other barriers are privacy concerns related to the misuse of shared data (Greenbaum et al. 2011). Concerns about Open Data are particularly acute in the Global South and to indigenous communities, including fear of ‘parachute science’ (data from the Global South being used by scientists into the Global North without benefit to Global South scientists; e.g. Serwadda et al. 2018), the right of indigenous communities to data ownership (‘Indigenous Data Sovereignty’; e.g. Kukutai & Taylor 2016), and the illegal extraction of resources (e.g. wildlife poaching based on publicly available animal tracking data; Tulloch et al. 2018).

Our Perspective: The ECRs who responded to our survey were, in general, familiar with Open Data and approximately two-thirds had used Open Data in their teaching or research. When asked about obstacles, they mentioned ethics and confidentiality issues, institutional policies that discourage publishing Open Data, and the practice being generally uncommon in some disciplines (e.g., Humanities/Social Sciences). Respondents were worried about the quality of Open Data (that they were not ‘well-curated’) and about others using these data without proper attribution or without detailed knowledge of the dataset. For example, one respondent shared their colleagues’ concerns about publishing data: ‘1) being robbed of the hard work they had done collecting those data, 2) people not understanding their data enough to accurately use it.’

Open Publishing

Open access to the scholarly literature (usually ‘open access’, OA) means that original research results are made freely accessible on the web—free of most embargoes, copyright, and licensing restrictions. While the focus has mainly been on open access to journal articles, OA principles also apply to books and other research outputs. Open access to journal articles has been classified in green, gold and diamond OA (amongst others), but such codes have lost much their utility in recent years with the expansion of new and sometimes complex publishing, licensing and financing models for OA such as transformative agreements, rights-retention strategies, collaborative and sharing networks, and OA platforms.

While OA started as a grassroots movement, funders and policymakers are now playing a significant role in the expansion of open access by mandating policies and supporting OS through infrastructures such as repositories and publishing platforms or by paying for author publication charges (Schiltz 2018; UNESCO 2021). Major progress has been made in OA in recent years, although in 2017, around 75% of journal articles were still paywalled. OA has been shown to positively impact citation counts, benefit innovation, reduce costs in the publication system, and advance citizen science; it may also provide new opportunities for low- and middle- income country scholars and ECRs (Tennant et al. 2016; Piwowar et al. 2018; cOAlition S 2020).

Our Perspective: OA is the best known and most adopted facet of OS among the ECRs we surveyed. Nevertheless, information is distributed unevenly, and two well-qualified ECRs from the Global South were still unsure about what OA involves. The main concern among ECRs is the perceived (high) cost associated with OA publishing, indicating that OA is generally interpreted as a form of ‘gold’ or author-pays OA. The second major concern is the fact that there are usually no incentives to publish OA; current research assessment practices often favor traditional outlets. One survey respondent wrote: ‘I try to do it more and more, though the prestige hierarchy in my field is still going in a completely different direction. There are rather few (pure) open access journals to choose from.’ The ECR continued by referring to the transformative agreements offered to researchers but added: ‘though I’m not sure that’s a model for the future, because only rich countries can do this’, hinting at important global inequalities in opportunities to publish OA. Interestingly, none of our respondents addressed making available a version of the manuscript (pre-print, accepted manuscripts, the final published version or the ‘Version of Record’) on a website or repository, which indicates that the ‘author pays’ model dominates the perception of OA publishing among ECRs.

Open Peer-Review

Open Peer-Review (OPR) mainly refers to ‘open identities’ of authors and reviewers (Ford 2013), but can also refer to ‘open reports’ (making peer-review reports publicly available), ‘open participation’ (crowdsourced peer-review) and other open review practices (Ross-Hellauer 2017). Peer-review is seen as the gold standard for scientific quality but has also been criticized, which has led to the statement that peer review is ‘the worst way to judge research, except for all the others’ (Carroll 2018). A few forms of OPR have been proposed to remedy some of these problems (e.g. Tennant et al. 2017). Despite its promise, OPR itself faces challenges such as difficulties finding reviewers (van Rooyen 1999) and concerns about re-introducing bias in favor of senior researchers. There are unique obstacles for ECRs to adopt OPR, such as the lack of journals that use this practice and research assessment incentivizing publication in traditional journals.

Our Perspective: Open Review had the second highest average knowledge score among the OA practices included in our survey. However, though respondents were generally familiar with this practice, only approximately 50% of respondents reported participating in OPR. A specific concern voiced among ECRs was that OPR exposes them too much as author and reviewer. As authors, they fear that harsh critique in print may affect their career prospects. As reviewers, ECRs are reluctant to sign reports and, as a result, may evaluate manuscripts more positively with OPR than they would otherwise. One ECR in our survey writes: ‘As a young researcher, I have been very hesitant to publish my name with reviews. I feel that there will be backlash if I do not positively review something’.

Gender and cultural factors also play a key role in the dynamics of peer review and peer criticism, and this will be reinforced in OPR. A young researcher from Sri Lanka pointed out that, culturally, some may take the reviews personally. OPR may suit ‘Western values’ better than regions where ‘saving face’ is an important cultural value. These perspectives highlight the fact that, to increase adoption of OPR, research culture as a whole will need to become more transparent and collaborative.

Citizen Science

Citizen science (CS) has three interrelated goals: creating knowledge, sharing scientific skills and knowledge with the public, and promoting civic engagement in science (Turrini et al. 2018). Because of the wealth and diversity of data available, CS may provide ECRs with the opportunity to publish high-impact research without the need for a long-established research program. Involvement in CS also provides ECRs with the opportunity to interact with community members and develop research programs of relevance to these communities. Although the number of CS projects has increased exponentially since the late 1980s (annual rate of increase of 10%; Pocock et al. 2017), CS is still relatively uncommon as a primary data source (Burgess et al. 2016) and has not been used to its fullest potential towards tracking global sustainability goals (Lepenies & Zakari 2021).

Barriers to wider implementation of CS projects and use of the resulting data include funding limitation and staff resources, time limitation, lack of awareness among scientists, and mistrust of data collected by non-experts (Burgess et al. 2017; Turrini et al. 2018). ECRs may be less aware of relevant CS projects due to smaller scientific networks and may lack the community ties, training, and time needed to work with citizen scientists. ECRs may also be more hesitant than established researchers to publish using non-traditional data sources, particularly since CS data can be ‘messy’. Furthermore, given the dynamic nature of collaborative projects with citizens, CS projects are difficult to plan and require significant project management skills and capacities.

Our Perspectives: Citizen science was one of the less well-known and used OA practices among our survey respondents. An ECR who does work with citizens to collect data voiced challenges similar to those listed above ‘Although I use it/try to use it – there are some obstacles – mainly lack of knowledge of the participants. It takes some time convincing them.’ Others mentioned that citizen science was not relevant to their research approach but that they interfaced with the public in other ways, that they ‘give lots of public talks and participate in public discussions’.

Open Educational Resources

The term ‘Open Educational Resources’ (OER), first coined during a UNESCO forum in 2002, is used to describe any educational materials that are fully open. Being open means that educators and learners must be able to retain, reuse, revise, remix and redistribute these materials (Al Abri & Dabbagh 2018). OER range from open software, through open textbooks, to fully open classes (e.g. Massive Open Online Courses or MOOCs). Although faculty and students who use OER generally report that these resources meet their pedagogical needs while reducing financial barriers, adoption of these materials has been slow (e.g. 42% of faculty in the United States are not aware of OER). Rates of use are, however, increasing in the United States (e.g. from 5% to 14% of US faculty between 2015–2016 and 2018–2019) (Seaman and Seaman 2020).

Frequently reported barriers to OER adoption include a lack of awareness of available OER and options for finding reputable OER, a lack of training in OER development and open licensing, issues or perceived issues with OER quality, and the resources needed to develop, maintain, and adopt OER (Hassall & Lewis 2017; Al Abri & Dabbagh 2018; Koseoglu & Bozkurt 2018; Luo et al. 2020). These barriers may be particularly acute for ECRs, whose publishing records and teaching evaluations are under close scrutiny. Time constraints, which are one of the largest barriers to OER adoption, are best addressed through institutional support, including financial incentives and OER training (Luo et al. 2020; McGowan 2020). In some countries, there may be language barriers or technological barriers to adopting OER. Additionally, most OERs come from a small number of well-resourced countries, which can lead to the dominance of one cultural perspective and resources that do not address local educational needs.

Our Perspective: Most, though not all, of our survey respondents were familiar with OER and approximately two-thirds had used it in their teaching. Among our survey respondents, some mentioned informal use of OER (‘This works mostly informally in my field, with people sharing syllabuses etc.’). Other respondents echoed the barriers above, with one respondent noting the language barrier as a major obstacle to using OER, stating that ‘most content [is] available in English, many students are not comfortable [with English]’. Respondents mentioned a lack of time and lack of OER that suited their needs ‘off the shelf’. For example, one respondent stated that ‘I do not have enough time to resolve obstacles to solve copyright issues and simply clean up the resources’.