Open science (OS) is currently dominated by a small subset of practices that occur late in the scientific process. Early career researchers (ECRs) will play a key role in transitioning the scientific community to more widespread use of OS from pre-registration to publication, but they also face unique challenges in adopting these practices. Here, we discuss these challenges across the OS life cycle. Our essay relies on the published literature, an informal survey of 32 ECRs from 14 countries, and discussions among members of the Global Working Group on Open Science (Global Young Academy and National Young Academies). We break the OS life cycle into four stages—study design and tracking (pre-registration, open processes), data collection (citizen science, open hardware, open software, open data), publication (open access publishing, open peer review, open data), and outreach (open educational resources, citizen science)—and map potential barriers at each stage.
The most frequently discussed barriers across the OS life cycle were a lack of awareness and training, prohibitively high time commitments, and restrictions and/or a lack of incentives by supervisors. We found that OS practices are highly fragmented and that awareness is particularly low for OS practices that occur during the study design and tracking stage, possibly creating ‘path-dependencies’ that reduce the likelihood of OS practices at later stages. We note that, because ECRs face unique barriers to adopting OS, there is a need for specifically targeted policies such as mandatory training at the graduate level and promotion incentives.
Open Science (OS) includes a diverse set of practices based on the principles of ‘transparent and accessible knowledge that is shared and developed through collaborative networks’ (Vicente-Saez & Martinez-Fuentes 2018). OS can, and should, be practiced by design (NASEM 2018) throughout the life cycle of the scientific process, from the planning stage through to publication and outreach (Figure 1). There are many potential benefits of widespread adoption of OS, including scientific transparency, reproducibility, and efficiency (e.g. Allen & Mehler 2019). The Covid-19 pandemic has highlighted and escalated the need for OS, which will also be crucial to tracking and reaching the Sustainable Development Goals and other global targets. However, OS is not yet the norm and, to date, has been dominated by a small subset of practices such as open-access publishing.
Early career researchers (ECRs) will play a key role in transitioning the scientific community to OS. Among ECRs, general barriers to OS include reduced scientific flexibility, increased time and cost demands, a lack of training, and a lack of institutional incentives (Allen & Mehler 2019; Mwelwa et al. 2020). These barriers are likely to exist at any career stage, but are particularly critical for ECRs, who are trying to develop research programs and who may not yet have career stability. Barriers to OS practices also vary with discipline and location. For example, variation in primary language spoken across Africa (English, French, Portugese, Spanish, indigenous languages) is a major barrier to the development of OS infrastructure and formation of collaborative networks (Mwelwa et al. 2020).
Though barriers to OS have been discussed in the literature, less common is explicit consideration of barriers at each entry point to the OS life cycle (Gagliardi, Cox, & Li 2015; UNESCO 2020 contain survey results broken down by OS practices). Here, we map the barriers to participation that can occur at each stage of the OS life cycle using published literature and an informal survey of 32 ECRs from 14 countries. After detailing our survey design and results, we discuss obstacles to OS practices based on pre-existing literature and the perspective of our survey respondents.
We developed a simple survey (Appendix 1) to assess knowledge and frequency of open science practices throughout the open science life cycle. In addition to collecting demographic information, the survey contained one section for each of the nine open science practices we identified through working group discussions (‘Preregistration’, ‘Open Processes’, ‘Open Hardware’, ‘Open Software’, ‘Open Data’, ‘Open Publishing’, ‘Open Review’, ‘Citizen Science’, ‘Open Education’). Each section included a 5-point Likert Scale question about knowledge of this practice (‘knowledge score’), a yes/no/unsure question about the use of this practice in research and/or teaching, and an open-ended question about obstacles to using this practice. The survey ended by asking respondents if they consented to being quoted in a publication. The perspectives included in the discussion below come only from those respondents who answered ‘Yes’ to this question.
In April 2021, this survey was sent over email to two groups: the Global Working Group on Open Science, which consists of members of the National Young Academies (NYAs) and the Global Young Academy (GYA), and the Global Young Academy Open Science Working Group, which only consists of members of the GYA. The GYA is restricted to 200 members, currently from 85 countries. There are a total of 49 NYAs worldwide. Members of GYA and NYA represent young researchers who are passionate about collaborating across disciplinary and geographical boundaries to address global issues.
In addition to a link to the survey, our email included a link to the Open Science Training Handbook (Bezjak et al. 2018) so that, if interested, members of these working groups could learn more about open science practices. If respondents viewed this handbook prior to taking the survey, it may have biased their responses. Additionally, as members of NYA and GYA open science working groups, the survey respondents are likely to be more informed on OS practices than many ECRs. Therefore, if this survey were sent to a broader group of ECRs, we expect that the proportion of respondents with knowledge and experience in OS practices would have been lower.
We received responses from 32 respondents from 14 countries, with the greatest representation from Southeast Asia (six respondents from Thailand and six from Malaysia). Most respondents worked at an academic institution, with nearly equal split between respondents who worked at primarily research (13 respondents) and primarily teaching (10 respondents) institutions; other respondents worked at academic institutions that balanced teaching and research (4 respondents), at governmental organizations (3 respondents), or in industry (2 respondents). Most respondents identified as working in a STEM-related field (e.g., bioinformatics, computer science, economics, materials science) while others identified as working in the social sciences or humanities. Our question about career stage was open-ended and thus not well-defined, but nearly all (29 of 32) respondents identified as being early or mid-career. To be eligible for membership at the GYA, individuals must be between 30 and 40 years old, and the GYA tenure is five years, so we expect that most GYA respondents were between the ages of 30 and 45 years old.
When averaged across respondents, the two highest scores for knowledge of the practice (potential range of 1 to 5) were ‘Open Publishing’ (4.19 ± 0.64 S.D.) and ‘Open Review’ (4.19 ± 0.64 S.D.), both of which fall within the publishing phase of the open science life cycle. The lowest average score was for ‘Open Hardware’ (2.19 ± 1.15 S.D.) and, in the open-ended response section, several respondents noted that they had never heard of this practice. The second lowest average score was for ‘Preregistration’ (2.41 ± 1.36 S.D.), which is the first step in the life cycle of a fully open scientific process. Interestingly, this open science practice also showed the highest variation in scores. Some respondents were very knowledgeable about this practice while others did not know what it meant (Figure 2).
Unsurprisingly, respondents were most likely to apply the open science practices that they were most familiar with (i.e., with the highest average knowledge scores). ‘Open Publishing’ (75.00% of respondents) and ‘Open Software’ (68.75%) were the most commonly used practices, though some respondents were unsure whether they used ‘Open Software’ (Figure 3). In contrast, few respondents used ‘Preregistration’ (18.75%), ‘Open Processes’ (18.75%), and/or ‘Open Hardware’ (15.63%) and there was a substantial amount of uncertainty regarding use of these practices.
Our survey responses highlight considerable gaps in awareness and uptake at some points of the OS life cycle, particularly at early planning stages (Preregistration and Open Notebooks). This lack of awareness is particularly surprising given that the survey was sent to two global working groups with a specific focus on OS. Our discussion below draws on pre-existing literature, on the quantitative survey results outlined above, and on responses to the open-ended question about obstacles to each OS practice.
Pre-registration is a practice in which scientists submit their study design to a free public registration platform (e.g. Open Science Framework, Registered Reports) in advance of conducting the research itself. This practice can help researchers to be intentional about and receive feedback on their research plans. It can improve transparency, reproducibility, and rigor and can reduce publication bias (Gonzales et. al. 2015; Kupferschmidt 2018). From 2012–2018, the number of journals that offer pre-registration, as well as the number of pre-registered studies on Open Science Framework, have grown exponentially (Kupferschmidt 2018). However, the practice is still largely restricted to certain fields (e.g. pre-clinical research; Nosek et al. 2018).
Barriers to pre-registration include the time commitment, an inability to make changes to the procedure once committed to pre-registration, and uncertainties about how to register projects involving pre-existing data (Kupferschmidt et. al. 2018; Nosek et al. 2018). For example, researchers may have to collect more data than they want because of a pre-registered commitment to a sample size, even if it seems clear that the results will be negative.
Additionally, effective pre-registration requires practice and enough experience to anticipate workflows ahead of time, which may be more difficult for ECRs who lack decades of experience in their field. Because pre-registration takes time and heightens scientific vulnerability, it may not be appealing to ECRs unless there are institutional incentives. Perhaps most importantly, there is a lack of awareness and training in pre-registration among ECRs, which may lead them to believe that this practice is not relevant to their scientific approach.
Our Perspective: Our survey respondents, who have a demonstrated interest in OS, showed low familiarity with pre-registration. Open-ended responses suggested that, in large part, the low frequency of pre-registration among the survey respondents stemmed from lack of knowledge. Comments like ‘I have never heard of it before’ (or similar sentiments) were very common in response to discussions on and questions about pre-registration. ECRs also voiced concerns about the time commitment and about being ‘scooped’. One respondent noted that, previously, the ideas they shared during pre-registration had been copied.
Open processes begin at the planning stage of research, with organization of documentation and research processes in an appropriate central repository, and continue throughout the life of a project. Since ‘open notebook science’, a central component of open processes, was coined (Bradley 2006), several online platforms have been established for openly tracking research projects (e.g. Openlabnotebooks, Github, Wellcome Open Research publishing platform/Open Research Fund, Zenodo).
Open processes represent a major departure from current practices and hold a mix of promises and risks. Advantages include accessibility of information prior to peer-reviewed publication, allowing for more efficient use of resources and knowledge, and the ability to reproduce detailed protocols (Powell 2016; Wallach, Boyack, & Loannidis 2018; Schapira & Harding 2019). Negative data, which may not be published beyond open notebooks, can provide important scientific insight (Carroll et al. 2017; Mlinarić, Horvat, & Smolčić et al. 2017). However, developing fully reproducible workflows presents a major challenge and it is still unclear that open notebooks or other documented processes will be read and used (Schapira & Harding 2019). Furthermore, misleading or non-rigorous science can still find its way into open platforms, so open comments are a necessity (e.g. healthy discussions at OpenSource Malaria) (Williamson et al. 2016). Other obstacles to the uptake of open processes include a fear of being ‘scooped’ (Grubb & Easterbrook 2011), disagreements among collaborators or other constraints on sharing data and ideas (Bourne et al. 2013), time constraints, and language barriers for non-native English speakers. These obstacles are likely to be exacerbated for ECRs.
Our Perspective: Similar to pre-registration, open processes were among the least well-known and least frequently implemented by the ECRs that we surveyed. This lack of knowledge about open processes was the most-frequently mentioned barrier to their use. One of the respondents stated that, ‘I know what open notebooks are but not sure how best to integrate them into my lab activities…some of the projects I have are industry-facing with…a different electronic reporting process…’ Another respondent stated that, ‘Finding the time to learn the tools necessary for this has been the major barrier for me.’ In addition to a lack of knowledge, respondents mentioned ethics, time, lack of incentives, irrelevance to the field, fear of being ‘scooped’, and protection of shared data as barriers to using open processes.
Open hardware is ‘hardware whose design is made publicly available so that anyone can study, modify, distribute, make, and sell the design or hardware based on that design’ (OSHWA 2016). It is based on the assumption that publishing a ‘design’ realizes the four freedoms of the open source concept; in the context of tangible products, the freedom to study, modify, make, and distribute. At the moment, researchers often use proprietary instrumentation or hardware in their research that is not freely accessible, reusable, or adaptable. While the open source hardware movement is a young phenomenon overall, it is increasingly growing beyond the domain of electronics and DIY to that of non-electronic and complex open source hardware products (Bonvoisin et al. 2017). As the inaccessibility to ventilators during the COVID crisis has highlighted (Pearce 2020), there is an ethical imperative that research infrastructure is accessible to researchers worldwide—be it a specific sensor, state-of-the-art biomedical laboratories, or particle colliders.
There is a push by the scientific community and policy makers to move towards Open Hardware (e.g. GOSH 2018; UNESCO 2021). Open Hardware should therefore also be seen in the context of open infrastructures, i.e. the sharing or giving access to the existing material culture and infrastructure of science. Even trends such as migration of ECRs need to be viewed through the lens of access to research infrastructure and the need for equal access to research infrastructure around the world (POST 2008). However, challenges remain in moving beyond the hobbyist level towards research-grade devices and outputs (Hansen & Howard 2013); this movement is being facilitated by collaborative approaches in the open hardware community (Maher, Paulini, & Murty 2011). The primary limitation is that effective development of novel, open source hardware requires a specific set of skills, tools, and infrastructure (Raasch, Herstatt, & Balka 2009).
Our Perspective: Our survey and discussions indicate that familiarity with and use of open hardware is limited among ECRs, more so than any other OS practice that we categorized under Data Collection. This lack of awareness is detrimental because ECRs specifically face problems with accessing relevant hardware. Among all of the OS practices included in the survey, respondents were most likely to select ‘unsure’ when asked if they used this OS practice in their teaching or research. One respondent stated ‘I still feel very uncertain of the meaning of this or how it might pertain to the type of research that I do.’ This sentiment was echoed by several respondents.
To be considered open software, source code must be freely available with a license (or terms and conditions) that allows for free dissemination and adaptation. Open source software has been highlighted as ‘the forgotten pillar of open science implementation’ (Vermeir et al. 2018) and one of the top needs for scientific research (UNESCO 2020). There are risks, however, of commercial use of open-source materials without compensation of creators (UNESCO 2020).
A study by the GYA’s Global Access to Research Software Working Group showed that researchers in Bangladesh, Ghana, and Nigeria have only limited knowledge about and experience with open research software, despite the clear benefits it may yield to their research (Vermeir et al. 2018). There also seems to be little interaction between the open research software communities in high-income and low-income countries. If these results are generalizable, the lack of active promotion and training may mean that Open Research Software remains inaccessible to most researchers in low-income countries, where it is most needed.
Our Perspective: Most of our survey respondents reported at least some familiarity with open software and many reported using it in their research and/or teaching. They did, however, report some obstacles in using open software. Reported obstacles included a lack of skills and the time investment needed to build these skills, confidentiality aspects of the research, and restrictive software policies from employers. One respondent stated that ‘[Organization Name] is strict with its software policies. Employees can do [limited] things with their computers.’
Open Data provides opportunities for collaboration and encourages further analysis and reuse of data that is often expensive and time-consuming to collect. It is also key to ensuring scientific reproducibility and transparency (Penders & Janssens 2018). The core set of principles that optimize reusability are the FAIR Data Principles: data or any digital object should be Findable, Accessible, Interoperable, and Reusable (Wilkinson et al. 2016). From the perspective of ECRs, Open Data can provide opportunities for research using pre-existing data even if they have limited funding and are unable to generate data themselves.
However, for all its promise, the Open Data movement has met with a number of challenges. Some of the challenges are tied to the Digital Divide—middle to low income countries seem to have unique challenges related to ICT infrastructure and internet connectivity, which has resulted in dis-incentivetizing data sharing (Bezuidenhout & Chakauya 2018). Other barriers are privacy concerns related to the misuse of shared data (Greenbaum et al. 2011). Concerns about Open Data are particularly acute in the Global South and to indigenous communities, including fear of ‘parachute science’ (data from the Global South being used by scientists into the Global North without benefit to Global South scientists; e.g. Serwadda et al. 2018), the right of indigenous communities to data ownership (‘Indigenous Data Sovereignty’; e.g. Kukutai & Taylor 2016), and the illegal extraction of resources (e.g. wildlife poaching based on publicly available animal tracking data; Tulloch et al. 2018).
Our Perspective: The ECRs who responded to our survey were, in general, familiar with Open Data and approximately two-thirds had used Open Data in their teaching or research. When asked about obstacles, they mentioned ethics and confidentiality issues, institutional policies that discourage publishing Open Data, and the practice being generally uncommon in some disciplines (e.g., Humanities/Social Sciences). Respondents were worried about the quality of Open Data (that they were not ‘well-curated’) and about others using these data without proper attribution or without detailed knowledge of the dataset. For example, one respondent shared their colleagues’ concerns about publishing data: ‘1) being robbed of the hard work they had done collecting those data, 2) people not understanding their data enough to accurately use it.’
Open access to the scholarly literature (usually ‘open access’, OA) means that original research results are made freely accessible on the web—free of most embargoes, copyright, and licensing restrictions. While the focus has mainly been on open access to journal articles, OA principles also apply to books and other research outputs. Open access to journal articles has been classified in green, gold and diamond OA (amongst others), but such codes have lost much their utility in recent years with the expansion of new and sometimes complex publishing, licensing and financing models for OA such as transformative agreements, rights-retention strategies, collaborative and sharing networks, and OA platforms.
While OA started as a grassroots movement, funders and policymakers are now playing a significant role in the expansion of open access by mandating policies and supporting OS through infrastructures such as repositories and publishing platforms or by paying for author publication charges (Schiltz 2018; UNESCO 2021). Major progress has been made in OA in recent years, although in 2017, around 75% of journal articles were still paywalled. OA has been shown to positively impact citation counts, benefit innovation, reduce costs in the publication system, and advance citizen science; it may also provide new opportunities for low- and middle- income country scholars and ECRs (Tennant et al. 2016; Piwowar et al. 2018; cOAlition S 2020).
Our Perspective: OA is the best known and most adopted facet of OS among the ECRs we surveyed. Nevertheless, information is distributed unevenly, and two well-qualified ECRs from the Global South were still unsure about what OA involves. The main concern among ECRs is the perceived (high) cost associated with OA publishing, indicating that OA is generally interpreted as a form of ‘gold’ or author-pays OA. The second major concern is the fact that there are usually no incentives to publish OA; current research assessment practices often favor traditional outlets. One survey respondent wrote: ‘I try to do it more and more, though the prestige hierarchy in my field is still going in a completely different direction. There are rather few (pure) open access journals to choose from.’ The ECR continued by referring to the transformative agreements offered to researchers but added: ‘though I’m not sure that’s a model for the future, because only rich countries can do this’, hinting at important global inequalities in opportunities to publish OA. Interestingly, none of our respondents addressed making available a version of the manuscript (pre-print, accepted manuscripts, the final published version or the ‘Version of Record’) on a website or repository, which indicates that the ‘author pays’ model dominates the perception of OA publishing among ECRs.
Open Peer-Review (OPR) mainly refers to ‘open identities’ of authors and reviewers (Ford 2013), but can also refer to ‘open reports’ (making peer-review reports publicly available), ‘open participation’ (crowdsourced peer-review) and other open review practices (Ross-Hellauer 2017). Peer-review is seen as the gold standard for scientific quality but has also been criticized, which has led to the statement that peer review is ‘the worst way to judge research, except for all the others’ (Carroll 2018). A few forms of OPR have been proposed to remedy some of these problems (e.g. Tennant et al. 2017). Despite its promise, OPR itself faces challenges such as difficulties finding reviewers (van Rooyen 1999) and concerns about re-introducing bias in favor of senior researchers. There are unique obstacles for ECRs to adopt OPR, such as the lack of journals that use this practice and research assessment incentivizing publication in traditional journals.
Our Perspective: Open Review had the second highest average knowledge score among the OA practices included in our survey. However, though respondents were generally familiar with this practice, only approximately 50% of respondents reported participating in OPR. A specific concern voiced among ECRs was that OPR exposes them too much as author and reviewer. As authors, they fear that harsh critique in print may affect their career prospects. As reviewers, ECRs are reluctant to sign reports and, as a result, may evaluate manuscripts more positively with OPR than they would otherwise. One ECR in our survey writes: ‘As a young researcher, I have been very hesitant to publish my name with reviews. I feel that there will be backlash if I do not positively review something’.
Gender and cultural factors also play a key role in the dynamics of peer review and peer criticism, and this will be reinforced in OPR. A young researcher from Sri Lanka pointed out that, culturally, some may take the reviews personally. OPR may suit ‘Western values’ better than regions where ‘saving face’ is an important cultural value. These perspectives highlight the fact that, to increase adoption of OPR, research culture as a whole will need to become more transparent and collaborative.
Citizen science (CS) has three interrelated goals: creating knowledge, sharing scientific skills and knowledge with the public, and promoting civic engagement in science (Turrini et al. 2018). Because of the wealth and diversity of data available, CS may provide ECRs with the opportunity to publish high-impact research without the need for a long-established research program. Involvement in CS also provides ECRs with the opportunity to interact with community members and develop research programs of relevance to these communities. Although the number of CS projects has increased exponentially since the late 1980s (annual rate of increase of 10%; Pocock et al. 2017), CS is still relatively uncommon as a primary data source (Burgess et al. 2016) and has not been used to its fullest potential towards tracking global sustainability goals (Lepenies & Zakari 2021).
Barriers to wider implementation of CS projects and use of the resulting data include funding limitation and staff resources, time limitation, lack of awareness among scientists, and mistrust of data collected by non-experts (Burgess et al. 2017; Turrini et al. 2018). ECRs may be less aware of relevant CS projects due to smaller scientific networks and may lack the community ties, training, and time needed to work with citizen scientists. ECRs may also be more hesitant than established researchers to publish using non-traditional data sources, particularly since CS data can be ‘messy’. Furthermore, given the dynamic nature of collaborative projects with citizens, CS projects are difficult to plan and require significant project management skills and capacities.
Our Perspectives: Citizen science was one of the less well-known and used OA practices among our survey respondents. An ECR who does work with citizens to collect data voiced challenges similar to those listed above ‘Although I use it/try to use it – there are some obstacles – mainly lack of knowledge of the participants. It takes some time convincing them.’ Others mentioned that citizen science was not relevant to their research approach but that they interfaced with the public in other ways, that they ‘give lots of public talks and participate in public discussions’.
The term ‘Open Educational Resources’ (OER), first coined during a UNESCO forum in 2002, is used to describe any educational materials that are fully open. Being open means that educators and learners must be able to retain, reuse, revise, remix and redistribute these materials (Al Abri & Dabbagh 2018). OER range from open software, through open textbooks, to fully open classes (e.g. Massive Open Online Courses or MOOCs). Although faculty and students who use OER generally report that these resources meet their pedagogical needs while reducing financial barriers, adoption of these materials has been slow (e.g. 42% of faculty in the United States are not aware of OER). Rates of use are, however, increasing in the United States (e.g. from 5% to 14% of US faculty between 2015–2016 and 2018–2019) (Seaman and Seaman 2020).
Frequently reported barriers to OER adoption include a lack of awareness of available OER and options for finding reputable OER, a lack of training in OER development and open licensing, issues or perceived issues with OER quality, and the resources needed to develop, maintain, and adopt OER (Hassall & Lewis 2017; Al Abri & Dabbagh 2018; Koseoglu & Bozkurt 2018; Luo et al. 2020). These barriers may be particularly acute for ECRs, whose publishing records and teaching evaluations are under close scrutiny. Time constraints, which are one of the largest barriers to OER adoption, are best addressed through institutional support, including financial incentives and OER training (Luo et al. 2020; McGowan 2020). In some countries, there may be language barriers or technological barriers to adopting OER. Additionally, most OERs come from a small number of well-resourced countries, which can lead to the dominance of one cultural perspective and resources that do not address local educational needs.
Our Perspective: Most, though not all, of our survey respondents were familiar with OER and approximately two-thirds had used it in their teaching. Among our survey respondents, some mentioned informal use of OER (‘This works mostly informally in my field, with people sharing syllabuses etc.’). Other respondents echoed the barriers above, with one respondent noting the language barrier as a major obstacle to using OER, stating that ‘most content [is] available in English, many students are not comfortable [with English]’. Respondents mentioned a lack of time and lack of OER that suited their needs ‘off the shelf’. For example, one respondent stated that ‘I do not have enough time to resolve obstacles to solve copyright issues and simply clean up the resources’.
Here, we leveraged the experiences of a diverse group of ECRs to provide some insight into barriers to OS adoption. Our essay reflects the views of a small group of ECRs who have a shared interest in OS through their involvement with GYA and NYA working groups. Due to our small sample size, we could not consider the synergistic effects of challenges linked to career stage, gender, nationality, etc. As mentioned above, given that our respondents are, in theory, relatively informed in OS, we expect the gaps in knowledge about less-known OS practices (e.g., pre-registration) to be even more pronounced among ECRs as a whole.
Our perspectives influenced the structure of our survey and, as a result, the conclusions we can draw from it. For the most part, the coauthors of this paper and the survey respondents work in STEM-related fields at an academic institution. Given our small sample size, we can only speak to this anecdotally, but several of the respondents who identified as social scientists mentioned a lack of relevance of the OS practices included in the survey. This perceived lack of relevance could, in part, be due to a larger issue—as Sidler (2014) notes, the use of the word ‘science’ in ‘Open Science’ perpetuates long-lasting silos of natural sciences, social sciences, and humanities and limits the perceived relevance of OS practices to non-STEM fields. As a result, some have suggested changing the phrasing from ‘Open Science’ to ‘Open Knowledge’ (Sidler 2014).
The composition of our group also influenced which OS practices we focused on in our survey questions. What constitutes OS varies depending on who you ask and on their goals. Fecher and Friesike (2014) suggest that there are five schools of thought surrounding OS: those interested in making research more efficient through open tools (‘Infrastructure School’) or sharing of knowledge (‘Pragmatic School), those interested in making science accessible to the public (‘Public School’) or in making this access more equitable (‘Democratic School’), and those concerned with how scientific success is assessed (‘Measurement School’). While we do not fit neatly into one of these schools, we recognize that our highly structured ‘life cycle’ approach to OS is characteristic of our experiences working in STEM fields in academia.
Additionally, our survey failed to sufficiently capture the intersection of OS and public engagement and trust in science. While we did include citizen science, we did not include other aspects of the ‘Public School’ of OS (Fecher and Friesike 2014), such as making science accessible (not just available) to the public. One respondent highlighted this gap in their response to the open-ended question on citizen science, mentioning that they gave public talks and otherwise participated in outreach. Similar to other OS practices, while this type of ‘impact work’ often benefits both society and researchers, it is undervalued in assessments at academic institutions (Friesike et al. 2021).
Furthermore, in many cases (e.g., ‘Open Data’, ‘Open Software’, ‘Open Hardware’) we did not distinguish between the creation and use of OS materials. In the case of ECRs, the barriers to creating OS are likely to be greater than those to using OS, particularly if there are no incentives for the creation of these materials. For example, creating an open textbook for a class takes substantially more time than adopting a pre-existing open textbook. Therefore, the proportion of ECRs who develop OS (rather than only using existing OS resources) will be lower than suggested by our survey. Robson et al. (2021) point out that studies suggest that 70% of researchers use open data while only 10% share their own data (statistics from Houtkoop et al. 2018; Fane et al. 2019).
The state of OS is highly fragmented. OS communities often do not know of each other, and even in individual research practices, it is mostly only a subset of scientific activities that can be considered ‘open’. Examining OS barriers at each stage of the scientific process can highlight less visible parts of the OS landscape and how they might influence the movement as a whole.
Our discussions and survey results show that awareness of OS practices is particularly low at early stages of the scientific life cycle. This lack of awareness can lead to ‘path-dependencies’, wherein historical decisions influence the suite of possible future outcomes (Sydow et al. 2008). For example, pre-registration of research should lessen concerns among ECRs about reviews of their work being publicly available (and therefore promote open peer-review), since the research approach would be assessed prior to being carried out. As another example, openly documenting the entire research process (open processes/notebooks) undoubtedly improves the reusability of open data. We suggest that the earlier in the scientific life cycle that researchers enter the open ecosystem, the likelier it is that subsequent activities will also be truly open.
When aligning aspects of the OS life cycle with the barriers that researchers encounter, we find that many of these barriers are common to multiple stages of this cycle. It is clear that ECRs face unique barriers to adopting OS practices, suggesting the need for specifically targeted policies (e.g. promotion incentives). The most frequently cited barriers across the OS life cycle are a lack of awareness and training, increased time commitments, and restrictions on open practices by supervisors. There were also barriers that were unique to specific stages of the life cycle. ECRs, for example, fear retribution for critical open peer reviews and may not have the learned experience to predict research workflows and pre-register their studies. Importantly, barriers to OS also vary globally. A few examples highlighted by the ECR perspectives included here are: 1) many OER are available only in English and may suffer from educational colonialism, 2) in some regions, there are cultural barriers to open scrutiny (such as noted for open peer-review), and 3) among ECRs from the Global South, there are concerns of data colonialism and questions of who benefits from OS.
While it may be difficult to convince some supervisors, we hope that there will be a widespread generational change towards a broader adoption of OS practices. Junior scientists could try to convince university leadership to sign up to OS commitments and then, through this, try to effect institutional change that might then convince individual faculty members. The UNESCO Recommendation on Open Science, adopted by the General Conference of UNESCO in November 2021, specifically encourages that institutions ‘adopt statements of principle in line with this Recommendation to encourage open science practice’ (UNESCO 2021). This recommendation recognizes the importance of aligning incentives for career progression with open science practices. In general, there is a need to rethink assessment. Resources like the San Francisco Declaration on Research Assessment, developed by journal editors and publishers at a professional conference in 2012, provide guidance to funding agencies, institutions, and publishers on how to improve assessment. This declaration has been signed by over 2,200 organizations (Robson et al. 2021).
Eventually, supervisors will have no choice but to promote OS practices. Journals are increasingly requiring that data and code be published alongside manuscripts (Hrynaszkiewicz et al. 2020; Tedersoo et al. 2021) and federal funding agencies often require data management plans, with some requiring data sharing. For example, the United States National Institutes of Health (NIH) has a data sharing policy that requires those receiving grants in excess of $500,000 to make their data publicly available while maintaining participant confidentiality (NIH 2020). The NIH is currently building on this policy to promote data sharing that follows FAIR principles.
Training is a low-hanging fruit for promoting OS practices among ECRs, yet there remains a lack of training on most OS topics. Even knowledge of OA, the most well-known aspect of OS, was limited to a subset of OA (i.e. ‘author pays’ models) among ECRs in this study. Free introductory courses on reproducible and transparent research do exist and have been shown to be successful at promoting OS practices, but these need to be integrated more formally into education at the undergraduate and/or graduate level and, potentially, mandated. For example, a 60-hour course (15-20 hours of lecture) on OS resulted in adoption of OS practices or plans for adoption of these practices by over half of the participants over the next six months; over 80% agreed that these practices improved the quality of science (Toelch & Ostwald 2018).
Similar to calls to include science communication training in graduate curricula (Hundey et al. 2016), we suggest graduate students should be introduced to and trained in OS practices early in their academic career. These concepts could easily be introduced in a brief seminar then reinforced throughout graduate classes where relevant. For example, in classes that involve coding, students should be graded on how reusable and well-documented their code is in addition to the accuracy of the code itself.
Reducing barriers to use of OS will require effort at multiple levels: at the individual level, the departmental level, the institutional level, and by journals and funding agencies (Robson et al. 2021). Once ECRs are themselves educated in these practices, they can help to promote OS through low-effort sharing in seminars, journal clubs, or discussions with colleagues. An ECR who uses OS in their teaching and research is likely to promote the use of these practices, whether explicitly (e.g., by requiring graduate students to pre-register their theses) or not (by setting an example for their students), among their trainees (Robson et al. 2021). Efforts to train ECRs in OS and to incentivize these practices will therefore result in knock-on effects that ultimately lead to widespread adoption of OS by upcoming generations of scientists.
The additional file for this article can be found as follows:Appendix
Obstacles to Open Science: YoungResearcher Perspectives. DOI: https://doi.org/https://doi.org/10.5334/dsj-2022-002.s1
The authors are members or alumni of the Global Young Academy (GYA) or National Young Academies (NYA). The GYA administrative Office is publicly funded and hosted at the German National Academy of Sciences Leopoldina. The wide array of GYA and NYA activities are supported by a range of international public and private funders and we are thankful for their support. We would like to thank the GYA Office staff for their review of the draft of this article and the 32 survey respondents, GYA Global Working Group, and NYA/GYA Global Working Group on Open Science for their insight and perspectives. In particular, Wataru Iwasaki contributed insight to earlier versions of this manuscript.
MIB is a shareholder and director of Arctoris Ltd.
Members of the GYA Open Science Group, including KV, have received funding from Open Research Europe.
All authors conceived of the idea and contributed intellectually. N. Gownaris led project management and N. Gownaris and K. Vermeir led paper writing. Authors were each responsible for finding sources and writing sections on specific facets of open science and for conducting extensive peer reviews of other sections. N. Gownaris led writing of the introduction and the open educational resources and citizen science sections, K. Vermeir led writing of the open publishing and open peer review sections, L. Gunawardena led writing of the pre-registration and open data sections, S. Kaur-Ghumaan led writing of the open processes section, G.N. Ntsefong contributed to the open publishing section, R. Lepenies led writing of the conclusions section and figure drafting, and I.S. Zakari led writing of the open software section. N. Gownaris led manuscript revisions and all authors reviewed and approved of the final manuscript.
Abri, MA and Dabbagh, N. 2018. Open educational resources: A literature review. Journal of Mason Graduate Research, 6(1): 83–104. DOI: https://doi.org/10.13021/G8jmgr.v6i1.2386
Allen, C and Mehler, DMA. 2019. Open science challenges, benefits and tips in early career and beyond. PLoS Biology, 17(5): e3000246. DOI: https://doi.org/10.1371/journal.pbio.3000246
Bezuidenhout, L and Chakauya, E. 2018. Hidden concerns of sharing research data by low/middle-income country scientists. Global Bioethics, 29(1): 39–54. DOI: https://doi.org/10.1080/11287462.2018.1441780
Bonvoisin, J, Thomas, L, Mies, R, Gros, C, Stark, R, Samuel, K, Jochem, R and Boujut, JF. 2017. Current state of practices in open source product development. In: DS 87-2 Proceedings of the 21st International Conference on Engineering Design (ICED 17) Vol 2: Design Processes, Design Organisation and Management, Vancouver, Canada in August 2017, pp. 111–120.
Bradley, J. 2006. Open Notebook Science. Available at http://drexel-coas-elearning.blogspot.com/2006/09/open-notebook-science.htm. [Last accessed 31 May 2021].
Burgess, HK, DeBey, LB, Froehlich, HE, Schmidt, N, Theobald, EJ, Ettinger, AK, HilleRisLambers, J, Tewksbury, J and Parrish, JK. 2017. The science of citizen science: Exploring barriers to use as a primary research tool. Biological Conservation, 208: 113–120. DOI: https://doi.org/10.1016/j.biocon.2016.05.014
Carroll, AE. 2018. Peer review: the worst way to judge research, except for all the others. The New York Times. Available at https://www.nytimes.com/2018/11/05/upshot/peer-review-the-worst-way-to-judge-research-except-for-all-the-others.html. [Last accessed 31 May 2021].
Carroll, HA, Toumpakari, Z, Johnson, L and Betts, JA. 2017. The perceived feasibility of methods to reduce publication bias. PloS one, 12(10): e0186472. DOI: https://doi.org/10.1371/journal.pone.0186472
cOAlition, S. 2020. Input for the development of the UNESCO Recommendation on Open Science. Hosted by European Science Foundation, 1, Quai Lezay-Marnésia, 67080 Strasbourg, France, 8 pages.
Fane, B, Ayris, P, Hahnel, M, Hrynaszkiewicz, I, Baynes, G and Farrell, E. 2019. The State of Open Data Report 2019. Digital Science. DOI: https://doi.org/10.6084/m9.figshare.9980783.v2
Fecher, B and Friesike, S. 2014. Open science: One term, five schools of thought. In: Bartling, S and Friesike, S (eds.) openingscience: The Evolving Guide on How the Internet is Changing Research, Collaboration, and Scholarly Publishing, 17–47. Germany: SpringerOpen. DOI: https://doi.org/10.1007/978-3-319-00026-8
Ford, E. 2013. Defining and characterizing open peer review: A review of the literature. Journal of Scholarly Publishing, 44(4): 311–326. DOI: https://doi.org/10.3138/jsp.44-4-001
Friesike, S, Dobusch, L and Heimstädt, M. 2021. Striving for societal impact as an early-career researcher: Reflections on five common concerns. SocArXiv. May 3. DOI: https://doi.org/10.31235/osf.io/kjehy
Gagliardi, D, Cox, D and Li, Y. 2015. Institutional Inertia and Barriers to the Adoption of Open Science. In: The Transformation of University Institutional and Organizational Boundaries. The Netherlands: Brill. pp. 107–133. DOI: https://doi.org/10.1007/978-94-6300-178-6_6
Gonzales, JE and Cunningham, CA. 2015. The promise of pre-registration in psychological research, Psychological Science Agenda. Available at: http://www.apa.org/science/about/psa/2015/08/pre-Registration. [Last accessed 31 May 2021].
GOSH (Gathering for Open Science Hardware). 2018. Global open science hardware roadmap: making open science hardware ubiquitous by 2025. Available at: https://openhardware.science/global-open-science-hardware-roadmap/. [Last accessed 31 May 2021].
Greenbaum, D, Sboner, A, Mu, XJ and Gerstein, M. 2011. Genomics and privacy: implications of the new reality of closed data for the field. PLoS Comput Biol, 7(12): e1002278. DOI: https://doi.org/10.1371/journal.pcbi.1002278
Grubb, AM and Easterbrook, SM. 2011. On the lack of consensus over the meaning of openness: an empirical study. PloS one, 6(8): e23420. DOI: https://doi.org/10.1371/journal.pone.0023420
Hansen, A and Howard, TJ. 2013. The current state of open source hardware: The need for an open source development platform. In: Chakrabarti, A and Prakash, R (eds), ICoRD’13. Lecture Notes in Mechanical Engineering, 977–988. India: Springer. DOI: https://doi.org/10.1007/978-81-322-1050-4_77
Hassall, C and Lewis, DI. 2017. Institutional and technological barriers to the use of open educational resources (OERs) in physiology and medical education. Advances in Physiology Education, 41(1): 77–81. DOI: https://doi.org/10.1152/advan.00171.2016
Houtkoop, BL, Chambers, C, Macleod, M, Bishop, DV, Nichols, TE and Wagenmakers, EJ. 2018. Data sharing in psychology: A survey on barriers and preconditions. Advances in methods and practices in psychological science, 1(1): 70–85. DOI: https://doi.org/10.1177/2515245917751886
Hrynaszkiewicz, I, Simons, N, Hussain, A, Grant, R and Goudie, S. 2020. Developing a research data policy framework for all journals and publishers. Data Science Journal, 19(1): 5. DOI: https://doi.org/10.5334/dsj-2020-017
Hundey, EJ, Olker, JH, Carreira, C, Daigle, RM, Elgin, AK, Finiguerra, M, Gownaris, NJ, Hayes, N, Heffner, L, Razavi, NR and Shirey, PD. 2016. A shifting tide: recommendations for incorporating science communication into graduate training. Limnology and Oceanography Bulletin, 25(4): 109–116. DOI: https://doi.org/10.1002/lob.10151
Koseoglu, S and Bozkurt, A. 2018. An exploratory literature review on open educational practices. Distance Education, 39(4): 441–461. DOI: https://doi.org/10.1080/01587919.2018.1520042
Kukutai, T and Taylor, J. 2016. Indigenous data sovereignty: Toward an agenda. Research monograph Australian National University, Centre for Aboriginal Economic Policy Research: no. 38. Australia: Anu Press. DOI: https://doi.org/10.22459/CAEPR38.11.2016
Kupferschmidt, K. 2018. ‘More and more scientists are preregistering their studies. Should you?’. Science. Available at https://www.sciencemag.org/news/2018/09/more-and-more-scientists-are-preregistering-their-studies-should-you. [Last accessed 31 May 2021]. DOI: https://doi.org/10.1126/science.aav4786
Lepenies, R and Zakari, IS. 2021. Citizen Science for Transformative Air Quality Policy in Germany and Niger. Sustainability, 13(7): 3973. DOI: https://doi.org/10.3390/su13073973
Luo, T, Hostetler, K, Freeman, C and Stefaniak, J. 2020. The power of open: Benefits, barriers, and strategies for integration of open educational resources. Open Learning, 35(2): 140–158. DOI: https://doi.org/10.1080/02680513.2019.1677222
Maher, ML, Paulini, M and Murty, P. 2011. Scaling up: From individual design to collaborative design to collective design. In: Design Computing and Cognition’10. Dordrecht: Springer pp. 581–599. DOI: https://doi.org/10.1007/978-94-007-0510-4_31
McGowan, V. 2020. Institution initiatives and support related to faculty development of open educational resources and alternative textbooks. Open Learning: The Journal of Open, Distance and e-Learning, 35(1): 24–45. DOI: https://doi.org/10.1080/02680513.2018.1562328
Mlinarić, A, Horvat, M and Šupak Smolčić, V. 2017. Dealing with the positive publication bias: Why you should really publish your negative results. Biochemia medica, 27(3): 447–452. DOI: https://doi.org/10.11613/BM.2017.030201
Mwelwa, J, Boulton, G, Wafula, JM and Loucoubar, C. 2020. Developing open science in Africa: barriers, solutions and opportunities. Data Science Journal, 19(31): 1–17. DOI: https://doi.org/10.5334/dsj-2020-031
NASEM (National Academies of Sciences, Engineering, and Medicine). 2018. Open Science by Design: Realizing a Vision for 21st Century Research. Washington, DC: The National Academies Press. DOI: https://doi.org/10.17226/25116
NIH (National Institutes of Health). 2020. NIH Data Sharing Policy and Implementation Guide. Available at https://grants.nih.gov/grants/policy/data_sharing/data_sharing_guidance.htm [Last accessed 19 December 2021].
Nosek, BA, Ebersole, CR, DeHaven, AC and Mellor, DT. 2018. The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11): 2600–2606. DOI: https://doi.org/10.1073/pnas.1708274114
OSHWA (Open Source Hardware Association). 2016. Open Source Hardware (OSHW) Statement of Principles 1.0. Available at https://www.oshwa.org/definition/. [Last accessed 31 May 2021].
Pearce, JM. 2020. A review of open source ventilators for COVID-19 and future pandemics. F1000Research, 9. DOI: https://doi.org/10.12688/f1000research.22942.2
Penders, B and Janssens, ACJ. 2018. Finding Wealth in Waste: Irreplicability ReExamined. BioEssays, 40(12): 1800173. DOI: https://doi.org/10.1002/bies.201800173
Piwowar, H, Priem, J, Larivière, V, Alperin, JP, Matthias, L, Norlander, B, Farley, A, West, J and Haustein, S. 2018. The state of OA: a large-scale analysis of the prevalence and impact of Open Access articles. PeerJ, 6: e4375. DOI: https://doi.org/10.7717/peerj.4375
Pocock, MJ, Tweddle, JC, Savage, J, Robinson, LD and Roy, HE. 2017. The diversity and evolution of ecological and environmental citizen science. PLoS One, 12(4): e0172579. DOI: https://doi.org/10.1371/journal.pone.0172579
POST (Parliamentary Office of Science and Technology). 2008. International migration of scientists and engineers. Available at https://www.parliament.uk/globalassets/documents/post/postpn309.pdf. [Last accessed 31 May 2021].
Powell, K. 2016. The waiting game. Does it take too long to publish research. Nature. Available at https://www.nature.com/news/does-it-take-too-long-to-publish-research-1.19320. [Last accessed 31 May 2021]. DOI: https://doi.org/10.1038/530148a
Raasch, C, Herstatt, C and Balka, K. 2009. On the open design of tangible goods. R&D Management, 39(4): 382–393. DOI: https://doi.org/10.1111/j.1467-9310.2009.00567.x
Robson, SG, Baum, MA, Beaudry, JL, Beitner, J, Brohmer, H, Chin, J, Jasko, K, Kouros, CD, Laukkonen, RE, Moreau, D, Seartson, RA, Slagter, HA, Steffens, NK, Tangen, JM and Thomas, A. 2021. Promoting Open Science: A holistic approach to changing behaviour. PsyArXiv. April 1. DOI: https://doi.org/10.31234/osf.io/zn7vt
Ross-Hellauer, T. 2017. What is open peer review? A systematic review. F1000Research, 6. DOI: https://doi.org/10.12688/f1000research.11369.2
Schapira, M, The Open Lab Notebook Consortium and Harding, RJ. 2019. Open laboratory notebooks: good for science, good for society, good for scientists. F1000Research, 8: 87. DOI: https://doi.org/10.12688/f1000research.17710.2
Schiltz, M. 2018. Science Without Publication Paywalls a Preamble to: cOAlition S for the Realisation of Full and Immediate Open Access. Available at https://www.coalition-s.org/why-plan-s/. [Last accessed 31 May 2021]. DOI: https://doi.org/10.3389/fnins.2018.00656
Seaman, JE and Seaman, J. 2020. Digital Texts in the Time of COVID: Educational Resources in U.S. Higher Education, 2020. Available at https://www.bayviewanalytics.com/oer.html. [Last accessed 31 May 2021].
Serwadda, D, Ndebele, P, Grabowski, MK, Bajunirwe, F and Wanyenze, RK. 2018. Open data sharing and the Global South—Who benefits? Science, 359(6376): 642–643. DOI: https://doi.org/10.1126/science.aap8395
Sidler, M. 2014. Open science and the three cultures: Expanding open science to all domains of knowledge creation. In: Bartling, S and Friesike, S (eds.), openingscience: The Evolving Guide on How the Internet is Changing Research, Collaboration, and Scholarly Publishing, 81–85. Germany: SpringerOpen. DOI: https://doi.org/10.1007/978-3-319-00026-8
Sydow, J, Schreyögg, G and Koch, J. 2008. Organizational path dependence: opening the black box. Academy of Management Review, 34(4): 689–709. DOI: https://doi.org/10.5465/amr.34.4.zok689
Tedersoo, L, Küngas, R, Oras, E, Köster, K, Eenmaa, H, Leijen, Ä, Pedaste, M, Raju, M, Astapova, A, Lukner, H and Kogermann, K. 2021. Data sharing practices and data availability upon request differ across scientific disciplines. Scientific data, 8(1): 1–11. DOI: https://doi.org/10.1038/s41597-021-00981-0
Tennant, JP, Dugan, JM, Graziotin, D, Jacques, DC, Waldner, F, Mietchen, D, Elkhatib, Y, Collister, LB, Pikas, CK, Crick, T and Masuzzo, P. 2017. A multi-disciplinary perspective on emergent and future innovations in peer review. F1000Research, 6. DOI: https://doi.org/10.12688/f1000research.12037.3
Tennant JP, Waldner F, Jacques D C, Paola, M, Lauren, BC and Chris, HJH. 2016. The academic, economic and societal impacts of Open Access: an evidence-based review. F1000Research, 5: 632. DOI: https://doi.org/10.12688/f1000research.8460.3
Toelch, U and Ostwald, D. 2018. Digital open science—Teaching digital tools for reproducible and transparent research. PLoS biology, 16(7): e2006022. DOI: https://doi.org/10.1371/journal.pbio.2006022
Tulloch, AI, Auerbach, N, Avery-Gomm, S, Bayraktarov, E, Butt, N, Dickman, CR, Ehmke, G, Fisher, DO, Grantham, H, Holden, MH and Lavery, TH. 2018. A decision tree for assessing the risks and benefits of publishing biodiversity data. Nature ecology & evolution, 2(8): 1209–1217. DOI: https://doi.org/10.1038/s41559-018-0608-1
Turrini, T, Dörler, D, Richter, A, Heigl, F and Bonn, A. 2018. The threefold potential of environmental citizen science-Generating knowledge, creating learning opportunities and enabling civic participation. Biological Conservation, 225: 176–186. DOI: https://doi.org/10.1016/j.biocon.2018.03.024
UNESCO (United Nations Educational, Scientific, and Cultural Organization). 2020. Report on UNESCO’s global online consultation on open science. Available at https://en.unesco.org/science-sustainable-future/open-science/consultation. [Last accessed on 31 May 2021].
UNESCO (United Nations Educational, Scientific, and Cultural Organization). 2021. Draft text of the UNESCO Recommendation on Open Science. SS-PCB-SPP/2021/OS-IGM/WD3.
van Rooyen, S, Godlee, F, Evans, S, Black, N and Smith, R. 1999. Effect of open peer review on quality of reviews and on reviewers’ recommendations: a randomised trial. BMJ, 318(7175): 23–27. DOI: https://doi.org/10.1136/bmj.318.7175.23
Vermeir, K, Leonelli, S, Shams Bin Tariq, A, Olatunbosun Sojinu, S, Ocloo, A, Ashraful Islam Khan, Md and Bezuidenhout, L. 2018. Global Access to Research Software:The Forgotten Pillar of Open Science Implementation. The Global Young Academy. Available at https://globalyoungacademy.net/wp-content/uploads/2018/03/18013_GYA_Report_GARS-Web.pdf. [Last accessed 31 May 2021].
Vicente-Sáez, R and Martínez-Fuentes, C. 2018. Open Science now: A systematic literature review for an integrated definition. Journal of business research, 88: 428–436. DOI: https://doi.org/10.1016/j.jbusres.2017.12.043
Wallach, JD, Boyack, KW and Loannidis, JP. 2018. Reproducible research practices, transparency, and open access data in the biomedical literature, 2015–2017. PLoS biology, 16(11): e2006930. DOI: https://doi.org/10.1371/journal.pbio.2006930
Wilkinson, MD, Dumontier, M, Aalbersberg, IJ, Appleton, G, Axton, M, Baak, A, Blomberg, N, Boiten, JW, da Silva Santos, LB, Bourne, PE and Bouwman, J. 2016. Comment: The FAIR Guiding Principles for scientific data management and stewardship. Scientific data, 3: 160018. DOI: https://doi.org/10.1038/sdata.2016.18
Williamson, AE, Ylioja, PM, Robertson, MN, Antonova-Koch, Y, Avery, V, Baell, JB, Batchu, H, Batra, S, Burrows, JN, Bhattacharyya, S, Calderon, F, Charman, SA, Clark, J, Crespo, B, Dean, M, Debbert, SL, Delves, M, Dennis, AS, Deroose, F, Duffy, S, Fletcher, S, Giaever, G, Hallyburton, I, Gamo, FJ, Gebbia, M, Guy, RK, Hungerford, Z, Kirk, K, Lafuente-Monasterio, MJ, Lee, A, Meister, S, Nislow, C, Overington, JP, Papadatos, G, Patiny, L, Pham, J, Ralph, SA, Ruecker, A, Ryan, E, Southan, C, Srivastava, K, Swain, C, Tarnowski, MJ, Thomson, P, Turner, P, Wallace, IM, Wells, TN, White, K, White, L, Willis, P, Winzeler, EA, Wittlin, S and Todd, MH. 2016. Open source drug discovery: Highly potent antimalarial compounds derived from the tres cantos arylpyrroles. ACS Central Science, 2(10): 687–701. DOI: https://doi.org/10.1021/acscentsci.6b00086