Summary

Members of the Arctic and Antarctic data communities held a workshop to interactively engage with researchers and program leaders. They discussed how data management expertise can help collate datasets and improve the trust that researchers can have in data compilations and data products. This workshop enabled interested participants to come and discuss their project together with a range of data management experts and acted as a forum to provide information, offer advice, and help to build networks.

Hackathon proceedings and outcomes

This Hackathon Session brought together data managers and researchers to explore ways that they can facilitate sharing datasets for both data providers and users. One of the key challenges that was highlighted by the community was to balance how much effort to put into managing raw data and producing analysis-ready products. A further challenge, especially relevant in an interdisciplinary community, is that data formats chosen by a data provider may not be readily usable by scientists from other backgrounds. Thus, data managers in the session highlighted the increasing trend towards separating data formats from back-end storage, which improves both machine- and human-readability by making datasets available in a range of formats. Furthermore, there was discussion on the best practices for storing code to support the reproducibility of data processing, analysis, and modelling. Finally, it was agreed that the handling of highly bespoke data types will continue to be a challenge. However, this challenge can be minimised by scientists standardising collection processes and formats through developing and following best practices wherever possible. It can also be minimised by making data management part of the experimental design, rather than an afterthought.

Summary

At the Polar Data Forum III a process was initiated to develop a basis for alignment of polar data policies. The process, involving data managers and experts from international polar data committees in both the northern and southern hemispheres, resulted in a report recommending ten fundamental principles for polar data policies. During this Hackathon Session, those principles were examined as well as other recent developments relating to international data policies. During this Hackathon experts discussed the next steps towards further alignment across polar and global scientific communities and observation systems. Representatives of the relevant organisations were invited to present their views on the continued alignment process.

Hackathon proceedings and outcomes

This session began with an update on the progress of a community-agreed set of recommendations for aligning polar data policies. Some recent developments were highlighted for incorporation in the policy alignment document, including the UNESCO Recommendation on Open Science (UNESCO 2021). Certain aspects of Indigenous data management were also discussed, inter alia the issue of sensitivity and how to allow metadata on sensitive data to be made available without issues. The following discussion on the data policy recommendations led to the finalisation of the reporting for the policy statements (Tronstad et al. 2021).

There were discussions on the data policy revision processes of the World Meteorological Organization (WMO) (WMO 2021) and the Intergovernmental Oceanographic Commission (IOC) (UNESCO-IOC 2019) of UNESCO. Planning discussions focussed on the processes the group needs to follow to update the data policies of the Arctic Data Committee (ADC), Scientific Committee on Antarctic Research (SCAR), and Southern Ocean Observing System (SOOS), so that all three polar data committees can produce policies that follow the common set of recommendations. Alignment of these processes with the initiative to develop a data policy for the Arctic Council was also discussed. This initiative is being coordinated by the Arctic SDI Group and was planned to be pursued at an Arctic data policy workshop during the Arctic Science Summit Week (ASSW) in Tromsø in March 2022. However, following the Russian invasion of Ukraine in February 2022 and subsequent ongoing war, the Arctic Council has suspended its activities.

Summary

With several countries building new polar research vessels and the associated data systems, there is a growing need to standardise data handling processes. One example is the development of an event logger deployment vocabulary to standardise description of actions and processes for ship instrument deployments. To minimise duplication of efforts, it makes sense to discuss development of standardised terminology that can be broadly applied on any marine platform.

Hackathon proceedings and outcomes

Organised as a joint session with the UNESCO-IOC Ocean Best Practices System’s Fifth Annual Community Workshop, this Hackathon Session brought together participants from oceanographic data centres, data aggregation portals, and the Ocean Best Practices System. All participants acknowledged the need for more transparency in data sources and how the data is being aggregated. Such transparency would support users of portals to improve their assessment of the completeness of what they’re accessing and how data has been treated up to the portal presentation. It would also help portal administrators to better identify potential gaps and duplications in their holdings, and more readily test for and fix errors in data feeds.

Participants developed a list of core fields to describe a harvesting relationship between a single source data centre and one aggregator. There was extensive discussion on the best ways to document multiple steps of Quality Control/Quality Assurance, when there are nested aggregations that involve individual datasets going through multiple data centres before they reach a particular data aggregator. It was agreed that each aggregator should pass on all metadata and processing descriptions from all previous steps.

The group concurred to engage the Global Ocean Observing System (GOOS) and explore establishing a new task team to agree on standards for documenting these relationships, as there is a distinct community need for these relationships to be publicly documented.

Summary

Within the polar and oceanographic communities, there is an increasing interest to integrate databases of observing assets, networks, and logistical resources, including research stations, projects, vessels, and expeditions. Integrating these databases requires crosswalks of metadata models and the agreement of best practice methods for sharing the information between catalogues in order to achieve a unified view within and beyond the polar regions.

Hackathon proceedings and outcomes

This session began with updates on progress from a number of projects that aim to improve the interoperability of databases holding information on polar observing networks and polar observing assets. The group reported on Polardex – a new online application for discovery of polar research infrastructures and observing assets, logistical planning, and coordination, developed by the European Polar Board (EPB) in cooperation with several partners (European Polar Board, 2021). Polardex served as a use case, demonstrating the need and value of improved interoperability in polar observing networks and logistical resources, and the need for a metadata standard when describing these. Furthermore, the Sustaining Arctic Observing Networks (SAON) Polar Observing Assets Working Group (POAwg) discussed their efforts to improve integration by focussing on interoperability parameters about research and monitoring assets (SAON-POAwg, 2021). Finally, there were discussions on an EU-PolarNet 2 initiative to develop a procedure for ongoing collection and collation of European polar observing capacities and activities. The session concluded with a broad invitation for participants to join one or more of the polar asset cross walking activities.

Summary

Federated metadata search for the polar regions is dependent on the data centres that host polar-relevant data being able to present discovery metadata in a common way. The Polar Federated Search Working Group (POLDER) is currently developing a Best Practices guidance for the implementation of schema.org as a potentially lightweight discovery metadata standard serving long-tail data in particular. Schema.org is a collaborative, community activity with a mission to create, maintain, and promote schemas for structured data on the Internet, on web pages, in email messages, and beyond. The polar guidance contributes to and draws on similar efforts in related science communities. This session made progress on the development of that Best Practices documentation and informed on the development of the pilot federated search tool (POLDER, 2021) that is currently being supported by the World Data System (WDS).

Hackathons proceeding and outcomes

During this session, POLDER continued to develop its Best Practices guidance for using schema.org as a potentially lightweight interchange standard for federating discovery metadata. This is an ongoing effort that goes beyond the Polar Data Forum, through the Polar to Global (P2G) Hackathons Sessions.

A team from Carleton University in Canada introduced the Mapping the Polar Data Ecosystem (MPDE) – a tool for storing, updating, and sharing information on metadata harvesting relationships among polar data centres that POLDER collated during 2018 and 2019. The new tool takes information that took significant community time to collate and puts it in a data structure that readily supports filtering and updating of the data, as well as providing tools to visually explore the harvesting relationships among metadata catalogues.

The final section of the Hackathon discussed the direction of the activities of the World Data System – International Technology Office (WDS-ITO) pilot federated search tool. The time allotted to POLDER’s Polar Pilot Federated Search (PPFS) was used to discuss recent developments in the project. This included the host domain, potential repositories to be included, and to use the space to announce the second advisory team meeting where these topics could be discussed in further detail. Additionally, this was the first time the polar community had the opportunity to meet the dedicated web developer hired for this project, and were able to voice any comments, questions, and concerns in their direction. Overall, this session represented the official beginning of development for the PPFS tool.

Summary

The Semantics and Vocabularies Hackathon worked on populating a Gap Analysis regarding the current state of the Polar Semantic Landscape. Stemming from the work of the Semantic harmonisation Cluster from Earth Science Information Partners (ESIP) where the sea ice vocabularies of Semantic Web for Earth and Environmental Terminology (SWEET) and Environment Ontology (ENVO) were aligned, this session was used to identify where resources can be allocated moving forward. The Hackathon Session identified the most common vocabularies, and how they are being used in polar settings, and prioritised which would be useful to align next. This can be used to ensure inclusion of Indigenous semantic resources, and/or outline a plan for how to include these in future.

Hackathon proceedings and outcomes

The community was able to collaboratively identify its strengths, weaknesses, opportunities, and threats and determined what an ideal minimum state would look like. This information was then converted into a SWOT Analysis visualisation (Verhey 2022). The group expanded upon the ADC Semantic Working Group’s compilation of polar specific ontological resources, with Hackathon attendees reviewing the current state of the list and adding any additional resources they were not already included. Additionally, it was decided that the resource would benefit from inclusion of information such as a description of each resource, the label of resource hierarchy, and simple use cases. The further development of this work will be carried out in the space of the ADC Semantic Vocabulary Working Group. Finally, the Hackathon participants brainstormed an ideal state of polar semantic resources. This entailed identifying a need for a set of controlled vocabularies and building tools around a small subset; continued collaboration on an international level and sustainable funding to continue this work; and continued outreach to scientific community and Indigenous communities to ensure semantic priorities support specific practical use cases. Moving forward, the community intends to utilise semantics to further enhance meta(data) interoperability. By incorporating standardised vocabularies, with structured identifiers in metadata records, the community is able to utilise existing infrastructure to enhance the Findability, Accessibility, and Interoperability of the FAIR principles.

Summary

Virtual Research Environments (VREs), also known as Science Gateways or Virtual Labs, are digital platforms or programs that provide varying degrees of collaboration, computational power and services designed to support collaboration and communication in teams of researchers. The goal of this session was to articulate the needs of a VRE for the polar data community. It was both a needs assessment and visioning exercise that asked participants who represent many different roles what features or components they need in a VRE. Additional discussion points included compiling existing VREs used by the polar research community, and what makes polar data unique.

Hackathon proceedings and outcomes

Virtual research environments are an emerging concept in polar marine environments. Discussions in this session revealed highly varied understandings of the concept of virtual research environments. Participants’ backgrounds included software developers, database administrators, and researchers (biology, meteorology, geosciences, and oceanography, etc.).

Participants had highly diverse needs in terms of platforms, data types, and analysis tools. The group identified a crucial need for stakeholder-specific solutions for each VRE. This included discussions about the accessibility of web applications. While these applications were described as more democratic and accessible for many users, bandwidth requirements can restrict use for those in remote communities and in the field. It was acknowledged that the applications’ analytic capabilities should cater to the intended audience. The group recognised that public-facing visualisation tools should be simple and intuitive to encourage use by people outside the polar data community, while community-specific tools may expect more knowledge of back-end calculations. Participants also identified a need for the polar marine community to focus on developing analysis-ready data products. To ensure reproducibility, the metadata associated with these products should be made complete (specifically, metadata associated with version control).

Summary

Observing and understanding the Southern Ocean presents significant challenges due to its remoteness and extreme weather conditions. To address these challenges, changes are needed with regards to the science community’s approach to developing technology and deploying infrastructure. Therefore, it is essential to discuss how the changing landscape of infrastructure investment focused on Southern Ocean observing will enable a seed change in observations for the next decade. Topics for this Hackathon Session included trends in infrastructure investment, autonomy, maximising asset use and coordination, and interoperability.

Hackathon proceedings and outcomes

This session began with a discussion on the problems that emerging polar technologies need to solve, including being interoperable, able to operate at scale and able to provide more flexibility in multi-mission parameters. These pressures are being increasingly driven by the need to improve return on investment for funders, who need to collect more data at lower costs. There was general agreement that the future of ocean observing will involve a mix of monolithic and distributed assets. These solutions are more likely to be found when communication and engagement are well established between scientists and the engineers/technicians who are creating the new technologies.

The group allocated significant discussion time to focus on cheap, capable, and sacrificial assets which can provide observations in hard-to-access areas (e.g., under ice), where there are higher risks of equipment being lost. Consequently, this raised questions about the impact of polar technology on the equity of access to polar science; with new technologies often being expensive to develop and purchase. This inequity may leave less-affluent or less-developed polar programmes behind. The group also addressed the challenges for sacrificial assets, where agreement needs to be struck between the technical providers and science programs about what are considered as acceptable risks.

Finally, the group highlighted some of the common issues between providers and scientists, such as requirements for data obtained from polar technology to be made FAIR. This issue can be problematic for companies that provide technology in terms of maintaining the confidentiality of the data where their for-profit structure requires. Additionally, vastly larger data streams will increase demands on data managers. Engineers in the session discussed the need for clearly-defined Essential Ocean Variables (EOVs) or key variables in the ocean sciences, to enable them to better prioritise sensor designs and allocation of payloads on observing platforms.