OPERATION OF A DATA ACQUISITION, TRANSFER, AND STORAGE SYSTEM FOR THE GLOBAL SPACE-WEATHER OBSERVATION NETWORK

,


INTRODUCTION
The Earth's magnetosphere is formed by interaction between the solar wind and the Earth's magnetic field.Solar wind conditions change according to changes in solar-activity.Thus, disturbances in the space environment around the Earth, called 'geospace', are driven by both transient and recurrent solar activities.The geospace has been recognized as a key area for human endeavors in space and also for social infrastructure, which is vulnerable to geospace disturbances driven by solar activities.To mitigate the risks caused by geospace disturbances, space weather forecasts are of fundamental importance.
The National Institute of Information and Communications Technology (NICT) is the institute responsible for space-weather forecasting in Japan and is a Regional Warning Centre of the International Space Environment Service.To provide nowcasting and forecasting of space-weather information as operational services, real-time monitoring of solar activity and the geospace environment are essential.Moreover, real-time monitoring is useful to check the current status of observational facilities and the condition of the data network.With these considerations in mind, we have been developing a near-real time data acquisition and transfer system for space-weather monitoring, following recent progress in information and communications technologies (Ishibashi & Nozaki, 1997;Nagatsuma, Obara, Ishibashi, Hayashi, & McEwen, 1999).NICT has been promoting the NICT Space Weather Monitoring Network (NICT-SWM), a project with the aim of establishing a global network of space-weather observations (Nagatsuma, 2009;2013).The basic concept of the project is to improve the reliability of space-weather forecasting by introducing real-time data obtained by our global network of space-weather related observational facilities, namely, ionosondes, magnetometers, high-frequency radars, Global Positioning System (GPS) receivers, solar radio telescopes, and a satellite data reception system.Data analyses of archived data are also important for further development of space-weather forecasting models.
Collection in near-real time of space weather monitoring data from a large number of observatories distributed throughout the world and in space is a nontrivial task.Especially, the Arctic and Antarctic regions are important gateways of solar wind energy, which flows from the magnetosphere to the ionosphere.NICT operates approximately 30 observatories in the NICT-SWM project.These observatories cover a wide-area of the Earth, including the Arctic and Antarctica, and all observational data are transferred on a real-time basis to NICT and deposited in a large storage system in the NICT Science Cloud (Murata, Watari, Nagatsuma, Kunitake, Watanabe, Yamamoto, et al., 2013;Watanabe, Yamamoto, Tsugawa, Nagatsuma, Watari, Murayama, et al., 2013).However, managing the entire operation has become increasingly complex using the legacy system because it contains a vast array of observational instruments, each having its own characteristics and conditions.Problems are beginning to amplify as the data transfer network connects additional observatories, and there is a shortage of human resources to maintain the observational systems.
To overcome these issues, we have developed a new integrated management system of global multipoint observations.The designed and implemented system is named the Wide-area Observation Network Monitoring (WONM) system; the concept and current operation of which are shown throughout the remainder of this paper.

WONM SYSTEM CONCEPT
A schematic of the basic WONM system concept is shown in Figure 1.This system consists of a "client server" at each observation site, a "data transfer" part (via the Internet), and "a central system" at a terminal site.It is necessary for the WONM client to have an "automatic recovery" function with high-level tolerance and redundancy characteristics to assure stable operation of the system.Furthermore, the use of a small-size, low-power, and fan-less personal computer (PC) server is essential for minimizing the load at the observation site.The software for the WONM client can, in contrast, be installed on pre-existing servers at the observation sites to reuse the current hardware resources.Data transfer is a central issue of this system.Preparing a high-performance network band is an optimal solution for rapid and continuous data transfer from the remote sites.However, realizing such a network performance is difficult in practice because our observation sites are often located in isolated regions worldwide.To avoid data gaps due to interruption of the network, functions that retry data transfer and that perform consistency checks of the data files must be included in the system.Moreover, because network policies are site-dependent, flexibility is ensured by preparing different types of protocols for data transfer in advance.
The central system installed at the terminal site needs a vast storage capacity with an appropriate level of redundancy so that a large number of data files can be held with high reliability.Such a hardware environment is available in the NICT Science Cloud (Murata et al., 2013), and it is therefore employed as the WONM central system.This system also requires monitoring software to warn of the condition of the network and of data transfer, and data and status information are transferred from each observational facility to the terminal site using the 'data crawler' and 'status crawler' software, which operate on a PC server at the terminal site or on a WONM appliance server installed at the observatory site.When both of the 'data crawler' and 'status crawler' software are in operation, and data and status information are successfully transferred, the details are archived within the terminal site storage.If the PC server at the observational facility flags that conditions are abnormal, or if the data and status information are not being transferred, the WONM system will give an alert message as warning information.
Combining the aforementioned three components, the WONM system is expected to acquire, transfer, and store data produced by a global observation network.Although this system concept is specified as a use case for space weather monitoring, it is applicable to a variety of research fields operating a network of observational instruments distributed worldwide.

CURRENT OPERATION OF THE WONM SYSTEM
The WONM system was developed according to the conceptual design shown in Section 2, and a trail implementation commenced operation in February 2013.Its technical details will be described in Murata, Nagatsuma, Yamamoto, Watanabe, Ukawa, Muranaga, et al. ( 2014).Once we had confirmed that the WONM system was fully functional, we started to replace our present data acquisition and monitoring system for NICT-SWM with this new system.Locations of observatories currently being managed by the WONM system are plotted on a map in Figure 2, where the site of each observatory is denoted by a PC server.At the time of writing, only part of the NICT-SWM is managed by this system.Table 1 lists the frequency and number (size) of data transferred via the WONM system.Since July 2013, we have already archived about 2.3 TB of data, and we are receiving about 8500 data files per day on average (equivalent to around 4 GB of data) from globally distributed observatories.
Table 1.Daily frequency and accumulated number of data transferred via the WONM system (as of October 2013) A time series of the size and number of data files recorded in the NICT Science Cloud is plotted in Figure 4.The discontinuity in May 2013 indicates the presence of a long network interruption and/or problems at a local data transfer site.After the problem was detected and repaired, file transfer quickly increased and recovered to the nominal level of data transfer, suggesting that data acquisition using the WONM system is smooth and stable.

CONCLUSION
We have shown that the WONM system provides us with an integrated and efficient means to manage a number of space weather observatories distributed worldwide.We have also shown that an interruption in data acquisition can be recovered automatically by this system.Future developments include increasing the number of observation sites to improve the space weather monitoring and forecasting ability currently offered by the WONM system.Moreover, because many projects employing global observation networks experience similar difficulties with data stewardship, the basic design of our system can be applied to other research fields.

Figure 1 .
Figure 1.A schematic showing the basic concept of the WONM system

Figure 2 .
Figure 2. (Top) Map of the observatory network managed by the WONM system; (Bottom) Key of status icons used in the mapFigure3is a screen grab of the world-map window produced from the Web application installed in the WONM system.The status of each observatory can be monitored by this Web application, and the current statuses of the observatories located in Antarctic, Arctic, and Southeast Asia regions are displayed using different icons, as shown in Figure2.If one wishes to check the details of a facility, simply select that observatory from the list or map to view the status time history and specific information.This Web application thus enables us to monitor the entire network in a single display.

Figure 3 .
Figure 3. Screen grab of Web application for browsing WONM information and the status of each observatory node

Figure 4 .
Figure 4. Time series of size and number of data files recorded in the NICT Science Cloud: (Left) GPS data from Chumphon; (Right) Frequency-Modulated Continuous-Wave Ionosonde data from Cebu