Delivering data to facilitate new discoveries in Earth science
The Earth Science Database Engineering and Informatics group plays a crucial role in managing data related to the Atmospheric Radiation Measurement (ARM) project's Data Center (ADC) and other environmental science projects located at Oak Ridge National Laboratory. Our responsibilities encompass various aspects of data management and database infrastructure development including:
- Enabling data science with state-of-the-art metadata management: We are responsible for end-to-end metadata management for the collection and archiving of ARM data. Metadata is essential information that provides context and details about the data itself, including when it was collected, where it came from, how it was measured, and more. Properly managed metadata helps users understand and utilize the data effectively. We are currently developing AI and machine learning strategies to improve metadata creation and quality.
- Next-generation database infrastructure management: Our group is responsible for developing and managing the foundational database infrastructure that supports the operations of the ADC and effective data discovery. This involves designing, implementing, and maintaining the complex database systems that store and manage the ARM data. A well designed and efficient database infrastructure is essential for quick and reliable data retrieval and analysis. The team is currently researching various database architectures, including NOSQL database technologies for data extraction from petabytes of atmospheric data.
- Data center operations: Our work is fundamental to the overall operations of the ARM Data Center. The ADC serves as a hub for storing, processing, and distributing ARM data to researchers, scientists, and other stakeholders. This effort ensures that the data center functions smoothly and efficiently.
- Informatics and development: Our group leverages informatics to enhance the efficiency of data management processes. This involves developing tools, applications, and workflows that streamline data-related tasks and improve accessibility to the data.
The group's contributions are integral to the success of the ARM project and its mission of advancing understanding of atmospheric processes. The work we do ensures that researchers and scientists can access accurate and well-organized data, enabling them to gain new insights and make informed conclusions about atmospheric processes and their effects.