SHARE
Research Highlight

ADIOS 2.8 software release with new file format

Topic:

Achievement: A new file format, BP5, and accompanying serialization class has been developed in the ADaptable I/O System (ADIOS) framework. The new file format is optimized for consuming less memory for I/O and for speeding up the management of its own internal metadata that is used for reading back the self-describing data sets. At the same time, the new serialization class was implemented in way that allowed for new features, including asynchronous I/O, the ability to restart writing data at collective datasets at well-defined points that was requested by many applications, better ability to filtering unwanted sets and thus accelerating the processing of wanted data, and multithreaded read to accelerate processing in serial programs. 

Significance and Impact: The new file format was designed for the extraordinary I/O challenges of applications in the Exascale Computing Program. The new file format enables particle simulations to output nearly the entire output of all the memory of all GPUs in a supercomputer, as required by for example, the WarpX and PIConGPU particle accelerator codes and the XGC fusion code. It was also optimized to handle many scientific variables by applications like the climate simulation code E3SM. These codes have seen improvements up to 33% memory saving for the same output as before, and be able to output more variables and more data for longer than before. 

Research Details

  • The metadata of the self-describing data is organized and encoded on two levels, to catch repeating information once at a higher level, and individual pieces of information at the lower level, thus reducing the overall metadata size. 
  • A new buffering mechanism was developed (list of chunks of limited sizes) to avoid unnecessary reallocations, as well as writing user variables above a certain size directly to storage to avoid unnecessary duplications where I/O performance does not suffer from not buffering.  
  • The output steps of a simulation to a single growing data set are handled more explicitly in the metadata, allowing for flexible filtering of steps at reading as well as restarting writing from a selected step (usually at the last checkpoint step of an aborted simulation).
  • The new data aggregation algorithm across processes is implemented in a new way that allows for turning on asynchronous mode, i.e., writing buffered data while the simulation continues its computation. 

Facility: ADIOS is developed at ORNL, Kitware, LBNL and Georgia Tech.

Sponsor/Funding: DOE Exascale Computing Program

PI and affiliation: Scott Klasky, ORNL 

Team: Norbert Podhorszki, Ana Gainaru, Dmitry Ganyushin, Ruonan Wang, Lipeng Wan (ORNL), Vicente Bolea, Caitlin Ross, Chuck Atkins (Kitware), Greg Eisenhauer (Georgia Tech), John K. Wu, Junmin Gu (LBNL)

Citation and DOI: https://github.com/ornladios/ADIOS2/releases/tag/v2.8.0
 

ADIOS 2.8 software release with new file format
Host-memory consumption of PIConGPU with the old buffering scheme vs new buffering scheme. The old format doubled the memory need to output the GPU content, while BP5 can use the user memory directly
Fig. 2. New, two-level metadata format of BP5 allows for storing the same information on 1/3-1/4 of the previous metadata size for the E3SM climate simulation code, and bringing better performance as well for writing and reading the same dataset.
New, two-level metadata format of BP5 allows for storing the same information on 1/3-1/4 of the previous metadata size for the E3SM climate simulation code, and bringing better performance as well for writing and reading the same dataset.
Fig. 3. While the previous format, BP4, focused on performance improvements for ECP applications, the new format, BP5 supports applications to run at even larger scales and produce more data, while the performance is as good as ever. The asynchronous I/O feature in BP5, however, can further improve I/O performance as well.
While the previous format, BP4, focused on performance improvements for ECP applications, the new format, BP5 supports applications to run at even larger scales and produce more data, while the performance is as good as ever. The asynchronous I/O feature in BP5, however, can further improve I/O performance as well.