SHARE
Research Highlight

Cabana particle library feature expansion, publication, and release

Topic:

Achievement: A collaborative team of researchers from Oak Ridge National Laboratory (ORNL) and four additional labs have published a new article in the Journal of Open Source Software paired with the release of a new version of the Cabana library for particle applications. The article discusses the intent, capabilities, design, and usage of the Cabana library and was accompanied by a detailed review of not only the manuscript, but also the software itself. The new version 0.5 release includes updates across the library with particular emphasis on tighter integration of the particle and particle-grid sub-packages, new interfaces to external load balancing and particle output libraries, additional performance benchmarks, as well as significant improvement to the library documentation for both users and developers.

Significance and Impact: The Cabana library is built on Kokkos for performance portable particle simulation across many scientific domains and application areas. Two Exascale Computing Project applications, the XGC plasma particle-in-cell (PIC) code and PicassoMPM additive manufacturing material point method (MPM) code, are built on top of Cabana, enabling their effective use of Summit and coming Frontier hardware through particle data structures, particle algorithms, and particle communication. In addition, many other particle applications are using Cabana directly or for rapid prototyping, including HACC for cosmology, ESPResSo++ for multi-scale molecular dynamics, PUMI-PIC for plasma physics, and CabanaPD for fracture mechanics. The new features, updates, and expanded documentation will greatly improve current and future experiences in using and adopting Cabana.

Research Details

  • Continued integration of core particle and grid sub-packages with new, more user-friendly, multi-node particle migration based on the underlying grid
  • New interfaces to the ALL and SILO libraries provide multi-node grid load balancing and multi-node particle visualization output, respectively
  • Grid sub-package performance benchmarks were added to assess performance across hardware and particle-grid systems 
  • The existing documentation was updated to reflect library changes and a full set of tutorial examples for the grid sub-package was added; in addition, the online wiki was expanded to discuss all features and the automated developer API documentation was improved
  • Numerous updates to better use and match Kokkos coding style added to improve maintainability

Facility: The Oak Ridge Leadership Computing Facility (OLCF) is used for Cabana development and includes multiple primary systems for Cabana applications.

Sponsor/Funding: Exascale Computing Project

PI and affiliation: Sue Mniszewski, Los Alamos National Laboratory (LANL)

Team: Samuel Reeve, Kwitae Chong, Austin Isner, Damien Lebrun-Grandié, Stuart Slattery (ORNL), Robert Bird, Guangye Chen, Shane Fogerty, Christoph Junghans, (LANL), Stan Moore (Sandia National Laboratories), Jim Belak (Lawrence Livermore National Laboratory), Aaron Scheinberg (Jubilee Development), Yuxing Qiu (UCLA), Stephan Schulz (Jülich Supercomputing Centre)

Citation and DOI:

Cabana particle library feature expansion, publication, and release CSED Computational Sciences and Engineering Division ORNL
The ExaMPM proxy application (built on Cabana) for the material point method (MPM) using the new load balancing capability of Cabana through the ALL library interface. Particle workload imbalance over time with multiple rebalancing frequencies (left), shown with single snapshot in time (right, marked with the vertical red line). Run by Stephan Schulz on 1 node of Summit (6 V100 GPU).

Summary: Performance portable solutions are critical in order to effectively use the broadening list of high-performance computing resources. Libraries such as Kokkos enable separation of concerns between application physics and hardware parallelism. In a similar spirit, Cabana provides infrastructure common to all particle applications to more effectively share development effort, further separate physics from computational bookkeeping, and allow focus on both domain science and improved performance. The feature updates in this release continue to make Cabana a useful library across many scientific domains, while the software updates and documentation upgrades continue to make it accessible to developers and users.