Skip to main content
SHARE
Publication

Creating Continuous Integration Infrastructure for Software Development on DOE HPC Systems

by Ryan M Adamson, Paul J Bryant, Erik Palmer, Ryan C Prout
Publication Type
Journal
Journal Name
Computing in Science & Engineering
Publication Date
Page Numbers
1 to 9
Volume
TBD
Issue
TBD

The Exascale Computing Project (ECP) software deployment effort developed and advanced DevOps capabilities. One goal was to enable robust continuous integration (CI) workflows that span the protected high performance computing (HPC) environments found within many of the Department of Energy’s (DOE) national laboratories. This article highlights several challenges encountered with enabling automation, such as charging models for CI jobs, and meeting individualized security requirements that revolve around strongly associating running code with a human identity. It also describes how the Jacamar CI tool evolved to meet latter requirements and became a key aspect of the solutions currently offered. Derived from this experience, we offer a conceptual framework for understanding current and future CI challenges at DOE facilities and offer suggestions for long-term solutions.