Abstract
As HPC systems and applications get bigger and more complex, we
are approaching an era in which resiliency and run-time elasticity concerns become
paramount. We offer a building block for an alternative resiliency approach
in which computations will be able to make progress while components fail, in
addition to enabling a dynamic set of nodes throughout a computation lifetime.
The core of our solution is a hierarchical scalable membership service providing
eventual consistency semantics. An attribute replication service is used for
hierarchy organization, and is exposed to external applications. Our solution is
based on P2P technologies and provides resiliency and elastic runtime support at
ultra large scales. Resulting middleware is general purpose while exploiting HPC
platform unique features and architecture. We have implemented and tested this
system on BlueGene/P with Linux, and using worst-case analysis, evaluated the
service scalability as effective for up to 1M nodes.
