Abstract
Global Address Space (GAS) programming models are attractive because they
retain the easy-to-use addressing model that is the characteristic of
shared-memory style load and store operations. The scalability of GAS models
depends directly on the design and implementation of runtime libraries on the
targeted platforms. In this paper, we examine the memory requirement of a
popular GAS run-time library, Aggregate Remote Memory Copy Interface (ARMCI)
on petascale Cray XT 5 systems. Then we describe a new technique, cooperative
server clustering, that enhances the memory scalability of ARMCI communication
servers. In cooperative server clustering, ARMCI servers are organized into
clusters, and cooperatively process incoming communication requests among
them. A request intervention scheme is also designed to expedite the return
of responses to the initiating processes. Our experimental results
demonstrate that, with very little impact on ARMCI communication latency and
bandwidth, cooperative server clustering is able to significantly reduce the
memory requirement of ARMCI communication servers, thereby enabling highly
scalable scientific applications. In particular, it dramatically reduces the
total execution time of a scientific application, NWChem, by 45% on 2400
processes.
