Skip to main content
SHARE
Publication

High-Performance data flows using analytical models and measurements ...

Publication Type
Conference Paper
Publication Date
Conference Name
Workshop on Modeling & Simulation of Systems and Applications
Conference Location
Seattle, Washington, United States of America
Conference Date
-

The combination of analytical models and measurements provide practical configurations and parameters to achieve high data transport rates: (a) buffer sizes and number of parallel streams for improved memory and file transfer rates, (b) Hamilton and Scalable TCP congestion control modules for memory transfers in place of default CUBIC, and (c) direct IO mode for Lustre file systems for wide-area transfers. Conventional parameter selection using full sweeps is impractical in many cases since it takes months. By exploiting the unimodality of throughput profiles, we developed the d-w method that significantly reduces the number of measurements needed for parameter identification. This heuristic method was effective in practice in reducing the measurements by about 90% for Lustre and XFS file transfers.