Abstract
Wide-area file transfers are an integral part of several
High-Performance Computing (HPC) scenarios. Dedicated
network connections with high capacity, low loss rate and low
competing traffic, are increasingly being provisioned over current
HPC infrastructures to support such transfers. To gain insights
into these file transfers, we collected transfer rate measurements
for Lustre and xfs file systems between dedicated multi-core
servers over emulated 10 Gbps connections with round trip times
(rtt) in 0-366 ms range. Memory transfer throughput over these
connections is measured using iperf, and file IO throughput on
host systems is measured using xddprof. We consider two file
system configurations: Lustre over IB network and xfs over SSD
connected to PCI bus. Files are transferred using xdd across
these connections, and the transfer rates are measured, which
indicate the need to jointly optimize the connection and host
file IO parameters to achieve peak transfer rates. In particular,
these measurements indicate that (i) peak file transfer rate is
lower than peak connection and host IO throughput, in some
cases by as much as 50% or lower, (ii) xdd request sizes that
achieve peak throughput for host file IO do not necessarily lead
to peak file transfer rates, and (iii) parallelism in host IO and
TCP transport does not always improve the file transfer rates.
