A delicious combination of high-energy physicists, engineers, and computer scientists from Caltech and the University of Victoria have broken the world record for sustained, computer-to-computer transfer over a network. Between the SuperComputing 2011 (SC11) convention in Seattle and the University of Victoria Computer Centre, Canada — a distance of 134 miles (217km) — a transfer rate of 186 gigabits per second was achieved over a 100Gbps bidirectional fiber optic link; 98Gbps in one direction, 88Gbps in the other.
At first blush, when you remember that scientists have transferred terabits per second over fiber optics, you might question the veracity of this world record. In those cases, though, the transfers were done over private networks under laboratory/testbed conditions — Caltech and the University of Victoria set their virtual land speed record over a standard, commercially-available 100Gbps link. Furthermore, the scientists didn’t simply shoot data down the pipe and into the digital ether, which is relatively easy — there were computers on either end of the link!
At SC11, Caltech employed 13 computer servers, equipped with a variety of PCIe 2.0 and 3.0 solid-state drives and 40Gbps LAN connections; on the Canadian side, University of Victoria used 10 Dell servers each with 10Gbps Ethernet links and six SSDs. At its peak, the team achieved a disk to disk transfer rate of 60Gbps — around 7.5 gigabytes per second. The 186Gbps record (23.25 gigabytes per second) was a memory-to-memory transfer between the servers. The max per-computer speed was 35Gbps, performed by one of the PCIe 3.0 servers. The 60 and 186Gbps transfers, incidentally, were sustained for no less than 11 hours. For a complete listing of the hardware and infrastructure used, hit up the Caltech SuperComputing 2011 site. There’s an overly-dramatic video demonstrating the link-up at the bottom of this story.
By this point you are probably wondering what all of those gigabits are actually for. Seattleites could use the connection as the mother of all cross-border digital smuggling routes — you could transfer 100,000 Blu-ray rips per day — or, more realistically, it could be used by scientists and physicists to share vast amounts of raw data, like that collected by the CMS and ATLAS detectors at CERN’s Large Hadron Collider. So far, hundreds of petabytes of data has been created by the LHC, but the only way for scientists to get a hold of it is by downloading it across the web (or loading up a van with terabyte tapes and driving it across the world). Currently, the fastest links are via Internet2 — but even that can’t sustain more than 10Gbps.
The fact is, many large-scale science experiments produce more data than a single institution can handle — and the only way to get around that is by sharing the load. The link demonstrated by Caltech and the University of California, beyond a promising glimpse of future at-home internet connections, directly improves our chance of discovering awesome things like the Higgs boson.
Read more at Caltech
[Image credit: Chrys Omori]