If you want to do big, serious science, you’ll need a serious machine. You know, like a giant water-cooled computer that’s 200,000 times more powerful than a top-of-the-line laptop and that sucks up enough energy to power 12,000 homes.
You’ll need Summit, a supercomputer nearing completion at the Oak Ridge National Laboratory in Tennessee. When it opens for business next year, it'll be the United States’ most powerful supercomputer and perhaps the most powerful in the world. Because as science gets bigger, so too must its machines, requiring ever more awesome engineering, both for the computer itself and the building that has to house it without melting. Modeling the astounding number of variables that affect climate change, for instance, is no task for desktop computers in labs. Some goes for genomics work and drug discovery and materials science. If it’s wildly complex, it’ll soon course through Summit’s circuits.
Summit will be five to 10 times more powerful than its predecessor, Oak Ridge’s Titan supercomputer, which will continue running its science for about a year after Summit comes online. (Not that there's anything wrong with Titan. It's just that at 5 years old, the machine is getting on in years by supercomputer standards.) But it’ll be pieced together in much the same way: cabinet after cabinet of so-called nodes. While each node for Titan, all 18,688 of them, consists of one CPU and one GPU, with Summit it'll be two CPUs working with six GPUs.
Think of the GPU as a turbocharger for the CPU in this relationship. While not all supercomputers use this setup, known as a heterogeneous architecture, those that do get a boost―each of the 4,600 nodes in Summit can manage 40 teraflops. So at peak performance, Summit will hit 200 petaflops, a petaflop being one million billion operations a second. "So we envision research teams using all of those GPUs on every single node when they run, that's sort of our mission as a facility," says Stephen McNally, operations manager.
Performing all those operations sucks up a lot of power and generates a ton of heat. That poses a daunting challenge for Heery, the company charged with preventing Summit from overheating and powering the building that houses it. Heery's piping in 20 megawatts of electricity (the supercomputer itself will run on 15 megawatts), enough juice to power a decent-sized city. “12,000 Southern homes with their air conditioners cranking would be roughly 20 megawatts of power,” says George Wellborn, senior associate at Heery. Luckily, Oak Ridge is hooked up to the Tennessee Valley Authority, which in Tennessee alone has a generating capacity of nearly 20,000 megawatts from 19 hydroelectric dams, two nuclear power plants, and too many other sources to get into here.
Another engineering pickle: Each of the supercomputer's 4,600 nodes needs to be cooled individually. Summit will use water. (Titan uses a refrigerant. You could also cool your electronics in a bath of mineral oil, if you were so inclined.) “Every one of those nodes is using a cold plate technology, where we're putting water through a cold plate that's directly on top,” says Jim Rogers, director for computing and facilities. “So 70 percent of the heat that's generated by this thing can be absorbed by that cold plate.”
Curiously, this isn’t super-chilled water―it’s a comfortable 70 degrees Fahrenheit. Why? Because if you drop the temperature too much, you’ll form dew, which is a great way to ruin a supercomputer. “You have to have higher flow rates to carry the heat away,” Rogers says (we’re talking a max flow of nearly 8,000 gallons per minute), “but that tradeoff is good in terms of energy efficiency and operating cost.”
Summit still has to ... summit some final steps before it can start crunching heavy-duty science. Its cabinets should all be installed by late October, then it will undergo a year of testing and debugging. But soon enough, one of the most impressive devices humankind has ever assembled will go node to node with the best supercomputers in the world.
