Confession. I once left my computer on during takeoff. I know I shouldn’t have. I really shouldn’t have. But it was acting up and half the time when I shut it down it wouldn’t turn back on again unless I plugged it in, so I left it on going through security in case the screeners wanted it turned on, and then I forgot, and… well, it’s not a very good excuse. And I really really will never do it again.
Imagine a world in which a car's performance is judged solely by the time it takes to go from 0 to 100 kilometers per hour, ignoring fuel efficiency and other metrics. This, in essence, is the state of supercomputing today, says a group of U.S. computer scientists. People today typically judge supercomputers in terms of their raw number-crunching power, for example by asking how many linear algebra problems they can solve in a second. But, the scientists argue, the lion's share of challenging supercomputing problems in the 2010s requires quick and efficient processing of petabyte and exabyte-size data sets. And good number crunchers are sometimes bad exascale sifters.
It's time, the researchers say, for high-performance computers to be rated not just in petaflops (quadrillions of floating-point operations per second) but also in "gigateps" (billions of traversed edges per second).
An "edge" here is a connection between two data points. For instance, when you buy Michael Belfiore's Department of Mad Scientists from Amazon.com, one edge is the link in Amazon's computer system between your user record and the Department of Mad Scientists database entry. One necessary but CPU-intensive job Amazon continually does is to draw connections between edges that enable it to say that 4 percent of customers who bought Belfiore's book also bought Alex Abella'sSoldiers of Reason and 3 percent bought John Edwards's The Geeks of War.
"What we're most interested in is being able to traverse the whole memory of the machine," says Richard Murphy, a senior researcher at Sandia National Laboratory, in Albuquerque, N.M. "There's no equivalent measure for these problems that's accepted industry-wide."
So Murphy and his colleagues from other U.S. national laboratories, academia, and industry have put together a benchmark they're calling the Graph 500. The name comes from the field of mathematics (graph theory) that the benchmark draws most heavily from. And the 500 is, Murphy says, an "aspirational" figure representing what they hope someday will be a "top 500" ratings list of the highest-performing supercomputers around the world, measured in gigateps instead of gigaflops.
The current biannual Top 500 supercomputers list recently made headlines whenChina's Tianhe-1A took the top position, coming in at 2.57 petaflops. The supercomputers on the list are ranked using a benchmark package of calculation speed tests called the High-Performance Linpack.
Crucially, Murphy says, the point of the Graph 500 is not to run a horse race on a new racetrack. Rather, he says, they've designed the benchmark to spur both researchers and industry toward mastering architectural problems of next-generation supercomputers. And the only way to know if you've solved those problems is for the industry to include those problems in its metrics.
In fact, by a Graph 500–type standard, supercomputers have actually been getting slower, says computer science and electrical engineering professor Peter Kogge of Notre Dame University. For the past 15 years, he says, every thousandfold increase in flops has brought with it a tenfold decrease in the memory accessible to each processor in each clock cycle. (For more on this problem, see Kogge's feature article in next month's issue of IEEE Spectrum.)
This means bigger and bigger supercomputers actually take longer and longer to access their memory. And for a problem like sifting through whole genomes or simulating the cerebral cortex, that means newer computers aren't always better.
"Big machines get embarrassingly bad gigateps results for their size," Kogge says.
Today only nine supercomputers have been rated in gigateps. The top machine, Argonne National Laboratory's IBM Blue Gene–based Intrepid, clocked in at 6.6 gigateps. But to score this high, Intrepid had to be scaled back to 20 percent of its normal size. (At full size, Intrepid ranks No. 13 on the conventional Top 500 list, at 0.46 petaflops.)
"I think Graph 500 is a far better measure for machines of the future than what we have now," Kogge says. Supercomputing, he says, needs benchmarks that measure performance across both memory and processing.
However, Jack Dongarra, professor of electrical engineering and computer science at the University of Tennessee and one of the developers of the Top 500 list, notes that the Graph 500 isn't the first new benchmark to challenge the ... Read more »
