In Defense of Custom Computing

The Top 500 List of supercomputers has quite a number of Intel CPU machines running Linux on the list. These are considered commodity supercomputers, even though they use custom backplanes and highly modified Linux kernels.

But several of the Top 10 do not meet that criteria, including the 3 IBM BlueGene systems that have custom chips. But the most interesting to me is the NEC Earth Simulator. This machine is well off the Top 10, in mere 31st place, but the thing that jumps out when you look at the chart is that it only has 1280 processors. In the Top 100, there are no other machines with fewer than 2048 processors, and most of the machines with under 5000 processors are based on the IBM PowerComputing architecture.

Simply dividing Rmax (which is in tflops) by the # of processors, we get 0.95 for the Earth Simulator, or about 95gflops per CPU. The #1 machine, Cray's "Jaguar" XT system, is 0.0078 or about 780mflops per CPU. Though there are other conflating factors in supercomputer design that make a CPU-to-CPU comparison difficult, a rough estimate of 100 times faster per CPU ought to pique interest in the NEC architecture. But it hasn't.

The Earth Simulator is the only machine in the Top 100 running the NEC Vector architecture and Super-UX operating system. The Intel-Linux pairing is quite cost competitive, and therefore often wins out on bids. But this is still the main advantage of Intel's commodity CISC architecture. The per-CPU compute rate is still a lot better when R&D dollars are applied to other computing architectures. By always going with the cheap solution, as an industry we're holding ourselves back. We've been stuck with the same Intel x86-ish architecture now for a while. RISC machines officially left the desktop mainstream in 2006 when Apple dumped the PowerPC architecture, but x86 dominance of the chip market has been a reality for a while. As of 2009, Intel has about 80% of the CPU market (including the mobile market, according to TGdaily), and its biggest rival is x86 clone maker AMD.

Most people don't need more computing power, more efficient CPUs, and less bloated / more compute efficient operating systems. But for those of us who do, it's a shame that so few resources are going into RISC CPU, whole-system architecture, and OS R&D in favor of trying to solve the problem with hacks on top of commodity systems. It's not like the industry is at a standstill, far from it. And sometimes those hacks work, which has all sorts of interesting results. But just imagine a 224162 CPU Earth Simulator instead of a 224162 Intel-based Cray XT, and what kinds of complex physics and earth science computations that could do.