ad info  technology > computing
    Editions | myCNN | Video | Audio | Headline News Brief | Feedback  




Consumer group: Online privacy protections fall short

Guide to a wired Super Bowl

Debate opens on making e-commerce law consistent



More than 11,000 killed in India quake

Mideast negotiators want to continue talks after Israeli elections


4:30pm ET, 4/16










CNN Websites
Networks image

Supercomputers on the cheap


April 13, 2000
Web posted at: 10:16 a.m. EDT (1416 GMT)

(IDG) -- What do you get when you cross off-the-shelf hardware and Linux? You get a powerful, highly standardized supercomputer called Beowulf. And you get it cheap.

How cheap? By supercomputing standards very cheap indeed. The team at the Los Alamos National Laboratory built Avalon, the 256th fastest supercomputer in the world for $152,000. By their estimate, a conventional supercomputer with equivalent performance would cost about $1 million today. Or, if $152,000 is too much for you, you can always do what the researchers at the University of Tennessee and Oak Ridge National Laboratories did, and build a Linux supercomputer for free from old 486 boxes that some government facilities were throwing away.

Even if you purchase a system from a vendor, Linux-based supercomputers are still cheap. "A 16-node cluster is in the $25,000 range," says Tom Leingerber, the sales manager at Aspen Systems, a vendor of Linux-based supercomputers in Wheat Ridge, Colo. "In today's world you can get supercomputer power for workstation prices, and do tasks that used to take weeks in days or hours."

  Supercomputers for the rest of us
  Blue Gene, a super computer for the history books
  Supercomputer sale draws fire from Congress
  Why the mainframe won't die
  Reviews & in-depth info at
  Questions about computers? Let's editors help you
  Subscribe to's free daily newsletter for IT leaders
  Search in 12 languages
  News Radio
  * Fusion audio primers
  * Computerworld Minute

The key to this kind of bargain-basement supercomputing is tying standard PCs together with Linux. The combination of low-cost, universally available source code and the open source development model that Linux offers makes it practical to build high-performance computers cheaply. Linux also offers a way around the software bottleneck that has plagued supercomputers for years. (Although there are cognate efforts with Windows NT, they aren't nearly as far along. NT isn't modular, isn't nearly as scalable, and certainly isn't as open as Linux, all factors that hinder the development of massively parallel Windows systems.)

Don't think that using standard processors is limiting. The current supercomputer champ, ASCI Red at Sandia National Labs, uses 9,632 Intel processors to crank out a 2379.6 on the standard Linpack Rmax benchmark. (See Resources for a link to information about Linpack.) That score's a lot higher than the 48.6 that Avalon achieved using 140 Alpha machines, but ASCI Red's builders didn't create their system for $152,000 either.

The reason these systems are so effective is that there are a great many very big, very complicated problems that naturally break down into a bunch of iterations of the same, much simpler, problem. That describes everything from forecasting the weather to doing computer animation. While the parts of the problem may be mind-numbingly complex by human standards, the processor in even a $500 PC can handle them without breaking a sweat. If can you divide the project among multiple processors and let them work on their parts at the same time, you can reduce enormously the time it takes to get a solution. Putting ten processors on the problem doesn't mean you finish in one-tenth the time -- there is overhead, after all. But if everything is well designed and carefully optimized, you can come pretty close.

Not every functionality is amenable to massive parallelism. A massively parallel word processor, for example, would be just plain silly. Likewise, there are some problems that, by their nature, don't divide into chunks that you can parcel out among processors. And there are others whose chunks are too large to run on a single microprocessor, or even an SMP (symmetrical multiprocessing) cluster. Still, there is an awful lot of territory out there that can be covered by Beowulf-type computers running Linux.

People are exploring that territory, too. A Beowulf-class computer is now a recognized designation in the supercomputer community, and the list of such computers is up in the hundreds and growing every day.

What is Beowulf?

Beowulf isn't a piece of software. It's an architecture, or a family of architectures, for building massively parallel systems out of commercially available pieces. "To say 'Beowulf supercomputer' is something of a misnomer," says Curtis Regetin, a senior engineer at Penguin Computing, a maker of Linux systems in San Francisco. "Beowulf was a [super] computer with one of the first Linux-based architectures." For example, Regetin points out, Beowulf-class computers use a lot of different methods to interconnect their processors. There is also a wide variation in the amount of local disk storage, the amount of memory, and even the number of copies of the operating system used. Still, the term "Beowulf supercomputer" is commonly used to refer to all these systems.

Basically, a Beowulf system consists of one or more master nodes, which are fully configured systems with I/O devices, and a lot of slave nodes, which are usually just motherboards or PCs with no display, no keyboard, and minimal or no hard disk storage. The master node divides the problem and parcels the pieces out to the slave nodes; it then gets the results back and does any final computation needed to combine the results. It also takes care of housekeeping and other chores for the computer. Communication is done by passing messages. Because the nodes are not closely coupled and do not share memory, they can be heterogeneous. You can mix 486 systems, Pentium systems, and systems using the Compaq Alpha processor (or anything else that runs Linux) in the same computer. If you want to be fancy, you can mount the nodes in racks so the result looks like a "real" supercomputer. Many Beowulf users are too cheap to spring for rack-mounted computers, so they just fill a room with PC boxes. For example, the creators of Avalon saved about $20,000 by forgoing the rack mounting for its Alpha workstations.

By convention, Beowulf systems are divided into Class I and Class II machines. A Class I Beowulf system passes what is called the "Computer Shopper test." That is, it contains nothing but standard components you can buy off the pages of a magazine from at least three different sources each. A Class II system uses customized components for some jobs. A customized component is defined as one that's available from fewer than three suppliers. There's no reason you couldn't develop custom chips for use in a Beowulf class supercomputer, but no one does so because it's just not worthwhile for most applications.

Although many Beowulf systems run ordinary Linux distributions on their nodes, you do need some additional pieces. One is a messaging application programming interface with the right characteristics. Currently the two most common ones are MPI (Message Passing Interface) and PVM (Parallel Virtual Machine).

If you're the adventurous type, or if your budget is really cramped, you can always build your own Beowulf system. (See sidebar.) The mechanics of putting together a Beowulf-type computer aren't too challenging, since it's basically a bunch of off-the-shelf motherboards with local memory, connected with the appropriate networking cards.

Beowulf being Linux, and Linux being Linux, there are a lot of sources of information and advice to help you set up a Beowulf system. (See Resources.) This project definitely isn't for someone who gets sweaty palms from partitioning a hard disk to support Linux and Windows simultaneously, but it's not quantum mechanics either. (Rocket science, maybe, but not quantum mechanics.)

However, if you don't want to roll your own, a number of companies will sell you the hardware and work with you to configure a Beowulf system to your needs. Currently more than a dozen companies offer Beowulf systems commercially.

IBM to announce industry's first copper-based supercomputer
February 9, 2000
Intel to showcase 1GHz 'Coppermine' Pentium III
February 1, 2000
A supercomputer in every garage?
January 20, 2000
Big Blue's SMASHing Blue Gene
December 7, 1999
A $1000 supercomputer?
June 15, 1999
Open-source supercomputer beats Cray
March 16, 1999
PC prices continue to freefall
March 23, 1999

Blue Gene, a super computer for the history books
Supercomputers for the rest of us
(PC World)
IBM builds Linux supercluster
(Network World Fusion)
Supercomputer sale draws fire from Congress
China unveils new supercomputer
(Computerworld Hong Kong)
Toyota acquires world's most advanced supercomputer
Why the mainframe won't die
Mac G4 stopped at borders
(Computerworld Hong Kong)

The Avalon Beowulf computer:
The Top 500 Supercomputer List
An introduction and overview of Beowulf:
The Stone Soupercomputer (built for free out of discarded 486s)

Note: Pages will open in a new browser window
External sites are not endorsed by CNN Interactive.


Back to the top   © 2001 Cable News Network. All Rights Reserved.
Terms under which this service is provided to you.
Read our privacy guidelines.