DARPA Challenge: billion-way parallelism, 1 PFLOPS system in one rack, 57 kW max power

By Gary Stiehr, June 28, 2009 2:37 pm
DOE's Roadrunner Supercomputer

DOE's Roadrunner Supercomputer

DARPA has issued an RFI (pdf) to help enable what they are calling Ubiquitous High Performance Computing (UHPC).  According to the June 2009 TOP500 supercomputer list, the fastest supercomputer available, Roadrunner, runs at just over 1 PFLOP.  It uses around 2.5 million watts of electricity and requires around 278 racks of equipment [1].

DARPA would like to fit the same computational power into one air-cooled rack and use no more than 57,000 watts (including cooling).  That’s 100% of Roadrunner’s computational power in 0.4% of the space using 2% of the electrical power. Also, while the most energy efficient system now achieves 536 Mflops/watt [2], DARPA is looking for 50 Gflops/watt.

What’s more, is that they would like to minimize the overhead associated with thousand-way to billion-way parallelism.  Why billion-way parallelism?  I suppose this implies an anticipation of systems containing billions of execution units.  This  may not be unreasonable.  For example, take a look at the proposed Sequoia supercomputer, which is proposed to include one million cores.

Beyond these astounding requirements, there are also requests for a “Self Aware OS” that is introspective, goal-oriented, adaptive, self-healing and approximate.  I’d recommend reading page eight of the RFI above for more details.  The hope is that the system will be able to continue operations in the face of failures and “attack” (see page 4 of RFI).

Well, while the OS and application capabilities will be huge challenges, the restrictions put on the physical aspects of the hardware are also challenging.  With GPUs and Cell processors leading to increased computations per watt, perhaps we may be able to significantly improve overall system power efficiencies.  In addition, DARPA is looking for this to take place potentially in 9 years (proposals are due by July 27, 2009) if it is feasible.  With top supercomputers sometimes becoming more powerful than the 500 most powerful supercomputers combined from four years prior, we can definitely see overall computational ability increase quickly but this doesn’t necessarily translate into the density and energy efficiencies.

Aside from the RFI above, you can read more here or here.  Also, thanks to @HPC_Guru from whom I first heard about this RFI.

The Fastest Supercomputer Became Faster Than the Top 500 Combined Four Years Prior

By Gary Stiehr, June 25, 2009 12:11 am

TOP500 Performance over time

After reading a perspective of the latest TOP500 Supercomputer List from @Chris P_Intel I took another look at the progress of the systems on the list shown above.  The June 2009 list just released puts the RoadRunner supercomputer in the number one spot with 1105 TFLOPS.  In June 2004, just five years ago, all 500 supercomputers combined summed to 813 TFLOPS, with the most powerful single system being the Earth-Simulator at 36 TFLOPS.  So in just five years, a single supercomputer has become more powerful than the 500 most powerful supercomputers from June 2004.

Upon taking a closer look, I saw that RoadRunner was actually in the number one spot in June 2008 at 1026 TFLOPS.  So the top supercomputer on the list in June 2008  was actually faster than all of the top 500 supercomputers combined from four years prior!

Ok, and going back to November 2005, it seems that the #1 system may have been more powerful than the sum of the top 500 supercomputers in November 2002.  So perhaps we are down to three years…I haven’t verified exact numbers though.  Has anyone officially tracked the record for how quickly the #1 supercomputer on the TOP500 list had achieved the performance of all of the supercomputers on a previous TOP500 list?

147,000 Processors Used for Atom-by-Atom Simulation of Nanoscale Transistor

By Gary Stiehr, June 23, 2009 1:29 am

Using 147,000 processors for 15 minutes from the Jaguar system (a Cray XT5) at the Oak Ridge Leadership Computing Facility, “a simulation of electrical current moving through a futuristic electronic transistor has been modeled atom-by-atom in less than 15 minutes by Purdue University researchers.”

“Professor Klimeck and his colleague have demonstrated the unique transformational scientific opportunity that comes from scaling a science application to fully exploit the capabilities of petascale systems like the Cray XT5 at the Oak Ridge Leadership Computing Facility,” Kothe says.

Freely available nanoelectrics software (OMEN) was used from nanoHUB.org to do this simulation.  I am curious about how else this could be applied.  What other nanostructures might we be able to simulate in this way?

For more information, see the source article.

Illumina Offers $48,000 Personal Genome Sequencing–How Will Data be Handled?

By Gary Stiehr, June 12, 2009 11:03 pm
dna_transcription

A depiction of the structure of DNA

Illumina will offer a service to sequence a person’s genome for $48,000 (a doctor’s prescription is required).  Note that this is only the sequencing and not the actual analysis of that sequence data.  The consumer must choose from a few different providers to do the actual analysis of the genome sequence data.  Currently, the representation of a human genome as Illumina is proposing (30-fold coverage of your DNA sequence) would require the transfer of terabytes of data to the company doing the analysis.  Of course, there are various parts to “analysis” so depending on where Illumina stops and the other companies take over, this actually could be a lot less data (e.g., gigabytes).

So this raises at least a couple of possible challenges for Illumina:

  • How will the data be transferred?
  • How will the data be secured?

Transferring the Data

One could see that data transfer of on the order terabytes of data would not be a problem if the turnaround time is long enough.  Although if the service becomes more and more popular, scaling may be a problem (or at least synchronizing network abilities with analysis providers).  Nevertheless, will Illumina establish encrypted network connections with the consumer’s/doctor’s chosen analysis provider?  Will they transfer the data encrypted on external hard drives?  If on external hard drives, how will tracking of the multiple pieces be tracked?

Securing the Data

I’m assuming the security/encryption questions may have answers based off of current electronic health records implementations although I’m not sure if electronic patient information systems are typically interconnected between different health care organizations.  That is, aren’t these systems usually secured/confined within the network of a particular health care organization?  If it is placed on external hard drives and shipped, would the encryption of terabytes of data per patient be challenging?

Using HPC to Understand Swine Flu

By Gary Stiehr, June 8, 2009 11:29 pm

Here is another great example of how an HPC site can function as a versatile resource for a wide variety of problem domains.  A priority queue was setup on TACC’s Ranger cluster to provide 2,000 to 3,000 processors for two weeks to allow a team to assess the way in which the underlying structure of the Swine Flue virus (H1N1A) has or could mutate and lead to drug resistance.  With this data, “they believe it will be possible to intelligently design a drug or vaccine that can’t be resisted.”

This still from a Quicktime movie represents a view of the drug buried in the binding pocket of the A/H1N1 neuraminidase protein. The animation also shows a 3D surface view of a neuraminidase protein and footage from the actual drug binding simulation.

This still from a Quicktime movie represents a view of the drug buried in the binding pocket of the A/H1N1 neuraminidase protein. The animation also shows a 3D surface view of a neuraminidase protein and footage from the actual drug binding simulation.

From the article cited below:

Supercomputers routinely assist in emergency weather forecasting, earthquake predictions, and epidemiological research. Now, says Schulten, they are proving their usefulness in biomedical crises.

“It’s a historic moment,” he said. “For the first time these supercomputers are being used for emergency situations that require a close look with a computational tool in order to shape our strategy.”

Find more details at Inside the Swine Flu Virus (found via this HPCwire article).

Panorama theme by Themocracy