JPL Technical Report Server

Evaluating the efficacy of the cloud for cluster computation

Show simple item record Knight, David Shams, Khawaja Chang, George Soderstrom, Tom 2015-03-10T23:34:51Z 2015-03-10T23:34:51Z 2012-03-03
dc.identifier.citation 2012 IEEE Aerospace Conference, Big Sky, Montana, March 3-10, 2012 en_US
dc.identifier.clearanceno 12-0549
dc.description.abstract Computing requirements vary by industry, and it follows that NASA and other research organizations have computing demands that fall outside the mainstream. While cloud computing made rapid inroads for tasks such as powering web applications, performance issues on highly distributed tasks hindered early adoption for scientific computation. One venture to address this problem is Nebula, NASA's homegrown cloud project tasked with delivering science-quality cloud computing resources. However, another industry development is Amazon's high-performance computing (HPC) instances on Elastic Cloud Compute (EC2) that promises improved performance for cluster computation. This paper presents results from a series of benchmarks run on Amazon EC2 and discusses the efficacy of current commercial cloud technology for running scientific applications across a cluster. In particular, a 240-core cluster of cloud instances achieved 2 TFLOPS on High-Performance Linpack (HPL) at 70% of theoretical computational performance. The cluster's local network also demonstrated sub-100 μs inter-process latency with sustained inter-node throughput in excess of 8 Gbps. Beyond HPL, a real-world Hadoop image processing task from NASA's Lunar Mapping and Modeling Project (LMMP) was run on a 29 instance cluster to process lunar and Martian surface images with sizes on the order of tens of gigapixels. These results demonstrate that while not a rival of dedicated supercomputing clusters, commercial cloud technology is now a feasible option for moderately demanding scientific workloads. en_US
dc.description.sponsorship NASA/JPL en_US
dc.language.iso en_US en_US
dc.publisher Pasadena, CA : Jet Propulsion Laboratory, National Aeronautics and Space Administration, 2012 en_US
dc.subject Cloud Computing en_US
dc.subject Scientific Computing en_US
dc.title Evaluating the efficacy of the cloud for cluster computation en_US
dc.type Preprint en_US

Files in this item

This item appears in the following Collection(s)

Show simple item record



My Account