Blog Post

Amazon's Cloud Gets a Supercomputing Cluster

Can Amazon's cloud beat IBM's Cray's Jaguar supercomputer?

Amazon (s amzn) today said it would now also offer high-performance computing (HPC) through Cluster Compute Instances, which use more powerful processors and clustered nodes to help get around some of the latency issues associated with distributed computing. HPC has long taken advantage of many nodes working in tandem, but has typically connected those processors using Infiniband or low-latency networks. However, as the need for HPC becomes more popular, and the next generation of faster supercomputers grows more difficult to build (GigaOM Pro sub req’d), the cloud may offer some hope.

The rise of cloud computing owes a bit to HPC, but for the most part demanding workloads couldn’t take advantage of the cost benefits and flexibility associated with the cloud because of the latency within the cloud and the cost and time associated with moving terabytes of data back and forth from a lab to the cloud. Amazon has solved the data problem by using the postal service and also by acting as a free repository for some types of scientifically valuable information. From the Amazon release:

Cluster Compute Instances can be grouped as cluster using a “cluster placement group” to indicate that these are instances that require low-latency, high bandwidth communication. When instances are placed in a cluster they have access to low latency, non-blocking 10 Gbps networking when communicating [with] the other instances in the cluster. Next, Cluster Compute Instances are specified down to the processor type so developers can squeeze optimal performance out of them using compiler architecture-specific optimizations. At launch Cluster Computer Instances for Amazon EC2 will have 2 Intel Xeon X5570 (also known as quad core i7 or Nehalem) processors.

Since enterprises are also turning to HPC for certain jobs, from analytics to making movies, it seems that Amazon may also help draw those jobs into the cloud. Of course, there are companies out there providing access to an actual supercomputer in an on-demand fashion, but Amazon’s decision could shake up the world of HPC. If Amazon succeeds, it proves the value of offering different hardware platforms for different workloads in a compute cloud. Perhaps we’ll see other heterogeneous cloud architectures follow suit.

7 Responses to “Amazon's Cloud Gets a Supercomputing Cluster”

    • Hey Derrick –

      Summer inserted a delay in my response, but here’s my quick take.

      Univa was the first to test and validate CCI in a private beta. Amazon EC2 is one of the ‘targets’ we support, but not the only one. We abstract where and how an application (or cluster) is provisioned – from bare metal to VMware ESXi, Oracle VM, Amazon EC2 (including CCI), Rackspace or GoGrid.

      This allows us to automate the addition of server “resources” from multiple clouds to an internal cluster simultaneously sharing the same network address space.

      What Amazon has done is upped the ante in the available functionality and performance of IaaS, that, as a result, is now suitable for a wider array of applications. If anything this helps Univa and the entire industry take advantage of Cloud. Escalating functionality gains – from both service providers and software companies – benefits the users directly.

      However, users still require management software that creates the repeatability and seamlessly manages the migration of workload to the appropriate resource, regardless of where it is.

      We applaud Amazon.

      Hope this helps….