7 Comments

Summary:

Amazon today said it would now also offer high-performance computing through Cluster Compute Instances, which use more powerful processors and clustered nodes to help get around some of the latency issues associated with distributed computing. Amazon could advance HPC while also promoting specialized clouds.

Can Amazon's cloud beat <del >IBM's</del> Cray's Jaguar supercomputer?

Amazon today said it would now also offer high-performance computing (HPC) through Cluster Compute Instances, which use more powerful processors and clustered nodes to help get around some of the latency issues associated with distributed computing. HPC has long taken advantage of many nodes working in tandem, but has typically connected those processors using Infiniband or low-latency networks. However, as the need for HPC becomes more popular, and the next generation of faster supercomputers grows more difficult to build (GigaOM Pro sub req’d), the cloud may offer some hope.

The rise of cloud computing owes a bit to HPC, but for the most part demanding workloads couldn’t take advantage of the cost benefits and flexibility associated with the cloud because of the latency within the cloud and the cost and time associated with moving terabytes of data back and forth from a lab to the cloud. Amazon has solved the data problem by using the postal service and also by acting as a free repository for some types of scientifically valuable information. From the Amazon release:

Cluster Compute Instances can be grouped as cluster using a “cluster placement group” to indicate that these are instances that require low-latency, high bandwidth communication. When instances are placed in a cluster they have access to low latency, non-blocking 10 Gbps networking when communicating [with] the other instances in the cluster. Next, Cluster Compute Instances are specified down to the processor type so developers can squeeze optimal performance out of them using compiler architecture-specific optimizations. At launch Cluster Computer Instances for Amazon EC2 will have 2 Intel Xeon X5570 (also known as quad core i7 or Nehalem) processors.

Since enterprises are also turning to HPC for certain jobs, from analytics to making movies, it seems that Amazon may also help draw those jobs into the cloud. Of course, there are companies out there providing access to an actual supercomputer in an on-demand fashion, but Amazon’s decision could shake up the world of HPC. If Amazon succeeds, it proves the value of offering different hardware platforms for different workloads in a compute cloud. Perhaps we’ll see other heterogeneous cloud architectures follow suit.

  1. Jaguar is a Cray/Sun supercomputer… not IBM. IBM’s fastest HPC is Roadrunner….

    Share
    1. Serge, you’re right and I fixed. I was in too much of a hurry with too little coffee.

      Share
  2. I wonder how this affects the HPC vendors (e.g., Univa UD) selling EC2-centric cluster management products.

    Share
    1. Hey Derrick -

      Summer inserted a delay in my response, but here’s my quick take.

      Univa was the first to test and validate CCI in a private beta. Amazon EC2 is one of the ‘targets’ we support, but not the only one. We abstract where and how an application (or cluster) is provisioned – from bare metal to VMware ESXi, Oracle VM, Amazon EC2 (including CCI), Rackspace or GoGrid.

      This allows us to automate the addition of server “resources” from multiple clouds to an internal cluster simultaneously sharing the same network address space.

      What Amazon has done is upped the ante in the available functionality and performance of IaaS, that, as a result, is now suitable for a wider array of applications. If anything this helps Univa and the entire industry take advantage of Cloud. Escalating functionality gains – from both service providers and software companies – benefits the users directly.

      However, users still require management software that creates the repeatability and seamlessly manages the migration of workload to the appropriate resource, regardless of where it is.

      We applaud Amazon.

      Hope this helps….

      -gt

      Share
  3. [...] portfolio by offering servers that will run graphics processors. The move comes on the heels of AWS releasing its Cluster Compute Instances in July, and validates the idea that specialized hardware may be better suited for certain types of [...]

    Share
  4. [...] Amazon’s Cloud Gets a Supercomputing Cluster [...]

    Share
  5. [...] Amazon’s Cloud Gets a Supercomputing Cluster [...]

    Share

Comments have been disabled for this post