Comparing major cloud-service providers: virtual processor performance

1. Summary

Cloud computing continues to play an increasingly important role for IT organizations in 2014. In partnership with Cloud Spectator, Gigaom Research conducted research to determine how three major cloud providers — Amazon EC2, Microsoft Azure and Rackspace Cloud — perform with regards to processor and memory.

The three providers were tested using Geekbench 3 standardized tests over a three-day period of time. Individual test results proved a variability of as much as 25 percent. The complex of variables between cloud providers beyond just processor and memory can contribute heavily to this variability in results. Tests were conducted over a three-day period to normalize the results. With each provider, different configurations of each solution were tested to demonstrate the variability and value between configurations. In addition, the provider’s updated hardware configurations were used to ensure currency.

It should be noted that the tests in this research focused on the performance of processor and memory only. There are a number of other variables (network, architecture, etc) that can vary the results for a specific application or service.

Key findings in this report include:

• Rackspace led in performance, though this was offset by a higher variability.
• Microsoft and Amazon showed stable results, but at a lower-performance rate.
• Rackspace demonstrated a 2x value over Amazon and Microsoft. Value is defined as the ratio of cost and performance.
• Amazon’s Reserve Instances provide an alternative that brings the cost/performance ratio closer to that of Rackspace. Depending on the specific use case, one solution may be more appropriate than the other.