Blog Post

It’s Black Friday, so who has the lowest cloud prices?

Who’s really got the cheapest cloud? The answer, alas, is “it depends.”

In today’s brave new IT world, there are few things more downright confusing than public cloud pricing. Every provider has a different way of T-shirt sizing infrastructure as well as pricing compute, memory, storage, bandwidth, OS, and support.

RBC Capital Markets recently advanced the cause of comparative cloud pricing by showing how a single metric — spend per GB of RAM — could be used to compare cloud providers. Though convenient, this method is not enough to figure out which is the lowest cost cloud provider for your particular application and workload needs.

To get to a more accurate pricing comparison, you would need to use a technique similar to the market basket pricing approach used in economics — i.e., a collection of components in a basket (e.g. storage, networking, RAM, CPU, support). But there is no one-size-fits-all cloud market basket when it comes to applications. This is because each app requires a different mix of infrastructure and services to meet its needs. For example, a typical medium-sized collaboration application might need eight app servers, four database servers, 5 TB of storage and 2,400 GB/month, while a medium-sized CRM app might need eight web servers, four app servers, four database servers, 3 TB of storage and 800 GB/month.

Since each cloud provider prices the individual infrastructure and services components differently (and changes price on each component differently and asynchronously), an accurate comparison of cloud provider pricing would need to be application-specific, workload-specific, and time-stamped.

Using an application- and workload-specific approach, we compared four major cloud providers (AWS, Google, IBM and Microsoft Azure) across nine applications. We assumed the application operated on a light workload and on optimized infrastructure. Infrastructure sizing was based on what we have seen as typical in customer deployments. We also included support with less than 1-hour SLA (anything else wouldn’t be enterprise-class).

We ran two scenarios: First, a medium-sized application infrastructure across these applications and, second, a large-sized infrastructure. All prices were current as of November 26, 2014 and included the most recent Google price cuts as well as the September introduction of new (D class) instance sizes at Microsoft Azure.

The results, showing approximate monthly pricing, are shown in the tables below. Note that the prices derived are for specific application deployments — your own mileage will vary depending on the specifics of your application and workload and the size of infrastructure needed.

Table 1: Provider comparison — Medium sized application infrastructure

Application Lowest 2nd Lowest 3rd Lowest
Business Intelligence Amazon $1,616 Microsoft $2,929 IBM $4,570
CDN Web App IBM $4,866 Amazon $5,071 Google $6,075
Collaboration Google $10,360 IBM $12,346 Amazon $12,713
Disaster Recovery IBM $1,126 Amazon $1,284 Google $1,924
Enterprise Portal App IBM $5,295 Amazon $5,429 Google $5,836
Enterprise Web App IBM $13,051 Microsoft $13,545 Amazon $13,828
HPC Amazon $3,291 Microsoft $6,232 IBM $8,493
Mobile Development Amazon $2,422 Google $2,480 IBM $2,655
OLTP IBM $2,543 Amazon $2,772 Google $3,397

We see that the lowest provider depends by application and there is no single provider that is lowest for all applications. The mean price difference between lowest and second lowest price was 25 percent,with a standard deviation of 33 points.

If we change the size of the application infrastructure, we get different comparison results:

Table 2: Provider comparison — Large sized application infrastructure

Application Lowest 2nd Lowest 3rd Lowest
Business Intelligence Amazon $6,469 Microsoft $8,867 IBM $18,531
CDN Web App Amazon $15,642 IBM $16,464 Microsoft $17,678
Collaboration Google $27,358 IBM $33,156 Amazon $33,495
Disaster Recovery IBM $4,756 Amazon $5,120 Google $6,496
Enterprise Portal App IBM $21,181 Amazon $21,417 Microsoft $22,138
Enterprise Web App Microsoft $44,652 IBM $45,252 Amazon $47,565
HPC Amazon $13,109 Microsoft $21,878 IBM $34,221
Mobile Development Google $8,163 Amazon $9,004 IBM $9,912
OLTP IBM $10,421 Microsoft $10,785 Amazon $11,016

What drives the difference in results between table 1 and table 2? In part, the answer is how each provider prices components (such as memory) as they scale in size, but it is also because of offering granularity. Granularity refers to the resolution of instance sizes available by a particular provider. The more granular the offering, the easier it is to match to your application sizing need without having to step up to the next highest instance level.

Granularity also helps explain the large differences in pricing between the lowest and second-lowest provider for some of the applications (e.g. HPC and BI). Both Amazon and Microsoft offer low-priced high compute and memory instances (lower than IBM). But Amazon wins because Microsoft’s lower instance resolution (i.e. less granular offering) requires customers to bump up to the next instance level to meet minimum performance requirements, thus raising the price.

IBM comes out ahead in many applications because it allows users to independently choose an instance’s CPU and memory at a granular level, making it easier to match the application needs.

Google comes out ahead in Collaboration applications because its high memory instances are a strong match for such applications and are competitively priced against Amazon and IBM.

However, price isn’t everything. We caution cloud users from just choosing the lowest cost provider for any given application. Cost is only one decision variable. Other important criteria are performance, location and features (e.g. security, VM scalability, file based storage, etc). A customer may decide to choose a more expensive cloud provider because they get higher performance or desired support, location, network security, etc.

At the end of the day, the real question should not be “which is the cheapest cloud?” but “which is the best-fit cloud for my application and business need?”

Praveen Asthana is chief marketing and strategy officer at Gravitant, a software company that enables IT organizations to become cloud service brokers. Thanks to Aaron Yan, pricing analyst at Gravitant, for crunching the numbers.

7 Responses to “It’s Black Friday, so who has the lowest cloud prices?”

  1. johnmartin1

    I appreciate the author’s effort to create these comparisons, but their usefulness is severely limited by using each vendor’s nominal resource units in the comparison. An AWS virtual compute unit is not equivalent to one from IBM or Azure. And oversubscribed RAM is not as useful as RAM that is not. And a storage network that delivers wide variations in IOPs is not as valuable as one that has better throughput or more consistent performance. And variations in the way each provider charges for elements like network can swamp any apparent savings on the compute and RAM. And so on.

    In order to come up with an actionable comparison, the author would need to start with a third-party performance test such as Cloud Harmony’s and then price application stacks that deliver equivalent performance, not just numerically equivalent vendor-defined components.

    • praveenasthana

      Thank you for the feedback. Fully agree that relative performance is also an important comparison metric. Indeed as pointed out in the blog post, we encourage a multi-dimensional comparison which includes relative performance and feature coverage (for example see http://tools.gravitant.com/cloud-compare/enterprise-web/select-vendor/?deployment=medium in which we look at effective IOPs and effective MBps). Sometimes it is worth skipping the lowest cost provider and paying more with higher performance.

      On your second point of incorporating price and performance directly, that is something we have looked at (starting with Cloud Harmony benchmarks as well as our own benchmarking); however one thing we have seen is that performance can vary quite a bit for a given provider (seemingly at random) depending on when the job is run (same run, 90 mins apart can give different performance numbers), not to mention which region etc. Indeed performance variability at a given provider is another valuable comparison metric. Some of this was discussed in this report by Gigaom research: http://research.gigaom.com/report/comparing-major-cloud-service-providers-virtual-processor-performance/

  2. Mahesh Kumar

    Well structured piece that goes beyond obvious pricing comparisons which tend to be (as the author states) at an ‘element level’. agree one has to look at what the cloud is being purposed for. the app stack, the type of application and its own unique needs dictate which vendor one wants to go with. What is interesting is the author is able to atleast directionally point to clouds that are suitable for specific applications (collaboration @google, IBM for many others).

    Would love a follow on article which compares the same application stacks and relative pricing across these clouds in say 6 months. Also would love to see how his data deviates with other inputs/restrictions (security, performance, storage etc)

  3. chrisjwilson

    I’m with Rich, how exactly are you deriving these numbers and what, in this context, is an “application”? Your research is not reproducible without those details. I recently went through a similar exercise for Coursera’s Data Products course using a linear model (http://bit.ly/1HP6Gqx) and Stephen O’Grady at Redmonk has published as-is and historical price comparisons – which is quite valuable. Disappointed Gigaom would publish this.

    • praveenasthana

      For each application category, a list of instances were generated to meet the infrastructure requirements for sample applications within the category (as seen in our customer deployments). Please find the sample app infrastructure details at: (http://bit.ly/1rOwLfi). The list of instances were determined using a lowest-cost and minimum-spec approach: the type of instance was chosen if it provided the required specs (per the app template) by a 15% margin of error and produced the lowest cost for the on-demand US-East region. Storage options were paired with the respective storage types (block vs. file) and networking prices used the provider-specific $/GB/month conversions.

      Below is an example outlining the pricing methodology for the medium Mobile Development application category and AWS pricing:

      Mobile Development Template – compute, storage, and network requirements:
      9x – dev servers – 4 cores, 16GB memory, 100GB local, linux, 97% util
      4x – test servers – 2 cores, 8GB memory, 100GB local, linux, 92% util
      100GB cloud storage, 200GB/month inbound, 200 GB/month outbound

      AWS ($2422/month):
      9x – m3.xlarge – 4 cores, 15GiB* memory, 32GB local, linux @ $1791/month
      4x – m3.large – 2 cores, 7.5GiB* memory, 40GB local, linux @ $377/month
      128GB EBS & 100GB S3 @ $10/month
      200 GB/month outbound @ $24/month
      AWS Business Support @ $220/month
      *1GiB to 1.07374GB

      Prices were calculated independently and then validated with pricing calculators (http://bit.ly/1rP9Xfr).

      This type of calculation was repeated for each of the other providers, mapping the above template infrastructure to a design specific for each provider. For example, for Google Compute instance assignments, we used:
      9x – n1-standard-4 – 4 cores, 15GB memory, linux
      4x – n1-standard-2 – 2 cores, 7.5GB memory, linux

      Aaron and I are happy to discuss/compare notes off line.