12 Comments

Summary:

In the wake of CloudStack’s announcement, Cloudscaling’s CTO, Randy Bias, takes issue with its claims of “AWS compatibility” and “true Amazon-style architecture.” According to Bias, “no one should be under the illusion that CloudStack is more AWS/Amazon compatible than any other open source cloud software.”

Gilad_Moving to The Cloud Part 2_brain image

The hubbub around Tuesday’s Citrix announcement has died down a bit, but I’d like to highlight one particular area of confusion: the cavalier use of phrases like “AWS compatibility” and “Amazon-style architectures.” These terms have been used interchangeably, but are they really the same thing?

As part of the Citrix announcement, Sameer Dholakia, their general manager of cloud platforms said:

….we believe the biggest winners in the Cloud Era will be clouds built on a platform that is designed from the ground up with a true Amazon-style architecture, proven at scale in real production clouds, compatible with the Amazon architecture and fully committed to open source. With the significant momentum CloudStack has gained….it is the only cloud platform on the market that even comes close to meeting these requirements.”

Of course, this is a shot across the bows for VMware, but most pundits, bloggers, and journalists focused, somewhat rightfully, on what it will mean for OpenStack.

But what does this statement mean? How much is real, and how much is marketing-speak? What does it mean for VMware and OpenStack? What does it do for Citrix’s strategy?

Before digging into these questions, I want to make clear that I am absolutely biased. Cloudscaling, the company I cofounded, is a longstanding member of the OpenStack community, and the company’s Open Cloud OS is based on OpenStack software.

I’m also in a better position to understand the reality here. Working with KT in 2010 and 2011, Cloudscaling designed and built one of the largest CloudStack deployments to date. Working with a global top 10 carrier in 2011 and 2012, Cloudscaling helped them kick off one of the largest OpenStack deployments to date.

Parsing the quote

Citrix makes several claims:

  • Winners in the “Cloud Era” will be “designed from the ground up with a true Amazon-style architecture”
  • These winners are “proven at scale in real production clouds”
  • The winners are also “compatible with the Amazon architecture”
  • The winners are “fully committed to open source”

Many of these claims seem to focus on “Amazon-style architecture” and Citrix’s right to claim the mantle of Amazon-ness:

“[CloudStack] is the only cloud platform on the market that even comes close to meeting these requirements.”

A bold claim, but is it true? Eucalyptus would claim otherwise and I won’t be bashful in saying that I think our OpenStack-powered Open Cloud OS will be more compatible with Amazon in every way when compared to Citrix. Vanilla OpenStack can also make similar claims.

So, what is “Amazon-style” and what is “compatible”?

Amazon-style architecture

There are a number of ways to interpret “Amazon-style,” but to be meaningful, we can assume that an amount of Amazon compatibility is implied. If “Amazon-style” means inexpensive, it means nothing. Anyone can build a cheap cloud.

Fortunately, Citrix provides a way forward when they also claim “compatible with the AWS architecture,” a more clear-cut claim.

AWS compatibility

What does AWS compatibility mean? Surely it means more than ordering virtual machines on demand. If not, then every cloud is “AWS compatible.” It must mean more. AWS-style networking, storage, and compute? Similar hypervisors, perhaps, given Citrix is the owner of Xen? Perhaps, it’s just providing AWS API compatibility?

How is an “Amazon-style architecture” compatible with AWS? Well, for most, I think it’s about replicating Amazon’s architecture (or, at least, elements of it) besides the APIs.

CloudStack versus AWS

I have heard that many of today’s CloudStack deployments were influenced by the architecture Cloudscaling designed at KT. This architecture involved the following key elements:

  • Scalable layer-2 hardware virtual area network (VLAN) using Arista switches
  • Local, in-rack storage area network using NexentaStor
  • XenServer clusters
  • CloudStack as the VM control system
  • Cloudscaling’s provisioning system and accompanying chef recipes
  • Cloudscaling’s methodology for building hardware blueprints
  • Supporting open source software for monitoring, logging and related functions

Now let’s examine these elements and related technology differences between AWS and CloudStack.

Hardware VLANs are not AWS native networking

Obviously, standard AWS doesn’t provide hardware VLANs in its architecture. The closest capability that is a cloud service provided by AWS is Virtual Private Cloud (VPC). However, it’s not their default networking but an add-on. The default AWS networking is a flat layer-3 network, which CloudStack can support, but typically does not. Much of their value-added functionality (e.g. load balancing, network area translation) disappears when using flat networking. Default CloudStack deployments aren’t network compatible with AWS. Maybe you could claim that they’re compatible with AWS VPC, but even that is a bit of a stretch (more on that below).

AWS EBS is not a SAN

AWS Elastic Compute Cloud doesn’t use SAN storage for standard VMs, although it does provide a similar type of capability in Elastic Block Storage (EBS). Instead, EC2 provides primarily direct-attached storage (DAS) that is surfaced as ephemeral storage for each VM. No guarantees are provided around the persistency of local VM storage. This is in contrast to standard CloudStack, which typically uses XenServer clusters and persistent storage to provide high availability (HA) for a VM, where a VM is guaranteed to never be lost.

Clearly, this is different from AWS, which means the storage architecture for CloudStack is different. OpenStack, in comparison, uses a default storage model which is exactly like AWS.

Xen is XenServer?

AWS doesn’t use XenServer, but has their own customized version of Xen. This is close enough that we can call it the same. If “Amazon-style” and “AWS compatible” means the Xen hypervisor, Citrix claims are good; however, anyone running Xen is now AWS compatible.

1999 called and wants its application architecture back

CloudStack is a single monolithic piece of Java code. Most of the code resides in a single .jar file and runs on a single Java app server by default. This is a common architecture — from 1999. As you might expect, AWS has much more software, mostly written in a distributed service-oriented-architecture (SOA) fashion. The core software architecture is not similar at all between AWS and CloudStack.

CloudStack’s code focuses on features desirable in smaller deployments such as HA or high-availability of VMs (a common request in VPS-type environments that is irrelevant for customers who use DevOps-style automation). Amazon doesn’t provide HA for VMs. You can boot a VM off of a persistent EBS volume, but that is about as close as you can get. Again, this is dissimilar in architecture and capability.

Advanced networking in CloudStack and AWS

Regarding advanced networking capabilities in CloudStack that are comparable to AWS EC2 and VPC, the picture is rather bleak. When deployed with hardware VLANs, the CloudStack networking model depends on providing a router-VM for every customer. This router-VM provides networking functionality such as NAT, routing, load balancing and DHCP. All router-VMs are a certain size (number of cores, amount of RAM, etc.). The size of the router-VM directly impacts amount of throughput and number of TCP sessions possible through the load balancing software.

At KT, we did a significant amount of tuning of the router-VMs to support bigger deployments. This is problematic in that if all router-VMs are upsized, to say 2GB RAM with 2vCores, you give up significant capacity. If downsized to something smaller, they are unable to provide acceptable performance. It’s impossible to right-size these router-VMs or to scale them dynamically.

Regardless, this is nothing like Amazon. Amazon uses edge networking services that are horizontally scalable and are multi-tenanted rather than one router VM per customer.

CloudStack’s native API is not AWS compatible

The CloudStack API is not AWS compatible. Instead CloudStack provides a piece of software, called CloudBridge, that is installed co-resident with CloudStack. CloudBridge translates AWS EC2 API calls to CloudStack native API calls.

CloudBridge only provides EC2, not S3. In this regard, stacks like Eucalyptus and OpenStack provide more AWS API capability by providing both EC2 and S3 APIs. How then is CloudStack “the only cloud platform on the market,” “designed from the ground up with a true Amazon-style architecture” and “AWS compatibility”?

There are additional issues with Citrix’s claims that CloudStack was designed from the ground up with an Amazon-style architecture. It’s not even close nor can it be without a ground-up re-write.

Where does CloudStack play?

Given this comparison, it doesn’t sound very good for CloudStack. Yes, CloudStack has a number of deployments and traction. It’s been very successful. However, when we see customers choosing CloudStack, they are almost never building an AWS-style open cloud. Instead, they want a cheaper Vblock.

CloudStack, if anything, is an open source attack on VCE and VMware that is largely irrelevant in the open cloud space. When talking to other OpenStack-based open cloud vendors, they tell me that CloudStack is rarely competing against their products except when customers don’t understand the difference between an enterprise-style cloud and an open cloud.

If you want a cheaper VCE Vblock, I think CloudStack is a great option. If you want an open cloud that is AWS compatible for both architecture and APIs, CloudStack is a terrible option. OpenStack is a true open cloud solution. It has more architectural affinity with AWS than CloudStack.

Bringing reality to the unreal

No one should be under the illusion that CloudStack is more AWS/Amazon compatible than any other open source cloud software. Eucalyptus and OpenStack have similar or better claims to AWS compatibility.

To suggest otherwise is misleading.

Randy Bias is the co-founder and chief technology officer at Cloudscaling, which provides an OpenStack-powered open cloud solution with support for AWS and Rackspace APIs.

Image courtesy of Flickr user Radio Saigón.

  1. mohammedfarooq Saturday, April 7, 2012

    Randy, great analysis between the different cloud technology stacks and AWS. This helps clarify what’s real and whats marketing. As we move forward on the cloud journey, cloud inter-operatability will be the key driver that drives cloud adoption.

    Share
  2. its easy to bash a competitor when a. you don’t have a product and b. your opencloud isn’t open. would love for randy to actually spread something other rumor and hype. CloudWashing, CloudBashing and nothing but “Baised” crap of an article

    Share
  3. Randy, I just got through updating the EC2 API comparison matrix on the OpenStack wiki: http://wiki.openstack.org/Nova/APIFeatureComparison While it’s true CloudStack requires an API proxy to enable EC2 support, they do support most of the AWS API functionality that both Eucalyptus and OpenStack provide.

    Regarding S3 APIs for CloudBridge, there don’t appear to be any docs or statements of support for it on their website. However, the CloudBridge Github repo appears to have the code for supporting it, including the SOAP calls: http://wiki.openstack.org/Nova/APIFeatureComparison. Maybe someone from cloud.com can weigh in on it and clarify.

    Share
    1. My point is that CloudStack’s API proxy is not ‘native’. It’s an add-on that was added post 2.x. So it’s hard to argue that CloudStack was designed ‘from the ground up’ for AWS if they never started out with the AWS APIs, their first API was not AWS, and their AWS API support is via a proxy/translation bridge.

      Additionally, I think it’s important to distinguish between ‘supporting’ an API call and supporting the *exact* AWS behavior when the API call is made. Is it AWS compatible if I attach an EBS volume to a VM instance that comes from the same storage that the VM’s local disk drive comes from? This breaks an implicit contract that AWS made, which is that if your local storage fails, your EBS won’t, and vice versa. I would argue that isn’t compatible as all in that the behavior is different because the implied contract isn’t met: EBS volumes and local disk drives are NOT from the same storage system.

      Share
      1. Thanks for sorting through this and helping filter signal from noise! Its great that GigaOm invited you to guest author, and its disappointing that GigaOm initially took the bait (marketing noise) without further analysis!

        Share
  4. Sorry. Pasted the wrong link for the repo. It’s here: https://github.com/CloudStack/CloudBridge/tree/master/src/com/amazon/s3

    Share
  5. BTW… When referring to Vblock™ Infrastructure Platforms the correct spelling is Vblock.

    Share
    1. Robert Cathey Monday, April 9, 2012

      Fixed.

      Share
  6. Who the @#$%^^%$# cares AWS is the worst piece of crap ever, so don’t even waste your time, will take you a month, just to get this install right.

    Clustering on Linux, which has been around for well over 30 years, does all that cloud crap seamlessly, so do not bother thinking cloud, think clustering instead!!

    Nyle

    Share
  7. Randy,

    Great post and as always, enjoy your writing!

    One thing you need to clarify is that CloudStack’s “CloudBridge” is different from CloudBridge that Citrix touts… http://www.citrix.com/English/ps2/products/product.asp?contentID=2311976&ntref=prod_cat

    This is my old product, so I should know. Disclaimer: I used to be an employee of Citrix Systems and I am now an employee of BMC Software. Anything I write is my own words and does not represent the company I am part of.

    I think what would be interesting from your viewpoint… while CloudStack is different from OpenStack, what is your opinion or insight on how much Citrix is contributing to OpenStack? I feel this matters in the argument.

    Second, a more pointed question… You allude to that Citrix is pushing marketing hype (in my words) and essentially point out some key flaws with CloudStack BUT could you say this blog post is the same ploy for your company and products, a marketing ploy to pump up your efforts around OpenStack? Fair question, right!

    Finally, it seems to me you’d want to hold a “friend” like Citrix closer, why the sudden shift? I mean I know you guys have a sudden shift in strategy but wouldn’t you want to spend time targeting guys like Piston Computing, Nimbula, Nebula and Morph Labs who are your primary competition in the OpenStack space?

    Finally… I’d love a chance to debate… what is “open” really mean.

    Regards,
    Pete Downing
    http://www.linkedin.com/in/pdowning

    Share
    1. Pete Downing Monday, April 9, 2012

      I just answered some of my comment by catching up on my reading… interesting:
      http://www.crn.com/news/cloud/232800325/citrix-shakes-up-cloud-with-split-from-openstack.htm

      Share
  8. KT called — they want the respect for their cloud back…

    Share

Comments have been disabled for this post