7 Comments

Summary:

If you think about it, Netflix’s metamorphosis into a company that runs its infrastructure completely atop cloud-based resources is truly remarkable. For many companies, such as site-optimization and CDN provider Yottaa, the bigger they get, the harder it is to justify the cloud’s cost and performance.

parachute

If you think about it, Netflix’s metamorphosis into a company that runs its infrastructure completely atop cloud-based resources is truly remarkable. It’s a very large company with a very large IT operation and, presumably, a rather large bill in the mail every month from Amazon Web Services. Engineering effort aside, the fact that Netflix has decided it’s worth it to pay the cloud computing premium is the most amazing part. With many companies, the bigger they get, the faster they come down from the clouds.

Case in point: Yottaa. The web-optimization startup, which also launched its own CDN service in March, is transitioning its network into a hybrid model of cloud-based and physical servers after launching in the cloud exclusively. It’s a significant shift considering the company was actually a finalist in the 2010 Amazon Web Services Start-Up Challenge and touted its cloud-based approach when the company launched last April.

Don’t get me wrong, the cloud-only model has served Yottaa well. Its network is actually spread across multiple providers, including AWS, Microsoft and Voxel, and that distribution helped the company reroute traffic to avoid any major downtime during last year’s four-day AWS outage. And even as it moves to a hybrid model, the cloud still has benefits. “We can literally scale to hundreds of thousands of machines in a matter of hours,” said Yottaa Founder and CEO Coach Wei.

As the company grew, however, cost, performance and security issues meant Yottaa had to decrease its cloud dependency:

  • Cost: According Wei, it’s easy to get started in the cloud — you can spin up only as many servers as you need at any given time and don’t have to invest in 100,000 physical servers to match Akamai’s architecture — but “when you get to a certain scale, it’s actually not cost-effective anymore.” For Yottaa, which is serving 100 million unique visitors across its network every month for more than 80,000 web sites, that time has already come. Actually, Wei said, Yottaa always planned to move to a hybrid architecture, but even still he was surprised at how much the cloud could cost. Before he started Yottaa, he’d never thought about paying a million dollars a year to AWS.
  • Performance: The performance trade-offs in the cloud can be problematic, too. Wei said network performance is the biggest problem for Yottaa, as it’s typically about 50 percent slower in the cloud and variable at that. You never know what type of performance you’ll get at any given time. If someone else is using a lot of bandwidth, your service might suffer. And while some cloud providers throttle bandwidth at 100 Mbps per user, Wei said, “at the high end of the scale, that’s just way too low.”
  • Security: Wei notes all sorts of security problems with cloud computing, but the major one is the inability to use tried-and-true physical appliances for security. Not only do physical appliances provide a lot in terms of traffic-monitoring and load-balancing, but they can store thousands of IP addresses for SSL certificates on a single box. A cloud provider, Wei said, might only give you a handful of IP addresses.

With its new hybrid model, Yottaa still leverages the cloud when necessary — like when it would be faster serving an Australian end-user through a cloud provider there than a physical server in the United States — but it targets physical resources whenever possible. Its network now includes cloud and/or physical servers in 24 cities across the globe.

Yottaa isn’t alone in its transition to a hybrid architecture after a cloud-heavy beginning. Zynga, for one, famously reversed its AWS-to-internal-cloud usage ratio from 80-20 to 20-80. Fellow gaming startup Digital Chocolate also moved a good portion of its operations back onto internal infrastructure. At a larger scale, you see similar requirements of cost, performance and customization driving web platforms such as Facebook and Twitter to build their own data centers.

Of course, for every company (or several companies) that decides to switch from a cloud-centric architecture, there’s a Netflix or Animoto that decides to stay all in the cloud. It’s really a matter of knowing what’s best for your business. We’ll talk all about ideal infrastructure choices at our Structure conference in June, which includes top executives from Zynga, Amazon and Netflix among others.

Image courtesy of Flickr user Thomas Claveirole.

  1. I think it’s worth mentioning that Netflix is transitioning to hosting their own streaming.

    Netflix uses Amazon for their webpage but that’s not really where their infrastructure challenges are.

    They use CDNs (Level3, Akamai, Limelight) to deliver their streaming content.

    However, it appears they are moving to using their own CDN where possible.

    See their peeringDB page.
    Netflix streams our content payload via one of four methods:

    A) An embedded cache inside of your network, provided you meet certain traffic criteria

    B) Public Peering (SFI) via IXs at common public peering locations

    C) PNI (SFI) at common private peering locations, provided you meet certain traffic criteria

    D) CDNs such as Akamai, Level3, and Limelight
    https://www.peeringdb.com/private/participant_view.php?id=457

    Share
    1. Derrick Harris Sunday, April 22, 2012

      There’s no denying streaming is important — and a company like Netflix certainly wouldn’t use AWS for that — but it still does a fair amount of computing in the cloud. Enough to make its AWS reliance remarkable, at any rate.

      Share
  2. Good article and why we are living in a “post cloud” era – not that the cloud is done nor bad, however, it was another evolutionary step towards unified IaaS/PaaS – the cloud was over hyped and not the “silver bullet” nor “one ring to rule them all” – now being on the backside of the hype cycle, we are seeing still unique infrastructure demands by companies who grow to certain scales as you mentioned Zynga, etc.

    http://rhdonaldson.wordpress.com/2012/04/09/we-are-in-a-post-cloud-era/

    Share
  3. Cloud will always be good enough for 99.9999% of services if it’s not your probably swimming in cash and it’s a problem I’d love to have

    Share
  4. it’s ironic that this article even exists, since it shows how misunderstood Cloud is (in the press at least). Cloud makes sense for exactly two reasons: outsourcing IT and bursty workloads. there’s plenty of space for companies to win by avoiding the public cloud – after all, Amazon is making a generous profit from all their customers.

    I think there’s also a widespread misunderstanding of the concept of “economy of scale”. people tend tend to use that phrase as a talisman when they want to outsource, but at least for IT, above a quite small installation, costs are linear. that means Amazon spends almost exactly as much for another server as any company that has a well-managed datacenter of similar hardware/density. where Amazon has an advantage is in comparison to poorly managed in-house IT.

    Share
  5. Sarbjeet Johal Sunday, April 22, 2012

    There are so many variables which impact coud choices. Agility, time to market is biggest reason for selection of cloud. Another thing I notice is that try before you buy is the model, Zyngas of the world are following. They tried cloud at AWS, when it was proved that it works, they built their own. It’s like renting/leasing a 16 wheeler for a longer test dive before you buy it! I think this trend will continue in enterprise cloud adoption arena!

    Share
  6. @sonian is multi-cloud for just this purpose: Avoid worrying about outgrowing a single cloud.

    http://www.gregarnette.com/blog/2012/02/a-2007-multi-cloud-fantasy-becomes-a-2012-reality/

    Share

Comments have been disabled for this post