Blog Post

Why Netflix’s CDN should scare the storage industry

Lest storage vendors thought they were immune to disruption that open source hardware is having on the server industry, Netflix’s (s nflx) new Open Connect content-delivery network might make them think again. While Open Connect directly targets commercial CDNs, it’s based upon (or at least inspired by) open source storage designs first released by Backblaze almost three years ago. Backblaze’s design evolving and expanding its range into the data centers of a Fortune 1000 company is significant in the same way the evolution of modern man was for neanderthals.

By way of background, Backblaze is a cloud storage provider focused solely on backing up lots of data for cheap (like $5 a month for unlimited capacity cheap). In order to do that, it had to build a storage system that could hold massive amounts of data without breaking the bank. As of last July, Backblaze’s architecture had evolved to a point where a 135TB pod cost less than $7,400 to build from scratch.

Understandably, the architecture generated a lot of interest from companies and organizations wanting to leverage it to soothe their own IT budgets, but none of them are Netflix. EMC’s (e emc) Pat Gelsinger said recently that the storage component of Facebook’s (s fb) Open Compute Project, called Open Vault, isn’t yet ready for primetime because nobody is running — or would run — mission-critical workloads on it. That might be true of Open Vault today — the project just launched earlier this year — but it likely won’t be for long. If you consider a CDN that serves Netflix streaming video mission-critical, the criticism is already invalid for Backblaze’s designs as Netflix has adapted them.

Netflix’s 4U, 100TB server

It’s worth noting, too, that open source hardware isn’t the only piece of the stack threatening legacy storage vendors such as EMC. I’ve heard it suggested recently by someone experienced in building out large-scale cloud infrastructure that the Hadoop Distributed File System has the potential to become the default file system for large infrastructures once it works out some of the limitations around performance and availability. One of the biggest of those limitations — the NameNode— has been eliminated in the latest version of Apache Hadoop and is already integrated into Cloudera’s new CDH4 release.

Can storage deal with the open source disruption?

As with Open Compute’s effects on the server industry, though, open source storage doesn’t need to spell doom for legacy vendors if they’re willing to adapt. One reason is that, at least in the short term, there are still plenty of customers that don’t operate at Facebook or Netflix scale and can afford to pay a premium on smaller deployments that offer the features (and vendor support) those customers demand.

If the server shipments tell us anything, though, it’s that the rise of cloud computing and web giants will ultimately take a toll on the storage market, too. Fewer, but very large, customers will be responsible for a greater percentage of sales, and they won’t necessarily want all the bells and whistles that make enterprise storage products so expensive. And if VMware (s vmw) is correct, even mainstream enterprises will soon want to follow the examples of web giants like Google (s goog) and Facebook by running relatively dumb hardware managed by really smart software.

If this scenario plays out, storage vendors will have to reassess how they deliver value and earn their money. That might mean adopting open source designs in their own gear while shifting their focus a lot more heavily toward software and services, or perhaps unlocking their storage-management software from the hardware and certifying it to run on open source gear.

Perhaps we’ll get some ideas for what the future storage and markets look like at our Structure conference June 20 and 21, where we’ll dive into the topic with Facebook’s Frank Frankovsy, Netflix’s Adrian Cockroft and VMware Steve Herrod. Whatever the case, it looks like something will have to give.

Feature image courtesy of Shutterstock user Zadorozhnyi Viktor.

4 Responses to “Why Netflix’s CDN should scare the storage industry”

  1. Mark Wojtasiak

    This is yet another example of larger “cloud” players moving to their own custom designed solutions. I just did a post on a little-known company named Intequus that announced they were the ones building custom solutions for Netflix. Why the movement from the big brands to smaller builders/integrators? (from the post) The reason is that many cloud providers know exactly what they want, what they need – especially the larger players that have invested in cloud architects, software developers, and technical talent… If these guys know what it takes to deliver their cloud services in the most cost efficient manner, what value would a big brand name offer them? Some argue none, hence the movement to flexible system builders like Intequus. No doubt the big brands will have a play, but I’m more interested in what’s happening with the up-and-coming “white-box” builders.

  2. Jason Thibeault

    Sigh. I have to say that I am usually quite impressed with the level of analysis that you guys put to topics. And although this one really digs into the whole storage angle of the Netflix decision, you toss in the comment, “While OpenConnect directly targets commercial CDNs” without any sort of explanation. First of all, OpenConnect is not really a CDN. It’s managed storage (hence, why I think your analysis is ultimately solid in this piece). But I’d like to understand better how you came to the conclusion that OpenConnect targets the commercial CDN market. Is that simply because of their decision to replace their CDN services with this? Or is it based on the poor analysis in the financial markets? Did you stop and ask yourself “why?” Do you have any idea about the relationship Netflix content has with access networks (that access networks are feeling increasing strain because of the amount of bandwidth Netflix content is hogging on their networks and the cost for them to backhaul it from the Internet; in case you didn’t know, CDNs terminate on access networks)? I’d expect you to understand this relationship before making a content about what is or isn’t targeting a commercial sector (I’ve seen enough of that bad analysis coming from the financial markets since Monday). I provided my own speculation and analysis of the “why” in a blog post but, more importantly, what the why points to: a tipping point in video content consumption. Netflix’s decision will be forgotten. But the tipping point it illustrates will not. I’m not asking that you don’t speculate. I’m just asking that when you elect to make casual statements that may be inflammatory (and possible erroneous), you do so with some authority based on analysis.


    • Derrick Harris


      My comment was simply alluding to the fact when one sees Netflix moving to its own CDN, the directly affected parties are its CDN providers. while the storage aspect swims under the surface.

  3. adrian cockcroft

    s/Cockroft/Cockcroft/ – Thanks.

    Also Redundant Array of Inexpensive Nodes – RAIN – is how low cost cloud storage disrupts the mainstream storage vendors.