14 Comments

Summary:

When we watch YouTube or stream TV through a Boxee, the assumption is that aside from some buffering, the experience will be fine. But add live content and interactive elements to those video streams, and it gets complicated. A new paper shows us how complicated.

Akamai CEO Paul Sagan

Akamai CEO Paul Sagan

Update: When it comes to consumers watching YouTube or even streaming TV through a Boxee, the assumption is that aside from some buffering or pauses while the streams catch up, the experience will be fine. But when we add live content and interactive elements to those video streams, it gets complicated. Thanks, to a paper detailing Akamai’s content delivery network in minute detail, we can see exactly how complicated it is today, and what sort of havoc interactivity might wreak.

In an academic paper written by researchers at Akamai, Harvard and the University of Massachusetts, readers get an in-depth look at how Akamai’s distributed CDN works, how the web itself works, and what the shift from static to interactive content means for content providers and the network itself. Details such as Akamai’s 61,000 servers located across 70 countries and nearly 1,000 networks pale in comparison to the section on how streaming video is going to get much more challenging in the coming years. Updated: Akamai emailed to say this paper was published last year and since then it now has more than 90,000 servers in more than 1,800 locations in 1,000 networks in more than 70 countries.

The paper offers up a timeline of big web streaming events, beginning with Steve Jobs’ MacWorld keynote in 2001 that drew 35,500 viewers and required 5.3 gigabytes Gbps of capacity. President Obama’s inauguration in 2009 drew 7 million simultaneous streams and required nearly 2 terabytes Tbps of capacity. Akamai noted it hit a peak record in 2010 of delivering 3.45 Tbps of data. But those numbers don’t keep Akamai engineers up at night. The future does. From the paper:

In the near term (two to five years), it is reasonable to expect that throughput requirements for some single video events will reach roughly 50 to 100 Tbps (the equivalent of distributing a TV quality stream to a large prime time audience). This is an order of magnitude larger than the biggest online events today. The functionality of video events has also been increasing to include such features as DVR-like-functionality (where some clients may pause or rewind), interactivity, advertisement insertion, and mobile device support.

At this scale, it is no longer sufficient to simply have enough server and egress bandwidth resources. One must consider the throughput of the entire path from encoders to servers to end users. The bottleneck is no longer likely to be at just the origin data center. It could be at a peering point, or a network‘s backhaul capacity, or an ISP‘s upstream connectivity—or it could be due to the network latency between server and end user, as discussed earlier in Section 3. At video scale, a data center‘s nominal egress capacity has very little to do with its real throughput to end users.

The paper goes on to say that because even an awesome data center can only provider a few hundred gigabytes per second of throughput to end users, it’s almost impossible to create a service with the scale to deliver the hundreds of terabytes needed to support video. And while the paper reads like a highly technical advertisement for Akamai (and why the way it has built out its CDN is superior to other CDNs,) it’s also a pretty detailed look into the complexity of the web.

I know many of us take it for granted that the animated GIFs that once slowed down our GeoCities page load times are now so commonplace we drop them into comment threads, but that’s the beauty of driving ever-faster broadband speeds. Sometimes it’s nice to look behind the curtain and see how our infrastructure is keeping up with the increasingly complicated elements we’re throwing at it.

Hat tip to High Scalability, which featured the paper.

You’re subscribed! If you like, you can update your settings

  1. It’s a bit disappointing to see a publication like GigaOm report on something with such old data behind it. The numbers in the story on Akamai’s platform are several years old, as is the referenced paper. The network and offerings has changed significantly since, as has the number of servers deployed globally, with the network now having over 95,000 servers.

    1. Stacey Higginbotham Mike Friday, August 19, 2011

      Mike, the paper is a year old, which I was not aware of. I have updated the data. But what convinced me to write it up was the information in there about delivering interactive video and what that might mean from an infrastructure perspective, and that’s still relevant.

      1. Hi Stacey, sounds good. It is interesting stuff, was just surprised that GigaOM and several other outlets just picked up on this now and that the data written up was so old. :)

    2. In the TelCo/ISP arena it is *very* difficult to get any publicly available, meaningful statistics. So if the data is 2-4 years old, I’d say that’s probably as good as it’s gonna get.

  2. Erik Schwartz Friday, August 19, 2011

    This is why all those backbone providers selling cheap CDN services by using excess backbone capacity are not a long term solution.

  3. I’m sure that what you’d meant to write was GigaBITS (Gbps) per second and TeraBITS (Tbps) ? (not GigaBYTES and TeraBYTES).

    1. Stacey Higginbotham Matt Friday, August 19, 2011

      Matt, you are correct, and I should stop writing so late at night. Fixed.

  4. This is written by Akamai. It references many phrases like “our network”. It’s just a high-level overview of their services. There is nothing mentioned in here we haven’t seen or heard before and it is anything but “in-depth”. It’s a high-level marketing document.

    The most in-depth paper we have seen to date on Akamai’s network, from a third party, was the one done by Microsoft Research and the Polytechnic Institute of NYU in 2008, which Akamai had Microsoft pull from the web after it was published.
    http://blog.streamingmedia.com/the_business_of_online_vi/2008/10/microsoft-resea.html

    The data in the paper on Akamai’s stream count is wrong. Akamai did not delivery 7M simultaneous streams of the Obama inauguration. That day, their network peaked at 7.7M streams amongst all of their customers, of which 3.8M were specifically the Obama webcast. Akamai broke out these numbers in detail:
    http://blog.streamingmedia.com/the_business_of_online_vi/2009/01/akamai-and-numbers.html

    Also, I would not agree that just moving from an on-demand stream to live makes it “complicated”. There are plenty of live streaming services from MLB, NBA, NHL etc. who have been delivering very good quality video for years, without complications. And when they do have issues, it’s usually issues with the functionality in the player, not the delivery itself.

    1. “And when they do have issues, it’s usually issues with the functionality in the player, not the delivery itself” would it be possible to elaborate ? Thanks

      1. adaptive streaming can allow for initial ease of delivery. Complex players can choke some devices that support Flash, so often slowdowns can happen because the device is having more difficulty keeping up with the experience than the bandwidth needed to at least display the lowest bitrate in the bunch…

  5. No matter how hard we try to push monolithic systems, there’ll always be a point where the cost/value ratio enters the is-it-worth-all-that zone.

    Wouldn’t it be about time to think about expanding local network structures and use a true peer-to-peer model in the sense of Bittorrent to push content? Sure, this could have an impact on the biz model of Akamai and friends, but it could safe Joe Consumer from an upcoming payment model which, without a doubt, content publishers are thinking about already just because the delivery becomes more expensive.

    Would I really care if I watch a live keynote presentation and have a 20 seconds time shift compared to my neighbor who’s sharing the stream with me?

  6. Steve Crowley Sunday, August 21, 2011

    For single-video events, maybe someone could invent a wireless system that sends the video over-the-air, casting it broadly to many people at once. It might be supported by advertising so viewers don’t have to pay. Such a system would have no bottlenecks. Except for soft-drink commercials.

    1. Someone already has, Steve. It’s called digital broadcast television. :-)

      1. Indeed, and it’s around since 1992. Just happens to be largely ignored in the Web & Internet communities. Needless to mention that modern DTV systems can of course carry IP datagrams directly?

Comments have been disabled for this post