Forget Net Neutrality; Comcast Might Break the Web
The fight that erupted today between Level 3 and Comcast involves an esoteric agreement between two of the Internet’s big players colliding with a series of equally arcane policy arguments, but at its core this fight is about money. Yet what began as a commercial dispute may end up fundamentally changing how the web works and who pays for it.
So what’s the issue? Level 3 told the world that Comcast had hit it up for more money in order to deliver traffic from Level 3′s customers (such as Netflix) to Comcast’s 17 million broadband subscribers. Level 3 said Comcast’s demand for more dough violated the principles of the Open Internet, which is shorthand for net neutrality. On the other side, Comcast, said Level 3 was trying to sell itself as a CDN while not having to pay fees to Comcast as other CDNs do. In short Level 3, was calling itself a CDN to its customers and a backbone provider to Comcast. This (plus the fact that Level 3 owns one of the largest Internet backbone networks) enabled it to undercut its competitors in the CDN business because it didn’t have to pay the fees that Akamai or Limelight did to get content onto Comcast’s network.
For example, Level 3 even told people back in 2007 that it could deliver CDN services for the same price as Internet access, a feat made possible because it owned its own networks. So when Comcast pointed out the traffic Level 3 was sending to its network would more than double to reach a 5:1 ratio when compared to the Comcast traffic sent over Level 3′s network, it was justifying its decision to act, something covered in Comcast’s peering agreement . (For detailed analysis of Comcast’s peering agreement check out this post from Vijay Gill.)
Peering is the face of this issue — the idea that Internet Service Provider A allow traffic from similarly sized and loaded networks to traverse its own for free because ISP A‘s traffic gets a pass when it’s on networks owned by ISP B or ISP C. However, the soul of this issue is how it exposes how uncompetitive the nation’s broadband networks really are. The very threat that Level 3 alleges Comcast made — essentially that Level 3 could accept the proposed fee or Comcast wouldn’t deliver Level 3′s content — should lead to concern.
This is a problem the Congress and regulators cannot ignore. Just as in the recent retransmission fights in the pay TV world, these rumblings between giant companies leaves consumers in the lurch, even though they’ve actually paid for access to the Internet — that is, the whole Internet, not one approved by Comcast or some other company. The problem, of course, is lack of competition in the broadband markets.
For the consumers who aren’t confused by their inability to access certain content and decide to switch to a provider working with Level 3, there aren’t a lot of choices. Typically, areas have only two ISPs — a cable company and a telco — and many ISPs are now offering service with annual contracts which could lock a consumer in. Plus, what happens if AT&T or Verizon decide to address this imbalance of traffic with Level 3? It is, after all, fairly common for there to be an imbalance of traffic given that consumers tend to request data from Level 3 and backbone providers far more often than they upload content to Level 3′s end customers.
It’s not far-fetched, given that by getting Level 3 to pay more for delivering a CDN service that essentially is the same as its Internet access, but does send more Level 3-specific traffic onto Comcast’s network, Comcast is getting Level 3 to pay for the increase in traffic on its network. One can wonder if Akamai’s CDN fees are calculated on the traffic it sends to an ISP or how much space its servers take up in the ISP’s data center, but with Level 3 and Comcast, there’s no need to wonder. It’s about the traffic. This idea of content providers paying ISPs to deliver the traffic to consumers, while consumers pay ISPs for access to the pipe isn’t a new one.
If that flies, then companies such as Google or Hulu may find themselves paying more for peering. That’s great for the ISPs, but again, it’s not like there are myriad opportunities for a company like Google to exert its market power short of building its own networks. For example, if a large content provider wants its services to reach folks in Rochester, N.Y., it has to work with either Frontier or Time Warner Cable. So while Comcast and Level 3 fight their commercial disagreement over peering in the press and possibly in front of regulators, the real people to suffer will be those who depend on the web. Not because Comcast has decided to call Level 3 on it being a CDN, but because of the lack of real competition in our broadband networks.
Yup, we have a problem!
Related GigaOM Pro Content (sub req’d):
Sorry, but what does CDN stand for? Doesn’t GigaOM have a Style Guide (SG) for acronyms? It’s Journalism 101: adopt a SG.
Type CDN into Google. Takes 2 secs
http://www.google.com/search?client=opera&rls=en&q=CDN&sourceid=opera&ie=utf-8&oe=utf-8
CDN = Content Delivery Network. I’m no expert but if I had to explain it, it’s a “data delivery” service you buy from Akamai, Limelight, … if you have a lot of data (e.g., pictures, videos) on your server/”warehouse” and want to make sure several 100m people all over the world can access it speedier and more reliably than if you were to host it yourself from a single location – or, if you’re FB, GOOG, …, you can of course set up your own CDN.
CDN is a Content Distribution Network or Content Delivery Network. It works under a “all your apples in a bunch of baskets” concept. Multiple servers with your content would be placed in various areas of a network, less hops means better use of your available bandwidth. Hope this helps, and yes I agree, a Style Guide is warranted.
Guys, my bad on not spelling out CDN on first reference. We do have a style guide but editors enforcing it are rare at 10 pm. Thank you to our commenters who helped people out on that.
http://lmgtfy.com/?q=define:+cdn
Great post! Thanks for explaining this so clearly.
Thanks for the nice post. Hope it all works out for the good of everyone. mainstreethost
Again, Stacey Higginbotham continues to spout the usual BS, denying the existence of competitive ISPs and promoting stifling, onerous regulation that would kill them. (This, of course, is all consistent with Google’s corporate party line.) Stacey likewise denies the market power of content providers such as Google – perhaps because Google is a sponsor of this site and pays a goodly portion of her salary. Given this financial motivation, we probably can’t ever expect unbiased or truthful reporting from Stacey or from this site.
Brett, you do see some of these problems don’t you? You have L3, Akamai and Limelight jamming more and more video traffic onto your network without paying you anything, don’t you?
Yes, I see the problem inherent in bandwidth-hogging content; in fact, I was the among the first small ISPs to speak about it. I was also one of the first ISPs to point out that there is often discrimination in the other direction. Akamai was (and is) discriminating against us and our customers by refusing to deal with US; they would not place a cache with us and thus greatly increased our backbone bandwidth load. This is a big issue for ISPs without nationwide backbones (and, therefore, without the ability to peer).
What’s sad to see, though, is that publications like GigaOm toe the Google anti-ISP party line and are also prejudiced against (and even deny the existence of!) small and competitive ISPs. This is unethical and destructive. But, as I mention above, they obviously care more about getting the maximum amount of money from Google than they do about the truthfulness of their reporting.
@Brett: Rochester, NY, as cited in the article. Metro population nearing 1 million, and we have as options Time Warner and Frontier, as cited in this article. Ostensibly, I have a choice between the two, but my little suburban ranch is in a nice little area equidistant from two substations, putting me at the very end range of both of them. Having tried to lose Time Warner (we were the lucky ones who got the “tier trials”), it can’t be done. DSL put me at a level below AOL dial-up users with a 14.4 modem. I have no competition here. Zero. Forget FiOS; I just want options for cable! If I want to be able to work out of my home, as I’ve done for the past 11 years, I pay Time Warner, and that’s it. If I want Netflix streaming or Xbox Live or even porn, it’s all Time Warner. So yes, markets like this do exist, and anyone in telecom will point to Rochester as the face of what’s wrong with the industry in the first place.
Cyndy, I see three wireless broadband providers (WISPs) in your area – not including cellular. You’ve got quite a lot of options.
WISP coverage here is flaky, at best. As an example, I use Clear for redundancy since I work out of my home, and we have flaky electric here in inclement weather. The signal drops in and out even though I’m in a solid coverage area for them, so while I CAN watch a streaming video, I’m going to be restarting it approximately three times in the course of a 30-minute TV episode. It’s fine for working, where I won’t notice blipping out, but not anything that requires constant up-time. There is no WISP here that’s going to provide anywhere near the broadband capability of TWC, so competition on paper != actual competition.
First up, we have Glass who operates a small ISP in Wyoming. Obviously, he wants to be able to charge his local customers for web access, and he wants to be able to charge content providers that his customers access. He doesn’t seem to understand that he could do a lot more for his cause by explaining to us all of the issues he has and laying out as much detail as possible.
Be specific, Glass! Don’t claim silly conflicts of interest where there is ZERO evidence. It makes you look ridiculous and will not gain you sympathy,
Wow Brett, thats some crazy tinfoil hattery there.
What evidence do you have that Google has anything to do with this article monetary or otherwise? You remove your own credibility by exposing the fact that you own/run an ISP so you have your own agenda in this fight, namely to double-dip the consumers AND the content providers for more and more fees.
In short, you and other ISPs want to have your cake, and eat it too.
Certainly there are other third party ISPs around, but almost every single one of them are just reselling services from a major carrier, like Comcast, AT&T or others, then re-branding it as their own, and charging far more than the service they resell.
This fight is about money, and how much more of it the ISPs can wring out of everyone involved, no matter who wins this fight, the consumer is going to lose. Increased costs passed on from the ISPs because they cant soak the content providers, or increased costs from the providers because the ISPs are going to hold them hostage from delivering PAID FOR CONTENT to the consumers.
This is the same thing as paying rent for an apartment building you live in, but the apartment owner demanding more money from the resident or USPS for the privilege of receiving mail or they will stop the postal carrier from getting to your mailbox.
Another analogy: you sign an agreement with some postal carrier, say UPS, to deliver all packages to and from your residence for a flat monthly rate.
However then UPS notices that most of the packages it delivers for you come come from a few key distributors. UPS threatens not to deliver any packages from those distributors unless they pay a premium to UPS.
It really is that ridiculous.
No, it’s like paying a monthly fee to have a phone (land-line or cellular, doesn’t matter) and then paying a per-minute charge for calls.
“Ken,” you obviously know nothing about the ISP business, or you would realize how ludicrous your statements above are. As for the connection between GigaOm and Google: GigaOm appears to make most of its money from ads placed by Google. Follow the money. They can’t be objective when they’re dependent upon Google for the majority of their revenue, which explains why their reporting has been slanted toward Google on every issue for many years.
“you obviously know nothing about the ISP business, or you would realize how ludicrous your statements above are.”
So…explain it?
I’m not going to write a treatise. Especially not on Om’s site, since he not only is not paying me but is trying to destroy my business and harm my customers.
“… denying the existence of competitive ISPs …”
Boy, I wish I had “competitive ISPs where I live. In my neck of the woods, a metropolitan area approaching a million souls, there are only two: Comcast and Qwest. Both have something to loose when third-party content providers like Netflix use their pipe’s to deliver content. (Qwest is partnered with DirectTV in my location.) Currently I’m paying Qwest three times the amount for half the bandwidth I got when DSL first became available in my neighborhood. Comcast would be even more expensive.
I like you believe that free markets are the best solution to the issues that Stacey Higginbotham brings up. However, free markets only exist where there is true competition and consumer choice. That hardly applies to the duopoly which is most U.S. broadband markets.
There is no US city with a population of nearly a million that does not have MANY competitive options. That you are ignorant of them does not mean they do not exist.
Wait, what’s wrong with charging for more traffic? If there is a balance of traffic in a peering agreement, then all are happy. But if there is a 5:1 ratio I agree that the ISP has a right to charge to deliver that extra traffic.
The point of net neutrality is that all *types* of traffic be treated equally. There’s nothing wrong with charging for added traffic volume by source… unless of course those prices are not controlled by any real competition. (The real issue!)
@Mark: The point of Net Neutrality is also that all *sources* of traffic be treated equally. As a comcast user i should have the same access to Akamai’s content as i do to Level 3′s content. Remember that Level 3 is already paying for their bandwidth (they’re maintaining their own servers, etc.). If Comcast can’t find a way to be profitable with the amount of bandwidth they’re serving, they need to raise the rates for their subscribers, the people who are actually consuming Level 3′s bandwidth.
Jim, I have to disagree. Both sides (consumer and provider) should have a stake in paying for bandwidth utilization/transmission. The electric utilities have been doing so for a long time.
IPV6 will help, but will not solve this problem, since there will inevitably be disagreements on classification and sub-segmenting of traffic beyond what IPV6 allows. But it’s a start.
Additionally, while we bash Comcast and the infrastructure providers as being “greedy”, their operating margins are a FRACTION of those of Google et al (and yet they still pay a whole lot more in taxes than Google, but that’s another story).
There does need to be a tiering of QoS and pricing based on the nature of the content if we’re ever going to enable mission critical applications to leverage the public internet (telemedicine, smart transportation, smart energy and water distribution, etc.).
@Tony Content delivery network
super post thanks for the info!
Most peering relationships have stipulations that if there is an imbalance, then the company responsible for the imbalance pays.
The irony of Level 3 doing this is that they’ve been on the “comcast side” of this a couple of times, including when they went as far as temporarily cutting off their relationship with Cogent in 2005.
Man, who hasn’t cut off Cogent? :)
Anyhow, the imbalance issue is real and I tried to address it in the story, however I do want more information on what Comcast charges Akamai and what those fees actually buy. Comcast makes it seem as if Akamai can undercut Level 3 simply because it doesn’t pay a CDN fee, but Level 3 does own its own network, which Akamai doesn’t. Akamai buys bandwidth from folks like Level 3. So even if if this Comcast fee is charged to both Akamai and Level 3 as CDNs, Level 3 should still have a price advantage.
Why should you, a blogger, have the right to ask for proprietary and sensitive information such as a contract between Akamai and Comcast? Further, why should you (or anyone for that matter) get to decide if these contracts are “fair”? My favorite part is how you blame Comcast for the lack of competition. Blame local governments for restricting entry!
Well, she’s actually a journalist who happens to work for a news site operating on a blog platform. Regardless, this is America; anyone is entitled to ask questions, it’s up to the other side to decide if it wants to respond.
But back to the point: If Akamai has a different peering agreement with Comcast, isn’t that just the free market in action, i.e. Akamai did a better job of negotiating its contract? Could this really be all about Level 3 trying to gin up public leverage pursuant to renegotiating its deal with Comcast? Or is there some law/regulation that all peering agreements are supposed to be equal?
My, how quickly they forget. I wrote a guest blog for GigaOm a year ago pointing out that the FCC’s proposed net neutrality rules could easily end up regulating peering and transit agreements, and now we see what? Net neutrality has been expanded by L3 to apply to peering and transit agreements.
The comments to my guest blog featured adamant denials from employees of Google, Akamai, Netflix, YouTube and a host of frequent fliers to the effect that I was offering a wholly unreasonable interpretation of the Net Neutrality rules. I don’t want to say I told you so, but my goodness was I ever right.
That blog post is: http://gigaom.com/2009/11/22/how-video-is-changing-the-internet/
The comments are fun, especially those from Patrick Gilmore disputing his own published claims and Daniel Golding’s assertions that there is no possible way for net neutrality to affect network interconnection.
What we’re seeing is the unbearable lightness of net neutrality: a vague cause that can be, and regularly is, redefined to cover all many of real or imaginary ills. Be careful what you wish for, kids. Net neutrality used to be about blocking access to web sites; now it’s about peering, retransmission consent for TV shows, down-level cable modem certification, and traffic engineering. What’s next, recycling and animal rights?
Richard, you’re right. I wish I had remembered your post last night. However, Level 3 hiding behind net neutrality is pretty bogus. This is not so much a net neutrality issue so much as a lack of competition issue. Although I suppose if we had real competition we’d also have less to worry about with regards to net neutrality because people would just switch providers.
Stacey, Akamai pays big time to be inside cable headends, Netflix is not dumb, they knew full well L3 is on outside looking in….
No mam, the baddies here are Netflix and L3, let them decide how much they’ll ante up to Time Warner to compete with Akamai, everyone has their price – let them pay it.
There is NO free ride. One way or the other WE pay, stop siding with companies and let them fight each other to the death – and we’ll pay the least.
The last time your type yammered on and on about this kind of thing, we had the Government investigating MSFT for giving away a browser Netscape wanted to charge $40 for.
There’s no difference between NN and the “lack of competition issue,” Stacy: The argument for NN all along has been that the ISP market isn’t sufficiently competitive for the market to discipline ISP behavior, so the government has to apply rules. I’ve always basically agreed with that, my beef is that the rules tend to not be narrow, clear, and well-focused. If cable modem certification and peering are NN issues, NN is too broadly defined.
@morgan warstler: Akamai isn’t inside Comcast’s cable head-ends and never was. Comcast doesn’t sell rack space, they interconnect at various places far removed from DOCSIS.
Next up in the complainer line-up, We have Bennet who it is said takes most of his consulting $ from “big telco” (e.g. AT&T). Again, obvious bias toward ISPs hoping to get $ from both content providers and consumers.Pretty straight forward.
coldbrew
whatever your feelings about richard might be, but the issue at at the heart of this matter is going to impact all of us, the Internet users. :-)
I like the diverse crowd here, Om, you’ve got the full range.
so the internet gets owned by big companies and small guys will be kicked out if they don’t pay? What about economies of scale where Google gets preferential rates of say $0.01/mb and individuals pay $0.04/mb… Scary
STOP THIS NOW
I agree that the lack of real competition in the broadband networks such as Time Warner Cable & Comcast have created monopolies where people are held hostage to them and their fees. We went through this before with the telephone company The Bell System, a.k.a. “Ma Bell”. The cable company franchising & licensing has to be changed to allow local competition. Competition has always driven prices down. The way it is working now is the towns, cities, etc are getting paid off by the cable company to allow a cable company an monopoly of their area under the disguise of an franchise.