43 Comments

Summary:

Comcast is moving away from the much criticized practice of blocking P2P traffic, and instead is looking at alternatives to rein in a small group of Comcast customers that can be labeled bandwidth hogs. Comcast Cable’s CTO explains.

Comcast recently announced a deal with BitTorrent that left me dazed and confused. It was basically a roundabout way for the cable company to backtrack from its P2P traffic-blocking gaffe. In describing the deal, Comcast tried to shift the focus away from their so-called “network management” — and by extension, the limitations of their network that prompted them to resort to traffic manipulation in the first place.

On Friday, I caught up with Tony Werner, chief technology officer of Comcast Cable, to get the real skinny. When asked to explain the so-called announcement in language a simpleton like me could understand, Werner said: “Historically we had looked at a basket of P2P protocols during peak load times and would slow them down. In the new approach, we don’t do this any more.” In short, no P2P blocking!

Werner said that between one half and two percent of Comcast’s customers can be described as “bandwidth hogs” — users that consume so much bandwidth that it can cause network quality degradation. According to Werner, the company is currently experimenting with software (including that from Sandvine) that would allow them to fractionally de-prioritize the traffic from these bandwidth hogs during peak load times, while at other times, leaving them alone.

Comcast will not discriminate against any protocol, but bandwidth baddies are going to be the ones to suffer. Or at least that’s what I took away from our conversation.

Problem is who’s to say they’re not going to manage everyone’s traffic? Although a company spokesperson assured us Comcast will be clear and transparent with anything related to traffic management, my skepticism stems for Comcast’s past actions. When it comes to traffic management, the Philadelphia-based operator has a checkered past.

Comcast assured the FCC during the Network Neutrality deliberations in 2005 that it would not degrade traffic; it repeated the assurance again in 2006. Yet the company started “traffic managing” that very same year. And now they’re cleaning up their act?

I asked Werner, why manage traffic to begin with? Why not just add more capacity? “You can’t quadruple the size of the streets and take away all the traffic rules,” Werner said.

He said Comcast is not alone in traffic management, that even in places like Japan, fiber operators that sell 100-megabits-per-second connections are managing traffic, too. “A vast majority of ISPs do perform traffic management, including NTT, and the reason we do it is because we want to have balanced traffic performance at peak times,” Werner said. (See here for “Why Shaping Traffic Isn’t Just A Comcast Issue.“)

Of course, my views on broadband align with those of French broadband maverick Xavier Niel, who believes giving people more bandwidth — not getting in their way. Still, his view (and mine) are the minority in a broadband world dominated by large incumbents.

For their part, the incumbents have started to talk about taking a protocol-agnostic approach to traffic management. They have to, otherwise we’ll have more snafus like the ones experienced by the Canadian Broadcasting Corporation. Although the CBC released a torrent legitimately, downloaders had a hard time grabbing the video shows. Werner’s comments and recent throat-clearing by Verizon and AT&T reveals a thaw in ISP views on P2P.

On a larger scale, Werner said traffic management is “very tricky.” “We need to get the whole industry together and tackle this issue,” he said.

  1. what is a “bandwidth baddie?” someone who actually uses what they were offered and paid for? ahem!

    Share
  2. I believe Comcast is doing the right thing. Traffic mgt is necessary. You can’t just keep solving the problem by adding capacity. You have to do both. Make the baddies pay more or slow them down.

    Share
  3. To poorly paraphrase Star Wars episode three…”This is how Net Neutrality dies, with thunderous applause…”

    Share
  4. P.S. Comcast you and all the other ISPs are just “dumb pipes” the sooner you accept that reality, a better place the world will be.

    http://en.wikipedia.org/wiki/Dumb_pipe

    Share
  5. Rick has a point – obviously, there will always be some bottleneck somewhere which must be managed in some way, even if it’s as simple as letting the router at that point drop packets from the relevant buffer, the default “traffic management” policy which has prevailed across the Internet for as long as it has existed.

    If Comcast applies a cap fairly – so that if 20 people all try to pull data across the same 155Mbps (OC-3) link somewhere, they each get a fair share of 7 Mbps or so – it’s reasonable – my objection comes when they start killing off one protocol (like BitTorrent) to leave more room for others like HTTP, and that seems to be precisely what Comcast were doing previously. I don’t mind not being able to max out the 8Mbps capacity of my ADSL line when the network’s busy (although obviously I expect my ISP to invest in maintaining performance as traffic grows) – but when they start artificially restricting the connection’s performance based on how they think I’m using it, I object.

    Share
  6. So if they are being transparent about what they are doing, any chance they can clue us in where the high traffic times? It seems like if they published these times, some of the bandwidth hogs would reduce their consumption at peak times and increase during non-peak times, so as to work with Comcast versus getting comcasted by whatever software (i.e. expensive, customer off putting, etc etc.) they come up with.

    Share
  7. Lexi Cographer Monday, March 31, 2008

    change to “rein in”

    Share
  8. Re ‘just add more capacity’

    In one of the press releases or articles they did in fact say they were adding network capacity, which is good. But I encourage anyone to read what the Japanese ISPs are experiencing, and to read the recent ZDNet article on this topic (George Ou wrote it I think). Basically it appears that other countries are seeing their pipes fill to capacity as soon as they are built, and the ZDNet article points out some short-comings in TCP itself on this matter.

    Re ‘Problem is who’s to say they’re not going to manage everyone’s traffic? Although a company spokesperson assured us Comcast will be clear and transparent with anything related to traffic management, my skepticism stems for Comcast’s past actions. When it comes to traffic management, the Philadelphia-based operator has a checkered past.’

    Makes sense to expect greater transparency. Didn’t the press release say they were sharing this practices with BitTorrent and in some way taking this to the IETF? Transparency is good.

    Share
  9. Re: ‘P.S. Comcast you and all the other ISPs are just “dumb pipes” the sooner you accept that reality, a better place the world will be.
    http://en.wikipedia.org/wiki/Dumb_pipe

    Just read that wiki and it appears suited to mobile operators with a walled garden, so perhaps that is more applicable to the vintage AOL walled garden experience?

    And the further scenario cited is that of the iPhone, where any device can be connected to the network, which is of course how any broadband service already works today.

    But if network operators are to believed, just providing basic bandwidth and speed necessitates some form of ‘smart’ network management. So even if someone runs a dumb network they still need smart people to operate and optimize it, and good software/hardware/infrastructure.

    Share
  10. @JL Plenty of smart people own and operate my chosen provider of electricity ( TXU here in Texas ) but they have no desire to “control” or “dictate” what I do with the juice – Can you imagine?!?!?! The more electricity I use, the bigger the monthly bill, that’s it.

    There is no mention of who can ( or cannot )use the electricity for Power company “approved” uses.

    Share
  11. The downstream leg is not the problem, really. Comcast made a decision very early in the build out to limit the channel capacity back to the headend. So, upload congestion started becoming the norm when torrent clients started the dance. Also, the growth of video uploads in general was never anticipated back in the day.

    The current best the Comcast can do is 90-120KBPS up. and, that’s a damn shame in this day and age.

    Share
  12. Om

    Tony is wrong about congestion on the 100 megabit pipes in Japan. Ain’t so, a lie repeated by many. I was on the original D.C. panel where a reporter misunderstood a chart.

    Dave Burstein

    Share
  13. @JL Plenty of smart people own and operate my chosen provider of electricity ( TXU here in Texas ) but they have no desire to “control” or “dictate” what I do with the juice – Can you imagine?!?!?! The more electricity I use, the bigger the monthly bill, that’s it. There is no mention of who can ( or cannot )use the electricity for Power company “approved” uses.

    So your power provider has never had brownouts, when demand outstrips capacity? Isn’t that similar to network congestion at peak times?

    Share
  14. @JL Plenty of smart people own and operate my chosen provider of electricity ( TXU here in Texas ) but they have no desire to “control” or “dictate” what I do with the juice – Can you imagine?!?!?! The more electricity I use, the bigger the monthly bill, that’s it. There is no mention of who can ( or cannot )use the electricity for Power company “approved” uses.

    Oh — and one other IMPORTANT difference. Your electricity is METERED, no flat rate. So are you suggesting that ISPs should move (back) to metered rates?

    Share
  15. Om,

    One important thing to note. Tony Werner talks as though Comcast has ceased its resetting of BitTorrent traffic immediately, but that is not the case. I asked a Comcast rep about this, and he said the practice will continue until the new method is in place. So don’t celebrate quite yet.

    See the updates at the end of this post: http://blogs.chron.com/techblog/archives/2008/03/comcast_and_bittorrent_actually_working_toget_1.html

    Share
  16. VTFootballGrad Monday, March 31, 2008

    It beats the alternative. I had to use a service that wouldn’t let you route their service and kicked you off the internet if you downloaded anything. I’ll take the bad comcast service over that terrible service any day.

    Share
  17. If Comcast had been forthright with its’ advertising to customers in the first place, the customer would know the limitations and make their buying decision based on the facts and not the fiction portrayed by Bill and Carolyn Slowski and the mercury like substance they call “High-Speed”. When people have taken Comcast at their word and later find out that they have been lied to based on the term unlimited meaning something different to Comcast, this is where Comcast gets themselves in trouble with truth in advertising.
    Many of the new products and services that Comcast has rolled out in the past couple of years are the biggest “Bandwidth Hogs” of all. With Comcast not preparing for their increased bandwidth usage in advance to the deployment of these new products and services and trying to place the blame for the bottlenecks on their users which use the most bandwidth that they paid for, I call that criminal.
    Comcast is now trying to placate their users by saying that they will only be slowing from ½ to 2% of their customers who they claim to be “Bandwidth Hogs” leading customers to believe that they won’t be affected by their so-called “Network Management”. Two days ago Comcast claimed that only ½% of their customers were “Bandwidth Hogs” … now the numbers have jumped to ½ to 2% of their customers. I trust Comcast about as much as I trust a fox in the henhouse.

    Share
  18. I am strongly opposed to bandwidth “management” as it has frequently been applied for p2p and other high bandwidth applications. In the case of Comcast, the real reason for their “management” is preemptive – it is intended to prevent traffic congestion before it happens.

    Intelligent network management is reactive, not preemptive. it should occur only when congestion actually begins to occur. This is congestion management. On those RARE OCCASIONS when congestion management is required, the managers should do several things:

    First of all, the network’s managers should, as you suggest, look at increasing available bandwidth if congestion is occurring regularly or for extended periods of time. There is no shortage of fiber available to Comcast.

    Secondly, the managers should look at what protocols can stand delay during congestion with little impact on the users. First target should be e-mail. Users will likely not even notice delays in this traffic. There are other protocols, including Comcast’s own network management traffic, that can be delayed with little impact on users.

    But in a correctly managed network, congestion should occur RARELY. If it occurs more frequently, the managers are simply not doing their job. And Comcast’s preemptive management philosophy is simply BAD network management. No professional I know would approve of this approach.

    Share
  19. Om –
    I posted a second comment to a previous “network management” post of yours the other day that applies directly to this. It looks like they are applying “fair queuing” or “max-min fairness” to the issue.

    My further thoughts with links to additional info on fair queuing and max/min – http://rfreeborn.blogspot.com/2008/03/traffic-shaping-network-management-and.html

    r.

    Share
  20. [...] Read the rest of this post Print all_things_di220:http://voices.allthingsd.com/20080401/om-4/ Sphere Comment Tagged: BitTorrent, Tony Werner, GigaOm, Om Malik, Voices, Comcast | permalink [...]

    Share
  21. There are newer technologies, such as P2P caching, that enable ISPs to intelligently manage P2P traffic, in a way that simultaneously provides a superior experience to everyone – P2P users and non-P2P users alike (or hogs and non-hogs, as the case may be).

    http://en.wikipedia.org/wiki/P2P_caching

    Share
  22. As long as they are fair, its fair. No protocol discrimination. When a hog on your pipe takes your bandwidth, you want Comcast on your side. Now if they want to mess with something, how about filtering SPAM! SPAM is probably screwing the net far more than Bittorrent, etc.

    Share
  23. I like this analogy (below) on rationing resources. This excerpt is taken from the NetEqualizer blog site. It is relevant to this discussion because many smaller ISPs around the world delegate bandwidth based on behavior without actually looking at a customers data. It sounds like Comcast may be adopting some of their techniques.

    The NetEqualizer methodology for application shaping has been agnostic with respect to type of data for quite some time. We have shown through our thousands of customers that you can effectively control and give priority to Internet traffic based on behavior. We did not feel comfortable with our Layer 7 application shaping techniques and hence we ceased to support that methodology almost two years ago. We now manage traffic as a resource much the same way a municipality would/should ration water if there was a shortage.

    Customers understand this, for example if you simply tell somebody they must share a resource such as: water, the internet , or butter (as in WWII), and that they may periodically get a reduced amount, they will likely agree that sharing the resource is better than one person getting all of the resource while others suffer.

    Well that is exactly what a NetEqualizer does with Internet resources albeit in real time. Internet bandwidth is very spiky, it comes and goes in milliseconds and there is no time for a quorum.

    taken from

    http://netequalizer.wordpress.com/2008/03/28/comcast-should-adopt-behavior-based-shaping-to-stay-out-of-trouble-opinion/

    Share
  24. [...] me his company would try and deal with the tiny number of subscribers who use most of the bandwidth by slowing down their connections during peak times. (Personally, I find that to be a distasteful solution, and I believe that folks should learn from [...]

    Share
  25. [...] Malik, Thursday, May 1, 2008 at 7:03 AM PT Comments (0) So all the noise, anger and finger pointing at Comcast’s cheap traffic tricks didn’t impact their broadband business. They just [...]

    Share
  26. [...] Malik, Thursday, May 1, 2008 at 7:03 AM PT Comments (0) So all the noise, anger and finger pointing at Comcast’s cheap traffic tricks didn’t impact their broadband business. They just [...]

    Share
  27. [...] has said it will start targeting bandwidth hogs, and this could be the way they do it. If you’re wondering if you’d have to pay extra, [...]

    Share
  28. [...] already know how Comcast plans to manage its network because Comcast CTO Tony Werner told Om how bandwdith hogs would experience slowdowns. In an emailed statement, Comcast spokeswoman Sena Fitzmaurice denied the traffic blocking. [...]

    Share
  29. [...] that 90 percent of P2P upload traffic isn’t managed at all. And by the end of this year, as Om has detailed, Comcast plans to shift the way it manages its network to slow down those using unreasonable [...]

    Share
  30. [...] you know I don’t care much for Comcast’s traffic managing ways — having written about it time and again — but this just seeks like an opportunistic and populist-baiting move [...]

    Share
  31. [...] year to stop its existing network managment practices, and tell its subscribers more about what the new network managment practices might entail. However, the FCC did not fine Comcast. Comcast said in a statement issued this morning: “We [...]

    Share
  32. [...] Compressing or managing data more efficiently is becoming increasingly important as providers attempt to clamp down on large amounts of traffic while consumers and corporations demand ever more bandwidth intensive [...]

    Share
  33. [...] maggior parte dei cap sono imposti in nome del network management, ma non si può negare che creano benefici economici per gli [...]

    Share
  34. [...] than 1 percent of its users. As expected the plan hews closely to what Om wrote about back in March in an interview with Comcast CTO Tony Werner. Essentially, folks using a lot of bandwidth at any one time will see their traffic slowed [...]

    Share
  35. [...] all:&nbspNews You’d think T-Mobile would have learned from all the hubbub surrounding the Comcast bandwidth throttling mess, and Rogers’ fights with Canadian customers over its paltry iPhone 3G plans. Customers want [...]

    Share
  36. [...] and with their video franchises threatened, newly independent Time Warner Cable and Comcast embraced the concept of metered broadband. Phone companies joined in, and now we are faced with a rather [...]

    Share
  37. [...] which got in trouble with the FCC for its P2P-throttling approach to network management, has now completed its plan that deals with bandwidth hogs by slowing all broadband traffic for heavy Internet users during times of network congestion. The nation’s largest cable [...]

    Share
  38. [...] to your home an incredibly frustrating experience.  And, this will happen, eventually…. read this. (Thanks [...]

    Share
  39. [...] money from their broadband pipes, generally in the name of needing to upgrade the network or to stop bandwidth hogs.  But in truth, caps and tiered plans are about limiting competition (especially from online [...]

    Share
  40. [...] money from their broadband pipes, generally in the name of needing to upgrade the network or to stop bandwidth hogs.  But in truth, caps and tiered plans are about limiting competition (especially from online [...]

    Share
  41. [...] money from their broadband pipes, generally in the name of needing to upgrade the network or to stop bandwidth hogs.  But in truth, caps and tiered plans are about limiting competition (especially from online [...]

    Share
  42. I think we’re all forgetting that Comcast charges egregious amounts for internet service and obviously spends next to nothing on customer service. They also outsource tech support so they’re not paying a premium there either. My point is that there’s no real reason why Comcast can’t provide virtually unlimited bandwidth for everyone.

    Think of it this way: Comcast moves into an area and spends X on setting up their bandwidth infrastructure. Now, imagine that they’ve covered an entire city. Every time they moved into a new part of town they had to have X amount of money to spend on setting up that infrastructure. After they’ve covered the whole city, they continue to make money that they would normally have spent on moving into new areas except now the whole city is covered. That money can now be used to add bandwidth to the areas they cover.

    The money is there. Comcast could be offering unlimited bandwidth if they wanted to but since they have no real competition in most areas where they are, they don’t have to.

    Share
  43. [...] as the one 80legs is offering is pretty interesting.  Since the pipes delivering the web to users can become congested at certain times of the day, finding programs and applications to take advantage of underutilized bandwidth would seem to be a [...]

    Share

Comments have been disabled for this post