54 Comments

Summary:

Level 3 is accusing 5 U.S. ISPs of using their market power to limit the amount of traffic that can get on their networks, thus degrading the consumer broadband experience in hopes of charging content providers money.

Level 3 Communications, a company that provides bandwidth for a wide variety of customers trying to get content from point A to point B on the internet, just accused five U.S. ISPs and one European ISP of using their market power to interfere with how traffic flows from Level 3 onto the ISPs’ last-mile network. The result is that customers of those ISPs experience degraded quality for services going over Level 3’s network.

This is a so-called peering problem. The topic has been in the news since early this year, when consumers began complaining about the quality of their Netflix, Hulu and Amazon Instant Video streams on networks like Comcast, AT&T, Verizon and Time Warner Cable. The issue is that at the interconnection points where Netflix traffic attempts to enter the last-mile ISP’s network, there isn’t enough capacity. Usually, when that happens, the transit provider or the content provider negotiate to add more capacity by opening up more ports. (I explain the process in this story.)

However, in recent months Level 3, Netflix and Cogent have all gone public accusing some ISPs of keeping those ports congested while trying to charge above-market rates for direct interconnection. Netflix has signed such a direct interconnection agreement with both Comcast and Verizon. But it isn’t happy about it and accuses the ISPs of abusing their market power to extract payments from content companies trying to serve the last mile.

In covering this issue, I’ve argued that the FCC needs to get data on these peering agreements and practices before we can prove that there’s a problem. It’s clear there is consumer harm, but what’s unclear is whether ISPs are actively blocking traffic, or hurting specific providers. It’s also unclear if ISPs are charging abusive rates to get direct interconnection agreements.

These peering agreements are deeply held secrets, which means the FCC will have to force providers to disclose the terms of their agreements and what traffic looks like on their networks. By sharing some of its network details, Level 3 is offering us a transparency strip tease, giving us (and the FCC) a glimpse of the data it has without naming names.

From the Level 3 post:

“The average utilization across all those interconnected ports is 36 perfect. So you might be asking — what is all the fuss about with peering?…Well, our peers fall into two broad categories; global or regional Internet Services providers like Level 3…and Broadband consumer networks like AT&T. If I use that distinction as a filter to look at congested ports, the story looks very different.

A port that is on average utilized at 90 percent will be saturated, dropping packets, for several hours a day. We have congested ports saturated to those levels with 12 of our 51 peers. Six of those 12 have a single congested port, and we are both (Level 3 and our peer) in the process of making upgrades — this is business as usual and happens occasionally as traffic swings around the Internet as customers change providers.

That leaves the remaining six peers with congestion on almost all of the interconnect ports between us. Congestion that is permanent, has been in place for well over a year and where our peer refuses to augment capacity. They are deliberately harming the service they deliver to their paying customers. They are not allowing us to fulfill the requests their customers make for content.

Five of those congested peers are in the United States and one is in Europe. There are none in any other part of the world. All six are large Broadband consumer networks with a dominant or exclusive market share in their local market. In countries or markets where consumers have multiple Broadband choices (like the UK) there are no congested peers.”

I’d love to see Cogent, Google and other providers release their data next, so even if the FCC doesn’t want to pursue this, a growing cry of consumer outrage could push the agency to do something about a very real and difficult problem that’s crippling access to video content on 5 U.S. broadband networks. Level 3 didn’t name names, but based on the deals Netflix has signed and the complaints it has made about AT&T, I’m confident that AT&T, Verizon and Comcast are among the five.

  1. While it may cost next to nothing to open a larger port on a router, accepting additional inbound traffic implies an obligation to deliver it to a destination. It would make no sense to accept traffic if it will be dropped at a point of congestion further downstream, e.g., at a neighborhood router. And surely it is likely that such congestion would exist, or else everyone has previously been overpaying to build capacity that was not needed. Now that additional capacity is needed due to the rising popularity of streaming video, someone needs to pay for it, and it is hard to argue how it should be anyone other than the originators of that data.

    I know there is an intuitive notion that paying for an Internet connection, has somehow already paid for the ability to fill that pipe from any source you choose. But, that simply is not how the Internet works. The sender also needs to pay according to the volume they need delivered, like pretty much every other transportation system. It doesn’t take that much thought to realize why it must work that way.

    Of course Level 3, NetFlix, etc. will try to get consumer opinion involved in these business to business negotiations. But, what is the end result if they succeed? Every ISP customer will see their bill go up to subsidize the people who watch Netflix, Hulu, etc.

    Share
    1. Call a shovel a shovel Monday, May 5, 2014

      “Now that additional capacity is needed due to the rising popularity of streaming video, someone needs to pay for it, and it is hard to argue how it should be anyone other than the originators of that data.”
      The subscribers are already paying for it, 10meg, 20meg, whatever the ISP is promising and then intentionally not delivering.

      Share
    2. Dilip Andrade Monday, May 5, 2014

      Tim,

      You say that it’s hard to argue that someone other than the originator of the data should pay for the additional capacity, but isn’t the consumer of the data paying their service provider for access to that very infrastructure?

      Do we have to move to a marketplace in which Netflix will end up charging differentiated fees based on which ISP you want to use? Would that really be any better?

      In a marketplace in which most ISPs are either a monopoly (or at best are part of a duopoly) isn’t the role of the service provider to provide their customer access to services?

      Share
      1. The consumer pays for access, and for the volume of data they send, but not for the volume they receive. As a practical matter, you have no control over what data is sent to your IP address. But, you do not generally get bombarded with unwanted data, because someone has to pay to do that. If there were no charge for sending, there would be nothing to stop NetFlix from saturating your connection with additional content, e.g., trailers and promotions for other products, simply to block you (or your neighbor) from doing anything else with their connection? Making senders pay insures that people are (mostly) sent only what they want, as efficiently as possible.

        I agree the monopoly power of the ISP’s is a real risk. But, one reason we see so little competition is it’s such a crappy business. Google + Facebook combined are worth more than ATT + Verizon + Comcast. No wonder nobody is jumping into the ISP business. Not sure how anyone thinks that preventing ISP’s from collecting more revenue from content providers is going to improve service.

        Also, most people do have a choice between cable and DSL. So, the monopoly is really only on high speed video streaming, which the cable companies had a monopoly on long before anyone was streaming video over the Internet. Most of the truly innovative modes of communication which the Internet has enabled, like this www site, work just fine on DSL.

        Share
        1. Tim, you’re speaking as if you’re from planet comcast. The vast majority of home consumers on earth don’t sign up with an ISP for the amount of data they are allowed to send. Nobody cares, because mouse clicks and e-mail falls into the negligible bandwidth bucket. ISP heavily advertise the download speed with no caps to homes, upload is in fine print if listed. Call it like it is Tim. Dominant ISP providers want to charge customers premium for higher bandwidth service, and they want to charge streaming sources for higher bandwidth delivery, because they can. Their dominant position can threaten the existence of content providers, so they use it to leverage money out of them at the detriment of consumers and content providers. Not only is the monopolistic powers of ISP a real risk, but they are already acting on it as evidence here. The argument that bandwidth is so expensive for ISP’s, is the same argument AT&T and Verizon use to justify charging $0.19 per incoming text message for those not on a text plan. They charge crazy prices, because they can, and justify it to the public by saying infrastructure support costs are high.

          Share
          1. I get the point that everyone hates their ISP. But, there is no evidence in these peering disputes that they are yet abusing their monopoly power. There has always been a fast lane for content providers who buy faster connections to the Internet. The only difference is that some huge content providers are now dealing directly with ISP’s, rather than via 3rd party aggregators (transit providers and CDN’s). If Netflix accounts for 30% of Internet traffic, how much aggregation can be going on anyway? You just have Cogent or Level 3 acting as an unnecessary middle man.

            I agree there is a theoretical risk. Yet Comcast, which has 50 times as many employees as Netflix, is worth only 7 times as much. Netflix earns around $2 million in profit per employee for doing exactly what? Using networks operated by others to distribute content created by others so we can watch more TV.

            Share
            1. Patrick W. Gilmore Tuesday, May 6, 2014

              Tim:

              You are highly confused.

              First, “there is no evidence in these peering disputes that they are yet abusing their monopoly power”. Bullshit. When was the last time you negotiated a peering agreement with a monopoly provider – in the US or anywhere?

              Second, Netflix’s 30% is of US broadband during certain hours. The Internet is a lot more than home broadband from 6 to midnight.

              Third, content has been peering with eyeballs for -decades-. This is not new. What is new is that broadband providers are monopolies, while dialup ISPs were not.

              Fourth, fast lane for people who buy faster connections? Possibly, but that is a complex issue that you gloss over without the slightest inkling of what is behind your statement. Plus, that doesn’t work any more. Content providers can buy terabits from transit providers and still get a trickle into Comcast & Verizon.

              I could go on, but something has become obvious to me, so allow me one final question: Which broadband provider pays your salary?

              Share
            2. Isn’t it obvious that if I run a www site off a 100 Mbps connection, my visitors will get a “faster lane” to my content than if I have only a 10 Mbps connection? Why is that so hard to understand? Most people can accept the fact that a content provider is responsible for capacity on the last hop to their content, and how much they are willing to pay for that determines the quality of service their customers will see.

              If transit providers are selling terabit connections but only delivering a trickle, they are ripping off their customers. No wonder big content providers are choosing to deal directly with networks who can in fact deliver the stated capacity to the desired destinations.

              I have never worked for the broadband industry, but have been writing real time video software for more than 20 years. Before dial-up modem became popular, most people were connected to the Internet by 10 Mbps ethernet, but with 100’s of those connections sharing a 45 Mbps connection at best, and a nationwide backbone that was not much larger than that. So, there was a big issue of multiple bottlenecks and how that capacity could be fairly allocated.

              I think the dial-up modem period gave people a faulty idea about how the Internet works. As if everyone could just pay for first hop capacity and the rest would sort itself out. In fact, that has worked remarkably well through the system of peering worked out privately among network operators. I think the main reason that is breaking down now is because you have a few applications like Netflix that require orders of magnitude more bandwidth than other applications. Dumping it all together and saying fight it out is not going to work.

              Share
            3. I’d give up on the poor deserving comcast argument…. At a $135 biillion, they’re bigger than Ford, GM and Harley combined. It’s hard to feel sorry for a company that charges customers triple digits each month for mereTV channels and abuses their dominant position. I dont’ think even the best paid Comcast shill could sell that argument. Comcast charges customers for internet access, with an expected 10-20 Mbps service that rarely materializes since they’ve imposed their choking the pipe. Netflix streaming is well below that at 2Mbps for SD, 5Mbps for HD. Regardless, Comcast double charges Netflix for internet service customers have already paid for. Why? Because they can get away with it. They’re an abusive monopoly. What is Netflix going to do? not pay and hemorrhage customers until the company bleeds to death? It’s not like there is any choice in the matter. Meanwhile, Google offers 1GB service for about about the same price as Comcast charges for 50x to 100x slower speeds, without throttling. Comcast stagnates on internet services as a monopoly does, Impacting customers, alternate content providers, and innovative company startups. A merger with Time Warner, will increase market dominance, and content ownership furthering their abusive use of power to the detriment of society.

              Share
            4. I think you are right that the 10-20 Mbps service rarely materializes, but I don’t believe there is any switch they can flip to change that. I would guess they simply have not built out their network with enough capacity to fill all of those modems simultaneously. The old rule of thumb used to be a 30:1 contention ratio, meaning you could only get full access speed if 29 out 30 customers were idle. Or, put another way, if everyone was active, you could get about 1/30th of your access speed.

              This turned out to work okay for classic www surfing. Even if everyone was sitting at their computer, there was still a lot of idle time, with people reading what they downloaded, commenting, searching, etc.. High quality long form video is an entirely different matter. You can turn it on, and let it stream for an hour while you fall asleep. The Internet is simply not scaled for that.

              I don’t know what it would cost to scale it up, but it could turn out that the Netflix business model simply does not scale. Initially, they were able to exploit idle capacity that was paid for by www sites people don’t use and consumers who don’t use their broadband. But, as their business has grown, they have eaten up the idle capacity, and will increasingly bear the full cost of transport themselves if they want to scale up.

              Share
            5. Patrick W. Gilmore Friday, May 9, 2014

              Tim:

              Please stop guessing. You clearly do not understand how large broadband providers, especially comcast, are architected.

              There is a trivial switch BB providers can flip to greatly increase their capacity – simply not congest their transit pipes. Would this guarantee every customer gets exactly the speed they purchased? Of course not. But the difference would be night & day.

              If there were no monopolies, I wouldn’t care. BB providers could congest all they want, and people could choose providers based on price, service, features, etc. In fact, Sky in the UK did exactly this, and still sold to lots of users because of price. However, since users have no choice, I do care. And I find it sad you do not.

              As for whether NF’s business model scales, again, please stop guessing. Your statements are provably false by the simple fact where there is no monopoly in broadband, things work many times better. So I would appreciate it if you quit making blanket statements which are trivially shown to be factually incorrect.


              TTFN,
              patrick

              Share
    3. “It would make no sense to accept traffic if it will be dropped at a point of congestion further downstream, e.g., at a neighborhood router. And surely it is likely that such congestion would exist, or else everyone has previously been overpaying to build capacity that was not needed.”

      Your right — their is no reason to increase capacity at the border if that traffic can’t reach the customer edge anyways due to choke points elsewhere on the network.

      Howver, immediatly after Netflix moved their Comcast-bound traffic away from Cogent to paid peering, streaming performance improved and the average Netflix stream to a Comcast customer improved dramatically in terms of throughput and quality. Comcast did not need to spend billions to upgrade their neighborhood-level networks (or metro or regional networks, for that matter). All they needed to do is provision more ports on their existing external-facing border routers (i.e. new 10G/40G/100G ports with Netflix). The rest of their network was demonstrably NOT the choke point.

      Complaining about imbalanced peering ratios, while running a network that provides assymetric speeds to the vast majority if its customers, is not logically sound; Comcast will ALWAYS have inbalanced traffic at their border (more in than out), and their peering policies should reflect that reality. But their market power is large enough that common sense does not dictate policy; they just want to double-bill as much traffic as they can out of greed because they can abuse their market position to get away with it at the expense of their customers.

      My question is, now that Comcast is getting paid by Netflix to flood my broadband connection with traffic, when is Comcast going to start paying *me* to receive that unfair traffic on my home cable modem? Heck, I’ll be generous: Comcast won’t even have to pay me to be their customer… I’ll accept the offer that Cogent & Level3 have been making to Comcast… just upgrade the connection between our networks to a fatter pipe at no cost to the recipient of the traffic — i.e. give me a free speed upgrade!. (To those that don’t get the joke, Comcast previously argued that Netflix was flooding their broadband network with unfair traffic… and when I watch Netflix on my Comcast connection at home, Comcast is therefore “flooding” my home network with “unfair” traffic).

      -Adam

      Share
      1. Haha, yes, look at your own peering ratio! Why isn’t Comcast paying you? The answer of course is that Comcast made a deal with whoever sent the data to deliver it to you, so once you have it, their job is done. If, in fact, you maintained a private network to serve an entire apartment building, Comcast probably would pay you to complete that last mile.

        As to your point that Comcast in fact had excess capacity already, that is like saying the grocery store should give away whatever is on the shelves, because it is there already, and costs them nothing more to let you walk away with it.

        Now, really, I have no idea whether Comcast is gouging everyone unfairly. The fact that there are no major players in the ISP business unless they have some other big business to helps subsidize it tells me it is probably not a great business. Or, just look at the valuations of the companies. ATT + Verizon + Comcast combined are worth less than Google + Facebook. That’s why there is so little competition.

        Then, people get duped into waiving the Network Neutrality flag as if it is something about keeping the Internet safe for the little guy, when it is really all about the biggest players, Netflix, Google, Facebook, etc., padding their bottom lines.

        Share
        1. No, there is a dearth of competition in the last mile, because they are regulated monopolies. You do realize that cable companies are prohibited from having overlapping franchises, right? By and large, you can have one MSO and by default one telco in a typical residential scenario.

          This crappy, last-mile ISP/telco/video business you say is so terrible produces billions and billions of cash flow and profit. Don’t be fooled by valuations, which are largely driven by growth. Comcast, for example, is a company that produces tons of cash.

          Share
    4. Jason McKenzie Tuesday, May 6, 2014

      I know how the internet works, but that’s not the point. I pay for service BECAUSE I want to watch Hulu, listen to Spotify, etc. If TimeWarner is throttling uplevel providers and degrading my service, I will just go back to not having TV at all, and reducing my service level as I won’t need the additional speed. Not to mention, we’re about to have Google Fiber in Austin, so with my experience of both TimeWarner AND AT&T, I cannot WAIT to give someone else my money.

      Share
      1. Jason McKenzie Tuesday, May 6, 2014

        Also, I would HAPPILY pay more for the service I want… just as soon as I get my money’s worth for the service I have already paid for. To me it’s not about money, it’s about satisfying my expectations as a customer first, BEFORE asking me for more money (or expecting it from someone else).

        Share
    5. Your comments simply ignore fact that ISPs oversell the capacity and start crying when that much capacity is used. End users DO PAY for the capacity and in that ISPs promise to deliver their packets to any destination and vice versa.

      Share
  2. Whenever there’s a peering dispute on the Internet, chances are high that Cogent is involved. Dog bites man, no film at 11:00.

    Share
    1. So now L3 follows that well-worn path.

      Share
      1. Ryan Singel Monday, May 5, 2014

        Whenever an ISP’s crappy practices are exposed, you always show up up to defend them.

        Share
        1. Robert J. Berger Friday, May 9, 2014

          Thank you Ryan, I was about to say the same thing. Clear that Tim works for the Oligopolists as well.

          Share
        2. Claiming an ISP is involved in crappy dealing isn’t the same as proving it, but the evidence against Cogent is clear and convincing.

          Share
  3. Patrick W. Gilmore Monday, May 5, 2014

    Tim:

    Just so I am clear, when I pay my triple-digit monthly fee to Comcast for my sub-triple-digit Mbps link, your argument is that Comcast has no need to actually allow me to download at those speeds unless L3, Netflix, Google, Akamai, etc. pay Comcast even more money?

    If Exxon tells you that you can’t have any gas unless GM also pays them, how would you feel?

    As for “every isp customer will see their bill go up”, I call BS. ONLY where there is Market Power (i.e. monopolies) are prices as high as they are in the US – yet we still cannot get the service for which we pay. In places like the UK, prices are lower and service is better.

    (And don’t bring up things like “the UK is not as big as the US”. Rural parts of the UK pay less than downtown Manhattan, get better service, and content providers still do not pay ISPs.)


    TTFN,
    patrick

    Share
  4. Patrick W. Gilmore Monday, May 5, 2014

    One last question for Tim: What other transportation system has both side pay?

    If I send you a package on FedEx, you do not pay. If I call you, you do not pay. If I call your 800#, I do not pay. If I take a train to you, you do not pay….

    Yeah, I don’t see any other transportation system (as if analogies like this mattered anyway) where both sides pay. Could you enlighten me please?


    TTFN,
    patrick

    Share
    1. You still need to pay for a telephone in order to receive a phone call. It really isn’t both sides paying. You are paying for what you send; the sender is paying for what you receive. Just like with a telephone. Otherwise, people could run up your bill by sending you stuff you did not want.

      This is how the Internet has always worked. If I want to operate a www site, I need to buy a connection to the Internet. If I buy a 100 Mbps connection my customers will get faster downloads than if I buy a 10 Mbps connection.

      Share
      1. @TIM

        Main point being, if you pay for 100Mps for your customers, but they decide to throttle your connection because you stream video. Then what is the point of this 100Mps plan?

        Maybe you are suggesting 100Mps for https protocol, 2mbps for torrent protocol, 500kpbs for VOIP protocol. That don’t make sense.

        The fact is this, if you offer 100Mps whether to sender, receiver or the dog or cat, you honor that. Whether I saturate 100Mps every single second is up to buyer.

        Share
        1. I was talking about what kind of connection the content provider buys for themselves. If the www site you want to visit has a 100 Mbps connection, and 100 simultaneous visitors, you are not going to get 100 Mbps of content from them, even if you have a 100 Mbps connection. And, it is not the business of your ISP to go upgrade their connection for them.

          Share
      2. Patrick W. Gilmore Tuesday, May 6, 2014

        How the Internet has always worked is there was no monopoly on eyeball networks.

        Do you even know how the first peering dispute, where BBN threatened Above.Net, Exodus, and Global Center, was resolved? (Interestingly, none of those companies exist any more, although I could argue that Zayo is kinda-sortta Above.net.) Hint: Not by content paying the eyeballs, because BBN was not a monopoly.

        The reasoning behind the peering dispute was something called bitmiles based on something call hot-potato routing. Content broke that back in the 90s, and CDNs like Akamai invalidate the bitmile argument by their very existence. If you do not understand the preceding couple of sentences, please stop posting as you are ignorant of the basics and are therefore unqualified to comment on the conclusions.

        As for the phone analogy, I note that I do not have to pay more on my phone if I get more incoming calls, but I have to pay more for a bigger broadband pipe.

        But let’s go ahead and use phone analogies: When I make a call, I pay for it. (Not talking 800 #s here.) If I place the call and you do all the talking, I still pay for it. How many content sites push traffic to users, and how many wait for a user to request the content? Put another way, the user is the ‘calling’ party. Why is content paying again?

        Or how about content vs. distribution. The original mass distribution channel was book & newspapers. Do writers pay publishers, or the other way around. Then there was radio and TV – you know, like cable providers. When a content provider, e.g. studios, make content, do they pay cable providers to show their content? No, Comcast pays ESPN. Hrmm, wait a minute….

        Yeah, analogies work great. Wanna try some more?

        Share
        1. Data originators have to pay to insure they send only what people want, and as efficiently as possible. Otherwise, everybody would be blasting content at you whether you want it or not. The network would be filled with a lot of garbage and the cost for that would be passed on to everyone.

          Share
          1. Patrick W. Gilmore Wednesday, May 7, 2014

            Huh? Data originators did not pay broadband providers for decades, yet people who did not want the content did not get it.

            I think you are confused again…..

            No, you definitely are confused again. This is the Internet, not the postal system. You can’t make a video appear on my screen just by sending me the video.

            Share
            1. Data originators pay at the point where they connect to the Internet. As for making stuff you don’t want appear on your screen, it happens all the time when you visit most www sites. And, people can drop packets on your doorstep all day long, but the don’t because it costs them money, unless they control a bot army.

              Share
            2. Patrick W. Gilmore Friday, May 9, 2014

              Tim:

              Is “banner ads are unwanted” really the best you can do? Your previous arguments, while factually incorrect or patently absurd, showed you were at least trying. This is beneath you.

              Instead of boring everyone with a fight over semantics, allow me to just state that if you type “www.example.com” into your browser, or you click on a link, you -requested- all data that shows up. if you decide afterwards you didn’t like it (for instance, your comment on this blog), that doesn’t mean you didn’t request it.

              As for dropping packets on my doorstep, again, beneath you. Better step up your game or whatever BB provider is paying you might get upset.

              Share
        2. The problem with Comcast paying ESPN is it means that Comcast is choosing what content to make available to its subscribers. What I like about the Internet is that anyone can publish content, as long as they are willing to pay the cost of transport. For most content, the cost of transport is small enough that there is a fair chance the provider can cover the cost with a small charge to the end-recipient, e.g., their attention to some ads. Streaming high quality long form video begins to cost real money. It is no surprise that Netflix would like to see their transport costs subsidized by broadband subscriber fees rather than having to pass that cost directly to their own customers.

          Share
          1. Patrick W. Gilmore Wednesday, May 7, 2014

            Analogies are not perfect, not terribly surprising. Shall we ask the assembled group which is closer to reality?

            And using charged words like ‘subsidize’ is a cute trick, but avoids the root of the problem. Realize that if I am a website with a tiny footprint sending Kbps or less to a few subscribers, I still cannot get good quality connectivity to most US broadband end users without paying the broadband provider directly.

            Personally, I don’t really care if they choose that – as long as I can choose another broadband provider. Private businesses should be allowed to do as they please. But monopolies, companies with Market Power, should not be allowed to charge Rents. (Those are economic terms any first-year econ major can explain to you, or you can just use Wikipedia.)

            Physical infrastructure is a natural monopoly. See previous post about electricity. If the physical wires were regulated, and the services provided were open to a free market, we would not have this discussion. That is not hypothetical, it has happened in other countries (see previous post about UK).

            But that will never happen in the US, because broadband providers know the vast majority of their profit is based on their monopoly (duopoloy) position, not on market forces (e.g. providing a better service than their competition and being paid for it).

            Same question I posed to Dan (who refuses to answer): Why do you not want a free market with consumer choice? Why do you think monopolistic capture of the market is a better idea?


            TTFN,
            partick

            Share
            1. “Realize that if I am a website with a tiny footprint sending Kbps or less to a few subscribers, I still cannot get good quality connectivity to most US broadband end users”

              I agree this is a serious problem. But, it seems like the solution is to let you pay the broadband provider. Of course, a small site wouldn’t need to deal directly with each broadband provider, but could enter into an SLA with a 3rd party aggregator, who could enter into an SLA with the ISP’s that allowed them to deliver on their SLA with you. Because your data volume is small, the charges should be negligible. It is the high volume content providers who are frightened of this, not because it is unfair, but because it is fair.

              I like the idea of municipal broadband networks operated as public utilities. Within the context of the current system, I like that idea that senders pay the transport cost to reach the end users.

              Share
  5. .> a growing cry of consumer outrage

    Hmm, what might the slogan be? OPEN UP NOW! ?

    Share
  6. Claude Rallins Monday, May 5, 2014

    @ Tim

    By your logic, if I call someone, and have my wife pick-up in another room so we can both talk. Or, when I turn a call into a conference call at the office… then I should pay more for the ‘volume’ of the content.. and not the call itself.

    And, movie theaters should charge by the length of the film, instead of a flat rate ticket price, or subscription fee?

    This model would be a big hit on Broadway. Not to mention LIVE concerts. — A show goes longer than expected so charge a Added Performance fee, of $25 a head. Perhaps a discount on encores would be in order?

    Bottom line, ISPs Do Not pay ‘by the volume’ for public easement access. Nor do they pay US for the volume of content traffic they send over the public airwaves.

    You can’t have it both ways, but don’t let that stop (you) / ISPs from trying.

    In the end, the People need to reclaim the Public Airwaves, and employ the Interstate Highway model to the Information Superhighway. Primarily because the fragmented U.S. ISP model is performing so poorly on the global competitive stage, while charging amongts the highest rates… and “asking” / (buying policy), to charge even more.

    Finally, your wealth comparison is equally flawed… Google and Facebook are global, and service billions of people. ISPs are national, regional players, serving millions.

    Share
  7. Tim,

    Consumers pay for what they receive. I pay for a 2M, 6M ,10M , or 20M… pipe to my house (whether it is DSL or cable). That is the receive bandwidth not he upload or transmit bandwidth. I don’t even care what my upload speed for the most part. All I transmit is small packets with HTTP requests. Then I wait, and wait, and wait for all that data that I requested to be received. This model works for everything else we buy. Electricity, water, gas, groceries… We pay for what we receive.

    A system whereby the content provider must pay for transport will only serve to limit what the consumer can get. The consumer has no real ability to affect the traffic engineer practices of the ISPs and is stuck with sub-par performance. At best there are only 2 options (DSL or Cable). Not real competition in the last mile delivery market.

    Worse yet, the small ISPs are no better off than the customers of the big ISPs. A small locally owned ISP (serving maybe thousands) usually does not have the facilities to reach a Giga-pop on their own and probably must buy internet from one of the 5 ISPs that Level 3 is referencing. They can’t provide any better quality service to their customers than what the big guys provide to them. If the content providers don’t pay, then everyone suffers due to the traffic engineering policies (i.e business practices) of the big 5. Until the content providers and the big ISPs play nice, the FCC will need to sort thing out. I just hope the FCC will consider the consumer that they are supposed to protect.

    Share
    1. streamingmediablog Tuesday, May 6, 2014

      “A system whereby the content provider must pay for transport will only serve to limit what the consumer can get.”

      This is the way it has worked for 20 years. I get that some argue it should not work that way, and that’s fine to debate, but too many people think what’s taking place is new, when it is the norm. Google, Microsoft, AOL, Yahoo etc. are all content providers that have been doing paid interconnects for a really long time.

      Also, not that Level 3 is not asking for free access to the ISPs, they simply want “fair” terms.

      Netflix wants it all for FREE. Big difference between the two.

      Share
      1. Patrick W. Gilmore Tuesday, May 6, 2014

        Oh, Dan, I was wondering when you would show up.

        Twenty years ago, none of the big providers (Above.net, Exodus, Global Center) paid for private peering. BBN tried to make them do it 16 years ago, and they did not. Which means your very first statement is false-to-fact.

        You could argue they paid for transit, which they all did. But there is a difference between paying for access to the entire Internet and paying for access to a single network. Trying to equate them is disingenuous at best.

        As for the rest, see my other mentions about countries without monopoly providers. Strangely, things seem to “just work” when the physical infrastructure is treated like a utility & the services on top of that are provided in a fair market. Everyone is happier, except maybe the people who had monopolies.

        Any particular reason you are against a market-based solution to these issues?

        Share
        1. streamingmediablog Wednesday, May 7, 2014

          AOL and Yahoo were doing paid interconnect deals with ISPs in the late 90’s.

          “market-based solution”. So far neither Netflix, Level 3 or Cogent have outlined, detailed or defined any type of “solution”, technical or commercial. They keep saying they want changes, but none have outlined what that means. They talk vague, high-level terminology. No details.

          Netflix and Cogent want access for free. Level 3 is ok with paying, but wants “fair” terms. But they won’t define what thet means. How will it be monitored? Regulated? Based on what technical terms?

          What I am against is companies complaning they want change, but then not outlining any details on how that change could happen, what it will look like, what impact it will have on their business (with numbers) etc. – yet they want the governement to regulate it.

          Share
    2. In your examples where receivers pay, the shipper is also the provider. You are paying the electric company for the production of electricity as well as the transport. If the Internet were like AOL, you would be right. But, the Internet is not like AOL. Anyone may serve Internet content simply by purchasing a connection to the Internet. But, they must purchase that connection themselves. The ISP’s do not choose which content providers to subsidize.

      Share
      1. Patrick W. Gilmore Tuesday, May 6, 2014

        Tim, Tim, Tim, Tim….

        First, I do not have to buy electricity from my electric company. The wires are regulated, the power can be bought from anyone, and it doesn’t matter which company supplies the power, I pay the same for the wires.

        As for the rest, even ISPs that buy transit cannot get their bits into VZ, Comcast, AT&T, etc. without _also_ paying the broadband provider.

        Or were you arguing that every content provider should pay each broadband provider individually? And those broadband providers should never accept bits without getting paid?

        I take it back, that is EXACTLY what you are arguing.

        Which broadband provider pays your salary again?

        Share
  8. Paging Dr FeelGoogleSP, Paging Dr FeelGoogleSP, pickup on line 3 you are wanted by the Internet,

    Share
  9. streamingmediablog Tuesday, May 6, 2014

    “what’s unclear is whether ISPs are actively blocking traffic, or hurting specific providers”

    That’s not unclear. Comcast is not allowed, by law, to do any kind of prioritization when it comes to delivering content on their network. No ISP has been accused by any provider, including Netflix, of “blocking” or “throttling” traffic.

    This isn’t about “blocking” or “slowing” down traffic, it’s about capacity. If a provider does not have enough capacity, then there are quality issues associated with that lack of capacity. But it has nothing to do with “blocking” traffic.

    Share
    1. Patrick W. Gilmore Tuesday, May 6, 2014

      Dan:

      While I’ll grant you that blocking accusations are exceedingly rare, this is about slowing down traffic. The method used is simply congestion, which you say is “about capacity”, but the result is the same.

      Comcast’s transit link graphs have – probably improperly – been posted public. They clearly show zero capacity. Comcast had plenty of backbone & other capacity, but intentionally did not upgrade their transit.

      Why did they do this? Some say it is because they wanted to force content providers to pay for peering. Doesn’t matter how much transit the content provider buy, if Comcast’s transit is full, more bits will not get to end user without paying Comcast for peering. Others argue .. uh, actually, I don’t even remember what the other arguments were.

      You can call this ‘capacity’ instead of ‘slowdown’, but the results are the same. You pay for peering (“fast lane”), you can deliver bits. You don’t, you don’t.

      Perhaps more importantly, without NN, that is the way everything will be. The “not fast lane” will suck, so everyone will be presented with the choice of pay, or things will not work.

      For a more eloquent explanation, try CGP Grey: .


      TTFN,
      patrick

      Share
      1. streamingmediablog Tuesday, May 6, 2014

        Not how everything “will be”, it’s how it’s been for years. ISPs have public peering policies. (comcast.com/peering). It says when they will traffic for free and when they won’t.

        I don’t see Google complaining about these same deals they have done with the ISPs. Or Yahoo. Or Microsoft. Or AOL. They all understand that there are cost associated with building out your own CDN. Even Level 3 in their dispute with Comcast didn’t say they wanted it for free, they just wanted fair terms.

        Netflix wants it for FREE. No one seems to mention that. Does free sound rational? Of course now. Both sides should come to a business agreement to add capacity, which is what Netflix and Comcast and Level 3 and Comcast have done – a business agreement.

        Level 3 wants it regulated but won’t say how they want it regulated. Price? Quality? Monitored how? They won’t say.

        Netflix is saying they should get all the capacity they want, with all ISPs, at no cost to them. That’s not a fair argument.

        Share
        1. Patrick W. Gilmore Wednesday, May 7, 2014

          Dan:

          First, I challenge your assumption that ‘free’ is not reasonable.

          Let’s flip this around: No one seems to mention that AT&T, Verizon, Comcast, TWC, etc. want the content companies to deliver that content for free. Does free sound rational? To quote you from above: “Of course now”! (Er, “not”. :)

          But perhaps more importantly, where there is a functioning market, as the conical example of the UK proves, “free” is _exactly_ what the peering is worth. Not because there is no value, but precisely because both sides see value.

          In fact, in the US where there is no Market Power, no monopoly, then free is still the price the market decides.

          Simply saying “of course no[t]” is not an argument, it is an assumption. One that you have no actual experience or competence to make. But make it you do, with exactly zero supporting logic. It is a good gambit, one that many people fall for.

          As for the idea that this should be a business agreement, we agree. Unfortunately, the fact L3 / NF pay people like CC proves there is a functioning market because that is a “business agreement” is circular at best. NF agreed to pay CC because They Had No Choice. Companies with Market Power frequently charge Rents by restricting choice. Using that lack of choice to prove there is a functioning market is absurd.

          When two companies with a mutual customer have a disagreement, usually the mutual customer votes with their wallet. Where I live (and millions of others), I have exactly one broadband provider, but I can use iTunes, NF, Hulu, iPlayer, etc. for streaming. If I cannot get good streaming from one, I -must- move to another streaming provider because I -cannot- change BB providers.

          That is not choice. That is not a functioning market.

          Pardon me, but I seem to have forgotten why you think restricting consumer choice and lack of a functioning market is a good idea? Care to enlighten me?


          TTFN,
          patrick

          Share
          1. streamingmediablog Wednesday, May 7, 2014

            Why isn’t Google complaining when they pay? Or AOL? Or Yahoo? or Microsoft? Or when Akamai paid $100M to get access to AT&T’s network? None of them are asking for free. The reason for this is that they know free is not “reasonable”. They understand there are costs associated with building out their own CDN.

            Even Level 3 isn’t asking for free.

            Share
            1. Patrick W. Gilmore Wednesday, May 7, 2014

              I note you do not answer the question. But I expected that, the answer to most of these questions would not suit your particular agenda, so why bother answering when you can distort the issue?

              Why did $FOO pay to access $BAR? Because the latter made it impossible to do business without paying. This works outside the Internet too.

              The difference for our discussion is that this is not an open market. If I run a content site, I have no choice but to pay broadband providers because no matter how good my product, pricing, service, features, etc. are, it will be irrelevant if there is massive packet loss getting to the end users.

              And those end users are a captive audience behind a monopoly or, if they are lucky, a duopoly. They cannot decide “I like patrick better than BB provider, so I’ll change providers”. They Have No Choice, so they “change” me, unless I pay.

              That is not a free market, that is not consumer choice.

              I’m getting a massive sense of deja vu. Oh, wait, I’ve explained this about 5 times, yet you still will not explain why you are against having a market-based solution or consumer choice. Just like looking at Comcast’s congested transit links, not hard to figure out why.


              TTFN,
              patrick

              Share
            2. streamingmediablog Thursday, May 8, 2014

              “If I run a content site, I have no choice but to pay broadband providers ”

              Not true. MOST content owners don’t pay broadband providers, they pay CDNs.

              I’m not for or against a “market-based solution”, because to date, no one has suggsted an alternative, with details.

              Share
            3. Patrick W. Gilmore Thursday, May 8, 2014

              First, MOST content owners pay a single transit provider. There are far, far, far more than 100K websites out there, yet there are far fewer than 50K customers amongst all CDNs on the planet. Unfortunately, most of them also have pretty lousy connectivity to much of the US user base.

              Back on topic, saying “content owners don’t have to pay BB providers, they pay CDNs” is evasive and, frankly, silly. So the CDNs pay BB providers. Doesn’t change the underlying point.

              Finally, multiple people have suggested multiple alternatives, including details. I myself have mentioned the UK system about half a dozen times. How much more detailed you can get than a production system serving 10s of millions of people?

              Feel free to ask some other off-topic question to make a blanket (if factually incorrect) statement, since I can’t really expect you to actually address any of the substantive points.

              Share
            4. streamingmediablog Monday, May 12, 2014

              File your “multiple alternatives” with the FCC. Make it official.

              “First, MOST content owners pay a single transit provider. ”

              No, they don’t. If you don’t have your own CDN, you have NO reason to buy transit.

              Share
  10. Patrick W. Gilmore Tuesday, May 6, 2014

    Try that again, CGP Grey’s YouTube video on network Neutrality:
    CGP Grey

    Or, really, any of CGP Grey’s videos. :)

    Share

Comments have been disabled for this post