How Video Is Changing the Internet
The rise of video streaming is dramatically affecting the Internet, according to a two-year study of Internet traffic trends that Arbor Networks recently presented to the North American Network Operators Group. Two years ago, Internet traffic was distributed evenly among a dozen Tier-1 network providers, but today the majority of traffic flows through direct peering agreements among large content providers, content delivery networks and ISPs. Consequently, Tier-1 networks have shifted their business models from simple packet delivery to richer cloud computing and content hosting services, and new players Google and Comcast have joined the top 10 list of Internet traffic producers — and the more traffic they put on the Internet, the more control it gives them over your online experience.
Traffic is growing much faster than the 50 percent year-to-year rate found by studies such as the Minnesota Internet Traffic Study; yet the “exaflood” of video traffic hasn’t drowned the Internet because network operators have found more efficient paths. The dramatic shift in traffic patterns has to do with the rise of what Arbor calls “the Hyper Giants,” 30 large companies that contribute 30 percent of Internet traffic. Thanks to YouTube, Google alone is responsible for 7 percent of all the traffic on today’s Internet, which puts it in the privileged position of prioritizing its VoIP and video calling services over YouTube without FCC permission.
The onslaught of video is also changing the nature of peering agreements. Traditionally, peering and so-called transit were very distinct from a revenue perspective: Peering agreements were “settlement free” arrangements in which packets changed hands between networks of roughly equal size and scope, but money didn’t. Fee-based network interconnects were confined to “transit agreements” in which a large network operator connected a small player to the entire Internet for a fee; peering is also strictly a “one network to one other network” arrangement. The new wrinkle is “paid peering” agreements in which a large operator permits direct connection for a small fee. Paid peering replaces transit fees that run $2-9 per Mbps with direct connection at $1-3, and enhances service, according to an article on Bill Norton’s “Ask Dr. Peering” web site which explains the value of Comcast’s paid peering and its potential collision with net neutrality regulations:
Paid peering provides better performance than transit, since the traffic takes a less circuitous route. Paid peering allows Google competitors to more easily compete with Google on performance and price without having to reach Google scale.
But paid peering may be forbidden by Question 106 of the FCC’s proposed Open Internet rules because it’s essentially two-tiered network access, Norton points out.
Paid peering illustrates how hard it is to write an anti-discrimination rule for the Internet that doesn’t have harmful side effects for all but the largest content networks. Paid peering is a better level of access to an ISP’s customers for a fee, but the fee is less than the price of generic access to the ISP via a transit network. The practice of paid peering also reduces the load on the Internet core, so what’s not to like? Paid peering agreements should be offered for sale on a non-discriminatory basis, but they certainly shouldn’t be banned.
Video is rising on the Internet, with more of it coming from legal sources such as content delivery networks and less from piracy-oriented systems like eDonkey and BitTorrent. Regulators need to look before they leap into wholesale bans on practices like paid peering that enable the Internet to carry increasing volumes of traffic. The FCC’s last net neutrality order (issued against Comcast in 2008) was an unintentional gift to purveyors of pirated content because it banned P2P throttling; going forward, the FCC should be at least as kind to network operators coping with the rise of video traffic by creative means.
Richard Bennett is a research fellow with the ITIF with 30 years of network architecture experience.
Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.
Great read. I am concerned with one thing you make it sounds like everyone uses BittTorrent as this negative thing. Although many people us it to download illegal things there are legal purposes. I know a lot of people that use it for 100% legal purposes. Other then that youtube using 7% of all internet traffic that is insane.
Most P2P use today is for piracy, but there is a large and significant segment of P2P use that’s perfectly legal. Going forward, when P2P is fully domesticated and doesn’t clog ISP networks as aggressively as it does today, I expect that it will find a niche as the preferred delivery system for ad hoc events like video streaming live news events. It’s not there yet, but the folks at BitTorrent, Inc. and a number of academics are working on refinements to the congestion-sensing logic of P2P systems to move them in that direction.
As a general rule, P2P is not a very good system for video distribution on the Internet because its use of long, bi-directional paths means that it can take 20-50 times more Internet resources to deliver a file than a CDN does; just count the number of hops a P2P packet has to traverse to get from source to destination and compare them to the hops a packet traverses from a CDN. P2P can place enormous stress on undersea cables, the scarcest resource on the Interenet, and that’s not a good thing.
The comments from the author clearly determine that he has absolutely no clue about the subject matter (“on-net vs. paid-peering”).
Look at the source. Bill Norton has made his career on writing white papers on Peering and Interconnection. Does it surprise us that he’s trying to whip up some controversy surrounding this NRPM?
And to back up Vijay’s satements, Paid Peering is nothing new, it was a service that was offered as far back as ’98-99 (a service that my company offered, and sold). And for you to say this is a new service that is popping up just shows that you haven’t done the most basic research.
Furthermore, the newer networks who are offering this as a service can only do so because the only very recently (within the last few years) have built national networks!
Whether or not you believe paid peering is new, the shift of an enormous amount of Internet traffic from transit to paid peering is new, that’s what the data in the Arbor Networks study shows. To ignore this trend in favor of some hair-spitting over terminology shows a lack of willingness to engage in the substance of the debate.
It’s also obviously not true to say that Norton is professional writer. He was one of the co-founders of Equinix and for a long time a pioneer in building carrier-neutral colos.
The combination of these two errors in your comment, smeuse, tells me you’d simply rather not deal with the implications of current traffic trends in the Internet core and their potential collision with proposed FCC regs. If the topic makes you uncomfortable, fine, but there’s no need for the personal attacks.
Steve Meuse is absolutely correct. I’m sure, Richard, you know who Steve is, since you have a 30 year history in this industry. At any rate, paid peering is nothing new, nor is there any indication from Arbor’s data that more traffic is traversing it. Bill spent his career writing white papers, not building anything. His job at Equinix was purely evangelical, not operational. Check it out with someone who knows.
“Hair-splitting” over terminology has major impacts when you are dealing with the FCC. If you get the base assumptions wrong, you are going to have lasting effects of bad legislation….
Let me pose you this question. What is the difference between Paid Peering and an ISP selling full transit to a content provider, and that content provider filtering out the prefixes they don’t care about?
Is the FCC going to ban ISPs from selling transit? That kinda breaks things, don’t you agree?
Again, get the base assumption wrong, the rest of the argument is invalid.
FWIW, I’ve know Bill Norton for a long time, and I even contributed to some of his earlier documents. I don’t think his most recent work has been grounded in any sense of reality and only seems to exist to stir up controversy where none exists.
You guys are repeating yourselves.
Daniel Golding at 12:30 PM on November 23, 2009 wrote:
“Bill spent his career writing white papers, not building anything. His job at Equinix was purely evangelical, not operational. Check it out with someone who knows.”
So I guess organizing NANOG doesn’t count for ‘building anything’ in Daniel’s book?
Irrespective, anyone who’s successfully built a company well understands the integral importance derived from communicating their solution’s core benefits to prospective early adopting customers.
With the lack of respect insinuated by the ad hominem attacks on Bill Norton’s track record of creating value for others, I can only surmise that Daniel either:
- Does not carry such experience himself;
- Is unable to logically, unemotionally challenge Bill’s articulated positions, or;
- Finds Bill’s writings, which shed light and understanding on how network operators currently commercially integrate with one another (a necessary first step towards more efficiently matching network resources to consumer-driven demands for bandwidth-intensive applications) to be in conflict with his current employer, contractor or other vested interest.
For the purpose of maximizing value to the conversation, may I suggest to those unproductively exerting energy making ‘playground’ attacks in this comment section focus their attention on the facts and matter at hand? Thank you in advance.
First you say paid peering may be banned by the NPRM. Then in the next paragraph you jump to the conclusion that you can’t have a non discrimination rule that doesn’t ban paid peering. It seems to me this is a issue that should be raised in the rulemaking, but that the truth is far from certain at this point. I’d think this could easily fall under network management exceptions, but we shall see.
And what does paid peering’s merits have to do with a prohibition on throttling certain types of applications which have legal uses? Two separate issues, but lumping them together reeks of an attempt to mud pie Net Neutrality in the face.
I think it’s possible to have a proper non-discrimination rule that doesn’t ban paid peering, but that’s not what the draft rule in the NPRM actually does. If the focus were on offering the service for sale on a non-discriminatory basis, it would be fine; but saying that “all packets are equal” is a big problem for everyone but Google.
Paid peering is not and will not be banned. Bill, unfortunately, made this up. There is no way to read the proposed rulemaking this way – its simply not in the document, at all.
Actually, it’s perfectly reasonable to read the NPRM that way, not only from Bill’s analysis but on the basis of discussions I’ve had with people at the FCC. The anti-discrimination rule has been kicked around since 2005 as a reaction to former Bell South CEO Bill Smith’s conjecture that ISPs could offer a service that accelerates traffic for paying service provider customers. A major element of net neutrality advocacy is to prevent ISPs from being in that business. Generally, that would entail the ISP offering a CDN or other short path service, and it’s hard to see how paid peering doesn’t fall into that category.
Paid peering is simply one example of a (perhaps) unintended consequence of an overly-broad anti-discrimination rule.
My friend Dick Bennett isn’t a lawyer or a policy analyst, so I don’t fault him for misreading the FCC’s NPRM. It most certainly wouldn’t impact peering, or any other paid transport arrangement. Para 106 is about discriminatory treatment for delivery by retail ISPs.
What Dick didn’t mention is he currently works for ITIF, and his paid by incumbents like Comcast to shill againt Net Neutrality. GigaOm, you should be ashamed for misleading your readers.
Jane
It is good to have fly by readers like yourself. If you have been a long time reader of the blog — which you aren’t — then you would know where I personally stand on this issue. And just because I stand on one side of the issue doesn’t mean that I am deaf or blind to other opinions such as Richard’s opinion. I am not sure anyone is misleading anyone here.
Fair enough, I suppose. And I do think its unreasonable to infer any editorial intention to ‘fool’ the readership. But even this *non*-’fly by reader’ would suggest that the provided postscript (‘[...] a research fellow with the ITIF with 30 years of network architecture experience.’) just barely qualifies as full disclosure…unless, of course, we presume that all GigaOm readers are either attentive industry insiders or remarkably well-read amateur geeks.
I’d bet a good number of civilians have more-than-passing familiarity with neutrality issues. Yet these people are hardly likely to have read Om’s opinions, much less to have a remote idea what in heck ITIF is – something, I might argue that _no_ one outside the group truly knows. Heck, even a WSJ op/ed would offer a link or exposition (eg: ‘communication industry think tank’, ‘independent consultancy’, etc.), to point out inherently intrusive business interests.
Ever heard of Google search, Justa Notherguy?
Om,
Actually, I have been reading this site for about 2 years. I certainly am aware of your and Stacey’s positions on network neutrality.
What I am asking for is some full disclosure on the part of contributors. ITIF, whom Dick works for, was originally founded by ITI, and was at the time a Cisco-IBM-even eBay supported group. But tech funding dried up, and ITIF went with open hand to the Bell companies and to the cable companies. I happen to know for a fact that after the Comcast-Bit Torrent case blew up, NCTA and Comcast funneled a ton of cash to ITIF so they could hire Dick Bennett and George Ou to be flies in the ointment on the network neutrality issue.
It would help your readers if they knew that this article was written by an author who is paid by Comcast to attack this policy. It doesn’t matter if Dick had these views since he was born, it just matters for transparency’s sake.
I agree with Jane on this one, Om, though I usually respect what’s on your site. There is little defense in my opinion about not having better disclosure. We come to your site because we trust and respect you, so please don’t spend down that capital you’ve built up by making many mistakes like this.
On the plus side, you are allowing an open discussion as far as I can tell, so that at least is consistent with my image of your credibility. Hopefully this type of thing won’t happen anymore. Otherwise, I will begin to worry you’ve sold out too somehow, and that will truly be a sad sad day for consumer internet, as you are one of its strongest allies.
It’s really quite hysterical that so much of the pro-regulation crowd would rather deal in personal attacks than the actual issues, but it’s nothing new.
The text from the NPRM says:
“We understand the term “nondiscriminatory” to mean that a broadband Internet access service provider may not charge a content, application, or service provider for enhanced or prioritized access to the subscribers of the broadband Internet access service provider, as illustrated in the diagram below.”
Jane, it’s an open ended rule that prohibits ISPs from charging content providers for ENHANCED or PRIORITIZED access. That’s an open ended rule that can and will be interpreted by lawyers filing complaints at the FCC who will argue on the basis of the literal meaning of the rule.
The concern over the current wording of the NPRM is reasonable in the context of Net Neutrality. For example, Rep. Ed Markey’s latest H.R.3458 would ban ISPs from charging content providers for anything. Here’s the exact text.
“not impose a charge on any Internet content, service, or application provider to enable any lawful Internet content, application, or service to be offered, provided, or used through the provider’s service, beyond the end user charges associated with providing the service to such provider;”
That bill would go even further to ban devices like the Amazon Kindle since Amazon has to pay Sprint for Kindle connectivity.
So Net Neutrality proponents have for a long time been trying to ban ISPs from charging content providers under the argument that such charges would be too much of a burden for smaller content providers to overcome. The problem is that this is a foolish and misguided argument because all it does is force content providers to pay more money for inferior transit access.
Now it’s perfectly possible that this is not the intent of the current NPRM. But why not clarify that?
But even if this bill doesn’t specifically ban paid peering, it does ban ISPs from charging content providers for prioritization or enhanced access. But I can think of two cases where this would be a good thing. One, an ISP would offer CDN services to the content provider coming though transit. Two, an ISP could offer content providers the option of boosting a broadband subscriber’s bandwidth beyond what the subscriber paid for. That would be no different than Amazon paying Sprint to provide bandwidth to Kindle users.
So the bottom line is that the NPRM goes too far to prohibit ISPs from providing these services under the assumption that it is always the bad form of discrimination. The more reasonable thing would be to allow ISPs to offer services under non-discriminatory terms.
I wasn’t aware that we were friends, Jane, but you can never have too many.
I’ve been commenting on net neutrality since 2002, and only went to work as a policy analyst in June, so my opinions precede my employment by a considerable margin. If you were familiar with the nature of the non-discrimination rule as it’s been discussed in DC and in Europe since 2005, you’d be aware that it bans payments to an ISP – such as Comcast – by a service provider that will give the service provider better QoS than would be afforded to a service provider with no specific contract with the ISP. Because paid peering is a short path to the consumer, it’s also a high-priority path, due to the statistical nature of the Internet and the impact of Round-Trip Time on TCP. So your analysis is clearly wrong, and you resort to smear tactics to hide its weakness.
But thanks for playing.
Any by the way, the tag line to my post discloses my employment, so I don’t know where you get off saying I don’t mention that work for ITIF. I do, and ITIF has many sponsors on all three sides of the net neutrality debate.
Here’s a clue, no charge: Bring some substance.
@Richard bennett:
> [...] ITIF has many sponsors on all three sides of the
> net neutrality debate.
Really? And who would those ‘sponsors’ be? Last I checked, ITIF have steadfastly refused to reveal their sources of funding. Nor have I found any such information on-line, via SourceWatch, etc.
Therefore, I’d very much like to see a list of said sponsors, showing the exact nature of their relationship(s) – money? technical help? good wishes? – with ITIF. And I would bet that I am not alone in this.
Given your aggressively defensive attitude, here, I anxiously await _your_ ‘substantial’ response.
I don’t do fund raising or public relations, so I couldn’t give you the full list of who funds ITIF even if I wanted to. But I will reiterate that I haven’t modified my views as a consequence of my employment, as everyone who’s followed my writing over the years can attest.
So rather than mud-slinging, why not argue a real point?
Richard Bennett wrote:
“[...] ITIF has many sponsors on all three sides of the net neutrality debate. ”
Intrigued, Justa Notherguy posited a logical follow-up:
“Really? And who would those ’sponsors’ be? Last I checked, ITIF have steadfastly refused to reveal their sources of funding.”
Richard Bennett answered:
“I don’t do fund raising or public relations, so I couldn’t give you the full list of who funds ITIF even if I wanted to.”
To which, I respond:
I never asked for a/the ‘full list’. A representative one will do, just fine…say, one well-known sponsor from each of the ‘three sides’ you cited.
Neither did I ask what you do/don’t do, at ITIF. Let’s recall that _you_ volunteered the varied nature of ITIF’s sponsors, as affirmative defense of both their objectivity and your own. Thus, my interest in seeing some examples from that diverse group. Ipso facto.
So, if you now hope to tap-dance around this subject (‘[...] why not argue a real point?’), here’s a clue – no charge: save the puerile, Debate Club distractions for a less-rigorous venue. You claimed evidence of objectivity; I asked for it. Bring some substance.
You’re the one who’s dodging the issues, anonymous Justa Notherguy, not me. I’m using my real name as I always have on the Internet, and posting my own opinions. You’re claiming that my opinions are bought and paid for and demanding full disclosure of my current employer’s sponsors, while you too much of a coward to even post blog comments using your real name. That’s a bit of double standard, don’t you think?
Richard Bennett wrote:
> You’re claiming that my opinions are bought and paid for [...]
No. I never wrote any such thing, nor have I implied it. But anyone can see that, just by reading my responses (above). Come on, now – this saber-rattling is merely yet another distraction from my question. You implied the diversity of ITIF’s sponsors-list is evidence of objectivity. Ok, then…show me. That’s all I’ve asked.
> [...] and demanding full disclosure of my current employer’s
> sponsors [...]
Not ‘full’ – just a few. Three, in fact. Have you neglected, to re-read my previous response? Besides, it was you who brought them up in the first place. Remember? Up to that point, it would never have occurred to me to ask. Must be a bit embarrassing, knowing that you put yourself in this position, to no real purpose.
> [...] while you too much of a coward to even post
> blog comments using your real name.
Tsk, tsk…name-calling? Richard, really! I expected more from a man of your background and education.
> That’s a bit of double standard, don’t you think?
This from a guy who volunteered un-named parties as references for his veracity, only to admit (see prev.) that he has no such names to provide and couldn’t get them, even if he tried? Then, when asked to explain this discrepancy, rather than admit to his error(s) he starts flinging epithets? And you see a ‘double standard’…where? LOL
Seriously, Richard, you need to get a grip. I believe this kind of behavior reflects poorly on your employer, as well as on yourself. And, heaven knows, ITIF seems to have enough detractors. Surely, they don’t need the additional burden of being associated with schoolyard grandstanding and shameless weaseling.
@Anonymous Justa Notherguy:
If you’re not implying that my opinions are bought and paid for, why do you keep harping on ITIF’s sponsors? You know as well as I do that ITIF’s policy is not to disclose, so I couldn’t tell you who they are if I wanted to.
Given that you refuse to give your name or say who pays you, I’d think you’d be sympathetic to ITIF’s policy in this respect. Well, you would be except for some sort of aversion to consistency.
So come on now, I’ve disclosed by name and my employment, and you can visit my web site, bennett.com, and read my resume and my past writing on this and other subjects.
I wrote the previous comment, for some reason the blog didn’t pick up my Facebook login.
A few clarifications:
“Thanks to YouTube, Google alone is responsible for 7 percent of all the traffic on today’s Internet, which puts it in the privileged position of prioritizing its VoIP and video calling services over YouTube without FCC permission.”
I wasn’t aware that since I’ve been prioritizing vonage over Hulu on my home network using my linksys WRT, I was in violation of the law. Perhaps richard can help me get the proper permission from the FCC? In the meantime I’ve reverted my home network QoS settings back to factory default.
“The new wrinkle is “paid peering” agreements in which a large operator permits direct connection for a small fee. ”
Perhaps richard isn’t aware, but ISPs have been providing “on-net” access as a form of limited transit product for over a decade, I know I had turned this on for customers as far back as 1999.
In the meantime, this line demonstrates quite clearly the competence and level of research in both articles:
“Google wants Paid Peering to be illegal, along with any form of access tiering. ” from the dr. peering article.
I haven’t seen a statement to that effect anywhere yet, perhaps Richard can point that out?
/vijay
Excellent snark from Google’s Director of Net Ops.
1. The home network point is one that I made to the FCC as a witness at the hearing on P2P at Harvard, because it illustrates the gap between current practice and the rules Google’s Rick Whitt and others have petitioned the FCC to adopt. It’s nice to have my arguments thrown back at me, but you and Rick should talk.
2. The paid peering service is new for the major ISPs, but not unprecedented in world history.
3. Google’s opposition to enhanced interconnect for a fee is a matter of clear public record. See Rick Whitt’s statements of the Google Policy blog, http://googlepublicpolicy.blogspot.com/2007/06/what-do-we-mean-by-net-neutrality.html :
“…in Google’s view, what should the broadband carriers not be allowed to do? …
* Levying surcharges on content providers that are not their retail customers;
* Prioritizing data packet delivery based on the ownership or affiliation (the who) of the content, or the source or destination (the what) of the content; or
* Building a new “fast lane” online that consigns Internet content and applications to a relatively slow, bandwidth-starved portion of the broadband connection. ”
As you’re a technologist, Vijay, these prohibitions don’t make sense to you, but they are the program your employer is seeking to impose on their competitors. Like the unbindling issue we’ve discussed before, they compromise the future to protect the status quo and are therefor bad for progress.
Actually I’d be a lot more interested if you would answer the question I asked, not what you want to say: Specifically I asked (quoting you) the following: “Thanks to YouTube, Google alone is responsible for 7 percent of all the traffic on today’s Internet, which puts it in the privileged position of prioritizing its VoIP and video calling services over YouTube without FCC permission.”
How does that actually work again? Alternatively, how does one get from a claimed 7% of traffic to the statement “privileged position of prioritizing its voip and video calling over youtube without fcc permission?” And could you help me with how I need to comply with the FCC since I didn’t ask their permission for my home network.
As for “2. The paid peering service is new for the major ISPs, but not unprecedented in world history.”
Again, do you actually know what you talking about? I said I was turning up paid peering in 1999. On-net or limited transit was being sold by all major ISPs back then.
As for #3, again, I don’t see anything about paid peering.
Maybe my sentence wasn’t clear. As a private network operator, Google doesn’t need FCC permission to prioritize (by order) the traffic that leaves its network in any way that it sees fit; it’s exempt from NN regs since it’s not an ISP.
If Google wants to offer VoIP in the future (in conjunction with Google Voice, for example,) it can ensure that its own VoIP packets are interleaved with YouTube in such a way that YouTube doesn’t cause excessive jitter. This makes perfect sense, and the FCC won’t complain. But Google’s proposed regulations could make it difficult for Comcast, say, to provide Vonage with the same service. A bit inconsistent, don’t you think?
How old is Paid Peering? Bill Norton is generally regarded as an authority since he co-founded Equinix and has more experience in the area than anyone else, so this will be he said-she said for the time being. If you have some data, please share.
The words “paid peering”aren’t in Rick’s statement, but the effects he lists are consequences of it. In the policy world, we tend to think abstractly.
“How old is Paid Peering? Bill Norton is generally regarded as an authority since he co-founded Equinix and has more experience in the area than anyone else, so this will be he said-she said for the time being. If you have some data, please share.”
Co-founding equinix and actually taking two ISPs to “tier-1″ status in us and eu isn’t exactly the same thing. Like I said, I’ve been turning up paid peering or on-net routes as a product since 1999 and so have other ISPs. Perhaps you should talk to someone who actually does this for a day job vs writing white papers?
Lets try this one more time shall we? “how does one get from a claimed 7% of traffic to the statement “privileged position of prioritizing its voip and video calling over youtube without fcc permission?””
“The words “paid peering”aren’t in Rick’s statement, but the effects he lists are consequences of it. In the policy world, we tend to think abstractly.”
Would that we actually thought at all. Like I said, perhaps you could boil it down to concrete words that idiots like me can understand, not being up with high faultin “policy world” thinking.
/vijay
I’m not sure that your “on-net routes” is the same product as the Paid Peering that Nortion is interpreting; the Arbor study found a large increase in the traffic that moves through these transit bypass paths, and that’s the actual story. While this service may have been available for a while, its use is radically increasing. That’s data, BTW, not anecdote, so if you have a problem with the Arbor data, you’ll need some data of your own to refute it.
The significance of the 7% isn’t that difficult, is it? If you pump the ISP’s pipe full of packets of all kinds, you’re going to cause jitter for third party services whenever you deliver a clump of packets back-to-back. You can order your own voice packets optimally in each clump, as you’re immune from FCC regulation, but Vonage has to deal with the jitter you cause. How are they going to do that without a little help fro the ISP?
Paying for a prioritization service – even as Paid Peering – would probably be forbidden under the regulations Rick wants. Do you read the statement seeking a ban on “Prioritizing data packet delivery based on the ownership or affiliation (the who) of the content, or the source or destination (the what) of the content” differently? If so, what does it say to you?
“I’m not sure that your “on-net routes” is the same product as the Paid Peering that Nortion is interpreting; the Arbor study found a large increase in the traffic that moves through these transit bypass paths, and that’s the actual story. While this service may have been available for a while, its use is radically increasing. That’s data, BTW, not anecdote, so if you have a problem with the Arbor data, you’ll need some data of your own to refute it.”
I think that is the entire problem statement right there – “I’m not sure that your “on-net routes” is the same product as the Paid Peering that Norton is interpreting”
I think you actually don’t know what you are talking about basically, so there isn’t much further point in discussion, but for completenesses sake, I’ll go through your points.
“The significance of the 7% isn’t that difficult, is it? If you pump the ISP’s pipe full of packets of all kinds, you’re going to cause jitter for third party services whenever you deliver a clump of packets back-to-back. You can order your own voice packets optimally in each clump, as you’re immune from FCC regulation, but Vonage has to deal with the jitter you cause. How are they going to do that without a little help fro the ISP?”
This one is relatively easy – once the traffic hits the ISP, it is all the same to them, voip, video etc. When they get congested they are doing to drop packets at random when the queue gets full. The standard drop discipline is juniper routers which are in every ISP bar one or two, is RED, which stands for RANDOM Early Drop. It would be sort of hard to predict ahead of time and re-order packets such that a *random* drop could be predicted ahead of time. If I could do that, I’d be in the futures business, not moving packets around.
“Paying for a prioritization service – even as Paid Peering – would probably be forbidden under the regulations Rick wants. Do you read the statement seeking a ban on “Prioritizing data packet delivery based on the ownership or affiliation (the who) of the content, or the source or destination (the what) of the content” differently? If so, what does it say to you?”
So now we are in to the “probably” now. I did read the statement on prioritizing data packet and affiliation – nowhere does it say to me you can’t conduct a commercial transaction for getting access to a providers routes by direct interconnection. Thats a commercial transaction.
With all respect to Bill, he is not a co-founder of Equinix. The founders of Equinix were Jay Adelson and Al Avery.
And, he is wrong about paid peering, wrong about Google’s intent, and wrong about the proposed rulemaking. I have told him this directly.
I am not an employee of Google and I take no money from them. Heck, I don’t even agree with them on Net Neutrality. But I do think that this sort of misinformation is terribly harmful to the debate. I suggest asking people other than “Dr. Peering”.
Once again,complete wrongness. Bill Norton is the ONLY person in the Internet industry who thinks 106 affects peering. One source is extremely poor reporting, but as this is apparently an editorial sloppy/no research is OK [sorry, I too am a 'fly-by' reader because I'm issue focused; google alerts tell me abut relevant stuff interesting to my niche, so not I'm not a faithful lapdog^Wreader anywhere, nor do i wish to be...].
From the NPRM:
“We understand the term “nondiscriminatory” to mean that a broadband Internet access service provider may not charge a content, application, or service provider for enhanced or prioritized access to the subscribers of the broadband Internet access service provider, as illustrated in the diagram below.”
Paid peering, which is not new and has nothing to do with video, does not prioritize to *subscribers*, therefore there is no applicability. If Richard thinks it is new to “major ISPs” then maybe he should say whichones. Likely they are minor or telephant dinosaurs who were behind the curve for the last decade. Direct interconnection to a network is much further away from the subscriber edge. Anyone with network design clue would get this.
As to this lovely bit: “not impose a charge on any Internet content, service, or application provider to enable any lawful Internet content, application, or service to be offered, provided, or used through the provider’s service, beyond the end user charges associated with providing the service to such provider;” This text is more likely to cause issues, but if is is a different service offering, and is optional, is that IMPOSED? I think not, but IANAL.
“Because paid peering is a short path to the consumer, it’s also a high-priority path, due to the statistical nature of the Internet and the impact of Round-Trip Time on TCP. So your analysis is clearly wrong, and you resort to smear tactics to hide its weakness.”
Short != high priority. If you honestly buy this argument, then you must desire legislation of queing disciplines and specific TCP implementations. Completely ludicrous.
I think this article is wildly inaccurate an providing distribution for a broken interpretation of the proposed rule making.
Obviously, Bill is not “The Only Person in the Internet Industry” who thinks 106 might have an affect on peering and transit. Try to make a serious argument next time.
Name someone else then, Anonymous. I’ll happily ask them to actually read 106 and look at the diagram that clearly states “End user’s broadband Internet access service provider may not charge CAS provider for enhanced or prioritized delivery to end user over this link.” where “this link” is the last mile, where all QoS discussions have been based.
All that’s necessary to for the FCC to apply the non-discrimination rule to the peering side of an ISP network is the realization that the Last Mile and the Middle Mile are seamless parts of a single network. You don’t have to be very smart to make that leap.
That would then be a different entity than THIS proposed rule-making, which clearly points to the last mile. Should there be a different proposed rule-making, the concern might have a shred of weight, but that would extend beyond where the scare-resource, duopoly claims have a reasonable foothold … the justification forr this whole pile of junk.
Wouldn’t it be nice if GigaOm had someone on staff (or even in consultancy) who actually knew how the internet worked? And I don’t mean like “click the link, look at the site”, I mean more like “hey, the internet has packets, how do they get to where they go”.
Seriously guys, please get some clues.
Articles like this are why network engineers think you guys lack credibility.
I find it interesting that P2P is really decreasing as viewers want to stream content, and not always download content to their computers. With new viruses coming out every day I certainly can’t blame users for doing this and I assume it’s a trend that will continue.
I like Bill, he’s a nice guy, and a friend of mine. I’ve helped him with many of his white-papers and he has thanked me publicly, so I am not unfamiliar with his work. However, he is, unfortunately, occasionally confused about certain things.
I do not know you, but you claim to have 30 years of experience and put yourself out as an expert in the field. Then you make statements which prove you are quite ignorant of the topics at hand (e.g. not knowing “on-net routes” and “paid peering” are the same as “6″ and “half dozen” to anyone with more than a passing familiarity with the subject matter).
My _guess_ is that you suffer from the same problem as Bill. You have talked to a lot of people, you have hung out at conferences, you have read about the topic, etc., etc. But you have never logged into a single router, never run a network, never had to make decisions while live traffic was running over the equipment being modified, and in general never had to actually deal with these issues “in real life”. I could be wrong, but it would be surprising that you truly are experienced yet make such sophomoric errors.
People like JZP, Daniel Golding, Vijay Gill, and I are experienced with these subjects. While we are all experienced, our histories and points of view are about as widely separated as it is possible to be. Yet we are united in believing that you and Bill are both wrong on several objectively provable points. Perhaps faced with a rather large contingent of experience and viewpoints, you should re-consider your stance? Or at the very least stop trying to win semantic arguments when we are trying to educate you.
For instance, Vijay has actually done what and when you claim was not done. Is it not better a better strategy to listen and learn, rather than try to argue over the meanings of individual terms? Especially when you are wrong on those meanings?
Your blog, your choice. However, I find people get farther working together. Would you consider working with us?
–
TTFN,
patrick
Interesting comment, Patrick, and one that illustrates the problem with having this kind of discussion. I’m not a network operations person, for sure, and I don’t claim to be. I’m one of the people who first created Ethernet over twisted pair, Wi-Fi, and some related LAN and WLAN things. I’ve been around network protocols and standards for a long time, but not in an operations role. So my interest in the subject comes from observing how the Internet is adapting to changed patterns of usage, and how the FCC’s proposed rules may affect it. These are not questions that one needs to be a network admin to consider, are they? And in fact being too close to the action – as Vijay clearly is – can warp one’s perspective to the point where it’s difficult for him to consider the wider implications either.
It doesn’t really matter, for purposes of this discussion, whether some ISPs started selling paid peering in the last couple of years or ten years ago as much as it matters that service providers started buying and using it on a large scale. Vijay wants to obsess over the history of the service, I want to talk about the adoption of it. So to jump up and down and complain as he is is equivalent to attacking spelling or grammar; it’s a nit-picking detail that doesn’t go to the actual issue. And you can understand why: it’s embarrassing to Google to find that there really is (or may be, if you will) a self-serving element to the Non-Evil company’s long-standing support for a highly-regulated Internet.
And similarly, the ability to interpret the text of the proposed rules in the NPRM is not something that comes from network operations experience, it comes from an understanding of how the FCC works, the history of how similar rules have applied in the past, and the nature of the commercial transactions and packet flows that they affect.
Knowing the Cisco CLI doesn’t really arm a person to deal with these questions, so let’s not pretend otherwise.
> Knowing the Cisco CLI doesn’t really arm a person to deal with these questions,
> so let’s not pretend otherwise.
And having protocol history has nothing to do with operations and policy. Those of us who have operated networks in the US have dealt with the impact of policies (pre & post telecom act of 96, pre & post data CALEA, pre & post interLATA releif…) and are not blinded by such experience. Rather, we are informed by our legal teams, years of intercompany co-operation, years or talking with LEOs and regulators, time spent working with data collection, on federal advisory councils, etc. We *are* the people who know what we’re doing, and we call BS.
Please do address any of my previous comments regarding specific language. The language is targeted for “access to the subscribers of the broadband Internet access service provider” which has diddly to do with paid peering.
This has nothing to do with knowing how to configure a router, this has to do with having actual experience with peering and operating a network from a policy perspective. Now don’t get me wrong I like Bill Norton personally, I think he’s a really nice guy and I wish him the best, but at the same time he has been in way over his head trying to write about an industry he doesn’t really understand for many many years. From a technical perspective he has absolutely no clue what he is talking about, he is simply regurgitating what others have said with no ability to distinguish fact from fiction.
You would do well to listen to the insightful comments posted here, by people who have doing more with peering than you will ever dream of understanding. Vijay, Dan, and Patrick are not only some of the most authoritative sources on peering anywhere in the world, but they also happen to be correct this time too. :)
…I’m not a network operations person, for sure, and I don’t claim to be. I’m one of the people who first created Ethernet over twisted pair, Wi-Fi, and some related LAN and WLAN things. I’ve been around network protocols and standards for a long time, but not in an operations role.
[...]
well, in your resume (http://bennett.com/resume.pdf) you state you are a “broadband network architect” and a “network engineer”
how can one fulfill either of those roles, if they do not understand basic interconnection methods such as paid-peering/partial/on-net routes
you should have some understanding about the operation of a broadband network and how it interconnects with other networks, if you are a broadband netarch..
You are very good at ‘nit picking’ while missing the wider point, which is pretty ironic given your reply.
Do you honestly believe Google’s Director of Network Operations spends time logging into routers these days? Of course not. Neither does Mr. Golding or I. (I do not know if JZP does these days.) But without some experience in how networks work, and whether something is useful in a real, live, business setting, it is impossible to make educated decisions regarding those things.
I have completely and perfect faith in Mr. Golding’s ability to read the proposed rule and tell me whether it means what you say it means. I have complete and perfect faith in Mr. Gill’s ability to tell me whether something happened and whether it was common. I have complete and perfect faith in JZP’s judgement in whether a shorter path is “high priority” in an _operational_ setting.
More importantly, my own opinion concurs with theirs. Additional commentary by people like Steve Meuse only emphasize the point.
Alternatively, you make claims based on papers which are written by people who have exactly zero experience with the subject matter. You argue that X is important or new when those who actually have to run a real network with real traffic say otherwise. You buttress your claim with _factual_ errors, on points which are not open to interpretation. When we point out these errors, you ignore them or further populate your blog with additional errors trying to cover your original error.
I am confused why you think this is a useful course of action?
Let’s be clear about just a few things:
1) Wide adoption of paid peering is not terribly new.
2) The “Tier Ones” have not been the center of the Internet for years.
It doesn’t matter how smart you are, if you base your logic on faulty assumptions, your conclusion is almost certain to be wrong. Anyone with any real operational experience would know these points without having to read a paper by Mr. Norton. That you read the paper and claim otherwise only makes anyone who has actually done this work think poorly of you.
Still think that operational background is still irrelevant to the question at hand?
That said, it is true that operational experience is not required to read a proposed rule. We disagree on what the rule says. But even if you can read the rule, I have zero faith you understand the repercussions of the rule. Not only do you have no experience in the subject, but you have proven you cannot take instruction from those who do. So while you may be able to read the rule, why does that matter if you cannot explain what the rule will mean to those actually doing business on the Internet?
In short, you add no value. Just the opposite, you confuse the issue, which is bad for everyone.
–
TTFN,
patrick
Actually, Patrick, I have another source on paid peering, a paper presented at TPRC 2007 by several people from MIT and one other author. See: http://people.csail.mit.edu/wlehr/Lehr-Papers_files/Clark%20Lehr%20Faratin%20Complexity%20Interconnection%20TPRC%202007.pdf
This paper says: “This line of thought has two implications—more formal peering contracts and the emergence, as we will discuss below, of paid peering” and “The two new types of agreements we will discuss are paid peering and partial transit” and “The emergence of paid peering, although non-standardized in its details, is not surprising as a concept”.
This suggests that paid peering is “a new wrinkle” in the Internet interconnection space. Oh, BTW, the other author of that paper is somebody named Patrick Gilmore; that would be you.
Does any of this ring a bell?
And BTW, I don’t buy the argument that “paid peering” is the same thing as “on-net access”. Paid peering is going to take you into one and only one network, while on-net access simply gives you a closer point of presence to connect with a transit network, which is not the same thing at all.
So no, I don’t know that on-net access is the same thing as paid peering; in fact, I know that it isn’t.
Richard,
You don’t “buy” the argument that paid peering is the same as on-net? Are you out of your #$%^&ing mind? Do you think there is some kind of conspiracy theory to hide the name from you? Let me clear this up for you with some definitions:
Peering is when two networks agree to exchange on-net (i.e. their customer) routes with each other for free.
On-net transit is when one person pays money for on-net (i.e. their customer) routes.
What do you think paid peering is? That’s right, money in exchange for on-net routes. You fail at even a basic understanding of the concepts, let alone the terms. Please, I’m begging you, stop embarrassing yourself.
Sorry, Steenbergen, but you’re wrong again. A range of “On-net services” that can be purchased from large network operators, some of which are simply interconnection and routing within the large operators own space, some of which are transit to the whole Internet, and some of which are combinations of carriage and routing. Level 3 has something they call “Extended on-net access” which is both carriage and transit: http://www.level3.com/index.cfm?pageID=454
You insist that these terms have the meanings you want them to have, but the industry has adopted a set of meanings that overlap with yours only slightly. Given the many competing authorities in this field, and the wide range of uses among providers, I’ll stick with the definitions I’ve found from credible sources.
Oooooook wait, I think I see the problem. You don’t understand the difference between on-net in a transport sense (i.e. a building you service to via your own network, without having to buy a tail/loop from another carrier) and on-net in an IP sense (your customer routes). One of these things is not like the others, one of these things just doesn’t belong. Again, basic concepts 101.
In other words, “on-net” as a prefix implies many different things, some of them very much like peering, some like transit, and some like carriage. Because of that ambiguity, it’s best to use the term “paid peering” when we’re talking about a service in which a content provider pays an eyeball network for access to the eyeball network’s retail customers.
Most credible sources – from David Clark to Patrick Gilmore to Bill Norton – regard paid peering as an emerging and valuable phenomenon.
In other words, there are some concepts which share similar words from the english language, but which are entirely different things with absolutely no relationship to each other. Those of us who understand the meaning of the words we are trying to use know the difference, while those of us that don’t (that would be you) flail wildly in a sea of fail.
On-net as a prefix implies that something is… yes you guessed it, ON YOUR NETWORK. When you’re talking about a building that is on your network, it’s called an on-net building. When you’re talking about an IP customer route which is on your network, it’s called an on-net route. But BUILDINGS ARE NOT THE SAME THING AS THE INTERNET ARE THEY?
I’m going to stop talking to you now, because this is absolutely pointless. It’s like arguing with a 4 year old, it doesn’t matter if I’m right because you wouldn’t know it if it bit you in the ass. You have absolutely no knowledge basis to be making any claims or conclusions about peering or interconnection on the Internet in general, but at least you’ve done a great job embarrassing yourself and proving that point. :)
That’s a good move, Steenbergen. Please note that I didn’t use the term “on-net” on my blog post, I used the precise terms such as peering, paid peering, and transit. “On-net access” and “on-net routes” were introduced in the comments by Vijay Gill in some sort of attempt to prove that my claim that paid peering is a new wrinkle is false. Gill’s claim, even if it were true, would be beside the point. The FCC regulations seek to broadly ban arrangements under which content or service providers have enhanced access to eyeball network customers. It’s plausible to consider paid peering as one such arrangement, and most responsible parties agree that this would be a bad outcome.
That’s the point of my blog, so this terminology war is simply obfuscation.
JZP demands comment on: “access to the subscribers of the broadband Internet access service provider” which (in his opinion) has diddly to do with paid peering.
You really don’t see this? It’s a mystery ow an ISP who sells a service (whatever you want to call it) that provides faster and higher-capacity access to the ISP’s network is not providing faster and higher-capacity access to the ISP’s subscribers?
WTF, dude.
Well, “Dude”, your argument means that any broadband ISP can sell no other internet access. Do you honestly believe that limiting the business opportunities of broadband ISPs large and small is the intent? You think that Version Business will be interested in shedding the as701 enterprise customer so that they are not providing “preference” to the path between people buying their service?
The language is specific regarding broadband subscribers and QoS (enhanced or prioritized access). The reference diagram is clearly the last mile. FFS it says “End user’s broadband Internet access service provider may not charge CAS provider for enhanced or prioritized delivery to end user over this link”. [diagram 2, page 42, where you can find the text which has been quoted several times to you.
That is plain as day. Not even a regulator could contort that to the meaning you and Bill seem to wish there to be.
See my previous reply to your first comment implying that ISP Last Mile and ISP Middle Mile are separate networks, JZP.
The end user’s broadband experience is affected by the nature of the interconnect between the ISP’s network and the service provider’s network; Vijay’s employer didn’t spend $10-15 billion on an unregulated private network and its peering and transit agreements because it didn’t make any difference.
Look, I think it’s really noble that Vijay has come in here guns blazing to defend the honor of his employer, and that his friends are standing up for him, but running the discussion down a rat hole of terminology isn’t really advancing the discussion or helping to move the US toward a good Internet policy framework.
The issue underlying this discussion is how much freedom the ISPs need to have in terms of interconnect agreements to provide the needed Quality of Service to diversify Internet applications in the future. The network neutrality advocates (and some in the operations community) argue that the best way forward for the Internet is to keep on doing things the way they’ve always been done; Vint Cerf argues, for example, that treating all packets in the same, “Best Efforts” way has given rise to a rich set of applications, and therefore has great utility. On the other side, people who want to see more diverse applications on the Internet (those that require QoS or volume-sensitive discount pricing) argue that allowing ISPs to sell interconnect at varying levels of QoS would be beneficial.
The Paid Peering example illustrates that business arrangements that provide poor man’s QoS by offering a short path (TCP throughput is inversely proportional to RTT, all other things being equal) show that such arrangements are beneficial and should be encouraged. Some net ops people are simply phobic about change, since they’re so busy bailing water under the current scenario. I understand that.
But Net Ops people aren’t the best people to dictate policy to the FCC. As one of you has pointed out, you learn about regulations from talking to your legal staffs, and the legal staffs learn what to teach you from seeing how the FCC and the courts interpret policy. This system is inherently retrospective. The questions before the US are prospective, namely which network operations and business policies stimulate innovation and wide adoption of broadband and which don’t. There’s a great body of empirical and analytical literature that shows that the policies that Vijay’s employer advocates – open access, un-bundling, line sharing and single service level – inhibit broadband adoption in both the current (copper-based) generation and in the fiber-based next generation.
These are the issues that the FCC’s proposed rules will impact, not whether the right term for direct access to an ISP network is “on-net routes” or “paid peering.” We’re talking about the direction in which we want to take network policy, not the place that it comes from.
Once again, the blog ate my name; the previous comment is mine.
Yeah and once again, reply buttons come and go…
> See my previous reply to your first comment implying that ISP Last Mile and ISP
> Middle Mile are separate networks, JZP.
From a regulatory perspective -both as proposed here and as implemented in many
countries- that is not relevant as those network elements are traded in an open market.
You are sidestepping the point that 106 does not say anything on-point to external
network edges. There *might* be any number of changes proposed, but *might* isn’t
what is on the table. What is on the table expressly points to the last mile, and you
keep failing to acknowledge that.
> The end user’s broadband experience is affected by the nature of the interconnect
> between the ISP’s network and the service provider’s network;
You are creating dichotomies where there aren’t any. You avoided touching my VZ
example; why? Without a strictly segmented and regulated last mile business entity,
how can you claim that there are such things as broadband providers with no other
content or connectivity lines of business?
> Vijay’s employer didn’t spend $10-15 billion on an unregulated private network
> and its peering and transit agreements because it didn’t make any difference.
So you have a bone to pick with every unregulated packet switched network that
exists? Neat, and the answer is to regulate them all?
> Look, I think it’s really noble that Vijay has come in here guns blazing to defend
> the honor of his employer, and that his friends are standing up for him, but
> running the discussion down a rat hole of terminology isn’t really advancing the
> discussion or helping to move the US toward a good Internet policy framework.
Colleagues concerned about people propigating misinformation are not friends
standing up for anything but intellectual honesty. If you bothered to look up the
people commenting, you will note they are long-standing members of the operations
and engineering community with a diverse set of opinions. We happen to agree on
9.8m/s^2(rounded) as standard acceleration due to gravity, but somehow defending
that fact doesn’t require “nobility”.
> The Paid Peering example illustrates that business arrangements that provide
> poor man’s QoS by offering a short path (TCP throughput is inversely proportional
> to RTT, all other things being equal) show that such arrangements are beneficial
> and should be encouraged. Some net ops people are simply phobic about change,
> since they’re so busy bailing water under the current scenario. I understand that.
Once again, you ignore the question raised by smeuse, myself and others. If you
buy this argument then how can any transit sales not violate such principles? There
is no differentiation visible externally between a standard transit agreement and any
paid peering agreement, so address how you think proposing such a restriction on
commercial offerings will fly with any business?
> But Net Ops people aren’t the best people to dictate policy to the FCC. As one
> of you has pointed out, you learn about regulations from talking to your legal staffs,
> and the legal staffs learn what to teach you from seeing how the FCC and the courts
> interpret policy.
It was me and I said “informed” and was in the context of not being blinkered nor
blinded. Most of the folks participating on this thread have been the ones *instructing*
legal staff over the past few decades.
I personally don’t care what google or comcast advocates. Nor do I see how either is
relevant to your claims that:
- google can magically prioritize traffic on other folks’ networks
- “recent” video is the cause of paid peering
- broadband companies can’t offer any other connectivity without being unfair
- …I’ve lost track of what else I need to correct; sorry
> These are the issues that the FCC’s proposed rules will impact, not whether
> the right term for direct access to an ISP network is “on-net routes” or “paid
> peering.”
The proposed rules in front of us do not deal with the non-access network
elements; where is the text that says otherwise? Thus far when pressed you
have said rules “might” be changed, etc. Other than maybes and mights, where
is the basis for the concern? Smells like floating the bogus idea only to champion
protecting all of us from it.
Have a good night!
Point out an actual error? You didn’t even know what “on-net” and “paid peering” were, that alone should be all the example you’ll never need. Each and every error that vgill/etc have pointed out was spot on, so I don’t need to repeat what they’ve already said.
As for “people who have a stake in the Internet”, I’d certainly hope that would include everyone. Yes we do have some similar opinions, they stem from the the “we all have extensive experience running large multi-national networks which deliver many hundreds of gigabits per second of traffic via peering” club, and thus we have some idea of what we’re talking about. If you really can’t find the clearly identified points where we’ve collectively proven you have no clue what you’re talking about then you fail at the English language as well as at routing.
“The language is specific regarding broadband subscribers and QoS (enhanced or prioritized access)”
I just opened up the NPRM word document and ran a search for the word “QoS” and it’s nowhere in the document. The string “Quality of Service” shows up elsewhere not in the context of paragraph 106. Paragraph 106 also doesn’t mention anything about the sale of “Assured Forwarding” or “Express Forwarding” in the context of DiffServ. It merely states that it doesn’t want ISPs charging Content/Application/Service providers for either enhanced or prioritized service.
Furthermore, the FCC’s chief technologist Jon Peha has stated that the NPRM was intentionally vague to get as much discussion and comment as possible. Dr. Peha also stated that the FCC doesn’t really understand this stuff and they want someone to explain it to them. Since we have a lot of lawyers like Tim Wu and Lessig and Free Press saying that the restrictions against ISPs aren’t stringent enough in the current NPRM draft, and since the FCC considers what this group of people say, it’s very reasonable to come out and argue that products like Paid Peering should not be banned.
Now if it turns out that the FCC doesn’t intend to ban Paid Peering and they come out and say Bennett and Norton misinterpreted the vague language of the NPRM, then nothing would make Bennett and Norton happier. But they way you guys are turning this into some kind of holy war against Norton and Bennett and using some of the most inappropriate personal attacks is just ridiculous.
If you don’t agree with their interpretation, fine. But you surely must admit that the issue needs to be cleared up by the FCC.
George Ou wrote:
‘I just opened the the NPRM word document[...]“
It does clearly say “enhanced or prioritized access”, which anyone with expertise can translate to traffic shaping, queue juggling and the passel of techniques we in IP networking call “Quality of Service”. Unlike ‘professional bloggers’ I don’t suffer from logorrhoea and prefer economy of words, therefroe used the well-known acronym QoS to cover the entire rubric. Next time I’ll show each steps so you can keep up.
While you have the document open, please read the remainder of the sentence ” as illustrated in the diagram
below.” and refer to Diagram 2 on that same page. It CLEARLY points to the last mile, which is appropriate as that is generally the oversubscribed segment where mitigation of abusers to avoid harming the other users on the segement takes place.
Further you say “If you don’t agree with their interpretation, fine. But you surely must admit that the issue needs to be cleared up by the FCC.” No. There is plain simple English and a pretty picture – it is NOT up for ‘interpretation’. While much of the NPRM is vague, this is pretty bloody precise. To date, no inter-provider peering takes place on this segment so until 106 or the diagram changes, or 107 or another section is deemed as abl to trump this VERY SPECIFIC limit on the scope, it is a waste of time.
JZP says: “It does clearly say “enhanced or prioritized access”, which anyone with expertise can translate to traffic shaping, queue juggling and the passel of techniques we in IP networking call “Quality of Service”.”
It never says “Quality of Service” in regards to the FCC prohibiting ISPs from selling access to content/app/service providers. Yet the FCC was very clear in multiple places to use the full term “Quality of Service” in other places in the document where it wanted to explicitly refer to any kind of Priority Queuing technology. It was notably vague here in paragraph 106.
And as Richard Bennett mentioned, those of us who are familiar with some of the internal affairs at the FCC have some good reason to fear that “Enhanced and Prioritized” does apply Paid Peering.
JZP says: “It CLEARLY points to the last mile, which is appropriate as that is generally the oversubscribed segment where mitigation of abusers to avoid harming the other users on the segement takes place.”
This obsession with the last-mile is pretty ludicrous. It makes little difference whether you apply pressure on the second hop or a little further upstream in terms of the shaping on the traffic and you know it. There are all kinds of way to shape traffic and Priority Queuing is only one of the mechanisms and probably less used mechanisms compared to the practice of just buying lower contention higher exclusivity circuits. Just having that private 10 Gbps Paid Peering connection directly into an ISP gives your traffic a hell of a boost over other traffic.
The last mile does offer some congestion challenges but these are largely intra-subscriber issues rather than inter-subscriber issues and that makes a huge difference. The broadband subscriber has quite a bit of control in intra-subscriber congestion just based on the active applications they use but they have very little control over the inter-subscriber congestion issues and they have little control over how congested or unreliable a long-haul transit connection gets.
JZP says: “Further you say ‘If you don’t agree with their interpretation, fine. But you surely must admit that the issue needs to be cleared up by the FCC.’ No. There is plain simple English and a pretty picture – it is NOT up for ‘interpretation’.”
Your response is surprisingly unreasonable, childish, and tyrannical. It’s one thing to say “your interpretation is wrong”, it’s another to just say “shut up, you have no right to ask for clarification”. This is clearly against the spirit of the FCC’s NPRM when the FCC’s chief technologist says they need to figure this stuff out and they want dialog.
JZP says: “While much of the NPRM is vague, this (paragraph 106) is pretty bloody precise”
Actually, that’s the problem. The NPRM is very explicit in most parts to specifically use the term “Quality of Service”. It was notably missing and notably vague in paragraph 106.
“Look, I think it’s really noble that Vijay has come in here guns blazing to defend the honor of his employer, and that his friends are standing up for him, but running the discussion down a rat hole of terminology isn’t really advancing the discussion or helping to move the US toward a good Internet policy framework.”
Richard, I think what people are saying and you are demonstrating that you do not have a clue as to how things work. The basic point people are bringing up is that you are completely unencumbered by facts, so your opining about things is considered harmful.
Well Vijay, what I see is you and your buddies spreading a fog of disinformation. Arbor Networks did a two year study of Internet traffic trends and found a shift away from transit and toward paid peering. This has implications for some of the proposed regulations for the Internet, under a common interpretation of the regulations. This impact happens to benefit Google.
So you come along and very aggressively start calling names a questioning the competence of people who had nothing to do with the study for reasons of your own, and then you obviously enlist some of your friends to help you out because you can’t turn the discussion around by yourself. So instead of talking about Internet traffic and regulation, we’re now talking about who’s entitled to have an opinion on the subject.
Let’s face it, Vijay, you don’t like a wide-ranging, free-wheeling debate, you want to spin Google’s policy preferences. I understand why you want to do that – you’re a loyal employee and you have your stock options – but your tactics are a dubious. If you have a point to make, or some data to offer, bring it on, but your argument that anyone who disagrees with you is necessarily clueless is not really working for you.
Right Vijay, your first comment was to call Norton and me clueless, and you’ve simply repeated the charge several times; and you’ve asked your friends for help.
I think everyone reading this far down knows this is your argument: Norton and Bennett are clueless, so how dare they criticize Google, the Not Evil Company. But the action here kinda belies your claim; it’s pretty evil to bully and pile on, even to protect your stock options, but that’s who you are and we all know that about you, so no big deal.
Forget about me and forget about Norton, just go read the Arbor Networks study. Are they right that there’s been a shift in Internet traffic dynamics to accommodate the rise of video streaming? Does this have implications for ISP business plans? Does the net neutrality agenda need to protect the ability of ISPs to deal with the traffic that YouTube is dumping on them?
I’m happy for people to consider those questions, and to do it in their own time away from all the tantrums, name-calling, and bullying. If they do that, I win.
you didn’t even know what the difference between on-net routes and paid peering +was+ richard. you didn’t know. The arbor study showed a shift from tier-1 traffic patterns to direct connection. How do you know that it was paid peering? Paid peering and direct interconnection are indistinguishable at the technical level.
I would have a debate but since you don’t even _understand_ the problem, I am at a loss on how to proceed. Its like saying ‘the earth is round’ and the answer is ‘I like french fries.’ And then claiming that there is no reasoned debate. Richard, you can’t get there from here.
Google’s Vijay Gill asks: “The arbor study showed a shift from tier-1 traffic patterns to direct connection. How do you know that it was paid peering? Paid peering and direct interconnection are indistinguishable at the technical level.”
I think the shift is mainly to paid peering since the smaller networks don’t generally qualify for settlement-free peering as they don’t have the size and scope of the larger networks. While most interconnect agreements are made under NDA and therefore aren’t public, there are a few operators – Comcast is one example – who publish their criteria for settlement-free peering, and from these we can deduce that there’s likely been an increase in paid peering.
So that’s the basis for the reasoning. Any questions?
Richard,
I think you’re misunderstanding the reason for the number of comments on your post. We aren’t defending Vijay because we’re his friends, we’re defending him because he’s right. Vijay and I (and Patrick, and Dan, and Joe, and many others who have commented here) do not agree with each other on a great many issues, most of us don’t work for Google, and many of us come from very diverse backgrounds with very different philosophies on peering (both personally and professionally). On any other day you would find us all arguing with each other, but your post is so wrong on so many fundamental levels that it has brought us all together to try and help explain where you went wrong.
I realize you’re probably just going to dismiss this as another part of the grand conspiracy against you and your ideas, but try to consider the facts here for a moment. You have a literal army of people who have many many years of ACTUAL EXPERIENCE building and managing some of the largest peering networks in the world, banging at your door collectively telling you that you are confused on even the most basic concepts of this subject matter.
This is like watching someone argue with the top 10 heart surgeons in the world because they’ve been watching medical dramas on TV for the last 20 years and think that equals experience. You simply don’t have a prayer of knowing what you’re talking about no matter how well intentioned you may be. We’re not trying to be mean here, but you are being so resistant to clue that you’ve left us with no other options. If you have any interest in actually learning the subject matter so you can stop embarrassing yourself, you’ve actually managed to collect the right people for the job. If not, just tell us that you’re refusing to listen to anyone who might know better so we can get back to doing better things with our time. The choice is yours.
@Steenbergen: I know for a fact that Gillmore, Temkin, et. al. are Vijay’s buddies and that they work for companies with a stake in the Internet. It’s not a “conspiracy”, it’s just a collection of NANOG buddies who happen to think a certain way.
The only “factual error” that Vijay has pointed out in my post is something that he interpreted in a unique way: I said that the use of paid peering is a “new wrinkle”, and he took that to mean that I was saying that paid peering itself is a new wrinkle. That’s the extent of the so-called factual errors.
What’s happening here is a typical Internet pile-on by a group of people with technicians disease who are going to be terribly embarrassed by the comments they’re left here in a few hours; well, that and the typical case of people trying to protect their stock options by stifling innovation. We see that all the time, so it’s no big deal.
Be specific, Steenbergen: point to an actual error in my post instead of waving your hands. The only “error” that Gill has been able to find is my calling paid peering a “new wrinkle” because it’s much more widely used now than in the past.
Calling thing of this sort “errors’ indicates a very poor level of analytical reasoning on the part of the critic.
That’s basically just a bunch of hand-waving, Steenbergen. The only specific factual issue anyone of Vijay’s buddies has raised to far is the assertion that the wide-scale use of paid peering “isn’t terribly new;” that was from the anonymous JZP. Others simply declare me unentitled to opine, for no particular reason.
The Arbor study is evidence that traffic is shifting, and the carrier-neutral peering site managers I’ve spoken with tell me they’re making something like 300 cross-connects a month. Do you think all those cross-connnects are implementing settlement-free peering or conventional transit agreements? I’m surmising that they aren’t.
If you have some data on this, then please share, but if you’re just here to play Friends of Google, then keep on mumbling.
Richard Bennett once claimed to have invented blogging. He pulled that claim from his own blog — which he humbly named “The Original Blog” — after several people showed it to be untrue.
Now he’s telling us he’s among a cadre of elite engineers who understand the Internet better than anyone else. And yet this article misses so many basic points about network engineering to cast serious doubt on that claim, too.
At least his banter was believable to the folks at ITIF, who hired him — courtesy of their phone company sponsors — to spread doubt about efforts in Congress and now at the FCC. Om seems to have fallen for it as well, alas.
This is as much as Reuters could uncover about ITIF:
“The group gets some funding from industry, including International Business Machines Corp and Cisco Systems Inc, and also is supported by telecommunications companies.”
http://www.reuters.com/article/marketsNews/idUSN0749664720090107
Simply pointing out that you are being paid by companies that have a stake in the outcome of the NPRM is not resorting to “personal attacks,” Richard. It is important to know who we’re talking to and who is standing behind him with a paycheck and a pat on the back.
Another scurrilous personal attack from someone who doesn’t want to use a real name.
I never claimed to have invented blogging, Kathy Marie, I claimed to have a web site in 1996 that looked kinda like a blog in that it was in newest first log format. Sometimes people do something called “tongue in cheek” commentary; you can look that up.