Reports have been filtering in during the last few days that consumers on Verizon and Time Warner Cable’s network are experiencing degraded service when they try to watch Netflix or YouTube videos. It may seem trite to whine about someone’s Arrested Development episode buffering, but the real issue is how big ISPs are trying to remake the agreements that underpin how the internet works.

As they do so, they are taking agreements that used to be negotiated by engineers based on web traffic and changing them into disputes negotiated behind closed doors in boardrooms, executives fighting over who has access to the end consumer. This could fundamentally change the way the internet works — making it more expensive to do business and erecting unnecessary barriers to innovation.

This is the new battle for the internet. The telcos and cable providers, intent on protecting their margins and their pay TV businesses, have taken network neutrality from the public world of consumer pricing and throttling to the data centers. Instead of banning Skype, or charging more for it on their networks, they want to change they way they charge content providers, demanding that they pay more for ports on the network when traffic starts filling them up.

The weapon in this battle is a concept known as peering. Peering is essentially an arrangement between two bandwidth providers — the companies that control the physical backbone of the internet — in which they send and receive traffic from each other for free. The logic is that the traffic sent from one network to another is reciprocated without adding extra costs and hurdles. This makes the web more efficient and redundant because companies don’t need to build out a network to connect every single service to every person who wants to consume that service.

There are about 50 major internet bandwidth providers around the world that connect to each other and work under that arrangement. There are about 5,000 other networks who then work with these major bandwidth providers to bring the internet to our homes, offices, iPads and iPhones. Of course, I’m being simplistic when I describe the notion of peering, but you get the gist. And this idea of peering — on which much of the modern internet is built — is coming under attack, thanks to ISPs like Verizon and Time Warner Cable that provide us (the consumers) with connectivity.

Two examples of this shift

Here’s how these battles tend to play out in the real world.

Picture a conference room in Arizona. In such a room last year, executives from Cable One, a cable broadband provider with 720,000 subscribers, asked the chief technology officer at one of world’s largest bandwidth providers to pay Cable One a fee each time it needed to add more ports to deliver content to Cable One’s end subscribers due to an increase in demand for certain types of content. The CTO of that bandwidth provider, who declined to be named, refused. He walked away rather than add a new, and to his mind, unsustainable, cost to the company’s bottom line. (A representative from Cable One did not immediately respond to a request for comment on this incident.)

Now consider another scenario, which is more in tune with how the internet currently works. Picture a data center in California, where Sonic.net, a Santa Rosa, Calif.-based independent broadband and phone company, also keeps its servers. Sonic.net, which has about 50,000 subscribers, sees a bit under 10 gigabits per second of Netflix traffic a day.

In order to deal with that traffic, Dane Jasper, the CEO of Sonic.net, has put a Netflix-designed box to cache Netflix content closer to the edge of the network where the ISP network meets the big bandwidth providers’ networks. This helps Sonic.net cut the cost that Netflix traffic can impose on its operations. Essentially, Netflix pays for its traffic to travel across most of the network before dropping it at Sonic.net’s door.

Telcos want to make the internet like the phone system

Open Connect, Netflix’s content delivery network, is a new wrinkle in the old idea of networking peering. But it tries to keep the same relationship, namely that both parties (those sending and those receiving content) share in the costs of delivering traffic to the end consumer. This model has worked for years, but some ISPs want to change it.

They see their pipes becoming commodified — dumb, as it were. So they want to build a new internet model: one based on terminating traffic that looks a lot like the old-school telephone networks, where ISPs serving the last mile can charge content companies and bandwidth providers for upgrades they make to the network in order to carry the increase in traffic. The point of connection and negotiation has become the ports where traffic hops from the bandwidth providers onto the ISPs’ network.

An OECD report on peering and internet exchange points from last October explained that because many areas of the world have efficient and cost-free peering relationships, the prices for data are around 100,000 times lower than the price of a voice minute.

As consumers, we are experiencing this underground tussle via poor video streaming and what are effectively throttled connections. For example, earlier this week Om and I wrote about Verizon’s refusal to work with Cogent because it was carrying traffic for a large video provider. Verizon was letting its connections to Cogent (multiple 10 gigabit per second ports) run hot — a euphemism for getting crammed with traffic. And that’s because Verizon has refused to open more ports — the equivalent of opening lanes of traffic on a highway — to Cogent.

Normally when this happens the two parties come to some sort of agreement about adding another port, and then the problem is solved. But in this case, Verizon doesn’t want to play. We don’t have details about what’s happening in the Verizon/Cogent spat, but we can share how these fights have played out in the past, thanks to a public peering squabble that happened in 2010.

We’ve been here before

On the final weekend of November 2010, engineers at Comcast were stunned to find themselves overloaded on ports that connected to Level 3 Communications. These ports — where Level 3 terminated its traffic destined for Comcast’s last mile subscribers — were getting slammed with 10 to 100 times the bits they normally saw. And since overloaded ports result in a crappy experience for the end customer, Comcast needed to figure out what was happening.

Engineers trying to determine where the huge influx of traffic was coming from finally settled on the culprit — a deal that Level 3 had signed earlier that month to carry traffic for streaming video service Netflix. At the time, Netflix traffic made up 33.7 percent of North American web traffic, and in that one weekend most of that traffic suddenly came into Comcast’s network from Level 3 as opposed to Akamai, Netflix’s previous content delivery partner.

The internet is flexible and redundant, but it’s still grounded in actual physical infrastructure. So when all those bits suddenly came into Level 3′s ports instead of the ones Comcast had designated for Netflix/Akamai traffic, packets started dropping and Netflix customers on Comcast network experienced poor service. It’s not all that different from when the police shut off access to a major highway and send people onto local streets: Traffic backs up and everyone’s unhappy.

Luckily, building new roads on the Comcast network isn’t as labor-intensive as building out roads in real life. Even so, when it came to adding new ports for Level 3, Comcast took the opportunity to try to renegotiate peering contracts with Level 3. “These fights over peering can be like third-graders squabbling on the playground,” said Patrick Gilmore, chief architect, network infrastructure for Akamai, in an interview last summer.

While the fuss over the Comcast and Level 3 spat eventually died down after the two parties agreed to some form of give-and-take on getting new ports on the Comcast network, it highlighted a basic fact of life on today’s internet. In many cases, as the big get bigger, the internet’s core weakness isn’t technical; it’s the fact that the main players are now so large that they are in the midst of negotiating a new balance of power.

It’s an IP version of the treaties and shifting alliances in Europe in the late 1800s that then fell apart during the first World War. And while lives aren’t exactly at stake in these peering battles, the outcome of these fights might change the way the internet works, making it more expensive to build a business on the internet and allowing ISPs to become a new gatekeeper.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Connected consumer first-quarter 2013: Analysis and outlook
• Who and what to watch in the new era of the living room
• Netflix may suffer from limited mobility

As the networking world changes, and companies that range from Broadcom to Intel attempt to eat away at the proprietary switch business that Cisco, Juniper and even Arista have built, there are huge opportunities for startups trying to rebuild in more open components the software stack that resides in those proprietary boxes. Cumulus, founded by JR Rivers and Nolan Leake is one of those startups. Rivers and Leake helped develop what is now Cisco’s Unified Computing System that combines servers and networking on one box.

Cumulus has developed a Linux-based operating system to control the commodity switches that are being built by Quanta, Accton and Foxconn. It has raised funding from investors that include Andreessen Horowitz, Battery Ventures and VMware founders Diane Greene and Mendel Rosenblum.

I’ve covered the startup before, as well as the overarching problems in the networking space, including calls by webscale companies such as Facebook for exactly the sort of product that Cumulus is offering. But James Hamilton, the distinguished engineer who helps keep Amazon Web Services up and running, is apparently also a fan based on a Tuesday blog post. The post also has me suspecting that Amazon might be the big unnamed Cumulus user Rivers referenced in our call.

In his post about Cumulus, Hamilton writes:

Because networking gear is complex and, despite them all implementing the same RFCs, equipment from different vendors (and sometimes the same vendor) still interoperates poorly. It’s very hard to deliver reliable networks at controllable administration costs from multiple vendors freely mixing and matching. The customer is locked in, the vendors know it, and the network equipment prices reflect that realization.

And that is what’s so exciting about Cumulus, and the entire wave of change sweeping over the networking world. Technologies such as OpenFlow, the rise of merchant silicon and even the promise of software-defined networks are all coming together to enable networks to make the same leap that servers made two and half decades ago, when Intel disrupted the proprietary chip model in play and opened up the ecosystem.

The silicon providers have been doing that for a while with their own chips in networking, but people building out real, production networks still needed software running on that silicon that was robust enough to manage a variety of networking functions and protocols communicating to that chip. With Cumulus, they get that operating system and perhaps the real key to innovation in networks.

Cumulus plans to open source the operating system and then sell support and new services on top of the OS, much like Red Hat does around Linux.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Cloud computing infrastructure: 2012 and beyond
• The promise of SDNs in the enterprise
• OpenFlow and beyond: future opportunities in networking

Recently a frustrated Huawei EVP and co-CEO Eric Xu took a rather flip position on the matter, saying his company was no longer interested in the U.S. market and had essentially stopped paying attention to the controversy surrounding it here. Xu was obviously posturing. No global equipment maker would just simply ignore the world’s largest telecommunications market.

But there is a bit of truth to his words. Huawei has done quite well for itself without landing a single major U.S. infrastructure deal. Domestic operators may have resisted Huawei’s allure, but carriers in Canada, Europe, Asia, Latin America and Africa certainly haven’t. By some measurements, Huawei has already surpassed Ericsson as the largest telecom vendor in the world.

What’s more, there might even be a veiled threat in Xu’s statement. Huawei may not need the U.S. to be successful, but the U.S. needs Huawei if it wants to keep the telecom equipment market competitive.

Pickings are slim

I’ll be the first to admit that I’m in no position to judge the national security implications of Huawei operating in the U.S. (My colleague Stacey Higginbotham does a great job examining the controversy, though). But I do know the competitive situation of the U.S. mobile industry. If Huawei gets banned, then one of the key industry forces keeping equipment prices low suddenly disappears. That’s not a situation U.S. mobile industry wants to see.

There has been a lot of consolidation in the telecom equipment market in the last several years. In many ways, Huawei was directly responsible for that consolidation, driving smaller players out of the market or into mergers through its aggressive pricing. A decade ago there were about a dozen companies that could sell you a cellular base station. Today there are really only three or four dominant mobile players, and Huawei is one of them.

Huawei R&D facility in Shenzhen

Huawei has evolved from a mere vendor of cheap telecom gear into a technology powerhouse that can throw enormous engineering and research resources at its product lines. But it still enjoys huge cost advantages due to China’s cheap labor and engineering pools. The dwindling competitive landscape is particularly evident in the U.S. where historical reliance on CDMA technologies has led two companies to dominate: Ericsson and Alcatel-Lucent. No U.S. operator relies solely on a single network maker, so AT&T, Verizon Wireless and Sprint all use both vendors as primary suppliers.

But Alcatel-Lucent is suffering, and it might not be long before Alcatel-Lucent finds itself broken up and sold for parts. If that happens, there will be a big vacuum, and there aren’t that many companies capable of filling it. There’s Nokia Siemens Networks and Huawei, and that’s pretty much it. Smaller network players like Samsung have been asserting themselves of late, but they’re still small fish in an ever-shrinking pond.

Huawei’s hidden influence

Over the last few years, U.S. mobile industry has undergone a big transformation, building its next generation of LTE mobile networks. Typically the relationship between carrier and vendor is rock solid, but whenever a big generational shift like this comes along, the mobile operators get to re-evaluate their partners. Contracts are suddenly up in the air, presenting the perfect opportunity for a new equipment maker to penetrate the U.S. market.

Huawei bid on all of those contracts, but except for a WiMAX deal with Clearwire and a few minor networks with regional providers, it failed to win any of them. Every analyst and industry insider I’ve talked to, however, said that Huawei’s presence was felt during those negotiations.

Established vendors were forced to underbid Huawei or risk losing key contracts. A deal with a nationwide U.S. operator is a marquee deal, involving billions of dollars and tens of thousands of cellsites. To lose a nationwide U.S. contract to Huawei would be a major black eye for an Ericsson. In the case of Alcatel-Lucent, its contracts with AT&T, Verizon and Sprint are the main things keeping its wireless business afloat.

All of the major U.S. operators have already named their LTE suppliers, so there won’t be much opportunity to disrupt the market for years to come. And when that time does come it will take a lot to pry those carriers away from their current suppliers. But the presence of Huawei would at least gives those operators leverage.

Look at this way: AT&T loves Ericsson and Alcatel-Lucent. It probably will never leave them. But if I’m AT&T, I want Huawei around to keep those two vendors honest. And if Alcatel-Lucent suddenly goes belly up, I’d rather have more than one choice to replace them than NSN.

Feature photo courtesy of Flickr user minghong

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• CES 2012: a recap and analysis
• The future of mobile: a segment analysis by GigaOM Pro
• New solutions for the evolving mobile network

Koruta, which I profiled last November, makes a photonics chip that allows signals to pass between chips using light (photons) instead of electrons. This makes communications between chips faster, something becoming more important inside the data center as networks become flatter, faster and fatter.

The Kotura chip is a fiber-based transceiver that can deliver 100 gigabits per second inside the data center. The transceiver could live on a board next to the CPU or inside a switch and could eventually expand to deliver a terabit per second (Tbps). Currently it’s used in high-performance compute clusters, which are also the most-popular home for Infiniband. From my post in November:

While one 1 Tbps is crazy fast when you consider that many data centers are currently upgrading to 10 gigabit Ethernet between servers, it’s going to be necessary. Arlon Martin, VP of Marketing, Government Contracts & Industry Relations at Kotura, tells me that customers are building products for the high-performance computing sectors but also for real-time data processing. The goal is bringing a low-power and less expensive optical part into a rack of servers, able to scale up to terabit per second capacities.

Kotura isn’t the only company trying to bring fiber optics into the data center. Plexxi is building fiber-based switches, while Facebook and the Open Compute Project in January announced their own plans to integrate photonics into their open hardware program. Intel, Cisco and IBM all have research or have acquired startups in this space as well. With its emphasis on fast networking, it makes sense for Mellanox to follow suit.

The company plans to continue offering the Kotura transceiver and open a research center in Monterrey Park, Calif., where Kotura is headquartered. From the Mellanox release:

“Operating networks at 100 Gigabit per second rates and higher requires careful integration between all parts of the network. We believe that silicon photonics is an important component in the development of 100 Gigabit InfiniBand and Ethernet solutions, and that owning and controlling the technology will allow us to develop the best, most reliable solution for our customers,” said Eyal Waldman, president, CEO and chairman of Mellanox Technologies.

At the time of my profile, Kotura had raised undisclosed millions from ARCH Venture Partners, Fuse Capital, GF Private Equity and others. It has an established customer base in the telecommunications business where it has sold product since 2006. But last year it began targeting the data center, where it apparently attracted Mellanox’s eye.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Infrastructure Q4: Big data gets bigger and SaaS startups shine
• Infrastructure Q2: Big data and PaaS gain more momentum
• Big Data, ARM and Legal Troubles Transformed Infrastructure in Q4

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Infrastructure Q1: Cloud and big data woo enterprises
• 2012: The Hadoop infrastructure market booms
• Sector RoadMap: SQL-on-Hadoop platforms in 2013

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Cloud computing infrastructure: 2012 and beyond
• Takeaways from the second quarter in cloud and data
• Infrastructure Q2: Big data and PaaS gain more momentum

Compass-EOS has been quietly working on these problems for six years and is now ready for its formal launch with a product aimed at the service providers’ core networks. The company has raised $120 million and spent more than half a decade developing a silicon-photonics-based router that would be faster, consume less power and is also more modular.

Pitango Venture Capital, Benchmark Capital, Northbridge Venture Partners, Crescent Point, Cisco Systems, Comcast Ventures and T-Ventures have all backed the company, which is based in Milpitas, Calif., and Netanya, Israel.

Routers are the workhorses of the internet. They determine where and how packets should travel around the web, and for years they’ve done this using specialty silicon that relied on electronic signals. But Compass-EOS’ technology is a new type of optical chip that allows the router to think at the speed of light — not at the speed of electrons.

The resulting box is faster, consumes less energy and takes up less space. All of these things will help network operators keep pumping more traffic over their infrastructure, especially as they deploying faster networks. After all, if routers are the brains of the operation, they need to speed up too.

Optical chips are coming not just to the core of telco networks, but also to the data center as companies such as Intel, Cisco and IBM invest in photonics research, but also as startups like Plexxi and Skorpios, Kortura build chips and equipment that can process optical signals, as opposed to electric ones.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• A near-term outlook for big data
• What converged infrastructure means for the future of the data center staff
• 12 tech leaders’ resolutions for 2012

(download)

iTunes

Stitcher Radio

SHOW NOTES
Host: Adam Lesser
Speakers: Isabelle Guis and Lee Doyle

Instgram’s Twit-storm, Netflix nabs Disney, GMail’s Pretty iPad App

RoadMap re-run, our talk with Instagram’s Kevin Systrom

iTunes 11, When Things Connect, Sun Volt

What Aspiring New Media Stars Should Know About Agents and Managers

Holiday Gadget Gift Guide

War Tweets, Google TV and Nexus 4

Director Jay Duplass on low-fi movies through high-tech

Election Dissection, Ditching DSL and Dumping the iPad

Sandy’s Social, Infrastructure Impact and Forstall

Windows 8 Surfaces, and disruption eruption

iPad Mini, iMac gets skinny

Boxee Cloud DVR, Apple Rumors and Chromebook

Commutist interview: Joy of X author Steven Strogatz

Commutist podcast: Patent trolls, Costco ban and Passbook’s home run

Commutist, meet Nerdist, and interview with Chris Hardwick

T-Metro, Broadband Caps, Remembering Steve Jobs

Apple’s iO-Mess, Dirty Data Centers and Tesla

News from the Mobilize Conference

Paul Tough: How Children Succeed and what you can learn from them

The iPhone 5 Event

Come on, Kindle, Light My 4G Fire

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Takeaways from the second quarter in cloud and data
• Cloud computing infrastructure: 2012 and beyond
• Who and what to watch in the new era of the living room

So a team of researchers from around Europe, with some help from the European Commission, Alcatel-Lucent and BT, is aiming to revise a standard data-transport protocol to effectively re-engineer how the internet runs — without the need for pricey new equipment.

It’s a tall order, but they’re deadly serious. And from the vendor side, it’s not hard to see why: Alcatel-Lucent wants to slash latency for better videoconferencing, and BT wants to do the same in order to improve its Radianz Cloud platform, which serves the very time-sensitive financial services industry.

The project is called Reducing Internet Transport Latency (RITE). It quietly kicked off last November but the University of Aberdeen, where some of the researchers are based, has only just started making noise about it — largely because the researchers are about to set off for an Internet Engineering Task Force (IETF) meeting in Florida next month to show off what they’re up to.

In a statement, Professor Gorry Fairhurst said:

“It’s a problem we all notice when you’re using a program like Skype. If anyone else in the house is watching a video at the same time, your video connection becomes jerky and often crashes. This affects gamers who want to play online in real time and companies doing stock training – both end up buying special and expensive internet connections to make these work, but often it’s not more bandwidth that’s needed to go faster – it’s less delay.

We think we can reduce this delay by making a set of small but important changes to the way computers and the network process the internet data.”

So, what sorts of changes are we talking about? Fairhurst told me on Thursday that there are two main strands to the RITE project’s work: revising the core Transmission Control Protocol (TCP, a.k.a the flipside to the Internet Protocol) and changing how network routers handle buffering.

“We’re trying to change TCP so that it works better with thin applications — applications that don’t send a huge amount of data and aren’t really interactive, like media streaming and conference calls,” he said. “We would make a small change to the timer mechanism so that you can recover data when you send a burst and lose part of it — it can take quite a lot of time to recover one lost packet — and we also have to do something to the way the congestion window works.

On the other side, to make this work effectively you have to change the way the routers behave as well. People know routers have lots of memory in them, but TCP tends to fill up all the buffers inside. We’re going to make recommendations on how to avoid buffer blocks — this is more directed at operators than people building PC software, but these things have to be done at the same time.”

Fairhurst noted that the TCP revisions would be piloted in Linux (the team is also working on mechanisms for UDP-based streaming applications). After those Linux patches have been rolled out over the next year, the RITE researchers will try to get the standards community to do its thing and then “hopefully convince Microsoft, Apple and everyone else” to incorporate the changes into their applications. Google, which has developed the SPDY (pronounced “speedy”) protocol for reducing load time, has already shown interest, he claimed.

Apart from better videoconferencing and faster trading platforms, Fairhurst also suggested that success could “make the internet available to a whole new raft of applications,” such as proper virtual reality.

“It’s a small fix that really came from the gaming community originally,” he added. “Gamers really hate delay.”

Apart from the University of Aberdeen (Scotland), other research facilities involved in RITE include Simula Research Labs (Norway), the University of Oslo (Norway), Karlstad University (Sweden) and the Institut Mines-Telecom (France). The consortium has received just over €3.5 million ($4.7 million) from the European Commission.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• How intelligent networks address enterprise cloud issues
• CES 2012: a recap and analysis
• LTE changes everything; LTE changes nothing

Lyatiss has raised $4 million in Series A funding from Idinvest Partners and others. It has headquarters in Santa Clara, Calif. and a research office in Lyon France, where the company originally began as a spin out from Inria, the French National Computer Science Research Institute.

Lyatiss combines the ideas associated with software-defined networking –such as monitoring flow-level data and treating the networking layer as an abstraction — and applies them to business and performance rules associated with applications. In short, Lyatiss says it can do what many of the people who are excited about SDN really want — a way for the network to react and deliver what the application needs.

Pascale Vicat-Blanc, CEO of Lyatiss

The service differs from a cloud-based networking monitoring program product such as Boundary’s, which tracks individual packets at the network level in that it tracks the entire flow — which includes where the packet is going and what might be stopping it or slowing it down along the way as opposed to just noticing that it has slowed down or stopped. This level of information, which can include details like the performance of the CPU and how that affects the network, is far more detailed.

This brings us back to TCP. This protocol helps define how devices talk to the web, ensuring that all packets sent around the web join up with their buddies at your end device. The protocol helps divide and track the packets that comprise an email, a movie or a digital photograph. In much the same way CloudWeaver’s application-defined networking hopes to use flow information to track how an application performs across multiple virtual machines, web services and eventually clouds.

It has started with an Amazon-based service, and promises to monitor and then let developers tweak their AWS settings when something is running a bit too hot or slowly. Developers could use this to see an outage before it occurs and then take swift action using the same software.

Eventually the plan is for such things to happen automatically if the developer sets it up that way. For those working in the cloud where scale is essential, the ability to monitor such flows and then have the hardware react to the needs of the application is somewhat of a holy grail. Lyatiss hasn’t managed to achieve this yet, but that’s where it is heading.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Takeaways from the second quarter in cloud and data
• Cloud computing infrastructure: 2012 and beyond
• The promise of SDNs in the enterprise