Despite the love people have for email, Twitter and even Facebook, the real star of the web in terms of sheer traffic is video. And not only is all this real-time video streaming possibly rotting our brains, congesting our broadband networks and threatening our pay TV businesses, it’s driving wholesale changes in how we pay for broadband and the future of television.

A great illustration of these changes comes from Sandvine’s Global Internet Phenomena Report: 1H 2013. Sandvine provides deep packet inspection and networking management tools to wireless and wireline ISPs, which is how it gets some of its data. While, many people already knew that Netflix traffic comprises about a third of the web traffic in the U.S., they might not know that YouTube is gaining rapidly with 17.11 percent of web traffic downloaded on wireline networks, up from 13.8 percent a year ago.

Video makes bandwidth hogs of us all

Few people are immune to the siren song of cat videos or Arrested Development. In fact, it’s changing the profile of what broadband usage looks like to the point where it’s normal to be a bandwidth hog. According to the Sandvine report in North America, the top 1 percent of subscribers who make the heaviest use of the network’s downstream resources account for 10.1 percent of downstream traffic.

However, those top 1 percent of users don’t look too much different from the top 30 percent. At the bottom, the network’s lightest 50 percent of users account for only 6.4 percent of total monthly traffic. In fact it’s those laggards at the bottom we should be worried about. Did they somehow miss Gangnam Style?

The average and median usage on both wireline and wireless networks in North America is on the rise. On wireline networks mean usage was 44.7 GB, a 39 percent year-over-year increase from 32.1 GB. Over the same period, median monthly usage increased at an even by 56.5 percent, jumping from 10.3 GB to 18.2 GB. On mobile networks mean monthly usage increased by 25 percent from 312.8 MB to 390.1 MB. Yet, median usage more than doubled from 25.5MB to 58.7 MB over the past year, driven in part by more people buying smartphones.

And mobile is even bigger than these numbers make it look like (or something like that). One out of every five bits — or 20 percent of the traffic on wireline network is generated by a smartphone or a tablet. And as Wi-Fi expands and is easier to connect too, that number should continue to increase.

Yes, video traffic will always be big, because videos are big

Before people accuse me of being unfair, let me note that sending video is one of the most data heavy options around. A two-hour HD movie file can contain 4GB of data or more, while a book that might also take two hours to read would top out at several megabytes.

The sheer volume of data is one reason video strikes fear into the hearts of both wireless and wireline network operators, while the loss of revenue from pay TV subscriptions keeps wireline providers up at night. Unfortunately for those implementing usage-based billing plans perhaps in hopes of influencing subscribers to keep their pay TV subscriptions, Sandvine shows that real-time entertainment usage goes up on networks with usage-based billing. In fact, the only thing reduced appears to be file-sharing traffic.

The rest of the report is chock full of great data such as this tidbit that confirms North America’s love of Apple products:

So what single home roaming device consumes the most Real-Time Entertainment traffic at over 10percent? It’s the iPad. In fact, Apple devices as a whole play a large role in the consumption of Real-Time Entertainment. If you add up all Apple manufactured devices (which includes iPads, iPhones, iPods, AppleTVs, and Mac computers), they consume over 35% of all streaming audio and video on North America fixed access networks.

There’s also some good data from Europe that shows that the lowered availability of over the top options like Netflix or the BBC’s video player cause the amount of real-time streaming traffic to drop. Additionally the report shows that in Europe file sharing is higher than in North America, something the report’s authors attribute to a lack of access to certain popular content because of geo-blocking.

But taken in its 40-page entirety, the data and case studies show how our love of video is causing both wireline and wireless ISPs to get creative to boost revenue and meet the challenges posed by the demand for video. Just like we said it would.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• What Amazon’s new Kindle line means for Apple, Netflix and online media
• How consumer media will change in 2013
• Who and what to watch in the new era of the living room

And guess which country is leading the charge, offering its citizens some of the fastest Netflix speeds? That’s right, the United States. However, U.S. broadband only came in first because of Google Fiber, whose very few actual customers saw an average Netflix speed of 3.35 Mbps in February. Second in is the U.K., where Virgin customers averaged 2.37 Mbps during the same month. At the bottom of the list is Mexico, where the fastest ISP averaged at 2.10 Mbps.

How U.S. ISPS are performing for Netflix viewing.

Of course, these speeds are far below what most ISPs advertise for their services, but the averages include lower-bitrate SD fare, network slowdowns due to poor Wifi performance and all kinds of other factors. Or, as Netflix puts it:

“The average is well below the peak performance due to many factors including home Wi-Fi, the variety of devices our members use, and the variety of encodes we use to deliver the TV shows and movies we carry. Those factors cancel out when comparing across ISPs, so these relative rankings are a good indicator of the consistent performance typically experienced across all users on an ISP network.”

Still, the site is an interesting tool to compare broadband speeds both within the countries in which Netflix is active as well as between those markets — and for the company, it’s another way to nudge ISPs toward signing up for Netflix’s own CDN.

Photo courtesy of Shutterstock user Sashkin.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Who and what to watch in the new era of the living room
• The Evolution of Over-the-Top Video
• How consumer media will change in 2013

The new rules, which have been in the works for over a year and have been repeatedly delayed, are being administered by the Center For Copyright Information — a non-profit entity made up of theoretically independent representatives from agencies like the Internet Education Foundation and the Future of Privacy Forum, and includes Jerry Berman, a former director of the Electronic Frontier Foundation, as well as Gigi Sohn of Public Knowledge. They have partnered with five of the largest ISPs, including Verizon and Comcast.

Part of what makes this new strategy difficult to understand is that each service provider’s method for implementing the rules is different. Verizon says that after several warnings via email and popup message, users who are downloading or sharing copyrighted content will be given several options, including a temporary reduction in their internet speed. AT&T’s policy apparently says that after several warnings a user’s ability to access popular websites will be blocked until they complete a course in understanding piracy and copyright infringement.

So should you be afraid of these new rules? That depends. Are you are only worried about how they might affect you directly, or are you concerned about the ways in which private corporations are seeking to snoop on and limit your behavior? Let’s break these two viewpoints down:

Why you shouldn’t be worried:

It doesn’t affect all internet service providers: Although providers like Comcast and Verizon are huge, they don’t cover all internet users in the United States, so it’s possible that you might not even be affected by the new restrictions even if you do download a lot of copyrighted content.

You get six strikes, which is probably more than you need: Copyright owners and the Center for Copyright Information say that the intent of these new rules is to go after the most egregious downloaders and sharers of content, not the person who occasionally downloads a new song or a movie. So if you don’t do a lot of peer-to-peer file-sharing, you probably won’t be affected.

You won’t get cut off, just lectured and irritated: Even if you do get flagged for something, the worst that most of the ISPs say they will do is limit your download speeds, show you popup warnings or send annoying emails. And some have said even if you ignore them, nothing will happen (although they could always change their minds about that later).

There are lots of ways around these restrictions: One of the criticisms of such rules isn’t that they are too invasive, but that they don’t work against the really hard-core file-sharers that are allegedly the target of this strategy — since virtual private networks, proxy addresses, cloaking software and other tools can make it almost impossible to detect infringing downloads.

Why you should be worried:

Your ISP is going to be doing some heavy snooping: One of the broader risks that groups like the EFF point to in their criticism of these new restrictions is that they rely on ISPs snooping on their users to an almost unprecedented degree — and this raises the same issues about privacy that debates around technology like “deep packet inspection” have. The potential downside is fairly significant.

The new rules don’t take into account fair use: Much of the material produced by the Center for Copyright Information makes it sound as though anyone downloading or sharing any copyrighted content is breaking the law — but that’s not the case at all. There are many instances in which the principle of fair use applies, and these rules don’t take that into account.

Copyright holders are unlikely to stop here: One fear about the six-strikes process is that it is just the latest move in an ongoing attempt by copyright holders and content companies to exert more and more control over what users can do, and that allowing it to proceed only encourages them to pursue even harsher measures such as SOPA and PIPA.

This puts commercial entities in place of laws: One of the biggest criticisms from free speech and open-web advocates is that the six-strikes rules essentially allow private corporations — movie studios, music labels and large telecom providers — to set up a quasi-legal process for pursuing their copyright claims, when the legal system is the appropriate place for those arguments.

The bottom line: There’s reason for concern

In the end, while this move may not affect you directly — or may only be a minor irritation in your daily life — the fact remains that it marks another attempt by content owners to exert their influence in areas that should belong to the courts and should in principle be protected by things like the First Amendment and the principle of fair use, neither of which are even mentioned by the promoters of this process.

Not only that, but as my colleague Jeff Roberts notes, focusing on these kinds of efforts feels a lot like what the music industry did while it was trying hard not to innovate as the web grew bigger and bigger. The risk for copyright owners is that they rely too much on these kinds of measures, instead of working to create a market and a digital ecosystem that fosters the creation, sale and distribution of content in a way that works with the web instead of against it.

Post and thumbnail images courtesy of Shutterstock / Cienpies and Flickr user Pew Center

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Pinterest reawakens Napster-style debate over copyright
• A near-term outlook for big data
• SOPA, OPEN and the fight for the Internet

In the last month, cable companies have switched from justifying their caps as a means to handle congestion — or bandwidth hogs — and are now saying it’s about recovering the billions invested in their network. In January the NCTA president (and former FCC Chairman Michael Powell) said when asked about caps as a means of controlling congestion: “That’s wrong. Our principal purpose is how to fairly monetize a high fixed cost.”

After the FCC has decided to take a half-hearted interest in caps and users and industry participants questioned the accuracy of how such caps are measured and the implemented, the cable industry is changing its justification for their policies. The problem is that its latest justification is just as false as its previous one.

The fixed costs to deliver broadband are refuted by the high profit margins broadband delivers to cable firms, the fact that upgrades to higher speeds costs relatively little and that most of the infrastructure cable providers built is already paid for.

The Open Technology Institute, a policy group that, yes, takes money from Google, has released a report attempting to quell this line of argument form the cable industry. Data points cited in the report include the ever popular fact that cablecos get 95-97 percent profit margins on their broadband services, that the billions invested in their networks in the early 2000s were to compete with new pay TV products from the telcos and the satellite broadcasters, and that adding broadband to existing cable infrastructure costs very little.

To back up that last claim check back to Cablevision’s comments to Wall Street that upgrading to DOCSIS 3.0 which provided faster upload and download speeds, were $70 per home (Cablevision doesn’t have a cap.) Other analysts pegged that number at about $100.

From the report:

Yes, cable companies and others have invested billions in building networks, but they have received more than healthy returns on those investments for several decades. According to analyst estimates listed on the NCTA website, cable companies invested over $185 billion in capital expenditures between 1996 and 2011. But these networks generated close to $1 trillion in revenue in the same time period. Moreover, both Comcast and Time Warner Cable are now spending less on capital expenses relative to revenue than in past years.

I’m not actually against the 95 percent profit margins or even caps if the market for broadband were competitive. Unfortunately, when more than thee-quarters of American homes have the choice between only two providers it’s clear that competition for the basic broadband service is limited. And when you look at how competitive the services are based on speeds there are big differences. For example, AT&T U-Verse tops out at 24 Mbps on the download side while cable tops out at 50 Mbps or even 100 Mbps.

But as someone who has documented legitimate questions about caps; their size, their spread, their rationale and their effect on innovation, I’m hoping that more and more consumers, lawmakers and regulators wake up to the fact that these caps aren’t necessary and that they pose a tax on innovation.

The NCTA has issued a statement in response to the report. I’ve included it below:

“It is regrettable that New America’s latest salvo merely repeats misleading statistics and shop-worn arguments. Their entire analysis is based on a flawed understanding of the broadband business, and the historic and ongoing investments necessary to build and operate world class networks. As the FCC and numerous economists, scholars and commenters have pointed out, tiered pricing models promote fairness by more equitably apportioning burdens between high volume and low volume users. New America’s old advocacy has gotten stale.”

This story was updated at 3 pm with a statement form the NCTA.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Report: The Ongoing Battle for the Digital Home
• Who and what to watch in the new era of the living room
• What the shift to the cloud means for the future EPG

Funambol won’t just get the $5.75 million cash infusion -– its last round in 2008 raised $12.5 million -– it will also have access to a working capital line, which it will use to expand into new international markets. The company started out as a cross-platform mobile e-mail synchronization service, but it’s evolved into a white-label cloud services company that lets wireless and wireline operators, ISPs and device makers to launch their own branded cloud storage services.

It’s biggest growth, however, has been among mobile operators who are pitting Funambol’s platform against the growing number of “siloed” clouds that limit storage to particular set of devices, operating systems or apps. Funambol and its carrier partners figure that they can carve a niche for themselves by opening up cloud content storage to any device or platform.

Funambol has already signed deals with Sprint, Clearwire, Telefónica, BT and a handful of smaller international operators, but it’s biggest customer is still unnamed. Funambol claims that customer is among the global top 5 in both customers and revenues. The only operator to fit that definition besides Telefónica is either China Mobile or Vodafone.

Feature photo courtesy of Flickr user getinet

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• The future of mobile: a segment analysis by GigaOM Pro
• How tomorrow’s mobile-centric data centers will look
• Consumer privacy in the mobile advertising era

The protests focused on the government’s broad power to monitor, intercept and block any information from the Internet as well as to force companies to remove any material it finds objectionable from their servers. Foreign Policy explains the issue in a detailed article published last week:

The trouble started with the 2008 passage of the Information Technology (Amendment) Act, whose Section 69 empowers the government to direct any Internet service to block, intercept, monitor, or decrypt any information through any computer resource. Company officials who fail to comply with government requests can face fines and up to seven years in jail. Then, in April 2011, the Ministry of Communications and Information Technology issued new rules under which Internet companies are expected to remove within 36 hours any content that regulators designate as “grossly harmful,” “harassing,” or “ethnically objectionable” — designations that are open to a wide variety of interpretations and that free speech advocates argue have opened the door to abuse. It is thanks to these rules that the website of the hunger-striking cartoonist, [Aseem] Trivedi, was taken offline. Also thanks to the 2011 rules, Facebook and Google are facing trial for having failed to remove objectionable content. If found guilty, the companies could face fines, and executives could be sentenced to jail time.

The more immediate spark for the protests, however, was a recent court order requiring ISPs to block two Bollywood films from file and video sharing sites such as the Pirate Bay, Vimeo and Pastebin. Rather than block individual videos, the ISPs chose to block the services entirely, effectively removing them from India’s Internet.

Anonymous organized the protests through its Posterous blog and over its Operation India Twitter feed. The group also coordinated multiple distributed denial of service attacks throughout the week against the websites of government departments, political parties and the ISPs doing the blocking.

Image courtesy of Flickr user zigazou76

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Controversy, courtrooms and the cloud in Q1
• Report: Monetizing Digital Content
• How consumer media will change in 2013

According to a report this week from Open Rights Group and the London School of Economics, many local mobile operators are using aggressive — but haphazard — child protection filters by default, leaving adult customers unable to see perfectly ordinary websites instead of preventing kids from accessing adult material.

As the report says (PDF):

“There are serious consequences to badly implemented, default child protection blocking systems. They include restrictions on markets, censorship, a failure to address young people’s diverse needs and a false sense of security for parents.”

The document outlines more than 60 reported cases where websites have been erroneously flagged as containing adult content — and these are just the small number of cases reported to the Open Rights Group’s blocked.org.uk complaint service.

This really isn’t just an oddity. I regularly run into blocks when browsing news or data online on my phone, which is on a business tariff with Vodafone — surely a product most kids wouldn’t be using.

And in fact, just yesterday we received a message saying that the adult filter for France Telecom-owned Orange was blocking GigaOM.

Now, I know we’re a site for grown-ups, but that’s just silly.

If your operator is deciding on your behalf that what we write is off limits — including now, of course, the fact that we’re telling you that these blocks are faulty — then there’s really no reason to suspect it couldn’t happen to anybody, at any time.

Spreading censorship

And it’s not just in Britain, either. This sort of approach is happening all over Europe, in a variety of ways.

In a piece for Foreign Policy, the author and activist Rebecca Mackinnon outlines some of the incursions being made — and points out that, crucially, none of this is happening because of regulatory pressure.

“This type of problem is serious enough, in enough countries, to have made its way to the U.N. Human Rights Council. Last year, the U.N. special rapporteur on freedom of expression, Frank La Rue, delivered an official report to the council that not only condemned the censorship and surveillance practices of authoritarian countries, but also warned of dangerous trends in the democratic world that threaten citizen rights to free expression in the Internet age.

“One of his major concerns is ‘over-broad private censorship, often without transparency and the due process of the law’. He singled out two examples of how governments are, ironically, using law to delegate enforcement responsibilities and functions to the private sector: Britain’s Digital Economy Act, which could potentially disconnect Internet users suspected of illegal downloading, and France’s similar ‘three strikes’ law.

The result of all this?

In the name of protecting us, mobile operators are now becoming the de facto censors of the web, whether we’ve asked them to or not.

Photograph copyright Shutterstock/Pixel 4 images

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Facebook’s IPO filing: ideas and implications
• Connected world: the consumer technology revolution
• Mobile 2012 and beyond

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Comcast’s long game
• Why Apple Could Be a Loser In The Comcast-NBC Deal
• OTT technologies and strategies for broadcasters

After spending the last decade developing products for these ISPs and hearing firsthand the challenges they’re experiencing, I founded a new company, Qwilt, at work on a solution I feel will benefit both operators and consumers.

Consider: Americans today watch an average of five hours a day of regular TV (depressing but true). Yet they consume only a few minutes of online video per day. That balance is changing dramatically with the rapid growth of video delivered through Web services like Netflix, HBO-Go, Amazon.com, Facebook, YouTube, Hulu, Xfinity-TV and others. It may not be too long before many of us will be hitting the volume caps set by carriers while consuming legitimate services of this kind.

In five years, we’re going to be watching about an hour a day of online video, according to TDG Research. That’s roughly 16 times more than consumption today. And it implies that Internet networks will need up to 10 times more capacity than they have now to handle the deluge.

What to do? Service providers are obviously not sitting still while this is happening. But their current options are limited. One solution is to simply throw more capacity at the problem — buy more gear from the usual telecom-gear suspects in the hopes that the boxes can handle all the new traffic. This is what many service providers have done so far.

But in many ways that’s a wasteful solution: carriers are still transmitting the same video time after time, again and again across the network every time a new user calls it up. This approach doesn’t take advantage of technology that would streamline this process and allow carriers to cache popular videos and serve them more efficiently. It also cannot scale with the exponential growth of video. Plus, it’s expensive — all that new gear isn’t free.

A second solution is for carriers to build their own content-delivery networks. Some have built internal CDNs to mainly distribute their own content to subscribers. Others have gone further and built “wholesale” CDNs that compete head-to-head with giants like Akamai, Limelight and Level3. Some of the challenges to these approaches, however, include:

• Content providers would rather not deal with dozens of small CDNs to get widespread geographic coverage — today, they deal with just a couple, or even one, to achieve global reach. The content providers we’re speaking with these days tell us that dealing with multiple, small CDNs is an operational headache.
• The economics of CDNs don’t work in favor of small providers. There are economies of scale at play here: the more traffic you bring as a content provider, the better prices you get per bit. So it’s cheaper to concentrate traffic with a smaller number of large CDNs.
• Carriers’ sales teams aren’t skilled at engaging with online content providers. Big guys like Akamai have been doing this for years and have a big advantage.

A third solution for carriers, which attempts to address some of the problems inherent in solution #2, is to create a federation of new CDNs. This would allow content providers like Netflix and YouTube to deal with just one CDN on a commercial level, instead of dozens of smaller ones, even though multiple, regional CDNs would be propagating and distributing the video content for carriers.

This is a nice idea, but it’s far from being a reality. Think about it: It won’t be easy to get operators from different countries, often with competing business objectives and different regulatory frameworks, to work together on a project like this. This doesn’t even take into account the technical challenges of having different content-delivery products made by different vendors, and owned by different carriers, work seamlessly together.

I think there is a better way.

Some new solutions, including Blue Coat and PeerApp, are being developed today that attack the problem through what is known as “transparent caching.” That means inserting a layer of network-optimization technology into networks to help make them more efficient and cut costs — specifically, reducing 60 percent to 80 percent of video traffic on the network.

Given how much video traffic is expected to soar in the coming years, that’s a technology that can make quite an impact in freeing up network resources. In basic terms, this technology temporarily stores popular videos at the edge of the network, so that they can be delivered faster, using less bandwidth, to geographic areas where those videos are in high demand.

This type of technology could be installed by such carriers as Comcast and Verizon to make sure the most popular content they’re transmitting is delivered in the highest quality. For example, let’s assume that the latest Super Bowl video summary is available at www.nfl.com and becomes very popular in New York. Carriers offering this kind of technology would automatically detect the high consumer demand for the video. They would then make any subsequent requests available to viewers from network points close to the various customers’ locations, both improving viewing experience and reducing costs.

There are several upsides to this approach. They include significantly lower costs and also more flexibility for carriers. A transparent video-delivery solution doesn’t require any specific commercial engagements for it to work. It can be used with any type of CDN. Such a system also doesn’t require any changes to existing network IP architectures, modifications to any system or browser settings, any special HTML code or integration with different vendors’ equipment. It’s an intelligent system that could be deployed and managed very easily, almost like a consumer product.

Such a solution has the potential to be a win for the entire video value chain — carriers, content providers, consumers and CDNs. The carriers get a low-cost, easy-to-integrate and flexible solution to deal with the current flood of video traffic. Content providers would have reassurance that their videos would actually get through crowded networks without bumping up against usage caps. (Recently, GigaOM reported that Charter Cable would impose such caps to combat the surge in online video that is overwhelming its network.) Consumers would benefit because they’d be able to see the video they want, in the highest-possible quality, without delays or extra costs. CDNs would also benefit, but that’s a topic for a post of its own.

With today’s technologies and the right algorithms, it is possible to unify real-time network intelligence with high-volume storage and video-delivery capabilities in a very compact, form-factor appliance. These products may then get distributed across the broad edge of carriers’ networks for delivering any form of video at a higher quality and much lower cost for all parties.

What we’ve seen in the last few months is that many carriers worldwide have realized the significant benefits of a transparent video-delivery technology and have started to roll out projects accordingly.

We’ll see how the landscape shakes out this year. As content providers expand their video offerings, delivering more and more bits to networks, it’s becoming critical for carriers to deal with their video-overload challenges.

Alon Maor is the CEO of Qwilt, a startup backed by Redpoint and Accel Partners that is developing new transparent video-delivery technology.

Image courtesy of Flickr user covilha.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• What the shift to the cloud means for the future EPG
• Who and what to watch in the new era of the living room
• Controversy, courtrooms and the cloud in Q1

It's communal broadband, man.

A Boston company wants to offer people free access to a communal broadband network if a user pays for about $300 in equipment, according to an interesting summary over at Stop the Cap. The site profiled NetBlazr, which hopes to deliver free broadband with speeds of up to 60 Mbps to companies that will share that access with others in range while turning over management of the gear to NetBlazr. From the post:

NetBlazr starts with gigabit fiber from Cogent Communications, and then delivers free or low-cost access to any customer willing to do two things:

Spend $299 for the basic installation kit, which includes a high-speed router, three antennas, and some cabling;
Use the included equipment to receive service from NetBlazr and agree to share it with anyone in range of the wireless antennas included in the kit.
Reception of the wireless broadband signal, comparable to Business Class DSL, comes with no ongoing fees. If you want dedicated, guaranteed speeds, NetBlazr will sell them to you at an added cost. The more customers exchanging signals, the more robust and faster the network becomes, says NetBlazr CEO Jim Hanley.

This is a pretty cool idea (seriously go to Stop the Cap and watch the video), although it doesn’t look like it would help with the rural broadband problem. What’s cooler is Hanley’s idea that services like this could commoditize broadband. Has he talked to Google yet?

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• A Town Hall Talk on the Future of Work
• Confused about the wireless markets? Here’s a breakdown
• The future of Wi-Fi in the enterprise