Blog Post

Breaking Down the Numbers on Crackle’s The Bannen Way

The Bannen Way, produced and distributed by the digital arm of Sony Entertainment (s SNE), has already proven itself as a bit of a lightning rod for controversy. And when Crackle told us that the crime noir web series had racked up 8.4 million streams since its early release on Dec. 29, that statistic was met by a fair amount of skepticism.

Because some of the concerns raised by our readers were intriguing, I’ve spent the last week trying to get a better understanding of Bannen Way‘s multi-million viewcount and how it fits into the web video marketplace. So here are a few of your FAQs hopefully answered:

If Bannen Way really earned 8.4 million views, why wasn’t it on Mashable’s top 10 web series of January chart?

That one’s simple: According to Matt Fiorentino of Visible Measures, which creates the chart every month for Mashable, the reason Bannen Way didn’t make it on the list was that only shows with public view counts are included. Because Crackle no longer makes its numbers generally viewable on their site, Bannen Way was ineligible.

What did Crackle use to measure views on Bannen Way?

A traffic analysis solution called SiteCatalyst, created by online marketing and analytics company Omniture (which is now owned by Adobe). Media companies like Turner Broadcasting. MTV Networks and, yep, Crackle use SiteCatalyst to measure their viewcounts, and according to a Omniture product manager, SiteCatalyst’s settings can be customized however the client wishes to determine what counts as a view.

So what counts as a view, then?

According to not one but two Crackle representatives, when a Crackle video finishes its 15-second pre-roll ad and begins playing the actual content, it counts as a view — it doesn’t matter how much of the actual video the viewer watches, so long as the pre-roll has finished playing. This means that if you visit Bannen Way‘s Crackle page and the video auto-plays, it’ll count as a view if you don’t click away before the pre-roll’s done, like one out of six users do according to TubeMogul.

(Fun fact: In the process of writing and researching this piece alone I visited Crackle’s Bannen Way page several times, and probably added about 10-20 views to the official count all on my own, thanks to the autoplay and the short length of the pre-roll.)

According to comScore, received only 2.7 million unique visitors in January. How does that correspond with the claimed viewcount?

This is the big issue. Let’s do some math. First, if we average out Bannen Way’s 8.4 million views over a period of approximately six weeks (Dec. 29-Feb. 9), we get 1.4 million views per week, which means an estimated 5.6 million views over the course of January (the real number may be higher or lower, depending on traffic thanks to promotion around the show’s premiere, but that’s what we have to go on today).

Second, about comScore — because the online measurement organization relies on sampling to gather its numbers, the comScore reporting’s accuracy isn’t as good as Crackle’s internal metrics, which measure actual visitation. A Crackle representative, speaking on background, stated that the comScore number, when measured against Crackle’s internal metrics, has in the past been between 20 percent and 200 percent lower than its actual numbers.

Let’s say that the comScore number is 20 percent lower than Crackle’s allegedly more accurate internal visitor count, which would be 3.24 million unique visitors for the month of January. For the purpose of this exercise, let’s assume that half of those unique visitors visited Crackle to watch Bannen Way instead of the other programming available, given the effort that Sony and Crackle have put into promoting the series. That comes to 1.62 million uniques for Bannen Way alone.

Third, comScore is only measuring unique visitors, not the actual number of video streams that occurred — their statistic for January will be available at the end of the month. Given the heavy amount of good press and positive buzz surrounding Bannen Way, it’s not impossible to imagine people tuning in multiple times to watch episodes — which wouldn’t affect the unique visitor count, but would contribute to the number of views.

Thus, if our 5.6 million streams in January estimation is at least vaguely accurate, that means the typical estimated unique visitor (1.62 million) at least started to watch approximately three to four episodes of the show over the course of the month.

Three to four episodes of a high-profile 16-episode show per site visitor over the course of a month isn’t out of the question, and it’s even more believable when you consider the fact that when you visit the main Crackle page for Bannen Way, the first episode is always the one that auto-plays. This means that if you come to the page hoping to watch a later episode and don’t scroll down and click on the desired ep before the pre-roll finishes, that counts as two views instead of one.

Crackle would not make their stats on repeat viewership available for publication, but it did confirm that over 90 percent of those 8.4 million streams occurred not through embeds on other sites, but on

Let me be clear about this, though — the only confirmed numbers in this piece are comScore’s reported 2.7 million unique visitors and Bannen Way‘s 8.4 million streams. I’m still stuck with a fair amount of conjecture, the fundamentals of which you may or may not agree with. But what all of this discussion cements is that, more than ever, the need for a independent reporting service for metrics, similar to Nielsen, for the web video world, to provide us with reliable numbers we trust.

Because once we get to that point, viewers know what’s legitimately popular across all sites, companies don’t have to juggle several different services in order to put out the numbers they ought to take pride in, and advertisers know what they’re paying for. In short: Everyone wins.

Related Pro Content (subscription required): Is There a Future for Original Web Video Shows?

6 Responses to “Breaking Down the Numbers on Crackle’s The Bannen Way

  1. I suppose it doesn’t necessarily count in terms of measuring popularity, but in measuring effectiveness in ad-sales, I think watching the pre-roll definitely counts as a view. You fulfilled your contract as a viewer – you watched the commercials. I think that’s pretty fair to count as a view, depending on what you’re trying to say with those numbers.

  2. Nice follow up.

    So basically a large fraction of the views could come from autoplay and be from people who did not actually watch the series or even an entire video or even a fraction of a video….


    Any chance we can get the numbers for each episode and also an idea of how many people actually watched the videos to the end… which for this type of show is the only way to measure a legitimate view.

  3. By any measure, The Bannen Way is a success with audiences and artistically whether you go with ComScore’s reported 2.7 million unique visitors or Bannen Way’s 8.4 million streams.

    We are, after all talking, about millions of viewers here for a web series and this is the tip of the iceberg as far as the long shelf life on different media that The Bannen Way the movie can expect to reap the benefit of even an initial impact with 2.7 million viewers.

    They will tell other people about the movie, they will repurchase the movie streaming or on DVD. This word of mouth will travel to other territories i.e. the entire rest of the world that has yet to enjoy The Bannen Way and this genre, well done, plays VERY well in certain foreign markets yet unexploited.

    Before we start passing a final judgement on the performance why not look at it for what it is these are the numbers for the “pilot” of the movie which will live on as a commodity and artistic success for years and make money for the studio. At a low cost to make, it already has.

    It’s that not a success for Jesse and Mark no matter how you choose to quantify the project?

  4. “A Crackle representative, speaking on background, stated that the comScore number, when measured against Crackle’s internal metrics, has in the past been between 20 percent and 200 percent lower than its actual numbers.”

    If a source is speaking on background, you’re not supposed to report it this way!

    • Ryan Lawler

      Well, there was no direct quote used, but attribution to an unnamed Crackle representative might cause some confusion. In general, Liz probably didn’t have to report that the conversation was ‘on background,’ but I don’t think any rules were broken by citing an unnamed representative of the company.