Blog Post

We are drowning in data about readers and attention, but which metrics really matter? You won’t like the answer

Stay on Top of Enterprise Technology Trends

Get updates impacting your industry from our GigaOm Research Community
Join the Community!

Thanks to the web and real-time measurement tools, the media industry has gone from having virtually no hard data on readers and attention to an embarrassment of riches — not only can we measure what people click on, but we can measure how far down the page they got when they were reading, whether they posted a comment, which social networks they came from, and a hundred other pieces of data. The only problem is that this is very much a double-edged sword.

New York Times media writer David Carr recently looked at some examples of media companies that are rewarding their writers based on traffic statistics and other measurements, including The Oregonian — whose efforts I wrote about here. But is paying your journalists based on pageviews or other metrics a smart way to align their incentives with your goals as a business, or does it poison the well when it comes to enhancing or encouraging creativity?

This fear of well-poisoning has even led some outlets — including The Verge and MIT’s Technology Review — to deny their journalists access to the statistics about readers and attention, because they’re concerned that it might distort their judgement about which stories to cover or how much time to devote to them. But then how do writers know whether their work is reaching an audience?

Be careful what kind of incentives you use

Grumpy Cat

In a piece he wrote for the American Journalism Review this week, Chartbeat CEO Tony Haile (who is also an adjunct professor of journalism at Columbia) looked at both sides of the data sword. One danger of using the wrong metrics to reward your journalists, he noted — as I also tried to point out in a recent post — is that you wind up incentivizing the wrong thing, and that can take your site far away from what its original goals were:

“What we choose to measure defines what we aspire to. It is thus unsurprising that when a newsroom prioritizes a measure of link performance, that newsroom learns how to optimize links, but not content. Instead of metrics teaching the primacy of good content, we get clickbait and slideshows.”

BuzzFeed founder Jonah Peretti, who some might think would be a natural proponent of traffic-boosting statistics given his site’s focus on viral content, has also talked about the downside of this phenomenon. “If you are a slave to the numbers, then you start creating more stuff like that… and pretty soon you will have a site full of trash and salacious garbage,” the BuzzFeed CEO said in an interview with Capital New York last fall.

As Haile put it in a similar piece he wrote for Time magazine earlier this year, media companies are “getting a lot wrong about the web these days — we confuse what people have clicked on for what they’ve read. We mistake sharing for reading.” As absurd as it sounds, according to some of Chartbeat’s research, whether someone shares a link to a particular story or blog post has virtually no relationship to whether they have actually read it or not.

[tweet 430008055263408128 hide_thread=’true’]

Who is your customer — reader or advertiser?

So if publishers and media companies shouldn’t be looking at raw pageviews — which many advertisers still seem to prefer as a measurement of how effective their ads are going to be — and they shouldn’t be looking at sharing statistics because that doesn’t measure whether people actually read the content, then what should they be looking at? Not surprisingly, that question is more complicated than it seems, because there is no single answer.

Part of the problem is that most content publishers are serving two very different masters: one is the reader, and the other is the advertiser (the exception being reader-funded sites like Andrew Sullivan’s The Daily Dish or Jessica Lessin’s The Information). As Jeff Jarvis argued in a Twitter conversation with Haile — which I have Storified here, and also embedded below — if you want to measure service to the reader then you have to look at different data.

Sites like Upworthy and even Chartbeat itself, Haile points out, choose to focus on metrics that measure something approaching “engagement” — so not only whether a page was loaded but how long the reader spent, whether they were a returning visitor, and so on. Upworthy’s version of this is called “attention minutes” (which I wrote about here) and Chartbeat calls it “engaged time.”

Unfortunately, many advertisers and brands continue to look at what they call “reach,” and that encourages a focus not on engagement but on raw numbers like pageviews, as News Corp. executive Raju Narisetti noted in a recent piece for the Poynter Institute. And the larger those numbers are, the better advertisers like them — even if that encourages gaming strategies, such as buying traffic from clickfarms, etc. And that is often exactly what happens.

[tweet 456059858354466816 hide_thread=’true’]

How do you measure engagement?

Driving pageviews is not difficult even if you don’t want to buy cheap traffic from clickfarms: all you need is someone like former Gawker writer Neetzan Zimmerman, who drove more pageviews per month than all of the other writers combined by focusing exclusively on viral content. Scott DeLong, who created a site called Viral Nova to do something very similar, built a massive traffic-generating engine singlehandedly from his house in rural Ohio.

The downside -of this kind of behavior – as Jarvis points out in a recent post at Medium that is part of a series about the future of news — is that focusing on these metrics drives journalistic organizations even farther away from what should their goal, which is not just encouraging engagement but actual interaction. Haile makes a similar point in his AJR piece, and comes to a fairly grim conclusion:

“Newsrooms are mission-driven businesses that have more complex standards of success than just profit. That complexity requires a more thoughtful approach than simply Revenue = Impressions = Pageviews. Otherwise, like the oily salesperson who wrings every last dollar out of your visit but ensures he will never see you again, media companies risk destroying their ability to build a long-term business in exchange for a few additional dollars today.”

Embedded below is a Storify of the conversation that Jarvis and Haile had on Twitter about the metrics media organizations should be paying attention to:

[protected-iframe id=”80a5f2cccb6830e5b43c4fad15786aa8-14960843-8890″ info=”//” width=”100%” height=”750″]

Post and photo thumbnails courtesy of Thinkstock / Sergey Nivens

7 Responses to “We are drowning in data about readers and attention, but which metrics really matter? You won’t like the answer”

  1. Chris Casquilho

    Time on page isn’t even accurate. One of the ways I remember to read something is to see the link in twitter and open it in a tab. After awhile (2 minutes? 2 hours?) I get back to the tab and sometimes read the article and sometimes not. For instance, I opened this article two hours ago, got up, did a bunch of stuff, then read it. Assuming the metrics weed out the outliers like a two-hour pageview, how sure can you be that a two minute page view wasn’t just sitting there? The fact that no real empirical data about actual reading can be gleaned reminds me of those weird motion-trackers the Nielsen people tried to use to ensure people were actually sitting on the sofa when the set was on…

  2. Gene Radcliffe

    Sorry, but Storifying tweets is a ridiculous way to share what should be a meaningful, rich and deep conversation. Unless, of course, you are writing for The Onion

  3. Pete Dooley

    I was on one of my daily reads, The Raw Story, they were mentioning that they needed more support. I mentioned that they should put a little PayPal link ( I know it sounds gauche) at the bottom of the page. I read many articles, and link to them, that I would be glad to drop a fiver on. This article being one of them… Thanks for the great information.

  4. Mark Woodward

    Great post.
    Whenever i hear somebody saying that you ‘should’ do something it rings alarm bells for me.
    In my experience media companies understand what they ‘should’ ideally do around engagement and building long term but few are in the luxurious position of being able to wait for the revenues to roll in. In transitioning cost-heavy legacy businesses to digital they need revenues today rather than tomorrow and therefore chase page views out of necessity.

  5. the problem is not content and it’s misleading messages, i.e getting some scintillating news to draw the viewer, hits increase etc…, everyone thinks it’s working, the problem should be, retaining readership, and over a period of time, it’s easy to work out what works and what does not. as we all are dependent on media, and we all have the 24 hour news channel at our fingertips, it should be a no brainer, what ought to be done, apply the soup principle, it usually works.