14 Comments

Summary:

As newspapers try and re-engineer their businesses to adapt to the disruption caused by the web and social media, they will have to confront a crucial question: How can they measure the effectiveness of the journalism they are producing — or is pleasing advertisers enough?

112082907_8c282f0761_z

One of the things that publishers of all kinds love about online media is that it can be measured in a thousand different ways: every pageview and every click can be tracked, and the amount of time a user spends on the page can be quantified, as well as where they came from and where they went after they visited. But as Jonathan Stray notes in a recent post at the Nieman Journalism Lab, a much harder question is why we are measuring these things at all. Is it for the benefit of advertisers? Is it to prove that we are accomplishing something worthwhile for society as a whole? Before we can properly measure whether online journalism — of any kind — is effective or not, we have to answer the question: What is journalism for?

As we’ve described a number of times, the advent of the web and social tools like blogs and Twitter and Facebook has disrupted virtually every aspect of the media industry, from books to movies. And one of the hardest hit has been the traditional newspaper business, which finds itself competing against a growing number of online entities for both audience attention and advertising revenue. In response, some publishers such as the Newhouse family’s Advance Publications have been cutting back on printing newspapers in places like New Orleans, leaving that city without a printed paper for several days of the week — and causing an outpouring of criticism.

How do we measure the impact of a newspaper?

Much of the reaction to these moves — including an open letter from famous New Orleans residents criticizing Advance and responses from media analysts such as New York Times media writer David Carr — have focused on the idea that an online newspaper can’t possibly have as much of an impact on a city like New Orleans as a printed paper can. How do we know? The short answer is that we don’t, because there is no easy or reliable way to measure the impact that a printed newspaper has on the society around it, apart from pure sales. As Stray puts it:

“Evaluating the impact of journalism is a maddeningly difficult task. To begin with, there’s no single definition of what journalism is. It’s also very hard to track what happens to a story once it is released into the wild, and even harder to know for sure if any particular change was really caused by that story.”

Newspapers have always argued that they are important because the stories they cover result in changes to legislation or in corrupt companies being investigated, or have other social benefits such as highlighting health problems, but in virtually all cases the evidence is anecdotal at best. And as Stray notes, this problem doesn’t become any easier online — where everything can and is measured — because it isn’t clear what constitutes success for any given piece of journalism. Is it the number of readers it draws, or the “engagement” they produce (via things like comments or tweets)? Can you find it by looking at the number of links to that story from other sources over time, the way that Google’s Page Rank does?

There are some efforts to try and answer some of these questions, including a project from Aron Pilhofer — a developer and head of the data-journalism team at the New York Times — that involves a Knight-Mozilla News fellowship. It’s not clear from Pilhofer’s description what exactly the project plans to focus on: he says he hopes to come up with a framework and a methodology that will allow news entities to measure something approximating their social impact. But again, the problem is what to measure. As Pilhofer puts it:

“We are awash in metrics, and we have the ability to engage with readers at scale in ways that would have been impossible (or impossibly expensive) in an analog world. The problem now is figuring out which data to pay attention to and which to ignore.”

Is serving subscribers enough, or is there more to it?

Most of the efforts at measurement that publishers have made so far consist of tracking eyeballs and responses so that advertisers can be sure they are reaching the right audience, because when you rely on ads for the bulk of your revenue that is the most important factor in your survival. But what about newspapers like the New York Times and Financial Times, which are either close to or have already become primarily funded by reader subscriptions? Surveying those readers to determine whether they feel satisfied is one way to quantify your success — but is that enough? What about the broader public mandate that journalism is supposed to have?

One of the things that complicates this whole process is the fact that “journalism” is so poorly defined in the first place, as Stray and others have noted. What do we mean we say that word? In most cases, people seem to mean investigative or meaningful reporting on global events like the war in Afghanistan or the crimes committed by hedge funds and banks. But the reality is that those kinds of stories make up a tiny fraction of the journalism produced by major newspapers and other outlets — they are vastly overwhelmed by entertainment news, “service” journalism about things like how to file your taxes, gardening columns and so on.

As news developer Stijn Debrouwere pointed out recently, much of what traditional media companies are competing with for attention and ad revenue doesn’t even look like journalism: things like Reddit’s “Ask Me Anything” features, or Tumblr’s viral content, or BuzzFeed’s animated GIFs. Those services don’t worry about what the social impact of their content is — all they care about is clicks, because they don’t have a social mandate. Newspapers don’t have that luxury, but nor do they have an easy metric to demonstrate their success.

That is going to become a crucial issue as the disruption of the industry continues to increase: How does a mainly digital media entity determine whether it is having an impact or not — or having the kind of social impact that we have come to associate with journalism? And if it can’t answer that question, then why do we need it?

Post and thumbnail images courtesy of Flickr users Kevin Lim and Woodley Wonderworks

You’re subscribed! If you like, you can update your settings

  1. Our country is in peril because the free press is now just another arm of the DNC. They have chosen sides instead of seeking truth. If this economy and lack of action happened under a republican , they would be advocating riots and repeated stories of doom and gloom.

    1. Oct 2008, George Bush and Henry Paulson, with somber & almost tearful faces, announced weekly bailouts of national banks and Wall Steet investment banks. Then the next administration actually had to enact Paulson’s extensive proposed bailout plan.

      I was reading and watching the news from our free media … and they never incited riots. Where we’re you, in diapers?

      1. Actually, I think both of you are somewhat right and somewhat wrong.

        To deny that a (thankfully dying) MSM has not been historically biased to the Left (enormously) is to be delusional.

        From the 60’s through the 80’s the three network oligolpoly (along with local newspaper monopolies) never saw a State-based “solution” they didn’t like – regardless of how self-serving this was for the political class (defined as elected officials, lobbyists, and anyone making their living from taxes).

        It was an extremely phony political consensus forged from media monopolies (and political gerrymandering) and it willed us generational ruin in the form of unfunded “entitlements” and politically created distortions of the free market (of which I would include the 2008 bailouts – *passed* by a Democratically-controlled Congress and a Republican President).

        The Republicans and the Democrats are simply two political-class mafias fighting over the protection-racket money while the MSM historically covered the political Kabuki as though it were the real, underlying truth of what was going on.

        It wasn’t – but the MSM was too corrupt, too stupid, and too self-serving to admit that.

        And too unchallenged to have to.

        The MSM’s main job was to skullf*ck 80% of the population into believing that our “betters” in the political class could run our personal and economic lives better than…*we* could.

        Our present decay and promised ruin is where such thinking (MSM bred) has led.

        But Bob is wrong in that, with ever increasing power, the Internet is giving rise to an ever more diverse and wide reaching range of voices and alternative solutions – progressively disemboweling a historically corrupt MSM.

        The terminally poisoned MSM (yea, Internet!) is still running its tired old scams – but the Internet is showing them up for the bad, sick joke that they are.

  2. JohnnyJournalist Tuesday, August 21, 2012

    One of the metrics we can probably measure on this article of yours is that the picture appears to have cost Gigaom nothing, since you guys appropriated it from Flicker. The next question is, did the writing cost anything? Are senior writers paid on the Internet?

    What I’m getting at, is the economy of scale that Internet compositions seem to take. How low is the economy of scale? How close are the Internet entities running this race to the bottom.

    It looks very close to the bottom.

    Matt?

    1. Not sure what your point is, er… Johnny. Yes, we use Creative Commons-licensed Flickr photos (for which we credit the owner) and yes, writers for GigaOM are paid — and yes, digital-only media has a lower cost-base than most traditional media.

  3. Kevin Colligan Tuesday, August 21, 2012

    Bob’s ridiculous comment about the press being an arm of the DNC is part of the problem. Professional journalism has long been besieged by several forces:
    1. Large corporations which gobbled up newspapers (Gannett and other chains) and television news outlets, and which are now viciously trimming expenses (ie: investments in quality content) and tarting up the news with entertainment fluff and opinion while chasing readers/viewers/advertising dollars
    2. The right wing (and sometimes left wing) which has demonized the media since the Nixon administration in order to inoculate themselves from legitimate criticism
    3. Feckless “journalists” who act as showbiz hucksters or stenographers to the powerful to please one or both of the above
    4. Worst of all, the general public doesn’t seem to be all that interested in serious news (eg: PBS NewsHour.) Compare CNN International and CNN’s American offering to see how news execs view their own country’s tastes. Very depressing

  4. As a PR professional, what I find interesting is that in spite of the digital media takeover, clients still put more stock in being featured in hardcopy publications. I’ve wondered if this is simply a holdover mentality–and they just haven’t adjusted to the new media environment–or it there’s more to it than that…if there is something about a hardcopy publication and its tangibility that actually makes it more impactful/influential somehow.

    1. I think it’s probably a little bit of both. Thanks for the comment.

    2. I’m in PR also and my clients don’t care really as they want the media outlet that has the right audience and most eyeballs. I think what you are talking about is credibility, which is what PR is all about. How many people can say they were printing in the NYTs or WSJ? It’s more of a “wow” factor and unfortunately impossible to measure – I can’t tweet a piece of paper.

  5. Trying to measure journalism’s immediate effect or impact is a distraction: results are often diffuse, elusive and often long-term. Get over it. The benchmark questions need to be: what are you doing? Why are you doing it? And, given the first two answers, are you doing it well? Some of my approaches to these questions are here: http://www.scribd.com/doc/28560140/George-Brock-Is-News-Over

    1. George Brock, I’ve shared this lecture with my colleagues. I thought that the comment about competing intelligences — situational knowledge versus expert knowledge was especially spot on.
      Michelle Ferrier, Associate Professor, Elon University, USA

  6. To touch on Mingram’s point on the preference of print advertising over digital: Once printed an advertisement will not change. Seeing an ad on a digital news page and returning to said page does not guarantee that same ad will appear the next time. There’s a “permanence” to print that digital does not have.

    I’ve been following the “how can print outlets survive in a digital world?” debate for the past 15 years or so, from the inside and out. (Disclosure: I was a newspaper reporter and editor between the years of 1999-2006.) The author is right in that there is no way to quantify with numbers the impact of journalism, specifically a printed newspaper, has on an audience. But that mentality is a digital mentality, where metrics and analytics are abundant and readily available — it can’t be applied with the same purpose or effect to a different medium. From printed content, to televised content, to Internet-delivered content, the ability to measure what we consider impact has improved. That’s well and good, but for all our fine tuning of measuring views we’re still just measuring views. Impact, or influence as called by another name, is something else entirely. As soon as we figure out how to measure accurately the influence of something someone sees, this argument will be academic. Until then, we can measure ad clicks that may lead to conversion, but even measuring “views” calls into question just what that word means.

    Newspapers face, and have been facing, declining advertising revenue. Call it lack of readership, a shrinking pie of advertising dollars, the emergence of new media to attract a limited ad spend, whichever is the culprit of the day there is less advertising money to go around and because of that print media is starving. The result of this decreasing revenue source is a decline in the ability of a media outlet to accurately and adequately report the news. The result of that decline is a less-informed public and less-governed commercial and governmental sectors. If the question is “what is journalism for?” the answer has to be to serve the public good, do no harm, and report the truth as it is, not just how it appears.

    I’ve often heard people who say they prefer to get their news from the Internet because they don’t have to pay for it. This is short sighted and disappointing. News is produced by fewer and fewer sources these days but ends up being rebroadcast or linked to by more and more channels. This isn’t source proliferation, it’s source amplification. The Internet — in this regard — hasn’t unleashed the inner publisher in all of us, it’s merely given everyone a megaphone and a parrot.

    To be sure, there are notable blogs and other web-based reportage sources to break news and do excellent reporting, but they lack both the clout and authority of a long-standing watchdog that a newspaper with decades of public service does. Newspapers would do well to reframe the way they see themselves — not as producers of news but as deliverers of content. It’s their distribution network that matters, their ability to reach people with the content they deliver. Their goal shouldn’t be survival but evolution. Journalism has existed for thousands of years, starting with epic poetry and the reportings of Herotodus — it will survive. Newspapers and their respective news outlets may not, but that’s up to them to decide what they want to be when they grow up.

  7. Journalism used to be for investigative news when you had lead-time to produce thoughtful and well-researched stories in print an OWNED the medium. Today this is still true but you are competing at a different game now because it’s all about speed and who broke the story first. Having a PR background, this made our lives easier and harder as well – there are more outlets for our clients to be featured on but because of the fragmentation, it was harder to measure the direct result from it (though we don’t have too but we still try).

    I want to make it clear though that is isn’t the journalists fault for this happening, it’s the large media companies that work for whose business modeling’s are failing them. And at the end of the day, the general public loses out because you can’t cover everything new and we have to go to multiple websites to get our news.

    Measurement is a whole new discussion… Pinterest is driving more traffic to websites than like Google, Twitter and Facebook combined.

  8. There are a number of kernels here. Journalism is storytelling, and it will continue to evolve – people are storytellers, always have been, always will be. So the question is, can established media continue to evolve and make money? Yes, I think so. I think the NYTimes is a good model – what they do well that others don’t is 1) provide great presentation in print, online and mobile, 2) highlight their unique and important content in such a way as to demonstrate clear value to the consumer, to such a degree that 3) NYTimes enjoys a dual revenue stream (online subs, print subs, AND advertising). Other papers that have resources and unique content can do the same. Clearly it’ll be harder for smaller papers to match that model, so they’ll have to develop their own version. Nevertheless, I think there will be room for a solid organized – and profitable – journalistic enterprise in most towns and cities, with social media and bloggers et al contributing to the general discourse.

Comments have been disabled for this post