Is tracking engaged time the secret to measuring media effectiveness, or is it just another blind alley?

3 Comments

As media companies and advertisers search for better ways of measuring the effectiveness of content, a lot of attention has been focused on what some like to call “engaged time.” Designed by companies like Chartbeat, the publishing-analytics company, Upworthy — which recently open-sourced its “attention minutes” measurement — and the Financial Times, these metrics try to measure the time that a reader actually spends with a story instead of just tracking clicks. But is engaged time a red herring? ProPublica president and former Wall Street Journal assistant publisher Dick Tofel thinks it might be.

Tofel lays out his argument in an essay he wrote for the Media Impact Project, a venture launched by the USC Annenberg School for Communication and Journalism that’s trying to improve the theory and practice of media measurement.

In a nutshell, Tofel argues that while “engaged time” might be an improvement over blunt or fatally flawed instruments like raw pageviews or even unique visitors, it still suffers from some fairly major problems. For one thing, he says the simple fact that a person spends more time with a specific story doesn’t necessarily mean that they are actually more engaged. It could mean that the story is just really long, or it’s a video clip — and the longest stories aren’t always the ones with the most impact.

“The metric implicitly posits that longer stories — assuming they are read for longer periods — are more valuable, more engaging, more salient than shorter stories. But our experience teaches us that this is not the case. The most memorable stories are not always (or perhaps even frequently) the longest.”

Most content gets no attention at all

Tofel also argues that encouraging media companies to focus on engaged time or “attention minutes” could cause publishers to lengthen the content they are producing, in the same way that focusing on pageviews drove some media companies to spend all their time pumping out tiny chunks of viral clickbait. In other words, the metric itself could have unintended consequences: “Do we really want reporters and editors to undertake a constant effort to take up more of a reader’s time, to always lengthen stories where feasible… to use more video simply because people may mindlessly watch?”

tv frame video generic

And what do some of the companies pushing the idea of engaged time or attention minutes have to say? The Media Impact Project has gotten essays in response from Chartbeat CEO Tony Haile as well as Daniel Mintz from Upworthy, Anthony Hitchings from the Financial Times and Jonathan Stray of the Overview Project, a document-mining system the Associated Press is building. Haile says he agrees with Tofel that measuring the actual impact of a specific story is difficult, but argues that the idea behind engaged time is somewhat different:

“The fight is not between the short form content that receives 30 seconds of attention and the long form that receives 300… the fight is with content that does not receive any attention at all. Simply distinguishing the value of content able to capture a meaningful expression of attention from that which does not would have more impact on the economics of digital publishing than anything else.”

By implication, there are two main strategies publishers can follow, Haile says — they can either create great short-form content that gets people to return regularly, or they can create long-form stories that get readers to engage for a long time. The only people who suffer under this scenario, he says “are those who create content that no one reads and to which no one returns.”

Perhaps there is no holy grail

Jonathan Stray says like Tofel, he is unconvinced that engaged time is the holy grail of reader measurement: “Journalism is very much a multi-stakeholder endeavor, so why should we imagine that a single number can capture all aspects of the activity?” Daniel Mintz of Upworthy, meanwhile, says he agrees that engaged time isn’t the ultimate barometer for media measurement, but argues that it is another tool that can be used in addition to all of the existing metrics in order to get a better handle on what readers are doing and why:

“We’re not advocating… that publishers and advertisers should ignore pageviews and uniques and shares, in favor of attention as the only metric that matters. Instead, we’re arguing that right now, we measure the input (pageviews and uniques) and the output (shares) of a system with no real understanding of what is happening in-between. Measuring attention provides invaluable insight into that gap between input and output.”

It’s more than a little ironic that while publishing online provides an almost infinite ability to measure every twitch and click and eye movement of the reader, media companies and advertisers aren’t really that much closer to figuring out how to tell whether what they are publishing is actually having any effect or not. About the only thing we can say for sure is that as long as there are companies publishing content online and trying to appeal to advertisers, this debate will likely continue.

Post and thumbnail images courtesy of Thinkstock / Darren Klimek and Shutterstock / Zastolskiy Victor

3 Comments

TrendTopics

I like the idea of an engagement metric as part of a broader measurement model, but there are some holes in the general notion of an engagement number. The consumption of news content, whether, read, heard, seen, alerted, overheard or otherwise absorbed from some source is not as linear as an engagement number would suggest. There’s much more nuance to news consumption then an engagement number would imply.

It seems the holy grail they’re all seeking is in support of advertising – analytic feedback to the revenue source, aka advertisers. A noble endeavor. Reader interaction on a site generates various data points, i.e. clicks, paths, identity (with possible demographics) and timestamps (which are used to determine engagement metrics). Advertisers need these numbers to help make placement decisions. You can hear the publishers’ sales reps already, “our users are more engagement, just look at our engagement numbers”. This could actually mean they’re reading more content and ignoring the ads! Versus the other sales rep, “our users are less engaged in our content and more available to view your ads”.

But in all seriousness.

If your noble cause is to understand how readers interact with a news subject, i.e. Syrian Conflict and not just a site’s content about the Syrian Conflict, then there are other approaches that do leverage the nuances of media consumption. These models take into account multiple channels, blend social and traditional media sources, add search metrics and various sentiment analytics. They also further segment and weight source data, as a tweet from your neighbor is probably less likely to shape your opinion then a cover piece from the New York Times (unless your neighbor is a reporter for the Times). The interplay between broadcast, print, radio, online, social vs traditional and momentum behind a given subject needs to be assessed.

I know a company pursuing the later, but I’ll keep the viral marketing out of my comment.

Paul Senatori
TrendTopics.com

TH

maybe a little poll under articles, or star system, for people to give feedback ?

defcon_5

Did the folks promoting engagement metrics say it’s the Holy Grail? Nope. They are saying that the info gives a much better sense of the content readers value – as measured by the amount of time they actually spend with it. Why wouldn’t editorial folks want to know that? Why wouldn’t advertisers want to know that?

Comments are closed.