Arnon Mishkin is a partner with Mitchell Madison Group, where he consults for media companies on improving legacy businesses as well as maki…

Arnon Mishkin (new headshot)

Arnon Mishkin is a partner with Mitchell Madison Group, where he consults for media companies on improving legacy businesses as well as making the internet profitable. Prior to MMG, he was a partner at the Boston Consulting Group, where he did some of the firm’s earliest work on the web.

People who “get the web” will explain to you that the economics of the web have everything to do with linking and getting linked to. The more links one can get, the better off one is. Few disagree with that guidepost.

So when the AP and the newspaper owners demanded that they get revenue from the linkers, it was clear that they just didn’t understand the web and didn’t appreciate all the value they were receiving from link traffic. (Here are just a few examples of that critique.)

Well, the data suggests that the web – the “blogosphere”- is less an ecosystem than a one-way street.

The vast majority of the value gets captured by aggregators linking and scraping rather than by the news organizations that get linked and scraped. We did a study of traffic on several sites that aggregate purely a menu of news stories. In all cases, there was at least twice as much traffic on the home page as there were clicks going to the stories that were on it. In other words, a very large share of the people who were visiting the site were merely browsing to read headlines rather than using the aggregation page to decide what they wanted to read in detail. Obviously, this has major ramifications for content creators’ ability to grow ad revenue, as the main benefit of added traffic is the potential for higher CPMs. (Disclosure: I have consulted for the AP and other content creators, though not on this particular issue.)

Even in an absolute best-case scenario for producers of original content, the aggregators get at least as much traffic on linked stories as the creators of those stories because anyone who clicks on the link does so from the aggregator’s site (so each site gets a page view). If you don’t believe the data, consider how often in an average day you visit the home page of your favorite news site vs. how often you click through to the underlying story.

Actually, it shouldn’t be surprising to anyone who’s thought about how people have historically read a newspaper: They’ve scanned the headlines and then turned to the sports, movie listings or recipe pages, depending on their real interest. As the saying goes, “People don’t check the news to read about the fire, they check it to learn that there wasn’t a fire.”

Historically, the value of those casual browsers was captured by the newspaper because the readers would have to buy a copy. Now all the value gets captured by the aggregator that scrapes the copy and creates a front page that a set of readers choose to scan. And because creating content costs much more scraping it, there is little rational economic reason to create content.

This is not just a reflection of economic reality but also captures the changing nature of reporting. Take Iran. When the Shah fell in 1979, the journalistic hero was Mike Wallace, who flew to Tehran and pointedly asked the Ayatollah whether he was, as Anwar Sadat called him, a “lunatic.” This year, after the June election there, the journalistic hero was the blogger for The Huffington Post who stayed in D.C. and linked to every piece of information from Tehran he could find.

Moreover, because they can scrape essentially every content provider on the web, each aggregator gets to build a “front page” to target and win over their chosen segment, or enable each user to tailor a front page perfectly suited to his or her needs. And they can do that by leveraging all the resources of the global journalistic community without paying any part of its cost. As a result, aggregators – often from a standing start – are able to create substantial amounts of daily traffic and viable web businesses.

People will argue that the scrapers create value by pointing to many obscure stories that captured the imagination of linkers and got unexpectedly high traffic for a very obscure site. Fine, but was that site able to monetize the jump in traffic? And, how likely is that site to create a sustainable business by consistently winning a surfing game of serendipity?

Others will say that the site that gets linked to can keep the user using the site. But the opposite is happening – users are being trained to increase their usage of (and thus value to) the linker rather than the creator.

Can anything be done about this? I think publishers need to take three steps:

• Assess how much value the aggregators are getting by virtue of using their content and use that to seek an equitable economic relationship. And be willing to drop the links rather than submit to an unfair deal.
• Consider partnering with other content makers and developing appropriate aggregation sites of their own.
• Develop new ways of sharing their content with other sites – in ways that allow them to monetize some of the traffic on the aggregator site. For example, rather than offering a pure “widget” of the site’s material, the site should offer “wadgets” that contain a combination of content and advertising that could run instead of an aggregator’s scraping.

  1. folks that continue to refer to web properties as "sites" consisting of "pages" have no clue.

  2. People who "get" the internet know that linking is a LAZY way to marry audience to platform, like SEO. And like SEO driven traffic, wouldn't be surprised to find out that session times aren't that great. 1 mil unique visitors who are on page less than a minute and do not return aren't really traffic.

    Smart people forget trying to cut corners and marry a real audience and customer base to their platform.

  3. Andrew Waterman Thursday, August 13, 2009

    Traffic to the successful aggregators (SeekingAlpha.com, HuffingtonPost.com, etc.) is quite disproportionately larger than the traffic to their content providers, leaving many content providers feeling this is a "raw deal" as the aggregators get the ads dollars.

    However most aggregators are not the size of SeekingAlpha and HuffPost and are not the real culprits. Here the problem lies with the ads model we have all grown dependent on. Pageviews, not just clicks, have become cheap currency when monetized on a CPM basis alone. More value can be extracted from high-quality content than just $0.01/pageview that we are all fighting each other over now.

    Most professional content producers have premium content to sell as reports or subscriptions that ought to be marketed alongside the headlines/free content on the aggregators' sites. Arnon's "wadgets" suggestion could accomplish this well, as well as other monetization systems that recognize this new "syndication economy."

    Andrew Waterman

  4. So it's not a "link economy" because the currency is distributed unequally and tends to accumulate to certain "winners." I guess the U.S. economy is a "fallacy" too, by your reasoning.

    Were you calling for literal communism — an "equitable economic relationship" between subjects and publishers — when it was newspapers and magazines repurposing information provided by unpaid sources?

  5. That's not to imply I'm conceding your premise, by the way. All evidence indicates that actual scrapers are low-rent operations making little if any money.

    Google News, the most prominent, is not monetized; HuffPo, which publishes reams of original content alongside some scraping, is famously unprofitable; the rest are mainly spam blogs that clog up Google results and disappear as quickly as they pop up.

    The latter are grasping for AdSense pennies and usually operating in clear violation of copyright law. But their clickthrough rates are so low most original copyright holders don't bother to shut them down.

  6. i'm guessing the author would consider a twitter account as a useless tool as well.

    since one can just "scan the headlines" using the (unmonetized) twitter feed most news orgs. have fallen all over each other in setting up and feeding , why would one bother to follow the embedded links to their ad supported property?

  7. Or, you can consider that aggregators are like digital newsstands. You come in, browse the content and "buy" it or not. In newsstands, many of us browse and don't buy. Same idea with aggregators. No? The aggregator does not charge to display your "content teasers" and makes money with advertising. Do print publications charge newsstands because they are selling cafe while people are browsing/reading magazines for free? Maybe they should after all. ;)
    Not sure the media industry is looking in the right direction. You said strategy?

  8. This is so ridiculously tired. Can we just stop already?

    There are thousands of businesses making money on the Web. Lots of money. The fact that newspapers cannot figure out how to be one of them is simply sad. These are lazy businesses, unable or unwilling to keep up with the competition.

    What do we do with those types of businesses? Bailout! I mean, we let them fail.

  9. While opposing the response to aggregators by some newspapers and AP, this article reveals a good deal of truth.

    Here are three similar posts I wrote some time back (this article reads almost like they were used in prep of the story).




    Many advocates of the link economy do not fully understand how links actually work.

  10. Ya know, I've been called many things, but I can't remember someone saying I was a communist. Several of my clients would be seriously upset and several detractors would be seriously surprised.

    To me an "equitable economic relationship" is one in which buyer and seller each feel they have gotten the best deal possible. I think Adam Smith himself opined on this.

    In addition to not being a communist, I'm not a lawyer, but as I understand the basics of "fair use" the core test is whether the person using the content has damaged the business of the person who created it. The fact that Google News and Huffington Post have not chosen or been able to monetize the content they've scraped is orthogonal to whether they've taken traffic that otherwise would have gone to the creators of that which was scraped.

    Now several of the comments suggest or imply that there are "thousands" of ad-based content businesses making piles of money from their content on the web. That is news to me. I suspect it is also news to the investors in Huffington Post, Daily Beast and others. Gawker claims they are now in the black, but I bet that has more to do with their buyer-guide sites than with the sweet commentary and gossip that they are known for.

    Who are these thousands of companies and where I can I invest in them?


Comments have been disabled for this post