Google Gives Publishers a Way to Close the Free Back Door

The Federal Trade Commission on Tuesday, day one of its two-day workshop, How Will Journalism Survive the Internet Age?, gave a highly visible platform to News Corp. (s nws) Chairman Rupert Murdoch and about a dozen other “old media” bigwigs from which to bash what Murdoch called “the wholesale misappropriation” and “theft” of newspaper content by aggregators and search engines. Today, Google (s goog) announced a new program to let publishers limit the amount of paid content Google News users can access for free.

The timing of the two developments is likely a coincidence (even new media companies like Google can’t respond to events that quickly). But the juxtaposition is striking nonetheless, and illustrates the growing pressures Google faces, from publishers and perhaps now from government regulators, to tread more lightly on the news media’s traditional business model.

Under Google’s new “Five Clicks Free” program, publishers will be able to limit the number of times in a single day that a Google News user can jump the pay wall and click through to a news story for free. Until now, Google News has followed a “first click free” rule, under which users could click through directly to any page Google’s crawlers indexed. Google set it up that way to limit “cloaking,” the sort of bait-and-switch game in which a web publisher lets one page be indexed but redirects users to another when they try to click through. The new program will still allow direct click through but will let publishers set a limit on the number of direct clickthroughs per user, per day, after which the user would encounter a registration roadblock.

Alternatively, Google said it will crawl and index specially created “preview pages,” which would be free to users but might contain no more than a headline and excerpt from the news story. “We think this approach still protects the typical user from cloaking, while allowing publishers to focus on potential subscribers who are accessing a lot of their content on a regular basis,” Google’s senior business product manager Josh Cohen wrote in a blog post.

It also provides a partial answer to Microsoft’s (s msft) recently reported efforts to encourage publishers to “de-list” their content from Google through direct payments or other types of beneficial treatment on Bing.

Whether Google’s attempts at compromise will appease publishers remains to be seen. But if broadly adopted they could accelerate the nascent movement among publishers toward placing more news content behind pay walls. That, in turn, could eventually limit the amount of free content from trusted sources available to the average web surfer.

Certainly, it creates an incentive for news organizations that are currently on the fence about a pay wall, such as the New York Times, to go ahead and take the plunge.

The risk to a publisher in erecting a paywall is that its search-driven web site traffic would plummet, because its content would be invisible to the crawlers. A recent analysis by Hitwise, for instance, estimated that the Wall Street Journal ‘s web site would lose 25 percent of its traffic if it blocked all crawlers. By making paid content discoverable, Google’s new programs would partly alleviate that risk.

On the other hand, Google’s new approach could also benefit business and specialized publishers, whose content is now hidden from crawlers, by allowing them to make their content more discoverable without risking the loss of paying subscribers. That could drive traffic to their sites and bring new opportunities to promote their paid content without having to spend for outbound marketing or advertising. It would also make for richer search results by adding more expert content to the mix.

As Google’s Cohen wrote, “[W]hether you’re offering your content for free or selling it, it’s crucial that people find it.”