52 Comments

Summary:

Google has responded to criticism about the declining quality of its results by saying it plans to come down hard on so-called “content farms” that try to game its algorithm with low-quality pages. That could mean pain for Demand Media, which is planning a closely-watched IPO.

1660643334_7da894dfaf_z (1)

Over the past few months, there has been a growing chorus of criticism — much of it anecdotal, but coming from a number of respected technology observers — about Google’s increasingly useless search results. Now the web giant has responded to these criticisms in a blog post, saying it doesn’t believe its search is getting any worse (if anything, it claims results have improved), but noting that it is going to be coming down hard on so-called “content farms” that try to game its algorithm with low-quality pages filled with keywords. And that could mean some pain for Demand Media, which is planning a closely-watched initial public offering that could launch as soon as next week.

The Google blog post came from Matt Cutts, who heads up the company’s search-spam team. Cutts said that in contrast to some of the criticisms, “according to the evaluation metrics that we’ve refined over more than a decade, Google’s search quality is better than it has ever been in terms of relevance, freshness and comprehensiveness.” He also said that English-language spam in the web company’s results is less than half what it was five years ago, although he admitted there had been a “slight uptick of spam in recent months.” But while pure spam has gotten better (the Google staffer provided this example of what the term means to Google) Cutts admitted that there was more work to be done in the area of low-quality pages from “content farms.”

As we discussed recently, the criticisms about Google’s results — from tech analysts such as financial blogger Paul Kedrosky, entrepreneur Vivek Wadhwa, programmer Jeff Atwood and Instapaper developer Marco Arment — almost all focus their hate on content from mass-production outfits such as eHow, which is a subsidiary of Demand Media. Kedrosky wrote about companies whose business model he described as: “find some popular keywords that lead to traffic and transactions, wrap some anodyne and regularly-changing content around the keywords so Google doesn’t kick you out of search results, and watch the dollars roll in.” This is almost a perfect description of what Demand Media and other similar companies do.

In a securities filing related to its IPO, Demand — which is run by former MySpace chairman Richard Rosenblatt — says that “while traditional media companies create content based on anticipated consumer interest, we create content that responds to actual consumer demand.” What this means in practice is that Demand produces text, images and video that are designed to attract keyword ads. The company looks at what keywords are being searched for most, and pays contractors to produce content that fits that description. So if Demand sees that keywords related to winter tires are fetching a high price on auction markets like Google’s, it will pay someone to write an article about how to put snow tires on your car.

Some of these articles are filled with useful information, but others are closer to what Marco Arment described when he railed against content “generated by penny-hungry affiliate marketers and sleazy web ‘content’ startups to target long-tail Google queries en masse, scraping content from others or paying low-wage workers to churn out formulaic, minimally nutritious pages to answer them.” Demand isn’t the only “content farm,” of course. Associated Content, which was bought by Yahoo for $100 million last year, also produces tens of thousands of articles a month on similar kinds of topics, and there are several smaller sites such as Suite101.com that take a similar approach.

Demand Media is by far the largest player, however, and that means it could bear the brunt of Google’s wrath. Demand mentioned this in the “risk factors” section of its S-1 filing, referring to “the possibility that our relationship with Google — from which a significant portion of our revenue is generated — may be terminated or renewed on less favorable terms,” and “the “current dependence of our Content & Media service offering on the success of eHow.” The filing noted that Google could easily “change its existing… methodologies and metrics for valuing the quality of Internet traffic and delivering cost-per-click advertisements” in a way that would negatively affect Demand’s business.

In his Google blog post, Matt Cutts made a point of refuting the allegation made by some critics that the search giant takes a less-than-rigorous approach to pages from content farms like Demand because they carry Google advertising, and therefore the company shares in some of that revenue. This is not the case, Cutts said:

One misconception that we’ve seen in the last few weeks is the idea that Google doesn’t take as strong action on spammy content in our index if those sites are serving Google ads. To be crystal clear:

  • Google absolutely takes action on sites that violate our quality guidelines regardless of whether they have ads powered by Google
  • Displaying Google ads does not help a site’s rankings in Google, and
  • Buying Google ads does not increase a site’s rankings in Google’s search results.

That seems to make it clear that Demand Media can’t expect any favors just because it carries a lot of Google ads (which Demand says accounted for 28 percent of its revenue in the nine months through September 2010). At the very least, Google’s saber-rattling on the content-farm issue could make investors a little more nervous about Demand’s stock offering — and put more pressure on the company to boost the quality of some of its content so that it doesn’t get caught in Google’s spam filters.

Related GigaOM Pro content (sub req’d):

Post and thumbnail courtesy of Flickr user Atli Haroarson

  1. I sure as hell hope so.

    That comment was surely a shot across the bow, and I’m sure Google and Demand will be in close talks if DemandMedia knows what’s good for it.

    Demand Media’s existence relies on Google’s good graces.

    Share
  2. It had to happen sooner or later. Google and Demand had a sort of Faustian bargain because Demand was filling a short-term need of helping to monetize YouTube. Ultimately though bad content will undermine Google’s main breadwinner and therefore it’s very existence.

    Demand was always one algorithm change away from oblivion. Their entire business model relied on the Google looking the other way as it violated — in spirit if not in letter — Google’s TOS and the ethos on which it built its business. Not much of a business model when you think about it.

    Share
  3. Thanks for digging up the link to Google’s spam def. But that doesn’t look like spam to me, that looks like text nonsense.
    Hence the disconnect between Google and their users. Our definition of spam and theirs seems not be aligned. Maybe we have evolved faster than Google on what we expect. Or what we call Context, Information, News, while Google is still fighting yesterdays battle(text nonsense).

    Share
    1. That’s a fair point, Ronald — I think the definition of spam for many people has evolved to mean “useless crap,” rather than the more technical definition that Google still uses.

      Share
  4. Invest at your own risk!

    Share
  5. Another problem is detecting the original, or canonical source of indexed content. Increasingly, a lot of sites scrap others’ and then rank better than the original source — even though Google is indexing the same material from both sites.

    Share
  6. If I were Google, I would pull the cord on all content farms. This would cause them all to spam the hell out of the competition to try to make up for lost revenue – making Google’s search competitors product even worse and Google’s product even better.

    It’s a no brainer.

    Share
  7. 7 letters to define spam GROUPON

    Share
  8. Mahalo is right up there as well. Possibly the #1 farm.

    Jason says Mooooo

    Share
  9. It’s interesting to ponder Demand Media’s response to Google’s ongoing and imminent algorithmic solutions to Web spam.

    We’ve all see examples of disappointing Demand Media content, especially on websites like eHow.

    However, not all of it is awful, and some of it serves particular niches very well. Google knows this, understands it, and I’m certain their algorithmic solutions (with advanced LDA) will weight accordingly.

    The interesting aspect arises when Demand Media’s internal KPIs detect index removal or significant down rankings of certain content.

    Demand’s vast indexed library of content affords them the luxury of quickly detecting patterns in the degraded or de-indexed content.

    Will they flag those articles for rewrites? Remove them altogether? Redevelop their internal writing guides and practices to circumvent algorithmic changes?

    It will be fascinating to see how this story plays out over the next year.

    Share
    1. I agree — those are some good questions. What kind of response will Demand come up with? It will be interesting to see.

      Share
  10. I see Demand receiving all the heat but there are other very prominent and low quality content farms that are running disguised as something else like Mahalo, but nobody seems to be saying anything about it.

    Share

Comments have been disabled for this post