61 Comments

Summary:

A chorus of complaints about spammy results in Google searches highlights a growing problem the search company is going to have to solve — and doing so will inevitably bring it into conflict with Demand Media, currently planning a high-profile IPO for later this year.

428065782_ca05c6c402_o

There’s been a chorus of criticism recently about Google’s spam-filled search results, including pointed blog posts from a number of technology and web luminaries, complaining that the company’s links are in many cases virtually unusable, because they are filled with keyword-riddled ad content disguised as helpful tips. Although such complaints are routine for search engines like Google, there is some truth to them — and trying to stamp out that kind of content is likely to mean pain for at least one giant web company. Demand Media, which is currently planning a high-profile IPO for later this year, is often singled out by critics as the kind of content “farm” that generates a lot of those spammy search results.

The most recent cavalcade of complaints about Google started in mid-December 2009, with a rant from Bloomberg business columnist and blogger Paul Kedrosky about the difficulty of finding information about dishwashers, and how the results displayed what he called “the drive-by damage done by keyword-driven content,” and the work of aggregators and content farms whose business model he described as:

Find some popular keywords that lead to traffic and transactions, wrap some anodyne and regularly-changing content around the keywords so Google doesn’t kick you out of search results, and watch the dollars roll in.

This was echoed a week ago by Vivek Wadhwa, an entrepreneur who is now the director of research at the Center for Entrepreneurship and Research Commercialization at Duke University. In a blog post entitled “Why We Desperately Need a New and Better Google,” Wadhwa said the search engine has “become a jungle: a tropical paradise for spammers and marketers. Almost every search takes you to websites that want you to click on links that make them money, or to sponsored sites that make Google money.” (A new Google competitor called Blekko has created a site called the SpamClock to track bogus web content, which it claims is being created at the rate of about 1 million new pages an hour.)

More recently, Instapaper developer Marco Arment joined the fray with a blog post describing how entire categories of searches seem swamped with spammy results — including those, like Kedrosky’s search for dishwashers, that involve specific products. As he put it, massive numbers of content sites are “generated by penny-hungry affiliate marketers and sleazy web ‘content’ startups to target long-tail Google queries en masse, scraping content from others or paying low-wage workers to churn out formulaic, minimally nutritious pages to answer them.”

Searching Google is now like asking a question in a crowded flea market of hungry, desperate, sleazy salesmen who all claim to have the answer to every question you ask.

Both Wadhwa and Arment — as well as others writing about the same issue — point the finger for much of this quasi-spam at “content farms” such as Demand Media (particularly its eHow.com unit) and Associated Content. The former, founded by CEO Richard Rosenblatt, is working on an IPO that is expected to value the company at more than $1.5 billion, while the latter was acquired by Yahoo last year for $100 million. As Wadhwa notes, Associated Content produces more than 10,000 new articles a month about topics such as how to change a tire or a diaper, while Demand Media has more than 8,000 writers who produce roughly 10 times that number every month.

Both companies argue that they produce valuable content that people (and companies) find worthwhile, and that they are not “content farms” trying to rig Google’s algorithms. But the reality is that much of their content is produced quickly, is often of fairly low quality, and is targeted — in Demand’s case, by the company’s own algorithms — to match the keywords that people are likely to search for, because those are the ones that will produce the most advertising revenue. Demand Media has noted in the “risk factors” section of its IPO filing that one of the big risks to the content business is that Google might devalue that content by changing its algorithms to make it show up lower in search.

Until recently, Google might have been able to ignore the kinds of criticisms that Kedrosky and Wadhwa and others have raised, because it was so dominant in both search and advertising. But the web giant can’t afford to do so for much longer when it’s already under competitive pressure from Facebook, which is not only growing rapidly and now has a $50-billion market value, but is also seen to be attracting increasing interest from the advertisers who represent Google’s bread and butter. Search results and user loyalty are about the only weapons that the search company has left — and that could make things unpleasant for Demand Media and its ilk.

Related GigaOM Pro content (sub req’d):

Post and thumbnail photo courtesy of Flickr user The U.S. Army

  1. [...] This post was mentioned on Twitter by GigaOM, Paul Papadimitriou, You Mon Tsang, TechBytZ, Samuel Maina and others. Samuel Maina said: Why Google and Demand Media Are Headed For a Showdown http://bit.ly/hVXlPX #samdemic [...]

    Share
  2. You said of Demand Media that “the reality is that much of their content is produced quickly, is often of fairly low quality…”

    Everyone seems to buy into this low-quality mantra, but no one’s willing to do any homework. Pull up 100 eHow articles at random and read them through, or better yet, have a panel of readers rate them for quality.

    You’ll find some corkers, no doubt. But overall, the Demand Media content strikes me as useful and well-written material. There are no Pulitizers in DM’s future, but that’s hardly the gauge for what is or isn’t quality material on the web.

    Google may indeed have some issues with the type of search results it’s returning, though I haven’t noticed huge problems myself. But whatever Google’s shortcoming may be, Demand Media isn’t part of the problem.

    David

    P.S. And yes, I do write for DM now and then as one of their freelancers.

    Share
    1. Thanks for the comment, David. I have read a lot of Demand articles, although I have not done an exhaustive comparison of them. While I’m sure there are some that are excellent quality, there are quite a lot that are poorly written, confusingly worded and contain little useful information apart from the keywords they require to maintain a high search ranking.

      Share
      1. While you’re checking the date on Kedrosky’s articles, take a look at the dates on any eHow material that strikes you as poor quality.

        Older materials, circa 2008, were posted at eHow with no QC, and a lot of garbage got published. eHow is slowly going through their library and deleting content that isn’t up to par, but you can still find some junk if you try.

        Demand Media’s newer materials are all extensively QC’d and meet pretty rigourous editorial standards. The high-quality materials are rapidly coming to dominate DM’s online content as the older materials get deleted or simply fade from search results over time.

        And as long as I have your attention…happy new year!

        Share
      2. “there are quite a lot that are poorly written, confusingly worded and contain little useful information” That is similar to what I always say about most newspapers and jornals. If they can write low quality material cheaper than “the professionals”, who cares.

        Share
    2. David, you are one in a long line of Demand Media Studios writers who also serve as apologists for Richard Rosenblatt’s empire. I think it’s a self-esteem issue. Some who write for Demand deny the quality is poor, because to do so would be to admit that they write work of inferior quality. Maybe some Demand Media content producers actually do produce higher quality material, but they are an extremely rare exception. The system is designed at its absolute core to churn out low quality material. If you have any doubts about what I say, go look at what Rosenblatt himself has contributed. It’s a joke!

      It is a fact, not an opinion, that the content on eHow.com is inferior quality. I write for eHow. They simply do not get my best quality work. Heck no! They get what they pay for. I give them exactly $15 worth, which I try to average out to about 2 articles per hour. That’s the only sane strategy to take, quite honestly, when you are paid per piece in a system that is very clearly about churning out pages for Google. Let’s be honest about the target audience. The Demand Media system is designed to create inferior quality, from the types of references that are not allowed (starting with competing sites and forums!!) to the quality of the copy editors, who are seriously underpaid and somehow consistently introduce factual errors.

      In my writing, I do a *LOT* of checking other similar eHow.com articles to the titles I am writing. I can truthfully say that they are terrible as a whole. I see flagrant misrepresentation of facts. I see words with no meaningful information. I see generally useless drivel. Mainly technical articles that I look at. To be clear, these are recent articles, not the old system either. Personally, when I need an answer to a vexing technical issue, I will trust (1) wikipedia, (2) Yahoo answers, (3) forums. None of those are allowable sources at Demand.

      Some Demand Media contributors probably do sweat out their articles and turn in high quality material but they are in a very small minority, which is only exacerbated by their likely short tenures and limited output. My priorities are 1) write it, 2) quickly, 3) accurately, 4) do no harm. I’ll be the first to admit that my content suffers because of this. The solution for Demand Media is to pay more. They get the quality they deserve commensurate with the amount they pay. If a simple solution gets my article written and meets my priorities, you’ll be sure that I’m not giving the best solution to a technical issue. My goal is to steer the poor sap who really needs help from eHow on a technical issue in the right direction, so they’re one step closer to a resolution. Many have conjectured that Demand’s goal is to drive the reader to click the links by making them more compelling than the content. Who’s to say? Anyone who argues about the quality of Demand writing is either an apologist with self-esteem issues or doesn’t actually understand what quality writing reads like. I understand the difference, and my client gets what they pay for and nothing more.

      Share
  3. Hey Mathew –

    Didn’t Paul K’s article get published in mid-December of 2009? This article and so many of the others seem to think it was just published a couple weeks ago.

    Share
    1. Wow — thanks for noticing that, Matt. So many other posts I read referred to Kedrosky’s as being recent that when I checked the date to confirm that it was December, I never even noticed that it was a year ago! My bad. I have updated the post to correct the date. Thanks for the heads up.

      Share
  4. It’s worth also discussing how the various domain squatters are using bogus content to generate revenues from search engine advertisers…

    Share
    1. Demand Media is a double-dipper into the sleaze there. Look up the ownership of eNOM.

      Share
  5. Google’s spam problem extends well beyond Demand Media. If that’s all it was it would be easy to deal with. Also Google gets advertising revenues from Demand so it can’t all be bad.

    The showdown with spammers isn’t coming, it already happened and Google lost. At least for now.

    Share
  6. I don’t think that Google will have to change to much to get rid of these content farms. What these content farms have discovered is that it doesn’t take much to rank in Google for oddball terms because there isn’t very much competition. Over the next 5 to 10 years I think a lot of these demand media companies are going to be pushed out by sites like StackOverflow. Q and A sites like StackOverflow and the like capture a lot of long tail keywords, but actually provide much better information and they do this without paying the writers.

    Share
    1. I’m curious…what leads you to think that StackOverflow produces better content than Demand Media?

      Share
      1. eHow has no date on the articles(maybe I’m just stupid and can’t find it). People like to know how fresh the content is and without a date there is really no way of knowing how relevant the content is today.

        eHow doesn’t appear to ever link to the stuff they’re talking in the content. For example, if there is an article about getting a job they will have the words Monster, eLance ect., but it will never be a live link. The only live links in the content of the article are to eHow category pages.

        They might not have useful links in the content of their articles but they do put those annoying double underlined ads that pop over top of the content just by hovering over them.

        Also, the only way users can contribute to an article is through comments. There is no wiki mode and the comments don’t allow for very much formatting, they’re there pretty much so users can say “Wow, Great how to !!!111!” and that’s it.

        Share
      2. I do have to give Demand Media credit for Cracked.com. I used to read this a lot and there was some pretty funny stuff on it.

        Share
    2. Sounds like you don’t like the way eHow is formatted (no dates, links at bottom instead of in article, etc). Those are legitimate beefs, but doesn’t really have much to do with the quality of the articles, which — as I said earlier — seems to keep to pretty high standards.

      Share
  7. Fascinating that the very industry that championed the idea to game SEO is now yelling at Google for it.

    Your last two posts have been really great topics and points.

    Share
    1. Thanks a lot, Patricia — appreciate the comment.

      Share
  8. So true, google search is full of sleazy salesmen who claimed to have the answers…..really pathetic and time wasting.

    It is only a matter of time when the loyal will go in search of a better search engine.

    Share
    1. Its going to take a very big force or take a very long time for the loyal to really move over to another Search Engine. People will get used to/have got used to Content Farms being at the end of what appear to be highly relevant links and just search again. Its simply categorized in their mind as as poor/irrelevant content and they go to the next search result. If its persists its put down to poor search terms by them (not Google serving awkward results) and they will change their Search criteria, not their Search Engine.

      Share
  9. I think the biggest problem here is Google’s lack of willingness to solve this problem.

    Is not only content farms, but bogus classified sites are also wreaking havoc on search results.

    Considering Google’s main product (and source of revenue) it’s still search they should get to work to solve this, specially since both Facebook and Bing are gaining marketshare.

    Share
  10. I’ve noticed that most spam sites tend to run Google Adsense ads. There is a part of me that thinks Google doesn’t do anything about spam because Google probably makes more money off spam than anyone else.

    So long as they have the search and advertising market share, it doesn’t really matter how good the quality of the search results are because people will keep coming back. The only thing that is going to fix this is competition, and right now that is probably only going to come from Facebook.

    Share
    1. “There is a part of me that thinks Google doesn’t do anything about spam because Google probably makes more money off spam than anyone else.”

      Yep, I’ve noticed the same thing and often wondered it. If so, this is a very short-sighted strategy.

      George

      Share
      1. Try to get your hands on a “Google” search evaluation hand book, or what ever it’s called now.
        They were at least when I read it heavily tilted to quantity instead of quality. If you can’t study/talk to them directly, one has to go via proxy.
        They also seem to be tilted to complexity, they are not alone in this. See here Notice the emphasis on brain size and complexity. One problem. Magpies have shown to be self aware, Uni Frankfurt Germany. If I recall it right. Crows use cars at red traffic lights to place Nuts in front of wheels to be cracked at the next green cycle and collected at the following red.
        High probability that brain size and complexity of algorithms have nothing to do with being smart. Bird brains a very different. I don’t think crows run statistical models about the probability of lights turning red,green,red and cars recognizing/ack that :-).

        Share
    2. That’s a good point, Gary — I didn’t get into it in this post, but as a number of people have pointed out, Google makes money from the AdSense on many of the pages that show up high in its results, including those from Demand Media and similar services, so there is what appears to be a significant disincentive to go after those sites. Hopefully Google will see that there is more to be gained by doing so, at least in the long term.

      Share

Comments have been disabled for this post