61 Comments

Summary:

A chorus of complaints about spammy results in Google searches highlights a growing problem the search company is going to have to solve — and doing so will inevitably bring it into conflict with Demand Media, currently planning a high-profile IPO for later this year.

428065782_ca05c6c402_o

There’s been a chorus of criticism recently about Google’s spam-filled search results, including pointed blog posts from a number of technology and web luminaries, complaining that the company’s links are in many cases virtually unusable, because they are filled with keyword-riddled ad content disguised as helpful tips. Although such complaints are routine for search engines like Google, there is some truth to them — and trying to stamp out that kind of content is likely to mean pain for at least one giant web company. Demand Media, which is currently planning a high-profile IPO for later this year, is often singled out by critics as the kind of content “farm” that generates a lot of those spammy search results.

The most recent cavalcade of complaints about Google started in mid-December 2009, with a rant from Bloomberg business columnist and blogger Paul Kedrosky about the difficulty of finding information about dishwashers, and how the results displayed what he called “the drive-by damage done by keyword-driven content,” and the work of aggregators and content farms whose business model he described as:

Find some popular keywords that lead to traffic and transactions, wrap some anodyne and regularly-changing content around the keywords so Google doesn’t kick you out of search results, and watch the dollars roll in.

This was echoed a week ago by Vivek Wadhwa, an entrepreneur who is now the director of research at the Center for Entrepreneurship and Research Commercialization at Duke University. In a blog post entitled “Why We Desperately Need a New and Better Google,” Wadhwa said the search engine has “become a jungle: a tropical paradise for spammers and marketers. Almost every search takes you to websites that want you to click on links that make them money, or to sponsored sites that make Google money.” (A new Google competitor called Blekko has created a site called the SpamClock to track bogus web content, which it claims is being created at the rate of about 1 million new pages an hour.)

More recently, Instapaper developer Marco Arment joined the fray with a blog post describing how entire categories of searches seem swamped with spammy results — including those, like Kedrosky’s search for dishwashers, that involve specific products. As he put it, massive numbers of content sites are “generated by penny-hungry affiliate marketers and sleazy web ‘content’ startups to target long-tail Google queries en masse, scraping content from others or paying low-wage workers to churn out formulaic, minimally nutritious pages to answer them.”

Searching Google is now like asking a question in a crowded flea market of hungry, desperate, sleazy salesmen who all claim to have the answer to every question you ask.

Both Wadhwa and Arment — as well as others writing about the same issue — point the finger for much of this quasi-spam at “content farms” such as Demand Media (particularly its eHow.com unit) and Associated Content. The former, founded by CEO Richard Rosenblatt, is working on an IPO that is expected to value the company at more than $1.5 billion, while the latter was acquired by Yahoo last year for $100 million. As Wadhwa notes, Associated Content produces more than 10,000 new articles a month about topics such as how to change a tire or a diaper, while Demand Media has more than 8,000 writers who produce roughly 10 times that number every month.

Both companies argue that they produce valuable content that people (and companies) find worthwhile, and that they are not “content farms” trying to rig Google’s algorithms. But the reality is that much of their content is produced quickly, is often of fairly low quality, and is targeted — in Demand’s case, by the company’s own algorithms — to match the keywords that people are likely to search for, because those are the ones that will produce the most advertising revenue. Demand Media has noted in the “risk factors” section of its IPO filing that one of the big risks to the content business is that Google might devalue that content by changing its algorithms to make it show up lower in search.

Until recently, Google might have been able to ignore the kinds of criticisms that Kedrosky and Wadhwa and others have raised, because it was so dominant in both search and advertising. But the web giant can’t afford to do so for much longer when it’s already under competitive pressure from Facebook, which is not only growing rapidly and now has a $50-billion market value, but is also seen to be attracting increasing interest from the advertisers who represent Google’s bread and butter. Search results and user loyalty are about the only weapons that the search company has left — and that could make things unpleasant for Demand Media and its ilk.

Related GigaOM Pro content (sub req’d):

Post and thumbnail photo courtesy of Flickr user The U.S. Army

  1. [...] This post was mentioned on Twitter by GigaOM, Paul Papadimitriou, You Mon Tsang, TechBytZ, Samuel Maina and others. Samuel Maina said: Why Google and Demand Media Are Headed For a Showdown http://bit.ly/hVXlPX #samdemic [...]

    Share
  2. You said of Demand Media that “the reality is that much of their content is produced quickly, is often of fairly low quality…”

    Everyone seems to buy into this low-quality mantra, but no one’s willing to do any homework. Pull up 100 eHow articles at random and read them through, or better yet, have a panel of readers rate them for quality.

    You’ll find some corkers, no doubt. But overall, the Demand Media content strikes me as useful and well-written material. There are no Pulitizers in DM’s future, but that’s hardly the gauge for what is or isn’t quality material on the web.

    Google may indeed have some issues with the type of search results it’s returning, though I haven’t noticed huge problems myself. But whatever Google’s shortcoming may be, Demand Media isn’t part of the problem.

    David

    P.S. And yes, I do write for DM now and then as one of their freelancers.

    Share
    1. Thanks for the comment, David. I have read a lot of Demand articles, although I have not done an exhaustive comparison of them. While I’m sure there are some that are excellent quality, there are quite a lot that are poorly written, confusingly worded and contain little useful information apart from the keywords they require to maintain a high search ranking.

      Share
      1. While you’re checking the date on Kedrosky’s articles, take a look at the dates on any eHow material that strikes you as poor quality.

        Older materials, circa 2008, were posted at eHow with no QC, and a lot of garbage got published. eHow is slowly going through their library and deleting content that isn’t up to par, but you can still find some junk if you try.

        Demand Media’s newer materials are all extensively QC’d and meet pretty rigourous editorial standards. The high-quality materials are rapidly coming to dominate DM’s online content as the older materials get deleted or simply fade from search results over time.

        And as long as I have your attention…happy new year!

        Share
      2. “there are quite a lot that are poorly written, confusingly worded and contain little useful information” That is similar to what I always say about most newspapers and jornals. If they can write low quality material cheaper than “the professionals”, who cares.

        Share
    2. David, you are one in a long line of Demand Media Studios writers who also serve as apologists for Richard Rosenblatt’s empire. I think it’s a self-esteem issue. Some who write for Demand deny the quality is poor, because to do so would be to admit that they write work of inferior quality. Maybe some Demand Media content producers actually do produce higher quality material, but they are an extremely rare exception. The system is designed at its absolute core to churn out low quality material. If you have any doubts about what I say, go look at what Rosenblatt himself has contributed. It’s a joke!

      It is a fact, not an opinion, that the content on eHow.com is inferior quality. I write for eHow. They simply do not get my best quality work. Heck no! They get what they pay for. I give them exactly $15 worth, which I try to average out to about 2 articles per hour. That’s the only sane strategy to take, quite honestly, when you are paid per piece in a system that is very clearly about churning out pages for Google. Let’s be honest about the target audience. The Demand Media system is designed to create inferior quality, from the types of references that are not allowed (starting with competing sites and forums!!) to the quality of the copy editors, who are seriously underpaid and somehow consistently introduce factual errors.

      In my writing, I do a *LOT* of checking other similar eHow.com articles to the titles I am writing. I can truthfully say that they are terrible as a whole. I see flagrant misrepresentation of facts. I see words with no meaningful information. I see generally useless drivel. Mainly technical articles that I look at. To be clear, these are recent articles, not the old system either. Personally, when I need an answer to a vexing technical issue, I will trust (1) wikipedia, (2) Yahoo answers, (3) forums. None of those are allowable sources at Demand.

      Some Demand Media contributors probably do sweat out their articles and turn in high quality material but they are in a very small minority, which is only exacerbated by their likely short tenures and limited output. My priorities are 1) write it, 2) quickly, 3) accurately, 4) do no harm. I’ll be the first to admit that my content suffers because of this. The solution for Demand Media is to pay more. They get the quality they deserve commensurate with the amount they pay. If a simple solution gets my article written and meets my priorities, you’ll be sure that I’m not giving the best solution to a technical issue. My goal is to steer the poor sap who really needs help from eHow on a technical issue in the right direction, so they’re one step closer to a resolution. Many have conjectured that Demand’s goal is to drive the reader to click the links by making them more compelling than the content. Who’s to say? Anyone who argues about the quality of Demand writing is either an apologist with self-esteem issues or doesn’t actually understand what quality writing reads like. I understand the difference, and my client gets what they pay for and nothing more.

      Share
  3. Hey Mathew –

    Didn’t Paul K’s article get published in mid-December of 2009? This article and so many of the others seem to think it was just published a couple weeks ago.

    Share
    1. Wow — thanks for noticing that, Matt. So many other posts I read referred to Kedrosky’s as being recent that when I checked the date to confirm that it was December, I never even noticed that it was a year ago! My bad. I have updated the post to correct the date. Thanks for the heads up.

      Share
  4. It’s worth also discussing how the various domain squatters are using bogus content to generate revenues from search engine advertisers…

    Share
    1. Demand Media is a double-dipper into the sleaze there. Look up the ownership of eNOM.

      Share
  5. Google’s spam problem extends well beyond Demand Media. If that’s all it was it would be easy to deal with. Also Google gets advertising revenues from Demand so it can’t all be bad.

    The showdown with spammers isn’t coming, it already happened and Google lost. At least for now.

    Share
  6. I don’t think that Google will have to change to much to get rid of these content farms. What these content farms have discovered is that it doesn’t take much to rank in Google for oddball terms because there isn’t very much competition. Over the next 5 to 10 years I think a lot of these demand media companies are going to be pushed out by sites like StackOverflow. Q and A sites like StackOverflow and the like capture a lot of long tail keywords, but actually provide much better information and they do this without paying the writers.

    Share
    1. I’m curious…what leads you to think that StackOverflow produces better content than Demand Media?

      Share
      1. eHow has no date on the articles(maybe I’m just stupid and can’t find it). People like to know how fresh the content is and without a date there is really no way of knowing how relevant the content is today.

        eHow doesn’t appear to ever link to the stuff they’re talking in the content. For example, if there is an article about getting a job they will have the words Monster, eLance ect., but it will never be a live link. The only live links in the content of the article are to eHow category pages.

        They might not have useful links in the content of their articles but they do put those annoying double underlined ads that pop over top of the content just by hovering over them.

        Also, the only way users can contribute to an article is through comments. There is no wiki mode and the comments don’t allow for very much formatting, they’re there pretty much so users can say “Wow, Great how to !!!111!” and that’s it.

        Share
      2. I do have to give Demand Media credit for Cracked.com. I used to read this a lot and there was some pretty funny stuff on it.

        Share
    2. Sounds like you don’t like the way eHow is formatted (no dates, links at bottom instead of in article, etc). Those are legitimate beefs, but doesn’t really have much to do with the quality of the articles, which — as I said earlier — seems to keep to pretty high standards.

      Share
  7. Fascinating that the very industry that championed the idea to game SEO is now yelling at Google for it.

    Your last two posts have been really great topics and points.

    Share
    1. Thanks a lot, Patricia — appreciate the comment.

      Share
  8. So true, google search is full of sleazy salesmen who claimed to have the answers…..really pathetic and time wasting.

    It is only a matter of time when the loyal will go in search of a better search engine.

    Share
    1. Its going to take a very big force or take a very long time for the loyal to really move over to another Search Engine. People will get used to/have got used to Content Farms being at the end of what appear to be highly relevant links and just search again. Its simply categorized in their mind as as poor/irrelevant content and they go to the next search result. If its persists its put down to poor search terms by them (not Google serving awkward results) and they will change their Search criteria, not their Search Engine.

      Share
  9. I think the biggest problem here is Google’s lack of willingness to solve this problem.

    Is not only content farms, but bogus classified sites are also wreaking havoc on search results.

    Considering Google’s main product (and source of revenue) it’s still search they should get to work to solve this, specially since both Facebook and Bing are gaining marketshare.

    Share
  10. I’ve noticed that most spam sites tend to run Google Adsense ads. There is a part of me that thinks Google doesn’t do anything about spam because Google probably makes more money off spam than anyone else.

    So long as they have the search and advertising market share, it doesn’t really matter how good the quality of the search results are because people will keep coming back. The only thing that is going to fix this is competition, and right now that is probably only going to come from Facebook.

    Share
    1. “There is a part of me that thinks Google doesn’t do anything about spam because Google probably makes more money off spam than anyone else.”

      Yep, I’ve noticed the same thing and often wondered it. If so, this is a very short-sighted strategy.

      George

      Share
      1. Try to get your hands on a “Google” search evaluation hand book, or what ever it’s called now.
        They were at least when I read it heavily tilted to quantity instead of quality. If you can’t study/talk to them directly, one has to go via proxy.
        They also seem to be tilted to complexity, they are not alone in this. See here Notice the emphasis on brain size and complexity. One problem. Magpies have shown to be self aware, Uni Frankfurt Germany. If I recall it right. Crows use cars at red traffic lights to place Nuts in front of wheels to be cracked at the next green cycle and collected at the following red.
        High probability that brain size and complexity of algorithms have nothing to do with being smart. Bird brains a very different. I don’t think crows run statistical models about the probability of lights turning red,green,red and cars recognizing/ack that :-).

        Share
    2. That’s a good point, Gary — I didn’t get into it in this post, but as a number of people have pointed out, Google makes money from the AdSense on many of the pages that show up high in its results, including those from Demand Media and similar services, so there is what appears to be a significant disincentive to go after those sites. Hopefully Google will see that there is more to be gained by doing so, at least in the long term.

      Share
  11. Didn’t you forget about the content scrapers? If I would know that the content originated at a certain point I would look around more, since I found something interesting to begin with. If one looks around on a content scraped site its just noise.

    So what Google needs is a Context driven system. But unlike many Journalist who think Google spreads itself to thin with autonomous cars I’m not sure. I studied vision too, for my foray into a Context Orientated language. I mean any vision system organizes more data into context with a ?Mhz system than most data centers can do with TFlops systems. Why? Certainly there must be something better than OO for that kind of modeling, which [not] surprisingly has got some attention from other companies working in that field.

    Either Google is up to something, or they are just a big company not knowing how to use [their] research to improve their own main product. In other words I find it curious that they research autonomous cars. If they build a context driven evaluation system the content farm problem solution will be a fallout. Another fallout is identifying the originator of a given text, given some previous/other texts.

    But on the other hand they are not comparing ideas with other people, there are only very few people working on context without statistics (see above) for modeling as an example. So either Google will surprise us or they will be in for a surprise. Based on how long it took to get here it’s most likely the later, but who knows.

    Share
  12. This is exactly why I’m counting on Blekko to succeed. Consumers deserve better.

    http://thenowledge.com/2010/11/02/blekko-scores-one-for-human-curation/

    Share
  13. Mathew, I expect that old mother “necessity” will be fostering invention soon. People are now recognizing what Paul Kedrosky concluded 13 months ago – that there is a fundamental conflict for the ad revenue-based search engines and the choice of volume or quality of content being served.

    The question for me is not if, but how the market will be address the situation. Will we see the successful introduction and adoption of new search engines, 3rd party extensions for existing search engines or significant changes to the ranking methodologies by the existing players?

    Share
  14. [...] This post was mentioned on Twitter by Ben Israel. Ben Israel said: RT @alansoon: Why Google and Demand Media are headed for a showdown. Good post on the need for curated search results http://t.co/AnzB7u6 [...]

    Share
  15. While I don’t know how technically feasible it is right now, a quick fix that Google could do would be to allow the user to remove certain domains from search results. That is, if you are logged in with your Google ID, the search results page could display an ‘x’ next to each result that would add the domain to your personal blacklist when clicked.

    Granted, this wouldn’t do much against the one-off spam domains, but it would allow people to remove sites like eHow if they so desire.

    Share
    1. That’s a good point, Jay — as I recall, Google experimented with that kind of user input through what it called the “search wiki” feature, but that was shelved a while back. I’m not sure why the company didn’t go ahead with it.

      Share
  16. It’s nice to see the tech media finally stating the obvious after watching Google’s search quality results go down year after year.

    Share
    1. Of course, as you note, this has been building for awhile and some tech writers have been discussing it all along. I just appreciate the critical mass that’s currently building.

      Share
  17. I find it interesting that people pound eHow. The site is about simple ways to solve problems or issues. To give an example I needed to clean my laundry machine and found 2 extremely useful ways to do so on eHow. The same for cleaning my microwave and clipping my dogs nails.

    To listen to people freak out over this type of content amazes me. eHow doesn’t write news articles.

    Share
  18. Not sure who is running Blekko’s pr campaign but whoever it is has been doing a great job. An article on Gigaom and Techcrunch in a couple of weeks. Damn you are awesome!

    Share
  19. The article was interesting … but I was hoping you had an alternative search engine to suggest … I agree that Google has become tedious to use … wading through all those irrelevant pages trying to find answers to a question or information about a product is no longer a simple matter.

    Share
  20. Google has work to do, but they can and will beat this problem. Here’s one idea how:

    “Beating the Content Farms: Google Can Automate the Like Button”

    http://designpete.com/2011/01/09/beating-the-content-farms-google-can-automate-the-like-button/

    Share
  21. I still think if you get “spam” in Google search results you don’t know how to use Google. Plenty of operator searches to find exactly what you want. Most of the linked articles just show singular examples too – I’d like to see some data behind Google actually delivering a bad user experience. Seems pretty good to me :)

    Share
  22. How quickly was this article written? Is this article of low quality or high? I personally think that gigaom.com articles are of low quality and gigaom.com should be either removed from Google index or at least pushed back to 100th+ pages.

    How did you like this statement? I did not think so. Then who are you to say that other websites should be removed just because they have in your opinion low quality articles?

    Share
    1. Clearly unbiased.

      Share
  23. The ‘quality’ argument is flawed and completely misses the mark. The fact is that Google is so good at search that we are all almost ignorant of the fact that they get better every year! Try to think back to the kind of queries you had to enter five years ago…and the type of results you’d get…Sure its easy to find spam – but you really have to go looking for it. Their batting average is sooo much better than it used to be, and still much, much better than most any alternative out there. And screw ‘semantic search’ – we don’t need it, and I’d posit that Google is getting pretty close – or close enough for 90% of complex queries.
    The whiners about Demand media are typically ‘traditional’ media pundits, who think that Google should somehow acknowledge the education and work experience of every writer for every piece of content – as opposed to analyzing whether or not the question asked is answered accurately.

    Share
  24. Vivek Wadhwa wrote an awesome article totally keyword targeted on Blekko. Hello Kettle, it’s the pot calling – you’re black.

    And Adam Singer, you’re spot on.

    Share
  25. To the commenter that said eHow does not give dates on articles, that is just plain wrong. The date is given right under the title. Here’s an example page.

    http://www.ehow.com/facts_7741249_plant-gladiolus-corms.html

    eHow also does not put in the double underlined link ads as the same poster claimed. Clearly this guy is just bashing.

    As for links embedded in articles, resources and references are linked at the bottom of the page. All articles undergo editing (not always perfect) and have specific requirements to pass before publishing. One thing that is not done, the articles are never stuffed with keywords. We are not given keywords to write, just a title to write to.

    Know your facts before you open your mouth and at least look at the articles before you blast them.

    MJ Logan

    Share
  26. They will complain if this causes fewer people to trust their results, or more importantly to not trust Google and therefore stop using ad sense.

    Share
  27. [...] Ingram does a good overview while claiming Google is on a collision course with Demand Media.  He [...]

    Share
  28. [...] Ingram does a good overview while claiming Google is on a collision course with Demand Media.  He [...]

    Share
  29. [...] Why Google and Demand Media Are Headed For a Showdown, GigaOM [...]

    Share
  30. I’m also a writer for Demand Media, primarily eHow.com, and I have to agree with Zziggy. I rarely give Demand my best work, and often try to find the easiest way to cut corners. I don’t try to omit necessary information or come up with anything patently false, but I’m not going to give my A game for a whopping $15 a piece.

    The quality of Demand’s output material is directly linked to how much it pays both it’s writers and it’s copy-editors. The old expression “you get what you pay for” applies in spades with Demand, and with the freelance staff so grievously underpaid (something to the tune of 100 times less than the regular going rate for similar sized articles, I’ve heard), what do they expect?

    The simple fact is that Demand really is about advertising, and the content given on each article is little more than a vague justification of Google ad dollars, so that they don’t face penalties from Google. If the writers were given decent wages, and stifling (and often baffling) rules were loosened up by management, the content could easily see an upswing in quality.

    But of course, this would require Demand to actually shell out more than a pittance to it’s writers, which they will never do. Our value does not come from our ability to create, our writing skills or even our ability to hunt down a couple of reference links. Our value comes from being a cheap source of low-grade information that’s just enough to pass Google’s rules. If Demand was interested in producing truly high quality informative work, do you really think they’d pay a bunch of us low-rent freelancers en masse for pennies on the dollar?

    We’re drones, and while Demand Zombies will tell you it’s amazing, I can tell you first hand it’s not. It’s sweatshop writing at it’s finest, bought and sold for pennies and now trying to gain a 1.5 billion dollar IPO that NONE of it’s freelance staff will benefit from.

    After all, we’re just “independent contractors.” We don’t get any of the pie we’re churning out endless piles of crap to pay for.

    TL;DR: Demand sucks because meaningful content is it’s LAST priority. The freelance staff are next in line. It’s about ad dollars. The end.

    /rant

    Share
  31. The shares you buy at this IPO won’t be worth the toilet paper they are printed on, after the owners of DM cash out – all $58 million worth. Forget the quality of their material, its the value of the company that won;t be worth much. Before you saw them, after you won’t.

    Share
  32. [...] Why Google and Demand Media Are Headed For a Showdown (gigaom.com) [...]

    Share
  33. [...] gehaltvoller Suchergebnisse (Kritik an der sich verschlechternden Qualität der Google-Resultate hört man in diesem Kontext häufiger). Je mehr Firmen dem Konzept von Demand Media folgen (in Deutschland z.B. contentfleet), desto eher [...]

    Share
  34. [...] are also getting more vocal about the rapid deterioration of Google’s search results, its core business, thanks in part to the contributions of “content farms” such as [...]

    Share
  35. [...] we discussed recently, the criticisms about Google’s results — from tech analysts such as Bloomberg blogger [...]

    Share
  36. [...] some observations from a recent article on GigaOm that raised a number of interesting points, of which I’ve highlighted just a couple I [...]

    Share
  37. [...] few before it from notable people like Jeff Atwood), including New York Magazine, Business Insider, GigaOm, TechCrunch, CNN, and The Wall Street [...]

    Share
  38. [...] few before it from notable people like Jeff Atwood), including New York Magazine, Business Insider, GigaOm, TechCrunch, CNN, and The Wall Street [...]

    Share
  39. [...] A big part of that effort is generating massive amounts of content and then trying to monetize it via Google AdWords and other search-based advertising — at least according to a leaked strategy document. It’s not clear when the document was written, but it describes how the company is banking on increasing the number of stories it runs per month by more than 60 percent over the next four months, to 55,000, and hoping to increase the number of pageviews per story by more than 400 percent to 7,000. Writers at the company are expected to write between 5 and 10 stories per day, and story selection is to be driven by analyzing search traffic to see what kind of content people want to read about — exactly the same strategy taken by Demand Media. [...]

    Share
  40. [...] value of more than $1.5 billion. Much of that attention has been less than favorable, however, focusing on Demand’s status as a “content farm” that produces huge volumes of disposable, low-quality articles that are designed primarily to show [...]

    Share
  41. [...] When I saw Dan on the Mac version I did the requisite Google search for an answer and got the usual spam-filled and out-of-date search [...]

    Share
  42. [...] is concluding that if people are so up in arms about its perceived declining search results, then it will let the masses get to work in helping refine its search technology. The company [...]

    Share
  43. [...] 3.5 or higher). The new feature only applies more pressure on so-called content farms, which are increasingly being targeted by Google, Blekko and [...]

    Share

Comments have been disabled for this post