9 Comments

Summary:

Metafilter, a pioneering online community, has been forced to lay off almost half of its moderation staff and is on financial life support after its traffic from Google suddenly declined by 40 percent. The reasons for the drop — as with most things involving Google — remain a mystery

For long-time web users, the announcement this week from Metafilter — a pioneering online community that has been around since 1999 — was a little like hearing that an old friend is in hospital with a fatal illness: founder Matt Haughey said that due to a dramatic decline in traffic and related advertising revenue over the past year and a half, the site has had to lay off several of its moderators and is essentially on financial life-support. And the name at the center of this drama likely won’t come as much of a surprise: Google.

In a post he wrote on Medium (and in a somewhat shorter update on Metafilter itself), Haughey describes how traffic to the site suddenly cratered in mid-2012, with visits falling by more than 40 percent. At around the same time, Google was rolling out an update to its indexing algorithm — an update known as Panda — which was designed to promote high-quality content and down-grade spam sites, as well as those using a variety of search-engine optimization or SEO tricks.

Not surprisingly, this update hit some “content farm” producers such as Demand Media fairly hard. But Metafilter is the polar opposite of that kind of site: its discussion threads — especially the ones on its Q & A forums at Ask Metafilter — are almost always excellent, with thoughtful responses and high-level conversation. Until recently, the site employed eight full-time moderators to keep the site clean, far more than many much larger sites use.

“The money situation changed one day in November 2012, when I saw a drastic reduction in traffic and revenue to Ask MetaFilter. I double-checked to make sure the initial estimates were correct, and it appeared that Ask MetaFilter lost 40% of its traffic overnight.”

The cause of the traffic decline is a mystery

As with most things involving Google and its algorithms, what happened to Metafilter is almost impossible to diagnose, because the search giant’s methods — and the motivation for any changes — are a black box. Haughey theorizes that the change occurred around the Panda update and was exacerbated by subsequent updates, but Search Engine Land founder and Google expert Danny Sullivan pointed out that the dramatic drop-off in traffic doesn’t really line up with any of the company’s major algorithm tweaks.

Metafilter traffic

If you’re interested, Sullivan has since written a much longer description of what Google does to sites via its various algorithm updates and “filters” like Panda or Top Heavy, and he agrees that Google could do a much better job of explaining to sites like Metafilter what they might have done and why, and how they might be able to alter their ranking or indexing to make it better.

Although it’s difficult to quantify how much Metafilter has fallen in Google’s search results, David Auerbach at Slate magazine did an informal survey of some results that should arguably have included pages from the site, given the amount of content and links related to that topic or search phrase, and found that in a significant number of cases Metafilter ranked a lot lower than it did on Bing: In one case the result was #2 on Bing and #60 on Google.

Google’s methods remain a black box

At one point, Haughey says that he tried to get Google to talk about what the site was doing wrong, or suggest steps that he could take to ensure that Metafilter’s content was ranked higher, but all he got was suggestions from AdSense executives that he should buy more advertising — although Sullivan notes that Google’s top spam cop, Matt Cutts, said on Twitter Wednesday that he has discussed the issue with Haughey at some length.

Whether Cutts is able to help or not, Metafilter’s experience sums up one of the main concerns that many publishers have about Google’s dominance in both search and search-related advertising. If having high-quality content like Metafilter isn’t enough to satisfy the search giant, then what hope do other sites or publishers have of catering to the company’s wishes?

Google HQ

It’s worth noting that Haughey points out a few other macro factors that could be affecting his site’s performance, including the decline of advertising itself: In other words, the theory that the web has reached “Peak Ads,” and traditional web strategies — including Google’s AdSense and other methods — are just not as effective as they used to be, and likely never will be again. As Haughey notes in his Medium post, this state of affairs is arguably exacerbated by the shift to mobile, where traditional web advertising also doesn’t work as well.

Shouldn’t quality matter?

One factor Haughey doesn’t really dwell on is that many online communities have a natural life-span, and it’s not uncommon for them to gradually decline in popularity — it happened to Slashdot (whose founder Rob “Cmdr Taco” Malda now works for a content-recommendation startup called Trove) and to Digg, and to plenty of other pioneers as well. That said, however, a gradual decline in popularity doesn’t really jibe with the sudden 40-percent drop Metafilter saw.

In the grand scheme of things, the fact that a small online community run by a handful of people is in financial trouble isn’t an earth-shattering development — Haughey has always run Metafilter as a kind of “lifestyle business,” without any of the usual Silicon Valley pretensions of grandeur or world-changing billion-dollar IPOs, and so even if Metafilter ceased to exist it probably wouldn’t be noticed outside a small group of old-time web users.

But at the same time, when a site that is so obviously concerned with quality — and so clearly driven by the pursuit of thoughtful content and interaction — gets penalized to such an extent, it makes you wonder what else is happening inside the confines of Google’s black box. Unfortunately, we will probably never know.

Post and thumbnail images courtesy of Flickr user Affiliate and Thinkstock / BrianAJackson

You’re subscribed! If you like, you can update your settings

  1. I don’t buy the argument that this is a natural decline of an admittedly very old community, or that MeFi’s retro/minimalist design is somehow to blame. According to graphs of traffic and revenue, 2012 was the most successful year on record, and as a longtime user I can vouch that posting and comment volumes remained as high as ever throughout 2013 and 2014. MetaFilter was doing just fine for itself even in the age of Twitter, Facebook, and Reddit.

    It needs to be emphasized: this is entirely a Google-created problem. Traffic to AskMetafilter dropped 40% *overnight*. Literally. And in the process of diagnosing it, Matt Haughey caught plenty of hints that Google now considered his highly-regarded community a malicious linkfarm, even telling third-party webmasters to ask Matt to delete links pointing to their site, like we were dragging them down by association. This, despite the fact that MetaFilter’s crack team of moderators scrub even subtle attempts at spam nigh-instantly and foster famously civil discussion.

  2. Interesting that the Jan 2014 “viral article” traffic spike matches up so closely with the former average traffic levels. This suggests that the missing MeFi audience hasn’t vanished from the planet, but are simply being poorly matched by market-maker Google.

    When Michael Lewis is finished with HFT algorithms, perhaps he can take on Panda.

    In the meantime, what can be done?

    CONSUMER

    Use DuckDuckGo, which allows !g and !b for manually routing queries to Google and Bing when needed for secondary searches.

    DEVELOPERS

    Build a browser-based, client-pull metasearch (“Streisand”) engine that provides a three-column view of Google, Bing & DDG, automating the search audit analysis done by Slate. For inspiration, look at the recent research done on HFT algorithms, or work done on identifying content that is censored in Turkey, China, etc.

    If discrepancies are found, have a client-side “Flag” button to report the anomaly to a public archive.org location for forensic analysis. There is a large academic & research community who can make sense of the data. Surely Google would love to have free, quantitative feedback.

    PUBLISHERS

    Start disclosing baseline traffic data to assist academic researchers who are doing forensic analysis of search engine. Publish this data in machine-readable format to archive.org.

    The Internet was designed to survive a nuclear war. It can survive confused algos.

  3. This is interesting, only in the sense of difficulty of analysis. all I see is aggregate traffic reduction, but no detailed breakdown, or mapping of traffic to regions or any other analysis that may have told me. eg. european traffic was down by 60% or some such. what if coincidental with the timing of google algorithm changes, there were changes at bing, yahoo or duck-duck-go? I obviously have to take founder’s word for it: must have something to do with google…

    1. Yes, hence the need for an “Internet Wayback Machine” open dataset for analytics.

      In the Slate article linked above, there is an informal comparison of search results across a few search engines, based on current site rankings:

      http://www.slate.com/articles/technology/bitwise/2014/05/metafilter_layoffs_why_has_google_forsaken_the_legendary_internet_forum.html

      It would indeed be good if MeFi publishes more data for public analysis, including queries, target pages and geographical origin.

  4. Bob Wan Kim, CEO Thursday, May 22, 2014

    Recommendation to content producers / startups: Don’t allow any single points of failures to exist in your business model. Don’t let more than 20% of your business depend on any one source.

    That’s BUSINESS–not revenues.

    In other words, your business can be crushed by a Cable network, Google, the IRS, Your master agent… you’re in trouble

  5. We all remember the difference between a COMMUNITY and a BUSINESS, right? It is easy to bully the neighborhood bake sale to make way for a digital condo that sits shiny .. and empty. MeFi is not a VC-backed startup, it is closer in spirit to a library or town hall.

    From the remnants of crushed community can arise more than one new BUSINESS. One where monopolies cannot buy default placement in browsers and phones. One where antitrust enforcement is not expected nor needed, so no one can be disappointed.

    Remember that all BUSINESS depends on recruiting human TALENT, who can choose which algorithmic revenue source will pay their bills. Taking the COMMUNITY for granted is one way to burn hard-earned DEVELOPER goodwill, putting future algorithms at risk, and ultimately hurting the BUSINESS.

    1. > “Mobile Q&A site ChaCha has laid off most of its employees, saying it came up on the losing side of an algorithm change by Internet powerhouse Google.”

      Another Q&A site. Were Quora & StackOverflow affected by Panda?

      In this week’s update, one estimate is that Ask.com traffic dropped ~70%:

      http://marketingland.com/googles-panda-filter-negatively-impacted-ask-com-ebay-google-backed-retailmenot-com-84708

      eBay ranking decrease was done “manually”, not as part of Panda. Implies the ability to also increase ranking “manually”, i.e. the black-box is in fact a white-box.

      http://recode.net/2014/05/23/yes-google-punished-ebay-for-bad-seo-practices-but-it-wasnt-part-of-panda-update/

  6. In fact, MetaFilter’s quality has steadily declined over recent years as they hired progressively more aggressive moderators try to keep the behavior squeaky clean. There are numerous older users who abandoned the site after finding it progressively more and more annoying to post to.

    At present metafilter does not really resemble most internet commentary because they discussion is scrubbed so clean that no conflict occurs. It’s fine to link drop there, but actually saying anything contentious is now pointless. It’s vaguely possible that metafilter shot themselves in the foot by trying to not look like other sites, which makes the text appear “inorganic” to Panda or similar.

    Anyways I read metafilter for years, still do, but I stopped commenting there, just pointless. I do not feel sorry for them if their incessant censorship turned google’s panda against them.

Comments have been disabled for this post