11 Comments

Summary:

As we’re hosted on WordPress.com, we were affected by an outage of their network of blogs today that’s been attributed to a core router change. The company’s 10.2 million hosted blogs were down for 110 minutes, for a projected page view loss of 5.5 million.

As we are hosted on WordPress.com through their VIP enterprise publishing service, we were affected by an outage of their network of blogs today that’s been attributed to a core router change. All GigaOM sites (besides GigaOM Pro and OStatic, which are hosted independently) were down today from approximately 1:30 to 3:30 PT.

Automattic founder and frontman Matt Mullenweg explained in a blog post after the sites came back up that the company’s 10.2 million hosted blogs were down for 110 minutes, for a projected page view loss of 5.5 million. He attributed the downtime, Automattic’s worst in four years, to “an unscheduled change to a core router by one of our datacenter providers.” Mullenweg said the problem “broke all the mechanisms for failover between our locations in San Antonio and Chicago.” He assured users that all their data was safe and promised a “concrete plan” to better handle such situations in the future.

WordPress being down was only the latest of an outbreak of outages this week affecting us or companies we work closely with — given pretty much all of Palo Alto had no power yesterday after a plane crash and then our office Internet and phones (we are based in San Francisco) went out yesterday. It seemed almost fitting (but nonetheless frustrating) when our sites went down. We joked that it was due to rogue reporters’ illicit use of Chrome for Mac, which doesn’t seem to get along with WordPress.com, so we’ve been kindly reminded not to use it over and over again by our lead developer. (Sorry Chancey!) All you cloud haters out there can feel free to voice in about how dumb we are for trusting a hosted service.

All joking aside, I found it surprising that Automattic users — the majority of whom use its services for free — seemed to be fairly forgiving in the comments to Mullenweg’s apology post. It seems the company has enough goodwill to spare a couple hours of failure. But one thing’s for sure, people won’t be so friendly if it happens again.

Related content from GigaOM Pro (sub req’d): The 5 Biggest Cloud-Computing Stories of 2009

Disclosure: Automattic, maker of WordPress.com, is backed by True Ventures, a venture capital firm that is an investor in the parent company of this blog, Giga Omni Media. Om Malik, founder of Giga Omni Media, is also a venture partner at True.

  1. I swear I wasn’t using the newest version of Google Chrome for Mac beta. However, I’d be lying if I said I wasn’t using a Chromium Developer build on the Mac today. You can’t really trace this to me, Liz, can you?!? ;)

    Share
    1. Dude I am sure it was you. We are all convinced it was you. We are actually starting a Facebook page group called No Chromium for Kevin. I am convinced I can trace this back to you.

      Share
      1. do i need to get the cement boots om?

        Share
  2. [...] } Tonight we have the 2 hour failure of WordPress. Here we go with yet another major failure from a poorly planned and badly executed data center [...]

    Share
  3. [...] attributed to a core router change. All GigaOM sites (besides GigaOM Pro and OStatic, which … Read Full Article (No Ratings Yet)  Loading … Word [...]

    Share
  4. Given the timing, those of us who know WP uses a server farm in Texas just figured it was an overload crash from people searching for info on the nutball who tried to take out the IRS offices in Austin with a Piper Apache.

    The good news is that it was just a hardware/administration problem.

    Share
  5. [...] estimates that 5.2 million page views were lost during the outage, and top sites like TechCrunch, GigaOm, CNN, and Flickr had to suffer through the 110 minute down time.  This is WordPress.com’s [...]

    Share
  6. [...] of the big boys like Tech Crunch and GigaOm were down and have been considered in their responses to the [...]

    Share
  7. [...] down such high-profile clients as technology news blog GigaOm (where a somewhat acerbic Liz Gannes has commented that, much to her surprise, “it seems the company has enough goodwill to spare a couple hours of [...]

    Share
  8. [...] Mention: WordPress.com: The hosted blogging service went down for 110 minutes in February 2010 due to networking configuration problems. It [...]

    Share

Comments have been disabled for this post