45 Comments

Summary:

Updated: Facebook these days is doing everything in its power to imitate Google, recruiting the search giant’s sales people, poaching its senior executives and — most importantly — using infrastructure as a competitive advantage. Like Google, Facebook has figured out that the right web infrastructure is […]

Updated: Facebook these days is doing everything in its power to imitate Google, recruiting the search giant’s sales people, poaching its senior executives and — most importantly — using infrastructure as a competitive advantage. Like Google, Facebook has figured out that the right web infrastructure is the difference between user delight and dismay. And like Google, Facebook is finding out that it isn’t cheap.

I’ve been trying to get a handle on Facebook’s infrastructure for some time, but so far have been unable to get the company to open up. The last time I reached out to them, back in January, I was hearing that they had between 1,200 and 1,500 servers, along with storage and switches from EMC Corp. and Force 10 Networks respectively. As it turns out, those server numbers weren’t even close to the total servers used by them.

The company is running around 10,000 servers, according to Data Center Knowledge, citing comments made by Facebook VP of technology, Jeff Rothschild, at a recent MySQL user conference. (See video of the panel.) Of the 10,000 servers, 1,800 are from MySQL and around 805 of them are memcached servers. In order to house its sprawling infrastructure, Facebook has leased data center space from DuPont Fabros in Ashburn, Va., and Digital Realty Trust in Santa Clara, Calif., DCK reports.

How much is Facebook spending on its infrastructure? The company isn’t going to tell us, but there are clues. Server and storage company Rackable today reported first-quarter 2008 sales of around $69 million. Facebook is one of its largest customers, accounting for around 10 percent of Rackable’s sales (that number could be higher, but we’ll have to wait for Rackable’s 10-Q to get a clearer picture), so some quick, back-of-the-envelope math reveals $7 million in spending by the social networking company. A well placed source of mine just let me know that Facebook is going to spend over $9 million more on servers this year. That should be good news for Rackable. Next on my list is an estimate of Facebook’s bandwidth and data center costs.

The hardware spending by startups like Facebook will be a topic of discussion at our Structure 08 conference, where we are hoping to learn more about the infrastructure secrets of all of today’s top (and fast-growing) web players.

  1. Isn’t Google employing strategy of 100 cheap used hardware instead of 1 costly powerful server? It would be interesting to confirm because its working for Google and maybe facebook should go that route. (Although its a different story that Google loads up those cheap machines with its own version of linux and file system for better performance).

    Share
    1. No, Google doesn’t use100 cheap computers instead of 1 powerful server. They *might* use a couple of slightly underpowered computers instead of 1 high end server. Google uses cost effective Intel and AMD processors. Intel uses standard PC disk drives spinning at 5400 to 7200 RPM instead of faster, more reliable SCSI. Google’s real strategy is that they customize the hardware and software for the problem at hand. They design and build their own mother boards, customize the OS, supply their own cluster software: file system, map-reduce, database, remote procedure call, distributed locking, distributed cluster managers; the whole kit and kaboodle.

      Share
      1. I’m going to dream about understanding what the hell you just said tonight

        Share
  2. That strategy only works when power and space is cheap and processing power is expensive. We’re starting to see that shift in a major way. Big iron is making a comeback in power effecient packaging.

    Share
  3. Re server hardware choices, Facebook is a big customer of Rackable, apparently their forth-largest after Microsoft, Yahoo and Amazon.

    http://www.datacenterknowledge.com/archives/2007/Nov/30/facebook_effect_rackable_server_sales_soar.html

    Share
  4. “between 1,200 and 1,500 servers from EMC Corp. and Force 10 Networks.”

    EMC makes storage systems and Force 10 makes Gigabit Ethernet switches…not really “servers” in my opinion. Was this sentence supposed to read differently?

    Share
    1. Obviously, you didn’t read the whole sentence.

      Share
      1. Obviously you didn’t read two comments down.

        Share
  5. how does this compare with myspace? ha

    Share
  6. @ GaryO

    You were right – few words got dropped. I fixed the sentence. Thanks for looking out

    Share
  7. @ Rich Miller

    I also updated on Facebook’s likely spending for rest of the year on the post: about $9 million.

    Share
  8. and that’s just on servers.

    Share
  9. I can tell you, accessing Facebook from Europe is always a real pain, loading so slowly.

    Share

Comments have been disabled for this post