Twitter will move into its own data center soon as it seeks to scale its social messaging service, according to a presentation by John Adams, one of the messaging service’s operations engineers. Speaking at the Chirp developer conference yesterday in a session on scale, he laid out Twitter’s strategy to keep the fail whale at bay — which included plans to soon move to its own data center (see slide below).
It’s unlikely Twitter is building and operating this data center itself. In 2008 Twitter signed a contract to host its servers at NTT because the latency in the cloud was too high for its service, and last August NTT said it had leased 15,000 square feet in Santa Clara, Calif. for expanding its data center operations in part because of Twitter’s success. So my hunch is that Twitter is moving into NTT’s data center dedicated to the messaging service, as opposed to building and operating its own, which would take a while. I’ve reached out to NTT and Twitter for more information. In 2008 Twitter saw year-over-year traffic growth of 752 percent and from 2008 to 2009, traffic rose 1,358 percent. It serves 55 million tweets a day.
Twitter has gone from hosting aspects of its service on Amazon Web Services and via Joyent to discarding the cloud because latency was too high (we’re going to have a talk about improving latency in the cloud at our Structure 10 conference in June). Essentially, like most IT folks, Twitter’s operations group plays a game of whack-a-mole attempting to add hardware, improve algorithms or add new code to solve the next engineering bottleneck.
Even if the dedicated NTT data center space manages to improve the service for a while, will Twitter ever end up following Facebook, which this year said it would build its own data center so it could control costs and reduce its energy consumption? And what does this say about the evolution of scalable infrastructure?
Chirp 2010: Scaling Twitterhttp://static.slidesharecdn.com/swf/ssplayer2.swf?doc=johnadams-scalingtwitter-100416061113-phpapp01&rel=0&stripped_title=billions-of-hits-scaling-twitter
View more presentations from John Adams.