Blog Post

Twitter open sources its MySQL secret sauce

Stay on Top of Enterprise Technology Trends

Get updates impacting your industry from our GigaOm Research Community
Join the Community!

Updated: Twitter has shared numerous aspects of its infrastructure over the past few years, and its decision to open source its work on MySQL might be the social media platform’s most useful contribution yet. Sure, open source big data tools are valuable, but they’re not MySQL.

Used by millions of web developers, MySQL is hugely popular; it’s the “M” in the LAMP stack that still underpins many web applications. But it has its problems, among them scalability and performance under the pressure of high transaction rates. This is part of the reason that NoSQL databases came into existence and continue to flourish. So, for anyone concerned with making MySQL scale, being able to see and build upon what a site like Twitter did with the code should be a big deal.

It’s not like Twitter is alone in working on MySQL’s shortcoming or anything — companies such as Tokutek and ScaleBase are centered around this very premise, and Facebook has done some amazing things with the database — but Twitter’s willingness to open source its code is critical. No vendor licenses, no support contracts and no lock-in, just a tried and true MySQL fork from one of the world’ most successful web platforms.

As it turns out, however, Twitter’s move in open sourcing its core MySQL code comes after the company already open sourced various components that sit atop MySQL. In 2010, it open sourced Gizzard, a middleware component for creating distributed databases that can serve tens of thousands of queries per second, and FlockDB shortly thereafter.

Update: Someone alerted me that Facebook does have a site for sharing its MySQL patches, although it’s hosted at Launchpad. Earlier on Monday, commenters to a thread on the MySQL at Facebook page strongly urged Facebook to move its efforts to the far more popular Github, which is where Twitter hosts its code.

3 Responses to “Twitter open sources its MySQL secret sauce”

  1. Chris Aniszczyk

    Just to further expand why we did this (and not directly post patches to MySQL upstream), most of the changes apply generally and would be beneficial to the MySQL codebase. However, it’s easier for us to go ahead and work in the open and to be able to pass these changes upstream, rather than pass them off privately and wait potentially a year or more to get them back in a public release. On top of that, we want to be able to collaborate with Percona, Facebook, Google, MariaDB, etc., on some of the actual changes, to make things better for everyone.

    This is simply the first step towards that goal.

    • Kristopher Wong

      I also have to assume that Oracle would be slow to react and implement changes because highly scalable MySQl is competition with Oracle which is a lock for enterprise. With these published change I am going to need to assume you could probably run an enterprise level application on MySQL with little to no issues.

      Kudos to Twitter for championing open source products