Blog Post

Facebook — with help from Google, LinkedIn, Twitter — releases MySQL built to scale

Stay on Top of Enterprise Technology Trends

Get updates impacting your industry from our GigaOm Research Community
Join the Community!

Facebook announced on Thursday a new open source project called WebScaleSQL. The technology — a branch of the incredibly popular MySQL database — already includes contributions from Google, LinkedIn and Twitter, and Facebook promises the WebScaleSQL community will accept contributors from anyone with experience operating MySQL beyond its natural limits.

Facebook’s mastery of MySQL has been a point of pride within the company even as skeptics questioned the wisdom of sticking with the database. The last time I checked in, for a tech talk the company’s engineers gave in late 2011 (when Facebook was at a mere 800 million users), its MySQL environment was growing like crazy. Engineers talked about 60 million queries and 4 million row changes per second, and about rapidly multiplying data volumes. They also touched on the importance of flash storage, something the company has really picked up on during the ensuing couple of years.

WebScaleSQL incorporates some of these learnings, as well as those from the other contributing companies. The project will keep up to date with the current production-ready MySQL release (right now, that’s MySQL 5.6) and is designed to move fast. In a blog post introducing the project, Facebook software manager Steaphan Greene details some of the specific improvements in the initial WebScaleSQL release, as well as those Facebook intends to add in the near future.

Beyond the potential utility of another good open source technology, the introduction of WebScaleSQL might help put to rest any lingering claims that NoSQL databases are the only option for applications that need scalability and performance. New SQL technologies have tackled those challenges and picked up a lot of momentum in the process; the availability of a scalable, open source, MySQL branch is just another step further. Now, the choice over which database to use — even within companies such as Facebook, which runs multiple NoSQL databases — is often focused around use cases rather than sheer scale.

WebScaleSQL isn’t the only for bringing scalable MySQL to the masses of web startups, though. In February, Facebook MySQL leader Mark Callaghan wrote a blog post proposing a mentorship program for startup database architects to learn the guts of MySQL so they can grow their environments themselves rather than trying to hire experts in a super-competitive field. In 2012, Twitter open sourced its MySQL code base.

Now more than ever, being able to scale a database — any database — in a hurry is critical. Mobile and web applications can attract millions of users over the course of weeks, in some cases, and they need databases that can handle all that user data and all those requests. Going viral is great — unless, of course, it crashes your system.

Feature image courtesy of Shutterstock user Semisatch.

15 Responses to “Facebook — with help from Google, LinkedIn, Twitter — releases MySQL built to scale”

  1. While it’s always good news to have big brains working to improve the scalability of MySQL, there doesn’t seem to be a fundamental breakthrough. Facebook is justifiably proud of their massive, sharded MySQL database, but they also have dozens of engineers doing nothing more than writing code to clean up database inconsistencies. Not something everyone can afford or will want to do.

  2. the issue with NuoDB is the need to migrate all their data to their proprietary newsql (or whatever sql..) …wondering what’s the effort/project time to use webscaledql.. we currently using AWS RDS (costs significant) ..

  3. @ST – why? do you know the details? we are looking for a solution.. we actually look at scalebase and even thought about migrating to nosql. need help!

    • Doron Levari

      I won’t answer for @Derrick, but from what I see in this article and in Github as well, it seems like this project is focused on improving the MySQL database itself to be better, faster, more robust and fit big guys’ web scale.

      Most big guys are achieving data distribution over MySQL databases using application sharding.

      I work for ScaleBase, specializes in – the automated and logical distribution of relational data using MySQL backend databases, and optimized for the cloud. ScaleBase can create a distributed database based on any MySQL building blocks, very much like MongoDB is gives right out of the box, with their mongos in front of the mongod – without the need to shard your application to use it!… So it seems you can use this exciting new WebScaleSQL as a great building block in a general-purpose distributed database with ScaleBase.

      I’ve authored some blogs, showing how modern needs can be met by MongoDB as well as with MySQL, looking at autosharding, data distribution and query models if anyone is curious to know more.

        • Doron Levari

          The problem is right there, ask WebScaleSQL contributes, but as Mark commented below, “they also have dozens of engineers doing nothing more than writing code to clean up database inconsistencies. Not something everyone can afford or will want to do.”.

          This is the problem for every Facebook-wannabes or just every other web/mobile startup, and you can find them solving this problem with ScaleBase.