Facebook launches an open source organization and releases new routing tech

2 Comments

Facebook announced a pair of open source efforts Monday at its inaugural @Scale conference in San Francisco, including an organization called TODO that will help other companies get started down their own open source paths, and a new networking tool called mcrouter.

TODO, which is short for “talk openly, develop openly,” is a collaboration between [company]Facebook[/company] and other large technology companies that are strong open source advocates. They include Box, Dropbox, Github, Google, Khan Academy, Stripe, Square, Twitter, and Walmart Labs. Facebook promised more details on the project in the coming weeks, but James Pearce, the company’s head of developer advocacy and open source, shared the following description on a blog post:

[O]ur overall goal in this collaboration is to make open source easier for everyone. We want to run better, more impactful open source programs in our own companies; we want to make it easier for people to consume the technologies we open source; and we want to help create a roadmap for companies that want to create their open source programs but aren’t sure how to proceed.

This is not the first time Facebook has collaborated with other web companies in order to advance open source technology. In March, it announced a collaboration with three other companies — including TODO partners Google and Twitter — on a database project called WebScaleSQL. The technology is a version of the popular MySQL database that’s designed to help other companies run it at large scale without requiring them to spend untold amounts time and resources re-engineering it like the WebScaleSQL contributors had to.

A diagram of mcrouter; mc means memcached. Credit: Facebook

A diagram of mcrouter. Credit: Facebook

Facebook’s other announcement, mcrouter, is the latest in a string of open source releases the company has made over the past several years, including popular technologies such as Cassandra and HipHop, and the hardware-based Open Compute Project. The gist is that it’s a tool for handling traffic on Facebook’s cache layer, which is where the company stores in memory data that needs to be accessed and served faster than is possible if it was on hard disk inside a traditional database system.

There are a lot more details on the blog post announcing mcrouter, but here is Facebook’s high-level explanation:

Mcrouter is a memcached protocol router that is used at Facebook to handle all traffic to, from, and between thousands of cache servers across dozens of clusters distributed in our data centers around the world. It is proven at massive scale — at peak, mcrouter handles close to 5 billion requests per second. Mcrouter was also proven to work as a standalone binary in an Amazon Web Services setup when Instagram used it last year before fully transitioning to Facebook’s infrastructure.

2 Comments

curious

“Mcrouter was also proven to work as a standalone binary in an Amazon Web Services setup when Instagram used it last year before fully transitioning to Facebook’s infrastructure”

Did they stop using it after transitioning to Facebook’s infrastructure?

Nikohl Vandel

so, basically, its this layer where the players get to pitch a tent, and decide which content to shuffle where to maximize their commitment 2 the fB game? like a base camp.

Comments are closed.