Summary:

Here are some highlights from a Reddit Ask Me Anything featuring members of Twitter’s open source engineering team. It’s a pretty informative look at Twitter’s open source culture, the statuses of its various projects and its high hiring standards.

Reddit

A handful of Twitter employees who work on the company’s open source software projects took to Reddit for an Ask Me Anything on Wednesday. The questions were wide ranging, from questions about specific projects to how to get a job at Twitter. Here are some of the highlights.

On why companies care about open source

“Here are some detailed reasons why to open source code:

  • Community Feedback: more usage translates into more bug reports which improves quality over time
  • Attract Talent: in our opinion, some of the best talent these days works in open source communities
  • Better Hiring: what better way to do a technical interview than looking at someones open source contributions?
  • Retain Talent: we believe great engineers like working in the open
  • Reduce Duplication: when you open source things, someone on the outside (or inside) will let you know
  • Modularization: when you open source internal code you tend to break it apart into smaller “reusable modules”
  • The Right Thing To Do: these days it’s difficult to build anything without benefiting from open source in some fashion. The old saying of “rising tides lift all boats” applies here.”

The whole thread is available here.

On how Twitter evolved its open source stack

“Using open source in the beginning was just practical when you’re a small scrappy startup. Since the first Tweet was sent, Twitter has been built on top of open source software (see opensource.twitter.com). Our early stack was mostly MySQL, Rails and Memcached. But then we then rapidly evolved from a simple application into a service-oriented architecture, leading us to integrate even more open source technologies such as OpenJDK (JVM), Netty, Apache Lucene, Apache Thrift, Apache Hadoop and Redis. This had to happen to allow us to scale and we discussed some of this in this blog post: https://blog.twitter.com/2013/new-tweets-per-second-record-and-how”

The whole thread is available here.

On getting hired at Twitter

“Generally we look for strong candidates. It is hard to give a recipe for interviews. Everyone must be able to code well. After that, some know more math, some are stronger with systems, some have more DB experience, some are functional programming PhDs. Find your niche and be great in your niche, BUT eagerly learn new things that inspire you.”

That whole thread is available here. Here’s another on the same topic.

On the importance of Mesos

“Apache Mesos is one of the bigget parts of Twitter infrastructure. It currently powers our compute cloud (with Apache Aurora) and part of our real time analytics (with Storm on Mesos). Almost all new services at Twitter are being written on top of Mesos. Twitter has been recently investigating/using YARN but its use case so far has been targeted towards Hadoop based work loads. In contrast, Mesos has been running in production at Twitter for more than 3 years now and has been thoroughly battle tested to run at Twitter’s scale.”

The whole thread is available here.

On not open sourcing its Marathon database … yet

“Right now we don’t have plans to open source it in the near future but plans may change. But meanwhile, you might investigate Mongo, which we understand is also webscale: http://www.youtube.com/watch?v=b2F-DItXtZs

The whole thread is available here.

The MongoDB comment is a classic burn straight from 2010.

Comments have been disabled for this post