Building a service-oriented architecture for a web application is probably the right way to go, but managing so many services — all interconnected but all living and running in their own little bubbles — can be tricky. Airbnb solved the problem for its service by building a system called SmartStack, which the company has open sourced on GitHub.
The company detailed SmartStack in a Wednesday afternoon blog post, but the gist is that it keeps applications up by automating and decentralizing the communications between them. A component called Nerve makes sure services are healthy (“is this app reachable over TCP or HTTP?” the blog post gives as an example) and, if they are, a component called Synapse makes them discoverable by other applications and services. If services aren’t healthy, Synapse takes them out of the rotation.
The blog post, of course, offers a much deeper explanation of how this works, including SmartStack’s heavy reliance on haproxy and how the company built a tool on top of SmartStack to load balance its Amazon Web Services-based resources (which is all of them).
Beyond this specific piece of software, though, I’m perpetually amazed at the ingenuity of web startups that find new ways to automate the management of their increasingly distributed systems. And the great thing is that it’s not just large companies like Facebook automating MySQL management, LinkedIn building entirely new databases or Netflix building its own take of service-management. Now, it’s also smaller companies like Airbnb getting in on the act.
That so many things get open sourced to some degree just drives more innovation. It’s no wonder there’s a whole emerging dedicated to making application developers’ lives easier and that web and mobile applications are getting better, faster and more reliable with each passing day. To think (broadband issues notwithstanding) there’ll be a whole generation of consumers who might never have to get used to fail whales or “Oops, something went wrong” messages.
Feature image courtesy of Shutterstock user Praisaeng.