Mesosphere, a startup trying to commercialize the Mesos cluster-management software, has raised $10.5 million in series A funding from Andreessen Horowitz, Data Collective and Fuel Capital. The company, which was founded in 2013, has now raised $12.75 million in total.
Mesos is an open-source project, initially developed at the University of California, Berkeley, designed to turn collections of individual into one big pool of computing. Developers can launch services on a Mesos cluster, dictate the amount of resources each needs and then let the system ensure they stay running. Twitter is probably the largest Mesos users, although companies such as Airbnb, eBay, HubSpot and many more also use it to manage their infrastructure.
Mesos was inspired by Borg, the system Google famously built to manage its sprawling data centers. The overall goals are to minimize the impact of any given server failing and to minimize the number of staff needed to monitor a data center, because the system will automatically rebalance the load across the rest of the pool. Applications are isolated by running in their own separate Linux containers.
Our Structure conference, which takes place June 18 and 19 in San Francisco, will feature talks with engineering leaders from companies including Google, Twitter, Facebook and Airbnb. They’ll talk all about the technologies they’re building and deploying in order to manage their ever-expanding scale.
Florian Leibert, the founder and CEO of Mesosphere, was on the Structure Show podcast recently and explained why Mesos is catching on right now:
“If we look at VMs and how VMs have evolved in maybe the last 10 years or so … the hardware was really growing in terms of capacity and applications were relatively small, so virtual machines came in and allowed you to place — manually — multiple small applications on these big servers. Today, if we look at a lot of applications that are being written — like Spark, like Hadoop and so forth — they are pretty much distributed systems from the get-go. And we are also using commodity hardware. So, today, this VM model doesn’t make as much sense any more. Rather than splitting up the applications onto multiple machines, we have to aggregate all the machines and present them to the application as a pool of resources.”
Mesos can run on bare metal, cloud instances or virtual machines, as long as they’re running the Linux operating system. Mesosphere, the company, is building plugins to make Mesos more consumable by mainstream enterprises and also providing some support to companies deploying it. It has also developed a framework called Marathon that makes it easier to launch and manage applications on the Mesos pool (such as Rails or JBoss applications, or even additional Mesos clusters) while Mesos focuses on long-running services and scheduled tasks.
The hope for Mesosphere is that more web companies and even mainstream companies will be inspired to move their applications to Mesos, lured by the promise of dynamism promised but not necessarily delivered by server virtualization. Leibert put it pretty succinctly when I spoke with him in February for a profile of the company. “Marathon is like the PaaS … on top of Mesos,” he said. “[You can] build your own Heroku, basically, within your data center.”