Avinash Lakshman isn’t yet saying exactly how his stealth-mode startup called Hedvig will disrupt the storage space, only that the future will be software-defined. Yes, it’s a vague prediction, but you should probably pay attention nonetheless. If there are two things Lakshman isn’t short on, they’re big ideas about distributed systems and determination.
Lakshman is best known for creating the Cassandra NoSQL database while working at Facebook from 2007 through 2011. Prior to that, he spent three years at Amazon and helped create the Dynamo database system that underpins much of Amazon.com and inspired both Cassandra and Amazon Web Services’ DynamoDB offering. Lakshman doesn’t really recall what he was doing before joining the Dynamo team (“something inconsequential, for sure,” he joked), but it’s how he got to be part of that team at Amazon that speaks volumes about his desire to succeed.
Lakshman’s driving force has always been pushing the intellectual envelope, he says. So when Werner Vogels joined Amazon as CTO about three months after Lakshman did and began putting together a team to build the next generation of Amazon’s infrastructure, Lakshman knew he had to be on it. He heard other engineers around the Seattle campus chattering about their work, and one day he approached one of them with a proposition.
Admittedly, he didn’t know what they were working on, but Lakshman told the guy, “You have my word that I will burn the midnight oil and get up to speed.” Word got back to Vogels, who invited Lakshman to join the team, and the rest is history.
Software-defined storage, inspired by the cloud
Fast-forward six years, and Lakshman says he isn’t totally satisfied with Dynamo or Cassandra. That’s why he’s on this list — because he’s hoping he finally nails it with Hedvig, his vision for how storage must function in a software-defined and cloud-savvy world.
His idea sounds simple enough: Take the operational techniques that exist inside companies like Amazon, Facebook and Google, and then package them up for all the world to use. Everything should be automated, programmable and easy to provision, Lakshman explained, like you’re using a public cloud platform. He looks at Amazon Web Services, in particular, as “a good blueprint for how it should be done.”
Large enterprises have now seen what’s possible and they want that experience inside their own data centers. They want to build new types of scale-out, service-oriented and virtualized applications, and they want them to perform better for less money. The storage system is a critical component of these new architectures, especially in an era of big-and-growing data.
However, Lakshman cautioned, “In order to enable that, you need to rethink how systems are built.”
Does failing at webscale lead to software that won’t?
Spending seven years inside some of the world’s biggest web companies helps put one in the rethinking mindset. Only it’s not just the sheer scale of operations that’s so influential, but what you learn from failure at that scale. “That is something you can’t teach someone,” Lakshman said. “You have to have your fingers burned.”
Looking back, he’s able to laugh, but it’s not always easy when you’re in the thick of it. Once while his mother was visiting from India, Lakshman was on pager duty at Amazon just like all senior engineers were required to do. It was a nightmare of a week, with stuff going wrong nearly around the clock.
“If this is how you’re working to support us,” Lakshman recalls his mom saying, “I think you should get a different job.”
Still, he wouldn’t trade those experiences for anything because they helped shape his thinking in terms of how one has to build infrastructure at such a large scale. When people who don’t have “systems administrator” in their job titles are expected to manage such large systems, complexity in the management process tends to disappear really fast. “It’s all about ease of management and ease of use,” Lakshman said.
That translates well in the world of next-generation enterprise software, too. “You want what you’re building to take care of all those headaches,” Lakshman said. “But in order to build that, you have to recognize what happens when things go wrong [and how to architect against that].”