Chris Lynch has been watching the database industry closely for decades as both the CEO of Vertica Systems and now as a partner with Boston-based venture capital firm Atlas Venture. And, as he told us on this week’s Structure Show podcast, the database industry is under some serious fire from open source technologies like Hadoop.
Here are some highlights from an insightful 15-minute conversation that you’ll probably want to hear in its entirety.
[soundcloud url=”https://api.soundcloud.com/tracks/117852161?secret_token=s-LHCTJ” width=”100%” height=”166″ iframe=”true” /]
“We’re starting to see the finical impact that Hadoop can have on the ecosystem of database vendors,” he said, referring to the latest earnings numbers from analytic-database pioneer Teradata.
But Teradata isn’t alone in feeling the pinch. “In my tenure, we were charging $100,000-$150,000 a terabyte,” Lynch said of his days at Vertica, which is now part of HP. “… Those now-generation companies like Vertica, like Greenplum, have, in fact, their own set of pressures now with Hadoop, as Hadoop becomes commercially viable, more robust,” he added.
The pressure is coming from everywhere. Cloudera is now positioning itself as a data-management platform more than a Hadoop distribution, and even SQL-on-Hadoop startup Hadapt (one of Lynch’s investments) is “going gangbusters,” Lynch said.
The problem for legacy vendors, he noted, isn’t always about vision — they see big data happening — but about economic pressures. “How do you embrace a new business and a new market that sub-optimizes what you’ve worked so hard to build?” Lynch asked. “That’s hard to embrace. It’s certainly hard to embrace because of the pressures of public companies to make quarterly numbers.”
It’s especially challenging considering the reasons Hadoop is becoming so popular. “The reason [Hadoop] allows you to aggregate [all your data] is less about the technology and more about the economics,” Lynch said. “Because they’ve made it affordable to put all that data in one place.”
Asked which legacy vendor is best poised to ride the Hadoop and greater big data wave, Lynch was quick to point to IBM. “I think that the best-positioned company is, in my opinion, by far IBM, because of the breadth of their offers and the ways that they can make money as this big data marketplace develops,” he said, noting its product range from databases up to BI applications and the billions it has spent acquiring and building new technologies.
“If you look at the investments someone like IBM has made compared to Oracle, that tells the story,” he said.
Lynch also talked about the hard work it can involve to take great technology — like that from his latest investment, machine learning startup Nutonian — and turn it from literally a science project into enterprise software. “That’s what we do at Atlas — we invest very early-stage in companies ’cause we like to build them,” he said. “We’re not financial engineers, we don’t have a billion-dollar fund and we’re not buying lottery tickets. We’re actually building companies.”