There is no other way to put it: 2014 was a huge year for the big data market. It seems years of talk about what’s possible are finally giving way to some real action on the technology front — and there’s a wave of cash following close behind it.
Here are the five stories from the past year that were meaningful in their own rights, but really set the stage for bigger things to come. We’ll discuss many of these topics in depth at our Structure Data conference in March, but until then feel free to let me know in the comments what I missed, where I went wrong or why I’m right.
5. Satya Nadella takes the reins at Microsoft
Microsoft CEO Satya Nadella has long understood the importance of data to the company’s long-term survival, and his ascendance to the top spot ensures Microsoft won’t lose sight of that. Since Nadella was appointed CEO in February, we’ve already seen Microsoft embrace the internet of things, and roll out new data-centric products such as Cortana, Skype Translate and Azure Machine Learning. Microsoft has been a major player in nearly every facet of IT for decades and how it executes in today’s data-driven world might dictate how long it remains in the game.
4. Apache Spark goes legit
It was inevitable that the Spark data-processing framework would become a top-level project within the Apache Software Foundation, but the formal designation felt like an official passing-of-the-torch nonetheless. Spark promises to do for the Hadoop ecosystem all the things MapReduce never could around speed and usability, so it’s no wonder Hadoop vendors, open source projects and even some forward-thinking startups are all betting big on the technology. Databricks, the first startup trying to commercialize Spark, has benefited from this momentum, as well.
3. IBM bets its future on Watson
Big Blue might have abandoned its server and microprocessor businesses, but IBM is doubling down on cognitive computing and expects its new Watson division to grow into a $10 billion business. The company hasn’t wasted any time trying to get the technology into users’ hands — it has since announced numerous research and commercial collaborations, highlighted applications built atop Watson and even worked Watson tech into the IBM cloud platform and a user-friendly analytics service. IBM’s experiences with Watson won’t only affect its bottom line; they could be a strong indicator of how enterprises will ultimately use artificial intelligence software.
2. Google buys DeepMind
It’s hard to find a more exciting technology field than artificial intelligence right now, and deep learning is the force behind a lot of that excitement. Although there were a myriad of acquisitions, startup launches and research breakthroughs in 2014, it was Google’s acquisition of London-based startup DeepMind in January that set the tone for the year. The price tag, rumored to be anywhere from $450 million to $628 million, got the mainstream technology media paying attention, and it also let deep learning believers (including those at competing companies) know just how important deep learning is to Google.
1. Hortonworks goes public
Cloudera’s massive (and somewhat convoluted) deal with Intel boosted the company’s valuation past $4 billion and sent industry-watchers atwitter, but the Hortonworks IPO in December was really a game-changer. It came faster than most people expected, was more successful than many people expected, and should put the pressure on rivals Cloudera and MapR to act in 2015. With a billion-plus-dollar market cap and public market trust, Hortonworks can afford to scale its business and technology — and maybe even steal some valuable mindshare — as the three companies vie to own what could be a humongous software market in a few years’ time.
- Tableau’s mad growth: Tableau has some tough competition from well-funded startups and even new products from legacy vendors including Salesforce.com, Microsoft and IBM, but its revenues are nearly doubling every quarter. It’s hard to see who’ll steal its place as the next-gen business intelligence poster child, especially if Tableau keeps expanding its product lineup.
- Twitter embraces its data: Twitter has always been a tempting target for enterprise data efforts, but getting data wasn’t always easy and projects weren’t always well-conceived. Twitter buying Gnip in April opened the social media platform up for business on the data front, some early fruits of which include the inaugural Twitter data grants and an interesting partnership with IBM.
- Cloud computing gets its big data act together: There’s no single event that epitomizes this shift, but cloud computing providers and big data vendors alike really stepped up their offerings last year. Running big data workloads on the Amazon, Google and Microsoft clouds never looked so good — which is important if cloud computing and big data are the paradigm shifts we like to say they are.
- The internet of things: There were some big acquisitions in 2014 (Nest, Dropcam, SmartThings), but none of them can do justice to the enormity of the internet of things. It spreads across untold millions of consumer devices and industrial machines, and might well represent the biggest opportunity yet for companies selling data infrastructure and analytics.