How big data got its mojo back

3 Comments

Credit: Jakub Mosur

Big data never really went anywhere, but as a business, it did get a little boring over the past couple years.

Big data technologies (and not just Hadoop) proved harder to deploy, harder to use and were a lot more limited in scope than all the hype suggested. Machine learning became the new black as startups infused it into everything, but most often marketing and sales software. So much ink and breath were wasted trying to define (or disprove) the idea of data science, probably because the tools of the trade were still so foreign to most people.

But while the early days of the big data movement hinted at greatness, it’s probably fair to say they didn’t deliver — even if the resulting tools were very useful and very necessary to set the stage for things to come. And, realistically, many companies still haven’t adopted these technologies or these techniques.

sd2015

Things are changing, however, and they’re changing fast. That’s what next year’s Structure Data conference, which takes place March 18 and 19 in New York, is all about. All of a sudden, those early ideas companies might have had about how data could transform their business don’t seem so crazy. Real-time processing, the internet of things and intelligent applications are all more possible now than ever before.

We’ll be examining the evolution of Hadoop as a technology and a marketplace; talking to early technology adopters from business large and small; highlighting entrepreneurs and researchers in fields ranging from robotics to quantum computing; and discussing what all this means to companies and society as a whole. Confirmed speakers so far include:

  • CEOs of all the major Hadoop players — Cloudera, Hortonworks and MapR
  • Rob Fergus, research scientist, Facebook Artificial Intelligence Research
  • Don Duet, co-head of technology, Goldman Sachs
  • Hilary Mason, founder and CEO, Fast Forward Labs
  • Dharmendra Modha, senior manager of cognitive computing, IBM Research
  • Ion Stoica, co-founder and CEO, Databricks; co-director of AMPLab, University of California, Berkeley
  • Ron Brachman, chief scientist and head, Yahoo Labs
  • Jeff Hawkins, co-founder, Numenta

You can see the complete list here. They’re all great, and we’ll be adding even more in the weeks to come.

Tom Reilly, CEO of Cloudera, at Structure Data 2014

Tom Reilly, CEO of Cloudera, at Structure Data 2014

In part, this evolution is happening because technology is maturing to a point where it’s faster, more flexible and generally better. Things like Spark and Kafka are making their way out of web companies and research labs and into the mainstream. The same goes for machine learning and artificial intelligence techniques, most notably the field of deep learning.

But people also matter. The bleeding-edge folks who cut their teeth on these technologies — often times even created them — are now venturing out on their own, or at least out of the academic world and into the corporate world. They’re taking their experiences and turning them into startups and products, sometimes in entirely new realms such as biotech and agriculture.

It has been amazing to watch the transformation from the time the idea of big data really caught on several years ago to today. Data used to mean numbers, tables and business intelligence; today it means all of those things, as well as documents, voice, text, pictures, connected devices, automation, you name it. Join us in New York and find out how the future of data will unfold.

3 Comments

Peter Fretty

I would argue a lot of the hype died down because organizations failed to invest in actually training the team to accomplish many of the crucial big data tasks. According to a recent IDG survey, the less than 15 percent of respondents would rate their organizations as highly effective at any of the key big data responsibilities. Investing in tools alone really doesn’t accomplish what most desire — an analytical culture.

Peter Fretty, IDG blogger working on behalf of SAS

Larry

When it comes to data everything still boils down to good ol’ data structures and algorithms. If there’s no break through on the DS/A front then forget about progress on data processing.

When I look at big data I see no major BTs, it’s the same ol’ arrays, lists, hash tables, trees and their associate algorithms. Nothing is new. All we get is shuffling data from one DS into another. So of course there’s no real MOJO. It’s just hype, good for an IPO or two but no real break through.

Comments are closed.