6 Comments

Summary:

As data volumes skyrocket, startups looking to take advantage of the opportunity in Big Data need to focus on the art of statistical-learning algorithms, says the founder of Dataspora. Modeling and analysis through algorithms is what will determine the winners and losers in Big Data.

As data volumes skyrocket, startups looking to take advantage of the opportunity in Big Data need to focus on the art of statistical-learning algorithms, says Michael Driscoll, founder of Dataspora and co-founder of Metamarkets (see full disclosure below.)

Driscoll took the stage of the IA Ventures Big Data Conference in New York today and laid out three skills necessary for data-driven start-ups: data munging, the corralling and wrestling of data; modeling, the statistical analysis of data through algorithms; and visualization, the presentation of all the data. While all three are necessary for success, Driscoll believes that modeling and analysis through algorithms is what will determine winners and losers in Big Data.

In the past, analysis might have meant just organizing the data into counts, sums and averages. But new data-product companies will have to take analysis to the next step toward powering recommendations and predictions, the kind of work being done by Pandora, Amazon or Netflix to suggest content to users

“The secret sauce is predictive analysis powered by data,” said Driscoll. “It’s less about what you did and more about what you should do, and not even telling you what you should do … it should just do it for you.”

This means that the two other skills — munging and visualization — must recede into the background to some extent. According to Driscoll, data-driven companies are spending some 80 percent of their time data munging, basically ingesting data and preparing it for analysis. He said that’s due to the hard work involved in wrestling with various data sets and managing the friction as data comes in, something he said should become easier over time. There’s also a lot of time spent on visualization, which allows customers to see the data and make decisions from it.

For companies looking to get ahead, he said the key lies in investing in machine-learning algorithms. He pointed to the demise of Swivel, a data visualization company, as a sign that visualization isn’t enough. Swivel’s executives said recently that the company had trouble articulating the usefulness of visualization and prioritizing new features.

“Visualization gets them in the door,” Driscoll said, “but what matters is search results or recommendations.”

DisclosureMetamarkets is backed by True Ventures, a venture capital firm that is an investor in the parent company of this blog, Giga Omni Media. Om Malik, founder of Giga Omni Media, is also a venture partner at True.

Post and thumbnail photos courtesy of Flickr user nataliej.

Related content from GigaOM Pro (subscription req’d):

  1. I actually think what Mike said at the conclusion of his talk was something to the effect that “visualizations are what get folks in the door, but analytics are what keeps them…”

    Share
    1. Yeah, I grabbed my quote from talking with Mike afterward. So what’d you think of the conference?

      Share
  2. While it’s true that firms that can analyze data stand to gain – it’s also worth noting that those who have the data aren’t always one in the same with those that have the skills to analyze it.

    Social network data is a case in point: which firms have the most valuable social graph on the planet? Telecoms! And what have telecoms done with this incredibly valuable data to date? Almost nothing.

    The next age of innovation will come when these dark pools of data are unlocked and made available to the machine learning algorithms that you praise.

    Share
  3. [...] Metamarkets (AdExchanger.com Q&A) co-founder Michael Driscoll participated in yesterday's IA Ventures Big Data Conference in New York. Metamarkets has IA and its chief, Roger Ehrenberg, as an investor. During a panel discussion covered by GigaOm, Driscoll identified "three skills necessary for data-driven start-ups: data munging, the corralling and wrestling of data; modeling, the statistical analysis of data through algorithms; and visualization, the presentation of all the data." Read which skill determines the "winner." [...]

    Share
  4. [...] Three skills necessary for data-driven start-ups: data munging, the corralling and wrestling of data… [...]

    Share
  5. I’m completely with you.

    Following Michael’s line of thought you should check the Honey-Lizer. It comes to help publishers monetize better by streaming the part of user data acquisition, user data analysis and user ads/offer targeting. Moreover, it uses Facebook API to get the data so you also get the advantage of controlled user privacy with 0% overhead of storing, managing and protecting the data.

    Share

Comments have been disabled for this post