17 Comments

Summary:

Data scientists are becoming employees as companies try to tap the river of data they’re generating to improve their products or build new business opportunities. Here’s a look at how companies are turning more to these data gurus to help lead them to success.

2353441514_68d83a4465_z

The fact that Foursquare is looking for a data scientist, first reported earlier this week, is hardly news. They’re just pursuing what many companies will eventually do as they try to tap the river of data they’re generating to improve their products or build new business opportunities. Indeed, data scientists will increasingly become vital employees as companies create and use more and more data.

But what is a data scientist? Hilary Mason, a data scientist at Bit.ly, has a good definition. It’s someone who can obtain, scrub, explore, model and interpret data, blending hacking, statistics and machine learning. It’s a set of skills that go beyond many existing job titles and it’s increasingly in demand.  Here’s a look at some recent news on the data scientist front:

Data scientists won’t just be important for social networks sitting on mountains of user data. Many companies that deal with consumers are able to generate a lot of data on users using analytical and consumer behavioral tools. And with data-as-a-service providers and others able to provide additional datasets, companies are able to work with a lot more information. The key will be digging deep into all that data and figuring out how to apply it to maximize revenue and improve products.

Katie Tucker, a senior partner at executive recruiting firm Korn/Ferry, said the hunt on data gurus, czars and architects has kicked into high gear in the last year. She said there was a 200 percent increase from 2008 to today in searches for executives with sophisticated data mining or data analytics capabilities. As companies emerged from the recession and expanded their marketing budgets, they started to look for more efficient ways to build their businesses and become even more relevant to their customers. With newer data warehousing appliances and management and processing tools such as Hadoop helping companies capture and manage more data, executives have looked to the information right under their noses for insight. Tucker said that, in turn, has prompted a big upswing in searches for data professionals, similar to the way the Sarbanes-Oxley Act’s reporting requirements elevated the importance of auditors.

“This is the first time in my 15 yeas I’ve ever seen data as such a big focus,” for companies, said Tucker. “Just as with Sarbanes-Oxley, you have something that was good discipline and good business becoming front and center.”

Todd Levy, CTO of URL shortener Bit.ly, said companies have built out more obvious products without having to delve deep into data. Now, the challenge is to get better insights out of the data emanating from a company to help steer it toward new opportunities. That requires more data knowhow. “We’re moving into an area where the low-hanging fruit has been culled and you need to take advantage of more advanced resources now,” he said.

Over the last year, location-based service and ad network Where has built up a data team of 12 people, made up of quantitative analysts, PhDs and data scientists. CTO Ivan Mitrovic said the ramp-up was necessary as the company launched its own mobile ad network. The team has not only made sense of the 3 billion monthly actions on the ad network but they were also able to point the company toward a new business: a local discovery engine for consumers that recommends places to go. That’s the value of data scientists, said Mitrovic, in helping turn raw data into new business opportunities. “When you give them the data, they find products in the data that you didn’t know existed,” he said.

Foursquare’s Dennis Crowley seemed to echo this point recently at the AllThingsD mobile conference, where he seemed almost unsure what Foursquare will turn into. Why is that? Because all that data could help the company transition in a number of ways. It could be as I wrote earlier about just being a better recommendation engine for local places. But the data could help Foursquare become a larger “crowdsourced city guide,” “a social utility that intersects with the real world,” or just a “stats engine.” That’s what his data scientist will help him sort out.

Roger Ehrenberg, founder of IA Ventures which focuses on big data startups, said the promise of all this data is creating highly personal experiences for consumers, anticipating what they want and providing it to them. He said successful companies will lean on data scientists to start delivering contextual and relevant information to users, without them having to ask for it.

“I think one of the things underlying this theme is the issue of a user’s ability to be passive but have an experience created for them on the basis of this data,” he said. “Companies will look at what’s happening, who’s around you and will intuit by time of day and your interests and will serve up contextual offers and content.”

Now to be sure, many companies won’t need a data scientist right away. Most just don’t deal with the kinds of volume that necessitates such a hire. But make no mistake, this is where the world is going. With the price of processing, storage and broadband dropping and data tools emerging, it’s easier than ever for companies to wade into big data. And as they do, they’ll need more scientists to make sense of it all and show them the way.

Related content from GigaOM Pro (subscription req’d):

Thumbnail image courtesy of Flickr user bettina n

You're subscribed! If you like, you can update your settings

Related stories

  1. Only most companies/people don’t even understand the relation between a point,data and information nor what it is.
    Hence what they build is point sorters.

    Share
  2. [...] On GigaOm, Ryan Kim says that "Data Scientists" are the latest key positions in start-ups as big data swarms/enables the consumer and the enterprise. A Bit.ly data scientist defines the role: "Someone who can obtain, scrub, explore, model and interpret data, blending hacking, statistics and machine learning. It’s a set of skills that go beyond many existing job titles and it’s increasingly in demand." Read more. [...]

    Share
  3. this is just another geek merit badge. over the past fifteen years i’ve watched a dozen or so efforts in which postdocs and research wonks proclaim that they’ll use support vectors or kernel machines or [insert new NLP jargon here] to provide some stunning insight…but after a year of coding you end up with obvious conclusions that you could derive in ten minutes by just talking to users.

    Share
  4. LOL :D

    Ever heard of GIGO?

    LMFAO!

    ;) nmw

    Share
  5. data doesn’t lie

    a great data analyst can easily find opportunities that wouldnt have been apparent before …

    it sure beats the old way of some exec getting some brilliant (according to him) idea and throwing money at it

    as long as the people in charge understand how numbers work and that correlation is not causation … data is a companies most valuable resource right after their customers and employees

    banks, telecos, retailers, search engines, or anyone with any real data have been exploiting this for years if not decades

    the other related issue which execs dont usually get is experimentation … assumptions need to be validated and causation proven … most execs dont even get the need for control groups

    Share
  6. Data is truly important to many of these business, especially people like foursquare and twitter. they have created completely new streams of data that can be mixed matched, blended and twisted to find extraordinary insight into almost any topic. The biggest issue is finding people that have novel ideas on how to use this data, and eventually monetize it in the proper way for these companies.
    Garbage in garbage out is a standard problem with many of the current data providers, no data is perfect, but everyone pretends it is. the great thing about foursquare and twitter is that the community is constantly iterating on the data, they know it isn’t perfect, but its good enough, and its right now, which is huge!

    Share
  7. Good observations of the trend, Ryan. Data and analytics are a big thing now because of the potential of enriched information that can be extracted from the new data sources. Just look at the recent acquisitions by IBM.

    As a company which is focused on extracting business value from social data, we live and die by this belief!

    Babar
    Founder, MutualMind.

    Share
  8. [...] gigs that materialized out of the ether nearly three years ago. Data scientists are officially the hot new hire of choice even though their particular mix of formal skills is still rare.Now, community managers will not [...]

    Share
  9. [...] gigs that materialized out of the ether nearly three years ago. Data scientists are officially the hot new hire of choice even though their particular mix of formal skills is still [...]

    Share
  10. [...] Data Scientists – As more and more data is made available, people are needed to make sense of it. Companies such as bit.ly, LinkedIn and Foursquare are hiring. If I was going back to school, this is a career that I would target. Cancel reply [...]

    Share

Comments have been disabled for this post