5 ways big data is going to blow your mind and change your world


Some people say big data is wallowing in the trough of disillusionment, but that’s a limited worldview. If you only look at it like an IT issue it might be easy to see big data as little more than business intelligence on steroids. If you only see data science as a means to serving better ads, it might be easy to ask yourself what all the fuss is about.

If you’re like me, though, all you see are the bright lights ahead. They might be some sort of data nirvana, or they might be a privacy-destroying 18-wheeler bearing down on us. They might be both. But we’re going to find out, and we’re we’re going to find out sooner rather than later.

This is because there are small pockets of technologists who are letting their imaginations lead the way. In a suddenly cliché way of saying it, they’re aiming for 10x improvement rather than 10 percent improvement. They can do that because they now have a base set of analytic technologies and techniques that are well positioned to solve, with relatively little effort, whatever data problems are thrown their way.

Here are some themes from our just-concluded Structure: Data conference that I think highlight the promise of data, but also the challenges that lie ahead.

Man and machine unite

Machine learning is already infiltrating nearly every aspect of our digital lives, but its ultimate promise will only be realized when it becomes more human. That doesn’t necessarily mean making machines think like human brains (although, granted, that’s a vision currently driving billions of research dollars), but just letting people better interact with the systems and models trying to discover the hidden patterns in everything around us.

Whatever shape it takes, the results will be revolutionary. We’ll treat diseases once thought untreatable, tackle difficult socio-economic and cultural issues, and learn to experience the world around in entirely new ways. Maybe that consumer-experience scourge known as advertising might actually become helpful rather than annoying.

That would really be something.

Data science, or data intelligence?

I’m not sure there needs to be a distinction between data science and data intelligence, but the latter does connote a grander goal. It’s about trying to solve meaningful problems rather than just serving ads; about trying to understand why things happen just as well as when they’ll happen. This means learning to work with smaller, messier data than we might like — certainly smaller and messier than the data sets underneath most of the massive web-company data science undertakings.

But just think about being able to go beyond predictive models and into a world of preventative — or even professorial — models. If you know what I like, where I go and who my friends are, it might be fairly easy to predict what I want to buy. Figuring out how my decision to buy something might affect my overall well-being and then telling me why? That’s a little more difficult and a lot more beneficial.

Telling stories with data

Have you ever looked at a chart and wondered what the heck it was supposed to be telling you? Or downloaded a report of your Facebook activity only to ask yourself if all the disparate data points come together to paint a bigger picture? Or tried — and failed — to stop a terrorist before his movement to recruit an army of followers gained critical mass?

A big problem with a lot data analysis right now is that it still treats data points as entities unto themselves, largely disconnected from those around them. However, data needs context in order to be really useful; it’s context that turns disparate data points into a story. Don’t just tell me how many steps I took today or the time of day I’m most active on Facebook, but tell me how that relates to the rest of my life.

And don’t just tell me that someone said he wants to kill Americans. Rather, tell me a story about how much more frequently he’s saying it and how much more inciteful his words are becoming.

The internet of things knows all

The mobile phone in your pocket is tracking your every movement and can also monitor the sounds that are surrounding you. That fitness tracker you’re wearing is identifying you by how you walk. Your smart meter data shows when you’re home, when you’re away and when you’re in the shower. Sensors in everything from toothbrushes to cars are quantifying every aspect of our lives.

This volume of data can still be a lot to deal with in terms of its volume, velocity and variety, and we’re still not quite sure what to do with it even if the right tools were in place. But all sorts of entrepreneurs, powerful institutions and intelligence agents have ideas. The technological pieces are coming along nicely, too. Just sayin’ …

This semantic life

The semantic web lives on; only it’s spreading well beyond our search engines and even our web browsers. Soon enough, we’ll be able to surface relevant content and people simply by highlighting passage of text in whatever we’re reading — web page or not — on any type of device. When we speak to our devices, they’ll not only know what we’re saying, but also what we really want even without the help of specific commands or keywords.

That’s a powerful proposition in a world where we increasingly expect our interactions to be hands-free and our answers to come as fast as our questions. Of course, what’s powerful in the hands of consumers driving in their cars or sitting on their couches is even more powerful in the hands of doctors trying to diagnose difficult diseases or aid workers trying lend a helping hand in places where they don’t know the customs or even speak the language.

Feature image courtesy of Shutterstock user GrandeDuc.


Tina Groves

The kinds of questions asked at GigaOM this year illustrated a maturing of the technology. Far fewer questions about ‘what is Big Data’ and far more about ‘what is possible’ were asked. Active discussions could be heard around the Exhibition Hall about the potential benefits and business models to monetize these new ideas.

A lot of work goes into analyzing and matching data originating from multiple applications but many people expect the end result to be be “free”. Once a viable business model is established, watch out for Big Data to go mainstream.

Ram Babu SEO Expert

thanks derrick, yes it is either not an easy task to collect or figure out the importance and effectiveness of such big data getting from any specific industry we market in.

Jack Rivkin

Very good summary and some nice original thoughts. The important story about Big Data has little to do with social media and more to do with health and well being IMHO. In Abu Dhabi and missed the conference. See http://bit.ly/YCd7DE “What is the Big Deal About Big Data?” Not what you might think…

Steve Ardire

Very good post Derrick with the 5 key themes and a ‘glueware theme’ is sentient computing with automation of cognitive and anticipatory thinking.

Shashank Garg

Very good article Derrick ..absolutely spot on when said data needes to tell story and semantic analysis is key than just numbers and last but not least we are heading to singularity when it comes to data and machine … Dearsrk.wordpress.com


The mobile phone in your pocket is tracking your every movement and can also monitor the sounds that are surrounding you. That fitness tracker you’re wearing is identifying you by how you walk. Your smart meter data shows when you’re home, when you’re away and when you’re in the shower. Sensors in everything from toothbrushes to cars are quantifying every aspect of our lives.

Really? Without my permission for access to my GPS info?


Of course, your carrier (service provider) will know where your phone is connected and know its location. They don’t need to ask to get that information. Your phone is connected, they have your location.

Comments are closed.