Comments on: Why big data might be more about automation than insights

By: Apptegic

Apptegic — Wed, 09 Jan 2013 15:09:59 +0000

Great article Derrick. I think the term “big data” creates a lot of confusion because of the ambiguity. Humans are still the critical element in the decision making process and it’s technologies that enable us to perform better that really make the difference. I’m excited to see big data and machine learning continue to evolve from an overflow of raw data to actionable intelligence.
- @MrRyanConnors

By: Achille

Achille — Fri, 04 Jan 2013 19:38:33 +0000

I do not agree with most of you. I don’t find it a so great article. It is pretty straight forward that in first place the incentative to analyze data is for someone to make a decision or better informed decision at the end. This being said, the article is auto-feeding itself with this reasoning saying the analysis methods aren’t making any decisions by themselves because they are not making the final decision.

Well, yes, there is a lot of marketing bullshit out there, but this was always the case with any product since the human kind is clever enough to communicate. And what is the whole point about this article? Selling us a conference in March sponsored by some of these bullshitters.

By: Sam Mikkelsen

Sam Mikkelsen — Thu, 03 Jan 2013 10:28:05 +0000

Great article Derrick – Our system helps advertisers analyse their data and see which channels have performed. A lot of human interaction is needed to take findings from the results but we automate the hard and frankly boring part of the job of collating and analysing the data and visualising the results!!

By: Chris Taylor

Chris Taylor — Thu, 03 Jan 2013 07:20:05 +0000

Derrick, this is an excellent piece and overdue. There’s a certain amount of hazy enthusiasm about big data that hides the fact that it still comes down to humans having to make great decisions. We’re automating the feeding of those decisions but not necessarily changing the outcome (unless the outcome is a faster decision based on more information).

More or less in response to the hype, we wrote up what we’ve nicknamed our Big Data Manifesto: http://successfulworkplace.com/2012/10/28/big-data-must-not-be-an-elephant-riding-a-bicycle/

This article focuses on the wide variety of human decisions and complementary automation that makes big data work. Comments welcome.

By: Drake Pruitt

Drake Pruitt — Thu, 03 Jan 2013 06:07:36 +0000

Great post, and I fully agree. Machine learning (both supervised and unsupervised) is now practical for big data applications, thanks to the decline in storage, compute and bandwidth costs. Like factory automation, ‘machines’ can be taught the skills of humans for repetitive tasks (pattern recognition, for example). The business analyst/data scientists can focus on higher order questions as a result.

By: Derrick Harris

Derrick Harris — Thu, 03 Jan 2013 05:12:50 +0000

Good points. I actually wasn’t considering IoT or M2M, except of course that people might want to work device data into their analyses or algorithms. I was thinking about humans as the intended consumers of analysis, and automation in the manufacturing sense.

By: Steve Ardire

Steve Ardire — Thu, 03 Jan 2013 03:26:58 +0000

Depends on how you define Big Data and if you include M2M and IoT yes indeed it’s more about automation

Big Data: Rise of the Machines http://nyti.ms/Vsnj3R

Machine-generated sensor data will be become a far larger portion of the Big Data world, according to a recent report by IDC. The IDC forecast also suggests that there is a lot of substance to the vision of machine-to-machine communication and intelligence that W. Brian Arthur terms “the second economy.”

> Whether it’s corporate business intelligence or the consumer web, though, all of this is about automation. Data-minded people have always used data to aid in decision-making without ignoring their instincts. Big data just lets them learn a lot more, a lot faster.

Your point in discussed here with lots of comments

We don’t need more data scientists — just make big data easier to use http://gigaom.com/2012/12/22/we-dont-need-more-data-scientists-just-simpler-ways-to-use-big-data/