4 Comments

Summary:

So the big data backlash begins. The Hadoop framework does a lot, but some experts — including those who push non-Hadoop options — say it’s not enough for many specialized apps where a build-your-own Hadoop implementation costs too much to be a real contender.

289861766_8d34a5d1d9_z

And so the backlash begins. Hadoop, the open-source framework for handling tons of distributed data, does a lot, and it is a big draw for businesses wanting to leverage the data they create and that is created about them. That means it’s a hot button as well for the IT vendors who want to capture those customers. Virtually every tech vendor from EMC to Oracle to Microsoft has announced a Hadoop-oriented “big data” strategy in the past few months.

But here comes the pushback. Amid the hype, some vendors are starting to point out that building and maintaining a Hadoop cluster is complicated and — given demand for Hadoop expertise — expensive. Larry Feinsmith, the managing director of JPMorgan Chase’s office of the CIO, told Hadoop World 2011 attendees recently that Chase pays a 10 percent premium for Hadoop expertise — a differential that others said may be low.

Manufacturing, which typically generates a ton of relational and nonrelational data from ERP and inventory systems, the manufacturing operations themselves, and product life cycle management, is a perfect use case for big data collection and analytics. But not all manufacturers are necessarily jumping into Hadoop.

General Electric’s Intelligent Platforms Division, which builds software for monitoring and collecting all sorts of data from complex manufacturing operations, is pushing its new Proficy Historian 4.5 software as a quicker, more robust way to do what Hadoop promises to do.

“We have an out-of-the-box solution that is performance comparable to a Hadoop environment but without that cost and complexity. The amount of money it takes to implement Hadoop and hire Hadoop talent is very high,” said Brian Courtney, the GM of enterprise data management for GE.

Proficy Historian handles relational and nonrelational data from product manufacturing and testing — data like the waveforms generated in the process. GE has a lot of historical data about what happens in the production and test phases of building such gear, and it could be put to good use, anticipating problems that can occur down the pike.

For example, the software can look at the electrical signature generated when a gas turbine starts up, said Courtney. “The question is what is the digital signature of that load in normal startup mode and then what happens if there’s an anomaly? Have you ever seen that anomaly before? Given this waveform, you can go back five years and look at other anomalies and whether they were part of a subsequent system failure.”

If similar anomalies caused a system failure, you can examine how much time it took after the anomaly for the failure to happen. That kind of data lets the manufacturer prioritize fixes.

The new release of Proficy software seeks to handle bigger amounts of big data, supporting up to 15 million tags, up from two million in the previous release.

Joe Coyle, the CTO of Capgemini, the big systems integrator and consulting company, said big data is here to stay but that many businesses aren’t necessarily clued in to what that means. “After cloud, big data is question number two I get from customers. CIOs will call and say, ‘Big data, I need it. Now what is it?’”

Coyle agrees that Hadoop has big promise but is not quite ready for prime time. “It’s expensive and some of the tools aren’t there yet. It needs better analytics reporting engines. Right now, you really need to know what to analyze. Hadoop brings a ton of data, but until you know what to ask about it, it’s pretty much garbage in, garbage out.”

The companies who make the best use of big data are those that know what to ask about it.

For example, Coyle explained, “Victoria’s Secret harvests information from Facebook and can tell you in detail about every 24-year-old who bought this product in the last 12 months. It’s very powerful, but that’s because of the humans driving it.”

Elephant photo courtesy of Flickr user sarahemcc

Waveform photo courtesy of Flickr user altemark

  1. Hadoop ain’t all that http://t.co/5IusYfOx #Cloudcomputing

    Share
  2. Hadoop ain’t all that http://t.co/Aqqvrw8b via Barb Darrow

    Share
  3. Hadoop ain’t all that: So the big data backlash begins. The Hadoop framework does a lot, but some experts –… http://t.co/TFVppBSz

    Share
  4. Hadoop ain’t all that http://t.co/O4K1Uexo @GigaOM

    Share
  5. Hadoop ain’t all that http://t.co/jMi1vxWS #cloud #gigaom

    Share
  6. Hadoop ain’t all that http://t.co/xXbqJMxK

    Share
  7. Hadoop ain’t all that http://t.co/kzEMhvb7

    Share
  8. Hadoop ain’t all that http://t.co/AY2oGYPj

    Share
  9. Hadoop ain’t all that http://t.co/TaMKMGdW

    Share
  10. Hadoop ain’t all that http://t.co/nkFDbjdO

    Share
  11. gigaom: Hadoop ain’t all that http://t.co/KRyrLy7L

    Share
  12. Hadoop ain’t all that http://t.co/nkc4bNwN

    Share
  13. RT @gigaom: Hadoop ain’t all that http://t.co/vRyqpsak

    Share
  14. [GigaOM] Hadoop ain’t all that: So the big data backlash begins. The Hadoop framework does a lot, but some exper… http://t.co/tirzx7OM

    Share
  15. Hadoop ain’t all that http://t.co/8KtkLYUH

    Share
  16. Hadoop ain’t all that http://t.co/ToNmUylQ @amarchugg #news

    Share
  17. Hadoop ain’t all that http://t.co/ETgite7U @amarchugg #news

    Share
  18. Hadoop ain’t all that: So the big data backlash begins. The Hadoop framework does a lot, but some experts — inc… http://t.co/2DFBJVwz

    Share
  19. Hadoop ain’t all that: So the big data backlash begins. The Hadoop framework does a lot, but some experts — inc… http://t.co/9W10RWRk

    Share
  20. Hadoop ain’t all that: So the big data backlash begins. The Hadoop framework does a lot, but some experts — inc… http://t.co/KyCaUikj

    Share
  21. Hadoop ain’t all that: So the big data backlash begins. The Hadoop framework does a lot, but some experts — inc… http://t.co/RebqNtww

    Share
  22. Hadoop ain’t all that: So the big data backlash begins. The Hadoop framework does a lot, but some experts — inc… http://t.co/ayW6xXND

    Share
  23. #cloud Hadoop ain’t all that – So the big data backlash begins. The Hadoop framework does a lot, but some experts –… http://t.co/wB9qQRPp

    Share
  24. RT @gigaom: Hadoop ain’t all that http://t.co/nkivcTJF (KD – this will certainly get more than a handful of responses! Thoughts?) #hadoop

    Share
  25. Hadoop ain’t all that /by @gigabarb http://t.co/WIsfX9TG. Well its that old chessnut idiots at the top of big corps..

    Share
  26. Hadoop ain’t all that: And so the backlash begins. Hadoop, the open-source framework for… http://t.co/RtlRYLry

    Share
  27. Hadoop ain’t all that: So the big data backlash begins. The Hadoop framework does a lot, but some experts — inc… http://t.co/KH5HQYNR

    Share
  28. Hadoop aint all that Cloud Computing News http://t.co/m2wWbplE

    Share
  29. Why some execs think Hadoop ain’t all that http://t.co/nkFDbjdO

    Share
  30. RT @gigabarb: Why some execs think Hadoop ain’t all that http://t.co/nkFDbjdO

    Share
  31. Hadoop ain’t all that http://t.co/TuBuSp6m

    Share
  32. Hadoop ain’t all that http://t.co/BvZNDjvb >more critical look at TCO of Hadoop

    Share
  33. Why some execs think Hadoop ain’t all that – So the big data backlash begins. The Hadoop framework does a lot, but s… http://t.co/x23PsNsS

    Share
  34. Why some execs think Hadoop ain’t all that – So the big data backlash begins. The Hadoop framework does a lot, but s… http://t.co/pobU2iO0

    Share
  35. MT @gigaom: Hadoop ain’t all that – companies saying it’s complicated and expensive #bigdata http://t.co/Oj6X6JUT

    Share
  36. Good News for Hadoop Developers http://t.co/sA9LJ5Fa via @gigaom

    Share
  37. It’s a great time to be a #Hadoop expert. http://t.co/BsSIrKOA #BigData

    Share
  38. Why some execs think Hadoop ain’t all that http://t.co/ZzVNzcgE

    Share
  39. Hadoop having a hard time. Not too surprising. http://t.co/oosDZlYW

    Share
  40. Great piece by @gigabarb on how @GE_IP is simplifying #BigData http://t.co/46pn5Owv

    Share
  41. RT @GE_IP: Great piece by @gigabarb on how @GE_IP is simplifying #BigData http://t.co/46pn5Owv

    Share
  42. Manufacturing will eventually leverage cheap Hadoop clusters but right now, Hadoop ain’t all that – http://t.co/5C013Jzm

    Share
  43. Why some execs think Hadoop ain’t all that: And so the backlash begins. Hadoop, the open-source framework for ha… http://t.co/7eJUGNgb

    Share
  44. Why some execs think Hadoop ain’t all that: And so the backlash begins. Hadoop, the open-source framework for ha… http://t.co/GgFoCt9L

    Share
  45. Why some execs think Hadoop ain’t all that http://t.co/kLxyJLN5 (via @gigaom)

    Share
  46. Why some execs think Hadoop ain’t all that: And so the backlash begins. Hadoop, the open-source framework for ha… http://t.co/wA4TpQbG

    Share
  47. the beginning of #hadoop decent into the “Trough of Disillusionment” – #splunk can help – http://t.co/i4N0EPrG #in

    Share
  48. RT @gigaom: Hadoop ain’t all that http://t.co/vRyqpsak

    Share
  49. Why some execs think Hadoop ain’t all that http://t.co/kRry0oEN

    Share
  50. Some Hadoop backlash starting? http://t.co/NaO96DAn

    Share
  51. Hadoop backlash? http://t.co/Xqms1jbC MR is valuable even if not Hadoop, lots of things that seem “big” aren’t. #bigdata #hadoop

    Share
  52. RT @aarondelp: Some Hadoop backlash starting? http://t.co/wPkxJTiF

    Share
  53. Why some execs think Hadoop ain’t all that http://t.co/BQxGxIEG

    Share
  54. RT @GE_IP: Great piece by @gigabarb on how @GE_IP is simplifying #BigData http://t.co/46pn5Owv

    Share
  55. RT @arosenbaum: Hadoop backlash? http://t.co/Xqms1jbC MR is valuable even if not Hadoop, lots of things that seem “big” aren’t. #bigdata #hadoop

    Share
  56. RT @lespider: the beginning of #hadoop decent into the “Trough of Disillusionment” – #splunk can help – http://t.co/i4N0EPrG #in

    Share
  57. Hadoop headache? @GigaOm says hadoop is hard – http://t.co/eGvpECQn. But there’s an app for that! (@StackIQ). #hadoop #bigdata

    Share
  58. RT @brianmesic: Hadoop headache? @GigaOm says hadoop is hard – http://t.co/eGvpECQn. But there’s an app for that! (@StackIQ). #hadoop #bigdata

    Share
  59. RT @timmcintire: RT @brianmesic: Hadoop headache? @GigaOm says hadoop is hard – http://t.co/GB4DApV6. But there’s an app for that! (@StackIQ). #hadoop

    Share
  60. Reading @gigabarb blog post: “Why some execs think Hadoop ain’t all that” – http://t.co/DJj5kK9z #BigData (via @LoraineLawson)

    Share
  61. Why some execs think Hadoop ain’t all that — Cloud Computing News http://t.co/EEFsraKQ

    Share
  62. Very insightful article Barb. It is also worth mentioning the HPCC System’s platform is a great fit for all data models. Unlike Hadoop distributions, HPCC is a mature platform and provides for a data delivery engine together with a data transformation and linking system equivalent to Hadoop. The main advantages over other alternatives are the real-time delivery of data queries and the extremely powerful ECL language programming model. Also, the ROI for HPCC is significantly better than Hadoop due to the fact that they require less nodes and less programmers. More information visit: http://hpccsystems.com

    Share
  63. reading Why some execs think Hadoop ain’t all that http://t.co/bPWzWXqY #bigdata

    Share
  64. Why some execs think Hadoop ain’t all that: http://t.co/iPWFn9kv

    Share
  65. Why some execs think Hadoop ain’t all that http://t.co/iCD5O3Ye

    Share
  66. Whaddaya think of this? – Why some execs think Hadoop ain’t all that http://t.co/ZLrSLxxu

    Share
  67. Why some execs think Hadoop ain’t all that http://t.co/Kg7neTWf #BigData

    Share
  68. Why some execs think Hadoop ain’t all that http://t.co/eviFJBRV #Bigdata

    Share
  69. Why some execs think Hadoop ain’t all that – http://t.co/R2btR9kT: “After cloud, big data is qu… http://t.co/M8Is8nJe #Cloud #Analytics

    Share
  70. Why some execs think Hadoop ain’t all that – http://t.co/m6cBrJCr: Joe Coyle, the CTO of Capgemin… http://t.co/9bPZDjBr #bigdata #cloud

    Share
  71. Mining data today is the same as was mining for any other commodity. It’ll take alot of hard work with the raw material before any golden nuggets are found with any long lasting or true valuem

    Share
  72. RT @Cloud_Analytics: Why some execs think Hadoop ain’t all that – http://t.co/R2btR9kT: “After cloud, big data is qu… http://t.co/M8Is8nJe #Cloud #Analytics

    Share
  73. If we believe that the future of video is to be driven by data, thoughts about #Hadoop bear some thought. http://t.co/xx4olfWa

    Share
  74. Why Some Execs Think Hadoop Ain’t All That. http://t.co/2IYD8eJW @hadoop

    Share
  75. RT @gigaom Hadoop ain’t all that http://t.co/yaUIrxwl

    Share
  76. the backlash begins…Why some execs think #Hadoop ain’t all that http://t.co/4arDxBXK #bigdata

    Share
  77. RT @williammcknight: the backlash begins…Why some execs think #Hadoop ain’t all that http://t.co/4arDxBXK #bigdata

    Share
  78. My talented colleagues at #JMP work w #bigdata like this in manufacturing. The value is tremendous to our customers. http://t.co/LgKMmTPv

    Share
  79. Why some execs think #Hadoop ain’t all that. #bigdata http://t.co/P3wgyDii

    Share
  80. Is building and managing Hadoop cluster complicated? http://t.co/djxSUlqc #Hadoop #BigData

    Share
  81. “Hadoop Ain’t all That” from @gigaom http://t.co/MgXv1YvQ could have seen that Trough coming, oh wait I did…http://tinyurl.com/42hvhme

    Share
  82. Is this a flea market where every merchant tout their own mellon, radish is better, more nutrient than the other guy?
    GE is a great Company. Intelligent Platform is may be 1/10 of 1 percent and perhaps 2000 employee? Should I go with a proprietary solution from such a player?

    Share
    1. It is all about making sense of the filter/processed data coming out. An more importantly, for man applications it is about visualizing it effectively.
      http://grandlogic.com

      Share
  83. Why some execs think Hadoop ain’t all that http://t.co/iYB0N0db Because they are stupid as shit!

    Share
  84. “CIOs will call and say, ‘Big data, I need it. Now what is it?’” http://t.co/BQxGxIEG Uh.Boy.

    Share
  85. First time I’ve seen data measured in tags. 15 million tags of data. http://t.co/BQxGxIEG

    Share
  86. Why some execs think Hadoop ain’t all that http://t.co/Jp0c45s7 #bigdata

    Share
  87. RT @LoraineLawson: “CIOs will call and say, ‘Big data, I need it. Now what is it?’” http://t.co/BQxGxIEG Uh.Boy.

    Share
  88. One problem with commercial software: Bandwagon Hopping – http://t.co/fpBt0Kvz

    Share
  89. Barb Darrow: Why some execs think Hadoop ain’t all that ::: http://t.co/WYQnUhGe :: #Hadoop #BigData

    Share
  90. RT @arosenbaum: Hadoop backlash? http://t.co/Xqms1jbC MR is valuable even if not Hadoop, lots of things that seem “big” aren’t. #bigdata #hadoop

    Share
  91. Why some execs think #Hadoop ain’t all that | via @Gigaom http://t.co/XGffURig > It’s a framework, not a solution. #analytics

    Share
  92. RT @Quantivo: Why some execs think #Hadoop ain’t all that | via @Gigaom http://t.co/XGffURig > It’s a framework, not a solution. #analytics

    Share
  93. Why some execs think Hadoop ain’t all that http://t.co/kQ61uOlx

    Share

Comments have been disabled for this post