Comments on: What it really means when someone says ‘Hadoop’

By: tomaskuzar

tomaskuzar — Mon, 20 Feb 2012 12:55:18 +0000

Yes, the overview is great, but do you know some overview of use cases of for Hadoop implementation for mid-size business? Thank you.

By: Nick Trendov

Nick Trendov — Fri, 17 Feb 2012 12:15:36 +0000

Nice overview.

Hadoop and Risk
Hadoop appears to follow a. Familiar and successful silicon valley pattern that started with the creation, promotion and sales of databases, ERP systems, Yahoo-like portals, Google-like advertising engines, CRM applications and most recently Social Media communities like Twitter and Facebook.

In each case competing vendors created their own products and educated the market about how their offerings were different and better.

Now vendors are smarter, they first say how they use Hadoop as a critical value element and then they differentiate by highlighting their unique value.

Hadoop is more than a product, it has become a cue for success and Risk mitigation.

Hadoop is a successful product and in Risky times everyone wants a guarantee.

Hadoop is a ‘success cue’ for companies with the resources to learn how and where to fit it in to their business model.

Odd, it seems that companies flock to Hadoop to avoid Risk but take on new Risk in learning, building and maintaining.

This behaviour pattern shows the power of well placed cues, and why cues have to be found and measured by sellers and buyers to achieve the outcome that they need rather than wahts good for someone else

Cheers,
Nick @ManyCUES

By: Washroom

Washroom — Fri, 10 Feb 2012 19:22:01 +0000

Oh, this is great. I had no idea what Hadoop was before this. Really good information.

By: Mark Samson

Mark Samson — Fri, 10 Feb 2012 11:35:28 +0000

IBM InfoSphere BigInsights actually does include IBM’s own distribution of Hadoop, with multiple proprietary extensions in analytics, usability and operational improvements.

By: sematext

sematext — Fri, 10 Feb 2012 06:21:25 +0000

Derrick, have a look at this recent LinkedIn Poll created by Otis Gospodnetic from Sematext on exactly this topic – “When you say Hadoop, which of these do you mean?” – http://www.linkedin.com/groupAnswers?viewQuestionAndAnswers=&discussionID=88676755&gid=988957

By: Keith

Keith — Wed, 08 Feb 2012 14:46:04 +0000

I hope all this stuff works, otherwise we have all been Hadoop, in the Yorkshire sense of the word!

By: Chris

Chris — Tue, 07 Feb 2012 17:31:09 +0000

Excellent overview!

By: Jubal Ince

Jubal Ince — Tue, 07 Feb 2012 17:03:27 +0000

Good Hadoop overview, thanks! On a side note, Microsoft is hiring Big Data engineers. Email or @ me if interested: jubaliATmicrosoft.com or @jubal_ince.

By: Paul K

Paul K — Tue, 07 Feb 2012 16:45:04 +0000

Few Corrections: Pig is a job control language while Hive is a SQL APL (HiveQL). HBase is a Columnar datbase built on HDFS.

By: Fred Holahan

Fred Holahan — Tue, 07 Feb 2012 14:38:34 +0000

Derek, your Hadoop taxonomy seems more informed and accurate than Forrester’s. What’s often lost in the Hadoop discussion is the question of how data gets into a Hadoop infrastructure in the first place. A case in point – the Karmasphere infographic shows example data sources (the blue boxes) and Hadoop (the yellow box). If we’re to believe the diagram, between the blue and the yellow, a miracle happens.

To be clear, I’m not picking on Karmasphere. Their diagram serves a purpose and is necessarily simplistic. My point is that miracles don’t happen; organizations need to be incredibly thoughtful about what data they’re going to deliver to Hadoop, where it’s coming from and how it’s connected.

Equally important is the reality that some data (e.g., from sensor scans, wireless apps, systems monitors, etc.) travels at such high velocities that it will quickly swamp a typical Hadoop/HDFS infrastructure. For high velocity “firehose” applications, the data tier needs a front-end cache that can ingest incoming data at very high speeds, manage that data statefully for real-time analytics, and deliver it to Hadoop in a controlled way. Products like VoltDB (I work for the company) offer an excellent solution for managing the “impedance mismatch” between high velocity data sources and high volume analytic infrustructures like Hadoop.