Comments on: Why the days are numbered for Hadoop as we know it

By: igorcarron

igorcarron — Thu, 16 Aug 2012 15:08:50 +0000

You forgot GraphLab, they just had a workshop (slides are here: http://graphlab.org/workshop2012/agenda/ ). Of interest within GraphLab is GraphChi (http://bickson.blogspot.com/2012/07/new-dog-on-block-graphchi.html)

By: grauw

grauw — Tue, 17 Jul 2012 11:42:33 +0000

I think that doesn’t do justice to open source. It is not merely the low-cost inferior alternative. The goal to aim for is to be better than proprietary solutions, or equally good at the least, and if it is not then there is work to be done! They are equals and I do not see why you could not compare them.

By: grauw

grauw — Tue, 17 Jul 2012 11:39:50 +0000

By: Ed H. Chi

Ed H. Chi — Sun, 15 Jul 2012 18:28:20 +0000

Link to per olator is broken. Should be
http://research.google.com/pubs/pub36726.html

By: Bradford S.

Bradford S. — Fri, 13 Jul 2012 20:54:13 +0000

The big thing missing from the Hadoop ecosystem is a database. One key component that Google developed after MapReduce was BigTable, which scaled but had little functionality beyond KV-access and scans. Then they built MegaStore, which added indexing and better serialization. They recently release a paper on F1, which is a SQL database built on BigTable: http://research.google.com/pubs/pub38125.html

Likewise, Drawn to Scale has built a SQL database on HBase, which is getting a ton of adoption even at the early stage: http://www.drawntoscale.com

By: alinnemet

alinnemet — Thu, 12 Jul 2012 13:55:56 +0000

good point man ;)

By: amitpandey

amitpandey — Wed, 11 Jul 2012 11:25:56 +0000

Hadoop is unviable and companies such as Cloudera would make a killing selling storage systems. I can’t imagine you would always require 3 times the hardware. So for a data of 10 PB you would require 30 PBs of clusters and that would grow linearly. I fail to understand when people talk about commodity systems, having so many clusters, network wires, Store rooms etc.

It clearly calls for a more robust way to handle really large data rather than using Hadoop.

By: Sankar Nagarajan

Sankar Nagarajan — Wed, 11 Jul 2012 08:26:11 +0000

Good point. The industry will need solutions similar to what Google has built and there is a good scope for ISVs , opensource and commercial alike..

By: Sankar Nagarajan

Sankar Nagarajan — Wed, 11 Jul 2012 08:20:07 +0000

Timely article when there is a lot of hype that Hadoop is the panacea for all types of big data problems

By: rem120

rem120 — Wed, 11 Jul 2012 03:09:58 +0000

“Yes, Hadoop is developing into an ecosystem of projects that touch nearly all parts of data management and processing. But, at its core, it is a MapReduce system. Your code is turned into map and reduce jobs, and Hadoop runs those jobs for you.”

This is not accurate, you do not have to use MapReduce to use Hadoop.

- HBase is a very popular low-latency database on top of Hadoop that does not use MapReduce as part of its normal operation.
- Applications built with ZooKeeper co-ordination do not necessarily require MapReduce.
- YARN in Hadoop 2.0 allows non-MapReduce jobs to be executed on the cluster, eg. MPI. (http://wiki.apache.org/hadoop/PoweredByYarn)

I do agree that Hadoop would benefit from an OLAP-like solution similar to Dremel.