Comments on: Because Hadoop isn’t perfect: 8 ways to replace HDFS

By: Cameron

Cameron — Fri, 17 Aug 2012 01:19:04 +0000

The ParaScale distributed storage and computation platform perhaps deserves a mention here too as one of the early vendors that integrated Hadoop into the platform and submitted the filesystem plugin to the open source community back in 2009/2010 time frame. The ParaScale filesystem provided fast parallel ingest via standard NFS protocol, full read/write POSIX semantics, a distributed and replicated object store, global namespace, and many other enterprise features. Hitachi Data Systems acquired ParaScale in 2010.

References:
http://en.wikipedia.org/wiki/Apache_Hadoop
https://issues.apache.org/jira/browse/HADOOP-6704

By: Derrick Harris

Derrick Harris — Sun, 29 Jul 2012 18:32:07 +0000

I think it depends on your use case. If you’re a Fortune 500 company, you might want to pay for performance, reliability, etc. And some alternative, such as Gluster (mentioned in comments), are open source but vendor-backed.

By: gengstrand

gengstrand — Sun, 29 Jul 2012 17:35:25 +0000

I think that it is disingenuous to propose expensive proprietary solutions as alternatives to an open source solution. Perhaps a more valuable article would be to propose open source alternatives instead. At Zoosk, we used http://www.dynamicalsoftware.com/nosql/solr instead of Hadoop for the implementation of our activity feed. The Apache Solr project is our open source alternative to Hadoop.

By: Derrick Harris

Derrick Harris — Thu, 26 Jul 2012 14:40:57 +0000

I saw your post, and Charles’ (from Cloudera) post. I think you’re both probably right that HDFS wins out in the end. By percentage, it’s probably very dominant now. But there will always be people looking for alternatives.

By: Toast to HDFS

Toast to HDFS — Thu, 26 Jul 2012 10:16:50 +0000

We wanted to make a sandwich/toast with HDFS, but could not. We had to design our own toaster. So is this a problem with HDFS?

By: Jay

Jay — Thu, 26 Jul 2012 06:38:41 +0000

Looks like most of the alternatives being promoted are highly expensive proprietary hardware based storage vendors who are feeling threatened by the RISE OF HDFS and crying wolf. Everyone knows that HDFS has its flaws but the benefits outweigh the drawbacks. It has got huge traction and in a few years will be the most dominant technology for data crunching. It is important for the storage vendors to re-invent themselves and not fight for a piece of the hadoop pie. That will mean slow decay for them.

By: eric baldeschwieler

eric baldeschwieler — Wed, 25 Jul 2012 22:56:35 +0000

Hi Derrick,

I’ve blogged my thoughts on HDFS vs other systems on the Hortonworks website.

http://hortonworks.com/blog/thinking-about-the-hdfs-vs-other-storage-technologies/

E14 – CTO Hortonworks & hadoop architectural contributor

By: scarpe messi 2012

scarpe messi 2012 — Tue, 17 Jul 2012 03:51:50 +0000

Dispersed Storage Network

By: Bill McColl

Bill McColl — Thu, 12 Jul 2012 22:16:56 +0000

Good post. At Cloudscale we tried using HDFS a while back. Given that we wanted to run realtime analytics, graph analytics, MPI, and bulk synchronous parallelism (BSP), as well as Hadoop, in a unified big data architecture, HDFS was a non-starter. For our purposes, Lustre has proved to be a far better option as a super-fast and super-scalable file system foundation for a unified big data platform. Running Lustre on AWS clusters is new. We are probably the only company currently doing it. To help others get started, we posted how to do it here

http://www.cloudscale.com/index.php/technology/lustre-on-aws-cloud

if you want to try out Lustre on cloud clusters for yourself.

By: Dave Mackey

Dave Mackey — Thu, 12 Jul 2012 16:28:07 +0000

Great list of alternatives. I wonder how Microsoft’s Storage Spaces will compete?