Comments on: Systems to handle big data might be this generation’s moon landing http://gigaom.com/2012/04/01/systems-to-handle-big-data-might-be-this-generations-moon-landing/ Mon, 20 May 2013 13:46:43 +0000 hourly 1 http://wordpress.com/ By: Christopher Glenn http://gigaom.com/2012/04/01/systems-to-handle-big-data-might-be-this-generations-moon-landing/#comment-826242 Wed, 04 Apr 2012 17:11:17 +0000 http://gigaom.com/?p=505931#comment-826242 Good fodder. Thanks for the post. http://bit.ly/HeYEJq

]]> By: Joe http://gigaom.com/2012/04/01/systems-to-handle-big-data-might-be-this-generations-moon-landing/#comment-825471 Mon, 02 Apr 2012 14:59:05 +0000 http://gigaom.com/?p=505931#comment-825471 “It will generate data like a corpse breeds maggots …”
This really should have read “It will generate noise like a corpse breeds maggots …”. This is what happens when you approach an infrastructure vendor with a problem that may benefit from their infrastructure – they go over the top! These systems do NOT need so much data being processed centrally, because the vast majority of it is noise, or simply the same signal re-sampled countless times. Requiring this much data capacity to, in effect, take a high resolution picture indicates the system has not been designed carefully from ground up. Surely there are ways around it, like performing more computation at each individual node (think a massive distributed cluster), other than this brute force approach?

]]> By: Stacey Higginbotham http://gigaom.com/2012/04/01/systems-to-handle-big-data-might-be-this-generations-moon-landing/#comment-825430 Mon, 02 Apr 2012 12:54:06 +0000 http://gigaom.com/?p=505931#comment-825430 It was late. I had no editor :)

]]> By: Rich http://gigaom.com/2012/04/01/systems-to-handle-big-data-might-be-this-generations-moon-landing/#comment-825379 Mon, 02 Apr 2012 07:09:35 +0000 http://gigaom.com/?p=505931#comment-825379 “It will generate data like a corpse breeds maggots.”

Nice comparison, Stacey.

]]> By: JCW Press http://gigaom.com/2012/04/01/systems-to-handle-big-data-might-be-this-generations-moon-landing/#comment-825349 Mon, 02 Apr 2012 04:09:59 +0000 http://gigaom.com/?p=505931#comment-825349 By 2024 it’s reasonable to think that an exabyte of storage and the related network capacity to transmit will be within reason and economical. While it’s difficult to state now that it could be trivial, moors law and the related exponential growth in transmission and storage capacity has been substantial. Today we have services that store and transmit dozens if not hundreds of petabytes. Simply look at where we were 10 years ago. 2024 is a long way off and my sense is that we will more than make the progress needed to support goals like this.

]]>