Commentüs on: Out of Cloud Chaos Comes Structure http://gigaom.com/2008/06/30/out-of-cloud-chaos-comes-structure/ The Business of Technology Fri, 05 Dec 2008 18:00:30 +0000 http://wordpress.org/?v=MU hourly 1 By: Rahul Dave http://gigaom.com/2008/06/30/out-of-cloud-chaos-comes-structure/#comment-886452 Rahul Dave Tue, 01 Jul 2008 02:00:13 +0000 http://gigaom.com/?p=13998#comment-886452 For folks like me, working in Cloud oriented aspects of science, these issues have been on the radar for a while. In astronomy, we expect 10-100TB/night data rates very soon. While we can compress by using at telescope-cluster computation the data by an order of magnitude, this still leaves us with 10TB/night or so to be analyzed/searched/stored and on which astronomical pipelines must run. What does this mean? It means that we must (a) deal with the financial aspects of poor scientists having to pay for bandwidth from remote telescope sites to data centers (b) deal with the programming aspects of situating computation at the data rather than the traditional paradigm of situating data at the computation (like the scientists desktop) (c) wholeheartedly use web services to make the composition of pipelines across multiple data centers (and thus computational centers) possible. (d) deal with huge data redundancy at low cost issues It should be a fun time. Some progress has been made on grid models of computation...the 'run your programs where you can get cycles' model..but it is the data oriented cloud computing notion that is more important..computation is now cheap but must be localized to the data. The web 2.0 paradigms, cloud computing, and experimental science are meeting in a big way! For folks like me, working in Cloud oriented aspects of science, these issues have been on the radar for a while. In astronomy, we expect 10-100TB/night data rates very soon. While we can compress by using at telescope-cluster computation the data by an order of magnitude, this still leaves us with 10TB/night or so to be analyzed/searched/stored and on which astronomical pipelines must run. What does this mean?

It means that we must (a) deal with the financial aspects of poor scientists having to pay for bandwidth from remote telescope sites to data centers (b) deal with the programming aspects of situating computation at the data rather than the traditional paradigm of situating data at the computation (like the scientists desktop) (c) wholeheartedly use web services to make the composition of pipelines across multiple data centers (and thus computational centers) possible. (d) deal with huge data redundancy at low cost issues

It should be a fun time. Some progress has been made on grid models of computation…the ‘run your programs where you can get cycles’ model..but it is the data oriented cloud computing notion that is more important..computation is now cheap but must be localized to the data.

The web 2.0 paradigms, cloud computing, and experimental science are meeting in a big way!

]]>