Insufficient bandwidth has the potential to limit the cloud because it can take a long time to send large files over thin pipes. Amazon Web Services now addresses this problem with a new data delivery service called AWS Import/Export, which uses the postal system rather than the Internet to deliver data. Yes, it means customers now have the option to send their data to Amazon’s cloud via the mail.
Werner Vogels, Amazon’s CTO, explains in a blog posting that it would take up to 13 days to sling a terabyte of data across a 10 Mbps network, which is pretty darn slow. So Amazon is offering customers the chance to store their data on an external device, ship it via post, and Amazon will load it into S3. I outlined this problem of needing fat pipes to transfer our increasing loads of data back in April, but was hoping that instead of using FedEx, we’d have faster networks. Interestingly, Vogels doesn’t think our networks will keep up with our data generation — a feeling common also in the supercomputing and cloud storage world. Vogels writes:
No matter how much we have improved our network throughput in the past 10 years, our datasets have grown faster, and this is likely to be a pattern that will only accelerate in the coming years. While network may improve another other of magnitude in throughput, it is certain that data sets will grow two or more orders of magnitude in the same period of time.
I don’t want to believe that faster networks are a lost cause, or that we’ll have to resort to the mail for delivery to the cloud, but as a stopgap measure this offering says a lot about the state of our pipes. For reference, here’s Amazon’s cheat sheet for when it will be faster to turn your packets into a package.