Blog Post

Amazon's New Service Goes Postal Over Slow Broadband

logo_aws2Insufficient bandwidth has the potential to limit the cloud because it can take a long time to send large files over thin pipes. Amazon (s amzn) Web Services now addresses this problem with a new data delivery service called AWS Import/Export, which uses the postal system rather than the Internet to deliver data. Yes, it means customers now have the option to send their data to Amazon’s cloud via the mail.

Werner Vogels, Amazon’s CTO, explains in a blog posting that it would take up to 13 days to sling a terabyte of data across a 10 Mbps network, which is pretty darn slow. So Amazon is offering customers the chance to store their data on an external device, ship it via post, and Amazon will load it into S3. I outlined this problem of needing fat pipes to transfer our increasing loads of data back in April, but was hoping that instead of using FedEx, we’d have faster networks. Interestingly, Vogels doesn’t think our networks will keep up with our data generation — a feeling common also in the supercomputing and cloud storage world. Vogels writes:

No matter how much we have improved our network throughput in the past 10 years, our datasets have grown faster, and this is likely to be a pattern that will only accelerate in the coming years. While network may improve another other of magnitude in throughput, it is certain that data sets will grow two or more orders of magnitude in the same period of time.

I don’t want to believe that faster networks are a lost cause, or that we’ll have to resort to the mail for delivery to the cloud, but as a stopgap measure this offering says a lot about the state of our pipes. For reference, here’s Amazon’s cheat sheet for when it will be faster to  turn your packets into a package.

aws

40 Responses to “Amazon's New Service Goes Postal Over Slow Broadband”

    • Back when I got in this business the fastest was to get 40 megabytes cross country was a syquest cartridge and a fedex envelope.

      Now fedex is the fastest way to get a terabyte cross country.

    • Back when I got in this business the fastest way to get 40 megabytes cross country was a syquest cartridge and a fedex envelope.

      Now fedex is the fastest way to get a terabyte cross country.

  1. primate

    one thing to keep in mind is that at any network speed over ~1GbE (1000Mbps), the bottleneck is going to shift to writing the media.
    eg. for 1TB or higher transfer…
    Tmed = Tread = 1hr/500GB
    Tship = 16hours (FedEx one day)
    Ttotal = 20hours
    T(1000Mbps) = 17.5hrs

    Tmed = Time to write data to Media
    Tship = Time to ship Media
    Tread = Time to read/copy Media to cloud server/HD array
    Tmed + Tship + Tread >> Tship for large data sets…
    Plus the total hourly cost of the operator (intern?) doing the media transfer (for large data sets) is higher than an operator drag/drop dataset to cloud…

  2. anonymous

    At work we trade data with other companies. If it takes more than an hour or two to FTP, we consider shipping a disk. If it takes more than a day, we almost always ship a disk.

  3. ajolie1

    This was recommended in a UC Berkeley paper (Feb 09) on the economics of cloud computing — see link. Amazon must have just gotten around to reading this paper or finally listened to customers or did a back of the envelope calculation to figure out that snail mail is better than US broadband. Nonetheless, it shows that bandwidth is still a problem in the US.

    Maybe Kinkos/FedEx can build high speed fiber connections to Amazon; then customers can drive to Kinkos and upload a terrabyte while making copies of the bill to submit to the boss who will probably yell at the poor worker for not using the Internet to upload the file.

    http://berkeleyclouds.blogspot.com/

  4. Gumby

    What is wrong with sending satellites up in space and do the cloud stuff up there?? Military and scientists already have that… Why not the masses?

  5. Richard Smith

    Back in the days of BITnet (anyone remember that?) email would backlog between Canada and the US (we had one link, I think a 56k line to Columbia in NY) and so the mail would be spooled onto tape and trucked to Canada. This is in the 1980s, and I might have the details wrong, but it looks like we’re going back to the old days…