Blog Post

Tweaking Storage for The Cloud

Stay on Top of Enterprise Technology Trends

Get updates impacting your industry from our GigaOm Research Community
Join the Community!

As we keep more and more of our valuable content online, do we need a new type of storage? A crop of venture-backed companies — among them Storwize and Ocarina Networks to Gear6 — certainly seems to think so. These companies solve one of two problems: how to access the data faster and how to store it more compactly.

Unlike the previous storage paradigm, which focused on backing up relatively static enterprise data and storing frequently accessed database information, storage today must be more nimble. Everything from photo sites to online email companies are offering and even encouraging consumers to store more and more bits of data. At last count, there were 281 exabytes of data being created each year. Much like cramming clothes into a suitcase, the more information you can store on a given box, the less you have to pay for the boxes — as well as the infrastructure to keep them running.

Last year, large storage vendors such as NetApp and EMC pushed the concept of de-dupulication, essentially storing only the new files at each backup, rather than the entire system, to reduce storage costs. Ocarina and Storwize solve this problem by going further than ignoring previously stored files, with an appliance that unpacks each file and then compresses it using proprietary algorithms.

The flip side of storing more is accessing that data faster. We’ve written about Gear6 and Atrato before, which both use caching and software to keep frequently accessed files easily available. The end road for most of these startups, however, is an acquisition, perhaps by bigger storage players such as EMC, or maybe even by Dell, HP or IBM.

Photo of the First IBM disk storage system courtesy of the Computer History Museum

Interested in web infrastructure? Want to learn more about Green Data Centers? Check out our upcoming conference, Structure 08.

17 Responses to “Tweaking Storage for The Cloud”

  1. What is interesting in the field of Primary Storage is Native File Optimization (NFO), i.e. optimizing single files systematically to the size they need to be. With our FILEminimizer technology, companies can optimize all unstructured files and gain back over 70% of existing storage space.
    The philosophy is different, the effect is powerful: why do companies allow to store Office files of 100 MB and more when the same content in optimized form takes up only 5 MB?
    This is storage waste and it is incredible how much you can save there without changing existing storage infrastuctures.

  2. Storwize is the only proven solution at the market with tens of customers and with enough money at the bank to the next few years. The saving in Storwize is across the board: Money, power, space, air conditioning etc. You should evaluate it.

  3. a much cheaper solution the NetApp and EMC for enteprise storage are the true Cloud Storage providers, like Nirvanix. they recently announced services for enterprise backup and online archiving, and they claim to be pretty “nimble” while staying cheap.
    for those interested

  4. Stacey, Today, the client that reassembles the data resides in a device which we sell called an Accesser. In the near future, the client will be imbedded in media players, cameras, cell phones, etc. to save and access dispersed data. The number of servers that can simultaneously fail while maintaining access to the data is user configurable. A typical configuration will have 16 storage servers spread across 8 locations with a minimum of number of slices needed to create the data being 10. That’s a 16 / 10 / 8 Cleversafe configuration. In this scenario, with 2 servers at each location you can simultaneously lose 3 sites or 6 individual servers and still be able to access the data. I’ll have someone from our PR agency schedule a briefing with you.

  5. I think your last pont is key. For any of these new ideas to really take off the companies either have to be bought out by one of the big players or they have to license out their knowledge. The cost to develop these ideas is huge and I think the VC folks don’t like spending dollars on actual equipment.

  6. Stacey Higginbotham

    Russ, Cleversafe is interesting. Who needs to have copies of the client in order to reassemble the data? What percentage of the servers can fail and still keep a viable copy of the information? I’d really like to learn more.

  7. Since you are identifying new technologies that are tweaking storage for the cloud, you may want to look at what Cleversafe is doing to address the needs of content storage and distribution. Cleversafe’s core technology is build on information dispersal which is a fundamentally better way to store and distribute large digital content objects, not just at tweak. Dispersed Storage takes in data and slices the data into a number of pieces that are transformed (optionally encrypted) and then dispersed across a network of storage servers which can be geographically distributed. Data is rebuilt using intelligence in a light weight client by having access to a predefined subset of the original pieces. The client dynamically determines the fastest and most efficient method for retrieving the slices and recreating the original data which is ideal for content distribution. Cleversafe’s products address both a more efficient way to store and protect digital content for extended periods of time as well as an efficient method of distributing that content to millions of users simultaneously.