Do Service Providers Forecast the Enterprise Storage Future?

Public clouds are the new storage service providers. In addition to Amazon S3, there are now compelling offerings from Microsoft, Rackspace, EMC, and soon, Google. Even though the public clouds have garnered more than their share of attention, however, corporations are still a long way off from wholesale adoption; tens of billions of dollars of enterprise storage software and hardware will continue to be sold for years to come.

But for those keeping an eye on the enterprise storage future, it makes sense to consider whether the infrastructure challenges being faced by these large service providers today indicate what the largest enterprises will face tomorrow.

After all, the two share many common challenges, such as grappling with enormous amounts of data, a high number of overall data objects, a large user population and wide geographic reach. Public storage clouds have solved the challenges through innovative architectures and approaches, many of which are working their way into the enterprise. Let’s take a closer look at what, specifically, they’ve achieved.

Commodity hardware — The embrace of commodity hardware on the part of public service providers as a way to drive down per-gigabyte storage costs is among the biggest advantages they offer over enterprises, which still rely on storage vendors to provide integrated solutions. That’s partly because the largest public clouds are by default infrastructure experts — not necessarily a skill set that can be found in every enterprise. I expect these hardware purchasing preferences to remain distinct for now.

Multitenancy — Most public clouds were designed from the ground up to segment customers. The same can’t be said of enterprise storage technology, however, even though the behavior of large corporations is very similar to that of multiple customers. For example, it’s common for large companies to manage internal chargebacks for technology resources across groups and departments and therefore the same segmentation, quality-of-service guarantees and billing mechanisms apply.

Everything through an API — Part of the very notion of cloud computing is the abstraction of physical resources whereby new servers or storage repositories can be provisioned via a web browser as opposed to a screwdriver. This is hardly the case in conventional corporate infrastructure, where direct access to hardware resources has led to very differently designed products. But management and administration occupies a large portion of storage budgets; meanwhile, the sheer volume of data and number of objects continues to make more manual administrative processes impossible even for enterprises. So we are likely to see enterprise storage technology evolve to streamline management. Specifically, vendors need to expose some of their basic functionality through open APIs that can be passed all the way to the application.

Geographic reach and availability — Cloud customers need to know that their data is always available, which is made possible by having it in multiple locations. Leading cloud providers accomplish this by having different availability zones, such as with Amazon, or through maintaining specific software features, like GeoProtect within EMC Atmos. Enterprises need the same capabilities, which means the storage industry must evolve from basic two-way replication solutions to geographically distributed storage objects where the distribution benefits both the availability and the delivery of the data. For example, second copies should alleviate the primary system load by being able to serve read-only requests similar to a content delivery network.

Security, ongoing maintenance and administration, performance management and data protection are additional areas in which leading public cloud storage providers have seen problems at a size and scale that many enterprises will see soon enough. And in most cases, the solutions they’ve come up with will serve the needs of enterprises accordingly. So for the time being, the public cloud storage providers are filling a role as enterprise barometer as well.

Want to know more about how massive public cloud storage deployments are forecasting the future? Check out these sessions at the upcoming Structure conference:

  • Scaling the Database in the Cloud
  • Dealing With the Data Tsunami: The Big Data Panel
  • Supermassive Storage: Cutting-edge Practices from the Service Provider Industry

Gary Orenstein is host of The Cloud Computing Show