Every so often a new technology emerges that changes everything. In the world of storage, the last major media shift was the move to hard disk drives (HDDs) from tape. While tape is still around today as a target for backups and archiving (it’s cheap, durable, and portable for offsite storage), disk owns the vast majority of primary storage.
Today we find ourselves on the cusp of a transition in storage as massive as the move from tape to hard disk drives — the move from hard disks to solid-state disks.
CAPEX, OPEX and IOPS
From a capital expense (CAPEX) perspective (specifically the cost of acquiring a unit of storage), disk was more expensive than tape when HDDs started the last storage media disruption. Tape was also arguably cheaper from an operating expense (OPEX) perspective (specifically the maintenance cost including power, space, and labor). But from a cost-of-input/output-operations-per-second (IOPS) perspective, there was no contest. Anything that required fast access to storage moved to disk in short order and a new set of applications was enabled. Cost per IOPS won.
While it may appear that the transition from HDDs to SSDs (also popularly known as flash memory) will mean higher CAPEX on a cost-per-unit-of-storage basis, that may be misleading. Amdahl’s Law helps us to understand the maximum improvement to a system when only part of the system is improved. Mainframes leveraged Amdahl’s Law to maintain balance among the various components (compute, storage, and so on) in order to optimize overall system performance. Throughout the evolution of open systems this sense of balance has been lost. While Moore’s Law has been applicable to storage in terms of cost per unit of storage, storage IOPS have not kept pace with compute IOPS. The system is out of balance. Solid-state disks will help us regain the balance between storage and compute, so CAPEX per IOPS may well decrease substantially.
OPEX will also decrease as SSDs replace hard disks. Power consumption is lower with SSDs than with hard disks, so electricity costs per IOPS will also drop. Solid-state disks will bring better performance, reducing the number of systems and IT professionals required to do the same job. And since SSDs have no moving parts costs related to maintenance should decrease.
Just as with the last storage discontinuity, total cost per IOPS will be much lower for SSDs relative to hard disks in many, if not most, cases. Consequently I believe that, in the coming 5-10 years, SSDs will replace HDDs en masse. Cost per IOPS will win again.
And just as the last transition enabled a new set of applications, so too will this one. Already consumer devices like laptops, iPods, and the iPhone have put SSDs to good use. This is just the tip of the iceberg. Here are a few opportunities that come to mind:
1. Enterprise storage.
The next EMC and NetApp will likely be built off the back of the move to SSDs. Incumbents will have difficulty changing their mental models to address the new world. Existing vendors are already introducing hybrid solutions into the market, which is exactly what you would expect of incumbent players. But their software and systems were designed with assumptions about spinning media that are not appropriate in a solid-state world. There is room for at least one massive, pure-play SSD enterprise storage vendor.
2. Enterprise applications.
The vast majority of today’s application software was also written with hard drives in mind. Just as with storage, there will be an opportunity for applications to be written from the ground up with solid-state disks as the sole target. Will a new type of database emerge? If you designed the perfect database today with solid state in mind, would it look like Oracle or MySQL? Will SSDs empower real-time analytics, supply chain management and financial reporting? How will an order of magnitude improvement in IOPS change the art of the possible in customer-facing applications?
3. Consumer applications.
The largest consumer Internet players spend vast sums of capital on in-memory solutions and on operating costs associated with massive farms of servers and hard disk drives. Delivering results to a search query, placing the right advertisement in front of the right consumer at the right time, running a massively multiplayer online game, and building personalized pages are IOPS-intensive applications operating on gargantuan data stores. SSDs will dramatically lower the cost of providing services like these, ushering in a wave of innovation. And hopefully this will empower a bunch of hackers with little capital to disrupt some very large businesses as well as to invent all new ones.
While the world is suffering from a nasty recession, innovation marches on. Solid-state drives will help us regain balance between compute and storage, which will lead to lower overall costs and usher in a new class of applications. I can’t wait.
*I would like to thank John Colgrove for reading a draft of this note and providing his feedback. John is an EIR at Sutter Hill Ventures, previously served as a Symantec Fellow and VP of Technology Strategy for the company’s Data Center Management group, and was a founding engineer of storage software leader VERITAS Software which was acquired by Symantec.
Mike Speiser is a Managing Director at Sutter Hill Ventures. His thoughts on technology, economics and entrepreneurship will appear at this time every week.