Table of Contents
For quite some time, users have been asking for object storage solutions with better performance characteristics. To satisfy such requests, several factors must first be considered:
- Data consolidation: Combining and storing various types of data in a single place can help to minimize the number of storage systems, lower costs, and improve infrastructure efficiency.
- New workloads and applications: Thanks to the cloud and other technology, developers have finally embraced object storage APIs, and both custom and commercial applications now support object storage. Moreover, there is a high demand for object storage for AI/ML and other advanced workflows in which rich metadata can play an important role.
- Better economics at scale: Object storage is typically much more cost effective than file storage and easier to manage at the petabyte scale. And $/GB is just one aspect; generally, the overall TCO of an object storage solution is better than it is for file and block systems.
- Security: Some features of object stores, such as the object lock API, increase data safety and security against errors and malicious attacks.
- Accessibility: Object stores are easier to access than file or block storage, making it the right target for IoT, AI, analytics, and any workflow that collects and shares large amounts of data or requires parallel and diversified data access.
A growing number of applications find object storage to be a natural repository for their data because of its scalability and ease of access. However, older object stores were not designed for flash memory, nor were they optimized to deal with very small files (512KB and less). Many vendors are redesigning the backend of their solution to respond to these new needs, but in the meantime, a new generation of fast object stores has become available for these workloads.
These new object stores usually offer a subset of the features of traditional object stores, in particular geo-replication or S3 API compatibility, but they excel in other ways that are even more important for interactive and high-performance workloads, including strong consistency, small file optimization, file-object parity, and features aimed at simplified data ingestion and access with the lowest possible latency. Their design is based on the latest technology: flash memory, persistent memory, and high-speed networks are usually combined with the latest innovations in software optimization. Even though object stores will never provide the performance of block or file storage, it is important to note that they are more secure and easier to manage at scale than the others, offering a good balance among performance, scalability, and TCO.
The ability to maintain a consistent response time under multiple different workloads is also very important. On the one hand, there are the primary workloads for which these object stores are usually selected in the first place, but on the other, it is unusual to find fast object stores serving only a single workload over a long period. Users tend to consolidate additional data and workloads, and multitenancy quickly becomes another important requirement. These solutions typically offer good file storage capabilities as well, allowing data to be consolidated even further.
At the moment, high-performance object stores do not really overlap with traditional object stores except for a limited set of use cases. This distinction will change over time because both traditional and high-performance object stores will eventually add the features necessary for parity. The products with the most balanced architecture and the ability to optimize for the latest media will end up in the leading positions.
How to Read this Report
This GigaOm report is one of a series of documents that helps IT organizations assess competing solutions in the context of well-defined features and criteria. For a fuller understanding consider reviewing the following reports:
Key Criteria report: A detailed market sector analysis that assesses the impact that key product features and criteria have on top-line solution characteristics—such as scalability, performance, and TCO—that drive purchase decisions.
GigaOm Radar report: A forward-looking analysis that plots the relative value and progression of vendor solutions along multiple axes based on strategy and execution. The Radar report includes a breakdown of each vendor’s offering in the sector.
Vendor Profile: An in-depth vendor analysis that builds on the framework developed in the Key Criteria and Radar reports to assess a company’s engagement within a technology sector. This analysis includes forward-looking guidance around both strategy and product.