As the march towards data-centric computing continues unabated, many organizations are finding that the traditional ideologies of the public cloud are no longer appropriate. The biggest issue is that the public cloud may not offer the level of security and speed needed by organizations that are transforming their intellectual property into actionable data sets. Simply put, there are enterprises that share a common problem, they cannot or will not send their proprietary data into a public cloud.
Factors such as compliance, privacy, and speed to market have driven many organizations to attempt to build internal data lakes, instead of turning to the cloud. However, in many cases, data lakes become far too complex to manage and increasingly resource intensive. Nowhere is that truer than in the burgeoning IoT market, where sensors and devices are providing constant streams of data.
To get a better feel for the problems facing businesses adopting a data-centric approach, GigaOm spoke with the founders of Igneous Systems, a company created to bring IaaS (Infrastructure as a Service) capabilities to businesses seeking to eschew public clouds for their sensitive data streams.
Kiran Bhageshpur and Steve Pao (Igneous’ respective CEO and CMO) offered observations that the two trends are related in that the falling price of storage has been encouraging even greater desires to store and retain data, rather than to throw it away. That one-two punch of increased data generation growing faster than dropping storage costs translates to something that seems unthinkable; the expenditures for storing data in-house and in the public cloud are actually rising as a whole.
Pao offered that the economic challenges of higher expenditures make some new storage ideologies possible, ones that combine the elasticity and ease of use offered by the cloud, paired with the locality, control and security offered by on-premises storage solutions. However, it takes more than just economic pressure to create a storage paradigm shift, and that is exactly where Igneous Systems comes into the picture.
Bhageshpur said, “Our customers had a common problem, one where they could not use the cloud, either because their data sizes are so large that the latency of the internet created problems, or their data was proprietary and a core part of their company IP.”
Pao added, “Another issue is that much of this critical data is being curated by line of business functions and not by enterprise IT groups normally responsible for managing a complex IT infrastructure, creating a situation where data can easily be managed outside of enterprise policy controls . Those problems have an even broader impact on other industries. For example, we are hearing that in the scientific computing segment, biologists, chemists, and physicists are struggling to deal with the large data sets generated by their equipment.”
Those realizations drove the creation of the Igneous Data Service, which is akin to building an on-premises, private cloud that acts as a scalable content store for large unstructured data. Bhageshpur said, “The Igneous Data Service offers true cloud for local data and supports the S3 API, which is rapidly becoming the de facto standard for cloud-based object storage.”
The advantages of adopting the S3 API are many, starting with the fact that many application developers are already familiar with developing applications for S3, and numerous applications are readily available. What’s more, the Igneous Data Service offers much of the same experience as the public cloud, in that Igneous handles all monitoring, maintenance, software updates and troubleshooting of on-premises equipment, all for the price of an annual subscription, which is based on installed capacity. That creates a zero-touch experience, which is one of the major benefits a public cloud service may offer.
From a technology perspective, Igneous Systems incorporated four key attributes into the Igneous Data Service which it considers as “first principles” to enable cost-effective deployment of an on-premises Infrastructure-as-a-Service (IaaS) offering.
- On-premises data with cloud management. While the data always stays on the customer’s networks. Igneous continually monitors, maintains and troubleshoots its fleet of inventory using automated, software-based cloud management techniques employed in today’s hyperscale cloud environments.
- RatioPerfect™ architecture. Behind the Igneous architecture is a nano-server design where each disk has its own dedicated CPU and Ethernet connection. This design enables a highly-distributed architecture that provides the scalability and resiliency of the public cloud but scaled to run on a customer premises.
- Extensible Data Path. Large data sets typically require inline processing of data before uploads or downloads “complete.” With its extensible data path, higher level operations (such as auto-tagging for search) can be performed on a mirror of incoming and outgoing data streams without slowing down low-level system functions.
- Cloud native services. To enable new application workflows, the Igneous architecture is built utilizing a modern microservices approach and incorporates stream processing, an event-driven framework, and container services.
“We started Igneous three years ago to deliver the type of distributed architectures enterprises need that will provide the public cloud with its scalability and resiliency,” said Kiran Bhageshpur. “We see data as a rapidly growing asset base, and we look forward to helping our enterprise customers curate that data to speed decision making and support new business models.”
One thing is certain; there is change in the air, fueled by the adoption of data-centric computing and enterprises need to think long and hard about their options, before drowning in a data lake.