Laptop Displaying the GigaOm Research Portal

Get your Free GigaOm account today.

Access complimentary GigaOm content by signing up for a FREE GigaOm account today — or upgrade to premium for full access to the GigaOm research catalog. Join now and uncover what you’ve been missing!

GigaOm Radar for Data Pipelinesv1.0

Table of Contents

  1. Summary
  2. Key Criteria Comparison
  3. GigaOm Radar
  4. Vendor Insights
  5. Adjacent Solutions
  6. Analyst’s Take
  7. About Andrew Brust

1. Summary

In a data-obsessed world, dealing with big data has its advantages and challenges. In order to mitigate the challenges and make better use of the advantages—such as quick business insights—it is important to be in control of data flow and be certain that data quality is ensured and insights are quickly reached.

Data pipelines take raw data, transform it so its format is suitable for the target system, and then load it to a unified data store. That process, which normally takes weeks and requires writing chunks of code, can be simplified to take less time and little-to-no coding with data pipeline tools. Considering the constantly growing amounts of data and the competitive advantage it can provide when analyzed efficiently, adopting a data pipeline tool to automate the data integration process can offer an unbeatable edge.

In an environment that offers a broad array of solutions, users should consider features like scalability, performance, monitoring, alerts, ease of maintenance, and collaboration before adopting a data pipeline solution and getting into technical debt.

In this report, we explore the major data pipeline platforms, outline the key criteria by which they should be evaluated, and assess each platform based on how they cover these key criteria.

Key findings:

  • Most platforms offer codeless solutions and enable an environment for citizen data professionals to utilize data.
  • The most complete offerings are AI-powered, provide support for any combination of cloud and hybrid infrastructures, and offer serverless processing.
  • The marketplace for classic on-premises data pipeline solutions is mature and stable. The newest developments are premised on a single streamlined tool for data integration and increasingly, machine learning. Vendors are also providing richer built-in tools for complex data transformations.

How to Read this Report

This GigaOm report is one of a series of documents that helps IT organizations assess competing solutions in the context of well-defined features and criteria. For a fuller understanding consider reviewing the following reports:

Key Criteria report: A detailed market sector analysis that assesses the impact that key product features and criteria have on top-line solution characteristics—such as scalability, performance, and TCO—that drive purchase decisions.

GigaOm Radar report: A forward-looking analysis that plots the relative value and progression of vendor solutions along multiple axes based on strategy and execution. The Radar report includes a breakdown of each vendor’s offering in the sector.

Solution Profile: An in-depth vendor analysis that builds on the framework developed in the Key Criteria and Radar reports to assess a company’s engagement within a technology sector. This analysis includes forward-looking guidance around both strategy and product.