Table of Contents
- Summary
- Data Pipelines – A Primer
- The Cloud and Data Pipelines
- Major Cloud Data Pipeline Platforms
- Extending the Data Pipeline
- Conclusion
- About Andrew Brust
- About Yiannis Antoniou
- About GigaOm
- Copyright
1. Summary
Cloud Data Pipelines have matured and evolved to meet the increasing volume, diversity, and velocity of data flowing through the enterprise. In this report, GigaOm Analysts Andrew Brust and Yiannis Antoniou explore the basics of data pipeline platforms, including their legacy as ETL products and their evolution to the modern era of cloud-based, ELT-supporting extensible frameworks. They then analyze cloud data pipeline services from Microsoft, Amazon, and Google—Azure Data Factory, AWS Glue and AWS Data Pipeline, and GCP Dataflow and GCP Cloud Data Fusion.