Skip to content

Main Navigation

Gigaom Search
  • Menu

  • About Us
  • ResearchExpand
    • Cloud & Infrastructure
    • Data Infrastructure, AI & Analytics
    • Edge & Networking
    • Security & Risk
    • DevOps & Applications
  • Analysts
  • Search
  • Sign in
Become a Client
Binary Data Transfer Process. Big Data Stream. Information Techn
Image credit: GennadiCh
Sponsored by

Related Research

Adventist Risk Management Data Protection Infrastructure Enrico Signoretti
GigaOm Radar for Evaluating Machine Learning Operations (MLOps) Andrew J. Brust
William McKnight, Jake Dolezal Mar 2, 2018 (Oct 13, 2020) -- Report

Benchmarking Enterprise Streaming Data and Message Queuing Platforms v1.0

Table of Contents

  1. Summary
  2. The Current Landscape for Streaming Data and Message Queuing
  3. Evaluating Potential Solutions
  4. Conclusion and takeaways
  5. Appendix – Deploying and Executing the Benchmark
  6. About William McKnight
  7. About Jake Dolezal
  8. About GigaOm
  9. Copyright

Summary

Historical paradigms around data are being shifted due to exponential data growth, hybrid cloud architectures, real-time data needs, massive computing at scale, globalization, 24×7 uptime, and the Internet of Things. The list could go on. Organizations are either seizing opportunities, scrambling to keep up, or coping with the changes in their industry verticals due to technological shifts. Many want or need to adopt these newer technologies, but sometimes struggle with how to evaluate or select them based on their current and future needs.

The needs and uses of data have evolved to a new level. In today’s climate, businesses must be able to ingest, analyze, and react to data immediately. With artificial intelligence and machine learning progressing by leaps and bounds, we see increasing numbers of technologies emerging as data-driven, autonomous, decision-making applications—where data is produced, consumed, analyzed, and reacted to (or not) in real-time. In this way, the technology becomes sentient of what’s going on inside and around it—making pragmatic, tactical decisions on its own. We see this being played out in the highly publicized development of self-driving cars, but beyond transportation, we see it in telephony, health, security and law enforcement, finance, manufacturing, and in most sectors of every industry.

Prior to this evolution, the analytical data was derived by humans long after the event that produced or created the data had passed. We labored under the assumption that by analyzing the past, we could set the course for the future—in our businesses, in our systems, in our world. Now, however, technology has emerged that allows us to capture and analyze what is going on right now. Much data value is time-sensitive. If the essence of the data is not captured within a certain window of time, its value is lost and the decision or action that needs to take place as a result never occurs.

This category of data is known by several names: streaming, messaging, live feeds, real-time, event-driven, and so on. This type of data needs special attention, because delayed processing can and will negatively affect its value—a sudden price change, a critical threshold met, an anomaly detected, a sensor reading changing rapidly, an outlier in a log file—all can be of immense value to a decision maker, but only if he or she is alerted in time to affect the outcome.

Much has already been said and written about artificial intelligence and autonomous decision making. The focus of this paper is the gap between the production and consumption of real-time data and how these autonomous, sentient systems can be fed at scale and with reliable performance. There are a number of big data technologies designed to handle large volumes of time-sensitive, streaming data.

We will introduce and demonstrate a method for an organization to assess and benchmark—for their own current and future uses and workloads—the technologies currently available. We will begin by reviewing the landscape of streaming data and message queueing technology. They are alike in purpose—process massive amounts of streaming data generated from social media, logging systems, clickstreams, Internet-of-Things devices, and so forth. However, they also have a few distinctions, strengths, and weaknesses.

Full report available to GigaOm Subscribers.

Subscribe to GigaOm Research
Content Management, Data Infrastructure, AI & Analytics, Streaming Data, Technology
  • Tweet
  • Share
  • Post

Related Research

Internet of things, wireless sensor network
Image credit: chombosan
Iben Rodriguez, Geoff Uyleman Apr 1, 2021 (Apr 1, 2021) -- Key Criteria & Market Landscape

Key Criteria for Evaluating Vulnerability Management Tools

Vulnerability management tools scan your IT estate to help identify and mitigate security risks and weaknesses. These tools can facilitate the development…

Read More
Security & Risk
imageLikeEmbed
Chris Grundemann, Logan Andrew Green Mar 26, 2021 (Mar 26, 2021) -- Market Radar

GigaOm Radar for Edge Platforms

Edge platforms leverage distributed infrastructure to deliver services, bringing content, computing, and security closer to end devices, offloading networks, and improving performance.…

Read More
Edge Infrastructure
computer-2930704_1280
Image credit: ItNeverEnds
Sponsored by
William McKnight, Jake Dolezal Mar 26, 2021 (Mar 26, 2021) -- TCO & Benchmark

Cassandra Total Cost of Ownership Study

This study examines the full cost and true value of Cassandra self-managed on Google Cloud (GCP) and the cost of a fully…

Read More
provetas-118222_1920
Image credit: Makunin
Michael Delzer Mar 19, 2021 (Mar 19, 2021) -- Key Criteria & Market Landscape

Key Criteria for Evaluating Performance Testing Tools

Performance testing is a process that determines how responsive a system is under varying loads and provides actionable feedback as part of…

Read More
computer-chip-5936602_1920
Image credit: Gerd Altmann
Anand Joshi Mar 18, 2021 -- Key Criteria & Market Landscape

Key Criteria for Evaluating AI Inference Chipsets

Many AI applications that started as proofs of concept a few years ago are now moving into production. The compute infrastructure for…

Read More
Artificial Intelligence, Data Infrastructure, AI & Analytics
computer-2760136_1920
Image credit: Thomas Ulrich
Enrico Signoretti Mar 17, 2021 -- Key Criteria & Market Landscape

Key Criteria for Evaluating Enterprise Object Storage

The object storage market is more active than ever. It is very popular with developers, and just about every IT organization is…

Read More
Data Storage, Object Storage, Scale-out File Storage

Stay on top of emerging trends impacting your industry with updates from our GigaOm Research Community.

Join Research Community
Gigaom
  • About Us
  • Analysts
  • Press Room
  • Contact
  • Twitter
  • Facebook
  • LinkedIn
  • RSS Feed
  • Newsletter
  • Privacy Policy
  • Terms of Service
  • © GigaOm All Rights Reserved 2021
This website uses cookies; by continuing you are a agreeing to our Privacy Policy Accept
Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.
Necessary
Always Enabled

This is an necessary category.

Save & Accept