Key Criteria for Evaluating Application Performance Monitoring (APM)v1.0

An Evaluation Guide for Technology Decision Makers

Table of Contents

  1. Summary
  2. APM Primer
  3. Report Methodology
  4. Decision Criteria Analysis
  5. Evaluation Metrics
  6. Key Criteria: Impact Analysis
  7. Analyst’s Take
  8. About Ron Williams

1. Summary

Application performance monitoring (APM) seeks to understand how an application reacts when in use. Historically, developers would write bits of code into their applications that would signal what was executing and use other data to determine application performance. Those days have long passed. Within the complicated world of modern distributed applications, microservices, cloud computing, and hybrid environments, heightened business and customer expectations are the norm.

The increase in operating functions and capability places an additional load on developers. They must handle not just the application code complexity but also the use of underlying infrastructure. Applications may run in on-premises infrastructure, public cloud environments, private clouds, SaaS applications, and hybrids of these models.

With the additional emphasis on DevOps, developers are now a part of the operations landscape and must have the tools to participate. APM tools now must handle a much more complex environment with both operations personnel and developers, and within low code/no code environments, encompassing citizen programmers.

The explosion of the Internet of Things (IoT) has impacted APM tools as well. In many cases, an agent or code injection is not possible with IoT objects, yet these same objects are becoming more critical to the enterprise. APM strategies such as synthetic transactions interrogate the edge device to determine functionality without detailed knowledge of the inner workings of the device, so they have become essential as IoT networks, devices, and their applications proliferate.

Standards within the APM world are minimal; however, OpenTelemetry, an open source project, holds the promise of a common method of transferring data. OpenTelemetry is currently a Cloud Native Computing Foundation (CNCF) project with the goal of providing a set of vendor-agnostic libraries and APIs for collecting and sending data, such as traces, metrics, and logs.

This report will explore application performance monitoring in its current state, determine the key criteria for decision making, and explore technologies that may impact the future of APM. Its companion Radar report will discuss vendors within the APM marketplace.

This report defines the following market segments:

  • Large enterprises: We assessed offerings on their ability to support large and business-critical projects. Optimal solutions in this category will have a strong focus on flexibility, performance, data services, and features to improve security and data protection. Scalability is another big differentiator, as is the ability to deploy the same service in different environments. (Managed service providers or MSPs, may be a target for some vendors, but are not addressed in this report).
  • Mid-market or business unit: In this category, we assess solutions on their ability to meet the needs of organizations ranging from medium-sized businesses to business units within a larger corporation. Ease of use and deployment are more important than extensive management functionality, data mobility, and feature set.
  • Small to medium business (SMBs): This category includes solutions that address the needs of SMBs that don’t require IT expertise across multiple domains. Only SaaS solutions are appropriate for this segment of users.

How to Read this Report

This GigaOm report is one of a series of documents that helps IT organizations assess competing solutions in the context of well-defined features and criteria. For a fuller understanding consider reviewing the following reports:

Key Criteria report: A detailed market sector analysis that assesses the impact that key product features and criteria have on top-line solution characteristics—such as scalability, performance, and TCO—that drive purchase decisions.

GigaOm Radar report: A forward-looking analysis that plots the relative value and progression of vendor solutions along multiple axes based on strategy and execution. The Radar report includes a breakdown of each vendor’s offering in the sector.

Solution Profile: An in-depth vendor analysis that builds on the framework developed in the Key Criteria and Radar reports to assess a company’s engagement within a technology sector. This analysis includes forward-looking guidance around both strategy and product.

Full content available to GigaOm Subscribers.

Sign Up For Free