Table of Contents
1. Executive Summary
This GigaOm Maturity Model report was commissioned by Catchpoint.
In today’s digital-first world, where every interaction can influence customer satisfaction and business success, the ability to monitor and optimize internet performance is critical. Organizations are at different stages in their journey toward achieving comprehensive Internet Performance Monitoring (IPM) capabilities, and this maturity level greatly affects their capacity to deliver resilient, high-quality user experiences.
An IPM maturity model offers a strategic framework that helps organizations evaluate their observability practices and guide their progress toward more advanced, value-driven outcomes. By evaluating their existing monitoring strategies, organizations can identify where they fall on the spectrum. The ultimate goal is to move toward what we refer to as “value-based observability,” where organizations not only track and respond to performance metrics, they also anticipate and address potential issues before they affect users.
This advanced approach focuses heavily on user experience and resiliency, ensuring digital services are consistently reliable, responsive, and adaptable to changing demands. It goes beyond traditional observability by aligning technical performance with broader business objectives, driving both operational efficiency and strategic value. To reach this level of observability, most organizations need to move beyond the basics to embrace a more comprehensive approach that directly contributes to business outcomes.
As companies advance through the stages of the IPM maturity model, they shift from meeting the minimum requirements of viable observability to using it as a strategic asset that enhances user experience, builds resilience, and drives business value. This journey is essential for maintaining a competitive edge in an increasingly complex digital environment.
GigaOm Maturity Model
This GigaOm Maturity Model provides context and expected outcomes for organizations that seek to transform Internet Performance Monitoring (IPM) observability. It illustrates common landmarks of meaningful progress and describes the long-term value that can be expected from evolving current observability practices.
2. Technology Primer
Prior to describing the Maturity Model, it is important to understand what we mean by IPM.
IPM encompasses a suite of solutions and practices designed to monitor, analyze, and optimize the performance of internet-based applications and services. Its primary objective is to ensure reliability, speed, and efficiency across all aspects of internet communications by delivering enhanced visibility, diagnostics, independent observability, and end-user experience monitoring.
IPM enables proactive optimization and fosters internet resilience, providing organizations with critical visibility into segments of the internet stack that are otherwise opaque to traditional or agent-based monitoring tools, while also guiding them to evolve.
IPM platforms should include the following ingestion/data collection capabilities:
- Real User Monitoring (RUM)
- Synthetics
- Endpoint
- Web page performance metrics
- Core internet Border Gateway Protocol (BGP) routing data
Accurate visibility in IPM platforms is only possible if data is collected from the end-user perspective and across internet infrastructure including:
- Backbone
- Cloud
- Last Mile and Wireless
- Private network vantage points
- Browser
- Desktops endpoint
- Border Gateway Protocol (BGP) Autonomous System Numbers (ASNs)
IPM should cover all layers of the internet delivery stack:
- Applications – (APIs, response times, performance)
- Media (ad serving, analytics, video)
- Cloud services (Clouds, API, Hosting)
- Internet Core (DNS, CDN, BGP)
- Protocols (TCP, SSH, IPV4/6)
- Network (Backbone, wireless, last mile, SASE, WAN, LAN)
Understanding the Complexity of IPM Across Organizational Roles
IPM is a multifaceted discipline that intersects with various aspects of an organization’s operations. From the chief information officer (CIO) to the digital transformation leader, each role within an organization has distinct priorities, challenges, and perspectives on how IPM contributes to achieving their goals.
Table 1 provides an overview of how different personas within an organization focus on specific aspects of IPM, their associated pain points, and the relevance of IPM in addressing these challenges.
Table 1. Persona Overview
Persona | Focus | Pain Points | IPM Relevance |
---|---|---|---|
Chief Information Officer (CIO) | IT initiatives aligned with business goals. Maintaining SLAs | Business continuity, downtime, IT budgets | Maintain resilient digital environment, ensure business operations run smoothly, reduce risk of costly outages |
VP/Director of IT Operations | Operations of IT systems, infrastructure, networks, and data centers | Mean time to resolution (MTTR), IT incidents, high availability of critical systems | Deep visibility into performance and reliability, quickly identify and resolve issues, reduce disruption. |
Network Engineer/Architect | Infrastructure supporting digital services | Network performance, traffic efficiency, secure and reliable connectivity | Monitor performance and resiliency, optimize routing, troubleshoot issues, ensure uninterrupted service |
Chief Information Security Officer (CISO) | Protect information and IT infrastructure from cyberthreats | Security and integrity of data, vulnerabilities, cyber risks | Insights to detect and respond to security incidents that impact system availability or performance |
Source: GigaOm 2024 |
3. Maturity Model
Within IPM, organizations navigate from basic reactive monitoring practices to more advanced strategies that emphasize proactive management and resilience. The GigaOm Maturity Model provides a framework to evaluate and guide this progression. By understanding and applying this model, organizations can systematically advance their IPM capabilities, ultimately achieving a more refined and effective approach to observability. Advancing an organization’s IPM involves progressing through various maturity levels, each representing a distinct stage in the development of an organization’s monitoring capabilities and business processes. These levels present a structured path that organizations can follow to systematically enhance their IPM strategies, evolving from initial, reactive practices to more defined, proactive, and ultimately optimized processes.
Table 2 outlines five maturity levels of IPM, detailing the evolution from basic practices to advanced, predictive capabilities that align closely with business objectives. Each level is defined by specific capabilities, characteristics, and goals that guide organizations in their progression toward more sophisticated and integrated IPM strategies.
Components of the Maturity Model
- Capabilities: This component details the type of tools, technologies, and techniques typically available at each level, highlighting the progression from basic monitoring tools to advanced AI-driven analytics and real-time optimization.
- Characteristics: This component outlines the nature of IPM processes at each maturity level. It reflects the evolution from minimal approaches to standardized, controlled systems. As organizations progress, these systems increasingly incorporate continuous improvement, experience, and resiliency alongside technology.
- Recommended actions: The recommended actions at each maturity level define the strategic objectives that organizations should strive to achieve as they advance through the maturity model. These actions range from establishing foundational monitoring practices to driving continuous innovation and aligning IPM efforts with broader business objectives.
Table 2. Maturity Model Components
Level 1 | Initial | Level 2 | Managed | Level 3 | Defined | Level 4 | Quantitative | Level 5 | Optimizing | |
---|---|---|---|---|---|
Capabilities | Private network focus. Reactive. Basic and manual tools. | Basic monitoring. Some automated alerts. Manual performance analysis. | Comprehensive monitoring, automated alerts, regular reviews. Integration with other IT systems. | Predictive analytics. Detailed metrics. Regular reporting. SLAs & KPIs. | AI-driven analytics. Real-time optimization. Dynamic resource allocation. Business objective integration. |
Characteristics | No formal IPM processes. Ad hoc, reactive, and manual issue resolution. | Basic, repeatable IPM processes. Mix of reactive & proactive measures. | Standardized, documented IPM processes. Proactive management focus. | Measured, controlled processes. Data driven improvements.. Focus on experience and resiliency | Continuous improvement via quantitative feedback. Innovation and adaptability focus. |
Recommended Actions | Establish basic monitoring and performance tracking. | Standardize IPM processes. Improve visibility. Automate monitoring. | Enhance proactive monitoring. Improve IT integration. Establish baselines. | Implement predictive analytics. Set benchmarks. Continuously optimize. | Real-time management. Align to business goals. Prioritize experience & value. |
Source: GigaOm 2024
4. Value-Based Observability
The IPM Maturity Model above serves as a critical foundation, outlining the progression and technical evolution of IPM processes. It can serve as a roadmap for organizations to improve their monitoring tools and capabilities and is meant to guide technical development at different maturity levels.
Value-based observability reframes this foundational view to more directly guide broader strategic development and business value realization. To align to this approach, organizations must shift their views from technology-centric and systems-based IPM to one that emphasizes overall experience. At these advanced stages, observability is no longer defined by simply ensuring systems are functioning; it is defined as a strategic solution that directly contributes to business value by optimizing performance and resilience along the network stack, enhancing and prioritizing user experience and satisfaction.
This progression can be understood through four key stages: Technology, Systems, Experience, and Value. Each of these stages reflects a deeper level of observability maturity, defined by specific structures, technologies, and outcomes. While similar to the general maturity model, the conversation is reframed into the following focus areas defining observability maturity levels:
- Technology focus: At the initial stage, organizations concentrate on the fundamental tools and technologies needed to monitor and manage their infrastructure. The focus here is on establishing the basic observability structures necessary for tracking performance metrics and responding to issues as they arise.
- Systems focus: As organizations mature, the emphasis shifts to developing more standardized and integrated systems. This stage involves not only refining the technologies in use but also establishing consistent processes that enable proactive management of performance and reliability across the entire IT environment.
- Experience focus: Moving into the experience-focused stage, organizations begin to prioritize the user experience—internal and external—along the network stack as a critical component of their observability strategy. The focus expands to include end-to-end visibility of the user journey, ensuring digital services are not only available but also optimized for performance and user satisfaction.
- Value focus: The final stage represents the culmination of the observability journey—where observability practices are fully aligned with business objectives. At this stage, organizations leverage advanced analytics, AI/ML, and other strategic technologies to drive continuous improvement, ensuring observability contributes directly to achieving business value and maintaining competitive advantage.
The image in Figure 1 illustrates this reframe in organizational thinking and reflects a minimum viable target level for IPM.
Figure 1. Stages to Value-Based Observability
5. Assessing Your Organization’s Maturity Level
As organizations seek to advance and evolve their IPM capabilities and strategies, they should first establish their current maturity level. Conducting an assessment of existing capabilities versus the desired future state enables creation of a roadmap for desired evolution and end state.
The following steps can be followed for this assessment:
- Identify key domains: Begin by identifying the critical domains that are central to your IPM strategy. For IPM, these would typically include components of the network stack such as network, application, website, internal clients, and external clients. Each domain represents a significant area of focus within your observability practices and should be evaluated to determine the desired coverage and experience desired.
- Evaluate core dimensions: Within each domain, you can further refine assessment by defining dimensions to monitor, measure, and score. Examples of dimensions for IPM might include such things as performance, availability, experience, and resilience. Each dimension provides insights into how well your systems within your domains are functioning, where improvements may be needed, and how evolved observability may be within the organization.
- Determine current maturity levels: For each defined dimension within each applicable domain, determine the current level of maturity. At a high level, this can be simply assessed against the general maturity model, which can then be overlaid onto the value-based observability model as desired. Some providers have assessment services that might be preferred, delivering scorecards and ratings against benchmarks that provide more specificity and detail. Regardless of approach, this step should define, at minimum, the ‘As Is’ state of IPM in your organization.
- Determine desired maturity levels: Once the current maturity levels have been evaluated, determine your desired or ‘To Be’ state. This can be done by comparing against benchmarks, desired scores, or high level stages in the maturity model. You might wish to prioritize within domains and dimensions to more definitively prioritize next steps.
- Develop a roadmap for improvement: Based on the assessment, develop a roadmap that outlines the actions needed to advance through the maturity levels to your desired state. This roadmap should focus on strengthening weak areas, optimizing current practices, and aligning observability efforts with broader business objectives. The recommendation for best success is to achieve at least a minimum viable observability level in your domains and dimensions, evolving to value-based observability levels from a solid foundation.
Indicative Descriptions for Assessing Dimensions
To assist organizations in assessing their IPM maturity level, the following examples are provided for dimensions to assess. These descriptions offer a general understanding of what to evaluate during the assessment process, aligned with both the baseline maturity model and the evolved observability model.
- Performance and capacity: This dimension assesses the efficiency and effectiveness of systems in managing latency, response times, throughput, and saturation. It involves understanding how well systems can handle current and projected traffic loads, ensuring they meet performance requirements under varying conditions.
- Availability: This dimension focuses on the ability of systems to remain operational and accessible. It includes evaluating uptime, stability, and functionality, with a goal of minimizing errors, outages, and disruptions to maintain continuous and reliable operations.
- Experience (cross-dimensional): This dimension evaluates the overall user experience. It encompasses how well systems deliver performance, capacity, availability, and resilience from the user’s perspective. It considers the impact of observability practices on user satisfaction and the consistency of the user experience, making it a critical measure that spans multiple aspects of IPM.
- Resilience (cross-dimensional): This dimension examines the capacity of systems to withstand and recover from disruptions. It involves assessing the reliability, reachability, availability, and performance of systems under stress, ensuring they can maintain consistent functionality and recover quickly from adverse conditions. Resilience is a fundamental aspect that influences and is influenced by all other dimensions, underscoring its cross-dimensional nature.
By following this structured assessment process, organizations can gain a deeper understanding of their current IPM maturity level and take actionable steps toward achieving a more evolved and value-driven observability practice. Conducting a thorough assessment ensures IPM strategies are not only robust and technically sound but also fully aligned with the strategic goals of the organization.
While various IPM vendors offer assistance in conducting these assessments, their approaches differ in completeness and detail. Catchpoint, for example, offers a detailed assessment and scoring across all mentioned dimensions, including the critical cross-dimensional aspects of experience and resilience. This comprehensive approach ensures all key elements are evaluated, providing a holistic view of an organization’s IPM maturity. By focusing on these critical dimensions, Catchpoint helps organizations achieve a more complete understanding of their observability practices, enabling more informed decision-making.
6. Expected Outcomes
Organizations that follow the proposed IPM maturity models can expect steady and measurable improvements in their digital operations. By systematically advancing through the maturity levels—from basic reactive monitoring to a more evolved, value-based observability—organizations will enhance their ability to efficiently manage performance, availability, experience, and resilience. The model supports and enables stronger alignment between IT operations and business objectives, ultimately leading to more reliable and effective digital services.
At the foundational levels, organizations begin by establishing essential monitoring capabilities, enabling them to track and respond to performance issues more effectively. As they progress, they develop standardized, proactive processes that enhance visibility, reduce downtime, and improve response times. Reaching the higher maturity levels allows organizations to integrate advanced predictive analytics and AI-driven insights into their IPM strategies, enabling real-time performance optimization and dynamic resource allocation.
Ultimately, organizations that evolve toward value-based observability will align their IPM practices with broader business objectives, prioritizing user experience, resiliency, and continuous improvement. This alignment ensures digital services are reliable, efficient, and capable of adapting to changing demands, providing tangible business value and a competitive edge in a complex digital landscape.
A natural next course of action would be to conduct a self-assessment and develop a roadmap for evolving maturity. Your provider can be leveraged for this effort.
7. About Darrel Kent
Darrel is an industry veteran with several decades of experience bridging technology and business disciplines with designed-for-purpose enablement of people and process to drive desired business outcomes. He’s an expert in Cloud, Infrastructure, Data Management and Governance, Sales and Marketing, Product Management and Technical Leadership. For thirty-seven years, he served as a technical leader at Hitachi, driving their Technical Sales department and providing Data Infrastructure Solutions. He’s been an IT advisory board member at Regis University, a board advisor at the University of Colorado, and is a founding member of the Colorado Institute of Technology.
8. About GigaOm
GigaOm provides technical, operational, and business advice for IT’s strategic digital enterprise and business initiatives. Enterprise business leaders, CIOs, and technology organizations partner with GigaOm for practical, actionable, strategic, and visionary advice for modernizing and transforming their business. GigaOm’s advice empowers enterprises to successfully compete in an increasingly complicated business atmosphere that requires a solid understanding of constantly changing customer demands.
GigaOm works directly with enterprises both inside and outside of the IT organization to apply proven research and methodologies designed to avoid pitfalls and roadblocks while balancing risk and innovation. Research methodologies include but are not limited to adoption and benchmarking surveys, use cases, interviews, ROI/TCO, market landscapes, strategic trends, and technical benchmarks. Our analysts possess 20+ years of experience advising a spectrum of clients from early adopters to mainstream enterprises.
GigaOm’s perspective is that of the unbiased enterprise practitioner. Through this perspective, GigaOm connects with engaged and loyal subscribers on a deep and meaningful level.
9. Copyright
© Knowingly, Inc. 2024 "GigaOm Maturity Model: Internet Performance Monitoring (IPM)" is a trademark of Knowingly, Inc. For permission to reproduce this report, please contact sales@gigaom.com.