GigaOm Survey Report: Delivering Application Performance in a Hybrid World

1. Executive Summary

This GigaOm survey, of 352 senior and technical decision makers across North America and Western Europe, assessed architectures, challenges, and approaches to building and managing performant applications. The survey was commissioned by SolarWinds, following a similar survey conducted two years ago. This has enabled comparisons to be drawn between evolving behaviors, challenges, and responses.

Key findings are:

• There is an imbalance between strategy and reality for cloud-based versus hybrid approaches. Whereas only 43% of organizations favor a hybrid strategy for their cloud applications, 56% have a hybrid application architecture. 70% of respondents saw customer experience as a primary driver for cloud-first. Only 50% of respondents saw the lower cost of delivery as a primary driver, suggesting a move beyond saving money as a primary criterion.
• Application complexity is the biggest operational challenge organizations face, according to 51% of the overall sample. This is driving organizations that would prefer a cloud-based approach towards unplanned hybrid models.
• Looking at operational management and observability, real-time performance measurement is the highest priority operational capability for 64% of respondents. We can also see the role of large language models (LLMs) and artificial intelligence (AI) to aid operational management.
• Drilling into features, existing tooling is making a difference: identifying performance improvements is the number one benefit for 64%. Most in need of improvement are higher-order features such as traces and business/retail metrics.
• For organizations struggling with their cloud-first aspirations, we learn from more advanced organizations regarding DevOps adoption and success in adopting cloud-based models.
• We found 60% of organizations with limited DevOps experience face complexity challenges to operations, compared with 46% that are optimizing their DevOps use. Similarly, 51% of those with limited DevOps experience struggle to build a picture of performance, compared with just 41% of the more advanced group. This can be associated with skills investment. 46% of the limited DevOps group say they lack operational skills, compared with 30% of the optimizing group.
• Similarly, 81% of the cloud-native group prioritize a real-time view of performance, compared to 60% working in legacy/virtualized environments. Meanwhile, 60% of cloud-first and 58% of cloud-native respondents favor a complete picture of performance across apps and infrastructure, compared with 48% of hybrid and 44% of legacy/virtualized groups.
• Some 65% of cloud-native respondents considered linking application performance to business outcomes important. Cloud-native organizations prioritize the business, a lesson all organizations should learn.

From the research overall, we see how performance management tools are being prioritized to address the complexity challenge and deliver on their observability goals. More advanced organizations prioritize an integrated, holistic view of application performance, drawing on measures from the top to the bottom of the stack.

To avoid creating unnecessary complexity by getting stuck in a halfway-hybrid house, we recommend taking such steps in advance. This means building skills around cloud-based and DevOps approaches, such that both become a viable destination, rather than being trapped in an unplanned hybrid state.

2. The Nature of Application Architectures Today

There is an imbalance between strategy and reality for cloud-based versus hybrid approaches, as shown in the chart for Question 1.

As shown in the first bar, 57% of respondent organizations currently have a hybrid application architecture; 20% are cloud first; 7% are fully cloud native; and 16% are running legacy or virtualized infrastructure architectures. This compares with 49%, 22%, 7%, and 22% in 2022 respectively. Broadly speaking, 6% of legacy systems have moved to the cloud in the past two years.

Question 1. What is the strategy around cloud-based approaches? And how is your application architecture structured?

This shift toward cloud-based models is not huge. Plus, whereas 57% currently have a hybrid application architecture, only 43% favor a hybrid strategy (as shown in the third bar). This suggests a gap of 14% between aspiration and reality – organizations that have adopted hybrid by default, rather than by design. If we exclude the subset of organizations favoring a legacy/on-premises architecture, 16%, the gap increases to 23% (this is the second bar).

To flip this around, 40% of organizations would rather have a cloud-first strategy, with both legacy and new applications running in the cloud. However, only 19% have succeeded thus far (or 23% when not including on-premises applications). In summary, of organizations looking to adopt a cloud-first application architecture, more than half have not yet succeeded.

Question 2. What is driving your organization to choose cloud-first?

Why is this? Let’s first drill into what is driving organizations towards the cloud-first approach. As shown in the chart, we can see that 70% of respondents saw customer experience (CX) as a primary driver for cloud-first, an increase from 64% in 2022; this is followed by improving the ability to scale, at 57% in 2024 (up from 56% in 2022).

Other drivers are important but not as important as CX. Interestingly, only 50% of respondents saw lower cost of delivery as a primary driver, suggesting decision makers are moving beyond saving money as a primary criterion. Scrutiny has moved to, “Yes, but can the application meet customer needs?”

Question 3. In which scenarios are cloud-native approaches being considered or used?

This makes sense when considering which approaches are being adopted for each scenario. Cloud-first approaches are the primary option for customer-facing applications, websites, and mobile apps, above cloud-native and on-premises approaches.

Cloud-first approaches lead other approaches across the board, with only big data/analytics being a close second from the cloud-native camp. There is relatively little adoption of on-premises approaches: only in legacy development is this greater than a quarter, or 27%, of responses.

Question 4. What are your biggest operational challenges, given your application environment?

What, then, is preventing organizations from adopting cloud-first approaches over hybrid approaches? We gain some insight when we look at the operational challenges organizations face.

Across the board, complexity presents itself as the biggest operational challenge with 51% of the overall sample struggling with application complexity. If we drill into the subsample of respondents running legacy or virtualized applications on physical infrastructure, the number of respondents increases to 67% (as shown in the bottom bar of the chart).

Indeed, the top 4 (out of 10) options are complexity-related—managing systems in parallel, getting a clear picture of application performance, or having process complexity in development and deployment. Complexity is an inevitable cause of inertia, driving organizations towards adopting hybrid, rather than cloud-first architectures.

3. Bringing in Observability and Performance

Question 5. What is a reasonable use of the term “observability” in this context?

Before we drill into the impact of this complexity challenge, we want to know: is observability seen as a marketing term or something only applicable to cloud-native applications?

Some 59% of respondents told us that the term is broadly applicable to all architecture types, and a further 10% saw it as a more general expression of operational management. Few respondents, 3%, see observability as a marketing term, and none said they defined or thought about it differently.

Question 6. What are your strategic drivers for operational management and observability across your application environment?

We gain insights into how organizations address their complexity challenges by looking at strategic drivers to operational management and observability. The need to identify application performance improvements, 68%, is top of the list, above fault diagnosis and resolution and even security. Complex environments inevitably cause performance issues, which then need to be addressed.

Question 7. How important are each of the following to deliver on your application performance and observability goals?

We reviewed which capabilities respondents see as most important. Real-time performance measurement is the highest priority operational management capability to deliver, 64%, followed by integration of database metrics at 55%. Interestingly, broader ways of using performance information are seen as less of a priority, for example, integrating with business/ops dashboards (49%) or synthesizing user experience. Everything is important, but the biggest priority right now is getting a clear picture at the point of need.

Question 8. Which areas of operational tooling do you have covered or are considering?

Looking at broader operational needs, we see the role of large language models (LLMs) and artificial intelligence (AI) to aid operational management, fault diagnosis, and resolution. These two areas are the largest areas of deployment today, at 19% and 18% of the sample, respectively.

Question 9. To deliver on your operational goals, what features have you deployed or are considering?

Drilling into deployment of features, we see that database performance features are considered the most satisfactory, followed by network performance and log/error monitoring, at 55%, 53%, and 53%, respectively. Nonetheless, work must be done to improve capabilities and features across the board: the average “mostly satisfactory” score is 48%.

Features most in need of improvement are higher-order, more recent features such as traces and business/retail metrics, each at 42%. Traces are architecturally harder to collate, and business/retail metrics have always struggled with being more important but less urgent than other features.

Question 10. Where have you seen the most benefit from adopting operational management and observability tools?

Nonetheless, current tooling is making a difference. Identifying application performance improvements is the number one primary benefit at 64%, which is the same as two years ago. While we continue to see the positive impact of tooling, keeping on top of complexity remains a chore: Only 48% of respondents see management of complex applications as a primary benefit, versus 45% in 2022, and 47% see managing across cloud and on-premises environments as a primary benefit.

4. What To Do? Learning from Leaders

So far, we have seen organizations prioritizing CX as a goal for their cloud-first applications, and many are being held back from their cloud-first aspirations due to performance challenges brought about by complexity. What can organizations do to address these challenges and deliver on their goals?

Question 11. How far are you on your DevOps journey?

We can learn from the behaviors of more advanced organizations, both through best practices and results. For best practices, we can look to those that have adopted DevOps, as this approach enables organizations to deliver better. For results, we can build a picture from those that have succeeded in being cloud-based, that is, cloud-first or cloud-native.

To confirm this line of thinking, we see alignment between more advanced DevOps organizations and the use of cloud-based approaches. Overall, 23% of organizations are advanced in DevOps, optimizing its use across the organization. By application architecture, 31% and 46% respectively are at the optimizing DevOps level in cloud-first and cloud-native environments, whereas 23% of hybrid organizations and only 5% with legacy/virtualized architectures are in this advanced DevOps group. Whether by default or by design, hybrid is not a good option for best practice.

This begs a correlation-versus-causation question: Are cloud-based organizations better at DevOps, or is DevOps experience driving better architectural practice? For our purposes, we can be ambivalent about this. DevOps practices are available to, and can benefit, all organizations and all architecture types, cloud and on-premise. We can nonetheless learn from the more advanced groups in both DevOps approaches and cloud-based environments.

Question 12. What are your biggest operational challenges given your application environment? (Broken out by DevOps engagement stage)

Looking at DevOps first, let us revisit the question of operational challenges (Question 4). When we compare the different DevOps groups, we see that those with limited DevOps experience face greater challenges than those further down the road. 60% with limited DevOps experience face complexity challenges to operations, compared with 46% that are optimizing their DevOps use. Similarly, 51% starting out find they struggle to get a clear picture of performance, compared to 41% of the optimizing group.

We can associate some of this with a lack of investment in skills. Some 46% starting out on DevOps say they lack operational skills, compared with 30% of the optimizing group. The adage of “people, process, technology” rings true—it is important to invest in skills as well as tooling.

To put a positive spin on this, more mature DevOps are better skilled, have a clearer performance picture of their applications, and are more aware of complexity than other groups. As Question 12 indicates, investing in DevOps is clearly worth making.

Question 13. What are your strategic drivers for operational management and observability?

Turning to cloud-based (cloud-first and cloud-native) versus legacy/virtualized or hybrid environments, we can see cloud-based groups prioritizing operational efficiency. Some 69% of cloud-first and 77% of cloud-native environments see increasing operational efficiency as a primary factor, compared with 46% for the legacy/virtualized group. In part, this is because they can, having already addressed areas such as complexity and performance.

For security, cloud-based and hybrid environments are similar, at 63% and 60%, respectively. However, 81% of cloud-native respondents see managing security risks and responding to incidents as a priority. We know the threat surface is more complex for cloud-native applications than for other groups. On the positive side, dealing with more of the application complexity frees up resources to address security concerns.

Question 14. In terms of operational tooling, have you deployed or are you considering infrastructure automation and configuration management?

Further analysis of the responses suggests a “don’t throw the baby out with the bathwater” approach. It might feel like the world is saying, “Legacy bad, cloud good,” but when it comes to operations, traditional environments get a lot of things right.

We can see this when we look at operational tooling. In certain areas, legacy environments held their own—for example, infrastructure automation and configuration management. As we can see, 74% of respondents in the Legacy group feel they have infrastructure automation and configuration management covered. This drops to 37% and 44% in the virtualized and hybrid groups, respectively.

The lesson from this is that virtualization/hybrid models have created a burden for operations by removing some traditional tooling and practices—cloud-first and cloud-native groups are more mature and better able to recognize their value. In transitioning from legacy-based to cloud-based approaches, the “mushy middle” of virtualization and lift-and-shift cloud migration is real, creating problems and risks if tools and practices are not considered part of the migration.

Question 15. How important are the following to deliver on application performance and observability?

What can we learn from cloud-based respondents in terms of capabilities to deliver on application performance and observability? Real-time visibility across applications and infrastructure and linkage to business outcomes jump out as factors. For example, 81% of the cloud-native group prioritizes a real-time view of performance, compared with only 60% working in legacy/virtualized environments.

Meanwhile, 60% of cloud-first and 58% of cloud-native respondents favor creating a complete picture of performance across apps and infrastructure, compared with 48% of the hybrid and 44% of the legacy/virtualized groups. This integrated view is important to enable holistic management of application performance and observability.

In addition, 65% of cloud-native respondents considered that linking application performance to business outcomes was important. Cloud-native organizations prioritize the business first, which is a lesson all organizations should learn.

5. Conclusion: Unplanned Hybrid is The Worst of Both Worlds

What have we learned? Across the board, cloud-based models have become the preferred option for both new and existing applications. In this research, we haven’t gone into detail on hosting options—some organizations might prefer to work with a single hyperscaler, while others may look to multicloud or a local provider. But architecturally, more impetus exists toward the cloud than alternatives.

It’s fair to assume that board-level executives will be ambivalent about the details of their application architectures. They want platforms for the digital elements of their businesses to ‘just work’ cost-effectively. This isn’t currently the case for every organization – we have seen how architectural complexity is an impediment to progress in this area.

While a good proportion of respondents (43%) see hybrid architectures as their strategic choice, many are stuck with these as their default position. Unplanned, hybrid-by-default, could well be the worst of both (on-premises and cloud) worlds, resulting in further complexity, which must be managed.

In response to this complexity, we have seen how performance management tools are being prioritized to deliver the real-time visibility operations leaders need in response and address the complexity they face. More advanced organizations prioritize an integrated, holistic view of application performance, drawing on measures from the top to the bottom of the stack. Performance management is thus an on-ramp to broader observability and manageability.

Question 16: What strategies are you considering to further improve application delivery, operations, observability, and your overall application architecture?

Looking more broadly at strategies to improve operational delivery, work remains to be done. Top of the list is the need for simplification and automation – toolchains can be fragmented, creating challenges that distract from creating and running applications. Meanwhile, while 53% of respondents see their performance and observability tooling as broadly satisfactory, 46% see it as a work in progress – either needing improvement, or not yet in place.

Overall, we see the need for organizations to take steps in advance to avoid getting stuck in the halfway house of hybrid cloud architectures. This means building skills around cloud-based and DevOps approaches, such that these can drive progress. We have also seen how older-school principles of good operational practice still apply in the cloud world, so organizations can build on this important heritage.

The technology landscape may have changed from five years ago, but the architectural building blocks now available to organizations—cloud-based and platform-oriented, using microservices approaches—have largely stabilized. Whether you choose a cloud-native, cloud-first, or hybrid strategy for your applications’ architecture, do so with intention and the right elements in place from the outset to minimize risks, reduce complexity, drive observability, and assure your success.

6. Annex – Summary Slides

This research considered organizations that had either broadly adopted or partially adopted cloud-native technologies. These organizations had also deployed some aspect of an application monitoring or observability solution.

A total of 352 technical and senior decision makers responded to the survey, across the USA and Canada, the United Kingdom, France, and Germany. Organization sizes were greater than 1,000 employees. (Questions 17-19)

Question 17. Which of the following describes the adoption of cloud-native technology in your organization?

Question 18. How many people are employed by your organization?

Question 19. Where are you personally located?

7. About Jon Collins

Jon Collins has over 35 years of experience in IT. He has worked as an industry analyst for a number of years and has advised some of the world’s largest technology companies, including Cisco, EMC, IBM, and Microsoft in product and go-to-market strategy. He has acted as an agile software consultant to a variety of enterprise organizations, advised government departments on IT security and network management, led the development of a mobile healthcare app, and successfully managed a rapidly expanding enterprise IT environment. Jon is frequently called on to offer direct and practical advice to support IT and digital transformation initiatives, has served on the editorial board for the BearingPoint Institute thought leadership program, and is currently a columnist for IDG Connect.

Jon wrote the British Computer Society’s handbook for security architects and co-authored The Technology Garden, a book offering CIOs clear advice on the principles of sustainable IT delivery.

8. About GigaOm

GigaOm provides technical, operational, and business advice for IT’s strategic digital enterprise and business initiatives. Enterprise business leaders, CIOs, and technology organizations partner with GigaOm for practical, actionable, strategic, and visionary advice for modernizing and transforming their business. GigaOm’s advice empowers enterprises to successfully compete in an increasingly complicated business atmosphere that requires a solid understanding of constantly changing customer demands.

GigaOm works directly with enterprises both inside and outside of the IT organization to apply proven research and methodologies designed to avoid pitfalls and roadblocks while balancing risk and innovation. Research methodologies include but are not limited to adoption and benchmarking surveys, use cases, interviews, ROI/TCO, market landscapes, strategic trends, and technical benchmarks. Our analysts possess 20+ years of experience advising a spectrum of clients from early adopters to mainstream enterprises.

GigaOm’s perspective is that of the unbiased enterprise practitioner. Through this perspective, GigaOm connects with engaged and loyal subscribers on a deep and meaningful level.

9. Copyright

© Knowingly, Inc. 2024 "GigaOm Survey Report: Delivering Application Performance in a Hybrid World" is a trademark of Knowingly, Inc. For permission to reproduce this report, please contact sales@gigaom.com.