Table of Contents
1. Introduction
This GigaOm Maturity Model report was commissioned by CloudBees.
Innovation and excellence in software delivery are core components of digital transformation. Modern applications development and CI/CD tools have expanded to include enterprise value-added features.
Most large companies are at an early stage in the effort and investment necessary to master the tools and practices of software transformation. In most cases, it takes several years of commitment to master CI/CD and reap the full rewards. However, incremental progress can be made quickly, along with increasing returns on investment.
The maturity model shows IT leaders how commitment to CI/CD produces results early and breaks down how the investment continues to grow in value. It is a powerful model for organizations considering the investment to develop buy-in among teams and decision-makers. Finally, it is a valuable reference for setting up business and DevOps teams to accelerate consistently and succeed at software delivery transformation.
GigaOm Maturity Model
This GigaOm Maturity Model provides context and expected outcomes for organizations that seek to transform software delivery with CI/CD tools and practices. It illustrates common landmarks of meaningful progress and describes the long-term value that can be expected from continuous improvement.
The maturity model begins at Level 0, which maps common characteristics of organizations with no maturity in CI/CD. It progresses through each level, characterized by growing consistency, quality, and speed in Levels 1 through 3. These stages take the longest time to achieve but return great value on investment. The final levels of the maturity model illustrate the experiences typical of organizations who have become proficient in CI/CD practices and have even become industry leaders. Not all organizations will have a goal to achieve this level of mastery, but any modern software development effort should strive for a level of maturity that ensures speed and quality.
2. CI/CD Maturity Model
2.1 Strategy
| LEVEL 0 | LEVEL 1 | LEVEL 2 | LEVEL 3 | LEVEL 4 | LEVEL 5 |
|---|---|---|---|---|---|
| • Little collaboration between IT and business partners • KPIs and other metrics are not measures of value • Inconsistent release schedule | • Identify KPIs and process to validate • Standards and policies defined • Identify experts and champions in the org • Identify business partners | • Ability to measure progress across all KPIs • Product teams spend less time on admin • Product teams have better focus due to less context switching | • Product teams focused on the right features • Development work streams are driven by KPIs and connected to the value stream • Releases composed of individual bug fixes or features • Excellent quality control | • Users are accustomed to bug-free software • Effective communication and actionable feedback loop between IT and business • Deployments routinely use progressive delivery for a gradual and methodical rollout | • Providing software products that meet customers’ expectations • Innovation is brought to market fast and incrementally • Releases transparent to the end user, the software is like a living document, constantly improving with each sprint |
| Source: GigaOm 2023 | |||||
2.2 Automation
| LEVEL 0 | LEVEL 1 | LEVEL 2 | LEVEL 3 | LEVEL 4 | LEVEL 5 |
|---|---|---|---|---|---|
| • Manual builds • Manual deployments • Manual Testing | • Automated tests available for critical functionality • Critical functionality has unit test coverage • Builds can be deployed by script | • All builds are automated • Release orchestration harmonizes manual tasks with automated DevOps • Critical aspects of the DevOps workflow represented as code | • Builds are automatically initiated and deployed to appropriate environments at every commit • Automated tests prevent breaking changes from disrupting integration environments • All DevOps workflow represented as code | • Automated rollbacks prevent downtime in critical pipelines • Stakeholders have confidence that all environments are up-to-date • Helpdesk and user training are part of the release pipeline | • All features released via managed processes and committed via user acceptance • Training documents and processes are integral to the release pipeline and are necessarily up to date |
| Source: GigaOm 2023 | |||||
2.3 Insight
| LEVEL 0 | LEVEL 1 | LEVEL 2 | LEVEL 3 | LEVEL 4 | LEVEL 5 |
|---|---|---|---|---|---|
| • Few defined DevOps metrics • Little to no visibility | • Teams collect and report key DevOps metrics • Consistency of data varies across teams • Data is available but not yet actionable • Metrics seem disconnected from results | • Visibility into process failures enables meaningful improvement • Demonstrable improvements in MTTR • Team perception of DevOps metrics has switched from scorecard to internally valuable tool | • Important Devops metrics and product KPIs are universally accessible • Problems or bottlenecks in workstreams can be more easily identified and remedied • Test environments mirror production | • DevOps metrics are readily and consistently available for all systems and products • Value can be mapped to different project streams and workflows • Using data to become truly customer centric | • Robust value stream management guides investment in future work, ensuring every iteration delivers maximum return • Impact of all new features is measured through end user analytics |
| Source: GigaOm 2023 | |||||
2.4 Processes
| LEVEL 0 | LEVEL 1 | LEVEL 2 | LEVEL 3 | LEVEL 4 | LEVEL 5 |
|---|---|---|---|---|---|
| • Separate environments are inclusive of individual systems • Releases have monolithic feature sets | • Environments “map” to analogs in interdependent systems or products • Merges are difficult and time consuming • Rapid onboarding of new DevOps team members XXX QA primarily in dedicated integration environment • Changesets span multiple bugs/features | • Developers merge their local repositories at least daily • Changesets are aligned to individual issues • Merges become less time consuming • Practiced backlog management • Feature flags are used occasionally | • All changesets align to individually tracked issues and distinct branching path • Release orchestration eliminates scheduling bottlenecks between product teams • Selective deployments (A/B testing, canary testing, blue/green deployments) for major releases | • Consistent set of integration environments used natively across the business vertical • Bug fixes and small changes are released routinely and seamlessly | • Consistent set of integration environments used natively across the enterprise |
| Source: GigaOm 2023 | |||||
2.5 Governance
| LEVEL 0 | LEVEL 1 | LEVEL 2 | LEVEL 3 | LEVEL 4 | LEVEL 5 |
|---|---|---|---|---|---|
| • No code repository or used only in an archival capacity • Inconsistent application of security and architecture guidance • Security is an afterthought to development | • All developers use a code repository • Development and production have controls to facilitate separation of concern • Incremental alignment to stages of a secure software dev lifecycle | • DevOps workflows are secure and repeatable • Developer effort is not tied up reactively to compliance or security activities | • Compliance audits conducted easily and comprehensively • New guidance from security or architecture is folded into the development process early and seamlessly | • Compliance metrics generated proactively • All products advance toward architectural goals | • Consistently exceeds standards for security and data privacy • Automated real-time verification of security compliance |
| Source: GigaOm 2023 | |||||
3. CI/CD Primer
At the heart of CI/CD is the “make it happen, and fast” principle of software-based innovation, driven by concepts such as:
- In the time taken to develop something properly, the world may have changed.
- End-users do not necessarily know what they want.
- It can be challenging to work out the ramifications of a solution in advance.
- There are benefits to having all teams (developers, operations, and infrastructure engineers) collaborating on the same CI/CD platform.
- Value must be delivered continuously to customers.
- Failing fast is a virtue: Managers must determine whether deployment is good enough (or not) through a test rollout.
- New business models are best discovered through experimentation and fast feedback.
- It can be more cost-effective to develop a new solution than to maintain an old one.
Many enterprises see software-based innovation as the key to unlocking digital transformation success. As less-advanced organizations seek to gain first-mover advantage, a significant metric for CI/CD is delivery frequency and masters of the art reporting multiple deliveries per day or even per hour. This need for speed continues across agile and DevOps practices. However, the core mantra of CI/CD is evolving beyond speed and into a key practice and necessary source of data for product teams to seamlessly cement their daily work into a managed value stream.
The tools and practices that enable CI/CD have grown to emphasize and better support this transformation, as addressed in Figure 1.
Figure 1. CI/CD Transformation Support
4. Expected Outcomes
As organizations grow in CI/CD maturity, they can expect to undergo a number of important changes, which progressively impact consistency, quality, and speed while returning significant value on investment. Ultimately, organizations proficient in CI/CD practices can become proactive and transparent, establishing a feedback loop that informs effort and improves value across the board. Among the changes to expect:
- Frequent software delivery: Speed and volume of delivery is an early and easy-to-measure benefit of CI/CD initiatives.
- Aligned effort: As CI/CD matures, organizations can identify and focus effort on things that bring real value to the business (value stream management) rather than simply increasing the pace of delivery.
- Increased productivity: Automation, orchestration, and process optimization remove bottlenecks to enable more efficient operation across teams.
- Quality focus: Over time, the focus shifts from delivering quantity to delivering value, resulting in higher-quality software with fewer defects and a welcome decrease in operational interruptions.
- Compliance confidence: Auditing and regulatory compliance snaps into focus in a mature CI/CD environment, where software is continuously refined to meet targets.
- Frictionless collaboration: The adoption of DevOps practices across the operation enables seamless interaction between DevOps and various enterprise teams such as security and architecture. Everyone speaks the same language.
- Intentful progress: Effort is informed by detailed and timely insight into the progress product teams are making toward enterprise initiatives and key performance indicators.
5. About Matt Jallo
Matt has over twenty years of professional experience in information technology as a computer programmer, software architect, and leader. An expert in Cloud, Infrastructure and Management, as well as DevOps, he’s been an Enterprise Architect at American Airlines, has established disaster recovery systems for critical national infrastructure, oversaw integration during the merger of US Airways with American Airlines, and developed web-based applications to help small businesses succeed in e-commerce while an engineer at GoDaddy.com. He can help improve enterprise software delivery, optimize systems for large scale, modernize or integrate software systems and infrastructure, and provide disaster recovery.
6. About GigaOm
GigaOm provides technical, operational, and business advice for IT’s strategic digital enterprise and business initiatives. Enterprise business leaders, CIOs, and technology organizations partner with GigaOm for practical, actionable, strategic, and visionary advice for modernizing and transforming their business. GigaOm’s advice empowers enterprises to successfully compete in an increasingly complicated business atmosphere that requires a solid understanding of constantly changing customer demands.
GigaOm works directly with enterprises both inside and outside of the IT organization to apply proven research and methodologies designed to avoid pitfalls and roadblocks while balancing risk and innovation. Research methodologies include but are not limited to adoption and benchmarking surveys, use cases, interviews, ROI/TCO, market landscapes, strategic trends, and technical benchmarks. Our analysts possess 20+ years of experience advising a spectrum of clients from early adopters to mainstream enterprises.
GigaOm’s perspective is that of the unbiased enterprise practitioner. Through this perspective, GigaOm connects with engaged and loyal subscribers on a deep and meaningful level.
7. Copyright
© Knowingly, Inc. 2023 "GigaOm Maturity Model: CI/CD" is a trademark of Knowingly, Inc. For permission to reproduce this report, please contact sales@gigaom.com.