Key Criteria for Evaluating Feature Flag Solutionsv1.0

An Evaluation Guide for Technology Decision Makers

Table of Contents

  1. Summary
  2. Feature Flags Primer
  3. Report Methodology
  4. Decision Criteria Analysis
  5. Evaluation Metrics
  6. Key Criteria: Impact Analysis
  7. Analysts’ Take
  8. About Bill Witter

1. Summary

Organizations today are looking to deliver software faster, and to create versions for specific geographies and audiences (such as beta testers). This creates challenges for software development teams expected to release partial functionality early on for soliciting feedback or to release full functionality in a phased manner. Feature flags (FFs) have emerged in response to these challenges.

Moving from continuous delivery to continuous deployment is an extremely valuable way for organizations to maintain a competitive edge. Continuous delivery is partly manual, whereas continuous deployment requires higher levels of automation, which also calls for a higher level of governance to be in place. FFs are used to toggle specific application capabilities on and off at execution time without deploying new code. Software development teams using continuous delivery or continuous deployment in an agile environment use FFs to manage the lifecycle of features being developed and released The benefits and use cases of this approach are summarized in Table 1 below.

Table 1: Comparative Summary of Development Stages With and Without Feature Flags

Feature Experimentation Due to manual overheads, it’s difficult to conduct and may be delayed until the feature is ready to be deployed in staging. Since the implementation is guarded by the feature flag, experimentation for the feature can begin very early in the development process by opening up the feature to a specific user/developer or group of test users/developers.
Feature Deployment Harder to do progressive rollouts or canary releases at any scale  Feature flags are a great way to implement progressive rollout across the user base. 
Feature Release Feature release is tightly coupled with the development team’s schedule.  Feature release is independent of the development team’s schedule. The feature may be complete, but not yet released. Product teams can time the release for opportune moments.
Feature Rollback Possible if a configuration variable was set up to be used as a kill switch in code; otherwise it requires rolling back a release to a previous one. A flag could be set up as a kill switch, which could be disabled in the event of an anomaly. When the flag is disabled, the feature is also disabled, so rolling back to a previous release is not required. 
Dynamic Invocation of Secondary Workflows Not possible. All flags need to be in code. As a result, it may get difficult to handle dynamic invocation of secondary workflows without developer intervention.  Flags can be set in the feature flag solution to dynamically invoke secondary workflows without the need for a developer.
Source: GigaOm 2022

FFs enable developers to wrap new functionality in inactive code paths and activate the functionality at a later time. This approach helps with validating functionality in the production environment and reducing the number of code branches and environments.

FFs are also useful in situations when features must be released for a time-bound period (especially in connection with an event) or on a fractional basis, especially if the feature is partially complete, and there is a need to solicit feedback.

FFs assist with selectively releasing new features for canary releases, which are made available to a subset of the user base. FFs help development and operations (DevOps) teams understand user behaviors regarding new functionality as part of experimentation via A/B testing. Only a subset of users are offered the new functionality and their usage behaviors can be compared with the remaining user base. In addition, FFs enable experimentation to be delegated to product teams, conserving developer time and avoiding upsetting release schedules.

This GigaOm Key Criteria report details the criteria and evaluation metrics for selecting an effective FF solution. The companion GigaOm Radar report identifies vendors and products that excel on those criteria and metrics. Together, these reports provide an overview of this category and its underlying technology, identify leading feature flag offerings, and help decision-makers evaluate these solutions so they can make a more informed investment decision.

How to Read this Report

This GigaOm report is one of a series of documents that helps IT organizations assess competing solutions in the context of well-defined features and criteria. For a fuller understanding, consider reviewing the following reports:

Key Criteria report: A detailed market sector analysis that assesses the impact that key product features and criteria have on top-line solution characteristics—such as scalability, performance, and TCO—that drive purchase decisions.

GigaOm Radar report: A forward-looking analysis that plots the relative value and progression of vendor solutions along multiple axes based on strategy and execution. The Radar report includes a breakdown of each vendor’s offering in the sector.

Solution Profile: An in-depth vendor analysis that builds on the framework developed in the Key Criteria and Radar reports to assess a company’s engagement within a technology sector. This analysis includes forward-looking guidance around both strategy and product.

Full content available to GigaOm Subscribers.

Sign Up For Free