Key Criteria for Evaluating Penetration Testing as a Service (PTaaS) Solutionsv2.0

An Evaluation Guide for Technology Decision-Makers

Table of Contents

  1. Summary
  2. PTaaS Primer
  3. Report Methodology
  4. Decision Criteria Analysis
  5. Evaluation Metrics
  6. Key Criteria: Impact Analysis
  7. Analyst’s Take
  8. About Chris Ray

1. Summary

Penetration testing, a cornerstone in cybersecurity, has for many years been the go-to technique for security professionals looking to uncover vulnerabilities in their systems and applications. This method of simulating attacks aids in achieving better practical security outcomes and helps organizations to be compliant with ever-evolving regulatory mandates. The in-depth insights provided by penetration testing are invaluable because they shine a light on hidden flaws and vulnerabilities, empowering security teams to reinforce their defense mechanisms more effectively.

Yet, traditional penetration testing, sometimes referred to as legacy pen testing, presents its own set of challenges. Typically, such tests hinge on the proficiency of a handful of experts, usually one or two penetration testers (or “pen testers”). This dependence can sometimes act as a bottleneck, potentially narrowing down the scope of the test to less than what was originally intended or impacting its overall quality. Given the limited availability of specialized pen testers within many legacy service providers, arranging for such tests frequently necessitates a prolonged wait—sometimes weeks or even months. Additionally, after the completion of the test, organizations might still find themselves waiting for an extended period before they receive a comprehensive report detailing all identified vulnerabilities.

In contrast, penetration testing as a service (PTaaS) magnifies the effectiveness of traditional penetration testing and introduces functionalities reminiscent of modern software as a service (SaaS) platforms. This includes a user-friendly interface, allowing clients to seamlessly access consolidated findings, potentially as they’re uncovered, and facilitates real-time interactions with the pen testers. PTaaS also offers a more systematic approach with standardized testing methodologies and robust integrations with a range of other contemporary technologies.

Even though penetration testing methodologies have been refined over many years, PTaaS is still in its nascent stages. Given this novelty, we can expect the definition and the services encompassed by PTaaS to undergo significant evolution in the near term. There’s potential for integration of newer services, including but not limited to attack surface management (ASM) and continuous vulnerability management (CVM). These services naturally align with the overarching goals of PTaaS.

This is the second year that GigaOm has reported on the PTaaS space in the context of our Key Criteria and Radar reports, and the need to streamline penetration testing has only continued to grow. This report builds on our previous analysis and considers how the market has evolved over the last year.

This GigaOm Key Criteria report details the capabilities (table stakes, key criteria, and emerging technologies) and non-functional requirements (evaluation metrics) for selecting an effective PTaaS solution. The companion GigaOm Radar report identifies vendors and products that excel in those capabilities and metrics. Together, these reports provide an overview of the category and its underlying technology, identify leading PTaaS offerings, and help decision-makers evaluate these solutions so they can make a more informed investment decision.

How to Read this Report

This GigaOm report is one of a series of documents that helps IT organizations assess competing solutions in the context of well-defined features and criteria. For a fuller understanding, consider reviewing the following reports:

Key Criteria report: A detailed market sector analysis that assesses the impact that key product features and criteria have on top-line solution characteristics—such as scalability, performance, and TCO—that drive purchase decisions.

GigaOm Radar report: A forward-looking analysis that plots the relative value and progression of vendor solutions along multiple axes based on strategy and execution. The Radar report includes a breakdown of each vendor’s offering in the sector.