Table of Contents
- Executive Summary
- Data Security and Governance
- Competitive Approaches and Platforms
- Test and Results
- About William McKnight
- About Jake Dolezal
1. Executive Summary
Data security has become an undeniable part of the technology stack for modern applications. No longer an afterthought, protecting application assets—namely data—against cybercriminal activities, insider threats, and basic human negligence needs to happen early and often during the application development cycle and beyond.
The need for data security has grown well beyond the simplistic features of data platforms, and a competitive industry has emerged to manage the security layer. Not only must the capabilities be thorough, but they must be easy to use and add a minimum of overhead in the process.
To measure the policy management burden, we designed a reproducible test suite that included a standardized, publicly available dataset and several data security policy management scenarios based on real-world use cases we have observed in the field. We tested three categories of data security approaches:
- Role-Based Access Control (RBAC)
- Column-Tagging Role-Based Access Control (CT-RBAC)
- Object-Tagging Attribute-Based Access Control (OT-ABAC)
We chose vendor solutions from each of these three categories. For RBAC we selected Apache Ranger, for CT-RBAC we chose Satori, and for OT-ABAC we chose Immuta. Of course, other vendors address these categories, and we have listed them below for reference. Note that while the three selected vendors were tested, their results should be closely representative of the category overall and not just their particular software.
- Apache Ranger
- AWS Lake Formation (Redshift & Athena only)
- Alation (Snowflake only)
- Informatica CDGC
- Apache Ranger + Atlas
- Collibra Protect (Snowflake only)
This benchmark report captures the number of policy changes required to manage ever-evolving data security policies seen in a modern data-driven enterprise. The more policy changes required, the more likely a required change will not take place or an error is made when implementing the change. With this study, we show the impacts of data security governance policy management in terms of:
- Dynamic versus static
We learned you should select the policy management category first, before the product.
As shown in the chart in Figure 1, Object-Tagging Attribute-Based Access Control (OT-ABAC) was the clear winner. Across both our basic and row-level security scenarios, Role-Based Access Control (RBAC) produced a total of 745 policy changes, versus 401 policy changes produced by Column-Tagging Role-Based Access Control (CT-RBAC). By contrast, OT-ABAC produced just 8 total policy changes to accomplish the same data security objectives. For our advanced use cases, OT-ABAC was the only approach able to tackle the security requirement; neither RBAC nor CT-RBAC were able to fulfill the requirement at all.
Figure 1. Basic and Row-Level Security Policy Changes by Category (lower is better)
This study exposed the strengths and limitations of extending legacy security approaches into cloud use cases. Role-Based Access Control uses static policies with very limited support for attributes. Column-Tagging Role-Based Access Control adds some dynamic and scalability advantages over traditional RBAC, but as the scenarios became more complex, we saw the policy burden grow and become fragile. The difference between these approaches and Object-Tagging Attribute-Based Access Control became clear. By leveraging dynamic variables, nested attributes, global row-level policies, and row-level security, OT-ABAC can be quickly implemented and updated compared to the two role-based methods.
Using both conventional and column-tagging, RBAC as a data security mechanism creates a heavy policy-management burden compared to OT-ABAC. Furthermore, OT-ABAC is shown here to provide scalability, clarity, and evolvability in meeting a complex enterprise’s data security and governance needs.
The assessment and scoring rubric and methodology are detailed in this report. We leave the issue of fairness for the reader to determine. We strongly encourage the reader to look past marketing messages and discern what is of value. We hope this report is informative and helpful in uncovering the challenges and nuances of data governance platform selection. You are encouraged to compile your own representative use cases and workflows and review these platforms in a way that applies to your requirements.