Say what you will about the National Security Agency, but you can’t say it doesn’t know how to share — or how to build technology that can scale. In fact, Accumulo, the petabyte-scale database technology the agency built, has been available as an open-source project for a couple of years. Now, however, a more-polished version of Accumulo is up for sale to the general public thanks to a Cambridge, Mass.-based startup called Sqrrl.
On Wednesday the company announced the general availability of its product, Sqrrl Enterprise, which is a cleaned-up and more-functional version of the open source Accumulo software. That means users will get an experience a lot more similar to what NSA data analysts get than what the core database code allows.
How do we know this? Because Sqrrl’s co-founder and CTO Adam Fuchs helped build Accumulo and the applications that run on top of it during his previous life working for the spy agency. (If you want to know more about the history of Accumulo and the types of massive graph analyses the NSA is using it for, you can check out my coverage of the NSA citizen-spying scandal from two weeks ago (here and here).) So, instead of just downloading an open-source take on Google’s BigTable data store, Sqrrl users get things like built-in analytic functions and search; support for JSON data structures; and data encryption both at rest and in motion.
It’s the latter features around security that Sqrrl Co-founder and VP of Business Development Ely Kahn said have many early Sqrrl users most excited. Health care companies, in particular, highlight an ideal use case for security features like those that Sqrrl provides. Because of its cell-level security and access control, Kahn explained, providers can try to do new things around data sharing while still keeping compliant with regulations such as HIPAA and the data requirements that come along with the Affordable Care Act.
But the applications of Accumulo and Sqrrl could be much broader across industries. Because it’s based on Hadoop, Sqrrl gives companies peace of mind when it comes to storing big data securely, Kahn said, which has been a big reason that many companies are afraid to do Hadoop in production. And Sqrrl’s analytic capabilities make it easier to analyze all that data, including log files and network data that could help a company track down the causes of any cyberattacks they might suffer.
At this point, said Kahn, who was previously director of cybersecurity strategy at the National Security Staff in the White House, that should be a major concern. For most organizations, he said, it’s not a question of whether they’ve been breached but “a question of whether they know that they know they’ve been breached.”