Bringing big data to bear on big security

If you weren’t paranoid about data security before, you probably are now after a week’s worth of  headlines:

• Cyber 9-11 threat could be imminent says Homeland Security Secretary Janet Napolitano.
• the Pentagon will quintuple its cybersecurity forces in response to fears of a “Cyber Pearl Harbor”; 
• Your own Linksys or D-link router may have betrayed you to hackers.

It’s enough to make you unplug, if that were an option.

But big security vendors are scrambling to meet the threat (and take advantage of these headlines) by touting how they are incorporating big data smarts into their latest offerings. RSA on Wednesday announced RSA Security Analytics -- which brings together the company’s NetWitness forensics and Envision log management and the GreenPlum MapR distribution, according to Amit Yoran, SVP of and GM of RSA’s Security Management & Compliance Business.

Amit Yoran, SVP and GM of RSA’s Security Management & Compliance Business Unit .

IBM’s on board with IBM Security Intelligence with Big Data, that ties together IBM Security QRadar SIEM and Big Data Platform among other things. Sandy Bird, CTO of IBM’s security systems division told the Wall Street Journal that this software can help “CIOs detect internal and external security threats in new ways—and can even scan email and social media to flag apparently ‘disgruntled’ employees who might be inclined to reveal company secrets.”

And Cisco Systems is buying Cognitive Security, a privately-held network security company that uses artificial intelligence to detect advanced cyber threats, according to the Cisco blog announcing the acquisition.

Big opportunity, crowded market

Big data security analytics, says Jon Oltsik, senior principal analyst with Enterprise Strategy Group, is obviously a hot market. Other contenders include SAIC and startups like RedLambda, PacketLoop, ZettaSet, Sumo Logic and  Palantir Palentir. The PacketPig open source project focuses on security and analytics, he said.

All these players see a chance to apply big data expertise to tilt the balance of power against cyber villains and because of big data’s ability to handle streams of log data and credentials, it does change the game, making it easier to prioritize threats and examine user behavior to spot anomalies, as GigaOM’s Derrick Harris has reported.

Traditionally, companies found threats by examining server and network logs to tell them if their systems had been accessed. It was a rear-guard, after-the-fact reaction. Proper use of big data analytics can accelerate the process. The idea is to apply analytics to this data as  it flows into the network and watch for patterns. “That lets you spot things that you would not have seen till after the fact before or would have thought was a blip,” said Judith Hurwitz, president of Hurwitz & Associates, an IT consultancy. The upshot is companies can shut threats down faster or possibly even prevent a breach by detecting hacker activity before it gets expensive.

Goal: better, faster info on digital evil doers

These new offerings promise to give companies a full picture of who’s coming into their network, who’s talking to whom, and spot anomalies or atypical user behavior while it is still actionable.

“If Johnny used five IP addresses and four user IDs and 12 different accounts  … the time to detect that activity will go from many hours to a few minutes.  This is a simple use case, not sci-fi.”  said Paul Stamp, director of product marketing for RSA.

Of course, this is an arms race. The good guys build and deploy cool new technologies, then the bad guys — no fools — use the same types of technology to overcome those defenses.

RSA’s executive chairman Art Coviello conceded as much at a press briefing Tuesday at RSA headquarters in Bedford, Mass. “It’s not about perfect security, its all about ratcheting down risk as much as you can.”

To learn more about the opportunities and challenges in the era of big data, check out GigaOM’s Structure Data event March 20-21 in New York.