Why big data needs real-time intelligent systems

Zubin Dowlaty was watching the movie the Fifth Element when he saw the future of real-time intelligence laid out in a pithy quote: “Anticipation denotes intelligence.” Dowlaty, vice president and head of innovation and development of analytics-outsourcing firm Mu Sigma, said that’s what companies need to be striving for and in this era of big data, the barriers to achieving that have fallen away.

Zubin Dowlaty of Mu Sigma at Structure:Data 2012
(c) 2012 Pinar Ozger. [email protected]

Speaking at GigaOM’s Structure: Data conference, Dowlaty gave an overview of how companies can stitch together their own real-time intelligent systems that can handle a growing stream of big data. He said companies first need to get into the right mindset and that means worrying more about consumption of analytics and less about the models and technology used.

He said the bigger goal is to help companies create predictive intelligent systems that can handle real-time data and shrink the amount of time it takes to act on events. He said by observing the companies that are successfully building these systems, often high-frequency traders, the necessary components come down to: messaging-oriented middleware; an advanced analytics engine; a business process modeling system; and a rules engine. Assembling these parts is becoming easier now, but companies still need to make the move.

“The biggest mistake is not doing anything,” Dowlaty told me after his talk. “Real time is hard but it’s not that hard. Big data is removing some of the excuses from a computational perspective. The technology exists, we just need to stitch it together and we’re off to the races.”

He said if pursued, this kind of intelligent systems can be applied to fraud detection, exception reporting, offer optimization, quantitive trading and sensor fusion. With the expected onslaught of sensor data coming from smart devices, there’s going be even more data to handle. But if you can anticipate what’s coming next, you’re on your way to becoming intelligent.

“It’s about man plus machine, heuristic and algorithmic coming together to operate analytics on a much quick scale,” he said.

Watch the livestream of Structure:Data here.