Over time, we’re generating massive amounts of new data, thanks to a growing number of connected devices and services. As this Big Data gets bigger, it becomes a challenge to gain insights through traditional database queries: More data to sift through means more lag time between a query and actionable results. Parallel processing is one way to solve this problem, but streaming queries can help.
Using the gaming world as an example, Black likened this to a continuously updated real-time leaderboard. “By streaming big data, analyzing and processing millions of data bits and reacting to the output,” a massive amount of data can be queried in a far shorter time period.
But this approach isn’t unique to online gaming, Black suggested. Any data that fits what he called “S3 data” can take advantage of this approach; specifically data from sensors, systems and services.
That suggests a broad range of uses, especially as more connected consumer devices gain the use of sensors. And the number of services is expanding too; Black noted that streaming queries are ideally suited to services such as text messaging, Twitter and even real-time pricing information. The days of instant price changes based on true supply and demand may be a SQL query away.
Watch the livestream of Structure:Data here.