Data — massive amounts of it — are emerging more quickly than ever before thanks to always-on networks, sensors and the web, and companies are increasingly turning to new software and database tools to help make sense of it. But how do you know if your company is experiencing the influx of “big data?” According to the panelists at the GigaOM Network’s Structure conference on Wednesday, here are five signs:
- You Do Complex Calculations: Amr Awadallah, CTO and co-founder of Cloudera, says it’s not necessarily about the size of the data stream, but what you want to do with it. Cloudera, which has developed a business of commercializing the Hadoop open data base platform, has one customer with a tiny amount of data, but the need to do very complex calculations with it in a matter of seconds every night. In contrast eBay (s ebay), one of Cloudera’s biggest customers, runs complex algorithms across a massive amount of data in order to try to find new ways to match products with people, rank search results and do fraud detection.
- You’re Drowning in Discs: Once data goes to tape it’s in a storage graveyard, and you’ll never see it again, says Awadallah. Making sense of the data in real time, using new tools, means that companies can make use of data that would normally move into storage before any use is drawn from it. “You know you have big data when you have to run out every week to buy discs,” says Awadallah.
- You Need New Tech to Stay Afloat: Damian Black CEO, SQLstream, says you know you have big data when you need new technology to just stay on top of it and survive. “When you’re unable to keep up with rapidly arriving data and need to do equations on the data in real time.”
- You Want Your Info Now: Barry Zane Founder and CTO, ParAccel, says when you need answers from your data in “seconds, minutes or hours,” and meeting that time commitment with traditional databases and approaches becomes a daunting task.
- You to Have to Deliver Answers: Will your company move in front of the competition if it can mine data in new ways to deliver answers for customers, asks Zane? That kind of flexibility can be invaluable and if the answer is “yes,” then you’ve got big data. For example, companies that fall into this category are financial and banking services, credit card companies, Internet search and advertising companies.