Weekly Update

Big data growth spirals upward to reach downward

The hype surrounding big data is starting to bear fruit beyond the enterprise market. A recent report from Gartner has projected that big data infrastructure expenditures will surpass $28 billion by the end of 2012 and grow to $34 billion for 2013.

Obviously, that is only a small percentage of worldwide IT budgets; however the rate of growth shows that big business is taking big data very seriously. Nevertheless, big data’s growth is most likely to surpass current predictions because very few analysts take into account the pervasiveness of the technology.

Today, big data and its associated analytics play mostly in the large enterprise space, where two primary elements make it viable: an abundance of data and budgets to fund the deployment of the associated infrastructure.

However, some 80 percent of businesses fall into the realm of the SMB (small and medium business) markets. Those SMBs can derive extensive benefits from big data analytics, yet do not have the infrastructure or budgets to pursue the advantages offered. That is a situation that has not gone unnoticed by the budding community of web services providers, many of which are developing platforms to offer big data analytics in bite sized chunks, either on a per use basis or a time share model.

For example, Hortonworks offers a hosted Hadoop Data Platform (HDP) on a per-cluster basis with 10 nodes costing $12,500 per year. Other hosts charge based upon processing time and total data set size — for example, Amazon Elastic MapReduce, which charges $0.32 per hour per on-demand server instance and $0.16 per hour of M5 Level MapReduce processing. Of course, you will need to buy the appropriate hosted storage as well.

That said, having access to affordable platforms is only the beginning for a small business seeking to process big data. Those businesses, of course, also need the data to analyze as well. That creates a conundrum: How can a business benefit from big data analytics if they don’t have any big data to begin with?

Luckily, there is no hard-set rule that dictates a business can only use internal data for analytics. Small businesses can turn to multiple sources to leverage public data, such as census information, demographics, weather, financial data, and many other potential sources.

For example, the Small Business Administration has made available a tool called SizeUp, which offers data related to all the small businesses across the United States. That data can be integrated into an analytics platform to create several determinations, such as where to open a branch office or target a specific demographic for new services. If combined with other data sources, such as sales demographics, weather, census, and so on, the potential for analysis is almost unlimited.

Another example of obtainable data comes from Square’s small business register, which is an alternative point-of-sale system for small businesses that provides free business analytics. While what Square offers isn’t really big data, the granular data, such as your average payment amount by time period, can be incorporated into formulas that process big data from public sources, can perhaps give additional insight to market potentials. Square makes it a little simpler by combining analytics and data gathering capabilities together, so that small businesses can understand their customers and payment data — analysis that was previously complicated to track and crunch.

Those are just a few examples of where SMBs can gather data. Others include the ever-growing sphere of social networking, where Facebook posts, Twitter feeds, and multiple other sources can all be incorporated into analytics to determine everything from customer sentiment to fashion trends to political decisions.

Small businesses, like large enterprises, have a lot to gain by understanding the big data surrounding their businesses. With the rise of new cloud-based business intelligence tools, small businesses are now empowered to improve their operations and efficiency by analyzing the data surrounding their business.

Once the word gets out and aggressive marketing takes hold, many SMBs will undoubtedly jump on the big data bandwagon, thanks to hosted services and abundantly available public data. Let’s just see how the market growth numbers will be affected once the sleeping giant of SMB awakens.