There’s more than a little truth in Google Chief Economist Hal Varian’s quip that “— the sexy job in the next ten years will be statisticians.” With enterprise data volumes moving past terabytes to tens of petabytes and more, business and IT leaders face significant opportunities and challenges from big data. For a large enterprise, big data may be in the petabytes or more; for a small or mid-size enterprise, data volumes that grow into tens of terabytes may become challenging to analyze and manage. Big data, though, is not just about cost control — there are sales-growth opportunities and even new business models from more advanced analytics.
Using cloud-computing technologies, organizations are experimenting with distributed data stores, cloud compute capacity for data analytics, hosted data integration and even operational databases in the cloud. Hadoop/MapReduce has moved past test/development to become a viable extension or alternative to traditional relational databases for managing and analyzing huge data sets. All of this makes 2011 an exciting year for benefiting from and managing big data.