The new economics of data warehousing provide attractive alternatives in both costs and benefits. While big data gets most of the attention, evolved data warehousing will play an important role for the foreseeable future. In order to be relevant, data-warehouse design and operation need to be simplified, taking advantage of greatly improved hardware, software, and methods.
On the benefits side, careful location of processing to separate data integration and stewardship activities from operational and analytical activities is clearly required. The industry has moved beyond a single relational database to do everything. There are more choices today than ever before. New BI tools are arriving every day that take reporting and analysis to new levels of visualization, collaboration, and distribution. Embedded analytical processes in operational systems for decision making rely on fast, deep, and clean data from data warehouses.
Data warehousing can trace its antecedents to traditional information technology (IT). Big data and especially its most prominent tool, Hadoop, trace their lineage to search and e-business. While these two streams are more or less incompatible, they are converging. Data-warehouse providers are quickly adding Hadoop distributions, or even their own versions of Hadoop, into their architecture, adding further cost advantages to collections of extremely large data sets. Finding the talent to manage this newly converged environment will not be easy, but it presents tremendous opportunity for companies willing to take some risk.
- Data warehouse economics: yesterday and today
- A look at the numbers
- A look at the requirements
- Total cost of ownership
- “Big data:” what it really means
- Skills and staffing
- What these changing economics mean to you
- How the new realities of data warehousing are affecting operations and strategy
- What the technology market will look like in 5 years and what you should do about it
- Appendix A: the role of columnar databases
- About Neil Raden
- About GigaOM Research