Analyst Report: Instinct meets evidence: using operational data to drive planning

1 Summary

Many believe big data analytics are always the answer. In reality, most sales, financial, and operational planning is still run on incomplete data sets. Why? It’s hard to find the right people who can help enterprises understand their data. Understanding the efficiencies of product lines and market factors are perhaps the most important aspects of understanding the business. However, the traditional approaches to analytics largely overlook these aspects. Business is lost, a company’s value moves downward, and most companies have no clue why it happened.

A sale gained and a sale lost are not typical data points that are considered in the planning process alongside the historical sales of accounts, market growth, and new product offerings. While some IT-built reporting tools provide on-demand information, they tend to work within very rigid constraints. Without timely access to the data needed to make daily decisions, operations managers rely upon instinct over evidence. This is no way to run a business.

To take full advantage of what modeling and analytics can offer for effective planning and execution and integrate data into every decision process, businesses must put flexible tools directly into the hands of operations. This paper will evaluate the challenges and potential for creating models and applications and making those tools available and useful to those who need the information the most.

Key findings include:

  • The democratization of data – putting the right data into the hands of all of those who need it – has a huge potential to provide a positive change to the efficiency of business planning operations.
  • Automated, data-driven planning tools will empower those on the front lines of the business with on-demand data to provide effective and dynamic business planning.
  • A single modeling engine should work across all cloud and on-premise applications to provide an abstraction layer of information that will optimize enterprise resources without involving strategic leadership.
  • New approaches to leverage operational data are essential. Planning is no longer in the realm of the business executive. Now it’s the operationally focused manager who makes tactical plans that drive value directly to the bottom line of the business.

2 Understanding the problem

According to an article by Mike Myatt that appeared in Forbes:

“The first key in understanding how to make great decisions is learning how to synthesize the overwhelming amount [of] incoming information leaders must deal with on a daily basis while making the best decisions possible in a timely fashion. The key to dealing with the voluminous amounts of information is as simple as becoming discerning surrounding the filtering of various inputs.”

The problem is that those charged with the operations of a business have very little insights into the information that they need. Thus, those who lead the business don’t have a solid and up-to-date understanding of exactly what the business is doing. In turn, they can’t lead effectively and the business suffers.

There are three core issues that businesses must consider around the use of data to drive planning:

  • Lack of pertinent data refers to the fact that most in management positions and charged with business planning can’t see all of the data they need to do their jobs effectively and create the proper plans to guide the business. The frontline managers and executive leaders need information that will provide the best understanding of the “as is” and the likely “to be” state of the business. IT has not done a great job of getting this information delivered to the right people, at the right time, in the right format.
  • Lack of timely availability of insights to operational teams refers to the fact that those who are actually on the front lines of the business can’t gain access to the business data they need. For example, a warehouse manager lacks the ability to see that a chronic lack of inventory cost 30 percent of sales in the last year, and the manager lacks information to solve the problem or provide better planning.
  • Operations don’t have the flexibility to model and view their business. Not only is required data and information lacking but operations also lack the ability to see the information in ways that are meaningful to them. This means seeing the right levels of data abstraction and details and having the ability to compare current and historical data to make key determinations around the health of the business and thus the actions that must be taken immediately.

Inefficiencies in current manual and automated planning

Considering the issues we’ve covered thus far, the core problem is the lack of efficiencies with manual and automated planning. The solution? The right information must go to the right people in the organization who will allow them to create more accurate plans.

The end result is the ability to create meaningful and accurate plans that can drive the business in the right tactical and strategic directions. Indeed, the efficiencies gained by creating the right plans with accurate and timely information may actually lead to the realization that it might have been better for businesses to not plan rather than plan using bad or out-of-date data.

The value of planning

According to a study by Entrepreneur Weekly, the Illinois Small Business Development Center at Bradley University, and University of Tennessee Research as reported by Statistic Brain, the No. 1 reason for startup business failure cited in the study was incompetence, coming in at 46 percent, with a major specific pitfall listed as lack of planning (see figure below).

Startup business failure rate by industry. The study also revealed that the major cause of the failure is lack of planning.


Source: Statistic Brain

Furthermore, as stated in a Deloitte report*,when leveraging the proper planning approaches and planning technology, businesses are able to avoid the planning and forecasting issues that plague other companies.

Simi Mehta, a senior manager at Deloitte Consulting and part of the Sales Force Effectiveness Practice, has some very practical and useful advice. In providing sales strategy and talent consulting as well as sales execution planning and performance management consulting, she has seen firsthand the issues caused by the lack of planning and modeling technology around sales planning and forecasting. Mehta advises that organizations need to enable people to focus on more value-added activity, thus removing them from having to manage the data and instead focusing on what the data means. Automated planning tools provide not only access to the data that’s important for those tasked with planning but also the ability to make sense of the data without becoming data experts themselves. This provides planning automation as well as data accessibility and turnkey analytics that get the automated planning system user to the right plan at the right time with the correct data to support the plan. In the case of Mehta’s clients, sales forecasting planning becomes more accurate and responsive to changes in the business.

The Deloitte report supports Mehta’s assertions, including the planning technologies’ ability to provide:


Mehta also advises that the planning tool’s ability to create and support the right planning model is wholly dependent upon the quality of data that’s provided. Moreover, the use of cloud-based tools provides cost efficiencies, the ability to quickly scale the planning activities, and the ability to quickly access the information required in support of planning.

*Undertaking successful Sales Execution Planning & Management requires insights, capabilities, and collaboration tools. An inconsistent mix of these aspects can lead to uncertainty, surprises, and missed targets — all costly mistakes. 

3 Defining a new path

To take full advantage of what modeling and analytics can offer for effective planning and execution and to integrate data into every decision process, businesses must put flexible tools directly in the hands of operations. While this seems like a simple task, it means a change to many of the existing business and planning processes to incorporate these automated tools as well as to make sure that required information is accessible and available.

What works and what does not work

Limitations of existing approaches, including the lack of efficiencies for manual and automated planning, are clearly core to the problem. The continuing movement to business process automation and big data systems lacks effective support for the planning process. This includes a lack of methods to arm those who drive the business with the right information at the right time, in the right format, and automated whenever possible.

The need for a modeling engine that can work across all cloud and on-premise applications within an organization is another approach that most enterprises should consider. Those who believe that we can easily make good planning sense of the widely diverse data that lives in complex structures will soon find the shotgun method is not an effective approach to bringing the right information to those who drive the company. While single model approaches are not always effective, the ability to meld the data into a single unified model specifically built for the operational planning process is becoming a best practice.

An effective approach when it comes to planning is to perform sophisticated analytics that remove the user from the underlying complexity of the data as well as unifying large amounts of transactional data that is easily accessible by the end user. This allows technology to place the right abstraction layers across the existing data, which allows the business user to view data in ways that are more effective to support the short- and long-term plans they need to create and the operational decisions that they need to make.

Moving to data-driven planning

New approaches to leverage operational data seem to be the key to most of these problems. As businesses learn more about analytics, it’s been well-documented that the value is higher within operational data that may be analyzed and abstracted for use in automated planning operations. This is no longer just in the realm of the business executive but also with the operationally-focused managers who make tactical plans that drive value directly to the bottom line of the business. Removing most of the planning inefficiencies means more potential revenue and a smaller amount of lost revenue.

Data-driven planning means that we attack planning from the data to the plan and rely more on real and current information. The plans created are able to take into account dynamic and transactional data, thus the planning and execution processes defined are adjusted automatically as the operational data changes. This provides a few key benefits, including:

  • The ability to take the guesswork out of planning, including leveraging data to drive the process instead of just the experience of the planner.
  • The ability to spot and capitalize on trends in the data, such as areas of the market that should get more focus.
  • The ability to automatically adjust to changing market and operational dynamics, such as understanding the effect of changes in key market indicators and how the indicators affect the business. For example, some businesses are inversely affected by economic downturns but by very different percentages. These are easily modeled by allowing users to build driver-based plans where they can see what-if forecasts and test different scenarios
  • The ability to customize planning for specific industries, considering that planning in the world of consumer packaged goods (CPG) is very different than planning in the world of retail. Moreover, you have to consider the regulatory environment and build those into your automated planning processes and use of data.

The democratization of data is directly related to data-driven planning. It is the process of expanding business information and the tools to analyze it out to a much broader audience than has traditionally had access. In this case, it’s about putting the data-driven planning tools in the hands of those with operational responsibilities in the business who can also make the best use of the data within the planning tool.

In most companies, the IT department has long been the gatekeeper of the data, business intelligence (BI), and analytical tools. This is not because of a desire to control information but out of necessity. The tools available were simply too complex for the average employee to use effectively. Also, there is a lack of patience for two- to three-year implementations with requirements that are gathered using “waterfall” types of approaches and thus are out of date by the time of implementation What’s more, analyzing data was a cumbersome and complicated process that required layers of data and BI experts to actually make heads or tails of the outcome.

As a result, reporting and planning was most often done to satisfy the needs of a handful of top executives. In most cases, those executives did nothing helpful with that information or often did not trust the authenticity of the data. Complex and distributed technology landscapes compounded these issues, as siloed systems made it more difficult to get a unified view of the business. Indeed, most companies find it difficult to get a single version of the truth across all areas of the enterprise. It’s a systemic problem that grew worse in recent years with the use of big data and the migration of systems out to public cloud platforms.

Thus, the democratization of data movement is the process of putting the data in the right hands, in the right ways, at the right times, and in the context of planning tools that can be driven by the end users without specialized expertise. This empowers the end user to make use of the data planning processes and thus have a better understanding of the state of the business without having to wait for reports to be generated by the IT department, which are typically too late and inaccurate and lack field-level knowledge

Open data model

Rather than track relationships at a cell, formula, and dependency level, emerging data models track dependencies in much more fine-grained ways. These data models are built specifically for data-driven planning. They have the intelligence to update only those portions of the dataset that have changed. Thus, your data volumes scale, but complexity doesn’t.

The purpose of open models is to build a structure that closely resembles your business, letting process drive technology requirements. Information is dynamically brought into these models. The models become the single version of truth for those in operational roles who have to perform planning activities and want automated approaches and tools to provide the value we’ve outlined in this paper thus far.

Physical and logical data models enable you to update or change models of any size instantaneously online. You need to simply update a key driver and see changes reflected throughout your model. Thus, you’re able to gain visibility into the impact your decisions will have on business outcomes.

The use of this data model as a means to provide physical abstraction of the data for use in data-driven and automated planning is a requirement. Consider the limitations of the existing database models and database technologies. The objective is to simplify the use of the data by hiding the complexity of the distributed enterprise data behind a single pane of glass for use by operational managers that require a turnkey system to access the data for planning purposes.

New use cases and new business value

Given the use of data-driven planning, the democratization of data, the introduction of new data models, and automated planning tools, let’s focus on the emerging use cases and the value that this technology brings. While the number of use cases is virtually unlimited for automated and data-driven planning technology, a few examples that provide good value include:

  • The ability to provide product planning that’s better aligned with past and emerging market trends. This means understanding the likely direction of the market, in terms of demand for a product or service. This may include what the market will require in terms of changes to the product and how many additional sales can be expected once the product changes are implemented. The data is put into the hands of the people who can become most effective with it, and thus, millions of dollars in additional revenue can be generated by updating the product line to better align with customer demand.
  • The ability to properly analyze human resources information within a professional services company to adjust for changing client demands. HR needs the ability to access relevant data that shows when the demand will appear as client markets evolve. Thus, they need to plan accordingly for staffing. This planning system should save the professional services organization millions of dollars per year in terms of increased utilization of resources and client satisfaction.

Moving stepwise to data-driven planning

So how do you adopt data-driven and automated planning to get the value you seek? This is the seven-step process that we recommend:

  • Step 1: Define your business, including what’s important to your business. This means the key ways to measure success and the models that show how the business gets revenue. This is important because it layers down to your planning process, including selection of your automated planning tools and processes.
  • Step 2: Define existing planning approaches, and include how manual planning currently occurs. This is important because you build automated planning processes off the existing manual approaches and then expand upon those approaches. Be sure to empower end users to access the data directly using automated planning tools that they can drive without the assistance of IT.
  • Step 3: Define existing data and data types, including a common model that combines and represents most of the major business entities, such as sales, inventory, products, services, etc.
  • Step 4: Define data gathering approaches, including how data will get from operational data stores to combined data models and storage that will power the automated planning tools.
  • Step 5: Define a target data model and access approach. Determine where the planning data will reside, including the data model, and how the tools will access the data.
  • Step 6: Select and implement tools. Look for the best-of-breed automated planning tools, and figure out how those tools will be implemented within the enterprise, including the technology rollout to end users.
  • Step 7: Measure value. Define the metrics to measure the value of these new processes and new tools. Use the data provided in this paper as a jumping-off point to define your own metrics. You’ll have to understand how your business values affect planning and define your own approach to get to the reality of the value.

4 Key takeaways

While the use of data-driven automated planning clearly provides value to most businesses, each enterprise is different. The approaches and technologies leveraged should be defined within the context of the business requirements. Key problems and concepts provided in this paper that should be considered during this process include:

  • Those charged with the operations of a business have very little insight into the information they need. They don’t have a solid and up-to-date understanding of exactly what the business is doing. Thus, planning can’t occur effectively for those who lead the business, and the business suffers.
  • Existing approaches include the lack of efficiencies for manual and automated planning that have limited our ability to get the most out of our businesses. The use of automated data-driven planning tools, when put in the hands of those on the front lines of the business, will quickly deliver value by empowering them to have the data they need to provide effective and dynamic planning for the business.
  • A single modeling engine should work across all cloud and on-premise applications to provide an abstraction layer of information. This single version of the truth can be exploited by those with operational responsibilities who will use this information to determine the current state of the business as well as to understand how to create plans to optimize enterprise resources more effectively without involving strategic leadership.
  • Sophisticated analytics remove the user from the underlying complexity of the data. We empower those users to be independently successful. The democratization of data, or putting the right data in the hands of all those who need it, has a huge potential to provide positive change to the efficiency of business planning operations.
  • New approaches to leverage operational data seem to be the key. As businesses learn more about analytics, it’s been well-documented that the value is higher within operational data that may be analyzed and abstracted for use in automated planning operations. This is no longer in the realm of the business executive but with operationally focused managers who make tactical plans that drive value directly to the bottom line of the business.

5 About David S. Linthicum

David S. Linthicum is an internationally recognized industry expert and thought leader in the world of cloud computing and the author or co-author of 15 books on computing, including the best-selling Enterprise Application Integration and his latest book, Cloud Computing and SOA Convergence. He is a blogger for InfoWorld, Intelligent Enterprise,, and Forbes, and he conducts his own podcast, the Cloud Computing Podcast. His industry experience includes tenures as the CTO and CEO of several successful software companies and upper-level management positions in Fortune 100 companies. In addition, Linthicum was an associate professor of computer science for eight years and continues to lecture at major technical colleges and universities.

6 About Gigaom Research

Gigaom Research gives you insider access to expert industry insights on emerging markets. Focused on delivering highly relevant and timely research to the people who need it most, our analysis, reports, and original research come from the most respected voices in the industry. Whether you’re beginning to learn about a new market or are an industry insider, Gigaom Research addresses the need for relevant, illuminating insights into the industry’s most dynamic markets.

Visit us at:

7 Copyright

© Knowingly, Inc. 2014. "Instinct meets evidence: using operational data to drive planning" is a trademark of Knowingly, Inc. For permission to reproduce this report, please contact