What is data-driven

decision making? Step-by-

step guide
 NOVEMBER 13, 2020
 6 MINUTE READ

Written by Cem Dilmegani

https://research.aimultiple.com/data-driven-decision-making/

Most successful organizations treat data-driven decision making

as a primary objective and pursue it with religious zeal.
However, data-driven decision making, the steps leading to it
and how AI is changing it are not well-defined.

In an AI-first company, data-driven decision making means

 Strategic decisions are made by a diverse group

including executives that rely on a sufficiently
comprehensive, high quality set of information. By
definition, a bad strategic decision should be important
enough to lead to failure of the company.
 There will be operational decisions with significant
business implications where out-of-the-box solutions
do not produce satisfactory results. These are
generated by custom-built machine learning
models. These models can be built by running data science
 competitions, in-house teams or by working with industry
leaders.
 Most other operational decisions are handled
by continuously learning machine learning
models which produce explainable decisions.
Operational decisions are frequent (once a week or more
frequent) and not critical (a single mistake is unlikely to lead
to failure of the company).
 Operational decisions which can not be automated with good
accuracy are delegated to humans.
o If data is lacking, opinion based decisions are made.
o If data exists and has been analyzed, decision maker
relies on analysis.
o If data exists but is not analyzed yet, cost of analysis
determines whether an opinion or data based decision
will be made.

Before settling on this framework for decision making for

modern corporations, we need to identify how we can evaluate
different decision making models. However, if you like you can
directly skip to the sections that interest you:

Why is it important?
Even though it is obvious that data-driven decision making is
important, it would be a shame if we were not being data-
driven about proving it. Though quantitative data about data-
driven decision making is hard to come by, there is significant
evidence that

 volume of data generated is increasing. Every day we create

2.5 quintillion bytes of data according to Domo’s analysis in
2017. 90% of the data in the world today has been created
in the last two years.
 companies are investing more in data and machine learning
technology to get insights from that data.
A relevant survey, was conducted by New Vantage
Partners about Big Data investments of companies. Results
highlight that businesses are trying to get an edge over each
other by implementing Big Data across the company. Here are
the figures in the survey:

 Percentage of companies investing more than $500M in Big

Data/AI tech has increased from 13% (2018) to 21.1%
(2019).
 Percentage of companies investing between $50M and
$500M has increased from 27% (2018) to 34% (2019).
 Percentage of companies investing less than $50M has
decreased from 60% (2018) to 45% (2019).

How to assess different decision

making models?
Decision making has 3 KPIs: Speed, quality, auditability. Lack of
any of these can lead to criticial failures.

 Slow decision making cripples organizations and allows

competitors to gain market share.
 Bad decisions lead to failure. A bad decision, no matter
how well executed, is unlikely to generate value. For
example, Motorola’s decision to continue investing in Iridium
satellites which cost billions of dollars resulted in a big write-
off for the company. Great execution could not change the
outcome as mobile communication was providing a cheaper
and higher quality alternative to satellite phones by the time
Iridium phones were launched.
 If decisions can not be audited, they are hard to
improve. Decision quality is difficult to measure at the time
of the decision but can be estimated later by considering
other alternatives and current market conditions. Such
postmortem analysis should lead to better decisions
What are different decision
making models?
Decision making models starting from the lowest level of
sophistication are:

1- Opinion based

While it provides fast decisions, decision quality can be

low which is dangerous for strategic decisions. However, it can
be applied for fast decisions making in cases where:

 High quality data is not available

 Cost of analysis is more than the value of a good decision

2- Driven by limited amount of data

Organizations with data quality or availability issues are likely

to rely on limited amounts of data to make decisions. This is
potentially the most dangerous decision making model
as decision makers could use limited data to support
their opinions, leading to widely supported but opinion based
decisions.

A significant share of consulting projects fall under this

category where consultants could be incentivized to find limited
data points to support opinion based hypotheses.

Decision quality can be better than opinion based decision

making if decision makers are open to new information, take
into account psychological biases, vary their approach to
making prediction (inside view vs outside view) and look for
answers rather than searching for data to support their opinion.
3- Data-driven decision making via manual analysis

For this model, data needs to be available, high quality and

there needs to be consensus about the correctness of data. It
is the preferable approach for important decisions
where timing is not critical and where cost of
automation is prohibitive compared to its benefits.

To enable this, companies need to:

 manage their data efficiently and effectively

 ensure the quality of their data
 rely on powerful analytics/business intelligence tools

4- Automated data-driven decision making

Data is available and high quality and there is consensus about

the correctness of data and it has been processed so decisions
are made automatically. Based on outcomes, the decision
making model continuously improves itself over time. However,
audit is challenging when reasons for decisions are not
provided.

To achieve this, companies need to apply the latest machine

learning and AI approaches to build automated, operational
models based on their data.

5- Automated data-driven explainable decision making

In addition to automated data-driven decision

making, explainable machine learning models are used to make
decisions. Explanations help audit decisions and continuously
improve the underlying model via manual interventions on top
of automated continuous learning.
Why I wrote this?
I spent most of my professional career at McKinsey where data-
driven decision making is one of the core principles. It was a
great principle but quite challenging to achieve as a consultant
without access to internal data. Given my current experience
with AI, I wanted to think how we can better define data driven
decision making.

How can your company become a

data-driven enterprise in the age
of AI?
1. Companies need to achieve digital transformation to ensure
that they have digital processes. For example, we have a
detailed guide on using RPA to digitize and automate
repetitive processes. Digital transformation enables the
company to generate data which will be used to build
models and operationalize decision making. Data needs to
be high quality and accessible to enable further
transformation.
2. AI transformation can start as processes are digitized. High
value processes where mature out-0f-the-box solutions exist
are the first areas for AI transformation.
3. As companies deploy more machine learning/AI based
solutions, there will be areas where out-of the box solutions
are inadequate. Companies need to start deploying custom
AI solutions to automate decision making in such areas.
4. As models are more frequently used in decision making,
companies need to ensure that they are explainable and free
from bias.
5. Along this journey, management needs to embrace data
driven decision making as well. Conscious trade-offs need to
 be made about the effort for data analytics vs the quality of
the decision. This cultural change is not a separate step but
should be developed as the company’s analytical capabilities
mature. A company with mature digital processes can afford
to make more of its decisions in a data driven way vs a
company with offline processes since it takes significant
effort to analyze results of such processes in detail.

After these steps, the company will have achieved the level of
maturity described at the beginning of the article. The exact
steps to get there will depend on the current maturity of the
company. Companies can work with various partners to get
support in this journey:

 digital transformation consulting companies to identify

the dx roadmap and implement key initiatives
 analytics consultants to set up the analytics framework of
the enterprise which allows critical decisions to be
completed with a mix of automated and manual analysis
 data science / AI consultants to operationalize company’s
data by building custom models that autonomously make
business decisions such as processing loans or claims
 Data science competition organizers to help companies
leverage wisdom of crowds to build effective models at
limited cost