C ER TI FI ED A N ALYTIC S

P RO FE SS I ON AL ( CAP )
®

E XAM IN AT IO N STUDY GUI DE

5521 Research Park Drive, Suite 200, Catonsville, MD 21228 USA

855-249-2589
C H A P T ER 3
DOMAIN II – ANALYTICS PROBLEM FRAMING

WHAT WILL YOU LEARN IN THIS CHAPTER?

This chapter is all about the dialogue between the business people who have a
problem that they need to solve and the analytics folks who will give them the
information required to solve the problem. This dialogue is mediated by the
analytics professional (YOU) who is trusted by both sides because you are fluent in
the language and culture of each side. As with any translation effort between two
different groups, much of what follows are simple precepts to keep the sense of
the business problem while decomposing it into actionable analytics pieces.

Learning Objectives

1. Reformulate a problem statement as an analytics problem

2. Develop a proposed set of drivers and relationships to inputs

3. State the set of assumptions related to the problem

4. Define key metrics of success

5. Obtain stakeholder agreement on the approach

Key Concepts/Fundamentals

OBJECTIVE 1. REFORMULATING THE BUSINESS PROBLEM STATEMENT AS AN

ANALYTICS PROBLEM

There’s an apocryphal story of a Black & Decker sales convention. The VP of sales
gets up to the dais, and says, “Folks, I have some bad news for you. We’ve done
some detailed customer surveys to find out what our customers care about. They
couldn’t care less about our carbide tips, or the voltage rating of our drills. In fact,
they’d rather not think about drills at all! What our customers want is to hang a
picture, or put up drywall, or do any number of other jobs. Our job is to help them
do just that.” Similarly, your business and operational stakeholders likely could not
care less about how you and your team are going to solve their problem. They just
want it to be solved reliably and deliver the results.

The first step is to decode the business problem statement to get to the analytics
problem. There are many ways to do this, some more formal than others. In simple

23
 terms, you are translating the “what” of the business problem into the “how” of
the analytics problem.

1. What result do we want?

2. Who will act?

3. What will they do?

4. What will change in the organization as a result of the new information

generated?

For example, a company wishes to increase market share, but what is the underlying
problem they need to address? Are they, for instance, emphasizing carbide-tipped
drills to someone who only wants to hang a picture?

One formal method of decomposition is quality function deployment (QFD) (http://

www.ieee.li/tmc/quality_function_deployment.pdf). This is a rigorous process that
maps the translation of requirements from one level to the next, e.g., from the
business level to the first analytics level, from the first analytics level to the second
level, etc.

Whether you are formally decomposing and parsing a complex business statement,
or you are less formally brainstorming with a project sponsor, it is critically important
to account for tacit as well as formal requirements. The best known model in this
area is Kano’s requirements model (Figure 2). It distinguishes between unexpected
customer delights, known customer requirements, and customer must-haves that
are not explicitly stated.

CUSTOMER SATISFACTION Often there are business or

operational requirements that
are taken for granted by those
exciting
requirements
stakeholders that if not surfaced will
normal
requirements result in customer dissatisfaction,
particularly items that come
DON'T DO under the heading of “that’s
FULFILL FULFILL the way we always do things.”
EXPECTATIONS EXPECTATIONS
Now, there are times when those
expected requirements or assumptions need
requirements
to be challenged, but they can’t be
challenged until they are brought
to light. When you ask your
CUSTOMER DISSATISFACTION
business stakeholders for a list of
what requirements they have, they
Figure 2: Kano’s Requirements Model (used under Creative Commons,
http://creativecommons.org/licenses/by-nc-nd/3.0/us/) will tend to focus on the “normal

24
requirements,” not the “expected requirements.” As the analytics professional
charged with translating business requirements into the problem statement, you
really need to probe to make sure that you have the entire appropriate context as
well, including the expected requirements.

OBJECTIVE 2. DEVELOP A PROPOSED SET OF DRIVERS & RELATIONSHIPS TO OUTPUTS

These next three items are related. Your input/output functions are strongly related
to your assumptions about what is important about this problem as well as the key
metrics by which you’ll measure the organizational response to the problem.

We’ll start by defining the input/output functions of the problem at hand. As with
any of these areas, you can be as formal or informal as you like, but sketches
and diagrams certainly help communicate with your stakeholders and help get
everyone on the same page.

Here’s a very simple example: An organization wants to predict the number of

detected software defects over the next six months. That’s the output. The inputs
would be elicited from stakeholder interviews, using questions like, “What future
activities will add to our rate from where we are today?”, “What will decrease our
rate from where we are today?”, “Will we add interfaces or components to the
testing?”, “Will we materially change the size of the test team?”, etc. Bear in mind
that you aren’t looking for causation at this stage, just ideas around which you’ll
form some hypotheses against which you’ll test your model later.

Once you have these inputs and a general sense of their predicted effects, you
have a choice of how to communicate them to the team at large. A simple table
(Figure 3) is one approach. A black box sketch (Figure 4) is another approach.
How you do it isn’t nearly as important as doing it in a way that the people you’re
working with will understand.

INCREASING FACTORS DECREASING FACTORS

NAME SCALE NAME SCALE

NEW INTERFACES LESS THAN 1 TEST TEAM SIZE LESS THAN 1

CUSTOMER SITE TIME SINCE LAST

1-10 LESS THAN 1
DEPLOYMENT NEW FUNCTIONALITY
… … … …

Figure 3: Input Table

25
 Test Team Size

Test Intensity

Test Level
Rate of software
Remaining Defects
defect detection
Interface Changes

Location Changes

Figure 4: Black Box Sketch

Even these simple examples help illustrate the concept. The idea here is to make
the inputs visible and start getting agreement among the team on the direction
and scale of the relationships to bound the problem and to create the related
hypotheses that you’ll use later to attack the data. A point you’ll want to emphasize
to the team is that these are preliminary assumptions and while your best estimate
is needed, it is still just an estimate and is subject to change depending on what
reality turns out to be. The danger we’re trying to avoid here is what Kahneman
calls “anchoring.” People have a tendency to hang on to views that they’ve seen
and held before, even if they are incorrect. Reminding them that these are initial
and preliminary, rather than finalized views, helps mitigate the anchoring effect.

OBJECTIVE 3. STATE THE SET OF ASSUMPTIONS RELATED TO THE PROBLEM

This is where you set the boundaries of the problem. As you look at your input
drivers, each likely has one or more assumptions embedded in it that needs to be
surfaced and listed. Additionally, some complexities can be trimmed away if their
presumed effect on the answer is less than the effort required to handle them.

As Stephen R. Covey (2004, p. 24) said, “We simply assume that the way we see
things is the way they really are or the way they should be. And our attitudes and
behaviors grow out of these assumptions.” Common practice assumptions in your
organization also need to be listed and questioned regularly to ensure that they
are either still valid or that the problem statement needs to change to incorporate
changes to them.

26
OBJECTIVE 4. DEFINE THE KEY METRICS OF SUCCESS

There is a truism quoted by many people that “what is measured, improves”

(cf. Drucker, Pearson’s Law, Hawthorne effect). This ties directly to the business
problem statement, but goes down one level further to the items that comprise the
key success metric. For example, if the business problem is that the organization
wants to increase return on sales from 10% to 12%, you might decompose that a
few different ways. One way is to give each business group that goal, or even to
give each group the objective of reaching 13%, figuring that some won’t make it
and on average we’ll be okay. Another way is to look at your value chain and give
each group a target: cost of goods reduction of five percentage points, general
and administrative reduction of three percentage points, etc. These metrics need
to be negotiated, published, committed to, and tracked so that your team knows
where you are and what to do next. As is the case throughout this chapter, you
have to make sure that all facets of the business problem are incorporated in the
metrics. After all, if you don’t say how something will be measured, you don’t know
how you’re doing, and you can’t succeed.

OBJECTIVE 5. OBTAIN STAKEHOLDER AGREEMENT

Although you’ve been in touch with your business stakeholders at some level all
along, this is when you come back to them to walk them through your assumptions
and approach and what the final answer will look like to be sure that you really are
answering the business problem. Whether in the form of a formal presentation,
you want your assumptions acknowledged along with the reframing you did from
the business problem, and the key metrics you will be using to mark progress
toward the solution.

Many people tend to think of stakeholders as people in positions “above” the

analytics team. It is true that there is a group of stakeholders that are the ones with
the business need and who are paying for the effort. But just as importantly, you
must also have an agreement with the people executing the analytics work that
your methods and hypotheses are workable in the time and budget allotted to get
the work done.

The output of this stakeholder agreement will vary by organization, but should
include the budget, timeline, interim milestones (if any), goals, and any known
effort that is excluded as out of scope. The key is to get all the pieces we’ve
noted in this chapter verbally discussed, documented, and visibly agreed to by
all parties. It can be tempting to settle for e-mails or written documents only and
desk-side reviews. For all but the simplest problems, this is a mistake. Translation
of problems from the business domain to the analytics domain, or truly from any

27
 given domain to another domain, requires that all parties agree to definitions
and terms, which really does require full and frank discussion. Otherwise, errors
will creep in and what was delivered will miss critical unstated requirements. If
you allow your project to rely on written communication only, you’ve missed the
opportunity to correct misapprehensions when it is still cheap to do so.

SUMMARY OF KEY TERMS

Decomposition: the act of breaking down a higher-level requirement to multiple

lower-level requirements (http://www.hq.nasa.gov/office/codeq/software/
ComplexElectronics/l_requirements2.htm).

Requirements: a requirement should be unitary (no conjunctions such as and,

but, or or), positive, and testable.

SUMMARY

Faithful translation of the business problem statement into an analytics problem

statement requires the following:

• Understanding the business case for solving this particular problem

Framing the business case as an actionable analytics problem by:

• Defining the key input and output drivers

• Surfacing and understanding individual and organizational assumptions

• Assigning goals to each sub-group affected by the problem

Full and frank review of the approach with the business stakeholders and the
analysts to ensure that the problem can be attacked as planned and that a
successful attack will yield the desired business result.

FURTHER READING

Albright SC, Winston W, Zappe C (2011) Data Analysis and Decision Making, 4th
ed. (South-Western Cengage Learning, Mason, OH).

Covey S (2004) The 7 Habits of Highly Effective People (Simon & Schuster, New
York).

28
Crow KA (1992) Quality Function Deployment, http://www.ieee.li/tmc/quality_
function_deployment.pdf.

National Aeronautics and Space Administration (2009), Assurance Process

for Complex Electronics, http://www.hq.nasa.gov/office/codeq/software/
ComplexElectronics/l_requirements2.htm.

Tversky A, Kahneman D (1974) Judgment under uncertainty: Heuristics and

biases. Science 185(4157):1124–1131.

29