11/03/2024

CHAPTER 03
Performing the Test Plan and
Analyzing Results

Objectives

• Understand four categories of Data Analytics.

• Describe some descriptive analytics approaches, including summary statistics and
data reduction.
• Explain the diagnostic approach to Data Analytics, including profiling and
clustering.
• Understand predictive analytics, including regression and classification.
• Describe the use of prescriptive analytics, including machine learning and artificial
intelligence.

1
 11/03/2024

Contents

• Four main categories of data analytics.

• Descriptive analytics
• Diagnostics analytics
• Predictive analytics
• Prescriptive analytics

Four main categories of data analytics.

• Descriptive analytics are procedures that summarize existing

data to determine what has happened in the past.
• Diagnostic analytics are procedures that explore the current
data to determine why something has happened the way it has,
typically comparing the data to a benchmark.
• Predictive analytics are procedures used to generate a model
that can be used to determine what is likely to happen in the
future.
• Prescriptive analytics are procedures that model data to enable
recommendations for what should be done in the future.

2
 11/03/2024

Four main categories of data analytics.

Each stage takes additional effort but provides additional value.

Exhibit 3-1 Four Main Categories of Data Analytics

5

Descriptive analytics
Descriptive analytics help summarize what has
happened in the past.
• A financial accountant would sum all the sales transactions within a
period to calculate the value for Sales Revenue that appears on the
income statement.
• An analyst would count the number of records in a data extract to
ensure the data are complete before running a more complex analysis.
• An auditor would filter data to limit the scope to transactions that
represent the highest risk. In all these cases, basic analysis provides
an understanding of what has happened in the past to help decision
makers achieve good results and correct poor results.

3
 11/03/2024

Descriptive analytics

Descriptive analytics examples:

• Summary statistics describe a set of data in terms of their location
(mean, median), range (standard deviation, minimum, maximum), shape
(quartile), and size (count).
• Data reduction or filtering is used to reduce the amount of
observations to focus on relevant items (that is, highest cost, highest
risk, largest impact, etc.). It does this by taking a large set of data
(perhaps the population) and reducing it to a smaller set that has the
vast majority of the critical information of the larger set.

Descriptive analytics

Summary statistics
Statistic Excel formula Description
Sum SUM() The total value of all numerical values
• Summary statistics The center value; sum of all observations divided by the
Mean =AVERAGE()
describe the number of observations
The middle value that divides the top half of the data from the
location, spread, Median =MEDIAN()
bottom half
shape, and Minimum =MIN() The smallest value
Maximum =MAX() The largest value
dependence of a set Count =COUNT() The number of observations
of observations. Frequency =FREQUENCY() The number of observations in each of a series of numerical or
categorical buckets
Standard The variability or spread of the data from the mean; a larger
=STDEV()
deviation standard deviation means a wider spread away from the mean
The value that divides a quarter of the data from the rest;
Quartile =QUARTILE()
indicates skewness of the data
Correlation How closely two datasets are correlated or predictive of one
=CORREL()
coefficient another

Exhibit 3-3 Description of Summary Statistics

8

4
 11/03/2024

Descriptive analytics

Summary statistics
Mean vs Median
Examples Mean Median

5, 3, 9, 7

9, 7, 8, 5, 4

8, 4, 4, 6, 2, 6, 6

7, 3, 5, 7, 1, 17, 13, 7
9

Descriptive analytics
Summary statistics
Mean vs Median: When to use?

It’s best to use the

Mean to describe the
center of a dataset when
the distribution is mostly
symmetrical and there
are no outliers.

10

5
 11/03/2024

Descriptive analytics
Summary statistics
Mean vs Median: When to use?

When a distribution is
skewed, the Median does a
better job of describing the
center of the distribution

11

Descriptive analytics
Summary statistics
Mean vs Median: When to use?

The Median also does a

better job of capturing the
central location of a
distribution when there are
outliers present in the data

12

6
 11/03/2024

Descriptive analytics

Summary statistics
Standard deviation

Examples Variance Std. deviation

10,15,20,25,30

5,10,15,20,20,25,30,30,35,40

13

Descriptive analytics

Summary statistics

Quartile

14

7
 11/03/2024

Descriptive analytics

Summary statistics
Quartile

Examples Q1 Q2 Q3 IQR Min Max

40, 42, 47, 53, 54, 59, 65

5,10,15,20,20,25,30,30,35,40

15

Descriptive analytics

Data reduction involves the following steps:

• Identify the attribute you would
like to reduce or focus on.
• Filter the results.
• Interpret the results.
• Follow up on results.

Exhibit 3-4 Use Filters to Reduce Data

16

8
 11/03/2024

Descriptive analytics

Data reduction
Fuzzy matching locates approximate matches
• Useful for
identifying
relationships in
imperfect data.

Exhibit 3-5 A Fuzzy Matching Shows a Likely Match of an

Employees and Vendor
17

Diagnostic analytics

Diagnostic analytics provide insight into why things happened or

how individual data values relate to the general population.

Two common methods of diagnostic analytics include Profiling and

Clustering.

More diagnostic analytics include Similarity matching and Co-

occurrence grouping

18

9
 11/03/2024

Diagnostic analytics

Diagnostic analytics methods:

• Profiling identifies the “typical” behavior of an individual, group,
or population by compiling summary statistics about the data
(including mean, standard deviations, etc.) and comparing
individuals to the population.
• Clustering helps identify groups (or clusters) of individuals (such
as customers) that share common underlying characteristics—in
other words, identifying groups of similar data elements and the
underlying drivers of those groups.

19

Diagnostic analytics

Diagnostic analytics methods:

• Similarity matching is a grouping technique used to identify
similar individuals based on data known about them.
• Co-occurrence grouping discovers associations between
individuals based on common events, such as transactions they
are involved in.

20

10
 11/03/2024

Diagnostic analytics

Profiling
• Profiling involves gaining an understanding of the typical behavior
of an individual, group, or population (or sample).
• Profiling can be used to develop complex models to predict
potential fraud.
• Profiling is done primarily using structured data—data that are
stored in a database or spreadsheet and are readily searchable.

21

Diagnostic analytics

Data profiling typically involves the following steps:

1. Identify the objects or activity you want to profile: What data do you want to
evaluate? Sales transactions? Customer data? Credit limits?
2. Determine the types of profiling you want to perform: What is your goal? Do
you want to set a benchmark for minimum activity? Have you set a budget
that you wish to follow?
3. Set boundaries or thresholds for the activity: This is a benchmark that may be
manually set or automatically set.

22

11
 11/03/2024

Diagnostic analytics

Data profiling typically involves the following steps:

4. Interpret the results and monitor the activity and/or generate a list of
exceptions: Here is where dashboards come into play.
5. Follow up on exceptions: A plan should be taken to validate, correct, or
identify the causes of the abnormal behavior.

23

Diagnostic analytics

Profiling
Z-Scores - Standardizing Data for Comparison

Where:
• z = Z-score
• x = the value being evaluated
• μ = the mean
• σ = the standard deviation
24

12
 11/03/2024

Diagnostic analytics

Profiling
Z-Scores shows spread and outliers.

The higher the Z-score

(farther away from the
mean), the more likely
a customer will have a
delayed shipment
(blue circle).

Exhibit 3-7 Z-Scores Provide an Example of Profiling That Helps Identify Outliers 25

Diagnostic analytics
Profiling
Box plots or whisker plot
• Displays the five-number summary of a set of data including the
minimum, first quartile, median, third quartile, and maximum
• The five-number summary divides the data into sections that each
contain approximately 25% of the data in that set

26

13
 11/03/2024

Diagnostic analytics
Profiling
Box plots show spread and outliers

EXHIBIT 3-8 Box Plots Provide an Example of Profiling That Helps Identify Outliers
(in This Case, Categories with Unusually High Average Days to Ship) 27

Diagnostic analytics
Data profiling in management accounting
Variance analysis
• Internal auditors analyze
travel and entertainment
expenses for violations of
internal controls.
• Managers use profiling to
compare variances from
target ranges.

Exhibit 3-9 Variance Analysis Is an Example of Data

28
Profiling

14
 11/03/2024

Diagnostic analytics
Data profiling in auditing
Benford’s Law

• In the continuous audit,

an auditor may use
Benford’s Law to evaluate
the frequency distribution
of the first digits from a
large set of numerical
data.

Exhibit 3-10 Benford’s Law Applied to Large Numerical

Data Sets (including Employee Transactions) 29

Diagnostic analytics
Benford’s Law is a diagnostic analytics that compares
actual to expected values.

30

15
 11/03/2024

Diagnostic analytics
Cluster analysis shows natural groupings of data.

• Clustering is used to identify

groups of similar data
elements and the underlying
drivers of those groups.
• Clustering algorithms
calculate the minimum
distance of all observations
and groups those elements.
Exhibit 3-11 Clustering Is Used to Find Three Natural
Groupings of Vendors Based on Purchase Activity
31

Diagnostic analytics

Clustering in auditing
• Internal auditors can use
clustering to identify
groups of transactions
that may indicate risk or
fraud in insurance or
other payments.

Exhibit 3-12 Cluster Analysis of Insurance Payments

32

16
 11/03/2024

Diagnostic analytics

Hypothesis Testing for Differences in Groups

• A two-sample t-test for equal means is used to determine if the
difference between the means of two different populations is
significant or not.
• Begin by setting the Null Hypothesis H0 (no relationship) and
the Alternative Hypothesis HA (expected relationship).

33

Diagnostic analytics

Hypothesis Testing for Differences in Groups

• Significance level (α)
• A measure of the strength of the evidence before rejecting the null
hypothesis and concluding that the effect is statistically significant.
• The probability of rejecting the null hypothesis when it is true.
• The p-value: a number calculated from a statistical test to help
decide whether to reject the null hypothesis.
• How?: Compare p-value to α.

34

17
 11/03/2024

Diagnostic analytics
Hypothesis Testing for Differences in Groups

EXHIBIT 3-13 T-Test Assessing for Significant Differences in Average Shipping Times across Categories 35

Predictive analytics

Predictive analytics examples:

• Regression estimates or predicts the numerical value of a
dependent variable based on the slope and intersect of a line and
the value of an independent variable.
• Classification predicts a class or category for a new observation
based on the manual identification of classes from previous
observations.
• Link prediction predicts a relationship between two data items,
such as members of a social media platform.

36

18
 11/03/2024

Predictive analytics

Regression helps predict expected outcomes.

• Regression is a statistical method that

attempts to determine the strength and
character of the relationship between one
dependent variable (usually denoted by Y)
and a series of other variables (known as
independent variables).
• Regressions allow the accountant to develop
models to predict expected outcomes.
Exhibit 3-14 Regression
37

Predictive analytics
Regression helps predict expected outcomes.

Regression analysis involves the following process:

1. Identify the variables that might predict an outcome.
2. Determine the functional form of the relationship (linear of
nonlinear?).
3. Identify the parameters of the model (β, P-value).
4. Evaluate the goodness of fit (R2)

38

19
 11/03/2024

Predictive analytics

What are some examples of regression?

• In managerial accounting, regression may predict employee
turnover:
• Sales volume = f(advertising spending, and economic indicators such as
GDP or inflation)
• In auditing, regression may be used to determine the
appropriateness of allowance accounts:
• Allowance for loan losses amount = f(current aged loans, loan type,
customer loan history, collections success)
39

Predictive analytics
Classification predicts which class an individual
belongs to
• Identify the classes you wish to predict.
• Manually classify an existing set of records.
• Select a set of classification models.
• Divide your data into training and testing sets.
• Generate your model.
• Interpret the results and select the “best” model.

40

20
 11/03/2024

Predictive analytics

Classification begins with decision boundaries.

• Training data are existing data
that have been manually
evaluated and assigned a class.
• Test data are existing data used
to evaluate the model.
• Decision trees are used to divide
data into smaller groups.
• Decision boundaries mark the
split between one class and
another.
Exhibit 3-16 Example of Decision Trees and Decision
Boundaries
41

Predictive analytics

What else do you need to know about classification?

• Pruning removes branches
from a decision tree to avoid
overfitting the model.

Exhibit 3-17 Illustration of Pruning a Decision Tree

42

21
 11/03/2024

Predictive analytics

What else do you need to know about classification?

• Linear classifiers are useful for
ranking items rather than simply
predicting class probability.
• These are useful for determining
the important values, such as
valuable customers, or which
transactions are most likely
fraudulent.

Exhibit 3-18 Illustration of Linear Classifiers

43

Predictive analytics

What else do you need to know about classification?

• Support vector machine is a
discriminating classifier that is
defined by a separating
hyperplane that works first to
Exhibit 3-19 Support Vector Machines
find the widest margin (or
biggest pipe) and then works to
find the middle line.

Exhibit 3-20 Support Vector Machine Decision Boundaries

44

22
 11/03/2024

Predictive analytics

How do we evaluate classifiers?

• Try to avoid overfitting, or
models that are too accurate.
They are bad at predicting a
future observation.

Exhibit 3-21 Illustration of Underfitting and

Overfitting the Data with a Predictive Model

45

Predictive analytics

How do we evaluate classifiers?

• Look for the sweet spot where
we maximize the accuracy of the
testing data.

Exhibit 3-22 Illustration of the Trade-Off between the

Complexity of the Model and the Accuracy of the
Classification
46

23
 11/03/2024

Prescriptive analytics

Once other diagnostic and predictive analyses have been

performed, the decision process can be aided by rules-based
decision support systems, machine learning models, or Machine learning and Artificial intelligence are two
added to an existing artificial intelligence model to forms of Prescriptive approach to Data Analytics work.
improve future predictions.

47

Prescriptive analytics

Prescriptive analytics examples:

• Decision support systems are rule-based systems that gather
data and recommend actions based on the input.
• Machine learning and artificial intelligence are learning
models or intelligent agents that adapt to new external data to
recommend a course of action.

48

24
 11/03/2024

Prescriptive analytics
DSS use rules to guide the accountant.

• The rules are derived from past

behavior to help guide the
accountant through a process.
• For example, the classification
of leases is based on evaluating
several rules.

Exhibit 3-23 Lease Classification Flowchart

49

Prescriptive analytics
Machine learning learns from past data to predict
better outcomes.
• What these all have in common is the use of algorithms and statistical models
to generate a previously unknown model that relies on patterns and
inferences.
• For most application of artificial intelligence models, most companies will
outsource the underlying system from companies like Microsoft, Amazon, or
Google rather than develop it themselves.
• These companies have large datasets to create more accurate prediction and
recommendation engines.
50

25
 11/03/2024

Summary

• In this chapter, we addressed the third and fourth steps of the IMPACT
cycle model: the “P” for “performing test plan” and “A” for “address and
refine results.” That is, how are we going to test or analyze the data to
address a problem we are facing?
• We identified descriptive analytics that help describe what happened with
the data, including summary statistics, data reduction, and filtering.
• We provided examples of diagnostic analytics that help users identify
relationships in the data that uncover why certain events happen through
profiling, clustering; similarity matching, and co-occurrence grouping.

51

Summary

• We introduced some specific models and terminology related to these tools,

including Benford’s law, test and training data, decision trees and boundaries,
linear classifiers, and support vector machines. We identified cases where creating
models that overfit existing data are not very accurate at predicting the future.

• We explained examples of predictive analytics and introduced some data mining

concepts related to regression, classification, and link prediction that can help
predict future events or values. We discussed prescriptive analytics, including
decision support systems and artificial intelligence and provided some examples
of how these systems can make recommendations for future actions.
52

26