0% found this document useful (0 votes)

12 views

Visual Guide To Machine Learning

The document provides an overview of machine learning concepts including supervised and unsupervised learning techniques as well as advanced topics. It also covers the machine learning process, with a focus on the importance of data quality assurance (QA) as the first step. Key aspects of QA include understanding variable types, handling empty values, calculating ranges and counts to identify outliers, and ensuring proper data structure and formatting.

Uploaded by

Michelle Saver

Available Formats

Download as PPTX, PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

12 views

Visual Guide To Machine Learning

Uploaded by

Michelle Saver

Available Formats

Download as PPTX, PDF, TXT or read online on Scribd

You are on page 1/ 349

*Copyright Maven Analytics, LLC

MACHINE LEARNING
LANDSCAPE
MACHINE LEARNING

Supervised Learning Unsupervised Learning Advanced

Topics

Clustering/Segmentation
Classification Regression Reinforcement Learning
K-Means (Q-learning, deep RL, multi-armed-bandit, etc.)

K-Nearest Neighbors Least Squares Outlier Detection

Natural Language Processing
Naïve Bayes Linear Regression Markov Chains (Latent Semantic Analysis, Latent Dirichlet Analysis,
relationship extraction, semantic parsing,
Logistic Regression Forecasting contextual word embeddings, translation, etc.)
Matrix factorization, principal components,
Non-Linear Regression factor analysis, UMAP, T-SNE, topological data
Sentiment Analysis
analysis, advanced clustering, etc.
Computer Vision
Monte Carlo (Convolutional neural networks, style translation, etc.)
Random forest, support
vector machines, LASSO/RIDGE, state-
gradient boosting, space, advanced Deep Learning
neural nets/deep generalized linear (Feed Forward, Convolutional, RNN/LSTM,
learning, etc. methods, VAR, DFA, Attention, Deep RL, Autoencoder, GAN. etc.)
etc.
MACHINE LEARNING
PROCESS
PREPARING YOUR UNDERSTANDING YOUR MODELING YOUR
DATA DATA DATA

QUALITY ASSURANCE

UNIVARIATE PROFILING

MULTIVARIATE PROFILING

MACHINE LEARNING

Quality Assurance (QA) is about preparing & cleaning data prior to analysis. We’ll cover common QA
topics
including variable types, empty/missing values, range & count calculations, censored data, etc.
PRELIMINARY DATA
QA
Data QA (otherwise known as Quality Assurance or Quality Control) is the first
step in the analytics and machine learning process; QA allows you to identify and
correct underlying data issues (blanks, errors, incorrect formats, etc.) prior to analysis

TOPICS WE’LL COMMON USE CASES:

COVER: • Minimizing the risk of drawing false conclusions
Variable Types Empty Values or
misunderstanding your data
• Confirming that dates are properly formatted
for analysis (as date values, not text strings)
Range Count Calculations
Calculations
• Replacing blanks or errors with appropriate
values to prevent summarization errors
Table Structures Left/Right Censors
• Calculating ranges (max & min) to spot-check
for outliers or unexpected values
PRELIMINARY DATA
QA

POP QUIZ: When should you QA your

data?

EVERY. SINGLE.
TIME. (no exceptions!)
WHY IS QA
IMPORTANT?
As an analyst, Preliminary Data QA will help you answer questions
like:

Are there any missing Was the data our client Is there any risk that Are there any outliers
or empty values in the captured from the the data capture that might skew the
data shared by the HR online survey encoded process was biased in results of our
team? properly? some way? analysis?
VARIABLE
TYPES
Variable types give us information about our variables
Variable Types
• January 1, 2000 as a number simply displays a date
• January 1, 2000 as a date implies a set of information we can use (first day of
Empty the year, winter, Saturday, weekend, etc.) all of which can be used to build
Values strong machine learning models

Range Calculations
Common variable types
include:
Count Calculations Numeric Discrete Date
• Customer count • Count • 1/1/2020

Ordinal Categorical Complex

Left/Right Censored • Small, Medium, • Gender • 4 + 2i
Large
Interval Nominal Logical
• Temperature • Nationality • True/False
Table Structure
Ratio Binary Monetary
• Weight • Yes/No • $4.50
VARIABLE
TYPES
Understanding variable types is fundamental to the machine
Variable Types
learning process, and can help avoid common QA issues,
including:
Empty Values • Numeric variables formatted as characters or strings, preventing
proper aggregation or analysis
• String/character variables formatted as numeric, preventing proper
Range Calculations
text-
based operations

Count Calculations • Values that Less obvious cases like variables that are re-coded from
raw values to buckets (surveys)
Here we’re formatting zip codes (which will never
be analyzed as values), as a numeric rather than
string
Left/Right

When we talk about variables, we might also refer to

them as metrics, KPIs, dimensions, or columns. Similarly,
Censored Table we may use rows, observations, and records
interchangeably

Structure
EMPTY VALUES

Variable Types Investigating empty values, and how they are recorded in your
data, is a prerequisite for every single analysis
Empty
Values
Empty values can be recorded in many ways (NA, N/A, #N/A,
NaN, Null, “-”, “Invalid”, blank, etc.), but the most common
Range Calculations mistake is turning empty numerical values into zeros (0)

Count Calculations

Left/Right

Average Age = Average Age =

Censored Table 28.6 35.8

Structure
EMPTY VALUES

Empty values can be handled in 3 ways: keep, remove, or

Variable Types
impute
• Keep empty or zero values if you are certain that they are accurate and
meaningful (i.e. no sales of a product on a specific date)
Empty
• Remove empty values if you have a large volume of data and can confirm
Values
that there is no pattern or bias to the missing values (i.e. all sales from a
specific product category are missing)
Range Calculations
• Impute (substitute) empty values if you can accurately populate the data or
if you are working with limited data and can use statistical methods (mean,
conditional mean, linear interpolation, etc.) without introducing bias
Count Calculations

For a missing Units Sold value,

you would likely remove the
Left/Right row unless you are certain
that an empty value
represents 0 sales

Censored Table
For a missing Retail Price, you
would likely be able to impute the
value since you know the product
Structure name/ID
RANGE CALCULATIONS

Variable Types One of the simplest QA tools for numerical variables is to

calculate the range of values in a column (minimum and
maximum values)
Empty Values

Range calculation is a helpful tool to understand your

Range
variables, confirm that ranges are realistic, and identify
Calculations potential outliers
Is there a clear lower or upper
min(Age) = 18 limit
Count Calculations max(Age) = 65 (i.e. 18+ or capped at 65)?

min(Income) = 0 Is your variable transformed to a

Left/Right max(Income) = 100 standard max/min scale (i.e. 1-10, 0-
100)?

min(height) = -10
max(height = 10 Is your variable normalized
Censored Table around a central value (i.e. 0)?

Structure
COUNT CALCULATIONS

Count calculations help you understand the number of records or

Variable Types
observations that fall within specific categories, and can be used
to:
Empty Values • Identify categories you were not expecting to see
• Begin understanding how your data is distributed
• Gather knowledge for building accurate ML models (more on this later)
Range

Distinct counts can

• Understand be particularly
the granularity useful
or “grain” for
of your QA, and help to:
data
Calculations
Count Calculations
• Identify how many unique values a field contains
• Ensure consistency by identifying misspellings or categorization errors
which might otherwise be difficult to catch (i.e. leading or trailing
Left/Right
spaces)
Censored

PRO TIP: For numerical variables with many unique values (i.e. long decimals),
use a
Table Structure histogram to plot frequency based on custom ranges or “bins” (more on that soon!)
LEFT/RIGHT
CENSORED
Variable Types
When data is left or right censored, it means that due to
some circumstance the min or max value observed is not the
natural minimum or maximum of that metric
Empty Values • This can be difficult to spot unless you are aware of how the data is
being
recorded (which means it’s a particularly dangerous issue to watch out
Range for!) Left Right
Censored Censored

Calculations

Count
Left/Right Censored

Calculations
Table Structure
M all Shopper Survey Results Ecommerce Repeat Purchase Rate
Only tracks shoppers over the age of 18 due to legal Sharp drop as you approach the current date has nothing to
reasons, so anyone under 18 is excluded (even do with customer behavior, but the fact that recent
though there are plenty of mall shoppers under customers haven’t have the opportunity or need to
18) repurchase yet
TABLE STRUCTURE

Variable Types Table structures generally come in two flavors: long or

wide
Long Table Wide Table
Empty Values

Range Calculations

PIVOT

Count Calculations
UNPIVOT

Left/Right

Pivoting is the process of adjusting a table

from long to wide by transforming rows into
Censored
Table Structure columns, and Unpivoting is the opposite
(wide to long)
TABLE STRUCTURE

Long tables typically contain a single, distinct column for each field
Variable Types
(Date, Product, Category, Quantity, Profit, etc.)
• Easy to see all available fields and variable types
Empty Values • Great for exploratory data analysis and aggregation (i.e. PivotTables)

Range Calculations Wide tables typically split the same metric into multiple columns or
categories (i.e. 2018 Sales, 2019 Sales, 2020 Sales, etc.)
• Typically not ideal for human readability, since wide tables may contain
Count Calculations
thousands of columns (vs. only a handful if pivoted to a long format)
• Often (but not always) the best format for machine learning model input
Left/Right • Great format for visualizing categorical data (i.e. sales by product category)

Censored
Table Structure There’s no right or wrong table structure; each type has strengths &
weaknesses!
CASE STUDY: PRELIMINARY
QA
THE You’ve just been hired as a Data Analyst for Maven Market, a local
SITUATION grocery store looking for help with basic data management and analysis.

The store manager would like you to conduct some analyses on

THE product inventory and sales, but the data is a mess.
ASSIGNMENT You’ll need to explore the data, conduct a preliminary QA, and help
clean it up
to prepare the data for further analysis.

1. Look at common data profiling metrics to identify potential

THE
issues
OBJECTIVES 2. Take note of any issues you find in the product inventory sample
3. Correct the issues to prepare the data for further analysis
BEST PRACTICES: PRELIMINARY
QA
Review all fields to ensure that variable types are configured for
proper analysis (i.e. no dates formatted as strings, text formatted as
values, etc.)

Remember that NA and 0 do not mean the same thing! Think carefully
about how to handle missing data and the impact it may have on your
analysis

Run basic diagnostics like Range, Count, and Left/Right Censored

checks against all columns in your data set...every time

Understand your table structure before conducting any analysis to reduce

the risk of double counting, inaccurate calculations, omitted data, etc.
*Copyright Maven Analytics, LLC
MACHINE LEARNING
PROCESS
PREPARING YOUR UNDERSTANDING YOUR MODELING YOUR
DATA DATA DATA

QUALITY ASSURANCE

UNIVARIATE PROFILING

MULTIVARIATE PROFILING

MACHINE LEARNING

Univariate profiling is about exploring individual variables to build an understanding of your data. We’ll
cover
common topics like normal distributions, frequency tables, histograms, etc.
UNIVARIATE PROFILING

Univariate profiling is the next step after preliminary QA; think of univariate profiling
as conducting a descriptive analysis of each variable by itself

TOPICS WE’LL COMMON USE CASES:

COVER: • Developing a deeper understanding of the
Categorical fields
Categorical Variables
Distributions you’re working with (outliers, distribution, etc.)
• Preparing to build ML models (necessary
Histograms & for selecting the correct model type)
Numerical Variables
Kernel
Densities • Exploring individual variables before conducting
a deeper multivariate analysis
Normal Distribution Data
Profiling
CATEGORICAL VARIABLES

VARIABLE
Categorical Variables TYPES

Categorical CATEGORICAL NUMERICAL

Distributions

Categorical variables contain categories as values, instead of numbers

Numerical (i.e. Product Type, Customer Name, Country, Month, etc.)
Variables
• Categorical fields are exceptionally important for data analysis, and are typically
used as dimensions by which we filter or “cut” numerical values (i.e. sales by
Histograms & store)
Kernel Densities • Many types of predictive models are used to predict categorical dependent
variables, or “classes” (more on that later!)
Normal
Distribution Terms like discrete, categorical, multinomial, and classes may all be used interchangeably.
Binary is a special type of categorical variable which takes only 1 of 2 cases: true for false
(or 1 or 0) and is also known as a logical variable or a binary flag

Data Profiling
DISCRETIZATION

Discretization is the process of creating a new categorical variable from

Categorical Variables an existing numerical variable, based on the values of the numerical
variable
Categorical
Distributions

Numerical
Variables Discretization
Rules:
If Price <100 then Price Level =
Low
If Price >=100 & Price <500 then Price Level =
Histograms & Med
If Price >=500 then Price Level =
Kernel Densities High

Normal
Distribution

Data Profiling
NOMINAL VS. ORDINAL
VARIABLES
VARIABLE
Categorical Variables TYPES

Categorical CATEGORICAL NUMERICAL

Distributions

Numerical NOMINAL ORDINAL

Variables

Histograms &
Kernel Densities
There are two types of categorical variables: nominal and ordinal
• Nominal variables contain categories with no inherent logical rank, which can
Normal be re-ordered with no consequence (i.e. Product Type = Camping, Biking or
Distribution Hiking)

• Ordinal variables contain categories with a logical order (i.e. Size = Small,
Medium, Large), but the interval between those categories has no logical
Data Profiling interpretation
CATEGORICAL
DISTRIBUTIONS
Categorical Variables Categorical distributions are visual and/or numeric representations
of the unique values a variable contains, and how often each occurs
Categorical
Distributions
Common categorical distributions include:
Numerical • Frequency tables: Show the count (or frequency) of each distinct value
Variables • Proportions tables: Show the count of each value as a % of the total
• Heat maps: Formatted to visualize patterns (typically used for multiple variables)
Histograms &
Kernel Densities
Understanding categorical distributions will help us gather knowledge
for building accurate machine learning models (more on this later!)
Normal
Distribution

Data Profiling
CATEGORICAL
DISTRIBUTIONS
Categorical Variables
Section
Distribution:
Categorical Camping Frequency
14 6 Biking
Distributions table

Campin Biking
g Proportions
Numerical 70% 30% table
Variables

Histograms & Size & Section

Kernel Densities Distribution:
S Camping
6 4 Biking
Heat
6 4
Normal L 8 2 Map
Distribution

Data Profiling
NUMERICAL
VARIABLES
VARIABLE
Categorical Variables TYPES

Categorical CATEGORICAL NUMERICAL

Distributions

Numerical variables contain numbers as values, instead of categories

Numerical Variables
(i.e. Gross Revenue, Pageviews, Quantity, Retail Price, etc.)
• Numerical fields are typically aggregated (as a sum, count, average, max, min,
Histograms & etc.)
Kernel Densities and broken down by different dimensions or categories
• Numerical data is also known as quantitative data, while categorical data is
Normal often referred to as “qualitative”
Distribution
You may hear numeric variables described further as interval and ratio, but the distinction is
trivial and rarely makes a difference in common use cases

Data Profiling
HISTOGRAMS

Categorical Variables
Histograms are used to plot a single, discretized numerical
variable
Imagine taking a numerical variable (like age), defining ranges or
Categorical “bins” (1-5, 6-10, etc.), and counting the number of observations
Distributions
which fall into each bin; this is exactly what histograms are
designed to do!
Numerical Variables
Age
Values:
8 29 45 8

Histograms & 25 33 37
7

Frequenc
Kernel 6

Densities 19 43 21 5

28 32 40 4

y
Normal Distribution 24 17 28 3

2
5 22 39
1
15 47 12
Data 0- 11- 21- 31- 41-
Profiling 10 20 30 40 50
Age
Range
KERNEL
DENSITIES
Categorical Variables
Kernel densities are “smooth” versions of histograms, which can
help
to prevent users from over-interpreting breaks between bins
Categorical • Technical definition: “Non-parametric density estimation via
Distributions smoothing”
• Intuitive definition: “Wet noodle laying on a histogram”
Numerical Variables
Age
Values:
8 29 45 8

Histograms & 25 33 37
7

Frequenc
Kernel 6

Densities 19 43 21 5

28 32 40 4

y
Normal Distribution 24 17 28 3

2
5 22 39
1
15 47 12
Data 0- 11- 21- 31- 41-
Profiling 10 20 30 40 50
Age
Range
HISTOGRAMS & KERNEL
DENSITIES
Categorical Variables When to use histograms and kernel density charts:
• Visualizing how a given variable is distributed
Categorical • Providing a visual glimpse of profiling metrics like mean, mode, and
Distributions skewness
Things to watch out
Numerical Variables
for: • Bin sensitivity: Bin size can significantly change the shape and ”smoothness” of
a
histogram, so select a bin width that accurately shows the data distribution
Histograms & • Outliers: Histograms can be used to identify outliers in your data set, but
Kernel you may need to remove them to avoid skewing the distribution
Densities
• Sample size: Histograms are best suited for variables with many observations,
Normal Distribution to reflect the true population distribution

PRO TIP: If your data is relatively symmetrical (not skewed), you can use Sturge’s Rule
Data as a quick “rule of thumb” to determine an appropriate number of bins: K = 1 + 3.322
log (N) (where K = number of bins, N = number of observations)
Profiling
CASE STUDY:
HISTOGRAMS
THE You’ve just been promoted as the new Pit Boss at The Lucky Roll
SITUATION Casino. Your mission? Use data to help expose cheats on the casino
floor.

Profits at the craps tables have been unusually low, and you’ve been asked
THE to
investigate the possible use of loaded die (weighted towards specific numbers).
ASSIGNMENT
Your plan is to track the outcome of each roll, then compare your results
against the expected probability distribution to see how closely they
match.

1. Record the outcomes for a series of individual dice rolls

THE
2. Plot the frequency of each result (1-12) using a histogram
OBJECTIVES
3. Compare your plot against the expected frequency
distribution
NORMAL
DISTRIBUTION
Many numerical variables naturally follow a normal distribution, also
Categorical Variables known as a Gaussian distribution or “bell curve”
• Normal distributions are symmetrical and dense around the center, with flared
Categorical out
Distributions “tails” on both ends (like a bell!)
• Normal distributions are essential to many underlying assumptions in ML, and
a helpful tool for comparing distributions or testing differences between them
Numerical
Variables Symmetrical

Histograms &
Kernel Densities

Normal Distribution

Most dense around

center
Data
Profiling You can find normal distributions in many real-world examples: heights, weights, test scores,
etc.
NORMAL
DISTRIBUTION
Categorical Variables Parabola
centered around
the mean
Categorical
Distributions
1 𝑥−𝜇 2
1 −
𝑒 2 𝜎
Numerical Variables

Histograms &
Kernel
Densities
𝜎 2𝜋 Turn the
parabola
Normal Distribution upside down
Make the
tails flare
Data out
Profiling
CASE STUDY: NORMAL
DISTRIBUTION
THE It’s August 2016, and you’ve been invited to Rio de Janeiro as a Data
SITUATION Analyst
for the Global Olympic Committee.

THE Your job is to collect demographic data for all female athletes competing in
the Summer Games and determine how the distribution of Olympic
ASSIGNMENT athlete heights compares against the general public.

1. Gather heights for all female athletes competing in the 2016 Games
THE 2. Plot height frequencies using a Histogram, and test various bin widths
OBJECTIVES 3. Determine if athlete heights follow a normal distribution, or “bell
curve”
4. Compare the distributions for athletes vs. the general public
DATA PROFILING

Data profiling describes the process of using statistics to describe

Categorical Variables
or summarize information about a particular variable
• Data profiling is critical for understanding variable characteristics which
Categorical
cannot be
Distributions
seen or easily visualized using tools like histograms
• Data profiling communicates meaning, using simple, concise, and
Numerical universally understood metrics
Variables
Common data profiling metrics include:
• Mode
Histograms & • Mean
Kernel Densities
• Median PRO TIP: Combining visual distributions
with data profiling metrics is a powerful way
• Percentile to communicate meaning in advanced
Normal analytics
• Variance
Distribution
• Standard
Data
Deviation
Profiling
• Skewness
MODE

The mode is the most frequently observed value

Categorical Variables
• With a numerical variable (most common), it’s the general range around the
highest ”peak” in the distribution
Categorical
• With a categorical variable, it’s simply the value that appears most often
Distributions

Numerical
Variables Mode of City =
“Houston”

Mode of Sessions = 24
Histograms &
Mode of Gender = F, M
Kernel Densities (this is a bimodal field!)

Normal Common
Distribution
uses:
• Understanding the most common values within a
Data
Profiling dataset
• Diagnosing if one variable is influenced by another
MODE

Categorical Variables While modes typically aren’t very useful on their own, they can
provide
helpful hints for deeper data exploration
Categorical
Distributions • For example, the right histogram below shows a multi-modal distribution,
which indicates that there may be another variable impacting the age
distribution
Numerical Single Mode (21- Two Modes (0-10, 41-
Variables 30) 50)
8 8
7 7
Histograms & 6 6
Frequency

Frequency
Kernel Densities 5 5
4 4
3 3

Normal 2 2

Distribution 1 1

0- 11- 21- 31- 41- 0- 11- 21- 31- 41-

Data 10 20 30 40 50 10 20 30 40 50
Profiling Age Age
Range Range
MEAN

Categorical Variables
The mean is the calculated “central” value in a discrete set on
numbers
• Mean is what most people think of when they hear the word “average”, and is
Categorical
calculated by dividing the sum of all values by the count of all observations
Distributions
• Means can only be applied to numerical variables (not categorical)

Numerical
Variables 𝑠𝑢𝑚 𝑜𝑓 𝑎𝑙𝑙 𝑣𝑎𝑙𝑢𝑒𝑠
𝑚𝑒𝑎𝑛 =
𝑐𝑜𝑢𝑛𝑡 𝑜𝑓 𝑜𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠
Histograms & 5,220
Kernel Densities =
5 = 𝟏,
Normal Common 𝟎𝟒𝟒
Distribution
uses:
• Making a “best-guess” estimate of a value
Data
Profiling • Calculating a central value when outliers are not
present
MEDIAN

The median is the middle value in a list of values sorted from

Categorical Variables
highest to lowest (or vice versa)
• When there are two middle-ranked values, the median is the average of the
Categorical two
Distributions
• Medians can only be applied to numerical variables (not categorical)

Numerical
Variables

Median =
Histograms &
Kernel Densities 19.5
(average of 15 and 24)

Normal
Distribution Common uses:
Data • Identifying the “center” of a distribution
Profiling • Calculating a central value when outliers may be
present
PERCENTILE

Percentiles are used to describe the percent of values in a column

Categorical Variables which fall below a particular number
• If you are in the 90th percentile for height, you’re taller than 90% of the
Categorical population
Distributions • If the 50th percentile for test scores is 86, half of the students scored lower
(which means 86 is also the median!)

Numerical
Variables

Histograms &
Kernel Densities
Bob is the 3rd tallest in a group of
75 12. Since he’s taller than 9 others,
% Bob is in the 75th percentile for
Normal
height!
Distribution Common uses:
Data • Providing context and intuitive benchmarks for how values “rank” within a sample
Profiling (test scores, height/weight, blood pressure, etc.)
VARIANCE

Variance, in simple terms, describes how thin or wide a distribution is

Categorical Variables
• Variance measures how far the observations are from the mean, on average,
and
Categorical help us concisely describe a variable’s distribution
Distributions • The wider a distribution, the higher the variance (and vice versa)

Numerical
Variables Variance = 5

Variance = 15
Histograms &
Kernel Densities Variance = 30

Normal
Distribution
Common uses:
Data
Profiling • Comparing the numerical distributions of two different groups (i.e. prices of
products ordered online vs. in store)
VARIANCE

Categorical Variables

σ 𝑛𝑖=1(𝑥𝑖 − 𝜇)2
Categorical
Distributions 𝑛−1
Numerical Variables Calculation
Steps:
1) Calculate the average of the
Histograms &
variable
Kernel
2) Subtract that average from the first
Densities Average squared
row
3) Square that distance from the
Normal Distribution
difference mean
4) Do steps 1-3 for every row and sum it
up
Data
5) Divide by the number of observations (-
Profiling
1)
STANDARD DEVIATION

Standard Deviation is the square root of the variance

Categorical Variables
• This converts the variance back to the scale of the variable itself
• In a normal distribution, ~68% of values fall within 1 standard deviation of the
Categorical mean,
Distributions ~95% fall within 2, and ~99.7% fall within 3 (known as the “empirical rule”)
~68 ~95 ~99.7
Numerical % % %
Variables

Histograms &
Kernel Densities
1 s.d. 2 s.d. 3 s.d.
Normal
Distribution Common uses:
Data • Comparing segments for a given metric (i.e. time on site for mobile users vs.
Profiling desktop)
• Understanding how likely certain values are bound to occur
SKEWNESS

Categorical Variables Skewness tells us how a distribution varies from a normal

distribution
Categorical • This is commonly used to mathematically describe skew to the left or right
Distributions
Left Normal Right
skew Distribution skew
Numerical
Variables

Histograms &
Kernel Densities

Normal
Distribution Common uses:
Data
• Identifying non-normal distributions, and describing them
Profiling
mathematically
BEST PRACTICES: UNIVARIATE
PROFILING
Make sure you are using the appropriate tools for profiling categorical
variables vs. numerical variables

Distributions are a great way to quickly and visually explore variables

QA still comes first! Profiling metrics are important, but can lead to
misleading results without proper QA (i.e. handling outliers or missing
values)

One single distribution, visualization, or metric isn’t enough to fully

understand a variable; always explore your data from multiple angles
*Copyright Maven Analytics, LLC
MACHINE LEARNING
PROCESS
PREPARING YOUR UNDERSTANDING YOUR MODELING YOUR
DATA DATA DATA

QUALITY ASSURANCE

UNIVARIATE PROFILING

MULTIVARIATE PROFILING

MACHINE LEARNING

Multivariate profiling is about understanding relationships between multiple variables. We’ll cover
common
tools for exploring categorical & numerical data, including kernel densities, violin & box plots, scatterplots,
etc.
MULTIVARIATE PROFILING

Multivariate profiling is the next step after univariate profiling, since single-
metric distributions are rarely enough to draw meaningful insights or conclusions

TOPICS WE’LL COMMON USE

COVER: CASES:
• Exploring relationships between multiple
Categorical-Categorical Categorical-Numerical
categorical and/or numerical variables
Distributions Distributions
• Informing which specific machine learning models
Multivariate Kernel or techniques to use
Violin & Box
Densities Plots
Multivariate distributions are generally called “joint”
Numerical-Numerical Scatter Plots distributions, and it’s clear why—it’s joining two
Distributions & variables into the same distribution!
Correlation
CATEGORICAL-CATEGORICAL
DISTRIBUTIONS
Categorical-Categorical Categorical/categorical distributions represent the frequency
Distributions of
unique combinations between two or more categorical variables
Categorical-Numerical
Distributions This is one of the simplest forms of multivariate profiling, and
leverages the same tools we used to analyze univariate distributions:
Multivariate
Kernel • Frequency tables: Show the count (or frequency) of each distinct combination
Densities • Proportions tables: Show the count of each combination as a % of the total
Violin & Box
• Heat maps: Frequency or proportions table formatted to visualize patterns
Plots

Numerical-Numerical Common uses:

Distributions
• Understanding product mix based on multiple characteristics (i.e. size &
category)
Scatter Plots
& • Exploring customer demographics based on attributes like location, gender, etc.
Correlation
CATEGORICAL-CATEGORICAL
DISTRIBUTIONS
Categorical-Categorical In this example we’re looking
Distributions at product inventory using a
distribution of two Frequenc
y Table
Categorical-Numerical categorical fields: Design and
Distributions Size

Multivariate We can show this joint

Kernel distribution as a simple
Densities
count (Frequency Table), as Proportion
s Table
Violin & Box a percentage of the total
Plots (Proportions Table), or as a
conditionally formatted table
Numerical-Numerical
Distributions
(Heat Map)

Scatter Plots Heat

Map
&
Correlation
CASE STUDY: HEAT
MAPS
THE You’ve just been hired by the New York Department of
SITUATION Transportation
(DOT) to help analyze traffic accidents in New York City from 2019-
2020

The DOT commissioner would like to understand accident frequency by time

of day and day of week, in order to support a public service campaign
THE
promoting safe driving habits.
ASSIGNMENT Your role is to provide the data that she needs to understand when traffic
accidents are most likely to occur.

THE 1. Create a table to plot accident frequency by time of day and day of week
OBJECTIVES 2. Apply conditional formatting to the table to create a heatmap showing the
days and times with the fewest (green) and most (red) accidents in the
sample
CATEGORICAL-NUM ERICAL
DISTRIBUTIONS
Categorical-numerical distributions are used for comparing
Categorical-Categorical
Distributions
numerical distributions across classes in a category (i.e. age
distribution by gender)
Categorical-Numerical These are typically visualized using variations of familiar
Distributions
univariate numerical distributions, including:
Multivariate • Histograms & Kernel Densities: Show the count (or frequency) of values
Kernel
Densities
• Violin Plots: Kernel density “glued” to its mirror image, and tilted on its side
• Box Plots: Like a kernel density, but formatted to visualize key statistical
Violin & Box values
Plots (min/max, median, quartiles) and outliers

Numerical-Numerical Common
Distributions
uses:
• Comparing key business metrics (i.e. customer lifetime value, average order
Scatter Plots size, purchase frequency, etc.) by customer class (gender, loyalty status,
& location, etc.)
Correlation • Comparing sales performance by day of week or hour of day
KERNEL
DENSITIES
Categorical-Categorical Remember: kernel densities are just smooth versions of histograms
Distributions
• To visualize a categorical-numerical distribution, kernel densities can be
repeated to represent each class within a particular category
Categorical-Numerical
Distributions

Multivariate Teal class has a mean of ~15 and relatively

Kernel low variance (highly concentrated around the
Densities mean)

Violin & Box Yellow class has a mean of ~20 and

Plots moderate variance relative to other
categories

Numerical-Numerical
Distributions Purple class has a mean of ~25, overlaps
with yellow, and has relatively high
variance
Scatter Plots
&
Correlation
VIOLIN
PLOTS
Categorical-Categorical A violin plot is essentially a kernel density flipped vertically
Distributions and combined with its mirror image

Categorical-Numerical • Violin plots use the exact

Distributions same data as kernel densities,
just visualized slightly
Multivariate differently
Kernel • These can help you visualize
Densities the shape of each distribution,
and compare them across
Violin & Box classes more clearly
Plots

Numerical-Numerical
Distributions

Scatter Plots
& Correlation
BOX
PLOTS
Box plots are like violin plots, but
Categorical-Categorical
Distributions
designed to show key statistical attributes
rather than smooth distributions,
Categorical-Numerical including:
Distributions • Median
• Min & Max (excluding
Multivariate outliers)
Kernel
Densities • 25th & 75th Percentiles

Violin & Box • Outliers

Plots
Box plots provide a ton of information in a
Numerical-Numerical
Distributions
single visual, and can be used to quickly
compare statistical characteristics
Scatter Plots between classes
&
Correlation
LIMITATIONS OF CATEGORICAL
DISTRIBUTIONS
Categorical profiling works for simple cases, but breaks down
Categorical-Categorical quickly
• Humans are pretty good at visualizing 1, 2, or maybe even 3 variables, but
Distributions
how would you visualize a joint distribution for 10 variables? 100?
Categorical-Numerical
Distributions

Multivariate
Kernel
?
Densities

Violin & Box Categorical profiling can’t answer prescriptive or predictive

Plots questions
• Suppose you randomized several elements on your sales page (font, image,
layout, button, copy, etc.) to understand which ones drive conversions
Numerical-Numerical
Distributions • You could count conversions for individual elements, or some combinations of
elements, but categorical distribution alone can’t measure causation
Scatter Plots
&
Correlation This is when you need machine
learning!
NUMERICAL-NUM ERICAL
DISTRIBUTIONS
Numerical-Numerical distributions are common and intuitive, but
Categorical-Categorical
Distributions the most complex mathematically

Categorical-Numerical They are typically visualized using scatter plots, which plot points
Distributions
along the X and Y axis to show the relationship between two
Multivariate
variables
Kernel
Densities
• Scatter plots allow for simple, visual intuition: when one variable increases
or decreases, how does the other variable change?
Violin & Box • There are many possibilities: no relationship, positive, negative, linear, non-
Plots linear,
cubic, exponential, etc.
Numerical-Numerical Common uses:
Distributions
• Quickly visualizing how two numerical variables relate
Scatter Plots • Predicting how a change in one variable will impact another (i.e. square footage
& and house price, marketing spend and sales, etc.)
Correlation
CORRELATION

Univariate profiling metrics aren’t much help in the multivariate

Categorical-Categorical
Distributions world;
we now need a way to describe relationships between variables
Categorical-Numerical Correlation is the most common multivariate profiling metric, and
Distributions
is used to describe how a pair of variables are linearly related
Multivariate • In other words: for a given row, when one variable’s observation goes above its
Kernel mean, does the other variable’s observation also go above its mean (and vice
Densities versa)?
Violin & Box Plots

Numerical-Numerical
Distributions

Scatter Plots
& Correlation
No Positive Strong positive
correlation correlation correlation
CORRELATION

Correlation is an extension of
Categorical-Categorical
Distributions variance
• Think of correlation as a way to measure the variance of both variables at one
time
(called “co-variance”), while controlling for the scales of each variable
Categorical-Numerical
Distributions
Variance Correlation
formula formula
Multivariate

σ 𝑛𝑖=1(𝑥𝑖 − 𝑥)(𝑦𝑖 − 𝑦)
Kernel 𝑛
Densities
σ 𝑖=1 (𝑥 𝑖 − 𝜇)2
Violin & Box 𝑛−1 (𝑛 − 1)𝑠𝑥𝑠𝑦
Plots

Numerical-Numerical
Distributions • Here we multiply variable X’s difference from its mean with variable Y’s
difference
Scatter Plots from its mean, instead of squaring a single variable (like we do with variance)
& • Sx and Sy are the standard deviations of X and Y, which puts them on the same
Correlation scale
CORRELATION VS.
CAUSATION

CORRELATIO
Categorical-Categorical
Distributions

Categorical-Numerical
Distributions

Multivariate
Kernel
Densities

Violin & Box

N DOES NOT IMPLY

CAUSATIO
Plots

Numerical-Numerical
Distributions

N
Scatter Plots
&
Correlation
CORRELATION VS.
CAUSATION
Categorical-Categorical
Distributions

Drowning
Categorical-Numerical

Deaths
Distributions

Multivariate
Kernel Densities
Ice Cream Cones
Sold

Violin & Box Consider the scatter plot above, showing daily ice cream sales and
Plots drowning deaths in a popular New England vacation town
Numerical-Numerical • These two variables are clearly correlated, but do ice cream cones CAUSE people
Distributions to drown? Do drowning deaths CAUSE a surge in ice cream sales?

Scatter Plots
&
Of course not, because correlation does NOT imply
Correlation causation!
So what do you think is really going on here?
PRO TIP: VISUALIZING A THIRD
DIMENSION
Categorical-Categorical Scatter plots show two dimensions by default (X and Y), but
Distributions using symbols or color allows you to visualize additional
variables and expose otherwise hidden patterns or trends
Categorical-Numerical
Distributions

Multivariate
Kernel Densities

Violin & Box Plots

Numerical-Numerical
Distributions

Visualizing more than 3-4 dimensions is beyond human

Scatter Plots capability.
& This is where you need machine learning!
Correlation
CASE STUDY: CORRELATION

THE You’ve just landed your dream job as a Marketing Analyst at Loud & Clear,
SITUATION the hottest ad agency in San Diego.

Your client would like to understand the impact of their digital media spend,
THE and how it relates to website traffic, offline spend, site load time, and sales.
ASSIGNMENT Your role is to collect and visualize these metrics at the weekly-level in order
to begin exploring the relationships between them.

1. Gather weekly spend, traffic, website, and sales data

THE
2. Create a scatterplot to visualize any two given variables
OBJECTIVES 3. Compare the relationship between Digital Media Spend and each
variable
4. What patterns do you see, and how would you interpret them?
BEST PRACTICES: MULTIVARIATE
PROFILING
Univariate profiling is a great start, but multivariate profiling is
necessary when working with more than one variable (pretty much
always)

Understand the types of variables you’re working with (categorical vs.

numerical) to determine which profiling and visualization techniques to
use

Use categorical variables to filter or “cut” your data and quickly

compare profiling metrics or distributions across classes

Remember that correlation does not imply causation, and that variables
can be related without one causing a change in the other
PART
2:
ABOUT THIS
SERIES
This is Part 2 of a 4-Part series designed to help you build a deep, foundational understanding of
machine learning, including data QA & profiling, classification, forecasting and unsupervised
learning

PART 1 PART 2 PART PART

QA & Data Profiling Classification 3 &
Regression 4
Unsupervised Learning
Forecasting

*Copyright Maven Analytics, LLC

COURSE
OUTLINE
Machine Learning landscape & key concepts, classification
1 Intro to process, feature engineering, data splitting, overfitting, etc.
Classification

2 Classification
Models

K-Nearest Naïve Decision Random Logistic Sentiment

Trees Regression Analysis
Neighbors Bayes Forests

3 Model Selection & Model performance, hyperparameter tuning, confusion

matrices, imbalanced classes, model drift, etc.
Tuning
*Copyright Maven Analytics, LLC
*Copyright Maven Analytics, LLC
MACHINE LEARNING
LANDSCAPE
MACHINE LEARNING

Supervised Learning Unsupervised Learning Advanced

Topics

Clustering/Segmentation
Classification Regression Reinforcement Learning
K-Means (Q-learning, deep RL, multi-armed-bandit, etc.)

K-Nearest Neighbors Least Squares Outlier Detection

LASSO/RIDGE, state-
Support vector
space, advanced Deep Learning
machines, gradient
generalized linear (Feed Forward, Convolutional, RNN/LSTM,
boosting, neural
methods, VAR, DFA, Attention, Deep RL, Autoencoder, GAN. etc.)
nets/deep learning, etc.
etc.
MACHINE LEARNING
LANDSCAPE
MACHINE LEARNING

Supervised Learning Unsupervised Learning

Has a “label” Does NOT have a “label”

A “label” is an observed variable which you are trying to predict

(purchase (1/0), sentiment score (positive/negative), product type (A/B/C),
etc.)

Focused on describing or organizing

Focused on predicting a label or value
the data in some non-obvious
way
MACHINE LEARNING
LANDSCAPE
MACHINE
LEARNING
Supervised Unsupervised
Learning Learning

Classification Regression CLASSIFICATION

EXAMPLES:
• What type of subscribers are most likely to cancel or
• churn?
Which segment does this customer fall into?
Used to predict Used to
classes for predict values • Is the sentiment of this review positive, negative or
CATEGORICA for neutral?
L NUMERICAL
variables variables REGRESSION EXAMPLES:
• How much revenue are we forecasting for the quarter?
• How many units do we expect to sell next season?
• How will seasonality impact our customer base next
year?
RECAP: KEY
CONCEPTS
These are categorical
All tables contain rows and variables

columns
• Each row represents an individual record/observation
Session ID Browser Time (Sec) Pageviews Purchase

1 Chrome 354 11 1
• Each columns represent an individual variable
2 Safari 94 4 1

3 Safari 36 2 0
Variables can be categorical or numerical 4 IE 17 1 0

• Categorical variables contain classes or categories (used for 5 Chrome 229 9 1

filtering)
These are numerical
• Numerical variables contain numbers (used for aggregation) variables
Each row represents a record of a unique web
QA and data profiling comes first, every session

time
• Without quality data, you can’t build quality models (“garbage in, garbage out”)
• Profiling is the first step towards conditioning (or filtering) on key variables to understand their
impact

Machine Learning is needed when visual analysis falls short

• Humans aren’t capable of thinking beyond 3+ dimensions, but machines are built for it
• ML is a natural extension of visual analysis and univariate/multivariate profiling
CLASSIFICATION 101

The goal of any classification model is to predict a dependent variable using independent
variables

𝒚 Dependent variable
• The dependent
•
(DV)
variable is commonly referred to as “Y”, “predicted”, “output”, or “target” variable
Classification is about understanding how the DV is impacted by, or dependent on, other variables in the
• model
This is the variable you’re trying to predict

𝐱 Independent variables
• These are the
•
variables which help you predict the dependent variable
Independent variables are commonly referred to as “X’s”, “predictors”, “features”, “dimensions”, “explanatory variables”, or
(IVs) “covariates”
• Classification is about understanding how the IVs impact, or predict, the DV
CLASSIFICATION 101

EXAMPLE: Using data from a CRM database (sample below) to predict if a customer will churn next
month

Cust. ID Gender Status HH Income Age Sign-Up Date Newsletter Churn Churn is our dependent
1 M Bronze $30,000 29 Jan 17, 2019 1 0 variable,
since it’s what we want to
2 F Gold $60,000 37 Mar 18, 2017 1 1
predict
3 F Bronze $15,000 24 Oct 1, 2020 0 0

4 F Silver $75,000 41 Apr 12, 2019 1 0

5 M Bronze $40,000 36 Jul 23, 2020 1 1

6 M Gold $35,000 31 Oct 22, 2017 0 1

7 F Gold $80,000 46 May 2, 2019 0 0

8 F Bronze $20,000 33 Feb 20, 2020 0 0

9 M Silver $100,000 51 Aug 5, 2020 1 1

10 M Silver $45,000 29 Sep 15, 2019 1 0

Gender, Status, HH Income, Age, Sign-Up Date and Newsletter are

all
independent variables, since they can help us explain, or predict,
Churn
CLASSIFICATION 101

EXAMPLE: Using data from a CRM database (sample below) to predict if a customer will churn next
month

Cust. ID Gender Status HH Income Age Sign-Up Date Newsletter Churn

1 M Bronze $30,000 29 Jan 17, 2019 1 0

2 F Gold $60,000 37 Mar 18, 2017 1 1

3 F Bronze $15,000 24 Oct 1, 2020 0 0 We use records with observed values for
4 F Silver $75,000 41 Apr 12, 2019 1 0 both independent and dependent
5 M Bronze $40,000 36 Jul 23, 2020 1 1
variables to “train” our classification
model...
6 M Gold $35,000 31 Oct 22, 2017 0 1

7 F Gold $80,000 46 May 2, 2019 0 0

8 F Bronze $20,000 33 Feb 20, 2020 0 0

9 M Silver $100,000 51 Aug 5, 2020 1 1

10 M Silver $45,000 29 Sep 15, 2019 1 0

...then apply that model to new,

11 F Bronze $40,000 40 Dec 12, 2020 0 ??
unobserved
? values containing IVs but no DV
This is what our model will predict!
CLASSIFICATION WORKFLOW

Project
Scoping
Remember, these steps ALWAYS come first
Preliminary Before building a model, you should have a deep understanding of both the project scope (stakeholders,
QA framework, desired outcome, etc.) and the data at hand (variable types, table structure, data quality, profiling
metrics, etc.)
Data
Profiling

Feature Data Model Selection &

Engineering Splitting Training Tuning

Adding new, calculated Splitting records into Building classification models Choosing the best
variables (or “features”) “Training” and “Test” from Training data and performing model for a
to a data set based on data sets, to validate applying to Test data to given prediction, and
existing fields accuracy and avoid maximize prediction tuning it to prevent drift
overfitting accuracy over time
FEATURE ENGINEERING

Feature engineering is the process of enriching a data set by creating

additional independent variables based on existing fields
• New features can help improve the accuracy and predictive power of your ML models
• Feature engineering is often used to convert fields into “model-friendly” formats; for
example,
one-hot encoding transforms categorical variables into binary (1/0) fields

Original features Engineered

features

Cust. ID Status HH Income Age Sign-Up Date Newsletter Gold Silver Bronze Scaled Income Log Income Age Group Sign-Up Year Priority

1 Bronze $30,000 29 Jan 17, 2019 1 0 0 1 .25 10.3 20-30 2019 1

2 Gold $60,000 37 Mar 18, 2017 1 1 0 0 .75 11.0 30-40 2017 1

3 Bronze $15,000 24 Oct 1, 2020 0 0 0 1 0 9.6 20-30 2020 0

4 Silver $75,000 41 Apr 12, 2019 1 0 1 0 1 11.2 40-50 2019 0

5 Bronze $40,000 36 Jul 23, 2020 1 0 0 1 .416 10.6 30-40 2020 1

FEATURE ENGINEERING
TECHNIQUES
Common feature engineering
techniques:
Cust. ID Status HH Income Age Sign-Up Date Newsletter Gold Silver Bronze Scaled Income Log Income Age Group Sign-Up Year Priority

1 Bronze $30,000 29 Jan 17, 2019 1 0 0 1 .25 10.3 20-30 2019 1

2 Gold $60,000 37 Mar 18, 2017 1 1 0 0 .75 11.0 30-40 2017 1

3 Bronze $15,000 24 Oct 1, 2020 0 0 0 1 0 9.6 20-30 2020 0

4 Silver $75,000 41 Apr 12, 2019 1 0 1 0 1 11.2 40-50 2019 0

5 Bronze $40,000 36 Jul 23, 2020 1 0 0 1 .416 10.6 30-40 2020 1

One-Hot Log Date

Scalin Transformatio Discretization Extractio Boolean
Encoding g n n Logic
Splitting categorical Standardizing Converting a range of Grouping Transforming a date Using and/or logic
fields into separate numerical ranges values into a continuous values or datetime value to encode
binary features common scales (i.e. 1- compressed, “less- into discrete into individual “interactions”
10, 0-100%) skewed” distribution segments or bins components between variables
DATA SPLITTING

Splitting is the process of partitioning data into separate sets of records for
the purpose of training and testing machine learning models
• As a rule of thumb, ~70-80% of your data will be used for Training (which is what your
model
learns from), and ~20-30% will be reserved for Testing (to validate the model’s accuracy)

Cust. ID Gender Status HH Income Age Sign-Up Date Newsletter Churn

1 M Bronze $30,000 29 Jan 17, 2019 1 0

Test data is NOT used to optimize
models
2 F Gold $60,000 37 Mar 18, 2017 1 1

3 F Bronze $15,000 24 Oct 1, 2020 0 0 Using Training data for optimization and Test
Training
4 F Silver $75,000 41 Apr 12, 2019 1 0 data for validation ensures that your model can
data
5 M Bronze $40,000 36 Jul 23, 2020 1 1 accurately predict both known and unknown
6 M Gold $35,000 31 Oct 22, 2017 0 1
values, which helps to prevent overfitting
7 F Gold $80,000 46 May 2, 2019 0 0

8 F Bronze $20,000 33 Feb 20, 2020 0 0

9 M Silver $100,000 51 Aug 5, 2020 1 1 Test

data
10 M Silver $45,000 29 Sep 15, 2019 1 0
OVERFITTING
Splitting is primarily used to avoid overfitting, which is when a model predicts known (Training) data
very well but unknown (Test) data poorly
• Think of overfitting like memorizing the answers to a test instead of actually learning the material; you’ll
ace the
test, but lack the ability to generalize and apply your knowledge to unfamiliar questions

OVERFIT WELL-FIT UNDERFIT model

model model
• Models the Training data too well • Models the Training data just • Doesn’t model Training data well
• Doesn’t generalize well to Test data right enough
(high variance, low bias) • Generalizes well to Test data • Doesn’t generalize well to Test data
(balance of bias & variance) (high bias, low variance)
OVERFITTING
Splitting is primarily used to avoid overfitting, which is when a model predicts known (Training) data
very well but unknown (Test) data poorly
• Think of overfitting like memorizing the answers to a test instead of actually learning the material; you’ll
ace the
test, but lack the ability to generalize and apply your knowledge to unfamiliar questions

OVERFIT WELL-FIT UNDERFIT model

In this section we’ll introduce common classification models used to predict

categorical
outcomes, including KNN, naïve bayes, decision trees, logistic regression, and more

TOPICS WE’LL COMMON USE

COVER: CASES:
• Predicting if a subscriber will stay or churn
K-Nearest Naïve Bayes
Neighbors • Predicting which product a customer will purchase
• Predicting tone or sentiment from raw text
Decision Trees Random Forests (product reviews, survey responses, tweets, etc.)
• Predicting if an email is spam, or a bank transaction
Logistic Regression Sentiment Analysis is fraudulent
*Copyright Maven Analytics, LLC
K-NEAREST NEIGHBORS
(KNN)
K-Nearest Neighbors (KNN) is a classification technique designed
K-Nearest
Neighbors
to predict outcomes based on similar observed values

• In its simplest form, KNN creates a scatter plot with training data, plots a new
Naïve Bayes unobserved value, and assigns a class (DV) based on the classes of nearby
points

Decision Trees • K represents the number of nearby points (or “neighbors”) the model will
consider
when making a prediction
Random Forests

$10,000
No Purchase

AGE
20 25 30 35 40 45 50 55 60 65 70 75
Logistic Regression

???
X Y K 3 Purchase
50 $80,000 6 3 No
Sentiment Analysis Purchase
UNOBSERVED VALUE NEIGHBORS PREDICTION
K-NEAREST NEIGHBORS
(KNN)

K-Nearest What if there’s a tie?

Neighbors • In practice, KNN also factors distance between neighbors; if there are
multiple modes, the class with the shortest total distance to the
Naïve Bayes observation wins

How do we figure out the “right” value for K?

Decision Trees
• This is where Machine Learning comes in! The model tests multiple K values
to see which one is most accurate when applied to your Test data
Random Forests

What if we have more than 2 IVs?

Logistic • Scatter plots are great for demonstrating how KNN works for 2 independent
variables, but humans can’t visualize beyond 3 or 4 dimensions
• Computers can easily apply the same logic and calculations to many IVs
Regression
CASE STUDY: K-NEAREST
NEIGHBORS
THE Congratulations! You just landed your dream job as a Machine Learning
SITUATION Engineer at Spotify, one of the world’s largest music streaming
platforms

Your first assignment is to analyze user listening behavior to help improve

THE Spotify’s recommendation engine.
ASSIGNMENT You’ll need to use a KNN model to predict if a user will listen, skip or favorite
a given track, based on various attributes.

THE 1. Collect sample data containing listener behaviors and song

attributes
OBJECTIVES
2. Visualize outcomes for a given pair of attributes using a Scatter Plot

3. Calculate the predicted outcome (listen, skip or favorite) and level

of confidence for any unobserved value
*Copyright Maven Analytics, LLC
NAÏVE
BAYES
Naïve Bayes is a classification technique which uses
K-Nearest
Neighbors conditional probabilities to predict multi-class or binary
outcomes
Naïve
Bayes
• Naïve Bayes essentially creates frequency tables for each combination of
variables, then calculates the conditional probability of each IV value for a given
• For new observations, the model looks at the probability that each IV value
outcome
Decision Trees would be observed given each outcome, and compares the results to make a
prediction

Random Forests
Example use cases:
• Predicting purchase probability for prospects in a marketing database
Logistic
• Credit risk scoring in banking or financial industries

Regression
NAÏVE
BAYES
These are the independent variables
Each record represents a (IV’s)
customer which will help us make a prediction

K-Nearest Cust. ID
Subscribed Followed Visited
Purchase?
to FB Websit
Neighbors Newsletter Page e
1 0 1 0 1 This is the dependent variable
(DV) we are predicting
Naïve 2 1 0 1 0
Bayes
3 1 1 0 1

4 0 0 0 0
Decision Trees
5 1 0 0 0 These are our observed values,
which we use to train the model
6 1 1 1 1
Random Forests 7 1 0 1 0

8 0 1 1 1

9 1 0 1 1
Logistic
10 1 1 0 0

This is an unobserved
11 0 1 1 ???
Regression value; which our model
will predict
NAÏVE
BAYES
Frequency tables
between each IV and
the DV

K-Nearest Cust. ID
Subscribed Followed Visited
Purchase?
to FB Websit
Neighbors Newsletter Page e
PURCHASE
1 0
1 0 1 0 1
1
Naïve 2 1 0 1 0

NEW
Bayes 0

S
3 1 1 0 1

4 0 0 0 0
Decision Trees 1 0

5 1 0 0 0 1

B
F
6 1 1 1 1 0
Random Forests 7 1 0 1 0
1 0
8 0 1 1 1
1
9 1 0 1 1
Logistic

SIT
E
0
10 1 1 0 0

11 0 1 1 ???
Regression
NAÏVE
BAYES

K-Nearest Cust. ID
Subscribed Followed Visited
Purchase?
to FB Websit
Neighbors Newsletter Page e
PURCHASE
1 0
1 0 1 0 1
Naïve 1 3
2 1 0 1 0

NEW
Bayes 0

S
3 1 1 0 1

4 0 0 0 0
Decision Trees 1 0

5 1 0 0 0 1

B
F
6 1 1 1 1 0
Random Forests 7 1 0 1 0
1 0
8 0 1 1 1
1
9 1 0 1 1
Logistic

SIT
E
0
10 1 1 0 0

11 0 1 1 ???
Regression
NAÏVE
BAYES

K-Nearest Cust. ID
Subscribed Followed Visited
Purchase?
to FB Websit
Neighbors Newsletter Page e
PURCHASE
1 0
1 0 1 0 1
Naïve 1 3 4
2 1 0 1 0

NEW
Bayes 0

S
3 1 1 0 1

4 0 0 0 0
Decision Trees 1 0

5 1 0 0 0 1

B
F
6 1 1 1 1 0
Random Forests 7 1 0 1 0
1 0
8 0 1 1 1
1
9 1 0 1 1
Logistic

SIT
E
0
10 1 1 0 0

11 0 1 1 ???
Regression
NAÏVE
BAYES

K-Nearest Cust. ID
Subscribed Followed Visited
Purchase?
to FB Websit
Neighbors Newsletter Page e
PURCHASE
1 0
1 0 1 0 1
Naïve 1 3 4
2 1 0 1 0

NEW
Bayes 0 2

S
3 1 1 0 1

4 0 0 0 0
Decision Trees 1 0

5 1 0 0 0 1

B
F
6 1 1 1 1 0
Random Forests 7 1 0 1 0
1 0
8 0 1 1 1
1
9 1 0 1 1
Logistic

SIT
E
0
10 1 1 0 0

11 0 1 1 ???
Regression
NAÏVE
BAYES

K-Nearest Cust. ID
Subscribed Followed Visited
Purchase?
to FB Websit
Neighbors Newsletter Page e
PURCHASE
1 0
1 0 1 0 1
Naïve 1 3 4
2 1 0 1 0

NEW
Bayes 0 2 1

S
3 1 1 0 1

4 0 0 0 0
Decision Trees 1 0

5 1 0 0 0 1

B
F
6 1 1 1 1 0
Random Forests 7 1 0 1 0
1 0
8 0 1 1 1
1
9 1 0 1 1
Logistic

SIT
E
0
10 1 1 0 0

11 0 1 1 ???
Regression
NAÏVE
BAYES

K-Nearest Cust. ID
Subscribed Followed Visited
Purchase?
to FB Websit
Neighbors Newsletter Page e
PURCHASE
1 0
1 0 1 0 1
Naïve 1 3 4
2 1 0 1 0

NEW
Bayes 0 2 1

S
3 1 1 0 1

4 0 0 0 0
Decision Trees 1 0

5 1 0 0 0 1 4 1

B
F
6 1 1 1 1 0 1 4
Random Forests 7 1 0 1 0
1 0
8 0 1 1 1
1 3 2
9 1 0 1 1
Logistic

SIT
E
0 2 3
10 1 1 0 0

=
Overall
11 0 1 1 ??? 50% 50%
Regression :
NAÏVE
BAYES
Frequency tables give us the conditional probability of each
outcome
• For example, P (News | Purchase) tells us the probability that a customer
K-Nearest
Neighbors subscribed to the newsletter, given (or conditioned on) the fact that they
purchased
PURCHASE P (News | Purchase) 60
Naïve %
1 0 P (No News |
Bayes
1 3 4 40
Purchase) P (FB | Probability of each independent
%

NEW
Purchase) variable, given that a purchase was
Decision Trees 0 2 1 80

S
made
P (No FB | %
1 0 20
Purchase) P (Site |
1 4 1 %
Random Forests Purchase)
60
B
F

0 1 4 P (No Site | Purchase) %

P (News | No 40 Probability of each independent
Logistic Regression 1 0 % variable, given that a purchase was
Purchase)
NOT made
1 3 2 P (No News | No 80
%
SIT

Purchase)
E

0 2 3 20
P (FB | No Purchase)
Sentiment Analysis %
P (No FB | No Overall probability of
20 purchase
Purchase) P (Site | No %
NAÏVE
BAYES
Unobserved
P (News | Purchase) 60 value:
NEWS FB SITE
%
K-Nearest P (No News | 0 1 1
40
Neighbors Purchase) P (FB |
%
Purchase)
80
Naïve P (No FB | Purchase) % Prob. given Purchase = Prob. given Purchase =
1 0
Bayes

=
20
P (No FB | No %
Logistic Regression
Purchase) P (Site | No 20
%
Purchase)
80
P (No Site | No
Sentiment Analysis %
Purchase) P (Purchase)
40
P (No Purchase) %
60
NAÏVE
BAYES
Unobserved
P (News | 60 value:
NEWS FB SITE
Purchase) %
K-Nearest P (No News | 40 0 1 1
Purchase) %
Neighbors P (FB | 80
Purchase) %
P (No FB | 20
Purchase) % Prob. given Purchase =
Naïve P (Site | 60 Prob. given Purchase =
Purchase) % 1 0
Bayes

=
80
P (Purchase)
Purchase) P (Site | No 50
Logistic
%
%
9.6%
P (No Purchase)
Purchase) 50
40
%
%
P (No Site | No
Purchase) 60
Regression %
NAÏVE
BAYES
Unobserved
P (News | Purchase) 60 value:
NEWS FB SITE
%
K-Nearest P (No News | 0 1 1
40
Neighbors Purchase) P (FB |
%
Purchase)
80
Naïve P (No FB | Purchase) % Prob. given Purchase = Prob. given Purchase =
1 0
Bayes

=
%
Purchase) P (Purchase)
Logistic 50
9.6% 0.8%
P (No Purchase) 50
%
%

Regression
NAÏVE
BAYES
Unobserved
P (News | Purchase) 60 value:
NEWS FB SITE
%
K-Nearest P (No News | 0 1 1
40
Neighbors Purchase) P (FB |
%
Purchase)
80
Naïve P (No FB | Purchase) % Prob. given Purchase = Prob. given Purchase =
1 0
Bayes

=
20
P (No FB | No
Logistic
% 9.6% 0.8%
Purchase) P (Site | No 20
%
Purchase)
80
P (No Site | No
Regression %
Overall Purchase = PURCHASE
Purchase) P (Purchase) 9.6%
Probability: 40
92.3%
PREDICTION
P (No Purchase) % (9.6% +

60 0.8%)
NAÏVE
BAYES

K-Nearest
Neighbors Who has time to work through all this math?

Naïve • No one! These probabilities are all calculated automatically by the

Bayes model
(no manual effort required!)

Decision Trees
Doesn’t multiplying many probabilities lead to very small
values?
Random Forests
• Yes, which is why the relative probability is so important; even so,
Naïve Bayes can struggle with a very large number of IVs
Logistic Regression

Sentiment Analysis
CASE STUDY: NAÏVE
BAYES
THE You’ve just been promoted to Marketing Manager at Cat Slacks, a global
SITUATION retail powerhouse specializing in high-quality pants for cats

Your boss wants to understand the impact of key customer interactions

THE on purchase behavior, including subscribing to the newsletter, following
ASSIGNMENT the Facebook page, and visiting the Cat Slacks website

1. Collect sample data containing customer interactions and purchase

THE
events
OBJECTIVES
2. Compare the purchase probability for specific interactions
3. Calculate the overall purchase probability for any given set of
interactions
*Copyright Maven Analytics, LLC
DECISION
TREES
Decision Trees use a series of binary (true/false) rules to predict
K-Nearest
Neighbors
multi- class or binary outcomes
• For each rule, a decision tree selects a single IV and uses it to split
observations into two groups, where each group (ideally) skews towards one
Naïve Bayes class
• The goal is to determine which rules and IVs do the best job splitting up
Decision the classes, which we can measure using a metric known as entropy
Trees
• This is NOT a manual process; decision trees test many variables and select
rules based on the change in entropy after the split (known as information
Random Forests gain)

Logistic Example use cases:

Regression
• Predicting if a customer will cancel/churn next month
• Identifying fraudulent bank transactions or insurance
claims
Sentiment Analysis
DECISION
TREES
Dependent variable:
CHURN
Independent
Q: Will this subscriber churn next month?
K-Nearest Variables

Neighbors Age

10 10 NO
Gende YES
HH Income
Naïve Bayes r Gender
Sign-Up Date
M F

Decision Last Log-In

Trees Lifetime Value

4 YES 4 NO 6 YES 6 NO
Random Forests

Splitting on Gender isn’t effective, since our classes are still evenly distributed after the
Logistic
split (in other words, Gender isn’t a good predictor for churn)
Regression

Sentiment Analysis
DECISION
TREES
Dependent variable:
CHURN
Independent Variables
Q: Will this subscriber churn next month?
K-Nearest
Neighbors Age

Gende 10 10 NO
YES
HH Income
r
Naïve Bayes Last Log-In
Sign-Up Date
<7 >7
days days

Decision Last Log-In

Trees Lifetime Value

4 YES 8 NO 7 YES 1 NO
Random Forests

How do we know which independent variables to split?

Logistic • Entropy measures the “evenness” or “uncertainty” between classes, and can
Regression be
compared before/after each split to measure information gain

Sentiment Analysis
• This is where Machine Learning comes in; the model tries all IVs to
determine which splits reduce entropy the most
DECISION
TREES
Probability of Class Probability of Class
1 2
(Count of Class 1 / N) (Count of Class 2 / N)

K-Nearest
Neighbors 𝑙𝑜𝑔2 𝑃1 − 𝑃2 ∗ 𝑙𝑜𝑔2(𝑃2)
ENTROPY = −𝑃1 ∗
Curve between 0 -
1
Naïve Bayes
1.00

Decision
Trees Entropy = 1
0.75
(max)
50/50 split between
Random Forests ENTROP classes
0.50
Y

Logistic
Regression 0.25
Entropy = 0 Entropy = 0
(min) (min)
All Class 2 (P1=0) All Class 1 (P2=0)
P1
Sentiment Analysis
0.25 0.50 0.75 1.00
DECISION
TREES
Dependent variable:
CHURN
Independent Variables
Q: Will this subscriber churn next month?
K-Nearest
Age
Neighbors Entropy =
1
Gender
10 10 NO
YES
HH Income
Naïve Bayes Last Log-In
Sign-Up Date
<7 >7
days days

Decision Last Log-In

Trees Lifetime Value

Entropy Entropy
= .91 = .54
4 YES 8 NO
(- 0.09) 7 YES 1 NO
(- 0.46)
Random Forests

The reduction in entropy after the split tells us that we’re gaining information,
Logistic and teasing out differences between those who churn and those who do
Regression not

Sentiment Analysis
DECISION
TREES
Dependent variable:
CHURN
Independent Variables
Q: Will this subscriber churn next month?
K-Nearest
Age
Neighbors Entropy =
1
Gender
10 10 NO
YES
HH Income
Naïve Bayes Last Log-In
Sign-Up Date
<7 >7
days days

Decision Last Log-In

Trees Lifetime Value

Entropy Entropy
= .91 = .54
4 YES 8 NO
(- 0.09) 7 YES 1 NO
(- 0.46)
Random Forests
Lifetime Value Age

<$50 >$50 <35 >35

Logistic
Regression

2 YES 3 2 YES 5 0 YES 1 NO 7 YES 0 NO

NO NO
Sentiment Analysis Entropy = 0 Entropy = 0
Entropy Entropy
= .97 (- 0.54) (- 0.54)
= .86
DECISION
TREES
Root node

Decision node
K-Nearest 10 YES 10 NO

Neighbors Last Log-In

Leaf
<7 >7
days days node

Naïve Bayes
4 YES 8 NO 7 YES 1 NO

Decision Lifetime Value Age

Trees <$50 >$50 <35 >35

Random Forests

2 YES 3 NO 2 YES 5 NO 0 YES 1 NO 7 YES 0 NO

Sign-Up Date
Logistic
<60 >60
Regression days days
UNOBSERVED VALUE PREDICTION

52-year-old customer who

last CHURN
logged in 21 days ago
Sentiment Analysis
1 YES 0 NO 1 YES 3 NO
DECISION
TREES

K-Nearest 10 YES 10 NO

Neighbors Last Log-In

<7 >7
days days

Naïve Bayes
4 YES 8 NO 7 YES 1 NO

Decision Lifetime Value Age

Trees <$50 >$50 <35 >35

Random Forests

2 YES 3 NO 2 YES 5 NO 0 YES 1 NO 7 YES 0 NO

Sign-Up Date
Logistic
<60 >60
Regression days days
UNOBSERVED VALUE PREDICTION

Customer who logged in 3

days ago, has an LTV of $28, NO
and signed up 90 days ago
Sentiment Analysis CHURN
1 YES 0 NO 1 YES 3 NO
DECISION
TREES

K-Nearest 10 YES 10 NO

Neighbors Last Log-In

<7 >7
days days

Naïve Bayes
4 YES 8 NO 7 YES 1 NO

Decision Lifetime Value Age

Trees <$50 >$50 <35 >35

Random Forests

2 YES 3 NO 2 YES 5 NO 0 YES 1 NO 7 YES 0 NO

Sign-Up Date
Logistic
<60 >60
Regression days days
UNOBSERVED VALUE PREDICTION

Customer who logged in 5

days NO
Sentiment Analysis
ago and has an LTV of $70
CHURN
1 YES 0 NO 1 YES 3 NO
DECISION
TREES

K-Nearest How do we know when to stop splitting?

Neighbors
• Splitting too much can lead to overfitting, so you can adjust model
inputs
(known as “hyperparameters”) to limit things like tree depth or leaf size
Naïve Bayes

Decision How do we know where to split numerical IVs?

Trees
• In practice, decision trees will test different numerical splits and optimize
based on information gain (just like testing individual IV splits)
Random Forests

Logistic Does the best first split always lead to the most accurate model?
Regression
• Not necessarily! That’s why we often use a collection of multiple decision
trees, known as a random forest, to maximize accuracy
Sentiment Analysis
RANDOM
FORESTS
A Random Forest is a collection of individual decision trees, each
built using a random subset of observations
K-Nearest
Neighbors • Each decision tree randomly selects variables to evaluate at each split
• Each tree produces a prediction, and the mode, or most frequent prediction,
wins
Naïve Bayes TREE 1
PREDICTION
SPLIT SPLIT SPLIT
1 2 N CHURN
Age Age Age

Decision Trees Gender Gender Gender TREE 2

PREDICTION
Income Income Income NO
Random CHURN
FINAL PREDICTION

Forests
CHURN
Sign- Sign- Sign-
Up Up Up
TREE 3
Log- Log- Log- PREDICTION

Logistic
In In In
CHURN
LTV LTV LTV
Regression
TREE N
PREDICTION
CHURN
Sentiment
Analysis
CASE STUDY: DECISION
TREES
THE You are the founder and CEO of Trip Genie, an online subscription
SITUATION service designed to connect global travelers with local guides.

You’d like to better understand your customers and identify which types of
THE behaviors can be used to help predict paid subscriptions.
ASSIGNMENT Your goal is to build a decision tree to help you predict subscriptions based
on multiple factors (newsletter, Facebook follow, time on site, sessions, etc.).

1. Collect Training data at the customer-level

THE
OBJECTIVES 2. Split on each independent variable and compare changes in entropy
3. Build a decision tree to best predict subscriptions based on available
IVs
*Copyright Maven Analytics, LLC
LOGISTIC REGRESSION

Logistic Regression is a classification technique used to predict

K-Nearest
Neighbors the probability of a binary (true/false) outcome
• In its simplest form, logistic regression forms an S-shaped curve between 0 -
Naïve Bayes 1, which represents the probability of a TRUE outcome for any given value
of X
• The likelihood function measures how accurately a model predicts outcomes,
Decision Trees and
is used to optimize the “shape” of the curve
• Although it has the word “regression” in its name, logistic regression is not used
for predicting numeric variables
Random Forests

Example use cases:

Logistic
Regression • Flagging spam emails or fraudulent credit card transactions
• Determining whether to serve a particular ad to a website
Sentiment Analysis visitor
LOGISTIC REGRESSION

TRUE
K-Nearest Neighbors (1)

Naïve Bayes

0.5

Decision Trees

Random Forests
FALSE (0)
X1

Logistic
Regression
• Each dot represents an observed value, where X is a numerical
independent variable and Y is the binary outcome (true/false) that we
Sentiment Analysis want to predict
LOGISTIC REGRESSION

TRUE
K-Nearest Neighbors (1)

Naïve Bayes

PROBABILIT
0.5

Decision Trees

Y
Random Forests
FALSE
(0)
X1

Logistic
Regression
• Logistic regression plots the best-fitting curve between 0 and 1, which tells us
the probability of Y being TRUE for any given value of X1
Sentiment Analysis
LOGISTIC REGRESSION

SPAM
K-Nearest Neighbors

Naïve Bayes

PROBABILIT
0.5

Decision Trees

Y
Random Forests
NOTFALSE
SPA M #
(0) 5 10 15 20 25
RECIPIENTS
Logistic
Regression
• Here we’re using logistic regression to predict if an email will be marked as
spam,
Sentiment Analysis based on the number of email recipients (X1)
LOGISTIC REGRESSION

SPAM
K-Nearest
P = 95%
Neighbors Prediction: SPAM

Naïve Bayes

PROBABILIT
0.5

Decision Trees P = 50%

Prediction: ??

Y
?

Random Forests P = 1%
Prediction: NOT
SPAM
NOTFALSE
SPA M #
(0) 5 10 15 20 25
RECIPIENTS
Logistic
Regression
• Using this model, we can test unobserved values of X1 to predict the probability
that
Sentiment Analysis Y is true or false (in this case the probability that an email is marked as spam)
LOGISTIC REGRESSION

SPAM
K-Nearest
0.9
Neighbors
In this case the risk of categorizing a real email

PROBABILIT
as spam is high, so our decision point may be
>50%
Naïve Bayes
By increasing the threshold to 90%,
0.5
we:
1. Correctly predict more of the legit
(not spam) emails, which is really
Decision Trees

Y
important
2. Incorrectly mark a few spam emails
as
“not spam”, which is not a big deal
Random
NOTFALSE
SPA M #
(0) 5 10 15 20 25
RECIPIENTS
Logistic
Forests
Regression Is 50% always the right decision point for logistic models?
Sentiment • No. It depends on the relative risk of a false positive (incorrectly predicting a
Analysis TRUE outcome) or false negative (incorrectly predicting a FALSE outcome)
LOGISTIC REGRESSION

QUIT
K-Nearest
Neighbors
In this case the risk of a false positive

PROBABILIT
is
Naïve Bayes low, so our decision point may be
<50%
By decreasing the threshold to 10%, we:
0.5

1. Correctly predict more cases where

someone
is likely to quit, which is really important
Decision Trees

Y
2. Incorrectly flag some employees who plan
to stay, which is not a big
deal
0.1
Random
STAY %
25% 50% 75% 100%
NEGATIVE
FEEDBACK
Logistic
Forests
Regression • Now consider a case where we’re predicting if an employee will quit based on
negative feedback from HR
Sentiment • It’s easier to train an employee than hire a new one, so the risk of a false positive
Analysis is
low but the risk of a false negative (incorrectly predicting someone will stay) is high
LOGISTIC REGRESSION

K-Nearest
Makes the output
fall between 0 1
and 1 Linear equation, where
Neighbors
1 + 𝑒− (𝛽0 +𝛽1 𝑥1 ) 𝖰𝟎 is the intercept, X
is the IV value and 𝖰 𝟏
is
Naïve Bayes the weight (or slope)

Decision Trees

Random Forests
𝖰𝟏 =
0.5 0.5
𝖰𝟏 =
Logistic
Regression 1.0
𝖰𝟏 =
Sentiment Analysis 0 X1 5.0
LOGISTIC REGRESSION

K-Nearest
Makes the output
fall between 0 1
and 1 Linear equation, where
Neighbors
1 + 𝑒− (𝛽0 +𝛽1 𝑥1 ) 𝖰𝟎 is the intercept, X
is the IV value and 𝖰 𝟏
is
the weight (or slope)
Naïve Bayes
1

Decision Trees

𝖰𝟏 =
Random 0.5 0.5
𝖰𝟏 =
Logistic
Forests
Regression 1.0
𝖰𝟏 =
Sentiment
Analysis
0 X1 5.0
LOGISTIC REGRESSION

Decision Trees

𝖰𝟏 =
Random 0.5 0.5
𝖰𝟏 =
Logistic
Forests
Regression 1.0
𝖰𝟏 =
Sentiment
Analysis
0 X1 5.0
LOGISTIC REGRESSION

Decision Trees

𝖰𝟏 =
Random 0.5 0.5
𝖰𝟏 =
Logistic
Forests
Regression 1.0
𝖰𝟏 =
Sentiment
Analysis
0 X1 5.0
LOGISTIC REGRESSION

Decision Trees

𝖰𝟎 = 0
Random 0.5
𝖰𝟎 = 2
Logistic 𝖰 𝟎 = -2
Forests
Regression

Sentiment 0 X1
Analysis
LOGISTIC REGRESSION

Decision Trees

𝖰𝟎 = 0
Random 0.5
𝖰𝟎 = 2
Logistic 𝖰 𝟎 = -2
Forests
Regression

Sentiment 0 X1
Analysis
LOGISTIC REGRESSION

Decision Trees

𝖰𝟎 = 0
Random 0.5
𝖰𝟎 = 2
Logistic 𝖰𝟎 = -
Forests
Regression
2
Sentiment 0 X1
Analysis
LOGISTIC REGRESSION

K-Nearest How do we determine the “right” values for 𝛽? LIKELIHOOD!

Actual Observation
Random Forests
Y=1 Y=0 • When our model output is close to the actual
Y, we want likelihood to be HIGH (near 1)
Logistic ~1 HIGH LOW
Regression Model Output • When our model output is far from the actual
~0 LOW HIGH Y, we want likelihood to be LOW (near 0)

Sentiment Analysis
LOGISTIC REGRESSION

K-Nearest How do we determine the “right” values for 𝛽? LIKELIHOOD!

Actual Observation
Random Forests
Y=1 Y=0 LIKELIHOOD FUNCTION:

=
Logistic ~1 HIGH LOW
𝒚 𝟏−𝒚
Regression Model Output 𝒐𝒖𝒕𝒑𝒖𝒕 ∗ 𝟏 − 𝒐𝒖𝒕𝒑𝒖𝒕
~0 LOW HIGH

Sentiment Analysis
LOGISTIC REGRESSION

K-Nearest How do we determine the “right” values for 𝛽? LIKELIHOOD!

K-Nearest
Neighbors Y?
SPAM
Naïve Bayes

PROBABILIT
Decision Trees

Y
Random

NOT SPAM

Forests
Logistic
5 10 15 20 25

Regression #
RECIPIENTS

• # of Recipients can help us detect spam, but so can other variables like the
Sentiment Analysis
number of typos, count of words like “free” or “bonus”, sender reputation
score, etc.
LOGISTIC REGRESSION

But what if multiple X variables could help predict

K-Nearest
Neighbors Y?

Naïve Bayes

Decision Trees

Random Forests

Logistic

• Logistic regression can handle multiple independent variables, but the

Regression
Sentiment Analysis visual
interpretation breaks down at >2 IV’s (this is why we need machine
learning!)
LOGISTIC REGRESSION

But what if multiple X variables could help predict

K-Nearest
Neighbors Y?

Naïve Bayes
Makes the output
fall between 0 1
and 1
Decision Trees
1 + 𝑒− (𝛽 0 +𝛽 1 𝑥 1 +𝛽 2 𝑥 2 +⋯+𝛽 𝑛 𝑥 𝑛 )
Random Forests Weighted independent
variables (x1 , x2 ...xn)

Logistic
Regression • Logistic regression is about finding the best combination of weights (𝛽1, 𝛽2...𝛽𝑛)
for
Sentiment Analysis a given set of independent variables (x1, x2...x𝑛) to maximize the likelihood
function
CASE STUDY: LOGISTIC
REGRESSION
THE You’ve just been promoted to Marketing Manager for Lux Dining, a
SITUATION wildly popular international food blog.

The CMO is concerned about unsubscribe rates and thinks it may be

THE related to the frequency of emails your team has been sending.
ASSIGNMENT Your job is to use logistic regression to plot this relationship and predict if
a user will unsubscribe based on the number of weekly emails received.

1. Collect customer data containing email frequency and subscription

THE activity
OBJECTIVES 2. Plot the logistic regression curve which maximizes likelihood
3. Determine the appropriate email frequency (based on a 50% decision
point)
*Copyright Maven Analytics, LLC
SENTIMENT ANALYSIS

Sentiment analysis is a technique used to determine the emotional

K-Nearest
Neighbors
tone or “sentiment” behind text-based data

• Sentiment analysis often falls under Natural Language Processing (NLP), but
is typically applied as a classification technique
Naïve Bayes
• Sentiment models typically use a “bag of words” approach, which
involves calculating the frequency or presence (1/0) of key words to
Decision Trees convert text (which models struggle with) into numerical inputs

• Unlike other classification models, you must ”hand-score” the sentiment

(DV)
Random Forests
values for your Training data, which your model will learn from

Logistic Regression
Example use cases:
• Understanding the tone of product reviews posted by
Sentiment customers
Analysis • Analyzing open-ended survey responses
SENTIMENT ANALYSIS

K-Nearest
Neighbors
This word cloud is *technically* a
very simple version of sentiment
Naïve Bayes
analysis

Limitations
Decision Trees : • Based entirely on word count
• Straight-forward but not flexible

Random Forests • Requires manual interpretation

• Subject to inconsistency/human
error

Logistic Regression

Sentiment
Analysis
SENTIMENT ANALYSIS

The first step in any sentiment analysis is to clean and QA the text
to remove noise and isolate the most meaningful information:
K-Nearest
Neighbors • Remove punctuation, capitalization and special characters
• Correct spelling and grammatical errors
• Use proper encoding (i.e. UTF-8)
Naïve Bayes
• Lemmatize or stem (remove grammar tense, convert to “root” term)
• Remove stop words (“a”, “the”, “or”, “of”, “are”, etc.)
Decision Trees
The computer is running hot because I’m mining
bitcoin!
Random Forests
the computer is running hot because I’m mining
bitcoin!
Logistic Regression
NOTE: This process cancomputer
vary basedrun hotcontext;
on the mine bitcoin
for example, you may want to
Sentiment preserve capitalization or punctuation if you care about measuring intensity (i.e.
Analysis “GREAT!!” vs. “great”), or choose to allow specific stop words or special characters
SENTIMENT ANALYSIS

Once the text has been cleaned, we can transform our text into
K-Nearest numeric data using a “bag of words” approach:
Neighbors
• Split cleaned text into individual words (this is known as tokenization)

Naïve Bayes • Create a new column with a binary flag (1/0) for each word
• Manually assign sentiment for observations in your Training data

Decision Trees • Apply any classification technique to predict sentiment for unobserved
text
Sentiment is our
Each word is an independent dependent
Random Forests variable variable

hate book sentiment

garbage love
I HATE this 1 1 0 0 negative
Observe
Logistic Regression garbage! d values
I love this 0 0 1 1 positive
book. Unobserve
Sentiment This book is 0 1 0 1 ??? d value
Analysis garbage
SENTIMENT ANALYSIS

K-Nearest How do you address language nuances like ambiguity,

Neighbors double negatives, slang or sarcasm?
• Generally speaking, the more observations you score in your Training data,
Naïve Bayes the better your model will be at detecting these types of nuances
• No sentiment model is perfect, but feature engineering and
advanced techniques can help improve accuracy for more complex
Decision Trees cases

“If you like watching If you like watching paint like watch
Random Forests paint dry, you’ll love this dry paint dry love
movie!” you’ll love this movie movie

Logistic Regression “The new version is awfully The new version is awfully new version
good, not as bad as good not as bad as awful good bad
expected!” expected expect
Sentiment
Analysis
CASE STUDY: SENTIMENT
ANALYSIS
THE You’re an accomplished author and creator of the hit series Bark Twain the
SITUATION Data Dog, featuring a feisty chihuahua who uses machine learning to solve
crimes.

Reviews for the latest book in the Bark Twain series are coming in, and they
THE aren’t looking great...
ASSIGNMENT To apply a bit more rigor to your analysis and automate scoring for future
reviews, you’ve decided to use this feedback to build a basic sentiment
model.

1. Collect raw text reviews and tokenize key words

THE
2. Hand-score the sentiment for observed values in the Training
OBJECTIVES data
3. Apply any classification technique and validate using Test data
*Copyright Maven Analytics, LLC
MODEL SELECTION &
TUNING
In this section we’ll discuss techniques for selecting & tuning classification models,
including hyperparameter optimization, class balancing, confusion matrices and
more

TOPICS WE’LL COMMON USE

COVER: CASES:
• Adjusting model parameters to optimize
Hyperparameters Imbalanced performance
Classes
• Rebalancing classes to reduce bias
Confusion Matrix Selection & • Comparing performance across multiple
Drift classification models to select the best performing
option
• Retraining models with fresh data to minimize drift
HYPERPARAMETERS

Hyperparameters are inputs/settings you can control while training a

Hyperparameters
model
• Adjusting parameters to maximize accuracy is known as “hyperparameter optimization”
Imbalanced
Classes

Confusion
K-Nearest Naïve Decision Random Logistic
Trees Regression
Neighbors Bayes Forests

Matrix Model Examples Examples: Examples Examples Examples

: : : :
• K • Smoothing • Criterion • # Trees • Intercept
• Distance parameters (entropy, • # Samples • Penalty
definition gini)
Selection Model • # Features Term
• NN • Tree depth • Re- • Solver
Algorithm • Leaf size Sampling
• Max
Features
Drift • Min gain
IMBALANCED
CLASSES
Imbalanced classes occur when you are predicting outcomes
Hyperparameters
with
drastically different frequencies in the Training data
• The class or outcome which occurs more frequently is known as the
Imbalanced majority
Classes class, while the class which occurs less frequently is the minority class
• Imbalanced classes can bias a model towards always predicting the majority
Confusion class, since it often yields the best overall accuracy (i.e. 99%+)
Matrix
• This is a significant issue when predicting very rare and very important events,
like a nuclear meltdown
Model

There are several ways to balance classes in your Training

Selection data, including up-sampling, down-sampling and
weighting

Model Drift
IMBALANCED
CLASSES

Hyperparameters Up-sampling
Minority class observations
are duplicated to balance
Imbalanced the data
Classes
Down-sampling
Majority class observations
Confusion
are randomly removed to
Matrix balance the data

Model Weighting
For models that randomly sample
observations (random forests),
increase the probability of
Selection selecting the minority class

Model Drift
IMBALANCED
CLASSES

Hyperparameters x
Up-sampling
x Minority class observations
x are duplicated to balance
Imbalanced the data
Classes x
Down-sampling
x
Majority class observations
Confusion x
are randomly removed to
Matrix balance the data
x
x
x

x
Model Weighting
x For models that randomly sample
observations (random forests),
x increase the probability of
Selection selecting the minority class
x

Model Drift x

x
IMBALANCED
CLASSES

Hyperparameters Up-sampling
Minority class observations
are duplicated to balance
Imbalanced 1 the data
Classes
Down-sampling
Confusion
Majority class observations
Matrix are randomly removed to
balance the data

Model Weighting
For models that randomly sample
observations (i.e. random
forests), increase the probability
Selection of selecting the minority class

Model Drift
CONFUSION MATRIX

A Confusion Matrix is a table summarizing the frequency of predicted

Hyperparameters
vs.
actual classes in your Test data
Imbalanced • This is the most common and concise way to evaluate performance and
Classes compare classification models against one another

Confusion • Confusion matrices can be used to derive several types of model

Matrix performance metrics, including accuracy, precision and recall
ACTUAL CLASS
Model
1 0 1 0
Selection
True Positive False Positive
1 (TP) (FP)
1 100 5
PREDICTED
Model Drift CLASS
False Negative True Negative
0 (FN) (TN)
0 15 50
CONFUSION MATRIX

Confusion matrices are used to derive model performance

Hyperparameters metrics

ACTUAL CLASS
1 0
Imbalanced
True Positive False Positive
Classes 1 (TP) (FP)
PREDICTE
D
Confusion CLASS False Negative True Negative
Matrix 0 (FN) (TN)

Model
Selection
Accuracy Precision Recall
(TP+TN) / (TP+TN+FP+FN) TP /
TP / (TP+FP)
Model Drift (TP+FN)
Of all predictions, Of all predicted Of all actual positives,
what % were positives, what % what
correct? were correct? % were predicted
correctly?
CONFUSION MATRIX

Confusion matrices are used to derive model performance

Hyperparameters metrics

ACTUAL CLASS
1 0
Imbalanced
Classes 1 100 5
PREDICTE
D
Confusion CLASS
Matrix 0 15 50

Model
Selection
Accuracy Precision Recall
(TP+TN) / (TP+TN+FP+FN) TP /
TP / (TP+FP)
Model Drift (TP+FN)
=

=
(100+50)/(100+50+5+15) 100/(100+5) 100/(100+15)
=

=
.8 .9 .8
8 5 7
CONFUSION MATRIX

Will a Confusion Matrix work for multi-class predictions too?

Hyperparameters • Yes! Consider a model classifying multiple products (A, B, C, D); ideally,
we want the highest frequencies along the top-left to bottom-right
diagonal
Imbalanced ACTUAL PRODUCT ACTUAL PRODUCT
Classes
A B C D A B C D

Confusion
A A
Matrix

B B
Model PREDICTED PREDICTED
Selection PRODUCT PRODUCT
C C

D D
Model Drift

Good confusion Bad confusion

matrix matrix
CONFUSION MATRIX

A confusion matrix can also provide insight into relationships

Hyperparameters
and interactions between classes, to help inform model
improvements
Imbalanced
ACTUAL PRODUCT
Classes
A B C D
In this case our model has a hard time
Confusion differentiating products B and C, often predicting
A
Matrix the wrong one
• Are these products very similar? Do we need to
B predict them separately?
Model PREDICTED
PRODUCT • If so, can we engineer features to help
Selection C distinguish B from C, and improve model
accuracy?
D

Model Drift
CONFUSION MATRIX

To score a multi-class confusion matrix, calculate metrics for each

Hyperparameters predicted class, then take a weighted average to evaluate the model as a
whole
ACTUAL PRODUCT PRODUCT
Imbalanced A:
A B C D Accuracy
Classes
( TP + TN ) / ( TP + TN + FP + FN )
A ( 214 + 452 + 1 + 1123 + 19 + 2 + 12 + 34 ) /
Confusion ( 1886 )

=
Matrix .9846
B
PREDICTED
PRODUCT Precision
Model C
Selection TP / ( TP + FP )
( 214 ) / ( 214 + 1 + 8 + 2 )

=
D In thiswhere
cases case, (TN) includes
Product A all .9511
was not predicted OR
observed Recall
Model Drift
TP / ( TP + FN )
( 214 ) / ( 214 + 15 +
3)

=
.9224
CONFUSION MATRIX

To score a multi-class confusion matrix, calculate metrics for each

Hyperparameters predicted class, then take a weighted average to evaluate the model as a
whole
ACTUAL PRODUCT PRODUCT
Imbalanced B:
A B C D Accuracy
Classes
( TP + TN ) / ( TP + TN + FP + FN )
A ( 452 + 214 + 8 + 2 + 1123 + 19 + 3 + 12 + 34 ) /
Confusion ( 1886 )

=
Matrix .9899
B
PREDICTED
PRODUCT Precision
Model C
Selection TP / ( TP + FP )
( 452 ) / ( 452 + 15 + 1 )

=
D
.9658
Model Drift Recall
TP / ( TP + FN )
( 452 ) / ( 452 + 1 + 2 )

=
.9934
CON FUSION
MATRIX
To score a multi-class confusion matrix, calculate metrics for each
Hyperparameters predicted class, then take a weighted average to evaluate the model as a
whole
ACTUAL PRO DUCT PRODUCT
Imbalanced Classes C:
A B C D Accuracy
( TP + TN ) / ( TP + TN + FP + FN )
A ( 1123 + 214 + 1+ 2 + 15 + 452 + 3 + 2 + 34 ) /
Confusion ( 1886 )

=
Matrix
B .9788
PREDICTED
PRODUCT Precision
Model Selection C
TP / ( TP + FP )
( 1123 ) / ( 1123 + 19 )

=
D
Model Drift .9834
Recall
TP / ( TP + FN )
( 1123 ) / ( 1123 + 8 + 1 + 12 )

=
.9816
CONFUSION MATRIX

To score a multi-class confusion matrix, calculate metrics for each

Hyperparameters predicted class, then take a weighted average to evaluate the model as a
whole
ACTUAL PRODUCT PRODUCT
Imbalanced D:
A B C D Accuracy
Classes
( TP + TN ) / ( TP + TN + FP + FN )
A ( 34 + 214 + 1 + 8 + 15 + 452 + 1 + 1123 ) /
Confusion ( 1886 )

=
Matrix .9799
B
PREDICTED
PRODUCT Precision
Model C
Selection TP / ( TP + FP )
( 34 ) / ( 34 + 3 + 2 + 12 )

=
D
.6667
Model Drift Recall
TP / ( TP + FN )
( 34 ) / ( 34 + 2 + 19 )

=
.6182
CON FUSION
MATRIX
To score a multi-class confusion matrix, calculate metrics for each
Hyperparameters predicted class, then take a weighted average to evaluate the model as a
whole
Imbalanced Classes
# Accuracy Precisio Recall
Obs. n

Confusion A 232 .9846 .9511 .9224

Matrix
B 455 .9899 .9658 .9934

Model Selection
C 1,144 .9788 .9834 .9816

D 55 .9799 .6667 .6182

Model Drift
WEIGHTED AVG: .9822 .9659 .9666
CON FUSION
MATRIX
To score a multi-class confusion matrix, calculate metrics for each
Hyperparameters predicted class, then take a weighted average to evaluate the model as a
whole
Imbalanced Classes
# Accuracy Precisio Recall
Obs. n

Confusion A 143 .7500 .1284 .0979

Matrix
B 403 .4795 .2732 .1390

Model Selection
C 207 .5498 .2239 .4348

D 200 .5792 .0759 .0900

Model Drift
WEIGHTED AVG: .5563 .1994 .1868
MODEL
SELECTION
Model selection describes the process of training multiple models,
Hyperparameters
comparing performance on Test data, and choosing the best
option
Imbalanced
When comparing performance, it’s important to prioritize the most
relevant metrics based on the context of your model
Classes • RECALL may be the best metric if it’s critical to predict ALL positive outcomes
correctly, and false negatives are a major risk (i.e. nuclear reactor)

• PRECISION may be the best metric if false negatives aren’t a big deal, but
Confusion
Model Matrix false
Selection positives are a major risk (i.e. spam filter or document search)

Model • ACCURACY may be the best metric if you care about predicting positive
Drift and negative outcomes equally, or if the risk of each outcome is
comparable
MODEL
DRIFT
Drift is when a trained model gradually becomes less accurate over
Hyperparameters
time, even when all variables and parameters remain the same
• As a best practice, all ML models used for ongoing prediction should
Imbalanced be updated or retrained on a regular basis to combat drift

Tips to correct drift:

Classes Confusion
• Benchmark your model performance (accuracy, precision, recall) on Day 1

Matrix Model • Monitor performance against your benchmark over time

• If you notice drift compared to your benchmark, retrain the model using
updated Training data and consider discarding old records (if you have enough
Selection
Model volume)
Drift
• Conduct additional feature engineering as necessary
PART
3:
ABOUT THIS
SERIES
This is Part 3 of a 4-Part series designed to help you build a deep, foundational understanding of
machine learning, including data QA & profiling, classification, forecasting and unsupervised
learning

PART 1 PART 2 PART PART 4

QA & Data Profiling Classification 3 &
Regression Unsupervised Learning
Forecasting

*Copyright Maven Analytics, LLC

COURSE
OUTLINE
Review the ML landscape and key supervised learning concepts,
1 Intro to Regression including regression vs. classification, feature engineering, overfitting,
etc.

Learn the key building blocks of regression analysis, including linear

2 Regression Modeling relationships, least squared error, multivariate and non-linear
models

Understand and interpret model outputs and common diagnostic

3 Model Diagnostics metrics like R-squared, mean error, F-significance, P-values, and more

Explore powerful tools & techniques for time-series forecasting,

4 Time-Series Forecasting including seasonality, linear and non-linear trends, intervention
analysis, etc.

*Copyright Maven Analytics, LLC

*Copyright Maven Analytics, LLC
MACHINE LEARNING
LANDSCAPE
MACHINE LEARNING

Supervised Learning Unsupervised Learning Advanced

Topics

Clustering/Segmentation
Classification Regression Reinforcement Learning
K-Means (Q-learning, deep RL, multi-armed-bandit, etc.)

K-Nearest Neighbors Least Squares Outlier Detection

Supervised Learning Unsupervised Learning

Has a “label” Does NOT have a “label”

A “label” is an observed variable which you are trying to predict

(purchase (1/0), sentiment score (positive/negative), product type (A/B/C),
etc.)

Focused on describing or organizing

Focused on predicting a label or value
the data in some non-obvious
way
MACHINE LEARNING
LANDSCAPE
MACHINE
LEARNING
Supervised Unsupervised
Learning Learning

Classification Regression CLASSIFICATION

EXAMPLES:
• What type of subscribers are most likely to cancel or
• churn?
Which segment does this customer fall into?
Used to predict Used to
classes for predict values • Is the sentiment of this review positive, negative or
CATEGORICA for neutral?
L NUMERICAL
variables variables REGRESSION EXAMPLES:
• How much revenue are we forecasting for the quarter?
• How many units do we expect to sell next season?
• How will seasonality impact the size of our customer base?
RECAP: KEY
CONCEPTS
These are categorical
All tables contain rows and variables

columns
• Each row represents an individual record/observation
Session ID Browser Time (Sec) Pageviews Purchase

1 Chrome 354 11 1
• Each columns represent an individual variable
2 Safari 94 4 1

3 Safari 36 2 0
Variables can be categorical or numerical 4 IE 17 1 0

• Categorical variables contain classes or categories (used for 5 Chrome 229 9 1

filtering)
These are numerical
• Numerical variables contain numbers (used for aggregation) variables
Each row represents a record of a unique web
QA and data profiling comes first, every session

Machine Learning is needed when visual analysis falls short

• Humans aren’t capable of thinking beyond 3+ dimensions, but machines are built for it
• ML is a natural extension of visual analysis and univariate/multivariate profiling
REGRESSION
101
The goal of regression is to predict a numeric dependent variable using independent
variables

𝒚 Dependent variable
• The dependent
•
(DV)
variable – for regression,
is commonly this
referred to as “Y”, must be
“predicted”, numerical
“output”, or “target”(not categorical)!
variable
Regression is about understanding how the numerical DV is impacted by, or dependent on, other variables in the
• model
This is the variable you’re trying to predict

𝐱 Independent variables
• These are the
•
variables which help you predict the dependent variable
Independent variables are commonly referred to as “X’s”, “predictors”, “features”, “explanatory variables”, or
(IVs) “covariates”
• Regression is about understanding how the IVs impact, or predict, the DV
REGRESSION
101
EXAMPLE: Using marketing and sales data (sample below) to predict revenue for a given
month

Socia Competitiv Marketin Promotio

Month ID
l e g n
Revenue Revenue is our dependent variable,
Post Activity Spend Count since
s
it’s what we want to predict
1 30 High $130,000 12 $1,300,050

2 15 Low $600,000 5 $11,233,310 Since Revenue is numerical, we’ll use

3 11 Medium $15,000 10 $1,112,050 regression (vs. classification) to predict
it
4 22 Medium $705,000 11 $1,582,077

5 41 High $3,000 3 $1,889,053

6 5 High $3,500 3 $1,200,089

7 23 Low $280,000 2 $1,300,080

8 12 Low $120,000 11 $700,150

9 8 Low $1,000,000 51 $41,011,150

10 19 Medium $43,000 10 $1,000,210

Social Posts, Competitive Activity, Marketing Spend and Promotion Count are
all
independent variables, since they can help us explain, or predict, monthly
Revenue
REGRESSION
101
EXAMPLE: Using marketing and sales data (sample below) to predict revenue for a given
month

Socia Competitiv Marketin Promotio

Month ID Revenue
l e g n
Post Activity Spend Count
s
1 30 High $130,000 12 $1,300,050

2 15 Low $600,000 5 $11,233,310

We’ll use records with observed values
3 11 Medium $15,000 10 $1,112,050 for both independent and dependent
4 22 Medium $705,000 11 $1,582,077 variables to “train” our regression
5 41 High $3,000 3 $1,889,053 model...
6 5 High $3,500 3 $1,200,089

7 23 Low $280,000 2 $1,300,080

8 12 Low $120,000 11 $700,150

9 8 Low $1,000,000 51 $41,011,150

10 19 Medium $43,000 10 $1,000,210 ...then apply that model to new, unobserved

11 7 High $320,112 8 ??? values
containing IVs but no DV
This is what our regression model will
predict!
REGRESSION WORKFLOW

Measurement
Planning
Remember, these steps ALWAYS come first
Preliminary Before building a model, you should have a clear measurement plan (KPIs, project scope, desired outcome, etc.)
QA
and an understanding of the data at hand (variable types, table structure, data quality, profiling metrics, etc.)
Data
Profiling

Feature Data Model Selection &

Engineering Splitting Training Tuning

Adding new, calculated Splitting records into Building regression models Choosing the best
variables (or “features”) “Training” and “Test” from Training data and performing model for a
to a data set based on data sets, to validate applying to Test data to given prediction, and
existing fields accuracy and avoid maximize prediction tuning it to prevent drift
overfitting accuracy over time
FEATURE ENGINEERING

Feature engineering is the process of enriching a data set by creating

Original Engineered
features features

Socia Competitiv Marketin Promotio Competitiv Competitiv Competitiv Promotion >10 &
Month ID Revenue Log Spend
l e g n e High e e Low Social > 25
Post Activity Spend Count Medium
s
1 30 High $130,000 12 $1,300,050 1 0 0 1 11.7

2 15 Low $600,000 5 $11,233,310 0 0 1 0 13.3

3 8 Medium $15,000 10 $1,112,050 0 1 0 0 9.6

4 22 Medium $705,000 11 $1,582,077 0 1 0 0 13.5

5 41 High $3,000 3 $1,889,053 1 0 0 0 8.0

DATA SPLITTING

Socia Competitiv Marketin Promotio

Month ID Revenue
l e g n
Post Activity Spend Count
s
Test data is NOT used to optimize
1 30 High $130,000 12 $1,300,050
models

2 15 Low $600,000 5 $11,233,310 Using Training data for optimization and Test
3 11 Medium $15,000 10 $1,112,050 Training data for validation ensures that your model can
data accurately predict both known and unknown
4 22 Medium $705,000 11 $1,582,077
values, which helps to prevent overfitting
5 41 High $3,000 3 $1,889,053

6 5 High $3,500 3 $1,200,089

7 23 Low $280,000 2 $1,300,080

8 12 Low $120,000 11 $700,150 Test

data
9 8 Low $1,000,000 51 $41,011,150

10 19 Medium $43,000 10 $1,000,210

OVERFITTING
Splitting is primarily used to avoid overfitting, which is when a model predicts known (Training) data
very well but unknown (Test) data poorly
• Think of overfitting like memorizing the answers to a test instead of actually learning the material; you’ll
ace the
test, but lack the ability to generalize and apply your knowledge to unfamiliar questions

OVERFIT WELL-FIT UNDERFIT model

There are two common use cases for linear regression: prediction and root-cause
analysis

PREDICTION ROOT-CAUSE
ANALYSIS
• Used to predict or forecast a • Used to determine the causal impact
numerical dependent variable of individual model inputs
• Goal is to make accurate predictions, • Goal is to prove causality by
even if causality cannot necessarily be comparing the sensitivity of each IV
proven on the DV
*Copyright Maven Analytics, LLC
REGRESSION
MODELING
In this section we’ll introduce the basics of regression modeling, including linear
relationships, least squared error, simple and multiple regression and non-linear
models

TOPICS WE’LL GOALS FOR THIS

COVER: SECTION:
• Understand basic linear relationships and the
Linear Relationships Least Squared concept of least squared errors
Error
• Explore the difference between simple (univariate)
Univariate Linear Multiple and multiple linear regression
Regression Linear
Regression • Demonstrate how regression can be used to
model linear or non-linear relationships
Non-Linear Regression
LINEAR
RELATIONSHIPS
A linear relationship means that when you increase or decrease
Linear your X variable, your Y variable increases or decreases at a steady
Relationships rate •
If you plot a linear relationship over many X/Y values, it will look like a straight
line
Least Squared
Error • Mathematically, linear relationships can be described using the following
equation: (x,
Y-intercept X
value y)
Univariate
Linear y=𝘢+ β

βx
Regression 𝘢
Y Slope
value (rise/run)
Multiple Linear
Regression • NOTE: Not all relationships are linear (more on that
later!)
Non-Linear Common examples:
Regression • Taxi M ileage (X) and Total Fare (Y)
• Units Sold (X) and Total Revenue
(Y)
LINEAR
RELATIONSHIPS
Positive Negative
Linear Linear
Linear
Relationships

Least Squared
Error

Univariate
Linear
Non-Linear No
Regression
(logarithmic) Relationship
Multiple Linear
Regression

Non-Linear

Regression

Sometimes there’s no relationship at

all!
LINEAR
RELATIONSHIPS

Linear 50

Relationships
40
Consider a line that fits
Least Squared every single point in the plot
Error 30
This is known as a
perfectly linear
20
Univariate relationship
Linear
Regression 10

Multiple Linear
Regression 10 20 30 40 50 60 70 80

Non-Linear
In this case you can simply calculate the exact value of Y for any given
Regression value
of X (no Machine Learning needed, just simple math!)
LINEAR
RELATIONSHIPS

Linear 50

Relationships
40
In the real world, things
Least Squared aren’t quite so simple
Error 30
When you add variance,
it means that many
20
Univariate different lines could
Linear potentially fit through the
Regression 10
plot
Multiple Linear
Regression 10 20 30 40 50 60 70 80

Non-Linear
To find the equation of the line with the best possible fit, we can use
Regression a
technique known as least squared error or “least squares”
LEAST SQUARED
ERROR
Least squared error is used to mathematically determine the line
Linear
Relationships that best fits through a series of data points
• Imagine drawing a line through a scatterplot, and measuring the distance
Least Squared between your line and each point (these distances are called errors, or residuals)
Error
• Now square each of those residuals, add them all up, and adjust your line until
you’ve minimized that sum; this is how least squares works!
Univariate
Linear
Regression
Why “squared” error?
Multiple Linear
Regression • Squaring the residuals converts them into positive values, and prevents
positive and negative distances from cancelling each other out (this helps the
model optimize more efficiently, too)
Non-Linear

Regression
LEAST SQUARED
ERROR

Linear 50
X Y (actual) Y (line) Error Sq.
Relationships 10 10
Error

15 5 25
40 20 25
20 -5 25
30 20
Least Squared 25 5 25
35 30
Error 30
27.5 -2.5 6.25
40 40
30 -10 100
50 15
20
Univariate 60 40
35 20 400

Linear 40 0 0
65 30
Regression 10 42.5 12.5
70 50 156.25
80 40
Multiple Linear 45 -5 25

Regression 10 20 30 40 50 60 70 80
50 10 100

Non-Linear
STEP 1: Plot each data point on a scatterplot, and record the X and Y
Regression
values
LEAST SQUARED
ERROR
y = 10 +
Linear 50 0.5x
X Y (actual) Y (line) Error Sq.
Relationships 10 10
Error

15 5 25
40 20 25
20 -5 25
30 20
Least Squared 25 5 25
35 30
Error 30
27.5 -2.5 6.25
40 40
30 -10 100
50 15
20
Univariate 60 40
35 20 400

Linear 40 0 0
65 30
Regression 10 42.5 12.5
70 50 156.25
80 40
Multiple Linear 45 -5 25

Regression 10 20 30 40 50 60 70 80
50 10 100

Non-Linear
STEP 2: Draw a straight line through the points in the scatterplot,
Regression
and calculate the Y values derived by your linear equation
LEAST SQUARED
ERROR
y = 10 +
Linear 50 0.5x
X Y (actual) Y (line) Erro Sq.
Relationships 10 10 15
r Error

5 25
40 20 25 20
-5 25
30 20 25
Least Squared 5 25
35 30 27.5
Error 30
-2.5 6.25
40 40 30
-10 100
50 15 35
20
Univariate 60 40 40
20 400

Linear 0 0
65 30 42.5
Regression 10 12.5 156.25
70 50 45
-5 25
80 40 50
Multiple Linear 10 100
Regression 10 20 30 40 50 60 70 80

Non-Linear
STEP 3: For each value of X, calculate the error (or residual) by
Regression
comparing the actual Y value against the Y value produced by your
linear equation
LEAST SQUARED
ERROR
y = 10 +
Linear 50 0.5x
X Y (actual) Y (line) Error Sq.
Relationships 10 10 15 5
Error

25
40 20 25 20 -5
25
30 20 25 5
Least Squared 25
35 30 27.5 -2.5
Error 30
6.25
40 40 30 -10
100
50 15 35 20
20
Univariate 60 40 40 0
400

Linear 0
65 30 42.5 12.5
Regression 10 156.25
70 50 45 -5
25
80 40 50 10
Multiple Linear 100
Regression

=
10 20 30 40 50 60 70 80

SUM OF SQUARED ERROR: 862.5

Non-Linear STEP 4: Square each individual residual and add them up to determine
Regression
the
sum of squared error (SSE)
• This defines exactly how well your line “fits” the plot (or in other words, how
well the linear equation describes the relationship between X and Y)
LEAST SQUARED
ERROR
y = 12 +
Linear 50 0.4x
X Y (actual) Y (line) Error Sq. Error
Relationships 10 10 16 6 36
40 20 25 20 -5 25

30 20 24 4 16
Least Squared
35 30 26 -4 16
Error 30

40 40 28 -12 144

50 15 32 17 289
20
Univariate 60 40 36 -4 16
Linear
65 30 38 8 64
Regression 10
70 50 40 -10 100

80 40 44 4 16
Multiple Linear
Regression

=
10 20 30 40 50 60
72
70 80
SUM OF SQUARED
ERROR:
2
Non-Linear STEP 5: Plot a new line, repeat Steps 1-4, and continue the process until
Regression
you’ve found the line that minimizes the sum of squared error
• This is where Machine Learning comes in; human trial-and-error is
completely
impractical, but machines can find an optimal linear equation in seconds
UNIVARIATE LINEAR
REGRESSION
Univariate (“simple”) linear regression is used for predicting a
Linear
Relationships numerical output (DV) based on a single independent variable
• Univariate linear regression is simply an extension of least squares; you use the
Least Squared linear equation that minimizes SSE to predict an output (Y) for any given input
Error (X)

Coefficient/
Univariate parameter
Dependent variable
Linear (DV) (sensitivity of Y to X) This is just the equation
Regression
y =𝘢 + βx + Error/residual of a line, plus an error
term
Multiple Linear
Regression 𝜀
Y-intercept
(IV)
Independent variable

Non-Linear
Simple linear regression is rarely used on its own; think of it as a primer
Regression for understanding more complex topics like non-linear and multiple
regression
CASE STUDY: UNIVARIATE LINEAR
REGRESSION
THE You are the proud owner of The Cone Zone, a mobile ice cream
SITUATION cart operating on the Atlantic City boardwalk.

You’ve noticed that you tend to sell more ice cream on hot days, and want to
THE understand how temperature and sales relate. Your goal is to build a simple linear
ASSIGNMENT regression model that you can use to predict sales based on the weather
forecast.

1. Use a scatterplot to visualize a sample of daily temperatures (X) and sales

THE (Y)
OBJECTIVES 2. Do you notice a clear pattern or trend? How might you interpret this?
3. Find the line that best fits the data by minimizing SSE, then confirm
by plotting a linear trendline on the scatter plot
MULTIPLE LINEAR
REGRESSION

Linear Can we use more than one IV to predict the DV?

Relationships

Multiple linear regression is used for predicting a numerical

Least Squared output (DV) based on multiple independent variables
Error
• In its simplest form, multiple linear regression is simply univariate linear
regression with additional x variables:
Univariate
Linear
Regression
Multiple Linear DV
Regression
y =𝘢 + β1x1 +β2x2 + β3x3 + … + βnxn Error/
residua
l
Non-
Linear
+𝜀
Y-intercept
Instead of just 1 IV, we have a whole set of independent
Regression variables
(and associated coefficients/weights) to help explain our DV
MULTIPLE LINEAR
REGRESSION
To visualize how multiple regression works with 2 independent
Linear
Relationships
variables, imagine fitting a plane (vs. a line) through a 3D scatterplot:

Least Squared
Error

Univariate
Linear
Regression

Multiple Linear
Regression

Non-Linear
Multiple regression can scale well beyond 2 variables, but this is
Regression
where visual analysis breaks down (and why we need machine
learning!)
MULTIPLE LINEAR
REGRESSION
EXAMPLE You are preparing to list a new property on AirBnB, and want to
estimate
Linear
Relationships
(or predict) an appropriate price using the listing data below

Least Squared
Error

Univariate
Linear
Regression
Multiple Linear
Regression

Non-
Linear
Regression
MULTIPLE LINEAR
REGRESSION
EXAMPLE You are preparing to list a new property on AirBnB, and want to
estimate
Linear
Relationships
(or predict) an appropriate price using the listing data below
MODEL 1: Predict price (Y) based on accommodation (X1):

Y =55.71 + (16.6*X1 )
Least Squared
Error
MODEL 2: Predict price (Y) based on accommodation (X1) and number of bedrooms

(X2): Y =52.59 + (15.4X1 ) + (5.1X2 )

Univariate
Linear
Regression
Multiple Linear MODEL 3: Predict price (Y) based on accommodation (X1), number of bedrooms (X2), and
Regression room type (entire.place (X3), hotel.room (X4), private.room (X5)):

Y =43.82 + (5.7X1 ) + (5.1X2 ) + (63.7X3 ) + (65.4X4 ) + (9.8*X5 )

Non-
Linear MODEL 4: Predict price (Y) based on accommodation (X1), number of bedrooms (X2), room
Regression type
(entire.place (X3), hotel.room (X4), private.room (X5)), and district (manhattan (X6), Brooklyn
(X7)):

Y =26.1 + (6.3X1) + (6.7X2) + (60.5X3) + (54.5X4) + (10.6X5) + (28.5X6) +

MULTIPLE LINEAR
REGRESSION
EXAMPLE You are preparing to list a new property on AirBnB, and want to
estimate
Linear
Relationships
(or predict) an appropriate price using the listing data below

Least Squared
Error

Univariate
Linear
Regression
Multiple Linear
Regression

Non-
Linear
Regression

Model 1 Model 2 Model 3 Model 4

Sum of Squared Error 226,577 224,259 172,201 158,591
(SSE):
NON-LINEAR
REGRESSION

Linear What if the relationship between variables isn’t linear?

Relationships

Non-linear regression is used when variables have a non-

linear relationship, but can be transformed to create a linear
Least Squared
Error one
• This works exactly like linear regression, except you use transformed
versions of
Univariate your dependent or independent variables: Coefficient/parameter
Linear Dependent variable
Regression (DV)

y = 𝘢 + β*ln(x) Error/residual

Multiple Linear
Regression
+𝜀
Y-intercept
Log-transformed independent variable
Non-Linear (IV)

Regression
All we’re really doing is transforming the data to create linear relationships between each IV and the
DV,
then applying a standard linear regression model using those transformed values
NON-LINEAR
REGRESSION
EXAMPLE You are predicting Sales (Y) using Marketing Spend (X). As you
#1 spend
Linear more on marketing, the impact on sales eventually begins to
Relationships
diminish.

Least Squared
Error

Univariate
Linear
Regression

Multiple Linear The relationship between Sales and ...but the relationship between Sales and
Marketing Spend is non-linear the
Regression
Non-Linear (logarithmic)... log of Marketing Spend is linear!

Regression y = 𝘢 + βx + y = 𝘢 + β*ln(x)
𝜀 +𝜀
NON-LINEAR
REGRESSION
EXAMPLE You are predicting population growth (Y) over time (X) and notice
#2 an increasing rate of growth as the population size increases.
Linear
Relationships

Least Squared
Error

Univariate
Linear
Regression

The relationship between Time and ...but the relationship between Time and
Multiple Linear Population is non-linear the
Regression (exponential)... log of Population is linear!
Non-Linear
y = 𝘢 + βx + 𝜀 ln(y) = 𝘢 + βx + 𝜀
Regression
NOTE: There are multiple ways to transform variables based on the type of
relationship
(log, exponential, cubic, etc.), and multiple techniques to model them (more on that
CASE STUDY: NON-LINEAR
REGRESSION
THE You work as a Marketing Analyst for Maven Marketing, an
SITUATION international advertising agency based in London.

Your client has asked you to help set media budgets and estimate sales for
THE an upcoming campaign. Using historical ad spend and revenue, your goal is
ASSIGNMENT to build a regression model to help predict campaign performance.

THE 1. Collect historical data for daily ad spend and revenue

OBJECTIVES 2. Create a linear regression, and gauge the fit. Does it look accurate?

3. Transform the ad spend values to fit a logarithmic relationship, and re-

run the model. How does this compare to the linear version?
*Copyright Maven Analytics, LLC
MODEL
DIAGNOSTICS
In this section we’ll explore common diagnostic metrics used to evaluate
regression models and ensure that predictions are stable and accurate

TOPICS WE’LL GOALS FOR THIS

COVER: SECTION:
• Understand how to interpret regression model
Sample Model R-Squared
Output outputs

Mean Error Homoskedasticity

• Define common diagnostic metrics like
Metrics R^2, MSE/MAE/MAPE, P-Values, F-
statistics, etc.
F-Significance P-Values & T-
Statistics • Explore the difference between homoskedasticity
and heteroskedasticity in a linear regression model
Multicollinearity Variance Inflation
Factor • Understand how to identify and measure
multicollinearity using variance inflation factor
(VIF)
SAMPLE MODEL
OUTPUT
Sample Model Output

R-Squared

Mean Error

Metrics

Homoskedasticity

F-Significance

P-Values & T-
Statistics

Multicollinea

rity Variance
R-SQUARED

Sample Model R-Squared measures how well your model explains the variance in
Output the dependent variable you are predicting
R-Squared
• The higher the R-Squared, the “better” your model predicts variance in the DV
and the more confident you can be in the accuracy of your predictions
Mean Error Metrics
• Adjusted R-Squared is often used as it “penalizes” the R-squared value
Homoskedasticity based on the number of variables included in the model

F-Significance Total distance between predicted and

actual values, squared (aka squared
error)

P-Values & T- SSE Sum of Squared = ෍(𝑦 𝑖 − 𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑖𝑜𝑛𝑖)2

Statistics
R =1 – =
2 Error 𝑖

Total Sum of = ෍(𝑦 𝑖 −

Multicollinearity TSS Squares 𝑦ത)2
= 𝑖
Total distance between each y value and the
Variance mean, squared (basically variance, without
dividing by n)

Inflation
R-SQUARED
EXAMPLE
Sample Model y = 12 +
Output 50 0.4x x y prediction (y–prediction)2 y ( y– y )2

10 10 16 36 30 400

R-Squared 40 20 25 20 25 30 25
30
30 20 24 16 100

Mean Error Metrics 30 35 30 26 16 30 0

40 40 28 144 30 100
30
20 50 15 32 289 225
Homoskedasticity 60 40 36 16 30 100

10 65 30 38 64 30 0
30
F-Significance 70 50 40 100 400

80 40 44 16 30 100
10 20 30 40 50 60 70 80

=
P-Values & T-
Statistics
722 1,450
SSE TSS

Multicollinearity 72
R2 =1 – SSE =
෍𝑖 (𝑦𝑖 − 𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑖𝑜𝑛𝑖 )2
1 2
1,45
= 0.50
෍(𝑦 𝑖 − –
Variance
TSS 𝑦ത)2 0 2
Inflation = 𝑖
MEAN ERROR
METRICS
Mean error metrics measure how well your regression model
Sample Model predicts, as opposed to how well it explains variance (like R-
Output
Squared)
R-Squared • There are many variations, but the most common ones are Mean Squared
Error
Mean Error
Metrics
(MSE), Mean Absolute Error (MAE) and Mean Absolute Percentage Error
(MAPE)
Homoskedasticity
• TheseMSE
metrics provide “standards” which
MAEcan be used to compare predictive
MAPE
accuracy across multiple regression models
F-Significance
σ 𝑖 (𝑦 𝑖 − 𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑖𝑜𝑛𝑖)2 σ 𝑖 |𝑦𝑖 − 𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑖𝑜𝑛𝑖| σ𝑖
|𝑦𝑖 − 𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑖𝑜𝑛𝑖|
𝑛 𝑛 𝑦𝑖
𝑛
P-Values & T-
Statistics Average of the squared distance Average of the absolute Mean Absolute Error,
between actual & predicted distance between actual & converted to a
values predicted values percentage
Multicollinearity
Mean error metrics can be used to evaluate regression models just like performance metrics like
Variance accuracy,
precision and recall can be used to evaluate classification models

Inflation
MSE
EXAMPLE
y = 12 +
Sample Model 50 0.4x
X Y (actual) Y (line) Error Sq. Error
Output
10 10 16 6 36
40 20 25 20 -5 25
R-Squared 30 20 24 4 16

35 30 26 -4 16
Mean Error 30

Metrics 40 40 28 -12 144

50 15 32 17 289
20
Homoskedasticity 60 40 36 -4 16

65 30 38 8 64
10
F-Significance 70 50 40 -10 100

80 40 44 4 16

=
P-Values & T- 10 20 30 40 50 60
72
70 80
SUM OF SQUARED
Statistics ERROR:
2
Multicollinearity
MSE
722
Variance
σ 𝑖(𝑦𝑖 − 𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑖𝑜𝑛 𝑖 )2
𝑛
=
10
= 72.
2
Inflation
MAE
EXAMPLE
y = 12 +
Sample Model 50 0.4x
X Y (actual) Y (line) Error Abs
Sq. Error
Output Error
10 10 16 6 36
20 25 20 -5 65
40 2
R-Squared 30 20 24 4
5
14
6
35 30 26 -4 14
Mean Error 30
40 40 28 -12 6
12
Metrics 14
4
50 15 32 17 17
28
20
Homoskedasticity 60 40 36 -4
9
14
65 30 38 8 68
6
10 4
F-Significance 70 50 40 -10 10
10
0
80 40 44 4 14
6

=
P-Values & T- 10 20 30 40 50 60
7
70 80
SUM OF ABSOLUTE
Statistics ERROR:
4
Multicollinearity MAE
74
Variance
σ 𝑖 |𝑦𝑖 − 𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑖𝑜𝑛𝑖|
𝑛
=
10
= 7.
4
Inflation
MAPE
EXAMPLE
y = 12 +
Sample Model 50 0.4x X
Y Y
Error
Abs Abs
(actual) (line) Erro %
Output r Error
10 10 16 6 6 0.6
40
20 25 20 -5 5 0.2
R-Squared
30 20 24 4 4 0.2

Mean Error 30 35 30 26 -4 4 0.133

Metrics 40 40 28 -12 12 0.3

20 50 15 32 17 17 1.133
Homoskedasticity
60 40 36 -4 4 0.1

65 30 38 8 8 0.267
10
F-Significance 70 50 40 -10 10 0.2

80 40 44 4 4 0.1

=
P-Values & T- 10 20 30 40 50 60 70 80

Statistics SUM OF ABSOLUTE % ERROR: 3.233

Multicollinearity
MAPE
|𝑦𝑖 − 𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑖𝑜𝑛𝑖| 3.23
Variance
σ𝑖 𝑦𝑖 = 3
10
= 32.33
𝑛
%
Inflation
MEAN ERROR
METRICS
Sample Model
Output
When should I use each type of error
metric?
R-Squared
• Mean Squared Error (MSE) is particularly
Mean Error
useful when outliers or
Metrics Mean Absolute Error (MAE) to
is useful
• extreme values are important predictif you want to minimize the
Homoskedasticity impact of outliers on model selection

• Mean Absolute Percentage Error (MAPE) is useful when your DV

F-Significance is on a very large scale, or if you want to compare models on a
more intuitive scale
P-Values & T-
Statistics

PRO TIP: In general we recommend considering all of them, since they can
Multicollinearity be calculated instantly and each provide helpful context into model
performance
Variance

Inflation
HOMOSKEDASTICITY

Homoskedasticity is a term used to describe a model with consistent

Sample Model “scatter”, or variance in residuals, across all IV values
Output
• In order to make accurate predictions across the full range of IV values,
models
R-Squared should have normally distributed residuals (errors) with a zero mean
• Heteroskedasticity describes a model with inconsistent residual variance,
Mean Error meaning that it predicts poorly for certain IV values and will fail to
Homoskedasticity generalize Homoskedasticity Heteroskedasticity

Metrics

F-Significance

P-Values & T-
Statistics

Multicollinearity Residuals are consistent Residuals increase at higher IV values, indicating that
over the entire IV there is some variance that the IVs are unable to
range explain

Variance
Breusch-Pagan tests can report a formal calculation of Heteroskedasticity, but usually a simple visual check is
enough

Inflation
NULL
HYPOTHESIS

NULL HYPOTHESIS (H0) [null hy·poth·e·sis]

noun
1. In a statistical test, the hypothesis that there is no significant
difference between specified populations, any observed difference
being due to sampling or experimental error *

The hypothesis that your model is garbage

Our goal is to reject the null hypothesis and prove (with a high level of confidence) that our
model can produce accurate, statistically significant predictions and not just random
outputs
*Oxford
F-STATISTIC & F-
SIGNIFICANCE
Sample Model The F-Statistic and associated P-Value (aka F-Significance) help
Output us understand the predictive power of the model as a whole

R-Squared • F-Significance is technically defined as “the probability that the null

hypothesis
Mean Error
cannot be rejected”, which can be interpreted as the probability that your
model predicts poorly

Metrics • The smaller the F-Significance, the more useful your regression is for prediction
F-Significance • NOTE: It’s common practice to use a P-Value of .05 (aka 95%) as a threshold
Homoskedasticity to
determine if a model is “statistically significant”, or valid for prediction
P-Values & T-
Statistics
PRO TIP: F-Significance should be the first thing you check when you
Multicollinearity evaluate a regression model; if it’s above your threshold, you may need more
training, if it’s below your threshold, move on to coefficient-level significance
(up next!)
Variance

Inflation
T-STATISTICS & P-
VALUES
Sample Model T-Statistics and their associated P-Values help us understand
Output the predictive power of the individual model coefficients

R-Squared T-Statistics tell us the degree to

which we can
“trust” the coefficient estimates
Mean Error
• Calculated by dividing the
coefficient by its
Metrics standard error
• Primarily used as a stepping-stone to
calculate P-Values, which are easier to
Homoskedasticity interpret and more commonly used for
diagnostics
P-Values & T-
• The smaller the P-value, the more confident
F-Significance
Statistics P-Values youtell
canusbethe
thatprobability thatis the
the coefficient valid (not
coefficient
0) is meaningless, statistically
Multicollinearity speakingStatistical significance is calculated as (1 –
•
P)P-Value Statistical Significance

0.001 99.9%***
Variance 0.01 99%**
Inflation 0.05 95%*
MULTICOLLINEARITY

Sample Model Output Multicollinearity occurs when two or more independent variables
are highly correlated, leading to untrustworthy model coefficients
R-Squared
• Correlation means that one IV can be used to predict another (i.e. height
and
Mean Error weight), leading to many combinations of coefficients that predict equally
well
Metrics
• This leads to unreliable coefficient estimates, and means that your model will
fail to generalize when applied to non-training data
Homoskedasticity

F-Significance
How do I measure multicollinearity, and what can I do about it?
P-Values & T-
Multicollinearity • Variance Inflation Factor (VIF) can help you quantify the degree of
Statistics
multicollinearity, and determine which IVs to exclude from the model
Variance
Inflation
VARIANCE INFLATION FACTOR

Sample Model Output To calculate Variance Inflation Factor (VIF) you treat each
individual IV as the dependent variable, and use the R2 value to
R-Squared measure how well you can predict them using the other IVs in the
model
Mean Error

Y = 𝘢 + β1 x1 + β2 x2 + β3 x3 + … + βnxn + 𝜀
Metrics

Homoskedasticity x1 = 𝘢 + β2 x2 + β3 x3 + … + βnxn 1 VIF for X1

+𝜀
1-R2 VIF for
F-Significance X2
x2 = 𝘢 + β1 x1 + β3 x3 + … + βnxn
+𝜀
P-Values & T-
PRO TIP: As a rule of thumb, a VIF >10 indicates that multicollinearity is a
problem, and that one or more IVs is redundant and should be removed from the
Variance
Statistics model
Inflation

Multicollinearity
VARIANCE INFLATION FACTOR

Sample Model Output

EXAMPLE You are predicting the price of an AirBnB listing, and notice the
following results after calculating the VIF for each independent
variable:
R-Squared

Mean Error
Entire.place and Private.room produce high VIF values, since they essentially measure the same
thing;
Metrics if a listing isn’t a private room, there’s a high probability that it’s an entire place (and vice versa)

SOLUTION: Pick one high-VIF variable to remove (arbitrarily), re-run the

Homoskedasticity
model, and recalculate the VIF values to see if multicollinearity is gone

F-Significance

P-Values & T-
Private.room
Adios NO YES
multicollinearity! PRO TIP: Use a

Entire.place
Variance
Statistics
NO 816 15,815
frequency
Inflation YES 13,166 0

table to confirm
correlation!
Multicollinearity
RECAP: SAMPLE MODEL
OUTPUT
Formula for the regression (variables & data
set)
Sample Model
Output

R-Squared
Profile of
Residuals/Errors
Mean Error

Coefficien
Metrics t
P-Values
Y-Intercept &
Homoskedasticity IV
coefficients

F-Significance
Coefficient
Standard
P-Values & T- R2 and Errors & T-
Statistics Adjusted R2 Values

Multicollinearity F-Statistic and P-Value (aka F-

Significance)

Variance
*Copyright Maven Analytics, LLC
TIME-SERIES
FORECASTING
In this section we’ll explore common time-series forecasting techniques, which
use regression models to predict future values based on seasonality and trends

TOPICS WE’LL GOALS FOR THIS

COVER: SECTION:
• Understand when, why and how to use
Forecasting 101 Seasonality
common time-series forecasting techniques

Linear Trending Smoothing

• Learn how to identify and quantify seasonality
using auto correlation and one-hot encoding

Non-Linear Trends Intervention • Explore common models for non-linear

Analysis forecasting (like ADBUDG and Gompertz)
• Apply forecasting techniques to analyze the
impact of
key business decisions (aka “interventions”)
FORECASTING
101

Forecasting 101 Time-series forecasting is all about predicting future values of a

single, numeric dependent variable
Seasonality • Forecasting works just like any other regression model, except your data
must contain multiple observations of your DV over time (aka time-series
data)
Linear Trending • Time-series forecast models look for patterns in the observed data – like
seasonality or linear/non-linear trends – to accurately predict future
values
Smoothing

Common Examples:
Non-Linear Trends • Forecasting revenue for the next fiscal
year
• Predicting website traffic growth over time
Intervention
Analysis • Estimating sales for an new product launch
SEASONALITY

Forecasting 101 Seasonality is a repeatable, predictable pattern that a

dependent variable may follow over time
Seasonality • Seasonality often aligns with specific time periods (calendar months, fiscal
periods, days of the week, hours of the day, etc.) but that isn’t always the case

Linear Trending • We can identify seasonal patterns using an Auto Correlation Function (ACF),
then apply that seasonality to forecasts using techniques like one-hot
encoding or moving averages (more on that soon!)
Smoothing

Common
Non-Linear Trends
examples:
• Website traffic by hour of the day
• Seasonal product sales
Intervention
• Airline ticket prices around major
Analysis
holidays
AUTO CORRELATION FUNCTION

ACF essentially involves calculating the correlation between time-

Forecasting 101
series data and lagged versions of itself, then plotting those
correlations
Seasonality
• This allows you to visualize which lags are highly correlated with the original
data, and reveals the length (or period) of the seasonal trend
Linear Trending

Forecasting 101
series data and lagged versions of itself, then plotting those
correlations
• This allows you to visualize which lags are highly correlated with the original
Seasonality data, and reveals the length (or period) of the seasonal trend

Linear Trending
Strong correlation every 7th lag,
which indicates a weekly
cycle
Smoothing
100%

80%

60%
CORRELATIO

Non-Linear Trends 40%

20%

-20%
N

-40%

Intervention
-60% 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

LAG
Analysis
CASE STUDY: AUTO
CORRELATION
THE You are a Business Intelligence Analyst for Accucorp Accounting,
SITUATION a national firm specializing in tax preparation services.

The marketing team is hoping to better understand customer behavior, and

THE would like to know if online searches for “tax”-related keywords follow a
ASSIGNMENT seasonal pattern. Your goal is to analyze monthly search data and use
auto correlation to test for seasonality.

1. Collect monthly search volume for relevant tax keywords

THE
2. Create 36 monthly lags, and calculate the correlation between each
OBJECTIVES lag and the original values
3. Plot the correlations and determine the period of seasonality, if it
exists
(monthly, quarterly, annual, etc.)
SEASONALITY: ONE-HOT
ENCODING
Once you’ve identified a seasonal pattern, you can use one-hot
encoding to create independent variables which capture the effect of
Forecasting 101
those time periods (days, weeks, months, quarters, etc.)

Seasonality
• NOTE: When you use one-hot encoding, you must exclude one of the options
rather than encoding all of them (it doesn’t matter which one you exclude)

• Consider the equation A+B=5; there are an infinite combination of A & B values
Linear Trending that can solve it. One-hot encoding all options creates a similar problem for
regression
Quarter ID Revenue Q1 Q2 Q3 Q4
Smoothing
1 $1,300,050 1 0 0 0

2 $11,233,310 0 1 0 0

Non-Linear Trends 3 $1,112,050 0 0 1 0

4 $1,582,077 0 0 0 1

Intervention
Analysis PRO TIP: If your data contains multiple seasonal patterns (i.e. hour of day + day of
week), include both dimensions in the model as one-hot encoded independent
variables
CASE STUDY: ONE-HOT
ENCODING
THE You’re are a Senior Analyst for Weather Trends, a Brazilian
SITUATION weather station boasting the longest and most accurate forecasts
in the biz.

You’ve been asked to help prepare temperature forecasts for the upcoming
THE year. To do this, you’ll need to analyze ~5 years of historical data from Rio
ASSIGNMENT de Janeiro, and use regression to predict monthly average temperatures.

1. Collect historical average monthly temperatures from Rio de Janeiro

THE
2. Create independent variables for each month using one-hot
OBJECTIVES encoding
3. Build a regression model to produce a forecast for the following
year
LINEAR
TRENDING

Forecasting 101
What if the data includes both seasonality and a linear
trend?

Seasonality
Trend describes an overarching direction or movement in a time
series, not counting seasonality

Linear Trending • Trends are often linear (up/down), but can be non-linear as well (more on that
later!)
• To account for linear trending in a regression, you can include a time step IV;
Smoothing this is simply an index value that starts at 1 and increments with each time
period
• If the time step coefficient isn’t statistically significant, it means you don’t have
Non-Linear Trends a
meaningful linear trend

Intervention PRO TIP: It’s common for time-series models to include trending AND seasonality; in
Analysis this case, use a combination of one-hot encoding and time step variables to account
for both!
LINEAR
TRENDING

Forecasting 101
What if the data includes both seasonality and a linear
trend?

Seasonality
Trend describes an overarching direction or movement in a time
series, not counting seasonality
Month Revenue T-Step Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov
Linear Trending
Jan $1,300,050 1 1 0 0 0 0 0 0 0 0 0 0

Feb $1,233,310 2 0 1 0 0 0 0 0 0 0 0 0

Mar $1,112,050 3 0 0 1 0 0 0 0 0 0 0 0
Smoothing
Apr $1,582,077 4 0 0 0 1 0 0 0 0 0 0 0

May $1,776,392 5 0 0 0 0 1 0 0 0 0 0 0

Jun $2,110,201 6 0 0 0 0 0 1 0 0 0 0 0
Non-Linear Trends Jul $1,928,290 7 0 0 0 0 0 0 1 0 0 0 0

Aug $2,250,293 8 0 0 0 0 0 0 0 1 0 0 0

Sep $2,120,050 9 0 0 0 0 0 0 0 0 1 0 0

Intervention Oct $2,479,293 10 0 0 0 0 0 0 0 0 0 1 0

Analysis Nov $2,560,203 11 0 0 0 0 0 0 0 0 0 0 1

Dec $2,739,022 12 0 0 0 0 0 0 0 0 0 0 0
CASE STUDY: SEASONALITY +
TREND
THE You are a Business Intelligence Analyst for Maven Muscles, a large
SITUATION national chain of fitness centers.

The Analytics Director needs your help building a monthly revenue

THE forecast for the upcoming year. Memberships follow a clear seasonal
ASSIGNMENT pattern, and revenue has been steadily rising as the gym continues to open
new locations.

1. Collect historical monthly revenue data

THE
2. Use one-hot encoding and a time-step variable to account for
OBJECTIVES both seasonality and linear trending
3. Fit a regression model to forecast revenue for the upcoming year
SMOOTHING

Forecasting 101 What if my data is so noisy that I can’t tell if a trend

exists?

Seasonality If your data is highly volatile, smoothing techniques can be used

to reveal underlying patterns or trends
Linear Trending • Common techniques like moving averages or weighted smoothing remove
random variation and “noise” from the data to help expose seasonality or
trends
Smoothing • If the volatility is truly just noise (and something we want our model to ignore),
averaging or weighting the values around each time step can help us
“smooth” the data and produce more accurate forecasts
Non-Linear Trends

Intervention PRO TIP: Smoothing is a great way to expose patterns and trends that otherwise might
Analysis be tough to see; make this part of your data profiling process!
SMOOTHING

Forecasting 101 What if my data is so noisy that I can’t tell if a trend

exists?

Seasonality If your data is highly volatile, smoothing techniques can be used

to reveal underlying patterns or trends
Linear Trending

Smoothing

Non-Linear Trends

Intervention
Analysis
CASE STUDY:
SMOOTHING
THE You’ve just been hired as an analytics consultant for Maven Motel, a
SITUATION struggling national motel chain.

Management has been taking steps to improve guest satisfaction, and has
THE asked you to analyze daily data to determine how ratings are trending. Your
ASSIGNMENT task is to use a moving average calculation to discern if an underlying
trend is present.

1. Collect daily average guest ratings for the motel. Do you see any
THE clear patterns or trends?
OBJECTIVES 2. Calculate a moving average, and compare various windows from 1-
12 weeks
3. Determine if an underlying trend is present. How would you describe
it?
NON-LINEAR
TRENDS
Time-series data won’t always follow a seasonal pattern or linear
Forecasting 101 trend;
it may follow a non-linear trend, or have no predictable trend at all
Seasonality • There are many formulas designed to forecast common non-linear trends:

Linear Trending • Exponential

• Cubiq
• Step-Functions
Smoothing • ADBUDG
• Gompertz

a
specific change (or “intervention”) on the dependent variable
Seasonality • Simply put, intervention analysis is about predicting what would have happened
if the change or intervention never took place

Linear Trending • By fitting a model to the “pre-intervention” data (up to the date of the change),
you can compare predicted vs. actual values after that date to estimate the
impact of the intervention
Smoothing

Common examples:
Non-Linear Trends
• Measuring the impact of a new website or check-out page on conversion
rates
Intervention • Quantifying the impact of a new HR program to reduce employee churn
Analysis
INTERVENTION ANALYSIS

Forecasting 101 Intervention analysis is a technique used to estimate the impact of

a
specific change (or “intervention”) on the dependent variable
Seasonality
Intervention

Linear Trending

Smoothing

Non-Linear Trends

Intervention
Analysis
INTERVENTION ANALYSIS

Forecasting 101 STEP 1: Fit a regression model to the data, using only observations from the
pre- intervention period:
Seasonality
Intervention

Linear Trending

Smoothing

Non-Linear Trends

Intervention
Analysis
INTERVENTION ANALYSIS

Forecasting 101 STEP 1: Fit a regression model to the data, using only observations from the
pre- intervention period:
Seasonality
Intervention

Linear Trending

Smoothing

Non-Linear
Trends

Intervention
Analysis
INTERVENTION ANALYSIS

Forecasting 101 STEP 2: Compare the predicted and observed values in the post-
intervention
period, and sum the daily residuals to estimate the impact of the change:
Seasonality
Intervention

Linear Trending

Smoothing

Non-Linear
Trends

Intervention
Analysis
CASE STUDY: INTERVENTION
ANALYSIS
THE You are a Web Analyst for Alpine Supplies, an online retailer
SITUATION specializing in high-end camping and hiking gear.

The company recently rolled out a new product landing page, and the
THE
CMO has asked you to help quantify the impact on conversion rate
ASSIGNMENT (CVR) and sales. You’ll need to conduct an intervention analysis, and make
sure to capture day of week seasonality and trending in your model.

1. Collect data to track sessions and CVR before and after the change
THE
2. Fit a regression model to predict CVR using data from before the
OBJECTIVES redesign
3. Use the model to forecast ”baseline” CVR after the change, and calculate
both incremental daily sales and the total cumulative impact
PART
4:
ABOUT THIS
SERIES
This is Part 4 of a 4-Part series designed to help you build a deep, foundational understanding of
machine learning, including data QA & profiling, classification, forecasting and unsupervised
learning

PART 1 PART 2 PART 3 PART

QA & Data Profiling Classification Regression & Forecasting 4
Unsupervised Learning

*Copyright Maven Analytics, LLC

COURSE OUTLINE

Review the ML landscape and key unsupervised learning concepts,

1 Intro to Unsupervised ML including unsupervised vs. supervised, feature engineering, workflow, etc.

Learn the key building blocks of segmentation and clustering,

2 Segmentation & and review two of the most-used algorithms: K-Means and
Clustering Hierarchical Clustering

Explore common techniques for association mining and when to use each
3 Association Mining of them, including A Priori and Markov models

Learn how to detect cross-sectional and time-series outliers using

4 Outlier Detection techniques like Nearest Neighbor Distance and Time-Series
models

Understand the core foundations of dimensionality reduction and how

5 Dimensionality it can be used in the context of BI and unsupervised learning
Reduction
*Copyright Maven Analytics, LLC
*Copyright Maven Analytics, LLC
MACHINE LEARNING LANDSCAPE

MACHINE LEARNING

Supervised Learning Unsupervised Learning Advanced

Topics

K-Means Clustering
Classification Regression Reinforcement Learning
Hierarchical (Q-learning, deep RL, multi-armed-bandit, etc.)
Clustering
K-Nearest Neighbors Least Squares Markov Chains
Natural Language Processing
Naïve Bayes Linear Regression Apriori (Latent Semantic Analysis, Latent Dirichlet Analysis,
relationship extraction, semantic parsing,
Decision Trees Forecasting Cross-sectional Outlier Detection contextual word embeddings, translation,
etc.)
Logistic Regression Non-Linear Regression Time-series Outlier Detection
Computer Vision
Sentiment Analysis Intervention Analysis Dimensionality Reduction (Convolutional neural networks, style translation, etc.)

LASSO/RIDGE, state-
Support vector space, advanced Matrix factorization, principal components, factor Deep Learning
machines, gradient generalized linear analysis, UMAP, T-SNE, topological data (Feed Forward, Convolutional, RNN/LSTM,
boosting, neural methods, VAR, DFA, analysis, advanced clustering, etc. Attention, Deep RL, Autoencoder, GAN. etc.)
nets/deep learning, etc.
etc.
MACHINE LEARNING LANDSCAPE

MACHINE LEARNING

Supervised Learning Unsupervised Learning

Has a Does NOT have a

“label” “label”

Classification Regression Used to DESCRIBE or ORGANIZE the data in some non-obvious way

Unsupervised learning isn’t about making predictions,

Used to predict Used to predict it’s about finding insights & patterns hidden in the
labels for values for
CATEGORICAL NUMERICAL data
variables variables
• Unlike classification or regression, we don’t care about IVs, DVs or
data splitting – we just care about understanding our observed values
MACHINE LEARNING LANDSCAPE

MACHINE LEARNING

Supervised Learning Unsupervised Learning

K-Means Clustering UNSUPERVISED

Clustering/Segmentation EXAM PLES:specific groups or
• Are there
Hierarchical Clustering
“clusters” of customers who behave
in similar ways?
Markov Chains
Association Mining/Basket
• When customers purchase a
Analysis Apriori
particular item, which other items
tend to be purchased as well?
Cross-sectional Outlier Detection • Are there any anomalies in the data
Outlier that we wouldn’t expect to see
Detection Time-series Outlier Detection under normal conditions?
• Can we combine or reduce the number
Dimensionality Dimensionality Reduction of dimensions in our data to better
Reduction understand it?
UNSUPERVISED LEARNING WORKFLOW

Business
Objective
Remember, these steps ALWAYS come first
Preliminary Before building a model, you should have a clear understanding of the business objective (identifying clusters,
QA
detecting anomalies, etc.) and the data at hand (variable types, table structure, data quality, profiling metrics,
Data
etc.)
Profiling

Hyperparamet Model
Feature Model
er Tuning Selection
Engineering Application
Add new, calculated Apply relevant Adjust and tune model Select the model that
variables (or unsupervised ML parameters (this is yields the most useful
“features”) to the techniques, based on the typically an iterative or insightful results,
data set based on objective (you will process) based on the objective
existing fields typically test multiple at hand
models)

There often are no strict rules to determine which model is best; it’s about which one helps you best answer the question at
hand
RECAP: FEATURE ENGINEERING

Feature engineering is the process of enriching a data set by creating

additional variables (or dimensions) based on existing fields
• When preparing data for segmentation/clustering, features should be intuitive and actionable;
they should help you clearly describe each cluster and make practical business
recommendations

Original Engineered
features features

2 15 Low $600,000 5 $11,233,310 0 0 1 0 13.3

3 8 Medium $15,000 10 $1,112,050 0 1 0 0 9.6

4 22 Medium $705,000 11 $1,582,077 0 1 0 1 13.5

5 41 High $3,000 3 $1,889,053 1 0 0 0 8.0

KEY TAKEAWAYS

Unsupervised learning is about finding patterns, not making

predictions
• Unlike classification or regression, it’s not about labels, dependent/independent variables or splitting test
and training data; it’s about better understanding the observed values in a data set

For many unsupervised techniques, there’s no “right” or “wrong”

solution
• Unsupervised learning is often less about optimizing specific metrics or parameters, and more about
using Machine Learning to clearly answer the business question at hand

Always focus on the desired business

outcome
• Make sure that you can clearly interpret your model output and translate it into intuitive, practical
insights and recommendations (otherwise what’s the point?)
*Copyright Maven Analytics, LLC
CLUSTERING & SEGMENTATION

In this section we’ll introduce the fundamentals of clustering & segmentation

and compare two common unsupervised models: K-means and hierarchical
clustering

TOPICS WE’LL GOALS FOR THIS

COVER: SECTION:
Clustering Basics K-Means
• Understand how segmentation and clustering
models fundamentally work

Hierarchical Key • Explore common models including K-Means

Clustering Takeaways and hierarchical clustering

• Compare and contrast common clustering

models

Remember, there’s no “right” answer or single optimization metric when it comes to clustering and segmentation; the
best outputs are the ones which help you answer the question at hand and make practical, data-driven business
decisions
CLUSTERING BASICS

Clustering Basics Clustering allows you to find concentrations or groups of

observations which are similar to one another but distinct from other
K-Means
groups

Hierarchical

Clustering Key

Takeaways
CLUSTERING BASICS

Clustering Basics Clustering allows you to find concentrations or groups of

observations which are similar to one another but distinct from other
K-Means
groups

Hierarchical

Clustering Key

Takeaways
CLUSTERING BASICS

Clustering Basics Clustering allows you to find concentrations or groups of

Hierarchical

Clustering

Key
Takeaways

STEP 4: Continue to relocate each centroid to the mean of

its assigned observations, until the clusters no longer
change
K-M EANS

Clustering Basics How do we know what’s the “right” number of clusters

(K)?
• While there is no “right” or “wrong” number of clusters, you can use
K-Means the
within-cluster sum of squares (WSS) to help inform your decision

Hierarchical
Square these distances and sum them to calculate WSS for two clusters
Clustering
(K=2)

Key
Takeaways
K-M EANS

Clustering Basics How do we know what’s the “right” number of clusters

(K)?
K-Means • While there is no “right” or “wrong” number of clusters, you can use
the
within-cluster sum of squares (WSS) to help inform your decision
Hierarchical
Clustering
Rerun the model with additional
clusters, and plot the WSS for each
Key value of K
Takeaways
WS
S

2 3 4 5 7 8
6
NUMBER OF CLUSTERS
(K)
K-M EANS

Clustering Basics How do we know what’s the “right” number of clusters

(K)?
• While there is no “right” or “wrong” number of clusters, you can use
K-Means the
within-cluster sum of squares (WSS) to help inform your decision

Hierarchical
Clustering Look for an “elbow” or inflection point,
where adding another cluster has a
Key relatively small impact on WSS (in this
Takeaways case where K=5)
WS
S
PRO TIP: Think of this as
a guideline, not a strict
rule

2 3 4 5 7 8
6
NUMBER OF CLUSTERS
(K)
K-M EANS

Clustering Basics Can we create clusters based on more than 2

dimensions?
• Yes! K-Means and WSS plots can scale to any number of dimensions
K-Means
• At higher dimensions, profiling metrics are critical for understanding
what each cluster represents, since you can’t compare them visually
Hierarchical
Clustering

Key
Does the shape of the clusters matter?
Takeaways
• Yes, K-Means works best when the clusters are mostly circular in
shape; but other tools like Hierarchical Clustering (up next!) can
address this

Does the initial centroid location make a difference?

• It can, so a common best practice is to re-run the model K times to
ensure that you are minimizing WSS for the selected number of clusters
CASE STUDY: K-
MEANS
THE You’ve just been hired as a Data Science Intern for Pet Palz, an
SITUATIO ecommerce start-up selling custom 3-D printed models of household pets.
N

You’ve been asked to analyze customer-level data to help identify

meaningful audience segments and refine the company’s marketing
THE strategy.
ASSIGNMENT
As a first step, you’ll be exploring the relationship between how often
customers engage with promotional emails and how much they actually spend,
using a K- means model for clustering.

THE 1. Collect sample data containing email engagement and spend by

customer
OBJECTIVES
2. Use K-means to identify clusters
3. Interpret each cluster and suggest next steps based on your analysis
HIERARCHICAL CLUSTERING

Clustering Basics Hierarchical clustering is an unsupervised learning technique

that creates clusters by grouping similar data points together*
K-Means
In it’s simplest form (2 dimensions), here’s how it
works:
Hierarchical 1. Create a scatterplot, find the 2 closest points, and group them into a cluster
Clustering
2. Then find the next two closest points or clusters, and group them to a cluster
Key
Takeaways 3. Repeat the process of combining the closest pairs of points or clusters until
you eventually end up with one single cluster

This process is visualized using a tree diagram called a

dendrogram, which shows the hierarchical relationship between
clusters

*This is known as agglomerative or “bottom-up” clustering (vs. divisive or “top-down” clustering, which is much less
common)
HIERARCHICAL CLUSTERING

Clustering Basics

K-Means

p5
p6
Hierarchical

DISTANCE
p2
Clustering
p3

Key
Takeaways p1
p4

p1 p2 p3 p4 p5 p6

STEP 1: Find the two closest points, and group them into a
cluster
HIERARCHICAL CLUSTERING

Clustering
Basics

K-Means
p5
p6
Hierarchical

DISTANCE
p2
Clustering
p3

Key
Takeaways p1
p4

5
clusters

p1 p2 p3 p4 p5 p6

STEP 1: Find the two closest points, and group them into a
cluster
HIERARCHICAL CLUSTERING

Clustering Basics

K-Means

p5
p6
Hierarchical

DISTANCE
p2
Clustering
p3

Key
Takeaways p1
p4
4
clusters

p1 p2 p3 p4 p5 p6

STEP 2: Find the next two closest points/clusters, and group them
together
HIERARCHICAL CLUSTERING

Clustering Basics

K-Means

p5
p6
Hierarchical

DISTANCE
p2
Clustering
p3 3
clusters
Key
Takeaways p1
p4

p1 p2 p3 p4 p5 p6

STEP 3: Repeat the process until all points are part of the same
cluster
HIERARCHICAL CLUSTERING

Clustering Basics

K-Means
2
clusters
p5
p6
Hierarchical

DISTANCE
p2
Clustering
p3

Key
Takeaways p1
p4

p1 p2 p3 p4 p5 p6

STEP 3: Repeat the process until all points are part of the same
cluster
HIERARCHICAL CLUSTERING

Clustering Basics

K-Means

p5
p6
Hierarchical

DISTANCE
p2
Clustering
p3

Key
Takeaways p1
p4

p1 p2 p3 p4 p5 p6

STEP 3: Repeat the process until all points are part of the same
cluster
HIERARCHICAL CLUSTERING

Clustering Basics How do you know when to stop creating new

clusters?
K-Means
The height of
each branch tells us Vertical branches that
Hierarchical lead to splits are called
how close the
Clustering clades
clusters/data points
are to each other

DISTANCE
Key
(taller = longer
Takeaways
distance)

Clades help us understand

similarity; two leaves in
the same clade are more
At the end of each similar to each other than
clade is a leaf, they are to the leaves in
representing a
p1 p2 p3 p4 p5 p6 another clade
single data point
HIERARCHICAL CLUSTERING

Clustering Basics How exactly do you define the distance between clusters?
• There are a number of valid ways to measure the distance between
K-Means clusters, which are often referred to as “linkage methods”
• Common methods include measuring the closest min/max distance between
Hierarchical clusters, the lowest average distance, or the distance between cluster
Clustering centroids

Key When might you use hierarchical clustering over K-Means?

Takeaways
• Run both models and compare outputs; Hierarchical clustering may
produce more meaningful results if clusters are not circular or uniform in
shape

K-Means Hierarchical
KEY TAKEAWAYS

Cluster analysis is about finding concentrations of observations which

are
similar to one another and distinct from other groups
• Common use cases include identifying key customer segments, clustering stores based on performance,
etc.

Two common techniques are K-means and Hierarchical

clustering
• In practice, we recommend testing multiple models since each technique has strengths and
weaknesses

Models won’t tell you how to interpret what each cluster

represents
• Use multivariate analysis and profiling techniques to understand the characteristics of each
cluster
*Copyright Maven Analytics, LLC
ASSOCIATION MINING

In this section we’ll introduce the fundamentals of association mining and

basket analysis, and compare common techniques including Apriori and
Markov Chains

TOPICS WE’LL GOALS FOR THIS

COVER: SECTION:
• Understand the basics of association mining
Association Mining Apriori
and introduce several real-world use cases
Basics

Markov Key
• Explore common association mining
Chains Takeaways techniques like Apriori and Markov Chains

• Compare and contrast common unsupervised

models for basket analysis
ASSOCIATION MINING BASICS

Association Mining Basics

Association Mining is used to reveal underlying patterns, relationships
and correlations in the data, and translate them into IF/THEN
association rules (i.e. “If a customer buys product A, then they will likely
Apriori
buy product B”)
Markov Chains • Association Mining is often applied to transactional data, to analyze
which products tend to be purchased together (known as “basket
analysis”)
Key Takeaways Common use cases:
• Building product recommendation systems (i.e. Amazon,
Netflix)
• Sending targeted product offers based on purchase history
• Optimizing physical store layouts to maximize cross-selling

Association mining is NOT about trying to prove or establish causation; it’s just
about identifying frequently occurring patterns and correlations in large datasets
APRIOR
I
The Apriori algorithm is commonly used to analyze how
Association Mining Basics
frequently items are purchased together in a single transaction
(i.e. beer and diapers, peanut butter and jelly, etc.)
Apriori
• In its simplest form (2-item sets), Apriori models compare the frequency
of transactions containing item A, item B, and both items A and B
Markov Chains
• This allows you to understand how often these items tend to be
purchased together, and calculate the strength of the association between
Key Takeaways them
Apriori models typically include three key metrics:
• Support (frequency of transactions containing a given item or set of items) / total
transactions

• Confidence (conditional probability of item B occurring, given item A)

• Lift (measures the importance or “strength” of the association between items)

Support (A,B)

Support (A) x Support (B)

APRIOR
I
TRANS ITEM ITEM
. 1 2
1 EXAMPLE Calculating the association between bacon &
Association Mining Basics #1 eggs
2

3
1 Support ) = 10/20 =
4 (
Apriori ) 0.5
5 Support
( ) = 7/20 =
6

Markov Chains 7 Support ( 0.35

, ) = 6/20 = 0.3
8

9 Support ( , 0.
Key Takeaways 10 2 Confidence ) ) = 3 = 60%
) ( = Support ( 0.
11 ) 5
12

13 Support ( , 0.
14 3 Lift ) ) = 3 = 1.7
) ( = Support ( ) x Support ( 0.5 x
15
) 0.35
16

18
Since Lift > 1, we can interpret the association between bacon and
19 eggs as real and informative (eggs are likely to be purchased with
20 bacon)

*Inspired by KDNuggets
APRIOR
I
TRANS ITEM ITEM
. 1 2
1 EXAMPLE Calculating the association between bacon &
Association Mining Basics #2 basil
2

3
1 Support ) = 10/20 =
4 (
Apriori ) 0.5
5 Support
( ) = 5/20 =
6

Markov Chains 7 Support ( 0.25

, ) = 1/20 = 0.05
8

9 Support ( , 0.0
Key Takeaways 10 2 Confidence ) ) = 5 = 10%
) ( = Support ( 0.
11 ) 5
12

13 Support ( , 0.0
14 3 Lift ) ) = 5 =
) ( = 0.5 x
15 Support ( ) x Support ( 0.4
) 0.25
16

18
Since Lift < 1, we can conclude that there is no positive
19 association between bacon and basil (basil is unlikely to be
20 purchased with bacon)

*Inspired by KDNuggets
APRIOR
I
TRANS ITEM ITEM
. 1 2
1 EXAMPLE Calculating the association between bacon &
Association Mining Basics #3 water
2

3
1 Support ) = 10/20 =
4 (
Apriori ) 0.5
5 Support
( ) = 1/20 =
6

Markov Chains 7 Support ( 0.05

, ) = 1/20 = 0.05
8

9 Support ( , 0.0
Key Takeaways 10 2 Confidence ) ) = 5 = 10%
) ( = Support ( 0.
11 ) 5
12

13 Support ( , 0.0
14 3 Lift ) ) = 5 = 2
) ( = Support ( ) x Support ( 0.5 x
15
) 0.05
16

18
Since Lift = 2 we might assume a strong association between bacon
19 and water, but this is skewed since water only appears in one
20 transaction

*Inspired by KDNuggets
APRIOR
I
TRANS ITEM ITEM
.
1
1 2
How do we account for infrequently purchased
Association Mining Basics 2 items?
3

4
To filter low-volume purchases, you can plot support for each
Apriori item and determine a threshold or cutoff value:
5

Markov Chains 7

SUPPORT
9
Key Takeaways 10

11 Drop-off where support =

0.15
12

16
In this case we might filter out any transactions containing items
17
with support <0.15 (transactions 4, 6, 8, 10, 15, 17)
18

19 • This helps us avoid misleading confidence and lift

20
calculations, and reduces the number of transactions we
need to analyze
*Inspired by KDNuggets
APRIOR
I
Association Mining Wouldn’t you want to analyze all possible item combinations,
Basics instead of calculating individual associations?
Apriori
• Yes! In reality that’s how Apriori works. But since the number of
configurations increases exponentially with each item, this is impractical for
Markov Chains humans to do
• To reduce or “prune” the number of itemsets to analyze, we can use the
apriori principle, which basically states that if an item is infrequent, any
Key
Takeaways
combinations containing that item must also be infrequent
1-item 2-item 3-item 4-item
Consider a shop that sells 4 sets: sets: sets: sets:
items
(chocolate, cheese, bananas &
There are 15 possible
bread)
combinations
or itemsets (ignoring duplicates)

*Inspired by KDNuggets
APRIOR
I
Association Mining Basics Wouldn’t you want to analyze all possible item combinations,
instead of calculating individual associations?
Apriori
• Yes! In reality that’s how Apriori works. But since the number of
configurations increases exponentially with each item, this is impractical for
Markov Chains humans to do
• To reduce or “prune” the number of itemsets to analyze, we can use the
apriori principle, which basically states that if an item is infrequent, any
Key Takeaways
combinations containing that item must also be infrequent

STEP 1: 1-item
sets:
2-item
sets:
3-item
sets:
4-item
sets:
Calculate support for each 1-item
set, and filter out all
transactions containing items
below the threshold (in this
case cheese)

STEP 2: 1-item
sets:
2-item
sets:
3-item
sets:
4-item
sets:
Based on the remaining itemsets,
calculate support for each 2-item
set, and filter out transactions
containing any pairs below the
threshold (in this case chocolate &
bread)

STEP 3: 1-item
sets:
2-item
sets:
3-item
sets:
4-item
sets:
Repeat until all infrequent
itemsets have been eliminated,
and filter transactions accordingly

STEP 4: 1-item
sets:
2-item
sets:
3-item
sets:
4-item
sets:
Based on the filtered
transactions, you can use an
apriori model to calculate
confidence and lift and identify
the strongest associations

STEP 4:
Based on the filtered
transactions, you can use an
apriori model to calculate
confidence and lift and identify
the strongest associations

Associations can be visualized

using
network graphs like this one

*Inspired by KDNuggets
APRIOR
I
Can you calculate associations between multiple items, like coffee
Association Mining Basics
being purchased with bacon and eggs?
• Yes, you can calculate support, confidence and lift using the same exact logic as
you would with individual items
Apriori

ID ITEM ITEM ITEM

1 2 3 Support ( , ) = 3/10 = 0.3
Markov Chains 1
Support
) = 4/10 = 0.4
2 (
3
Key Takeaways Support ( , , ) = 2/10 = 0.2
4

5 Support ( , , )
Confidence ( & ) 0.2 = = 67%
6
= Support , )
7 ( 0.3
8

9 Support , , ) 0.
Lift ( & ) ( = 2 =
10
= ) 0.1
Support ( , ) x Support ( 1.7
2

Calculating all possible associations would be impossible for a human; this is where Machine Learning comes
in!

*Inspired by KDNuggets
CASE STUDY:
APRIORI
THE You are the proud owner of Coastal Roasters, a local coffee shop
SITUATIO and bakery based in the Pacific Northwest.
N

You’d like to better understand which items customers tend to purchase

THE together, to help inform product placement and promotional strategies.
ASSIGNMENT As a first step, you’ve decided to analyze a sample of ~10,000 transactions over
a 6-month period, and conduct a simple basket analysis using an apriori model.

THE 1. Collect transaction-level data, including date, time, and items purchased

OBJECTIVES 2. Determine an appropriate support threshold, based on product frequency

3. Calculate confidence and lift to measure association between popular
items
MARKOV CHAINS

Markov Chains are used to describe the flow or transition

Association Mining Basics between “states” of a categorical variable
• Markov Chains calculate the probability of moving between states
Apriori (subscribers renewing or churning, customers purchasing one product after
another, etc.)
Markov Chains
• Observations must be tracked over time and tied to a unique ID, so that all
state transitions can be recorded as probabilities in a matrix (where all rows =
EXAMPLE:100%)
Monthly subscribers transitioning between Gold, Silver and Churn states each
Key Takeaways month:

80 40
TO % %
20
STATE %
GOLD SILVE
Gold Silver Churn 15% R
40
Gold 0.8 0.15 0.05 1 %
5 %
30
%
%
Silver 0.2 0.4 0.4
FROM
STATE

CHURN

Churn 0.01 0.3 0.69

69
%
MARKOV CHAINS
80 40
TO % %
STATE 20
%
Gold Silver Churn
Association Mining Basics GOLD SILVE
15 R
%
Gold 0.8 0.15 0.05 40
%
1%
Apriori Silver
5
30
0.2 0.4 0.4 %

FROM
%

STATE
CHURN
Churn 0.01 0.3 0.69
Markov Chains

69
Key Takeaways Example insights & %

recommendations:
• Most customers who churn stayed churned, but 31% do come back; of those who
return, nearly all of them re-subscribe to a Silver plan (vs. Gold)
 RECOMMENDATION: Launch targeted marketing to recently churned customers, offering a discount
to resubscribe to a Silver membership plan

• Once customers upgrade to a Gold membership, the majority 80% renew each month
 RECOMMENDATION: Offer a one-time discount for Silver customers to upgrade to Gold; while you
may sacrifice some short-term revenue, it will likely be profitable in the long term

To account for prior transitions (vs. just the previous) you can use more complex “higher-order” Markov
Chains
CASE STUDY: M ARKOV
CHAINS
THE You’ve just been promoted to Senior Web Analyst at Alpine Supplies, an
SITUATIO online retailer specializing in equipment and supplies for outdoor enthusiasts.
N

The VP of Sales just shared a sample of ~15,000 customer purchase paths, and
THE would like you to analyze the data to help inform a new cross-sell sales
strategy.
ASSIGNMENT
Your goal is to explore the data and build a simple Markov model to predict
which product an existing customer is most likely to purchase next.

THE 1. Collect sample data containing customer-level purchase paths

OBJECTIVES 2. Calculate a frequency and probability matrix for popular

products
3. For any given product, predict the most likely future purchase
KEY TAKEAWAYS

Association mining reveals patterns, relationships & correlations in

the data, and translates them into IF/THEN association rules
• Common use cases include building recommendation systems (i.e. Amazon, Netflix), targeting
product offers based on purchase history, optimizing physical store layouts to maximize cross-
selling, etc.

Apriori models measure how frequently specific items tend to

be purchased together (known as “basket analysis”)
• Rather than analzying every possible combination of itemsets, the apriori principle can be used to speed
up calculations by filtering out infrequent transactions

Markov Chains can be used to measure the probability of

transitioning between “states” of a categorical variable
• Markov models analyze sequences of events over time to determine what is most likely to happen
next
*Copyright Maven Analytics, LLC
OUTLIER DETECTION

In this section we’ll introduce the concept of statistical outliers, and review
common methods for detecting both cross-sectional and time-series outliers and
anomalies

TOPICS WE’LL GOALS FOR THIS SECTION:

COVER:
• Understand different types of outliers, and how
Outlier Detection Cross-sectional Outliers
they can impact an analysis
Basics
• Learn how to identify outliers in a cross-
Time-Series Key
sectional dataset using a “nearest neighbors”
Outliers Takeaways
approach
• Learn how to use regression analysis to
detect outliers in a time-series dataset

The terms outlier detection, anomaly detection, and rare-event detection are often used interchangeably; they all focus
on finding observations which are materially different than the others (outliers are also sometimes called “pathological”
data)
OUTLIER DETECTION
BASICS
Outlier Detection
Outlier detection is used to identify observations in a dataset which
Basics are either unexpected or statistically different from the others
Cross-Sectional Outliers
There are two general types of outliers you may encounter:

Time-Series • Data/recording issues: Values which were captured or categorized

incorrectly, and which must be removed or excluded to avoid skewing the
analysis
Outliers Key
• Business outliers: Legitimate values which provide real and meaningful
information, such as a fraudulent transaction or an unexpected spike in
Takeaways sales

Common use cases:

• Identifying store locations which are behaving abnormally compared to
others
• Detecting web traffic anomalies which would be impossible to detect
otherwise
CROSS-SECTIONAL
OUTLIERS
Outlier Detection Cross-sectional outlier detection is used to measure the similarity
Basics between observations in a dataset, and identify observations which
Cross-Sectional Outliers
are unusually different or dissimilar
• Detecting outliers in one or two dimensions is often trivial, using basic
Time-Series profiling metrics (i.e. interquartile range) or visual analysis (scatterplots, box
Outliers plots, etc.)

Key • Detecting outliers 3+ dimensions is tricker, and often requires more sophisticated
Takeaways techniques; this is where machine learning comes in!

A common technique is to calculate the distance

You’d like to analyze store-level sales by product category, to identify patterns

THE and
ASSIGNMENT see if any locations show an unusual composition of revenue.
To do this, you’ll be exploring revenue data across 100 individual store
locations, and detecting outliers using “nearest neighbor” calculations.

THE 1. Collect category-level sales data by store location

OBJECTIVES 2. Use scatterplots to visualize relationships and scan for outliers

3. Calculate the nearest neighbor for each observation and visualize
the data using a box plot and histogram. Do you see any outliers
now?
TIME-SERIES
OUTLIERS
Outlier Detection Time-series outlier detection is used to identify observations which
Basics fall well outside expectations for a particular point in time, after
Cross-Sectional Outliers
accounting for seasonality and trending
• In practice, this involves building a time-series regression, comparing each
Time-Series
observation to its predicted value, and plotting the residuals to detect
Outliers anomalies

Fit regression Calculate Plot

Key
model residuals residuals
Takeaways
TIME-SERIES
OUTLIERS
Real-world time-series data is often extremely granular, with recurring
Outlier Detection
Basics patterns and trends which make visual outlier detection virtually
impossible
Cross-Sectional Outliers
EXAMPL Hourly website sessions
E (n=8,760)
Time-Series Outliers

Key Takeaways
TIME-SERIES
OUTLIERS
EXAMPL Hourly website sessions
Outlier Detection (n=8,760)
Basics
E

Cross-Sectional Outliers

Time-Series

Outliers Key

Takeaways

Just like our cross-sectional example, univariate

distributions don’t reveal any outliers or
anomalies
TIME-SERIES
OUTLIERS
Outlier Detection Instead of simple visual analysis, we can fit a linear regression model
Basics to account for seasonality (hour of day) and trending
Cross-Sectional Outliers • From there, we can plot the distribution of residuals in order to quickly
identify any observations which significantly deviated from the model’s
prediction
Time-Series Outliers

Key Takeaways
TIME-SERIES
OUTLIERS
Outlier Detection
Basics

Cross-Sectional Outliers When we plot the distribution of

model residuals, we detect a
Time-Series clear outlier in the data
Outliers

Key
Takeaways PRO TIP: Not all outliers
are bad! If you find an
anomaly, understand what
happened and how you can
learn from it
KEY TAKEAWAYS

Outlier detection is used to identify observations in a dataset which

are either unexpected or statistically different from the others
• Common use cases include detecting web traffic anomalies, flagging fraudulent credit card
transactions, identifying unusually high or low-performing store locations, etc.

For cross-sectional analysis, plotting “nearest neighbor” distances can

help detect anomalies when visual profiling falls short
• This approach is ideal for exposing outliers in large, complex or highly-dimensional
datasets

For time-series analysis, you can fit a regression model to the data and
plot the residuals to clearly identify outliers
• This allows you to quickly find outliers, while controlling for seasonality and
trending
*Copyright Maven Analytics, LLC
DIMENSIONALITY REDUCTION

In this section we’ll review the fundamentals of dimensionality reduction, discuss

common Machine Learning and business use cases, and introduce core techniques
like Principal Component Analysis (PCA)

TOPICS WE’LL GOALS FOR THIS SECTION:

COVER:
Dimensionality Reduction Principal Component
• Understand how dimensionality reduction can be
Basics Analysis applied in business and Machine Learning
contexts
• Explore the intuition behind core linear
Advanced Techniques Key dimensionality reduction techniques, including
Takeaways Principal Component Analysis (PCA)

• Introduce more advanced models, including

techniques for non-linear dimensionality reduction
DIMENSIONALITY REDUCTION

Dimensionality Reduction Dimensionality reduction involves reducing the number of columns

Basics (i.e. dimensions) in a dataset while losing as little information as
possible
Principal Component
Analysis • This can be used strictly as a ML optimization technique, or to help develop a
better understanding of the data for business intelligence, market research,
etc.
Advanced Techniques COM M ON BUSINESS USE COMMON ML USE CASES:
CASES:
 Summarizing item-level sales data to  Improving a model’s predictive
better understand group or category power by reducing redundant or
Key Takeaways performance unnecessary dimensions in the data
 Analyzing survey data to understand
overarching trends and findings without  Visualizing clusters in limited dimensions
parsing through individual question to spot-check cluster/segmentation
responses outputs
 Understanding the main traits or  Reducing trivial correlations and
characteristics of your customers (website removing multicollinearity to build
activity, transaction patterns, survey more accurate and meaningful models
response data, etc.)
PRINCIPAL COMPONENT ANALYSIS

Dimensionality Reduction One of the most common dimensionality reduction techniques is

Basics Principal Component Analysis (PCA)

Principal Component
• In the simplest form, PCA finds lines that best fit through the observations in
Analysis a data set, and uses those lines to create new dimensions to analyze

Advanced Techniques

STEP 1 STEP STEP STEP STEP

2 3 4 5
Key Takeaways
Use high- Find “components” Interpret the Select the Use those principal
dimensional data or lines that fit weighting of each “principal” components in
that could be more through those component on the components which place of existing
useful if it could dimensions, to original dimensions explain a material dimensions for
be accurately potentially use as to build an proportion of the further analysis
represented with new dimensions to intuition for what variance in the
fewer dimensions represent the data they represent data
PRINCIPAL COMPONENT ANALYSIS

EXAMPL Student test scores by subject

Dimensionality
Reduction E (n=10)
Basics Plotting pairs of dimensions reveals some
correlation between
Principal Component Spelling/Vocabulary and
Analysis
Multiplication/Geometry test scores
• Linear dimensionality reduction
techniques will find lines that fit
Advanced through highly correlated dimensions
Techniques first

Spelling (X) / Vocabulary (Y) Vocabulary (X) / Multiplication Spelling (X) / Multiplication Multiplication (X) / Geometry
Key Takeaways (Y) (Y) (Y)
PRINCIPAL COMPONENT ANALYSIS

EXAMPL Student test scores by subject

Dimensionality
Reduction E (n=10)
Basics In the context of dimensionality reduction,
these lines are called components or
Principal Component factors
Analysis Observations are given a weighting for
each component, which essentially
measures its position relative to the length
Advanced of the line
Techniques

Key Takeaways
e
on
omp
c
Vocabulary

Geometry
tin
e igh
w
g

Spelling Multiplication
PRINCIPAL COMPONENT ANALYSIS

EXAMPL Student test scores by subject

Dimensionality
Reduction E (n=10)
Basics 2.8 7.8
Plot the component weightings
4.3 8.7 for each point, and record
Principal Component 7.3 3.2 them as new dimensions in
10.7 4.9
Analysis 11.4 2.6 the table
10.0 11.5
6.1 11.6
NOTE: We’re looking at only 2
9.6 11.3 dimensions here, but PCA
Advanced
6.6 13.6 scales to any number of
13.5 12.0
Techniques dimensions

Key Takeaways
Vocabulary

Geometry
10.
0 (8.3 ,
5.6)
3.2

(2.8 ,
1.6)

Spelling Multiplication
PRINCIPAL COMPONENT ANALYSIS

How do you determine the meaning of each

Dimensionality
Reduction component?
Basics • PCA will generate many potential components, and it won’t be immediately
clear what they actually represent
Principal Component
Analysis • In practice, PCA models calculate weights (or “loadings”) to help explain
the relationship between each component and the original dimensions
(think of loadings like coefficients in a linear regression)
Advanced
Techniques
X1 X2 X3 X4 Y
language
2.8 7.8
4.3 8.7 High coefficients on X 1 (spelling)
Key Takeaways 7.3 3.2 and X 2 (vocabulary), so we might
10.7 4.9 interpret component 1 as
11.4 2.6
10.0 11.5
“language”
6.1 11.6
9.6 11.3
6.6 13.6
13.5 12.0

Y (0.72)*X 1 + + (-0.02)*X 3 +
= (0.69)*X 2 (0.03)*X 4
PRINCIPAL COMPONENT ANALYSIS

How do you determine the meaning of each

Dimensionality
Reduction component?
Basics • PCA will generate many potential components, and it won’t be immediately
clear what they actually represent
Principal Component
Analysis • In practice, PCA models calculate weights (or “loadings”) to help explain
the relationship between each component and the original dimensions
(think of loadings like coefficients in a linear regression)
Advanced
Techniques
X1 X2 X3 X4 Y
language math
2.8 7.8
4.3 8.7 High coefficients on X 3 (multiplication)
Key Takeaways 7.3 3.2
and X4 (geometry), so we might
10.7 4.9
11.4 2.6 interpret component 2 as “math”
10.0 11.5
6.1 11.6
9.6 11.3
6.6 13.6 This requires human
13.5 12.0 intuition; your model won’t
tell you what the
components mean!
Y = (0.004)*X 1 + (0.68)*X 3 +
(0.001)*X 2 + (0.72)*X 4
PRINCIPAL COMPONENT ANALYSIS

Dimensionality How do you know how many components to

Reduction
Basics keep?
• You can use a Scree Plot to visualize the cumulative variance explained by each
Principal Component component, and look for the “elbow” where additional components add
Analysis relatively little value (similar to a WSS chart for clustering)

• The components up to and including the “elbow” become your

Advanced principal
Techniques components, and others can be disregarded

Key Takeaways In this plot, the first 2 components

explain most of the variance in the
data, and additional components have
minimal impact
PRINCIPAL COMPONENT ANALYSIS

Dimensionality
Reduction
Basics

Principal Component
Analysis

Advanced
Techniques

Dimensionality
Reduction
Basics

Principal Component

Math
Analysis

Advanced
Techniques

Language

Key Takeaways In this example, defining components for language and math
helped us simplify and better understand the data set
• Using the new components we derived, we can conduct further
analysis like predictive modeling, classification, clustering, etc.
• For example, clustering might help us understand student testing
patterns
(most skew towards either language or math, while a few excel in both
subjects)
ADVANCED TECHNIQUES

Dimensionality Reduction What about non-linear dimensionality

Basics
reduction?
• Linear dimensionality reduction is the most well-researched and widely used
Principal Component method, and can be implemented across a broad range of BI and ML use
Analysis cases

• Other, non-linear dimensionality reduction techniques do exist, but they are

Advanced Techniques significantly more specialized and complex in terms of the underlying
algorithms and math:
• Kernel PCA
• T-Distributed Stochastic Neighbors (T-SNE)
Key Takeaways
• Uniform Manifold Approximation & Projection (UMAP)
• Self-Organizing Maps
KEY TAKEAWAYS

Dimensionality reduction is used to reduce the number of columns in

a data set while losing as little information as possible
• This can be used for model optimization, or to identify overarching trends or patterns in a granular
dataset

One of the most common dimensionality reduction techniques is known

as
Principal
• PCA Component
essentially fits lines Analysis (PCA) in a data set to create new dimensions to
through the observations
analyze

Dimensionality reduction requires human logic and intuition to

interpret what each component actually means
• This will help you simplify your data, understand it on a deeper level, and conduct further
analysis
*Copyright Maven Analytics, LLC
WRAPPING
UP
CONGRATULATIONS!
You now have a solid foundational understanding of unsupervised learning, including
techniques like clustering, association mining, outlier detection, and dimensionality
reduction.

We hope you’ve enjoyed the entire Machine Learning for BI series, and that you find
an opportunity to put your skills to good use!

PART 1 PART 2 PART 3 PART 4

QA & Data Profiling Classification Regression & Unsupervised
Forecasting Learning

Statistics For Absolute Beginners (Second Edition) (Oliver Theobald
0% (1)
Statistics For Absolute Beginners (Second Edition) (Oliver Theobald
144 pages
Introduction To Data Mining and Analytics
No ratings yet
Introduction To Data Mining and Analytics
1,668 pages
Data Preparation Notebook
No ratings yet
Data Preparation Notebook
14 pages
Basic Economic Analytics Using Excel!
No ratings yet
Basic Economic Analytics Using Excel!
72 pages
CS109a Lecture1
No ratings yet
CS109a Lecture1
67 pages
ML Unit 1 Part 2
No ratings yet
ML Unit 1 Part 2
56 pages
Machine Learning Summer Training
No ratings yet
Machine Learning Summer Training
118 pages
Module 1_BCS602_chapter 02.pptx
No ratings yet
Module 1_BCS602_chapter 02.pptx
90 pages
Data preprocessing (1)
No ratings yet
Data preprocessing (1)
77 pages
DS_w3-4
No ratings yet
DS_w3-4
69 pages
3-Preparing The Data-10-01-2024
No ratings yet
3-Preparing The Data-10-01-2024
127 pages
UNIT02
No ratings yet
UNIT02
41 pages
Working With Data - Annotated
No ratings yet
Working With Data - Annotated
62 pages
Chapter 1 Data and Decisions
No ratings yet
Chapter 1 Data and Decisions
3 pages
Consolidate AmitRana (1)
No ratings yet
Consolidate AmitRana (1)
58 pages
FDS Unit 1 Notes
No ratings yet
FDS Unit 1 Notes
53 pages
Cognizant Data Analyst Interview Questions 1745235888
No ratings yet
Cognizant Data Analyst Interview Questions 1745235888
18 pages
Working With Data - Annotated
No ratings yet
Working With Data - Annotated
62 pages
DMML
No ratings yet
DMML
65 pages
Data Analytics Program - Introduction To Data Analytics - Lesson 1
No ratings yet
Data Analytics Program - Introduction To Data Analytics - Lesson 1
56 pages
Unit 3 Data Exploration (P)
No ratings yet
Unit 3 Data Exploration (P)
69 pages
Data Preprocessing
100% (1)
Data Preprocessing
109 pages
Unit 1 Big Data Analytics - An Introduction (Final)
No ratings yet
Unit 1 Big Data Analytics - An Introduction (Final)
65 pages
Data Visualization and Story Telling Notes
No ratings yet
Data Visualization and Story Telling Notes
31 pages
Data Cleaning Wrangling
No ratings yet
Data Cleaning Wrangling
42 pages
ML U2
No ratings yet
ML U2
62 pages
BA_CH01
No ratings yet
BA_CH01
14 pages
1st Unit Notes
No ratings yet
1st Unit Notes
22 pages
Data Analytics Course Session 1-5
100% (1)
Data Analytics Course Session 1-5
252 pages
Data Preprocessing
No ratings yet
Data Preprocessing
56 pages
Agenda: 1) Assign Homework #1 (Due Wednesday 6/30) 2) Lecture Over More of Chapter 2
No ratings yet
Agenda: 1) Assign Homework #1 (Due Wednesday 6/30) 2) Lecture Over More of Chapter 2
43 pages
MODULE 2_(Ask Questions to Make Data-Driven Decisions)
No ratings yet
MODULE 2_(Ask Questions to Make Data-Driven Decisions)
23 pages
ML Unit-Ii
No ratings yet
ML Unit-Ii
100 pages
Chapter - 3 Data Pre - Processing
No ratings yet
Chapter - 3 Data Pre - Processing
54 pages
Create A Pivottable To Analyze Worksheet Data
No ratings yet
Create A Pivottable To Analyze Worksheet Data
23 pages
Data Science UNIT 1 Final
No ratings yet
Data Science UNIT 1 Final
107 pages
Data-Preprocessing
No ratings yet
Data-Preprocessing
138 pages
Bab 2 Data: Created By: Arif Djunaidy (Ftif - Its)
No ratings yet
Bab 2 Data: Created By: Arif Djunaidy (Ftif - Its)
57 pages
Statistics Refresher
No ratings yet
Statistics Refresher
11 pages
data science slides
No ratings yet
data science slides
57 pages
Module 2 BDA
No ratings yet
Module 2 BDA
40 pages
Lecture3 More of Chapter 2
No ratings yet
Lecture3 More of Chapter 2
50 pages
E-Note_33325_Content_Document_20250319114322AM
No ratings yet
E-Note_33325_Content_Document_20250319114322AM
69 pages
Types of Data and Data Quality: KIT306/606: Data Analytics Unit Coordinator: A/Prof. Quan Bai University of Tasmania
No ratings yet
Types of Data and Data Quality: KIT306/606: Data Analytics Unit Coordinator: A/Prof. Quan Bai University of Tasmania
25 pages
Visual Guide To Machine Learning
No ratings yet
Visual Guide To Machine Learning
364 pages
Unit 3 Data Preprocessing - Data
No ratings yet
Unit 3 Data Preprocessing - Data
90 pages
Chapter 1
No ratings yet
Chapter 1
3 pages
Chapter1 2023
No ratings yet
Chapter1 2023
76 pages
Indian Food EDA
No ratings yet
Indian Food EDA
20 pages
Unit2
No ratings yet
Unit2
76 pages
Types of Data
No ratings yet
Types of Data
14 pages
AIML Unit 2 Understanding Data
No ratings yet
AIML Unit 2 Understanding Data
51 pages
Data Analytics Questions
No ratings yet
Data Analytics Questions
6 pages
Module 01 - STAT 101
No ratings yet
Module 01 - STAT 101
23 pages
Measurement Scale: Dr. Myint Moe Moe Khin Professor / Head Department of Statistics Monywa University of Economics
No ratings yet
Measurement Scale: Dr. Myint Moe Moe Khin Professor / Head Department of Statistics Monywa University of Economics
27 pages
Data Science Questions
No ratings yet
Data Science Questions
16 pages
Introduction to Business Intelligence 2
No ratings yet
Introduction to Business Intelligence 2
23 pages
Unit 1
No ratings yet
Unit 1
21 pages
DATA MINING and MACHINE LEARNING. PREDICTIVE TECHNIQUES: ENSEMBLE METHODS, BOOSTING, BAGGING, RANDOM FOREST, DECISION TREES and REGRESSION TREES.: Examples with MATLAB
From Everand
DATA MINING and MACHINE LEARNING. PREDICTIVE TECHNIQUES: ENSEMBLE METHODS, BOOSTING, BAGGING, RANDOM FOREST, DECISION TREES and REGRESSION TREES.: Examples with MATLAB
César Pérez López
No ratings yet
Statistics with Rust: 50+ Statistical Techniques Put into Action
From Everand
Statistics with Rust: 50+ Statistical Techniques Put into Action
Keiko Nakamura
No ratings yet
Python Data Visualization
No ratings yet
Python Data Visualization
174 pages
1754638164
No ratings yet
1754638164
34 pages
OpenStorageLogging060811 Symconnect 4
No ratings yet
OpenStorageLogging060811 Symconnect 4
25 pages
NBU 8.0 Blueprint BMR Jan 2017
No ratings yet
NBU 8.0 Blueprint BMR Jan 2017
45 pages
NearStore VTL Datasheet - LR
No ratings yet
NearStore VTL Datasheet - LR
2 pages
NetBackup Troubleshooting Refresher Introduction
No ratings yet
NetBackup Troubleshooting Refresher Introduction
26 pages
1447770917
No ratings yet
1447770917
23 pages
Master Governance Plan
No ratings yet
Master Governance Plan
21 pages
Affiliate Marketing and Customer Satisfaction
No ratings yet
Affiliate Marketing and Customer Satisfaction
2 pages
Oracle Integrated Cloud Applications and Platform Services
No ratings yet
Oracle Integrated Cloud Applications and Platform Services
2 pages
Strategy Proposal Pindo Deli (15062022)
No ratings yet
Strategy Proposal Pindo Deli (15062022)
20 pages
Organization: Project Overview Plan Template
No ratings yet
Organization: Project Overview Plan Template
9 pages
Talend Data Fabric Demo Request - Talend Real-Time Open Source Data Integration Software
No ratings yet
Talend Data Fabric Demo Request - Talend Real-Time Open Source Data Integration Software
6 pages
Ex 2
No ratings yet
Ex 2
15 pages
C3W Color Night Vision Datasheet
No ratings yet
C3W Color Night Vision Datasheet
8 pages
Template Format Test Case - BINAR Bootcamp Quality Assurance
No ratings yet
Template Format Test Case - BINAR Bootcamp Quality Assurance
4 pages
Softstart: Spare Part List For PSS18 300 PST30 300 PSTB370 1050
No ratings yet
Softstart: Spare Part List For PSS18 300 PST30 300 PSTB370 1050
10 pages
FLOWTITE-Installation
No ratings yet
FLOWTITE-Installation
9 pages
STM32C011x4/x6: Arm Cortex - M0+ 32-Bit MCU, 32 KB Flash, 6 KB RAM, 2 X USART, Timers, ADC, Comm. I/Fs, 2-3.6 V
No ratings yet
STM32C011x4/x6: Arm Cortex - M0+ 32-Bit MCU, 32 KB Flash, 6 KB RAM, 2 X USART, Timers, ADC, Comm. I/Fs, 2-3.6 V
92 pages
MN 63120 2101 en dp2 Instruction PDF
No ratings yet
MN 63120 2101 en dp2 Instruction PDF
4 pages
GITCHIA - Transition Plan From ISO 9001:2008 To ISO 9001:2015
No ratings yet
GITCHIA - Transition Plan From ISO 9001:2008 To ISO 9001:2015
2 pages
Instructional Designer Resume
100% (2)
Instructional Designer Resume
8 pages
Hood User Manual: CFB 6310 W CFB 6310 X CFB 5310 W CFB 5310 X
No ratings yet
Hood User Manual: CFB 6310 W CFB 6310 X CFB 5310 W CFB 5310 X
64 pages
1733998129135
No ratings yet
1733998129135
19 pages
IE3300
No ratings yet
IE3300
20 pages
Spectroquant Nova 60A: Operating Manual Bedienungsanleitung Mode D'emploi Modo de Empleo Manuale D'uso
No ratings yet
Spectroquant Nova 60A: Operating Manual Bedienungsanleitung Mode D'emploi Modo de Empleo Manuale D'uso
92 pages
Manual Bomba de Vacuo
No ratings yet
Manual Bomba de Vacuo
29 pages
Brief SCMPE M23
No ratings yet
Brief SCMPE M23
69 pages
Aol Fullz
67% (3)
Aol Fullz
44 pages
Joyance International Co.,Ltd: I Steel Sheet Pile Installation
No ratings yet
Joyance International Co.,Ltd: I Steel Sheet Pile Installation
11 pages
Merlin Gerin Compact NS400 N Data Sheet
No ratings yet
Merlin Gerin Compact NS400 N Data Sheet
3 pages
GPP in Tropical Ecosystems (Monteith, 1972)
No ratings yet
GPP in Tropical Ecosystems (Monteith, 1972)
21 pages
Chapter 2-Metal Casting Processes
No ratings yet
Chapter 2-Metal Casting Processes
87 pages
Concrete Construction Article PDF - Fire Ratings of Masonry Walls PDF
No ratings yet
Concrete Construction Article PDF - Fire Ratings of Masonry Walls PDF
3 pages
Module (Code &Name:Swdba401-Backend Application Development
No ratings yet
Module (Code &Name:Swdba401-Backend Application Development
62 pages
Project Members: Underground Cable Fault Detection by Using GSM
No ratings yet
Project Members: Underground Cable Fault Detection by Using GSM
14 pages
Premade Pouch 10 Questions
No ratings yet
Premade Pouch 10 Questions
18 pages
Delhi NCR Based Companies List 1680421970
100% (1)
Delhi NCR Based Companies List 1680421970
3 pages
Applications of Assembly Language
100% (4)
Applications of Assembly Language
8 pages
Configure Basic Settings
No ratings yet
Configure Basic Settings
12 pages
User Manual and Setup of Wintek32 Tekscope Ths720 and PC User Manual Start Wintek32
No ratings yet
User Manual and Setup of Wintek32 Tekscope Ths720 and PC User Manual Start Wintek32
10 pages
Grade 10-Learning Activity Sheet Q4M3
100% (1)
Grade 10-Learning Activity Sheet Q4M3
3 pages
Car Rental Srs Document
No ratings yet
Car Rental Srs Document
116 pages