Introduction to Machine Learning

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use

or distribution prohibited
Learning from Data

• Can we learn about the world around us using data?

• Model building from data
– Take data as input
– Find patterns in the data
– Summarize the pattern in a mathematically precise way
• Machine learning automates this model building.

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use

or distribution prohibited

2
The Challenge

• Data unfortunately contains noise. If not, machine learning

would be trivial!
• Think of Data = Information + Noise
• The challenge is to identify the information content and
distill away the noise.
• To help do this, machine learning uses a train and test
approach.

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use

or distribution prohibited

3
Over fitting Vs under fitting

• If the model we finish with ends up

– modeling the noise as well, we call it “over fitting” - bad for
prediction!
– not modeling all the information, we call it “under fitting” - bad for
prediction!
• The hope is that the model that does the best on testing
data manages to capture/model all the information but leave
out all the noise.

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use

or distribution prohibited

4
Machine Learning tasks

1. Supervised learning: Building a mathematical model using

data that contains both the inputs and the desired outputs
(ground truth).
– Examples:
• Determining if an image has a horse. The data would include images with
and without the horse (the input), and for each image we would have a
label (the output) indicating if there is a horse in that image.
• Determining is a client might default on a loan
• Determining if a call center employee is likely to quit
– Since we have desired outputs, model performance can be
evaluated by comparisons.

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use

or distribution prohibited

5
Machine Learning Tasks

2. Unsupervised learning: Building a mathematical model

using data that contains only inputs and no desired outputs.
– Used to find structure in the data, like grouping or clustering of
data points. To discover patterns and group the inputs into
categories.
– Example: an advertising platform segments the population into
smaller groups with similar demographics and purchasing habits.
Helping advertisers reach their target market with relevant ads.
– Since no labels are provided, there is no specific way to compare
model performance in most unsupervised learning methods.

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use

or distribution prohibited

6
Tools and techniques

• Supervised learning
– Regression: desired output is a continuous number
– Classification: desired output is a category
• Unsupervised learning
– Clustering: Grouping data
– Dimensionality reduction: Compressing data
– Association rule learning: If X then Y

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use

or distribution prohibited

7
 Intro to Clustering

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use

or distribution prohibited
Clustering

• Clustering is an Unsupervised Learning Technique

• A Cluster: collection of objects that are similar
• Objective is to group similar data points into a group
– Segmenting customers into similar groups
– Automatically organizing similar files/emails into folders
• Simplifies data by reducing many data points into a few
clusters

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use

or distribution prohibited

9
Distance

• Do define “similarity” you need a measure of distance

• Examples of common distance measures
– Manhattan Distance
– Eucledian Distance
– Chebyshev Distance

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use

or distribution prohibited

1
0
Types of Clustering

1. Connectivity based clustering (Hierarchical clustering): based on the idea that related
objects are closer to each other. Can we then create a hierarchy of clusters/groups.

– Useful when you want flexibility in how many clusters you ultimately want. For
example, imagine grouping items on an online marketplace like Etsy or Amazon.

– In terms of outputs from the algorithm, in addition to cluster assignments you

also build a nice tree (dendrogram) that tells you about the hierarchies between
the clusters. You can then pick the number of clusters you want from this tree.

– In a dendrogram, the y-axis marks the distance at which the clusters merge,
while the objects are placed along the x-axis.

– Algorithms can be agglomerative (start with 1 object and aggregate them into
clusters) or divisive (start with complete data and divide into partitions).

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use

or distribution prohibited

1
1
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use
or distribution prohibited

1
2
Types of Clustering
2. Centroid based clustering (Eg. K- Means clustering):
The objective is to find K clusters/groups. The way
these groups are defined is by creating a centroid for
each group. The centroids are like the heart of the
cluster, they “capture” the points closest to them
and add them to the cluster.
– Large K produces smaller groups and a small K produces
larger groups
– K-Means uses Eucledian distances and is the most popular
– Other variants like K-medians and K-mediods use other
distance measures

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use

or distribution prohibited

1
3
 Clustering

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use

or distribution prohibited
Data we will work with
– Customer Spend Data
• AVG_Mthly_Spend: The average monthly amount spent by customer
• No_of_Visits: The number of times a customer visited in a month
• Item Counts: Count of Apparel, Fruits and Vegetable, Staple Items purchased

• Can we cluster similar customers together?

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use 1
5
or distribution prohibited
Connectivity Based: Hierarchical Clustering

• Hierarchical Clustering techniques create clusters in a

hierarchical tree like structure
• Any type of distance measure can be used as a
measure of similarity
• Cluster tree like output is called Dendogram
• Techniques either start with individual objects and
sequentially combine them (Agglomerative ), or start
from one cluster of all objects and sequentially divide
them (Divisive)

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use

or distribution prohibited

1
6
Agglomerative
• Starts with each object as a cluster of one record each
• Sequentially merges 2 closest records by distance as a
measure of similarity to form a cluster.
• How would we measure distance between two
clusters?

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use

or distribution prohibited

1
7
Distance between clusters
• Single linkage – Minimum
distance or Nearest neighbor
• Complete linkage –
Maximum distance or
Farthest distance
• Average linkage – Average
of the distances between all
pairs
• Centroid method – combine
cluster with minimum
distance between the
centroids of the two clusters
• Ward’s method – Combine
clusters with which the
increase in within cluster
variance is to the smallest
degree
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use 1
8
or distribution prohibited
Distance between objects

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use 1

9
or distribution prohibited
Centroid based: K-Means Clustering

• K-Means is probably the most used clustering technique

• Aims to partition the n observations into k clusters so as to

minimize the within-cluster sum of squares (i.e. variance).

• Computationally less expensive compared to hierarchical

techniques.

• Have to pre-define K, the no of clusters

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use

or distribution prohibited

2
0
Lloyd’s algorithm

1. Assume K Centroids

2. Compute Squared Eucledian distance of each objects with

these K centroids. Assign each to the closest centroid forming
clusters.

3. Compute the new centroid (mean) of each cluster based on

the objects assigned to each clusters.

4. Repeat 2 and 3 till convergence: usually defined as the point

at which there is no movement of objects between clusters

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use

or distribution prohibited

2
1
Choosing the optimal K

• Usually subjective, based on striking a good balance between

compression and accuracy

• The “elbow” method is commonly used

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use

or distribution prohibited
2
2
Lloyd’s algorithm

1. Assume K Centroids

2. Compute Squared Eucledian distance of each objects with

these K centroids. Assign each to the closest centroid forming
clusters.

3. Compute the new centroid (mean) of each cluster based on

the objects assigned to each clusters.

4. Repeat 2 and 3 till convergence: usually defined as the point

at which there is no movement of objects between clusters

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use

or distribution prohibited

23
Market Basket Analysis (or) Association Rules

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use

or distribution prohibited
Market Baskets
– Transactions/Baskets

• Is it true that {Breakfast Cereals}->{Bread}

• How sure are you?

• Other patterns like, If {A,B,…} then {C,…}?

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use 2
5
or distribution prohibited
Association Rules Learning
• Rules-bases unsupervised learning:
– If X then Y. Written as X -> Y.
– X and Y can be sets of multiple items
• Market basket analysis is the term usually used to
when the context is the transactions in retail/e-
commerce.
• The rule X -> Y, indicating that if you have all items in X
then you are more likely to have items in Y as well. Of
course each rule might or might not be true in a given
data set and hence has to be appropriate qualified.
• Other Applications
– web usage mining
– intrusion detection, network traffic analysis
– bioinformatics, protein sequencing
– medical diagnosis
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use 2
6
or distribution prohibited
How good is a given Rule?
• {Breakfast Cereals}->{Bread}?
• If you think this is true
– Does it apply to a large number of transactions?
– Is it often correct?
– Are you sure it is not just a coincidence?
• Lets say for example, transactions looked like this
– Total: 415
– BC: 54
– Bread: 90
– Bread and BC: 44

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use

or distribution prohibited

2
7
 Support, Confidence and Lift

• Results of an actual analysis would look like this:

2
8
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use
or distribution prohibited