Scribd
Upload
13 views

Classification

This document discusses various machine learning algorithms for classification tasks. It describes supervised learning classification where the goal is to assign class labels to unknown data instances. It covers flavors of classification like single label, multi-label, and binary classification. Specific algorithms discussed include naive Bayes, decision trees, kernel-based classifiers like support vector machines, artificial neural networks, and unsupervised learning techniques like k-means clustering.

Uploaded by

Syed Hamza Ibrar Shah
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
13 views

Classification

This document discusses various machine learning algorithms for classification tasks. It describes supervised learning classification where the goal is to assign class labels to unknown data instances. It covers flavors of classification like single label, multi-label, and binary classification. Specific algorithms discussed include naive Bayes, decision trees, kernel-based classifiers like support vector machines, artificial neural networks, and unsupervised learning techniques like k-means clustering.

Uploaded by

Syed Hamza Ibrar Shah
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 40

Machine Learning

Machine Learning Extended

06/16/2023 2
Classification Task

• A supervised learning Task


• Assign class label to unknown instance.

Machine Learning 3
Flavors of Classification
Flavors of Classification
• Single Label
– For a given di at most one (di, ci) is true
– Train a system which takes a di and C as input and outputs a c i

• Multi-label
– For a given di zero, one or more (di, ci) can be true
– Train a system which takes a di and C as input and outputs C’, a subset
of C

• Binary
– Build a separate system for each c i, such that it takes in as input a d i
and outputs a boolean value for (di, ci)
– The most general approach
– Based on assumption that decision on (d i, ci) is independent of (di, cj)
Bayesian Based Classifiers

• Bayesian methods apply the Bayesian inference


theorem and again solve classification and regression
problems (**).
• Bayesian methods facilitate subjective probability in
modeling.
• The following are some of the Bayesian based
algorithms:
– Naïve Bayes
– Averaged one-dependence estimators (AODE)
– Bayesian belief network (BBN)

06/16/2023 6
The naive Bayes probabilistic model

• Abstractly, the probability model for a


classifier is a conditional model
• Using Bayes' theorem, we write

• In plain English the above equation can be


written as,

7
Example Naïve Bayes Classifier
Given:

How would the Naive Bayes classifier classify the data


instance
X = (sunny, hot, high, false)?
Example Naïve Bayes Classifier
Example Naïve Bayes Classifier
Example Naïve Bayes Classifier

=> Play(Yes)
Classification Task: Decision Tree
Classification Task: Decision Tree

A possible decision tree for the data:


Classification Task: Decision Tree
• How we build decision tree

Input: A data set, S


Output: A decision tree
 If all the instances have the same value for the target attribute then
return a decision tree that is simply this value (not really a tree –
more of a stump).
 Else
1. Compute Gain values (next slide) for all attributes and select an
attribute with the highest value and create a node for that
attribute.
2. Make a branch from this node for every value of the attribute
3. Assign all possible values of the attribute to branches.
4. Follow each branch by partitioning the dataset to be only
instances whereby the value of the branch is present and then go 14
Information Gain

15
Information Gain

16
Information Gain

17
Information Gain

18
Information Gain

19
Information Gain

20
Example

21
Tree Induction Example
Step 1: Calculate entropy of the target. 

22
Tree Induction Example

Step 2: The dataset is then split on the different attributes.

23
Tree Induction Example

Step 2: The dataset is then split on the different attributes.

24
Tree Induction Example

Step 2: The dataset is then split on the different attributes.

25
Tree Induction Example
Step 3: Choose attribute with the largest information gain
as the decision node

26
Tree Induction Example
Step 3: Choose attribute with the largest information gain
as the decision node

27
Tree Induction Example
Step 4a: A branch with entropy of 0 is a leaf node.

28
Tree Induction Example
Step 4b: A branch with entropy more than 0 needs further
splitting.

29
Tree Induction Example
Step 5: The ID3 algorithm is run recursively on the non-leaf
branches, until all data is classified.

30
Tree Induction Example
Decision Tree to Decision Rules.
A decision tree can easily be transformed to a set of rules by
mapping from the root node to the leaf nodes one by one.

31
Other Decision Tree Algorithms
• Some of the algorithms are as follows:
– Random forest
– Classification and Regression Tree (CART)
– C4.5 and C5.0
– Chi-square
– Gradient boosting machines (GBM)
– Chi-Squared Automatic Interaction Detection (CHAID)
– Decision stump
– Multi-adaptive regression splines (MARS)

06/16/2023 32
Kernel Method Based Classifiers

• Kernel methods are concerned with pattern analysis


• Crux of pattern analysis includes various mapping
techniques.
• Here, the mapping datasets include vector spaces.
• Some examples of kernel method based learning algorithms
are listed as follows:
– Support Vector Machines (SVM)
– Linear discriminant analysis (LDA)

06/16/2023 34
Artificial Neural Networks (ANN)
• Similar to kernel methods, artificial neural networks are again a
class of pattern matching techniques, but these models are
inspired by the structure of biological neural networks.
• These methods are again used to solve classifications and
regression problems.
• They relate to Deep learning modeling and have many subfields
of algorithms that help solve specific problems in context.

06/16/2023 35
Unsupervised Learning: Clustering

06/16/2023 36
Unsupervised Learning: Clustering types

06/16/2023 37
Unsupervised Learning: Clustering Algorithms…

• K-Means clustering
– Choses centroids that minimize within-cluster sum of squared distance
– Assume flat convex surfaces, not able to tackle manifolds
• Agglomerative (a hierarchical) clustering
– A bottom apporach to generate clusters (tree-like outcome)
– Mminimizes average of the distances between all observations of pairs
of clusters
• DBSCAN
– Density based clustering of samples in area of high density
– Number of clusters not required a priori

06/16/2023 38
Clustering : K-Means clustering Algorithm

06/16/2023 39
Clustering : K-Means clustering Algorithm
Details of K-means

06/16/2023 40

You might also like