UNIT – 4

Machine Learning
Introduction to Machine Learning
• A subset of artificial intelligence known as machine learning focuses primarily on the
creation of algorithms that enable a computer to independently learn from data and
previous experiences.
• Machine Learning is the field of study that gives computers the capability to learn
without being explicitly programmed.
• Machine learning algorithms create a mathematical model that, without being explicitly
programmed, aids in making predictions or decisions with the assistance of sample
historical data, or training data.

How does Machine Learning work

• A machine learning system builds prediction models, learns from previous data, and
predicts the output of new data whenever it receives it. The amount of data helps to
build a better model that accurately predicts the output, which in turn affects the
accuracy of the predicted output.
• Let's say we have a complex problem in which we need to make predictions. Instead of
writing code, we just need to feed the data to generic algorithms, which build the logic
based on the data and predict the output. Our perspective on the issue has changed as a
result of machine learning. The Machine Learning algorithm's operation is depicted in
the following block diagram:

Features of Machine Learning:

• Machine learning uses data to detect various patterns in a given dataset.

• It can learn from past data and improve automatically.

SAHANA K BGS FGC Mysuru

 • It is a data-driven technology.
• Machine learning is much similar to data mining as it also deals with the huge amount
of the data.

Classification/Types of Machine Learning

Machine learning can be classified into three types:

1. Supervised learning
2. Unsupervised learning
3. Semi supervised learning
4. Reinforcement learning

1) Supervised Learning
• In supervised learning, sample labelled data are provided to the machine learning
system for training, and the system then predicts the output based on the training data.
• The system uses labelled data to build a model that understands the datasets and learns
about each one. After the training and processing are done, we test the model with
sample data to see if it can accurately predict the output.
• The mapping of the input data to the output data is the objective of supervised learning.
• Spam filtering is an example of supervised learning.
• Supervised learning can be grouped further in two categories of algorithms:
i) Classification: A classification problem is when the output variable is a
category, such as “Red” or “blue” , “disease” or “no disease”.
ii) Regression: A regression problem is when the output variable is a real
value, such as “dollars” or “weight”.
Regression
Regression is a type of supervised learning that is used to predict continuous values, such as
house prices, stock prices, or customer churn. Regression algorithms learn a function that maps
from the input features to the output value.
Some common regression algorithms include:
Linear Regression
Polynomial Regression

Classification
Classification is a type of supervised learning that is used to predict categorical values, such as
whether a customer will churn or not, whether an email is spam or not, or whether a medical
image shows a tumour or not. Classification algorithms learn a function that maps from the
input features to a probability distribution over the output classes.
Some classification algorithms includes:
KNN, Logistic regression, Decision trees

SAHANA K BGS FGC Mysuru

 Applications of Supervised learning
• Spam filtering: Supervised learning algorithms can be trained to identify and classify
spam emails based on their content, helping users avoid unwanted messages.
• Image classification: Supervised learning can automatically classify images into
different categories, such as animals, objects, or scenes, facilitating tasks like image
search.
• Medical diagnosis: Supervised learning can assist in medical diagnosis by analysing
patient data, such as medical images, test results, and patient history, to identify patterns
that suggest specific diseases or conditions.
Advantages of Supervised learning
• Supervised learning allows collecting data and produces data output from previous
experiences.
• Supervised machine learning helps to solve various types of real-world computation
problems.
• It performs classification and regression tasks.
• It allows estimating or mapping the result to a new sample.
• We have complete control over choosing the number of classes we want in the training
data.
Disadvantages of Supervised learning
• Classifying big data can be challenging.
• Training for supervised learning needs a lot of computation time. So, it requires a lot of
time.
• Supervised learning cannot handle all complex tasks in Machine Learning.
• It requires a labelled data set.
• It requires a training process.

Regression Algorithm Classification Algorithm

The task of the regression algorithm is to The task of the classification algorithm is to map the
map the input value (x) with the continuous input value(x) with the discrete output variable(y).
output variable(y).

Regression Algorithms are used with Classification Algorithms are used with discrete data.
continuous data.

Regression algorithms can be used to solve Classification Algorithms can be used to solve
the regression problems such as Weather classification problems such as Identification of spam
Prediction, House price prediction, etc. emails, Speech Recognition, Identification of cancer
cells, etc.

SAHANA K BGS FGC Mysuru

The regression Algorithm can be further The Classification algorithms can be divided into Binary
divided into Linear and Non-linear Classifier and Multi-class Classifier.
Regression.

2) Unsupervised Learning
• Unsupervised learning is a learning method in which a machine learns without any
supervision.
• The training is provided to the machine with the set of data that has not been labelled,
classified, or categorized, and the algorithm needs to act on that data without any
supervision.
• The goal of unsupervised learning is to restructure the input data into new features or a
group of objects with similar patterns.
• In unsupervised learning, we don't have a predetermined result. The machine tries to
find useful insights from the huge amount of data.
• Example: Imagine you have a machine learning model trained on a large dataset of
unlabelled images, containing both dogs and cats. The model has never seen an image
of a dog or cat before, and it has no pre-existing labels or categories for these animals.
Your task is to use unsupervised learning to identify the dogs and cats in a new, unseen
image.

3) Semi-Supervised Learning
• Semi-supervised learning is a type of machine learning that falls in between supervised
and unsupervised learning. It is a method that uses a small amount of labelled data and
a large amount of unlabelled data to train a model.
• Example: Text classification: In text classification, the goal is to classify a given text
into one or more predefined categories. Semi-supervised learning can be used to train
a text classification model using a small amount of labelled data and a large amount of
unlabelled text data.

4) Reinforcement Learning
• In reinforcement learning, the agent interacts with the environment and explores it. The
goal of an agent is to get the most reward points, and hence, it improves its performance.
• Reinforcement learning is a feedback-based learning method, agent learns
automatically with these feedbacks and improves its performance.
• Example: Automated robots, Traffic control etc.,

The main differences between Supervised and Unsupervised learning are given below:

SAHANA K BGS FGC Mysuru

Supervised Learning Unsupervised Learning

Supervised learning algorithms are trained using Unsupervised learning algorithms are trained using
labelled data. unlabelled data.

In supervised learning, input data is provided to the In unsupervised learning, only input data is provided
model along with the output. to the model.

The goal of supervised learning is to train the model The goal of unsupervised learning is to find the
so that it can predict the output when it is given new hidden patterns and useful insights from the
data. unknown dataset.

Supervised learning model produces an accurate Unsupervised learning model may give less accurate
result. result as compared to supervised learning.

Supervised learning needs supervision to train the Unsupervised learning does not need any supervision
model. to train the model.

Decision Tree
• Decision Tree is a Supervised learning technique that can be used for both classification
and Regression problems, but mostly it is preferred for solving Classification problems.
It is a tree-structured classifier, where internal nodes represent the features of a dataset,
branches represent the decision rules and each leaf node represents the outcome.
• In a Decision tree, there are two nodes, which are the Decision Node and Leaf Node.
Decision nodes are used to make any decision and have multiple branches, whereas
Leaf nodes are the output of those decisions and do not contain any further branches.
• It is a graphical representation for getting all the possible solutions to a
problem/decision based on given conditions.
• It is called a decision tree because, similar to a tree, it starts with the root node, which
expands on further branches and constructs a tree-like structure.
• A decision tree simply asks a question, and based on the answer (Yes/No), it further
split the tree into subtrees.

SAHANA K BGS FGC Mysuru

Decision Tree Terminologies

• Root Node: Root node is from where the decision tree starts. It represents the entire
dataset, which further gets divided into two or more homogeneous sets.
• Leaf Node: Leaf nodes are the final output node, and the tree cannot be segregated
further after getting a leaf node.
• Splitting: Splitting is the process of dividing the decision node/root node into sub-
nodes according to the given conditions.
• Branch/Sub Tree: A tree formed by splitting the tree.
• Pruning: Pruning is the process of removing the unwanted branches from the tree.
• Parent/Child node: The root node of the tree is called the parent node, and other nodes
are called the child nodes.
Example: Let us take an example of playing football based on the weather condition. The
given table gives the data set.
In order to build a decision tree first we need to find out the entropy and Information Gain
(IG). The formula and calculations are given in the below image.

SAHANA K BGS FGC Mysuru

SAHANA K BGS FGC Mysuru
After calculation we got the values as
IG (weather) = 0.246
IG (temperature) = 0.029
IG (humidity) = 0.15
IG (wind) = 0.047
We have to consider the maximum value i.e, weather = 0.246. That will be the root node.
Weather will be classified as sunny, cloudy and rain. Cloud is having only yes parameter, so
the decision tree is given in the fig.1
Next, we have to consider the Sunny data under weather node. Calculating entropy and IG is
sunny data is given below.

SAHANA K BGS FGC Mysuru

From the calculation we got
IG (temperature) = 0.57
IG (humidity) = 0.97
IG (wind) = 0.019
The maximum value is humidity, therefore under weather node humidity is the sub node as
shown in the fig.2

SAHANA K BGS FGC Mysuru

Similarly, after calculating entropy and IG for rain, we will get wind as sub node. From this
data decision tree is constructed with weather as root node. The final decision tree is
constructed as below

Advantages of the Decision Tree

• It is simple to understand as it follows the same process which a human follow while
making any decision in real-life.
• It can be very useful for solving decision-related problems.
• It helps to think about all the possible outcomes for a problem.
• There is less requirement of data cleaning compared to other algorithms.
Disadvantages of the Decision Tree

• The decision tree contains lots of layers, which makes it complex.

• For more class labels, the computational complexity of the decision tree may increase.
Algorithm:
Step-1: Begin the tree with the root node, says S, which contains the complete dataset.
Step-2: Find the best attribute in the dataset using Attribute Selection Measure (ASM).
Step-3: Divide the S into subsets that contains possible values for the best attributes.
Step-4: Generate the decision tree node, which contains the best attribute.
Step-5: Recursively make new decision trees using the subsets of the dataset created in step -
Continue this process until a stage is reached where you cannot further classify the nodes and
called the final node as a leaf node.

Linear Regression
• Linear regression is one of the easiest and most popular Machine Learning algorithms.
• It is a statistical method that is used for predictive analysis. Linear regression makes
predictions for continuous/real or numeric variables such as sales, salary, age, product
price, etc.
• Linear regression algorithm shows a linear relationship between a dependent (y) and
one or more independent (y) variables, hence called as linear regression.
• Since linear regression shows the linear relationship, which means it finds how the
value of the dependent variable is changing according to the value of the independent
variable.
SAHANA K BGS FGC Mysuru
 • The linear regression model provides a sloped straight line representing the relationship
between the variables.

Types of Linear Regression

Simple Linear Regression:
If a single independent variable is used to predict the value of a numerical dependent variable,
then such a Linear Regression algorithm is called Simple Linear Regression.
Multiple Linear regression:
If more than one independent variable is used to predict the value of a numerical dependent
variable, then such a Linear Regression algorithm is called Multiple Linear Regression.
Note: Dependent variable: value which should be predicted.
Independent variable: on basis of which variable, we should predict

Mathematically, we can represent a linear regression as:

Y = mX + b
Y – dependent variable
X – independent variable
m – slope (how much Y changes for a unit change in X)
b – intercept (the value of Y when X is 0)

Linear Regression Line

A linear line showing the relationship between the dependent and independent variables is
called a regression line. A regression line can show two types of relationship:

Positive Linear Relationship:

If the dependent variable increases on the Y-axis and independent variable increases on X-axis,
then such a relationship is termed as a Positive linear relationship.

SAHANA K BGS FGC Mysuru

Negative Linear Relationship:
If the dependent variable decreases on the Y-axis and independent variable increases on the X-
axis, then such a relationship is called a negative linear relationship.

Example: Pizza price detection based on the diameter using linear regression

SAHANA K BGS FGC Mysuru

KNN Classification (K Nearest Neighbourhood)
• K-Nearest Neighbour is one of the simplest Machine Learning algorithms based on
Supervised Learning technique.
• K-NN algorithm assumes the similarity between the new case/data and available cases
and put the new case into the category that is most similar to the available categories.
• K-NN algorithm stores all the available data and classifies a new data point based on
the similarity. This means when new data appears then it can be easily classified into a
well suite category by using K- NN algorithm.

SAHANA K BGS FGC Mysuru

 • K-NN algorithm can be used for Regression as well as for Classification but mostly it
is used for the Classification problems.
• It is also called a lazy learner algorithm.
• KNN algorithm at the training phase just stores the dataset and when it gets new data,
then it classifies that data into a category that is much similar to the new data.
• Example: Suppose, we have an image of a creature that looks similar to cat and dog,
but we want to know either it is a cat or dog. So, for this identification, we can use the
KNN algorithm, as it works on a similarity measure. Our KNN model will find the
similar features of the new data set to the cats and dogs’ images and based on the most
similar features it will put it in either cat or dog category.

The K-NN working can be explained on the basis of the below algorithm:
Step-1: Select the number K of the neighbours
Step-2: Calculate the Euclidean distance of K number of neighbours
Step-3: Take the K nearest neighbours as per the calculated Euclidean distance.
Step-4: Among these k neighbours, count the number of the data points in each category.
Step-5: Assign the new data points to that category for which the number of the neighbour is
maximum.
Step-6: Our model is ready.

Example: Predicting movie genre (category) by KNN algorithm

SAHANA K BGS FGC Mysuru

From the calculated values it is predicted that “Barbie” movie belongs to “comedy” genre.

Advantages of KNN Algorithm:

• It is simple to implement.
• It is robust to the noisy training data
• It can be more effective if the training data is large.
Disadvantages of KNN Algorithm:
• Always needs to determine the value of K which may be complex some time.
• The computation cost is high because of calculating the distance between the data points
for all the training samples.

SAHANA K BGS FGC Mysuru

Naïve Bayes Classification
• Naïve Bayes algorithm is a supervised learning algorithm, which is based on Bayes
theorem and used for solving classification problems.
• Naïve Bayes Classifier is one of the simple and most effective Classification algorithms
which helps in building the fast machine learning models that can make quick
predictions.
• It is a probabilistic classifier, which means it predicts on the basis of the probability of
an object.
• Some popular examples of Naïve Bayes Algorithm are spam filtration, Sentimental
analysis, and classifying articles.

Bayes' Theorem:
Bayes' theorem is also known as Bayes' Rule or Bayes' law, which is used to determine the
probability of a hypothesis with prior knowledge. It depends on the conditional probability.
The formula for Bayes' theorem is given as:

Example: From the given data set we need to predict the person having flu and covid is
having fever r not. The below image shows the calculation

SAHANA K BGS FGC Mysuru

SAHANA K BGS FGC Mysuru
Advantages of Naïve Bayes Classifier:
• Naïve Bayes is one of the fast and easy ML algorithms to predict a class of datasets.
• It can be used for Binary as well as Multi-class Classifications.
• It performs well in multi-class predictions as compared to the other Algorithms.
• It is the most popular choice for text classification problems.

Disadvantages of Naïve Bayes Classifier:

• Naive Bayes assumes that all features are independent or unrelated, so it cannot learn
the relationship between features.

SAHANA K BGS FGC Mysuru

Support Vector Machine
• Support Vector Machine or SVM is one of the most popular Supervised Learning
algorithms, which is used for Classification as well as Regression problems. However,
primarily, it is used for Classification problems in Machine Learning.
• The goal of the SVM algorithm is to create the best line or decision boundary that can
segregate n-dimensional space into classes so that we can easily put the new data point
in the correct category in the future. This best decision boundary is called a
hyperplane.
• SVM chooses the extreme points/support vectors that help in creating the hyperplane.
These extreme cases are called as support vectors, and hence algorithm is termed as
Support Vector Machine.
• Consider the below diagram in which there are two different categories that are
classified using a decision boundary or hyperplane:

Hyperplane: There can be multiple lines/decision boundaries to segregate the classes in n-

dimensional space, but we need to find out the best decision boundary that helps to classify the
data points. This best boundary is known as the hyperplane of SVM.

Example: Suppose we see a strange cat that also has some features of dogs, so if we want a
model that can accurately identify whether it is a cat or dog, so such a model can be created by
using the SVM algorithm. We will first train our model with lots of images of cats and dogs so
that it can learn about different features of cats and dogs, and then we test it with this strange
creature. So as support vector creates a decision boundary between these two data (cat and dog)
and choose extreme cases (support vectors), it will see the extreme case of cat and dog. On the
basis of the support vectors, it will classify it as a cat.

SAHANA K BGS FGC Mysuru

Types of SVM
SVM can be of two types:
Linear SVM: Linear SVM is used for linearly separable data, which means if a dataset can be
classified into two classes by using a single straight line, then such data is termed as linearly
separable data, and classifier is used called as Linear SVM classifier.

Non-linear SVM: Non-Linear SVM is used for non-linearly separated data, which means if a
dataset cannot be classified by using a straight line, then such data is termed as non-linear data
and classifier used is called as Non-linear SVM classifier.

How does SVM works?

Linear SVM:
The working of the SVM algorithm can be understood by using an example. Suppose we have
a dataset that has two tags (green and blue), and the dataset has two features x1 and x2. We
want a classifier that can classify the pair (x1, x2) of coordinates in either green or blue.
Consider the below image:

SAHANA K BGS FGC Mysuru

So as it is 2-d space so by just using a straight line, we can easily separate these two classes.
But there can be multiple lines that can separate these classes. Consider the below image:

Hence, the SVM algorithm helps to find the best line or decision boundary; this best boundary
or region is called as a hyperplane.
The distance between the vectors and the hyperplane is called as margin. And the goal of SVM
is to maximize this margin. The hyperplane with maximum margin is called the optimal
hyperplane.

Non-Linear SVM:
If data is linearly arranged, then we can separate it by using a straight line, but for non-linear
data, we cannot draw a single straight line. Consider the below image:

SAHANA K BGS FGC Mysuru

So to separate these data points, we need to add one more dimension. we will add a third
dimension z. It can be calculated as:
z=x2 +y2
By adding the third dimension, the sample space will become as below image:

So now, SVM will divide the datasets into classes in the following way. Consider the below
image:

SAHANA K BGS FGC Mysuru

Artificial Neural Network
• The term "Artificial Neural Network" is derived from Biological neural networks
that develop the structure of a human brain.
• Similar to the human brain that has neurons interconnected to one another, artificial
neural networks also have neurons that are interconnected to one another in various
layers of the networks. These neurons are known as nodes.
• An Artificial Neural Network in the field of Artificial intelligence where it attempts to
mimic the network of neurons makes up a human brain so that computers will have an
option to understand things and make decisions in a human-like manner.
• The artificial neural network is designed by programming computers to behave simply
like interconnected brain cells.

Figure depicting the biological neuron and Artificial neuron

SAHANA K BGS FGC Mysuru

Dendrites from Biological Neural Network represent inputs in Artificial Neural Networks, cell
nucleus represents Nodes, synapse represents Weights, and Axon represents Output.

Relationship between Biological neural network and artificial neural network:

Biological Neural Artificial Neural

Network Network

Dendrites Inputs

Cell nucleus Nodes

Synapse Weights

Axon Output

The architecture of an artificial neural network:

• Let us look at various types of layers available in an artificial neural network.
• Artificial Neural Network primarily consists of three layers:
• Input Layer: As the name suggests, it accepts inputs in several different formats
provided by the programmer.
• Hidden Layer: The hidden layer presents in-between input and output layers. It
performs all the calculations to find hidden features and patterns.
• Output Layer: The input goes through a series of transformations using the hidden
layer, which finally results in output that is conveyed using this layer.

It determines weighted total is passed as an input to an activation function to produce the

output.
Activation: A mathematical function known as an activation function maps the input to the
output, and executes activations.

SAHANA K BGS FGC Mysuru

Types of Artificial Neural Networks
There are four basic types. They are
1) Single-layer feed-forward network: In this type of network, we have only two layers
input layer and the output layer. The output layer is formed when different weights are
applied to input nodes. After this, the neurons collectively give the output layer to
compute the output signals.

2) Multilayer feed-forward network: This layer has a hidden layer that is internal to the
network and has no direct contact with the external layer. The existence of one or more
hidden layers enables the network to be computationally stronger, a feed-forward
network because of information flow through the input function, and the intermediate
computations used to determine the output.

3) Single node with its own feedback: When outputs can be directed back as inputs to
the same layer or preceding layer nodes, then it results in feedback networks. Recurrent
networks are feedback networks with closed loops.

SAHANA K BGS FGC Mysuru

 4) Single-layer recurrent network: A single-layer network with a feedback connection
in which the processing element’s output can be directed back to itself or to another
processing element or both.

5) Multilayer recurrent network: In this type of network, processing element output can
be directed to the processing element in the same layer and in the preceding layer
forming a multilayer recurrent network.

Advantages of Artificial Neural Network (ANN)

• Parallel processing capability: Artificial neural networks have a numerical value that
can perform more than one task simultaneously.
• Storing data on the entire network: Data that is used in traditional programming is
stored on the whole network, not on a database.

SAHANA K BGS FGC Mysuru

 • Capability to work with incomplete knowledge: After ANN training, the information
may produce output even with inadequate data.

Disadvantages of Artificial Neural Network:

• Assurance of proper network structure: There is no particular guideline for determining
the structure of artificial neural networks. The appropriate network structure is
accomplished through experience, trial, and error.
• Unrecognized behaviour of the network
• Hardware dependence
• Difficulty of showing the issue to the network
• The duration of the network is unknown

Applications of Artificial Neural Networks

• Social Media: Artificial Neural Networks are used heavily in Social Media.
• Another common application of Machine Learning in social media is facial recognition.
This is done by finding around 100 reference points on the person’s face and then
matching them with those already available in the database using convolutional neural
networks.
• Marketing and Sales: When you log onto E-commerce sites like Amazon and Flipkart,
they will recommend your products to buy based on your previous browsing history.
• Healthcare: Artificial Neural Networks are used in Oncology to train algorithms that
can identify cancerous tissue at the microscopic level at the same accuracy as trained
physicians.
• Personal Assistants: I am sure you all have heard of Siri, Alexa, Cortana, etc., and also
heard them based on the phones you have!!! These are personal assistants and an
example of speech recognition that uses Natural Language Processing to interact with
the users and formulate a response accordingly.

SAHANA K BGS FGC Mysuru