Supervised

learning
Types of classification
Types of Classification

• 1. Binary Classification

• This is the simplest kind of classification. In binary

classification, the goal is to sort the data into two
distinct categories.
• Think of it like a simple choice between two options.
• Imagine a system that sorts emails into
either spam or not spam.
Multiclass Classification

• Here, instead of just two categories, the data needs to

be sorted into more than two categories. The model
picks the one that best matches the input. Think of an
image recognition system that sorts pictures of animals
into categories like cat, dog, and bird.
• Basically, machine looks at the features in the image
(like shape, color, or texture) and chooses which
animal the picture is most likely to be based on
the training it received.
 Classification

• Classification algorithms are used when the output

variable is categorical, which means there are two
classes such as Yes-No, Male-Female, True-false, etc.
• K nearest neighbours
• Decision Trees
• Random Forest
• Support vector Machines
• Logistic Regression
K nearest neighbours
• K-NN algorithm assumes the similarity between the new
case/data and available cases and put the new case into
the category that is most similar to the available
categories.
KNN
• Suppose there are two
categories, i.e., Category A
and Category B, and we
have a new data point x1,
so this data point will lie in
which of these categories.
To solve this type of
problem, we need a K-NN
algorithm. With the help of
K-NN, we can easily identify
the category or class of a
particular dataset.
 How does K-NN work?

• The K-NN working can be explained on the basis of the below

algorithm:
• Step-1: Select the number K of the neighbors
• Step-2: Calculate the Euclidean distance of K number of
neighbors
• Step-3: Take the K nearest neighbors as per the calculated
Euclidean distance.
• Step-4: Among these k neighbors, count the number of the data
points in each category.
• Step-5: Assign the new data points to that category for which the
number of the neighbor is maximum.
• Step-6: Our model is ready.
• As we can see the 3
nearest neighbors
are from category A,
hence this new data
point must belong to
category A.
Decision Tree Classification Algorithm

• Decision Tree is a Supervised learning

technique that can be used for both classification and
Regression problems, but mostly it is preferred for
solving Classification problems. It is a tree-structured
classifier, where internal nodes represent the
features of a dataset, branches represent the
decision rules, and each leaf node represents the
outcome
Continue…
• In a Decision tree, there are two nodes, which are
the Decision Node and Leaf Node. Decision nodes
are used to make any decision and have multiple
branches, whereas Leaf nodes are the output of those
decisions and do not contain any further branches.
Decision
tree
• Let’s say you want to
code a simple
classification
algorithm that
determines whether
or not you want to go
outside based on the
weather.
 Random Forest Classifier

• It is a set of decision trees (DT) from a randomly

selected subset of the training set and then It collects
the votes from different decision trees to decide the
final prediction.
• Consider the following
scenario: a dataset containing
several fruits images. And the
Random Forest Classifier is
given this dataset.
• Each decision tree is given a
subset of the dataset to work
with.
• During the training phase,
each decision tree generates a
prediction result.
• The Random Forest classifier
predicts the final decision
based on most outcomes when
a new data point appears.
 SVM
• Support Vector Machine (SVM) is a supervised machine
learning algorithm used for classification and regression
tasks.
• The key idea behind the SVM algorithm is to find the
hyperplane that best separates two classes by
maximizing the margin between them. This margin is
the distance from the hyperplane to the nearest data
points (support vectors) on each side.
Hence, the SVM algorithm helps to find the
best line or decision boundary; this best
boundary or region is called as a hyperplane.
SVM algorithm finds the closest point of the
lines from both the classes. These points are
called support vectors.
The distance between the vectors and the
hyperplane is called as margin.
And the goal of SVM is to maximize this
margin. The hyperplane with maximum margin
is called the optimal hyperplane.
Ex:SVM
How does SVM
classify the data?

• It’s simple! The blue ball in the

boundary of red ones is an
outlier of blue balls. The SVM
algorithm has the
characteristics to ignore the
outlier and finds the best
hyperplane that maximizes the
margin. SVM is robust to
outliers.
 Logistic Regression?

• Logistic regression is a supervised machine

learning algorithm used for classification
tasks where the goal is to predict the probability that
an instance belongs to a given class or not.
• Logistic regression predicts the output of a categorical
dependent variable. Therefore the outcome must be a
categorical or discrete value. It can be either Yes or No,
0 or 1, true or False, etc.
 Some examples of such classifications and
instances where the binary response is expected or
implied are:

• 1. Determine the probability of heart attacks: With the help of a logistic model,
medical practitioners can determine the relationship between variables such as the
weight, exercise, etc., of an individual and use it to predict whether the person will
suffer from a heart attack or any other medical complication.

• 2. Possibility of enrolling into a university: Application aggregators can

determine the probability of a student getting accepted to a particular university or
a degree course in a college by studying the relationship between the estimator
variables, such as GRE, GMAT, or TOEFL scores.

• 3. Identifying spam emails: Email inboxes are filtered to determine if the email
communication is promotional/spam by understanding the predictor variables and
applying a logistic regression algorithm to check its authenticity.
• Ex: consider an
organization in that
try to find weather
an employee will
get promotion or
not based on their
performance
 Regression

• Regression algorithms are used if there is a relationship

between the input variable and the output variable.
• It is used for the prediction of continuous variables, such as
Weather forecasting, Market Trends, etc. Below are some
popular Regression algorithms which come under supervised
learning:
• Linear Regression
• Regression Trees
• Non-Linear Regression
• Bayesian Linear Regression
• Polynomial Regression
• Regression analysis is a statistical method to model the
relationship between a dependent (target) and independent
(predictor) variables with one or more independent variables.
• More specifically, Regression analysis helps us to understand
how the value of the dependent variable is changing
corresponding to an independent variable when other
independent variables are held fixed.
• It predicts continuous/real values such as temperature, age,
salary, price, etc
Now, the company wants to do the
advertisement of $200 in the year
2019 and wants to know the
prediction about the sales for
this year. So to solve such type of
prediction problems in machine
learning, we need regression
analysis.

•Example: Suppose
there is a marketing
company A, who does
various advertisement
every year and get
sales on that. The
below list shows the
advertisement made
by the company in the
last 5 years and the
corresponding sales:
 Linear Regression:

• Linear regression is a statistical regression method which is used for

predictive analysis.
• It is one of the very simple and easy algorithms which works on
regression and shows the relationship between the continuous
variables.
• It is used for solving the regression problem in machine learning.
• Linear regression shows the linear relationship between the
independent variable (X-axis) and the dependent variable (Y-axis),
hence called linear regression.
• If there is only one input variable (x), then such linear
regression is called simple linear regression. And if
there is more than one input variable, then such linear
regression is called multiple linear regression.
• Here we are predicting the
salary of an employee on the
basis of the year of
experience.

• Y= aX+b
• Here, Y = dependent
variables (target
variables),
X= Independent
variables (predictor
variables),
a and b are the linear
coefficients
The Brain and Neuron
• Dendrites collects the input and
gives it to the cell body
• Cell body process the input and
produces the output
• Then the output is transferred
through the axons
• Axons transfer the output to the
next neuron through its dendrites
• A neuron in the brain works by receiving signals from other
neurons through its dendrites,
• Neurons are information messengers. They use electrical
and chemical signals to send information between different
areas of the brain, as well as between the brain, the spinal
cord, and the entire body.
• processing the information is done with in the cell body,
and finally sending an electrical impulse (action potential)
down its axon to transmit the signal to other neurons,
essentially acting as a communication unit within the brain
by releasing chemical messengers called neurotransmitters
• at the synapse where it connects to other neurons;
• this process allows for complex functions like thought,
memory, and movement.
Kinds of Neurons
• There are three kinds of neurons:
1.Sensory neurons carry information
from the sense organs (such as the
eyes and ears) to the brain.
2.Motor neurons control voluntary
muscle activity, such as walking and
talking, and carry messages from
nerve cells in the brain to the
muscles.
3.Other neurons, called interneurons,
make connections between sensory
and motor neurons.
 Neural network
• A neural network is a massively collection of Neurons
interconnected with each other like human brain
consists of billions of neurons interconnected with each
other.
 First computational neuron
• The first computational model of a neuron was proposed
by Warren MuCulloch (neuroscientist) and Walter Pitts
(logician) in 1943.
• It may be divided into 2 parts.
• Here x1,x2,x3………xn are the inputs
• The first part, g takes an input performs an aggregation
and based on the aggregated value the second
part, f makes a decision.
Artificial Neural Networks
• The term "Artificial Neural Network" is
derived from Biological neural networks that
develop the structure of a human brain.
• Similar to the human brain that has neurons
interconnected to one another, artificial
neural networks also have neurons that are
interconnected to one another in various
layers of the networks.
• These neurons are known as nodes.
• Since all these neurons are artificial
that’s why it is called as artificial
neural network.
• Dendrites from Biological Neural Network represent inputs in
Artificial Neural Networks, cell nucleus represents Nodes, synapse
represents Weights, and Axon represents Output.
 Simple neural network
architecture
A basic neural network has interconnected artificial neurons in three layers:

• Input Layer
Information from the outside world enters the artificial neural network from the input layer. Input
nodes process the data, analyze or categorize it, and pass it on to the next layer.

• Hidden Layer
Hidden layers take their input from the input layer or other hidden layers. Artificial neural networks
can have a large number of hidden layers. Each hidden layer analyzes the output from the previous
layer, processes it further, and passes it on to the next layer. All the computations are done in hidden
layer.

• Output layer

The output layer gives the final result of all the data processing by the artificial neural network. It
can have single or multiple nodes.
 What are the types of neural
networks?
• Artificial neural networks can be categorized by how the
data flows from the input node to the output node.
Below are some examples:
• Feedforward neural networks
• Feedforward neural networks process data in one
direction, from the input node to the output node. Every
node in one layer is connected to every node in the next
layer. A feedforward network uses a feedback process to
improve predictions over time.
 Backpropagation algorithm

• Artificial neural networks learn continuously by using corrective feedback

loops to improve their predictive analytics. In simple terms, you can think
of the data flowing from the input node to the output node through many
different paths in the neural network. Only one path is the correct one
that maps the input node to the correct output node. To find this path,
the neural network uses a feedback loop, which works as follows:
1.Each node makes a guess about the next node in the path.
2.It checks if the guess was correct. Nodes assign higher weight values to
paths that lead to more correct guesses and lower weight values to node
paths that lead to incorrect guesses.
3.For the next data point, the nodes make a new prediction using the
higher weight paths and then repeat Step 1.
• Convolutional neural networks
• The hidden layers in convolutional neural networks
perform specific mathematical functions, like
summarizing or filtering, called convolutions. They are
very useful for image classification because they can
extract relevant features from images that are useful for
image recognition and classification. The new form is
easier to process without losing features that are critical
for making a good prediction. Each hidden layer
extracts and processes different image features, like
edges, color, and depth.
Perspectives of Machine Learning
Perspectives of ML tell us how and in what way we train a model
 Perspectives of Machine Learning

One important perspective of Machine Learning is that it

involves searching a very large space of possible
hypothesis to determine one that best fits the observed
data and any prior knowledge held by learner.
Issues in Machine Learning
1. Inadequate Training Data
• The major issue that comes while using machine learning algorithms is the lack of
quality as well as quantity of data.
• Although data plays a vital role in the processing of machine learning algorithms,
many data scientists claim that inadequate data, noisy data, and unclean data are
extremely exhausting the machine learning algorithms.
• For example, a simple task requires thousands of sample data, and an advanced
task such as speech or image recognition needs millions of sample data examples.
Further, data quality is also important for the algorithms to work ideally, but the
absence of data quality is also found in Machine Learning applications.
2. Poor quality of data

• As we have discussed above, data plays a significant role in machine learning, and
it must be of good quality as well.
• Noisy data, incomplete data, inaccurate data, and unclean data lead to less
accuracy in classification and low-quality results.
• Hence, data quality can also be considered as a major common problem while
processing machine learning algorithms.
3. Non-representative training data

• To make sure our training model is generalized well or not, we have to ensure that
sample training data must be representative of new cases that we need to
generalize. The training data must cover all cases that are already occurred as well
as occurring.
• Further, if we are using non-representative training data in the model, it results in
less accurate predictions. A machine learning model is said to be ideal if it
predicts well for generalized cases and provides accurate decisions. If there is less
training data, then there will be a sampling noise in the model, called the non-
representative training set. It won't be accurate in predictions. To overcome this, it
will be biased against one class or a group.
 4. Overfitting and Underfitting

• Overfitting is one of the most common issues faced by Machine Learning

engineers and data scientists.
• Whenever a machine learning model is trained with a huge amount of data, it
starts capturing noise and inaccurate data into the training data set. It negatively
affects the performance of the model.
• Let's understand with a simple example where we have a few training data sets
such as 1000 mangoes, 1000 apples, 1000 bananas, and 5000 papayas. Then there
is a considerable probability of identification of an apple as papaya because we
have a massive amount of biased data in the training data set; hence prediction got
negatively affected.
• The main reason behind overfitting is using non-linear methods used in machine
learning algorithms as they build non-realistic data models. We can overcome
overfitting by using linear and parametric algorithms in the machine learning
models.
Methods to reduce overfitting

• Increase training data in a dataset.

• Reduce model complexity by simplifying the model by selecting one with fewer
parameters
• Ridge Regularization and Lasso Regularization
• Early stopping during the training phase
• Reduce the noise
• Reduce the number of attributes in training data.
• Constraining the model.
 Underfitting

• Underfitting is just the opposite of overfitting. Whenever a machine learning

model is trained with fewer amounts of data, and as a result, it provides
incomplete and inaccurate data and destroys the accuracy of the machine learning
model.
• Underfitting occurs when our model is too simple to understand the base structure
of the data, just like an undersized pant. This generally happens when we have
limited data into the data set, and we try to build a linear model with non-linear
data. In such scenarios, the complexity of the model destroys, and rules of the
machine learning model become too easy to be applied on this data set, and the
model starts doing wrong predictions as well.
 Methods to reduce Underfitting

• Increase model complexity

• Remove noise from the data
• Trained on increased and better features
• Reduce the constraints
• Increase the number of epochs to get better results.
 5. Monitoring and maintenance

• As we know that generalized output data is mandatory for any

machine learning model; hence, regular monitoring and maintenance
become compulsory for the same.
• Different results for different actions require data change; hence
editing of codes as well as resources for monitoring them also become
necessary.
6. Getting bad recommendations

• A machine learning model operates under a specific context which

results in bad recommendations and concept drift in the model.
• Let's understand with an example where at a specific time customer is
looking for some gadgets, but now customer requirement changed
over time but still machine learning model showing same
recommendations to the customer while customer expectation has
been changed. This incident is called a Data Drift.
• It generally occurs when new data is introduced or interpretation of
data changes. However, we can overcome this by regularly updating
and monitoring data according to the expectations.
7. Lack of skilled resources

• Although Machine Learning and Artificial Intelligence are continuously growing in the
market, still these industries are fresher in comparison to others.

• The absence of skilled resources in the form of manpower is also an issue.

• Hence, we need manpower having in-depth knowledge of mathematics, science, and

technologies for developing and managing scientific substances for machine learning.
8. Customer Segmentation

• Customer segmentation is also an important issue while developing a machine learning

algorithm. To identify the customers who paid for the recommendations shown by the
model and who don't even check them. Hence, an algorithm is necessary to recognize the
customer behavior and trigger a relevant recommendation for the user based on past
experience.
9.Data Bias

• Data Biasing is also found a big challenge in Machine Learning. These errors exist when
certain elements of the dataset are heavily weighted or need more importance than others.

• Biased data leads to inaccurate results, skewed outcomes, and other analytical errors.
However, we can resolve this error by determining where data is actually biased in the
dataset.

• Further, take necessary steps to reduce it.

Methods to remove Data Bias

• Research more for customer segmentation.

• Be aware of your general use cases and potential outliers.
• Combine inputs from multiple sources to ensure data diversity.
• Include bias testing in the development process.
• Analyze data regularly and keep tracking errors to resolve them easily.
• Review the collected and annotated data.
• Use multi-pass annotation such as sentiment analysis, content
moderation, and intent recognition.
10.Slow implementations and results

• This issue is also very commonly seen in machine learning models.

However, machine learning models are highly efficient in producing
accurate results but are time-consuming.

• Slow programming, excessive requirements and overloaded data take more

time to provide accurate results than expected. This needs continuous
maintenance and monitoring of the model for delivering accurate results.
11.Irrelevant features

• Although machine learning models are intended to give the best

possible outcome, if we feed garbage data as input, then the result will
also be garbage.

• Hence, we should use relevant features in our training sample. A

machine learning model is said to be good if training data has a good
set of features or less to no irrelevant features.