MACHINE LEARNING

MC4301
ASSIGNMENT 3

Differentiate perceptron, neural network,

convolutional neural network and deep learning

1
PERCEPTRON:
In Machine Learning and Artificial Intelligence, Perceptron is the most
commonly used term for all folks. It is the primary step to learn Machine
Learning and Deep Learning technologies, which consists of a set of weights,
input values or scores, and a threshold. Perceptron is a building block of an
Artificial Neural Network. Initially, in the mid of 19th century, Mr. Frank
Rosenblatt invented the Perceptron for performing certain calculations to
detect input data capabilities or business intelligence. Perceptron is a linear
Machine Learning algorithm used for supervised learning for various binary
classifiers. This algorithm enables neurons to learn elements and processes
them one by one during preparation. In this tutorial, "Perceptron in Machine
Learning," we will discuss in-depth knowledge of Perceptron and its basic
functions in brief. Let's start with the basic introduction of Perceptron.

What is the Perceptron model in Machine Learning?

Perceptron is Machine Learning algorithm for supervised learning of various
binary classification tasks. Further, Perceptron is also understood as an
Artificial Neuron or neural network unit that helps to detect certain input
data computations in business intelligence.

Perceptron model is also treated as one of the best and simplest types of
Artificial Neural networks. However, it is a supervised learning algorithm of
binary classifiers. Hence, we can consider it as a single-layer neural network
with four main parameters, i.e., input values, weights and Bias, net sum, and
an activation function.

What is Binary classifier in Machine Learning?

In Machine Learning, binary classifiers are defined as the function that helps in
deciding whether input data can be represented as vectors of numbers and
belongs to some specific class.Binary classifiers can be considered as linear
classifiers. In simple words, we can understand it as a classification algorithm
that can predict linear predictor function in terms of weight and feature
vectors.

2
Basic Components of Perceptron
Mr. Frank Rosenblatt invented the perceptron model as a binary classifier
which contains three main components. These are as follows:

Input Nodes or Input Layer:

This is the primary component of Perceptron which accepts the initial data into
the system for further processing. Each input node contains a real numerical
value.

Wight and Bias:

Weight parameter represents the strength of the connection between units.

This is another most important parameter of Perceptron components. Weight
is directly proportional to the strength of the associated input neuron in
deciding the output. Further, Bias can be considered as the line of intercept in
a linear equation.

Activation Function:

These are the final and important components that help to determine whether
the neuron will fire or not. Activation Function can be considered primarily as
a step function.

3
NEURAL NETWORKS:

Neural networks extract identifying features from data, lacking pre-

programmed understanding. Network components include neurons,
connections, weights, biases, propagation functions, and a learning rule.
Neurons receive inputs, governed by thresholds and activation functions.
Connections involve weights and biases regulating information transfer.
Learning, adjusting weights and biases, occurs in three stages: input
computation, output generation, and iterative refinement enhancing the
network’s proficiency in diverse tasks.
These include:
❖ The neural network is simulated by a new environment.
❖ Then the free parameters of the neural network are changed as a result
of this simulation.
❖ The neural network then responds in a new way to the environment
because of the changes in its free parameters.
The ability of neural networks to identify patterns, solve intricate puzzles,
and adjust to changing surroundings is essential. Their capacity to learn from
data has far-reaching effects, ranging from revolutionizing technology
like natural language processing and self-driving automobiles to automating
decision-making processes and increasing efficiency in numerous industries.
The development of artificial intelligence is largely dependent on neural
networks, which also drive innovation and influence the direction of
technology.

Working of a Neural Network

Neural networks are complex systems that mimic some features of the
functioning of the human brain. It is composed of an input layer, one or more
hidden layers, and an output layer made up of layers of artificial neurons
that are coupled. The two stages of the basic process are called
backpropagation and forward propagation.

4
Forward Propagation:
• Input Layer: Each feature in the input layer is represented by a node on
the network, which receives input data.
• Weights and Connections: The weight of each neuronal connection
indicates how strong the connection is. Throughout training, these weights
are changed.
• Hidden Layers: Each hidden layer neuron processes inputs by multiplying
them by weights, adding them up, and then passing them through an
activation function. By doing this, non-linearity is introduced, enabling the
network to recognize intricate patterns.
• Output: The final result is produced by repeating the process until the
output layer is reached.

Backpropagation:
• Loss Calculation: The network’s output is evaluated against the real goal
values, and a loss function is used to compute the difference. For a
regression problem, the Mean Squared Error (MSE) is commonly used as
the cost function.
Loss Function:
• Gradient Descent: Gradient descent is then used by the network to
reduce the loss. To lower the inaccuracy, weights are changed based on
the derivative of the loss with respect to each weight.
• Adjusting weights: The weights are adjusted at each connection by
applying this iterative process, or backpropagation, backward across the
network.

5
• Training: During training with different data samples, the entire process
of forward propagation, loss calculation, and backpropagation is done
iteratively, enabling the network to adapt and learn patterns from the
data.
• Actvation Functions: Model non-linearity is introduced by activation
functions like the rectified linear unit (ReLU) or sigmoid. Their decision on
whether to “fire” a neuron is based on the whole weighted input.

Types of Neural Networks:

There are seven types of neural networks that can be used.
• Feedforward Neteworks: A feedforward neural network is a simple
artificial neural network architecture in which data moves from input to
output in a single direction. It has input, hidden, and output layers;
feedback loops are absent. Its straightforward architecture makes it
appropriate for a number of applications, such as regression and pattern
recognition.
• Multilayer Perceptron (MLP): MLP is a type of feedforward neural
network with three or more layers, including an input layer, one or more
hidden layers, and an output layer. It uses nonlinear activation functions.
• Convolutional Neural Network (CNN): A Convolutional Neural
Network (CNN) is a specialized artificial neural network designed for
image processing. It employs convolutional layers to automatically learn
hierarchical features from input images, enabling effective image
recognition and classification. CNNs have revolutionized computer vision
and are pivotal in tasks like object detection and image analysis.
• Recurrent Neural Network (RNN): An artificial neural network type
intended for sequential data processing is called a Recurrent Neural
Network (RNN). It is appropriate for applications where contextual
dependencies are critical, such as time series prediction and natural
language processing, since it makes use of feedback loops, which enable
information to survive within the network.
• Long Short-Term Memory (LSTM): LSTM is a type of RNN that is
designed to overcome the vanishing gradient problem in training RNNs. It
uses memory cells and gates to selectively read, write, and erase
information.

6
CONVOLUTIONAL NEURAL NETWORK:
A Convolutional Neural Network (CNN) is a type of Deep Learning neural
network architecture commonly used in Computer Vision. Computer vision
is a field of Artificial Intelligence that enables a computer to understand and
interpret the image or visual data.
When it comes to Machine Learning, Artificial Neural Networks perform
really well. Neural Networks are used in various datasets like images, audio,
and text. Different types of Neural Networks are used for different purposes,
for example for predicting the sequence of words we use Recurrent Neural
Networks more precisely an LSTM, similarly for image classification we use
Convolution Neural networks. In this blog, we are going to build a basic
building block for CNN.
Convolution Neural Network
Convolutional Neural Network (CNN) is the extended version of artificial
neural networks (ANN) which is predominantly used to extract the feature
from the grid-like matrix dataset. For example visual datasets like images or
videos where data patterns play an extensive role.

CNN Architecture:
Convolutional Neural Network consists of multiple layers like the input layer,
Convolutional layer, Pooling layer, and fully connected layers.

The Convolutional layer applies filters to the input image to extract features,
the Pooling layer downsamples the image to reduce computation, and the

7
fully connected layer makes the final prediction. The network learns the
optimal filters through backpropagation and gradient descent.

Overall Structure of a CNN

A typical CNN architecture is made up of multiple layers organized in a

specific way to automatically learn and extract features from input data. Each
layer performs a distinct operation on the input data and passes the results to
the next layer. Key layers include:

• Convolutional layers (for feature extraction)

• Pooling layers (for downsampling)
• Fully connected layers (for classification)
• Activation functions (to introduce non-linearity)

Convolutional Layer

• The convolutional layer is the core building block of a CNN. This layer
uses a set of filters (or kernels), small matrices, that slide over the input
image or previous layer’s output. These filters are used to detect simple
patterns like edges, lines, and textures.
• Each filter produces an output called a feature map. As the model learns,
these filters automatically adjust to capture relevant features for the
task.
• Stride and padding are used to control the filter's movement across the
input, affecting the spatial dimensions of the output feature maps.

Pooling Layer

• Pooling layers help reduce the spatial dimensions of the feature maps
while retaining significant information. This makes the network more
computationally efficient and reduces the risk of overfitting.
• Max pooling and average pooling are common pooling techniques. Max
pooling takes the maximum value from a feature map region, while
average pooling computes the average value.
• Pooling helps the CNN become more translation invariant, meaning it
can recognize objects regardless of their position in the image.

8
Activation Functions

• Activation functions introduce non-linearity, enabling the network to

learn complex patterns. The most commonly used activation function in
CNNs is the ReLU (Rectified Linear Unit), which converts all negative
values to zero, preserving only the positive ones.
• Other activation functions include sigmoid and tanh, though they are
less common in CNNs.

Fully Connected (Dense) Layers

• The fully connected layers are typically used near the end of a CNN to
combine the features learned by the convolutional layers and make the
final classification.
• Each neuron in a fully connected layer is connected to every neuron in
the previous layer, allowing the model to learn a weighted combination
of high-level features.

DEEP LEARNING:

Deep learning is a type of machine learning that uses artificial neural

networks to learn from data. Artificial neural networks are inspired by the
human brain, and they can be used to solve a wide variety of problems,
including image recognition, natural language processing, and speech
recognition.
Deep learning algorithms:

Deep learning algorithms are typically trained on large datasets of labeled

data. The algorithms learn to associate features in the data with the correct
labels. For example, in an image recognition task, the algorithm might learn to
associate certain features in an image (such as the shape of an object or the
color of an object) with the correct label (such as "dog" or "cat").
Once a deep learning algorithm has been trained, it can be used to make
predictions on new data. For example, a deep learning algorithm that has been
trained to recognize images of dogs can be used to identify dogs in new
images.
How does deep learning work?

9
 Deep learning works by using artificial neural networks to learn from data.
Neural networks are made up of layers of interconnected nodes, and each
node is responsible for learning a specific feature of the data. Building on our
previous example with images – in an image recognition network, the first
layer of nodes might learn to identify edges, the second layer might learn to
identify shapes, and the third layer might learn to identify objects.
As the network learns, the weights on the connections between the nodes are
adjusted so that the network can better classify the data. This process is called
training, and it can be done using a variety of techniques, such as supervised
learning, unsupervised learning, and reinforcement learning.
Deep learning applications
Deep learning can be used in a wide variety of applications, including:

• Image recognition: To identify objects and features in images, such as people,

animals, places, etc.
• Natural language processing: To help understand the meaning of text, such as
in customer service chatbots and spam filters.
• Finance: To help analyze financial data and make predictions about market
trends
• Text to image: Convert text into images, such as in the Google Translate app.

Types of deep learning

There are many different types of deep learning models. Some of the most
common types include:
Convolutional neural networks (CNNs)

CNNs are used for image recognition and processing. They are particularly
good at identifying objects in images, even when those objects are partially
obscured or distorted.
Deep reinforcement learning

Deep reinforcement learning is used for robotics and game playing. It is a type
of machine learning that allows an agent to learn how to behave in an
environment by interacting with it and receiving rewards or punishments.

10
Recurrent neural networks (RNNs)

RNNs are used for natural language processing and speech recognition. They
are particularly good at understanding the context of a sentence or phrase,
and they can be used to generate text or translate languages.

Aspect Perceptron Neural Convolutional Deep

Network Neural Learning
Network
(CNN)
Definitio A single-layer A collection of A specialized A subset of
n binary interconnected neural network machine
classifier that layers of designed for learning
models a single neurons processing grid- focusing on
neuron like data, like deep neural
images, using networks with
convolution multiple hidden
operations layers
Structure Single layer Multiple layers, Consists of Deep networks
with an input including input, convolutional, with many
and an output hidden, and pooling, and fully hidden layers;
node output layers connected layers architectures
include CNNs,
RNNs, GANs,
and
transformers
Purpose Basic General- Specialized for Solving
classification, purpose visual tasks, such complex tasks
mainly linearly function as image and in vision,
separable approximation video analysis speech, NLP,
problems and pattern and
recognition reinforcement
learning
Learning Limited to Can model Highly effective Designed for
Complexi linearly complex for complex visual highly complex
ty separable functions, but patterns and data patterns
problems may struggle spatial and large
hierarchies, even datasets

11
 without enough with fewer
hidden layers parameters due
to shared weights
Training Simple Backpropagatio Backpropagation Uses various
Algorith perceptron n with gradient with optimizations
m learning rule, descent optimizations for for
adjusts weights convolutional backpropagatio
based on errors layers n; sometimes
employs
distributed or
parallel
computing
Activatio Typically a step Sigmoid, ReLU, Commonly uses Uses ReLU,
n function tanh, etc. ReLU for hidden Sigmoid, Tanh,
Function layers, Softmax Softmax, and
s for output others
depending on
the application
Input Numeric, low- Numeric, low to Images or grid- Can handle
Type dimensional high- like data (e.g., various data
dimensional audio types: text,
spectrograms) images, video,
and structured
or unstructured
data
Interpret Highly Moderate Interpretable to a Often regarded
ability interpretable interpretability certain extent; as a "black box"
however, due to the
understanding complexity of
individual filter layers and
significance in parameters
deep layers can
be challenging

12
13