UNIT

UNIT-4

- Artificial Neural Network

- Perceptron’s
- Gradient descent and the Delta rule
- Multilayer Networks

Prepared By: Deepti Singh

1. Artificial Neural Network
• The study of an ANN (information- processing model) has been inspired in part by
the observation that biological learning systems are built of very complex webs of
interconnected neurons.
• ANN are built out of a densely interconnected set of simple units, where each
unit takes a number of real-valued
valued inputs(possibly the outputs of other units) and
produce a single real-valued
valued output(which may become the input to many other
units).
- This model tries to replicate only the most basic functions of the brain.
• ANN is composed of large no. of highly interconnected processing units
(neurons) working in unison to solve specific problems.
• ANN also learn by examples as like humans.
.
• Each neuron is connected with the other by a connection link and each
connection link is associated with weights which contain information about the
input signal.
• This information is used by the neuron network to solve a particular problem.
• ANN’s collective behaviour is characterized by their ability to learn, recall, and
generalize training patterns or data similar to that of a human brain.
• ANN processing elements are called neurons or artificial neurons.
Biological and Artificial Neural Networks.

Fig. 4 Biological Neuron Fig. 5 Artificial Neural Networks

• A typical neuron consists of the following four parts with the help of
which we can explain its working −
• Dendrites − They are tree-likelike branches, responsible for receiving the
information from other neurons it is connected to. In other sense, we
can say that they are like the ears of neuron.
• Soma or cell nucleus − It is the cell body of the neuron and is
responsible for processing of information, they have received from
dendrites.
• Axon − It is just like a cable through which neurons send the
information.
• Synapses − It is the connec on between the axon and other neuron
dendrites.
• An ANN is a computational model composed of interconnected nodes or
“neurons”, which process information in a manner similar to the human
brain.
• Warren McCulloch & Walker Pitts modelled the first artificial neural circuit
in 1943, but it gain popularity around 1980’s in mainstream data
operations.
Function: Neurons process information using weights, biases and activation
functions (like Sigmoid etc.)
Training: It uses back propagation to adjust weights and minimize errors.
Application: Image Processing, NLP, Financial Forecasting, Medical Diagnosis
etc.
* ANN learning is well-suited to problems in which the training data
corresponds to noisy, complex sensor data, such as inputs from cameras
and microphones.
ANN Architecture
• Input neurons X1 and X2 are connected to the
output neuron Y, over a weighted
interconnection links (W1 and W2).
• For the above simple neuron net architecture,
the net input has to be calculated in the
following way:
• Yin = x1w1+ x2w2 … xnwn
• i.e., Net input Yin = ∑in xiwi
• The output can be calculated by applying the
activation function over the net input.
• Y = f(yin)
• Output= function netinputclaculated
Classification of ANN
1. Single Layer feed-forward Network:
- Single layer: only one computational layer(output layer), making it is a single layer
ANN, but it consist of two layers in total.(input and output layer).
- Feed-Forward
Forward Network: Information flows from the input layer to the output
layer without any feedback loops.
Cont…
2. Multilayer Feed-Forward Network:
• It includes an input layer, one or more intermediate (hidden) layers, and an
output layer.
• Hidden layer performs intermediate computations before passing the information
to the output layer.
Cont…

3.) 4.)

(a) Single node with own feedback (b) Single layered recurrent Network
Cont…
5.)

(c) Multilayer recurrent network

Activation Function:
• There are many activation function:
1. Identity Function: it is linear function and can be defined as
f(x) = x for all x
The output here remains same as input.
The input layer uses the identity function.
2. Binary Step function: These
hese function can be defined as

Where Ɵ represents threshold value.

This is mostly used in single layer networks to convert the net input to an output that is a
binary (1 or 0).
3. Bipolar Step function: These
hese function can be defined as

Where Ɵ represents threshold value.

This is also used in single layer networks to convert the net input to an output that is a
binary (+1 or -1).
4. Ramp Function: The ramp function is defined
5. Sigmoidal function: The sigmoidal functions are widely used in back-propagation
back
networks because of the relationship between the value of the derivative at that
point which reduces the computational burden during training. It is of two types:
• Binary Sigmoid function: It is also termed as a logistic sigmoid function or
unipolar sigmoid function. It can be defined as:

• Where λ is the steepness parameter. The derivation of this function is:

• Range(0 to 1).
• Bipolar Sigmoid function: It is defined as:

• Where λ is the steepness parameter, and the range is between -1 and

+1.
• The derivative of this function is:
Deep Learning
• Deep learning is a type of artificial intelligence that mimics the way the human
brain processes information.
• It’s a subset of machine learning that uses artificial neural networks with multiple
layers—hence the term "deep.“
• These networks learn from vast amounts of data and improve their performance
over time.
• It’s behind innovations like self-driving cars, automated medical diagnoses, and
realistic AI-generated images.
Convolutional Neural Network (CNN)
• Convolutional Neural Networks are a special type of feed-forward artificial neural network in
which the connectivity pattern between its neuron is inspired by the visual cortex.
• CNN are multilayer neural network that are designed to recognize visuals patterns directly from
image.
• The Convolutional Neural Networks, which are also called as covnets, are nothing but neural
networks, sharing their parameters.
• The CNN takes an image as an array of numbers and outputs the probability of the image
belonging to a class.
• Suppose that there is an image, which is embodied as a cuboid, such that it encompasses length,
width, and height. Here the dimensions of the image are represented by the Red, Green, and Blue
channels, as shown in the image given below.
• Now assume that we have taken a small patch of the same image, followed by running a small
neural network on it, having k number of outputs, which is represented in a vertical manner.
• Now when we slide our small neural network all over the image, it will result in another image
constituting different width, height as well as depth.
depth
• We will notice that rather than having R, G, B channels, we have come across some more
channels that, too, with less width and height, which is actually the concept of Convolution.
• In case, if we accomplished in having similar patch size as that of the image, then it would have
been a regular neural network. We have some wights due to this small patch.
Working of CNN
• Generally, a Convolutional Neural Network has three layers, which are as follows;
• Input: If the image consists of 32 widths, 32 height encompassing three R, G, B channels, then
it will hold the raw pixel([32x32x3]) values of an image.
• Convolution: It has a set of filters also known as masks or kernels. This layer reads an image
and examine a small area using filters(examined by receptive field).
- No. of filters
- Filter Size
- Stride
- Padding
It computes the output of those neurons, which are associated with input's local regions, such
that each neuron will calculate a dot product in between weights and a small region to which
they are actually linked to in the input volume.
volume For example, if we choose to incorporate 12
filters, then it will result in a volume of [32x32
32x12].
• ReLU Layer: It is specially used to apply an activation function element wise, like as max (0, x)
thresholding at zero. It results in ([32x32x1212]), which relates to an unchanged size of the
volume.
• Pooling: This layer is used to perform a downsampling operation along the spatial dimensions
(width, height) that results in [16x16x12] volume.
volume

• Locally Connected: It can be defined as a regular neural network layer that receives an input from
the preceding layer followed by computing the class scores and results in a 1 Dimensional array
that has the equal size to that of the number of classes.