Scribd
Upload
2 views

Backpropagation

Uploaded by

cse21298
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
2 views

Backpropagation

Uploaded by

cse21298
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 4

Backpropagation-

Back-propagation is the essence of neural net training. It is the method of fine-tuning the weights
of a neural net based on the error rate obtained in the previous epoch (i.e., iteration). Proper
tuning of the weights allows you to reduce error rates and to make the model reliable by
increasing its generalization.
Backpropagation is a short form for "backward propagation of errors." It is a standard method of
training artificial neural networks. This method helps to calculate the gradient of a loss function
with respects to all the weights in the network.
Backpropagation is one of the important concepts of a neural network. Our task is to classify our
data best. For this, we have to update the weights of parameter and bias, but how can we do that
in a deep neural network? In the linear regression model, we use gradient descent to optimize the
parameter. Similarly here we also use gradient descent algorithm using Backpropagation.
For a single training example, Backpropagation algorithm calculates the gradient of the error
function. Backpropagation can be written as a function of the neural network. Backpropagation
algorithms are a set of methods used to efficiently train artificial neural networks following a
gradient descent approach which exploits the chain rule.
The main features of Backpropagation are the iterative, recursive and efficient method through
which it calculates the updated weight to improve the network until it is not able to perform the
task for which it is being trained. Derivatives of the activation function to be known at network
design time is required for Backpropagation.

How Backpropagation Works: Simple Algorithm

Consider the following diagram


H
ow Backpropagation Works

1. Inputs X, arrive through the preconnected path


2. Input is modeled using real weights W. The weights are usually randomly selected.
3. Calculate the output for every neuron from the input layer, to the hidden layers, to the
output layer.
4. Calculate the error in the outputs

ErrorB= Actual Output – Desired Output

5. Travel back from the output layer to the hidden layer to adjust the weights such that the
error is decreased.

Keep repeating the process until the desired output is achieved

Why We Need Backpropagation?

Most prominent advantages of Backpropagation are:

● Backpropagation is fast, simple and easy to program


● It has no parameters to tune apart from the numbers of input
● It is a flexible method as it does not require prior knowledge about the network
● It is a standard method that generally works well
● It does not need any special mention of the features of the function to be learned.

What is a Feed Forward Network?

A feedforward neural network is an artificial neural network where the nodes never form a cycle.
This kind of neural network has an input layer, hidden layers, and an output layer. It is the first
and simplest type of artificial neural network.

Types of Backpropagation Networks

Two Types of Backpropagation Networks are:

● Static Back-propagation
● Recurrent Backpropagation

Static back-propagation:

It is one kind of backpropagation network which produces a mapping of a static input for static
output. It is useful to solve static classification issues like optical character recognition.

Recurrent Backpropagation:

Recurrent backpropagation is fed forward until a fixed value is achieved. After that, the error is
computed and propagated backward.

The main difference between both of these methods is: that the mapping is rapid in static back-
propagation while it is nonstatic in recurrent backpropagation.

Backpropagation Key Points

● Simplifies the network structure by elements weighted links that have the least effect on
the trained network
● You need to study a group of input and activation values to develop the relationship
between the input and hidden unit layers.
● It helps to assess the impact that a given input variable has on a network output. The
knowledge gained from this analysis should be represented in rules.
● Backpropagation is especially useful for deep neural networks working on error-prone
projects, such as image or speech recognition.
● Backpropagation takes advantage of the chain and power rules allows backpropagation to
function with any number of outputs.

Disadvantages of using Backpropagation


● The actual performance of backpropagation on a specific problem is dependent on the
input data.
● Backpropagation can be quite sensitive to noisy data
● You need to use the matrix-based approach for backpropagation instead of mini-batch.

Summary

● A neural network is a group of connected it I/O units where each connection has a weight
associated with its computer programs.
● Backpropagation is a short form for "backward propagation of errors." It is a standard
method of training artificial neural networks
● Backpropagation is fast, simple and easy to program
● A feedforward neural network is an artificial neural network.
● Two Types of Backpropagation Networks are 1)Static Back-propagation 2) Recurrent
Backpropagation
● In 1961, the basics concept of continuous backpropagation were derived in the context of
control theory by J. Kelly, Henry Arthur, and E. Bryson.
● Backpropagation simplifies the network structure by removing weighted links that have a
minimal effect on the trained network.
● It is especially useful for deep neural networks working on error-prone projects, such as
image or speech recognition.
● The biggest drawback of the Backpropagation is that it can be sensitive for noisy data.

You might also like