Scribd
Upload
4 views

37_Perceptron2

The document discusses the Hebb network and perceptron architecture, explaining the Hebb rule for weight updates and its limitations with binary data. It also covers the perceptron learning rule, its convergence properties, and limitations regarding linearly separable problems. Examples are provided to illustrate the concepts of weight updates and decision boundaries in neural networks.

Uploaded by

f20220996
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
4 views

37_Perceptron2

The document discusses the Hebb network and perceptron architecture, explaining the Hebb rule for weight updates and its limitations with binary data. It also covers the perceptron learning rule, its convergence properties, and limitations regarding linearly separable problems. Examples are provided to illustrate the concepts of weight updates and decision boundaries in neural networks.

Uploaded by

f20220996
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 21

AI for Robotics

Perceptron
DR. ABHISHEK SARKAR
ASSISTANT PROFESSOR
MECHANICAL ENGG., BITS
Hebb Network

• According to the Hebb rule, the weight vector is found to increase


proportionately to the product of the input and the learning
signal. The learning signal is equal to the neuron's output.
• In Hebb learning, if two interconnected neurons are 'on'
simultaneously then the weights associated with these neurons can
be increased by the modification made in their synaptic gap
(strength).
• Mathematically the weight update in Hebb rule is given by
wi(new) = wi(old) + xiy
29
Limitation with Binary data

• If binary data is used, the above weight update formula cannot


distinguish two conditions,
1. A training pair in which an input unit is "on" and target value is
"off."
2. A training pair in which both the input unit and the target value
are "off."
• Which means if either xi or y is equal to zero, the weight update will
not happen.

30
Hebb Network
Example

• Logical AND function (using bipolar inputs and targets).


• Initially the weights and bias are set to zero,
w 1 = w 2 = wb = 0
Inputs Targets
• First input [x1 x2 b] = [1 1 1] and
target = 1 [y = 1] x1 x2 b y

• Applying Hebb rule 1 1 1 1

wi(new) = wi(old) + xiy 1 -1 1 -1


-1 1 1 -1

31 -1 -1 1 -1
Hebb Network
Example

• So
w1(new) = w1 (old) +x1y = 0 + 1 x 1 = 1
w2(new) = w2 (old) +x2y = 0 + 1 x 1 = 1
Inputs Targets
b(new) = b(old) + y = 0 + 1 = 1
x1 x2 b y
• These updated weights are used as
the initial weights when the second 1 1 1 1
input pattern is presented. 1 -1 1 -1
-1 1 1 -1

32 -1 -1 1 -1
Hebb Network
Example

• The second input pattern is presented.


• Second input [x1 x2 b] = [1 - 1 1] and y = - 1:
• The new weights here are
Inputs Targets
w1(new) = w1(old) + x1y = 1 - 1 = 0
x1 x2 b y
w2(new) = w2(old) + x2y = 1 + 1 = 2
1 1 1 1
b(new) = b(old) + y = 1 - 1 = 0
1 -1 1 -1
• Similarly we have to update weights
-1 1 1 -1
for third and fourth input patterns.
33 -1 -1 1 -1
Hebb Network
Example
• The separating line equation is given by
−w 1 b
x2= x 1−
w2 w2
• Now for the first input [1 1 1), the separating line is given by
−1 1
x2= x − =−x 1−1
1 1 1
• Similarly we can draw separating line for second input

34
Perceptron Architecture

• The output of the network is given by

• First, consider the


network weight matrix:

• Now we can partition the weight matrix:

where elements of the ith row


of W
44
Perceptron Architecture

• The ith element of the network output vector is

• The hardlim transfer function is defined as

• Therefore, if the inner product of the ith row of the weight matrix
with the input vector is greater than or equal to -bi, the output will
be 1, otherwise the output will be 0.
45 • Each neuron in the network divides the input space into two regions.
Single-Neuron Perceptron

• The output of this network is determined by

• The decision boundary is determined by the


input vectors for which the net input n is zero

• Let’s assign some values for the weights and bias:

46
Multiple-Neuron Perceptron

• For perceptrons with multiple neurons, there will be one decision


boundary for each neuron.
• The decision boundary for neuron will be defined by

• Since its output can be either 0 or 1, a single-neuron perceptron can


classify input vectors into two categories.
• So, there are a total of 2S possible categories, where S is the number
of neurons.

51
Unified Learning Rule

• We can now rewrite the three rules

• First we will define a new variable e, the perceptron error


• Unified learning rule is

57
Training Multiple-Neuron Perceptrons

• We can generalize this rule for the multiple-neuron perceptron


• To update the ith row of the weight matrix use:

• To update the ith element of the bias vector use:

• The perceptron rule can be written conveniently in matrix notation:

and
58
Training Multiple-Neuron Perceptrons

• To test the perceptron learning rule, consider again the


apple/orange recognition problem
• The input/output prototype vectors will be

• We are using 0 as the target output for the orange pattern, p1,
instead of -1. This is because we are using the hardlim transfer
function, instead of hardlims.)
59
Training Multiple-Neuron Perceptrons

• Suppose that here we start with the initial weight matrix and bias:

• The first step is to apply the first input vector, p1, to the network:

• Then we calculate the error:

60
Training Multiple-Neuron Perceptrons

• The weight update is

• The bias update is

61
Training Multiple-Neuron Perceptrons

• The second iteration of the perceptron rule is:

• Error
• Weight and bias update

62
Training Multiple-Neuron Perceptrons

• The third iteration begins again with the first input vector:

63 Continue ...
Convergence

• Although the perceptron learning rule is simple, it is quite powerful.


• It can be shown that the rule will always converge to weights that
accomplish the desired classification (assuming that such weights
exist).
• The perceptron learning rule is guaranteed to converge to a solution
in a finite number of steps, so long as a solution exists.
• The boundary between the regions is defined by the equation

64
Limitations

• The perceptron can be used to classify input vectors that can be


separated by a linear boundary (hyperplane).
• We call such vectors linearly separable.
• Unfortunately, many problems are not linearly separable.
• Eg. the XOR gate.

65
Limitations

• Rosenblatt had investigated more complex networks, which he felt


would overcome the limitations of the basic perceptron, but he was
never able to effectively extend the perceptron rule to such
networks.

66

You might also like