Nural network ER.Abhishek k. upadhyay
Download as PPTX, PDF•0 likes•1,196 views
The document discusses the Hamming network, which is a two-layer neural network for pattern classification. The first layer, called the Hamming network, calculates the Hamming distance between input patterns and stored prototype patterns, and the second layer, called MAXNET, selects the output of the first layer with the minimum Hamming distance. The document provides details on the structure and learning algorithm of the Hamming network and demonstrates its ability to correctly classify patterns even with noise or missing information.
![ The purpose of the layer is to compute, in a feed forward
manner, the values of (n-HD).
Where HD is the hamming distance between the search
argument and the encoded class prototype vector.
For the Hamming net, we have input vector X
p classes => p neurons for output
output vector Y = [y1,……yp]
6/4/2015 8](https://image.slidesharecdn.com/sunita1-150604182504-lva1-app6892/85/Nural-network-ER-Abhishek-k-upadhyay-8-320.jpg)
![ for any output neuron ,m, m=1, ……p, we have
Wm = [wm1, wm2,……wmn]t and
m=1,2,……p
to be the weights between input X and each output
neuron.
Also, assuming that for each class m, one has the
prototype vector S(m) as the standard to be matched.
6/4/2015 9](https://image.slidesharecdn.com/sunita1-150604182504-lva1-app6892/85/Nural-network-ER-Abhishek-k-upadhyay-9-320.jpg)
![ Ex: To have a Hamming Net for classifying C , I , T
then
S(1) = [ 1 1 1 1 -1 -1 1 1 1 ]t
S(2) = [ -1 1 -1 -1 1 -1 -1 1 -1 ]t
S(3) = [ 1 1 1 -1 1 -1 -1 1 -1 ]t
So,
6/4/2015 23
111111111
111111111
111111111
HW](https://image.slidesharecdn.com/sunita1-150604182504-lva1-app6892/85/Nural-network-ER-Abhishek-k-upadhyay-23-320.jpg)
Nural network ER.Abhishek k. upadhyay
- 1. PRESENTED BY: ER.Abhishek k. upadhyay ECE(REGULAR),2015 6/4/2015 1
- 2. A neural network is a processing device, whose design was inspired by the design and functioning of human brain and their components. Different neural network algorithms are used for recognizing the pattern. Various algorithms differ in their learning mechanism. All learning methods used for adaptive neural networks can be classified into two major categories: Supervised learning Unsupervised learning 6/4/2015 2
- 3. Its capability for solving complex pattern recognition problems:- Noise in weights Noise in inputs Loss of connections Missing information and adding information. 6/4/2015 3
- 4. The primary function of which is to retrieve in a pattern stored in memory, when an incomplete or noisy version of that pattern is presented. This is a two layer classifier of binary bipolar vectors. The first layer of hamming network itself is capable of selecting the stored class that is at minimum HD value to the test vector presented at the input. The second layer MAXNET only suppresses outputs. 6/4/2015 4
- 5. 6/4/2015 5
- 6. The hamming network is of the feed forward type. The number of output neurons in this part equals the number of classes. The strongest response of a neuron of this layer indicated the minimum HD value between the input vector and the class this neuron represents. The second layer is MAXNET, which operates as a recurrent network. It involves both excitatory and inhibitory connections. 6/4/2015 6
- 7. 6/4/2015 7
- 8. The purpose of the layer is to compute, in a feed forward manner, the values of (n-HD). Where HD is the hamming distance between the search argument and the encoded class prototype vector. For the Hamming net, we have input vector X p classes => p neurons for output output vector Y = [y1,……yp] 6/4/2015 8
- 9. for any output neuron ,m, m=1, ……p, we have Wm = [wm1, wm2,……wmn]t and m=1,2,……p to be the weights between input X and each output neuron. Also, assuming that for each class m, one has the prototype vector S(m) as the standard to be matched. 6/4/2015 9
- 10. For classifying p classes, one can say the m’th output is 1 if and only if output for the classifier are XtS(1), XtS(2),…XtS(m),…XtS(p) So when X= S(m), the m’th output is n and other outputs are smaller than n. X= S(m) W(m) =S(m) => happens only 6/4/2015 10
- 11. Xt S(m) = (n - HD(X , S(m)) ) - HD(X , S(m)) ∴½ XtS(m) = n/2 – HD(X , S(m)) So the weight matrix: WH=½S )()( 2 )( 1 )2()2( 2 )2( 1 )1()1( 2 )1( 1 2 1 p n pp n n H SSS SSS SSS W 6/4/2015 11
- 12. By giving a fixed bias n/2 to the input then netm = ½XtS(m) + n/2 for m=1,2,……p or netm = n - HD(X , S(m)) To scale the input 0~n to 0~1 down, one can apply transfer function as f(netm) = 1/n(netm) for m=1,2,…..p 6/4/2015 12
- 13. 6/4/2015 13
- 14. So for the node with the the highest output means that the node has smallest HD between input and prototype vectors S(1)……S(m) i.e. f(netm) = 1 for other nodes f(netm) < 1 The purpose of MAXNET is to let max{ y1,……yp } equal to 1 and let others equal to 0. 6/4/2015 14
- 15. 6/4/2015 15
- 16. So ε is bounded by 0<ε<1/p and ε: lateral interaction coefficient )( 1 1 1 1 pp MW 6/4/2015 16
- 17. And So the transfer function 0 00 )( netnet net netf 6/4/2015 17 kk k M k netfY YWnet 1
- 18. Each entry of the updated vector decreases at the k’th recursion step under the MAXNET update algorithm, with the largest entry decreasing slowest. 6/4/2015 18
- 19. Step 1: Consider that patterns to classified are a1, a2 … ap,each pattern is n dimensional. The weights connecting inputs to the neuron of hamming network is given by weight matrix. pmpp n n H aaa aaa aaa W 21 22121 11211 2 1 6/4/2015 19
- 20. Step2: n-dimensional input vector x is presented to the input. Step3: Net input of each neuron of hamming network is netm = ½XtS(m) + n/2 for m=1,2,……p Where n/2 is fixed bias applied to the input of each neuron of this layer. Step 4: Out put of each neuron of first layer is, f(netm) =1/n( netm) for m=1,2,…..p 6/4/2015 20
- 21. Step 5: Output of hamming network is applied as input to MAXNET y0=f(netm) Step 6: Weights connecting neurons of hamming network and MAXNET is taken as, )( 1 1 1 1 pp MW 6/4/2015 21
- 22. Where ε must be bounded 0< ε <1/p. the quantity ε can be called the literal interaction coefficient. Dimension of WM is p×p. Step 7: The output of MAXNET is calculated as, k=1, 2, 3…… denotes the no of iterations. 0 00 )( netnet net netf k1k k M k netfY YWnet 6/4/2015 22
- 23. Ex: To have a Hamming Net for classifying C , I , T then S(1) = [ 1 1 1 1 -1 -1 1 1 1 ]t S(2) = [ -1 1 -1 -1 1 -1 -1 1 -1 ]t S(3) = [ 1 1 1 -1 1 -1 -1 1 -1 ]t So, 6/4/2015 23 111111111 111111111 111111111 HW
- 24. 6/4/2015 24
- 25. For And 6/4/2015 25 22 1 n XWnet H Y netf t 9 5 9 3 9 7 t t net X 537 111111111
- 26. Input to MAXNET and select =0.2 < 1/3(=1/p) So, And 6/4/2015 26 1 5 1 5 1 5 1 1 5 1 5 1 5 1 1 MW kk k M k netfY YWnet 1
- 27. K=o 6/4/2015 27 333.0 067.0 599.0 333.0 067.0 599.0 555.0 333.0 777.0 12.02.0 2.012.0 2.02.01 01 0 netfY o net
- 28. K=1 K=2 6/4/2015 28 t t Y net 2 0 1 120.00520.0 120.0120.0520.0 t t Y net 3 0 2 096.00480.0 096.014.0480.0
- 29. K=3 The result computed by the network after four recurrences indicates the vector x presented at i/p for mini hamming distance has been at the smallest HD from s1. So, it represents the distorted character C. 6/4/2015 29 t t Y net 4 7 0 3 00461.0 10115.0461.0
- 30. Noise is introduced in the input by adding random numbers. Hamming Network and MaxNet network recognizes correctly all the stored strings even after introducing noise at the time of testing. 6/4/2015 30
- 31. In the network, neurons are interconnected and every interconnection has some interconnecting coefficient called weight. If some of these weights are equated to zero then how it is going to effect the classification or recognition. The number of connections that can be removed such that the network performance is not affected. 6/4/2015 31
- 32. Missing information means some of the on pixels in pattern grid are made off. For the algorithm, how many information we can miss so that the strings can be recognized correctly varies from string to string. The number of pixels that can be switched off for all the stored strings in algorithm. 6/4/2015 32
- 33. Adding information means some of the off pixels in the pattern grid are made on. The number of pixels that can be made on for all the strings that can be stored in networks. 6/4/2015 33
- 34. The network architecture is very simple. This network is a counter part of Hopfield auto associative network. The advantage of this network is that it involves less number of neurons and less number of connections in comparison to its counter part. There is no capacity limitation. 6/4/2015 34
- 35. The hamming network retrieves only the closest class index and not the entire prototype vector. It is not able to restore any of the key patterns. It provides passive classification only. This network does not have any mechanism for data restoration. It’s not to restore distorted pattern. 6/4/2015 35
- 36. Jacek M. Zurada, “Introduction to artificial Neural Systems”, Jaico Publication House. New Delhi, INDIA Amit Kumar Gupta, Yash Pal Singh, “Analysis of Hamming Network and MAXNET of Neural Network method in the String Recognition”, IEEE ,2011. C.M. Bishop, Neural Networks for Pattern Recognition, Oxford University Press, Oxford, 2003. 6/4/2015 36
- 37. 6/4/2015 37