5.2 Mexican Hat Net In 1989, Kohonen developed the Mexican hat network which is a more generalized contrast enhancement network compared to the earlier Maxner. There exist several "cooperative neighbors" (neurons in close proximity) to which every other neuron is connected by excitatory links. Also, each neuron is connected over inhibitory weights to a number of" competitive neighbors" {neurons present farther away). There are several oilier fanher neurons ro which the connections between the neurons are nor established. Here, in addition to the connections within a particular laye-r Of neural net, the neurons also receive some other external signals. This interconnection pattern is repeated for several other neurons in the layer.5.2.1 Architecture of Mexican Hat Net The architecture of Mexican hat is shown in Figure 5-2, with the interconnection pattern for node Xi. The neurons here are arranged in linear order; having positive connections between Xi and near neighboring units, and negative connections between Xi and farther away neighboring units. The positive connection region is called region of cooperation and the negative connection region is called region of competition. The size of these regions depends on the relative magnitudes existing between the positive and negative weights and also on the topology of regions such as linear, rectangular, hexagonal grids, ere. In Mexican Hat, there exist two symmetric regions around each individual neuron. The individual neuron in Figure 5-2 is denoted by Xi. This neuron is surrounded by other neurons Xi+ 1, Xi-1, Xity, Xi-2, .... The nearest neighbors to the individual neuron Xi are Xi+, Xi- 1. Xit2* and Xi-2- Hence, the weights associated with these are considered to be positive and are denoted by WI and w2. The farthest neighbors m the individual neuron Xi are taken as Xi+3 and Xi-3, the weights associated with these are negative and are denoted by w3. It can be seen chat Xi+s and Xi-s are not connected to the individual neuron Xi, and therefore no weighted interconnections exist between these connections. To make it easier, the units present within a radius of 2 [query for unit] to the unit Xi are connected with positive weights, the units within radius 3 are connected with negative weights and the units present further away from radius 3 are not connected in any manner co the neuron Xi. Ws i W%y We () WW HOO GOS © & Figure 5.2 Structure of Maxican Hat5.4 Kohonen Self-Organizing Feature Maps Feature’s mapping is a process which converts the patterns of arbitrary dimensionality into a response of one- or two-dimensional arrays of neurons, i.e. it converts a wide pattern space into a typical feature space. The network performing such a mapping is called feature map. Apart from its capability to reduce the higher dimensionality, it has to preserve the neighborhood relations of the input patterns, i.e. it has to obtain a topology preserving map. For obtaining such feature maps, it is required to find a self-organizing array which consist of neurons arranged in a one-dimensional array or a two-dimensional array. To depict this, a typical network structure where each component of the input vector x is connected to each of nodes is shown in Figure 5-5. Figure 5.5 One-dimensional Feature mapping network On the other hand, if the input vector is two-dimensional, the inputs, say x(a, b), can arrange themselves in a two-dimensional array defining the input space (a, b) as in Figure 5-6. Here, the two layers are fully connected.The topological preserving property is observed in the brain, bur nor found in any other artificial neural network. 0000] 000 af 00 e°/ [o. | | | | L MV Figure 5.6. Two dimensional feature mapping network okey) 900 ° °O e oO S000 Oo 5.4.1 Architecture of Kohonen Self-Organizing Feature Maps Consider a linear array of cluster units as in Figure 5-7. The neighborhoods of the units designated by "o" of radii Ni(k1), Ni(k2) and Ni(k,), kl > k, > &, where k/ = 2,42 = 1,43 =0. Fora rectangular grid, a neighborhood (Ni) of radii k1, k2, and k3 is shown in Figure 5-8 and for a hexagonal grid the neighborhood is shown in Figure 5-9. In all the three cases (Figures 5-7-5-9), the unit with “#” symbol is the winning unit and the other units are indicated by “o." In both rectangular and hexagonal grids, k/ >k2 > k3, where kL =2,42=1,43=0. For rectangular grid, each unit has eight nearest neighbors but there are only six neighbors for each unit in the case of a hexagon grid. Missing neighborhoods may just be ignored. A typical architecture of Kohonen self-organizing feature map (KSOFM) is shown in Figure 5-10. Nik) Nata) Nk)Figure 5.7. Linear array of cluster unitsFigure 5.10. Kohonen self organizing feature map architecture Flowchart of Kohonen Self-Organizing Feature Maps Initialize the weights, earning rate x(") Initialize topologiéal neighborhood prramatars — [Calculate square of ‘Euctdean distance | DY) = 5 x~ Ht ‘Obiain winning unit dex J [__for DW) = Minimum ‘Calculate weights of winning unit w{new) = wold) + alx-W,(o')] [Reduce learning rate «| a(t+t) = 05 alt) Reduce radius of topological NAW Figure 5.11. Flowchart for training process of KSOFM5.6 Learning Vector Quantization (LVQ) LVQ is a process of classifying the patterns, wherein each output unit represents a particular class. Here, for each class several units should be used. The output unit weight vector is called the reference vector or code book vector for the class which the unit represents. This is a special case of competitive net, which uses supervised learning methodology. During the training the output units are found to be positioned to approximate the decision surfaces of the existing Bayesian classifier. Here, the set of training patterns with known classifications is given to the network, along with an initial distribution of the reference vectors. When the training process is complete, an LVQ net is found to classify an input vector by assigning it to the same class as that of the output unit, which has its weight vector very close to the input vector. Thus LVQ is a classifier paradigm that adjusts the boundaries between categories to minimize existing misclassification. LVQ is used for optical character recognition, converting speech mro phonemes and other application as well. 5.6.1. Architecture of LVQ: Figure 5-13 shows the architecture of LVQ. From Figure 5-13 it can be noticed that there exists input layer with "n” unit; and output layer with "m” units. The layers are found to be fully interconnected with weighted linkage acting over the links. Figure 5.13. Architecture of LVQ6.1. Simulated Annealing Network The concept of simulated annealing has it origin in the physical annealing process performed over metals and other substances. In metallurgical annealing, a metal body is heated almost to its melting point and then cooled back slowly to room temperature. This process eventually makes the metal's global energy function reach an absolute minimum value. If the metal's temperature is reduced quickly, the energy of the metallic lattice will be higher than this minimum value because of the existence of frozen lattice dislocations that would otherwise disappear due to thermal agitation. Analogous to the physical annealing behaviour, simulated annealing can make a system change its state to a higher energy state having a chance to jump from local minima or global maxima. There exists a cooling procedure in the simulated annealing process such that the system has a higher probability of changing to an increasing energy state in the beginning phase of convergence. Then, as time goes by, the system becomes stable and always moves in the direction of decreasing energy state as in the case of normal minimization produce. With simulated annealing, a system changes its state from the original state SA°™’ to anew stare SA" with a probability P given by 1 P= Ty exp CAE/T) where AE = E°4 — £" (energy change = difference in new energy and old energy) and T is the nonnegative parameter (acts like temperature of a physical system). The probability P as a function of change in energy (AE) obtained for different values of the remperature T is shown in Figure 6 — 1. From Figure 6 — 1, it can be noticed that the probability when AE > 0 is always higher than she probability when AE < 0 for any remperature.An optimization problem seeks to find some configuration of parameters X = (X, ...,X;,), hat minimizes some function f(X) called cost function. In an arcificial neural network, configuration parameters are associated with the set of weights and the cost function is associated with the error function. The simulated annealing concept is used in statistical mechanics and is called Metropolis algorithm. As discussed earlier, this algorithm is based on a material that anneals into a solid as temperature is slowly decreased. To understand this, consider the slope of a hill having local valleys. A stone is moving down the hill. Here, the local valleys are local minima, and the bottom of the hill is going to be the global or universal minimum. It is possible that the stone may stop at a local minimum and never reaches the global minimum. In neural nets, this would correspond to a set of weights that correspond to that of local minimum, but this is nm the desired solution. Hence, to overcome this kind of situation, simulated annealing perturbs the stone such that if it is trapped in a local minimum, it escapes from it and continues falling till it reaches its global minimum (optimal solution). At that point, further perturbations cannot move the stone to a lower position. Figure 6-2 shows the simulated annealing between a stone and a hill. SE Figure 6.1 Probability “P” as a function in energy(AE) for different values of temperature TStone Local minimum Global minimum Figure 6.2 Simulated annealing stone and hill components required for annealing algorithm are the following A basic system configuration: The possible solution of a problem over which we search for a best (optimal) answer. (In a neural ner, this is optimum steady- state weight.) The move set: A ser of allowable moves thar permit us to escape from local minima and reach all possible configurations. A cost function associated with the error function. Acooling schedule: Starting of the cost function and rules to determine when it should be lowered and by how much, and when annealing should be terminated. Simulated annealing networks can be used to make a network converge to its global minimum.6.11. SPIKING NEURAL NETWORKS(SNN) As itis well known that the biological nervous system has inspired the development of the artificial neural network models. On looking into the depth of working of biological neurons, it is noted that the working of these neurons and their computations are performed in temporal domain and the neuron firing depends on the timing between the spikes stimulated in the neurons of the brain. These fundamental biological understandings of the neuron operation lead the pathway to the development of spiking neural networks (SNN). SNNs fall under the category of third-generation neural networks and this is more closely related to the biological counterparts compared to the first- and second-generation neural networks. These developed spiking neural networks use transient pulses for performing the computations and require communications within the layers of the network designed. There exist different spiking neural models and their classification is based on their level of abstraction. 6. rehitecture of SNN Model Neurons in central nervous system communicate using short-duration electrical impulses called spikes or action potentials in which their amplitude is constant in the same structure of neurons. SNNs offer a biological plausible fast third-genera- tion neural connectionist model. They derive their strength and interest from an accurate modelling of synaptic interactions between neurons, taking into account the time of spike emission. SNNs overcome the computational power of neural networks made of threshold or sigmoidal units. Based on dynamic event-driven processing, they open up new horizons for developing models with an exponential capacity of memorizing and a strong ability to fast adaptation. Moreover, SNNs add a new dimension, the temporal axis, to the representationMoreover, SNNs add a new dimension, the temporal axis, to the representation capacity and the processing abi models one could use to model both the individual spiking neurons and also the nonlinear dynamics of the system. Neurons communicate with spikes, also known as action potentials. Since all spikes art identical (1-2 ms of duration and 100 mV of amplitude), the information is encoded by the liming of the spikes and not the spikes themselves. Basically, a neuron is divided into three parts: the dendrites, the soma and the axon. Generally speaking, the dendrites receive the input signals from the previous neurons. The received input signals are processed in the soma and the output signals are transmitted at the axon. The synapse is between every two neurons; if a neuron / sends a signal across the synapse to neuron i, the neuron that sends the signal is called pre-synaptic neuron and the neuron that receives the signal is called posi es of neural networks, There are many different synaptic neuron. Every neuron is surrounded by positive and negative ions. In the inner surface of the membrane there is an excess of negative charges and on the outer surface there is an excess of positive charges. Those charges create the membrane potential. Each spiking neuron is characterized by a membrane potential, When the membrane potential reaches a critical value called threshold it emits an action potential, also known as a spike (Figure 7-1), A neuron is said to fire when its membrane potential reaches a specific threshold. When it fires, it sends a spike towards all other connected neurons. Its membrane potentials then reset and the neuron cannot fire for a short period of time, this time period refractory period. The output of a spiking neuron is therefore binary (spike or not) but it can be converted to continuous signal over time. Hence the activity of a neuron over a short period of lime is converted into a mean firing rate. The spikes are identical to each other and their form does not change as the signal moves from a pre-synaptic to a post- synaptic neuron. The firing time of a neuron is called spike train.6.5. Probabilistic Neural Net The probabilistic neural net is based on the idea of conventional probability theory, such as Bayesian classification and other estimators for probability density functions, to construct a neural net for classification. This net instantly approximates optimal boundaries between categories, It assumes that the training data are original representative samples. The probabilistic neural net consists of two hidden layers as shown in Figure 6-4. The first hidden layer contains a dedicated node for each training pattern amd the second hidden layer contains a dedicated node for each class. The two hidden layers are connected on a class-by- class basis, that is, the several examples of the class in the first hidden layer are connected only to a single machine unit in the second hidden layer. Input Hidden layer layer 1 Figure 6.4, Probabilistic neural network The algorithm for the construction of the net is as follows: Step 0: For each training input pattern x(p),p = 1 to P, perform Steps | and 2. Step 1: Create pattern unit z, (hidden-layer-I unit). Weight vecror for unit zy is given by Wy = x(P) Unit zy is either z -class- 1 unit or z -class- 2 unic. Step 2: Connect the hidden-layer- 1 unit to the hidden-layer- 2 unic. Ifx(p) belongs to class 1, then connect the hidden layer unic 2}, ro the hidden layer unit Fy. Otherwise, connect pattern hidden layer unit z, to the hidden layer unit F,.5.5. Kohonen Self-Organizing Motor Map : {unsupervised or ‘suporssed ining) ia Figure 5.12. Architecture of kohonen self organizing motor map The extension of Kohonen feature map for a multilayer network involve the addition of an association layer to the output of the self-organizing feature map layer. The output node is found to associate the desired output values with certain input vectors. This type of architecture is called as Kohonen self-organizing motor map and layer that is added is called a motor map in which the movement command, are being mapped into two-dimensional locations of excitation. The architecture of KSOMM is shown in Figure 5-12. Here, the feature map is a hidden layer and this acts as a competitive network which classifies the input vectors.Q. Explain Fuzzy Relations. 7.10.1 Properties Classical Composition Fuzzy Composition ‘Associative (RS) = RSM) (R°S)°M = RSM) Commutative RSFSR RS#SR Inverse (res) T=S TR T [ (Res) T=s Ter tT 7.10.2 Equivalence Classical Composition Fuzzy Composition Reflexivity | yR(xi, xi) = Lor (xi,xi) ER | uR(i,xi) = 1 vx EX ARO, XP) = ARPA) (HI xP) | URC XA) Symmetry ER = (xj,xi)_ | = wR(xj,xi) Wxi,xj EX ER XR (xi, xj) and xR (xj, xk) UR(xi, xj) , = 1, $0 xR(xi, xk) = 1(xi,xf) | => 1 anduR(xj,xk) => 9 Transitivity . . . € R(xj,xk) € R, so (xi, xk) => UR(xi, xk) =» where ER >= min (> 1 2) Fuzzy Max-product transitive can be defined. It is given by [eg Garay = max: (Hg nx) * ug Cpa Ween) € X? “eX - 7.10.3 Tolerance Classical Composition Fuzzy Composition A tolerance relation RI on universe X is one where the only the properties of reflexivity & symmetry are satisfied. Abinary fuzzy relation that possesses the properties of reflexivity and symmetry is called fuzzy tolerance relation or resemblance relation. The tolerance relation can also be called proximity relation. The equivalence relations are a special case of the tolerance relation, ‘An equivalence relation can be formed from tolerance relation RI by (n-1) compositions with itself, where n is the cardinality of the set that defines R1, here it is X The fuzzy tolerance relation can be reformed into fuzzy equivalence relation in the same way as a crisp tolerance relation is reformed into crisp equivalence relation Rr =RoRo- ~ Tolerance tehaion BE Behe oR= & Feary Fay wlennee — ‘elon8.4 Methods of Membership Value Assignment Following are the methods for assigning membership value: Intuition Inference Rank ordering Angular fuzzy sets Neural Network Genetic Algorithm Inductive Reasoning 8.4.1 Intuition Intuition method is the base upon the common intelligence of human. It is capacity of the human to develop membership functions on the bi of their own intelligence and understanding capability. There should be an in-depth knowledge of the application to which membership value assignment has to be made. ” © very taht ght Normal Hee _Weyreey © oo 12 Weighting) Figure 8.5: Membership functions for the Fuzzy variable “weight” 8.4.2 Inference & Rank Ordering The inference method uses knowledge to perform deductive reasoning. Deduction achieves conclusion by means of forward inference.Rank ordering is carried on the basis of the preferences. Pairwise comparisons enable us to determine preferences & resulting in determining the order of membership. 8.4.3 Angular Fuzzy Sets Angular fuzzy set ‘s’ is defined on a universe of angles, thus repeating the shapes every 27 cycles. The truth value of the linguistic variable is represented by angular fuzzy sets. The logical proposition is equated to the membership value “1” is said to be “true” and that preposition with membership value 0 is said to be “false”. The intermediate values between 0 & 1 correspond to proposition being partially true or partially false. (onde 4a £8 enn a Figure 8.6: Model of Angular Fuzzy Set The values of the linguistic variable vary with “6” & their membership values are on the 4(@)axis, The membership value corresponding to the linguistic term can be obtained from equation pt(8)=t.tan (8) where t is the horizontal projection of radial vector8.4.4 Neural Network Asoralewacort ~Les] ~Le| Figure 8.7: Fuzzy Membership function evaluated from Neural Networks 8.4.5 Genetic Algorithm Genetic algorithm is based on the Darwin’s theory of evolution, the basic rule is “survival of the fittest”. Genetic algorithms use the following steps to determine the fuzzy membership function: © Fora particular functional mapping system, the same membership functions & shapes are assumed for various fuzzy variables to be defined. © These chosen membership functions are then coded into bit strings. © Then these bit strings are concatenated together . The fitness function to be used here is noted. In genetic algorithm, fitness function plays a major role similar to that played by activation function in neural network. . The fitness function is used to evaluate the fitness of each set of membership function. * These membership functions define the functional mapping of the system8.4.6 Inductive Reasoning Induction is used to deduce causes by means of backward inference. The characteristics of inductive reasoning can be used to generate membership functions. Induction employs entropy minimization principles, which clusters the parameters corresponding to the output classes. To perform inductive reasoning method, a well-defined database for the input-output relationship exist. Induction reasoning can be applied for complex systems where database is abundant & static. Laws of Induction: © — Given a set of irreducible outcomes of experiment, the induced probabilities are probability consistent with all the available information that maximize the entropy of the set. © The induced probability of a set of independent observation is proportional to the probability density of the induced probability of single observation. © The induced rule is that rule consistent with all available information of that minimizes the entropy The third law stated above is widely used for development of membership function. ‘The membership functions using inductive reasoning are generated as follow: * A fuzzy threshold is to be established between classes of data. © Using entropy minimization screening method, first determine the threshold line ¢ Then start the segmentation process ¢ The segmentation process results into two classes. . Again, partitioning the first two classes one more time, we obtain three different classes. . The partitioning is repeated with threshold value calculation, which lead us to partition the data set into a number of classes and fuzzy set. ¢ Then on the basis of shape, membership function is determined.Q. What is defuzzification? List and explain methods of defuzzification. 8.5 Overview of Defuzzification Defuzzification is mapping process from a space of fuzzy control actions defined over an output universe of discourse into space of crisp control action. A defuzzification process produces a nonfuzzy control action that represents the possibility distribution of an inferred fuzzy control action. Defuzzification process has the capability to reduce a fuzzy set into a crisp single-valued quantity or into a crisp set; to convert a fuzzy matrix into a crisp matrix; or to convert a fuzzy number into a crisp number. Mathematically, the defuzzification process may also termed as “rounding off”, Fuzzy set with a collection of membership values or a vector of values on the unit interval may be reduced to a single scalar quantity using defuzzification process. Defuzzification is the process of conversion of a fuzzy quantity into a precise quantity. The output of a fuzzy process may be union of two or more fuzzy membership functions defined on the universe of discourse of the output variable. ry Figure 8.8 (A): First part of fuzzy output, (B) second part of fuzzy output, (C) union of parts (A) and (B)Defuzzification Methods Max-membership principle Centroid method Weighted average method Mean-Max membership Centers of Sums Center of largest area First of maxima, last of maxima 8.8.1 Max-membership Principle This method is also known as height method and is limited to peak output functions. This method is given by the algebraic expression: gh) 2 uct forall x Figure 10-4 Muvmembenbipdefsuifcaion mahod Figure 8.9: Max-membership Defuzzification Method 8.8.2 Centroid Method This method is also known as center of mass, center of area, center of gravity,J is denotes an algebraic integration. ° ra : Figure 8.10: Centroid Defuzzification Method 8.8.3 Weighted Average Method This method is valid for symmetrical output membership functions only. Each membership function is weighted by its maximum membership value. Ya Luc) -* Leck) th > denotes algebraic sum and xi is the maximum of the i_ membership function. Figure 8.11: Weighted average defuzzification method (two symmetrical membership functions)8.8.4 Mean-Max Membership This method is also known as the middle of maxima. The locations of the maxima membership can be nonunique. a+b Figure 8.12: Mean-max membership defuzzification method 8.8.5 Centers of Sums This method employs the algebraic sum of the individual fuzzy subsets. Advantage: Fast calculation, Drawback: intersecting areas are added twice. The defuzzified value x* is given by: ao LD eee fDi ee; (de Figure 10-8. (Pan (0) eri fants () dein Figure 8.13: (A) First and (B) Second Membership functions, (C) DefuzzificationQ. How is an interval analysis obtained in fuzzy arithmetics? 8.9 Overview of Fuzzy Arithmetic Fuzzy arithmetic is based on the operations and computations of fuzzy numbers. Fuzzy numbers help in expressing fuzzy cardinalities and fuzzy quantifiers. Fuzzy arithmetic is applied in various engineering applications when only imprecise or uncertain sensory data are available for computation. The imprecise data from the measuring instruments are generally expressed in the form of intervals, and suitable mathematical operations are performed over these intervals to obtain a reliable data of the measurements (which are also in the form of intervals). This type of computation is called interval arithmetic or interval analysis. 8.10 Interval Analysis of Uncertain Values Fuzzy numbers are an extension of the concept of intervals. Intervals are considered at only one unique level. Fuzzy numbers consider them at several levels varying from 0 to 1. In interval analysis, the uncertainty of the data is limited between the intervals specified by the lower bound & upper bound. The following are the various types of intervals: . [a1, a2] = {x|a1 0. A fuzzy set whose support is a single element in X with wA(x) =1_ is referred to as a fuzzy singleton. Boundary: The support of a membership function for a fuzzy set A is defined as that region of universe containing elements that have nonzero but not complete membership. The boundary comprises of those elements of x of the universe such that 0 < 4A(x) < 1. The boundary elements are those which possess partial membership in fuzzy set A. (x) i Boundary-+! ‘Boundary; Figure 8.1: Properties of Membership FunctionsOther types of Fuzzy Sets “ny , 1 1 | A A 0 7 0 ~ x (A) . (8) Figure 8.2: (A) Normal Fuzzy Set and (B) Subnormal Fuzzy Set . Normal fuzzy set: A fuzzy set whose membership function has at least one element x in the universe whose membership value is unity. ° Prototypical element: The element for which the membership is equal tol. © Subnormal fuzzy set: A fuzzy set wherein no membership function has it equal to 1. © Convex fuzzy set: A convex fuzzy set has membership function whose membership values are strictly monotonically increasing or strictly monotonically decreasing or strictly monotonically increasing than strictly monotonically decreasing with increasing values for the elements in the universe.