LaboratoryManual

IT-802Soft ComputingLab
 INDEX
S NO Title Std on Sub on Remark

1. Draw and sketch neat and clean

diagram of biological neuron.
2. Write a program to implement
perceptron training algorithm in
3.
C/C++.

Write a program to implement

Error Back propagation algorithm
4. in C/C++.

Study of Linear Separability. How

XOR problem can be solved by
5. MLP or MADALINE

Implement following activation

function in C/C++
6. …………………

(i) Ramp function

7. (ii) Bipolar step function

Study and Analysis of Counter

Propagation Network….

Study and analysis of ART on the

8.
basis of following
(i) Architecture
(ii) Need of theory
(iii) Working Introduction

Study and analysis of Genetic

9. Algorithm on the basis of
following
(i) Basic concept
(ii) Working principle
(iii) Advantages and
10.
Disadvantage

Study and analysis of recognition

task on the basis of following-
 (i) Auto association
(ii) Hetro association

Explain fuzzy application on the

basis of following
(i) Components of FLC
(ii) FLC-GA based
structural optimization

EXPERIMENT-1

Object: Draw and sketch neat and clean diagram of biological neuron.
The human brain consists of a large number; more than a billion of neural cells that process
information. Each cell works like a simple processor. The massive interaction between all cells
and their parallel processing only makes the brain’s abilities possible.

Dendrites are branching fibres thatextend from the cell body or soma.
Soma or Cell Body of a neuron contains the nucleus and other structures, supportchemical
processing and production ofneurotransmitters.

Axonis a singular fibre carries information away from the soma to the synaptic sites of other
neuron (dendrites and somas), muscles, or glands.

Axon Hillockis the site of summation for incoming information. At any moment the collective

influence of all neurons that conducts impulses to a given neuron will determine whether or not
anaction potential will be initiated at theaxon hillock and propagated along the axon.

Myelin Sheathconsists of fat-containing cells that insulate the axon from electricalactivity. This
insulation acts to increase the rate of transmission of signals. A gapexists between each myelin
sheath cell along the axon. Since fat inhibits thepropagation of electricity, the signals jump from
one gap to the next.

Nodes of Ranvierare the gaps (about 1 µm) between myelin sheath cells long axonsare Since fat
serves as a good insulator, the myelin sheaths speed the rate oftransmission of an electrical
impulse along the axon.

Synapseis the point of connection between two neurons or a neuron and a muscle ora gland.
Electrochemical communication between neurons takes place at these junctions.

Terminal Buttonsof a neuron are the small knobs at the end of an axon that releasechemicals
called neurotransmitters.

Information flow in a Neural Cell

The input /output and the propagation of information are shown below.
 i. Dendrites receive activation from other neurons.

ii. Soma processes the incoming activations and converts them into output activations.
Axons act as transmission lines to send activation to other neurons.

iii. Synapses the junctions allow signal transmission between the axons and dendrites. .

iv. The process of transmission is by diffusion of chemicals called neuro-transmitters.

EXPERIMENT-2

Object: Write a program to implement perceptron training algorithm in C/C+

+.

Single Layer Perceptron Learning Algorithm and Flowchart of the Program and the Code
of the Program in C++.

The Single Layer Perceptron Learning Algorithm:

Step 1: Initialize all weights and threshold and values.
Step 2: Get the weighted sum of one input.
Step 3: Compare the weighted sum with the threshold and set value to output.
Step 4: If the input is of class A

If the desired output mismatches with the output

Decrease those weights which have corresponding 1 in the input
Else
Take the next input and go to Step 2
Else
If the desired output mismatches with the output
Increase those weights which have corresponding 1 in the input
Else
Take the next input and go to Step 2
Step 5: While all weights has been set, take input from the user.
Step 6: Get the weighted sum for this input
Step 7: Compare the weighted sum with the threshold
if weighted sum is less then the threshold
input is from Class A
Else
input is from class B
The Flowchart of the Program:
Part 01 of the Flowchart of the Program on Single Layer Perceptron Learning Algorithm
Part 02 of the Flowchart of the Program on Single Layer Perceptron Learning Algorithm
The Code of the Program:

/* Implement perceptron learning algorithm in C*/

#include<stdio.h>
int main()
{
signed int x[4][2],tar[4];
float w[2],wc[2],out=0;
int i,j,k=0,h=0;
float s=0,b=0,bc=0,alpha=0;
float theta;
printf("Enter the value of theta & alpha");
scanf("%f%f",&theta,&alpha);
for(i=0;i<=3;i++)
{
printf("Enter the value of %d Input row & Target",i);
for(j=0;j<=1;j++)
{
scanf("%d",&x[i][j]);
}
scanf("%d",&tar[i]);
w[i]=0;
wc[i]=0;
}
printf("\n\tNet \t Target\tWeight changes\tNew weights\t Bias changes\tBias\n");
printf("------------------------------------------------------------------\n");
mew:
printf("ITERATION %d\n",h);
for(i=0;i<=3;i++)
{
for(j=0;j<=1;j++)
{
s+=(float)x[i][j]*w[j];
}
s+=b;
printf("%.2f\t",s);
if(s>theta)
out=1;
else if(s<-theta)
out=-1;
else{out=0;}
printf("%d\t",tar[i]);
s=0;
if(out==tar[i])
{
for(j=0;j<=1;j++)
{
wc[j]=0;
bc=0;
printf("%.2f\t",wc[j]);
}
for(j=0;j<=1;j++)
printf("%.2f\t",w[j]);
k+=1;
b+=bc;
printf("%.2f\t\t",bc);
printf("%.2f\t",b);
}
else{
for(j=0;j<=1;j++)
{
wc[j]=x[i][j]*tar[i]*alpha;
w[j]+=wc[j];
printf("%.2f\t",wc[j]);
wc[j]=0;
}
for(j=0;j<=1;j++)
printf("%.2f\t",w[j]);
bc=tar[i]*alpha;
b+=bc;
printf("%.2f\t\t",bc);
printf("%.2f\t",b);
}
printf("\n");
}
if(k==4)
{printf("\nFinal weights\n");
for(j=0;j<=1;j++)
{printf("w[%d]=%.2f\t",j,w[j]);}
printf("Bias b=%.2f",b);
}
else
{
k=0;
h=h+1;

goto mew;
}
return 0;
}
The Output of the Program:

EXPERIMENT-3
Object: Write a program to implement Error Back propagation algorithm in
C/C++.
/*Backprop.c:

This file contains the back propagation routines, including functions for error
calculation and weight adjustments

Usage: See below for the detail of each function routine.

*/

#include <stdio.h>
#include "pnnet.h"
#define N_MAX_NEURONS PNN_N_MAX_NEURONS

/*Function errcalc_out(): Calculates err. of neuron in output layer Returns error value of a
neuron output - output value of this neuron target - target value of this neuron ratio - error ratio
*/

float errcalc_out(float output, float target, float ratio)

{
float error;

error = ratio * output * (1.0 - output) * (target - output);

#ifdef DEBUG
printf("Error = %f. ", error);
#endif

return error;
}

/*Function errcalc_hidden(): Calculates error of a neuron in hidden layer

output - output value of this neuron err_f - pointer to the error values of neurons in front layer
wt_f - pointer to the weight values connected to front layer neurons - number of neurons in
current layer neurons_f - number of neurons in front layer ratio - error ratio.
***/

float errcalc_hidden(float output, float *err_f, float *wt_f,

int neurons, int neurons_f, float ratio)
{
int k;
float sum = 0.0;
float *error_front;
float *weight_front;
float error;
 for(k=0;k<neurons_f;k++) {
error_front = (float *)(err_f + k);
weight_front = (float *)(wt_f + k * N_MAX_NEURONS);
sum = sum + *error_front * *weight_front;
}

error = ratio * output * (1.0 - output) * sum;

#ifdef DEBUG
printf("Error = %f. ", error);
#endif

return error;
}

/*Function weight_adj(): Adjusts the weights connected to this neuron. */

void weight_adj(float *wt, float *in, int neurons_b, float err, float ratio)
{
int k;
float *ptr_weight, *ptr_input;

for(k=0;k<neurons_b;k++) {
ptr_weight = (float *)(wt + k);
ptr_input = (float *)(in + k);
*ptr_weight = *ptr_weight + ratio * err * *ptr_input;
#ifdef NNET_DEBUG_X
if(k<6) printf("%6.3f ", *ptr_weight);
#endif
}
return;
}

EXPERIMENT-4
Object: Study of Linear Separability. How XOR problem can be solved by
MLP or MADALINE
Linearlyseparablefunctions

Theexampleofthelogicalfunctionsoftwovariables showsthattheproblem
ofperceptroncomputabilitymustbediscussedinmoredetail.Inthissection weprovide
thenecessary toolstodeal more effectivelywithfunctionsofn arguments.

LinearSeparability

WecandeducefromourexperiencewiththeXORfunctionthat manyother logicalfunctionsof

severalargumentsmustexistwhichcannotbecomputed withathresholdelement.This facthas
todowiththegeometryofthen- dimensional
hypercubewhoseverticesrepresentthecombinationof logicvaluesofthearguments.Each
logicalfunctionseparatestheverticesintotwo classes. If thepoints whose functionvalue is1
cannotbe separatedwith alinear cutfromthepointswhosefunctionvalue is0,thefunctionisnot
perceptron-computable. The f o l l o w i n g two
definitionsgivethisproblema moregeneralsetting.

Definition2:Two sets ofpoints Aand B in an n-dimensional space are calledlinearly

separable ifn+1real numbers w1,...,wn+1exist, such that every point

(x1 , x2 , . . . , xn ) ∈ A satisfies Pn wi xi ≥ wn+1and every point(x1 , x2 . , xn ) ∈ B satisfies

Pn wi xi < wn+1

Sinceaperceptron canonlycomputelinearlyseparablefunctions,aninter-
estingquestionishowmanylinearlyseparablefunctionsofnbinaryarguments thereare. When
n=2,14outofthe16possibleBooleanfunctions arelin- early separable. When
n=3,104outof256and whenn=4,1882outof65536possiblefunctionsarelinearly separable.
Althoughtherehasbeenextensiveresearchonlinearlyseparable
functionsinrecentyears,noformulafor expressing thenumber o f linearly
separablefunctionsasafunctionofnhas yetbeenfound.Howeverwewillprovidesomeupper
boundsforthisnumber inthefollowingchapters.

Dualityofinputspaceandweightspace
Thecomputationperformedbyaperceptron canbevisualizedasalinearsep-
arationofinputspace.However,whentryingtofindtheappropriateweights foraperceptron,the
searchprocesscanbebettervisualizedinweightspace. When mrealweights
mustbedetermined,thesearch spaceisthewholeofIRm .

For aperceptronwithn inputlines,finding theappropriatelinear sep- arationamounts

tofindingn+1freeparameters (n weights and thebias).
Thesen+1parametersrepresentapointin(n+1)-dimensional weightspace. Each
timewepickonepointinweightspacewearechoosingonecombina- tionofweights and
aspecificlinear separationofinputspace. This meansthateverypointin(n+1)-dimensional
w e i g h t spacecanbeassociatedwith ahyperplanein(n+1)-dimensional
e x t e n d e d inputspace.Figure shows anexample. Each
combinationofthreeweights,w1,w2,w3,whichrepresent apointinweightspace,
definesaseparationofinputspace withtheplane w1x1 +w2x2 +w3x3 =0.
There isthesamekindofrelationintheinversedirection,frominputtoweightspace.
Ifwewantthepointx1,x2,x3 tobe locatedin thepositive half-space definedbyaplane,
weneedtodeterminetheappropriateweights w1,w2 andw3.Theinequality
w1x1 +w2x2 +w3x3 ≥0
musthold.Howeverthisinequalitydefinesalinearseparationofweightspace,
thatis,thepoint(x1,x2,x3)definesacuttingplaneinweightspace.Pointsin onespacearemapped
toplanesintheotherandviceversa.Thiscomplementaryrelationiscalledduality.Inputandweights
pacearedualspacesandwe canvisualizethecomputationsdonebyperceptronsand
learningalgorithms inany oneofthem.We willswitchfromonevisualizationtotheotheras
necessaryorconvenient.

Theerrorfunctioninweightspace

Giventwosetsofpatternswhichmustbeseparatedbyaperceptron,alearn-
ingalgorithmshouldautomaticallyfindtheweightsand thresholdnecessary forthesolutionof
theproblem.Theperceptronlearning algorithmcanaccom-
plishthisforthresholdunits.AlthoughproposedbyRosenblatt itwasalready
knowninanothercontext. Assume thatthesetAofinputvectorsinn-dimensionalspacemustbe
separatedfromthesetB ofinputvectorsinsuch awaythataperceptron computes thebinary
functionfw withfw(x) =1forx∈Aand fw(x) =0forx ∈B. The
b i n a r y f u n c t i o n fwdepends o n thesetwofweights andthreshold.The
errorf u n c t i o n isthenumber
o f falseclassificationsobtainedusingtheweightvectorw.Itcanbedefinedas:
X X
E(w)= (1−fw(x))+ fw(x).

x∈A x∈B

This isafunctiondefinedoverallofweightspaceand theaimofperceptron

learningistominimizeit.SinceE(w) ispositiveorzero,wewantto reachthe
globalminimumwhereE(w)=0.Thiswillbedonebystartingwith arandom weightvectorw,and
thensearchinginweightspaceabetteralternative,in anattempttoreducetheerrorfunctionE(w)
ateachstep.
Generaldecisioncurves

Aperceptron makesadecisionbasedonalinearseparationoftheinputspace. This

reducesthekinds ofproblem solvable withasingleperceptron.More
generalseparationsofinputspacecanhelptodealwithotherkindsofproblem unsolvable
withasinglethresholdunit.Assumethat asinglecomputingunit
canproducetheseparationshowninFigure3.8.Suchaseparationoftheinput
spaceintotworegionswouldallowthe computationofthe XORfunctionwith
asingleunit.Functionsusedtodiscriminatebetweenregionsofinputspace arecalleddecision
curves.Someofthedecisioncurveswhichhavebeen studiedarepolynomials andsplines.
Instatisticalpatternrecognitionproblemsweassumethatthepatternstoberecognizedaregroupe
d inclustersininputspace.Usingacombinationof
decisioncurveswetrytoisolateoneclusterfromtheothers.Onealternative iscombining several
perceptronstoisolateaconvexregionofspace. Other alternativeswhich have been studiedare,
for example, so-called Sigma-Pi
unitswhich,foragiveninputx1,x2,...,xn,computethesumofallorsome
partialproductsoftheformxixj.

Fig : Non-linear separation of input space

In thegeneral casewewanttodistinguishbetweenregionsofspace. A neural

n e t w o r k mustlearntoidentifytheseregionsandtoassociatethemwiththecorrect
response.Themainproblemisdeterminingwhetherthefree
parametersofthesedecisionregionscanbefoundusingalearningalgorithm. In thenextchapter
weshow thatit isalways possible tofind these free parameters
forlineardecisioncurves,ifthepatternsto beclassifiedareindeed linearly separable. Finding
l e a r n i n g a l g o r i t h m s for otherkinds ofdecision curvesareanimportantresearch
topicnotdealtwithhere[45,4].

TheXORproblem

Wecannowdealwiththeproblemof determiningwhichlogicalfunctionscan
beimplementedwithasingleperceptron.Aperceptronnetworkiscapable
ofcomputinganylogicalfunction,sinceperceptronsareevenmorepowerful
thanunweightedMcCulloch–Pittselements.Ifwereduce thenetworktoa
singleelement,whichfunctionsarestillcomputable?

Taking thefunctionsoftwovariables asan example wecan gain some insightintothis problem.

Table showsall16possibleBooleanfunctions oftwovariables f0tof15.
Eachcolumnfishowsthevalueofthefunctionfor eachcombinationofthetwovariablesx1 and x2.
Thefunctionf 0,forexample,
isthezerofunc tionwhereasf
14istheOR- function.

Table:The 16Boolean functionsoftwovariables

Perceptron-computable functions are those for which the points whose function value is 0 can be
separated from the points whose function value is 1 using a line. Figure shows two possible
separations to compute the ORand the AND functions.
 OR AND
11 0 1

1 0 0
0

Separations of input space corresponding to OR and AND

It isclearthattwoofthefunctions inthetablecannotbecomputedinthisway.Theyarethe
functionXORandidentity(f6andf9 ).Itisintuitively evidentthatnolinecanproduce
thenecessaryseparationoftheinputspace. Thiscanalsobeshownanalytically.

Letw1 andw2 betheweightsof aperceptronwithtwoinputs,andθits

threshold.IftheperceptroncomputestheXOR functionthefollowingfour
inequalitiesmustbefulfilled:

x1 =0x2 =0
w1x1 +w2x2 =0 ⇒0<θ
x1 =1x2 =0
w1x1 +w2x2 =w1 ⇒w1 ≥θ x1 =0x2 =1 w1x1 +w2x2
=w2 ⇒w2 ≥θ x1 =1x2 =1
w1x1 +w2x2 =w1+w2 ⇒w1+w2 <θ

Sinceθispositive,accordingtothefirstinequality,w1a n d w2 a r e positive too,according

tothesecondand thirdinequalities.Therefore theinequality w1 +w2 < θcannotbetrue.This
contradictionimpliesthatnoperceptron capable
ofcomputingtheXORfunctionexists.Ananalogousproofholdsfor thefunctionf9.
EXPERIMENT-5

Object: Implement following activation function in C/C++…………………

(iii) Ramp function

(iv) Bipolar step function

Ramp Function

Ramp()
{
if (nargin < 3),
{
t0 = 0;
c=1;
}
elseif(nargin==3),
{
c=1;
}
ra=abs(t0/c);

if(ra<1|ra==0)
{
t = -2*T:.01:2*T;
}
elseif(ra>=1||ra==0)
{
t = -2*T*(ra):.01:2*T*(ra);
}
y = A*(c*t-t0).*(c*t>=t0);
y1 = zeros(1,length(t));

if(nargout==0)
{
plot(t,y,'k','LineWidth',2);
hold on;
plot(t,y1,':k');
hold on;
plot(y1,t,':k');
hold off;
title('Ramp signal');
 if(A<0&ra~= 0)
axis([-2*T*(ra) 2*T*(ra) 4*A/3 -A/4]);
elseif(A<0&&ra==0)
axis([-2*T 2*T 4*A/3 -A/4])

elseif A>0 & ra~=0

axis([-2*T*(ra) 2*T*(ra) -A/4 4*A/3]);
else
axis([-2*T 2*T -A/4 4*A/3]);

} }

Unit Step

Unit Step()
{
if (nargin < 3)
{
t0 = 0;
c=1;
}
elseif nargin == 3,
{
c=1;
}
ra = abs(t0/c);

if(ra<1||ra==0)
{
t = -2*T:.01:2*T;
}
elseif(ra>=1)
{
t = -2*T*(ra):.01:2*T*(ra);
}

y = A*(t>=t0/c);

y1 = zeros(1,length(t));

if(nargout==0)
{
plot(t,y,'k','LineWidth',2);
 hold on;
plot(t,y1,':k');
hold on;
plot(y1,t,':k')
hold off
title('Unit Step signal')

if (A<0 &&ra ~= 0)
{
axis([-2*T*(ra)2*T*(ra)4*A/3-A/4]);

elseif(A<0&&ra== 0)
axis([-2*T 2*T 4*A/3 -A/4])

elseif(A>0&&~=0)
axis([-2*T*(ra) 2*T*(ra) -A/4 4*A/3]);

else
axis([-2*T 2*T -A/4 4*A/3]);
}

}
Experiment: 6
Object: Study and Analysis of Counter Propagation Network….

Counter Propagation is not as general as back propagation,but it provides a solution for those
applicationsthat can not tolerate long training sessions. We Point out that in addition to
overcomeing the limitations of other networks, counter propagation has some interesting and
useful features of its own.

Counter propagation is sa combination of two well-known algorithms – The self-organizing map

of Kohonen and the Grossberg outstar.Together they posses properties not available in either one
alone.

Techniques like counter porpagation that combine network paradigims in building-block fashion
may produceNetworks closer to the brain’s architecture than any homogenous structure. It Does
indeed seem that the brainCascades varoius specialized modules to produce the desired
computation.

Counter propagation network functions as a lookup table capable of generalization. The training
process associatesInput vectors with corresponding output vectors. These vectors may be in
binary, consisting of ones and zeros, or continous. Once the network is trained, application of an
input vectors produces the desired output vector. The generalization capability of the network
allows it to produce a correct output even when it is given an input vectorThat is partially
incomplete or partially incorrect.

The advantage of the counter propagation network is that it can be trained to perform associative
mappings much faster than a typical two layer network. Counter propagation network has been
used successfully for function approxmiation, hetero-association and the data compression, and
its training occurs much faster than the back porpagation.

Counter propagations can be used as a continious function approximator. Assume that the
training pairs are (xi,zi) and zi- g(x) where g is a continous function on the set of input vectors
[x]. The mean square of approximation can be made small as desired by choosing a sufficiently
large number of p of Kohonen’s layer neurons. However, for the continous function
approximation, the network is not as efficeint as error back propagation trained networks since it
requires many more neurons for comparable accuracy. Counter propagation networks can be
used for rapid prototyping of a mapping and to speed up system development, since they
typically require orders of magnitude fewer training cycles than is usally needed in error back
propagation training.
 Fig. Conter Propagation Network

The full counter propagation network is shown in fig. In normal operation, input vectors X and Y
are applied and the trained network produces output vectors X’ and Y’, which are
approximations of X and Y, respectively. In this case, X and Y are assumed to be normalized
unit vectors; hence, they will tend to produce normalized vectors an the output.

During training, vectors X and Y are applied both as inputs to the network and as desired
outputs. X is used to train the X’ outputs, while Y is used to train the X’ outputs, while Y is used
to train the Y’ outputs of the Gossberg layer.

The full counter propagation network is trained using the same method as for the feedforward
network. The Kohonen neurons receive inputs form both the X and Y vectors, but these are
indistinguisable form a single larger vector composed of the X and Y vectors; thus this
arrangement does not affect the training algorithm.

The result is an identity mapping in which the application of pair of input vectors produces their
replicas on theOutput. They do not seem very intresting until one realizes that applying only the
X vector (With the Y vector set to 0) produces both the X’ and Y’ outputs, if F is a function
mapping X to Y’, then the network approximates it. Also, if the inverse of F exists, applying
only the Y vector (setting X to 0) produces X’. This unique ability to generate a function and its
inverse makes the counter propagation network useful in a number of applications.

The main diffrence between full and forward only counter porpagation neural network is that the
former treats both ip and dp without any special preference; both function similarly in finding the
winner cluster node of the hidden layer. The full counter propagation neural network is designed
to function in both directions-
 1. To predict ip, given dp
2. To predict dp, given ip

As in the forward counter popagation, the training set consists of many patterns – {(i p,dp):p=1,
…..,P} and the network is trained in two phases. Initially, all sets of weights assigned random
values between 0 and 1. The first phase adjusts the weights associated with connections leading
into the hidden nodes. This part of training algorithm is similar to the first phase of the forward
algorithm except that ip and dp are both used in this process. The main porpose of the second
phase is to adjust the weights associated with the connections leading away from the hidden
nodes. When training is completed the network can be used in either direction.
EXPERIMENT-7

Object: Study and analysis of ART on the basis of following

(i) Architecture
(ii) Need of theory
(iii) Working Introduction

Introduction-

One of the nice features of human memory is its ability to learn many new things without
necessarily forgetting things learned in the past. A frequently cited example is the ability to
recognize your parents even if you have not seen them for some time and have learned many
new faces in the interim. It would be highly desirable if we could impart this same capability to
an Artificial Neural Networks. Most neural networks will tend to forget old information if we
attempt to add new information incrementally. When developing an artificial neural network to
perform a particular pattern-classification operation, we typically proceed by gathering a set of
exemplars, or training patterns, then using these exemplars to train the system.

During the training, information is encoded in the system by the adjustment of weight values.
Once the training is deemed to be adequate, the system is ready to be put into production, and no
additional weight modification is permitted. This operational scenario is acceptable provided the
problem domain has well-defined boundaries and is stable. Under such conditions, it is usually
possible to define an adequate set of training inputs for whatever problem is being solved.
Unfortunately, in many realistic situations, the environment is neither bounded nor stable.
Consider a simple example. Suppose you intend to train a backpropagation to recognize the
silhouettes of a certain class of aircraft. The appropriate images can be collected and used to train
the network, which is potentially a time-consuming task depending on the size of the network
required. After the network has learned successfully to recognize all of the aircraft, the training
period is ended and no further modification of the weights is allowed. If, at some future time,
another aircraft in the same class becomes operational, you may wish to add its silhouette to the
store of knowledge in your neural network. To do this, you would have to retrain the network
with the new pattern plus all of the previous patterns. Training on only the new silhouette could
result in the network learning that pattern quite well, but forgetting previously learned patterns.
 The Adaptative Resonance Theory: ART

In 1976, Grossberg (Grossberg, 1976) introduced a model for explaining biological phenomena.
The model has three crucial properties:

1. A normalisation of the total network activity. Biological systems are usually very adaptive to
large changes in their environment. For example, the human eye can adapt itself to large
variations in light intensities;
2. Contrast enhancement of input patterns. The awareness of subtle di
erences in input patterns can mean a lot in terms of survival. Distinguishing a hiding panther
from a resting one makes all the diference in the world. The mechanism used here is contrast
enhancement;
3. Short-term memory (STM) storage of the contrast-enhanced pattern. Before the input pattern can
be decoded, it must be stored in the short-term memory. The long-term memory (LTM)
implements an arousal mechanism (i.e., the classication), whereas the STM is used to cause
gradual changes in the LTM.

The system consists of two layers, F1 and F2, which are connected to each other via the LTM

The input pattern is received at F1, whereas classification takes place in F2. As mentioned
before, the input is not directly classified. First a characterization takes place by means of
extracting features, giving rise to activation in the feature representation field. The expectations,
residing in the LTM connections, translate the input pattern to a categorization in the category
representation field. The classification is compared to the expectation of the network, which
resides in the LTM weights from F2 to F1. If there is a match, the expectations are strengthened,
otherwise the classification is rejected.

ART1: The simplified neural network model

The ART1 simplified model consists of two layers of binary neurons (with values 1 and 0),
called F1 (the comparison layer) and F2 (the recognition layer)

Each neuron in F1 is connected to all neurons in F2 via the continuous-valued forward long term
memory (LTM) Wf , and vice versa via the binary-valued backward LTM Wb. The other
modules are gain 1 and 2 (G1 and G2), and a reset module. Each neuron in the comparison layer
receives three inputs: a component of the input pattern, a component of the feedback pattern, and
a gain G1. A neuron outputs a 1 if and only if at least three of these inputs are high: the 'two-
thirds rule.' The neurons in the recognition layer each compute the inner product of their
incoming (continuous-valued) weights and the pattern sent over these connections. The winning
neuron then inhibits all the other neurons via lateral inhibition. Gain 2 is the logical 'or' of all the
elements in the input pattern x. Gain 1 equals gain 2, except when the feedback pattern from F2
 contains any 1; then it is forced to zero. Finally, the reset signal is sent to the active neuron in F2
if the input vector x and the output of F1 dier by more than some vigilance level.

Operation

The network starts by clamping the input at F1. Because the output of F2 is zero, G1 and G2 are
both on and the output of F1 matches its input.The pattern is sent to F2, and in F2 one neuron
becomes active. This signal is then sent back over the backward LTM, which reproduces a
binary pattern at F1. Gain 1 is inhibited, and only the neurons in F1 which receive a 'one' from
both x and F2 remain active. If there is a substantial mismatch between the two patterns, the reset
signal will inhibit the neuron in F2 and the process is repeated.

1. Initialisation:

where N is the number of neurons in F1, M the number of neurons in F2, 0

i < N,
and 0 ≤ j <M. Also, choose the vigilance threshold ρ, 0 ≤ ρ ≤ 1;

2. Apply the new input pattern x:

3. compute the activation values y0 of the neurons in F2:

4. select the winning neuron k (0 ≤ k <M):
5. vigilance test: if

where . denotes inner product, go to step 7, else go to step 6. Note that essentially is the
inner product ,which will be large if and near to each other;
6. neuron k is disabled from further activity. Go to step 3;
7. Set for all l, 0 ≤ l < N:

8. re-enable all neurons in F2 and go to step 2.

An example of the behaviour of the Carpenter Grossberg network for letter patterns. The binary
input patterns on the left were applied sequentially. On the right the stored patterns (i.e., the
weights of Wb for the first four output units) are shown.

ART1: The original model

In later work, Carpenter and Grossberg (Carpenter & Grossberg, 1987a, 1987b) present several
neural network models to incorporate parts of the complete theory. We will only discuss the first
model, ART1. The network incorporates a follow-the-leader clustering algorithm (Hartigan,
1975). This algorithm tries to fit each new input pattern in an existing class. If no matching class
can be found, i.e., the distance between the new pattern and all existing classes exceeds some
threshold, a new class is created containing the new pattern. The novelty in this approach is that
the network is able to adapt to new incoming patterns, while the previous memory is not
corrupted. In most neural networks, such as the backpropagation network, all patterns must be
taught sequentially; the teaching of a new pattern might corrupt the weights for all previously
learned patterns. By changing the structure of the network rather than the weights, ART1
overcomes this problem.
EXPERIMENT-8

Object: Study and analysis of Genetic Algorithm on the basis of following

(i) Basic concept

(ii) Working principle
(iii) Advantages & Disadvantage

(1) Basic concept-

In the computer science field of artificial intelligence, a genetic algorithm (GA) is
a search heuristic that mimics the process of natural evolution. This heuristic is routinely used to
generate useful solutions to optimization and search problems. Genetic algorithms belong to the
larger class of evolutionary algorithms (EA), which generate solutions to optimization problems
using techniques inspired by natural evolution, such as inheritance, mutation, selection,
and crossover.

It is a heuristic search algorithm based on the mechanism of biological evolution. On the

evolution of biological diversity is the variation among individual organisms of the chromosome.
Variations of this chromosome will affect the rate of reproduction and level of ability of
organisms to stay alive.
In these algorithms, search techniques performed well on a number of possible solutions are
known as populations. Individuals who present in a population referred to as chromosomes. This
chromosome is a solution that was shaped symbol. Initial population is built randomly, while the
next population is the result of the evolution of chromosomes through the iterations are referred
to as generations. In each generation, chromosomes are going through the evaluation process by
using a measuring instrument called a fitness function. The fitness of a chromosome will show
the quality of the chromosomes in the population. The next generation is known as a child
(offspring) formed from the combination of two chromosomes present generation that acts as a
parent (the parent) by crossing operator (crossover). In addition to crossing operators, a
chromosome can also be modified by using mutation operators. Population new generation is
formed by selecting the fitness of the parent chromosome (parent) and child fitness values of
chromosomes (offspring), and reject the other chromosomes so that the population size (number
of chromosomes in a population) constant. After going through several generations, then the
algorithm will converge to the best chromosome.

(2) Working principle-

In a genetic algorithm, a population of strings (called chromosomes or the genotype of

the genome), which encode candidate solutions (called individuals, creatures, or phenotypes) to
an optimization problem, evolves toward better solutions. Traditionally, solutions are represented
in binary as strings of 0s and 1s, but other encodings are also possible. The evolution usually
starts from a population of randomly generated individuals and happens in generations. In each
generation, the fitness of every individual in the population is evaluated, multiple individuals
are stochastically selected from the current population (based on their fitness), and modified
(recombined and possibly randomly mutated) to form a new population. The new population is
then used in the next iteration of the algorithm. Commonly, the algorithm terminates when either
a maximum number of generations has been produced, or a satisfactory fitness level has been
reached for the population. If the algorithm has terminated due to a maximum number of
generations, a satisfactory solution may or may not have been reached.

A typical genetic algorithm requires:

(1) A genetic representation of the solution domain,

(2) A fitness function to evaluate the solution domain.

A standard representation of the solution is as an array of bits. Arrays of other types and
structures can be used in essentially the same way. The main property that makes these genetic
representations convenient is that their parts are easily aligned due to their fixed size, which
facilitates simple crossover operations. Variable length representations may also be used, but
crossover implementation is more complex in this case. Tree-like representations are explored
in genetic programming and graph-form representations are explored in evolutionary
programming.

The fitness function is defined over the genetic representation and measures the quality of the
represented solution. The fitness function is always problem dependent. For instance, in
the knapsack problem one wants to maximize the total value of objects that can be put in a
knapsack of some fixed capacity. A representation of a solution might be an array of bits, where
each bit represents a different object, and the value of the bit (0 or 1) represents whether or not
the object is in the knapsack. Not every such representation is valid, as the size of objects may
exceed the capacity of the knapsack. The fitness of the solution is the sum of values of all objects
in the knapsack if the representation is valid, or 0 otherwise. In some problems, it is hard or even
impossible to define the fitness expression; in these cases, interactive genetic algorithms are
used.

Once the genetic representation and the fitness function are defined, a GA proceeds to initialize a
population of solutions (usually randomly) and then to improve it through repetitive application
of the mutation, crossover, inversion and selection operators.

Initialization

Initially many individual solutions are (usually) randomly generated to form an initial
population. The population size depends on the nature of the problem, but typically contains
several hundreds or thousands of possible solutions. Traditionally, the population is generated
randomly, allowing the entire range of possible solutions (the search space). Occasionally, the
solutions may be "seeded" in areas where optimal solutions are likely to be found.

Selection

During each successive generation, a proportion of the existing population is selected to breed a
new generation. Individual solutions are selected through a fitness-based process,
where fitter solutions (as measured by a fitness function) are typically more likely to be selected.
Certain selection methods rate the fitness of each solution and preferentially select the best
solutions. Other methods rate only a random sample of the population, as the latter process may
be very time-consuming.

Reproduction

The next step is to generate a second generation population of solutions from those selected
through genetic operators: crossover (also called recombination), and/or mutation.

For each new solution to be produced, a pair of "parent" solutions is selected for breeding from
the pool selected previously. By producing a "child" solution using the above methods of
crossover and mutation, a new solution is created which typically shares many of the
characteristics of its "parents". New parents are selected for each new child, and the process
continues until a new population of solutions of appropriate size is generated. Although
reproduction methods that are based on the use of two parents are more "biology inspired", some
research suggests that more than two "parents" generate higher quality chromosomes.

These processes ultimately result in the next generation population of chromosomes that is
different from the initial generation. Generally the average fitness will have increased by this
procedure for the population, since only the best organisms from the first generation are selected
for breeding, along with a small proportion of less fit solutions, for reasons already mentioned
above.

Although Crossover and Mutation are known as the main genetic operators, it is possible to use
other operators such as regrouping, colonization-extinction, or migration in genetic algorithms.

Termination

This generational process is repeated until a termination condition has been reached. Common
terminating conditions are:

(1) A solution is found that satisfies minimum criteria

(2) Fixed number of generations reached
(3) Allocated budget (computation time/money) reached
(4) The highest ranking solution's fitness is reaching or has reached a plateau such that
successive iterations no longer produce better results
(5) Manual inspection
(6) Combinations of the above
(7)

Simple generational genetic algorithm procedure:

(1) Choose the initial population of individuals

(2) Evaluate the fitness of each individual in that population
(3) Repeat on this generation until termination (time limit, sufficient fitness achieved, etc.):
(4) Select the best-fit individuals for reproduction
 (5) Breed new individuals through crossover and mutation operations to give birth
to offspring
(6) Evaluate the individual fitness of new individuals
(7) Replace least-fit population with new individuals

(3) ADVANTAGES & DISADVANTAGES-

Advantages-

 The first and most important point is that genetic algorithms are intrinsically parallel. Most other
algorithms are serial and can only explore the solution space to a problem in one direction at a
time, and if the solution they discover turns out to be suboptimal, there is nothing to do but
abandon all work previously completed and start over. However, since GAs have multiple
offspring, they can explore the solution space in multiple directions at once. If one path turns out
to be a dead end, they can easily eliminate it and continue work on more promising avenues,
giving them a greater chance each run of finding the optimal solution.
 Due to the parallelism that allows them to implicitly evaluate many schema at once, genetic
algorithms are particularly well-suited to solving problems where the space of all potential
solutions is truly huge - too vast to search exhaustively in any reasonable amount of time. Most
problems that fall into this category are known as "nonlinear". In a linear problem, the fitness of
each component is independent, so any improvement to any one part will result in an
improvement of the system as a whole.

Another notable strength of genetic algorithms is that they perform well in problems for which the
fitness landscape is complex - ones where the fitness function is discontinuous, noisy, changes over
time, or has many local optima. Most practical problems have a vast solution space, impossible to
search exhaustively; the challenge then becomes how to avoid the local optima - solutions that are
better than all the others that are similar to them, but that are not as good as different ones elsewhere
in the solution space. Many search algorithms can become trapped by local optima: if they reach the
top of a hill on the fitness landscape, they will discover that no better solutions exist nearby and
conclude that they have reached the best one, even though higherpeaks exist elsewhere on the map.
Evolutionary algorithms, on the other hand, have proven to be effective at escaping local optima and
discovering the global optimum in even a very rugged and complex fitness landscape.

 However, crossover is the key element that distinguishes genetic algorithms from other methods
such as hill-climbers and simulated annealing. Without crossover, each individual solution is on
its own, exploring the search space in its immediate vicinity without reference to what other
individuals may have discovered. However, with crossover in place, there is a transfer of
information between successful candidates - individuals can benefit from what others have
learned, and schemata can be mixed and combined, with the potential to produce an offspring
that has the strengths of both its parents and the weaknesses of neither.
 Another area in which genetic algorithms excel is their ability to manipulate many parameters
simultaneously Many real-world problems cannot be stated in terms of a single value to be
minimized or maximized, but must be expressed in terms of multiple objectives, usually with
tradeoffs involved: one can only be improved at the expense of another. GAs are very good at
solving such problems: in particular, their use of parallelism enables them to produce multiple
equally good solutions to the same problem.

Disadvantages-

Although genetic algorithms have proven to be an efficient and powerful problem-solving

strategy, they are not a panacea. GAs do have certain limitations; however, it will be shown that
all of these can be overcome and none of them bear on the validity of biological evolution.

 The first, and most important, consideration in creating a genetic algorithm is defining a
representation for the problem. The language used to specify candidate solutions must be robust;
i.e., it must be able to tolerate random changes such that fatal errors or nonsense do not
consistently result.
 The problem of how to write the fitness function must be carefully considered so that higher
fitness is attainable and actually does equate to a better solution for the given problem. If the
fitness function is chosen poorly or defined imprecisely, the genetic algorithm may be unable to
find a solution to the problem, or may end up solving the wrong problem.
 In addition to making a good choice of fitness function, the other parameters of a GA - the size
of the population, the rate of mutation and crossover, the type and strength of selection - must be
 also chosen with care. If the population size is too small, the genetic algorithm may not explore
enough of the solution space to consistently find good solutions. If the rate of genetic change is
too high or the selection scheme is chosen poorly, beneficial schema may be disrupted and the
population may enter error catastrophe, changing too fast for selection to ever bring about
convergence.
 One type of problem that genetic algorithms have difficulty dealing with are problems with
"deceptive" fitness functions those where the locations of improved points give misleading
information about where the global optimum is likely to be found
 One well-known problem that can occur with a GA is known as premature convergence. If an
individual that is more fit than most of its competitors emerges early on in the course of the run,
it may reproduce so abundantly that it drives down the population's diversity too soon, leading
the algorithm to converge on the local optimum that that individual represents rather than
searching the fitness landscape thoroughly enough to find the global optimum
EXPERIMENT-9

Object: Study and analysis of recognition task on the basis of following-

(i) Auto association

(ii) Hetro association

Auto Associative Memory Network

Theory

In the case of in autoassociative neural net, the training input and the target output vectors are the
same. The determination of weights of the association net is called storing of vectors.

This type of memory net needs suppression of the output noise at the memory output. The vectors
that have been stored can be retrieved horn distorted (noisy) input if the input is sufficiently similar
to it. The net’s performance is based on its ability to reproduce a stored pattern from a noisy input.
It should be noted that in the case auto associative net, the weights on the diagonal can be set to
zero. This can be called as auto associative net with no self connection. The main reason behind
setting the weights to zero is that improves the net’s ability to generalize or increase the biological
plausibility of the net. This may be more suited for iterative nets and when delta rule is being used.

Architecture
Fig shows that for an auto associative net, the training input and target output vectors are the
same, The input layer consists of n input and the output layer also consists of n output units. The
input and output layers are connected through weighted interconnections. The input and output
vectors are related component by component
Flow Chart For Training Process

Training Algorithm
Testing Algorithm

Heteroassociative Memory Network

Theory

In case of Heteroassociative neural net, the training Input and the target output vectors are different
the weights determined in a way that the net can store a set of pattern association. The association
here is a pair of training input target output vectors pairs(s (p), t (p)), with p=1,……, P.Each vector
s (p) has n components and each vector r (p) ha m component. The determination of weight is done
either by using Hebb rule or delta rule. The net finds an appropriate output vector, which
corresponds to an input vector x. that may be either one of the stored patterns or a new pattern.

Architecture

From figure can lie noticed that for a Heteroassociativenet, the training input and target output
vectors are different. The input layer consists of n number of input units and the output layer
consists of m number outputs units. There exits weighted interconnections between the input and
output layers. The input and output layer units are not correlated with each other. The flowchart of
the training process and the training algorithm, are the same as discussed above.
Testing Algorithm
EXPERIMENT-10

Object: Explain fuzzy application on the basis of following

(iii) Components of FLC

(iv) FLC-GA based structural optimization

1) Components of FLC

In this section, we will explain the main components of a fuzzy logic controller and also
implement a simple fuzzy control program. The three main actions performed by a fuzzy logic
controller are:

• Fuzzification
• Fuzzy processing
• Defuzzification
As shown in figure, when the fuzzy controller receives the input data, it translates it into a fuzzy
form. This process is called fuzzification. The controller then performs fuzzy processing, which
involves the evaluation of the input information according to IF…THEN rules created by the user
during the fuzzy control system’s programming and design stages. Once the fuzzy controller finishes
the rule-processing stage and arrives at an outcome conclusion, it begins the defuzzification process.
In this final step, the fuzzy controller converts the output conclusions into “real” output data (e.g.,
analog counts) and sends this data to the process via an output module interface. If the fuzzy logic
controller is located in the PLC rack and does not have a direct or built-in I/O interface with the
process, then it will send the defuzzification output to the PLC memory location that maps the
process’s output interface module.
 2) FLC-GA Based Structural Optimization

First, coding scheme is to be defined and the initial population is produced. The computation
with genetic operators is used to evaluate fitness function with respect to the objective function.
Figure shows the FLC-GA based optimization procedure. Using FLC we can get the expert’s
experience in fuzzy rule base of FLC. Hence, the search can react optimum solution quickly. As
a result, computing time is very much reduced. The probability and fuzzy representation of
design constraints causes FLC to reduce the risk of premature problem solution caused by
improper rule.