CS/IS stream paper

J N N COLLEGE OF ENGINEERING
SHIMOGA -577204
KARNATAKA

DEPARTMENT OF CSE

“Artificial Neural Networks”

PRESENTORS:
MEGHANA NAYAK K
4TH SEM CSE
E-Mail:[email protected]
MOB:7204042777

SAHANA B S
4TH SEM CSE
E-Mail:[email protected]
 other words, is an emulation of biological neural
system. An Artificial Neural Network (ANN) is an
information processing paradigm that is inspired
by the way biological nervous systems, such as the
ABSTRACT : brain, process information. It is composed of a
large number of highly interconnected processing
This report is an introduction to Artificial Neural
Networks. The various types of neural networks elements (neurones) working in unison to solve
are explained and demonstrated, applications of specific problems. ANNs, like people, learn by
neural networks like ANNs in medicine are example. An ANN is configured for a specific
described, and a detailed historical background is application, such as pattern recognition or data
provided. The connection between the artificial classification, through a learning process.
and the real thing is also investigated and Learning in biological systems involves
explained. Finally, the mathematical models
adjustments to the synaptic connections that exist
involved are presented and demonstrate Artificial
between the neurones. This is true of ANNs as
neural network-based air pollution prediction
well. A neural network is a powerful data
models have become very popular. The paper
modeling tool that is able to capture and represent
describes feature determination and pattern
selection strategies that help to improve
complex input/output relationships. The

significantly the performance of neural-network motivation for the development of neural network
based models. The most important methods for technology stemmed from the desire to develop an
pattern selection are meteorological knowledge- artificial system that could perform "intelligent"
based cluster determination and Kohonen neural tasks similar to those performed by the human
network-based cluster determination. brain. Neural networks resemble the human brain
in the following two ways:
 A neural network acquires
Introduction to neural
knowledge through learning.
networks :  A neural network's knowledge
is stored within inter-neuron
1.1 What is a Neural Network? connection strengths known as
synaptic weights.
An artificial neural network is a system based on
the operation of biological neural networks, in
 1.3Why use neural networks? Fault Tolerance via

Neural networks, with their remarkable ability Redundant.Information Coding:

to derive meaning from complicated or Partial destruction of a network leads

imprecise data, can be used to extract patterns to the corresponding

and detect trends that are too complex to be degradation of performance.

noticed by either humans or other computer However, some network capabilities

techniques. A trained neural network can be may be retained even with major

thought of as an "expert" in the category of network damage.

information it has been given to analyse. This

expert can then be used to provide projections Human and Artificial
given new situations of interest and answer Neurones -
"what if" questions. investigating the
Other advantages include: similarities
How the Human Brain Learns?
Adaptive learning: An ability to learn how to
do tasks based on the data given for training Much is still unknown about how the brain trains
or initial experience. itself to process information, so theories abound.
In the human brain, a typical neuron collects
Self-Organisation: An ANN can create its signals from others through a host of fine
own organisation or representation of the structures called dendrites. The neuron sends out
information it receives during learning time. spikes of electrical activity through a long, thin
stand known as an axon, which splits into
Real Time Operation: ANN computations
thousands of branches. At the end of each
may be carried out in parallel, and special
branch, a structure called a synapse converts the
hardware devices are being designed and
activity from the axon into electrical effects that
manufactured which take advantage of this
inhibit or excite activity from the axon into
capability.
electrical effects that inhibit or excite activity in
the connected neurones. When a neuron receives
excitatory input that is sufficiently large
 compared with its inhibitory input, it sends a spike
of electrical activity down its axon.

are necessarily gross idealisations of real networks

of neurones.

Components of a neuron

The synapse

The neuron model

An engineering approach
A simple neuron

An artificial neuron is a device with many inputs and one

output. The neuron has two modes of operation; the
training mode and the using mode. In the training mode,
the neuron can be trained to fire (or not), for particular
From Human Neurones to Artificial
Neurones input patterns. In the using mode, when a taught input
pattern is detected at the input, its associated output
We conduct these neural networks by first trying
becomes the current output. If the input pattern does not
to deduce the essential features of neurones and
belong in the taught list of input patterns, the firing rule is
their interconnections. We then typically program
used to determine whether to fire or not.
a computer to simulate these features. However
because our knowledge of neurones is incomplete
and our computing power is limited, our models
 the collection cause the node to fire if, on on
comparison, they have more input elements in common
with the 'nearest' pattern in the 1-taught set than with the
'nearest' pattern in the 0-taught set. If there is a tie, then
the pattern remains in the undefined state.

For example, a 3-input neuron is taught to output 1 when

the input (X1,X2 and X3) is 111 or 101 and to output 0
when the input is 000 or 001. Then, before applying the
firing rule, the truth table is;

A simple neuron

X1: 0 0 0 0 1 1 1 1
X2: 0 0 1 1 0 0 1 1
X3: 0 1 0 1 0 1 0 1
Firing rules

OUT: 0 0 0/1 0/1 0/1 1 0/1 1

The firing rule is an important concept in
neural networks and accounts for their high
flexibility. A firing rule determines how one
calculates whether a neuron should fire for
any input pattern. It relates to all the input As an example of the way the firing rule is applied,
patterns, not only the ones on which the node take the pattern 010. It differs from 000 in 1 element,
was trained. from 001 in 2 elements, from 101 in 3 elements and
from 111 in 2 elements. Therefore, the 'nearest'
A simple firing rule can be implemented by
pattern is 000 which belongs in the 0-taught set. Thus
using Hamming distance technique. The rule
the firing rule requires that the neuron should not fire
goes as follows:
when the input is 001. On the other hand, 011 is
equally distant from two taught patterns that have
Take a collection of training patterns for a node,
some of which cause it to fire (the 1-taught set different
of outputs and thus the output stays
patterns) and others which prevent it from doing undefined (0/1).
so (the 0-taught set). Then the patterns
 By applying the firing in every column the
following truth table is obtained;

X1: 0 0 0 0 1 1 1 1
X2: 0 0 1 1 0 0 1 1
X3: 0 1 0 1 0 1 0 1

OUT: 0 0 0 0/1 0/1 1 1 1

Figure 1.

For example:
The difference between the two truth tables is The network of figure 1 is trained to recognise the
called the generalization of the neuron. patterns T and H. The associated patterns are all
Therefore the firing rule gives the neuron a black and all white respectively as shown below
sense of similarity and enables it to respond
'sensibly' to patterns not seen during training.

Pattern Recognition - an example

An important application of neural networks is pattern

recognition. Pattern recognition can be implemented
a feed-forward (figure 1) neural network that has been trained
If we represent black squares with 0 and
accordingly. During training, the network is trained to
white squares with 1 then the truth tables for
associate outputs within put patterns. When the network is
the 3 neurones after generalization are;
used, it identifies the input pattern and tries to output
the associated output pattern. The power of neural X11: 0 0 0 0 1 1 1 1
networks comes to life when a pattern that has no X12: 0 0 1 1 0 0 1 1
output associated with it, is given as an input. In this X13: 0 1 0 1 0 1 0 1
case, the network gives the output that corresponds to a taught
inp out pattern that is least different from the given patOUT: 0 0 1 1 0 0 1 1

pat tern. Top neuron

 X21: 0 0 0 0 1 1 1 1
X22: 0 0 1 1 0 0 1 1
X23: 0 1 0 1 0 1 0 1

OUT: 1 0/1 1 0/1 0/1 0 0/1 0

Here also, it is obvious that the output should be

Middle neuron
all whites since the input pattern is almost the

X21: 0 0 0 0 1 1 1 1 same as the 'H' pattern.

X22: 0 0 1 1 0 0 1 1
X23: 0 1 0 1 0 1 0 1

OUT: 1 0 1 1 0 0 1 0

Bottom neuron

Here, the top row is 2 errors away from the a T and

3 from an H. So the top output is black. The middle
row is 1 error away from both T and H so the output
is random. The bottom row is 1 error away from T
and 2 away from H. Therefore the output is black.
The total output of the network is still in favour of
From the tables it can be seen the
the T shape.
following associasions can be extracted:

Architecture of neural
networks :
Feed-forward networks
In this case, it is obvious that the output should
Feed-forward ANNs (figure 1) allow signals to travel
be all blacks since the input pattern is almost the
one way only; from input to output. There is no
same as the 'T' pattern.
feedback (loops) i.e. the output of any layer does not
affect that same layer. Feed-forward ANNs tend to be
straight forward networks that associate inputs
to denote feedback connections in
withOutputs. They are extensively used in patternsingle-layer organizations.
recognition. This type of organisation is also referred
to as bottom-up or top-down.

Figure 1 An example of a simple feed

forward network

Feedback networks

Feedback networks (figure 1) can have

signals traveling in both directions by
An example of a complicated network
introducing loops in the network. Feedback
networks are very powerful and can get
extremely complicated. Feedback networks are
dynamic; their 'state' is changing continuously
until they reach an equilibrium point. They
remain at the equilibrium point until
the input changes and a new Applications of neural
equilibrium needs to be found.
networks
Feedback architectures are also
referred to as interactive or recurrent,
Neural Networks in Practice
although the latter term is often used
Since neural networks are best at identifying extensive application to biomedical systems
patterns or trends in data, they are well in the next few years.
suited for prediction or forecasting needs
including:

sales forecasting Modelling and Diagnosing the

Cardiovascular System
industrial process control
 Neural Networks are used experimentally to
customer research
model the human cardiovascular system. Diagnosis

data validation can be achieved by building a model of the

cardiovascular system of an individual and
risk management comparing it with the real time physiological
measurements taken from the patient.
ANN are also used in the following specific
paradigms: Electronic noses

recognition of speakers in communications; ANNs are used experimentally to implement

diagnosis of hepatitis; recovery of electronic noses. Electronic noses have
telecommunications from faulty software; several potential applications in telemedicine.
interpretation of multimeaning Chinese Telemedicine is the practice of medicine over
words; undersea mine detection; texture long distances via a communication link. The
analysis; three-dimensional object electronic nose would identify odours in the
recognition; hand-written word recognition; remote surgical environment. These identified
and facial recognition. odors would then be electronically transmitted
to another site where an door generation
system would recreate them.

Neural networks in medicine

 Artificial Neural Networks (ANN) Because the sense of smell can be an

are currently a 'hot' research area in medicine important sense to the surgeon, telesmell
and it is believed that they will receive would enhance telepresent surgery.
 Neural Networks in business The computing world has a lot to gain fron neural
networks. Their ability to learn by example makes
 Business is a diverted field them very flexible and powerful. Furthermore there
with several general areas of is no need to devise an algorithm in order to
specialisation such as accounting or perform a specific task; i.e. there is no need to
financial analysis. Almost any neural understand the internal mechanisms of that task.
network application would fit into one They are also very well suited for real time systems
business area or financial analysis. because of their fast response and computational
times which are due to their parallel architecture.
 There is also a strong potential
for using neural networks for database Neural networks also contribute to other areas of

mining, that is, searching for patterns research such as neurology and psychology.

implicit within the explicitly stored They are regularly used to model parts of living

information in databases organisms and to investigate the internal

mechanisms of the brain.

Marketing

 There is a marketing
Finally, I would like to state that even though
application which has been integrated
neural networks have a huge potential we will
with a neural network system. The
only get the best of them when they are
Airline Marketing Tactician (a trademark
intergrated with computing, AI, fuzzy logic and
abbreviated as AMT) is a computer
related subjects.
system made of various intelligent
technologies including expert systems. A
feedforward neural network is integrated
with the AMT and was trained using
back-propagation to assist the marketing
control of airline seat allocations References:
Conclusion
1. An introduction to neural computing.
Aleksander, I. and Morton, H. 2nd
edition
2. Neural Networks at Pacific Northwest National
Laboratory
http://www.emsl.pnl.gov:2080/docs/cie/neural/ne
ural.homepage.html
3. Industrial Applications of Neural Networks
(research reports Esprit, I.F.Croall, J.P.Mason)
4. A Novel Approach to Modelling
and Diagnosing the Cardiovascular
System
http://www.emsl.pnl.gov:2080/doc
s/cie/neural/papers2/keller.wcnn95
.abs.html
5. Artificial Neural Networks in
Medicine
http://www.emsl.pnl.gov:2080/doc
s/cie/techbrief/NN.techbrief.ht
6. Neural Networks by Eric Davalo
and Patrick Naim
7. Learning internal representations
by error propagation by
Rumelhart, Hinton and Williams
(1986).