 BACK PROPAGATION NEURAL NETWORKS

BACKPROPAGATION

 This allow you to perform a “backward pass”, which helps tune the weights of the inputs.

 Backpropagation performs iterative backward passes which attempt to minimize the “loss”,
or the difference between the known correct prediction and the actual model prediction.

 With each backward pass, the weights move towards an optimum that minimizes the loss
function and results in the most accurate prediction

 Back Propagation Neural (BPN) is a multilayer neural network consisting of the input layer,
at least one hidden layer and output layer.

 Back propagating takes place in the network by calculating the error at the output layer and
comparing the target output and the actual output which will be propagated back towards
the input layer

Neural network or deep learning model training occurs in six stages:

 Initialization: initial weights are applied to all the neurons.

 Forward propagation: the inputs from a training set are passed through the neural network
and an output is computed.

 Error function: because we are working with a training set, the correct output is known. An
error function is defined, which captures the delta between the correct output and the
actual output of the model, given the current model weights (in other words, “how far off” is
the model from the correct result).

 Backpropagation: the objective of backpropagation is to change the weights for the

neurons, in order to bring the error function to a minimum.

 Weight update: weights are changed to the optimal values according to the results of the
backpropagation algorithm.

 Iterate until convergence: because the weights are updated, a small delta step at a time,
several iterations are required in order for the network to learn. After each iteration, the
gradient descent force updates the weights towards less and less global loss function. The
amount of iterations needed to converge depends on the learning rate, the network meta-
parameters, and the optimization method used.

KOHONEN SELF-ORGANIZING MAPS

 Self-organizing neural networks are used to cluster input patterns into groups of similar
patterns.

 They're called "maps" because they assume a topological structure among their cluster
units; effectively mapping weights to input data.

 The Kohonen network is probably the best example, because it's simple, yet introduces the
concepts of self-organization and unsupervised learning easily.

 Each weight is representative of a certain input. Input patterns are shown to all neurons
simultaneously
  In SOM unsupervised learning, the weights of a neural network is modified without
specifying the desired output for any input patterns

 The advantage is that it allows the network to find its own solution, making it more efficient
with pattern association

 The disadvantage is that other programs or users have to figure out how to interpret the
output.

 The structure of a self-organizing map involves m cluster units, arranged in either a one- or
two-dimensional array, with vectors of n input signals.

 The weight vectors define each cluster.

 Input patterns are compared to each cluster, and associated with the cluster it best matches.

 The comparison is usually based on the square of the minimum Euclidean distance.

 When a best match is found, the associated cluster gets its weights and its neighboring units
updated.

 Weight vectors are arranged into lines or various grid structures.

 Some neighborhoods closer to the ends or edges will have fewer weights, because the
algorithm doesn't wrap around.

INFERENCE SYSTEM

 Fuzzy Inference System is the key unit of a fuzzy logic system having decision making as its
primary work.

 It uses the “IF…THEN” rules along with connectors “OR” or “AND” for drawing essential
decision rules.

CHARACTERISTICS OF FUZZY INFERENCE SYSTEM

 The following are some characteristics of FIS −

 The output from FIS is always a fuzzy set irrespective of its input which can be fuzzy
or crisp.

 It is necessary to have fuzzy output when it is used as a controller.

 A defuzzification unit would be there with FIS to convert fuzzy variables into crisp
variables.

FUNCTIONAL BLOCKS OF FIS

 The following five functional blocks will help you understand the construction of FIS −

 Rule Base − It contains fuzzy IF-THEN rules.

 Database − It defines the membership functions of fuzzy sets used in fuzzy rules.
  Decision-making Unit − It performs operation on rules.

 Fuzzification Interface Unit − It converts the crisp quantities into fuzzy quantities.

 Defuzzification Interface Unit − It converts the fuzzy quantities into crisp quantities.
The block diagram of fuzzy interference system is presented below

 Diagram present

WORKING OF FIS

 The working of the FIS consists of the following steps −

 A fuzzification unit supports the application of numerous fuzzification methods, and

converts the crisp input into fuzzy input.

 A knowledge base - collection of rule base and database is formed upon the
conversion of crisp input into fuzzy input.

 The defuzzification unit fuzzy input is finally converted into crisp output.

FUZZY QUERY SYSTEM

 A fuzzy query system is an interface to users to get information from the database using
(quasi) natural language sentences.

 Many fuzzy query implementations have been proposed, resulting in slightly different
languages.

 Although there are some variations according to the particularities of different

implementations, the answer to a fuzzy query sentence is generally a list of records, ranked
by the degree of matching

QUANTIFICATION

 In modelling natural language statements, quantified statements play an important role.

 It means that NL heavily depends on quantifying construction which often includes fuzzy
concepts like “almost all”, “many”, etc.

 The following are a few examples of quantifying propositions −

 Every student passed the exam.

 Every sport car is expensive.

 Many students passed the exam.

 Many sports cars are expensive.

 In the above examples, the quantifiers “Every” and “Many” are applied to the crisp
restrictions “students” as well as crisp scope “(person who) passed the exam” and “cars” as
well as crisp scope” sports”.

 Fuzzy logic is applied with great success in various control application.

  Almost all the consumer products have fuzzy control. Some of the examples include
controlling your room temperature with the help of air-conditioner, anti-braking system
used in vehicles, control on traffic lights, washing machines, large economic systems, etc

WHY USE FUZZY LOGIC IN CONTROL SYSTEMS

 A control system is an arrangement of physical components designed to alter another

physical system so that this system exhibits certain desired characteristics. The following are
some reasons of using Fuzzy Logic in Control Systems −

 While applying traditional control, one needs to know about the model and the
objective function formulated in precise terms. This makes it very difficult to apply in
many cases.

 By applying fuzzy logic for control we can utilize the human expertise and experience
for designing a controller.

 The fuzzy control rules, basically the IF-THEN rules, can be best utilized in designing a
controller

ASSUMPTIONS IN FUZZY LOGIC CONTROL (FLC) DESIGN

 While designing fuzzy control system, the following six basic assumptions should be made −

 The plant is observable and controllable − It must be assumed that the input,
output as well as state variables are available for observation and controlling
purpose.

 Existence of a knowledge body − It must be assumed that there exist a knowledge

body having linguistic rules and a set of input-output data set from which rules can
be extracted.

 Existence of solution − It must be assumed that there exists a solution.

 ‘Good enough’ solution is enough − The control engineering must look for ‘good
enough’ solution rather than an optimum one.

 Range of precision − Fuzzy logic controller must be designed within an acceptable

range of precision.

 Issues regarding stability and optimality − The issues of stability and optimality
must be open in designing Fuzzy logic controller rather than addressed explicitly

MAJOR COMPONENTS OF FLC

 The Following are the major components of the FLC as shown in the above figure −

 Fuzzifier − The role of fuzzifier is to convert the crisp input values into fuzzy values.

 Fuzzy Knowledge Base − It stores the knowledge about all the input-output fuzzy
relationships. It also has the membership function which defines the input variables
to the fuzzy rule base and the output variables to the plant under control.

 Fuzzy Rule Base − It stores the knowledge about the operation of the process of
domain.
  Inference Engine − It acts as a kernel of any FLC. Basically it simulates human
decisions by performing approximate reasoning.

 Defuzzifier − The role of defuzzifier is to convert the fuzzy values into crisp values
getting from fuzzy inference engine.

STEPS IN DESIGNING FLC

 The following are the steps involved in designing FLC −

 Identification of variables − Here, the input, output and state variables must be
identified of the plant which is under consideration.

 Fuzzy subset configuration − The universe of information is divided into number of

fuzzy subsets and each subset is assigned a linguistic label. Always make sure that
these fuzzy subsets include all the elements of universe.

 Obtaining membership function − Now obtain the membership function for each
fuzzy subset that we get in the above step.

 Fuzzy rule base configuration − Now formulate the fuzzy rule base by assigning
relationship between fuzzy input and output.

 Fuzzification − The fuzzification process is initiated in this step.

 Combining fuzzy outputs − By applying fuzzy approximate reasoning, locate the

fuzzy output and merge them.

 Defuzzification − Finally, initiate defuzzification process to form a crisp output.

ADVANTAGES OF FUZZY LOGIC CONTROL

 Let us now discuss the advantages of Fuzzy Logic Control.

 Cheaper − Developing a FLC is comparatively cheaper than developing model based

or other controller in terms of performance.

 Robust − FLCs are more robust than PID controllers because of their capability to
cover a huge range of operating conditions.

 Customizable − FLCs are customizable.

 Emulate human deductive thinking − Basically FLC is designed to emulate human

deductive thinking, the process people use to infer conclusion from what they know.

 Reliability − FLC is more reliable than conventional control system.

 Efficiency − Fuzzy logic provides more efficiency when applied in control system.

DISADVANTAGES OF FUZZY LOGIC CONTROL

 We will now discuss what are the disadvantages of Fuzzy Logic Control.

 Requires lots of data − FLC needs lots of data to be applied.

 Useful in case of moderate historical data − FLC is not useful for programs much
smaller or larger than historical data.
  Needs high human expertise − This is one drawback as the accuracy of the system
depends on the knowledge and expertise of human beings.

Needs regular updating of rules − The rules must be updated with time.

ADAPTIVE FUZZY CONTROLLER

 Adaptive Fuzzy Controller is designed with some adjustable parameters along with an
embedded mechanism for adjusting them.

 Adaptive controller has been used for improving the performance of controller.

Basic Steps for Implementing Adaptive Algorithm

 Let us now discuss the basic steps for implementing adaptive algorithm.

 Collection of observable data − The observable data is collected to calculate the

performance of controller.

 Adjustment of controller parameters − Now with the help of controller

performance, calculation of adjustment of controller parameters would be done.

 Improvement in performance of controller − In this step, the controller parameters

are adjusted to improve the performance of controller.

OPERATIONAL CONCEPTS

 Design of a controller is based on an assumed mathematical model that resembles a real

system.

 The error between actual system and its mathematical representation is calculated and if it
is relatively insignificant then the model is assumed to work effectively.

 A threshold constant that sets a boundary for the effectiveness of a controller, also exists.

 The control input is fed into both the real system and mathematical model

 Here we assume that x(t) is the output of the real system and y(t) is the output of the
mathematical model

 Then the error ϵ (t) can be calculated as follows: ϵ(t)=x(t)-y(t)

 Here, x desired is the output we want from the system and µ (t) is the output coming from
controller and going to both real as well as mathematical model.

 The following diagram shows how the error function is tracked between output of a real
system and Mathematical model −

PARAMETERS FOR SELECTING AN ADAPTIVE FUZZY CONTROLLER

 The following parameters need to be considered for selecting an adaptive fuzzy controller −

 Can the system be approximated entirely by a fuzzy model?

 If a system can be approximated entirely by a fuzzy model, are the parameters of

this fuzzy model readily available or must they be determined online?
  If a system cannot be approximated entirely by a fuzzy model, can it be
approximated piecewise by a set of fuzzy model?

 If a system can be approximated by a set of fuzzy models, are these models having
the same format with different parameters or are they having different formats?

 If a system can be approximated by a set of fuzzy models having the same format,
each with a different set of parameters, are these parameter sets readily available or
must they be determined online?

FUZZINESS IN NEURAL NETWORKS

 Artificial neural network (ANN) is a network of efficient computing systems. The central
theme of which is borrowed from the analogy of biological neural networks.

 ANNs are also named as “artificial neural systems,” “parallel distributed processing
systems,” “connectionist systems.”

 ANN acquires large collection of units that are interconnected in some pattern to allow
communications between units.

 These units, also referred to as nodes or neurons, are simple processors which operate in
parallel.

 Every neuron is connected with other neuron through a connection link. Each connection
link is associated with a weight having the information about the input signal.

 This is the most useful information for neurons to solve a particular problem because the
weight usually inhibits the signal that is being communicated

 WHY USE FUZZY LOGIC IN NEURAL NETWORK

 The following are some reasons for using fuzzy logic in neural networks

 Fuzzy logic is largely used to define the weights, from fuzzy sets, in neural networks.

 When crisp values are not possible to apply, then fuzzy values are used

 We have already studied that training and learning help neural networks perform
better in unexpected situations. At that time, fuzzy values would be more applicable
than crisp values.

 When we use fuzzy logic in neural networks, the values must not be crisp and the
processing can be done in parallel.

FUZZY COGNITIVE MAP

 It is a form of fuzziness in neural networks. Basically FCM is like a dynamic state machine
with fuzzy states (not just 1 or 0).

DIFFICULTY IN USING FUZZY LOGIC IN NEURAL NETWORKS

 Despite having numerous advantages, there is also some difficulty while using fuzzy logic in
neural networks.

 The difficulty is related with membership rules, the need to build fuzzy system, because it is
sometimes complicated to deduce it with the given set of complex data.
NEURAL-TRAINED FUZZY LOGIC

 The reverse relationship between neural network and fuzzy logic, i.e., neural network used
to train fuzzy logic is also a good area of study. The following are two major reasons to build
neural-trained fuzzy logic −

 New patterns of data can be learned easily with the help of neural networks hence,
it can be used to pre-process data in fuzzy systems.

 Neural network, because of its capability to learn new relationship with new input
data, can be used to refine fuzzy rules to create fuzzy adaptive system.

EXAMPLES OF NEURAL-TRAINED FUZZY SYSTEM

 Neural-Trained Fuzzy systems are being used in many commercial applications. Let us now
see a few examples where Neural-Trained Fuzzy system is applied −

 The Laboratory for International Fuzzy Engineering Research (LIFE) in Yokohama,

Japan has a back-propagation neural network that derives fuzzy rules. This system
has been successfully applied to foreign-exchange trade system with approximately
5000 fuzzy rules.

 Ford Motor Company has developed trainable fuzzy systems for automobile idle-
speed control.

 NeuFuz, software product of National Semiconductor Corporation, supports the

generation of fuzzy rules with a neural network for control applications.

 AEG Corporation of Germany uses neural-trained fuzzy control system for its water –
and energy conserving machine. It is having total of 157 fuzzy rules.

FUZZY LOGIC - APPLICATIONS

AEROSPACE

In aerospace, fuzzy logic is used in the following areas −

 Altitude control of spacecraft

 Satellite altitude control

 Flow and mixture regulation in aircraft de-icing vehicles

AUTOMOTIVE

In automotive, fuzzy logic is used in the following areas −

 Trainable fuzzy systems for idle speed control

 Shift scheduling method for automatic transmission

 Intelligent highway systems

 Traffic control

 Improving efficiency of automatic transmissions

BUSINESS
In business, fuzzy logic is used in the following areas −

 Decision-making support systems

 Personnel evaluation in a large company

DEFENCE

In defence, fuzzy logic is used in the following areas −

 Underwater target recognition

 Automatic target recognition of thermal infrared images

 Naval decision support aids

 Control of a hypervelocity interceptor

 Fuzzy set modelling of NATO decision making

ELECTRONICS

In electronics, fuzzy logic is used in the following areas −

 Control of automatic exposure in video cameras

 Humidity in a clean room

 Air conditioning systems

 Washing machine timing

 Microwave ovens

 Vacuum cleaners

FINANCE

In the finance field, fuzzy logic is used in the following areas −

 Banknote transfer control

 Fund management

 Stock market predictions

INDUSTRIAL SECTOR

In industrial, fuzzy logic is used in following areas −

 Cement kiln controls heat exchanger control

 Activated sludge wastewater treatment process control

 Water purification plant control

 Quantitative pattern analysis for industrial quality assurance

 Control of constraint satisfaction problems in structural design

 Control of water purification plants

MANUFACTURING

In the manufacturing industry, fuzzy logic is used in following areas −

 Optimization of cheese production

 Optimization of milk production

MARINE

In the marine field, fuzzy logic is used in the following areas −

 Autopilot for ships

 Optimal route selection

 Control of autonomous underwater vehicles

 Ship steering

MEDICAL

In the medical field, fuzzy logic is used in the following areas −

 Medical diagnostic support system

 Control of arterial pressure during anaesthesia

 Multivariable control of anaesthesia

 Modelling of neuropathological findings in Alzheimer's patients

 Radiology diagnoses

 Fuzzy inference diagnosis of diabetes and prostate cancer

SECURITIES

In securities, fuzzy logic is used in following areas −

 Decision systems for securities trading

 Various security appliances

TRANSPORTATION

In transportation, fuzzy logic is used in the following areas −

 Automatic underground train operation

 Train schedule control

 Railway acceleration

 Braking and stopping

PATTERN RECOGNITION AND CLASSIFICATION

In Pattern Recognition and Classification, fuzzy logic is used in the following areas −

 Fuzzy logic based speech recognition

  Fuzzy logic based

 Handwriting recognition

 Fuzzy logic based facial characteristic analysis

 Command analysis

 Fuzzy image search

PSYCHOLOGY

 In Psychology, fuzzy logic is used in following areas −

 Fuzzy logic based analysis of human behaviour

 Criminal investigation and prevention based on fuzzy logic reasoning