![Perceptron Model of Neuron
Input signals:
• Continuous or discrete values fed from previous neurons
• Each input associated with a Weight
Integration Function:
• Usually a weighted summation function
• Threshold/Bias regulates result of Integration Function
• Output is called neuron net input
Activation/Transfer Function:
• Usually a non linear function
• Output interval [0,1] or [-1,1]
• Output values continuous or discrete
Neural
Networks
Modeling
26
-1
-0.5
0
0.5
1
1.5
-2 -1 0 1 2
Sigmoid Heaviside Linear](https://image.slidesharecdn.com/14-150402000544-conversion-gate01/85/mohsin-dalvi-artificial-neural-networks-presentation-25-320.jpg)
mohsin dalvi artificial neural networks presentation
- 1. SUBMITTED TO: SUBMITTED BY: DR. Y. M. PURI MOHSIN DALVI ASSOCIATE PROFESSOR 13MT07IND014 COURSE COORDINATOR FOR M. TECH (INDL. ENGG) AUTOMATION IN PRODUCTION SEM-II, SUMMER 2014 PRESENTATION ON ARTIFICIAL NEURAL NETWORKS VISVESVARAYA NATIONAL INSTITUTE OF TECHNOLOGY, NAGPUR 440010 DEPARTMENT OF MECHANICAL ENGINEERING
- 3. What are Neural Networks? An artificial neural network is a biologically inspired computational model that consists of processing elements (neurons) and connections between them, as well as of training and recall algorithms. The network is usually implemented using electronic components or simulated in software on a digital computer. Neural Networks attempt to bring computers a little closer to the brain's capabilities by imitating certain aspects of information processing in the brain, in a highly simplified way. Introduction 3
- 4. The Brain vs A ComputerIntroduction Brain Computer Processing Elements 1010 neurons 108 transistors Element Size 10-6 m 10-6 m Energy Use 30 W 30 W (CPU) Processing Speed 102 Hz 1012 Hz Style Of Computation Parallel, Distributed Serial, Centralized Energetic Efficiency 10-16 joules/opn/sec 10-6 joules/opn/sec Fault Tolerant Yes No Learns Yes A little 4
- 5. Characteristics of a Biological Brain Massively parallel, distributed information processing High degree of connectivity among basic units Connections get reorganized based on experience Performance degrades gracefully if some units are removed (i.e. some nerve cells die) Learning is constant and usually unsupervised Learning is based only on local information Introduction 5
- 6. The Biological Brain Neurons: Fundamental information-processing units of the brain. Neurons contain axons (the transmission lines) and dendrites, (the receptive zones). Electrical signal flows from dendrites to axon. Introduction 6
- 7. The Biological Brain Synapses are elementary structural and functional units that mediate the interactions between neurons. Synapse converts a presynaptic electrical signal into a chemical signal and then back into a postsynaptic electrical signal. During the early stage of development (first two years from birth), about 1 million synapses are formed per second. In an adult’s brain, a neuron is connected to around 10,000 other neurons by synapses. Introduction 7
- 8. Evolution of Neural Networks 1911 - Ramon y Cajal introduced the idea of neurons as structural constituents of the brain 1943 - McCulloch and Pitts apply Boolean algebra to nerve net behaviour 1948 - Donald Hebb postulates qualitative mechanism for learning at cellular level in brains 1957 - Rosenblatt develops ‘perceptron’ neurocomputer Between 1960’s & 1980’s - Almost no research in ANN Middle 80's - John Hopfield revives ANN Today - ANN one of the most active current areas of research Introduction 9
- 9. Characteristics of Neural NetworksIntroduction Universal Regression Systems - Modeling of a system with an unknown input-output relationship Learning - Network with "no knowledge“ can be trained with set of paired input-output data to give desired outputs for known inputs. Generalization - Produce best output according to learned examples if a different vector is input into network. Adaptivity - Adapt response to changes in surrounding environment 10
- 10. Characteristics of Neural NetworksIntroduction Nonlinearity - Cope with nonlinear data and environment Massive parallel processing - Many neurons fire simultaneously during data processing Fault Tolerance - Good response even if input data is slightly incorrect Robustness - Whole system can still perform well even if some neurons "go wrong" 11
- 12. Input Output Node Node Node Node Node Node Node Node Node Representation of Neural Networks Neural Networks Modeling Connections 13
- 13. OUTPUTS B A INPUTS 2 1 3 / * + - AND OR IF GOTO Conventional Computer Model Neural Networks Modeling 14
- 14. HIDDEN LAYER OUTPUT LAYER INPUT LAYER Neural Network As A Computer Model Neural Networks Modeling Connections Nodes 15
- 15. HIDDEN LAYER OUTPUT LAYER INPUT LAYER Directed Connections Neural Network As A Computer Model Neural Networks Modeling 16
- 16. HIDDEN LAYER OUTPUT LAYER INPUT LAYER Weighted Connections Neural Network As A Computer Model Neural Networks Modeling 17
- 17. HIDDEN LAYER INPUT LAYER A B C Effect of Weighted Connections Neural Networks Modeling 18 + =A B C EQUAL PROPORTIONS: R - 0 G - 255 B - 0 R - 255 G - 0 B - 0 R - 255 G - 255 B - 0 + =A B C WEIGHTED PROPORTIONS: R - 0 G - 127 B - 0 R - 255 G - 0 B - 0 R - 255 G - 127 B - 0
- 18. OUTPUT LAYER HIDDEN LAYER Example of Weighted Connections Neural Networks Modeling INPUT LAYER 19
- 19. Decision Making in Neural Networks Neural Networks Modeling HIDDEN LAYERINPUT LAYER OUTPUT LAYER DESIRED OUTPUTS ACCURACY DECISION: HOW TO UPDATE WEIGHTS TO REDUCE ERROR? ERROR 20
- 20. Thresholding in Biological Neural NetworkIntroduction 21
- 21. Reflex Action in Biological Neural NetworkIntroduction 22
- 22. INPUT LAYER A B C Decision Making in Neural Networks Neural Networks Modeling ACTIVATION DECISION: WHEN SHOULD THE NODE FIRE ITS OUTPUT? 23
- 23. Neural Networks Modeling Aspects Fundamental issues in modeling of an artificial neural network: • How to assign weights to the connections? • How to determine the neuron output threshold? Major steps in modeling an artificial neural network: • Model a single neuron • Establish a pattern of neuron interconnectivity • Implement a learning mechanism Neural Networks Modeling 24
- 24. Modeling an Artificial Neuron Neural Networks Modeling 25 Perceptron Model: Developed by Rosenblatt in 1957. Other neuron models are adaptations of perceptron model μk=Σwkjxj νk=μk-bk νk yk=φ(νk) yk θk Threshold Output Activation Function Integration Function --- x2 wk2 x1 wk1 xn wkn ---
- 25. Perceptron Model of Neuron Input signals: • Continuous or discrete values fed from previous neurons • Each input associated with a Weight Integration Function: • Usually a weighted summation function • Threshold/Bias regulates result of Integration Function • Output is called neuron net input Activation/Transfer Function: • Usually a non linear function • Output interval [0,1] or [-1,1] • Output values continuous or discrete Neural Networks Modeling 26 -1 -0.5 0 0.5 1 1.5 -2 -1 0 1 2 Sigmoid Heaviside Linear
- 26. Perceptron Model Example Neural Networks Modeling 27 Summing Function: μ = w1x1 + w2x2 + w3x3 = 3(0.2) + 1(0.4) + 2(0.1) = 1.2 Activation Function: 77.0 1 1 1 1 2.1 ee y w2 = 0.4 x2 = 1 x3 = 2 x1 = 3 y
- 27. Connecting Neurons to Build Networks Neural Networks Modeling 28Recurrent Network Lattice Network Single Layer Feedforward Network Input Layer Output Layer Multi- Layer Feedforward Network Input Layer Output Layer Hidden Layer 1 Hidden Layer 2
- 28. Supervised Learning Unsupervised Learning Binary Valued Input • Hopfield Network • Boltzmann Machine • ART I Continuous Valued Input • Backpropagation • Percepteron • ART II • Self-Organising Feature Maps Learning Algorithms in Neural Networks Neural Networks Modeling 29
- 29. Compute Output Adjust Weights Stop Is Desired Output Achieved? Yes No Supervised Learning in Neural Networks Neural Networks Modeling 30
- 30. First Run of the Network Neural Networks Modeling HIDDEN LAYER OUTPUT LAYER INPUT LAYER 14 29 41 DESIRED OUTPUTS 35 45 17 ERROR 21 16 -24 31
- 31. ERROR 21 16 -24 Backpropagation of Output Error Neural Networks Modeling HIDDEN LAYER INPUT LAYER -11 19 3 19 -5 32 0 14 26 -12 -8 32
- 32. OUTPUT LAYER HIDDEN LAYER Second Run of the Network Neural Networks Modeling INPUT LAYER 20 32 29 DESIRED OUTPUTS 35 45 17 ERROR 15 13 -12 33
- 33. ERROR (RUN 1) 21 16 -24 Reduction of Output Error Neural Networks Modeling ERROR (RUN 2) 15 13 -12 ERROR (RUN N) 0 0.01 0 - - - - 34
- 34. After Many Runs of the Network Neural Networks Modeling HIDDEN LAYER INPUT LAYER OUTPUT LAYER 35 45 17 35
- 35. Collect Data Separate into Training and Test Sets Define Network Structure Select Learning Algorithm Select Parameters, Values, Initialise Weights Transform Data into Network Inputs Start Training and Determine and Revise Weights Stop and Test Implementation: Use the Network with New Cases Get More, Better Data Reseparate Redefine Structure Select Another Algorithm Reset Reset Developing a Neural Network Neural Networks Modeling 36
- 37. What are neural networks used for? Neural Networks Applications Classification: Assigning each object to a known specific class Clustering: Grouping together objects similar to each other Pattern Association: Presenting of an input sample triggers the generation of specific output pattern Function approximation: Constructing a function generating almost the same outputs from input data as the modeled process Optimization: Optimizing function values subject to constraints Forecasting: Predicting future events on the basis of past history Control: Determining values for input variables to achieve desired values for output variables 38
- 38. Unknown Character Feature Recognition Neurons 0 1 2 3 4 5 6 7 8 9 Classifier Neurons Recognised Character ANN Feature Recognition (OCR Software) Applications of Neural Networks 39
- 39. Neural Networks in Manufacturing Neural Networks Applications Manufacturing process decision problem: Neural network enabled decision support system: Manufacturing System State Variables Process Settings Process OutcomesProcess Variables Trained Neural Network Training Algorithm Production Data Step 1: Train the Network Optimization Procedure Recommended Process Setting Current Values of State and Process Variables Step 2: Use the network to aid in decision making process 40 Trained Neural Network
- 40. Neural Networks in Manufacturing Neural Networks Applications Modeling and Design of Manufacturing Systems • Cell Formation for Agile and Flexible Manufacturing • Optimization and Simulation of manufacturing system • Forecasting and Cost Estimation • AGV path determination Modeling, Planning, and Scheduling of Manufacturing Processes • Production and Machine-scheduling • Kanban Determination • Resource queuing and scheduling • Economic order quantity 41
- 41. Neural Networks in Manufacturing Neural Networks Applications Monitoring and Control of Manufacturing Processes • Parameter Selection • Automated Process Control eg: pressing, rolling, welding, EDM, WEDM • Condition Monitoring for Machines and Tools • Robot part handling Quality Control, Quality Assurance, and Fault Diagnosis • Recognizing Handwritten Characters and Graphs • Visual Edge Detection • Pattern recognition • Fault Diagnosis and Troubleshooting 42
- 42. Neural Networks in Injection Moulding Neural Networks Applications 43
- 43. Neural Networks in Injection Moulding Neural Networks Applications 44
- 44. Neural Networks in Injection Moulding Neural Networks Applications Neural Network Configuration: 27-10-1 Correlation coefficient = 0.9254 RMSE = 1.3%-19% 45
- 45. Final Words Applications of Neural Networks “ Artificial neural networks are still far away from biological neural networks , but what we know today about artificial neural networks is sufficient to solve many problems that were previously unsolvable or inefficiently solvable at best. ” 46