Scribd
Upload
51 views

Training and Testing Neural Networks

1) The document discusses training and testing neural networks. It describes parameters that control the training process like learning rate and momentum. 2) It explains that a neural network is trained when it meets specified accuracy criteria on the task, like classification or forecasting. Training continues until connection weights stabilize. 3) An iterative process is described where the model, data representation, and architecture are refined to avoid overfitting and ensure the network generalizes well to new examples.

Uploaded by

Brij Mohan Singh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
51 views

Training and Testing Neural Networks

1) The document discusses training and testing neural networks. It describes parameters that control the training process like learning rate and momentum. 2) It explains that a neural network is trained when it meets specified accuracy criteria on the task, like classification or forecasting. Training continues until connection weights stabilize. 3) An iterative process is described where the model, data representation, and architecture are refined to avoid overfitting and ensure the network generalizes well to new examples.

Uploaded by

Brij Mohan Singh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 28

Training and Testing

Neural Networks

Contents
Introduction
When Is the Neural Network Trained?
Controlling the Training Process with Learning
Parameters
Iterative Development Process
Avoiding Over-training
Automating the Process

Introduction (1)
Training a neural network
perform a specific processing function
1) parameter?
2) how used to control the training process
3) management of the training data - training process ?

Development Process
1) Data preparation
2) neural network model & architecture
3) train the neural network
neural network function
Application
trained

Introduction (2)
Learning Parameters for Neural Network
Disciplined approach to iterative neural network
development

Introduction (3)

When Is the Neural Network Trained?


When the network is trained?
the type of neural network
the function performing
classification
clustering data
build a model or time-series forecast

the acceptance criteria


meets the specified accuracy
the connection weights are locked
cannot be adjusted

When Is the Neural Network Trained?


Classification (1)
Measure of success : percentage of correct
classification
incorrect classification
no classification : unknown, undecided

threshold limit

When Is the Neural Network Trained?


Classification
(2)
confusion matrix
: possible output categories and the corresponding
percentage of correct and incorrect classifications

Category A Category B Category C


Category A 0.6

0.25

0.15

Category B 0.25

0.45

0.3

Category C 0.15

0.3

0.55

When Is the Neural Network Trained?


Clustering (1)
Output a of clustering network
open to analysis by the user

Training regimen is determined:


the number of times the data is presented to the neural
network
how fast the learning rate and the neighborhood decay

Adaptive resonance network training (ART)


vigilance training parameter
learn rate

When Is the Neural Network Trained?


Clustering (2)
Lock the ART network weights
disadvantage : online learning

ART network are sensitive to the order of the


training data

When Is the Neural Network Trained?


Modeling (1)
Modeling or regression problems
Usual Error measure
RMS(Root Square Error)

Measure of Prediction accuracy


average
MSE(Mean Square Error)
RMS(Root Square Error)

The Expected behavior


RMS error , stable
minimum

When Is the Neural Network Trained?


Modeling (2)

When Is the Neural Network Trained?


Modeling (3)

network fall into a local minima
the prediction error doesnt fall
oscillating up and down

reset(randomize) weight and start again


training parameter
data representation
model architecture

When Is the Neural Network Trained?


Forecasting

Forecasting (1)

prediction problem
RMS(Root Square Error)
visualize : time plot of the actual and desired network
output

Time-series forecasting
long-term trend
influenced by cyclical factor etc.

random component
variability and uncertainty

neural network are excellent tools for modeling complex


time-series problems
recurrent neural network : nonlinear dynamic systems
no self-feedback loop & no hidden neurons

When Is the Neural Network Trained?

Forecasting (2)

Controlling the Training Process with


Learning Parameters (1)
Learning Parameters depends on
Type of learning algorithm
Type of neural network

Controlling the Training Process with


Learning Parameters (2)
- Supervised training
Pattern
Pattern
Neural Network

Prediction
Prediction
Desired
Desired
Output
Output

1) How the error is computed


2) How big a step we take when adjusting the
connection weights

Controlling the Training Process with


Learning Parameters (3)
- Supervised training
Learning rate
magnitude of the change when adjusting the connection
weights
the current training pattern and desired output
large rate
giant oscillations

small rate

to learn the major features of the problem


generalize to patterns

Controlling the Training Process with


Learning Parameters (4)
- Supervised training
Momentum
filter out high-frequency changes in the weight values
oscillating around a set values
Error

Error tolerance

how close is close enough


0.1

net input must be quite large?

Controlling the Training Process with


Learning Parameters (5)
-Unsupervised learning

Parameter
selection for the number of outputs
granularity of the segmentation
(clustering, segmentation)

learning parameters (architecture is set)


neighborhood parameter : Kohonen maps
vigilance parameter : ART

Controlling the Training Process with


Learning Parameters (6)
-Unsupervised learning

Neighborhood
the area around the winning unit, where the non-wining
units will also be modified
roughly half the size of maximum dimension of the
output layer
2 methods for controlling
square neighborhood function, linear decrease in the learning
rate
Gaussian shaped neighborhood, exponential decay of the
learning rate

the number of epochs parameter


important in keeping the locality of the topographic
amps

Controlling the Training Process with


Learning Parameters (7)
-Unsupervised learning

Vigilance
control how picky the neural network is going to be
when clustering data
discriminating when evaluating the differences between
two patterns
close-enough
Too-high Vigilance
use up all of the output units

Iterative Development Process (1)


Network convergence issues
fall quickly and then stays flat / reach the global
minima
oscillates up and down / trapped in a local minima

some random noise
reset the network weights and start all again
design decision

Iterative Development Process (2)

Iterative Development Process (3)


Model selection
inappropriate neural network model for the function to
perform
add hidden units or another layer of hidden units
strong temporal or time element embedded
recurrent back propagation
radial basis function network

Data representation
key parameter is not scaled or coded
key parameter is missing from the training data
experience

Iterative Development Process (4)


Model architecture
not converge : too complex for the architecture
some additional hidden units, good
adding many more?
Just, Memorize the training patterns

Keeping the hidden layers as this as possible, get the


best results

Avoiding Over-training
Over-training

pattern
cannot generalize
pattern
switch between training and testing data

Automating the Process


Automate the selection of the appropriate number
of hidden layers and hidden units

pruning out nodes and connections


genetic algorithms
opposite approach to pruning
the use of intelligent agents

You might also like