Course-Topics-for-EEE-511
Course-Topics-for-EEE-511
PREREQUISITES:
Signals and systems, random signal analysis, linear systems, or instructor approval
COURSE DESCRIPTION
This course covers the principles of artificial neural networks (ANN), collective computational
phenomena emerging from simple interconnected elements. Biological neural motivation will be
introduced. However, the course emphasis is on "artificial" with a focus on engineering aspects of the
field. Our approach is based on engineering systems theory and statistical machine learning. We
address how do we make a network of neuron-like elements learn from a set of data that is
representative of the problem, to solve complex problems in life, such as pattern classification
(speech recognition, and target recognition), function approximation and system identification, control
of a robot or a plant, or anything else among a myriad of possible applications. Our purpose is to
obtain a coherent overview of the field, an ability to find and digest deeper knowledge in any
particular subfield, and most importantly, ability to apply an appropriate type of an ANN model to a
given problem.
COURSE TOPICS
Introduction of ANN, inspirations from biological neural networks, neuron models, relations to
other disciplines, applications, motivating demonstrations
Perceptrons, historical significance in learning, convergence theorem, perceptron vs. statistical
pattern recognition
Single layer networks and unconstrained optimization methods, Single layer networks: LMS,
LMS vs. Wiener filter
Multilayer perceptrons and back-propagation
Representational capabilities of MLPs, training issues
Generalization, overfitting with multilayer networks
Regularization, Cross-Validation
Optimization methods for MLPs
Regularization theory
Radial basis function networks, Introduction, Exact interpolation
Radial basis function networks, Regularization, Model selection
Radial basis function networks, Learning basis functions
Support vector machines, introduction
Support vector machines: Separable and nonseparable cases
Support vector machines: More on VC-dimension, the kernel trick
Support vector machines: Regression, examples
The self-organizing map, Introduction, The basic algorithm
The self-organizing map, Some example applications, Analysis of the algorithm
The self-organizing map, Variants of SOMs
The self-organizing map: Tree-Structurd SOMs, Applications in optimization.
Learning vector quantization, principal-component analysis
Information theoretic learning
Stochastic optimization methods
Dynamic Programming and Reinforcement learning
Temporal processing with feedforward networks
Filtering of stochastic processes
Neurodynamics and Recurrent networks