Artificial Neural Network
Artificial Neural Network
Lecture 4
2
Agenda
▪ Introduction
▪ Understanding the brain
▪ Artificial Neural Network
▪ Architecture of ANN
▪ How ANN Works
▪ Activation Function
▪ Backpropagation Algorithm
▪ Types of ANN
▪ The Perceptron
▪ Multilayer Perceptron
▪ Back propagation algorithm
▪ Nonlinear Regression
3
Introduction: ANN
▪“...a computing system made up of a number of simple,
highly interconnected processing elements, which process
information by their dynamic state response to external
inputs.“ … Dr. Robert Hecht-Nielsen
each neuron performs a dot product with the input and its weights, adds the bias
and applies the non-linearity (or activation function), in this case the
1
sigmoid 𝜎 𝑥 =
1 + 𝑒𝑥𝑝 −𝑥
22
ANN: How it works?...
▪Here’s how it works:
6. The model applies a loss function to the output layer to
deliver the output.
▪ a way to quantify how “good” it’s doing so that it can try to do
“better”
Final hidden
state of the
RNN
ANN Types: RNN…
▪ Feed-forward NN have no memory
of the input they receive and are
bad at predicting what’s coming
next.
▪ Only considers the current
input, it has no notion of order
in time.
▪ Simply can’t remember
anything about what happened
in the past except its training.
▪ Used for: speech recognition,
language modeling,
translation, image
captioning…
50
ANN: LSTM
51
C0 - Personal Information
54