Scribd
Upload
14 views26 pages

3-Neural Network

Uploaded by

Nguyen Quoc Khanh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
14 views26 pages

3-Neural Network

Uploaded by

Nguyen Quoc Khanh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 26

Introduction to Deep Learning

Neural Network
Simple Neural Network Model
• Logistic regression model:

• Two steps:
(1) Linear sum:

(2) Apply sigmoid function:

2
Simple Neural Network Model

Flow chart of logistic regression model

3
Simple Neural Network Model

Flow chat of logistic regression model


• W0 is called bias coefficient, or free coefficient
• Sigmoid function is called activation function

4
General Neural Network Model

• Input layer and output


layer are required
• Hidden layers are
optional
• Total layers = # layers – 1
• Each circle is called one
node

5
General Neural Network Model

Each node in hidden layer and output layer:


• Is connected with all nodes with previous layer with the coefficents w
• Has a bias coefficient w0
• Follows two steps of linear sum and appliance of activation function
(sigmoid)

6
General Neural Network Model

The neural network has: 3 layers, 2 nodes in input layer, 3 nodes in


hidden layer 1, 3 nodes in hidden layer 2, 1 node in output layer

7
General Neural Network Model

Note: node 1 is not considered as a node since it is used to calculate


bias of the node in the next layer

8
General Neural Network Model
• Node i in layer l with bias bi(l) has 2 steps:

(1) Linear sum:

(2) Apply activation function:

9
General Neural Network Model
• At node 2 of layer 1, we have:

• At node 3 of layer 2, we have:

10
Feedforward
• Let call input layer x = a(0), with size 2*1, we have:

11
Feedforward
• Similarly, we have:

12
Feedforward

feedforward neural network

13
Loss function

• Gradient descent algorithm


• Step of derivative calculation of
coefficients of loss function is done with
the backpropagation algorithm
Will be taught in the next lecture

14
Logistic Regression vs. Neural Network

Problem: AND

x1 AND x2

15
Logistic Regression vs. Neural Network

Flow chart for the problem x1 AND x2

16
Logistic Regression vs. Neural Network

Flow chart for the problem NOT (x1 AND x2)

17
Logistic Regression vs. Neural Network

Separation line y = 1.5 - 1 * x1 - 1 * w2

18
Logistic Regression vs. Neural Network

Problem: OR

x1 OR x2

19
Logistic Regression vs. Neural Network

Flow chart for the problem x1 OR x2

20
Logistic Regression vs. Neural Network

Separation line for OR problem

21
Logistic Regression vs. Neural Network

Problem: XOR

x1 XOR x2

22
Logistic Regression vs. Neural Network

Cannot draw a separation line for XOR problem


à Cannot solve the XOR problem using logistic regression
à Neural network ???

23
Logistic Regression vs. Neural Network

Rewrite XOR problem:

24
Logistic Regression vs. Neural Network

Solve XOR problem with several logistic regression models

25
26

You might also like