Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural Networks | Simplilearn

Geolocation with Neural Network
How does a neural network recognize the location of an image?
Artificial Neural Network New image of a placeImages of millions of places Identifies the place

Geolocation with Neural Network
Pixels of millions of images Feed to Neural Network Recognizes landmarks,
landscapes, architectural
styles, road signs, etc
New image of a place Identifies the place
How does a neural network recognize the location of an image?

What’s in it for you?
Types of Neural Networks
Applications of Deep Learning
Working of Neural Network
Introduction to TensorFlow
Use case implementation using TensorFlow
What is Deep Learning?
What is a Neural Network?
Implementing Logic Gates using Perceptron
What is a Perceptron?
Why do we need Deep Learning?

What is Deep Learning?
Deep Learning is a subset of Machine Learning that has networks which are capable of learning from data
that is unstructured or unlabeled and works similar to the functioning of a human brain.
Artificial Intelligence
Machine
Learning
Deep
Learning
Ability of a machine to imitate
intelligent human behavior
Application of AI that allows a system
to automatically learn and improve
from experience
Application of Machine Learning that
uses complex algorithms and deep
neural nets to train a model

Why do we need Deep Learning?
Machine Learning works
only with large sets of
structured data, while
Deep Learning can work
with both structured and
unstructured data
Deep Learning
algorithms can perform
complex operations
easily while Machine
Learning Algorithms
cannot
Machine Learning
algorithms use labelled
sample data to extract
patterns, while Deep
Learning accepts large
volumes of data as
input, analyze the input
to extract features out
of an object
Performance of
Machine Learning
algorithms decreases
as the amount of data
increase, so to
maintain the
performance of the
model we need Deep
Learning
Works with unstructured data Handle complex operations Feature Extraction Achieve best performance

Neuron
X1
X2
Xn
Y
Output
w1
w2
w n
Input 1
Input 2
Input n
Neural Networks are modeled after biological neural networks that allow computers to learn and interact
like humans do. It has interconnected neurons with dendrites that receive inputs, and based on those
inputs, it produces an electric signal i.e. output through the axon.
Biological Neuron

Neuron
X1
X2
Xn
Y
Output
w1
w2
w n
Input 1
Input 2
Input n
Neural Network has interconnected neurons that receive some inputs, processes those inputs in layers to
produce the output.
Artificial Neuron

Biological Neurons Vs Artificial Neuron
Dendrites -------> Inputs
Neuron
X1
X2
Xn
Y
Output
w1
w2
w n
Dendrites Input 1
Input 2
Input n
Inputs

Cell Nucleus
Neuron
Input 1
Input 2
Input n
Y
Output
w1
w2
w n
Nodes
X1
X2
Xn
Cell Nucleus -------> Nodes

X1
X2
Xn
Synapse
Neuron
Input 1
Input 2
Input n
Y
Output
Weights
w1
w2
wn
Synapse -------> Weights

X1
X2
Xn
Neuron
Input 1
Input 2
Input n
w1
w2
wn
Axon
Axon -------> Output
Y
Output

What is a Perceptron?
A Perceptron is the basic part of a neural network. It represents a single neuron of a human brain and is
used for binary classifiers.
X1
X2
Xn
Input 1
Input 2
Input n
w1
w2
wn
Y Output
Net Input Function Activation Function
ERROR

Perceptron Learning Rule
w
X1
X2
Xn
Input 1
Input 2
Input n
w2
Y Output
Activation Function
ERROR
w1
n
Net Input
Function
Inputs are multiplied with the weights and
a summation is performed, plus a bias is
added
Bias
i=1
n
w x + b
i i*
Step 1

w
X1
X2
Xn
Input 1
Input 2
Input n
w2
Y Output
Activation Function
ERROR
w1
n
Net Input
Function
The weighted sum of inputs is passed to
an activation function to determine if a
neuron will fire or not
Step 2

w
X1
X2
Xn
Input 1
Input 2
Input n
w2
Y Output
Activation Function
ERROR
w1
n
Net Input
Function
Perceptron receives input signals and if
the sum of the input signals exceeds a
certain threshold value, it either
outputs a signal or does not return an
output
Output =
0, if
i=1
n
w x + b < threshold
i i*
1, if
i=1
n
w x + b >= thresholdi i*
Step 3

w
X1
X2
Xn
Input 1
Input 2
Input n
w2
Y Output
Activation Function
ERROR
w1
n
Net Input
Function
The error in the output is backpropagated
and weights are updated to minimize the
error
Step 4

w
X1
X2
Xn
Input 1
Input 2
Input n
w2
Y Output
Activation Function
ERROR
w1
n
Net Input
Function
Repeat steps 1 to 4 in order to generate
the desired output

In 1943, neuroscientist Warren McCulloch and logician Walter Pitts published a paper showing how neurons could implement
the three logical functions (AND, OR, XOR). The neurons they used were simple threshold neurons which they named it
Perceptron.
Warren McCulloch Walter Pitts

A logic gate is the basic building block of a digital circuit. Most logic gates have 2 inputs and 1 output. At
any given moment, every terminal is in one of the two binary conditions low (0) or high (1), represented by
different voltage levels.
Popular Logic Gates
AND
OR
NAND
NOR
NOT
XOR

Implementing AND Gate
Using the Logic Gates, Neural Networks can learn on their own without having to manually code the logic
Input (A) Input (B) Output
0 0 0
0 1 0
1 0 0
1 1 1
AND
The output of Perceptron is true or positive if both
the inputs are true, else false

Implementing AND Gate
A and B are input neurons. The red neuron is our output neuron. The threshold value is 1. If the sum of the
weighted input neurons is greater than the threshold, the output is 1 else 0.
0.7*0 0.7*0 0
0.7*0 0.7*1 0.7
0.7*1 0.7*0 0.7
0.7*1 0.7*1 1.4
A
B
0.7
0.7
= 1
This exceeds
the threshold, so
output is 1
We have designed a neuron which implements a logical
AND gate
w x1 1* w x
2 2* w x +1 1* w x
2 2*

Implementing OR Gate
0 0 0
0 1 1
1 0 1
1 1 1
OR
The output of Perceptron is true or positive if
either of the inputs are true, else false

Implementing OR Gate
A and B are input neurons. The red neuron is our output neuron. The threshold value is 1. If the sum of the
weighted input neurons is greater than the threshold, the output is 1 else 0.
1.2*0 1.2*0 0
1.2*0 1.2*1 1.2
1.2*1 1.2*0 1.2
1.2*1 1.2*1 2.4
A
B
1.2
1.2
= 1
These exceed
the threshold, so
output is 1
w x1 1* w x
2 2* w x +1 1* w x
2 2*
We have designed a neuron which implements a logical
OR gate

Implementing XOR Gate
0 0 0
0 1 1
1 0 1
1 1 0
XOR
The output of Perceptron is true or positive if one
of the inputs is true, else false

XOR gate requires an intermediate hidden layer to achieve the logic of a XOR gate
XOR
Implementation
5
X
X
h
h
O
1
2
3
4
w13
w14
w23
w24
w45
w35
5
• X1 and X2 are inputs and h3 and h4 are the
hidden layers. B3, B4 and B5 are the biases
for h3, h4 and O5.
• h3 = sigmoid (X1*w13 + X2*w23 – B3)
• h4 = sigmoid (X2*w14 + X2*w24 – B4)
• O5 = sigmoid (h3*w35 + h4*w45 – B5)

XOR gate requires an intermediate hidden layer to achieve the logic of a XOR gate
5
X
X
h
h
O
1
2
3
4
20
-20
20
-20
20
20
5
Sigmoid(20*0+20*0-10) 0
Sigmoid(20*1+20*1-10) 1
Sigmoid(20*0+20*1-10) 1
Sigmoid(20*1+20*0-10) 1
~~
~~
~~
~~
Sigmoid(-20*0-20*0+30) 1
Sigmoid(-20*1-20*1+30) 0
Sigmoid(-20*0-20*1+30) 1
Sigmoid(-20*1-20*0+30) 1
~~
~~
~~
~~
b= -10
Sigmoid(20*0+20*1-30) 0
Sigmoid(20*1+20*0-30) 0
Sigmoid(20*1+20*1-30) 1
Sigmoid(20*1+20*1-30) 1
~~
~~
~~
~~
h3 h4
Output
These exceed the
threshold, so
output is 1
b= 30
b= -30
= 1
= 1
= 1

Types of Neural Networks
Pattern
Recognition
ANN CNN RNN DNN DBN
Image Processing Acoustic
Modeling
Recursive
Neural Network
Cancer Detection
Neural Networks are mainly classified into 5 types
Artificial Neural
Network
Convolution
Neural Network
Speech Recognition
Deep Neural
Network
Deep Belief
Network

Deep Learning allows us to build machines that can play games
Playing Games

Composing Music
Deep Neural Nets can be used to produce music by making computers
learn the patterns in a composition

Autonomous Driving Cars
Distinguishes different types of objects, people, road signs and drives
without human intervention

Building Robots
Deep Learning is used to train robots to perform human tasks

Medical Diagnosis
Deep Neural Nets are used to identify suspicious lesions and nodules in
lung cancer patients

Let’s find out how an Artificial Neural Network can be used to identify different shapes like Squares, Circles and Triangles
How to identify
various shapes
using a neural
network?
Feed the shapes to a neural network as
input
Artificial Neural Network

28
28
Strengths
28 28 = 784*
• 28*28 pixels of the input image is
taken as input i.e. 784 neurons
Lets consider the shape of a square
Shape of a Square

28
28
Strengths
28 28 = 784*
0.78 Activation
• Each neuron holds a number called
Activation that represents grayscale
value of the corresponding pixel ranging
from 0 to 1. 1 for white pixel and 0 for
black pixel

28
28
Strengths
• Each neuron is lit up when its
activation is close to 1
28 28 = 784*
• Each neuron holds a number called
Activation that represents grayscale
value of the corresponding pixel ranging
from 0 to 1. 1 for white pixel and 0 for
black pixel

X1
X2
Xn
w1
w2
wn
Step 1
i=1
n
w x + b
i i*
Activation Function
Step 2
i=1
n
w x + b
ii*( )
Lets find out how an Artificial Neural Network can be used to identify different shapes
Identifies different shapes
Squares Circles
Triangles
Pixels of images
fed as input

X1
X2
Xn
Y
W1
W2
W3
.
.
.
.
.
.
.
.
.
Wn
W22
W23
W33
.
.
.
.
.
.
.
.
.
Wn2
W11
W21
W31
.
.
.
.
.
.
.
.
.
Wn1
Apply the weights
Squares Circles
Triangles^
Pixels of images
fed as input

X1
X2
Xn
Y^
Apply the activation functions
Squares Circles
Triangles

X1
X2
Xn
Y^
Compare it with
actual output
Predicted Output
Actual
Output
Y

Applying the cost function to minimize the difference between predicted and actual output using gradient descent algorithm
Predicted Output
X1
X2
Xn
Y^
Actual
Output
Compare it with
actual output
Y
Cost Function: C = ½( Y – Y )
2

Neural Networks use Backpropagation method along with improve the performance of the Neural Net. A cost function is used
to reduce the error rate between predicted and actual output.
X
X
X
Y^
2
1
2
n
Y

Cost Function
The Cost value is the difference between the neural nets predicted output and the actual output from a set of
labelled training data. The least cost value is obtained by making adjustments to the weights and biases
iteratively throughout the training process.
X
X
Xn
Y Y^
W1
W2
W3
.
.
.
.
.
.
.
.
.
Wn
W22
W23
W33
.
.
.
.
.
.
.
.
.
Wn2
W11
W21
W31
.
.
.
.
.
.
.
.
.
Wn1
1
2
2

Gradient Descent
Slower Faster
The person will take more time to reach the base of the mountain if the
slope is gentle and will come down faster if the slope is steep.
Likewise, a Neural Net will train slowly if the Gradient is small and it will
train quickly if the Gradient is large.
Gradient Descent is an optimization algorithm for finding the minimum of a function
A person trying to reach the base of a
mountain

Gradient Descent
Gradient Descent is an optimization algorithm for finding the minimum of a functionGradient Descent is an optimization algorithm for finding the minimum of a function
C = ½( Y – Y )^ 2
C
W
C
W
Best
Gradient
Global Minimum
cost
Initial Weight

Stochastic Gradient Descent
C
W
Best
Local Minimum Global Minimum
It does not require the
cost function to be
convex
Takes the rows one by
one, runs the neural
network and then adjusts
the weights
Helps you avoid the
local minimum as it
performs 1 iteration at a
time

Deep Learning Platforms
Torch
Keras
4 main platforms
TensorFlow
DeepLearning4J
(java)

• TensorFlow is a Deep Learning tool to define and run computations involving tensors.
• A tensor is a generalization of vectors and matrices to potentially higher dimensions.
• The array of data passed in each layer of nodes is known as Tensor.
a
m
k
q
d
2
4
8
1
1
9
3
2
5
4
4
6
6
3
3
7
8
2
9
5
Tensor of Dimensions[5] Tensor of Dimensions[5,4]Tensor of Dimension[3,3,3]

• TensorFlow is a Deep Learning tool to define and run computations involving tensors.
• A tensor is a generalization of vectors and matrices to potentially higher dimensions.
• The array of data passed in each layer of nodes is known as Tensor.
1.2 3.5 2.2…
4.7 10 4.5…
2.2 7.8 8.1…
3.6 2.5 4.5…
Arrays of data with different
dimensions and ranks go as
input to the network

Tensor Ranks
m=V=[1,2,3],[4,5,6]
v=[10,20,30]
t=[[[1],[2],[3]],[[4],[5],[6]],[[7],[8],[9]]]
s= [107]
Tensor of Rank 0 Tensor of Rank 1
Tensor of Rank 2Tensor of Rank 3

Open source software
library developed by
Google
Most popular library in
Deep Learning
Can run on either CPU
or GPU Can create data flow
graphs that have nodes
and edges
Used for Machine
Learning applications
such as Neural
Networks

Use case Implementation using TensorFlow
Lets build a neural network to identify hand written digits using MNIST Database.
Hand written digits
from MNIST Database
Modified National Institute of Standards and Technology
Database
Has a collection of 70,000 handwritten digits
Digit labels identify each of the digits from 0 to 9

Lets build a neural network to identify hand written digits using MNIST Database.
Hand written digits
from MNIST Database
Pixels of digits fed as
input to the network
New image of a digit Identifies the digit
That’s
digit 3

Softmax Function
…………….
0 1 2 9
28*28=784 pixels
…………….
Softmax (Ln) =
Ln
e
e L

1. Import MNIST Data using TensorFlow
2. Check the type of Dataset

3. Array of Training images
4. Number of images for Training, Testing and Validation

5. Visualizing the Data

6. Maximum and minimum value of the pixels in the image

7. Create the Model

8. Create the Session
9. Evaluate the Trained model on Test data

Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural Networks | Simplilearn

Recommended

More Related Content

What's hot (20)

Similar to Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural Networks | Simplilearn (20)

More from Simplilearn (20)

Recently uploaded (20)

Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural Networks | Simplilearn

Editor's Notes