Scribd
Upload
2 views

Deep Learning_Lecture 4_CNNs

The lecture discusses Convolutional Neural Networks (CNNs), highlighting their ability to process high-dimensional inputs and recognize visual patterns with minimal preprocessing. It outlines the architecture of CNNs, including layers such as convolution, ReLU, pooling, and fully connected layers, and addresses the limitations of traditional neural networks. The presentation emphasizes the advantages of CNNs in handling image data, including local connectivity, parameter sharing, and pooling to enhance performance.

Uploaded by

hazemkotp14
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
2 views

Deep Learning_Lecture 4_CNNs

The lecture discusses Convolutional Neural Networks (CNNs), highlighting their ability to process high-dimensional inputs and recognize visual patterns with minimal preprocessing. It outlines the architecture of CNNs, including layers such as convolution, ReLU, pooling, and fully connected layers, and addresses the limitations of traditional neural networks. The presentation emphasizes the advantages of CNNs in handling image data, including local connectivity, parameter sharing, and pooling to enhance performance.

Uploaded by

hazemkotp14
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 53

Lecture 4

Deep Learning
Presented by : Dr.Hanaa Bayomi
[email protected]
Convolutional Neural Network
CNN
Multi-layer perceptron and image processing

• One or more hidden layers


• Sigmoid activations functions
MNIST digits classification
Drawbacks of previous neural networks
• The number of trainable parameters becomes extremely large
Drawbacks of previous neural networks
•Little or no invariance to shifting, scaling, and other forms of
distortion
Drawbacks of previous neural networks
• The topology of the input data is completely ignored
• work with raw data.
Motivation
➢We can design neural networks that are specifically adapted
for such problems

1- Must deal with very high-dimensional inputs


•150 x 150 pixels = 22500 inputs, or 3 x 22500 if RGB pixels

2- Can exploit the 2D topology of pixels (or 3D for video data)

3- Can build in invariance to certain variations we can expect


•Translations,Scalling, illumination, etc.
History
Convolutional neural Network

➢ CNN is a feed-forward network that can extract topological properties


from an image.

➢Like almost every other neural networks they are trained with a version of
the back-propagation algorithm.

➢Convolutional Neural Networks are designed to recognize visual patterns


directly from pixel images with minimal preprocessing.

➢They can recognize patterns with extreme variability (such as handwritten


characters).
Convolutional neural Network
Convolutional networks leverage these ideas

1. local connectivity
•Each hidden unit is connected only to a sub region (patch) of
the input image
• It is connected to all channels = receptive field
- 1 if greyscale image
- 3 (R, G, B) for color image
Convolutional neural Network
Convolutional networks leverage these ideas

2. parameter sharing

3. pooling / subsampling hidden units


Convolutional neural Network
➢ Convnets contain one or more of each of the
following layers:

1. convolution layer
2. ReLU (rectified linear units) layer (element wise threshold)
3. pooling layer
4. fully connected layer
5. loss layer (during the training process)
1- Convolution layer
a convnet processes an image using a matrix of weights called filters (or
features) that detect specific attributes such as diagonal edges, vertical
edges, etc. Moreover, as the image progresses through each layer, the
filters are able to recognize more complex attributes.
Convolution layer
The convolution layer is always the first step in a convnet. Let's say
we have a 10 x 10 pixel image, here represented by a 10 x 10 x 1
matrix of numbers:
Convolution Example

CS 678 – Deep Learning 18


Kernels as Feature Detectors
Can think of kernels as a ”local feature detectors”
Detect contour
stride
2- ReLU Layer
•The ReLU (short for rectified linear units) layer commonly follows the convolution
layer.

• The addition of the ReLU layer allows the neural network to account for non-linear
relationships, i.e. the ReLU layer allows the convnet to account for situations in which
the relationship between the pixel value inputs and the convnet output is not linear.

• the convolution operation is a linear one. y = w1x1 +w2x2 + w3x3 + ...

• The ReLU function takes a value y and returns 0 If y is negative and y if y is positive.
f(x) = max(0,y)
Rectified linear (ReLU) : max(0,y)

- Simplifies backprop
- Makes learning faster
- Make feature sparse
2- ReLU Layer f(x) = max(0,x)

Other functions such as tanh or the sigmoid function can


be used to add non-linearity to the network, but ReLU
generally works better in practice.
3- Pooling layer
• the pooling layer makes the convnet less sensitive to small changes in the location of a
feature
• Pooling also reduces the size of the feature map, thus simplifying computation in later layers.
4- fully connected NN + loss layers
The fully-connected layer is where the final "decision" is made.
1000 classes

You might also like