Scribd
Upload
31 views

9.0 CNN-Overview PDF

The document discusses topics in convolutional neural networks including convolution operations, motivations for using convolution, pooling, variants of convolution functions, and efficient convolution algorithms.

Uploaded by

suji.kaliki
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
31 views

9.0 CNN-Overview PDF

The document discusses topics in convolutional neural networks including convolution operations, motivations for using convolution, pooling, variants of convolution functions, and efficient convolution algorithms.

Uploaded by

suji.kaliki
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 11

Deep Learning Srihari

Convolutional Networks: Overview


Sargur Srihari
[email protected]

1
Deep Learning Srihari

Topics in Convolutional Networks


• Overview
1. The Convolution Operation
2. Motivation
3. Pooling
4. Convolution and Pooling as an Infinitely Strong Prior
5. Variants of the Basic Convolution Function
6. Structured Outputs
7. Data Types
8. Efficient Convolution Algorithms
9. Random or Unsupervised Features
10. The Neuroscientific Basis for Convolutional Networks
11. Convolutional Networks and the History of Deep Learning 2
Deep Learning Srihari

Plan of discussion

1. Overview of Convolutional Networks


2. Traditional versus Convolutional Networks
3. Topics in CNNs

3
Deep Learning Srihari

Overview of Convolutional Networks

• Convolutional networks, also known as Convolutional neural


networks (CNNs) are a specialized kind of neural network
• For processing data that has a known grid-like topology
• Ex: time-series data, which is a 1-D grid, taking samples at intervals

• Image data, which are 2-D grid of pixels

• They utilize convolution, which is a specialized kind of linear


operation
4
Deep Learning Srihari

Neural net matrix multiplication


• Each layer produces values that are obtained from previous
layer by performing a matrix-multiplication
Augmented network
• In an unaugmented network
• Hidden layer produces values z =h (W(1)T x + b(1))
• Output layer produces values y =σ(W(2)T x + b(2))
• Note:W(1) and W(2) are matrices rather than vectors
• Example with D=3, M=3 x=[x1,x2,x3]T
• We have two weight matrices W(1) and W(2)
⎪⎧⎪ T T T
⎪⎪ W columns :W1 = ⎡⎢⎣W11 W12 W13 ⎤⎥⎦ ,W2 = ⎡⎢⎣W21 W22 W23 ⎤⎥⎦ ,W3 = ⎡⎢⎣W31 W32 W33 ⎤⎥⎦
(1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1)

w =⎨
⎪⎪ (2) (2)
T T
⎡W (2)W (2)W (2) ⎤ ,W (2) = ⎡W (2)W (2)W (2) ⎤ ,W (2) = ⎡W (2)W (2)W (2) ⎤
T
⎪⎪ W columns :W1
= ⎢⎣ 11 12 13 ⎥⎦ 2 ⎢⎣ 21 22 23 ⎥⎦ 3 ⎢⎣ 31 32 33 ⎥⎦
⎩
First Network layer Network layer output In matrix multiplication notation

5
Deep Learning Srihari

Matrix multiplication for 2D Convolution

Far fewer weights needed than full matrix multiplication

6
Deep Learning Srihari

Convolutional Layer for Image Recognition

7
Deep Learning Srihari

CNN is a neural network with a convolutional layer


• CNNs are simply neural networks that use convolution in place
of general matrix multiplication in at least one of their layers
• Convolution can be viewed as multiplication by a matrix

8
https://towardsdatascience.com/a-comprehensive-guide-to-convolutional-neural-networks-the-eli5-way-3bd2b1164a53
Deep Learning Srihari

Specifying a convolutional layer in Keras


• Conv1D

• Conv2D

• Arguments

• filters: Integer, the dimensionality of the output space (i.e. the number of output filters in the convolution).

• kernel_size: An integer or tuple/list of 2 integers, specifying the height and width of the 2D convolution
window. Can be a single integer to specify the same value for all spatial dimensions.

• strides: An integer or tuple/list of 2 integers, specifying the strides of the convolution along the height and
width. Can be a single integer to specify the same value for all spatial dimensions. Specifying any stride value
!= 1 is incompatible with specifying any dilation_rate value != 1.
• padding: one of "valid" or "same" (case-insensitive). Note that "same" is slightly inconsistent across
backends with strides != 1, as described here
• data_format: A string, one of "channels_last" or "channels_first". The ordering of the dimensions in the
inputs. "channels_last" corresponds to inputs with shape (batch, height, width, channels) while
"channels_first" corresponds to inputs with shape (batch, channels, height, width). It defaults to the
image_data_format value found in your Keras config file at ~/.keras/keras.json. If you never set it, then it will
be "channels_last".
Runtime of Traditional vs Convolutional NetworksSrihari
Deep Learning

• Traditional neural network layers use matrix multiplication by a


matrix of parameters with a separate parameter describing the
interaction between each input unit and each output unit
s =g(WTx )

• With m inputs and n outputs, matrix multiplication requires m x n


parameters and O(m×n) runtime per example
• This means every output unit interacts with every input unit
• Convolutional network layers have sparse interactions

• If we limit no of connections for each input to k we need k x n parameters


and O(k×n) runtime
10
Deep Learning Srihari

Topics in Convolutional Networks

• What convolution is
• Motivation behind using convolution in a neural network
• Pooling, which almost all convolutional networks employ
• Usually the operation used in a convolutional neural network
does not correspond precisely to convolution in math
• We describe several variants on convolution function used in practice
• Making convolution more efficient
• Convolution networks stand out as an example of
neuroscientific principles in deep learning
• Very deep convolutional network architectures

11

You might also like