Abdulrahman Motasem Alanani 20174911 Assignment 5

Neural Networks

1)What are the Steps of Manual Feature Extraction?

1. Define Features: Identify and define specific characteristics or attributes in the input data
that are relevant for the classification task. These features can be based on domain
knowledge or prior understanding of the data.

2. Domain Knowledge: Utilize domain knowledge to determine which features are important
for the classification task. Domain knowledge helps in selecting relevant features that can
discriminate between different classes or categories.

3. Detect Features: Implement algorithms or methods to extract the defined features from the
input data. This process involves identifying patterns, structures, or attributes that are
indicative of the classes to be classified.

2) Discusses the Fully Connected Neural Network?

A Fully Connected Neural Network (FCNN) is a type of artificial neural network where each neuron
in one layer is connected to every neuron in the next layer. Key points about FCNNs include:

• Architecture: Neurons are organized in layers, with each neuron in a layer connected to
every neuron in the subsequent layer, enabling complex relationships to be learned
between input and output data.

• Training: FCNNs are trained using backpropagation to adjust connection weights between
neurons and minimize prediction errors.

• Activation Functions: Non-linear functions like ReLU or Sigmoid are used in FCNNs to
introduce non-linearity and learn complex patterns in the data.

• Limitations: FCNNs struggle with high-dimensional data like images due to a large number
of parameters and a lack of consideration for spatial relationships in the input data.

• Spatial Structure: FCNNs do not account for spatial structure in input data, which is
crucial for tasks like image recognition. This limitation led to the development of
Convolutional Neural Networks (CNNs) that leverage spatial relationships in data.

3) How can we use spatial structure in the input to inform the architecture of the network?

Convolutional Neural Networks (CNNs) are tailored to efficiently handle spatial data by utilizing
specific strategies:

• Convolutional Layers: CNNs employ convolutional layers with filters to capture local
patterns and spatial hierarchies in the input data, enabling the network to learn features at
various spatial scales.

• Pooling Layers: Pooling layers downsample spatial dimensions while retaining essential
features, such as through max pooling and average pooling, reducing complexity and
enhancing robustness to input variations.
Abdulrahman Motasem Alanani 20174911 Assignment 5
Neural Networks

• Shared Weights: CNNs share filter weights across different spatial locations, reducing
parameters and facilitating the learning of translation-invariant features crucial for
capturing spatial patterns effectively.

• Hierarchical Feature Learning: CNNs incorporate multiple layers to learn hierarchical

representations of input data. Lower layers detect simple features like edges, while higher
layers combine these features to learn complex patterns and object representations.

4)What are the steps of Feature Extraction using convolution?

1. Convolution Operation:

o Apply filters (kernels) to the input data by sliding them across the input.

o Compute element-wise multiplications between the filter and local regions of the
input.

o Sum the results to produce feature maps that capture patterns and features from
the input data.

2. Non-linear Activation:

o Apply a non-linear activation function (e.g., ReLU) to introduce non-linearity into the
network.

o Activation functions help the network learn complex patterns and relationships in
the data.

3. Pooling (Downsampling):

o Use pooling layers (e.g., max pooling, average pooling) to downsample the feature
maps.

o Pooling helps reduce spatial dimensions while retaining important features, aiding
in computational efficiency and generalization

5)Suppose we want to compute the convolution of a 5x5 image and a 3x3 filter what is the output?

10011

10101 101

11100 * 101

01100 010

11001

Feature map =
4 2 3
5 2 3
4 2 2
Abdulrahman Motasem Alanani 20174911 Assignment 5
Neural Networks

6) Draw the Steps of CNNs for Classification

1. Input Image: The initial input to the CNN is the image that needs to be classified.
Abdulrahman Motasem Alanani 20174911 Assignment 5
Neural Networks

2. Convolutional Layer: Apply filters with learned weights to the input image to generate
feature maps. Each filter detects specific patterns or features in the image.

3. Activation Function (ReLU): Introduce non-linearity through an activation function like

ReLU to add complexity and enable the network to learn more intricate patterns.

4. Pooling Layer: Perform downsampling operations on each feature map to reduce

dimensionality and preserve spatial invariance.

5. Fully Connected Layer: Use the high-level features extracted from the convolutional and
pooling layers for classifying the input image. The fully connected layer combines these
features to make a prediction.

6. Output Layer (Softmax): Express the output as the probability of the image belonging to a
particular class using the softmax function.

7. Training with Backpropagation: Learn the weights of the convolutional filters and fully
connected layers through backpropagation using a cross-entropy cost function.

7) What is the meaning of pooling ?

Abdulrahman Motasem Alanani 20174911 Assignment 5
Neural Networks

8) Compare between CNNs for Classification :Feature Extraction & Class Probabilities?

9) What is the Type of classification?

• Semantic segmentation
• Object detection
• Image captioning

10) Discuss the Idea of Object Detection with R-CNNs

1. Region Proposals: R-CNNs start by generating region proposals using selective search or
other region proposal methods. These region proposals are bounding boxes that potentially
contain objects of interest within the image.

2. Feature Extraction: For each region proposal, the image patch within the bounding box is
extracted and warped to a fixed size. A pre-trained CNN (such as AlexNet or VGG) is then
used to extract features from each region proposal.

3. Classification and Localization: The extracted features from each region proposal are fed
into a set of fully connected layers for classification and bounding box regression. The
classification network predicts the probability of each class being present in the region,
while the regression network refines the bounding box coordinates.
Abdulrahman Motasem Alanani 20174911 Assignment 5
Neural Networks

4. Non-Maximum Suppression: After classification and regression, a post-processing step

like non-maximum suppression is applied to remove duplicate detections and refine the
final bounding boxes for the detected objects.

5. Training: R-CNNs are trained in a multi-stage process where the CNN is pre-trained on a
large dataset (e.g., ImageNet) and fine-tuned on the object detection dataset using a
combination of classification and regression losses.

6. Advantages: R-CNNs are able to localize objects accurately within an image and are
capable of handling multiple object instances and classes. They have been successful in
various object detection challenges and applications.