DEPARTMENT OF ARTIFICIAL INTELLIGENCE AND DATA SCIENCE

AD3511-DEEP LEARNING LABORATORY

Viva Questions with answers
1. Define convolutional networks.

Convolutional networks, or convolutional neural networks (CNNs), are a class of

deep learning models primarily used for processing data that has a grid-like topology, such as
images. The core building block of CNNs. These layers apply convolution operations to the
input, using filters (or kernels) to extract features such as edges, textures, and patterns.

2. Would sparse interactions cause reduction on performance in convolutional

networks?

Sparse interactions in convolutional networks generally do not lead to a reduction in

performance.

1. Limited Capacity
2. Underfitting
3. Choice of Hyperparameters

3. What is equivariance representation?

Equivariance representation refers to a property of certain neural networks,
particularly convolutional neural networks (CNNs), where the output changes in a predictable
way in response to transformations applied to the input. Specifically, if an input is
transformed (e.g., translated, rotated, or scaled), the output of the network will change in a
corresponding manner.
4. List the types of pooling.

Pooling is a technique used in convolutional neural networks to reduce the spatial

dimensions of feature maps while retaining important information.

1. Max Pooling
2. Average Pooling
3. Global Average Pooling
4. Global Max Pooling
5. Min Pooling
6. Fractional Pooling
7. Stochastic Pooling
8. Adaptive Pooling

5. Which are four main operations in a CNN?

The four main operations in a convolutional neural network (CNN) are:

1. Convolution
2. Activation
3. Pooling
 4. Fully Connected Layer

6. Define Full convolution.

Full convolution refers to a type of convolution operation where the filter (or kernel)
is applied to the entire input, including the border areas, resulting in an output that is larger
than the input. This is in contrast to "valid" convolution, where the filter is applied only to the
valid portions of the input, resulting in a smaller output.
7. What is Gradient Descent?
Gradient descent is an optimization algorithm used to minimize the cost function in
machine learning and deep learning models. It works by iteratively adjusting the parameters
of the model in the direction that reduces the cost, which measures how well the model's
predictions match the actual data.
8. What is the difference between linear and rectified linear unit?

Linear Activation Function

o The linear activation function is defined as: f(x)=xf(x) = xf(x)=x

o It simply outputs the input value without any transformation.
o The output range is unbounded, covering all real numbers (-∞ to +∞).
o It is generally less common in hidden layers of deep networks.

Rectified Linear Unit (ReLU)

o The ReLU activation function is defined as: f(x)=max⁡(0,x)f(x) = \max(0,

x)f(x)=max(0,x)
o It outputs the input directly if it is positive; otherwise, it outputs zero.
o The output range is [0, +∞), which helps introduce non-linearity into the
model.
o Commonly used in hidden layers of neural networks due to its ability to
mitigate the vanishing gradient problem and accelerate training.

9. Define sparse interactions.

Sparse interactions refer to a property of certain neural network architectures,
particularly convolutional neural networks (CNNs), where each neuron or unit in a layer
interacts with only a small subset of neurons in the previous layer. This is in contrast to fully
connected layers, where each neuron connects to every neuron in the preceding layer.
10. What is loss function?
A loss function (or cost function) is a mathematical function used in machine learning
and deep learning to quantify the difference between the predicted output of a model and the
actual target values. It measures how well the model's predictions align with the true data,
guiding the optimization process during training.
11. What is the use of parameter sharing in CNN?
 Parameter sharing in convolutional neural networks (CNNs) refers to the practice of
using the same weights (or filter parameters) across different parts of the input data.

1. Reduced Number of Parameters

2. Improved Generalization
3. Efficiency in Learning
4. Local Feature Detection
5. Hierarchical Feature Learning

12. Explain padding in CNN.

Padding in convolutional neural networks (CNNs) refers to the process of adding

extra pixels (usually zeros) around the edges of the input data before applying convolutional
operations. Padding is an important technique that helps control the output size of feature
maps and affects the behaviour of the convolutional layer.

13. What are the types of padding?

1. Valid Padding
2. Same Padding
3. Full Padding

14. How many filters must a CNN have?

The number of filters in a convolutional neural network (CNN) isn't fixed and depends on
several factors. Here are some guidelines:

1. Early Layers
2. Deeper Layers
3. Task Complexity
4. Architecture Design
5. Available Resources

15. Define Mean Square Error.

Mean Squared Error (MSE) is a commonly used loss function in regression tasks that
measures the average squared difference between the predicted values and the actual target
values. It quantifies how well a model's predictions align with the true values, with a focus on
penalizing larger errors more than smaller ones.
16. What is recurrent neural network?
A class of neural network for processing sequential data is known as recurrent neural
networks. It is neural networks that are specialized for processing a series of values x(1),
….,x(n),just like convolutional networks are neural networks that are specialized for
processing a grid of values X, such as an image.
17. Why RNN is called as recurrent?
 RNNs are called recurrent because they perform the same task for every element of a
sequence, with the output being depends on the previous computations and user already know
that they have a “memory” which captures information about what has been calculated so far.

18. Explain advantages of RNN.

 RNN can process input of any length
 RNN model is modeled to remember each information throughout the time
which is very helpful in any time series predictor.
 Even if the input size is larger, the model size does not increase
 The weights can be shared across the time steps.
 RNN can use their internal memory for processing the arbitrary series of
inputs which is not the case with feed forward neural networks.
19. Explain disadvantages of RNN.
 Due to its recurrent nature, the computation is slow.
 Training of RNN models can be difficult.
 Prone to problems such as exploding and gradient vanishing.
20. List the advantages of unfolding process.
 Regardless of sequence length, learned model has same inputs size. Because it is
specified in terms of transition from one state to another state rather than specified in
terms of a variable length history of states.
 Possible to use same function f with same parameters at every step.
21. Explain echo state networks of RNN.
 Echo states networks provide an architecture and supervised learning principle for
recurrent neural networks.
 The main idea is to drive a random large fixed recurrent neural networks with the
input signal, thereby including in each neuron within this “reservoir” network a non-
linear response signal and combine a desired output signal by a trainable linear
combination of all of these response signals.
22. Explain recursive neural networks.
 Non-linear adaptive models that can learn in depth and structured information are
called recursive neural networks.
 RNNs were effective in natural language processing for learning sequences and
structures of the trees, primarily phrases and sentences based on word embedding.
 Recursive networks is use for processing data structures as inputs to neural nets in
natural language processing as well as in computer vision.
23. What is long short -term memory?
 Long short-term memory networks are type of recurrent neural networks capable of
learning order dependence in sequence prediction problems.
  LSTM has “memory cells” that can remember information for long periods of time.
24. List a components of LSTM networks.
There are four main components to an LSTM network:
 The forget gate
 The input gate
 The output gate
 The cell state
25. What is NLP?
 NLP stands for Natural Language Processing.
 It is the branch of Artificial Intelligence that gives the ability to machine
understand and process human languages.
 Human languages can be in the form of text or audio format.

26. What are the components of NLP?

There are two components of Natural Language Processing:
 Natural Language Understanding
 Natural Language Generation

27. What are the types of RNN?

There are four types of RNNs based on the number of inputs and outputs in the network.
 One to One
 One to Many
 Many to One
 Many to Many

28. What are the types of sequence to sequence learning?

1. Sequence to Sequence — Outputs are equal to the number of inputs.

2. Sequence to Vector — A single output is given for ’n’ number of inputs
3. Vector to Sequence — ’n’ number of outputs is received for 1 input
4. Vector to Vector — Single output is received for a single input

29. What is bidirectional RNN?

 An architecture of a neural network called a bidirectional recurrent neural network

(BRNN) is made to process sequential data.
 In order for the network to use information from both the past and future context in
its predictions, BRNNs process input sequences in both the forward and backward
directions.
 This is the main distinction between BRNNs and conventional recurrent neural
networks.

30. What are the reasons for training data can be limited?
1. The cost of collecting more data.
2. Data collection can require time, money or human suffering.
31. Explain learning rate.
 Learning rate controls model capacity in a more complicated way than other
hyperparamaters.
 Effective capacity is highest when learning rate is correct, not when it is large or
small.
 The amount that the weights are updated during training is referred to as the step size
or the learning rate.
32. What is grid search?
 Grid-search is used to find the optimal hyperparamaters of a model which results in
the most “accurate” predictions.
 When there are three or fewer hyperparamaters, the common practice is to perform
grid search.
33. What is hyperparamaters?
 Hyperparameters control algorithm behavior and deep learning algorithms come with
many hyperparameters.
 Hyperparameters are points of choice or configuration that allow a machine learning
model to be customized for a specific task or dataset.
34. Define random search.
 Random search uses random combinations of hyperparameters.
 This means that not all of the parameters values or tried and instead parameters will
be sampled with fixed numbers of iterations.
35. What is main reason? Why random search finds good solutions faster than grid-
search?
The main reason why random search finds good solutions faster than grid-search is
that there are no wasted experimental runs, unlike in the case of grid-search, when two values
of a hyperparamater would give the same results.
36. When manual hyperparameter works well?
Manual hyperparameter tuning can work very well when the user has a good starting
point such as one determined by others having worked on the same type of applications and
architecture ,or when the user has months or years of experience in exploring hyperparameter
values for neural networks applied to similar tasks.
37. Define precision.
 Precision is the ratio of true positive and total positives predicted.
 A precision score towards 1 will signify that your model didn’t miss any true positives
and is able to classify well between correct and incorrect labeling of data.
38. What is mean by performance metrices?
 To evaluate the performance or quality of the model, different metrics are used, and
these metrics are known as performance metrics or evaluation metrics.
  These performance metrics help us understand how well our model has performed for
the given data.
 In this way, we can improve the model's performance by tuning the hyper-parameters.
 Each ML model aims to generalize well on unseen/new data, and performance metrics
help determine how well the model generalizes on the new dataset.
39. Define accuracy.
The accuracy metric is one of the simplest Classification metrics to implement, and it can
be determined as the number of correct predictions to the total number of predictions.
Accuracy=number of correct predictions
Total number of predictions
40. What is mean by confusion matrix?
A confusion matrix is a tabular representation of prediction outcomes of any binary
classifier, which is used to describe the performance of the classification model on a set of
test data when true values are known.
41. Define Autoencoders.
A machine learning algorithm known as an artificial neural network (ANN), which
uses backpropagation and sets the target values to equal the input values, is used in an
autoencoder. In order to recreate the actual input, it is constructed in a way that can do both
data encoding and data decoding tasks.

42. What are the Uses of Autoencoders?

Today's world of images requires the usage of autoencoders for a variety of purposes.
These are some of the uses for them:

 Data compression
 Dimensionality reduction
 Image denoising
 Feature extraction
 Removing watermarks from Images

43. Describe the sparse constraint.

A sparse constraint is one that appears in the loss function of a sparse encoder.
When we set the many nodes in the hidden layer, the sparse constraint ensures that the
autoencoder is not overfitting the training data.

44. Name some of the Autoencoder Variations?

Some of the Autoencoder Variations are as follows:

 Convolutional Autoencoders
 Sparse Autoencoders
 Deep Autoencoders
 Contractive Autoencoders

45. Difference between overcomplete and undercomplete autoencoders

 The autoencoder is referred to as an overcomplete autoencoder when the dimension
of the code or latent representation is greater than the dimension of the input. On the other
hand, the autoencoder is known as an undercomplete autoencoder when the dimension of
the code or latent representation is smaller than the dimension of the input.