Deep Learning Techniques
Deep Learning Techniques
UNIT-1
1.Is AI a science or is it engineering? Or neither or both? Explain.
A. Artificial Intelligence is both a science and engineering.
Artificial Intelligence (AI) is a branch of research and engineering that
integrates science and engineering to construct intelligent machines. It draws
on work from philosophy, psychology, and computer science, as well as brain
science and languages.
AI Science and Engineering is about the study and building of "intelligent agents": any
system that perceives its environment and takes actions that maximize its chance of
achieving its goals.
AISE aims to create machines that mimic "cognitive" functions that humans associate
with the human mind, such as "learning" and "problem solving".
K(x1, x2) = x1 . x2
4. Exponential Kernel: Similar to the RBF kernel, but it decays much more quickly.
5. Laplacian Kernel: Similar to RBF Kernel, it has a sharper peak and faster decay.
6. Hyperbolic or the Sigmoid Kernel: It is used for non-linear classification problems. It transforms
the input data into a higher-dimensional space using the Sigmoid kernel.
k(x, y) = tanh(xTy + c)
7. Anova radial basis kernel: It is a multiple-input kernel function that can be used for feature
selection.
7.How random forests are related to Decision trees. (or) What is a Decision
tree algorithm? Explain.
A. The Random Forest Algorithm combines the output of multiple Decision Trees to
generate the final output. This process of combining the output of multiple individual
models is called Ensemble Learning.
Relationship between Decision Trees and Random Forests:
1. Random Forests are an extension of Decision Trees
2. Random Forests address Decision Trees' limitations: Overfitting, High variance.
3. Random Forests improve Decision Trees' performance:
- Increased accuracy
- Improved robustness
- Better handling of high-dimensional data
Overview of Random Forest vs Decision Tree
This process of combining the output of multiple individual models (also known as
weak learners) is called Ensemble Learning. If you want to read more about how the
random forest and other ensemble learning algorithms work, check out the following
articles:
8. What are the assumptions of Gradient boosting Algorithm?
A. Gradient Boosting Algorithm (GBA) relies on the following assumptions:
Assumptions:
1. Linearity: Relationship b/w features and target variable is approximately linear.
2. Independence: Observations are independent and identically distributed.
3. Constant Variance: Variance of residuals is constant across all levels of predictors.
4. Normality: Residuals follow a normal distribution.
5. No multicollinearity: Features are not highly correlated.
6. No significant outliers: Data is free from influential outliers.
7. Sufficient data: Large enough sample size for reliable estimates.
Gradient Boosting
Gradient Boosting is a powerful boosting algorithm that combines several weak
learners into strong learners, in which each new model is trained to minimize the loss
function such as mean squared error or cross-entropy of the previous model using
gradient descent.
12.Explain how random forests give output for classification and regression
problems.
A. Decision Making and Voting: When it comes to making predictions, each decision
tree in the Random Forest casts its vote. For classification tasks, the final prediction is
determined by the mode across all the trees. In regression tasks, the average of the
individual tree predictions is taken.
13. Discuss about Probabilistic modelling in detail.
A. Probabilistic models are used to optimize complex models with many parameters,
such as neural networks. Probabilistic models are statistical models that include one
or more probability distributions in the model to account for these additional factors.
They allow us to account for error or randomness in our statistical models of data.
Categories Of Probabilistic Models
These models can be classified into the following categories:
• Generative models
• Discriminative models.
• Graphical models
Generative models: Generative models aim to model the joint distribution of the
input and output variables. They can be used for tasks such as image and speech
synthesis
Discriminative models: The discriminative model aims to model the conditional
distribution of the output variable given the input variable. They can be used for
tasks such as image recognition, speech recognition.
Graphical models: These models use graphical representations to show the
conditional dependence between variables. They are commonly used for tasks such
as image recognition, natural language processing.
Advantages Of Probabilistic Models
• Probabilistic models are an increasingly popular method in many fields,
including artificial intelligence, finance, and healthcare.
• The main advantage of these models is their ability to take into account
uncertainty and variability in data.
Disadvantages Of Probabilistic Models
There are also some disadvantages to using probabilistic models.
• One of the disadvantages is the potential for overfitting, where the model is
too specific to the training data.
• Not all data fits well into a probabilistic framework.
Sounding Natural
We use a combination of a concatenative text to speech (TTS) engine and a synthesis
TTS engine to control intonation depending on the circumstance.
System Operation
The Google Duplex system has a self-monitoring capability, which allows it to
recognize the tasks it cannot complete autonomously.
Benefits for Businesses and Users
Duplex can call the business to inquire about open hours and make the information
available online with Google, making the information more accessible to everyone.
Duplex asking for holiday hours:
For users, Google Duplex is making supported tasks easier. Instead of making a phone
call, the user simply interacts with the Google Assistant, and the call happens
completely in the background without any user involvement.
5. What is optimization? What are the measures used to minimise cost.
A. Optimization is the process of finding the best solution among a set of possible
solutions to maximize or minimize a specific objective function, subject to
constraints.
Types of Optimization:
1. Maximization: Increase revenue, profit, or efficiency.
2. Minimization: Reduce cost, risk, or time.
Measures to Minimize Cost:
1. Linear Programming (LP)
2. Integer Programming (IP)
3. Dynamic Programming (DP)
4. Greedy Algorithms
5. Branch and Bound
Computer vision and human vision are two distinct yet interconnected domains that
shape our understanding of visual perception.
Computer vision refers to the ability of machines to understand and interpret visual
data. It is a field of artificial intelligence that utilizes algorithms and computational
models to analyze and make sense of images and videos.
Human vision is a remarkable biological process that allows us to perceive and
interpret the visual world. It begins with the eyes, which capture light and send
signals to the brain for processing.
Key Differences Between Computer Vision and Human Vision
1. Processing Mechanisms: Computer vision relies on algorithms and
computational models to process visual data, whereas human vision involves
complex neural networks and biological processes.
2. Adaptability and Efficiency: Human vision is highly adaptable and efficient in
recognizing patterns. Computer vision algorithms can struggle in such
situations.
3. Handling Complex Scenes and Varied Conditions: Human vision integrates
information from multiple sensory channels. Computer vision algorithms often
focus on specific visual features.
4. Update the weights and repeat until the system minimizes the error and makes
accurate predictions with new data.
11. Illustrate on computation representation of language in Human and
Machine language.
A. Here's an illustration of computational representation of language in human and
machine languages:
Human Language Representation:
1. Phonological: Sound waves (speech) or written symbols (text)
2. Lexical: Words and phrases
3. Syntactic: Grammar and sentence structure
4. Semantic: Meaning and context
5. Pragmatic: Communication goals and intentions
15. Discuss about the Softmax layer of a Fast Food – classifying Network.
A. The Softmax layer is a crucial component in a Fast Food-classifying Network,
enabling the model to predict probabilities for each food class.
Softmax Layer:
1. Normalizes output to ensure probabilities sum to 1.
2. Computes probability distribution over all classes.
3. Used for multi-class classification problems.
Fast Food-classifying Network:
1. Convolutional Neural Network (CNN) architecture.
2. Trained on dataset of images of various fast foods.
3. Softmax layer outputs probabilities for each food class.
By incorporating a Softmax layer, the Fast Food-classifying Network accurately
predicts probabilities for each food class, enabling efficient classification and
decision-making.
UNIT - III
1.Explain different types of neural networks.
A. Neural Networks are a fundamental component of Deep Learning, inspired by the
human brain's structure and function.
2. Explain the terms loss function and optimizers with respect to DL.
A. Loss function is a method of evaluating how well your algorithm is modeling your
dataset.
5.Plot the graph between training (i.e no. of epochs) and validation loss.
Explain.
A. Training loss is the calculated error when the model makes predictions on the
training data. It is updated after every forward and backward pass of the model
during the training process. A loss function quantifies the difference between the
predicted and actual labels.
Validation loss evaluates the model’s performance on a separate dataset that the
model has never seen during training. Validation loss is computed at the end of each
epoch during training but is not used to update the model weights.
6. Explain the anatomy of a neural network.
A. The Anatomy of a Neural Network
Neural Network: Neural networks are made up of multiple layers of interconnected
nodes or neurons that work together to process and analyze data.
1. Input Layer: The input layer is the first layer of the neural network and is
responsible for receiving data from the outside world.
2. Hidden Layer: The hidden layer is where the magic happens. This layer is
responsible for processing and analyzing the data received from the input layer.
3. Output Layer: The output layer is the final layer of the neural network and is
responsible for producing the output or prediction.
4. Activation Function: The activation function is a mathematical function that is
applied to the output of each node in the neural network.
5. Backpropagation: Backpropagation is a technique used to train neural networks.
6. Dropout: Dropout is a regularization technique that is used to prevent overfitting
in neural networks.
7. Convolutional Neural Networks: Convolutional neural networks (CNNs) are a type
of neural network that is particularly well-suited for image recognition tasks.
7. Elaborate on Reuters dataset in detail.
A. The Reuters-21578 dataset is one of the most widely used data collections for text
categorization research. It is collected from the Reuters financial newswire service in
1987.
Dataset Features
- Text Classification: each newswire is labelled with one or more topics.
- Multi-Label Classification: each newswire can belong to multiple topics.
- Word Indexing: Words are indexed by frequency for efficient filtering operations.
- Sequence Length: Sequences have length that can be specified .
Dataset Structure
The dataset is divided into training and testing sets, with 80% used for training and
20% for testing. Each sequence is represented as a list of word indexes, with
unknown words replaced by a special character.
Keras Architecture:
1. Frontend: User interface for building and training models.
2. Backend: Engine for computing tensors and gradients.
Key Components:
1. Models: Sequential, Functional, or Subclassing API.
2. Layers: Building blocks of models (Dense, Conv2D, LSTM, etc.).
3. Activations: Functions applied to layer outputs (ReLU, Sigmoid, etc.).
4. Optimizers: Algorithms for weight updates (Adam, SGD, etc.).
5. Loss Functions: Metrics for evaluating model performance.
Backend Engines:
1. TensorFlow: Default backend.
2. Theano: Optional backend.
3. CNTK: Optional backend.
12. Discuss the classification of newswires and explain with the dataset.
A. Classifying newswires: a multi-class classification example
This notebook contains the code samples found in Chapter 3, Section 5 of Deep
Learning with Python.
Classification of newswires involves categorizing news articles into predefined topics
or categories.
Classification Categories:
1. Business
2. Technology
3. Education
4. International News
Dataset:
The Reuters Corpus dataset is commonly used for newswire classification.
Reuters Corpus Dataset:
- 11,228 newswires from Reuters
- 46 topics (categories)
- 10,788 training samples
- 2,440 testing samples
- Text data with corresponding topic labels
13. With a neat sketch, enumerate the concept of the deep-learning software
and hardware stack.
A. The deep-learning software and hardware stack consists of several layers that
work together to facilitate the development and deployment of deep learning
models.
1. Hardware Layer: Provides the computational power for training deep learning
models.
Examples: GPUs, TPUs, CPUs, FPGAs, and ASICs.
2. System Software Layer: provides an environment for running deep learning
frameworks.
Examples: Os, device drivers, and resource managers.
3. Deep Learning Frameworks: handle complex mathematical operations and
optimize performance.
Examples: TensorFlow, PyTorch, Keras.
4. Libraries and Tools: Offer additional functionalities for data manipulation.
Examples: NumPy, OpenCV, and specialized libraries.
5. Model Development: The phase where deep learning models are created.
Examples: Data preprocessing, model design, training, and evaluation.
14. Explain the high-level building blocks required for developing deep-
learning models.
A. The building blocks of deep learning:
The fundamental data structure in neural networks is the layer.
A layer is a data-processing module that takes as input one or more tensors and that
outputs one or more tensors. Some layers are stateless, but more frequently layers
have a state.
Different layers are appropriate for different tensor formats and different types of
data processing.
Building deep-learning models in Keras is done by clipping together compatible layers
to form useful data-transformation pipelines.
Consider the following example
When using Keras, you don’t have to worry about compatibility, because the layers
you add to your models are dynamically built to match the shape of the incoming
layer. For instance, suppose you write the following
Convolution operations:
Convolution Kernels: A kernel is a 2D matrix that maps on the input image by simple
matrix multiplication and addition, the output obtained is of lower dimensions and
therefore easier to work with.
12. What do you mean by weight sharing? Explain weight sharing in CNNs.
A. weight sharing is to share the same weights across all four filters. It reduces the
number of weights that must be learned, which reduces model training time and
cost.
The weight-sharing property of convolutional neural networks (CNNs) has been a
revolutionary concept in the field of deep learning and computer vision. This article
explores the genesis, advantages, and disadvantages of weight-sharing in CNNs,
including a Python example to illustrate its practical application.
Advantages
1. Reduced Complexity: CNNs drastically reduce the number of parameters,
making the network less complex and easier to train.
2. Translation Invariance: helps in detecting features regardless of their position
in the input space.
3. Efficiency: With fewer parameters, CNNs are more computationally efficient
and require less memory.
Disadvantages
1. Limited Perception: Due to their local receptive field, CNN might have a limited
understanding of the overall context.
2. Spatial Invariance Limitation: CNNs are less effective with other
transformations like rotation and scaling without additional augmentation.
Python Example
import tensorflow as tf
model = Sequential()
model.summary()
5. Can you explain how an image sensor is in context with machine vision?
A. Image sensors: At the center of any vision system, an image sensor converts light
energy into electrical signals that can be analyzed by software. Image sensors are
solid-state semiconductor chips comprising millions of photodetectors called pixels,
which convert light into electrical signals.
Machine Vision Image Sensor Context:
1. Lighting: Proper lighting illuminates the object or scene, ensuring optimal image
quality.
2. Optics: Lenses focus light onto the image sensor, controlling field of view, depth of
field, and resolution.
3. Image Sensor: Converts light into electrical signals, capturing visual data (e.g.,
intensity, color, texture).
4. Image Processing: Software processes and enhances the captured image,
correcting distortions, noise, and artifacts.
5. Analysis: Algorithms analyze the processed image, extracting relevant information
(e.g., object detection, classification, measurement).
12. What are the steps involved in typical deep reinforcement learning
algorithm
A. Step 0: Defining Your Problem Space
Step 1: Start with a Pre-trained Model
The first step in developing AI applications involves starting with a pre-trained model,
which can be obtained from open-source providers.
Step 2: Supervised Fine-Tuning
Supervised fine-tuning is a crucial step in the development of generative AI
applications for large language models, allowing them to become adaptable to
specific use cases.
Step 3: Reward Model Training
Reward model training is an advanced technique that involves training a model to
recognize desirable outputs created by another model.
Step 4: Reinforcement learning via proximal policy optimization (PPO)
Reinforcement learning via proximal policy optimization (PPO) is a type of algorithm
that trains large language models to produce outputs that maximize a reward signal
through trial and error.
Step 5: Red teaming
It allows for human evaluators to provide real-world feedback on the performance of
the generative AI models.
Advantages:
1. Unsupervised Learning: DBNs can learn complex patterns in data.
2. Dimensionality Reduction: reduce the dimensionality of high-dimensional data.
3. Feature Learning: DBNs can learn relevant features from data.
4. Improved Classification: DBNs can improve classification accuracy.
5. Robustness to Noise: DBNs can handle noisy or missing data.
6. Efficient Training: DBNs can be trained efficiently using greedy layer-wise training.
7. Scalability: DBNs can handle large datasets.
8. Flexibility: DBNs can be used for various tasks.
Examples
• Image Denoising: DAEs are effective in removing noise from images, such as
Gaussian noise.
• Fraud Detection: reconstruct common transactions from their noisy
counterparts.
• Data Imputation: data imputation in datasets with incomplete information.
Image Denoising with Denoising Autoencoder
1.Importing Libraries and Dataset
!pip install tensorflow numpy matplotlib
from tensorflow.keras.datasets import mnist
(x_train, _), (x_test, _) = mnist.load_data()
2.Preprocess Data
noise_factor = 0.5
x_train_noisy = np.clip(x_train_noisy, 0., 1.)
x_test_noisy = np.clip(x_test_noisy, 0., 1.)
x_train_noisy = x_train_noisy / 255.
x_test_noisy = x_test_noisy / 255.
3.Define the model
autoencoder = Model(input_layer, decoded)
autoencoder.compile(optimizer='adam', loss='binary_crossentropy')
from tensorflow.keras.models import Model
input_layer = Input(shape=(28, 28, 1))
4. Train the model
autoencoder.fit(x_train_noisy, x_train, epochs=10, batch_size=128,
validation_data=(x_test_noisy, x_test))
denoised_images = autoencoder.predict(x_test_noisy)
5.Visualize the original, noisy, and denoised images.
import matplotlib.pyplot as plt
plt.imshow(x_test[0], cmap='gray')
plt.show()
plt.imshow(x_test_noisy[0], cmap='gray')
plt.show()
Output: