UNIT -2

INTRODUCTION TO DEEP LEARNING:

Deep Learning is a subset of Machine Learning (ML) that focuses on training
artificial neural networks to mimic the way humans learn and process data. It
is a core component of Artificial Intelligence (AI), enabling systems to
recognize patterns, make decisions, and perform complex tasks without explicit
programming.
Deep learning is a subfield of machine learning that is inspired by the structure
and function of the human brain, and it has become increasingly popular in
recent years due to its impressive ability to solve complex problems in a wide
range of domains. Deep learning algorithms are capable of learning from large
amounts of data, using artificial neural networks that are composed of many
layers of interconnected nodes or neurons.
The term "deep" refers to the depth of the neural network, meaning that it has
many layers, with each layer learning increasingly complex features of the input
data. These layers are typically trained using a form of supervised learning,
where the network is presented with input data and corresponding output
data, and the weights between the neurons are adjusted in order to minimize
the difference between the predicted output and the actual output.
Some of the most common applications of deep learning include image
recognition, speech recognition, natural language processing, and autonomous
driving. Deep learning has also been applied to many other fields, such as drug
discovery, financial modeling, and predictive maintenance.
Biological Vision in Deep Learning :
Biological vision is about understanding how the human visual
system works to process and interpret visual information from the
environment. It has inspired the creation of deep learning models like
Convolutional Neural Networks (CNNs), which try to mimic how the
brain processes visual data.
1. Human Visual System:
o The brain processes images through the visual cortex, which has
multiple layers for identifying edges, shapes, and patterns.
o It is very efficient, recognizing objects even in different
environments with little data.
2. Deep Learning Inspiration:
o Models like CNNs are designed to work like the visual cortex.
o They learn patterns step by step, starting with simple features like
edges and moving to complex ones like full objects.
3. Applications:
o These models are used in tasks like object recognition, image
classification, image segmentation, and more.
o Examples include self-driving cars, medical imaging, and artistic
style transfer.
4. Challenges:
o Large Data Requirement: Deep learning needs lots of labeled data
to train effectively.
o Overfitting: Models can become too focused on training data,
making them less effective on new, unseen data.
o Unlike humans, deep learning models struggle to learn from
minimal data or recognize objects in varying contexts.
5. Advancements:
 o Researchers are working on making models more adaptable and
context-aware, just like the human brain.
o New ideas like attention mechanisms and capsule networks are
being developed to improve these models.
Example of Biological Vision in Deep Learning
If you show a CNN a picture of a cat:
1. First Layer: Detects edges of the cat's body.
2. Next Layers: Recognize fur patterns, eyes, and whiskers.
3. Final Layer: Combines all these features and predicts it’s a cat.

Machine Vision:
 Machine vision is the study of how computers can process and analyze
visual information.
 It is a subfield of computer vision, which focuses on developing
algorithms that can interpret visual information.
  Deep learning has enabled significant advances in machine vision,
particularly in areas such as image recognition, object detection, and
autonomous driving.
 Deep learning models, such as convolutional neural networks (CNNs),
are capable of analyzing images and video and identifying objects and
patterns with high accuracy.
 Machine vision has numerous real-world applications, such as industrial
quality control, robotics, and security systems.
 Deep learning models used in machine vision can be fine-tuned to
improve their performance on specific tasks, such as identifying specific
types of objects or detecting subtle changes in images over time.
 Overall, machine vision is a powerful tool that is enabling new
applications and driving innovation in a wide range of industries.
Challenges in Machine Vision with Deep Learning
 Data Requirements:
Deep learning models require large, labeled datasets to train effectively,
which can be time-consuming and expensive to create.
 Overfitting:
Models may perform well on training data but fail to generalize to new,
unseen data.
 Computational Resources:
Deep learning models need high computational power for training and
inference.
 Environmental Variability:
Models must be robust to variations in lighting, angles, and object
appearances in real-world scenarios.
Example in Machine Vision: Image Classification
 Task: Classify images of animals (e.g., cats vs. dogs).
 Deep Learning Model: A CNN processes the input image by:
o Detecting edges (whiskers, ears).
o Recognizing textures (fur patterns).
o Combining these features to classify whether the image contains a
cat or a dog.

Human and Machine Language:

 Human language refers to the natural language used by humans to
communicate, while machine language refers to the language used by
machines to perform
computations.
 In deep learning, both human and machine language play important
roles in training and using neural networks.
 Human language is used to provide input data to the neural network,
such as text or speech, which the network processes and learns from.
 Machine language, on the other hand, is used to represent the internal
workings of the neural network, including the weights and biases of the
connections between neurons.
 Machine language is also used to represent the output of the neural
network, which can be in the form of text, speech, or other types of
data.
 Human Language refers to the natural languages we use to
communicate, like English, Hindi, or French. Machine Language is the
language computers understand, involving binary codes or
representations used for computations. In deep learning, both human
and machine language play crucial roles in training and using neural
networks.
Learning Methods:
 Supervised Learning: Human language is used as labeled training data
(e.g., "This is positive" or "This is negative" for sentiment analysis).
 Unsupervised Learning: Human language is unstructured input (e.g.,
paragraphs of text) that the model clusters or classifies by patterns.
 Example: Identifying different topics in news articles without pre-
labeled categories.
Real-Life Example:
 Google Translate:
o Input: "Hello, how are you?" (Human language).
o Machine language processes the sentence, identifies patterns, and
translates it.
o Output: "Hola, ¿cómo estás?" (Human language).
 Sentiment Analysis:
o Input: "The movie was amazing!" (Human language).
o Machine language analyzes patterns in the sentence.
o Output: Sentiment = "Positive".
Artificial Neural Network(ANN):
Artificial Neural Networks contain artificial neurons which are called units.
These units are arranged in a series of layers that together constitute the whole
Artificial Neural Networks in a system. A layer can have only a dozen units or
millions of units as this depends on the complexity of the system. Commonly,
Artificial Neural Network has an input layer, output layer as well as hidden
layers.
 Input Layer: Receives data from the outside world.
 Hidden Layers: Process and transform the input into meaningful
patterns.
 Output Layer: Provides the final result or prediction
How ANNs Learn.
1. Training: ANNs are trained using datasets.
o Example: To teach an ANN to recognize cats, it is shown thousands
of cat images.
2. Testing: After training, the ANN is tested with new images to check if it
can identify cats correctly.
3. Backpropagation: If the ANN makes mistakes, an error-correction
process called backpropagation adjusts the connection weights to
improve accuracy.
This iterative process continues until the ANN produces reliable results.

A Feedforward Neural Network (FNN):

 A Feedforward Neural Network (FNN) is a simple neural network
where data flows in one direction—from input to output—through
hidden layers. Each layer processes the data using weights and
activation functions to make predictions or classifications. FNNs are
widely used for tasks like image recognition, regression, and pattern
classification.
A Recurrent Neural Network (RNN) :
It is designed to process sequential data, making it suitable for tasks like
time series prediction and natural language processing. Unlike
feedforward networks, RNNs have connections that allow information to
be passed from previous time steps to the current one, enabling them to
retain context. They use hidden states to maintain memory of past
inputs and adjust weights over time to improve predictions. RNNs are
effective for modeling relationships between elements in sequences,
such as predicting the next word in a sentence or forecasting stock
prices.
A Convolutional Neural Network (CNN) :
It is designed for processing spatial data, such as images. It applies
convolutional layers to extract features from input data, followed by
pooling layers to reduce dimensions and capture important information.
CNNs are widely used in image recognition, object detection, and
computer vision tasks.
A Modular Neural Network:
consists of multiple independent neural networks working together to
solve complex tasks. Each module focuses on a specific sub-task,
allowing for specialization and reduced complexity. These modules
combine to produce a unified output by collaborating on different
aspects of the problem.
Radial Basis Function (RBF) Neural Network:
 Uses distance functions to process data.
 Often used for modeling trends or functions.

Applications of Artificial Neural Networks

 1. Social Media:
o Suggests "People You May Know" by analyzing your profile and
connections.
o Facial recognition uses CNNs to identify faces from photos.
2. Marketing and Sales:
o Platforms like Amazon recommend products based on browsing
and purchase history.
o Food delivery apps suggest restaurants matching user preferences.
3. Healthcare:
o Identifies cancerous tissues with accuracy similar to physicians.
o Detects rare diseases using facial analysis.
4. Personal Assistants:
o Tools like Siri and Alexa use Natural Language Processing (NLP)
powered by ANNs for speech recognition and user interaction.
Advantages of ANNs
 Parallel Processing: Can handle multiple tasks simultaneously.
 Distributed Memory: Data is stored across the entire network, so loss of
some data doesn’t stop functioning.
 Fault Tolerance: Can produce outputs even when some neurons fail.
 Handles Incomplete Data: Can work with missing information, though
performance may decrease.

Disadvantages of ANNs
 Unclear Structure: Deciding the correct network structure is based on
trial and error.
 Unexplainable Results: Outputs lack transparency, reducing trust.
 Hardware Requirements: Needs powerful processors for parallel
computations.
  Preprocessing Required: Problems must be converted into numerical
data before input.
 Time-Consuming: Training may take an unpredictable amount of time

Training Deep Networks

Training deep networks in deep learning involves using large amounts of data
to teach neural networks to recognize patterns and perform specific tasks.
These networks consist of multiple layers of interconnected nodes, each
responsible for learning different features of the input data. The goal is to
minimize the error between the predicted output and the actual target output
by adjusting the weights and biases through optimization algorithms like
gradient descent.
Steps in Training Deep Networks:
1. Data Collection: Gather a large dataset relevant to the task. For example,
in image classification, this could be thousands or millions of images
labeled with their corresponding categories (e.g., cats, dogs, cars).
2. Data Preprocessing: The raw data is cleaned and transformed into a
suitable format, such as resizing images, normalizing pixel values, or
tokenizing text for NLP tasks.
3. Model Architecture: Design a deep neural network architecture. For
instance, a Convolutional Neural Network (CNN) for image classification
may have multiple convolutional layers followed by pooling layers and
fully connected layers.
4. Training Process: During training, the network takes input data,
processes it through its layers, and outputs a prediction. The output is
compared to the actual label, and the difference is used to compute the
error (loss). The network then adjusts its weights and biases using
gradient descent to minimize this error. This process continues iteratively
until the network achieves acceptable performance.
5. Optimization and Evaluation: Hyperparameters such as learning rate,
batch size, and regularization strength are fine-tuned to improve
performance. Validation and test sets are used to assess how well the
model generalizes to unseen data.
Example – Image Classification:
Consider training a CNN to classify images of handwritten digits (e.g., the
MNIST dataset). The model starts by learning simple features like edges and
shapes through the convolutional layers. Then, deeper layers recognize more
complex features, such as digits or patterns within digits. After multiple layers,
the network outputs a prediction for each image—whether it's a '0', '1', '2', and
so on. During training, the model adjusts its weights and biases to minimize the
error between the predicted labels and the actual labels of the images. The
result is a trained model capable of accurately classifying unseen images with
high precision.

Improving Deep Networks:

 More data: One way to improve deep networks is to use more data to
train them. Larger datasets help the model learn more robust and
generalized features that can be applied to unseen data.
 Better initialization: Initializing the weights and biases in a deep
network can have a big impact on its performance. Techniques such as
 Xavier initialization can improve the convergence and accuracy of the
network.
 Regularization: Deep networks are prone to overfitting, especially when
the number of parameters is high. Regularization techniques such as
Dropout or L1/L2 regularization can help reduce overfitting and improve
generalization.
 Improved activation functions: The choice of activation function can
have a big impact on the performance of a deep network. Recent
advances in activation functions such as ReLU, LeakyReLU, and Swish
have shown improved performance over traditional functions such as
sigmoid and tanh.
 Advanced optimization algorithms: Gradient descent is the most
common optimization algorithm used in deep learning. Advanced
optimization algorithms such as Adam, Adagrad, and RMSprop can help
speed up the convergence and improve accuracy.
 Network architecture: The choice of network architecture can have a
significant impact on the performance of a deep network. Popular
architectures such as ResNet, VGG, and Inception have shown improved
performance on various tasks.
 Transfer learning: Transfer learning is a technique where a pre-trained
network is used as a starting point for a new task. This can help improve
performance and reduce the amount of data required for training.
 Data augmentation: Data augmentation involves generating new
training examples by applying various transformations to the existing
data. This can help improve the performance of the network by
reducing overfitting and improving generalization.
 Hyperparameter tuning: The performance of a deep network can be
highly dependent on the choice of hyperparameters such as learning
rate, batch size, and regularization strength. Careful tuning of these
hyperparameters can help improve the performance of the network.
 Ensemble methods: Ensemble methods involve combining the
predictions of multiple deep networks to improve performance.