Unit-5

Neural Network
Lecture – 2
Feedforward Neural Networks
• A Feedforward Neural Network is the most basic type of artificial neural
network where connections between nodes do not form a cycle.
Feedforward Neural Networks
• Data moves in one direction — from input to output — without looping
back.
• It’s called “feedforward” because the data is fed forward through the
layers of the network.
Feedforward Neural Networks
Structure of FNN
• A typical FNN consists of:
• Input Layer : Takes the features of the
input data (e.g., pixels of an image,
exam scores, etc.)
• Hidden Layer(s) : Performs
transformations using weights, biases,
and activation functions.
• Output Layer : Produces the final result,
e.g., prediction (yes/no, digit class, etc.)
Working of Feedforward Neural
Networks
• Input : Input features are passed to the input layer.
• Weighted Sum (Z): Each neuron does a weighted sum

Where,
Working of Feedforward Neural
Networks
• Activation Function : The weighted sum
goes through an activation function (like
ReLU, sigmoid, etc.) to introduce non-
linearity.

• Output Layer: Final activations from the

last hidden layer are passed to the output
layer.In classification problems, softmax is
commonly used.
Working of Feedforward Neural
Networks
• Loss Calculation : A loss function measures how far the output is
from the target (e.g., mean squared error, cross-entropy).
Backpropagation
• Backpropagation stands for "backward propagation of errors".
• It's the process used to update the weights and biases of a neural
network by minimizing the error (loss) between the predicted and
actual outputs.
Backpropagation
• A Simple Analogy
• Imagine you're throwing darts at a target.
• You miss a bit to the left.
• ou learn from that, adjust your angle slightly.
• Try again — closer!
• Repeat until you're hitting the bullseye.
• This is exactly what backpropagation does:
• Measure error → Adjust → Try again → Repeat
Backpropagation
Convolutional Neural Network
(CNN)
• Convolutional Neural Network (CNN) is an advanced version of artificial
neural networks (ANNs), primarily designed to extract features from
grid-like matrix datasets.
• This is particularly useful for visual datasets such as images or videos,
where data patterns play a crucial role.
• CNNs are widely used in computer vision applications due to their
effectiveness in processing visual data.
Convolutional Neural Network
(CNN)
• CNNs consist of multiple layers like the input layer, Convolutional layer,
pooling layer, and fully connected layers.
Convolutional Neural Network
(CNN)
• Input Layer : Takes in the raw image data as input to the network.
• Convolutional Layer : Detects patterns like edges or textures by
sliding filters over the image.
• Activation Layer (e.g., ReLU) : Adds non-linearity so the network can
learn complex patterns.
Convolutional Neural Network
(CNN)
• Pooling Layer : Reduces the size of feature maps to make processing
faster and prevent overfitting.
• Flatten Layer : Converts the 2D feature maps into a 1D vector for the
next layer.
• Fully Connected Layer : Connects all neurons and makes final
decisions based on the learned features.
• Output Layer : Produces the final prediction like class labels using
softmax or sigmoid.
Recurrent Neural Network
(RNN)
• Recurrent Neural Network (RNN) is a type of neural network designed
to work with sequential data — where the order of data matters.
• Example Use Cases:
• Text generation
• Machine translation
• Sentiment analysis
• Speech recognition
• Time series forecasting (like stock prices)
Recurrent Neural Network
(RNN)
• How is RNN Different?
• Unlike CNNs, RNNs have a “memory” — they remember information
from previous steps to influence future outputs.
• This is crucial when the context of earlier input affects later decisions.
Recurrent Neural Network
(RNN)
• Simple RNN Architecture
• An RNN processes data one step at a time and has:
• Input Layer
• Hidden Layer with Recurrent Connections
• Activation Function
• Output Layer
Recurrent Neural Network
(RNN)
• RNN Layers
• Input Layer : Feeds each item from the sequence (like a word or time point)
into the network.
• Hidden Layer : Processes the current input and remembers information from
previous inputs (acts like memory).
• Activation Function (usually tanh or ReLU) : Adds non-linearity to help the
network learn complex relationships in the sequence.
• Recurrent Connection : Feeds the output of the previous time step back into
the current step to maintain context.
• Output Layer : Produces a prediction for each time step or the final result
(like a translated sentence or class label).
Deep Learning
• Deep Learning is a subset of Machine Learning that uses algorithms
called neural networks — especially deep neural networks — to
automatically learn patterns from large amounts of data.
• It mimics how the human brain works (in a very simplified way) to
solve complex problems like image recognition, natural language
understanding, and more.
• It’s a way for computers to learn from data by using multiple layers
of artificial neurons that adjust themselves to make better
predictions.
Deep Learning
Deep Learning Applications
Computer vision
• In computer vision, deep learning models enable machines to identify and
understand visual data.
• Object detection and recognition: Deep learning models are used to identify and
locate objects within images and videos, making it possible for machines to perform
tasks such as self-driving cars, surveillance, and robotics.
• Image classification: Deep learning models can be used to classify images into
categories such as animals, plants, and buildings. This is used in applications such as
medical imaging, quality control, and image retrieval.
• Image segmentation: Deep learning models can be used for image segmentation
into different regions, making it possible to identify specific features within images.
Deep Learning Applications
2. Natural language processing (NLP)
• In NLP, deep learning model enable machines to understand and generate human
language.
• Automatic Text Generation: Deep learning model can learn the corpus of text and new
text like summaries, essays can be automatically generated using these trained models.
• Language translation: Deep learning models can translate text from one language to
another, making it possible to communicate with people from different linguistic
backgrounds.
• Sentiment analysis: Deep learning models can analyze the sentiment of a piece of text,
making it possible to determine whether the text is positive, negative, or neutral.
• Speech recognition: Deep learning models can recognize and transcribe spoken words,
making it possible to perform tasks such as speech-to-text conversion, voice search, and
voice-controlled devices.
Deep Learning Applications
3. Reinforcement learning
• In reinforcement learning, deep learning works as training agents to take
action in an environment to maximize a reward.
• Game playing: Deep reinforcement learning models have been able to
beat human experts at games such as Go, Chess, and Atari.
• Robotics: Deep reinforcement learning models can be used to train
robots to perform complex tasks such as grasping objects, navigation,
and manipulation.
• Control systems: Deep reinforcement learning models can be used to
control complex systems such as power grids, traffic management, and
supply chain optimization.
Challenges in Deep Learning
• Data availability: It requires large amounts of data to learn from. For
using deep learning it’s a big concern to gather as much data for training.
• Computational Resources: For training the deep learning model, it is
computationally expensive because it requires specialized hardware like
GPUs and TPUs.
• Time-consuming: While working on sequential data depending on the
computational resource it can take very large even in days or months.
• Interpretability: Deep learning models are complex, it works like a black
box. it is very difficult to interpret the result.
• Overfitting: when the model is trained again and again, it becomes too
specialized for the training data, leading to overfitting and poor
performance on new data.
Advantages of Deep Learning
• High accuracy: Deep Learning algorithms can achieve state-of-the-art
performance in various tasks, such as image recognition and natural
language processing.
• Automated feature engineering: Deep Learning algorithms can
automatically discover and learn relevant features from data without the
need for manual feature engineering.
• Scalability: Deep Learning models can scale to handle large and complex
datasets, and can learn from massive amounts of data.
• Flexibility: Deep Learning models can be applied to a wide range of tasks
and can handle various types of data, such as images, text, and speech.
• Continual improvement: Deep Learning models can continually improve
their performance as more data becomes available.
Disadvantages of Deep Learning
• High computational requirements: Deep Learning AI models require
large amounts of data and computational resources to train and
optimize.
• Requires large amounts of labeled data: Deep Learning models often
require a large amount of labeled data for training, which can be
expensive and time- consuming to acquire.
• Interpretability: Deep Learning models can be challenging to interpret,
making it difficult to understand how they make decisions.
• Overfitting: Deep Learning models can sometimes overfit to the training
data, resulting in poor performance on new and unseen data.