Modal Robustness:

It refers to the ability of a model to work well on unseen data, an essential requirement for real-world
applications.

Best Websites for Vocabulary

https://pythongeeks.org/deep-learning-key-terms/

https://iq.opengenus.org/key-terms-in-deep-learning/

https://www.simplilearn.com/tutorials/deep-learning-tutorial/deep-learning-algorithm

https://www.springboard.com/blog/data-science/machine-learning-terminology/

https://www.inforly.io/deep-learning-glossary/

Feature Engineering Explained

Feature engineering is s a crucial step in the machine learning pipeline as it converts raw data
into features that help to make predictions or classifications. It has a significant impact on the
performance of the resulting model. The goal of feature engineering is to create informative,
uncorrelated features and have a strong relationship with the target variable.

Steps

There are various steps involved in feature engineering that include:

1. Feature Selection: This step involves selecting the most relevant features from the raw data.
The goal is to choose features that are informative, uncorrelated, and have a strong relationship
with the target variable.
2. Feature Extraction: This step involves creating new features from the raw data. The goal is to
transform the data into a format that is more suitable for the machine learning algorithm.
3. Feature Transformation: This step involves transforming the features into a format that is
suitable for the machine learning algorithm. Common techniques for feature transformation
include normalization, scaling, or log transformations.
4. Feature Augmentation: This step involves adding new features to the dataset that can provide
additional information to the machine learning algorithm. Feature augmentation can involve
adding new features derived from external sources, such as weather data or demographic
information.

Feature Extraction: Feature Extraction aims to reduce the number of features in a dataset by
creating new features from the existing ones (and then discarding the original features). These new
reduced set of features should then be able to summarize most of the information contained in the
original set of features. In this way, a summarised version of the original features can be created
from a combination of the original set.

Techniques

The following techniques used in feature engineering are as follows –

 Feature Encoding: This step involves encoding categorical data into a format that can be used by
the machine learning algorithm. Common techniques for feature encoding include one-hot
encoding, label encoding, and binary encoding.
 Feature Scaling: This step involves scaling the features so that they are on the same scale. This
can be important if the features have different units or scales, as it can make it easier for the
machine learning algorithm to compare the features.
 One-Hot Encoding: This is a technique used to convert categorical variables into numerical
values by creating a binary column for each category. For example, if there is a categorical
feature like color with categories red, blue, and green, then one-hot encoding will create three
binary columns representing each category.
 Discretization: Discretization is a technique used to convert continuous variables into discrete
values to simplify the model. For example, age can be discretized into age groups like 0-10, 11-
20, 21-30, etc.
 Binning: Binning is a technique used to group continuous variables into bins based on specific
intervals. For example, income can be binned into income ranges like low-income, middle-
income, and high-income.
  Imputation: Imputation is a technique used to fill in missing values in a dataset. Various
imputation techniques are available like mean imputation, median imputation, and mode
imputation

Basis Feature Engineering Feature Selection Feature Extraction

Feature selection, on the Feature extraction involves
Feature engineering
other hand, involves transforming the original data
involves creating new
selecting a subset of the into a new feature space using
features from the existing
Meaning available features that are mathematical techniques such
data to improve the
most relevant for a given as Principal Component
performance of machine
predictive modeling Analysis (PCA) or Linear
learning models.
problem. Discriminant Analysis (LDA).
The goal of feature selection
This may involve tasks The goal of feature extraction is
is to reduce the
such as transforming to identify and extract the most
dimensionality of the data,
variables, creating important and relevant
Purpose which can help to reduce
interaction terms, or information from the original
overfitting, improve model
encoding variables in a data while reducing the
performance, and speed up
way that captures dimensionality of the data.
training
relevant information.

https://www.wallstreetmojo.com/feature-engineering/

What is an optimizer?
Optimizers are algorithms or methods used to minimize an error function (loss function) or to
maximize the efficiency of production. Optimizers are mathematical functions which are
dependent on model’s learnable parameters i.e Weights & Biases. Optimizers help to know how
to change weights and learning rate of neural network to reduce the losses.

Learning Rate
How big/small the steps are gradient descent takes into the direction of the local minimum are
determined by the learning rate, which figures out how fast or slow we will move towards the
optimal weights.
 Learning Rate

https://medium.com/mlearning-ai/optimizers-in-deep-learning-7bf81fed78a0

Feature Selection Concepts & Techniques

Feature selection is a process in machine learning that involves identifying and selecting the
most relevant subset of features out of the original features in a dataset to be used as inputs for
a model. The goal of feature selection is to improve model performance by reducing the number
of irrelevant or redundant features that may introduce noise or bias into the model.

The importance of feature selection lies in its ability to improve model accuracy and
efficiency by reducing the dimensionality of the dataset.

Feature importance technique for features selection

Feature importance techniques such as using estimator such as Random Forest algorithm to
fit a model and select features based on the value of attribute such as feature_importances_ .
The feature_importances_ attribute of the Random Forest estimator can be used to obtain the
relative importance of each feature in the dataset. The feature_importances_ attribute of the
Random Forest estimator provides a score for each feature in the dataset, indicating how
important that feature is for making predictions. These scores are calculated based on the
reduction in impurity (e.g., Gini impurity or entropy) achieved by splitting the data on that
feature. The feature with the highest score is considered the most important, while features with
low scores can be considered less important or even irrelevant. The code below

Feature Extraction Concepts & Techniques

Feature extraction is about extracting/deriving information from the original features set to
create a new features subspace. The primary idea behind feature extraction is to compress the
data with the goal of maintaining most of the relevant information. As with feature selection
techniques, these techniques are also used for reducing the number of features from the original
features set to reduce model complexity, model overfitting, enhance model computation
efficiency and reduce generalization error. The following are different types of feature extraction
techniques:

 Principal component analysis (PCA) for unsupervised data compression.

When to use Feature Selection & Feature Extraction

The key difference between feature selection and feature extraction techniques used for
dimensionality reduction is that while the original features are maintained in the case of
feature selection algorithms, the feature extraction algorithms transform the data onto a new
feature space.

Feature selection techniques can be used if the requirement is to maintain the original features,
unlike the feature extraction techniques which derive useful information from data to construct a
new feature subspace. Feature selection techniques are used when model explainability is a key
requirement.

Feature extraction techniques can be used to improve the predictive performance of the models,
especially, in the case of algorithms that don’t support regularization.

Unlike feature selection, feature extraction usually needs to transform the original data to
features with strong pattern recognition ability, where the original data can be regarded as
features with weak recognition ability.

https://vitalflux.com/machine-learning-feature-selection-feature-extraction/

What is feature extraction/selection?

Straight to the point:

 Extraction: Getting useful features from existing data.

 Selection: Choosing a subset of the original pool of features.
 

Important Deep Learning Terms

Before proceeding, there are a few terms that you should be familiar with.

 Epoch – The number of times the algorithm runs on the whole training dataset.
 Sample – A single row of a dataset.
 Batch – It denotes the number of samples to be taken to for updating the model parameters.
 Learning rate – It is a parameter that provides the model a scale of how much model weights
should be updated.
 Cost Function/Loss Function – A cost function is used to calculate the cost, which is the
difference between the predicted value and the actual value.
 Weights/ Bias – The learnable parameters in a model that controls the signal between two
neurons.

https://www.analyticsvidhya.com/blog/2021/10/a-comprehensive-guide-on-deep-learning-optimizers/