AAM UT-1 QB ANS

(2 Marks Questions):

Q.1) What is feature scaling?

Feature Scaling is a technique to standardize the independent features present in

the data in a fixed range.

It is performed during the data pre-processing to handle highly varying magnitudes

or values or units.

Types of Scaling:

Normalization: (Min-Max Scaling)

• Min-Max normalization scales the data to fit within a specified range,
usually between 0 and 1.

• The formula for min-max normalization is:

Standardization: (Z-Score Normalization)

• Z-score normalization, or standardization, scales the data so that it has a
mean of 0 and a standard deviation of 1.
• The formula for z-score normalization is:
Q.2) State any four applications of random forest.

 Banking: Banking sector mostly uses this algorithm for the identification of
loan risk.

 Medicine: With the help of this algorithm, disease trends and risks of the
disease can be identified.

 Land Use: We can identify the areas of similar land use by this algorithm.

 Marketing: Marketing trends can be identified using this algorithm.

Q.3) How to select value of ‘K’ in K-nearest neighbor Algorithm?
(Mention two methods)

There is no particular way to determine the best value for "K", so we need to try
some values to find the best out of them.

 The most preferred value for K is 5.

 A very low value for K such as K=1 or K=2, can be noisy and lead to the
effects of outliers in the model.

 Large values for K are good, but it may find some difficulties.

The two methods used are:

 Elbow Method:

Test different values of K and choose the one where the error rate stabilizes
or decreases marginally.

 Cross-validation:

Use techniques like k-fold cross-

validation to determine the best K
value by evaluating performance on
different subsets of data.
Q.4) List any two advantages and disadvantages of support vector machine

Advantages:

1. Effective in high-dimensional spaces: SVM is capable of handling data

with a large number of features, making it suitable for high-dimensional
datasets.

2. Works well with a clear margin of separation: SVM performs particularly

well when there is a distinct gap between classes, as it focuses on
maximizing the margin between them.

3. Robust to overfitting: Especially in high-dimensional spaces, SVM is less

prone to overfitting when used with appropriate regularization.

4. Versatile with different kernels: SVM can be adapted to various types of

data, including non-linear ones, by using kernel tricks, which help in
mapping the data to higher-dimensional spaces.

Disadvantages:

1. Computationally expensive for large datasets: Training an SVM can be

slow and memory-intensive for large datasets due to its complexity,
especially when using non-linear kernels.

2. Difficult to choose the correct kernel function: Selecting the right kernel
(e.g., linear, polynomial, RBF) and tuning the associated parameters (like C
and gamma) requires expertise and can be challenging.

3. Not suitable for noisy data: SVM may not perform well when the data
contains a lot of noise or overlapping classes.

4. Requires careful tuning: SVM has several hyper-parameters (like the

regularization parameter, kernel type, and margin) that need to be fine-tuned
for optimal performance, which can be time-consuming.
Q.5) Explain the types of support vector machines

Linear SVM: Linear SVM is used for linearly separable data, which means if a
dataset can be classified into two classes by using a single straight line, then such
data is termed as linearly separable data, and classifier is used called as Linear
SVM classifier.

Non-linear SVM: Non-Linear SVM is used for non-linearly separated data, which
means if a dataset cannot be classified by using a straight line, then such data is
termed as non-linear data and classifier used is called as Non-linear SVM
classifier.

Q.6) Enlist any FOUR decision tree terminology.

Root Node: Root node is from where the decision tree starts. It represents the
entire dataset, which further gets divided into two or more homogeneous sets.

Leaf Node: Leaf nodes are the final output node, and the tree cannot be segregated
further after getting a leaf node.

Splitting: Splitting is the process of dividing the decision node/root node into sub-
nodes according to the given conditions.

Branch/Sub Tree: A tree formed by splitting the tree.

Pruning: Pruning is the process of removing the unwanted branches from the tree.

Parent/Child node: The root node of the tree is called the parent node, and other
nodes are called the child nodes.
Q.7) State any TWO advantages of KNN algorithm

 Easy to use: SVM is simple to implement with available tools.

 Handles noisy data well: It’s good at ignoring errors or outliers in data.

 Scales to large datasets: Works well with lots of data, though it can slow
down with huge datasets.

 Versatile: Can be used for both classification and predicting continuous

values.

 Handles many features: Works well with datasets that have many different
characteristics or variables.
(4 Marks Questions):

Q.1) Describe process of feature engineering.

Feature Engineering is the process of creating new features or transforming

existing features to improve the performance of a machine-learning model.

It involves selecting relevant information from raw data and transforming it into a
format that can be easily understood by a model.

The goal is to improve model accuracy by providing more meaningful and relevant
information. The process of feature engineering is as given below:

 Feature Extraction: Identify relevant variables from raw data (e.g.,

extracting text length from a document).

 Feature Selection: Choose the most important features using methods like
correlation, mutual information, or Recursive Feature Elimination (RFE).

 Feature Transformation: Modify features using scaling (Normalization,

Standardization), encoding categorical variables (One-Hot Encoding, Label
Encoding), or creating polynomial features.

 Feature Creation: Generate new meaningful features, such as time-based

features from timestamps or domain-specific features.
 Handling Missing Values: Impute missing data using mean, median, or
predictive modeling.

 Feature Reduction: Reduce dimensionality using PCA (Principal

Component Analysis) or LDA (Linear Discriminant Analysis).
Q.2) Demonstrate Working of Decision Tree Algorithm with Attribute
Selection Measures (ASM)

Decision Tree working:

Step-1: Begin the tree with the root node, says S, which contains the complete
dataset.

Step-2: Find the best attribute in the dataset using Attribute Selection Measure
(ASM).

Step-3: Divide the S into subsets that contains possible values for the best
attributes.

Step-4: Generate the decision tree node, which contains the best attribute.

Step-5: Recursively make new decision trees using the subsets of the dataset
created in step -3. Continue this process until a stage is reached where you cannot
further classify the nodes and called the final node as a leaf node.

Attribute Selection Measure (ASM):

ASM is a technique used for the selecting best attribute for discrimination among
tuples.

It gives rank to each attribute and the best attribute is selected as splitting criterion.

There are two popular techniques for ASM, which are:

1. Information Gain:

It calculates how much information a feature provides us about a class.

According to the value of information gain, we split the node and build the
decision tree.

2. Gini Index: Gini Index aims to decrease the impurities from the root nodes
(at the top of decision tree) to the leaf nodes of a decision tree model.
Example:

Suppose there is a candidate who has a job offer and wants to decide whether he
should accept the offer or Not. So, to solve this problem, the decision tree starts
with the root node (Salary attribute by ASM).
Q.3) With suitable example, explain how Naïve Bayes Theorem is applied

Naïve Bayes’ Theorem:

P (A ∣ B) = Probability of event A occurring given B (Posterior Probability)

P (B ∣ A) = Probability of event B occurring given A (Likelihood)

P (A) = Prior probability of event A

P (B) = Prior probability of event B

Example of Naïve Bayes Theorem:

Below is a training data set of weather and corresponding target variable ‘Play’
(suggesting possibilities of playing).
Now, we need to classify whether players will play or not based on weather
condition.
Problem: Players will play if the weather is sunny. Is this statement correct?
We can solve it using the above-discussed method of posterior Probability.
Dataset: [ S– Sunny O– Overcast R– Rainy (N– No Y – Yes) ]

Step 1: Frequency Table (Counts for Weather & Play):

Step 2: Likelihood Table (Conditional Probabilities):

Given data:

Step 1: Apply Bayes’ Theorem

Step 2: Interpretation

Since (which is higher probability), players are

more likely to play when the weather is sunny.

Conclusion: The statement is likely correct, but not always certain.

Q.4) Describe types of ensemble learning methods

Ensemble learning is a machine learning technique where multiple models (weak

learners) are combined to improve overall prediction accuracy and robustness.

Instead of relying on a single model, ensemble methods aggregate the predictions

of multiple models to reduce variance, bias, and improve generalization.

Types of Ensemble Learning:

1. Bagging (Bootstrap Aggregating):

 Multiple models are trained on different subsets of the dataset

using bootstrapping (sampling with replacement).
 Final prediction is obtained by averaging (for regression) or
majority voting (for classification).
 Example: Random Forest (an ensemble of decision trees).

2. Boosting:

 Models are trained sequentially, with each new model correcting the
errors of the previous one.
 Boosting gives higher weights to misclassified instances to improve
performance.
 Example: AdaBoost, Gradient Boosting, XGBoost.
3. Stacking:

 Uses multiple base models and combines their outputs using a meta-
learner (a higher-level model).
 The meta-learner learns how to best combine the base models’
predictions.
 Example: Combining Decision Trees, SVM, and Neural Networks.

4. Voting & Averaging:

 Aggregates predictions from multiple models using majority voting

(for classification) or averaging (for regression).
 Example: Using Logistic Regression, KNN, and SVM together to
make a final decision.

Advantages:
 Increases model accuracy and reduces overfitting.
 Works well with both classification and regression tasks.
 Reduces variance and improves model robustness.
Q.5) Consider following training dataset of weather, apply Naive Bayes
Below is a training data set of weather and corresponding target variable ‘Play’
(suggesting possibilities of playing).
Now, we need to classify whether players will play or not based on weather
condition.
Problem: Players will play if the weather is sunny. Is this statement correct?
We can solve it using the above-discussed method of posterior Probability.
Dataset: [ S– Sunny O– Overcast R– Rainy (N– No Y – Yes) ]

Step 1: Frequency Table (Counts for Weather & Play):

Step 2: Likelihood Table (Conditional Probabilities):

Given data:

Step 1: Apply Bayes’ Theorem

Step 2: Interpretation

Since (which is higher probability), players are

more likely to play when the weather is sunny.

Conclusion: The statement is likely correct, but not always certain.

Q.6) Implement Python code for given steps to implement Naïve Bayes
Algorithm

from sklearn.datasets import load_iris scaler = StandardScaler()

from sklearn.model_selection import X_train =
train_test_split scaler.fit_transform(X_train)
from sklearn.preprocessing import X_test = scaler.transform(X_test)
StandardScaler
from sklearn.neighbors import
# Create and train the KNN
KNeighborsClassifier
classifier
from sklearn.metrics import
accuracy_score k = 3 # Number of neighbors
knn =
KNeighborsClassifier(n_neighbors=k)
# Load the Iris dataset
knn.fit(X_train, y_train)
iris = load_iris()
X, y = iris.data, iris.target
# Make predictions
y_pred = knn.predict(X_test)
# Split into training and testing sets
X_train, X_test, y_train, y_test =
train_test_split(X, y, test_size=0.2, # Evaluate the model
random_state=42) accuracy = accuracy_score(y_test,
y_pred)
print(f'KNN Model Accuracy:
# Standardize the dataset
{accuracy:.2f}')
(important for KNN)
Q.7) Describe Feature Selection Methods

Feature selection:

Feature selection is a process that chooses a subset of features from the original
features so that the feature space is optimally reduced according to a certain
criterion.
The goal is to reduce the dimensionality of the dataset while retaining the most
important features.
There are several methods for feature selection, including:
 Filter Methods
 Wrapper Methods
 Embedded Methods.

Filter Methods:
• These methods are generally used while doing the pre-processing step.
• These methods select features from the dataset irrespective of the use of any
machine learning algorithm
• In terms of computation, they are very fast and inexpensive and are very
good for removing duplicated, correlated, redundant features
• Selection of feature is evaluated individually which can sometimes help
when features are in isolation (don’t have a dependency on other features)
but will lag when a combination of features can lead to increase in the
overall performance of the model.
Wrapper Methods:

 These methods evaluate subsets of features by training a model and checking

performance.

 They are computationally expensive but often provide better results than
filter methods.

 Wrapper methods consider the interaction between features, which helps

improve model performance.

 Since they rely on the actual learning algorithm, they can be slower but more
accurate.

Embedded methods
• In embedded methods, the feature selection algorithm is blended as part of
the learning algorithm, thus having its own built-in feature selection
methods.
• Embedded methods encounter the drawbacks of filter and wrapper methods
and merge their advantages.
• These methods are faster like those of filter methods and more accurate than
the filter methods and take into consideration a combination of features as
well.
Q.8) Explain Random Forest Algorithm In Detail.

 Random forest is a supervised learning technique.

 It can be used for both Classification and Regression problems in Machine

Learning.

 It is based on the concept of ensemble learning, which is a process of

combining multiple classifiers to solve a complex problem and to improve
the performance of the model.

 As the name suggests, "Random Forest is a classifier that contains a

number of decision trees on various subsets of the given dataset and takes
the average (regression) or majority votes (classification) to improve the
predictive accuracy of that dataset."
Why use Random Forest?

 It takes less training time as compared to other algorithms.

 It predicts output with high accuracy, even for the large dataset it runs
efficiently.

 It can also maintain accuracy when a large proportion of data is missing.

Random Forest Working:

Step-1: Select random K data points Step-4: Repeat Step 1 & 2.

from the training set.
Step-5: For new data points, find the
Step-2: Build the decision trees predictions of each decision tree, and
associated with the selected data assign the new data points to the
points (Subsets).
category that wins the majority votes.
Step-3: Choose the number N for
decision trees that you want to build.

Advantages of Random Forest

1. Random Forest is capable of performing both Classification and Regression

tasks.

2. It is capable of handling large datasets with high dimensionality.

3. It enhances the accuracy of the model and prevents the overfitting issue.

Disadvantages of Random Forest


Although random forest can be used for both classification and regression tasks, it
is not more suitable for Regression tasks.
Q.9) Describe The Working Of KNN Algorithm With Example.

K-Nearest Neighbor is one of the simplest Machine Learning algorithms based on

Supervised Learning technique.

K-NN algorithm assumes the similarity between the new case/data and available
cases and put the new case into the category that is most similar to the available
categories.

K-NN algorithm stores all the available data and classifies a new data point based
on the similarity.

This means when new data appears then it can be easily classified into a well suite
category by using K- NN algorithm.

K-NN algorithm can be used for Regression as well as for Classification but
mostly it is used for the Classification problems.

It is also called a lazy learner algorithm because it does not learn from the
training set immediately instead it stores the dataset and at the time of
classification, it performs an action on the dataset.
Working of K-NN

Step-1: Select the number K of the neighbors

Step-2: Calculate the Euclidean distance of K number of neighbors
Step-3: Take the K nearest neighbors as per the calculated Euclidean distance.
Step-4: Among these k neighbors, count the number of the data points in each
category.
Step-5: Assign the new data points to that category for which the number of the
neighbor is maximum.
Step-6: Our model is ready.
Q.10) Explain Support Vector Machines with Suitable Example

Support Vector Machine or SVM is one of the most popular Supervised Learning
algorithms.
It is used for Classification as well as Regression problems. However, primarily, it
is used for Classification problems in Machine Learning.
The goal of the SVM algorithm is to create the best line or decision boundary that
can segregate n-dimensional space into classes so that we can easily put the new
data point in the correct category in the future.
This best decision boundary is called a hyperplane.
SVM chooses the extreme points/vectors that help in creating the hyperplane.
These extreme cases are called as support vectors, and hence algorithm is termed
as Support Vector Machine.
SVM algorithm can be used for Face detection, image classification, text
categorization, etc.
Hyperplane and Support Vectors in the SVM algorithm:

Hyperplane:

There can be multiple lines/decision boundaries to segregate the classes in n-

dimensional space, but we need to find out the best decision boundary that helps to
classify the data points.

This best boundary is known as the hyperplane of SVM.

Support Vectors:

The data points or vectors that are the closest to the hyperplane and which affect
the position of the hyperplane are termed as Support Vector.

Since these vectors support the hyperplane, hence called a Support vector.

Types of SVM

Linear SVM: Linear SVM is used for linearly separable data, which means if a
dataset can be classified into two classes by using a single straight line, then such
data is termed as linearly separable data, and classifier is used called as Linear
SVM classifier.
Non-linear SVM: Non-Linear SVM is used for non-linearly separated data, which
means if a dataset cannot be classified by using a straight line, then such data is
termed as non-linear data and classifier used is called as Non-linear SVM
classifier.

Extras:
Feature Selection Diagram: