AAM UT-1 QB ANS

(2 Marks Questions):

Q.1) What is feature scaling?

Feature Scaling is a technique to standardize the independent features present in

the data in a fixed range.

It is performed during the data pre-processing to handle highly varying magnitudes

or values or units.

Q.2) State any four applications of random forest.


 Banking: Banking sector mostly uses this algorithm for the identification of
loan risk.

 Medicine: With the help of this algorithm, disease trends and risks of the
disease can be identified.

 Land Use: We can identify the areas of similar land use by this algorithm.

 Marketing: Marketing trends can be identified using this algorithm.

Q.3) How to select value of ‘K’ in K-nearest neighbor Algorithm?
(Mention two methods)

There is no particular way to determine the best value for "K", so we need to try
some values to find the best out of them.

 The most preferred value for K is 5.

 A very low value for K such as K=1 or K=2, can be noisy and lead to the
effects of outliers in the model.

 Large values for K are good, but it may find some difficulties.

The two methods used are:

 Elbow Method: Test different values of K and choose the one where the
error rate stabilizes or decreases marginally.

 Cross-validation: Use techniques like k-fold cross-validation to determine

the best K value by evaluating performance on different subsets of data.
Q.4) List any two advantages and disadvantages of support vector machine

Advantages:

1. Effective in high-dimensional spaces: SVM is capable of handling data with a large

number of features, making it suitable for high-dimensional datasets.

2. Works well with a clear margin of separation: SVM performs particularly well when
there is a distinct gap between classes, as it focuses on maximizing the margin between
them.

3. Robust to overfitting: Especially in high-dimensional spaces, SVM is less prone to

overfitting when used with appropriate regularization.

4. Versatile with different kernels: SVM can be adapted to various types of data,
including non-linear ones, by using kernel tricks, which help in mapping the data to
higher-dimensional spaces.

Disadvantages:

1. Computationally expensive for large datasets: Training an SVM can be slow and
memory-intensive for large datasets due to its complexity, especially when using non-
linear kernels.

2. Difficult to choose the correct kernel function: Selecting the right kernel (e.g., linear,
polynomial, RBF) and tuning the associated parameters (like C and gamma) requires
expertise and can be challenging.

3. Not suitable for noisy data: SVM may not perform well when the data contains a lot of
noise or overlapping classes.

4. Requires careful tuning: SVM has several hyperparameters (like the regularization
parameter, kernel type, and margin) that need to be fine-tuned for optimal performance,
which can be time-consuming.
Q.5) Explain the types of support vector machines

There are 2 types of SVM given below,

Linear SVM: Linear SVM is used for linearly separable data, which means if a
dataset can be classified into two classes by using a single straight line, then such
data is termed as linearly separable data, and classifier is used called as Linear
SVM classifier.

Non-linear SVM: Non-Linear SVM is used for non-linearly separated data, which
means if a dataset cannot be classified by using a straight line, then such data is
termed as non-linear data and classifier used is called as Non-linear SVM
classifier.

Q.6) Enlist any FOUR decision tree terminology.

Root Node: Root node is from where the decision tree starts. It represents the
entire dataset, which further gets divided into two or more homogeneous sets.

Leaf Node: Leaf nodes are the final output node, and the tree cannot be segregated
further after getting a leaf node.

Splitting: Splitting is the process of dividing the decision node/root node into sub-
nodes according to the given conditions.

Branch/Sub Tree: A tree formed by splitting the tree.

Pruning: Pruning is the process of removing the unwanted branches from the tree.

Parent/Child node: The root node of the tree is called the parent node, and other
nodes are called the child nodes.
Q.7) State any TWO advantages of KNN algorithm

 Easy to use: SVM is simple to implement with available tools.

 Handles noisy data well: It’s good at ignoring errors or outliers in data.

 Scales to large datasets: Works well with lots of data, though it can slow
down with huge datasets.

 Versatile: Can be used for both classification and predicting continuous

values.

 Handles many features: Works well with datasets that have many different
characteristics or variables.
(4 Marks Questions):

Q.1) Describe process of feature engineering.

Feature Engineering is the process of creating new features or transforming

existing features to improve the performance of a machine-learning model.

It involves selecting relevant information from raw data and transforming it into a
format that can be easily understood by a model.

The goal is to improve model accuracy by providing more meaningful and relevant
information.

The process of feature engineering is as given below:

 Feature Extraction: Identify relevant variables from raw data (e.g.,

extracting text length from a document).

 Feature Selection: Choose the most important features using methods like
correlation, mutual information, or Recursive Feature Elimination (RFE).

 Feature Transformation: Modify features using scaling (Normalization,

Standardization), encoding categorical variables (One-Hot Encoding, Label
Encoding), or creating polynomial features.

 Feature Creation: Generate new meaningful features, such as time-based

features from timestamps or domain-specific features.

 Handling Missing Values: Impute missing data using mean, median, or

predictive modeling.

 Feature Reduction: Reduce dimensionality using PCA (Principal

Component Analysis) or LDA (Linear Discriminant Analysis).
Q.2) Demonstrate Working of Decision Tree Algorithm with Attribute
Selection Measures

Decision Tree is a Supervised learning technique that can be used for both
classification and Regression problems.

It is a tree-structured classifier, where internal nodes represent the features of a

dataset, branches represent the decision rules and each leaf node represents the
outcome.

Steps to build a Decision Tree:

1. Calculate the Entropy for the dataset:

2. Compute Information Gain for each feature:

The feature with the highest Information Gain (IG) is chosen for the first split.

3. Repeat the splitting process:

o Compute entropy and IG for remaining features.

o Continue splitting until stopping criteria (e.g., pure nodes, max depth) are met.

4. Make Predictions:

o Traverse the tree using feature values until a leaf node is reached.
o Assign the label of the leaf node as the prediction.

Example:
If we have a dataset with features Weather (Sunny, Rainy, Overcast) and a target Play
(Yes/No), we compute IG for Weather and split the tree accordingly.
Q.3) with the suitable example, explain how Bayes Theorem is applied

Bayes’ Theorem:

P (A ∣ B) = Is the conditional probability of event A occurring, given that B is true.

P (B ∣ A) = Is the conditional probability of event B occurring, given that A is true.

P (A) and P (B) = Are the probabilities of A and B occurring independently of one
another.
Example of Bayes Theorem:

Three bags contain 6 red, 4 black; 4 red, 6 black, and 5 red, 5 black balls
respectively. One of the bag is selected at random and a ball is drawn from it. If the
ball drawn is red, find the probability that it is drawn from the first bag.

Solution:
Let E1, E2, E3, and A be the events defined as follows:

E1 = bag first is chosen, E2 = bag second is chosen, E3 = bag third is chosen,

A = ball drawn is red

Each bag is equally likely to be chosen:

Step 1: Probabilities of Drawing a Red Ball from Each Bag:

Step 2: Find P (A) (Total Probability of Drawing Red):

Step 3: Apply Bayes’ Theorem

Thus, the probability that the red ball was drawn from the first bag is 2/5 or 40%
Q.4) Describe types of ensemble learning methods.

Ensemble learning is a machine learning technique where multiple models (weak

learners) are combined to improve overall prediction accuracy and robustness.

Instead of relying on a single model, ensemble methods aggregate the predictions

of multiple models to reduce variance, bias, and improve generalization.

Types of Ensemble Learning:

1. Bagging (Bootstrap Aggregating):

 Multiple models are trained on different subsets of the dataset using

bootstrapping (sampling with replacement).
 Final prediction is obtained by averaging (for regression) or majority voting
(for classification).
 Example: Random Forest (an ensemble of decision trees).

2. Boosting:

 Models are trained sequentially, with each new model correcting the errors of the
previous one.
 Boosting gives higher weights to misclassified instances to improve performance.
 Example: AdaBoost, Gradient Boosting, XGBoost.

3. Stacking:

 Uses multiple base models and combines their outputs using a meta-learner (a
higher-level model).
 The meta-learner learns how to best combine the base models’ predictions.
 Example: Combining Decision Trees, SVM, and Neural Networks.

4. Voting & Averaging:

 Aggregates predictions from multiple models using majority voting (for

classification) or averaging (for regression).
 Example: Using Logistic Regression, KNN, and SVM together to make a final
decision.

Advantages:
 Increases model accuracy and reduces overfitting.
 Works well with both classification and regression tasks.
 Reduces variance and improves model robustness.
Q.5) Consider following training dataset of weather, apply Naive Bayes
Below is a training data set of weather and corresponding target variable ‘Play’
(suggesting possibilities of playing).
Now, we need to classify whether players will play or not based on weather
condition.
Problem: Players will play if the weather is sunny. Is this statement correct?
We can solve it using the above-discussed method of posterior Probability.
Dataset:

Step 1: Frequency Table (Counts for Weather & Play):

Step 2: Likelihood Table (Conditional Probabilities):

Given data:

Step 1: Apply Bayes’ Theorem

Step 2: Interpretation

Since (which is higher probability), players are

more likely to play when the weather is sunny.

Conclusion: The statement is likely correct, but not always certain.