Scribd
Upload
5 views

Experiment No. 10 ML

The document outlines an experiment to analyze the performance of three classification algorithms: Support Vector Machine (SVM), Naïve Bayes, and Decision Tree. It details the implementation steps, including data preprocessing, model training, evaluation metrics, and visualization of results using confusion matrices. The objective is to compare the classifiers based on accuracy, precision, recall, and F1-score using a sample dataset.

Uploaded by

dhirruusinghh
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
5 views

Experiment No. 10 ML

The document outlines an experiment to analyze the performance of three classification algorithms: Support Vector Machine (SVM), Naïve Bayes, and Decision Tree. It details the implementation steps, including data preprocessing, model training, evaluation metrics, and visualization of results using confusion matrices. The objective is to compare the classifiers based on accuracy, precision, recall, and F1-score using a sample dataset.

Uploaded by

dhirruusinghh
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 3

Experiment No.

10

Title: Performance Analysis of Classification Algorithms (SVM, Naïve Bayes, Decision


Tree)

Objective:

 To implement and compare SVM, Naïve Bayes, and Decision Tree classifiers.
 To evaluate their performance using accuracy, precision, recall, and F1-score.
 To visualize results using confusion matrices.

Theory:

1. Support Vector Machine (SVM)

 A supervised learning algorithm that finds the best decision boundary to separate
different classes.
 Uses kernels like linear, polynomial, and RBF for better classification.

2. Naïve Bayes (NB)

 A probabilistic classifier based on Bayes' theorem.


 Assumes independence between features (hence "naïve").
 Works well for text classification and spam filtering.

3. Decision Tree (DT)

 A tree-based classification model that splits data based on feature values.


 Uses criteria like Gini impurity or entropy to create decision rules.

Libraries Required:

python
CopyEdit
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
from sklearn.naive_bayes import GaussianNB
from sklearn.tree import DecisionTreeClassifier
from sklearn.metrics import accuracy_score, classification_report, confusion_matrix
from sklearn.datasets import load_wine # Example dataset
Implementation Steps:

Step 1: Load and Preprocess Dataset

python
CopyEdit
# Load a sample dataset (Wine dataset)
data = load_wine()
X, y = data.data, data.target

# Standardize features for SVM


scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

# Split dataset into training and testing sets


X_train, X_test, y_train, y_test = train_test_split(X_scaled, y, test_size=0.2, random_state=42)

Step 2: Train and Evaluate SVM Model

python
CopyEdit
# Train SVM model
svm_model = SVC(kernel='linear', random_state=42)
svm_model.fit(X_train, y_train)

# Predict and evaluate SVM


y_pred_svm = svm_model.predict(X_test)
print("SVM Accuracy:", accuracy_score(y_test, y_pred_svm))
print("SVM Classification Report:\n", classification_report(y_test, y_pred_svm))

Step 3: Train and Evaluate Naïve Bayes Model

python
CopyEdit
# Train Naïve Bayes model
nb_model = GaussianNB()
nb_model.fit(X_train, y_train)

# Predict and evaluate Naïve Bayes


y_pred_nb = nb_model.predict(X_test)
print("Naïve Bayes Accuracy:", accuracy_score(y_test, y_pred_nb))
print("Naïve Bayes Classification Report:\n", classification_report(y_test, y_pred_nb))
Step 4: Train and Evaluate Decision Tree Model

python
CopyEdit
# Train Decision Tree model
dt_model = DecisionTreeClassifier(random_state=42)
dt_model.fit(X_train, y_train)

# Predict and evaluate Decision Tree


y_pred_dt = dt_model.predict(X_test)
print("Decision Tree Accuracy:", accuracy_score(y_test, y_pred_dt))
print("Decision Tree Classification Report:\n", classification_report(y_test, y_pred_dt))

Step 5: Compare Results Using Confusion Matrices

python
CopyEdit
# Plot Confusion Matrices
fig, axes = plt.subplots(1, 3, figsize=(18, 5))

for ax, model, y_pred, title in zip(


axes,
[svm_model, nb_model, dt_model],
[y_pred_svm, y_pred_nb, y_pred_dt],
["SVM", "Naïve Bayes", "Decision Tree"]
):
cm = confusion_matrix(y_test, y_pred)
sns.heatmap(cm, annot=True, fmt="d", cmap="Blues", ax=ax)
ax.set_title(f"{title} Confusion Matrix")
ax.set_xlabel("Predicted Label")
ax.set_ylabel("True Label")

plt.show()

You might also like