PRINCE DR.K.

VASUDEVAN COLLEGE OF
ENGINEERING AND TECHNOLOGY
PONMAR,CHENNAI-600 048
DEPARTMENT OF CSE (CYBER SECURITY)

CS3491 ARTIFICIAL INTELLIGENCE MACHINE LEARNING LABORATORY

RECORD
NAME :

REGISTER NO.:

SEMESTER :

BRANCH :
SNO DATE TITLE SIGNATURE

1(a)Implementation of Uninformed search

algorithm Breadth First Search Algorithm
1

1(b) Implementation of Uninformed searchalgorithm

depth First Search Algorithm

Implementation of Informed search algorithm A*

Algorithm
2

Implementation of Navie Bayes Classifier

Implementation of Bayesian Networks

Implementation of Regression models

6(a) Implementation of decision trees

6

6(b) Implementation of Random Forests

Implementation of SVM Models

7
 Implementation of Ensembling Methods

Implement clustering algorithms

Build simple NN models

10

Build simple NN models

11
 Reg.no:411622149007

Ex. No:1(a) Implementation of Uninformed search algorithms

Breadth First Search Algorithm

AIM:
To implement the uninformed Search Algorithms like Breadth First Search using Python.
ALGORITHM

• Step 1: Place the starting node into the OPEN list.

• Step 2: If the OPEN list is empty, Stop and return failure.
• Step 3: Remove the node n, from the OPEN list which has the lowest value of h(n),
and places it in the CLOSED list.
• Step 4: Expand the node n, and generate the successors of node n.
• Step 5: Check each successor of node n, and find whether any node is a goal node or
not. If any successor node is goal node, then return success and terminate the
search, else proceed to Step 6.
• Step 6: For each successor node, algorithm checks for evaluation function f(n), and
then check if the node has been in either OPEN or CLOSED list. If the node has not
been in both lists, then add it to the OPEN list.
• Step 7: Return to Step 2.

PROGRAM:

from queue import PriorityQueue

v = 14
graph = [[] for i in range(v)]

def best_first_search(actual_Src, target, n):

visited = [False] * n

pq = PriorityQueue()
pq.put((0, actual_Src))
visited[actual_Src] = True
while pq.empty() == False:
u = pq.get()[1]
 Reg.no:411622149007

print(u, end=" ")if u

== target:
break

for v, c in graph[u]:
if visited[v] == False:
visited[v] = True
pq.put((c, v))
print()

# Function for adding edges to graph

def addedge(x, y, cost):

graph[x].append((y, cost))

graph[y].append((x, cost))
# The nodes shown in above example(by alphabets) are
# implemented using integers addedge(x,y,cost);
addedge(0, 1, 3)
addedge(0, 2, 6)
addedge(0, 3, 5)
addedge(1, 4, 9)
addedge(1, 5, 8)
addedge(2, 6, 12)
addedge(2, 7, 14)
addedge(3, 8, 7)
addedge(8, 9, 5)
addedge(8, 10, 6)
addedge(9, 11, 1)
addedge(9, 12, 10)
addedge(9, 13, 2)
source = 0
target = 9
best_first_search(source, target, v)
 Reg.no:411622149007

OUTPUT:

013289

RESULT:
Thus the Program for uninformed search Algorithm like Breadth first search was executed and
output has been verified successfully.
 Reg.no:411622149007

Ex. No:1(b) Implementation of Uninformed search algorithms

Breadth First Search Algorithm

AIM:
To implement the uninformed Search Algorithms like Depth First Search using Python.
ALGORITHM:

• Step 1: Start the Program.

• Step 2: Add the node to the top of the stack.
• Step 3: Marked it as visited.
• Step 4: Check if this node has any adjacent nodes:
• Step 5: If it has adjacent nodes, then we ensure that they have not been visited already,
and then visited it.
• Step 6: Removed it from the stack if it had no adjacent nodes.
• Step 7: Stop the Program.

PROGRAM:
def dfs(node, graph, visited, component):
component.append(node) # Store answer
visited[node] = True # Mark visited

# Traverse to each adjacent node of a node

for child in graph[node]:
if not visited[child]: # Check whether the node is visited or not
dfs(child, graph, visited, component) # Call the dfs recursively

if name == " main ":

# Graph of nodes

graph = {
0: [2],
1: [2, 3],
2: [0, 1, 4],

3: [1, 4],
4: [2, 3]
 Reg.no:411622149007

}
node = 0 # Starting node
visited = [False]*len(graph) # Make all nodes to False initially

component = []
dfs(node, graph, visited, component) # Traverse to each node of a graph
print(f"Following is the Depth-first search: {component}") # Print the answer
 Reg.no:411622149007

OUTPUT:
Following is the Depth-first search: [0, 2, 1, 3, 4]

RESULT:
Thus the Program for uninformed search Algorithm like Depth first search was executed and
output has been verified successfully.
 Reg.no:411622149007

Ex. No:2 Implementation of Informed search algorithms

A* Algorithm

AIM:
To implement the Informed Search Algorithms like A* Algorithm using Python.
ALGORITHM:

• Step 1: Start the Program.

• Step 2: Place the starting node in the OPEN list.
• Step 3: Check if the OPEN list is empty or not, if the list is empty then return failure and
stops.
• Step 4: Select the node from the OPEN list which has the smallest value of evaluation
function (g+h), if node n is goal node, then return success and stop, otherwise
• Step 5: Expand node n and generate all of its successors, and put n into the closed list.
For each successor n', check whether n' is already in the OPEN or CLOSED list, if not then
compute evaluation function for n' and place into Open list.
• Step 6: Else if node n' is already in OPEN and CLOSED, then it should be attached to the
back pointer which reflects the lowest g(n') value.
• Step 7: Return to Step 3.
• Step 8: Stop the Program.

PROGRAM:

from collections import deque

class Graph:

def init (self, adjac_lis):

self.adjac_lis = adjac_lis

def get_neighbors(self, v):

return self.adjac_lis[v]

# This is heuristic function which is having equal values for all nodes

def h(self, n):

H={
 Reg.no:411622149007

'A': 1,

'B': 1,

'C': 1,

'D': 1

return H[n]

def a_star_algorithm(self, start, stop):

# In this open_lst is a lisy of nodes which have been visited, but who's

# neighbours haven't all been always inspected, It starts off with the start

#node

# And closed_lst is a list of nodes which have been visited

# and who's neighbors have been always inspected

open_lst = set([start])

closed_lst = set([])

# poo has present distances from start to all other nodes

# the default value is +infinity

poo = {}

poo[start] = 0

# par contains an adjac mapping of all nodes

par = {}

par[start] = start

while len(open_lst) > 0:

n = None

# it will find a node with the lowest value of f() -

for v in open_lst:

if n == None or poo[v] + self.h(v) < poo[n] + self.h(n):

 Reg.no:411622149007

n = v;

if n == None:

print('Path does not exist!')

return None

# if the current node is the stop

# then we start again from start

if n == stop:

reconst_path = []

while par[n] != n:

reconst_path.append(n)

n = par[n]

reconst_path.append(start)

reconst_path.reverse()

print('Path found: {}'.format(reconst_path))

return reconst_path

# for all the neighbors of the current node do

for (m, weight) in self.get_neighbors(n):

# if the current node is not presentin both open_lst and closed_lst

# add it to open_lst and note n as it's par

if m not in open_lst and m not in closed_lst:

open_lst.add(m)

par[m] = n

poo[m] = poo[n] + weight

# otherwise, check if it's quicker to first visit n, then m

# and if it is, update par data and poo data

# and if the node was in the closed_lst, move it to open_lst

 Reg.no:411622149007

else:

if poo[m] > poo[n] + weight:

poo[m] = poo[n] + weight

par[m] = n

if m in closed_lst:

closed_lst.remove(m)

open_lst.add(m)

# remove n from the open_lst, and add it to closed_lst

# because all of his neighbors were inspected

open_lst.remove(n)

closed_lst.add(n)

print('Path does not exist!')

return None

INPUT:

adjac_lis = {

'A': [('B', 1), ('C', 3), ('D', 7)],

'B': [('D', 5)],

'C': [('D', 12)]

graph1 = Graph(adjac_lis)

graph1.a_star_algorithm('A', 'D')
 Reg.no:411622149007

OUTPUT:

Path found: ['A', 'B', 'D']

['A', 'B', 'D']

RESULT:
Thus the Program for informed search Algorithm like A* was executed and output has been
verified successfully.
 Reg.no:411622149007

Ex. No:3 Implementation of Navie Bayes Classifier

AIM:
To implement the Navie Bayes Classifier using Python.
ALGORITHM:

• Step 1: Start the Program.

• Step 2: Import all the modules.
• Step 3: Load data from input file.
• Step 4: Convert the given dataset into frequency tables.
• Step 5: Generate Likelihood table by finding the probabilities of given features.
• Step 6: Now, use Bayes theorem to calculate the posterior probability.
• Step 7: Train the classifier.
• Step 8: Compute the accuracy.
• Step 9: Visualize the performance of the classifier.
• Step 10: Stop the Program.

PROGRAM:

# Importing all modules or library

import numpy as np
import matplotlib.pyplot as plt
from sklearn.naive_bayes import GaussianNB
from sklearn.model_selection import train_test_split, cross_val_score
input_file = '/content/data_multivar_nb.txt' # Giving the file path to
read our data

# Load data from input file

data = np.loadtxt(input_file, delimiter=',')

# Here the speration of the data is done {X, y}

X, y = data[:, :-1], data[:, -1]
# Have already import the GaussianNB in importing part
# From navie bayes, which is from sklearn
# Create Navies Bayes Classifier
classifier = GaussianNB()

# Train the Classifier

classifier.fit(X,y)
# Predict the values for training data
y_pred = classifier.predict(X)
 Reg.no:411622149007

# Compute accuracy
accuracy = 100.0 * (y == y_pred).sum() / X.shape[0]
print("Accuracy of Naive Bayes classifier =", round(accuracy, 2), "%")
# Defining the Visualizer
def visualize_classifier(classifier, X, y):
# Define the minimum and maximum values for X and Y
# that will be used in the mesh grid
min_x, max_x = X[:, 0].min() - 1.0, X[:, 0].max() + 1.0
min_y, max_y = X[:, 1].min() - 1.0, X[:, 1].max() + 1.0

# Define the step size to use in plotting the mesh grid

mesh_step_size = 0.01

# Define the mesh grid of X and Y values

x_vals, y_vals = np.meshgrid(np.arange(min_x, max_x, mesh_step_size
), np.arange(min_y, max_y, mesh_step_size))

# Run the classifier on the mesh grid

output = classifier.predict(np.c_[x_vals.ravel(), y_vals.ravel()])

# Reshape the output array

output = output.reshape(x_vals.shape)
# Create a plot
plt.figure()

# Choose a color scheme for the plot

plt.pcolormesh(x_vals, y_vals, output, cmap=plt.cm.gray)

# Overlay the training points on the plot

plt.scatter(X[:, 0], X[:, 1], c=y, s=75, edgecolors='black', linewi
dth=1, cmap=plt.cm.Paired)

# Specify the boundaries of the plot

plt.xlim(x_vals.min(), x_vals.max())
plt.ylim(y_vals.min(), y_vals.max())

# Specify the ticks on the X and Y axes

plt.xticks((np.arange(int(X[:, 0].min() -
1), int(X[:, 0].max() + 1), 1.0)))
plt.yticks((np.arange(int(X[:, 1].min() -
1), int(X[:, 1].max() + 1), 1.0)))
plt.title(f'{a}')
plt.show()
# Visualize the performance of the classifier
visualize_classifier(classifier, X, y)
# Cross validation
 Reg.no:411622149007

# Split data into training and test data

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2
, random_state=3)

#Creating a classifier
classifier_new = GaussianNB()

#Train the classifier

classifier_new.fit(X_train, y_train)

# Predict the test data

y_test_pred = classifier_new.predict(X_test)
# compute accuracy of the classifier
accuracy = 100.0 * (y_test == y_test_pred).sum() / X_test.shape[0]
print("Accuracy of the new classifier =", round(accuracy, 2), "%")

# Visualize the performance of the classifier

visualize_classifier(classifier_new, X_test, y_test)

print('Predicted output :-', y_test_pred)

num_folds = 3

accuracy_values = cross_val_score(classifier, X, y, scoring='accuracy',

cv=num_folds)
print("Accuracy: " + str(round(100*accuracy_values.mean(), 2)) + "%")

precision_values = cross_val_score(classifier, X, y, scoring='precision

_weighted', cv=num_folds)
print("Precision: " + str(round(100*precision_values.mean(), 2)) + "%")

recall_values = cross_val_score(classifier, X, y, scoring='recall_weigh

ted', cv=num_folds)
print("Recall: " + str(round(100*recall_values.mean(), 2)) + "%")

f1_values = cross_val_score(classifier, X, y, scoring='f1_weighted', cv

=num_folds)
print("F1: " + str(round(100*f1_values.mean(), 2)) + "%")
 Reg.no:411622149007

OUTPUT:

Accuracy of Naive Bayes classifier = 99.75 %

Accuracy of the new classifier = 100.0 %

 Reg.no:411622149007

Predicted output :- [0. 0. 1. 2. 3. 1. 0. 0. 2. 0. 1. 3. 3. 3. 3. 2. 1.

2. 2. 0. 2. 0. 3. 2.
1. 2. 3. 3. 0. 0. 2. 1. 0. 2. 3. 0. 1. 0. 1. 0. 1. 3. 2. 1. 3. 2. 3.
3.
0. 1. 2. 1. 3. 2. 3. 2. 3. 0. 3. 1. 0. 2. 0. 2. 2. 3. 2. 0. 1. 2. 1.
1.
0. 1. 0. 2. 2. 3. 2. 2.]

Accuracy: 99.75%
Precision: 99.76%
Recall: 99.75%
F1: 99.75%

RESULT:
Thus the Python Program for Navie Bayes Classifier was executed and output has been verified
successfully.
 Reg.no:411622149007

Ex. No:4 Implementation of Bayesian Networks

AIM:
To implement the Bayesian Networks using Python.
ALGORITHM:

• Step 1: Start the Program.

• Step 2: Import all the modules.
• Step 3: Load data from input file.
• Step 4: Convert the given dataset into frequency tables.
• Step 5: Generate Likelihood table by finding the probabilities of given features.
• Step 6: Now, use Bayes theorem to calculate the posterior probability.
• Step 7: Train the classifier.
• Step 8: Compute the accuracy.
• Step 9: Visualize the performance of the classifier.
• Step 10: Stop the Program.

PROGRAM:

import torch
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
x = np.linspace(0, 0.5, 1000)
ε = 0.02 * np.random.randn(x.shape[0])
y = x + 0.3 * np.sin(2 * np.pi * (x + ε)) + 0.3 * np.sin(4 * np.pi * (x
+ ε)) + ε

fig, ax = plt.subplots(figsize=(10, 5))

ax.plot(x, y, 'o', markersize=1);
import pyro
import pyro.distributions as dist
from pyro.nn import PyroModule, PyroSample
import torch.nn as nn
from pyro.infer.autoguide import AutoDiagonalNormal
from pyro.infer import SVI, Trace_ELBO, Predictive
from tqdm.auto import trange, tqdm
class Model(PyroModule):
def init (self, h1=20, h2=20):
super(). init ()
self.fc1 = PyroModule[nn.Linear](1, h1)
 Reg.no:411622149007

self.fc1.weight = PyroSample(dist.Normal(0., 1.).expand([h1, 1]

).to_event(2))
self.fc1.bias = PyroSample(dist.Normal(0., 1.).expand([h1]).to_
event(1))
self.fc2 = PyroModule[nn.Linear](h1, h2)
self.fc2.weight = PyroSample(dist.Normal(0., 1.).expand([h2, h1
]).to_event(2))
self.fc2.bias = PyroSample(dist.Normal(0., 1.).expand([h2]).to_
event(1))
self.fc3 = PyroModule[nn.Linear](h2, 1)
self.fc3.weight = PyroSample(dist.Normal(0., 1.).expand([1, h2]
).to_event(2))
self.fc3.bias = PyroSample(dist.Normal(0., 1.).expand([1]).to_e
vent(1))
self.relu = nn.ReLU()
def forward(self, x, y=None):
x = x.reshape(-1, 1)
x = self.relu(self.fc1(x))
x = self.relu(self.fc2(x))
mu = self.fc3(x).squeeze()
sigma = pyro.sample("sigma", dist.Uniform(0., 1.))
with pyro.plate("data", x.shape[0]):
obs = pyro.sample("obs", dist.Normal(mu, sigma), obs=y)
return mu
model = Model()
guide = AutoDiagonalNormal(model)
adam = pyro.optim.Adam({"lr": 1e-3})
svi = SVI(model, guide, adam, loss=Trace_ELBO())

pyro.clear_param_store()
bar = trange(20000)
x_train = torch.from_numpy(x).float()
y_train = torch.from_numpy(y).float()
for epoch in bar:
loss = svi.step(x_train, y_train)
bar.set_postfix(loss=f'{loss / x.shape[0]:.3f}')
predictive = Predictive(model, guide=guide, num_samples=500)
x_test = torch.linspace(-0.5, 1, 3000)
preds = predictive(x_test)

y_pred = preds['obs'].T.detach().numpy().mean(axis=1)
y_std = preds['obs'].T.detach().numpy().std(axis=1)

fig, ax = plt.subplots(figsize=(10, 5))

ax.plot(x, y, 'o', markersize=1)
ax.plot(x_test, y_pred)
ax.fill_between(x_test, y_pred - y_std, y_pred + y_std,
alpha=0.5, color='#ffcd3c');
 Reg.no:411622149007

OUTPUT:

RESULT:
Thus the Python Program for Bayesian Networks was executed and output has been verified
successfully.
 Reg.no:411622149007

Ex. No:5 Implementation of Regression models

AIM:
To implement the Linear and Multiple Linear Regression using Python.
ALGORITHM:

 Step 1: Start the program.

 Step 2: Importing the dataset
 Step 3: Data pre-processing
 Step 4: Splitting the test and train sets
 Step 5: Fitting the linear regression model to the training set
 Step 6: Predicting test results
 Step 7: Visualizing the test results
 Step 8: Stop the program.

PROGRAM:

Linear Regression

import numpy as np
import matplotlib.pyplot as plt

def estimate_coef(x, y):

# number of observations/points
n = np.size(x)

# mean of x and y vector

m_x = np.mean(x)
m_y = np.mean(y)

# calculating cross-deviation and deviation about x

SS_xy = np.sum(y*x) - n*m_y*m_x
SS_xx = np.sum(x*x) - n*m_x*m_x

# calculating regression coefficients

b_1 = SS_xy / SS_xx
b_0 = m_y - b_1*m_x

return (b_0, b_1)

def plot_regression_line(x, y, b):

# plotting the actual points as scatter plot
plt.scatter(x, y, color = "m",
marker = "o", s = 30)
 Reg.no:411622149007

# predicted response vector

y_pred = b[0] + b[1]*x

# plotting the regression line

plt.plot(x, y_pred, color = "g")

# putting labels
plt.xlabel('x')
plt.ylabel('y')

# function to show plot

plt.show()

def main():
# observations / data
x = np.array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
y = np.array([1, 3, 2, 5, 7, 8, 8, 9, 10, 12])

# estimating coefficients
b = estimate_coef(x, y)
print("Estimated coefficients:\nb_0 = {} \
\nb_1 = {}".format(b[0], b[1]))

# plotting regression line

plot_regression_line(x, y, b)

if name == " main ":

main()

Multiple Linear Regression:

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

dataset = pd.read_csv('/content/50_Startups.csv')
dataset.head()

# data preprocessing
X = dataset.iloc[:,:-1].values
y = dataset.iloc[:,4].values

from sklearn.preprocessing import LabelEncoder, OneHotEncoder

labelEncoder_X = LabelEncoder()
X[:,3] = labelEncoder_X.fit_transform(X[ : , 3])

from sklearn.compose import ColumnTransformer

 Reg.no:411622149007

ct = ColumnTransformer([('encoder', OneHotEncoder(), [3])], remainder='

passthrough')
X = np.array(ct.fit_transform(X), dtype=np.float)

X = X[:, 1:]

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0
.2, random_state = 0)

# Fitting the model

from sklearn.linear_model import LinearRegression
regressor = LinearRegression()
regressor.fit(X_train, y_train)

# predicting the test set results

y_pred = regressor.predict(X_test)

y_test

y_pred
 Reg.no:411622149007

OUTPUT:
Linear Regression
Estimated coefficients:
b_0 = 1.2363636363636363
b_1 = 1.1696969696969697

Multiple Regression:
array([103015.20159796, 132582.27760816, 132447.73845175,
71976.09851259, 178537.48221054, 116161.24230163, 67851.69209676,
98791.73374688, 113969.43533012, 167921.0656955 ])

RESULT:
Thus the Python Program for Regression models was executed and output has been verified
successfully.
 Reg.no:411622149007

Ex. No:6(a) Implementation of decision trees

AIM:
To implement the decision tress using Python.
ALGORITHM:

 Step 1: Start the program.

 Step 2: Divide the data based on target variables, choose the best feature
employing Attribute Selection Measures (ASM).
 Step 3: Then it will divide the dataset into smaller sub-datasets and designate
that feature as a decision node for that branch.
 Step 4: Once one of the conditions matches, the procedure is repeated recursively
for every child node to begin creating the tree.
 Step 5: Visualizing the test results
 Step 6: Stop the program.

PROGRAM:

# Python program to implement decision tree algorithm and plot the tree

# Importing the required libraries

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from sklearn import metrics
import seaborn as sns
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn import tree
# Loading the dataset
iris = load_iris()
#converting the data to a pandas dataframe
data = pd.DataFrame(data = iris.data, columns = iris.feature_names)
#creating a separate column for the target variable of iris dataset
data['Species'] = iris.target

#replacing the categories of target variable with the actual names of t

he species
target = np.unique(iris.target)
target_n = np.unique(iris.target_names)
 Reg.no:411622149007

target_dict = dict(zip(target, target_n))

data['Species'] = data['Species'].replace(target_dict)

# Separating the independent dependent variables of the dataset

x = data.drop(columns = "Species")
y = data["Species"]
names_features = x.columns
target_labels = y.unique()

# Splitting the dataset into training and testing datasets

x_train, x_test, y_train, y_test = train_test_split(x, y, test_size = 0
.3, random_state = 93)

# Importing the Decision Tree classifier class from sklearn

from sklearn.tree import DecisionTreeClassifier

# Creating an instance of the classifier class

dtc = DecisionTreeClassifier(max_depth = 3, random_state = 93)

# Fitting the training dataset to the model

dtc.fit(x_train, y_train)

# Plotting the Decision Tree

plt.figure(figsize = (30, 10), facecolor = 'b')
Tree = tree.plot_tree(dtc, feature_names = names_features, class_names
= target_labels, rounded = True, filled = True, fontsize = 14)
plt.show()
y_pred = dtc.predict(x_test)

# Finding the confusion matrix

confusion_matrix = metrics.confusion_matrix(y_test, y_pred)
matrix = pd.DataFrame(confusion_matrix)
axis = plt.axes()
sns.set(font_scale = 1.3)
plt.figure(figsize = (10,7))

# Plotting heatmap
sns.heatmap(matrix, annot = True, fmt = "g", ax = axis, cmap = "magma")

axis.set_title('Confusion Matrix')
axis.set_xlabel("Predicted Values", fontsize = 10)
axis.set_xticklabels([''] + target_labels)
axis.set_ylabel( "True Labels", fontsize = 10)
axis.set_yticklabels(list(target_labels), rotation = 0)
plt.show()
 Reg.no:411622149007

OUTPUT:
 Reg.no:411622149007

RESULT:

Thus the Python Program for Decision Tree was executed and output has been verified
successfully.
 Reg.no:411622149007

Ex. No:6(b) Implementation of Random Forests

AIM:
To implement the Random Forest Algorithm using Python.
ALGORITHM:

 Step 1: Start the program.

 Step 2: Design a specific question or data and get the source to determine the
required data.
 Step 3: Make sure the data is in an accessible format else convert it to the
required format.
 Step 4: Specify all noticeable anomalies and missing data points that may be
required to achieve the required data.
 Step 5: Create a machine-learning model.
 Step 6: Set the baseline model that you want to achieve
 Step 7: Train the data machine learning model.
 Step 8: Provide an insight into the model with test data
 Step 9: Now compare the performance metrics of both the test data and the
predicted data from the model.
 Step 10: If it doesn’t satisfy your expectations, you can try improving your model
accordingly or dating your data, or using another data modeling technique.
 Step 11: Visualizing the test results
 Step 12: Stop the program.

PROGRAM:

#import required libraries

import numpy as np # linear algebra
import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)
import seaborn as sns
import matplotlib.pyplot as plt
iris=pd.read_csv("/content/Iris.csv")
iris.head()
 Reg.no:411622149007

iris.shape
(150, 6)

iris.isnull().sum()

'''checking if there is any inconsistency in the dataset as we see ther

e
are no null values in the dataset, so the data can be processed. ... /''
'
iris.info()

iris["Species"].unique()
 Reg.no:411622149007

array(['Iris-setosa', 'Iris-versicolor', 'Iris-virginica'],

dtype=object)

'''dropping the Id column as it is unnecessary, axis=1 specifies that

it should be column wise, inplace =1 means
the changes should be reflected into the dataframe'''
iris.drop('Id',axis=1,inplace=True)
#checking data after droping "ID column".....
iris.head()

'''Some Exploratory Data Analysis With Iris'''

fig = iris[iris.Species=='Iris-
setosa'].plot(kind='scatter',x='SepalLengthCm',y='SepalWidthCm',color='
orange', label='Setosa')
iris[iris.Species=='Iris-
versicolor'].plot(kind='scatter',x='SepalLengthCm',y='SepalWidthCm',col
or='blue', label='versicolor',ax=fig)
iris[iris.Species=='Iris-
virginica'].plot(kind='scatter',x='SepalLengthCm',y='SepalWidthCm',colo
r='green', label='virginica', ax=fig)
fig.set_xlabel("Sepal Length")
fig.set_ylabel("Sepal Width")
fig.set_title("Sepal Length VS Width")
fig=plt.gcf()
fig.set_size_inches(10,6)
plt.show()
fig.savefig("Sepal Length VS Width.png")
 Reg.no:411622149007

'''Some Exploratory Data Analysis With Iris'''

fig = iris[iris.Species=='Iris-
setosa'].plot.scatter(x='PetalLengthCm',y='PetalWidthCm',color='orange'
, label='Setosa')
iris[iris.Species=='Iris-
versicolor'].plot.scatter(x='PetalLengthCm',y='PetalWidthCm',color='blu
e', label='versicolor',ax=fig)
iris[iris.Species=='Iris-
virginica'].plot.scatter(x='PetalLengthCm',y='PetalWidthCm',color='gree
n', label='virginica', ax=fig)
fig.set_xlabel("Petal Length")
fig.set_ylabel("Petal Width")
fig.set_title(" Petal Length VS Width")
fig=plt.gcf()
fig.set_size_inches(10,6)
plt.show()
fig.savefig("Petal Length VS Width.png")
 Reg.no:411622149007

'''let us see how are the length and width are distributed'''
iris.hist(edgecolor='Yellow', linewidth=1.2)
fig=plt.gcf()
fig.set_size_inches(12,6)
plt.show()

'''Let us see how the length and width vary according to the species'''
plt.figure(figsize=(15,10))
plt.subplot(2,2,1)
sns.violinplot(x='Species',y='PetalLengthCm',data=iris)
plt.subplot(2,2,2)
 Reg.no:411622149007

sns.violinplot(x='Species',y='PetalWidthCm',data=iris)
plt.subplot(2,2,3)
sns.violinplot(x='Species',y='SepalLengthCm',data=iris)
plt.subplot(2,2,4)
sns.violinplot(x='Species',y='SepalWidthCm',data=iris)
fig.savefig("variable with species.png")

#Separating dependent and independent values..

X=iris.iloc[:, :-1].values
X
array([[5.1, 3.5, 1.4, 0.2],
[4.9, 3. , 1.4, 0.2],
[4.7, 3.2, 1.3, 0.2],
[4.6, 3.1, 1.5, 0.2],
[5. , 3.6, 1.4, 0.2],
[5.4, 3.9, 1.7, 0.4],
[4.6, 3.4, 1.4, 0.3],
[5. , 3.4, 1.5, 0.2],
[4.4, 2.9, 1.4, 0.2],
[4.9, 3.1, 1.5, 0.1],
[5.4, 3.7, 1.5, 0.2],
[4.8, 3.4, 1.6, 0.2],
[4.8, 3. , 1.4, 0.1],
[4.3, 3. , 1.1, 0.1],
[5.8, 4. , 1.2, 0.2],
[5.7, 4.4, 1.5, 0.4],
[5.4, 3.9, 1.3, 0.4],
[5.1, 3.5, 1.4, 0.3],
[5.7, 3.8, 1.7, 0.3],
[5.1, 3.8, 1.5, 0.3],
[5.4, 3.4, 1.7, 0.2],
 Reg.no:411622149007

[5.1, 3.7, 1.5, 0.4],

[4.6, 3.6, 1. , 0.2],
[5.1, 3.3, 1.7, 0.5],
[4.8, 3.4, 1.9, 0.2],
[5. , 3. , 1.6, 0.2],
[5. , 3.4, 1.6, 0.4],
[5.2, 3.5, 1.5, 0.2],
[5.2, 3.4, 1.4, 0.2],
[4.7, 3.2, 1.6, 0.2],
[4.8, 3.1, 1.6, 0.2],
[5.4, 3.4, 1.5, 0.4],
[5.2, 4.1, 1.5, 0.1],
[5.5, 4.2, 1.4, 0.2],
[4.9, 3.1, 1.5, 0.1],
[5. , 3.2, 1.2, 0.2],
[5.5, 3.5, 1.3, 0.2],
[4.9, 3.1, 1.5, 0.1],
[4.4, 3. , 1.3, 0.2],
[5.1, 3.4, 1.5, 0.2],
[5. , 3.5, 1.3, 0.3],
[4.5, 2.3, 1.3, 0.3],
[4.4, 3.2, 1.3, 0.2],
[5. , 3.5, 1.6, 0.6],
[5.1, 3.8, 1.9, 0.4],
[4.8, 3. , 1.4, 0.3],
[5.1, 3.8, 1.6, 0.2],
[4.6, 3.2, 1.4, 0.2],
[5.3, 3.7, 1.5, 0.2],
[5. , 3.3, 1.4, 0.2],
[7. , 3.2, 4.7, 1.4],
[6.4, 3.2, 4.5, 1.5],
[6.9, 3.1, 4.9, 1.5],
[5.5, 2.3, 4. , 1.3],
[6.5, 2.8, 4.6, 1.5],
[5.7, 2.8, 4.5, 1.3],
[6.3, 3.3, 4.7, 1.6],
[4.9, 2.4, 3.3, 1. ],
[6.6, 2.9, 4.6, 1.3],
[5.2, 2.7, 3.9, 1.4],
[5. , 2. , 3.5, 1. ],
[5.9, 3. , 4.2, 1.5],
[6. , 2.2, 4. , 1. ],
[6.1, 2.9, 4.7, 1.4],
[5.6, 2.9, 3.6, 1.3],
[6.7, 3.1, 4.4, 1.4],
[5.6, 3. , 4.5, 1.5],
[5.8, 2.7, 4.1, 1. ],
[6.2, 2.2, 4.5, 1.5],
[5.6, 2.5, 3.9, 1.1],
[5.9, 3.2, 4.8, 1.8],
[6.1, 2.8, 4. , 1.3],
[6.3, 2.5, 4.9, 1.5],
[6.1, 2.8, 4.7, 1.2],
[6.4, 2.9, 4.3, 1.3],
[6.6, 3. , 4.4, 1.4],
[6.8, 2.8, 4.8, 1.4],
[6.7, 3. , 5. , 1.7],
[6. , 2.9, 4.5, 1.5],
[5.7, 2.6, 3.5, 1. ],
 Reg.no:411622149007

[5.5, 2.4, 3.8, 1.1],

[5.5, 2.4, 3.7, 1. ],
[5.8, 2.7, 3.9, 1.2],
[6. , 2.7, 5.1, 1.6],
[5.4, 3. , 4.5, 1.5],
[6. , 3.4, 4.5, 1.6],
[6.7, 3.1, 4.7, 1.5],
[6.3, 2.3, 4.4, 1.3],
[5.6, 3. , 4.1, 1.3],
[5.5, 2.5, 4. , 1.3],
[5.5, 2.6, 4.4, 1.2],
[6.1, 3. , 4.6, 1.4],
[5.8, 2.6, 4. , 1.2],
[5. , 2.3, 3.3, 1. ],
[5.6, 2.7, 4.2, 1.3],
[5.7, 3. , 4.2, 1.2],
[5.7, 2.9, 4.2, 1.3],
[6.2, 2.9, 4.3, 1.3],
[5.1, 2.5, 3. , 1.1],
[5.7, 2.8, 4.1, 1.3],
[6.3, 3.3, 6. , 2.5],
[5.8, 2.7, 5.1, 1.9],
[7.1, 3. , 5.9, 2.1],
[6.3, 2.9, 5.6, 1.8],
[6.5, 3. , 5.8, 2.2],
[7.6, 3. , 6.6, 2.1],
[4.9, 2.5, 4.5, 1.7],
[7.3, 2.9, 6.3, 1.8],
[6.7, 2.5, 5.8, 1.8],
[7.2, 3.6, 6.1, 2.5],
[6.5, 3.2, 5.1, 2. ],
[6.4, 2.7, 5.3, 1.9],
[6.8, 3. , 5.5, 2.1],
[5.7, 2.5, 5. , 2. ],
[5.8, 2.8, 5.1, 2.4],
[6.4, 3.2, 5.3, 2.3],
[6.5, 3. , 5.5, 1.8],
[7.7, 3.8, 6.7, 2.2],
[7.7, 2.6, 6.9, 2.3],
[6. , 2.2, 5. , 1.5],
[6.9, 3.2, 5.7, 2.3],
[5.6, 2.8, 4.9, 2. ],
[7.7, 2.8, 6.7, 2. ],
[6.3, 2.7, 4.9, 1.8],
[6.7, 3.3, 5.7, 2.1],
[7.2, 3.2, 6. , 1.8],
[6.2, 2.8, 4.8, 1.8],
[6.1, 3. , 4.9, 1.8],
[6.4, 2.8, 5.6, 2.1],
[7.2, 3. , 5.8, 1.6],
[7.4, 2.8, 6.1, 1.9],
[7.9, 3.8, 6.4, 2. ],
[6.4, 2.8, 5.6, 2.2],
[6.3, 2.8, 5.1, 1.5],
[6.1, 2.6, 5.6, 1.4],
[7.7, 3. , 6.1, 2.3],
[6.3, 3.4, 5.6, 2.4],
[6.4, 3.1, 5.5, 1.8],
[6. , 3. , 4.8, 1.8],
 Reg.no:411622149007

[6.9, 3.1, 5.4, 2.1],

[6.7, 3.1, 5.6, 2.4],
[6.9, 3.1, 5.1, 2.3],
[5.8, 2.7, 5.1, 1.9],
[6.8, 3.2, 5.9, 2.3],
[6.7, 3.3, 5.7, 2.5],
[6.7, 3. , 5.2, 2.3],
[6.3, 2.5, 5. , 1.9],
[6.5, 3. , 5.2, 2. ],
[6.2, 3.4, 5.4, 2.3],
[5.9, 3. , 5.1, 1.8]])

y=iris.iloc[:, -1].values
y
array(['Iris-setosa', 'Iris-setosa', 'Iris-setosa', 'Iris-setosa',
'Iris-setosa', 'Iris-setosa', 'Iris-setosa', 'Iris-setosa',
'Iris-setosa', 'Iris-setosa', 'Iris-setosa', 'Iris-setosa',
'Iris-setosa', 'Iris-setosa', 'Iris-setosa', 'Iris-setosa',
'Iris-setosa', 'Iris-setosa', 'Iris-setosa', 'Iris-setosa',
'Iris-setosa', 'Iris-setosa', 'Iris-setosa', 'Iris-setosa',
'Iris-setosa', 'Iris-setosa', 'Iris-setosa', 'Iris-setosa',
'Iris-setosa', 'Iris-setosa', 'Iris-setosa', 'Iris-setosa',
'Iris-setosa', 'Iris-setosa', 'Iris-setosa', 'Iris-setosa',
'Iris-setosa', 'Iris-setosa', 'Iris-setosa', 'Iris-setosa',
'Iris-setosa', 'Iris-setosa', 'Iris-setosa', 'Iris-setosa',
'Iris-setosa', 'Iris-setosa', 'Iris-setosa', 'Iris-setosa',
'Iris-setosa', 'Iris-setosa', 'Iris-versicolor', 'Iris-versicolor',
'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
 Reg.no:411622149007

'Iris-virginica', 'Iris-virginica', 'Iris-virginica',

'Iris-virginica', 'Iris-virginica'], dtype=object)
#splitting into training set and test.
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X,y,test_size = 0.2
, random_state = 123)
# Feature Scaling
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform(X_test)
# Fitting Random Forest Classification to the Training set
from sklearn.ensemble import RandomForestClassifier
classifier = RandomForestClassifier(n_estimators = 10, criterion = 'ent
ropy', random_state = 0)
classifier.fit(X_train, y_train)
# Predicting the Test set results
y_pred = classifier.predict(X_test)
# Making the Confusion Matrix
from sklearn.metrics import confusion_matrix
cm = confusion_matrix(y_test, y_pred)
cm
array([[13, 0, 0], [ 0, 5, 1], [ 0, 2, 9]])

from sklearn import metrics #for checking the model accuracy

print('The accuracy of the Random forest is:',metrics.accuracy_score(y_
pred,y_test))
 Reg.no:411622149007

OUTPUT:
The accuracy of the Random forest is: 0.9

RESULT:

Thus the Random Forest Algorithm using Python was executed and output has been
verified successfully.
 Reg.no:411622149007

Ex. No:7 Implementation of SVM Models

AIM:
To implement the Support Vector Machine Models using Python.
ALGORITHM:
Step 1: Start the Program.
Step 2: Import the Libraries- import numpy as np import matplotlib.pyplot as plt import pandas
as pd.
Setp 3: Load the Dataset.
Step 4: Split Dataset into X and Y.
Step 5: Split the X and Y Dataset into the Training set and Test set.
Step 6: Perform Feature Scaling.
Step 7: Fit SVM to the Training set.
Step 8: Predict the Test Set Results.
Step 9: Make the Confusion Matrix
Step 10. Stop
PROGRAM:

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
dataset = pd.read_csv('/content/Social_Network_Ads.csv')
X = dataset.iloc[:, [2, 3]].values
y = dataset.iloc[:, 4].values
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0
.25, random_state = 0)
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform(X_test)
from sklearn.svm import SVC
classifier = SVC(kernel = 'rbf', random_state = 0)
classifier.fit(X_train, y_train)
y_pred = classifier.predict(X_test)
from sklearn.metrics import confusion_matrix, accuracy_score
cm = confusion_matrix(y_test, y_pred)
print(cm)
accuracy_score(y_test,y_pred)
from matplotlib.colors import ListedColormap
X_set, y_set = X_test, y_test
 Reg.no:411622149007

X1, X2 = np.meshgrid(np.arange(start = X_set[:, 0].min() -

1, stop = X_set[:, 0].max() + 1, step = 0.01),
np.arange(start = X_set[:, 1].min() -
1, stop = X_set[:, 1].max() + 1, step = 0.01))
plt.contourf(X1, X2, classifier.predict(np.array([X1.ravel(), X2.ravel(
)]).T).reshape(X1.shape),
alpha = 0.75, cmap = ListedColormap(('red', 'green')))
plt.xlim(X1.min(), X1.max())
plt.ylim(X2.min(), X2.max())
for i, j in enumerate(np.unique(y_set)):
plt.scatter(X_set[y_set == j, 0], X_set[y_set == j, 1],
c = ListedColormap(('red', 'green'))(i), label = j)
plt.title('SVM (Test set)')
plt.xlabel('Age')
plt.ylabel('Estimated Salary')
plt.legend()
plt.show()
 Reg.no:411622149007

OUTPUT:
Confusion Matrix
[[64 4]
[ 3 29]]
0.93

RESULT:
Thus the Support Vector Machine models using Python was executed and output has
been verified successfully.
 Reg.no:411622149007

Ex. No:8 (a) Implementation of Ensembling Methods

AIM:
To implement the Ensembling Methods like Bagging, Boosting and Voting Classifier
Algorithms using Python.
ALGORITHM:
Step 1: Start
Step 2: Split the train dataset into n parts
Step 3: The base model is then fitted on the whole train dataset.
Step 4: The Steps 2 to 4 are repeated for another base model which results in another set of
predictions for the train and test dataset.
Step 5: The predictions on train data set are used as a feature to build the new model.
Step 6: Stop
PROGRAM:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

%matplotlib inline
sns.set_style("whitegrid")
plt.style.use("fivethirtyeight")
df = pd.read_csv("/content/diabetes.csv")
df.head()
df.info()
df.isnull().sum()
pd.set_option('display.float_format', '{:.2f}'.format)
df.describe()
categorical_val = []
continous_val = []
for column in df.columns:
# print('==============================')
# print(f"{column} : {df[column].unique()}")
if len(df[column].unique()) <= 10:
categorical_val.append(column)
else:
continous_val.append(column)
df.columns
# How many missing zeros are mising in each feature
feature_columns = [
 Reg.no:411622149007

'Pregnancies', 'Glucose', 'BloodPressure', 'SkinThickness',

'Insulin', 'BMI', 'DiabetesPedigreeFunction', 'Age'
]

for column in feature_columns:

print("============================================")
print(f"{column} ==> Missing zeros : {len(df.loc[df[column] == 0])}
")
from sklearn.impute import SimpleImputer

fill_values = SimpleImputer(missing_values=0, strategy="mean", copy=Fal

se)
df[feature_columns] = fill_values.fit_transform(df[feature_columns])

for column in feature_columns:

print("============================================")
print(f"{column} ==> Missing zeros : {len(df.loc[df[column] == 0])}
")
from sklearn.model_selection import train_test_split

X = df[feature_columns]
y = df.Outcome

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3

, random_state=42)
from sklearn.metrics import confusion_matrix, accuracy_score, classific
ation_report
Bagging:
def evaluate(model, X_train, X_test, y_train, y_test):
y_test_pred = model.predict(X_test)
y_train_pred = model.predict(X_train)

print("TRAINIG RESULTS: \n===============================")

clf_report = pd.DataFrame(classification_report(y_train, y_train_pr
ed, output_dict=True))
print(f"CONFUSION MATRIX:\n{confusion_matrix(y_train, y_train_pred)
}")
print(f"ACCURACY SCORE:\n{accuracy_score(y_train, y_train_pred):.4f
}")
print(f"CLASSIFICATION REPORT:\n{clf_report}")

print("TESTING RESULTS: \n===============================")

clf_report = pd.DataFrame(classification_report(y_test, y_test_pred
, output_dict=True))
print(f"CONFUSION MATRIX:\n{confusion_matrix(y_test, y_test_pred)}"
)
 Reg.no:411622149007

print(f"ACCURACY SCORE:\n{accuracy_score(y_test, y_test_pred):.4f}"

)
print(f"CLASSIFICATION REPORT:\n{clf_report}")
from sklearn.ensemble import BaggingClassifier
from sklearn.tree import DecisionTreeClassifier

tree = DecisionTreeClassifier()
bagging_clf = BaggingClassifier(base_estimator=tree, n_estimators=1500,
random_state=42)
bagging_clf.fit(X_train, y_train)

evaluate(bagging_clf, X_train, X_test, y_train, y_test)

scores = {
'Bagging Classifier': {
'Train': accuracy_score(y_train, bagging_clf.predict(X_train)),
'Test': accuracy_score(y_test, bagging_clf.predict(X_test)),
},
}
Random Forest:

from sklearn.ensemble import RandomForestClassifier

rf_clf = RandomForestClassifier(random_state=42, n_estimators=1000)

rf_clf.fit(X_train, y_train)
evaluate(rf_clf, X_train, X_test, y_train, y_test)
scores['Random Forest'] = {
'Train': accuracy_score(y_train, rf_clf.predict(X_train)),
'Test': accuracy_score(y_test, rf_clf.predict(X_test)),
}

Extra Tree Classifier:

from sklearn.ensemble import ExtraTreesClassifier

ex_tree_clf = ExtraTreesClassifier(n_estimators=1000, max_features=7, r

andom_state=42)
ex_tree_clf.fit(X_train, y_train)
evaluate(ex_tree_clf, X_train, X_test, y_train, y_test)
scores['Extra Tree'] = {
'Train': accuracy_score(y_train, ex_tree_clf.predict(X_train)),
'Test': accuracy_score(y_test, ex_tree_clf.predict(X_test)),
}
AdaBoost:

from sklearn.ensemble import AdaBoostClassifier

ada_boost_clf = AdaBoostClassifier(n_estimators=30)
ada_boost_clf.fit(X_train, y_train)
evaluate(ada_boost_clf, X_train, X_test, y_train, y_test)
 Reg.no:411622149007

scores['AdaBoost'] = {
'Train': accuracy_score(y_train, ada_boost_clf.predict(X_train)
),
'Test': accuracy_score(y_test, ada_boost_clf.predict(X_test)),
}
Gradient Boosting Classifier

grad_boost_clf = GradientBoostingClassifier(n_estimators=100, random_st

ate=42)
grad_boost_clf.fit(X_train, y_train)
evaluate(grad_boost_clf, X_train, X_test, y_train, y_test)
scores['Gradient Boosting'] = {
'Train': accuracy_score(y_train, grad_boost_clf.predict(X_train
)),
'Test': accuracy_score(y_test, grad_boost_clf.predict(X_test)),
}
Voting Classifier:

from sklearn.ensemble import VotingClassifier

from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC

estimators = []
log_reg = LogisticRegression(solver='liblinear')
estimators.append(('Logistic', log_reg))

tree = DecisionTreeClassifier()
estimators.append(('Tree', tree))

svm_clf = SVC(gamma='scale')
estimators.append(('SVM', svm_clf))

voting = VotingClassifier(estimators=estimators)
voting.fit(X_train, y_train)

evaluate(voting, X_train, X_test, y_train, y_test)

scores['Voting'] = {
'Train': accuracy_score(y_train, voting.predict(X_train)),
'Test': accuracy_score(y_test, voting.predict(X_test)),
}
Comparison:

scores_df = pd.DataFrame(scores)

scores_df.plot(kind='barh', figsize=(15, 8))

 Reg.no:411622149007

OUTPUT:

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 768 entries, 0 to 767
Data columns (total 9 columns):
# Column Non-Null Count Dtype

0 Pregnancies 768 non-null int64

1 Glucose 768 non-null int64
2 BloodPressure 768 non-null int64
3 SkinThickness 768 non-null int64
4 Insulin 768 non-null int64
5 BMI 768 non-null float64
6 DiabetesPedigreeFunction 768 non-null float64
7 Age 768 non-null int64
8 Outcome 768 non-null int64
dtypes: float64(2), int64(7)
memory usage: 54.1 KB

Pregnancies 0 Glucose 0 BloodPressure 0 SkinThickness 0 Insulin 0 BMI 0

DiabetesPedigreeFunction 0 Age 0 Outcome 0 dtype: int64

Index(['Pregnancies', 'Glucose', 'BloodPressure', 'SkinThickness',

'Insulin', 'BMI', 'DiabetesPedigreeFunction', 'Age', 'Outcome'],
dtype='object')
 Reg.no:411622149007

Bagging:
TRAINIG RESULTS:
===============================
CONFUSION MATRIX:
[[349 0]
[ 0 188]]
ACCURACY SCORE:
1.0000
CLASSIFICATION REPORT:
0 1 accuracy macro avg weighted avg
precision 1.00 1.00 1.00 1.00 1.00
recall 1.00 1.00 1.00 1.00 1.00
f1-score 1.00 1.00 1.00 1.00 1.00
support 349.00 188.00 1.00 537.00 537.00
TESTING RESULTS:
===============================
CONFUSION MATRIX:
[[119 32]

ACCURACY SCORE:
0.7576
 Reg.no:411622149007

CLASSIFICATION REPORT:
0 1 accuracy macro avg weighted avg
precision 0.83 0.64 0.76 0.73 0.76
recall 0.79 0.70 0.76 0.74 0.76
f1-score 0.81 0.67 0.76 0.74 0.76
support 151.00 80.00 0.76 231.00 231.00

Random Forest Classifier

TRAINIG RESULTS:
===============================
CONFUSION MATRIX:
[[349 0]
[ 0 188]]
ACCURACY SCORE:
1.0000
CLASSIFICATION REPORT:
0 1 accuracy macro avg weighted avg
precision 1.00 1.00 1.00 1.00 1.00
recall 1.00 1.00 1.00 1.00 1.00
f1-score 1.00 1.00 1.00 1.00 1.00
support 349.00 188.00 1.00 537.00 537.00
TESTING RESULTS:
===============================
CONFUSION MATRIX:
[[123 28]
[ 29 51]]
ACCURACY SCORE:
0.7532
CLASSIFICATION REPORT:
0 1 accuracy macro avg weighted avg
precision 0.81 0.65 0.75 0.73 0.75
recall 0.81 0.64 0.75 0.73 0.75
f1-score 0.81 0.64 0.75 0.73 0.75
support 151.00 80.00 0.75 231.00 231.00

Extra Tree Classifier:

TRAINIG RESULTS:
===============================
CONFUSION MATRIX:
[[349 0]
[ 0 188]]
ACCURACY SCORE:
1.0000
CLASSIFICATION REPORT:
0 1 accuracy macro avg weighted avg
precision 1.00 1.00 1.00 1.00 1.00
recall 1.00 1.00 1.00 1.00 1.00
f1-score 1.00 1.00 1.00 1.00 1.00
support 349.00 188.00 1.00 537.00 537.00
TESTING RESULTS:
===============================
CONFUSION MATRIX:
[[124 27]
[ 25 55]]
 Reg.no:411622149007

ACCURACY SCORE:
0.7749
CLASSIFICATION REPORT:
0 1 accuracy macro avg weighted avg
precision 0.83 0.67 0.77 0.75 0.78
recall 0.82 0.69 0.77 0.75 0.77
f1-score 0.83 0.68 0.77 0.75 0.78
support 151.00 80.00 0.77 231.00 231.00

AdaBoost Classifier
TRAINIG RESULTS:
===============================
CONFUSION MATRIX:
[[310 39]
[ 51 137]]
ACCURACY SCORE:
0.8324
CLASSIFICATION REPORT:
0 1 accuracy macro avg weighted avg
precision 0.86 0.78 0.83 0.82 0.83
recall 0.89 0.73 0.83 0.81 0.83
f1-score 0.87 0.75 0.83 0.81 0.83
support 349.00 188.00 0.83 537.00 537.00
TESTING RESULTS:
===============================
CONFUSION MATRIX:
[[123 28]
[ 27 53]]
ACCURACY SCORE:
0.7619
CLASSIFICATION REPORT:
0 1 accuracy macro avg weighted avg
precision 0.82 0.65 0.76 0.74 0.76
recall 0.81 0.66 0.76 0.74 0.76
f1-score 0.82 0.66 0.76 0.74 0.76
support 151.00 80.00 0.76 231.00 231.00

Gradient Boosting Classifier

TRAINIG RESULTS:
===============================
CONFUSION MATRIX:
[[342 7]
[ 19 169]]
ACCURACY SCORE:
0.9516
CLASSIFICATION REPORT:
0 1 accuracy macro avg weighted avg
precision 0.95 0.96 0.95 0.95 0.95
recall 0.98 0.90 0.95 0.94 0.95
f1-score 0.96 0.93 0.95 0.95 0.95
support 349.00 188.00 0.95 537.00 537.00
TESTING RESULTS:
===============================
CONFUSION MATRIX:
[[116 35]
[ 26 54]]
 Reg.no:411622149007

ACCURACY SCORE:
0.7359
CLASSIFICATION REPORT:
0 1 accuracy macro avg weighted avg
precision 0.82 0.61 0.74 0.71 0.74
recall 0.77 0.68 0.74 0.72 0.74
f1-score 0.79 0.64 0.74 0.72 0.74
support 151.00 80.00 0.74 231.00 231.00

Voting Classifier
TRAINIG RESULTS:
===============================
CONFUSION MATRIX:
[[327 22]
[ 82 106]]
ACCURACY SCORE:
0.8063
CLASSIFICATION REPORT:
0 1 accuracy macro avg weighted avg
precision 0.80 0.83 0.81 0.81 0.81
recall 0.94 0.56 0.81 0.75 0.81
f1-score 0.86 0.67 0.81 0.77 0.80
support 349.00 188.00 0.81 537.00 537.00
TESTING RESULTS:
===============================
CONFUSION MATRIX:
[[131 20]
[ 37 43]]
ACCURACY SCORE:
0.7532
CLASSIFICATION REPORT:
0 1 accuracy macro avg weighted avg
precision 0.78 0.68 0.75 0.73 0.75
recall 0.87 0.54 0.75 0.70 0.75
f1-score 0.82 0.60 0.75 0.71 0.75
support 151.00 80.00 0.75 231.00 231.00

Comparison:
 Reg.no:411622149007

Result:
Thus, the Ensemble Methods like Bagging, Boosting and Voting Classifiers using Python
was executed and output has been verified successfully.
 Reg.no:411622149007

Ex. No:9 Implement clustering algorithms

AIM:
To implement the clustering algorithms using Python.
ALGORITHM:
Step 1: Start the Program.
Step 2: Import the Libraries- import numpy as np import matplotlib.pyplot as plt import pandas
as pd.
Step 3: Select K data objects for each cluster.

Step 4: Calculate dissimilarities D(X,Q) and allocate each data object to nearest cluster.
Step 5: Calculate the new modes for all clusters.
Step 6: Repeat step 4 and 5 until the cluster will become stable.
Step 7. Stop
PROGRAM:

import numpy as np # linear algebra

import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)

# Input data files are available in the read-only "../input/" directory

# For example, running this (by clicking run or pressing Shift+Enter) w
ill list all files under the input directory

import os
for dirname, _, filenames in os.walk('/kaggle/input'):
for filename in filenames:
print(os.path.join(dirname, filename))

# You can write up to 20GB to the current directory (/kaggle/working/)

that gets preserved as output when you create a version using "Save & R
un All"
# You can also write temporary files to /kaggle/temp/, but they won't b
e saved outside of the current session
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import warnings
import seaborn as sns
from sklearn import preprocessing
from sklearn.decomposition import PCA
from sklearn.decomposition import KernelPCA
from sklearn.cluster import KMeans
 Reg.no:411622149007

from sklearn.metrics import silhouette_score

from sklearn.metrics import calinski_harabasz_score
from sklearn.metrics import davies_bouldin_score
from sklearn.manifold import TSNE
import plotly.express as px
import seaborn as sns
#read the data

df = pd.read_csv("/content/CC GENERAL.csv")

print('The shape of the dataset is:', df.shape)

df.head(5)
# Let's see the data types and non-null values for each column
df.info()
round(df.isnull().sum(axis=0)*100/df.shape[0],2)
# This will print basic statistics for numerical columns
df.describe().T
df.duplicated().sum()
numerical_features=[feature for feature in df.columns if df[feature].dt
ypes!='object']

df[numerical_features].hist(bins=15, figsize=(20, 20), layout=(6, 3));

plt.subplots(figsize=(20,20))

mask = np.triu(np.ones_like(df.corr()))
heatmap = sns.heatmap(df.corr(), mask=mask, vmin=-
1, vmax=1, annot=True, cmap='Greens')
heatmap.set_title('Correlation Heatmap', fontdict={'fontsize':21}, pad=
16);
#make a copy for the original dataset
df_copy=df.copy()
df_copy.set_index('CUST_ID', inplace=True)
#solution
df_copy["CREDIT_LIMIT"].fillna(df_copy["CREDIT_LIMIT"].mean(), inplace=
True)
df_copy["MINIMUM_PAYMENTS"].fillna( df_copy["MINIMUM_PAYMENTS"].mean(),
inplace=True)
#test
round(df_copy.isnull().sum(axis=0)*100/df_copy.shape[0],2)
df_scaled = df_copy.copy()
col_names =df_scaled.columns
features = df_scaled[col_names]
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
df_Standard_Scaler = df_copy.copy()

df_Standard_Scaler[col_names] = scaler.fit_transform(features.values)
df_Standard_Scaler.describe().T
 Reg.no:411622149007

from sklearn.preprocessing import RobustScaler

scaler = RobustScaler()

df_RobustScaler = df_copy.copy()
df_Standard_Scaler
df_RobustScaler[col_names] = scaler.fit_transform(features.values)
df_RobustScaler.describe()
from sklearn.preprocessing import MinMaxScaler
df_MinMaxScaler = df_copy.copy()

scaler = MinMaxScaler()
df_MinMaxScaler[col_names] = scaler.fit_transform(features.values)
df_MinMaxScaler.describe()
pca = PCA()
#Transform the data
df_pca = pca.fit_transform(df_Standard_Scaler)
plt.plot(df_pca[0], df_pca[1]) # Plot the chart
plt.show()
kernel_pca = KernelPCA(n_components=2,kernel='linear')
kernel_pca_df = kernel_pca.fit_transform(df_Standard_Scaler)
plt.plot(kernel_pca_df[0], kernel_pca_df[1]) # Plot the chart
plt.show()
# 3. Use elbow method
inertia_list=[]

for i in range(1, 10):

kmean_skl = KMeans(n_clusters=i, n_init=1,max_iter=200)
kmean_skl.fit(df_Standard_Scaler)
inertia_list.append(kmean_skl.inertia_)

plt.plot(range(1, 10), inertia_list, marker='o')

plt.xlabel('Num_of Clusters')
plt.ylabel('Distortion')
plt.show()
def kmean(df):
kmean = KMeans(n_clusters=4, max_iter=100)
kmean.fit(df)
kmean.fit_predict(df)
labels_kmean= kmean.labels_
#df_pred=pd.DataFrame(pred,index=df_copy.index,columns= ['Model_lab
el'])
return labels_kmean
from sklearn.cluster import AgglomerativeClustering
import scipy.cluster.hierarchy as shc

def cluster_Hierarchical(df):
cluster_Hierarchical = AgglomerativeClustering(n_clusters=4, affini
ty='euclidean')
 Reg.no:411622149007

cluster_Hierarchical.fit_predict(df)
labels_cluster_Hierarchical=cluster_Hierarchical.labels_
return labels_cluster_Hierarchical
"""
#linkage='complete'
plt.figure(figsize=(10, 7))
plt.title("Counters Dendograms")
dend = shc.dendrogram(shc.linkage(y=df_pca , method='complete',metric='
euclidean')) """
from sklearn.cluster import DBSCAN
def dbscan(df):
cluster_dbscan = DBSCAN(eps=8, min_samples=4).fit(df)
n=cluster_dbscan.labels_
return cluster_dbscan.labels_

n=dbscan(df_RobustScaler)
n.shape
unique, counts = np.unique(n, return_counts=True)
dict(zip(unique, counts))
print('Silhoutte score of dbscan is ' , silhouette_score(df_RobustScale
r,n))
from sklearn.cluster import DBSCAN
def dbscan(df):
cluster_dbscan = DBSCAN(eps=8, min_samples=4).fit(df)
return cluster_dbscan.labels_

n=dbscan(df_RobustScaler)
n.shape
unique, counts = np.unique(n, return_counts=True)
print('Silhoutte score of dbscan is ' , silhouette_score(df_RobustScale
r,n))
dict(zip(unique, counts))
from sklearn.ensemble import IsolationForest
def IForest(df):
lforest = IsolationForest().fit(df)
lforest_labels = lforest.predict(df)
return lforest_labels
from sklearn import mixture
def gmm(df):
gmm = mixture.GaussianMixture(n_components=3,covariance_type="full"
,max_iter = 100,init_params="random")
gmm.fit(df)
gmm_labels = gmm.predict(df)
return(gmm_labels)
#1-MinMaxScaler with diff algo
df_pca = pca.fit_transform(df_MinMaxScaler)
labels_kmean=kmean(df_pca)
labels_cluster_Hierarchical=cluster_Hierarchical(df_pca)
 Reg.no:411622149007

gmm_labels=gmm(df_pca)
cluster_dbscan_labels=dbscan(df_pca)
lforest_labels=IForest(df_pca)
print("scores of pca with MinMaxScaler")
print('Silhoutte score of kmean is ' , silhouette_score(df_pca, labels_
kmean))
print('Silhoutte score of Hierarchical is ' , silhouette_score(df_pca,
labels_cluster_Hierarchical))
print('Silhoutte score of EM is ' , silhouette_score(df_pca,gmm_labels)
)
print('Silhoutte score of IsolationForest is ' , silhouette_score(df_pc
a,lforest_labels))

print('davies bouldin score of kmean is ' , davies_bouldin_score(df_pca

, labels_kmean))
print('davies bouldin score of Hierarchical is ' , davies_bouldin_score
(df_pca, labels_cluster_Hierarchical))
print('davies bouldin score of EM is ' , davies_bouldin_score(df_pca, g
mm_labels))
print('davies bouldin score of IsolationForest is ' , davies_bouldin_sc
ore(df_pca, lforest_labels))
#2-RobustScaler with diff algo
df_pca = pca.fit_transform(df_RobustScaler)
labels_kmean=kmean(df_pca)
labels_cluster_Hierarchical=cluster_Hierarchical(df_pca)
gmm_labels=gmm(df_pca)
cluster_dbscan_labels=dbscan(df_pca)
lforest_labels=IForest(df_pca)
print("\nscores of pca with RobustScaler")
print('Silhoutte score of kmean is ' , silhouette_score(df_pca, labels_
kmean))
print('Silhoutte score of Hierarchical is ' , silhouette_score(df_pca,
labels_cluster_Hierarchical))
print('Silhoutte score of EM is ' , silhouette_score(df_pca,gmm_labels)
)
print('Silhoutte score of IsolationForest is ' , silhouette_score(df_pc
a,lforest_labels))
print('Silhoutte score of dbscan is ' , silhouette_score(df_pca,cluster
_dbscan_labels))

print('davies bouldin score of kmean is ' , davies_bouldin_score(df_pca

, labels_kmean))
print('davies bouldin score of Hierarchical is ' , davies_bouldin_score
(df_pca, labels_cluster_Hierarchical))
print('davies bouldin score of EM is ' , davies_bouldin_score(df_pca, g
mm_labels))
print('davies bouldin score of IsolationForest is ' , davies_bouldin_sc
ore(df_pca, lforest_labels))
 Reg.no:411622149007

print('davies bouldin score of dbscan is ' , davies_bouldin_score(df_pc

a,cluster_dbscan_labels))
#3-Standard_Scaler with diff algo
df_pca = pca.fit_transform(df_Standard_Scaler)
labels_kmean=kmean(df_pca)
labels_cluster_Hierarchical=cluster_Hierarchical(df_pca)
gmm_labels=gmm(df_pca)
cluster_dbscan_labels=dbscan(df_pca)
lforest_labels=IForest(df_pca)
print("\nscores of pca with Standard_Scaler")
print('Silhoutte score of kmean is ' , silhouette_score(df_pca, labels_
kmean))
print('Silhoutte score of Hierarchical is ' , silhouette_score(df_pca,
labels_cluster_Hierarchical))
print('Silhoutte score of EM is ' , silhouette_score(df_pca,gmm_labels)
)
print('Silhoutte score of IsolationForest is ' , silhouette_score(df_pc
a,lforest_labels))
print('Silhoutte score of dbscan is ' , silhouette_score(df_pca,cluster
_dbscan_labels))

print('davies bouldin score of kmean is ' , davies_bouldin_score(df_pca

, labels_kmean))
print('davies bouldin score of Hierarchical is ' , davies_bouldin_score
(df_pca, labels_cluster_Hierarchical))
print('davies bouldin score of EM is ' , davies_bouldin_score(df_pca, g
mm_labels))
print('davies bouldin score of IsolationForest is ' , davies_bouldin_sc
ore(df_pca, lforest_labels))
print('davies bouldin score of dbscan is ' , davies_bouldin_score(df_pc
a,cluster_dbscan_labels))

#1-MinMaxScaler with diff algo

df_pca = kernel_pca.fit_transform(df_MinMaxScaler)
labels_kmean=kmean(df_pca)
labels_cluster_Hierarchical=cluster_Hierarchical(df_pca)
gmm_labels=gmm(df_pca)
cluster_dbscan_labels=dbscan(df_pca)
lforest_labels=IForest(df_pca)
print("scores of kernel_pca with MinMaxScaler")
print('Silhoutte score of kmean is ' , silhouette_score(df_pca, labels_
kmean))
print('Silhoutte score of Hierarchical is ' , silhouette_score(df_pca,
labels_cluster_Hierarchical))
print('Silhoutte score of EM is ' , silhouette_score(df_pca,gmm_labels)
)
print('Silhoutte score of IsolationForest is ' , silhouette_score(df_pc
a,lforest_labels))
 Reg.no:411622149007

print('\ndavies bouldin score of kmean is ' , davies_bouldin_score(df_p

ca, labels_kmean))
print('davies bouldin score of Hierarchical is ' , davies_bouldin_score
(df_pca, labels_cluster_Hierarchical))
print('davies bouldin score of EM is ' , davies_bouldin_score(df_pca, g
mm_labels))
print('davies bouldin score of IsolationForest is ' , davies_bouldin_sc
ore(df_pca, lforest_labels))
#print('Silhoutte score of dbscan is ' , silhouette_score(df_pca,cluste
r_dbscan_labels))
#2-RobustScaler with diff algo
df_pca = kernel_pca.fit_transform(df_RobustScaler)
labels_kmean=kmean(df_pca)
labels_cluster_Hierarchical=cluster_Hierarchical(df_pca)
gmm_labels=gmm(df_pca)
cluster_dbscan_labels=dbscan(df_pca)
lforest_labels=IForest(df_pca)
print("\n\nscores of kernel_pca with RobustScaler")
print('Silhoutte score of kmean is ' , silhouette_score(df_pca, labels_
kmean))
print('Silhoutte score of Hierarchical is ' , silhouette_score(df_pca,
labels_cluster_Hierarchical))
print('Silhoutte score of EM is ' , silhouette_score(df_pca,gmm_labels)
)
print('Silhoutte score of IsolationForest is ' , silhouette_score(df_pc
a,lforest_labels))
print('Silhoutte score of dbscan is ' , silhouette_score(df_pca,cluster
_dbscan_labels))

print('\ndavies bouldin score of kmean is ' , davies_bouldin_score(df_p

ca, labels_kmean))
print('davies bouldin score of Hierarchical is ' , davies_bouldin_score
(df_pca, labels_cluster_Hierarchical))
print('davies bouldin score of EM is ' , davies_bouldin_score(df_pca, g
mm_labels))
print('davies bouldin score of IsolationForest is ' , davies_bouldin_sc
ore(df_pca, lforest_labels))
print('davies bouldin score of dbscan is ' , davies_bouldin_score(df_pc
a,cluster_dbscan_labels))
#3-Standard_Scaler with diff algo
df_pca = kernel_pca.fit_transform(df_Standard_Scaler)
labels_kmean=kmean(df_pca)
labels_cluster_Hierarchical=cluster_Hierarchical(df_pca)
gmm_labels=gmm(df_pca)
cluster_dbscan_labels=dbscan(df_pca)
lforest_labels=IForest(df_pca)
print("\n\nscores of kernel_pca with Standard_Scaler")
 Reg.no:411622149007

print('Silhoutte score of kmean is ' , silhouette_score(df_pca, labels_

kmean))
print('Silhoutte score of Hierarchical is ' , silhouette_score(df_pca,
labels_cluster_Hierarchical))
print('Silhoutte score of EM is ' , silhouette_score(df_pca,gmm_labels)
)
print('Silhoutte score of IsolationForest is ' , silhouette_score(df_pc
a,lforest_labels))
print('Silhoutte score of dbscan is ' , silhouette_score(df_pca,cluster
_dbscan_labels))

print('\ndavies bouldin score of kmean is ' , davies_bouldin_score(df_p

ca, labels_kmean))
print('davies bouldin score of Hierarchical is ' , davies_bouldin_score
(df_pca, labels_cluster_Hierarchical))
print('davies bouldin score of EM is ' , davies_bouldin_score(df_pca, g
mm_labels))
print('davies bouldin score of IsolationForest is ' , davies_bouldin_sc
ore(df_pca, lforest_labels))
print('davies bouldin score of dbscan is ' , davies_bouldin_score(df_pc
a,cluster_dbscan_labels))
tsne = TSNE(n_components=2).fit_transform(df_RobustScaler)
labels_kmean=kmean(tsne)
df_RobustScaler["cluster"]=labels_kmean.astype(str)

for c in df_RobustScaler:
grid= sns.FacetGrid(df_RobustScaler, col='cluster')
grid.map(plt.hist, c)
cl = []
for row in df_RobustScaler['cluster']:
if row == "0" :cl.append("People mostly doesn't pay by Cash in Adva
nce mostly Purchase in installment and Purchase Frequently")
elif row =="1":cl.append("People with high Purchase Frequently and
use all types of payments ")
elif row =="2":cl.append("People mostly pay by Cash in Advance with
high palance and less Purchase Frequently")
elif row =="3":cl.append("People with less Purchases and mostly doe
sn't Purchase in installment")
#sns.scatterplot(tsne[:,0], tsne[:,1],s=100)
cluster_dbscan_labels=dbscan(tsne)
plt.scatter(tsne[:,0], tsne[:,1] , c = cluster_dbscan_labels,s=100)
#sns.scatterplot(tsne[:,0], tsne[:,1] , c = cluster_dbscan_labels,s=10,
palette="Set2")
tsne_ss = TSNE(n_components=2).fit_transform(df_Standard_Scaler)
cluster_dbscan_labels=dbscan(tsne_ss)
plt.scatter(tsne_ss[:,0], tsne_ss[:,1] , c = cluster_dbscan_labels,s=10
)
 Reg.no:411622149007

OUTPUT:
Read the data Set

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 8950 entries, 0 to 8949
Data columns (total 18 columns):
# Column Non-Null Count Dtype

0 CUST_ID 8950 non-null object

1 BALANCE 8950 non-null float64
2 BALANCE_FREQUENCY 8950 non-null float64
3 PURCHASES 8950 non-null float64
4 ONEOFF_PURCHASES 8950 non-null float64
5 INSTALLMENTS_PURCHASES 8950 non-null float64
6 CASH_ADVANCE 8950 non-null float64
7 PURCHASES_FREQUENCY 8950 non-null float64
8 ONEOFF_PURCHASES_FREQUENCY 8950 non-null float64
9 PURCHASES_INSTALLMENTS_FREQUENCY 8950 non-null float64
10 CASH_ADVANCE_FREQUENCY 8950 non-null float64
11 CASH_ADVANCE_TRX 8950 non-null int64
12 PURCHASES_TRX 8950 non-null int64
13 CREDIT_LIMIT 8949 non-null float64
14 PAYMENTS 8950 non-null float64
15 MINIMUM_PAYMENTS 8637 non-null float64
16 PRC_FULL_PAYMENT 8950 non-null float64
17 TENURE 8950 non-null int64
dtypes: float64(14), int64(3), object(1)
CUST_ID 0.00
BALANCE 0.00
BALANCE_FREQUENCY 0.00
PURCHASES 0.00
ONEOFF_PURCHASES 0.00
INSTALLMENTS_PURCHASES 0.00
CASH_ADVANCE 0.00
PURCHASES_FREQUENCY 0.00
ONEOFF_PURCHASES_FREQUENCY 0.00
PURCHASES_INSTALLMENTS_FREQUENCY 0.00
CASH_ADVANCE_FREQUENCY 0.00
CASH_ADVANCE_TRX 0.00
PURCHASES_TRX 0.00
CREDIT_LIMIT 0.01
PAYMENTS 0.00
MINIMUM_PAYMENTS 3.50
 Reg.no:411622149007

PRC_FULL_PAYMENT 0.00
TENURE 0.00
dtype: float64
 Reg.no:411622149007

Data Preprocessing
BALANCE 0.0
BALANCE_FREQUENCY 0.0
PURCHASES 0.0
ONEOFF_PURCHASES 0.0
INSTALLMENTS_PURCHASES 0.0
CASH_ADVANCE 0.0
PURCHASES_FREQUENCY 0.0
ONEOFF_PURCHASES_FREQUENCY 0.0
PURCHASES_INSTALLMENTS_FREQUENCY 0.0
CASH_ADVANCE_FREQUENCY 0.0
CASH_ADVANCE_TRX 0.0
PURCHASES_TRX 0.0
CREDIT_LIMIT 0.0
PAYMENTS 0.0
MINIMUM_PAYMENTS 0.0
PRC_FULL_PAYMENT 0.0
TENURE 0.0
dtype: float64

Features Transformation:
 Reg.no:411622149007

PCA:
 Reg.no:411622149007

k-Means

Comparison:

Hierarchical Clustering:
 Reg.no:411622149007

Dbscan:

3- dbscan with and Standard_Scaler

Result:
Thus, the Clustering Algorithms using Python was executed and output has been
verified successfully.
 Reg.no:411622149007

Ex. No:10 Build simple NN models

AIM:
To Build simple NN models using Python.
ALGORITHM:
Step 1: Start the Program.
Step 2: Given a set of incomplete data, consider a set of starting parameters.
Step 3: Expectation step (E – step): Using the observed available data of the dataset, estimate
(guess) the values of the missing data.
Step 4: Maximization step (M – step): Complete data generated after the expectation (E) step is
used in order to update the parameters.
Step 5: Repeat step 3 and step 4 until convergence.
Step 6: Stop

PROGRAM:

import numpy as np
import pandas as pd

from sklearn.datasets import make_blobs

from scipy.stats import multivariate_normal
import seaborn as sns
import matplotlib.pyplot as plt

# matplotlib
plt.rc('font', size=15)
plt.rc('axes', titlesize=18)
plt.rc('xtick', labelsize=10)
plt.rc('ytick', labelsize=10)

# seaborn
sns.set(font_scale = 1.2)
sns.set_style('whitegrid')
# Configuration
RANDOM_SEED = 1
N_SAMPLE_SIZE = 1500
N_CLUSTER = 3
def plot_clusters(X, y=None, ax=None, **kwargs):
"""Displays the given data as a scatter plot."""
if ax is None:
ax = plt.gca()

sns.scatterplot(
x=X[:, 0],
 Reg.no:411622149007

y=X[:, 1],
palette='Set2',
hue=y,
ax=ax)

return ax
from matplotlib.patches import Ellipse

def plot_ellipse(x, covar, alpha=0.3, ax=None, **kwargs):

"""Displays for the given covarence matrix an ellipse that
which represents of the highest spread.
"""
if ax is None:
ax = plt.gca()

for f, a in zip([1, 2, 3], [1., 0.75, 0.5]):

U, s, Vt = np.linalg.svd(covar)
angle = np.degrees(np.arctan2(U[1, 0], U[0, 0]))
width, height = f * 2 * np.sqrt(s)

e = Ellipse(
x,
width,
height,
angle=angle,
facecolor='gray',
**kwargs)
e.set_alpha(a*alpha)
ax.add_artist(e)

return ax
def plot_center(x, ax=None, **kwargs):
"""Displays of a center point for the specified coordinate."""
if ax is None:
ax = plt.gca()

sns.scatterplot(
x=[x[0]],
y=[x[1]],
color='red',
s=80,
alpha=0.8,
ax=ax)

return ax
X, y = make_blobs(
n_samples=N_SAMPLE_SIZE,
centers=N_CLUSTER,
 Reg.no:411622149007

n_features=2,
random_state=RANDOM_SEED
)
plt.subplots(1, 1, figsize=(8, 8))

plot_clusters(X)
plt.show()

from typing import NamedTuple

class GaussianMixture(NamedTuple):
"""Gaussian mixture parameter set."""
# (K, dim) array -
each row corresponds to a gaussian component mean
mu: np.ndarray

# (K, ) array - each row corresponds to the variance of a component

var: np.ndarray

# (K, ) array = each row corresponds to the weight of a component

p: np.ndarray
def init(
X:np.ndarray,
num_clusters:int=3,
seed=None) -> GaussianMixture:
"""Initializes the gaussian mixture parameters.

Parameters

X: np.ndarray, (N, dim) numpy array of observed data points

num_clusters: int, number of clusters, optional, default 3
seed: int, random state, optional, default None

Returns

gm: initialized gaussian mixture parameters

"""
# data size (N) and dimension (d)
N, dim = X.shape
# init mixture component means (μ's)
sigma = [[1, 0], [0, 1]]
mu = multivariate_normal.rvs(
[0, 0],
sigma,
size=num_clusters, random_state=seed)

# init covariances (Σ's)

var = [np.identity(dim) for k in range(0, num_clusters)]
 Reg.no:411622149007

# init weights of mixture components

p = np.ones(num_clusters) / num_clusters

return GaussianMixture(mu, var, p)

E-Step
def e_step(X:np.ndarray, gm:GaussianMixture) -> np.ndarray:
"""Performing E-Step.

Parameters

X: np.ndarray, (N, dim) numpy array of observed data points

gm: the current gaussian mixture parameters

Returns

post: the posterior probability p(i|k) that

the data point x_i belongs in cluster k.
"""
N, _ = X.shape
K, dim = gm.mu.shape # number of clusters (K)

# init posteriors probabilities

post = np.zeros((N, K))

for k in range(K):
rv = multivariate_normal(gm.mu[k], gm.var[k])
post[:, k] = gm.p[k] * rv.pdf(X)

# normalize
c = np.sum(post, axis=1)[:, np.newaxis]
post /= c

return post
M-Step
def m_step(X:np.ndarray, gm:GaussianMixture, post:np.ndarray) -
> GaussianMixture:
"""Performing M-Step.

Parameters

X: np.ndarray, (N, dim) numpy array of observed data points.

gm: the current gaussian mixture parameters.
post: the posterior probability p(i|k).

Returns

gm: the updated gaussian mixture parameters.

 Reg.no:411622149007

"""
N, _ = X.shape
K, dim = gm.mu.shape # number of clusters (K)

# update weights
w = np.sum(post, axis=0)
p = w / N
# update mixture component means (μ's)
c = np.sum(post, axis=1)[:, np.newaxis]
mu = np.dot(post.T, X) / w[:, np.newaxis]

# update covariances (Σ's)

var = [np.identity(dim) for k in range(0, K)]
for k in range(K):
xs = X - gm.mu[k, :]

p_diag = np.diag(post[:, k])

p_diag = np.matrix(p_diag)

sigma = xs.T * p_diag * xs

var[k] = (sigma) / w[:, np.newaxis][k]

return GaussianMixture(mu, var, p)

def get_labels(post:np.ndarray) -> np.ndarray:
"""Returns the cluster belonging to the data point."""
return np.argmax(post, axis=1)
def plot_gaussian_mixture(
X:np.ndarray,
gm:GaussianMixture,
post:np.ndarray,
ax=None,
**kwargs):
"""Displays the current state of the
Gaussian mixture parameter set."""
if ax is None:
ax = plt.gca()

y = get_labels(post)
plot_clusters(X, y=y, ax=ax)

for k in range(N_CLUSTER):
μ = gm.mu[k]
Σ = gm.var[k]

plot_center(μ, ax=ax)
plot_ellipse(μ, Σ, alpha=0.3, ax=ax)

return ax
 Reg.no:411622149007

gm = init(X, num_clusters=N_CLUSTER, seed=RANDOM_SEED)

Visualization:

nrows = 2
ncols = 4

fig, axs = plt.subplots(nrows, ncols, figsize=(30, 15))

for (step, ax) in zip(range(nrows*ncols), axs.ravel()):

post = e_step(X, gm)
gm = m_step(X, gm, post)

plot_gaussian_mixture(X, gm, post, ax=ax)

ax.set_title(f'Step {step+1}')

plt.legend()
plt.show()
 Reg.no:411622149007

OUTPUT:

Result:
Thus, the EM Algorithm for Bayesian Network was built using Python was executed and
output has been verified successfully.
 Reg.no:411622149007

Ex. No:11 Build simple NN models

AIM:
To Build simple NN models using Python.
ALGORITHM:
Step 1: Start the Program.
Step 2: Import the libraries. For example: import numpy as np
Step 3: Define/create input data. For example, use numpy to create a dataset and an array of
data values.
Step 4: Add weights and bias (if applicable) to input features. These are learnable parameters,
meaning that they can be adjusted during training.
Weights = input parameters that influences output
Bias = an extra threshold value added to the output
Step 5: Train the network against known, good data in order to find the correct values for the
weights and biases.
Step 6: Test the Network against a set of test data to see how it performs.
Step 7: Fit the model with hyperparameters (parameters whose values are used to control the
learning process), calculate accuracy, and make a prediction.
Step 8: Stop
PROGRAM:

# Creating data set

# A
a =[0, 0, 1, 1, 0, 0,
0, 1, 0, 0, 1, 0,
1, 1, 1, 1, 1, 1,
1, 0, 0, 0, 0, 1,
1, 0, 0, 0, 0, 1]
# B
b =[0, 1, 1, 1, 1, 0,
0, 1, 0, 0, 1, 0,
0, 1, 1, 1, 1, 0,
0, 1, 0, 0, 1, 0,
0, 1, 1, 1, 1, 0]
# C
c =[0, 1, 1, 1, 1, 0,
0, 1, 0, 0, 0, 0,
0, 1, 0, 0, 0, 0,
 Reg.no:411622149007

0, 1, 0, 0, 0, 0,
0, 1, 1, 1, 1, 0]

# Creating labels
y =[[1, 0, 0],
[0, 1, 0],
[0, 0, 1]]

import numpy as np
import matplotlib.pyplot as plt
# visualizing the data, plotting A.
plt.imshow(np.array(a).reshape(5, 6))
plt.show()
# converting data and labels into numpy array

"""
Convert the matrix of 0 and 1 into one hot vector
so that we can directly feed it to the neural network,
these vectors are then stored in a list x.
"""

x =[np.array(a).reshape(1, 30), np.array(b).reshape(1, 30),

np.array(c).reshape(1, 30)]
# Labels are also converted into NumPy array
y = np.array(y)
print(x, "\n\n", y)
# activation function

def sigmoid(x):
return(1/(1 + np.exp(-x)))

# Creating the Feed forward neural network

# 1 Input layer(1, 30)
# 1 hidden layer (1, 5)
# 1 output layer(3, 3)
def f_forward(x, w1, w2):
# hidden
z1 = x.dot(w1)# input from layer 1
a1 = sigmoid(z1)# out put of layer 2

# Output layer
z2 = a1.dot(w2)# input of out layer
a2 = sigmoid(z2)# output of out layer
return(a2)
# initializing the weights randomly
def generate_wt(x, y):
l =[]
for i in range(x * y):
 Reg.no:411622149007

l.append(np.random.randn())
return(np.array(l).reshape(x, y))
# for loss we will be using mean square error(MSE)
def loss(out, Y):
s =(np.square(out-Y))
s = np.sum(s)/len(y)
return(s)
# Back propagation of error
def back_prop(x, y, w1, w2, alpha):

# hidden layer
z1 = x.dot(w1)# input from layer 1
a1 = sigmoid(z1)# output of layer 2

# Output layer
z2 = a1.dot(w2)# input of out layer
a2 = sigmoid(z2)# output of out layer
# error in output layer
d2 =(a2-y)
d1 = np.multiply((w2.dot((d2.transpose()))).transpose(),
(np.multiply(a1, 1-a1)))

# Gradient for w1 and w2

w1_adj = x.transpose().dot(d1)
w2_adj = a1.transpose().dot(d2)

# Updating parameters
w1 = w1-(alpha*(w1_adj))
w2 = w2-(alpha*(w2_adj))

return(w1, w2)
def train(x, Y, w1, w2, alpha = 0.01, epoch = 10):
acc =[]
losss =[]
for j in range(epoch):
l =[]
for i in range(len(x)):
out = f_forward(x[i], w1, w2)
l.append((loss(out, Y[i])))
w1, w2 = back_prop(x[i], y[i], w1, w2, alpha)
print("epochs:", j + 1, "======== acc:", (1-(sum(l)/len(x)))*100)
acc.append((1-(sum(l)/len(x)))*100)
losss.append(sum(l)/len(x))
return(acc, losss, w1, w2)

def predict(x, w1, w2):

Out = f_forward(x, w1, w2)
maxm = 0
 Reg.no:411622149007

k = 0
for i in range(len(Out[0])):
if(maxm<Out[0][i]):
maxm = Out[0][i]
k = i
if(k == 0):
print("Image is of letter A.")
elif(k == 1):
print("Image is of letter B.")
else:
print("Image is of letter C.")
plt.imshow(x.reshape(5, 6))
plt.show()

w1 = generate_wt(30, 5)
w2 = generate_wt(5, 3)
print(w1, "\n\n", w2)

"""The arguments of train function are data set list x,

correct labels y, weights w1, w2, learning rate = 0.1,
no of epochs or iteration.The function will return the
matrix of accuracy and loss and also the matrix of
trained weights w1, w2"""

acc, losss, w1, w2 = train(x, y, w1, w2, 0.1, 100)

import matplotlib.pyplot as plt1

# plotting accuracy
plt1.plot(acc)
plt1.ylabel('Accuracy')
plt1.xlabel("Epochs:")
plt1.show()

# plotting Loss
plt1.plot(losss)
plt1.ylabel('Loss')
plt1.xlabel("Epochs:")
plt1.show()

# the trained weights are

print(w1, "\n", w2)

"""
The predict function will take the following arguments:
1) image matrix
2) w1 trained weights
3) w2 trained weights
 Reg.no:411622149007

"""
predict(x[1], w1, w2)

OUTPUT

[array([[0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0,
0,
0, 1, 1, 0, 0, 0, 0, 1]]), array([[0, 1, 1, 1, 1, 0, 0, 1, 0,
0, 1, 0, 0, 1, 1, 1, 1, 0, 0, 1, 0, 0,
1, 0, 0, 1, 1, 1, 1, 0]]), array([[0, 1, 1, 1, 1, 0, 0, 1, 0,
0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0,
0, 0, 0, 1, 1, 1, 1, 0]])]

[[1 0 0]
[0 1 0]
[0 0 1]]

Activation Function:
[[-1.19606517 0.35701803 0.71317902 0.50827198 1.98620667]
[-1.59077486 0.41880816 0.57304877 -0.37205614 -1.04269024]
[-0.1941151 0.50861565 -1.56259308 -0.08872702 0.33786455]
[-0.20572917 3.39354129 -1.06391825 0.28943676 -1.52478848]
[-1.54570587 -0.17357302 -0.42439848 -2.06332659 1.74626094]
[-0.75078944 2.00375702 -0.02322824 -1.07577908 1.1380853 ]
[ 0.86302856 -0.64502336 -0.23851911 1.01381374 1.30492746]
[ 1.10468106 1.40883514 -1.89216469 0.27791238 -0.01079454]
[ 0.0486144 0.35403961 -0.24370201 -2.51911717 -0.52835615]
[ 1.10427439 -1.20811277 -0.51972944 -0.29924303 -2.39862386]
[ 0.18311864 -0.353889 1.02854945 -0.5052859 -0.30519449]
[-0.75303406 2.71893476 -0.28173665 0.04685361 -0.60211221]
 Reg.no:411622149007

[ 1.28267724 0.32642875 -1.55429807 -0.24404682 1.06219742]

[-0.94837659 -0.04879027 -0.52295672 0.96638115 -0.47169872]
[ 0.74201662 -0.12681501 -0.71983046 -0.26385494 -0.46228193]
[ 0.24062653 0.9108022 -1.21971755 0.21067048 0.50134678]
[-0.44890693 -0.94968496 -0.77635742 0.9483034 0.38962539]
[ 0.40462923 1.33102726 -0.46923999 -0.71115943 -1.10276262]
[ 0.47003265 1.42150683 1.59870017 -1.07844223 0.80849334]
[-0.53332495 -1.76367945 -0.25719279 0.83808537 -0.07525124]
[-0.42082677 -0.09650442 0.8275041 -0.17783027 0.27453112]
[-0.75790551 0.60631097 0.31681654 -0.20157639 0.79697155]
[ 0.19513623 -0.17678604 1.73468611 0.34478341 2.33183406]
[ 0.47045423 0.48074763 -0.03381237 -0.77425396 0.2795718 ]
[ 0.14941963 2.03651344 -0.32885339 -0.80571395 -1.04899057]
[-1.03560549 -1.55460166 -0.08698937 0.30126974 -0.64140744]
[-1.81294287 0.78174447 0.94985655 -0.26978401 -1.18139017]
[-0.00698634 0.47953131 -1.85294466 -1.16693706 -0.1639528 ]
[ 2.04133358 -1.06009322 -0.46057384 -0.78910481 -2.15511288]
[-0.3610823 -0.06351741 0.48011831 -0.45257641 0.61474472]]

[[ 0.33082373 -1.04916163 0.1591827 ]

[ 1.98346653 0.08033702 -1.82104583]
[ 0.47054958 -0.76540532 1.3667822 ]
[ 1.01484511 -0.92001887 1.64080433]
[ 1.41899555 -0.58029631 0.67309987]]
Training:
epochs: 1 ======== acc: 67.72583076546357
epochs: 2 ======== acc: 69.57743892457708
epochs: 3 ======== acc: 71.92191923734961
epochs: 4 ======== acc: 74.64780011769821
epochs: 5 ======== acc: 77.10121154572734
epochs: 6 ======== acc: 78.7909453312117
epochs: 7 ======== acc: 79.77890380226344
epochs: 8 ======== acc: 80.37605732370298
epochs: 9 ======== acc: 80.77258557608658
epochs: 10 ======== acc: 81.05792856360294
epochs: 11 ======== acc: 81.27578854555911
epochs: 12 ======== acc: 81.44945675041357
epochs: 13 ======== acc: 81.59239660982531
epochs: 14 ======== acc: 81.71293612939799
epochs: 15 ======== acc: 81.81652036985197
epochs: 16 ======== acc: 81.90687535859101
epochs: 17 ======== acc: 81.98664902056116
epochs: 18 ======== acc: 82.05778375429136
epochs: 19 ======== acc: 82.12174323698007
epochs: 20 ======== acc: 82.17965606484499
epochs: 21 ======== acc: 82.23240989131376
epochs: 22 ======== acc: 82.28071499277256
epochs: 23 ======== acc: 82.32514833251597
epochs: 24 ======== acc: 82.36618482468477
epochs: 25 ======== acc: 82.4042199811441
epochs: 26 ======== acc: 82.43958662420503
epochs: 27 ======== acc: 82.47256742851486
epochs: 28 ======== acc: 82.50340447685082
epochs: 29 ======== acc: 82.53230664185087
epochs: 30 ======== acc: 82.5594553604646
epochs: 31 ======== acc: 82.5850092033542
epochs: 32 ======== acc: 82.60910752908485
epochs: 33 ======== acc: 82.63187343491384
 Reg.no:411622149007

epochs: 34 ======== acc: 82.65341616098921

epochs: 35 ======== acc: 82.67383306545919
epochs: 36 ======== acc: 82.69321125952959
epochs: 37 ======== acc: 82.71162897064642
epochs: 38 ======== acc: 82.7291566865189
epochs: 39 ======== acc: 82.74585812111766
epochs: 40 ======== acc: 82.76179103502047
epochs: 41 ======== acc: 82.77700793578944
epochs: 42 ======== acc: 82.79155667891031
epochs: 43 ======== acc: 82.80548098582226
epochs: 44 ======== acc: 82.81882089243518
epochs: 45 ======== acc: 82.8316131390614
epochs: 46 ======== acc: 82.8438915107273
epochs: 47 ======== acc: 82.85568713526386
epochs: 48 ======== acc: 82.867028745314
epochs: 49 ======== acc: 82.87794290937478
epochs: 50 ======== acc: 82.88845423616291
epochs: 51 ======== acc: 82.89858555591306
epochs: 52 ======== acc: 82.90835808166
epochs: 53 ======== acc: 82.917791553095
epochs: 54 ======== acc: 82.92690436520334
epochs: 55 ======== acc: 82.93571368357114
epochs: 56 ======== acc: 82.94423554798227
epochs: 57 ======== acc: 82.95248496570096
epochs: 58 ======== acc: 82.96047599564692
epochs: 59 ======== acc: 82.96822182450822
epochs: 60 ======== acc: 82.9757348357013
epochs: 61 ======== acc: 82.98302667197058
epochs: 62 ======== acc: 82.99010829232081
epochs: 63 ======== acc: 82.99699002388944
epochs: 64 ======== acc: 83.00368160929298
epochs: 65 ======== acc: 83.0101922499175
epochs: 66 ======== acc: 83.01653064556854
epochs: 67 ======== acc: 83.02270503084823
epochs: 68 ======== acc: 83.02872320858542
epochs: 69 ======== acc: 83.03459258060865
epochs: 70 ======== acc: 83.04032017612069
epochs: 71 ======== acc: 83.04591267790488
epochs: 72 ======== acc: 83.05137644656976
epochs: 73 ======== acc: 83.05671754301737
epochs: 74 ======== acc: 83.06194174930114
epochs: 75 ======== acc: 83.06705458802372
epochs: 76 ======== acc: 83.07206134040976
epochs: 77 ======== acc: 83.07696706317617
epochs: 78 ======== acc: 83.08177660431105
epochs: 79 ======== acc: 83.08649461786244
epochs: 80 ======== acc: 83.09112557782898
epochs: 81 ======== acc: 83.09567379123689
epochs: 82 ======== acc: 83.10014341048088
epochs: 83 ======== acc: 83.10453844500036
epochs: 84 ======== acc: 83.10886277235669
epochs: 85 ======== acc: 83.1131201487733
epochs: 86 ======== acc: 83.11731421919552
epochs: 87 ======== acc: 83.12144852692379
epochs: 88 ======== acc: 83.12552652287107
epochs: 89 ======== acc: 83.12955157449258
epochs: 90 ======== acc: 83.13352697443315
epochs: 91 ======== acc: 83.13745594893709
 Reg.no:411622149007

epochs: 92 ======== acc: 83.1413416660627

epochs: 93 ======== acc: 83.14518724374365
epochs: 94 ======== acc: 83.148995757738
epochs: 95 ======== acc: 83.15277024950625
epochs: 96 ======== acc: 83.15651373405873
epochs: 97 ======== acc: 83.16022920781464
epochs: 98 ======== acc: 83.16391965651432
epochs: 99 ======== acc: 83.16758806322883
epochs: 100 ======== acc: 83.17123741651189

Trained weights:
[[-1.19606517 0.35701803 0.71317902 0.50827198 1.98620667]
[-1.69198646 -0.26861884 0.56666918 -0.50344874 -1.27665875]
[-0.21125776 -0.17732944 -1.5692321 -0.22550232 0.32036221]
[-0.22287183 2.7075962 -1.07055728 0.15266146 -1.54229081]
[-1.64691748 -0.86100003 -0.43077808 -2.1947192 1.51229243]
[-0.75078944 2.00375702 -0.02322824 -1.07577908 1.1380853 ]
[ 0.86302856 -0.64502336 -0.23851911 1.01381374 1.30492746]
[ 1.0875384 0.72289005 -1.89880372 0.14113708 -0.02829687]
[ 0.0486144 0.35403961 -0.24370201 -2.51911717 -0.52835615]
[ 1.10427439 -1.20811277 -0.51972944 -0.29924303 -2.39862386]
[ 0.15004908 -0.42548163 1.01105131 -0.87578889 -0.31919913]
[-0.75303406 2.71893476 -0.28173665 0.04685361 -0.60211221]
[ 1.36674619 0.32791067 -1.55455751 -0.24942951 1.27866359]
[-0.96551924 -0.73473536 -0.52959574 0.82960585 -0.48920106]
[ 0.70894706 -0.19840764 -0.7373286 -0.63435794 -0.47628657]
[ 0.20755697 0.83920957 -1.23721569 -0.15983252 0.48734214]
[-0.48197649 -1.02127759 -0.79385556 0.5778004 0.37562074]
[ 0.48869817 1.33250917 -0.46949942 -0.71654212 -0.88629644]
[ 0.5541016 1.42298874 1.59844073 -1.08382493 1.02495951]
[-0.63453655 -2.45110645 -0.26357239 0.70669276 -0.30921974]
[-0.42082677 -0.09650442 0.8275041 -0.17783027 0.27453112]
[-0.75790551 0.60631097 0.31681654 -0.20157639 0.79697155]
[ 0.07799772 -0.24986059 1.7174474 -0.0203369 2.10136325]
[ 0.55452318 0.48222955 -0.0340718 -0.77963666 0.49603798]
[ 0.23348858 2.03799535 -0.32911282 -0.81109665 -0.83252439]
[-1.1368171 -2.24202867 -0.09336897 0.16987714 -0.87537594]
[-1.91415448 0.09431746 0.94347695 -0.40117661 -1.41535867]
[-0.10819795 -0.2078957 -1.85932425 -1.29832966 -0.3979213 ]
[ 1.94012198 -1.74752022 -0.46695343 -0.92049742 -2.38908138]
[-0.27701335 -0.0620355 0.47985888 -0.4579591 0.83121089]]
[[ 0.78098127 -1.89716614 -0.77318318]
 Reg.no:411622149007

[ 1.88359722 -0.94157921 -2.49346205]

[ 0.45715572 -0.76582541 1.36622953]
[ 0.74224586 -0.97016698 1.66533937]
[ 1.51442739 -0.80964251 0.28257926]]

Predicted Output:

Result:
Thus, the simple NN models was built using Python was executed and output has been
verified successfully.
 Reg.no:411622149007

Ex. No:11 Build simple NN models

AIM:
To Build simple NN models using Python.
ALGORITHM:
Step 1: Start the Program.
Step 2: Import the libraries. For example: import numpy as np
Step 3: Define/create input data. For example, use numpy to create a dataset and an array of
data values.
Step 4: Add weights and bias (if applicable) to input features. These are learnable parameters,
meaning that they can be adjusted during training.
Weights = input parameters that influences output
Bias = an extra threshold value added to the output
Step 5: Train the network against known, good data in order to find the correct values for the
weights and biases.
Step 6: Test the Network against a set of test data to see how it performs.
Step 7: Fit the model with hyperparameters (parameters whose values are used to control the
learning process), calculate accuracy, and make a prediction.
Step 8: Stop
PROGRAM:

import numpy as np
import matplotlib.pyplot as plt
import random

#
import tensorflow as tf
import keras
from tensorflow import keras
import keras_tuner as kt
fashion_mnist = keras.datasets.fashion_mnist
(train_images, train_labels), (test_images, test_labels) = fashion_mnis
t.load_data()
train_images = train_images / 255.0
test_images = test_images / 255.0
plt.figure(figsize=(10,10))
for i in range(16):
plt.subplot(4,4,i+1)
plt.xticks([])
 Reg.no:411622149007

plt.yticks([])
plt.grid(False)
plt.imshow(train_images[i], cmap=plt.cm.binary)
plt.title("Image label is: {}".format(train_labels[i]))
plt.show()
x_train = train_images.reshape(-1,28,28,1)
x_test = test_images.reshape(-1,28,28,1)
early_stop = keras.callbacks.EarlyStopping(monitor='val_loss', patience
=3)
def build_model(hp):
model = keras.Sequential([

# First conv_block
keras.layers.Conv2D(
filters = hp.Choice('conv_1_filter', values=[16, 32, 64, 128]),
kernel_size=hp.Choice('conv_1_kernel', values = [3,4]),
activation='relu',
input_shape=(28,28,1)),
keras.layers.MaxPooling2D((2,2)),

# Second conv_block
keras.layers.Conv2D(
filters = hp.Choice('conv_2_filter', values=[16, 32, 64, 128]),
kernel_size=hp.Choice('conv_2_kernel', values = [3,4]),
activation='relu'),
keras.layers.MaxPooling2D((2,2)),

#
keras.layers.Flatten(),
keras.layers.Dense(units = hp.Choice('units', values=[16, 32, 64, 1
28, 256]),
activation='relu'),
keras.layers.Dropout(hp.Float('dropout', 0, 0.5, step=0.1, default=
0.5)),
#
keras.layers.Dense(10)
])

model.compile(optimizer=keras.optimizers.Adam(hp.Choice('learning_r
ate',
values=[1e-
1, 1e-2, 1e-3, 1e-4])),
loss=keras.losses.SparseCategoricalCrossentropy(from_logi
ts=True),
metrics=['accuracy'])
return model
tuner = kt.Hyperband(build_model,
objective="val_accuracy",
 Reg.no:411622149007

max_epochs=5,
factor=3,
hyperband_iterations=3)
tuner.search_space_summary()
tuner.search(x_train,train_labels, epochs=3, validation_split=0.2)
print(f"""conv_1_filter is {best_hps.get('conv_1_filter')}""")
print(f"""conv_1_kernel is {best_hps.get('conv_1_kernel')}""")
print(f"""conv_2_filter is {best_hps.get('conv_2_filter')}""")
print(f"""conv_2_kernel is {best_hps.get('conv_2_kernel')}""")
print(" ")
print(f"""units is {best_hps.get('units')}""")
print(f"""learning_rate is {best_hps.get('learning_rate')}""")
print(f"""dropout is {best_hps.get('dropout')}""")
model = tuner.hypermodel.build(best_hps)
history = model.fit(x_train, train_labels,
epochs=50, validation_split=0.2)

val_acc_per_epoch = history.history['val_accuracy']
best_epoch = val_acc_per_epoch.index(max(val_acc_per_epoch)) + 1
print('Best epoch: %d' % (best_epoch,))
hypermodel = tuner.hypermodel.build(best_hps)

history = hypermodel.fit(x_train, train_labels,

epochs=best_epoch,
validation_split=0.2,
callbacks=[early_stop])
hypermodel.summary()
keras.utils.plot_model(hypermodel, show_shapes=True)
successive_outputs = [layer.output for layer in hypermodel.layers[1:]]
visualization_model = keras.models.Model(inputs = hypermodel.input, out
puts = successive_outputs)
index = 20
plt.imshow(train_images[index], cmap=plt.cm.binary)

x = train_images[index]
x = x.reshape((1,) + x.shape)
x /= 255
successive_feature_maps = visualization_model.predict(x)
layer_names = [layer.name for layer in hypermodel.layers[1:]]

for layer_name, feature_map in zip(layer_names, successive_feature_maps

):
if len(feature_map.shape) == 4:
n_features = feature_map.shape[-1]
size = feature_map.shape[1]
display_grid = np.zeros((size, size * n_features))
for i in range(n_features):
x = feature_map[0, :, :, i]
 Reg.no:411622149007

x -= x.mean()
x /= x.std()
x *= 64
x += 128
x = np.clip(x, 0, 255).astype('uint8')
display_grid[:, i * size : (i + 1) * size] = x
scale = 20. / n_features
plt.figure(figsize=(scale * n_features, scale))
plt.title(layer_name)
plt.grid(False)
plt.imshow(display_grid, aspect='auto', cmap='viridis')

eval_result = hypermodel.evaluate(x_test, test_labels)

print("[test loss, test accuracy]:", eval_result)
pred = hypermodel.predict(x_test)

print("Prediction is -> {}".format(pred[12]))

print("Actual value is -> {}".format(test_labels[12]))
print("The highest value for label is {}".format(np.argmax(pred[12])))
import matplotlib.pyplot as plt
%matplotlib inline
acc = history.history['accuracy']
val_acc = history.history['val_accuracy']
loss = history.history['loss']
val_loss = history.history['val_loss']

epochs = range(len(acc))

plt.plot(epochs, acc, 'bo', label='Training acc')

plt.plot(epochs, val_acc, 'b', label='Validation acc')
plt.title('Training and validation accuracy')
plt.legend()

plt.figure()

plt.plot(epochs, loss, 'bo', label='Training loss')

plt.plot(epochs, val_loss, 'b', label='Validation loss')
plt.title('Training and validation loss')
plt.legend()
plt.show()
 Reg.no:411622149007

OUTPUT:

Search space summary

Default search space size: 7
conv_1_filter (Choice)
 Reg.no:411622149007

{'default': 16, 'conditions': [], 'values': [16, 32, 64, 128],

'ordered': True}
conv_1_kernel (Choice)
{'default': 3, 'conditions': [], 'values': [3, 4], 'ordered': True}
conv_2_filter (Choice)
{'default': 16, 'conditions': [], 'values': [16, 32, 64, 128],
'ordered': True}
conv_2_kernel (Choice)
{'default': 3, 'conditions': [], 'values': [3, 4], 'ordered': True}
units (Choice)
{'default': 16, 'conditions': [], 'values': [16, 32, 64, 128, 256],
'ordered': True}
dropout (Float)
{'default': 0.5, 'conditions': [], 'min_value': 0.0, 'max_value': 0.5,
'step': 0.1, 'sampling': None}
learning_rate (Choice)
{'default': 0.1, 'conditions': [], 'values': [0.1, 0.01, 0.001,
0.0001], 'ordered': True}

Trial 30 Complete [00h 00m 42s]

val_accuracy: 0.8424999713897705

Best val_accuracy So Far: 0.9085000157356262

Total elapsed time: 00h 10m 49s
Reg.no:411622149007
Reg.no:411622149007
 Reg.no:411622149007

Result:
Thus, the deep NN models was built using Python was executed and output has been
verified successfully.