Sri Venkateshwara College of Engineering

NH 7, Vidyanagar, KIA Road, Bengaluru - 562 157.

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

(Accredited by NBA)

A LABORATORY MANUAL

FOR
ARTIFICIAL INTELLIGENCE AND MACHINE LEARNING
LABORATORY
(18CSL76)

VII SEMESTER

Prepared by:
1. Suresh P
2. Surekha Gowda
3. Ashika
PREREQUISITES

 PROGRAMMING IN PYTHON

 KNOWLEDGE OF BASIC MACHINE LEARNINGALGORITHMS

 KNOWLEDGE OF COMMON STATISTICAL METHODS ANDDATA

ANALYSIS BESTPRACTICES.

SOFTWARE REQUIREMENTS

 ANACONDA VERSION 3 ANDABOVE

 MACHINE LEARNINGPACKAGES

 SCIKIT-LEARN

 NUMPY-MATRICES AND LINEARALGEBRA

 SCIPY-MANY NUMERICALVALUES

 PANDAS-FACILITATES STRUCTURED/TABULARDATA
MANIPULATION AND VISUALIZATIONS.

OPERATING SYSTEMS

 WINDOWS/LINUX

 ANACONDA PYTHON DISTRIBUTION IS COMPATIBLEWITH

WINDOWS ANDLINUX
 ARTIFICIAL INTELLIGENCE AND MACHINE LEARNING LABORATORY
(Effective from the academic year 2018 -2019)
SEMESTER – VII
Course Code 18CSL76 CIE Marks 40
Number of Contact Hours/Week 0:0:2 SEE Marks 60
Total Number of Lab Contact Hours 36 Exam Hours 03
Credits – 2
Course Learning Objectives: This course (18CSL76) will enable students to:
 Implement and evaluate AI and ML algorithms in and Python programming language.
Descriptions (if any):
Installation procedure of the required software must be demonstrated, carried out in groups
and documented in the journal.
Programs List:
1. Implement A* Search algorithm.
2. Implement AO* Search algorithm.
3. For a given set of training data examples stored in a .CSV file, implement and demonstrate
the Candidate-Elimination algorithm to output a description of the set of all hypotheses
consistent with the training examples.
4. Write a program to demonstrate the working of the decision tree based ID3 algorithm. Use an
appropriate data set for building the decision tree and apply this knowledge to classify a
new sample.
5. Build an Artificial Neural Network by implementing the Backpropagation algorithm and test the
same using appropriate data sets.
6. Write a program to implement the naïve Bayesian classifier for a sample training data set stored
as a .CSV file. Compute the accuracy of the classifier, considering few test data sets.
7. Apply EM algorithm to cluster a set of data stored in a .CSV file. Use the same data set for
clustering using k-Means algorithm. Compare the results of these two algorithms and comment
on the quality of clustering. You can add Java/Python ML library classes/API in the program.
8. Write a program to implement k-Nearest Neighbor algorithm to classify the iris data set. Print
both correct and wrong predictions. Java/Python ML library classes can be used for this problem.
9. Implement the non-parametric Locally Weighted Regression algorithm in order to fit data points.
Select appropriate data set for your experiment and draw graphs
Laboratory Outcomes: The student should be able to:
 Implement and demonstrate AI and ML algorithms.
 Evaluate different algorithms.
Conduct of Practical Examination:
 Experiment distribution
o For laboratories having only one part: Students are allowed to pick one experiment from
the lot with equal opportunity.
o For laboratories having PART A and PART B: Students are allowed to pick one
experiment from PART A and one experiment from PART B, with equalopportunity.
 Change of experiment is allowed only once and marks allotted for procedure to be made zero of
the changed part only.
 Marks Distribution (Courseed to change in accoradance with universityregulations)
q) For laboratories having only one part – Procedure + Execution + Viva-Voce: 15+70+15=
100Marks
r) For laboratories having PART A and PARTB
i. Part A – Procedure + Execution + Viva = 6 + 28 + 6 = 40Marks
ii. Part B – Procedure + Execution + Viva = 9 + 42 + 9 = 60Marks
 Contents

S.No Program Page No

1 A* Search Algorithm 7
12
2 AO* Search Algorithm
17
3 Candidate Elimination Algorithm
23
4 Decision Tree-ID3
31
5 Artificial Neural Network using Back propagation
38
6 Classification of data using Naïve Bayes
43
7 Data set for Clustering using K-means Algorithm
48
8 K-Nearest Neighbour algorithm
53
9 Locally Weighted Regression algorithm

Extra Lab Programs

11 Implement the binomial probability distribution 57
12 Implement the gradient descent algorithm 58
 Introduction

Machine learning

Machine learning is a subset of artificial intelligence in the field of computer science that often uses
statistical techniques to give computers the ability to "learn" (i.e., progressively improve performance
on a specific task) with data, without being explicitly programmed. In the past decade, machine
learning has given us self-driving cars, practical speech recognition, effective web search, and a vastly
improved understanding of the human genome.

Machine learning tasks

Machine learning tasks are typically classified into two broad categories, depending on whether there
is a learning "signal" or "feedback" available to a learning system:

1. Supervised learning: The computer is presented with example inputs and their desired
outputs, given by a "teacher", and the goal is to learn a general rule that maps inputs to outputs.
As special cases, the input signal can be only partially available, or restricted to specialfeedback:

2. Semi-supervised learning: the computer is given only an incomplete training signal: a

training set with some (often many) of the target outputsmissing.

3. Active learning: the computer can only obtain training labels for a limited set of instances
(based on a budget), and also has to optimize its choice of objects to acquire labels for. When
used interactively, these can be presented to the user forlabeling.

4. Reinforcement learning: training data (in form of rewards and punishments) is given only as
feedback to the program's actions in a dynamic environment, such as driving a vehicle or playing
a game against anopponent.

Page 2
 5. Unsupervised learning: No labels are given to the learning algorithm, leaving it on its own to
find structure in its input. Unsupervised learning can be a goal in itself (discovering hidden
patterns in data) or a means towards an end (featurelearning).

Machine Learning Applications

In classification, inputs are divided into two or more classes, and the learner must produce a model that
assigns unseen inputs to one or more (multi-label classification) of these classes. This is typically
tackled in a supervised manner. Spam filtering is an example of classification, where the inputs are
email (or other) messages and the classes are "spam" and "not spam".

In regression, also a supervised problem, the outputs are continuous rather than discrete.
In clustering, a set of inputs is to be divided into groups. Unlike in classification, the groups are not
known beforehand, making this typically an unsupervised task.

Density estimation finds the distribution of inputs in some space.

Dimensionality reduction simplifies inputs by mapping them into a lower dimensional space. Topic
modeling is a related problem, where a program is given a list of human language documents and is
tasked with finding out which documents cover similar topics.

Machine learning Approaches

1. Decision treelearning
Decision tree learning uses a decision tree as a predictive model, which maps observations about an
item to conclusions about the item's target value.

Page 3
2. Association rulelearning
Association rule learning is a method for discovering interesting relations between variables in large
databases.

3. Artificial neuralnetworks
An artificial neural network (ANN) learning algorithm, usually called "neural network" (NN), is a
learning algorithm that is vaguely inspired by biological neural networks. Computations are structured
in terms of an interconnected group of artificial neurons, processing information using a connectionist
approach to computation. Modern neural networks are non-linear statistical data modeling tools. They
are usually used to model complex relationships between inputs and outputs, to find patterns in data, or
to capture the statistical structure in an unknown joint probability distribution between
observedvariables.

4. Deeplearning
Falling hardware prices and the development of GPUs for personal use in the last few years have
contributed to the development of the concept of deep learning which consists of multiple hidden
layers in an artificial neural network. This approach tries to model the way the human brain processes
light and sound into vision and hearing. Some successful applications of deep learning are computer
vision and speech Recognition.

5. Inductive logicprogramming
Inductive logic programming (ILP) is an approach to rule learning using logic Programming as a
uniform representation for input examples, background knowledge, and hypotheses. Given an
encoding of the known background knowledge and a set of examples represented as a logical database
of facts, an ILP system will derive a hypothesized logic program that entails all positive and no
negative examples. Inductive programming is a related field that considers any kind of programming
languages for representing hypotheses (and not only logic programming), such as functional programs.
6. Support vector machines
Support vector machines (SVMs) are a set of related supervised learning methods used for
classification and regression. Given a set of training examples, each marked as belonging to one

Page 4
 of two categories, an SVM training algorithm builds a model that predicts whether a new example falls
into one category or the other.

7. Clustering
Cluster analysis is the assignment of a set of observations into subsets (called clusters) so that
observations within the same cluster are similar according to some pre designated criterion or criteria,
while observations drawn from different clusters are dissimilar. Different clustering techniques make
different assumptions on the structure of the data, often defined by some similarity metric and
evaluated for example by internal compactness (similarity between members of the same cluster) and
separation between different clusters. Other methods are based on estimated density and graph
connectivity. Clustering is a method of unsupervised learning, and a common technique for statistical
dataanalysis.

8. Bayesian networks
A Bayesian network, belief network or directed acyclic graphical model is a probabilistic graphical
model that represents a set of random variables and their conditional independencies via a directed
acyclic graph (DAG). For example, a Bayesian network could represent the probabilistic relationships
between diseases and symptoms. Given symptoms, the network can be used to compute the
probabilities of the presence of various diseases. Efficient algorithms exist that perform inference
andlearning.

9. Reinforcementlearning
Reinforcement learning is concerned with how an agent ought to take actions in an environment so as
to maximize some notion of long-term reward. Reinforcement learning algorithms attempt to find a
policy that maps states of the world to the actions the agent ought to take in those states.
Reinforcement learning differs from the supervised learning problem in that correct input/output pairs
are never presented, nor sub-optimal actions explicitly corrected.
10. Similarity and metriclearning
In this problem, the learning machine is given pairs of examples that are considered similar and pairs
of less similar objects. It then needs to learn a similarity function (or a distance metric function) that
can predict if new objects are similar. It is sometimes used in Recommendation systems.

Page 5
 11. Geneticalgorithms
A genetic algorithm (GA) is a search heuristic that mimics the process of natural selection, and uses
methods such as mutation and crossover to generate new genotype in the hope of finding good
solutions to a given problem. In machine learning, genetic algorithms found some uses in the 1980s
and 1990s. Conversely, machine learning techniques have been used to improve the performance of
genetic and evolutionary algorithms.

12. Rule-based machinelearning

Rule-based machine learning is a general term for any machine learning method that identifies, learns,
or evolves "rules" to store, manipulate or apply, knowledge. The defining characteristic of a rule-
based machine learner is the identification and utilization of a set of relational rules that collectively
represent the knowledge captured by the system. This is in contrast to other machine learners that
commonly identify a singular model that can be universally applied to any instance in order to make a
prediction. Rule-based machine learning approaches include learning classifier systems, association
rule learning, and artificial immunesystems.

13. Feature selectionapproach

Feature selection is the process of selecting an optimal subset of relevant features for use in model
construction. It is assumed the data contains some features that are either redundant or irrelevant, and
can thus be removed to reduce calculation cost without incurring much loss of information. Common
optimality criteria include accuracy, similarity and information measures.

Page 6
 Machine Learning Laboratory 18CSL76

Program 1
Implement A* Search Algorithm.(Ver1)
from collections import deque

class Graph:
def init (self, adjac_lis):
self.adjac_lis = adjac_lis

def get_neighbors(self, v):

return self.adjac_lis[v]

# This is heuristic function which is having equal values for all nodes
def h(self, n):
H={
'A': 1,
'B': 1,
'C': 1,
'D': 1
}

return H[n]

def a_star_algorithm(self, start, stop):

# In this open_lst is a lisy of nodes which have been visited, but who's
# neighbours haven't all been always inspected, It starts off with the start
#node
# And closed_lst is a list of nodes which have been visited
# and who's neighbors have been always inspected
open_lst = set([start])
closed_lst = set([])

# poo has present distances from start to all other nodes

# the default value is +infinity
poo = {}
poo[start] = 0

# par contains an adjac mapping of all nodes

par = {}
par[start] = start

while len(open_lst) > 0:

n = None

# it will find a node with the lowest value of f() -

for v in open_lst:
if n == None or poo[v] + self.h(v) < poo[n] + self.h(n):
n = v;

if n == None:

Department of CSE, Page 7

 Machine Learning Laboratory 18CSL76
print('Path does not exist!')
return None

# if the current node is the stop

# then we start again from start
if n == stop:
reconst_path = []

while par[n] != n:
reconst_path.append(n)
n = par[n]

reconst_path.append(start)

reconst_path.reverse()

print('Path found: {}'.format(reconst_path))

return reconst_path

# for all the neighbors of the current node do

for (m, weight) in self.get_neighbors(n):
# if the current node is not presentin both open_lst and closed_lst
# add it to open_lst and note n as it's par
if m not in open_lst and m not in closed_lst:
open_lst.add(m)
par[m] = n
poo[m] = poo[n] + weight

# otherwise, check if it's quicker to first visit n, then m

# and if it is, update par data and poo data
# and if the node was in the closed_lst, move it to open_lst
else:
if poo[m] > poo[n] + weight:
poo[m] = poo[n] + weight
par[m] = n

if m in closed_lst:
closed_lst.remove(m)
open_lst.add(m)

# remove n from the open_lst, and add it to closed_lst

# because all of his neighbors were inspected
open_lst.remove(n)
closed_lst.add(n)

print('Path does not exist!')

return None

Department of CSE, Page 8

 Machine Learning Laboratory 18CSL76
INPUT:
adjac_lis = {
'A': [('B', 1), ('C', 3), ('D', 7)],
'B': [('D', 5)],
'C': [('D', 12)]
}
graph1 = Graph(adjac_lis)
graph1.a_star_algorithm('A', 'D')

OUTPUT:
Path found: ['A', 'B', 'D']
['A', 'B', 'D']
OR
Version 2

from future import print_function

import matplotlib.pyplot as plt

class AStarGraph(object):
#Define a class board like grid with two barriers

def init (self):

self.barriers = []
self.barriers.append([(2,4),(2,5),(2,6),(3,6),(4,6),(5,6),(5,5),(5,4),(5,3),(5,2),(4,2),(3,2)])

def heuristic(self, start, goal):

#Use Chebyshev distance heuristic if we can move one square either
#adjacent or diagonal
D=1
D2 = 1
dx = abs(start[0] - goal[0])
dy = abs(start[1] - goal[1])
return D * (dx + dy) + (D2 - 2 * D) * min(dx, dy)

def get_vertex_neighbours(self, pos):

n = []
#Moves allow link a chess king
for dx, dy in [(1,0),(-1,0),(0,1),(0,-1),(1,1),(-1,1),(1,-1),(-1,-1)]:
x2 = pos[0] + dx
y2 = pos[1] + dy
if x2 < 0 or x2 > 7 or y2 < 0 or y2 > 7:
continue
n.append((x2, y2))

Department of CSE, Page 9

Machine Learning Laboratory 18CSL76
return n

def move_cost(self, a, b):

for barrier in self.barriers:
if b in barrier:
return 100 #Extremely high cost to enter barrier squares
return 1 #Normal movement cost

def AStarSearch(start, end, graph):

G = {} #Actual movement cost to each position from the start position

F = {} #Estimated movement cost of start to end going via this position

#Initialize starting values

G[start] = 0
F[start] = graph.heuristic(start, end)

closedVertices = set()
openVertices = set([start])
cameFrom = {}

while len(openVertices) > 0:

#Get the vertex in the open list with the lowest F score
current = None
currentFscore = None
for pos in openVertices:
if current is None or F[pos] < currentFscore:
currentFscore = F[pos]
current = pos

#Check if we have reached the goal

if current == end:
#Retrace our route backward
path = [current]
while current in cameFrom:
current = cameFrom[current]
path.append(current)
path.reverse()
return path, F[end] #Done!

Department of CSE, Page 10

 Machine Learning Laboratory 18CSL76
#Mark the current vertex as closed
openVertices.remove(current)
closedVertices.add(current)

#Update scores for vertices near the current position

for neighbour in graph.get_vertex_neighbours(current):
if neighbour in closedVertices:
continue #We have already processed this node exhaustively
candidateG = G[current] + graph.move_cost(current, neighbour)

if neighbour not in openVertices:

openVertices.add(neighbour) #Discovered a new vertex
elif candidateG >= G[neighbour]:
continue #This G score is worse than previously found

#Adopt this G score

cameFrom[neighbour] = current
G[neighbour] = candidateG
H = graph.heuristic(neighbour, end)
F[neighbour] = G[neighbour] + H

raise RuntimeError("A* failed to find a solution")

if name ==" main ":

graph = AStarGraph()
result, cost = AStarSearch((0,0), (7,7), graph)
print ("route", result)
print ("cost", cost)
plt.plot([v[0] for v in result], [v[1] for v in result])
for barrier in graph.barriers:
plt.plot([v[0] for v in barrier], [v[1] for v in barrier])
plt.xlim(-1,8)
plt.ylim(-1,8)
plt.show()

Output
route [(0, 0), (1, 1), (2, 2), (3, 1), (4, 1), (5, 1), (6, 2), (7, 3), (6, 4), (7, 5), (6, 6), (7, 7)]
cost 11

Department of CSE, Page 11

 Machine Learning Laboratory 18CSL76

3) Implement A* Search algorithm. –V3

#include <iostream> #include<bits/stdc++.h>
using namespace std;
struct node
{
int data;
vector< vector<node* >* >v;
bool mark;
bool solved;
};
int edge_cost=0;
void insert(node* root)
{
cout<<"Enter data of node
:"<<endl;
cin>>root->data;
//vector<vector<node*>
>vec=root->v;
cout<<"Enter number of OR
nodes for value "<<root->data<<"
:"<<endl;
int or_no;
cin>>or_no;
for(int i=0;i<or_no;i++)
{
vector<node*>* ans=new
vector<node*>;
cout<<"Enter number of AND
nodes for "<<i+1<<" or node for
value "<<root->data<<" :"<<endl;
int and_no;
cin>>and_no;
for(int j=0;j<and_no;j++)
{
node* n=new node;
n->solved=false;
n->mark=false;
insert(n);
(*ans).push_back(n);
//cout<<"inserted node
with value"<<n->data<<endl;
}
root->v.push_back(ans);
}

Department of CSE, Page 12

 Machine Learning Laboratory 18CSL76
void aostar(node* root)
{
vector<node*>* min_ans=new
vector<node*>;
(*min_ans).push_back(root);
while(!root->solved)
{
node* next_node=root;
stack<node*>st;
while(next_node &&
next_node->mark)
{
if((next_node-
>v).size()==0)
{
root->solved=true;
return;
}
int cost=INT_MAX;
st.push(next_node);
for(unsigned int
i=0;i<next_node->v.size();i++)
{

vector<node*>*ans=(next_node-
>v)[i];
vector<node*>
ans_v=*ans;
int temp_cost=0;
for(unsigned int
j=0;j<(ans_v.size());j++)
{
node*
n=ans_v[j];
temp_cost+=n-
>data;
}
if(temp_cost<cost)
{
min_ans=ans;

cost=temp_cost;
}
}
vector<node*>
min_ans_v=*min_ans;
next_node=NULL;
for(unsigned int

Department of CSE, Page 13

 Machine Learning Laboratory 18CSL76
j=0;j<min_ans_v.size();j++)
{
if(min_ans_v[j]-
>mark)
{

next_node=min_ans_v[j];
break;
}
}

vector<node*>
min_ans_v=*min_ans;
for(unsigned int
j=0;j<min_ans_v.size();j++)
{
node* n=min_ans_v[j];
cout<<"Exploring
:"<<n->data<<endl;
int
final_cost=INT_MAX;
if(n->v.size()==0)
{
n->mark=true;
}
else{
for(unsigned int
i=0;i<n->v.size();i++)
{

vector<node*>*ans=(n->v)[i];
vector<node*>
ans_v=*ans;
int temp_cost=0;
for(unsigned int
j=0;j<(ans_v.size());j++)
{
node*
n=ans_v[j];
temp_cost+=n-
>data;

temp_cost+=edge_cost;
}

if(temp_cost<final_cost)

Department of CSE, Page 14

 Machine Learning Laboratory 18CSL76
{

final_cost=temp_cost;
}
}
n- >data=final_cost;
n->mark=true;
}
cout<<"Marked : "<<n-
>data<<endl;
}

for(int i=0;i<20;i++)
cout<<"=";
cout<<endl;
while(!st.empty())
{
node* n=st.top();
cout<<n->data<<" ";
st.pop();
int
final_cost=INT_MAX;
for(unsigned int
i=0;i<n->v.size();i++)
{

vector<node*>*ans=(n->v)[i];
vector<node*>
ans_v=*ans;
int temp_cost=0;
for(unsigned int
j=0;j<(ans_v.size());j++)
{
node*
n=ans_v[j];
temp_cost+=n-
>data;

temp_cost+=edge_cost;
}

if(temp_cost<final_cost)
{
min_ans=ans;

final_cost=temp_cost;
}
}

Department of CSE, Page 15

 Machine Learning Laboratory 18CSL76
n->data=final_cost;
}
cout<<endl;
next_node=root;

}
}
void print(node* root)
{
if(root)
{
cout<<root->data<<" ";
vector<vector<node*>*
>vec=root->v;
for(unsigned int
i=0;i<(root->v).size();i++)
{
vector<node*>*
ans=(root->v)[i];
vector<node*>
ans_v=*ans;
for(unsigned int
j=0;j<ans_v.size();j++)
{
node* n=ans_v[j];
print(n);
}
}
}
return;
}

int main()
{
node* root=new node;
root->solved=false;
root->mark=false;
insert(root);cout<<endl;
cout<<"Enter the edge cost:
"<<endl;cin>>edge_cost;cout<<endl;
cout<<"The tree is as follows
:"<<endl;
print(root);
cout<<endl;
aostar(root);
cout<<"The minimum cost is :
"<<root->data<<endl;
return 0;

Department of CSE, Page 16

 Machine Learning Laboratory 18CSL76
}

Improvement in the Lab Manual is as Follows:

 Implementation of A* Algorithm using Python.

 Implementation of A* Algorithm with the different example.
 Implementation of A* Algorithm using C++ Program.

Department of CSE, Page 17