Artificial Intelligence BAD402

MODULE – 5
1) Instance-Based Learning: INTRODUCTION

 The learning in Machine learning can be either Generalized Learning (or) Memorized Learning.

 Instance-based learning comes under the category of Memorized Learning and hence it is also
known as Lazy Learning.

 In Generalized Learning, we use Training sets and store them, have labels to identify each features.

 Whereas in Instance-Based learning, we don‟t store anything neither any labels to identify the
instances.

 Since in Instance-Learning we don‟t store any information, we need to identify the instances as and
when they arrive.

 For new instances, it will delay the classification at runtime.

 Instance-Based Learning is done through three approaches as follows;

Instance Based
Learning

Lazy Learners(NN) Nearest Radial Basis Function (RBF) Case Based Reasoning (CBR)
Neighbor method) Weighted methods

 In instance-based learning, when we are given Large Set of Instances, and whenever each new
instances arrives, we will locally assign Target Function for each instances.

2) k-Nearest Neighbor Learning

 K-Nearest Neighbour is one of the simplest Machine Learning algorithms based on Supervised
Learning technique.

 K-NN is a non-parametric algorithm, which means it does not make any assumption on
underlying data.

 It is also called a lazy learner algorithm because it does not learn from the training set
immediately instead it stores the dataset and at the time of classification, it performs an action on
the dataset.
Dept. Of CSE in AIML Page 1 | 14
VVIT,B’lore
 Artificial Intelligence BAD402

 K-NN algorithm assumes the similarity between the new case/data and available cases and put the
new case into the category that is most similar to the available categories.

 Example: Suppose, we have an image of a creature that looks similar to cat and dog, but we want to
know either it is a cat or dog. So for this identification, we can use the KNN algorithm, as it works
on a similarity measure. Our KNN model will find the similar features of the new data set to the cats
and dogs images and based on the most similar features it will put it in either cat or dog category.

 The reason why we say KNN is a Lazy Learning algorithm is because it only tries to start the
process by memorizing and not by learning itself.

 It classifies the new points based on a Similarity Measure known as Euclidean distance.

 Suppose we have a new data point and we need to put it in the required category. Consider the
below image:

 Firstly, we will choose the number of neighbors, so we will choose the k=5.

Dept. Of CSE in AIML Page 2 | 14

VVIT,B’lore
 Artificial Intelligence BAD402

 Next, we will calculate the Euclidean distance between the data points. The Euclidean distance is
the distance between two points. It can be calculated as:

 By calculating the Euclidean distance we got the nearest neighbors, as three nearest neighbors in
category A and two nearest neighbors in category B. Consider the below image:

 As we can see the 3 nearest neighbors are from category A, hence this new data point must belong
to category A.

 The value of „k‟ must be chosen always as a ODD number, otherwise there will be a tie during the
classification.

Dept. Of CSE in AIML Page 3 | 14

VVIT,B’lore
 Artificial Intelligence BAD402

 The K-NN working can be explained on the basis of the below

algorithm: Step-1: Select the number K of the neighbors

Step-2: Calculate the Euclidean distance of K number of neighbors

Step-3: Take the K nearest neighbors as per the calculated Euclidean distance.

Step-4: Among these k neighbors, count the number of the data points in each category.

Step-5: Assign the new data points to that category for which the number of the neighbor is

maximum. Step-6: Our model is ready.

 Advantages of KNN Algorithm:

 It is simple to implement.
 It is robust to the noisy training data
 It can be more effective if the training data is large.

 Disadvantages of KNN Algorithm:

 Always needs to determine the value of K which may be complex some time.
 The computation cost is high because of calculating the distance between the data points for
all the training samples.

3) Locally weighted regression (LWR)

 Regression: is a statistical tool used to understand and quantify the relation between two or more
variables.

Linear Regression

Linear regression will work only for a Linearly Separable Data.

For non-linearly separable data, we will use the Locally Weighted Regression(LWR).

Dept. Of CSE in AIML Page 4 | 14

VVIT,B’lore
 Artificial Intelligence BAD402

 The nearest neighbor approach can be thought of as approximating the target function, whereas
Locally weighted regression is a Generalization of this approach.

 Locally Weighted Regression tries to overcome the problem faced in a Linear Regression method
by assigning weights to the training data.

 LWR requires the entire data set every time you try to make a prediction.

 It makes much more computationally expensive compared to the simple linear regression.

 Assigning a weight is not a simple process, but we should use some algorithm or methods in
assigning the weight.

 Kernel Smoothing is a method which is used to assign the weights to the training data.

Here, means the dot product of X – X0

Here, means X-Transpose.

Diagonal Matrix means, all the diagonal elements will have some values, whereas in other places it
will be Zero.

Dept. Of CSE in AIML Page 5 | 14

VVIT,B’lore
 Artificial Intelligence BAD402

4) Radial Basis Function (RBF)

 Used in ANN(Artificial Neural Network).

 Has only one Hidden node.

 Example:

So, the above example is a non-linearly separable data. Hence we need to first convert this Non-Linearly
separable into a Linearly Separable data.

There are 2 steps to do this conversion which is as follows;

1) Increase the dimensionality (2D-3D)

- But this step is not mandatory, only based on requirement.

2) Expand the direction (Horizontal)

Compress the direction (Vertical)

Dept. Of CSE in AIML Page 6 | 14

VVIT,B’lore
 Artificial Intelligence BAD402

 Now, in RBF we need to first consider one center point randomly.

 Then draw concentric circles from that same center point.

So, from here we need to calculate the radius „r‟, which is the distance from the center point till the star
that touched on the circle in this example.

This radius „r‟ is needed to use it for the functions used for Expanding and compressing as follows;

From the above 3 functions, we can use any ONE of the function.

5) Case Based Reasoning (CBR)

 In Instance based learning , there are basically three properties;

i) Lazy learners
ii) Classification is different for each instance
iii) Instances are represented using Euclidean space

Dept. Of CSE in AIML Page 7 | 14

VVIT,B’lore
 Artificial Intelligence BAD402

 In CBR, everything is considered as case, and based on previous cases we propose a solution.

 Instances are represented as symbols (not values).

 CBR has three components;

i) Similarity functions (or) distance measures
ii) Approximation / adjustment of instances
iii) Symbolic representation of instances

 For modeling CBR, we use CADET system (Case based Design Tool).

 CADET has 75 predefined libraries and using these libraries we will be generating any new
situations/cases/solutions.

 Example: Modern Water Taps

Q-> is the waterflow control

T-> is the Temperature control

This is a model of a Water Tap in which we can turn the Pipe towards Left side for Hot water, and
turn right side for Cold water, and if we keep in the middle we get a mix of both cold and hot level
water.

 Now, the requirement is, we need to construct another Tap which will control the Temperature &
Waterflow i.e either Hot or Cold water, but not both mixed level. So, for this we will take the help
of the already available system.

Dept. Of CSE in AIML Page 8 | 14

VVIT,B’lore
 Artificial Intelligence BAD402

Ct - > Control of temperature

Cq -> Control of waterflow
Qc -> Cold water
Qh -> Hot water
Tc -> Temperature cold
Tm -> Temperature medium
Th -> Temperature hot

6) Reinforcement Learning : Introduction

 Reinforcement Learning is a part of machine learning.

 Reinforcement learning is the art of optimal decision making process.

 Reinforcement learning is used to solve the problems that are reward based.

 The agent learns by trial and error and tries to get the maximum possible reward by
performing certain actions in the environment.
Dept. Of CSE in AIML Page 9 | 14
VVIT,B’lore
 Artificial Intelligence BAD402

Agent – is the sole decision-maker and learner

Environment – a physical world where an agent learns and decides the actions to be
performed

Action – a list of action which an agent can perform

State – the current situation of the agent in the environment

Reward – For each selected action by agent, the environment gives a reward. It’s
usually a scalar value and nothing but feedback from the environment

Policy – the agent prepares strategy (decision-making) to map situations to actions.

 Instead of one input producing one output, the algorithm produces a variety of outputs and is
trained to select the right one based on certain variables.

 It is a type of machine learning technique where a computer agent learns to perform a task
through repeated trial and error interactions with a dynamic environment.

Reinforcement Learning Workflow

– Create the Environment

– Define the reward
– Create the agent
– Train and validate the agent
– Deploy the policy

7) Q Learning

 The term „Q‟ stands here for “Quality”.

 Q learning is a type of reinforcement learning, which involves and AI agent operating in an

environment with States & Rewards(inputs) and Actions(outputs).

 Q learning involves model-free environments where;

 The AI agent is not seeking to learn about an underlying mathematical model or probability
distribution.

Dept. Of CSE in AIML Page 10 | 14

VVIT,B’lore
 Artificial Intelligence BAD402

 Instead, the AI agent attempts to construct an optimal policy directly by interacting with
the environment.

 Q learning uses a trial-and-error based approach where;

 The AI agent repeatedly tries to solve the problem using varied approaches, and
continuously updates its policy as it learns more and more about its environment.

 Characteristics of Q-Learning models

 All of the fundamental characteristics of reinforcement learning apply to Q-Learning

models:
 An input and output system, rewards, an environment, Markov decision processes, and
training & inferences.

 Q-learning includes two additional characteristics:

1. The number of possible states is finite.

- The AI agent will always be in one of a fixed number of possible situations.

2. The number of possible actions is finite

- The AI agent will always need to choose from among a fixed number of possible
actions.

 What are Q Values ?

 A „Q’ value indicates the quality of a particular action „a‟ in a given state „s‟ [ Q(s , a) ]

 Q values are our current estimates of the sum of future rewards

- That is , Q-values estimate how much additional reward we can accumulate
through all remaining steps in the current episode if the AI agent is in state „s‟ and
takes action „a‟.
- Q-values therefore increases as the agent gets closer and closer to the highest reward.

 Q-Values and the Q-Table Policy

 Q-values are stored in a Q-Table, which has one row for each possible state, and one
column for each possible action.
- An optimal Q-Table contains values that allow the agent to take the best action in any
possible state, thus providing the agent with the optimal path to the highest reward.
- Q-table therefore represents the AI agent‟s policy for acting in the current
environment.

 For example, if we consider the movement in a Maze where there is a Starting point
and End point as shown below;

Dept. Of CSE in AIML Page 11 | 14

VVIT,B’lore
 Artificial Intelligence BAD402

In this, once we start from the “Start”, and our destination is at “End”, in between the
movement we shouldn‟t fall on the “CLIFF”. The possible movements here can be any
of these based on each location that we visit, i.e UP, RIGHT, DOWN, LEFT.

The possible movement in the maze can be represented as shown below;

A possible Q-Table based on each location in this Maze can be represented in a Q-Table as
shown below;

Dept. Of CSE in AIML Page 12 | 14

VVIT,B’lore
 Artificial Intelligence BAD402

 What are Temporal Differences ?

 Temporal Differences (TDs) provide us with a method of calculating how much the Q-
value for the action taken in the previous state should be changed based on what the AI
agent has learned about the Q- values for the current state‟s actions. Previous Q-values are
therefore updated after each step.

 What is the Bellman Equation?

 The Bellman equation tells us what new value to use as the Q-value for the action taken in
the previous state.
- Relies on both the old Q-value for the action taken in the previous state and what
has been learned after moving to the next state.
- Includes a learning rate parameter (α) that defines how quickly Q-values are adjusted.

 How the Q-learning process works?

Dept. Of CSE in AIML Page 13 | 14

VVIT,B’lore
 Artificial Intelligence BAD402

 Inference Mode

 Once the Q-learning model is fully trained, it can be used for inference.

 In inference mode:
- Q-values are no longer updated.
- For any state, that action that the AI agent chooses to take is simply the action with
the largest Q-value.

Dept. Of CSE in AIML Page 14 | 14

VVIT,B’lore