S.D.M.

Jain matt Trust®

A.G.M RURAL COLLEGE OF ENGINEERING AND TECHNOLOGY, VARUR,
HUBLI
DEPARTMENT OF ELECTRONICS AND COMMUNICATION ENGINEERING

NOTES

Subject with code: Intelligent Systems and Machine Learning Algorithms (BEC515A)

Module 1

Prepared By

Prof. ASIF IQBAL MULLA

Department of Electronics and Communication

Syllabus :

Introduction: What is AI? Foundations and History of AI Intelligent Agents: Agents and
environment, Concept of Rationality, The nature of environment, The structure of agents.

Text book 1: Chapter 1- 1.1, 1.2, 1.3 Chapter 2- 2.1, 2.2, 2.3, 2.4

Text Book:

1. Stuart J. Russell and Peter Norvig , Artificial Intelligence, 3rd Edition, Pearson,2015
Intelligent Systems and Machine Learning Algorithms (BEC515A)

1 What is AI? (Refer Text book chapter 1.1)

2 Foundations and History of AI Intelligent Agents (Refer Text book chapter 1.2
&1.3)

3 AGENTS AND ENVIRONMENTS

Environment Sensor Actuator

An agent is anything that can be ENVIRONMENT viewed as perceiving its environment

through sensors andSENSOR acting upon that environment through actuators..This simple idea
is illustrated in Figure 2.1
 A human agent has sensory organs such as eyes, ears, nose, tongue and skin parallel to
the sensors, and other organs such as hands, legs, mouth, for effectors.
 A robotic agent replaces cameras and infrared range finders for the sensors, and various
motors and actuators for effectors.
 A software agent receives keystrokes, file contents, andnetwork packets as sensory inputs
and acts on the environment by displaying on the screen, Writing files, and sending
network packets.

Precept and precept sequence

 We use the term percept to refer to the agent’s perceptual inputs at any given instant. An
agent’s percept sequence is the complete history of everything the agent has ever
perceived.
 In general, an agent’s choice of action at any given instant can depend on the entire
perceptsequence observed to date, but not on anything it hasn’t perceived.

Agent Function

Prof. Asif Iqbal M, Dept of ECE 2

Intelligent Systems and Machine Learning Algorithms (BEC515A)

 Mathematically speaking, we say that an agent’s behavior is described by the AGENT

FUNCTION that maps any given percept sequence to an action.

Agent Program

 We can imagine tabulating the agent function that describes any given agent; for most
agents, this would be a very large table.
 Construct this table by trying out all possible percept sequences and recording which
actions the agent does in response. The table is, of course, an external characterization of
the agent.
 Internally, the agent function for an artificial agent will be implemented by an agent
programThe agent function is an abstract mathematical description; the agent program is
a concrete implementation, running within some physical system.

Example
To illustrate these ideas, we use a very simple example—the vacuum-cleaner worldshown in Figure 2.2.
This particular world has just twolocations: squares A and B. The vacuum agent perceives which square it
is in and whetherthere is dirt in the square. It can choose to move left, move right, suck up the dirt, or
donothing. One very simple agent function is the following: if the current square is dirty, then suck;
otherwise, move to the other square. A partial tabulation of this agent function is shownin Figure 2.3 and
an agent program that implements it appears in Figure 2.8 on page 48.

Prof. Asif Iqbal M, Dept of ECE 3

Intelligent Systems and Machine Learning Algorithms (BEC515A)

4 GOOD BEHAVIOR: THE CONCEPT OF RATIONALITY

RATIONAL AGENT& PERFORMANCE MEASURE

A rational agent is one that does the right thing. what does it mean to do the right thing?
When an agent is plunked down in an environment, it generates a sequence of actions according
to the percepts it receives. This sequence of actions causes theenvironment to go through a
sequence of states. If the sequence is desirable, then the agenthas performed well. This notion of
desirability is captured by a performance measure that evaluates any given sequence of
environment states.
Obviously, there is not one fixed performance measure for all tasks and agents;
typically,a designer will devise one appropriate to the circumstances. For example, the vacuum-
cleaner agent from the preceding section. We might propose to measure performance by the
amount of dirt cleaned up in a single eight-hour shift.
Asa general rule, it is better to design performance measures according to what one
actuallywants in the environment, rather than according to how one thinks the agent should
behave.

RATIONALITY

What is rational at any given time depends on four things:

The performance measure that defines the criterion of success.

The agent’s prior knowledge of the environment.
The actions that the agent can perform.
The agent’s percept sequence to date.

This leads to a definition of a rational agent:

For each possible percept sequence, a rational agent should select an action that is expected to
maximize its performance measure, given the evidence provided by the perceptsequence and
whatever built-in knowledge the agent has.

Consider the simple vacuum-cleaner agent that cleans a square if it is dirty and moves to the
other square if not; this is the agent function tabulated in Figure 2.3. Is this a rational agent?That
depends! First, we need to say what the performance measure is, what is known about the
environment, and what sensors and actuators the agent has. Let us assume the following:

Prof. Asif Iqbal M, Dept of ECE 4

Intelligent Systems and Machine Learning Algorithms (BEC515A)

 The performance measure awards one point for each clean square at each time step, over
a “lifetime” of 1000 time steps.
 The “geography” of the environment is known a priori (Figure 2.2).Clean squares stay
clean and sucking cleans the current square.
 The only available actions are Left , Right, and Suck.
 The agent correctly perceives its location and whether that location contains dirt.

We claim that under these circumstances the agent is indeed rational; its expected performance is
at least as high as any other agent’s.

OMNISCIENCE, LEARNING, AND AUTONOMY

Omniscience
We need to be careful to distinguish between rationality and omniscience. An
omniscientagent knows the actual outcome of its actions and can act accordingly; but
omniscience is impossible in reality.
Consider the following example: I am walking along the Champs Elys´ees one day and I
see an old friend across the street. There is no traffic nearby and I’m not otherwise engaged, so,
being rational, I start to cross the street. Meanwhile, at 33,000 feet, a cargo door falls off a
passing airliner,2 and before I make it to the other side of the street I am flattened. Was I
irrational to cross the street? It is unlikely that my obituary would read “Idiot attempts to cross
street.”
This example shows that rationality is not the same as perfection. Rationality maximizes
expected performance, while perfection maximizes actual performance.
Learning
Our definition requires a rational agent not only to gather information but also to learnas
much as possible from what it perceives. The agent’s initial configuration could reflectsome
prior knowledge of the environment, but as the agent gains experience this may be modified and
augmented. There are extreme cases in which the environment is completelyknown a priori. In
such cases, the agent need not perceive or learn; it simply acts correctly.

Autonomy

To the extent that an agent relies on the prior knowledge of its designer rather than on its
own precepts, we say that the agent lacks autonomy. A rational agent should be autonomous—it
should learn what it can to compensate for partial or incorrect prior knowledge. For example, a
vacuum-cleaning agent that learns to foresee where and when additional dirt will appear will do
better than one that does not. As a practical matter, one seldom requires complete autonomy
from the start: when the agent has had little or no experience, it would have to act randomly
unless the designer gave some assistance.

5 THE NATURE OF ENVIRONMENTS

Prof. Asif Iqbal M, Dept of ECE 5

Intelligent Systems and Machine Learning Algorithms (BEC515A)

Specifying the task environment

In our discussion of the rationality of the simple vacuum-cleaner agent, we had to specify
the performance measure, the environment, and the agent’s actuators and sensors. We group all
these under the heading of the task environment. For the acronymically minded, we call this the
PEAS (Performance, Environment, Actuators, Sensors) description.
let us consider a more complex problem: an automated taxi driver. Figure 2.4 summarizes
the PEAS description for the taxi’s task environment. We discuss each element in more detail in
the following paragraphs.

Prof. Asif Iqbal M, Dept of ECE 6

Intelligent Systems and Machine Learning Algorithms (BEC515A)

Properties of task environments

The range of task environments that might arise in AI is obviously vast. Lets discuss some task
environments as follows
Fully observable vs. partially observable: If an agent’s sensors give it access to the complete
state of the environment at each point in time, then we say that the task environment is fully
observable. A task environment is effectively fully observable if the sensors detect all aspects
that are relevant to the choice of action.
An environment might be partially observable because of noisy and inaccurate sensors or
because parts of the state are simply missing from the sensor data—for example, a vacuum agent
with only a local dirt sensor cannot tell whether there is dirt in other squares, and an automated
taxi cannot see what other drivers are thinking.

Single agent vs. multi agent: The distinction between single-agent and multi agent
environments may seem simple enough. For example, an agent solving a crossword puzzle by
itself is clearly in a single-agent environment, whereas an agent playing chess is in a two agent
environment. Thus, chess is a competitive multi agent environment. In the taxi-driving
environment, on the other hand, avoiding collisions maximizes the performance measure of all
agents, so it is a partially cooperative multi agent environment.

Deterministic vs. stochastic. If the next state of the environment is completely determined by
the current state and the action executed by the agent, then we say the environment is
deterministic; otherwise, it is stochastic. In principle, an agent need not worry about uncertainty
in a fully observable, deterministic environment.

If the environment is partially observable, however, then it could appear to be stochastic. Most
real situations are so complex that it is impossible to keep track of all the unobserved aspects; for
practical purposes, they must be treated as stochastic.
Taxi driving is clearly stochastic in this sense. The vacuum world as we described it is
deterministic

Episodic vs. sequential: In an episodic task environment, the agent’s experience is divided into
atomic episodes. In each episode the agent receives a percept and then performs a single action.
Crucially, the next episode does not depend on the actions taken in previous episodes. Many
classification tasks are episodic. For example, an agent that has to spot defective parts on an
assembly line bases each decision on the current part, regardless of previous decisions.

In sequential environments, on the other hand, the current decision could affect all future
decisions. Chess and taxi driving are sequential: in both cases, short-term actions can have long-
term consequences.

Static vs. dynamic: If the environment can change while an agent is deliberating, then we say
the environment is dynamic for that agent; otherwise, it is static. Static environments are easy to
deal with because the agent need not keep looking at the world while it is deciding on an action,
nor need it worry about the passage of time. Dynamic environments, on the other hand, are
continuously asking the agent what it wants to do; if it hasn’t decided yet, that counts as deciding

Prof. Asif Iqbal M, Dept of ECE 7

Intelligent Systems and Machine Learning Algorithms (BEC515A)

to do nothing. If the environment itself does not change with the passage of time but the agent’s
performance score does, then we say the environment is
Semi dynamic.
Taxi driving is clearly dynamic, Chess, when played with a clock, is semi dynamic. Crossword
puzzles are static.

Discrete vs. continuous: The discrete/continuous distinction applies to the state of the
environment, to the way time is handled, and to the percepts and actions of the agent. For
example, the chess environment has a finite number of distinct states (excluding the clock).
Chess also has a discrete set of percepts and actions. Taxi driving is a continuous-state and
continuous-time problem: the speed and location of the taxi and of the other vehicles sweep
through a range of continuous values and do so smoothly over time. Input from digital cameras
is discrete, strictly speaking, but is typically treated as representing continuously varying
intensities and locations.

Known vs. unknown: Strictly speaking, this distinction refers not to the environment itself but
to the agent’s (or designer’s) state of knowledge about the “laws of physics” of the environment.
In a known environment, the outcomes for all actions are given. Obviously, if the environment is
unknown, the agent will have to learn how it works in order to make good decisions.
Figure 2.6 lists the properties of a number of familiar environments.

6 THE STRUCTURE OF AGENTS

The job of AI is to design an agent program that implements the agent function— the mapping
from percepts to actions. We assume this program will run on some sort of computing device
with physical sensors and actuators—we call this the architecture:

agent = architecture + program

Prof. Asif Iqbal M, Dept of ECE 8

Intelligent Systems and Machine Learning Algorithms (BEC515A)

Obviously, the program we choose has to be one that is appropriate for the architecture. If the
program is going to recommend actions like Walk, the architecture had better have legs. The
architecture might be just an ordinary PC, or it might be a robotic car with several onboard
computers, cameras, and other sensors.

Agent programs

Agent programs take the current percept as input from the sensors and return an action to the
actuators. Notice the difference between the agent program, which takes the current percept as
input, and the agent function, which takes the entire percept history. The agent program takes
just the current percept as input because nothing more is available from the environment; if the
agent’s actions need to depend on the entire percept sequence, the agent will have to remember
the percepts.
Figure 2.7 shows a rather trivial agent program written in pseudo code that keeps track of the
percept sequence and then uses it to index into a table of actions to decide what to do.

Prof. Asif Iqbal M, Dept of ECE 9

Intelligent Systems and Machine Learning Algorithms (BEC515A)

Example

Prof. Asif Iqbal M, Dept of ECE 10

Intelligent Systems and Machine Learning Algorithms (BEC515A)

Prof. Asif Iqbal M, Dept of ECE 11

Intelligent Systems and Machine Learning Algorithms (BEC515A)

Prof. Asif Iqbal M, Dept of ECE 12

Intelligent Systems and Machine Learning Algorithms (BEC515A)

Types of Agent Programs

The four basic kinds of agent programs that embody the principles underlying almost all
intelligent systems:
• Simple reflex agents;
• Model-based reflex agents;
• Goal-based agents; and
• Utility-based agents.

Simple reflex agents

The simplest kind of agent is the simple reflex agent. These agents select actions on the
basis of the current percept, ignoring the rest of the percept history. For example, the vacuum
agent whose agent function is tabulated in Figure 2.3 is a simple reflex agent, because its
decision is based only on the current location and on whether that location contains dirt. An
agent program for this agent is shown in Figure 2.8.

Prof. Asif Iqbal M, Dept of ECE 13

Intelligent Systems and Machine Learning Algorithms (BEC515A)

Notice that the vacuum agent program is very small indeed compared to the
corresponding table. The most obvious reduction comes from ignoring the percept history, which
cuts down the number of possibilities from 4T to just 4. A further, small reduction comes from
the fact that when the current square is dirty, the action does not depend on the location.
Imagine yourself as the driver of the automated taxi. If the car in front brakes and its
brake lights come on, then you should notice this and initiate braking. Then, this triggers some
established connection in the agent program to the action “initiate braking.” We call such a
connection a condition–action rule,5 written as

if car-in-front-is-braking then initiate-braking.

Figure 2.9 gives the structure of this general program in schematic form, showing how
the condition–action rules allow the agent to make the connection from percept to action.
The agent program, which is also very simple, is shown in Figure 2.10. The INTERPRET
INPUT function generates an abstracted description of the current state from the percept, and the
RULE-MATCH function returns the first rule in the set of rules that matches the given state
description. The agent in Figure 2.10 will work only if the correct decision can be made on the
basis of only the current percept—that is, only if the environment is fully observable.

Prof. Asif Iqbal M, Dept of ECE 14

Intelligent Systems and Machine Learning Algorithms (BEC515A)

Model-based reflex agents

The most effective way to handle partial observability is for the agent to keep track of the
part of the world it can’t see now. That is, the agent should maintain some sort of internal
INTERNAL STATE state that depends on the percept history and thereby reflects at least some
of the unobserved aspects of the current state. For the braking problem, the internal state is not
too extensive—just the previous frame from the camera, allowing the agent to detect when two
red lights at the edge of the vehicle go on or off simultaneously. For other driving tasks such as
changing lanes, the agent needs to keep track of where the other cars are if it can’t see them all at
once. And for any driving to be possible at all, the agent needs to keep track of where its keys
are.
Updating this internal state information as time goes by requires two kinds of knowledge
to be encoded in the agent program. First, we need some information about how the world
evolves independently of the agent—for example, that an overtaking car generally will be closer
behind than it was a moment ago. Second, we need some information about how the agent’s own
actions affect the world—for example, that when the agent turns the steering wheel clockwise,
the car turns to the right. This knowledge about “how the world works”—whether implemented
in simple Boolean circuits or in complete scientific theories—is called a model of the world. An
agent that uses such a MODEL-BASED model is called a model-based agent.
Figure 2.11 gives the structure of the model-based reflex agent with internal state,
showing how the current percept is combined with the old internal state to generate the updated
description of the current state, based on the agent’s model of how the world works. The agent
program is shown in Figure 2.12.

Prof. Asif Iqbal M, Dept of ECE 15

Intelligent Systems and Machine Learning Algorithms (BEC515A)

Goal-based agents

Knowing something about the current state of the environment is not always enough to
decide what to do. For example, at a road junction, the taxi can turn left, turn right, or go straight
on. The correct decision depends on where the taxi is trying to get to. In other words, as well as a
current state description, the agent needs some sort of goal information that describes situations
that are desirable—for example, being at the passenger’s destination. The agent program can
combine this with the model to choose actions that achieve the goal. Figure 2.13 shows the goal-
based agent’s structure.
Sometimes goal-based action selection is straightforward—for example, when goal
satisfaction results immediately from a single action. Sometimes it will be more tricky—for
example, when the agent has to consider long sequences of twists and turns in order to find a
way to achieve the goal. Search and planning are the subfields of AI devoted to finding action
sequences that achieve the agent’s goals.

Utility-based agents

Goals alone are not enough to generate high-quality behavior in most environments. For
example, man action sequences will get the taxi to its destination (thereby achieving the goal) but
some are quicker, safer, more reliable, or cheaper than others. Goals just provide a crude binary
distinction between “happy” and “unhappy” states. A more general performance\ measure should
allow a comparison of different world states according to exactly how happy they would make
the agent. Because “happy” does not sound very scientific, economists and computer scientists
use the term utility instead. An agent’s utility function is essentially an internalization of the
performance measure. If the internal utility function and the external performance measure are in
agreement, then an agent that chooses actions to maximize its utility will be rational according to
the external performance measure.
Furthermore, in two kinds of cases, goals are inadequate but a utility-based agent can still
make rational decisions. First, when there are conflicting goals, only some of which can be
achieved (for example, speed and safety), the utility function specifies the appropriate tradeoff.

Prof. Asif Iqbal M, Dept of ECE 16

Intelligent Systems and Machine Learning Algorithms (BEC515A)

Second, when there are several goals that the agent can aim for, none of which can be achieved
with certainty, utility provides a way in which the likelihood of success can be weighed against
the importance of the goals.
The utility-based agent structure appears in Figure 2.14. Utility-based agent programs
appear in Part IV, where we design decision-making agents that must handle the uncertainty
inherent in stochastic or partially observable environments.

Learning agents

In his famous early paper, Turing (1950) considers the idea of actually programming his
intelligent machines by hand. The method he proposes is to build learning machines and then to
teach them. In many areas of AI, this is now the preferred method for creating state-of-the-art
systems. Learning has another advantage, as we noted earlier: it allows the agent to operate in
initially unknown environments and to become more competent than its initial knowledge alone
might allow.
A learning agent can be divided into four conceptual components, as shown in Figure
2.15. The most important distinction is between the learning element, which is responsible for
making improvements, and the performance element, which is responsible for selecting
external actions.
The performance element is what we have previously considered to be the entire agent: it
takes in precepts and decides on actions. The learning element uses feedback from the critic on
how the agent is doing and determines how the performance element should be modified to do
better in the future. The last component of the learning agent is the problem generator. It is
responsible for suggesting actions that will lead to new and informative experiences.

Prof. Asif Iqbal M, Dept of ECE 17

Intelligent Systems and Machine Learning Algorithms (BEC515A)

How the components of agent programs work

We have described agent programs (in very high-level terms) as consisting of various
components, whose function it is to answer questions such as: “What is the world like now?”
“What action should I do now?” “What do my actions do?” The next question for a student of AI
is, “How on earth do these components work?” It takes about a thousand pages to begin to
answer that question properly, but here we want to draw the reader’s attention to some basic
distinctions among the various ways that the components can represent the environment that the
agent inhabits.
Roughly speaking, we can place the representations along an axis of increasing
complexity and expressive power—atomic, factored, and structured. To illustrate these ideas,
it helps to consider a particular agent component, such as the one that deals with “What my
actions do.” This component describes the changes that might occur in the environment as the
result of taking an action, and Figure 2.16 provides schematic depictions of how those transitions
might be represented.

Prof. Asif Iqbal M, Dept of ECE 18