INTELLIGENT

AGENTS
Agents and
Environments
An agent is anything that can be viewed as perceiving its environment through
sensors and acting upon that environment through actuators
the term percept to refer to the agent’s perceptual inputs at any given instant
An agent’s percept sequence is the complete history of everything the agent has ever perceived
an agent’s choice of action at any given instant can depend on the entire percept sequence
observed to date, but not on anything it hasn’t perceived

an agent’s behavior is described by the agent function that maps any given percept sequence to
an action
[f: P*  A]
The agent program runs on the physical architecture to produce f, the agent function for an
artificial agent will be implemented by an agent program
agent = architecture + program
The agent function is an abstract mathematical description; the agent program is a concrete
implementation, running within some physical system.
The Vacuum Cleaner
World
This particular world has just two locations: squares A and B.
The vacuum agent perceives which square it is in and whether there is dirt in the square.
It can choose to move left, move right, suck up the dirt, or do nothing.
One very simple agent function is the following: if the current square is dirty, then suck;
otherwise, move to the other square.
Percepts: location and contents, e.g., [A,Dirty]
Actions: Left, Right, Suck, NoOp
Agent’s function  look-up table
For many agents this is a very large table
 Good Behaviour: Concept of
•Rationality
A rational agent is one that does the right thing
•what does it mean to do the right thing?
 by considering the consequences of the agent’s behavior
When an agent is plunked down in an environment, it generates a sequence of
actions according to the percepts it receives. This sequence of actions causes the
environment to go through a sequence of states. If the sequence is desirable,
then the agent has performed well.
This notion of desirability is captured by a performance measure that evaluates
any given sequence of environment states.
Example- Vacuum Cleaner
Revisited
We might propose to measure performance by the amount of dirt cleaned up in a single
eight-hour shift.
With a rational agent, of course, what you ask for is what you get.
A rational agent can maximize this performance measure by cleaning up the dirt, then
dumping it all on the floor, then cleaning it up again, and so on.
A more suitable performance measure would reward the agent for having a clean
floor.
For example, one point could be awarded for each clean square at each time step
(perhaps with a penalty for electricity consumed and noise generated).
As a general rule, it is better to design performance measures according to what one
actually wants in the environment, rather than according to how one thinks the agent
should behave.
Performance measure: An objective criterion for success of an agent's behavior.

Performance measures of a vacuum-cleaner agent: amount of dirt cleaned up,

amount of time taken, amount of electricity consumed, level of noise generated, etc.

Performance measures self-driving car: time to reach destination (minimize),

safety, predictability of behavior for other agents, reliability, etc.

Performance measure of game-playing agent: win/loss percentage (maximize),

robustness, unpredictability (to “confuse” opponent), etc.
 Rationalit
y– Performance measuring success
•What is rational at any given time depends on four things:

– Agents prior knowledge of environment

– Actions that agent can perform
– Agent’s percept sequence to date

•Rational Agent: For each possible percept sequence, a rational

agent should select an action that is expected to maximize its
performance measure, given the evidence provided by the
percept sequence and whatever built-in knowledge the agent
has.
 Example- Vacuum
Cleaner
• The performance measure awards one point for each clean square at
each time step
• The “geography” of the environment is known a priori but the dirt
distribution and the initial location of the agent are not. Clean squares
stay clean and sucking cleans the current square. The Left and Right
actions move the agent left and right except when this would take the
agent outside the environment, in which case the agent remains where
it is.
• The only available actions are Left, Right, and Suck.
• The agent correctly perceives its location and whether that
location contains dirt
Rational
Agent
For each possible percept sequence, a rational agent should select an action that
is expected to maximize its performance measure, given the evidence provided
by the percept sequence and whatever built-in knowledge the agent has.
Omniscience, learning, and
autonomy
Omniscient agent: knows the actual outcome of its actions and can act accordingly;
but omniscience is
impossible in reality.
rationality is not the same as perfection.
Rationality maximizes expected performance, while perfection maximizes actual
performance
information gathering
Exploration
learn as much as
possible from what it
perceives
rational agent should be autonomous—it should learn what it can to compensate for
partial or incorrect prior knowledge
 Task Environment
PEAS
PEAS (Performance,
Actuators, Sensors)
Environment,

In designing an agent, the first step must always

be to
specify the task environment as fully as possible
Example: the task of designing a self-
driving car measure Safe, fast, legal, comfortable trip
◦ Performance
◦ Environment Roads, other traffic, pedestrians
◦ Actuators Steering wheel, accelerator, brake,
signal, horn
◦ Sensors Cameras, LIDAR (light/radar), speedometer,
GPS, odometer
engine sensors, keyboard
 Task Environment
•Types
Fully observable (vs. partially observable)
•Single agent(vs. Multi agent)
•Deterministic (vs. stochastic)
•Episodic (vs. sequential)
•Static (vs. dynamic)
•Discrete (vs. continuous)
•Known (vs. unknown)
Fully observable vs. partially
observable
If an agent’s sensors give it access to the complete state of the environment at
each point in time, then we say that the task environment is fully observable.
A task environment is effectively fully observable if the sensors detect all
aspects that are relevant to the choice of action; relevance, in turn, depends on
the performance measure.
Fully observable environments are convenient because the agent need
not maintain any internal state to keep track of the world.
An environment might be partially observable because of noisy and
inaccurate sensors or because parts of the state are simply missing from the
sensor data
If the agent has no sensors at all then the environment is unobservable
Single agent vs.
multiagent
If only one agent is involved in an environment, and operating by itself then such an
environment is called single agent environment.
However, if multiple agents are operating in an environment, then such an
environment is called
a multi-agent environment.
chess is a competitive multiagent environment.
In the taxi-driving environment avoiding collisions maximizes the performance
measure of all
agents, so it is a partially cooperative multiagent environment.
◦It is also partially competitive because, for example, only one car can occupy a
parking space.
Deterministic vs.
stochastic
If the next state of the environment is completely determined by the current state and the action executed
by the agent, then we say the environment is deterministic; otherwise, it is stochastic.

an agent need not worry about uncertainty in a fully observable, deterministic

environment. If the environment is partially observable, however, then it could appear to
be stochastic.
environment
“stochastic”isgenerally
uncertainimplies
if it is not
thatfully observable
uncertainty or not
about deterministic.
outcomes is quantified in terms of probabilities
a nondeterministic environment is one in which actions are characterized by their possible outcomes,
but no probabilities are attached to them
Episodic vs.
sequential
Episodic task environment:
◦the agent’s experience is divided into atomic episodes.
◦In each episode the agent receives a percept and then performs a single action.
◦Crucially, the next episode does not depend on the actions taken in previous
episodes.
◦Many classification tasks are episodic.
Sequential environments
◦the current decision could affect all future decisions
◦Chess and taxi driving are sequential: in both cases, short-term actions can have long-
term consequences.
◦Episodic environments are much simpler than sequential environments because the
agent does not need to think ahead.
Static vs.
dynamic
If the environment can change while an agent is deliberating, then we say the environment
is dynamic for that agent; otherwise, it is static
A static environment does not change while the agent is thinking.
The passage of time as an agent deliberates is irrelevant.
Dynamic environments, on the other hand, are continuously asking the agent what it wants
to do;
if it hasn’t decided yet, that counts as deciding to do nothing.
If the environment itself does not change with the passage of time but the agent’s
performance
Chess, when played with a clock, is semi-dynamic.
score does, then we say the environment is semi-dynamic.
Crossword puzzles are static
Discrete vs.
continuous
If the number of distinct percepts and actions is limited, the environment is discrete, otherwise
it is continuous.
The chess environment has a finite number of distinct states (excluding the clock).
Chess also has a discrete set of percepts and actions.
Taxi driving is a continuous-state and continuous-time problem: the speed and location of the taxi and of the
other vehicles sweep through a range of continuous values and do so smoothly over time
Taxi-driving actions are also continuous (steering angles, etc.).
Input from digital cameras is discrete, strictly speaking, but is typically treated as representing
continuously varying intensities and locations.
Known vs.
unknown
In a known environment, the outcomes (or outcome probabilities if the environment is
stochastic) for all actions are given.
If the environment is unknown, the agent will have to learn how it works in order to make good
decisions
a known environment can be partially observable
◦ for example, in solitaire card games, I know the rules but am still unable to see the cards
that have not yet been turned over.
An unknown environment can be fully observable
◦ in a new video game, the screen may show the entire game state but I still don’t know
what
the buttons do until I try them.
Structure of
Agent
The job of AI is to design an agent program that implements the agent
function the mapping from percepts to actions.
his program will run on some sort of computing device with physical sensors and
actuators called the architecture
agent = architecture + program
architecture makes the percepts from the sensors available to the program, runs
the program, and feeds the program’s action choices to the actuators as they
are generated
Agent
programs
Agent program: use current percept as input from the sensors and return an action to the actuators
Agent function: takes the entire percept history

To build a rational agent in this way, we as designers must construct a table that contains the appropriate action for
every possible percept sequence.
Let P be the set of possible percepts and let T be the lifetime of the agent (the total number of percepts
it will receive)
The lookup table will contain entries
Consider the automated taxi: the visual input from a single camera comes in at the rate of roughly 27
megabytes per second (30 frames per second, 640 × 480 pixels with 24 bits of color information). This
gives a lookup table with over 10250,000,000,000 entries for an hour’s driving.
Even the lookup table for chess a tiny, well-behaved fragment of the real world would have at least
10150 entries.
The daunting size of these tables (the number of atoms in the observable universe is less than 1080)
means that
a) no physical agent in this universe will have the space to store the table,
b) the designer would not have time to create the table,
c) no agent could ever learn all the right table entries from its experience, and
d) even if the environment is simple enough to yield a feasible table size, the designer still has no
guidance about how to fill in the table entries.
Types of Agent
Programs
Four basic kinds of agent programs that embody the principles underlying almost
all intelligent systems:
1. Simple reflex agents;
2. Model-based reflex agents;
3. Goal-based agents; and
4. Utility-based agents
 Simple reflex
agents
Select actions on the basis of the current percept, ignoring the rest of the percept
history
Agents do not have memory of past world states or percepts.
So, actions depend solely on current percept.
Action becomes a “reflex.”

Uses condition-action rules.

 If tail-light of car in front is
red, then brake.

condition–action rule
if car-in-front-is-braking then initiate-braking
The INTERPRET-INPUT function generates an abstracted description of the
current state from the percept, and
the RULE-MATCH function returns the fifirst rule in the set of rules that
matches
the given state description. Note that the description in terms of “rules”
and “matching” is purely conceptual;
actual implementations can be as simple as a collection of logic gates
implementing a Boolean circuit
This will work only if the correct decision can be made on the basis of only the
current percept—that is, only if the environment is fully observable.
Even a little bit of unobservability can cause serious trouble. For example, the
braking rule given earlier assumes that the condition car-in-front-is-braking can
be determined from the current percept—a single frame of video.
This works if the car in front has a centrally mounted brake light.
Infinite loops are often unavoidable for simple reflex agents operating in
partially observable environments
Escape from infifinite loops is possible if the agent can randomize its
actions.
Model-based reflex agents
Key difference (wrt simple reflex agents):

◦Agents have internal state, which is used to keep track of past states of the world.

◦ Agents have the ability to represent change in the World.

 represents the
agent’s “best
guess”

“Infers potentially
dangerous driver
in front.”
If “dangerous driver in front,”
then “keep distance.”
internal state information as time goes by requires two kinds of knowledge to be encoded in the
agent program
1. we need some information about how the world evolves independently of the agent
2. we need some information about how the agent’s own actions affect the world

knowledge about “how the world works is called a model of the world. An agent that uses such a model
is called a model-based agent.
UPDATE-STATE, which is responsible for creating the new internal state description.
Goal-based agents
Key difference wrt Model-Based Agents:
In addition to state information, have goal information that
describes desirable situations to be achieved.

Search and planning are the subfields of AI devoted to finding action sequences that achieve the
agent’s goals

Agents of this kind take future events into consideration.

What sequence of actions can I take to achieve certain goals?

Choose actions so as to (eventually) achieve a (given or computed) goal.

Utility-based
agents
Goals alone are not enough to generate high-quality behavior in most
environments
Goals just provide a crude binary distinction between “happy” and
“unhappy”
states
Because “happy” does not sound very scientific, economists and
computer
scientists use the term utility instead
An agent’s utility function is essentially an internalization of the performance
measure. If the internal utility function and the external performance measure
are in agreement, then an agent that chooses actions to maximize its utility will
be rational according to the external performance measure.
Learning
agents
A learning agent can be divided into four conceptual components
1. learning element, which is responsible for making improvements
2. performance element, which is responsible for selecting external actions. The performance
element is what we have previously considered to be the entire agent: it takes in percepts
and decides on actions.
3. Feedback from the critic : on how the agent is doing and determines how the performance
element should be modified to do better in the future
4. problem generator. It is responsible for suggesting actions that will lead to new
and informative experiences