Agents and Environments

Agents and Environments

• An Agent is
Agent
anything that can
Sensors Percepts
be viewed as

Environment
perceiving its
environment ?
through sensors
and acting upon
that environment Actuators Actions

through actuators

AI: Chapter 2: Intelligent 2

Agents
 Agents and Environments
• The environment
Agent
is everything
outside the agent Sensors Percepts

Environment
that the agent
interacts with. ?
• It includes all the
conditions, objects,
and information Actuators Actions

that the agent

observes and acts
upon.
AI: Chapter 2: Intelligent 3
Agents
Agents and Environments

AI: Chapter 2: Intelligent 4

Agents
 Simple Terms

Percept
 Agent’s perceptual inputs at any given
instant

Percept sequence
 Complete history of everything that the
agent has ever perceived.
Agent function & program
Agent’s behavior is mathematically
described by
 Agent function
 A function mapping any given percept

sequence to an action
– f : P* -> A

Practically it is described by
 An agent program
 The real implementation
 Agents and Environments
• Example: Vacuum
Cleaner World
– Two locations: squares A
and B A B
– Perceives what square it is
in
– Perceives if there is dirt in
the current square
– Actions
• move left
• move right
• suck up the dirt
• do nothing
AI: Chapter 2: Intelligent 7
Agents
 Agents and Environments
• Agent Function: Percept Sequence Action

Vacuum Cleaner [A, Clean] Right

World
[A, Dirty] Suck
– If the current
square is dirty, then [B, Clean] Left
suck, otherwise
move to the other [B, Dirty] Suck
square [A, Clean], [A, Clean] Right

[A, Clean], [A, Dirty] Suck

AI: Chapter 2: Intelligent 8

Agents
AI: Chapter 2: Intelligent 9
Agents
 Agents and Environments
• But what is the right way to fill out the
table?
– is the agent
• good or bad
• intelligent or stupid
– can it be implemented in a small program?

Function Reflex-Vacuum-Agent([location, status]) return an action

if status == Dirty then return Suck
else if location = A then return Right
else if location = B then return Left

AI: Chapter 2: Intelligent 10

Agents
 Goal of agents
• High performance- performance
should be maximized.
• Optimal result- correct and short
procedure
• Rational Action- right action should
be performed.

AI: Chapter 2: Intelligent 11

Agents
Concept of Rationality
 Concept of Rationality
Rational agent
 One that does the right thing
 = every entry in the table for the

agent function is correct (rational).

What is correct?
 The actions that cause the agent to
be most successful
 So we need ways to measure

success.
 Performance measure

Performance measure
 An objective function that determines
 How the agent does successfully
 E.g., 90% or 30% ?

An agent, based on its percepts

 action sequence :
if desirable, it is said to be performing
well.
 No universal performance measure

for all agents

 Performance measure
A general rule:
 Design performance measures according to
 What one actually wants in the
environment
 Rather than how one thinks the agent should

behave
E.g., in vacuum-cleaner world
 We want the floor clean, no matter how
the agent behave
 We don’t restrict how the agent behaves
 Rationality

Rationality depends on four things:

 The performance measure defining the
criterion of success
 The agent’s prior knowledge

(environment)
 The actions that the agent can perform

(accurators)
 The agents’s percept sequence (sensor)
 Rational agent
For each possible percept sequence,
 an rational agent should select
 anaction expected to maximize its
performance measure, given the evidence
provided by the percept sequence and
whatever built-in knowledge the agent has
E.g., an exam
 Maximize marks, based on
the questions on the paper & your
knowledge
 Example of a rational
agent
Performance measure
 Awards one point for each clean square
 at each time step

Prior knowledge about the

environment
 The geography of the environment
 Only two squares
 Example of a rational
agent
Actions that can perform
 Left, Right, Suck and NoOp

Percept sequences
 Where is the agent?
 Whether the location contains dirt?

Under this circumstance, the agent

is rational.
 Omniscience

An omniscient agent
 Knows the actual outcome of its
actions in advance
 No other possible outcomes

 However, impossible in real world

An example
 crossing
a street but died of the fallen
cargo door from 33,000ft  irrational?
 Learning
Does a rational agent depend on
only current percept?
 No, the past percept sequence should
also be used
 This is called learning

 After experiencing an episode, the

agent
 should adjust its behaviors to perform
better for the same job next time.
 Autonomy

If an agent just relies on the prior

knowledge of its designer rather than its
own percepts then the agent lacks
autonomy
A rational agent should be autonomous-
it should learn what it can to compensate
for partial or incorrect prior knowledge.
E.g., a clock
 No input (percepts)
 Run its own algorithm (prior knowledge)
 No learning, no experience.
 Good Behavior and
Rationality
• Rational != omniscient
• Rational != successful

• Rational -> exploration, learning,

autonomy

AI: Chapter 2: Intelligent 23

Agents
The nature of environment
 Software Agents
Sometimes, the environment may not
be the real world
 E.g.,flight simulator, video games,
Internet
 They are all artificial but very complex

environments
 Those agents working in these

environments are called

 Software agent (softbots)
 Because all parts of the agent are software
 Rationality

Rationality depends on four things:

 The performance measure defining the
criterion of success
 The agent’s prior knowledge

(environment)
 The actions that the agent can perform

(accurators)
 The agents’s percept sequence (sensor)
 Task environments
Task environments are the problems
 While the rational agents are the solutions

Specifying the task environment

 PEAS description as fully as possible
 Performance
 Environment

 Actuators

 Sensors

In designing an agent, the first step must always be to

specify the task environment as fully as possible.
Use automated taxi driver as an example
 Task environments
Performance measure
 How can we judge the automated driver?

 Which factors are considered?

 gettingto the correct destination
 minimizing fuel consumption

 minimizing the trip time and/or cost

 minimizing the violations of traffic laws

 maximizing the safety and comfort, etc.

 Task environments
Environment
A taxi must deal with a variety of roads

 Traffic
lights, other vehicles,
pedestrians, stray animals, road works,
police cars, etc.

 Interact with the customer

 Task environments
Actuators (for outputs)
 Control over the accelerator, steering,
gear shifting and braking
 A display to communicate with the
customers

Sensors (for inputs)

 Detect other vehicles, road situations
 GPS (Global Positioning System) to
know where the taxi is
 Many more devices are necessary
 Task environments
A sketch of automated taxi driver
Task environments
 Properties of task
environments
• Fully Observable
– In a fully observable environment, the agent has complete access
to all the information relevant to making decisions. The agent can
perceive the entire state of the environment at any given time.
• Example: Chess is fully observable because the
agent can see the entire chessboard and know all the
positions of the pieces.
• Partially Observable
– In a partially observable environment, the agent only has access to
limited information about the environment at any given time. Some
aspects of the environment are hidden or unknown to the agent.
• Example: Poker is partially observable because the
agent (player) cannot see the cards held by other
players.
 Properties of task
environments
Deterministic Environment
In a deterministic environment, the outcome of every action is
predictable. Given the current state and action, the next state can
be determined with certainty.

Example: Chess is a deterministic environment because, given the

current position and the move, the outcome is fully predictable, with
no random factors.

Stochastic Environment
In a stochastic environment, there is uncertainty in the outcome of
actions. Even if the agent performs the same action in the same
state multiple times, the results may differ due to random factors.

Example: A dice game is stochastic because rolling the dice has

probabilistic outcomes, and the next state depends on random
chance.
 Properties of task
environments
Episodic Environment:
In an episodic environment, the agent's experience is divided into
independent episodes. Each episode is a separate task or interaction
that does not depend on the previous one.

Example: Each email is analyzed individually to determine if it is

spam or not. The classification of one email does not affect the
classification of another email..

Sequential Environment:
In a sequential environment, each action taken by the agent affects
future actions. The outcome of one action influences the next state or
scenario the agent faces.

Example: Driving a car is a sequential task. The decision to steer or

accelerate in one moment affects the next state (e.g., the car's
position on the road), and all future decisions depend on past actions.
 Properties of task
environments
Static Environment:
A static environment remains unchanged while the agent
performs its tasks. The environment does not evolve
independently, meaning no external factors alter the
environment except for the agent's actions.
Example:
Chess Game: The chessboard remains the same until a player
makes a move.

Dynamic Environment:
A dynamic environment is constantly changing, even if the
agent is not actively doing anything. The environment evolves
independently of the agent's actions.
Example:
Self-Driving Cars: The environment (roads, traffic,
pedestrians) changes continuously even when the car is not
moving.
 Properties of task
environments
Discrete Environment:
In a discrete environment, there are a finite number of distinct,
clearly defined states. The actions and changes between states
happen in specific, quantifiable steps.
Examples:
Chess or Board Games: The pieces have a set number of
positions, and each move occurs in a defined step-by-step manner.

Continuous Environment:
In a continuous environment, the state space is infinite, and actions
and changes happen smoothly over a continuous range. The
transitions are not step-based but gradual.
Examples:
Self-Driving Cars: The car's position on the road, speed, and
distance to objects can vary continuously.
 Properties of task
environments
Single-Agent Environment:
In a single-agent environment, there is only one agent interacting
with the environment. The agent is responsible for making decisions
and taking actions without competition or cooperation from other
agents.
Examples:
Chess against a computer: The AI operates alone, solving the
problem based on input from the human player.

Multi-Agent Environment:
In a multi-agent environment, multiple agents interact with each
other within the same environment. These interactions can be
competitive, cooperative, or neutral depending on the situation.
Examples:
Self-driving cars on a road: Multiple autonomous cars interacting
with each other.
 Properties of task
environments
• Known Environment:
• In a known environment, the agent has complete information about
the environment's structure, rules, and the consequences of its
actions. It knows the model of the environment and can plan its
actions accordingly.
• Examples:
– Chess: The agent knows all the rules, moves, and states of the board.

• Unknown Environment:
• In an unknown environment, the agent does not have prior
knowledge about how the environment works. It needs to explore,
learn from feedback, and adapt to make decisions.
• Examples:
– Robotics in an unfamiliar environment: The robot might not know
the layout of a room and must navigate by trial and error.
Examples of task
environments
Structure of agents
 Structure of agents
Agent = architecture +
program
 Architecture= some sort of
computing device (sensors +
actuators)
 (Agent) Program = some function that

implements the agent mapping = “?”

 Agent Program = Job of AI
 Agent programs
Input for Agent Program
 Only the current percept

Input for Agent Function

 The entire percept sequence
 The agent must remember all of them

Implement the agent program as

A look up table (agent function)
 Agent programs
• Basic algorithm for a rational agent
– While (true) do
• Get percept from sensors into memory
• Determine best action based on memory
• Record action in memory
• Perform action

• Most AI programs are a variation of this

theme

AI: Chapter 2: Intelligent 44

Agents
 Agent programs
Skeleton design of an agent
program
 Agent Programs
P = the set of possible percepts
T= lifetime of the agent
 The total number of percepts it receives
Size of the look up tablet 1 P
T t

Consider playing chess

P =10, T=150
 Will require a table of at least 10 150

entries
 Agent programs
Despite of huge size, look up table
does what we want.
The key challenge of AI
 Findout how to write programs that, to
the extent possible, produce rational
behavior
 From a small amount of code
 Rather than a large amount of table entries
Types of agent programs
Four types
 Simple reflex agents
 Model-based reflex agents

 Goal-based agents

 Utility-based agents
 Simple reflex agents
They choose action only based on current situation
ignoring history of perception
Perform action only based on simple situation
They work only if environment is fully observable.
The agent is based on condition action rule. ie “if
condition then action”
This can be done using predetermined rules that are
there in knowledge base
Eg. Chess is fully observable because the agent can
see the entire chessboard and know all the positions
of the pieces.
 The Structure of Agents
Simple Reflex Agent

What the world Sensors Percepts

is like now

Environment
Condition-Action What action I
Rules should do now Actuators Actions

AI: Chapter 2: Intelligent 50

Agents
 The Structure of Agents
• Simple Reflex Agent

function Simple-Reflex-Agent (percept) return action

static: rules, a set of condition-action rules

state <- INTERPRET-INPUT( percept )

rule <- RULE-MATCH( state, rules )
action <- RULE-ACTION[ rule ]
return action

AI: Chapter 2: Intelligent 51

Agents
 The Structure of Agents
• Simple Reflex Agent

function Simple-Reflex-Agent (percept) return action

static: rules, a set of condition-action rules

state <- INTERPRET-INPUT( percept )

rule <- RULE-MATCH( state, rules )
action <- RULE-ACTION[ rule ]
return action

AI: Chapter 2: Intelligent 52

Agents
Model-based Reflex Agents
Model based reflex agent is an intelligent agent that uses
percept history and internal memory to make decisions about
the “model ” of the world around it.
The model based agent can work in partially observable
environment and track the situation.
Eg. Poker is partially observable because the agent (player)
cannot see the cards held by other players.
Model- knowledge about “how things happen in the world”
Internal state- it is a representation of unobserved aspect
of the current state depending on percept history.
Updating the state requires the information about
How the world evolves
How the agent action effect the world
Model-based Reflex Agents
Reflex Agent With State

What the world Sensors Percepts

State is like now

Environment
How the world evolves

What my actions do

Condition-Action What action I

Rules should do now Actuators Actions

AI: Chapter 2: Intelligent 54

Agents
Model-based Reflex Agents
• Reflex Agent With State

function Reflex-Agent-With-State (percept) return action

static: state, a description of the current world state
rules, a set of condition-action rules
action, the most recent action, initially none

state <- UPDATE-STATE( state, action, percept )

rule <- RULE-MATCH( state, rules )
action <- RULE-ACTION[ rule ]
return action

AI: Chapter 2: Intelligent 55

Agents
 Goal-based agents
Current state of the environment is always not
enough
The goal is another issue to achieve
 Judgment of rationality / correctness
They choose their actions in order to achieve
goals.
This allows the agent a way to choose among
multiple possibilities selecting the one which
reaches a goal state
They usually require searching and planning
Eg. A GPS finding a path to certain destination.
 Goal-based agents
Goal Based Agent

What the world Sensors Percepts

State is like now

Environment
How the world evolves

What my actions do
What it will be like
if I do action A

What action I
Goals Actuators
should do now Actions

AI: Chapter 2: Intelligent 57

Agents
 Utility-based agents
Utility Based Agent

What the world Sensors Percepts

State is like now

Environment
How the world evolves
What it will be like
What my actions do if I do action A

How happy I will be

Utility
in such a state

What action I
should do now Actuators Actions

AI: Chapter 2: Intelligent 58

Agents
 Utility-based agents
Goals alone are not enough
 to generate high-quality behavior
A utility bsaed agent is an agent that acts based not
only on what the goal is, but the best way to reach that
goal.
They choose actions based on a preference(utility) for
each state.
Eg. A GPS system finding shortest/fastest/safest path
to destination.
Many action sequences  the goals
 some are better and some worse
 If goal means success,
 then utility means the degree of success (how successful it is)