Department of Information Systems School of Computing and Informatics

CHAPTER 2
INTELLIGENT AGENTS

Objectives:
 Understand the agent’s environment and its behavior
 Differentiate rationality from omniscience, understand how to specify task environment of an
agent
 Explain the properties of task environment
 Describe the structure of an agent and the working principle of Simple-Reflex agent
 Describe the working principle of Model-based reflex agent and Goal-based agent.
 Describe the working principle of Utility-based agent and Learning agent.

Agent: An agent is an entity that perceives and acts.

Computer Agent: Operate autonomously, perceive their environment, persist over a prolonged time period,
adapt to change and create and pursue goals.
Rational Agent: is one that acts so as to achieve the best outcome, or, when there is uncertainty, the best
expected outcome.

1. AGENTS AND ENVIRONMENTS:

o Agents include humans, robots, soft bots and thermostats.

o The agent function f: P*A maps from percept histories to actions.
o The agent program runs on the physical architecture to produce f.
o Percept – perceptual input of an agent – at any given time.

Human agent: sensors – eyes, ears, and other organs;

actuators – hands, legs, vocal tract, etc.
Robotic agent: sensors – cameras, infrared range finders;
actuators – various motors.
Software agents: sensors – receives keystrokes, file contents, network packets as inputs;
actuators – displaying on the screen, writing files, sending network packets.
Agent function is an abstract mathematical description: f: P*A
Agent program is a concrete implementation running within some physical system.

Mizan-Tepi University Page 1

 Department of Information Systems School of Computing and Informatics

Example: Vacuum-cleaner world with just two locations

Partial tabulation of simple agent function for the vacuum –cleaner world:

GOOD BEHAVIOR: THE CONCEPT OF RATIONALITY

Rational agent: is one that does the right thing conceptually, every entry in the table for the agent function
is filled out correctly.

What does it mean to do the right thing?

Ans: by considering the consequences of the agent’s behavior.

Performance measure:That evaluates any given sequence of environment states.

Example: vacuum-cleaner world
 One point per square cleaned up in time T?
 One point per clean square per time step, minus one per move?
 Penalize for >K dirty squares?

Obviously, there is not one fixed performance measure for all tasks and agents; typically, a designer will devise
one appropriate to the circumstances.
As a general rule, it is better to design performance measures according to what one actually wants in the
environment, rather than according to how one thinks the agent should behave.

Mizan-Tepi University Page 2

 Department of Information Systems School of Computing and Informatics

Rationality: What is rational at any given time depends on four things:

 The performance measure that defines the criterion of success.
 The agent’s prior knowledge of the environment.
 The action that the agent can perform
 The agent’s percept sequence to date.

Definition of Rational Agent:

For each possible percept sequence, a rational agent should select an action that is expected to maximize
its performance measure given the evidence provided by the percept sequence and whatever built-in knowledge
the agent has.

Whether Vacuum-cleaner agent is a rational agent?

Let us assume the following:

 The performance measure awards one point for each clean square at each time step, over a lifetime of
1000 time steps.
 The geography of the environment is known a priori but the dirt distribution and the initial location of
the agent are not, clean squares stay clean and sucking cleans the current square.
The left and right actions move the agent left and right except when this would take the agent outside the
environment, in which case the agent remains where it is.
 The only available actions are left, right and suck.
 The agent correctly perceives its location and whether that location contains dirt.

We claim that under these circumstances the agent is indeed rational; the same agent would be irrational under
different circumstances.

2. RATIONALITY vs. OMNISCIENCE

An omniscient agent knows the actual outcome of its actions and can act accordingly; but omniscience is
impossible in reality.
Example: A person is walking along the road and there is no traffic nearby and he is being rational, he start to
cross the street, meanwhile, at 33,000 feet, a cargo door falls off a passing airliner, and before he make it to the
other side of the street he flattened.

Was he irrational to cross the street?

This example shows that rationality is not the same as perfection.
 Rationality maximizes expected performance, while perfection maximizes actual performance.

The point is that if we expect an agent to do what turns out to be the best action after the fact, it will be
impossible to design an agent to fulfill this specification.
Mizan-Tepi University Page 3
 Department of Information Systems School of Computing and Informatics

Our definition of rationality does not require omniscience, then, because the rational choice depends only on the
percept sequence to date. Information gathering and exploration, is an important part of rationality – Doing
actions in order to modify future percepts.

Rational ≠ omniscient
Rational=> exploration ,learning and autonomy.

Learning and autonomy:

 A Rational agent should not only to gather information, but also to learn as much as possible from what
it perceives.
 A Rational agent should be autonomous-it should learn what it can to compensate for partial or incorrect
prior knowledge.

To the extent that an agent relies on the prior knowledge of its designer rather than on its own percepts, we say
that the agent lacks autonomy.

Example: A vacuum-cleaning agent that learns to foresee where and when additional dirt will appear will do
better than one that does not.

3. THE NATURE OF ENVIRONMENTS:

Taskenvironments: which are essentially the “problems” to which rational agents are the “solutions”.
Specifying the task environment:
In designing an agent, the first step must always be to specify the task environment as fully as possible.
We call this the PEAS (Performance, Environment, Actuators, and Sensors) description.

Ex 1:An automated Taxi driver agent.

Performance measure? safety, destination, profit, legality, comfort…..
Environment? streets, traffic, pedestrians, weather….
Actuators?  steering, accelerator, brake, horn, speaker, display….
Sensors video, accelerometers, gauges, engine sensors keyboard, GPS,…

Ex 2: Internet Shopping agent: (Software Agent)

Performance measure? price quality, appropriateness, efficiency.
Enviornment?  current and future WWW sites, vendors, shippers.
Actuators?  display to user, follow URL, fill in form.
Sensors? HTML pages (text, graphics, scripts).

Ex 3: Medical diagnosis system agent

Performance measure?  Healthy patient, reduced costs.
Environment?  Patient, Hospital, staff.
Actuators?  Display of questions, test, diagnosis, treatments, referrals.
Sensors?  Keyboard entry of symptoms, findings, patient’s answers.

Mizan-Tepi University Page 4

 Department of Information Systems School of Computing and Informatics

Properties of task environments:

This determines to a large extent, the appropriate agent design and the applicability of each of the principal
families of techniques for agent implementation.

1. Fully observable vs. Partially observable

If an agent’s sensors give it access to the complete state of the environment ateach point in time, then we say
that the task environment is fully observable.

An environment might be partially observable because of noisy and inaccurate sensors or because parts of the
state are simply missing from the sensor data.

Example forPartially observable:

Vacuum Agent – with only a local dirt sensor cannot tell whether there is dirt in other squares.
Automated Taxi driver agent – cannot see what other drivers are thinking.

Unobservable: if the agent has no sensors at all then the environment is unobservable.

2. Single Agent vs. Multiagent

Single Agent – Example – an agent solving a crossword puzzle by itself.

Multi agent - Example – an agent playing chess is in a two-agent environment.

Which entities must be viewed as agents?

Does an agent A (the taxi driver) have to treat an object B (another vehicle) as an agent or it can be
treated merely as an object”. The key distinction is whether B’s behavior is best described as maximizing a
performance measure whose value depends on agent A’s behaviour.

Competitive multiagent environment:

Example: In chess, the opponent entity B is trying to maximize its performance measure, which, by the
rules of chess, minimizes agent A’s performance measure.

Cooperative multiagent environment:

Example: In taxi driving agent environment,avoiding collisions maximizes the performance measure of all
agents. It is a partially cooperative environment. It is also partially competitive because, only one can occupy a
parking space.

3. Deterministic vs. Stochastic

If the next state of the environment is completely determined by the current state and the action executed
by the agent, then we say the environment is deterministic, otherwise, it is stochastic.

In principle, an agent need not worry about uncertainty in a fully observable, deterministic environment.
Example: A game can be deterministic even though each agent may be unable to predict the actions of
others.If the environment is partially observable, then it could appear to be stochastic.

Mizan-Tepi University Page 5

 Department of Information Systems School of Computing and Informatics

Example: Taxi driving is stochastic, because one can never predict the behavior of traffic exactly.

Example: vacuum-cleaner world as we described it is deterministic, but variations can include stochastic
elements such as randomly appearing dir and unreliable suction mechanism.

Uncertain environment:
An environment is uncertain if it is not fully observable or not deterministic.

Note:
 Our use of the word “Stochastic” generally implies that uncertainty about outcomes is quantified in
terms of probabilities.
 Non deterministic environment is one in which actions are characterized by their possible outcomes, but
no probabilities are attached to them.

4. Episodic vs. Sequential

Episodic: In this environment, the agent’s experience is divided into atomic episodes. In each episode the agent
receives a percept and then performs a single action. The next episode does not depend on the actions taken in
previous episodes.

Example:An agent that has to spot defective parts on an assembly line bases each decision in the
current part, regardless of previous decisions; moreover the current decision doesn’t affect whether the
next part is defective.

Sequential: In this environment, the current decision could affect all future decisions.

Example: Chess and taxi driving – in both cases, short term actions can have long-term consequences.

Episodic environments are much simpler than sequential environments because the agent does not need to think
ahead.

5. Static vs. Dynamic

If the environment can change while an agent is deliberating, then it is dynamic for that agent; otherwise
it is static.

Static environments: are easy to deal with because the agent need not keep looking at the work while it is
deciding on an action, nor need it worry about the passage of time.

Dynamic environments: are continuously asking the agent what it wants to do; if it hasn’t decided yet, that
counts as deciding to do nothing.

Semidynamic environment: If the environment itself does not change with the passage of time but the agent’s
performance sure does, then we say that it is semidynamic environment.

Mizan-Tepi University Page 6

 Department of Information Systems School of Computing and Informatics

Example:
Taxi driving – is dynamic
Chess – is semidynamic, when played with a clock.
Crossword puzzles – are static

6. Discrete vs. Continuous

These discrete/continuous distinction applies to the state of the environment, to the way time is handled, and to
the percepts and actions of the agent.

Example: Chess – has a finite number of distinct states (excluding clock)

- Also has discrete set of percepts and actions.

Taxi driving - is a continuous-state and continuous time problem: the speed and location of the taxi and
of the other vehicles sweep through a range of continuous values and so smoothly over time.
Taxi driving actions – are also continuous (steering angles….)

7. Known vs. Unknown

This distinction refers not to the environment itself but to the agent’s (or designer’s) state of knowledge about
the “laws of physics” of the environment.

Known environment:
The outcomes (or outcome probabilities of the environment is stochastic) for all actions are given.
Unknown environment:
Here the agent will have to learn how it works in order to make good decisions.

Note: The distinction between known and unknown environments is not the same as the one between fully and
partially observable environments.

Known environment can be partially observable:

Example: In solitaire games, I know the rules but a still unable to see the cards that have not yet been
turned over.

Unknown environment can be fully observable:

Example: In a new video game, the screen may show the entire game state but I still don’t know what
the buttons do until I try them.

Mizan-Tepi University Page 7