Artificial Intelligence

Lecturer 7 – Part I: Planning

Brigitte Jaumard
Dept of Computer Science and Software
Engineering
Concordia University
Montreal (Quebec) Canada

1
Outline
n Planning problem
n State-space search
n Partial-order planning
n Planning graphs
n Planning with propositional logic

2
2
Search vs. planning
n Consider the task get milk, bananas, and a cordless drill
n Standard search algorithms seem to fail miserably:

3
n After-the-fact heuristic/goal test inadequate
Planning problem

n Planning is the task of determining a sequence of

actions that will achieve a goal.
n Domain independent heuristics and strategies
must be based on a domain independent
representation
q General planning algorithms require a way to represent
states, actions and goals
q STRIPS, ADL, PDDL are languages based on propositional
or first-order logic
n Classical planning environment
q fully observable, deterministic, finite, static and discrete.

4
Additional complexities

n Because the world is …

q Dynamic
q Stochastic
q Partially observable
n And because actions
q take time
q have continuous effects

5
AI Planning background

n Focus on classical planning; assume none of

the above

n Deterministic, static, fully observable

q “Basic”
q Most of the recent progress
q Ideas often also useful for more complex
problems

6
CPM: Critical Path Method
[0,15] [30,45] [60,75]
AddEngine1 AddWheels1 Inspect1
30 30 10
[0,0] [85,85]
Start Finish

[0,0] [60,60] [75,75]

AddEngine2 AddWheels2 Inspect2
60 15 10

AddWheels1

AddEngine1 Inspect1

AddEngine2 Inspect2

AddWheels2

0 10 20 30 40 50 60 70 80 90
7
Critical Path Method (Cont’d)
n

8
Problem Representation
n State
q What is true about the (hypothesized) world?

n Goal
q What must be true in the final state of the world?

n Actions
q What can be done to change the world?

q Preconditions and effects

n We’ll represent all these as logical predicates

9
 STRIPS STRIPS

• Developed at Stanford in early 1970s (Stanford Research Institute

Planning System), for the first “intelligent” robot
• Domain: a set of typed objects; usually represented as propositions
• States are represented as first-order predicates over objects
– Closed-world assumption: everything not stated is false; the only
objects in the world are the ones defined
• Operators/Actions defined in terms of:
– Preconditions: when can the action be applied?
– Effects: what happens after the action?
No explicit description of how the action should be executed
• Goals: conjunctions of literals

10
STRIPS Representation
STRIPS representations

• States are represented as conjunctions

In(Robot,room) ¬ In(Charger, r) ...
• Goals are represented as conjunctions:
(implicit ⌥ r) In(Robot, r) In(Charger, r)
• Actions (operators):
– Name: Go(here, there)
– Preconditions: expressed as conjunctions
At(Robot, here) Path(here, there)
– Postconditions (effects): expressed as conjunctions
At(Robot, there) ¬ At(Robot, here)
• Variables can only be instantiated with objects of the correct type

11
 STRIPS Operator Representation
STRIPS Operator Representation

• Operators have a name, preconditions and postconditions or effects

• Preconditions are conjunctions of positive literals
• Postconditions/effects are represented in terms of:
– Add-list: list of propositions that become true after the action
– Delete-list: list of propositions that become false after the action

12
 Semantics
Semantics

• If the precondition is false in a world state, the action does not change
anything (since it cannot be applied)
• If the precondition is true:
– Delete the items in the Delete-list
– Add the items in the Add-list.
Order of operations is important here!

This is a very restricted language, which means we can do e⇥cient inference.

13
 Example: Buying Action
Example: Buying Action

• Action: Buy(x) (where x is a good)

• Precondition: At(s), Sells(s,x,p), HaveMoney(p) (where s is a store, p is
the price)
• Effect:
– Add-list: Have(x)
– Delete-list: HaveMoney(p)
• Note that many important details are abstracted away!
• Additional propositions can be added to show that now the store has the
money, the stock has decreased etc.

14
Example: Move Action
Example: Move Action

• Action: Move(object, from, to)

• Preconditions: At(object, from), Clear(to), Clear(object)
• Effects:
– Add-list: At(object, to), Clear(from)
– Delete-list: At(object, from), Clear(to)

15
Pros and Cons of STRIPS
Pros and cons of STRIPS

• Pros:
Example: Move action
– Since it is restricted, inference can be done e⇥ciently
– All operators can be viewed as simple deletions and additions of
• Action: to the knowledge base
propositions
• Cons: – Move(object, from, to)
– Assumes that a small number of propositions will change for each
• Preconditions:
action– (otherwise operators
At(object, from), Clear(to), are hard to write down, and reasoning
Clear(object)
becomes expensive)
• Effects:
– Limited language (preconditions and effects are expressed as
conjunctions,
– Add-list: implicit
At(object, quantifiers),
to), Clear(from) so not applicable to all domains
of interest.
– Delete-list: At(object, from), Clear(to)

16
Example: Block World
Welcome to the Blocks World!
Example: Blocks World

A
B
A B C C
Initial state Goal state

Initial state = On(A,table) ! On(B,table) ! On(C,table) ! Clear(A) ! Clear(B) ! Clear(C)

Goal state = On(A,B) !On (B,C)

Action = Move(b,x,y)
Precondition = On(b,x) ! Clear(b) ! Clear(y)
Effect = On(b,y) ! Clear(x) ! ¬On(b,x) ! ¬Clear(y)

Action = MoveToTable(b,x)
Preconditions = On(b,x) ! Clear(b)
Effect = On(b,Table) ! Clear(x) ! ¬On(b,x)

COMP-424: Artificial intelligence 14 Joelle Pineau

17
 STRIPS state transitions
State Transitions in the Blocks World
State Transitions in the Blocks World

18
 Two Basic Approaches to Planning
Two Basic Approaches to Planning

1. State-space planning works at the level of the states and operators

• Finding a plan is formulated as a search through state space, looking
Pros and cons of STRIPS
for a path from the start state to the goal state(s)
• Most similar to constructive search
2. Plan-space planning works at the level of plans
• Finding a plan is formulated as a search through the space of plans
Pros:
• We start with a partial, possibly incorrect plan, then apply changes to
it to make it a full, correct plan
– Since it is restricted, inference can be done efficiently.
• Most similar to iterative improvement/repair
– All operators can be viewed as simple deletions and additions of
propositions to the knowledge base.

Cons:
– Assumes that a small number of propositions will change for19each
Planning with state-space search

n Both forward and backward search possible

n Progression planners
q forward state-space search

q consider the effect of all possible actions in a given state

n Regression planners
q backward state-space search

q Determine what must have been true in the previous state

in order to achieve the current state

20
 Plan-Space Planning in the Blocks World

Plan-Space Planning in the Blocks World

• Start with plan: Put(A,B), Put(B,C)

• Plan fails because the precondition of the second action is not satisfied
after the first action
• So we can try to add a step, remove a step, or re-order the steps

21
State-Space Planners
State-Space Planners

• Progression planners reason from the start state, trying to find the
operators that can be applied (match preconditions)
• Regression planners reason from the goal state, trying to find the actions
that will lead to the goal (match effects or post-conditions)

In both cases, the planners work with sets of states instead of using
individual states, like in straightforward search

22
 Progression (Forward) Planning

Progression (Forward) Planning

1. Determine all operators that are applicable in the start state

2. Ground the operators, by replacing any variables with constants
3. Choose an operator to apply
4. Determine the new content of the knowledge base, based on the operator
description
5. Repeat until goal state is reached.

23
 Example: Supermarket Domain
Example: Supermarket Domain

• In the start state we have At(Home), which allows us to apply operators

of the type Go(x,y).
• The operator can be instantiated as Go(Home, HardwareStore),
Go(Home,GroceryStore), Go(Home, School), ...
• If we choose to apply Go(Home, HardwareStore), we will delete from the
KB At(Home) and add At(HardwareStore).
• The new proposition enables new actions, e.g. Buy

Note that in this case there are a lot of possible operators to perform!

24
Goal Regression
Goal Regression

• Introduced in Newell & Simon’s General Problem Solver

• Algorithm:
1. Pick an action that satisfies (some of) the goal propositions
2. Make a new goal by:
– Removing the goal conditions satisfied by the action
– Adding the preconditions of this action
– Keeping any unsolved goal propositions
3. Repeat until the goal set is satisfied by the start state

25
 Example: Supermarket Domain
Example: Supermarket Domain

• In the goal state we have At(Home) Have(Milk) Have(Drill)

• The action Buy(Milk) would allow us to achieve Have(Milk)
• To apply this action we need to have the precondition At(GroceryStore),
so we add it to the set of propositions we want to achieve
• The goal set becomes: At(Home) At(GroceryStore) Have(Drill)
• Next, we may want to achieve At(HardwareStore)

Note that in this case the order in which we try to achieve these propositions
matters!

26
 Variations of Goal Regression

Variations of Goal Regression

• Using a stack of goals - also called linear planning

This is not complete! I.e. we may not find a plan even if one exists
• Using a set of goals - also called non-linear planning
This is complete, but more expensive (need to decide what to work on
next)
• Both versions are sound: only legal plans will be found

27
COMP-424, Lecture 9 - February 4, 2013 49

Prodigy Planner

Prodigy Planner

• Do both forward search and goal regression at the same time.

• At each step, choose either an operator to apply or goal to regress
• Uses domain-dependent heuristics to guide the search
• General heuristics (e.g. number of propositions satisfied) do not work
well in planning, because subgoals interact

28
 Total vs. Partial Order
Total vs. Partial Order
Total vs. Partial Order
Total vs. Partial Order
• Total order: Plan is always a astrict sequence of actions
• Total order: Plan is
• Total always aalways
order: Plan is strict sequence
strict of actions
sequence of actions
E.g. E.g.
To paint thethe
To paint ceiling
ceiling
Plan 1:PlanStart
1: Start Get Get
Brush
Brush Get
GetLadder
Ladder Paint ceiling
Paint ceiling Finish
Finish

2: Start Get Get

Plan 2:PlanStart Ladder
Ladder Get
Get Brush
Brush Paint ceiling
Paint ceiling Finish
Finish

• Partial order: Plan steps may be unordered

• Partial order:• Plan stepsPlanmay
Partial order: steps be unordered
may be unordered
Get Brush
Start Get Brush Paint ceiling Finish
Start Get Ladder Paint ceiling Finish
Get Ladder

COMP-424: Artificial intelligence 3 Joelle Pineau

COMP-424: Artificial intelligence 3 Joelle Pineau

29
 –Search space is a set of partial plans
–Transitions
Can are
maintain both plan
state andoperators
plans (e.g. Prodigy)

Partial Order Planning

COMP-424: Artificial intelligence 4 Joelle Pineau

Partial Order Planning Can maintain both state and plans (e.g. Prodigy)

COMP-424: Artificial intelligence 4 Joelle Pineau

• Search in plan space and use least commitment whenever possible 2
• In state space search:
– Search space is a set of states of the world
– Actions cause transitions between states
– Plan is a path through state space
• In plan space search:
– Search space is a set of partial plans
– Plan operators cause transitions
– Goal is a legal plan
• Can maintain both state and plans (e.g. Prodigy)

30
 Plan-Space Planners
Plan-Space Planners
Plan is defined by ↵A, O, B, L :

• A is a set of actions/operators from the problem domain

• O is a set of ordering constraints of the form ai < aj
The constraint specifies that ai must come before aj but does not say
exactly when
• B is a set of bindings, of the form vi = C, vi ⇧= C, vi = vj or vi ⇧= vj ,
where vi, vj are variables and C is a constant
• L is a set of causal links, which records why a certain ordering has to
occur:
ai ⇥c aj means that action ai achieves effect c which is a precondition
of aj

31
COMP-424, Lecture 9 - February 4, 2013 53

Plan Transformations

Plan Transformations

• Adding actions
• Specifying orderings
• Binding variables

Constraint satisfaction is used along the way to ensure the consistency of

orderings

32
 Discussion of Partial-Order Planning
Discussion of Partial-Order Planning

• Advantages:
– Plan steps may be unordered (plan will be ordered, or linearized, before
execution)
– Handles concurrent plans
– Least commitment can lead to shorter search times
– Sound and complete
– Typically produces the optimal plan
• Disadvantages:
– Complex plan operators lead to high cost for generating every action
– Larger search space, because of concurrent actions
– Hard to determine what is true in a state

33
The real world
The real world
Things are usually not as expected:

• Incomplete information
– Unknown preconditions, e.g., Intact(Spare)
– Disjunctive effects, e.g., Inflate(x) causes Inflated(x) according to
the knowledge base, but in reality it actually causes Inflated(x)
SlowHiss(x) Burst(x) BrokenPump . . .
• Incorrect information
– Current state incorrect, e.g., spare NOT intact
– Missing/incorrect postconditions in operators
• Qualification problem: can never finish listing all the required
preconditions and possible conditional outcomes of actions

34
 Solutions
Solutions

• Conditional (contingency) planning:

1. Plans include observation actions which obtain information
2. Sub-plans are created for each contingency (each possible outcome of
the observation actions)
E.g. Check the tire. If it is intact, then we’re ok, otherwise there are
several possible solutions: inflate, call AAA....
Expensive because it plans for many unlikely cases
• Monitoring/Replanning:
1. Assume normal states, outcomes
2. Check progress during execution, replan if necessary
Unanticipated outcomes may lead to failure (e.g., no AAA card)
In general, some monitoring is unavoidable

35
 Monitoring

Monitoring

• Execution monitoring: “failure” means that the preconditions of the

remaining plan not met
• Action monitoring: “failure” means that the preconditions of the
next action not met (or action itself fails)

In both cases, need to replan

36
Replanning

Replanning

• Simplest: on failure, replan from scratch

• Better: plan to get back on track by reconnecting to best continuation
In this case, we can try to reconnect to the plan’s next action, or some
future action
The latter is typically more expensive in terms of planning computation
(lots of possible places to reconnect!) but usually yields better plans (e.g.
if it is very hard to achieve the preconditions of the very next action)

37
 Summary
Summary

• Planning is very related to search, but allows the actions/states have

more structure
• We typically use logical inference to construct solutions
• State-space vs.plan-space planning
• Least-commitment: we build partial plans, order them only as necessary
• In the real world, it is necessary to consider failure cases - replanning
• Hierarchy and abstraction make planning more e⇥cient
• Many varieties of planners that we have not looked at: case-based
planners, MDP planners (we will see this later on) etc.

38
Progression and regression
initial state

goal

39
Progression algorithm
n Formulation as state-space search problem:
q Initial state and goal test: obvious
q Successor function: generate from applicable actions
q Step cost = each action costs 1
n Any complete graph search algorithm is a complete planning
algorithm.
q E.g. A*
n Inherently inefficient:
q (1) irrelevant actions lead to very broad search tree
q (2) good heuristic required for efficient search

40
Forward Search Methods:
can use A* with some h and g

41
Regression algorithm
n How to determine predecessors?
q What are the states from which applying a given action leads to the
goal?
Goal state = At(C1, B) Ù At(C2, B) Ù … Ù At(C20, B)
Relevant action for first conjunct: Unload(C1,p,B)
Works only if pre-conditions are satisfied.
Previous state= In(C1, p) Ù At(p, B) Ù At(C2, B) Ù … Ù At(C20, B)
Subgoal At(C1,B) should not be present in this state.
n Actions must not undo desired literals (consistent)
n Main advantage: only relevant actions are considered.
q Often much lower branching factor than forward search.

42
Regression algorithm
n General process for predecessor construction
q Give a goal description G
q Let A be an action that is relevant and consistent
q The predecessors are as follows:
n Any positive effects of A that appear in G are deleted.

n Each precondition literal of A is added , unless it already

appears.
n Any standard search algorithm can be added to perform the
search.
n Termination when predecessor satisfied by initial state.
q In FO case, satisfaction might require a substitution.

43
Backward search methods

Regressing a
ground
operator

44
Regressing an ungrounded operator

45
Example of Backward Search
A partial order plan for putting
shoes and socks

47
Reading and Suggested Exercises

n Chapters 10
n Exercises: 10.3, 10.4, 10.15

48