Artificial Intelligence:

Search Methods for Problem Solving

Problem Decomposition
A First Course in Artificial Intelligence: Chapter 6

Deepak Khemani
Department of Computer Science & Engineering
IIT Madras

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
Problem Decomposition
• So far our view of problem solving using search
has been state centered
– the state space is the arena for search
– the solution is expressed as a sequence of states
– even when we talk of solution space, the solution, for example for the TSP
problem, is expressed in terms of states.
• Problem decomposition takes
a goal directed view of problem solving
– the emphasis is on breaking up a problem into smaller problems
• like in backward state space planning: goal à subgoals
– primitive problems are labeled SOLVED
– … otherwise they are LIVE and have to be refined
• like in the SSS* game playing algorithm

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
Motivation
• Consider the problem of planning an evening out with friends
• To plan for an activity, a movie followed by dinner
• Let us say the agent proposing the plan works with a MoveGen
as follows
– in the start state pick an activity
– having chosen an activity pick a movie
– having picked a movie pick a restaurant for dinner
– propose the plan
• if not accepted then backtrack
• essentially doing depth first search
• Then the search conducted by the agent may be as follows…

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
Depth First Search Start

Visit Mall Visit Beach

The Matrix The Matrix Everyone accepts the

solution
Pizza Hut Seven Samurai 1. visit the beach
2. watch The Matrix
Saravana Bhavan Bhuvan Shome Pizza Hut 3. eat at Saravana Bhavan
Pizza Hut Saravana Bhavan
Pizza Hut

Artificial Intelligence: AI Saravana Bhavan

Saravana Bhavan

Pizza Hut Saravana Bhavan

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
The culprit? Start

Visit Mall Visit Beach

The Matrix The Matrix It looks like the culprit is the

choice to visit the mall,
Pizza Hut Seven Samurai because,
the other two choices
Saravana Bhavan Bhuvan Shome Pizza Hut occurred in the left subtree
Saravana Bhavan
as well
Pizza Hut Pizza Hut

Artificial Intelligence: AI Saravana Bhavan

Saravana Bhavan

Pizza Hut Saravana Bhavan

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
Wasted search Start

Visit Mall Visit Beach

Chronological backtracking in
The Matrix The Matrix
DFS results in wasted effort
below the Visit Mall node.
Pizza Hut Seven Samurai

Saravana Bhavan Bhuvan Shome Pizza Hut Dependency directed

backtracking in Constraint
Saravana Bhavan
Pizza Hut Pizza Hut Processing would have jumped
Artificial Intelligence: AI Saravana Bhavan back to the culprit variable.

Saravana Bhavan

Pizza Hut Saravana Bhavan

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
The key choices Start

Visit Mall Visit Beach

The Matrix The Matrix

Seven Samurai

Pizza Hut Seven Samurai Bhuvan Shome

Bhuvan Shome Pizza Hut

Saravana Bhavan Pizza Hut
Saravana Bhavan
Pizza Hut Saravana Bhavan Pizza Hut
Pizza Hut

Artificial Intelligence: AI Saravana Bhavan Saravana Bhavan

Saravana Bhavan Artificial Intelligence: AI

Pizza Hut Saravana Bhavan

Pizza Hut Saravana Bhavan

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
An AND-OR tree PlanOuting The solution is a subtree!

AND arc Leaf nodes are SOLVED

Dinner
Outing
OR arc
Movie

Visit Mall Pizza Hut Saravana Bhavan

Visit Beach

Seven Samurai
The Matrix
Bhuvan Shome

Artificial Intelligence: AI
Another option

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
 x4 Symbolic Integration
-------------
dx
(1-x2)5/2
x= sin(y)

sin4y
dy
cos4y
z= tan(y/2)

cot–4y dy tan4y dy z4
32 dx
(1+z2)(1-z2)4

dx z4
dz
x4(1+x2) 1+z4

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
 Symbolic Integration
1 .
(-1 + z2 + 1+z2 ) dz

1 .tan3(arcsin x) – tan(arcsin x) + arcsin x

Integration by parts 3

dz
-dz z2 dz 1+z2

w = -z z = tan(w)

dw dw

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
Dendral (1965-1980)
One of the earliest successes of AI was the program Dendral
(for Dendritic Algorithm) developed at Stanford University.

The DENDRAL program was the first AI program to emphasize the

power of specialized knowledge over generalized problem-solving
methods.

To assist chemists in the task of determining the structure of a

chemical compound. The number of candidate structures for a
given compound can be very large.

Dendral led to a program called CONGEN (CONstrained

GENerator) that allows a chemist to constrain the generation of
candidates.

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
Structural formulas C C
Candidate structures for
C6H13NO2 generated by C C C
CONGEN [Buchanan 82]. C
C
The hydrogen atoms are N C N
C C O
not shown.
O O O
O C C C C

C C N O O C C

C C N C C C

O C

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
Dendral: an Expert System
The program DENDRAL
explored and And-Or graph.

It generated candidate
structures and generated a
synthetic spectrogram.

This was compared to the

spectrogram of the material.

Performed better than most

human chemists.

An Expert System

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
Goal Trees
The search space generated for solving And-Or (AO)
problems can be seen as a goal tree.
G
The figure shows that a goal G may be refined in two ways
• the left branch involves solving the subgoal A
• the right branch reduces in to subgoals B and C
• both have to be solved

A B C

The nodes in the search space have a heuristic value that is

an estimate of the cost of solving the node.

Edge costs indicate the cost of transforming a problem.

Solved leaf nodes may have a non-zero cost as well.

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
Layered Goal Trees G

A BC

A B C
B C

An AO graph with mixed nodes can be converted

into a graph with pure AND and OR nodes.

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
Solving Goal Trees
At any point the algorithm AO* maintains a graph generated so far.

Every choice point in the graph has a marker

marking the best choice as it appears at that point of time.

The algorithm follows the marked path leading to a set of live nodes.
It refines one of the LIVE nodes by expanding it.

It backs up the cost of the best hyper arc and propagates it upwards.

If the best choice leads to a SOLVED node

the node is also labeled SOLVED

The algorithm terminates when the root is labeled SOLVED

or there is no solution

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
AO*

The algorithm for solving the goal tree, known as the AO* algorithm
(Martelli and Montanari, 1978; Nilsson, 1980) has the following cycle.

• Starting at the root traverse the graph along marked paths till the
algorithm reaches a set of unsolved nodes U.
• Pick a node n from U and refine it.
• Propagate the revised estimate of n up via all ancestors.
• If for a node all AND successors along the marked path are marked
SOLVED mark it SOLVED as well.
• If a node has OR edges emanating from it, and the cheapest
successor is marked SOLVED then mark the node SOLVED.
• Terminate when the root node is marked SOLVED.

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
An illustration

G
46 51
G 2
4 3
h=45

A B C
h=42 h=22 h=24

Which is the best node to expand next?

Even though node B has the lowest heuristic value it is a part

of a more expensive looking option. Note that the choice is
made on the basis of backed up values 46 and 51.
Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
An illustration After node A is expanded...
49 G
Best option 46 51
marked ... there are two options
2 (D,E) and (F,G) with
4 3 estimated costs 54 and 45
New estimated respectively.
total cost. h=42
A B C
54 45 h=22 h=24

4 2
3 3
Best option
marked
D E F G
h=23 h=24 h=22 h=18
Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
 AO*: forward phase

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
 AO*: forward phase

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
 Algorithm AO*

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
Admissibility of AO*
Like A* the algorithm AO* is also admissible (finds the least cost solution)
when the heuristic function underestimates the actual cost.

The explanation is the same as before – as long as the partial solutions are
looking better it will keep exploring them.

Refine the best looking partial solution

Till the best solution is fully refined

We illustrate this by varying the cost of edges.

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
Edge cost = 1 21
Start Assume every arc costs
one unit to traverse.

Nodes are labeled with

5
6 7 heuristic values.
4

Solved nodes represented

by double lined boxes
3 4
have cost zero
7

6 8

10 10
9

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
AO*
11 Node to be expanded
Start
21

6 5
7 4

12
2
6 7 4
2
6 7 8

3 4

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
 11

13

2
6
7
7
2
6
7
9

1 4

3 4 10

10

10
10
9
10
10
9

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
Solution 8
Start The algorithm terminates as the
root (start) is labeled SOLVED

6 2
7 4

1 1
Solution found with cost = 8
7

6 8

10 10
9

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
Edge cost = 10 29
Start
21 Cost of each arc = 10

6 5
7 4

33

35
20
6 7 4
20
6 27 4

3 4 Note : cost of node on

expansion goes up due to
underestimation.
Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
Edge cost = 10
44
57

20
6 20
33 6
4 33 17

3 10
3 10

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
 66

59

20
6
40
20 29
6
33
29

10 10

3 10 19

19

10
10
9
10
10
9

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
Explores more nodes 70

69

20
10
40
30
20
10
40
29

10 10

10 10 20

19 6 8

6 8

10
10
10
10
10
9

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
Means Ends Analysis
In their seminal work on Human Problem Solving, Newell and Simon proposed a
general purpose strategy for problem solving, which they call the General
Problem Solver (GPS). GPS encapsulated the heuristic approach which they
called Means Ends Analysis.

• compare the current state with the desired state, and

• list out the differences between them
• evaluate the differences in terms of magnitude (in some way)
• consult an operator-difference table
• reduce largest (most important) differences first

• the differences characterize the ends that need to be achieved

• the operators define the means of achieving those ends.

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
Means Ends Analysis Achieve goal G from state S

Reduce largest difference

between goal G from state S Achieve goal G from state S’

Operator O1 •• Operator Oi •• Operator Ok

Achieve precond(Oi) from state S Apply Oi in S to generate S’

The MEA strategy generates an AND/OR tree by replicating the this

structure below the shaded nodes when a recursive call is made.

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras
 End : AO*

Artificial Intelligence: Search Methods for Problem Solving Deepak Khemani, IIT Madras