Intelligent Systems

Lecture 06: State Space Search (Cont.)

Collected and Edited by:

Dr. Mina Younan
Lecturer of Computer Science,
Faculty of Computers and Information, Minia University

20
21
Dr. Mina Younan
 Outline of this lecture
// this lecture collected and edited from Prof. Moheb Girgis Lectures in AI

Part One (cont.):

• Introduction
• Structures for State Space Search
• Graph Theory Basics
• Data-driven and Goal-driven Search
o Backtrack
o Breadth-First Search Algorithm
o Depth-First Search Algorithm
• Local Search Algorithms (Travel Sales man Problem)

20
21
Dr. Mina Younan
2
 Introduction
• Well-formed predicate calculus expressions provide a means of
describing objects and relations in a problem domain, and
inference rules such as modus ponens allow us to infer new
knowledge from these descriptions.
• These inferences define a space that is searched to find a
problem solution.
• By representing a problem as a state space graph, we can use
graph theory to analyze the structure and complexity of both the
problem and the search procedures that we employ to solve it.
• This lecture introduces the theory of state space search.

20
21
Dr. Mina Younan
3
 Structures for State Space Search
Graph Theory Basics:
A graph G(N, A) consists of two sets:
N: a set of nodes N1, N2, …, Nn, …, which need not be finite;
A: a set of arcs that connect pairs of nodes.
The directed arc connecting nodes N3 and N4 is represented by the ordered pair (N3,
N4). N N
3 4

The undirected arc connecting nodes N3 and N4 is represented by two ordered pairs
N4
N3 N4 N3
(N3, N4) and (N4, N3)

Terms used to describe relationships between nodes include parent, child, and
sibling. These are used in the usual familial fashion with the parentParent
preceding N3 its
child along a directed arc. The children of a node are called siblings.
A rooted graph has a unique node, called Child N 4 N5
Child

the root, from which all paths in the graph

originate. That is the root has no parent in the graph. Siblings
20
21
Dr. Mina Younan
4
 Graph Theory Basics
A tree is a graph with the following properties:
• There is a unique path between every pair of nodes.
• Each node has a unique parent.
• The paths contain no cycles.
The edges in a rooted tree are directed away from the root.

A rooted tree - a is the root

a - b is the parent of d, e, and f
- d, e, and f are children of b,
b c and siblings of each other
- a and c are ancestors of g
d e f g - g is a descendant of a and c.

20
21
Dr. Mina Younan
5
 Data-driven and Goal-driven Search
A state space may be searched in two directions:
• from the given data of a problem instance toward a goal, or
• from the goal back to the data.
In data-driven search, sometimes called forward chaining:
• The problem solver begins with the given facts of the problem and a set of
legal moves or rules for changing state.
• Search proceeds by applying rules to facts to produce new facts, which are
in turn used by the rules to generate more new facts.
• This process continues until it generates a path that satisfies the goal.
In goal driven search, sometimes called backward chaining:
• The problem solver takes the goal, and finds the rules or legal moves that
could produce this goal, and determines the conditions that must be true to
use them.
• These conditions become the new goals, or sub-goals, for the search.
• Search continues, working backward through successive sub-goals until it
works back to the facts of the problem.
20
21
Dr. Mina Younan
6
 Data-driven and Goal-driven Search
• The preferred strategy is determined by careful analysis of the problem,
considering such issues as:
1. The complexity of the rules.
2. The branching factor of rule application (on average how many new
states are generated by rule application in both directions?), i.e. the shape
of the state space.
3. The availability of data.
4. Ease of determining the potential goals.
• In solving a problem using either goal- or data-driven search, a problem
solver must find a path from a start state to a goal through the state space
graph.
• The sequence of arcs in this path corresponds to the ordered steps of the
solution.
• The problem solver must consider different paths through the space until it
finds a goal.

20
21
Dr. Mina Younan
7
 The Backtrack Search Algorithm
Backtracking is a technique for systematically trying all paths through a
state space. i.e., begins at the start state and pursues a path until it reaches
either a goal or a "dead end".
• If it finds a goal, it quits and returns the solution path.
• If it reaches a dead end, it "backtracks" to the most recent node on the path
having unexamined children and continues down one of these branches.
1
The algorithm continues until it finds A

a goal or exhausts the state space.

The following figure shows the 2
B 8 C 10 D
backtrack algorithm applied to
3
a hypothetical state space. E F G
6
The dashed arrows indicate the 9
4
progress of the search up and H
5
I J
7
down the space. The number beside each node indicates the order in
which it is visited.
20
21
Dr. Mina Younan
8
 The Backtrack Search Algorithm
We now define an algorithm that performs a backtrack using three lists to
keep track of nodes in the state space:
• SL, for state list, lists the states in the current path being tried. If a goal is
found, SL contains the ordered list of states on the solution path.
• NSL, for new state list, contains nodes whose descendants have not yet been
generated and searched.
• DE, for dead ends, lists states whose descendants have failed to contain a
goal node. If these states are encountered again, they will be eliminated from
consideration again.
In addition, the algorithm uses the variable CS to hold the state currently
under consideration.

20
21
Dr. Mina Younan
9
 The Backtrack Search Algorithm
Function backtrack
begin
SL = [Start]; NSL = [Start]; DE = [ ]; CS = Start; % initialize
while NSL  [ ] do
begin
//--------------------------------------------------
if CS = goal (or meets goal description) then
return SL; % on success, return list on states in path
//--------------------------------------------------
if CS has no children (excluding nodes already on DE, SL, and NSL) then
begin % backtrack
while SL is not empty and CS = 1st element of SL do
begin % This means the state has been visited before.
add CS to DE; % record state as dead end
remove 1st element from SL; % backtrack
remove 1st element from NSL;
CS = 1st element of NSL;
end
add CS to SL;
end
20
21
Dr. Mina Younan
10
 The Backtrack Search Algorithm
else % CS has children
begin
generate and place children of CS (except nodes
already on DE, SL, and NSL) on NSL;
CS = 1st element of NSL;
add CS to SL;
end
end;
return FAIL;
end.

 As presented here, backtrack implements data-driven search, taking the root

as a start state and evaluating its children to search for the goal.
 The algorithm can be viewed as a goal-driven search by letting the goal be the
root of the graph and evaluating descendants back in an attempt to find a start
20
state.
21
Dr. Mina Younan
11
 The Backtrack Search Algorithm
DEz NSL SL CS Iteration #
Initialize:
[] [A] [A] A 0
• SL = [A];
[] [BCDA] [BA] B 1
• NSL = [A];
[] [EFBCDA] [EBA] E 2
• DE = [ ]; [] [HIEFBCDA] [HEBA] H 3
• CS = A [H] [IEFBCDA] [EBA] I
[H] [IEFBCDA] [IEBA] I 4
[IH] [EFBCDA] [EBA] E
[EIH] [FBCDA] [BA] F
a [EIH] [FBCDA] [FBA] F 5
[EIH] [JFBCDA] [JFBA] J 6
b c d
[JEIH] [FBCDA] [FBA] F

e f g [FJEIH] [BCDA] [BA] B

[BFJEIH] [CDA] [A] C
h i j [BFJEIH] [CDA] [CA] C 7
[BFJEIH] [GCDA] [GCA] G 8

20
21
Dr. Mina Younan
12
 The Backtrack Search Algorithm
Initialize: DE NSL SL CS Iteration #
• SL = [A]; [] [A] [A] A 0
• NSL = [A]; [] [BCDA] [BA] B 1
• DE = [ ]; [] [EFBCDA] [EBA] E 2
• CS = A [] [HIEFBCDA] [HEBA] H 3
[H] [IEFBCDA] [EBA] I
[H] [IEFBCDA] [IEBA] I 4
a [IH] [EFBCDA] [EBA] E
[EIH] [FBCDA] [BA] F

b c d [EIH] [FBCDA] [FBA] F 5

[EIH] [JFBCDA] [JFBA] J 6

e f g [JEIH] [FBCDA] [FBA] F

[FJEIH] [BCDA] [BA] B

j [BFJEIH] [CDA] [A] C

h i
[BFJEIH] [CDA] [CA] C 7
[BFJEIH] [GCDA] [GCA] G 8
G is the Goal, Path = [GCA] 9
20
21
Dr. Mina Younan
13
 Depth-First and Breadth-First Search
In addition to specifying a search direction (data-driven or goal-driven), a
search algorithm must determine the order in which states are examined in
the tree or the graph, for example: depth-first and breadth-first search.
Depth-first search (DFS) goes deeper into the search space whenever this is
possible. Only when no further descendants of a state can be found are its
siblings considered.
• DFS examines the states in the previous graph in the order: A, B, E, H, I, F,
J, C, G, D.
• The backtrack algorithm implemented depth-first search.
Breadth-first search (BFS), in contrast, explores the space in a level-by-
level fashion. Only when there are no more states to be explored at a given
level does the algorithm move on to the next level.
• A BFS of the previous graph considers the states in the order: A, B, C, D,
E, F, G, H, I, J.

20
21
Dr. Mina Younan
14
 Depth-First and Breadth-First Search ..
BFS and DFS algorithms use two lists:
• open: lists states that have been generated but whose children
have not been examined. The order in which states are placed
in open determines the order of the search. It is implemented as
a queue in BFS and as a stack in DFS. (open is like NSL in
backtrack).
• closed: records states that have already been examined. (closed
is the union of the DE and SL lists of the backtrack algorithm).
The current state is stored in a variable X.
Child states are generated by inference rules, legal moves of a
game, or other state transition operators. Each iteration
produces all children of the state X and adds them to open.

20
21
Dr. Mina Younan
15
 Breadth-First Search Algorithm
Function BFS
begin
open = [Start]; closed = [ ]; % initialize
while open  [ ] do
begin
remove leftmost state from open, and store it in X;
if X is a goal then
return SUCCESS; % goal found
else
begin
generate children of X;
put X on closed;
discard children of X if already on open or closed;
% loop check
put remaining children on right end of open; % queue
end
end
return FAIL
end.

20
21
Dr. Mina Younan
16
 Breadth-First Search Algorithm
A trace of BFS on the previous graph, where the desired goal is G, appears
below:
closed open X Iteration #

right end of open; [] [A] - 0

a
% queue [A] [BCD] A 1
b c d
[BA] [CDEF] B 2
[CBA] [DEFG] C 3
e f g
[DCBA] [EFG] D 4
[EDCBA] [FGHI] E 5
h i j
[FEDCBA] [GHIJ] F 6
G is the goal G 7

Because BFS considers every node at each level of the graph before going deeper into the
space, all states are first reached along the shortest path from the start State, Breadth-first
search is therefore guaranteed to find the shortest path from the start state to the goal.
20
21
Dr. Mina Younan
17
 Depth-First Search Algorithm
Function DFS
begin
open = [Start]; closed = [ ]; % initialize
while open  [ ] do
begin
remove leftmost state from open, and store it in X;
if X is a goal then
return SUCCESS; % goal found
else
begin
generate children of X;
put X on closed;
discard children of X if already on open or closed;
% loop check
put remaining children on left end of open; % stack
end
end
return FAIL
end.

20
21
Dr. Mina Younan
18
 Depth-First Search Algorithm
A trace of DFS on the previous graph, where the desired goal is G,
appears below:
closed open X Iteration #
a
left end of open; [] [A] - 0
% stack c [A] [BCD] A 1
b d
[BA] [EFCD] B 2
e f g
[EBA] [HIFCD] E 3
h i j [HEBA] [IFCD] H 4
[IHEBA] [FCD] I 5
Unlike BFS, a DFS search is not [FIHEBA] [JCD] F 6
guaranteed to find the shortest path to [JFIHEBA] [CD] J 7
a state the first time that state is
[CJFIHEBA] [GD] C 8
encountered. Later in the search, a
different path may be found to any G is the goal G 9
state.
20
21
Dr. Mina Younan
19
 Local Search Algorithms
Travelling Salesman Problem
• Suppose a salesperson has five cities to visit and then must return home.
• The goal of the problem is to find the shortest path for the salesperson to travel,
visiting each city, and then returning to the starting city.
• The following figure gives an instance of this problem.

• The nodes of the graph represent cities, and

each arc is labeled with a weight indicating the
cost of traveling that arc.
• Assume that the salesperson lives in city A and
will return there.
• The path [A, D, C, B, E, A], with associated
cost of 450 miles, is an example of a possible
circuit.
• The goal description requires a complete circuit
with minimum cost.

20
21
Dr. Mina Younan
Intelligent Systems 20
 Local Search Algorithms
Travelling Salesman Problem
• The goal description is a property of the entire path, rather than of a single state.
• Now we present three different search techniques to solve the salesperson problem:

(1) Exhaustive Search:

• In this technique, beginning
with node A, possible next
states are added until all cities
are included and the path
returns home. The goal is the
lowest-cost path.
• The complexity of the
exhaustive search in the
traveling salesperson problem
is (N-1)!, where N is the
number of cities in the graph.

20
21
Dr. Mina Younan
Intelligent Systems 21
 Local Search Algorithms
Travelling Salesman Problem
• Complexity of N! grows so fast that very soon the search combinations become
intractable. The following two techniques can reduce the search complexity.

(2) Branch and Bound Technique:

• This technique generates paths one at a time, keeping track of the best circuit found
so far. This value is used as a bound on future candidates.
• As paths are constructed one city at a time, the algorithm examines each partially
completed path.
• If the algorithm determines that the best possible extension to a path, the branch, will
have greater cost than the bound, it eliminates that partial path and all of its possible
extensions.
• This reduces search considerably (1.26N rather than N!).

20
21
Dr. Mina Younan
Intelligent Systems 22
 Local Search Algorithms
Travelling Salesman Problem
(3) Nearest Neighbor Technique
• This technique constructs the path according to the rule "go to the closest
unvisited city".
• The nearest neighbor path through the above graph is [A, E, D. B, C, A], at
a cost of 375 miles.
• This method is highly efficient, as there is only one path to be tried!
• The nearest neighbor heuristic is fallible, as graphs exist for which it does
not find the shortest path, but it is a possible compromise when the time
required makes exhaustive search impractical.
• For example, the algorithm fails to find the shortest path if AC = 300.

20
21
Dr. Mina Younan
Intelligent Systems 23
 Local Search Algorithms
Travelling Salesman Problem
(3) Nearest Neighbor Technique
• An instance of the traveling salesperson problem with the nearest neighbor
path in bold.

• Note that this path

[A, E, D, B, C. A],
at a cost of 550, is not
the shortest path.

• The comparatively high

cost of arc (C, A)
defeated the heuristic.

20
21
Dr. Mina Younan
Intelligent Systems 24