Search Algorithms

Search algorithms
A search algorithm takes a search problem as
input and returns a solution, or an indication of
failure.
Finding out a solution is done by
 searching through the state space

All problems are transformed

 as a search tree

 generated by the initial state and successor

function
 Search tree
Each node in the search tree corresponds to a state
in the state space
Edges in the search tree correspond to actions.
The root of the tree corresponds to the initial state of
the problem.
Expanding
 applying successor function to the current state
 thereby generating a new set of states

leaf nodes
 the states having no successors
Fringe: Set of search nodes that have not been expanded
yet.
 State Spaces versus Search Trees
State Space
 Set of valid states for a problem

 Linked by operators

 e.g., 20 valid states (cities) in the Romanian travel problem

Search Tree
Root node = initial state
Child nodes = states that can be visited from parent
Note that the depth of the tree can be infinite
 E.g., via repeated states

Partial search tree

 Portion of tree that has been expanded so far

Fringe
 Leaves of partial search tree, candidates for expansion Search trees =

data structure to search state-space

Tree search example
Tree search example
 Search tree
The essence of searching
 in case the first choice is not correct
 choosing one option and keep others for later
inspection
Hence we have the search strategy
 which determines the choice of which state to
expand
 good choice  fewer work  faster

Important:
 state space ≠ search tree
 Search tree
State space
 has unique states {A, B}
 while a search tree may have cyclic paths:

A-B-A-B-A-B- …
A good search strategy should avoid
such paths
 Search data structures
Search algorithms require a data structure to keep track of
the search tree.
A node in the tree is represented by a data structure with
five components:
STATE: which state it is in the state space
 PARENT-NODE: from which node it is generated

 ACTION: which action applied to its parent-node to

generate it
 PATH-COST: the cost, g(n), from initial state to the
node n itself
 DEPTH: number of steps along the path from the initial
state
Data structure to store the frontier

The appropriate choice is a queue of some kind, because

the operations on a frontier are:
•IS-EMPTY(frontier) returns true only if there are no nodes
in the frontier.
•POP(frontier) removes the top node from the frontier and
returns it.
•TOP(frontier) returns (but does not remove) the top node
of the frontier.
• ADD(node, frontier) inserts node into its proper place in
the queue.
Three kinds of queues are used in search
algorithms:
•Priority queue first pops the node with the minimum
cost according to some evaluation function, It is used in
best-first search. g(node) f.
•FIFO queue or first-in-first-out queue first pops the node
that was added to the queue first; we shall see it is used in
breadth-first search.
•LIFO queue or last-in-first-out queue (also known as a
stack) pops first the most recently added node; we shall
see it is used in depth-first search.
•The reached states can be stored as a lookup table (e.g. a
hash table) where each key is a state and each value is the
node for that state.
Measuring problem-solving performance
The evaluation of a search strategy/properties
 Completeness:
 isthe strategy guaranteed to find a solution when
there is one?
 Optimality:
 doesthe strategy find the highest-quality solution
when there are several different solutions?
 Time complexity:
 how long does it take to find a solution?
 Space complexity:
 how much memory is needed to perform the search?
Measuring problem-solving performance
In AI, complexity is expressed in
 b, branching factor, maximum number of
successors of any node
 d, the depth of the shallowest goal node.

(depth of the least-cost solution)

 m, the maximum length of any path in the state
space
Time and Space is measured in
 numberof nodes generated during the search
 maximum number of nodes stored in memory
Measuring problem-solving performance

For effectiveness of a search algorithm

 we can just consider the total cost
 The total cost = path cost (g) of the solution
found + search cost
 search cost = time necessary to find the solution
Tradeoff:
 (long time, optimal solution with least g)
 vs. (shorter time, solution with slightly larger
path cost g)
General problem solving, Water-jug
problem, 8-puzzle problem
 Production System - PS
PS is a formation for structuring AI programs
which facilitates describing search process.
It consists of
 Initial or start state of the problem
 Final or goal state of the problem
 It consists of one or more databases containing
information appropriate for the particular task.
The information in databases may be structured
 using knowledge representation schemes.

12/02/24 Prof Saroj Kaushik 17

 Production Rules
PS contains set of production rules,
 each consisting of a left side that determines the
applicability of the rule and
 a right side that describes the action to be performed if
the rule is applied.
 These rules operate on the databases.
 Application of rules change the database.
A control strategy that specifies the order in which
the rules will be applied when several rules match
at once.
One of the examples of Production Systems is an
Expert System.

12/02/24 Prof Saroj Kaushik 18

 Advantages of PS
In addition to its usefulness as a way to
describe search, the production model has other
advantages as a formalism in AI.
 It is a good way to model the strong state driven
nature of intelligent action.
 As new inputs enter the database, the behavior of the
system changes.
 New rules can easily be added to account for new
situations without disturbing the rest of the system,
which is quite important in real-time environment.

12/02/24 Prof Saroj Kaushik 19

 Example : Water Jug
Problem
Problem statement:
 Given two jugs, a 4-gallon and 3-gallon having no
measuring markers on them. There is a pump that can be
used to fill the jugs with water. How can you get exactly 2
gallons of water into 4-gallon jug.
Solution:
 State for this problem can be described as the set of
ordered pairs of integers (X, Y) such that
 X represents the number of gallons of water in 4-gallon jug
and
 Y for 3-gallon jug.
 Start state is (0,0)
 Goal state is (2, N) for any value of N.
12/02/24 Prof Saroj Kaushik 20
 Production Rules
Following are the production rules for this problem.

R1: (X, Y | X < 4)  (4, Y)

{Fill 4-gallon jug}
R2: (X, Y | Y < 3)  (X, 3)
{Fill 3-gallon jug}
R3: (X, Y | X > 0)  (0, Y)
{Empty 4-gallon jug}
R4: (X, Y | Y > 0)  (X, 0)
{Empty 3-gallon jug}
R5: (X, Y | X+Y >= 4  Y > 0) (4, Y – (4 – X))
{Pour water from 3- gallon
jug into 4-gallon jug until
4-gallon jug is full}
12/02/24 Prof Saroj Kaushik 21
 Contd..
R6: (X, Y | X+Y >= 3  X > 0)  (X – (3 – Y), 3)
{Pour water from 4-gallon jug into
3- gallon jug until 3-gallon jug is full}
R7: (X, Y | X+Y <= 4  Y > 0)  (X+Y, 0)
{Pour all water from 3-gallon jug
into 4-gallon jug }
R8: (X, Y | X+Y <= 3  X > 0)  (0, X+Y)
{Pour all water from 4-gallon jug
into 3-gallon jug }

Superficial Rules: {May not be used in this problem}

R9: (X, Y | X > 0)  (X – D, Y)
{Pour some water D out from 4-gallon
jug}
R10: (X, Y | Y > 0)  (X, Y - D)
{Pour some water D out from 3- gallon
jug}
12/02/24 Prof Saroj Kaushik 22
 Trace of steps involved in solving the
water jug problem - First solution
Number Rules applied 4-g 3-g
of Steps jug jug
1 Initial State 0 0
2 R2 {Fill 3-g jug} 0 3
3 R7{Pour all water from 3 to 4-g jug } 3 0
4 R2 {Fill 3-g jug} 3 3
5 R5 {Pour from 3 to 4-g jug until it is full} 4 2
6 R3 {Empty 4-gallon jug} 0 2
7 R7 {Pour all water from 3 to 4-g jug} 2 0 Goal State

12/02/24 Prof Saroj Kaushik 23

 Trace of steps involved in solving the
water jug problem - Second solution
Note that there may be more than one solutions.

Number Rules applied 4-g 3-g

of steps jug jug
1 Initial State 0 0
2 R1 {Fill 4-gallon jug} 4 0
3 R6 {Pour from 4 to 3-g jug until it is full } 1 3
4 R4 {Empty 3-gallon jug} 1 0
5 R8 {Pour all water from 4 to 3-gallon jug} 0 1
6 R1 {Fill 4-gallon jug} 4 1
7 R6 {Pour from 4 to 3-g jug until it is full} 2 3
8 R4 {Empty 3-gallon jug} 2 0 Goal State
12/02/24 Prof Saroj Kaushik 24
 Important Points
For each problem
 there is an initial description of the problem.
 final description of the problem.

 more than one ways of solving the problem.

 a path between various solution paths based on some

criteria of goodness or on some heuristic function is

chosen.
 there are set of rules that describe the actions called

production rules.
 Left side of the rules is current state and right side describes
new state that results from applying the rule.

12/02/24 Prof Saroj Kaushik 25

 Summary: In order to provide a formal description of
a problem, it is necessary to do the following things:
 Define a state space that contains all the possible
configurations of the relevant objects.
 Specify one or more states within that space that describe
possible situations from which the problem solving process
may start. These states are called initial states.
 Specify one or more states that would be acceptable as
solutions to the problem called goal states.
 Specify a set of rules that describe the actions. Order of
application of the rules is called control strategy.
 Control strategy should cause motion towards a
solution.

12/02/24 Prof Saroj Kaushik 26

 Control Strategies
Control Strategy decides which rule to apply next during
the process of searching for a solution to a problem.
Requirements for a good Control Strategy
 It should cause motion
In water jug problem, if we apply a simple control strategy of
starting each time from the top of rule list and choose the first
applicable one, then we will never move towards solution.

 It should explore the solution space in a systematic manner

If we choose another control strategy, say, choose a rule
randomly from the applicable rules then definitely it causes
motion and eventually will lead to a solution. But one may
arrive to same state several times. This is because
control strategy is not systematic.

12/02/24 Prof Saroj Kaushik 27

 Systematic Control Strategies (Blind
searches)
Blind searches are exhaustive in nature.
These are uninformed searches.
If the problem is simple then
 any control strategy that causes motion and is systematic
will lead to an answer.
But in order to solve some real world problems,
 we must use a control strategy that is efficient.
Let us discuss these strategies using water jug
problem.
These may be applied to any search problem.

12/02/24 Prof Saroj Kaushik 28

 Breadth First Search – BFS : Water Jug
Problem
BFS
 Construct a tree with the initial state of the problem as its root.
 Generate all the offspring of the root by applying each of the applicable
rules to the initial state.
 For each leaf node, generate all its successors by applying all the rules
that are appropriate.
 Repeat this process till we find a solution, if it exists.

(0, 0)

(4, 0) (0, 3)

12/02/24 Prof Saroj Kaushik 29

 (0, 0)

(4, 0) (0, 3)

(4, 3) (0, 0) (1, 3) (4, 3) (0, 0) (3, 0)

12/02/24 Prof Saroj Kaushik 30

 Depth First Search - DFS
Here we pursue a single branch of the
tree until it yields a solution or some
pre-specified depth has reached.
If solution is not found then
 go back to immediate previous node and
 explore other branches in DF fashion.

Let us see the tree formation for water

jug problem using DFS

12/02/24 Prof Saroj Kaushik 31

 (0, 0) Start State

(4, 0)

(4, 3) (0, 0) (1, 3)

(0, 3) (4, 0)

 (4, 3)  (0, 0) (3, 0)

(3, 3) (0, 0) (0, 3)

 (3, 0)  (0, 3) (4, 2)

(0, 2) (3, 3) (4, 0)

 (4, 2)  (0, 0) (2, 0) Goal state

12/02/24 Prof Saroj Kaushik 32

Uninformed search strategies
Uninformed search strategies
Uninformed search
 No information about the number of steps
or the path cost from the current state to
the goal
 search the state space blindly

Informed search, or heuristic search

A cleverer strategy that searches toward
the goal, based on the information from the
current state so far
Uninformed search strategies
Breadth-first search
Dijkstra’s algorithm(Uniform cost

search)
Depth-first search
 Depth-limited search
 Iterative deepening search

Bidirectional search
 Breadth-first search
The root node is expanded first (FIFO)
All the nodes generated by the root
node are then expanded
And then their successors and so on
 Breadth First Search (BFS)
 It expands all the states one step away from the initial
state, then expands all states two steps from initial state,
then three steps etc., until a goal state is reached.
 It expands all nodes at a given depth before expanding any
nodes at a greater depth.
 All nodes at the same level are searched before going to
the next level down.
 For implementation, two lists called OPEN and CLOSED
are maintained.
 The OPEN list contains those states that are to be expanded
and CLOSED list keeps track of states already expanded.
 Here OPEN list is used as a queue.

12/02/24 37
 Algorithm (BFS)
Input: Two states in the state space START and GOAL
Local Variables: OPEN, CLOSED, STATE-X, SUCCS
Output: Yes or No
Method:
• Initially OPEN list contains a START node and CLOSED list is
empty; Found = false;
 While (OPEN  empty and Found = false)
Do {
 Remove the first state from OPEN and call it STATE-X;
 Put STATE-X in the front of CLOSED list;
 If STATE-X = GOAL then Found = true else
{- perform EXPAND operation on STATE-X,
producing a list of SUCCESSORS;
- Remove from successors those states, if any, that
are in the CLOSED list;
- Append SUCCESSORS at the end of the OPEN list
/*queue*/
} } /* end while */
 If Found = true then return Yes else return No and Stop
12/02/24 38
 Breadth-first search
S

A D

B D A E

C E E B B F
11

D F B F C E A C G
14 17 15 15 13
G C G F
19 19 17
G 25
 Breadth-First Strategy

New nodes are inserted at the end of FRINGE

1

2 3 FRINGE = (1)

4 5 6 7
 Breadth-First Strategy

New nodes are inserted at the end of FRINGE

1

2 3 FRINGE = (2, 3)

4 5 6 7
 Breadth-First Strategy

New nodes are inserted at the end of FRINGE

1

2 3 FRINGE = (3, 4, 5)

4 5 6 7
 Breadth-First Strategy

New nodes are inserted at the end of FRINGE

1

2 3 FRINGE = (4, 5, 6, 7)

4 5 6 7
Breadth-first search (Analysis)
Breadth-first search
 Complete – find the solution eventually
 Optimal, if step cost is 1 or (if all operators

have the same cost).

The disadvantage
 if the branching factor of a node is large,

 for even small instances (e.g., chess)(31 to 35

legal moves)
 the space complexity and the time complexity
are enormous
Properties of breadth-first search
Complete? Yes (if b is finite)

Time? 1+b+b2+b3+… +bd = b(bd-1) = O(bd+1)

Space? O(bd+1) (keeps every node in memory)

Optimal? Yes (if cost = 1 per step)

Space is the bigger problem (more than time)

Breadth-first search (Analysis)
assuming 10000 nodes can be processed per second, each
with 1000 bytes of storage
 Dijkstra’s algorithm(Uniform
cost search)
Breadth-first finds the shallowest goal state
 butnot necessarily be the least-cost solution
 work only if all step costs are equal

Uniform cost search

 modifies breadth-first strategy
 by always expanding the lowest-cost node
 The lowest-cost node is measured by the path
cost g(n)
 BEST-FIRSTSEARCH with PATH-COST as the

evaluation function
 Uniform cost search
the first found solution is guaranteed to be the cheapest
 least in depth

 But restrict to non-decreasing path cost

 Unsuitable for operators with negative cost

 Uniform-cost
Expand least-cost unexpanded node
search
Implementation:
 fringe = queue ordered by path cost

Equivalent to breadth-first if step costs all equal

Complete? Yes, if step cost ≥ ε
Time? # of nodes with g ≤ cost of optimal solution,
O(bceiling(C*/ ε)) where C* is the cost of the optimal solution
Space? # of nodes with g ≤ cost of optimal solution,
O(bceiling(C*/ ε))
Optimal? Yes – nodes expanded in increasing order of
g(n)
let
ε is possitive constant (every action cost) (lower bound on the
cost of each action)

C* be the cost of optimal solution.

 Depth-first search
Always expands one of the nodes at the
deepest level of the tree
Only when the search hits a dead end
 goes back and expands nodes at shallower levels
 Dead end  leaf nodes but not the goal

Backtracking search
 only one successor is generated on expansion
rather than all successors
 fewer memory
 Algorithms (DFS)

Input: Two states in the state space, START and GOAL

LOCAL Variables: OPEN, CLOSED, RECORD-X,
SUCCESSORS
Output: A path sequence if one exists, otherwise return No
Method:
 Form a stack consisting of (START, nil) and call it OPEN
list. Initially set CLOSED list as empty; Found = false;
 While (OPEN  empty and Found = false) DO
{
• Remove the first state from OPEN and call it RECORD-X;
• Put RECORD-X in the front of CLOSED list;
• If the state variable of RECORD-X= GOAL,
then Found = true
12/02/24 51
Else
{ - Perform EXPAND operation on STATE-X, a
state variable of RECORD-X, producing a list of
action records called SUCCESSORS; create each
action record by associating with each state its parent.
- Remove from SUCCESSORS any record whose
state variables are in the record already in the CLOSED
list.
- Insert SUCCESSORS in the front of the OPEN list
/* Stack */
}
}/* end while */
 If Found = true then return the plan used /* find it by tracing
through the pointers on the CLOSED list */ else return No
 Stop
12/02/24 52
 Depth-first search
Expand deepest unexpanded node
Implementation:
 fringe = LIFO queue, i.e., put successors at front

 Depth-first search
Expand deepest unexpanded node
Implementation:
 fringe = LIFO queue, i.e., put successors at front

 Depth-first search
Expand deepest unexpanded node
Implementation:
 fringe = LIFO queue, i.e., put successors at front

 Depth-first search
Expand deepest unexpanded node
Implementation:
 fringe = LIFO queue, i.e., put successors at front

 Depth-first search
Expand deepest unexpanded node
Implementation:
 fringe = LIFO queue, i.e., put successors at front

 Depth-first search
Expand deepest unexpanded node
Implementation:
 fringe = LIFO queue, i.e., put successors at front

 Depth-first search
Expand deepest unexpanded node
Implementation:
 fringe = LIFO queue, i.e., put successors at front
 Depth-first search
Expand deepest unexpanded node
Implementation:
 fringe = LIFO queue, i.e., put successors at front
 Depth-first search
Expand deepest unexpanded node
Implementation:
 fringe = LIFO queue, i.e., put successors at front

 Depth-first search
Expand deepest unexpanded node
Implementation:
 fringe = LIFO queue, i.e., put successors at front

 Depth-first search
Expand deepest unexpanded node
Implementation:
 fringe = LIFO queue, i.e., put successors at front

 Depth-first search
Expand deepest unexpanded node
Implementation:
 fringe = LIFO queue, i.e., put successors at front
 Depth-first search
S

A D

B D A E

C E E B B F
11

D F B F C E A C G
14 17 15 15 13
G C G F
19 19 17 G 25
Depth-first search (Analysis)
Not complete
 because a path may be infinite or looping
 then the path will never fail and go back try

another option
Not optimal
 it doesn't guarantee the best solution
It overcomes
 the time and space complexities
 Properties of depth-first search
Complete? No: fails in infinite-depth spaces,
spaces with loops
 Modify to avoid repeated states along path
 complete in finite spaces
Time? O(bm): terrible if m is much larger than d
 but if solutions are dense, may be much faster than
breadth-first
Space? O(bm), i.e., linear space!
Optimal? No


 Depth-Limited Strategy
Depth-first with depth cutoff k (maximal
depth below which nodes are not
expanded)
Three possible outcomes:
 Solution
 Failure (no solution)
 Cutoff (no solution within cutoff)
 Depth-limited search
It is depth-first search
 with a predefined maximum depth
 However, it is usually not easy to define
the suitable maximum depth
 too small  no solution can be found
 too large  the same problems are
suffered from
Anyway, the search is
 complete
 but still not optimal
Depth-limited
S
search
depth = 3
A D 3
6
B D A E

C E E B B F
11

D F B F C E A C G
14 17 15 15 13
G C G F
19 19 17
G 25
Iterative deepening search
No choosing of the best depth limit
It tries all possible depth limits:
 first
0, then 1, 2, and so on
 combines the benefits of depth-first and

breadth-first search
Iterative deepening search
 IDDFS Algorithm
procedure IDDFS(root)
for depth from 0 to ∞
 found ← DLS(root, depth)
if found ≠ null
return found

procedure DLS(node, depth) if depth = 0 and node is a goal

return node
else if depth > 0
For each child of node
found ← DLS(child, depth−1)
if found ≠ null return found return null
Iterative deepening search
(Analysis)
optimal
complete
Time and space complexities
 reasonable

suitable for the problem

 having a large search space
 and the depth of the solution is not known
Properties of iterative deepening
search
Complete? Yes
Time? (d+1)b0 + d b1 + (d-1)b2 + … + bd
= O(bd)
Space? O(bd)
Optimal? Yes, if step cost = 1
 Bidirectional search
Simultaneously searches forward from the initial state and
backwards from the goal state(s), the two searches will meet.
bd/2+ bd/2 is less than bd
The motivation is that is much less than (e.g., 50,000 times less
when ).
Bidirectional search for this to work, need to keep track of two
frontiers and two tables of reached states, and
need to be able to reason backwards: if state is a successor of
node in the forward direction, then need to know that is a
successor of node in the backward direction.
solution when the two frontiers collide.
 Bi-Directional Search
 For those problems having a single goal state and single
start state, bi-directional search can be used.
 It starts searching forward from initial state and backward
from the goal state simultaneously starting the states
generated until a common state is found on both search
frontiers.
 DFID can be applied to bi-directional search for k = 1,
2, .. as follows :
 kth iteration consists of a DFS from one direction to depth k
storing all states at depth k, and
 DFS from other direction : one to depth k and other to depth k+1
not storing states but simply matching against the stored states
from forward direction.
 The search to depth k+1 is necessary to find odd-length solutions.

12/02/24 77
Graph:
Start
1

2 3
4

5 6 7 8
9

10 11 12
13

14 15

16
12/02/24 Goal 78
 For k = 0

Start
1
______________

14 15

16
Goal

12/02/24 79
 For k = 1

Start
1

2 3 4
_________________________________

11 12 13

14 15

16
Goal
12/02/24 80
 For k = 2
Start
1

2 3 4

5 6 7 8 9
_____________________________________________

11

14

16
Goal
12/02/24 81
Comparing search strategies