Chapter 3

Search Algorithms
 Search Algorithms
 A search method is defined by picking the order of node expansion
 Strategies are evaluated along the following dimensions:
 completeness: does it always find a solution if one exists?
 time complexity: number of nodes generated
 space complexity: maximum number of nodes in memory
 optimality: does it always find the shortest path solution?
 Time and space complexity are measured in terms of
 b: maximum branching factor of the search tree
 d: depth of the shortest path solution
 m: maximum depth of the state space (may be
∞)
Uninformed Search
 A class of general purpose algorithms that operates in a brute
force way
 Search algorithms that explore a problem space without any
specific knowledge or information about the problem other than
the initial state and the possible actions to take.
 Also called blind search, or naïve search
 E.g. Random Search
 This method selects randomly a new state from the current
one
 If the goal state is reached, the search terminates
 Otherwise the methods randomly select an other operator to
move to the next state

 Prominent methods:  Depth-First Search  Breadth-First Search

Search Algorithms(Uninformed and informed
 Use an arbitrary ordering of operations to reach the goal
 Computationally (space and time) expensive
Search Algorithms
 Heuristic Search (informed search)
 Search algorithms that use knowledge to direct the
search such as how far we are from the goal, path
cost, how to reach to goal node, etc. This knowledge
help agents to explore less to the search space and
find more efficiently the goal node.
 We need this extra information to compute preference
among child nodes to explore and expand
 Each node has a heuristic function associated with it.
 Examples are Best First Search (BFS), A*, Hill Climbing,
Simulated Annealing, etc.
Search Algorithm
Begin
Open = initial state // open list is all generated states
// that have not been
“expanded”
While open not empty // one iteration of search algorithm
state = First(open) // current state is first state in open
Pop(open) // remove new current state from open
if Goal(state) // test current state for goal
condition
Return “succeed” // search is complete
else
// else expand the
current state by
// generating children
and
// reorder open list per
search strategy
open = QueueFunction(open, Expand(state))
Return “fail”
Search Strategies
 Search strategies differ only in QueuingFunction
 Features by which to compare search strategies
 Completeness (always find solution)
 Cost of search (time and space)
 Cost of solution, optimal solution
 Make use of knowledge of the domain
 “uninformed search” vs. “informed search”
Breadth-First Search
(BFS)
 Generate children of a state
 QueueingFn adds the children to the end
of the open list
 Level-by-level search
 Order in which children are inserted on
open list is arbitrary
 In tree, assume children are considered
left-to-right unless specified differently
 Number of children is “branching factor” b
BFS Example

Branch factor = b= 2, depth of tree = d

=2
BFS Example
BFS Example
 BFS Example Queue visited
1 S S

C B A S
3
2 A B C 4 D C B SA

E D C SAB
5 D 6 E F
7 F E D SABC

G F E SABCD
G
G F SABCDE
8
G SABCDEF

SABCDEFG
 Analysis
 Assume goal node at level d with constant branching factor b
 Time complexity (measured in #nodes generated)
 1+b+b2+b3+…+bd = =
O(bd+1)
 This assumes goal on far right of level
 Space complexity = O(bd+1)
 Exponential time and space
 Features
 Simple to implement
 Complete (always find a solution)
 Solution is not necessarily optimal
 Analysis
Depth (d) Nodes Time Memory
0.1111
3 1111 1 megabyte
seconds
 See what happens
with b=10 1.1111
4 11111 10 MB
seconds
 No. of Nodes =
6 106 1.66 min 1 GB
 expand 10,000
nodes/second 2.78
8 108 100 GB
hours
 1,000 bytes/node
11.57
10 1010 1 TB
days
3.17
12 1012 1 petabytes
years
3,170
15 1015 1 exabyte
years
Depth-First Search
(DFS)
 QueueingFn adds the children to the
front of the open list
 DFS emulates LIFO stack
 Follow leftmost path to bottom, then
backtrack
 Expand deepest node first
DFS Examples
https://www.youtube.com/watch?
v=0RKR3iYXy3Q&list=PLPBnj6azlABatXqkOgE4-Suu2ucfax42Fhttps://
www.youtube.com/watch?
v=Zt54zJmrAzs&list=PLPBnj6azlABatXqkOgE4-Suu2ucfax42F&index=2
DFS Examples
 Analysis
 Time complexity
 In the worst case, search entire space
 Goal may be at level d but tree may continue to level m, m>=d
 O(bm)
 Particularly bad if tree is infinitely deep
 Space complexity
 Only need to save one set of children at each level
 1 + b + b + … + b (m levels total) = O(bm)
 For previous example, DFS requires 120kb instead of 1 petabytes for d=12 (10 billion
times less)
 Benefits
 Simple to implement
 If many solutions, may find one quickly (quickly moves to depth d)
 Less Space, so more usable than BFS for large problems
 May not always find solution (not complete)
 Solution is not necessarily optimal
BFS vs DFS
BFS vs DFS
BFS vs DFS
 Comparison of Search Techniques
DFS BFS

Complete (find a solution) N (may be trapped Y

in infinite loop)
Optimal (least cost) N N

Time complexity bm bd+1

Space complexity bm bd+1

Data structure Stack (FILO) Queue (FIFO)

Memory usage less more

Speed May be faster slower

When to use Better when target Better when target is

is far from source close to source
 Uniform Cost Search (UCS)

 Uniform-cost search is an uninformed search algorithm that uses the lowest

cumulative cost to find a path from the source to the destination.
 Nodes are expanded, starting from the root, according to the minimum cumulative
cost. The uniform-cost search is then implemented using a Priority Queue.
 QueueingFn is Sort By CostSoFar = g(n)
 g(n)= Cost from root to current node n
 Add operator costs along path
 First goal found is least-cost solution (optimal)
 Space & time can be exponential because large subtrees with inexpensive steps
may be explored before useful paths with costly steps
 If costs are equal, time and space are O(bd)
 Otherwise, complexity related to cost of optimal solution
UCS Example

Open list: C
UCS Example

Open list: B(2) T(1) O(3) E(2) P(5)

UCS Example

Open list: T(1) B(2) E(2) O(3) P(5)

UCS Example

Open list: B(2) E(2) O(3) P(5)

UCS Example

Open list: E(2) O(3) P(5)

UCS Example

Open list: E(2) O(3) A(3) S(5) P(5) R(6)

UCS Example

Open list: O(3) A(3) S(5) P(5) R(6)

UCS Example

Open list: O(3) A(3) S(5) P(5) R(6) G(7)

UCS Example

Open list: A(3) S(5) P(5) R(6) G(7)

UCS Example

Open list: A(3) I(4) S(5) N(5) P(5) R(6) G(7)

UCS Example

Open list: I(4) P(5) S(5) N(5) R(6) G(7)

UCS Example

Open list: P(5) S(5) N(5) R(6) Z(6) G(7)

UCS Example

Open list: S(5) N(5) R(6) Z(6) F(6) G(7) D(8) L(10)
UCS Example

Open list: N(5) R(6) Z(6) F(6) G(7) D(8) L(10)

UCS Example

Open list: Z(6) F(6) G(7) D(8) L(10)

UCS Example

Open list: F(6) G(7) D(8) L(10)

UCS Example
Comparison of Blind Search Techniques

DFS BFS UCS

Complete N Y Y

Optimal N N Y

Time bm bd+1 bm

Space bm bd+1 bm
What are heuristics?
 A Heuristic is a technique to solve a problem faster than classic
methods, or to find an approximate (a good) solution in reasonable time
when classic methods cannot.
 The Heuristic is often effective but will not guarantee work in every
case and will not guarantee optimal solution for the problem
 Solution given by heuristic doesn’t have to be the best- an approximate
solution will do since it can be obtained fast enough.
Heuristic search
 Most search problem in AI are exponential (i.e., search space combinatorial
explode or exponentially increase with the length of the input n)
 These problems are called NP (Non-deterministic Polynomial) such as TSP
 Exhaustive search takes exponential time to find the optimal solution
 Exhaustive search systematically enumerates all possible candidates for the solution to
find the optimal solution
 Exact (deterministic) algorithm that finds the optimal solution in polynomial time
is not known
 A good solution can be found in polynomial time using heuristic (non-
deterministic) algorithm
 Informed (Heuristics) Searches
 At each branching step in the search algorithm, a Heuristic (or a heuristic function)
evaluates the available information and makes a decision on which branch to
follow. It does so by ranking alternatives.
 Best-first search, Hill climbing, Beam search, ….
 New parameters
 g(n) = estimated cost from initial state (root) to state n
 h(n) = estimated cost (distance) from state n to closest goal (G)
 h(n) is our heuristic
 Robot path planning, h(n) could be Euclidean distance
 8 puzzle, h(n) could be #tiles out of place

 Search algorithms which use h(n) to guide search are heuristic search algorithms
 Best First Search
1. Create 2 empty lists: OPEN and CLOSED
2. Start from the initial node (say N) and put it in the ‘ordered’ OPEN list
3. Repeat the next steps until GOAL node is reached
1. If OPEN list is empty, then EXIT the loop returning ‘False’
2. Select the first/top node (say N) in the OPEN list and move it to the CLOSED list.
3. If N is a GOAL node, then the loop returning ‘True’. The solution can be found by backtracking
the path
4. If N is not the GOAL node, expand node N to generate the ‘immediate’ next nodes linked to
node N and add all those to the OPEN list
5. Reorder the nodes in the OPEN list in ascending order according to a heuristic function h(n)
Greedy Best First Search

QueueingFn is sort-by f(n)=h(n), h(n) is the estimated distance to the

goal, choose the minimum after sorting them.
Example
Example
Example
Example
Example
Example
Example
Example
Comparison of Search Techniques

DFS BFS UCS Greedy

Best
First
Complete N Y Y N

Optimal N N Y N

Heuristic N N N Y

Time bm bd+1 bm bm

Space bm bd+1 bm bm
 Hill Climbing
 QueueingFn is sort-by-h (h is the estimated cost)
 Only keep lowest-h state on open list (keep the state that has the minimum cost)
 Best search is tentative
 Hill Climbing is irrecoverable (it is not able to track back or adaptable)
 Features
 Much faster
 Less memory
 Dependent upon h(n)
 If bad h(n), may prune (remove) away all goals
 Not complete
Example
Example
 Beam Search
 QueueingFn is sort-by-h
 Only keep best (lowest-h) n nodes on open list (keep n states with minimum cost in Open
list)
 n is the “beam width”
 If n = 1, Hill climbing
 If n = infinity, Best first search
Example
Example
Example
Example
Example
Example
Example
Example
Example
Comparison of Search Techniques

DFS BFS UCS Best HC Beam

Complete N Y Y N N N

Optimal N N Y N N N

Heuristic N N N Y Y Y

Time bm bd+1 bm bm bm nm

Space bm bd+1 bm bm b bn
 A* Search
 QueueingFn is sort-by-f
 f(n) = g(n) + h(n)
 g(n) is the cost from starting node to the current node
 H(n) is the cost from the current node to the goal
 Note that UCS and Best-first both improve search
 UCS keeps solution cost low
 Best-first helps find solution quickly
 A* combines these approaches
Example
Example
Example
Example
Example
Example
Example
Example
Comparison of Search Techniques

DFS BFS UCS Best HC Beam A*

first
Complete N Y Y N N N Y

Optimal N N Y N N N Y

Heuristic N N N Y Y Y Y

Time bm bd+1 bm bm bm nm bm

Space bm bd+1 bm bm b bn bm