INTRODUCTION TO

ARTIFICIAL INTELLIGENCE

Mahesh Madhushan
B.Sc. in ICT (RUSL), B.Eng. (Hons) in Software Engineering (UK), M.Sc. in IT (UOP)
 INTRODUCTION TO
ARTIFICIAL INTELLIGENCE

Session 06 – Problem solving by Searching

 SEARCH
• INTELLIGENT AGENTS: PROBLEM SOLVING AS SEARCH
• SEARCH PROBLEM:
• STATES (ABSTRACTION OF THE WORLD)
• ACTIONS (AND COSTS)
• SUCCESSOR FUNCTION (WORLD DYNAMICS):
• START STATE AND GOAL TEST

• A SEARCH TREE IS AN EFFECTIVE WAY TO REPRESENT THE

SEARCH PROCESS

• THERE ARE A VARIETY OF SEARCH ALGORITHMS,

INCLUDING
• DEPTH-FIRST SEARCH
• BREADTH-FIRST SEARCH ETC.
 SEARCHING FOR PATHS
• SEARCHING FOR A PATH FROM ONE VERTEX TO ANOTHER:
• SOMETIMES, WE JUST WANT ANY PATH (OR WANT TO KNOW THERE IS A PATH).
• SOMETIMES, WE WANT TO MINIMIZE PATH LENGTH (# OF EDGES).
• SOMETIMES, WE WANT TO MINIMIZE PATH COST (SUM OF EDGE WEIGHTS).

• WHAT IS THE SHORTEST PATH FROM MIA TO SFO?

WHICH PATH HAS THE MINIMUM COST?
PVD
ORD
SFO

LGA
HNL
LAX
DFW
$500 MIA
 SEARCH STRATEGIES
• UNINFORMED SEARCH (BLIND SEARCH STRATEGIES)
• UNINFORMED SEARCH STRATEGIES USE NO INFORMATION
ABOUT THE LIKELY “DIRECTION” OF THE GOAL NODE(S)
– BREADTH FIRST SEARCH
– DEPTH FIRST SEARCH
• INFORMED SEARCH (HEURISTIC SEARCH STRATEGIES)
•KNOW WHETHER ONE NON-GOAL STATE IS ―MORE
PROMISING‖ THAN ANOTHER
– A* SEARCH
 UNINFORMED SEARCH STRATEGIES
• UNINFORMED: WHILE SEARCHING YOU HAVE NO CLUE WHETHER
ONE NON-GOAL STATE IS BETTER THAN ANY OTHER. YOUR
SEARCH IS BLIND. YOU DON’T KNOW IF YOUR CURRENT
EXPLORATION IS LIKELY TO BE FRUITFUL.
• VARIOUS BLIND STRATEGIES:
• BREADTH-FIRST SEARCH
• UNIFORM-COST SEARCH
• DEPTH-FIRST SEARCH 6
EXAMPLE FOR ILLUSTRATING UNINFORMED
SEARCH STRATEGIES

S
3 1 8

A B C
3 15
7 20 5
D E G
BREADTH-FIRST SEARCH
 BREADTH-FIRST SEARCH

Strategy: expand a a G
shallowest node first b c
Implementation: Fringe e
d f
is a FIFO queue S h
p q r

d e p
Search
b c e h r q
Tiers
a a h r p q f

p q f q c G

q c G a

a
 BREADTH-FIRST SEARCH
• BREADTH-FIRST SEARCH (BFS): FINDS A PATH BETWEEN TWO NODES BY
TAKING ONE STEP DOWN ALL PATHS AND THEN IMMEDIATELY
BACKTRACKING.
• OFTEN IMPLEMENTED BY MAINTAINING A QUEUE OF VERTICES TO VISIT.

• BFS ALWAYS RETURNS THE SHORTEST PATH (THE ONE WITH THE FEWEST
EDGES) BETWEEN THE START AND THE END VERTICES.
• TO B: {A, B} a b c
• TO C: {A, E, F, C}
• TO D: {A, D} d e f
• TO E: {A, E}
• TO F: {A, E, F} g h
• TO G: {A, D, G}
• TO H: {A, D, H}
 BREADTH-FIRST SEARCH
• EXPAND SHALLOWEST UNEXPANDED NODE
• FRINGE: NODES WAITING IN A QUEUE TO BE EXPLORED, ALSO
CALLED OPEN
• IMPLEMENTATION:
• FRINGE IS A FIRST-IN-FIRST-OUT (FIFO) QUEUE, I.E., NEW SUCCESSORS GO AT END OF THE QUEUE.

Is A a goal state?

11
 BREADTH-FIRST SEARCH

• EXPAND SHALLOWEST UNEXPANDED NODE

• IMPLEMENTATION:
– FRINGE IS A FIFO QUEUE, I.E., NEW SUCCESSORS GO AT END

Expand:
fringe = [B,C]

Is B a goal state?

12
 BREADTH-FIRST SEARCH

• EXPAND SHALLOWEST UNEXPANDED NODE

•
• IMPLEMENTATION:
• FRINGE IS A FIFO QUEUE, I.E., NEW SUCCESSORS GO AT END
Expand:
fringe=[C,D,E]

Is C a goal state?

13
 BREADTH-FIRST SEARCH

• EXPAND SHALLOWEST UNEXPANDED NODE

• IMPLEMENTATION:
• FRINGE IS A FIFO QUEUE, I.E., NEW SUCCESSORS GO AT END

Expand:
fringe=[D,E,F,G]

Is D a goal state?

14
Example
BFS

15
 EXAMPLE: MAP NAVIGATION
A B C

S G

D E F

S = start, G = goal, other nodes = intermediate states, links = legal transitions

16
 INITIAL BFS SEARCH TREE
S
A D

B D E

C E
A B C

S G

D E F

Note: this is the search tree at some particular point in

in the search. 17
 SEARCH ALGORITHM PROPERTIES
• COMPLETE: GUARANTEED TO FIND A SOLUTION IF ONE EXISTS?
• RETURN IN FINITE TIME IF NOT?

• OPTIMAL: GUARANTEED TO FIND THE LEAST COST PATH?

• TIME COMPLEXITY?
1 node
b
• SPACE COMPLEXITY? … b nodes
b2 nodes
• CARTOON OF SEARCH TREE:
m tiers
• B IS THE BRANCHING FACTOR
• M IS THE MAXIMUM DEPTH
• SOLUTIONS AT VARIOUS DEPTHS

bm nodes
• NUMBER OF NODES IN ENTIRE TREE?
• 1 + B + B2 + …. BM = O(BM)
 BREADTH-FIRST SEARCH (BFS) PROPERTIES
• WHAT NODES DOES BFS EXPAND?
• PROCESSES ALL NODES ABOVE SHALLOWEST SOLUTION 1 node
• LET DEPTH OF SHALLOWEST SOLUTION BE S b
… b nodes
• SEARCH TAKES TIME O(BS) s tiers
b2 nodes
• HOW MUCH SPACE DOES THE FRINGE TAKE?
• HAS ROUGHLY THE LAST TIER, SO O(BS) bs nodes

• IS IT COMPLETE?
• S MUST BE FINITE IF A SOLUTION EXISTS, SO YES! (IF NO bm nodes
SOLUTION, STILL NEED DEPTH != ∞)

• IS IT OPTIMAL?
• ONLY IF COSTS ARE ALL 1 (MORE ON COSTS LATER)
 What is the Complexity of Breadth-First Search?

• Time Complexity
– assume (worst case) that there is 1 goal leaf at the
RHS d=0
– so BFS will expand all nodes
d=1
= 1 + b + b2+ ......... + bd
= O (bd) d=2
G

• Space Complexity
– how many nodes can be in the queue (worst-case)? d=0
– at depth d there are bd unexpanded nodes in the Q =
O (bd) d=1

d=2
Time and space of number of generated nodes is O
(b^(d+1)) G
20
• Properties of breadth-first search

• Complete? Yes (if b is finite)

• Time? 1+b+b2+b3+… +bd + b(bd -1) = O(bd)
• Space? O(bd) (keeps every node in memory)
• Optimal? Yes (if cost = 1 per step)
• (Can) expand each node once (if checks for duplicates)

• Space is the bigger problem (more than time)

21

• Drawback: requires exponential space

 EXAMPLES OF TIME AND MEMORY
REQUIREMENTS FOR BREADTH-FIRST SEARCH

Depth of Nodes
Solution Expanded Time Memory

0 1 1 millisecond 100 bytes

2 111 0.1 seconds 11 kbytes

4 11,111 11 seconds 1 megabyte

8 108 31 hours 11 giabytes

12 1012 35 years 111 terabytes

Assuming b=10, 1000 nodes/sec, 100 bytes/node

22
DEPTH-FIRST SEARCH
 DEPTH-FIRST SEARCH
Strategy: expand a a G
deepest node first b c

Implementation: e
d f
Fringe is a LIFO stack S h
p q r

d e p

b c e h r q

a a h r p q f

p q f q c G

q c G a

a
 DEPTH-FIRST SEARCH
• DEPTH-FIRST SEARCH (DFS): FINDS A PATH BETWEEN TWO VERTICES BY
EXPLORING EACH POSSIBLE PATH AS FAR AS POSSIBLE BEFORE
BACKTRACKING.
• OFTEN IMPLEMENTED RECURSIVELY.
• MANY GRAPH ALGORITHMS INVOLVE VISITING OR MARKING
VERTICES.

• DEPTH-FIRST PATHS FROM A TO ALL VERTICES (ASSUMING ABC EDGE

ORDER): a b c
• TO B: {A, B}
• TO C: {A, B, E, F, C} d e f
• TO D: {A, D}
• TO E: {A, B, E} g h
• TO F: {A, B, E, F}
• TO G: {A, D, G}
• TO H: {A, D, G, H}
 DEPTH-FIRST-SEARCH
1. PUT THE START NODE S ON OPEN
2. IF OPEN IS EMPTY EXIT WITH FAILURE.
3. REMOVE THE FIRST NODE N FROM OPEN AND PLACE IT ON
CLOSED.
4. IF N IS A GOAL NODE, EXIT SUCCESSFULLY WITH THE SOLUTION
OBTAINED BY TRACING BACK POINTERS FROM N TO S.
5. OTHERWISE, EXPAND N, GENERATING ALL ITS SUCCESSORS
ATTACH TO THEM POINTERS BACK TO N, AND PUT THEM AT THE
TOP OF OPEN IN SOME ORDER.
6. GO TO STEP 2.
26
 DEPTH-FIRST SEARCH
• EXPAND DEEPEST UNEXPANDED NODE
• IMPLEMENTATION:
• FRINGE = LAST IN FIRST OUT (LIPO) QUEUE, I.E., PUT SUCCESSORS AT
FRONT

Is A a goal state?

27
 DEPTH-FIRST SEARCH
• EXPAND DEEPEST UNEXPANDED NODE
• IMPLEMENTATION:
• FRINGE = LIFO QUEUE, I.E., PUT SUCCESSORS AT FRONT
queue=[B,C]

Is B a goal state?

28
 DEPTH-FIRST SEARCH
• EXPAND DEEPEST UNEXPANDED NODE
• IMPLEMENTATION:
• FRINGE = LIFO QUEUE, I.E., PUT SUCCESSORS AT FRONT
queue=[D,E,C]

Is D = goal state?

29
 DEPTH-FIRST SEARCH
• EXPAND DEEPEST UNEXPANDED NODE
• IMPLEMENTATION:
• FRINGE = LIFO QUEUE, I.E., PUT SUCCESSORS AT FRONT
queue=[H,I,E,C]

Is H = goal state?

30
 DEPTH-FIRST SEARCH
• EXPAND DEEPEST UNEXPANDED NODE
• IMPLEMENTATION:
• FRINGE = LIFO QUEUE, I.E., PUT SUCCESSORS AT FRONT

queue=[I,E,C]

Is I = goal state?

31
 DEPTH-FIRST SEARCH
• EXPAND DEEPEST UNEXPANDED NODE
• IMPLEMENTATION:
• FRINGE = LIFO QUEUE, I.E., PUT SUCCESSORS AT FRONT

queue=[E,C]

Is E = goal state?

32
 DEPTH-FIRST SEARCH

• EXPAND DEEPEST UNEXPANDED NODE

• IMPLEMENTATION:
queue=[J,K,C]
• FRINGE = LIFO QUEUE, I.E., PUT SUCCESSORS AT FRONT
Is J = goal state?

33
 DEPTH-FIRST SEARCH
• EXPAND DEEPEST UNEXPANDED NODE
• IMPLEMENTATION:
• FRINGE = LIFO QUEUE, I.E., PUT SUCCESSORS AT FRONT

queue=[K,C]

Is K = goal state?

34
 DEPTH-FIRST SEARCH
• EXPAND DEEPEST UNEXPANDED NODE
• IMPLEMENTATION:
• FRINGE = LIFO QUEUE, I.E., PUT SUCCESSORS AT FRONT
queue=[C]

Is C = goal state?

35
 DEPTH-FIRST SEARCH

• EXPAND DEEPEST UNEXPANDED NODE

• IMPLEMENTATION:
• FRINGE = LIFO QUEUE, I.E., PUT SUCCESSORS AT FRONT
queue=[F,G]

Is F = goal state?

36
 DEPTH-FIRST SEARCH

• EXPAND DEEPEST UNEXPANDED NODE

• IMPLEMENTATION:
• FRINGE = LIFO QUEUE, I.E., PUT SUCCESSORS AT FRONT
queue=[L,M,G]

Is L = goal state?

37
 DEPTH-FIRST SEARCH
• EXPAND DEEPEST UNEXPANDED NODE
• IMPLEMENTATION:
• FRINGE = LIFO QUEUE, I.E., PUT SUCCESSORS AT FRONT

queue=[M,G]

Is M = goal state?

38
Example DFS

39
40
SEARCH METHOD 2: DEPTH FIRST SEARCH (DFS)
S
A D
A B C
B D
S G

D E F
C E

Here, to avoid repeated states

assume we don’t expand any
D F child node which appears
already in the path from the root
S to the parent. (Again, one
could use other strategies) 41
G
SEARCH ALGORITHM PROPERTIES
 DEPTH-FIRST SEARCH (DFS) PROPERTIES
• WHAT NODES DFS EXPAND?
• SOME LEFT PREFIX OF THE TREE.
1 node
• COULD PROCESS THE WHOLE TREE! b
… b nodes
• IF M IS FINITE, TAKES TIME O(BM)
b2 nodes
• HOW MUCH SPACE DOES THE FRINGE TAKE? m tiers
• ONLY HAS SIBLINGS ON PATH TO ROOT, SO O(BM)

• IS IT COMPLETE?
• M COULD BE INFINITE, SO ONLY IF WE PREVENT CYCLES bm nodes
(MORE LATER)

• IS IT OPTIMAL?
• NO, IT FINDS THE “LEFTMOST” SOLUTION, REGARDLESS OF
DEPTH OR COST
 WHAT IS THE COMPLEXITY OF DEPTH-FIRST SEARCH?
• TIME COMPLEXITY d=0
• ASSUME (WORST CASE) THAT THERE IS 1
GOAL LEAF AT THE RHS d=1
• SO DFS WILL EXPAND ALL NODES d=2
(M IS CUTOFF) G
=1 + B + B2+ ......... + B^M
= O (B^M)
d=0

d=1
• SPACE COMPLEXITY
• HOW MANY NODES CAN BE IN THE d=2
QUEUE (WORST-CASE)?
d=3
• AT DEPTH L < D WE HAVE B-1 NODES
• AT DEPTH D WE HAVE B NODES d=4 44

• TOTAL = (M-1)*(B-1) + B = O(BM)

 A
PROPERTIES OF DEPTH-FIRST SEARCH
B C

• COMPLETE? NO: FAILS IN INFINITE-DEPTH SPACES

CAN MODIFY TO AVOID REPEATED STATES ALONG PATH

• TIME? O(BM) WITH M=MAXIMUM DEPTH

• TERRIBLE IF M IS MUCH LARGER THAN D
• BUT IF SOLUTIONS ARE DENSE, MAY BE MUCH FASTER THAN
BREADTH-FIRST

• SPACE? O(BM), I.E., LINEAR SPACE! (WE ONLY NEED TO

REMEMBER A SINGLE PATH + EXPANDED UNEXPLORED NODES)

• OPTIMAL? NO (IT MAY FIND A NON-OPTIMAL GOAL FIRST)

45
QUIZ: DFS VS BFS
 COMPARING DFS AND BFS

• SAME WORST-CASE TIME COMPLEXITY, BUT

• IN THE WORST-CASE BFS IS ALWAYS BETTER THAN DFS
• SOMETIME, ON THE AVERAGE DFS IS BETTER IF:
• MANY GOALS, NO LOOPS AND NO INFINITE PATHS

• BFS IS MUCH WORSE MEMORY-WISE

• DFS IS LINEAR SPACE
• BFS MAY STORE THE WHOLE SEARCH SPACE.

• IN GENERAL
• BFS IS BETTER IF GOAL IS NOT DEEP, IF INFINITE PATHS, IF MANY LOOPS, IF SMALL SEARCH SPACE
• DFS IS BETTER IF MANY GOALS, NOT MANY LOOPS,
• DFS IS MUCH BETTER IN TERMS OF MEMORY

47
 DFS VS BFS

• WHEN WILL BFS OUTPERFORM DFS?

• WHEN WILL DFS OUTPERFORM BFS?

 COST-SENSITIVE SEARCH
a GOAL

b c

e
d
f
START h

p r
q
 COST-SENSITIVE SEARCH
a GOAL
2 2
b c
3
2
1 8
2 e
3 d
f
9 8 2
START h
1 4 2

p 4 r
15
q

BFS FINDS THE SHORTEST PATH IN TERMS OF NUMBER OF ACTIONS.

IT DOES NOT FIND THE LEAST-COST PATH. WE WILL NOW COVER
How?
A SIMILAR ALGORITHM WHICH DOES FIND THE LEAST-COST PATH.
UNIFORM COST SEARCH
 UNIFORM COST SEARCH
2 a G
Strategy: expand a cheapest b c
node first: 1 8 2
2 e
3 d f
Fringe is a priority queue 9 2
S h 8
(priority: cumulative cost) 1
1 p q r
15

S 0

d 3 e 9 p 1

b 4 c e 5 h 17 r 11 q 16
11
Cost a 6 a h 13 r 7 p q f
contours
p q f 8 q c G

q 11 c G 10 a

a
 UNIFORM COST SEARCH
• EXPAND LOWEST-COST OPEN NODE (G(N))
• IN BFS G(N) = DEPTH(N)

53
 Requirement
 g(successor)(n))  g(n)
 FIND MINIMUM COST PATH
6 1
3 A D F 1

2 4 8
S B E G

1 20
C

The graph above shows the step-costs for different paths going from the start (S) to
the goal (G). On the right you find the straight-line distances.

Use uniform cost search to find the optimal path to the goal.

54
 UNIFORM COST SEARCH

1. PUT THE START NODE S ON OPEN

2. IF OPEN IS EMPTY EXIT WITH FAILURE.
3. REMOVE THE FIRST NODE N FROM OPEN AND PLACE IT ON
CLOSED.
4. IF N IS A GOAL NODE, EXIT SUCCESSFULLY WITH THE SOLUTION
OBTAINED BY TRACING BACK POINTERS FROM N TO S.
5. OTHERWISE, EXPAND N, GENERATING ALL ITS SUCCESSORS
ATTACH TO THEM POINTERS BACK TO N, AND PUT THEM AT THE END
OF OPEN IN ORDER OF SHORTEST COST
6.GO TO STEP 2. 55
 UNIFORM-COST SEARCH
Implementation: fringe = queue ordered by path cost
Equivalent to breadth-first if all step costs all equal.

Complete? Yes, if step cost ≥ ε

(otherwise it can get stuck in infinite loops)

Time? # of nodes with path cost ≤ cost of optimal solution.

Space? # of nodes on paths with path cost ≤ cost of optimal

solution.

Optimal? Yes, for any step cost. 56

 UNIFORM COST ISSUES
• REMEMBER: UCS EXPLORES INCREASING
COST CONTOURS … c1
c2
c3
• THE GOOD: UCS IS COMPLETE AND
OPTIMAL!

• THE BAD:
• EXPLORES OPTIONS IN EVERY “DIRECTION”
• NO INFORMATION ABOUT GOAL LOCATION
Start Goal

• WE’LL FIX THAT SOON!

[Demo: empty grid UCS (L2D5)]
[Demo: maze with deep/shallow water
DFS/BFS/UCS (L2D7)]
COMPARISON OF ALGORITHMS

58
QUESTIONS?

THANK YOU