Chapter Three

Solving Problem By Searching

1
 Problem Solving by Search
ØProblem Solving by Searching
vProblem Solving Agents:Searching problem
vProblem Formulations and Goal formulations
vSearch Strategies
ü Uniformed-Search strategy: blind search
ü Informed-search strategy: heuristic search
vAvoiding Repeated States
ØProblem solving by graph
ØConstraint Satisfaction Search
ØProblem solving by Game playing
2
 Problem solving
• Problem solving is an important aspect of Artificial
Intelligence.
• A problem can be considered to consist of a goal and a
set of actions that can be taken to lead to the goal.
• many problems can be viewed as reaching a goal state
from a given starting point
• At any given time, we consider the state of the search
space to represent where we have reached as a result of
the actions we have applied so far.

3
 Problem ….
• problems with a readily available formal specification
• initial state
• starting point from which the agent sets out
• actions (operators, successor functions)
• describe the set of possible actions
• state space
• set of all states reachable from the initial state by any sequence of actions
• path
• sequence of actions leading from one state in the state space to another
• goal test
• determines if a given state is the goal state

4
Problem….
• solution
• path from the initial state to a goal state
• search cost
• time and memory required to calculate a solution
• path cost
• determines the expenses of the agent for executing the actions in a
path
• sum of the costs of the individual actions in a path
• total cost
• sum of search cost and path cost
• overall cost for finding a solution

5
Exploration Aksum
100

200
Mekele
Gondar 80
180
Lalibela
110 250
150
Bahr dar
Dessie
170

Debre markos 330

Dire Dawa
230

400
330
Jima
Addis Ababa
100
430 Nazarez 370

Gambela 230 320 Nekemt

Awasa 6
Road map Ethiopia
Solution
• Current position of the agent: Awasa.
• Needs to arrive to: Gondar
• Formulate goal:
• be in Gondar
• Formulate problem:
• states: various cities
• actions: drive between cities
• Find a solution:
• sequence of cities, e.g., Awasa, Nazarez, Addis Ababa,
Dessie, Gondar

7
3 cat & 3 rat puzzle (class work)
 Three cat and three mice come to a crocodile infested river. There is a boat on their
sides that can be used by one or two “persons”. If cats outnumber the mice at any
time, the cats eat the mice. How can they use the boat to cross the river so that all
mice survive.
 State description
 [#of cats to the left side,
#of mice to the left side,
boat location,
#of cats to the right side,
#of mice to the right side]
 Initial state
 [3,3,Left,0,0]
 Goal
 [0,0,Right,3,3]

8
Search
• Search is a method that can be used by computers
to examine a problem space in order to find a goal.
• A problem space can also be considered to be a
search space because in order to solve the
problem, we will search the space for a goal state.

9
 Searching Defined
• Examine different possible sequences of actions & states,
and come up with specific optimal sequence of actions
that will take you from the initial state to the goal state
–Given a state space with initial state and goal state, find optimal
sequence of actions leading through a sequence of states to the
final goal state

• Searching in State Space

– select optimal (min-cost/max-profit) node/state in the state
space
– test whether the state selected is the goal state or not
– if not, expand the node further to identify its successor.

10
 Search algorithm
•Two functions needed for conducting search
– Generator (or successors) function: Given a state and action,
produces its successor states (in a state space)
– Tester (or IsGoal) function: Tells whether given state S is a goal state
IsGoal(S)  True/False
üIsGoal and Successors functions depend on problem domain.
•Two lists maintained during searching
–OPEN list: stores the nodes expanded but not explored
–CLOSED list: the nodes expanded and explored
üGenerally search proceeds by examining each node on the OPEN list;
performing some expansion operation that adds its children to the
OPEN list, & moving the node to the CLOSED list
• Merge function: Given successor nodes, it either append, prepend
to open list or rearrange based on evaluation cost
• Path cost: function assigning a numeric cost to each path; either
from initial node to current node and/or from current node to goal
node 11
 Search algorithm
• Input: a given problem (Initial State + Goal state + transit states and
operators)
• Output: returns optimal sequence of actions to reach the goal.
function GeneralSearch (problem, strategy)
open = (initialState); //put initial state in the List
closed = {}; //maintain list of nodes examined earlier
while (not (empty (open)))
f = remove_first(open);
if IsGoal (f) then return (f);
closed = append (closed, f);
succ = Successors (f)
left = not-in-closed (succ, closed );
open = merge (rest(open), left); //append or prepend l to open list
end while
return ('fail')
end GeneralSearch 12
 Algorithm Evaluation: Completeness
Optimality, Time and Space Complexity
• Completeness
– Is the algorithm guarantees in finding a solution whenever one
exists
– Think about the density of solutions in space and evaluate
whether the searching technique guaranteed to find all solutions
or not.
• Optimality
– is the algorithm finding an optimal solution; i.e. the one with
minimize cost or maximize profit
• how good is our solution?

13
Algorithm Evaluation: Time & Space Tradeoffs

• With many computing projects, we worry about:

– Speed versus memory
– Time complexity: how long does it take to find a solution
– Space complexity: how much space is used by the algorithm
• Fast programs can be written
– But they use up too much memory
• Memory efficient programs can be written
– But they are slow
• We consider various search strategies
– In terms of their memory/speed tradeoffs
14
 Searching Strategies
• Search strategy gives the order in which the search space is
examined
• Uninformed (= blind) search
• they do not need domain knowledge that guide them towards the
goal
• Have no information about the number of steps or the path cost
from the current state to the goal
• It is important for problems for which there is no additional
information to consider

• Informed (= heuristic) search

• have problem-specific knowledge (knowledge that is true from
experience)
• Have knowledge about how far are the various state from the goal
• Can find solutions more efficiently than uninformed search
15
Search Methods:

16
 Breadth first search
•Expand shallowest unexpanded node,
–i.e. expand all nodes on a given level of
the search tree before moving to the next
level
•Implementation: use queue data
structure to store the list:
–Expansion: put successors at the end of
queue
–Pop nodes from the front of the queue
–fringe is a FIFO queue, i.e. successors go
in at the end of the queue
•Properties:
– Takes space: keeps every node in memory
– Optimal and complete: guarantees to
find solution 17
BFS
• All the nodes reachable from the current node
are explored first
• achieved by the TREE-SEARCH method by appending
newly generated nodes at the end of the search
queue
• Time Complexity: bd+1 where , b=branching factor and d
depth of the tree
• Space Complexity: bd+1
• Completeness: yes (for finite b)
• Optimality: yes (for non-negative path costs)

18
19
 Exercise
• Apply BFS to find an optimal path from start node to
Goal node.

20
Depth-first search
• Depth-first search is so called because it follows
each path to its greatest depth before moving on to
the next path.

21
 Depth-first search
•Expand one of the node at the deepest
level of the tree.
•Only when the search hits a non-goal dead end
does the search go back and expand nodes at
shallower levels
•Implementation: treat the list as stack
•Expansion: push successors at the top of stack
•Pop nodes from the top of the stack
•fringe is a LIFO queue (a stack), i.e. successors
go in at front of queue
•Properties
•Incomplete and not optimal: fails in infinite-
depth spaces, spaces with loops.
• Modify to avoid repeated states along the path
•Takes less space (Linear): Only needs to
remember up to the depth expanded
22
 Algorithm for Depth first search
function DFS (problem){
open = (C_0); //put initial state C_0 in the List
closed = {}; /maintain list of nodes examined earlier
while (not (empty (open))) {
f = remove_first(open);
if IsGoal (f) then return (f);
closed = append (closed, f);
l = not-in-set (Successors (f), closed );
open = merge ( rest(open), l); //prepend to the list
}
return ('fail')
}

23
 Uniform cost Search
•The goal of this technique is to find the shortest path to the goal
in terms of cost.
• It modifies the BFS by always expanding least-cost unexpanded
node
• Expand least-cost unexpanded node (costs added up over paths from root to
leafs)
•Implementation: nodes in list keep track of total path length
from start to that node
• List kept in priority queue ordered by path cost
• fringe is queue ordered by increasing path cost
•Properties:
• This strategy finds the cheapest solution provided the cost of a
path must never decrease as we go along the path
g(successor(n)) ≥ g(n), for every node n
• Takes space since it keeps every node in memory
• Equivalent to depth-first search if all step costs are equal
24
 Examples
A

S S S
1 10
S
B
5 5
S G 0
A B C A B C A B C

1 5 15 5 15 15

G G G
15 5
11 11 10
C

25
 Algorithm for Uniform Cost search
function uniform_cost (problem){
open = (C_0); //put initial state C_0 in the List
g(s) = 0;
closed = {}; /maintain list of nodes examined earlier
while (not (empty (open))) {
f = remove_first(open);
if IsGoal (f) then return (f);
closed = append (closed, f);
l = not-in-set (Successors (f), closed );
g(f,li) = g(f) + c(f,li);
open = merge(rest(open), l, g(f,li)); //keep the open list sorted in
ascending order by edge cost
}
return ('fail')
}
26
 Iterative Deepening Search (IDS)
•IDS solves the issue of choosing the best depth limit by trying all
possible depth limit:
–Perform depth-first search to a bounded depth d, starting at d = 1 and increasing
it by 1 at each iteration.
• It is Depth-limited search or Depth-first search with depth limit or with ever
increasing limits
•It combines the benefits of DFS and BFS
–DFS is efficient in space, but has no path-length guarantee
–BFS finds min-step path towards the goal, but requires memory space
–IDS performs a sequence of DFS searches with increasing depth-cutoff until goal
is found
Examples:
Limit=0 Limit=1 Limit=2

27
Iterative deepening search with depth limit 0-3

Limit = 2

Limit = 3

28
Algorithm for IDS
function IDS (problem){
open = (C_0); //put initial state C_0 in the List
closed = {}; /maintain list of nodes examined earlier
while (not reached maxDepth) {
while (not (empty (open))) {
f = remove_first(open);
if (IsGoal (f)) then return (f);
closed = append (closed, f);
l = not-in-set (Successors (f), closed);
if (depth(l) < maxDepth) then
open = merge (rest(open), l); //prepend to the list
}
}
return ('fail')
}
29
 Bidirectional Search
• Simultaneously search both forward from the initial state
to the goal and backward from the goal to the initial state,
and stop when the two searches meet somewhere in the
middle
• Requires an explicit goal state and invertible operators (or
backward chaining).
• Decide what kind of search is going to take place in each half
using BFS, DFS, uniform cost search, etc.

30
Bidirectional Search
• Advantages:
• Only need to go to half depth
• It can enormously reduce time complexity, but is not
always applicable

• Difficulties
• Do you really know solution? Unique?
• Cannot reverse operators
• Memory requirements may be important: Record all paths
to check they meet
• Memory intensive

31
Exercise: Uninformed Search Strategies
• Assume that node 3 is the initial state and node 5 is
the goal state

32
Exercise: Apply Uninformed Search
Strategies to identify optimal path

S
1 5 8

A B C
3 9
7 4 5
D E G

33
Informed Search
• Best First Search
• Local Search
• Constraint satisfaction problem

34
Best First Search
• It is a generic name to the class of informed methods
• The two best first approaches to find the shortest path:
– Greedy search: minimizes estimated cost to reach a goal
– A*-search: minimizes the total path cost
• When expanding a node n in the search tree, greedy search
uses the estimated cost to get from the current state to the
goal state, defined as h(n).
– In route finding problem h(n) is the straight-line distance
• We also possess the sum of the cost to reach that node
from the start state, defined as g(n).
– In route finding problem; this is the sum of the step costs for the
search path.
• For each node in the search tree, A*-search uses an
evaluation function, f(n):
f(n) = g(n) + h(n) 35
Greedy Search
• A best first search that uses a heuristic function h(n) alone to
guide the search
• Selects node to expand that is closest (hence it’s greedy) to a goal
node
• The algorithm doesn’t take minimum path costs from initial to current
nodes into account, it just go ahead optimistically and never looks
back.
• Implementation:
• expand 1st the node closest to the goal state, i.e. with evaluation
function f(n) = h(n)
• h(n) = 0 if node n is the goal state
• Otherwise h(n) ≥ 0; an estimated cost of the cheapest path from the
state at node n to a goal state

Property
Not optimal and complete
Fast

36
Example

37
 A* Search
• It considers both estimated cost of getting from n to the goal node
h(n), and cost of getting from initial node to node n, g(n)
• avoid expanding paths that are already expensive
• Apply three functions over every nodes
• Evaluation function f(n) = g(n) + h(n)
1. g(n) = Cost of path found so far from initial state to n or to reach n
2. h(n) = Estimated cost of shortest path from n to z or to goal from n
3. f(n) = estimated total cost of path through n to goal or Estimated
total cost of shortest path from a to z via n
• A∗ search uses an admissible heuristic i.e., h(n) =< h∗ (n) where
h∗ (n) is the true cost from n to a goal.
• (Also require h(n) = 0, so h(G) = 0 for any goal G.)
• E.g., h(n) never overestimates the actual road distance
• Theorem: A∗ search is optimal if h is admissible.

38
 A* Search
• Implementation: Expand the node for which the evaluation function f(n) is lowest
–Rank nodes by f(n) that goes from the start node to goal node via given node
Example: Route finding problem
Properties:
– A* is complete whenever the branching factor is finite, and every
operator has a fixed positive cost
– When the condition h(n) <= h*(n) holds, we say that h is admissible.
– A* is admissible
– If h(n) = h*(n) for all n, then only the nodes on the optimal solution path
will be expanded. So, no extra work will be performed.
– If h(n) = 0 for all n, then this is an admissible heuristic and results in A*
performing exactly as the Uniform-Cost Search does
– If h1(n) < h2(n) <= h*(n) for all n that are not goal nodes, then h2 is
a better heuristic than h1. this means A1* expands at least as many
nodes as A2*. We say that A2* is better informed than A1*.
– The closer h is to h*, the fewer extra nodes that will be expanded

39
Thank You!!

40