Harambee University

Department of Computer Science

Chapter Three
Problem solving

Kebebe.S (MSc)

1
Objectives
 Problem Solving by Searching
 What is search and terminologies?
 Search Strategies
 Unformed Search Strategies
o DFS, BFS, Uniform cost search, DLS, Iterative
Deeping DFS, and Bi-directional search.
 Informed Search Strategies
o A star search, Best first search
 Local Search Strategies
o Hill climbing search, Means ends analysis,
Simulated annealing
2
 Problem Solving by Searching
What is search and terminologies?
 Searching is a step by step procedure to solve a search-
problem in a given search space.
 A search problem can have three main factors:
o Search Space: Search space represents a set of possible
solutions, which a system may have.
o Start State: It is a state from where agent begins the
search.
o Goal test: It is a function which observe the current
state and returns whether the goal state is achieved or
not.
 Search tree: A tree representation of search problem is
called Search tree.

3
 What is search and terminologies?...
 Actions: It gives the description of all the available actions
to the agent.
 Transition model: A description of what each action do, can
be represented as a transition model.
 Path Cost: It is a function which assigns a numeric cost to
each path.
 Solution: It is an action sequence which leads from the
start node to the goal node.
 Optimal Solution: If a solution has the lowest cost among
all solutions.

4
 Search Strategies
Unformed Search Strategies
 The uninformed search does not contain any domain
knowledge such as closeness, the location of the goal. It is
also called level order traversal.
 It operates in a brute-force way as it only includes
information about how to traverse the tree and how to
identify leaf and goal nodes.
 Uninformed search applies a way in which search tree is
searched without any information about the search space
like initial state operators and test for the goal, so it is also
called blind search.
 It examines each node of the tree until it achieves the goal
node.

5
 Types Of Uninformed Search Algorithms
 Six kinds of such search strategies will be
discussed and each depends on the order of
expansion of successor nodes.
1. Breadth-first search
2. Uniform-cost search
3. Depth-first search
4. Depth-limited search
5. Iterative deepening search
6. Bidirectional search

6
 Measuring problem-solving performance
 Completeness: Is the algorithm guaranteed to find a
solution when there is one?
 Optimality: Does the strategy find the optimal solution?
(optimal solution: the solution that has the lowest path-cost
among all solutions).
 Time complexity: How long does it take to find a solution?
 Space complexity: How much memory is needed to
perform the search?
 Complexity is represented in terms of three quantities:
b: The branching factor, or maximum number of successors of any node.
d: The depth of the shallowest goal node.
m: The maximum length of any path in the state space.

7
1. Breadth-first search (BFS)
 BFS algorithm starts searching from the root node of the
tree and expands all successor node at the current level
before moving to nodes of next level.
 It searches breadthwise in a tree or graph, so it is called
breadth-first search.
 Breadth-first search implemented using FIFO queue data
structure.
 If branching factor (average number of child nodes for a
given node) = b and depth = d, then number of nodes at
level d = bd.
 The total no of nodes created in worst case is
b + b2 + b3 + … + bd.

8
BFS (Cont’d…)
Advantage of BFS
 BFS will provide a solution if any solution exists.
 If there are more than one solution for a given problem,
then BFS will provide the minimal solution which requires
the least number of steps.
Disadvantage of BFS
 It requires lots of memory since each level of the tree must
be saved into memory to expand the next level.
 BFS needs lots of time if the solution is far away from the
root node.

9
BFS (Cont’d…)
Example:

10
BFS (Cont’d…)
 Time Complexity: is obtained by the number of nodes
traversed in BFS until the shallowest Node.
 T (b) =O(bd) where d= depth of shallowest solution
and b is a node at every state.
 Space Complexity: Space complexity of BFS algorithm is
given by the Memory size of frontier which is O(bd)
 Completeness: BFS is complete, which means if the
shallowest goal node is at some finite depth, then BFS will
find a solution.
 Optimality: BFS is optimal if path cost is a non-decreasing
function of the depth of the node.

11
2. Depth-First Search (DFS)
 Depth-first search is a recursive algorithm for traversing a
tree or graph data structure.
 It is called the depth-first search because it starts from the
root node and follows each path to its greatest depth node
before moving to the next path.
 DFS uses a stack data structure for its implementation.
 In a stack (LIFO Principles), elements are added and
removed from the top of the stack.
Note:
 Backtracking is an algorithm technique for finding all
possible solutions using recursion.
 It traverses a tree from Root node--->Left node ---- > right
node.

12
 Advantage and Disadvantage of DFS
Advantage:
o DFS requires very less memory as it only needs to store a
stack of the nodes on the path from root node to the
current node.
o It takes less time to reach to the goal node than BFS
algorithm (if it traverses in the right path).
Disadvantage:
o There is the possibility that many states keep re-occurring,
and there is no guarantee of finding the solution.
o DFS algorithm goes for deep down searching and sometime
it may go to the infinite loop.

13
DFS (Cont’d…)
Example

 Completeness: DFS search algorithm is complete within

finite state space as it will expand every node within a
limited search tree.

14
DFS (Cont’d…)
 Completeness: DFS search algorithm is complete within
finite state space as it will expand every node within a
limited search tree.
 Time Complexity: Time complexity of DFS will be equivalent
to the node traversed by the algorithm. T(n)=O(nm),
where m= maximum depth of any node and this can be
much larger than d (Shallowest solution depth)
 Space Complexity: DFS algorithm needs to store only single
path from the root node, hence space of DFS is equivalent
to the size of the fringe set, which is O(bm).
 Optimal: DFS search algorithm is non-optimal, as it may
generate a large number of steps or high cost to reach to
the goal node.

15
3. Depth limited search
 A depth-limited search algorithm is similar to depth-first
search with a predetermined limit.
 Depth- limited search can solve the drawback of the infinite
path in the Depth-first search.
 In this algorithm, the node at the depth limit will treat as it
has no successor nodes further.
 Depth-limited search can be terminated with two
Conditions of failure:
a. Standard failure value: It indicates that problem does
not have any solution.
b. Cutoff failure value: It defines no solution for the
problem within a given depth limit.
Advantages:
 Depth-limited search is Memory efficient.
16
3. Depth limited search (Cont’d…)
Disadvantages:
Depth-limited search also has a disadvantage of
incompleteness. It may not be optimal if the problem has
more than one solution.
Example: Find the path to get the goal ‗J‘ if depth limit is 2

17
3. Depth limited search (Cont’d…)
 Completeness: DLS search algorithm is complete if the
solution is above the depth-limit.
 Time Complexity: Time complexity of DLS algorithm is
O(bℓ).
 Space Complexity: Space complexity of DLS algorithm is
O(b×ℓ).
 Optimal: Depth-limited search can be viewed as a special
case of DFS, and it is also not optimal even if ℓ>d.

18
4. Uniform cost search
 Uniform-cost search is a searching algorithm used for
traversing a weighted tree or graph.
 This algorithm comes into play when a different cost is
available for each edge.
 The primary goal of the uniform-cost search is to find a
path to the goal node which has the lowest cumulative
cost.
 Uniform-cost search expands nodes according to their path
costs form the root node.
 It can be used to solve any graph/tree where the optimal
cost is in demand.
 A uniform-cost search algorithm is implemented by the
priority queue.

19
4. Uniform cost search (Cont’d…)
 It gives maximum priority to the lowest cumulative cost.
 Uniform cost search is equivalent to BFS algorithm if the
path cost of all edges is the same.
Advantages:
 Uniform cost search is optimal because at every state the
path with the least cost is chosen.
Disadvantages:
 It does not care about the number of steps involve in
searching and only concerned about path cost.
 Due to which this algorithm may be stuck in an infinite
loop.

20
4. Uniform cost search (Cont’d…)
Example : Find the goal node G‘ for the following search tree

21
4. Uniform cost search (Cont’d…)
 Completeness - Uniform-cost search is complete, such as if
there is a solution, UCS will find it.
 Time Complexity - Let C* is Cost of the optimal solution,
and ε is each step to get closer to the goal node. Then the
number of steps is = C*/ε+1. Here we have taken +1, as
we start from state 0 and end to C*/ε. Hence, the worst-
case time complexity of Uniform-cost search is O(b1 +
[C*/ε])/.
 Space Complexity - The same logic is for space complexity
so, the worst-case space complexity of Uniform-cost search
is O(b1 + [C*/ε]).
 Optimal - Uniform-cost search is always optimal as it only
selects a path with the lowest path cost.

22
5. Iterative Deeping DFS
 It is a combination of DFS and BFS algorithms.
 It finds out the best depth limit and does it by gradually
increasing the limit until a goal is found.
 This algorithm performs depth-first search up to a certain
"depth limit", and it keeps increasing the depth limit after
each iteration until the goal node is found.
 It the benefits of Breadth-first search's fast search and
depth-first search's memory efficiency.
 The iterative search algorithm is useful uninformed search
when search space is large, and depth of goal node is
unknown.

23
5. Iterative Deeping DFS (Cont’d…)
Advantages:
 It combines the benefits of BFS and DFS search algorithm
in terms of fast search and memory efficiency.
Disadvantages:
 The main drawback of IDDFS is that it repeats all the work
of the previous phase.

24
5. Iterative Deeping DFS (Cont’d…)
Example: find the goal node G

Solution
1'st Iteration ---- > A
2'nd Iteration --- > A, B, C
3'rd Iteration ----- >A, B, D, E, C, F, G
In the third iteration, the algorithm will find the goal node.
25
5. Iterative Deeping DFS (Cont’d…)
 Completeness - This algorithm is complete if the branching
factor is finite.
 Time Complexity - Let's suppose b is the branching factor
and depth is d then the worst-case time complexity is
O(bd).
 Space Complexity - The space complexity of IDDFS will be
O(bd).
 Optimal - IDDFS algorithm is optimal if path cost is a non-
decreasing function of the depth of the node.

26
6. Bidirectional search
 Bidirectional search algorithm runs two simultaneous
searches, one form initial state called as forward-search
and other from goal node called as backward-search, to
find the goal node.
 Bidirectional search replaces one single search graph with
two small sub-graphs in which one starts the search from
an initial vertex and other starts from goal vertex.
 The search stops when these two graphs intersect each
other.
 Bidirectional search can use search techniques such as BFS,
DFS, DLS, etc.

27
6. Bidirectional search (Cont’d…)
Advantage:
 Bidirectional search is fast.
 Bidirectional search requires less memory
Disadvantages:
 Implementation of the bidirectional search tree is difficult.
 In bidirectional search, one should know the goal state in
advance.

28
6. Bidirectional search (Cont’d…)
Example: Find the path to get the goal node 16‘

29
6. Bidirectional search (Cont’d…)
 Completeness: Bidirectional Search is complete if we use
BFS in both searches.
 Time Complexity: Time complexity of bidirectional search
using BFS is O(bd).
 Space Complexity: Space complexity of bidirectional search
is O(bd).
 Optimal: Bidirectional search is Optimal.

30
Informed Search Strategies

31