FUNDAMENTALS

IN
ARTIFICIAL INTELLIGENCE & MACHINE
LEARNING
CSA2001

Dr. N. PAZHANIRAJA
 MODULE DESCRIPTION

UNIT II PROBLEM SOLVING METHODS

Problem Solving Methods - Search Strategies- Uninformed - Informed - Heuristics - Local Search
Algorithms And Optimization Problems - Searching With Partial Observations – Constraint
Satisfaction Problems – Constraint Propagation - Backtracking Search - Game Playing – Optimal
Decisions In Games – Alpha - Beta Pruning - Stochastic Games
 DATA STRUCTURE
• Specialized format for organizing, processing, retrieving and storing data.
STACK & QUEUE &
LINKED LIST
TREE
GRAPH
 TYPES OF SEARCHES
❑ Uninformed Search
❑ Breadth-First Search
❑ Uniform-Cost Search
❑ Depth-First Search
❑ Depth-Limited Search
❑ Iterative Deepening Search

❑ Informed Search
❑ Best First Search
❑ Greedy Best First Search
❑ Recursive Best First Search

❑ Heuristic Functions
 UNINFORMED SEARCH
❑ General-purpose search algorithms which operates in Brute Force-way.

❑ Do not have additional information about state or search space other than
how to traverse the tree

❑ Also called blind search

 BREADTH-FIRST SEARCH
▪ Most common search strategy for traversing a tree or graph.

▪ Searches breadthwise in a tree or graph, so it is called breadth-first search.

▪ Starts searching from the root node of the tree and expands all successor node
at the current level before moving to nodes of next level.

▪ Example of a General-graph search algorithm.

▪ Implemented using FIFO queue data structure.

 BREADTH-FIRST SEARCH
❑ ADVANTAGES:
❑ Provide a solution if any solution exists.
❑ If there are more than one solutions for a given problem, then bfs will provide the
minimal solution which requires the least number of steps.
❑ DISADVANTAGES:
❑ It requires lots of memory since each level of tree must be saved into memory to
expand the next level.
❑ Needs lots of time if the solution is far away from the root node.
Time Complexity: Obtained by number of nodes
traversed in BFS until the shallowest Node.
Where the d= depth of shallowest solution and b is
a node at every state.
T (b) = 1+b2+b3+.......+ b d = O (bd )
Space Complexity: Given by the Memory size of
frontier which is O(bd).
Completeness: Shallowest goal node is at some
finite depth, then BFS will find a solution.
 DEPTH-FIRST SEARCH
❑ Recursive algorithm for traversing a tree or graph data structure.

❑ It starts from root node and follows each path to its greatest depth node before
moving to the next path.

❑ Uses stack data structure for implementation.

❑ Process similar to BFS algorithm.

 DEPTH-FIRST SEARCH
❑ ADVANTAGE:
❑ Requires very less memory as it only needs to store a stack of the nodes on the
path from root node to the current node.
❑ Less time to reach to the goal node than BFS algorithm (if it traverses in the right
path).
❑ DISADVANTAGE:
❑ There is possibility that many states keep re-occurring, and there is no guarantee
of finding the solution.
❑ Algorithm goes for deep down searching and sometime it may go to the infinite
loop.
Completeness: complete within finite state space as it will
expand every node within a limited search tree.

Time Complexity: Will be equivalent to node traversed by the

algorithm.

T(n)= 1+ n2+ n3 +.........+ nm=O(nm)

m= maximum depth of any node and this can be much larger

than d (Shallowest solution depth)

Space Complexity: Store only single path from the root node,
hence space complexity of DFS is equivalent to the size of the
fringe set, which is O(bm).
 DEPTH-LIMITED SEARCH ALGORITHM
❑ Similar to depth-first search with a predetermined limit.
❑ Solve drawback of infinite path in the depth-first search.
❑ Node at depth limit will treat as it has no successor nodes further.
❑ Terminated with two conditions of failure:
❑ Standard failure value:
❑ it indicates that problem does not have any solution.

❑ Cutoff failure value:

❑ it defines no solution for the problem within a given depth limit.
 DEPTH-LIMITED SEARCH ALGORITHM
• ADVANTAGES:

• Memory efficient.

• DISADVANTAGES:

• Incompleteness.

• It may not be optimal if the problem has more than one solution.
Completeness: complete if the solution is above the
depth-limit.

Time Complexity: Time complexity is O(bℓ).

Space Complexity: Space complexity is O(b×ℓ).

 UNIFORM-COST SEARCH ALGORITHM
❑ Used for traversing a weighted tree or graph.
❑ Play when a different cost is available for each edge.
❑ Find a path to goal node which has lowest cumulative cost.
❑ Search expands nodes according to their path costs form root node.
❑ Used to solve any graph/tree where optimal cost is demand.
❑ Algorithm is implemented by the priority queue.
❑ Gives maximum priority to the lowest cumulative cost.
❑ Equivalent to BFS algorithm if the path cost of all edges is the same.
 UNIFORM-COST SEARCH ALGORITHM
❑ ADVANTAGES:

❑ Uniform cost search is optimal because at every state the path with the
least cost is chosen.

❑ DISADVANTAGES:

❑ Doesn't care about number of steps involve in searching and only

concerned about path cost.

❑ Due to which algorithm may be stuck in an infinite loop.

Completeness:
complete, such as if there is a solution, UCS will find it.
Time Complexity:
Let C* is Cost of the optimal solution, and ε is each
step to get closer to the goal node. Then the number
of steps is = C*/ε+1. Here we have taken 1, as we
start from state 0 and end to C*/ε.
Hence, the worst-case time complexity of Uniform-
cost search isO(b1 + C*/ε]).
Space Complexity:
The same logic is for space complexity so, the worst-
case space complexity of Uniform-cost search is O(b1
+ C*/ε]
).
 ITERATIVE DEEPENING DEPTH-FIRST SEARCH
❑ Combination of DFS and BFS algorithms.
❑ Finds out the best depth limit and does it by gradually increasing the limit until a goal is
found.
❑ Performs depth-first search up to a certain "depth limit", and it keeps increasing the
depth limit after each iteration until the goal node is found.
❑ Combines benefits of breadth-first search's fast search and depth-first search's
memory efficiency.
❑ Useful uninformed search when search space is large, and depth of goal node is
unknown.
 ITERATIVE DEEPENING DEPTH-FIRST SEARCH
❑ ADVANTAGES:
❑ It combines benefits of BFS and DFS search algorithm in terms of fast search and
memory efficiency.

❑ DISADVANTAGES:
❑ The main drawback of IDDFS is that it repeats all the work of the previous phase.
1'st Iteration-----> A
2'nd Iteration----> A, B, C
3'rd Iteration------>A, B, D, E, C, F, G
4'th Iteration------>A, B, D, H, I, E, C, F, K, G
In fourth iteration, algorithm will find goal node.

Completeness:
This algorithm is complete is ifthe branching factor is
finite.
Time Complexity:
Let's suppose b is the branching factor and depth is d
then the worst-case time complexity is O(bd ).
Space Complexity:
Space complexity of IDDFS will be O(bd).
 BIDIRECTIONAL SEARCH ALGORITHM
❑ Runs two simultaneous searches
❑ One form initial state called as forward-search and other from goal node called
as backward-search, to find the goal node.
❑ Bidirectional search replaces one single search graph with two small subgraphs
in which one starts the search from an initial vertex and other starts from goal
vertex.
❑ The search stops when these two graphs intersect each other.
❑ Bidirectional search can use search techniques such as BFS, DFS, DLS, etc.
 BIDIRECTIONAL SEARCH ALGORITHM
❑ ADVANTAGES:

❑ Bidirectional search is fast.

❑ Bidirectional search requires less memory

❑ DISADVANTAGES:

❑ Implementation of the bidirectional search tree is difficult.

❑ In bidirectional search, one should know the goal state in advance.

Completeness: complete if we use BFS in both
searches.
Time Complexity: Time complexity of
bidirectional search using BFS is O(bd).
Space Complexity: Space complexity of
bidirectional search is O(bd).
 INFORMED SEARCH
❑ Contain information regarding the goal state.

❑ Contains an array of knowledge such as how far we are from the goal, path cost,
how to reach to goal node, etc.

❑ Knowledge help agents to explore less the search space and find more efficiently
goal node.

❑ Useful for large search space.

❑ Uses idea of heuristic, thereby called Heuristic search.

 HEURISTICS FUNCTION
❑ Function that ranks different in search algorithms at each branching step based on
available information to decide which branch to follow.

❑ Estimates how close a state is to the goal.

❑ Represented by h(n), and it calculates the cost of an optimal path between the pair of
states.

❑ Heuristic function is always positive.

❑ h(n) <= h*(n)

❑ h(n) heuristic cost & h*(n) estimated cost.

❑ Heuristic cost should be less than or equal to the estimated cost.

 PURE HEURISTIC SEARCH
❑ Simplest form of heuristic search algorithms.

❑ Expands nodes based on their heuristic value h(n).

❑ Maintains two lists, OPEN and CLOSED list.

❑ CLOSED list, it places those nodes which have already expanded.

❑ OPEN list, it places nodes which have yet not been expanded, estimates how close a
state is to the goal.

❑ On each iteration, each node n with the lowest heuristic value is expanded and
generates all its successors and n is placed to the closed list.
 INFORMED SEARCH ALGORITHMS

❑ BEST FIRST SEARCH ALGORITHM(GREEDY SEARCH)

❑ A* SEARCH ALGORITHM
BEST-FIRST SEARCH ALGORITHM (GREEDY SEARCH)
❑ Selects path which appears best at that moment.

❑ Combination of DFS and BFS algorithms.

❑ Uses heuristic function and search.

❑ Choose most promising node.

❑ We expand node which is closest to goal node and closest cost is estimated
by heuristic function f(n)= g(n).
 BEST-FIRST SEARCH ALGORITHM
Step 1: Place the starting node into the OPEN list.
Step 2: If the open list is empty, stop and return failure.
Step 3: Remove the node n, from the open list which has the lowest value of h(n), and places it
in the closed list.
Step 4: Expand the node n, and generate the successors of node n.
Step 5: Check each successor of node n, and find whether any node is a goal node or not. If any
successor node is goal node, then return success and terminate the search, else proceed to step
6.
Step 6: For each successor node, algorithm checks for evaluation function f(n), and then check if
the node has been in either open or closed list. If the node has not been in both list, then add it
to the OPEN list.
Step 7: Return to step 2.
 BEST-FIRST SEARCH ALGORITHM
❑ ADVANTAGES:
❑ Switch between BFS and DFS by gaining the advantages of both the algorithms.
❑ More efficient than BFS and DFS algorithms.
❑ DISADVANTAGES:
❑ Behave as an unguided depth-first search in the worst case scenario.
❑ Get stuck in a loop as DFS.
❑ Not optimal.
Expand the nodes of S and put in the CLOSED list
Initialization: Open [A, B], Closed [S]
Iteration 1: Open [A], Closed [S, B]
Iteration 2: Open [E, F, A], Closed [S, B]
: Open [E, A], Closed [S, B, F]
Iteration 3: Open [I, G, E, A], Closed [S, B, F]
: Open [I, E, A], Closed [S, B, F, G]
Final solution path will be: S----> B----->F----> G
 BEST-FIRST SEARCH ALGORITHM
❑ TIME COMPLEXITY:
❑ The worst case time complexity of greedy best first search is o(bm).

❑ SPACE COMPLEXITY:
❑ The worst case space complexity of greedy best first search is o(bm).
❑ Where, m is the maximum depth of the search space.

❑ COMPLETE:
❑ Incomplete, even if the given state space is finite.

❑ OPTIMAL: Not optimal.

 A* SEARCH ALGORITHM
❑ COMMONLY KNOWN FORM OF BEST FIRST SEARCH.

❑ USES HEURISTIC FUNCTION H N , AND COST TO REACH THE NODE N FROM

THE START STATE G N . 

❑ COMBINED FEATURES OF UCS AND GREEDY BEST FIRST SEARCH

❑ FINDS THE SHORTEST PATH THROUGH THE SEARCH SPACE USING THE
HEURISTIC FUNCTION.

❑ EXPANDS LESS SEARCH TREE AND PROVIDES OPTIMAL RESULT FASTER.

❑ SIMILAR TO UCS EXCEPT THAT IT USES G N H N INSTEAD OF G N .

A* SEARCH ALGORITHM
 A* SEARCH ALGORITHM
STEP1: Place the starting node in the OPEN list.

STEP 2: Check if the OPEN list is empty or not, if the list is empty then return failure and stops.

STEP 3: Select the node from the OPEN list which has the smallest value of evaluation function (g+h), if
node n is goal node then return success and stop, otherwise

STEP 4: Expand node n and generate all of its successors, and put n into the closed list. For each successor
n', check whether n' is already in the OPEN or CLOSED list, if not then compute evaluation function for n'
and place into open list.

STEP 5: Else if node n' is already in OPEN and CLOSED, then it should be attached to the back pointer
which reflects the lowest g(n') value.

STEP 6: Return to step 2.

 A* SEARCH ALGORITHM
❑ ADVANTAGES:
❑ A* search algorithm is the best algorithm than other search algorithms.

❑ A* search algorithm is optimal and complete.

❑ This algorithm can solve very complex problems.

❑ DISADVANTAGES:
❑ It does not always produce the shortest path as it mostly based on heuristics and approximation.

❑ A* search algorithm has some complexity issues.

❑ The main drawback of a* is memory requirement as it keeps all generated nodes in the memory, so it
is not practical for various large-scale problems.
 A* SEARCH ALGORITHM

Initialization:   S, 5
Iteration1   S A, 4 , S G, 10
Iteration2   S A C, 4 , S A B, 7 , S G,
10
Iteration3   S A C G, 6 , S A C D,
11 , S A B, 7 , S G, 10
Iteration 4 will give the final result, as S A
C G it provides the optimal path with cost 6.