Artificial Intelligence (IT)

UNIT-2
Search Algorithms
Search Algorithm teaches computers to “act rationally” by achieving a certain goal with a
certain input value

Search: Searching is a step by step procedure to solve a search-problem in a given search space.
A search problem can have three main factors:
 Search Space: Search space represents a set of possible solutions, which a system may
have.
 Start State: It is a state from where agent begins the search.
 Goal test: It is a function which observe the current state and returns whether the goal
state is achieved or not.

Search tree: A tree representation of search problem is called Search tree. The root of the search
tree is the root node which is corresponding to the initial state.

Search Trees
(a) The initial state Arad

Sibiu Timisoara Zerind

Arad Fagaras Oradea Rimnicu Vilcea Arad Lugoj Arad Oradea

(b) After expanding Arad Arad

Sibiu Timisoara Zerind

Arad Fagaras Oradea Rimnicu Vilcea Arad Lugoj Arad Oradea

(c) After expanding Sibiu Arad

Sibiu Timisoara Zerind

Arad Fagaras Oradea Rimnicu Vilcea Arad Lugoj Arad Oradea

Partial search trees for finding a route from Arad to Bucharest

function TREE-SEARCH(problem) returns a solution, or failure

initialize the frontier using the initial state of problem
 loop do
if the frontier is empty then return failure
choose a leaf node and remove it from the frontier
if the node contains a goal state then return the corresponding solution
expand the chosen node, adding the resulting nodes to the frontier
function GRAPH-SEARCH(problem) returns a solution, or failure
initialize the frontier using the initial state of problem
initialize the explored set to be empty
loop do
if the frontier is empty then return failure
choose a leaf node and remove it from the frontier
if the node contains a goal state then return the corresponding solution
add the node to the explored set
expand the chosen node, adding the resulting nodes to the frontier
only if not in the frontier or explored set

An informal description of the general tree-search and graph-search algorithms.

Properties of Search Algorithms:

Following are the four essential properties of search algorithms to compare the efficiency of
these algorithms:
 Completeness: A search algorithm is said to be complete if it guarantees to return a solution
if at least any solution exists for any random input.
 Optimality: If a solution found for an algorithm is guaranteed to be the best solution (lowest
path cost) among all other solutions, then such a solution for is said to be an optimal
solution.
 Time Complexity: Time complexity is a measure of time for an algorithm to complete its
task.
 Space Complexity: It is the maximum storage space required at any point during the search,
as the complexity of the problem.

Types of search algorithms:

Based on the search problems we can classify the search algorithms into uninformed (Blind
search) search and Informed search (Heuristic search) algorithms.
Uninformed/Blind Search:
 The uninformed search does not contain any domain knowledge such as closeness, the
location of the goal. It operates in a brute-force way as it only includes information about
how to traverse the tree and how to identify leaf and goal nodes.
 Uninformed search applies a way in which search tree is searched without any information
about the search space like initial state operators and test for the goal, so it is also called
blind search. It examines each node of the tree until it achieves the goal node.
 So Distance to goal not taken into account
 Ex: DFS, BFS, Iterative deepening DFS
Search with open and closed list
 Open List: Contains all nodes waiting to be expanded. And the search algorithm which
we selected will only clarify the order of child insertion into the list.

 Closed List: Contains all nodes already expanded. The closed list can be used to make
sure you don’t check the same node twice. So Closed list generally improves the speed
of algorithm.

Generic search algorithm

All searches essentially follow the same algorithm:
1) create an empty list called "closed set"; this list will contain states that have been visited
2) create a list called "open set" that contains just the starting state; this list contains states that
have not been visited but are known to exist
3) create an empty map (key/value pairs) called "parents"; this map contains the previous state
of each state that has been visited
4) while the open set is not empty:
a) grab a state from the open set (and remove it); put it in the closed set (since we're now
looking at it)
b) if that state is the goal, hey we're done!
i) return a reconstructed path from the goal state to the start state (this is easy:
recursively grab parent states from the "parents" map)
c) it's not the goal state; for each next state that is accessible from here:
i. if this next state is in the closed set (it has been visited before), ignore it
ii. if this next state is not in the open set, put it in the open set and record its parent d.
(repeat the loop).
5) if the open set is empty and we never found the goal

Step 4.a. is a key step. The "algorithm" above gives no guidance about how to choose the
next state to evaluate

Random Search
 A random search, while never practical as far as I know, is nevertheless interesting to
consider as a worst case search.
 To program this search, modify step 4.a. so that a random state is chosen.
 Given enough time, the goal state should be found, but a random search will probably
take more time (more states are checked) than any other method.

Random Search Algorithm

All searches essentially follow the same algorithm:
1) create an empty list called "closed set"; this list will contain states that have been visited
2) create a list called "open set" that contains just the starting state; this list contains states that
have not been visited but are known to exist
3) create an empty map (key/value pairs) called "parents"; this map contains the previous state
of each state that has been visited
4) while the open set is not empty:
a) grab a random state from the open set (and remove it); put it in the closed set (since
we're now looking at it)
b) if that random state is the goal, hey we're done!
i) return a reconstructed path from the goal state to the start state (this is easy:
recursively grab parent states from the "parents" map)
c) it's not the goal state; for each next random state that is accessible from here:
i. if this next random state is in the closed set (it has been visited before), ignore it
ii. if this next random state is not in the open set, put it in the open set and record
its parent d. (repeat the loop).
5) if the open set is empty and we never found the goal

Breadth First Search (BFS)

 Breadth-first search is a simple strategy in which the root node is expanded first, then all
the successors of the root node are expanded next, then their successors, and so on.
 In general, all the nodes are expanded at a given depth in the search tree before any nodes
at the next level are expanded.
 Breadth-first search is an instance of the general graph-search algorithm in which the
shallowest unexpanded node is chosen for expansion.
 This is achieved very simply by using a FIFO queue for the frontier. Thus, new nodes
(which are always deeper than their parents) go to the back of the queue, and old nodes,
which are shallower than the new nodes , get expanded first.

function BREADTH-FIRST-SEARCH(problem) returns a solution, or failure

node ←a node with STATE = problem . INITIAL-STATE, PATH-COST = 0
if problem . GOAL-TEST(node. STATE) then return SOLUTION(node)
frontier ←a FIFO queue with node as the only element
explored ←an empty set
loop do
if EMPTY?( frontier) then return failure
node ← POP( frontier ) /* chooses the shallowest node in frontier */
add node . STATE to explored
for each action in problem . ACTIONS(node. STATE) do
child ←CHILD-NODE(problem, node, action)
if child .STATE is not in explored or frontier then
if problem. GOAL-TEST(child. STATE) then return SOLUTION(child )
frontier ←INSERT(child , frontier )
BFS Attributes:
 Completeness – yes (b < ∞ , d < ∞)
 Optimality – yes, if graph is un-weighted.
 Time Complexity: O(1 + b + b 2 + ... + b d +1 − b) = O(b d +1 )
 Memory Complexity: O(b d +1 )
Where b is branching factor and d is the solution depth

Depth First Search (DFS)

 Depth-first search always expands DEPTH-FIRST the deepest node in the current
frontier of the search tree.
 The search proceeds immediately to the deepest level of the search tree, where the nodes
have no successors.
 As those nodes are expanded, they are dropped from the frontier, so then the search
“backs up” to the next deepest node that still has unexplored successors.
 The depth-first search algorithm is an instance of the graph-search algorithm , breadth-
first-search uses a FIFO queue, depth-first search uses a LIFO queue.
 A LIFO queue means that the most recently generated node is chosen for expansion.
 This must be the deepest unexpanded node because it is one deeper than its parent—
which, in turn, was the deepest unexpanded node when it was selected.
DFS Attributes:
 Completeness – No. Infinite loops or Infinite depth can occur.
 Optimality – No.
 Time Complexity: O(bm)
 Memory Complexity: O(bm)
Where b is branching factor and m is the maximum depth of search tree

Informed Search:
 Informed search algorithms use domain knowledge. In an informed search, problem
information is available which can guide the search. Informed search strategies can find a
solution more efficiently than an uninformed search strategy. Informed search is also
called a Heuristic search.
 A heuristic is a way which might not always be guaranteed for best solutions but
guaranteed to find a good solution in reasonable time.
 Informed search can solve much complex problem which could not be solved in another
way.
 The informed search algorithm is more useful for large search space. Informed search
algorithm uses the idea of heuristics, so it is also called Heuristic search.
 So Information about cost to goal taken into account
 Ex: Best first search , A* search
Heuristics function: Heuristic is a function which is used in Informed Search, and it finds the
most promising path.
 It takes the current state of the agent as its input and produces the estimation of how close
agent is from the goal.
 The heuristic method, however, might not always give the best solution, but it guaranteed
to find a good solution in reasonable time.
 Heuristic function estimates how close a state is to the goal. It is represented by h(n), and
it calculates the cost of an optimal path between the pair of states.
 The value of the heuristic function is always positive, it is also called Heuristic search.
 Admissibility of the heuristic function is given as:
h(n) <= h*(n)
Here h(n) is heuristic cost, and h*(n) is the estimated cost. Hence heuristic cost should
be less than or equal to the estimated cost.

Pure Heuristic Search

 Pure heuristic search is the simplest form of heuristic search algorithms. It expands nodes
based on their heuristic value h(n). It maintains two lists, OPEN and CLOSED list. In the
CLOSED list, it places those nodes which have already expanded and in the OPEN list, it
places nodes which have yet not been expanded.
 On each iteration, each node n with the lowest heuristic value is expanded and generates
all its successors and n is placed to the closed list. The algorithm continues unit a goal
state is found.
 In the informed search we will discuss two main algorithms which are given below:
o Best First Search Algorithm (Greedy search)
o A* Search Algorithm

Best-first Search Algorithm (Greedy Search)

 Greedy best-first search algorithm always selects the path which appears best at that
moment.
 It is the combination of depth-first search and breadth-first search algorithms.
 It uses the heuristic function and search. Best-first search allows us to take the advantages
of both algorithms.
 With the help of best-first search, at each step, we can choose the most promising node.
 In the best first search algorithm, we expand the node which is closest to the goal node
and the closest cost is estimated by heuristic function,
i.e. f(n)= g(n).
Where, h(n)= estimated cost from node n to the goal.
 The greedy best first algorithm is implemented by the priority queue.

Example:
 Let us see how this works for route-finding problems in Romania; we use the straight
line distance heuristic, which we will call hSLD.
 If the goal is Bucharest, we need to know the straight-line distances to Bucharest, which
are shown in below figure.
 For example, hSLD(In(Arad))=366. Notice that the values of hSLD cannot be computed
from the problem description itself.
 Moreover, it takes a certain amount of experience to know that hSLD is correlated with
actual road distances and is, therefore, a useful heuristic.

 The above shows the progress of a greedy best-first search using hSLD to find a path from
Arad to Bucharest.
 The first node to be expanded from Arad will be Sibiu because it is closer to Bucharest
than either Zerind or Timisoara.
 The next node to be expanded will be Fagaras because it is closest. Fagaras in turn
generates Bucharest, which is the goal.
 For this particular problem, greedy best-first search using hSLD finds a solution without
ever expanding a node that is not on the solution path; hence, its search cost is minimal. It is not
optimal, however: the path via Sibiu and Fagaras to Bucharest is 32 kilometers longer than the
path through Rimnicu Vilcea and Pitesti.
 This shows why the algorithm is called “greedy”—at each step it tries to get as close to the goal
as it can.
Properties of greedy best-first search:
 Greedy best-first tree search is also incomplete even in a finite state space, much like
depth-first search.
 Consider the problem of getting from Iasi to Fagaras. The heuristic suggests that Neamt
be expanded first because it is closest to Fagaras, but it is a dead end.
 The solution is to go first to Vaslui—a step that is actually farther from the goal
according to the heuristic—and then to continue to Urziceni, Bucharest, and Fagaras.
 The algorithm will never find this solution, however, because expanding Neamt puts Iasi
back into the frontier,
 Iasi is closer to Fagaras than Vaslui is, and so Iasi will be expanded again, leading to an
infinite loop.
 The worst-case time and space complexity for the tree version is O(bm),
where b is the maximum branching factor of the search tree
m is the maximum depth of the search space.
 With a good heuristic function, however, the complexity can be reduced substantially.
 The amount of the reduction depends on the particular problem and on the quality of the
heuristic.

A* search
 A* search is the most commonly known form of best-first search. It uses heuristic
function h(n), and cost to reach the node n from the start state g(n).
  A* search algorithm finds the shortest path through the search space using the heuristic
function.
 This search algorithm expands less search tree and provides optimal result faster.
 In A* search algorithm, we use search heuristic as well as the cost to reach the node.
Hence we can combine both costs as following, and this sum is called as a fitness
number.

Algorithm A*:
OPEN = nodes on frontier CLOSED=expanded nodes
OPEN = {<s,nil>}
while OPEN is not empty
remove from OPEN the node <n,p> with minimum f(n)
place <n,p> on CLOSED
if n is a goal node, return success (path p)
for each edge connecting n & m with cost c
if <m,q> is on CLOSED and {p|e} is cheaper than q
then remove n from CLOSED , put <m,{p|e}> on OPEN
else if <m,q> is on OPEN AND {p|e} is cheaper than q
then replace q with {p|e}
else if m is not on OPEN put <m,{p|e}> on OPEN
return failure

Example 1:

:
Solution:
Start State: {(S, 5)}
Iteration1: {(S--> A, 4), (S-->G, 10)}
Iteration2: {(S--> A-->C, 4), (S--> A-->B, 7), (S-->G, 10)}
Iteration 3: {(S--> A-->C--->G, 6), (S--> A-->C--->D, 11), (S--> A-->B, 7), (S-->G, 10)}
Iteration 4 will give the final result, as S--->A--->C--->G it provides the optimal path with cost
6

So A* avoid expanding paths that are already expensive

Bucharest appears on the fringe but not selected for expansion since its cost (450) is higher
than that of Pitesti (417). Important to understand for the proof of optimality of A*
Optimal Path Found: Arad --- Sibiu --- Rimnicu --- Pitesti --- Bucharest

Under some reasonable conditions for the heuristics, we have:

 Complete : Yes, unless there are infinitely many nodes with f(n) < f(Goal)
 Time Complexity: The time complexity of A* search algorithm depends on heuristic
function, and the number of nodes expanded is exponential to the depth of solution d. So the
time complexity is O(b^d) , where b is the branching factor.
 Space Complexity: The space complexity of A* search algorithm is O(b^d)
 Optimal: Yes
Also, optimal use of heuristics information!
 Widely used. E.g. Google maps.
 After almost 40 yrs, still new applications found.
 Also, optimal use of heuristic information.

A*: Difficulties
 It becomes often difficult to use A* as the OPEN queue grows very large.
 A solution is to use algorithms that work with less memory
 Memory bounded heuristic search algorithms reduce the memory requirements for A* by
introducing IDA*.
 IDA* is an iterative deepening algorithm.
 The cut-off for nodes expanded in an iteration is decided by the f-value of the nodes.

IDA* Algorithm

 IDA* = A* + Iterative Deepening DFS

 The key feature of the IDA* algorithm is that it doesn’t keep a track of each visited node
which helps in saving memory consumption and can be used where memory is
constrained.
 It is path search algorithm which finds shortest path from start node to goal node in
weighted graph.
 It borrows the idea to use a heuristic function from A*
  Its memory usage is weaker than in A*, thereby reducing memory problem in A*
 IDA* is optimal in terms of space and cost
 Problem: Excessive node generation

Algorithm:
 Set certain threshold/f-bound
 If f(node) > threshold/f-bound, prune the node
 Set threshold/f-bound = minimum cost of any node that is pruned
 Terminates when goal is reached.

IDA* Search Procedure

 Consider the threshold/f-bound value and cycle through the algorithm
 In every branch visit the depth till the f(node) is greater than the threshold/f-bound and
note down that f(node) value, do this till all branches are explored upto certain depth.
 Then the cycle continues from the starting node again with the new threshold/f-new value
that is the minimum of f(node) values noted down.
 This continues until the goal is found or the time limit is exceeded
Properties of IDA*:
 Complete: Yes, similar to A*
 Time: Depends strongly on the number of different values that the heuristic value can
take on. If A* expands N nodes, IDA* expands 1+2+……+N=O(N2) nodes.
 Space: It is DFS, it only requires space proportional to the longest path it explores.
 Optimal: Yes, similar to A*.