Artificial Intelligence and Machine Learning (21CS54)

MODULE- 02
HEURISTIC SEARCH

INFORMED (HEURISTIC) SEARCH TECHNIQUE

 Informed search strategy is one that uses problem-specific knowledge beyond the definition of the
problem itself. It can find solutions more efficiently than can an uninformed strategy.
 The general approach we consider is called best-first search.
 Best-first search is an instance of the general TREE-SEARCH or GRAPH-SEARCH algorithm in
which a node is selected for expansion based on an evaluation function, f(n).
 The evaluation function is construed as a cost estimate, so the node with the lowest evaluation is
expanded first.
 The implementation of best-first graph search is identical to that for uniform-cost search, except for
the use of f instead of g to order the priority queue.
 The choice of f determines the search strategy.
 Most best-first algorithms include as a component of f a heuristic function, denoted h(n) where h(n)
= estimated cost of the cheapest path from the state at node n to a goal state.
 Heuristic functions are the most common form in which additional knowledge of the problem is
imparted to the search algorithm.

GREEDY BEST-FIRST SEARCH

 Greedy best-first search8 tries to expand the node that is closest to the goal, on the grounds that this
is likely to lead to a solution quickly. Thus, it evaluates nodes by using just the heuristic function;
that is, f(n) = h(n).
 We use the straight line distance heuristic, which we will call hSLD.
 The values of hSLD cannot be computed from the problem description itself.
 It takes a certain amount of experience to know that hSLD is correlated with actual road distances
and is, therefore, a useful heuristic.
 Greedy best-first search using hSLD finds a solution without ever expanding a node that is not on the
solution path; hence, its search cost is minimal.
 Greedy best-first tree search is also incomplete even in a finite state space, much like depth-first
search.
 The worst-case time and space complexity for the tree version is O(bm), where m is the maximum
depth of the search space.

Prepared by Ms. Shilpa, Assistant Professor, Dept. of CSE, NCEH 1

 Artificial Intelligence and Machine Learning (21CS54)

 With a good heuristic function, however, the complexity can be reduced substantially.
 The amount of the reduction depends on the particular problem and on the quality of the heuristic.
 Figure 1 shows the progress of a greedy best-first search using hSLD to find a path from Arad to
Bucharest.

Best first search algorithm:

 Step 1: Place the starting node into the OPEN list.
 Step 2: If the OPEN list is empty, Stop and return failure.
 Step 3: Remove the node n, from the OPEN list which has the lowest value of h(n), and places
it in the CLOSED list.
 Step 4: Expand the node n, and generate the successors of node n.
 Step 5: Check each successor of node n, and find whether any node is a goal node or not. If any
successor node is goal node, then return success and terminate the search, else proceed to Step
6.

Prepared by Ms. Shilpa, Assistant Professor, Dept. of CSE, NCEH 2

 Artificial Intelligence and Machine Learning (21CS54)

 Step 6: For each successor node, algorithm checks for evaluation function f(n), and then check
if the node has been in either OPEN or CLOSED list. If the node has not been in both list, then
add it to the OPEN list.
 Step 7: Return to Step 2.

Advantages:
o Best first search can switch between BFS and DFS by gaining the advantages of both the
algorithms.
o This algorithm is more efficient than BFS and DFS algorithms.

Disadvantages:
o It can behave as an unguided depth-first search in the worst case scenario.

o It can get stuck in a loop as DFS.

o This algorithm is not optimal.

Example

Start with node S as it’s the root node. Expand the nodes of S and put in the CLOSED list. Check
whether it’s a goal node.

Iteration 1: Open [ ], Closed [ ]

Expand the node S and add the successors of S into the OPEN according to its Heuristic value. Open
[B, A], Closed [ S]

Prepared by Ms. Shilpa, Assistant Professor, Dept. of CSE, NCEH 3

 Artificial Intelligence and Machine Learning (21CS54)

Iteration 2: Open [B, A], Closed [S]

Since the heuristic value of B is less remove it from OPEN and add it to the CLOSED. Add the
successors of B into the OPEN

Open [F, B, E], Closed [S, B]

Iteration 3: Open [F, B, E], Closed [S, B]

Since the heuristic value of F is less remove it from OPEN and add it to the CLOSED. Add the
successors of F into the OPEN

Open [G, B, E, I], Closed [S, B, F]

Iteration 4: Open [G, B, E, I], Closed [S, B, F]

Since the heuristic value of G is less remove it from OPEN and add it to the CLOSED. Check whether
it is GOAL node. Since it’s a goal node stop it.

Open [B, E, I], Closed [S, B, F, G]

Hence the final solution path will be: S----> B----->F----> G

A* SEARCH: MINIMIZING THE TOTAL ESTIMATED SOLUTION COST

 The most widely known form of best-first search is called A∗ A search. It evaluates nodes by
combining g(n), the cost to reach the node, and h(n), the cost to get from the node to the goal:
 f(n) = g(n) + h(n) .
 Since g(n) gives the path cost from the start node to node n, and h(n) is the estimated cost of the
cheapest path from n to the goal, we have
 f(n) = estimated cost of the cheapest solution through n .
 Thus, if we are trying to find the cheapest solution, a reasonable thing to try first is the node with
the lowest value of g(n) + h(n).
 It turns out that this strategy is more than just reasonable: provided that the heuristic function
h(n) satisfies certain conditions.
 A* search is both complete and optimal. The algorithm is identical to Uniform-Cost-Search

except that A∗ uses g + h instead of g.

Prepared by Ms. Shilpa, Assistant Professor, Dept. of CSE, NCEH 4

 Artificial Intelligence and Machine Learning (21CS54)

Conditions for optimality: Admissibility and consistency

 The first condition we require for optimality is that h(n) be an admissible heuristic. An
admissible heuristic is one that never overestimates the cost to reach the goal. Because g(n) is
the actual cost to reach n along the current path, and f(n) = g(n) + h(n).
 f(n) never overestimates the true cost of a solution along the current path through n.
 Admissible heuristics are by nature optimistic because they think the cost of solving the problem
is less than it actually is.
 Straight-line distance is admissible because the shortest path between any two points is a straight
line, so the straight line cannot be an overestimate.
 A second, slightly stronger condition called consistency/monotonicity is required only for
applications of A∗ to graph search.
 A heuristic h(n) is consistent if, for every node n1 and every successor n of n generated by any
action a, the estimated cost of reaching the goal from n is no greater than the step cost of getting
to n1 plus the estimated cost of reaching the goal from n1
h(n) ≤ c(n, a, n1) + h(n1)
 This is a form of the general triangle inequality, which stipulates that each side of a triangle
cannot be longer than the sum of the other two sides.
 For an admissible heuristic, the inequality makes perfect sense: if there were a route from n to
Gn via n1 that was cheaper than h(n), that would violate the property that h(n) is a lower bound
on the cost to reach Gn.
 It is fairly easy to show that every consistent heuristic is also admissible. Consistency is
therefore a stricter requirement than admissibility, but one has to work quite hard to concoct
heuristics that are admissible but not consistent.
 The general triangle inequality is satisfied when each side is measured by the straight-line
distance and that the straight-line distance between n and n1 is no greater than c(n, a, n1). Hence,
hSLD is a consistent heuristic.

Optimality of A*
 A∗ has the following properties: the tree-search version of A∗ is optimal if h(n) is admissible,
while the graph-search version is optimal if h(n) is consistent.
 The argument essentially mirrors the argument for the optimality of uniform-cost search, with g
replaced by f—just as in the A∗ algorithm itself.
 The first step is to establish the following: if h(n) is consistent, then the values of f(n) along any
path are non-decreasing.
 The proof follows directly from the definition of consistency.

Prepared by Ms. Shilpa, Assistant Professor, Dept. of CSE, NCEH 5

 Artificial Intelligence and Machine Learning (21CS54)

 Suppose n1 is a successor of n; then g(n1 ) = g(n) + c(n, a, n1) for some action a, and we have
f(n1) = g(n1) + h(n1) = g(n) + c(n, a, n1 ) + h(n1) ≥ g(n) + h(n) = f(n) .
 The next step is to prove that whenever A∗ selects a node n for expansion, the optimal path to
that node has been found.
 Were this not the case, there would have to be another frontier node n1 on the optimal path from
the start node to n, by the graph separation property.
 From the two preceding observations, it follows that the sequence of nodes expanded by A∗
using GRAPH-SEARCH is in nondecreasing order of f(n).
 Hence, the first goal node selected for expansion must be an optimal solution because f is the
true cost for goal nodes (which have h = 0) and all later goal nodes will be at least as expensive.
 The fact that f-costs are nondecreasing along any path also means that we can draw contours in
the state space, just like the contours in a topographic map.

 If C∗ is the cost of the optimal solution path, then we can say the following:

o A∗ expands all nodes with f(n) < C∗.

o A∗ might then expand some of the nodes right on the “goal contour” (where f(n) = C∗)

before selecting a goal node.

 Completeness requires that there be only finitely many nodes with cost less than or equal to C∗,

a condition that is true if all step costs exceed some finite and if b is finite.

 A∗ expands no nodes with f(n) > C∗.

 In the general case of a graph, the situation is even worse. There can be exponentially many

states with f(n) < C∗ even if the absolute error is bounded by a constant.

 The complexity of A∗ often makes it impractical to insist on finding an optimal solution.

 One can use variants of A∗ that find suboptimal solutions quickly, or one can sometimes
design heuristics that are more accurate but not strictly admissible.
 Computation time is not, however, A∗’s main drawback.
 Because it keeps all generated nodes in memory, A∗ usually runs out of space long before it
runs out of time. For this reason, A∗ is not practical for many large-scale problems.

HEURISTIC FUNCTION
 The 8-puzzle was one of the earliest heuristic search problems.
 The object of the puzzle is to slide the tiles horizontally or vertically into the empty space until
the configuration matches the goal configuration.
 The average solution cost for a randomly generated 8-puzzle instance is about 22 steps.

Prepared by Ms. Shilpa, Assistant Professor, Dept. of CSE, NCEH 6

 Artificial Intelligence and Machine Learning (21CS54)

 The branching factor is about 3. (When the empty tile is in the middle, four moves are possible;
when it is in a corner, two; and when it is along an edge, three.) This means that an exhaustive
tree search to depth 22 would look at about 322 ≈ 3.1 × 1010 states.
 A graph search would cut this down by a factor of about 170,000 because only 9!/2 = 181, 440
distinct states are reachable.
 This is a manageable number, but the corresponding number for the 15-puzzle is roughly 1013,
so the next order of business is to find a good heuristic function.
 If we want to find the shortest solutions by using A∗, we need a heuristic function that never
overestimates the number of steps to the goal.

 h1 = the number of misplaced tiles, all of the eight tiles are out of position, so the start state
would have h1 = 8.
 h1 is an admissible heuristic because it is clear that any tile that is out of place must be moved at
least once.
 h2 = the sum of the distances of the tiles from their goal positions. Because tiles cannot move
along diagonals, the distance we will count is the sum of the horizontal and vertical distances.
This is sometimes called the city block distance or Manhattan distance.
 h2 is also admissible because all any move can do is move one tile one step closer to the goal.
Tiles 1 to 8 in the start state give a Manhattan distance of

h2 = 3 + 1 + 2 + 2 + 2 + 3 + 3 + 2 = 18 .

The effect of heuristic accuracy on performance

 One way to characterize the quality of a heuristic is the effective branching factor b∗
 If the total number of nodes generated by A∗ for a particular problem is N and the solution depth
is d, then b∗ is the branching factor that a uniform tree of depth d would have to have in order to
contain N + 1 nodes.
 Thus,
N +1 = 1+ b∗ + (b∗)2 + ··· + (b∗)d
 If A∗ finds a solution at depth 5 using 52 nodes, then the effective branching factor is 1.92.
 Experimental measurements of b∗ on a small set of problems can provide a good guide to the
heuristic’s overall usefulness.
 A well designed heuristic would have a value of b∗ close to 1, allowing fairly large problems to
be solved at reasonable computational cost.
 It is easy to see from the definitions of the two heuristics that, for any node n, h2(n) ≥ h1(n).

Prepared by Ms. Shilpa, Assistant Professor, Dept. of CSE, NCEH 7

 Artificial Intelligence and Machine Learning (21CS54)

INFORMED SEARCH V/S UNINFORMED SEARCH

Parameters Informed Search Uninformed Search

Utilizing It uses knowledge during the process It does not require using any knowledge during the process
Knowledge of searching. of searching.

Speed Finding the solution is quicker. Finding the solution is much slower comparatively.

Completion It can be both complete and It is always bound to be complete.

incomplete.

Consumption of Due to a quicker search, it consumes Due to slow searches, it consumes comparatively more
Time much less time. time.

Cost Incurred The expenses are much lower. The expenses are comparatively higher.

Suggestion/ The AI gets suggestions regarding The AI does not get any suggestions regarding what
Direction how and where to find a solution to solution to find and where to find it. Whatever knowledge
any problem. it gets is out of the information provided.

Efficiency It costs less and generates quicker It costs more and generates slower results. Thus, it is
results. Thus, it is comparatively comparatively less efficient.
more efficient.

Length of Implementation is shorter using AI. The implementation is lengthier using AI.
Implementation

Examples A few examples include Graph A few examples include Breadth-First Search or BFS and
Search and Greedy Search. Depth-First Search or DFS.

Prepared by Ms. Shilpa, Assistant Professor, Dept. of CSE, NCEH 8