Artificial Intelligence

Heuristic Search Techniques

• Many of the problems are too complex to be solved by different
techniques ; rather they must be attacked by appropriate search
methods armed with whatever direct techniques are available to
guide the search.
• They can be described independently of any particular task or
problem domain.
• But , when they are applied to a particular problem, their efficiency is
highly dependent on the way they exploit domain specific knowledge.
• They have to take care of Combinatorial Explosion.
• These techniques are often called weak methods.
• Even though they are called weak but they provide a solution to the
toughest of the problems.

10/1/2010
 Artificial Intelligence

Heuristic Search Techniques

• Some of the search strategies are as under
o Depth First
o Breadth First
o Generate & Test
o Hill Climbing
o Best- First Search
o Problem Reduction
o Constraint Satisfaction
o Means-ends analysis

10/1/2010
 Artificial Intelligence

Heuristic Search Techniques

• Generate & Test
o It is the simplest of the all.
o Algorithm
 Generate a possible solution. For some problems, this means
generating a particular point in the problem space. For others,
it means generating a path from a start state.
 Test to see if this is actually a solution by comparing the
chosen point or the endpoint of the chosen path to the set of
acceptable goal states.
 If a solution has been found, quit otherwise return to step 1.

10/1/2010
 Artificial Intelligence

Heuristic Search Techniques

• If the generation of possible solutions is done systematically, then
this procedure will find a solution eventually.
• If the problem space is large it may take a long time.
• It uses a Depth-First Search procedure, since complete solution must
be generated before it can be tested.
• The procedure is very exhaustive.
• It can also generate solutions randomly, but that will not guarantee
that we will find a solution.( British Museum Algorithm)
• The most straight forward way to implement systematic Generate &
Test is a depth-first search tree with backtracking.
• For simple problems, exhaustive G&T is often a reasonable
technique.
• Example – Four six sided cubes.

10/1/2010
 Artificial Intelligence

Heuristic Search Techniques

• Example of complex problem - DENDRAL
o It infers the structure of the organic compounds using mass
spectrogram and nuclear magnetic resonance.
o It uses a strategy called plan-generate-test.
o The major weakness is that it often produces somewhat
inaccurate solutions since there is no feedback from the world.

10/1/2010
 Artificial Intelligence

Heuristic Search Techniques

• Hill Climbing
o It is a variant of generate and test in which the feedback from the
test procedure is used to help the generator decide which
direction to move in the search space.
o In a pure generate and test procedure, the test function responds
with only a yes or a no.
o Hill climbing is often used when a good heuristic function is
available for evaluating states bur when no other useful
knowledge is available.
o Example – moving towards the tallest building in town.
o End a Hill Climbing search when there is no reasonable
alternative state to move to.

10/1/2010
 Artificial Intelligence

Heuristic Search Techniques

• Simple Hill Climbing
o Algorithm
 Evaluate the initial state. If it is also a goal state, then return it
and quit. Otherwise, continue with the initial state as the
current state.
 Loop until a solution is found or until there are no new
operators left to be applied in the current state.
 Select an operator that has not yet been applied to the
current state and apply it to produce a new state.
 Evaluate the new state.
 If it is a goal state then return it and quit.
 If it is not a goal state but it is better than the current state,
then make it the current state.
 If it is not better than the current state, then continue in the
loop.

10/1/2010
 Artificial Intelligence

Heuristic Search Techniques

• Steepest Ascent Hill Climbing
o It is a variation from the simple Hill Climbing, we consider all the
moves from the current state and select the best one possible.

o Both Simple Hill Climbing and Steepest Ascent may fail to find the
best solution, what kind of situations occur are discussed on the
next slide.

10/1/2010
 Artificial Intelligence

Heuristic Search Techniques

o A local maxima is a state that is better than all neighbors but is
not better than some other state farther away. At local maxima, all
moves appear to make things worse.
o A Plateau is a flat area of the search space in which a whole set
of neighboring states have the same value. It is difficult to pick the
best direction.
o A Ridge is a special kind of local maxima. It is an area of the
search space that is higher than surrounding areas.
• Ways to tackle these problems
o Backtrack
o Make a big jump
o Apply two or more rules before applying the test.

10/1/2010
 Artificial Intelligence

Heuristic Search Techniques

• Simulated Annealing
o It is another variation of hill climbing where at the beginning of the
process, some downhill moves may be made. The idea is to do
enough exploration of the while space early on so that the final
solution is relatively insensitive to the starting state. This should
lower the chances of getting caught at a local maximum, plateau
or a ridge.

10/1/2010
 Artificial Intelligence

Heuristic Search Techniques

• Best First Search
o It is the way of combining the advantage of both depth-first &
breath first into a single method.
o Depth-first is good as it allows a solution to be found without all
competing branches having to be expanded.
o Breath-first is good because it does not get trapped on dead-end
paths.
o One way to combine the both is to follow a single path at a time,
but switch paths whenever some competing path looks more
promising than the current one does.
o Usually, we start with BFS and when we sense a opportunity, then
we move to DFS.

10/1/2010
 Artificial Intelligence

Heuristic Search Techniques

• Best First Search
o It is very similar to the procedure followed in Steepest-ascent hill
climbing, exception being as under
 In hill climbing, one move is selected and all the others are
rejected, never to be considered again, in BFS one move is
selected, but the others are kept around so that they can be
revisited later if the selected path becomes less promising.

10/1/2010
 Artificial Intelligence

Heuristic Search Techniques

• Best First Search
o We can use a graph technique as well.
o To implement such a graph-search procedure, we will need two
use lists of nodes:
 OPEN : Nodes that have been generated and have had the
heuristic function applied to them but which have not been
examined yet.
 CLOSED : nodes that have been already examined.

10/1/2010
 Artificial Intelligence

Heuristic Search Techniques

• Best First Search
o Algorithm
 Start with OPEN containing just the initial state.
 Until a goal is found or there are no nodes left on OPEN do:
 Pick the best node on OPEN.
 Generate its Successors.
 For each successor do:
 If it has not been generated before, evaluate it, add it to
OPEN , and record its parent.
 If it has been generated before, change the parent if this
new path is better than the previous one. In that case,
update the cost of getting to this node and to any
successors that this node may already have.

10/1/2010
 Artificial Intelligence

Heuristic Search Techniques

• Problem Reduction
o We work on AND-OR technique graph (tree) , it is useful for
representing the solution of problems that can be solved by
decomposing them into a set of smaller problems, all of which
must then be solved.
o This decomposition generates arcs that we call AND arcs. One
AND arc may point to any number of successor nodes, all of
which must be solved in order for the arc to point to a solution.
o In OR graph, several arcs may emerge from a single node,
indicating a variety of ways in which the original might be solved.
o An AND arc are indicated with the line connecting all the
components.
o f’ , g and h’

10/1/2010
 Artificial Intelligence

Heuristic Search Techniques

• Problem Reduction
o .
Goal : Acquire TV Set

Goal : Steal TV Goal : Earn Money Goal : Buy TV Set

10/1/2010
 Artificial Intelligence

Heuristic Search Techniques

• Problem Reduction Algorithm
o Initialize the graph to start node.
o Loop until the starting node is labeled SOLVED or until its cost
goes above FUTILE.
 Traverse the graph, starting at the initial node and following
the current best path, and accumulate the set of nodes that are
on that path and have not yet been expanded or labeled as
solved.
 Pick one of these unexpected nodes and expand it. If there are
no successors, assign FUTILITY as the value of this node.
Otherwise , add its successors to the graph and for each of
them compute f’ ( use only h’ and ignore g) If f’ of any node is
0, mark it as SOLVED

10/1/2010
 Artificial Intelligence

Heuristic Search Techniques

• Problem Reduction
o Change the f’ estimate of the newly expanded node to reflect the
new information provided by its successors. Propagate this
change backwards through the graph. It any node contains a
successor arc whose descendants are all solved, label the node
itself as SOLVED. At each node that is visited while going up the
graph, decide which of its successors arcs is the most promising
and mark it as part of the current best path. This may cause the
current best path to change. This propagation of revised cost
estimates back up the tree was not necessary in the best-first
search algorithm because only unexpected nodes were
examined. But now expanded nodes must be reexamined so that
the best current path can be selected. Thus it is important that
their f’ values be the best estimate available.
o Fig : 3.8

10/1/2010
 Artificial Intelligence

Heuristic Search Techniques

• Difference between Best First & Problem
Reduction
o In the best first search algorithm, the desired path from one node
to another was always the one with the lowest cost. But the same
is not true when searching an AND-OR graph.
o Fig: 3.9 of text book

• Limitation , illustrated by fig 3.10

10/1/2010
 Artificial Intelligence

Heuristic Search Techniques

• Constraint Satisfaction
o The goal is to discover some problem state that satisfies a given
set of constraints. E.g. Puzzles, real-world perceptual labeling
problems, design tasks etc.
o Helps in reducing the amount of search required as compared
with a method that attempts to form partial solutions directly by
choosing specific values for components of the eventual solution.
o The fundamental issue is to include restrictions first that can be
inferred from the rules of arithmetic. This reduces the number of
guesses.
o The initial state contains the constraints that are originally given in
the problem description.
o A goal state is any state that has been constrained enough,
enough has to be defined for each problem.

10/1/2010
 Artificial Intelligence

Heuristic Search Techniques

• Constraint Satisfaction
o It is a 2 step process.
 First, constraints are discovered and propagated as far as
possible throughout the system.
 Then, if there is still no solution, search begins. A guess about
something is made and added as a new constraint.
Propagation can then occur with this new constraint and so
forth.

10/1/2010
 Artificial Intelligence

Heuristic Search Techniques

• Constraint Satisfaction
o Algorithm
 Propagate available constraints. To do this, first set OPEN to
the set of all objects that must have values assigned to them in
a complete solution. Then do until an inconsistency is detected
or until OPEN is empty :
 Select an object OB from OPEN. Strengthen as much as
possible the set of constraints that apply to OB.
 If this set is different from the set that was assigned the last
time OB was examined or it this is the first time OB has
been examined, then add to OPEN all objects that share
any constraints with OB.
 Remove OB from OPEN.

10/1/2010
 Artificial Intelligence

Heuristic Search Techniques

• Constraint Satisfaction
o Algorithm
 If the union of the constraints discovered above defines a
solution, then quit and report the solution.
 If the union of the constraints discovered above defines a
contradiction, then return failure.
 If neither of the above occurs, then it is necessary to make a
guess at something in order to proceed. To do this, loop until a
solution is found or all possible solutions have been eliminated
:
 Select an object whose value is not yet determined and
select a way of strengthening the constraints on that
object.
 Recursively invoke constraint satisfaction with the current
set of constraints augmented by the strengthening
constraint just selected.

10/1/2010
 Artificial Intelligence

Heuristic Search Techniques

• Means-Ends analysis
o So far, we have presented a collection of search strategies that
can reason either forward or backward, but for a given problem,
one direction or the other must be chosen. Often, however a
mixture of two directions is appropriate.
o By doing this we can solve the major part of problem first and then
go back and solve the small problems that arise in putting the big
picture together.
o The technique is called, means-ends analysis.
o It processes the centers around the detection of differences
between the current state and the goal state. Once that difference
is found , and operator that can reduce the difference must be
found.
o Divide the problem into sub-problems.

10/1/2010
 Artificial Intelligence

Heuristic Search Techniques

• Means-Ends analysis
o The first AI program to use Means-Ends analysis was
GPS( General Problem Solver)
o We need to have three things Operator, Preconditions & Result
o Eg. Push, Carry, Walk, Pickup, Putdown, Place

10/1/2010
 Artificial Intelligence

Heuristic Search Techniques

• Means-Ends analysis
o Compare CURRENT to GOAL. If there are no differences
between them then return.
o Otherwise , select the most important difference and reduce it by
doing the following until success or failure is signaled.
 Select an option that is untried, Operator O that is applicable to
the current differences. It there as no such operators then
signal FAILURE
 Attempt to apply O to CURRENT. Generate descriptions of two
states : O-START, a state in which O’s preconditions are
satisfied and O-RESULT, the state that would result if O were
applied in O-START.

10/1/2010
 Artificial Intelligence

Heuristic Search Techniques

• Means-Ends analysis
o If ( FIRST-PART  MEA ( CURRENT, O-START)
and
((LAST-PART  MEA(O-RESULT,GOAL)
are successful, then signal success and return the result of
concatenating FIRST-PART,O & LAST-PART

10/1/2010