Chapter 4

Beyond Classical Search

 Local Search Algorithms and
Optimization problems

• What/Why Local Search

• Hill-Climbing Search
• Simulated Annealing Search
• Genetic Algorithms
 Local Search
• local search algorithms in the state space,
evaluating and modifying one or more current
states rather than systematically exploring
paths from an initial state.
• These algorithms are suitable for problems in
which all that matters is the solution state, not
the path cost to reach it.
 Local search algorithms operate using a single current
node (rather than multiple paths) and generally move only
to neighbors of that node.

Typically, the paths followed by the search are not

retained. Although local search algorithms are not
systematic,

they have two key advantages:

(1) they use very little memory—usually a constant
amount
(2) they can often find reasonable solutions in large or
infinite state spaces for which systematic algorithms are
not suitable.
• In addition to finding goals, local search
algorithms are useful for solving pure
optimization problems, in which the aim is to
find the best state according to an objective
function
• A landscape has both “location” (defined by the
state) and “elevation”(height) (defined by the value
of the heuristic cost function or objective function).
• If elevation corresponds to cost, then the aim is to
find the lowest valley—a global minimum
• if elevation corresponds to an objective function,
then the aim is to find the highest peak—a global
maximum.
• An optimal algorithm always finds a global
minimum/maximum.
 Hill-Climbing Search
• The hill-climbing search algorithm (steepest-
ascent)- simply a loop that continually moves in the
direction of increasing value—that is, uphill. It
terminates when it reaches a “peak” where no
neighbor has a higher value.
• The algorithm does not maintain a search tree, so
the data structure for the current node need only to
record the state and the value of the objective
function.
• Hill climbing does not look ahead beyond the
immediate neighbors of the current state.
 8-queens problem
• Local search algorithms typically use a complete-state
formulation, where each state has 8 queens on the
board, one per column.
• The successors of a state are all possible states
generated by moving a single queen to another square
in the same column (so each state has 8×7=56
successors).
• The heuristic cost function h is the number of pairs of
queens that are attacking each other, either directly or
indirectly.
• The global minimum of this function is zero, which
occurs only at perfect solutions
• Figure 4.3(a) shows a state with h=17. The
figure also shows the values of all its
successors, with the best successors having
h=12.
• Hill-climbing algorithms typically choose
randomly among the set of best successors if
there is more than one.
• Hill climbing is sometimes called greedy local
search because it grabs a good neighbor state
without thinking ahead about where to go
next.
Figure 4.3
function h is the number of pairs of queens that are attacking each other, either
directly or indirectly.

• (1,2) (1,3) (1,5)

• (2,3) (2,4) (2,6) (2,8)
• (3,5) (3,7)
• (4,5) (4,6) (4,7)
• (5,6)(5,7)
• (6,7) (6,8)
• (7,8)
• ==17
 function h is the number of pairs of queens that are attacking
each other, either directly or indirectly.
When queen is moved from 5th to 4th row in 1st column
• (1,4)
• (2,3) (2,4) (2,6) (2,8)
• (3,5) (3,7)
• (4,5) (4,6) (4,7)
• (5,6)(5,7)
• (6,7) (6,8)
• (7,8)
• ==15
• Hill climbing often makes rapid progress
toward a solution because it is usually quite
easy to improve a bad state.
• For example, from the state in Figure 4.3(a), it
takes just five steps to reach the state in
Figure 4.3(b), which has h=1 and is very nearly
a solution.
Example
 Hill climbing
Hill climbing often gets stuck for the following
reasons
• Local maxima: a local maximum is a peak that
is higher than each of its neighboring states but
lower than the global maximum. Hill-climbing
algorithms once reaches a local maximum will
then be stuck with nowhere else to go.
• Ridges: a ridge is shown in Figure 4.4. Ridges
result in a sequence of local maxima that is very
difficult for greedy algorithms to navigate.

Figure 4.4
• Plateaux: a plateau is a flat area of the state-
space landscape. It can be a flat local maximum,
from which no uphill exit exists.
• A shoulder, from which progress is possible. A
hill-climbing search might get lost on the
plateau.
• If we always allow sideways moves when there
are no uphill moves, an infinite loop will occur
whenever the algorithm reaches a flat local
maximum.
• One solution is to put a limit on the number of
consecutive sideways moves allowed.
• For randomly generated 8-queens state, steepest-
ascent hill climbing gets stuck 86% of the time,
solving only 14% of problem instances.
• It works, taking just 4 steps on average when it
succeeds and 3 when it gets stuck—not bad for a
state space with 88 ≈ 17 million states.
• For example, we could allow up to, say, 100
consecutive sideways moves in the 8-queens
problem. This raises the percentage of problem
instances solved by hill climbing from 14% to 94%.
• the algorithm averages roughly 21 steps for each
successful instance and 64 for each failure.
 Variants of Hill Climbing

Stochastic hill climbing

•chooses at random from among the uphill moves;
•the probability of selection can vary with the steepness of
the uphill move.
•This usually converges more slowly than steepest ascent,
• but in some state landscapes, it finds better solutions.
First-choice hill climbing

• Implements stochastic hill climbing by generating

successors randomly until one is generated that is better
than the current state.
•This is a good strategy when a state has many (e.g.,
thousands) of successors.
•The hill-climbing algorithms described so far are
incomplete—they often fail to find a goal when one exists
because they can get stuck on local maxima.
 Random-restart hill climbing
•It conducts a series of hill-climbing searches from
randomly generated initial states, until a goal is found.
•It is trivially complete with probability approaching 1,
because it will eventually generate a goal state as the
initial state.
•If each hill-climbing search has a probability p of
success, then the expected number of restarts required
is 1/p.
• For 8-queens instances with no sideways moves allowed, p ≈
0.14, so we need roughly 7 iterations to find a goal (6 failures
and 1 success).
• The expected number of steps is the cost of one successful
iteration plus (1−p)/p times the cost of failure, or roughly 22
steps in all. (1 x 4)+ ((1-0.14)/0.14)x3=4+18=22steps
• When we allow sideways moves, 1/0.94 ≈ 1.06 iterations are
needed on average and (1×21)+(0.06/0.94)×64 ≈ 25 steps.

• For 8-queens, then, random-restart hill climbing is very effective

indeed. Even for three million queens, the approach can find
solutions in under a minute
• The success of hill climbing depends very
much on the shape of the state-space
landscape
• if there are few local maxima and plateaux,
random-restart hill climbing will find a good
solution very quickly.
 Simulated annealing
• A hill-climbing algorithm that never makes “downhill”
moves toward states with lower value (or higher cost)
is guaranteed to be incomplete, because it can get
stuck on a local maximum.
• In contrast, a purely random walk—that is, moving to a
successor chosen at random from the set of successors
—is complete but extremely inefficient.
• Simulated annealing algorithm : it is reasonable to
combine hill climbing with a random walk that yields
both efficiency and completeness.
• In metallurgy, annealing is the process used to temper or
harden metals and glass by heating them to a high
temperature and then gradually cooling them, thus
allowing the material to reach a low energy crystalline
state.

To avoid being stuck in a local maxima, it tries randomly

(using a probability function) to move to another state, if this new state
is better it moves into it, otherwise try another
move and so on.
•To explain simulated annealing, we switch our point of view
from hill climbing to gradient descent (i.e., minimizing cost)
•imagine the task of getting a ping-pong ball into the deepest
crevice in a bumpy surface.
• If we just let the ball roll, it will come to rest at a local
minimum. If we shake the surface, we can bounce the ball
out of the local minimum.
•The trick is to shake just hard enough to bounce the ball out
of local minima but not hard enough to dislodge or remove it
from the global minimum.
•The simulated-annealing solution is to start by shaking hard
(i.e., at a high temperature) and then gradually reduce the
intensity of the shaking (i.e., lower the temperature).
• The simulated-annealing algorithm is quite similar to hill climbing.
• Instead of picking the best move, however, it picks a random move.
• If the move improves the situation, it is always accepted. Otherwise,
the algorithm accepts the move with some probability less than 1.
• The probability decreases exponentially with the “badness” of the
move—the amount ΔE by which the evaluation is worsened.
• The probability also decreases as the “temperature” T goes down:
• “bad” moves are more likely to be allowed at the start when T is
high, and they become more unlikely as T decreases.
• If the schedule lowers T slowly enough, the algorithm will find a
global optimum with probability approaching 1.
• Simulated annealing was first used extensively to solve VLSI layout
problems in the early 1980s. It has been applied widely to factory
scheduling and other large-scale optimization tasks.
 Local beam search
• The local beam search algorithm keeps track
of k states rather than just one.
• It begins with k randomly generated states.
• At each step, all the successors of all k states
are generated.
• If any one is a goal, the algorithm halts.
• Otherwise, it selects the k best successors
from the complete list and repeats.
• At first sight, a local beam search with k states might
seem to be nothing more than running k random
restarts in parallel instead of in sequence.
• In fact, the two algorithms are quite different.
• In a random-restart search, each search process runs
independently of the others.
• In a local beam search, useful information is passed
among the parallel search threads.
• The states that generate the best successors say to the
others, “Come over here, the grass is greener!”
• The algorithm quickly abandons unfruitful searches and
moves its resources to where the most progress is
being made.
•local beam search can suffer from a lack of diversity among
the k states
•they can quickly become concentrated in a small region of
the state space, making the search little more than an
expensive version of hill climbing.
•A variant called stochastic beam search , analogous to
stochastic hill climbing, helps alleviate this problem.
• Instead of choosing the best k from the pool of candidate
successors, stochastic beam search chooses k successors at
random, with the probability of choosing a given successor
being an increasing function of its value.
• Stochastic beam search bears some resemblance to the
process of natural selection, whereby the “successors”
(offspring) of a “state” (organism)populate the next
generation according to its “value” (fitness).
 Genetic algorithms
• A genetic algorithm (or GA) - in which successor
states are generated by combining two parent
states
• GAs begin with a set of k randomly generated
states, called the population.
• Each state, is represented as a string of 0s and 1s.
• For example, an 8-queens state must specify the
positions of 8 queens, each in a column of 8
squares, and so requires 8 × log2 8 = 24 bits.
• Alternatively, the state could be represented as 8 digits,
each in the range from 1 to 8. Figure (a) shows a population
of four 8-digit strings representing 8-queens states.
• In (b), each state is rated by the objective function, the
fitness function. A fitness function should return higher
values for better states, for the 8-queens problem we use
the number of non attacking pairs of queens, which has a
value of 28 for a solution.
• The values of the four states are 24, 23, 20, and 11. In this
particular variant of the genetic algorithm, the probability
of being chosen for reproducing is directly proportional to
the fitness score, and the percentages are shown next to
the raw scores.
• non attacking pairs of queens, which has a
value of 28 for a solution.
• (1,2) (1,3) (1,4) (1,5) (1,6) (1,7) (1,8)
• (2,3)(2,4)(2,5)(2,6)(2,7)(2,8)
• (3,4)(3,5)(3,6)(3,7),(3,8)
• (4,5)(4,6)(4,7)(4,8)
• (5,6)(5,7)(5,8)
• (6,7)(6,8)
• (7,8)
• = 28
Fitness Function :
Calculate the probability of being regenerated in next generation. For example:
24/(24+23+20+11) = 31% ,
23/(24+23+20+11) = 29% ,
20/ (24+23+20+11) = 26%
11/(24+23+20+11) = 14%
• In (c), two pairs are selected at random for
reproduction, in accordance with the probabilities
in (b).
• One individual is selected twice and one not at all.
• For each pair to be mated, a crossover point is
chosen randomly from the positions in the string.
• In Figure the crossover points are after the third
digit in the first pair and after the fifth digit in the
second pair.
• In (d), the offspring themselves are created by
crossing over the parent strings at the crossover
point.
• For example, the first child of the first pair gets the
first three digits from the first parent and the
remaining digits from the second parent,
• whereas the second child gets the first three digits
from the second parent and the rest from the first
parent.
• The example shows that when two parent states
are quite different, the crossover operation can
produce a state that is a long way from either
parent state.
• It is often the case that the population is quite
diverse early on in the process, so crossover (like
simulated annealing) frequently takes large steps in
the state space early in the search process
• and smaller steps later on when most individuals
are quite similar
• Finally, in (e), each location is subject to random
mutation with a small independent probability.
One digit was mutated in the first, third, and
fourth offspring.
• In the 8-queens problem, this corresponds to
choosing a queen at random and moving it to a
random square in its column.
• In practice, genetic algorithms have had a
widespread impact on optimization problems,
such as circuit layout and job-shop scheduling.