Artificial Intelligence

Problem Solving by Search

Module -2
 Search
• Search parameters all of AI
• What choices are we searching through?
– Problem solving
Action combinations (move 1, then move 3, then move 2...)
– Natural language
Ways to map words to parts of speech
– Computer vision
Ways to map features to object model
– Machine learning
Possible concepts that fit examples seen so far
– Motion planning
Sequence of moves to reach goal destination
• An intelligent agent is trying to find a set or sequence of actions to
achieve a goal
• This is a goal-based agent
Problem-solving Agent

SimpleProblemSolvingAgent(percept)
state = UpdateState(state, percept)
if sequence is empty then
goal = FormulateGoal(state)
problem = FormulateProblem(state, g)
sequence = Search(problem)
action = First(sequence)
sequence = Rest(sequence)
Return action
 Assumptions
• Static or dynamic?

Environment is static
 Assumptions
• Static or dynamic?
• Fully or partially observable?

Environment is fully observable

 Assumptions
• Static or dynamic?
• Fully or partially observable?
• Discrete or continuous?

Environment is discrete
 Assumptions
• Static or dynamic?
• Fully or partially observable?
• Discrete or continuous?
• Deterministic or stochastic?

Environment is deterministic
 Assumptions
• Static or dynamic?
• Fully or partially observable?
• Discrete or continuous?
• Deterministic or stochastic?
• Episodic or sequential?
Environment is sequential
 Assumptions
• Static or dynamic?
• Fully or partially observable?
• Discrete or continuous?
• Deterministic or stochastic?
• Episodic or sequential?
• Single agent or multiple agent?
 Assumptions
• Static or dynamic?
• Fully or partially observable?
• Discrete or continuous?
• Deterministic or stochastic?
• Episodic or sequential?
• Single agent or multiple agent?
Search Example
Formulate goal: Be in
Bucharest.

Formulate problem: states

are cities, operators drive
between pairs of cities

Find solution: Find a

sequence of cities (e.g., Arad,
Sibiu, Fagaras, Bucharest) that
leads from the current state
to a state meeting the goal
condition
 Search Space Definitions
• State
– A description of a possible state of the world
– Includes all features of the world that are pertinent to the problem
• Initial state
– Description of all pertinent aspects of the state in which the agent starts
the search
• Goal test
– Conditions the agent is trying to meet (e.g., have $1M)
• Goal state
– Any state which meets the goal condition
– Thursday, have $1M, live in NYC
– Friday, have $1M, live in Valparaiso
• Action
– Function that maps (transitions) from one state to another
 Search Space Definitions
• Problem formulation
– Describe a general problem as a search problem
• Solution
– Sequence of actions that transitions the world from the initial state to a
goal state
• Solution cost (additive)
– Sum of the cost of operators
– Alternative: sum of distances, number of steps, etc.
• Search
– Process of looking for a solution
– Search algorithm takes problem as input and returns solution
– We are searching through a space of possible states
• Execution
– Process of executing sequence of actions (solution)
 Problem Formulation

A search problem is defined by the

1. Initial state (e.g., Arad)

2. Operators (e.g., Arad -> Zerind, Arad -> Sibiu, etc.)
3. Goal test (e.g., at Bucharest)
4. Solution cost (e.g., path cost)
Example Problems – Eight Puzzle
States: tile locations

Initial state: one specific tile configuration

Operators: move blank tile left, right, up, or

down

Goal: tiles are numbered from one to eight

around the square

Path cost: cost of 1 per move (solution cost

same as number of most or path length)

Eight puzzle applet

Example Problems – Robot Assembly
States: real-valued coordinates of
• robot joint angles
• parts of the object to be assembled

Operators: rotation of joint angles

Goal test: complete assembly

Path cost: time to complete assembly

Example Problems – Towers of Hanoi
States: combinations of poles and disks

Operators: move disk x from pole y to pole z

subject to constraints
• cannot move disk on top of smaller disk
• cannot move disk if other disks on top

Goal test: disks from largest (at bottom) to

smallest on goal pole

Path cost: 1 per move

Example Problems – Rubik’s Cube
States: list of colors for each cell on each face

Initial state: one specific cube configuration

Operators: rotate row x or column y on face

z direction a

Goal: configuration has only one color on

each face

Path cost: 1 per move

Example Problems – Eight Queens
States: locations of 8 queens on chess board

Initial state: one specific queens

configuration

Operators: move queen x to row y and

column z

Goal: no queen can attack another (cannot

be in same row, column, or diagonal)

Path cost: 0 per move

Example Problems –
Missionaries and Cannibals
States: number of missionaries, cannibals,
and boat on near river bank

Initial state: all objects on near river bank

Operators: move boat with x missionaries

and y cannibals to other side of river
• no more cannibals than missionaries on
either river bank or in boat
• boat holds at most m occupants

Goal: all objects on far river bank

Path cost: 1 per river crossing

Example Problems –Water Jug
States: Contents of 4-gallon jug and 3-gallon
jug

Initial state: (0,0)

Operators:
• fill jug x from faucet
• pour contents of jug x in jug y until y full
• dump contents of jug x down drain

Goal: (2,n)

Path cost: 1 per fill

Sample Search Problems
• Graph coloring
• Protein folding
• Game playing
• Airline travel
• Proving algebraic equalities
• Robot motion planning
 Visualize Search Space as a Tree
• States are nodes
• Actions are edges
• Initial state is root
• Solution is path
from root to goal
node
• Edges sometimes
have associated
costs
• States resulting
from operator are
children
Search Problem Example (as a tree)
 Search Function –
Uninformed Searches
Open = initial state // open list is all generated states
// that have not been “expanded”
While open not empty // one iteration of search algorithm
state = First(open) // current state is first state in open
Pop(open) // remove new current state from open
if Goal(state) // test current state for goal condition
return “succeed” // search is complete
// else expand the current state by
// generating children and
// reorder open list per search strategy
else open = QueueFunction(open, Expand(state))
Return “fail”
 Search Strategies
• Search strategies differ only in
QueuingFunction
• Features by which to compare search
strategies
– Completeness (always find solution)
– Cost of search (time and space)
– Cost of solution, optimal solution
– Make use of knowledge of the domain
• “uninformed search” vs. “informed search”
 Breadth-First Search
• Generate children of a state, QueueingFn
adds the children to the end of the open list
• Level-by-level search
• Order in which children are inserted on
open list is arbitrary
• In tree, assume children are considered left-
to-right unless specified differently
• Number of children is “branching factor” b
BFS Examples

b=2

Example trees

Search algorithms applet

 Analysis
• Assume goal node at level d with constant branching factor b

• Time complexity (measured in #nodes generated)

1 (1st level ) + b (2nd level) + b2 (3rd level) + … + bd (goal level) + (bd+1 – b) = O(bd+1)

• This assumes goal on far right of level

• Space complexity
At most majority of nodes at level d + majority of nodes at level d+1 = O(bd+1)
Exponential time and space

• Features
Simple to implement
Complete
Finds shortest solution (not necessarily least-cost unless all operators have equal cost)
 Analysis
• See what happens with b=10
– expand 10,000 nodes/second
– 1,000 bytes/node
Depth Nodes Time Memory

2 1110 .11 seconds 1 megabyte

4 111,100 11 seconds 106 megabytes

6 107 19 minutes 10 gigabytes

8 109 31 hours 1 terabyte

10 1011 129 days 101 terabytes

12 1013 35 years 10 petabytes

15 1015 3,523 years 1 exabyte

 Depth-First Search
• QueueingFn adds the children to the
front of the open list
• BFS emulates FIFO queue
• DFS emulates LIFO stack
• Net effect
– Follow leftmost path to bottom, then
backtrack
– Expand deepest node first
DFS Examples
 Analysis
• Time complexity
In the worst case, search entire space
Goal may be at level d but tree may continue to level m, m>=d
O(bm)
Particularly bad if tree is infinitely deep

• Space complexity
Only need to save one set of children at each level
1 + b + b + … + b (m levels total) = O(bm)
For previous example, DFS requires 118kb instead of 10 petabytes for d=12 (10 billion times less)

• Benefits
May not always find solution
Solution is not necessarily shortest or least cost
If many solutions, may find one quickly (quickly moves to depth d)
Simple to implement
Space often bigger constraint, so more usable than BFS for large problems
Comparison of Search Techniques

DFS BFS

Complete N Y

Optimal N N

Heuristic N N

Time bm bd+1

Space bm bd+1
 Avoiding Repeated States
Can we do it?

• Do not return to parent or grandparent state

– In 8 puzzle, do not move up right after down
• Do not create solution paths with cycles
• Do not generate repeated states (need to store
and check potentially large number of states)
 Maze Example
• States are cells in a maze
• Move N, E, S, or W
• What would BFS do
(expand E, then N, W, S)?
• What would DFS do?
• What if order changed to
N, E, S, W and loops are
prevented?
Uniform Cost Search (Branch&Bound)

• QueueingFn is SortByCostSoFar
• Cost from root to current node n is g(n)
– Add operator costs along path
• First goal found is least-cost solution
• Space & time can be exponential because large
subtrees with inexpensive steps may be explored
before useful paths with costly steps
• If costs are equal, time and space are O(bd)
– Otherwise, complexity related to cost of optimal
solution
 UCS Example

Open list: C
 UCS Example

Open list: B(2) T(1) O(3) E(2) P(5)

 UCS Example

Open list: T(1) B(2) E(2) O(3) P(5)

 UCS Example

Open list: B(2) E(2) O(3) P(5)

 UCS Example

Open list: E(2) O(3) P(5)

 UCS Example

Open list: E(2) O(3) A(3) S(5) P(5) R(6)

 UCS Example

Open list: O(3) A(3) S(5) P(5) R(6)

 UCS Example

Open list: O(3) A(3) S(5) P(5) R(6) G(10)

 UCS Example

Open list: A(3) S(5) P(5) R(6) G(10)

 UCS Example

Open list: A(3) I(4) S(5) N(5) P(5) R(6) G(10)

 UCS Example

Open list: I(4) P(5) S(5) N(5) R(6) G(10)

 UCS Example

Open list: P(5) S(5) N(5) R(6) Z(6) G(10)

 UCS Example

Open list: S(5) N(5) R(6) Z(6) F(6) D(8) G(10) L(10)
 UCS Example

Open list: N(5) R(6) Z(6) F(6) D(8) G(10) L(10)

 UCS Example

Open list: Z(6) F(6) D(8) G(10) L(10)

 UCS Example

Open list: F(6) D(8) G(10) L(10)

UCS Example
 Comparison of Search Techniques
DFS BFS UCS

Complete N Y Y

Optimal N N Y

Heuristic N N N

Time bm bd+1 bm

Space bm bd+1 bm
 Iterative Deepening Search
• DFS with depth bound
• QueuingFn is enqueue at front as with
DFS
– Expand(state) only returns children such that
depth(child) <= threshold
– This prevents search from going down
infinite path
• First threshold is 1
– If do not find solution, increment threshold
and repeat
Examples
 Analysis
• What about the repeated work?
• Time complexity (number of generated nodes)
[b] + [b + b2] + .. + [b + b2 + .. + bd]
(d)b + (d-1) b2 + … + (1) bd
O(bd)
 Analysis
• Repeated work is approximately 1/b of total
work
Negligible
Example: b=10, d=5
N(BFS) = 1,111,100
N(IDS) = 123,450
• Features
– Shortest solution, not necessarily least cost
– Is there a better way to decide threshold? (IDA*)
 Comparison of Search Techniques
DFS BFS UCS IDS

Complete N Y Y Y

Optimal N N Y N

Heuristic N N N N

Time bm bd+1 bm bd
Space bm bd+1 bm bd
 Bidirectional Search
• Search forward from
initial state to goal AND
backward from goal state
to initial state
• Can prune many options
• Considerations
– Which goal state(s) to use
– How determine when
searches overlap
– Which search to use for
each direction
– Here, two BFS searches
• Time and space is O(bd/2)
 Informed Searches
• Best-first search, Hill climbing, Beam search, A*, IDA*, RBFS, SMA*
• New terms
– Heuristics
– Optimal solution
– Informedness
– Hill climbing problems
– Admissibility
• New parameters
– g(n) = estimated cost from initial state to state n
– h(n) = estimated cost (distance) from state n to closest goal
– h(n) is our heuristic
• Robot path planning, h(n) could be Euclidean distance
• 8 puzzle, h(n) could be #tiles out of place
• Search algorithms which use h(n) to guide search are heuristic
search algorithms
 Best-First Search
• QueueingFn is sort-by-h
• Best-first search only as good as heuristic
– Example heuristic for 8 puzzle:
Manhattan Distance
Example
Example
Example
Example
Example
Example
Example
Example
Example
 Comparison of Search Techniques
DFS BFS UCS IDS Best

Complete N Y Y Y N

Optimal N N Y N N

Heuristic N N N N Y

Time bm bd+1 bm bd bm

Space bm bd+1 bm bd bm
 Generate-and-Test

Algorithm
1. Generate a possible solution.
2. Test to see if this is actually a solution.
3. Quit if a solution has been found.
Otherwise, return to step 1.

76
 Generate-and-Test

• Acceptable for simple problems.

• Inefficient for problems with large space.

77
 Generate-and-Test

• Exhaustive generate-and-test.
• Heuristic generate-and-test: not consider
paths that seem unlikely to lead to a solution.
• Plan generate-test:
- Create a list of candidates.
- Apply generate-and-test to that list.

78
 Generate-and-Test

Example: coloured blocks

“Arrange four 6-sided cubes in a row, with each
side of
each cube painted one of four colours, such that
on all four
sides of the row one block face of each colour is
showing.”

79
 Generate-and-Test

Example: coloured blocks

Heuristic: if there are more red faces than other

colours
then, when placing a block with several red
faces, use few
of them as possible as outside faces.

80
 Local search and optimization
• Previously: systematic exploration of search space.
– Backtrack search
– Can solve n-queen problems for n = 200

• Different algorithms can be used

– Local search
– Can solve n-queen for n = 1,000,000
 Local search and optimization
• Local search
– Keep track of single current state
– Move only to neighboring states
– Ignore paths

• Advantages:
– Use very little memory
– Can often find reasonable solutions in large or infinite (continuous) state spaces.

• “Pure optimization” problems

– All states have an objective function
– Goal is to find state with max (or min) objective value
– Does not quite fit into CSP formulation
– Local search can do quite well on these problems.
Hill Climbing (Greedy Search)
• QueueingFn is sort-by-h
– Only keep lowest-h state on open list
• Best-first search is tentative
• Hill climbing is irrevocable
• Features
– Much faster
– Less memory
– Dependent upon h(n)
– If bad h(n), may prune away all goals
– Not complete
Example
Example
 Hill Climbing Issues
• Also referred to as gradient descent
• Foothill problem / local maxima / local minima
• Can be solved with random walk or more steps
• Other problems: ridges, plateaus

global maxima
values

local maxima

states
Comparison of Search Techniques
DFS BFS UCS IDS Best HC

Complete N Y Y Y N N

Optimal N N Y N N N

Heuristic N N N N Y Y

Time bm bd+1 bm bd bm mn

Space bm bd+1 bm bd bm b
 Hill Climbing

• Searching for a goal state = Climbing to the

top of a hill

88
 Hill Climbing

• Generate-and-test + direction to move.

• Heuristic function to estimate how close a
given state is to a goal state.

89
 Simple Hill Climbing

Algorithm
1. Evaluate the initial state.
2. Loop until a solution is found or there are no
new operators left to be applied:
- Select and apply a new operator
- Evaluate the new state:
goal  quit
better than current state  new current state

90
 Simple Hill Climbing
• Evaluation function as a way to inject task-
specific knowledge into the control process.

91
 Simple Hill Climbing

Example: coloured blocks

Heuristic function: the sum of the number of

different
colours on each of the four sides (solution = 16).

92
 Steepest-Ascent Hill Climbing (Gradient
Search)

• Considers all the moves from the current

state.
• Selects the best one as the next state.

93
 Steepest-Ascent Hill Climbing (Gradient
Search)

Algorithm
1. Evaluate the initial state.
2. Loop until a solution is found or a complete
iteration produces no change to current state:
- SUCC = a state such that any possible successor of the
current state will be better than SUCC (the worst state).
- For each operator that applies to the current state, evaluate
the new state:
goal  quit
better than SUCC  set SUCC to this state
- SUCC is better than the current state  set the current
state to SUCC.
94
 Hill-climbing search
function HILL-CLIMBING( problem) return a state that is a local maximum
input: problem, a problem
local variables: current, a node.
neighbor, a node.

current  MAKE-NODE(INITIAL-STATE[problem])
loop do
neighbor  a highest valued successor of current
if VALUE [neighbor] ≤ VALUE[current] then return STATE[current]
current  neighbor

This version of HILL-CLIMBING found local maximum.

 Hill-climbing search
• “a loop that continuously moves in the direction of increasing value”
– terminates when a peak is reached
– Aka greedy local search

• Value can be either

– Objective function value
– Heuristic function value (minimized)

• Hill climbing does not look ahead of the immediate neighbors of the current state.

• Can randomly choose among the set of best successors, if multiple have the best value

• Characterized as “trying to find the top of Mount Everest while in a thick fog”
 Hill climbing and local maxima
• When local maxima exist, hill climbing is
suboptimal
• Simple (often effective) solution
– Multiple random restarts
 Hill-climbing example
• 8-queens problem, complete-state formulation
– All 8 queens on the board in some configuration

• Successor function:
– move a single queen to another square in the same column.

• Example of a heuristic function h(n):

– the number of pairs of queens that are attacking each other
(directly or indirectly)
– (so we want to minimize this)
 Hill-climbing example

(c1 c2 c3 c4 c5 c6 c7 c8) = (5 6 7 4 5 6 7 6)

Current state: h=17

Shown is the h-value for each possible successor in each column

 A local minimum for 8-queens

A local minimum in the 8-queens state space (h=1)

 Other drawbacks

• Ridge = sequence of local maxima difficult for greedy

algorithms to navigate

• Plateau = an area of the state space where the evaluation

function is flat.
 Performance of hill-climbing on 8-queens

• Randomly generated 8-queens starting states…

• 14% the time it solves the problem

• 86% of the time it get stuck at a local minimum

• However…
– Takes only 4 steps on average when it succeeds
– And 3 on average when it gets stuck
– (for a state space with ~17 million states)
Possible solution…sideways moves
• If no downhill (uphill) moves, allow sideways moves in hope that
algorithm can escape
– Need to place a limit on the possible number of sideways moves to avoid
infinite loops

• For 8-queens
– Now allow sideways moves with a limit of 100
– Raises percentage of problem instances solved from 14 to 94%

– However….
• 21 steps for every successful solution
• 64 for each failure
 Hill-climbing variations
• Stochastic hill-climbing
– Random selection among the uphill moves.
– The selection probability can vary with the steepness of
the uphill move.

• First-choice hill-climbing
– stochastic hill climbing by generating successors randomly
until a better one is found
– Useful when there are a very large number of successors

• Random-restart hill-climbing
– Tries to avoid getting stuck in local maxima.
 Hill-climbing with random restarts
• Different variations
– For each restart: run until termination v. run for a fixed time
– Run a fixed number of restarts or run indefinitely

• Analysis
– Say each search has probability p of success
• E.g., for 8-queens, p = 0.14 with no sideways moves

– Expected number of restarts?

– Expected number of steps taken?
 Hill Climbing: Disadvantages
Local maximum
A state that is better than all of its neighbours,
but not
better than some other states far away.

106
 Hill Climbing: Disadvantages
Plateau
A flat area of the search space in which all
neighbouring
states have the same value.

107
 Hill Climbing: Disadvantages
Ridge
The orientation of the high region, compared to
the set
of available moves, makes it impossible to climb
up.
However, two moves executed serially may
increase
the height.

108
 Hill Climbing: Disadvantages

Ways Out
• Backtrack to some earlier node and try going
in a different direction.
• Make a big jump to try to get in a new section.
• Moving in several directions at once.

109
 Hill Climbing: Disadvantages
• Hill climbing is a local method:
Decides what to do next by looking only at the
“immediate” consequences of its choices.
• Global information might be encoded in
heuristic functions.

110
Hill Climbing: Disadvantages

Start Goal
A D

D C

C B

B A

Blocks World

111
 Hill Climbing: Disadvantages

Start Goal
A D
0 4
D C

C B

B A

Blocks World
Local heuristic:
+1 for each block that is resting on the thing it is supposed to be
resting on.
-1 for each block that is resting on a wrong thing.

112
Hill Climbing: Disadvantages

0 A 2

D D

C C

B B A

113
Hill Climbing: Disadvantages

D 2
C

B A

A 0
0
D

C C D C 0

B B A B A D

114
 Hill Climbing: Disadvantages

Start Goal
A D
-6 6
D C

C B

B A

Blocks World
Global heuristic:
For each block that has the correct support structure: +1 to
every block in the support structure.
For each block that has a wrong support structure: -1 to
every block in the support structure.
115
Hill Climbing: Disadvantages

D -3
C

B A

A -6
-2
D

C C D C -1

B B A B A D

116
 Hill Climbing: Conclusion
• Can be very inefficient in a large, rough
problem space.

• Global heuristic may have to pay for

computational complexity.

• Often useful when combined with other

methods, getting it started right in the right
general neighbourhood.
117
 Simulated Annealing
• A variation of hill climbing in which, at the
beginning of the process, some downhill moves
may be made.

• To do enough exploration of the whole space

early on, so that the final solution is relatively
insensitive to the starting state.
• Lowering the chances of getting caught at a local
maximum, or plateau, or a ridge.
118
 Simulated Annealing
Physical Annealing
• Physical substances are melted and then gradually
cooled until some solid state is reached.
• The goal is to produce a minimal-energy state.
• Annealing schedule: if the temperature is lowered
sufficiently slowly, then the goal will be attained.
• Nevertheless, there is some probability for a
transition to a higher energy state: e-E/kT.
 Simulated Annealing

Algorithm
1. Evaluate the initial state.
2. Loop until a solution is found or there are no new operators
left to be applied:
- Set T according to an annealing schedule
- Selects and applies a new operator
- Evaluate the new state:
goal  quit
E = Val(current state) - Val(new state)
E < 0  new current state
else  new current state with probability e-E/kT.

120
 Best-First Search
• Depth-first search: not all competing branches
having to be expanded.

• Breadth-first search: not getting trapped on

dead-end paths.
 Combining the two is to follow a single path
at a time, but switch paths whenever some
competing path look more promising than
the current one.
121
 Best-First Search

A A A

B C D B C D
3 5 1 3 5
E F
4 6
A A

B C D B C D
5 5
G H E F G H E F
6 5 4 6 6 5 6
I J
2 1
122
 Best-First Search
• OPEN: nodes that have been generated, but
have not examined.
This is organized as a priority queue.

• CLOSED: nodes that have already been

examined.
Whenever a new node is generated, check
whether it has been generated before.
123
 Best-First Search

Algorithm
1. OPEN = {initial state}.
2. Loop until a goal is found or there are no
nodes left in OPEN:
- Pick the best node in OPEN
- Generate its successors
- For each successor:
new  evaluate it, add it to OPEN, record its parent
generated before  change parent, update successors

124
 Best-First Search
• Greedy search:
h(n) = estimated cost of the cheapest path
from node n to a goal state.

125
 Best-First Search
• Uniform-cost search:
g(n) = cost of the cheapest path from the
initial state to node n.

126
 Best-First Search
• Greedy search:
h(n) = estimated cost of the cheapest path
from node n to a goal state.
Neither optimal nor complete

127
 Best-First Search
• Greedy search:
h(n) = estimated cost of the cheapest path from
node n to a goal state.
Neither optimal nor complete

• Uniform-cost search:
g(n) = cost of the cheapest path from the initial
state to node n.
Optimal and complete, but very inefficient
128
 Best-First Search
• Algorithm A* (Hart et al., 1968):
f(n) = g(n) + h(n)

h(n) = cost of the cheapest path from node n to a

goal state.
g(n) = cost of the cheapest path from the initial
state to node n.

129
 Best-First Search
• Algorithm A*:
f*(n) = g*(n) + h*(n)

h*(n) (heuristic factor) = estimate of h(n).

g*(n) (depth factor) = approximation of g(n)

found by A* so far.

130
 Problem Reduction

Goal: Acquire TV set

Goal: Steal TV set Goal: Earn some money Goal: Buy TV set

AND-OR Graphs

Algorithm AO* (Martelli & Montanari 1973, Nilsson 1980)

131
 Problem Reduction: AO*

A A 6
5 9
B C D
3 4 5

A 9 A 11
9 12
B C D 10 B 6 C D 10
3 4 4
E F G H E F
4 4 5 7 4 4

132
 Problem Reduction: AO*

A 11 A 14

B 13 C 10 B 13 C 15

D 5 E 6 F 3 D 5 E 6 F 3

G 5 G 10

Necessary backward propagation H 9

133
 Constraint Satisfaction
• Many AI problems can be viewed as problems
of constraint satisfaction.

Cryptarithmetic puzzle:
SEND

MORE
MONEY

134
 Constraint Satisfaction
• As compared with a straightforard search
procedure, viewing a problem as one of
constraint satisfaction can reduce substantially
the amount of search.

135
 Constraint Satisfaction
• Operates in a space of constraint sets.
• Initial state contains the original constraints given in
the problem.

• A goal state is any state that has been constrained

“enough”.

136
 Constraint Satisfaction
Two-step process:
1. Constraints are discovered and propagated as
far as possible.
2. If there is still not a solution, then search
begins, adding new constraints.

137
Initial state:
• No two letters have
the same value. M=1
• The sum of the digits S = 8 or 9 SEND
must be as shown. O=0 
N=E+1 MORE
C2 = 1
N+R>8
E9
MONEY
E=2
N=3
R = 8 or 9
2 + D = Y or 2 + D = 10 + Y
C1 = 0 C1 = 1

2+D=Y 2 + D = 10 + Y
N + R = 10 + E D=8+Y
R=9 D = 8 or 9
S =8
D=8 D=9
Y=0 Y=1
138
 Constraint Satisfaction
Two kinds of rules:
1. Rules that define valid constraint
propagation.
2. Rules that suggest guesses when necessary.

139
 Local Beam Search
• QueueingFn is sort-by-h
– Only keep best (lowest-h) n nodes on open list
• n is the “beam width”
– n = 1, Hill climbing
– n = infinity, Best first search
Example
Example
Example
Example
Example
Example
Example
Example
Example
Comparison of Search Techniques
DFS BFS UCS IDS Best HC Beam

Complete N Y Y Y N N N

Optimal N N Y N N N N

Heuristic N N N N Y Y Y

Time bm bd+1 bm bd bm bm nm

Space bm bd+1 bm bd bm b bn
 A*
• QueueingFn is sort-by-f
– f(n) = g(n) + h(n)
• Note that UCS and Best-first both improve
search
– UCS keeps solution cost low
– Best-first helps find solution quickly
• A* combines these approaches
 Power of f
• If heuristic function is wrong it either
– overestimates (guesses too high)
– underestimates (guesses too low)
• Overestimating is worse than underestimating
• A* returns optimal solution if h(n) is admissible
– heuristic function is admissible if never
overestimates true cost to nearest goal
– if search finds optimal solution using admissible
heuristic, the search is admissible
 Overestimating
A (15)
3 3
2

B (6) C (20) D (10)

15 6 20 10 5

E (20) F(0) G (12) H (20) I(0)

• Solution cost: • Open list:

– ABF = 9 – A (15) B (9) F (9)
– ADI = 8 • Missed optimal solution
 Example

A* applied to 8 puzzle

A* search applet
Example
Example
Example
Example
Example
Example
Example
Example
 Optimality of A*
• Suppose a suboptimal goal G2 is on the open list
• Let n be unexpanded node on smallest-cost path to
optimal goal G1

f(G2) = g(G2) since h(G2) = 0

>= g(G1) since G2 is suboptimal
>= f(n) since h is admissible

Since f(G2) > f(n), A* will never select G2 for expansion

Comparison of Search Techniques
DFS BFS UCS IDS Best HC Beam A*

Complete N Y Y Y N N N Y

Optimal N N Y N N N N Y

Heuristic N N N N Y Y Y Y

Time bm bd+1 bm bd bm bm nm bm

Space bm bd+1 bm bd bm b bn bm
 IDA*
• Series of Depth-First Searches
• Like Iterative Deepening Search, except
– Use A* cost threshold instead of depth threshold
– Ensures optimal solution
• QueuingFn enqueues at front if f(child) <= threshold
• Threshold
– h(root) first iteration
– Subsequent iterations
• f(min_child)
• min_child is the cut off child with the minimum f value
– Increase always includes at least one new node
– Makes sure search never looks beyond optimal cost solution
Example
Example
Example
Example
Example
Example
Example
Example
Example
Example
Example
Example
Example
Example
Example
Example
Example
Example
Example
Example
Example
Example
Example
Example
Example
Example
Example
Example
 Analysis
• Some redundant search
– Small amount compared to work done on last
iteration
• Dangerous if continuous-valued h(n) values or if
values very close
– If threshold = 21.1 and value is 21.2, probably only
include 1 new node each iteration
• Time complexity is O(bm)
• Space complexity is O(m)
Comparison of Search Techniques
DFS BFS UCS IDS Best HC Beam A* IDA*

Complete N Y Y Y N N N Y Y

Optimal N N Y N N N N Y Y

Heuristic N N N N Y Y Y Y Y

Time bm bd+1 bm bd bm bm nm bm bm

Space bm bd+1 bm bd bm b bn bm bm
 Recursive Best First Search
• Recursive Best First Search
– Linear space variant of A*
• Perform A* search but discard subtrees when
perform recursion
• Keep track of alternative (next best) subtree
• Expand subtree until f value greater than bound
• Update f values before (from parent)
and after (from descendant) recursive call
 Algorithm
// Input is current node and f limit
// Returns goal node or failure, updated limit
RBFS(n, limit)
if Goal(n)
return n
children = Expand(n)
if children empty
return failure, infinity
for each c in children
f[c] = max(g(c)+h(c), f[n]) // Update f[c] based on parent
repeat
best = child with smallest f value
if f[best] > limit
return failure, f[best]
alternative = second-lowest f-value among children
newlimit = min(limit, alternative)
result, f[best] = RBFS(best, newlimit) // Update f[best] based on descendant
if result not equal to failure
return result
Example
Example
Example
Example
Example
Example
 Analysis
• Optimal if h(n) is admissible
• Space is O(bm)
• Features
– Potentially exponential time in cost of solution
– More efficient than IDA*
– Keeps more information than IDA* but benefits
from storing this information
 SMA*
• Simplified Memory-Bounded A* Search
• Perform A* search
• When memory is full
– Discard worst leaf (largest f(n) value)
– Back value of discarded node to parent
• Optimal if solution fits in memory
 Example
• Let MaxNodes = 3
• Initially B&G added to open list,
then hit max
• B is larger f value so discard but
save f(B)=15 at parent A
– Add H but f(H)=18. Not a goal and
cannot go deper, so set
f(h)=infinity and save at G.
• Generate next child I with
f(I)=24, bigger child of A. We
have seen all children of G, so
reset f(G)=24.
• Regenerate B and child C. This is
not goal so f(c) reset to infinity
• Generate second child D with
f(D)=24, backing up value to
ancestors
• D is a goal node, so search
terminates.
 Heuristic Functions
• Q: Given that we will only use heuristic
functions that do not overestimate, what type
of heuristic functions (among these) perform
best?
• A: Those that produce higher h(n) values.
 Reasons
• Higher h value means closer to actual distance
• Any node n on open list with
– f(n) < f*(goal)
– will be selected for expansion by A*
• This means if a lot of nodes have a low
underestimate (lower than actual optimum cost)
– All of them will be expanded
– Results in increased search time and space
 Informedness
• If h1 and h2 are both admissible and
• For all x, h1(x) > h2(x), then h1 “dominates” h2
– Can also say h1 is “more informed” than h2
• Example
– h1(x): | xgoal  x |
– h2(x): Euclidean distance ( x goal  x) 2  ( y goal  y ) 2
– h2 dominates h1
 Effect on Search Cost
• If h2(n) >= h1(n) for all n (both are admissible)
– then h2 dominates h1 and is better for search
• Typical search costs
– d=14, IDS expands 3,473,941 nodes
• A* with h1 expands 539 nodes
• A* with h2 expands 113 nodes
– d=24, IDS expands ~54,000,000,000 nodes
• A* with h1 expands 39,135 nodes
• A* with h2 expands 1,641 nodes
Which of these heuristics are admissible?
Which are more informed?

• h1(n) = #tiles in wrong position

• h2(n) = Sum of Manhattan distance between each tile and goal location for the tile
• h3(n) = 0
• h4(n) = 1
• h5(n) = min(2, h*[n])
• h6(n) = Manhattan distance for blank tile
• h7(n) = max(2, h*[n])
 Generating Heuristic Functions
• Generate heuristic for simpler (relaxed)
problem
– Relaxed problem has fewer restrictions
– Eight puzzle where multiple tiles can be in the
same spot
– Cost of optimal solution to relaxed problem is an
admissible heuristic for the original problem
• Learn heuristic from experience
Iterative Improvement Algorithms
• Hill climbing
• Simulated annealing
• Genetic algorithms
 Iterative Improvement Algorithms
• For many optimization problems, solution
path is irrelevant
– Just want to reach goal state
• State space / search space
– Set of “complete” configurations
– Want to find optimal configuration (or at
least one that satisfies goal constraints)
• For these cases, use iterative improvement algorithm
– Keep a single current state
– Try to improve it
• Constant memory
 Example
• Traveling salesman

• Start with any complete tour

• Operator: Perform pairwise exchanges
 Example
• N-queens

• Put n queens on an n × n board with no two

queens on the same row, column, or diagonal
• Operator: Move queen to reduce #conflicts
Hill Climbing (gradient ascent/descent)

• “Like climbing Mount Everest in thick fog with

amnesia”
 Local Beam Search
• Keep k states instead of 1
• Choose top k of all successors
• Problem
– Many times all k states end up on same local hill
– Choose k successors RANDOMLY
– Bias toward good ones
• Similar to natural selection
 Local beam search
• Keep track of k states instead of one
– Initially: k randomly selected states
– Next: determine all successors of k states
– If any of successors is goal  finished
– Else select k best from successors and repeat.

• Major difference with random-restart search

– Information is shared among k search threads.

• Can suffer from lack of diversity.

– Stochastic beam search
• choose k successors proportional to state quality.
 Search using Simulated Annealing
• Simulated Annealing = hill-climbing with non-deterministic search

• Basic ideas:
– like hill-climbing identify the quality of the local improvements
– instead of picking the best move, pick one randomly
– say the change in objective function is d
– if d is positive, then move to that state
– otherwise:
• move to this state with probability proportional to d
• thus: worse moves (very large negative d) are executed less often
– however, there is always a chance of escaping from local maxima
– over time, make it less likely to accept locally bad moves
– (Can also make the size of the move random as well, i.e., allow
“large” steps in state space)
Physical Interpretation of Simulated Annealing

• Annealing = physical process of cooling a liquid

or metal until particles achieve a certain
frozen crystal state
• simulated annealing:
– free variables are like particles
– seek “low energy” (high quality) configuration
– get this by slowly reducing temperature T, which particles
move around randomly
 Simulated annealing
function SIMULATED-ANNEALING( problem, schedule) return a solution state
input: problem, a problem
schedule, a mapping from time to temperature
local variables: current, a node.
next, a node.
T, a “temperature” controlling the probability of downward steps

current  MAKE-NODE(INITIAL-STATE[problem])
for t  1 to ∞ do
T  schedule[t]
if T = 0 then return current
next  a randomly selected successor of current
∆E  VALUE[next] - VALUE[current]
if ∆E > 0 then current  next
else current  next only with probability e∆E /T
More Details on Simulated Annealing
– Lets say there are 3 moves available, with changes in the objective
function of d1 = -0.1, d2 = 0.5, d3 = -5. (Let T = 1).
– pick a move randomly:
• if d2 is picked, move there.
• if d1 or d3 are picked, probability of move = exp(d/T)
• move 1: prob1 = exp(-0.1) = 0.9,
– i.e., 90% of the time we will accept this move
• move 3: prob3 = exp(-5) = 0.05
– i.e., 5% of the time we will accept this move

– T = “temperature” parameter
• high T => probability of “locally bad” move is higher
• low T => probability of “locally bad” move is lower
• typically, T is decreased as the algorithm runs longer
– i.e., there is a “temperature schedule”
Simulated Annealing in Practice
– method proposed in 1983 by IBM researchers for
solving VLSI layout problems (Kirkpatrick et al,
Science, 220:671-680, 1983).
• theoretically will always find the global optimum (the best
solution)

– useful for some problems, but can be very slow

– slowness comes about because T must be decreased very
gradually to retain optimality
• In practice how do we decide the rate at which to
decrease T? (this is a practical problem with this method)
 Genetic algorithms
• Different approach to other search algorithms
– A successor state is generated by combining two parent states

• A state is represented as a string over a finite alphabet (e.g. binary)

– 8-queens
• State = position of 8 queens each in a column
=> 8 x log(8) bits = 24 bits (for binary representation)

• Start with k randomly generated states (population)

• Evaluation function (fitness function).

– Higher values for better states.
– Opposite to heuristic function, e.g., # non-attacking pairs in 8-queens

• Produce the next generation of states by “simulated evolution”

– Random selection
– Crossover
– Random mutation
 Genetic algorithms

Fitness function: number of non-attacking pairs of queens (min = 0, max = 8 × 7/2 = 28)
•
• 24/(24+23+20+11) = 31%
•
• 4 states for
23/(24+23+20+11) 2 pairs
= 29% etc of 2 states randomly New states Random
• 8-queens selected based on fitness. after crossover mutation
problem Random crossover points applied
selected
 Genetic algorithms

Has the effect of “jumping” to a completely different new

part of the search space (quite non-local)
 Genetic algorithm pseudocode
function GENETIC_ALGORITHM( population, FITNESS-FN) return an individual
input: population, a set of individuals
FITNESS-FN, a function which determines the quality of the individual
repeat
new_population  empty set
loop for i from 1 to SIZE(population) do
x  RANDOM_SELECTION(population, FITNESS_FN) y
RANDOM_SELECTION(population, FITNESS_FN)
child  REPRODUCE(x,y)
if (small random probability) then child  MUTATE(child )
add child to new_population
population  new_population
until some individual is fit enough or enough time has elapsed
return the best individual
 Comments on genetic algorithms
• Positive points
– Random exploration can find solutions that local search can’t
• (via crossover primarily)
– Appealing connection to human evolution
• E.g., see related area of genetic programming

• Negative points
– Large number of “tunable” parameters
• Difficult to replicate performance from one problem to another

– Lack of good empirical studies comparing to simpler methods

– Useful on some (small?) set of problems but no convincing evidence that GAs are
better than hill-climbing w/random restarts in general
 Simulated Annealing
• Pure hill climbing is not complete, but pure random search is
inefficient.
• Simulated annealing offers a compromise.
• Inspired by annealing process of gradually cooling a liquid until
it changes to a low-energy state.
• Very similar to hill climbing, except include a user-defined
temperature schedule.
• When temperature is “high”, allow some random moves.
• When temperature “cools”, reduce probability of random move.
• If T is decreased slowly enough, guaranteed to reach best state.
 Algorithm
function SimulatedAnnealing(problem, schedule) // returns solution state
current = MakeNode(Initial-State(problem))
for t = 1 to infinity
T = schedule[t]
if T = 0
return current
next = randomly-selected child of current
= Value[next] - Value[current]
 E
if E > 0 e T
current = nextE // if better than accept state

else current = next with probability

Simulated annealing applet

Traveling salesman simulated annealing applet

 Genetic Algorithms
• What is a Genetic Algorithm (GA)?
– An adaptation procedure based on the mechanics of
natural genetics and natural selection
• Gas have 2 essential components
– Survival of the fittest
– Recombination
• Representation
– Chromosome = string
– Gene = single bit or single subsequence in string,
represents 1 attribute
 Humans
• DNA made up of 4 nucleic acids (4-bit code)
• 46 chromosomes in humans, each contain 3 billion DNA
• 43 billion combinations of bits
• Can random search find humans?
– Assume only 0.1% genome must be discovered, 3(10 6) nucleotides
– Assume very short generation, 1 generation/second
– 3.2(10107) individuals per year, but 101.8(107) alternatives
6
– 101810 years to generate human randomly
• Self reproduction, self repair, adaptability are the rule in natural
systems, they hardly exist in the artificial world
• Finding and adopting nature’s approach to computational design
should unlock many doors in science and engineering
 GAs Exhibit Search
• Each attempt a GA makes towards a solution is called a
chromosome
– A sequence of information that can be interpreted as a possible
solution
• Typically, a chromosome is represented as sequence of
binary digits
– Each digit is a gene
• A GA maintains a collection or population of
chromosomes
– Each chromosome in the population represents a different
guess at the solution
 The GA Procedure
1. Initialize a population (of solution guesses)
2. Do (once for each generation)
a. Evaluate each chromosome in the population
using a fitness function
b. Apply GA operators to population to create a new
population
3. Finish when solution is reached or number of
generations has reached an allowable
maximum.
 Common Operators
• Reproduction
• Crossover
• Mutation
 Reproduction
• Select individuals x according to their fitness
values f(x)
– Like beam search
• Fittest individuals survive (and possibly mate)
for next generation
 Crossover
• Select two parents
• Select cross site
• Cut and splice pieces of one parent to those of
the other

11111 11000
00000 00111
 Mutation
• With small probability, randomly alter 1 bit
• Minor operator
• An insurance policy against lost bits
• Pushes out of local minima
Population: Goal: 0 1 1 1 1 1

110000 Mutation needed to find the goal

101000
100100
010000
 Example
• Solution = 0 0 1 0 1 0
• Fitness(x) = #digits that match solution
A) 0 1 0 1 0 1 Score: 1
B) 1 1 1 1 0 1 Score: 1
C) 0 1 1 0 1 1 Score: 3
D) 1 0 1 1 0 0 Score: 3

Recombine top two twice.

Note: 64 possible combinations
 Example
• Solution = 0 0 1 0 1 0 • Next generation:
C) 0 1 1 0 1 1 E) 0 0 1 1 0 0
D) 1 0 1 1 0 0 F) 0 1 1 0 1 0

E) 0 | 0 1 1 0 0 Score: 4 G) 0 1 1 | 1 0 0 Score: 3
F) 1 | 1 1 0 1 1 Score: 3 H) 0 0 1 | 0 1 0 Score: 6
G) 0 1 1 0 1 | 0 Score: 4 I) 0 0 | 1 0 1 0 Score: 6
H) 1 0 1 1 0 | 1 Score: 2 J) 0 1 | 1 1 0 0 Score: 3

DONE! Got it in 10 guesses.

 Issues
• How select original population?
• How handle non-binary solution types?
• What should be the size of the population?
• What is the optimal mutation rate?
• How are mates picked for crossover?
• Can any chromosome appear more than once in a
population?
• When should the GA halt?
• Local minima?
• Parallel algorithms?
GAs for Mazes
 GAs for Optimization
• Traveling salesman problem
• Eaters
• Hierarchical GAs for game playing
 GAs for Control
• Simulator
 GAs for Graphic Animation
• Simulator
• Evolving Circles
• 3D Animation
• Scientific American Frontiers
 Biased Roulette Wheel
• For each hypothesis, spin the roulette wheel
to determine the guess
 Inversion
• Invert selected subsequence
• 1 0 | 1 1 0 | 1 1 -> 1 0 0 1 1 1 1
 Elitism
• Some of the best chromosomes from previous
generation replace some of the worst
chromosomes from current generation
 K-point crossover
• Pick k random splice points to crossover parents
• Example
–K=3
1 1 | 1 1 1 | 1 1 | 1 1 1 1 1 -> 1 1 0 0 0 1 1 0 0 0 0 0
00|000|00|00000 001110011111
 Diversity Measure
• Fitness ignores diversity
• As a result, populations tend to become
uniform
• Rank-space method
– Sort population by sum of fitness rank and
diversity rank
– Diversity rank is the result of sorting by the
function 1/d2
 Classifier Systems
• GAs and load balancing
• SAMUEL