PROBLEM SOLVING

BY SEARCH

Nguyễn Ngọc Thảo – Nguyễn Hải Minh

{nnthao, nhminh}@fit.hcmus.edu.vn
Outline
• Problem-solving agents
• Example problems
• Searching for solutions

2
Problem-solving agents

• Well-defined problems and solutions

• Formulating problems
A touring holiday in Romania

A simplified road map of part of Romania, with road distances in miles. 4

Problem-solving process
• Assume the agent always has access to information about
the world by using a map.
• Then, it follows the four-phase problem-solving process

Goal Problem
formulation formulation
Search Execution

5
Problem-solving process
• Goal formulation: Goals organize behavior by limiting the
objectives and hence the actions to be considered.
• E.g., in the touring holiday example, the goal is reaching Bucharest.

• Problem formulation: The agent devises a description of the

states and actions necessary to reach the goal.
• E.g., in the above example, one good model is to consider the
actions of traveling from one city to an adjacent city.
6
Problem-solving process
• Search: The agent simulates sequences of actions in its
model, searching until it finds a solution.
• This is done before taking any action in the real world.
• A solution is a sequence of actions that reaches the goal, e.g., going
from Arad to Sibiu to Faragas to Bucharest.

• Execution: The agent can now execute the actions in the

solution, one at a time.

7
Search problems and solutions
• A search problem can be defined formally as follows.
• A set of possible states that the environment can be in,
called the state space
• E.g., the cities, {Arad, Zerind,..}, in the touring holiday example
• The initial state that the agent starts in
• E.g., in the above example, the agent begins at Arad.
• A set of one or more goal states
• E.g., in the above example, the agent wants to arrive at Bucharest.
• There can be one or several goal states, or the goal is defined by a
property that applies to many states (potentially an infinite number).

8
Search problems and solutions
• A finite set of actions that are executable in state 𝑠, 𝐴𝑐𝑡𝑖𝑜𝑛(𝑠)
• E.g., 𝐴𝑐𝑡𝑖𝑜𝑛(𝐴𝑟𝑎𝑑) = {𝑇𝑜𝑆𝑖𝑏𝑖𝑢, 𝑇𝑜𝑇𝑖𝑚𝑖𝑠𝑜𝑎𝑟𝑎, 𝑇𝑜𝑍𝑒𝑟𝑖𝑛𝑑}

• A transition model, 𝑅𝑒𝑠𝑢𝑙𝑡(𝑠, 𝑎), returns the state that results

from doing action 𝑎 in state 𝑠.
• E.g., 𝑅𝑒𝑠𝑢𝑙𝑡 𝐴𝑟𝑎𝑑, 𝑇𝑜𝑍𝑒𝑟𝑖𝑛𝑑 = 𝑍𝑒𝑟𝑖𝑛𝑑

9
Search problems and solutions
• An action cost function, 𝐴𝑐𝑡𝑖𝑜𝑛 - 𝐶𝑜𝑠𝑡(𝑠, 𝑎, 𝑠’) , gives the
numeric cost of applying action 𝑎 in state 𝑠 to reach state 𝑠’.
• The cost function should reflect the agent’s performance measure.
• E.g., for route-finding agents, the cost might be the length in miles or
the time to complete the action.

• Action costs are assumed to be positive and additive.

• The total cost of a path is the sum of individual action costs.
• An optimal solution has the lowest cost among all solutions.

10
The state space
• The state space can be represented as a graph.
• The vertices are states and the directed edges between
them are actions.

A sequence of actions forms a path,

and a solution is a path from the
initial state to a goal state.

11
Formulating problems
• The formulation of the problem is a model—an abstract
mathematical description—and not the real thing.

How do you solve a

navigation problem?
vs.

Compare the simple atomic state description to an actual cross-country trip,

where the state of the world includes so many things: the traveling
companions, the condition of the road, the weather, the traffic, and so on.
The model deals with the change in location only while leaving out all the
other considerations.
12
Formulating problems
• Abstraction involves removing detail from a representation.
• It creates an approximate and simplified model of the world,
which is critical for automated problem solving.

• A valid abstraction lets any abstract solution be turned into a

solution in the more detailed world.
• An abstraction is useful if carrying out each of the actions in
the solution is easier than the original problem.

13
Formulating problems
• A good abstraction eliminates as much detail as possible.
• Meanwhile, it retains validity and ensures that the abstract
actions are easy to carry out.

14
Example problems

• Standardized problems
• Real-world problems
Example problems
Standardized problems
• Illustrate or exercise various problem-solving methods.
• Descriptions are concise and exact.
• Suitable as a benchmark for comparing the performance of algorithms.

Real-world problems
• Bring solutions to practical issues
• Formulations are idiosyncratic and not standardized.

16
The Vacuum-cleaner world
• States: determined by the agent location and the availability of dirt
• 2 × 22 = 8 possible world states (𝑛 × 2𝑛 for environment with 𝑛 cells)
• Initial state: Any state can be designated as the initial state.
• Actions: Left, Right, and Suck
• Larger models may include Upward and Downward.
• Transition model: The actions have their expected effects.
• Note that moving Left in the leftmost square, moving Right in the
rightmost square, and Sucking in a clean square have no effect.
• Goal states: The states in which every cell is clean.
• Action cost: Each step costs 1.

17
The Vacuum-cleaner world

The state space for the vacuum world

Links denote actions: L = Left, R = Right, S = Suck.
18
The 8-puzzle problem
• States: specified by the location of each of the eight tiles.
• Initial state: Any state can be designated as the initial state.
• Actions: The movement of the blank space—Left, Right, Up, Down
• Transition model: Map a state and action to a resulting state.

• Goal states: Any state could be the goal

• Action cost: Each step costs 1.

19
The 8-puzzle problem

• A member of the family of sliding-block puzzles, NP-complete

• 8-puzzle: 9! / 2 = 181,440 reachable states → easily solved.

• 15-puzzle: 16! / 2—over 10 trillion states → optimally solved in millisecs
• 24-puzzle: around 1025 states → optimally solved in several hours

• Note the parity property that partitions the state space—any given goal
can be reached from exactly half of the possible initial states.
20
The 8-queens problem
• Incremental formulation: add a queen
step-by-step to the empty initial state
• Complete-state formulation: start with
all 8 queens on the board and move
them around
• The action/path cost is trivial because
only the final state counts.

21
The 8-queens: Incremental formulation
• States: Any arrangement of 0 to 8 queens on the board.
• Initial state: No queens on the board
• Actions: Add a queen to any empty square.
• Transition model: Return the board with a queen newly added to
the specified square
• Goal states: 8 queens are on the board, none attacked.
• 64 ∙ 63 ⋯ 57 ≈ 1.8 × 1014 possible sequences to investigate

22
The 8-queens: Incremental formulation
• A better formulation would prohibit placing a queen in any square
that is already attacked.
• States: All possible arrangements of 𝑛 queens (0 ≤ 𝑛 ≤ 8), one
per column in the leftmost 𝑛 columns, no queen attacking another.
• Actions: Add a queen to any square in the leftmost empty column
such that it is not attacked by other queens

• 8-queens: from 1.8 × 1014 states to just 2,057 states

• 100-queens: from 10400 states to about 1052 states.
• A big improvement, yet insufficient to make the problem tractable,

23
Donald Knuth’s 4 problem (1964)
• Starting with the number 4, a sequence of
factorial, square root, and floor operations
will reach any desired positive integer.

620,448,401,733,239,439,360,000 steps

• States: positive numbers → infinite state space

• Initial state: 4
• Actions: Apply factorial (for integers only), square root, and floor operation
• Transition model: given by the operations’ mathematical definitions
• Goal states: The desired positive integer.
• Action cost: Each action costs 1.
• Infinite state spaces arise in tasks like mathematical expressions
generation, circuits, proofs, programs, etc.
24
The route-finding problem
• Consider the airline travel problems solved by a travel-planning Web site.
• States: Each state includes a location (e.g., an airport) and the current time.
• Extra information about “historical” aspects, e.g., previous segments, fare bases,
statuses as domestic or international, are needed.
• Initial state: The user’s home airport.
• Actions: Take any flight from the current location, in any seat class, leaving
after the current time, leaving enough time for within-airport transfer.
• Transition model: The state resulting from taking a flight will have the flight’s
destination as the new location and the flight’s arrival time as the new time.
• Goal states: A destination city.
• Action cost: Depend on different factors of the performance measure
• A combination of monetary cost, waiting time, flight time, customs and
immigration procedures, seat quality, time of day, type of airplane, and so on
25
The touring problems
• Touring problems describe a set of locations that must be
visited, rather than a single goal destination.
• Traveling salesperson problem (TSP): Every city must be
visited exactly once, and the tour is shortest (if required).
• Applications: Stocking machines on shop floors, plan movements of
automatic circuit-board drills, etc.
• It is a NP-hard problem instance.

26
Robot navigation
• Robot navigation generalizes the route-finding problem.
• The search space for a circular robot moving on a flat surface is
essentially two-dimensional.
• The robot has arms and legs that must also be controlled → the
space becomes many-dimensional, one dimension per joint angle.
• Advanced techniques are required just to make the essentially continuous
search space finite.

• Real robots must also handle errors in their sensor readings and
motor controls, with partial observability, and with other agents.

27
Quiz 01: The Towers of Hanoi
•• States: Any arrangement
Formulate the Towersofofdisks in three
Hanoi pegs such
problem with that anypegs
three disk is on
and
top of another larger disk.
three disks.
• Initial state:

• Actions: Take the upper disk from one of the rods and sliding it onto
another rod, on top of the other disks that may already be present on
that rod, without violating the rule of disk sizes.
• Transition model: Return a state given a state and an action
• Goal state:

• Action cost: Each step costs 1

28
Search algorithms

• Best-first search
• Search data structures
Search algorithms
• A search algorithm takes a search problem as input and
returns a solution, or an indication of failure.
• A search tree shows various paths from the initial state,
trying to find a path that reaches a goal state.
• Node = a state in the state space. Edge = action. Root = initial state.

A S

C A B
S
B C C
State space G Search tree
G G
30
State space vs. Search tree
• The state space describes the set of states in the world, and
the actions that allow transitions from one state to another.
• The search tree describes paths between these states,
reaching towards the goal.
• There may be multiple paths any given state, but each node in the
tree has a unique path back to the root.

31
Search tree: An example for Romania map

A partial search tree for finding a route from Arad to Bucharest. Nodes that have been
expanded are lavender with bold letters; nodes on the frontier that have been
generated but not yet expanded are in green; the set of states corresponding to these
two types of nodes are said to have been reached. Nodes that could be generated next
are shown in faint dashed lines.

32
Search tree: Terminologies
• A node is called expanded if we apply available actions onto
that state, leading to the generation of successor nodes.
• Those successors are added to a frontier, available for
expansion at any given point.
• Any state that has had a node generated for it has been
reached.
• Reached nodes = Expanded nodes + Nodes in the frontier.

33
Loopy path can’t be
an optimal path, so
search should not
continue from
there.

34
Search tree: An example for Romania map

A sequence of search trees generated by a graph search on the Romania problem. At each
stage, we have expanded every node on the frontier, extending every path with all
applicable actions that do not result in a state that has already been reached.

35
Graph-search separation property
• The frontier separates two regions of the state-space graph.
• An interior region is where every state has been expanded,
and an exterior region for states not yet been reached.

• The algorithm examines the

states in a systematic fashion,
until it finds a solution.

36
Graph-search separation property

The separation property of graph search, illustrated on a rectangular-grid problem. The

frontier (green) separates the interior (lavender) from the exterior (faint dashed). The
frontier is the set of nodes (and corresponding states) that have been reached but not yet
expanded; the interior is the set of nodes (and corresponding states) that have been
expanded; and the exterior is the set of states that have not been reached. In (a), just the
root has been expanded. In (b), the top frontier node is expanded. In (c), the remaining
successors of the root are expanded in clockwise order.
37
Best-first search
• Best-first search is a general approach that chooses a node
𝑛 with minimum value of some evaluation function, 𝑓 𝑛 .
• The choice of 𝑓 determines the specific search strategy.

38
function BEST-FIRST-SEARCH(problem, f) returns a solution node or failure
node  NODE(STATE=problem.INITIAL)
frontier  a priority queue ordered by f, with node as an element
reached  a lookup table, with one entry with key problem.INITIAL and value node
while not IS-EMPTY(frontier) do
node  POP(frontier)
if problem.IS-GOAL(node.STATE) then return node
for each child in EXPAND(problem, node) do
s  child.STATE
if s is not in reached or child.PATH-COST < reached[s].PATH-COST then
reached[s]  child
add child to frontier
return failure

function EXPAND(problem, node) yields nodes

s  node.STATE
for each action in problem.ACTIONS(s) do
s’  problem.RESULT(s, action)
cost  s.PATH-COST + problem.ACTION-COST(s, action, s’)
yield NODE(STATE=s’, PARENT=node, ACTION=action, PATH-COST=cost)
39
Search data structures
• We need some data structure to keep track of the search tree.
• A node node in the tree is represented by four components.
• node.STATE: The state to which the node node corresponds.
• node.PARENT: The node in the tree that generated node.
• node.ACTION: The action that was applied to the parent’s state to
generate node.
• node.PATH-COST: The cost of the path from the initial state to node.

• The solution is formed by following the PARENT pointers

back from the goal node to the initial node.

40
Search data structures
• The frontier can be implemented with a priority queue (UCS
and A*), FIFO queue (BFS), or LIFO queue (DFS).
• The reached states can be stored in a lookup table (e.g., a
hash table).
• Each element has its state as key and node for that state as value.

41
Redundant paths
• A redundant path can be either a loopy or suboptimal path,
which are both unavoidable.
• Following these paths may turn a tractable problem into an
intractable one.

Each state has four successors.

A search tree of depth 𝑑: 4𝑑 leaves with
repeated states, about 2𝑑 2 distinct states
within 𝑑 steps of any given state
For 𝑑 = 20: a trillion nodes but only about
800 distinct states.

42
Approaches to redundant paths
• Tree-search algorithms: No check of redundant paths.
• There are some problem formulations where it is rare or impossible
for two paths to reach the same state.
• Graph-search algorithms: Remember all previously reached
states and keep only the best path to each state.
• Appropriate for situations where there are many redundant paths,
and the table of reached states will fit in memory.

“Tree-search” refers to the fact that the state space is treated as if it were a tree, with
only one path from each node back to the root. The state space is still the same graph no
matter how we search it.
43
Approaches to redundant paths
• We can compromise and check for cycles, but not for
redundant paths in general.
• Idea: Follow up the chain of parents to see if the state at the
end of the path has appeared earlier in the path.
• This can be done with no need for additional memory.
• Some implementations follow this chain all the way up to
eliminate all cycles.
• Others trace only a few links (e.g., → parent → grandparent
→ great-grandparent), and thus just detects short cycles.

44
Measuring problem-solving performance
• Completeness: Does it always find a solution if one exists?
• Optimality: Does it always find a least-cost solution?
• Time complexity: How long does it take to find a solution?
• Space complexity: How much memory is needed to perform the
search?

• Time and space complexities are abstractly measured by the

number of states and actions considered.
• 𝑏: number of successors of a node that need to be considered
• 𝑑: number of actions in an optimal solution
• 𝑚: the maximum number of actions in any path (may be ∞)

45
Quiz 02: The Towers of Hanoi
• Draw the first two levels of the search tree for the Towers of
Hanoi problem with three pegs and two disks (identical
repeated states on a path can be ignored).

46
47