DS&A 1

 The 0-1 Knapsack problem (optimization

problem) solved so far.
 Via Greedy?
▪ No
 Via DP!
▪ O(nW) or O(2n)
 Any better?
▪ No one has ever found an algorithm whose worst-case time
complexity is better than exponential!
▪ Yet no one has proven that such an algorithm is not possible!
DS&A 2
 The Traveling Salesperson Problem
(optimization problem) solved so far.
 Via Greedy?
▪ No
 Via DP!
▪ O(n22n)
 Any better?
▪ No one has ever found an algorithm whose worst-case time
complexity is better than exponential!
▪ Yet no one has proven that such an algorithm is not possible!

DS&A 3
 Then, what?
 Still want to improve!
 How?

 Backtracking
 Branch-and-Bound (for optimization
problems)

DS&A 4
DS&A 5
 An approach/paradigm to designing
algorithms!

DS&A 6
 The branch-and-bound strategy is very
similar to backtracking in that a state space
tree is used to solve a problem.
 Does not limit us to any particular way of
traversing the state space tree.
 Is used only for optimization problems.

 Idea?

DS&A 7
 Does not limit us to any particular way of
traversing the state space tree:
 Depth-first search
▪ Using Stack
 Breadth-first search
▪ Using Queue
 Best-first search
▪ Using Priority Queue

DS&A 8
 A branch-and-bound algorithm computes a
number (bound) at a node to determine
whether the node is promising.
 The number is a bound on the value of the
solution that could be obtained by expanding
beyond the node.
 If that bound is no better than the value of the
best solution found so far, the node is
nonpromising.
 Otherwise it is promising.

DS&A 9
 void beadth_first_B&B (state_space_tree T, number best)
{ BFS
queue_of_node Q;
node u, v;
initialize(Q); //initialize queue to be empty
v = root of T;
visit v;
enqueue(Q, v); best = value (v);
while (!empty(Q))
{
dequeue(Q, v);
for (each child u of v)
{
if ( value(u) is better than best) best=value(u);
if ( bound(u) is better than best) “promising”
enqueue(Q, u);
}
}

}
DS&A 10
 Besides using the bound to determine
whether a node is promising,
 We can compare the bounds of promising
nodes and visit the children of the one with
the best bound.
 Best-first search with branch and bound
pruning!
 Does not limit us to any particular way of
traversing the state space tree.

DS&A 11
 void best_first_B&B (state_space_tree T, number best)
{ Best-First Search
priority_queue_of_node PQ;
node u, v;
initialize(PQ); //initialize priority_queue to be empty
v = root of T;
visit v;
enqueue(PQ, v); best = value (v);
while (!empty(PQ))
{
dequeue(PQ, v);
if ( bound(v) is better than best )
for (each child u of v)
{
if ( value(u) is better than best) best=value(u);
if ( bound(u) is better than best) “promising”
enqueue(PQ, u);
}
}
DS&A 12
 The branch and bound algorithms can be
exponential or worse at the worst-case (like
backtracking).
 The branch and bound algorithms can often
arrive at an optimal solution faster than
backtracking's depth-first search.
 The branch and bound algorithms are
efficient for many large instances.

DS&A 13
 An enhancement of backtracking
▪ Depth-first search
▪ Using Stack
▪ Breadth-first search
▪ Using Queue
▪ Best-first search
▪ Using Priority Queue

 Applicable to optimization problems only

DS&A 14
 Bound is used for?
 Ruling out certain nodes as “nonpromising” to
prune the tree!
▪ If a node’s bound is not better than the best solution
seen so far, the node is non-promising.
 Guiding the search through state-space!
▪ Compare the bounds of promising nodes and visit the
one with the best bound.

DS&A 15
 The 0-1 Knapsack Problem via B&B

DS&A 16
 The branch-and-bound design strategy by
applying it to the 0-1 Knapsack problem.
 First, a simple version called breadth-first
search with branch-and-bound pruning.
 Second, an improvement on the simple version
called best-first search with branch-and-
bound pruning.

DS&A 17
 Bound(v) (Potential profit upper bound) is
an upper bound on the profit we could
achieve by expanding beyond the node v!

 Pretending the Fractional Knapsack!

 We will use bound(v) to determine whether v

is non-promising!
DS&A 18
 Each node consists of

($) Profit

(lbs) Weight

($) Bound

DS&A 19
 A node is nonpromising if
 The total weight thus far is greater than or equal
to W.
 The bound (potential profit upper bond) is smaller
than or equal to the value of maxprofit.

DS&A 20
 First, a simple version called breadth-first
search with branch-and-bound pruning.

DS&A 21
 Example 6.1
 n= 4 & W = 16 lb Sort by profit/weight
 Item1= $40 & 2 lb
 Item2= $30 & 5 lb
 Item3= $50 & 10 lb
 Item4= $10 & 5 lb

 Item 1 & item 3

DS&A 22
 Total 31 nodes
W=16
Figure 6.2
Maxprofit = ?

The optimal solution!

Profit per unit weight!

17 nodes checked!
DS&A 23
 W = 6 lb
 Item1= $10 & 1 lb
 Item2= $18 & 2 lb
 Item3= $32 & 4 lb
 Item4= $14 & 2 lb

DS&A 24
 Second, an improvement called best-first
search with branch-and-bound pruning.

 Compare the bounds of promising nodes and visit

the children of the one with the best (largest)
bound.

 We often arrive at an optimal solution more

quickly than if we simply proceeded blindly in a
predetermined order.
DS&A 25
 Example 6.2
 n= 4 & W = 16 lb Sort by profit/weight
 Item1= $40 & 2 lb
 Item2= $30 & 5 lb
 Item3= $50 & 10 lb
 Item4= $10 & 5 lb

 Item 1 & item 3

DS&A 26
 Total 31 nodes
W=16
Figure 6.3
Maxprofit = ?
Best-First Search

The optimal solution!

Profit per unit weight! 11 nodes checked!

DS&A 27
 W = 6 lb
 Item1= $10 & 1 lb
 Item2= $18 & 2 lb
 Item3= $32 & 4 lb
 Item4= $14 & 2 lb

DS&A 28
 Recall:
 Section 5.7
 Using Backtracking for the 0-1 Knapsack
Problem!

DS&A 29
 Example 5.6
 n= 4 & W = 16 lb Sort by profit/weight
 Item1= $40 & 2 lb
 Item2= $30 & 5 lb
 Item3= $50 & 10 lb
 Item4= $10 & 5 lb

 Item 1 & item 3

DS&A 30
 Total 31 nodes
W=16
Figure 5.14 DFS
Maxprofit = ?

The optimal solution!

Profit per unit weight!
13 nodes checked!
DS&A x= nonpromising node ! 31
 Actually,
 Using Backtracking for the 0-1 Knapsack
Problem!
 This is a depth-first search with branch-and-
bound pruning.

DS&A 32
 Compare Brute-Force Approach vs. Branch-
and-Bound?

DS&A 33
 Compare Backtracking vs. Branch-and-
Bound?

DS&A 34
 Compare Brute-Force Approach vs.
Backtracking vs. Branch-and-Bound?

DS&A 35
 The 0-1 Knapsack Problem via B&B

DS&A 36
DS&A 37
 Chapter 6: Exercise #1 and #4 (Draw the
pruned state space tree.)

DS&A 38
 Chapter 6:
 6.1

DS&A 39