Branch and Bound

Searching Strategies

1
The personnel
assignment problem

A linearly ordered set of persons P={P1, P2,
…, Pn} where P1<P2<…<Pn

A partially ordered set of jobs J={J1, J2, …, Jn}

Suppose that Pi and Pj are assigned to jobs f
(Pi) and f(Pj) respectively. If f(Pi)  f(Pj), then
Pi  Pj. Cost Cij is the cost of assigning Pi to Jj.
We want to find a feasible assignment with t
he min. cost. i.e.

Minimize i,j CijXij,
where Xij = 1 if Pi is assigned to Jj and Xij = 0
otherwise.

2
The personnel
assignment problem
 E.g. J1 J2

↓ ↘ ↓
J3 J4
Fig. 6-21 A Partial Ordering of Jobs
 After topological sorting, one of the following
topologically sorted sequences will be
generated:
J1, J2, J3, J4
J1, J2, J4, J3
J1, J3, J2, J4
 One of feasible assignments: J2, J1, J3, J4

P1→J1, P2→J2, P3→J3, P4→J4 J2, J1, J4 J3

3
The personnel
assignment problem
 Cost matrix:
Jobs 1 2 3 4
Persons

1 29 19 17 12
2 32 30 26 28
3 3 21 7 9
4 18 13 10 15

Table 6-1 A Cost Matrix for a Personnel

Assignment Problem
4
The personnel
assignment problem
 Reduced cost matrix:
subtract a constant from each row and each
column respectively such that each row and
each column contains at least one zero.
Jobs 1 2 3 4
Persons
1 17 4 5 0 (-1
2)
2 6 1 0 2 (-2
6)
3 0 15 4 6 (-3)
4 8 0 0 5 (-1
0)
(-3)
A Reduced Cost Matrix 5
The personnel
assignment problem
 Total cost subtracted: 12+26+3+10+3
= 54
 This is a lower bound of our solution.

6
The personnel
assignment problem
 Solution tree:
0 Person
Assigned

1 2 1

2 3 1 2

3 4 2 3 4 3

4 3 4 4 3 4

7
 Steps to generate solution
tress
 Step 1: Add the original partial ordering into st
ack.
 Step 2: Remove the top partial ordering O from
stack. If O contains only one element E, then st
op (with the element E being an element in the
topologically sorted sequence). Otherwise, add
the partial orderings into the stack each of whi
ch is derived by removing an element E not pr
eceded by any other element in the partial ord
ering O. (Every E is an element in the topologic
ally sorted sequence.)
 Step 3: Goto Step 1.

8
The personnel
assignment problem
 Bounding of subsolutions:

9
The traveling salesperson
optimization problem
 Given a graph, the TSP Optimization
problem is to find a tour, starting
from any vertex, visiting every other
vertex and returning to the starting
vertex, with minimal cost.
 It is NP-complete
 We try to avoid n! exhaustive
search by the branch-and-bound
technique.
10
The traveling salesperson
optimization problem
 E.g. A Cost Matrix for a Traveling Salesperson Problem.

j 1 2 3 4 5 6 7
i
1 ∞ 3 93 13 33 9 57
2 4 ∞ 77 42 21 16 34
3 45 17 ∞ 36 16 28 25
4 39 90 80 ∞ 56 7 91
5 28 46 88 33 ∞ 25 57
6 3 88 18 46 92 ∞ 7
7 44 26 33 27 84 39 ∞
11
The basic idea
 There is a way to split the solution space
(branch)
 There is a way to predict a lower bound
for a class of solutions. There is also a
way to find a upper bound of an optimal
solution. If the lower bound of a solution
exceeds the upper bound, this solution
cannot be optimal and thus we should
terminate the branching associated with
this solution.
12
Splitting
 We split a solution into two groups:
 One group including a particular arc
 The other excluding the arc
 Each splitting incurs a lower bound
and we shall traverse the
searching tree with the “lower”
lower bound.

13
The traveling salesperson
optimization problem

Reduced cost matrix:
j 1 2 3 4 5 6 7
i
1 ∞ 0 90 10 30 6 54 (-3)
2 0 ∞ 73 38 17 12 30 (-4)
3 29 1 ∞ 20 0 12 9 (-16)
4 32 83 73 ∞ 49 0 84 (-7)
5 3 21 63 8 ∞ 0 32 (-25)
6 0 85 15 43 89 ∞ 4 (-3)
7 18 0 7 1 58 13 ∞ (-26)
reduced:84

A Reduced Cost Matrix. 14

The traveling salesperson
optimization problem
j 1 2 3 4 5 6 7
i
1 ∞ 0 83 9 30 6 50
2 0 ∞ 66 37 17 12 26
3 29 1 ∞ 19 0 12 5
4 32 83 66 ∞ 49 0 80
5 3 21 56 7 ∞ 0 28
6 0 85 8 42 89 ∞ 0
7 18 0 0 0 58 13 ∞
(-7) (-1) (-4)
Table 6-5 Another Reduced Cost Matrix.
15
Lower bound
 The total cost of 84+12=96 is
subtracted. Thus, we know the
lower bound of this TSP problem is
96.

16
The traveling salesperson
optimization problem
 Total cost reduced: 84+7+1+4 = 96 (lower
bound)
decision tree:

The Highest Level of a Decision Tree.

 If we use arc 3-5 to split, the difference on the
17
lower bounds is 17+1 = 18.
For the right subtree
j 1 2 3 4 5 7
i
1 ∞ 0 83 9 30 50
2 0 ∞ 66 37 17 26
3 29 1 ∞ 19 0 5
5 3 21 56 7 ∞ 28
6 0 85 8 ∞ 89 0
7 18 0 0 0 58 ∞

A Reduced Cost Matrix if Arc 4-6 is Included.

(We must set c6-4 to be )
18
For the left subtree

The cost matrix for all solution with arc 4-6:
j 1 2 3 4 5 7
i

1 ∞ 0 83 9 30 50
2 0 ∞ 66 37 17 26
3 29 1 ∞ 19 0 5
5 0 18 53 4 ∞ 25 (-
6 0 85 8 ∞ 89 0 3)
7 18 0 0 0 58 ∞
A Reduced Cost Matrix for that in Table 6-6.
 Total cost reduced: 96+3 = 99 (new lower
bound) 19
Upper bound
 We follow the best-first search sch
eme and can obtain a feasible solu
tion with cost C.
 C serves as an upper bound and m
any branchings may be terminated
if their lower bounds exceed C.

20
 1 2
5
6

4 3
7
Fig 6-26 A Branch-and-Bound Solution of a Traveling
Salesperson Problem. 21
Preventing an arc
 In general, if paths i1-i2-…-im and j1-j
2-…-jn have already been included
and a path from im to j1 is to be add
ed, then path from jn to i1 must be
prevented
(by assigning the cost of jn to i1 to
be )

22
The 0/1 knapsack
problem
 Positive integer P1, P2, …, Pn (profit)
W1, W2, …, Wn (weight)
M (capacity)
n
maximize PX
i i
i1
n
subject to WiXiM Xi =0or 1, i =1, …, n.
i
1
Theproblemismodified:
n
minimize  PX
i i
i
1
23
The 0/1 knapsack
problem

Fig. 6-27 The Branching Mechanism in the Branch-and-

Bound Strategy to Solve 0/1 Knapsack Problem.
24
How to find the upper
bound?
 Ans: by quickly finding a feasible s
olution: starting from the smallest
available i, scanning towards the la
rgest i’s until M is exceeded. The u
pper bound can be calculated.

25
The 0/1 knapsack
problem
 E.g. n = 6, M = 34

i 1 2 3 4 5 6

Pi 6 10 4 5 6 4

Wi 10 19 8 10 12 8

(Pi/Wi  Pi+1/Wi+1)
 A feasible solution: X1 = 1, X2 = 1, X3 = 0, X4 = 0,
X5 = 0, X6 = 0
-(P1+P2) = -16 (upper bound)
Any solution higher than -16 can not be an optimal solution.
26
How to find the lower boun
d?
 Ans: by relaxing our restriction from Xi = 0
or 1 to 0  Xi  1 (knapsack problem)
n
Let  PX
i i be an optimal solution for 0/1
i
1
n
knapsack problem and  PX
i 
i be an optimal
i
1
n
solution for knapsack problem. Let Y= PX
i i,
i
1
n
Y’ =  PX
i 
i.
i
1
 Y’ Y 27
 The knapsack problem
 We can use the greedy method to find an optimal
solution for knapsack problem.

For example, for the state of X1=1 and X2=1, we
have
X1 = 1, X2 =1, X3 = (34-6-10)/8=5/8, X4 = 0, X5 = 0,
X6 =0
-(P1+P2+5/8P3) = -18.5 (lower bound)
-18 is our lower bound. (only consider integers)

28
How to expand the tree?
 By the best-first search scheme
 That is, by expanding the node
with the best lower bound. If two
nodes have the same lower
bounds, expand the node with the
lower upper bound.

29
0/1 Knapsack Problem Solved by Branch-and-Bound Strategy
30
 Node 2 is terminated because its
lower bound is equal to the upper
bound of node 14.
 Nodes 16, 18 and others are
terminated because the local lower
bound is equal to the local upper
bound.
(lower bound  optimal solution 
upper bound)
31
The A* algorithm
 Used to solve optimization problems.
 Using the best-first strategy.
 If a feasible solution (goal node) is obtained, then it is
optimal and we can stop.
 Estimated cost function of node n : f(n)
f(n) = g(n) + h(n)
g(n): cost from root to node n.
h(n): estimated cost from node n to a goal node.
h*(n): “real” cost from node n to a goal node.
f*(n): “real” cost of node n
h(n)  h*(n)
 f(n) = g(n) + h(n)  g(n)+h*(n) = f*(n)

32
Reasoning
 Let t be the selected goal node. Let
n denote a node already expanded.
 f(t)f(n)
 f(t) f(n) f*(n)
 But one of f*(n)’s must be the optim
al value (cost). Let f*(s) denote the
value.
 We have f(t) f*(s) …………(a)
33
Reasoning
 Since t is a goal node, we have h(t)
=0. Thus, f(t)=g(t)+h(t)=g(t).
 By Eq. (a), we have f(t)=g(t)f*(s)
 Yet, f(t)=g(t) is the value of a feasi
ble solution. Consequently, g(t) ca
nnot be smaller than f*(s) by defini
tion. This means that g(t)=f*(s).

34
The A* algorithm
 Stop iff the selected node is also a goal no
de
 E.g.

Fig. 6-36 A Graph to Illustrate A * Algorithm. 35

The A* algorithm
 Step 1.

g(A)=2 h(A)=min{2,3}=2 f(A)=2+2=4

g(B)=4 h(B)=min{2}=2 f(B)=4+2=6
g(C)=3 h(C)=min{2,2}=2 f(C)= 3+2=5

36
The A* algorithm
 Step 2. Expand A

g(D)=2+2=4 h(D)=min{3,1}=1 f(D)=4+1=5

g(E)=2+3=5 h(E)=min{2,2}=2 f(E)=5+2=7
37
The A* algorithm
 Step 3. Expand C

g(F)=3+2=5 h(F)=min{3,1}=1 f(F)=5+1=6

g(G) =3+2=5 h(G)=min{5}=5 f(G) =5+5=10
38
 The A* algorithm
 Step 4. Expand D

g(H)=2+2+1=5 h(H)=min{5}=5 f(H)=5+5=10

g(I)=2+2+3=7 h(I)=0 f(I)=7+0=7
39
The A* algorithm
 Step 5. Expand B

g(J)=4+2=6 h(J)=min{5}=5 f(J)=6+5=11

40
 The A* algorithm
 Step 6. Expand F 
f(n) f*(n)
=

I is a goal node !
g(K)=3+2+1=6 h(K)=min{5}=5 f(K)=6+5=11
g(L)=3+2+3=8 h(L)=0 f(L)=8+0=8
41
The A* Algorithm
 Can be considered as a special
type of branch-and-bound
algorithm.

42