Fundamentals of Algorithms

Unit 7

Unit 7
Structure 7.1 Introduction Objectives 7.2 Branch and Bound Strategy 7.3 O/I Knapsack Problem 7.4 Travelling Salesman Problem 7.5 Summary 7.6 Terminal Questions 7.7 Answers

Branch and Bound

7.1 Introduction
In the previous units we discussed about the concept of backtracking and now we will study a new concept known as branch and bound. The term branch-and-bound refers to all state space search methods in which all children of the E-node are generated before any other live node can become the E-node. As in the case of backtracking, bounding functions are used to help avoid the generation of subtrees that do not contain an answer node. Objectives After studying this unit, you should be able to: solve the O/I Knapsack Problem find the solution using Traveling Salesman Problem.

7.2 Branch and Bound Strategy

The term branch-and-bound refers to all state space search methods in which all children of the E-node are generated before any other live node can become the E-node. In branch-and-bound terminology, a BFS-like state space search will be called FIFO (First In First Out) search as the list of live nodes is first-in-firstout list (or queue). A D-search-like state space search will be called LIFO (Last In First Out) search as the list of live nodes is a last-in-first-out list (or stack). As in the case of backtracking, bounding functions are used to help avoid the generation of subtrees that do not contain an answer node.
Sikkim Manipal University Page No. 138

Fundamentals of Algorithms

Unit 7

Self Assessment Question 1. The term branch-and-bound refers to all state space search methods in which all children of the E-node are generated before any other live node can become the.

7.3 O/l Knapsack Problem

To use the branch-and-bound technique to solve any problem, it is first necessary to conceive of a state space tree for the problem. The objective function
pi x i is replaced by the function pi x i . Clearly,

pi x i is maximized iff pi x i is minimized. This modified knapsack

problem is stated as given below

n

Minimize
i 1 n

pi x i

Subject to
i 1

w i xi

xi = 0 or 1, 1

n ------- (7.1)

We continue the discussion assuming a fixed tuple size formulation for the solution space. The discussion is easily extended to the variable tuple size formulation. Every leaf node in the state space tree represents an assignment for which
w i xi
1 i n

m is an answer (or solution) node. All

other leaf nodes are infeasible. For a minimum-cost answer node to correspond to any optional solution, we need to define c x every answer node x. The cost c(x) =
pi x i for
1 i n

for infeasible leaf nodes. For non-

leaf nodes, c(x) is recursively defined to be min{c(Ichild(x)), c(rchild(x))}

Sikkim Manipal University

Page No. 139

Fundamentals of Algorithms

Unit 7

Algorithm 7.1 1. Algorithm U Bound(cp, cw, k, m) 2. // cp is the current profit total, cw is the current 3. // weight total; k is the index of the last removed 4. // item; and m is the knapsack size 5. // w[i] and p[i] are respectively the weight and profit 6. // of the ith object 7. { 8. b:=cp; c:=cw; 9. for I:=k+1 to n do 10. { 11. if (c+w[i] m) then 12. { 13. c:c+w[i]; b:=b-p[i]; 14. } 15. } 16. return b; 17. }
(x) and u(x) such that c (x) We now need two functions c

c(x)

u(x) for

(.) and u(.) satisfying this requirement may be every node x. The cost c obtained as follows. Let x be a node at level j, 1 j n+1. At node x, assignments have already been made to xi, 1 i < j. The cost of these

assignments is
ux pi x i . If q
1 i j

pi x i . So,
1 i j

cx

pi x i
1 i j

and we may use

pi x i then an improved upper bound function u(x) is

1 i j

u(x) = Ubound q ,

w i x i , j 1, m , where Ubond is defined in Algorithm

1 i j

Self Assessment Question 2. To use the branch-and-bound technique to solve any problem, it is first necessary to conceive of a for the problem.

Sikkim Manipal University

Page No. 140

Fundamentals of Algorithms

Unit 7

7.4 Travelling Salesman Problem

An O(n22n) dynamic programming algorithm for the travelling sales person was arrived at in Unit 5. We now investigate branch-and-bound algorithms for this problem. Although the worst-case complexity of these algorithms will not be any better than O(n22n), the use of the good bounding functions will enable these branch and bound algorithms to solve some problem instances in much time less than that required by the dynamic programming algorithm. Let G=(V,E) be a directed graph defining an instance of the travelling sales person problem. Let cij equal the cost of edge <i, j>,cij = if < i, j > E, and let |V|=n. Without loss of generality, we can assume that every tour starts and ends at vertex 1. So, the solution space S is given by S = {1, , 1| is a permutation of (2,3 ,n)}. Then, | s |= (n 1)!. The size of S can be reduced by restricting S so that (1, i1,i2,in1,1) S iff < ij, ij+1 > E, 0 j n 1, and i0 = in = 1. To search the travelling salesperson state space tree, we need to define a (.) and u(.) such that c (r) c(r) cost function c (.) and two other functions c u(r) for all nodes r. The cost c(.) is such that the solution node with least c(.) corresponds to a shortest tour in G. One choice for c(.) is c(A) = length of tour defined by the path from the root to A, if A is a leaf and c(A) = cost of minimum-cost leaf in the subtrees A if A is not a leaf l(i) = length of the path defined at node i

Fig. 7.1: State space tree for the travelling sales person problem with n=4 and i0 = i4 = 1 Sikkim Manipal University Page No. 141

Fundamentals of Algorithms

Unit 7

20 30 10 11 15 16 4 2 3 5 2 4 19 6 18 3 16 4 7 16

10 17 0 12 11 2 0 3 0 15 3 12 11 0 0 12

1 0 2 0

a) Lost matrix

b) Reduced cost matrix L=25

Fig. 7.2: An example of cost matrix and reduced cost matrix

(.) such that c (A) A simple c

(A) to c(A) for all A is obtained by defining c (.) can be obtained be the length of the path defined at node A. A better c by using the reduced cost matrix corresponding to G. A row (column) is said to be reduced if it contains atleast one zero and all remaining entries are non-negative. A matrix is reduced iff every row and column is reduced. As an example of how to reduce the cost matrix of a given graph G, consider the matrix of fig. 7.2(a). This corresponds to a graph with five vertices. Since every tour on this graph includes exactly one edge <i, j> with i=k, 1 k 5, and exactly one edge <i, j> with j=k, 1 k 5, subtracting a constant t from every entry in one column or one row of the cost matrix reduces the length of every tour by exactly t. A minimum-cost tour remains a minimum-cost tour following this subtraction operation. If t is chosen to be the minimum entry in row i (column j), then subtracting it from all entries in row i (column j) introduces a zero into row i (column j) Repeating this as often as needed, the cost matrix can be reduced. The total amount subtracted from the columns and rows is a lower bound on the length of a minimum-cost tour value for the root of the state space tree. and can be used as the c Subtracting 10, 2, 2, 3, 4, 1 and 3 from rows 1, 2, 3, 4 and 5 and Column 1 and 3 respectively of the matrix of fig 7.2(a), yields the reduced matrix of fig. 7.2(b). The total amount subtracted is 25. Hence, all tours in the original graph have a length of at least 25.

We can associate a reduced cost matrix with every node in the travelling salesperson state space tree. Let A be the reduced cost matrix for node R. Let S be a child of R such that the tree edge (R, S) Corresponds to including edge <i,j> in the tour. If S is not a leaf, then the reduced cost matrix for S may be obtained as follows:
Sikkim Manipal University Page No. 142

Fundamentals of Algorithms

Unit 7

1) Change all entries in row i and column j and A to . This prevents the use of any more edge leaving vertex i or entering vertex j. 2) Set A(j,1) to . This prevents the use of edges <j,1>. 3) Reduce all rows and columns in the resulting matrix except for rows and columns containing only . Let the resulting matrix be B. Step (1) and (2) are valid as no tour in the subtree S can contain edges of the type <i,k> or <k,j> or <j,1> (except for edge <i,j>). If r is the total amount subtracted in step (3) then,
(S) = c (R) c + A(i, j)+r. For leaf nodes, (.)= c( c ) is easily computed as c each leaf defines a unique tour. For the upper bound function u, we can use u( R)= for all nodes R.

Self Assessment Question 3. We can associate a with every node in the travelling salesperson state space tree

7.5 Summary
The term branch-and-bound refers to all state space search method in which all children of the E-node are generated before any other live node can become the E-node. The Knapsack problem using branch-and-bound technique states as
n

Minimize
i 1 n

pi x i w i xi
i 1

Subject to

xi = 0 or 1, 1 i n. In travelling salesperson problem, one choice for c(.) is

cA length tour defined by the path from the root to A, is a leaf cost of minimum cost leaf in the subtrees A, if A is not a leaf

Sikkim Manipal University

Page No. 143

Fundamentals of Algorithms

Unit 7

7.6 Terminal Questions

1. Briefly explain the O/I Knapsack Problem 2. Explain the travelling Salesman Problem 3. Write the Algorithm for Knapsack Problem 4. Write the general method for Branch and Bound 5. What do you mean by Bounding?

7.7 Answers
Self Assessment Questions 1. E-node 2. state space tree 3. reduced cost matrix Terminal Questions 1. To use the branch-and-bound technique to solve any problem, it is first necessary to conceive of a state space tree for the problem. The objective function
pi x i is replaced by the function pi x i . Clearly

pi x i is maximized iff pi x i is minimized. This modified knapsack

problem is stated as given below

n

Minimize
i 1 n

pi x i w i xi
i 1

Subject to

xi = 0 or 1, 1

n. (Refer Section 7.3)

2. Let G=(V,E) be a directed graph defining an instance of the traveling sales person problem. Let cij equal the cost of edge <i, j>,cij = if < i, j > E, and let |V|=n. Without loss of generality, we can assume that every tour starts and ends at vertex 1. So, the solution space S is given by S = {1, , 1| is a permutation of (2,3 ,n)}. Then | s |= (n 1)!. The size of S can be reduced by restricting S so that (1, i1,i2,in1,1) S iff < ij, ij+1 > E, 0 j n 1, and i0 = in = 1.

Sikkim Manipal University

Page No. 144

Fundamentals of Algorithms

Unit 7

To search the traveling salesperson state space tree, we need to define (.) and u(.) such that c (r) a cost function c (.) and two other functions c c(r) u(r) for all nodes r. The cost c(.) is such that the solution node with least c(.) corresponds to a shortest tour in G. One choice for c(.) is c(A) = length of tour defined by the path from the root to A, if A is a leaf and c(A) = cost of minimum-cost leaf in the subtrees A and if A is not a leaf l(i) = length of the path defined at node i. (Refer Section 7.4). 3. To use the branch-and-bound technique to solve any problem, it is first necessary to conceive of a state space tree for the problem. The objective function Clearly,
pi x i is replaced by the function pi x i .

pi x i is maximized iff pi x i is minimized. This modified

n

knapsack problem is stated as given below

Minimize
i 1 n

pi x i w i xi
i 1

Subject to

xi = 0 or 1, 1

n ------- (7.1)

We continue the discussion assuming a fixed tuple size formulation for the solution space. The discussion is easily extended to the variable tuple size formulation. Every leaf node in the state space tree represents an assignment for which
w i xi
1 i n

m is an answer (or solution) node.

All other leaf nodes are infeasible. For a minimum-cost answer node to correspond to any optional solution, we need to define c x for every answer node x. The cost c(x) =
pi x i
1 i n

for infeasible leaf nodes. For

non-leaf nodes, c(x) is recursively defined to be min{c(Ichild(x)), c(rchild(x))}

Sikkim Manipal University

Page No. 145

Fundamentals of Algorithms

Unit 7

Algorithm 7.1 18. Algorithm U Bound(cp, cw, k, m) 19. // cp is the current profit total, cw is the current 20. // weight total; k is the index of the last removed 21. // item; and m is the knapsack size 22. // w[i] and p[i] are respectively the weight and profit 23. // of the ith object 24. { 25. b:=cp; c:=cw; 26. for I:=k+1 to n do 27. { 28. if (c+w[i] m) then 29. { 30. c:c+w[i]; b:=b-p[i]; 31. } 32. } 33. return b; 34. }
(x) and u(x) such that c (x) c(x) u(x) Now we need two functions c (.) and u(.) satisfying this requirement may for every node x. The cost c

be obtained as follows. Let x be a node at level j, 1 j n+1. At node x, assignments have already been made to xi, 1 i < j. The cost of these assignments is
ux pi x i . If q
1 i j

pi x i . So, c x
1 i j

pi x i
1 i j

and we may use

pi x i then, an improved upper bound function

1 i j

u(x) is u(x) = Ubound q , Algorithm. 4. The general method is

w i x i , j 1, m , where Ubond is defined in

1 i j

1. In branch and bound method a state space tree is built and all the children of E nodes ( a live node whose children are currently being
Sikkim Manipal University Page No. 146

Fundamentals of Algorithms

Unit 7

generated) are generated before any other node can become a live node. 2. For exploring new nodes either a BFS or D-Search technique can be used. 3. In Branch and bound technique, BFS-like state space search will be called FIFO (First in First Out) search. This is because the list of live node is First In First Out list (queue). On the other hand, the Dsearch like state space search will be called LIFO search because the list of live node is Last in First out list (stack). 4. In this method, a space tree of possible solutions is generated. Then partitioning (called as branching) is done at each node of the tree. We compute lower bound and upper bound at each node. This computation leads to selection of answer node. 5. Bounding functions are used to avoid the generation of subtrees that do not contain an answer node. 5. Bounding is having the following characteristics 1. As we know that the bounding functions are used to avoid the generation of sub trees that do not contain the answer nodes. In bounding lower bounds and upper bounds are generated at each node. 2. A cost function c (x) is such that c (x) c (x) is used to provide the lower bounds on solution obtained from node x. 3. Let upper is an upper bound on cost of minimum-cost solution. In that case, all the live nodes with c (x) upper can be killed. 4. At the start, the upper is usually set to . After generating the children of current E-node, upper can be updated by minimum cost answer node. Each time a new answer node can be obtained.

Sikkim Manipal University

Page No. 147