Unit.

9 Integer Programming

Xiaoxi Li

EMS & IAS, Wuhan University

Dec. 22 - 29, 2016 (revised)

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 1 / 58
Organization of this lecture

Contents:
Integer Programming: Modeling Approach
Basic Models and Classifications: IP, MIP, BIP
A Prototype Example and Some BIP Applications
Some Innovative Uses of Binary Variables in Model Formulation
Some Formulation Examples
Solving an IP (1): the Branch-and-Bound Algorithm
The Branch-and-Bound Algorithm for BIP
The Branch-and-Bound Algorithm for MIP
Solving an IP (2): the Cutting Plane Approach

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 2 / 58
Integer Programming: Modeling Approach

The notion of Integer Programming (IP) has the following two senses.
The broad sense. An IP is a Mathematical Programming (LP or
Non LP) with the restrictions that some decision variables take
integer values.
The narrow sense: An IP is a LP with the restrictions that some
decision variables take integer values.
In this course, we restrict ourselves to the narrow sense. Thus a LP
can be regarded as a relaxation of some IP.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 3 / 58
Integer Programming: Modeling Approach (cont.)

Classifications of IP:
MIP: some variables are asked to take integer values
Pure IP: all variables are asked to take integer values
BIP: all variables are asked to take 0-1 values

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 4 / 58
A Prototype Example with BIP Modeling: The
California Manufacturing Company

The California Manufacturing Company is a company with

factories and warehouses throughout California.
It is currently considering whether to build a new factory in Los
Angeles and/or San Francisco.
Management is also considering building one new warehouse
where a new factory has been recently built.
Should the CMC build factories and/or warehouses in Los Angeles
and/or San Francisco?

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 5 / 58
A Prototype Example with BIP Modeling: The
California Manufacturing Company (cont.)

There are (at most) four possible decisions to make:

To build a factory in LA? (x1 )
To build a factory in SF? (x2 )
To build a warehouse in LA (if a factory has been built there)? (x3 )
To build a warehouse in SF (if a factory has been built there)? (x4 )
So the decision variables take the binary form, to be Yes or No:

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 6 / 58
A Prototype Example with BIP Modeling: The
California Manufacturing Company

The following restrictions exist amongst xj :

x3 and x4 are mutually exclusive alternatives: x3 + x4 ≤ 1;
(since at most ONE warehouse is to be constructed)


x3 is a contingent variable on x1 : x3 ≤ x1 same for (resp. x4 ≤ x2 ) .
(a warehouse is constructed only where a factory is constructed)

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 7 / 58
A Prototype Example with BIP Modeling: The
California Manufacturing Company (cont.)

The BIP for the problem is as follows:

Max Z = 9x1 + 5x2 + 6x3 + 4x4


 6x1 + 3x2 + 5x3 + 2x4 ≤ 10

x3 + x4 ≤ 1



s.t. x3 ≤ x1

x4 ≤ x2





xj ∈ {0, 1}, j = 1, 2, 3, 4.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 8 / 58
Some Other Applications

Investment Analysis
Site Selection
Designing a Production and Distribution Network
Dispatching Shipments
Scheduling Interrelated Activities
Airline Application
...

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 9 / 58
Some Innovative Uses of Binary Variables in Model
Formulation

To name a few:
Either-Or Constraint
K out of N Constraints Must Hold
Functions with N Possible Values
The Fixed-Charge Problem

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 10 / 58
Either-Or Constraint

Suppose you have a situation where you have two potential

constraints, but only one of them can hold. For example:
either "5x1 + 9x2 ≤ 24" (Material 1 be used for production) – 1
or "8x1 + 6x2 ≤ 35" (Material 2 be used for production) – 2
To handle this, we introduce a binary variable y1 , a big M > 0, and
add to one constraint the term My1 to the RHS;
add to the other constraint the term M(1 − y1 ) to the RHS.
To obtain: "5x1 + 9x2 ≤ 24 + My1 and 8x1 + 6x2 ≤ 35 + M(1 − y1 )".
Why they are equivalent?
When y1 = 0, Constraint 1 is active and Constraint 2 is
automatically satisfied for any feasible solution (always bounded);
When y1 = 1, Constraint 2 is active and Constraint 1 is
automatically satisfied for any feasible solution (always bounded).

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 11 / 58
K out of N Constraints Must Hold

There are N constraints among which only K (must) hold (K < N).
Introduce binary variables yi , i = 1, ..., N; add to Constraint i the
term Myi , and another constraint ∑Ni=1 yi = N − K.
For an example (M = 3 and K = 1):
5x1 + 12x2 ≤ 34 + My1
6x1 + 11x2 ≤ 43 + My2
7x1 + 10x2 ≤ 57 + My3
y1 + y2 + y3 = 1
Why they are equivalent?
The same reasoning as the Either-Or Constraint (M = 2, K = 1);
"yi = 0" corresponds to the case that "Constraint i " is active, so
∑Ni=1 yi = N − K is the number of constraints that do not hold.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 12 / 58
Functions with N Possible Values

An example: the constraint is "5x1 + 31x2 = 25 or 50 or 75".

We introduce binary variables y1 , y2 and y3 to obtain the equivalent:

5x1 + 31x2 = 25y1 + 50y2 + 75y3 and y1 + y2 + y3 = 1.

Indeed, "the RHS taking the i-th value" corresponds to the case

”yi = 1 and yj = 0, ∀j 6= i.”

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 13 / 58
The Fixed-Charge Problem

The cost function for producing activity j is

(
kj + cj xj if xj > 0
fj (xj ) =
0 if xj = 0

where xj is the level of activity j, kj is the fixed (setup) cost and cj is

the marginal cost.
The objective function is not linear (mainly at the point 0).
To handle the issue, we introduce for each j
the binary variable yj for the decision "if activity j is produced";
the continuous variable xj for the level of activity j.
Thus yj is seen as a contingent variable of xj , i.e. "yj = 1 iff. xj > 0".

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 14 / 58
The Fixed-Charge Problem (cont.)

Here, it is somehow more complicated (than the previous case)

because xj can take very large value (a priori no given bounded).
Let M > 0 be sufficiently large that exceeds the maximum feasible
value of any xj , j = 1, ..., n.
Introduce the constraints "xj ≤ Myj , j = 1, ..., n".
The objective function is Z = ∑nj=1 (cj xj + kj yj ).
Why they are equivalent to the constraints "yj = 1 iff. xj > 0, j = 1, ..., n"?
First of all, "xj ≤ Myj " is
It is straightforward that "xj ≤ Myj " implies "yj = 1 if xj > 0";
On the other hand, if xj = 0, yj could be 0 or 1 as a feasible
solution. However, by the objective function, at optimum, we
should have yj = 0 whenever xj = 0.
This leads us to a MIP.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 15 / 58
Example 1: Making Choices When the Decision
Variables Are Continuous

Objective: Maximize Profits

Decision Variables: Product 1, Product 2, and Product 3
Constraints:
Production time available for Plants 1 and 2
At most two out of the three products can be produced
Only one of the two plants can produce the new products

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 16 / 58
Example 1: Making Choices When the Decision
Variables Are Continuous (cont.)

The data for the problem:

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 17 / 58
Example 1: Making Choices When the Decision
Variables Are Continuous (cont.)

We construct the following MIP for the problem:

Max Z = 5x1 + 7x2 + 3x3




x1 − My1 ≤ 0, x2 − My2 ≤ 0, x3 − My3 ≤ 0

y1 + y2 + y3 ≤ 2




x ≤ 7, x ≤ 5, x ≤ 9
1 2 3
s.t.


3x1 + 4x2 + 2x3 ≤ 30 + Mz1




4x1 + 6x2 + 2x3 ≤ 30 + M(1 − z1 )

x , x , x ≥ 0, y , y , y , z ∈ {0, 1}.
1 2 3 1 2 3 1

Interpretations. In class.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 18 / 58
Example 2: Proportionality Assumption Violated

Objective: Maximize Profits

Decision Variables: Number of TV spots for Product 1, Product 2,
and Product 3
Constraints: Number of TV spots allocated to the three products
cannot be more than five
Major Issue: Proportionality Assumption is violated

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 19 / 58
Example 2: Proportionality Assumption Violated
(cont.)

The data for the problem:

Note: Proportionality Assumption Violated!

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 20 / 58
Example 2: Proportionality Assumption Violated
(cont.)

We construct the following MIP for the problem (solution 1):

Max Z = y11 + 3y12 + 3y13 + 2y22 + 3y23 − y31 + 2y32 + 4y33



 y11 + y12 + y13 ≤ 1

y21 + y22 + y23 ≤ 1



s.t. y31 + y32 + y33 ≤ 1

y11 + 2y12 + 3y13 + y21 + 2y22 + 3y23 + y31 + 2y32 + 3y33 = 5





yij ∈ {0, 1}, i, j = 1, 2, 3.


For each i, the auxiliary binary variable yij is introduced for each
possible value "j" of the variable xi i.e. yij = 1 if xi = j and yij = 0 if xi 6= j.

Interpretations. In class.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 21 / 58
Example 2: Proportionality Assumption Violated
(cont.)
We construct the following MIP for the problem (solution 2):

Max Z = y11 + 2y12 + 2y22 + y23 − y31 + 3y32 + 2y33


y ≤ y11 , y13 ≤ y12
 12


y22 ≤ y21 , y23 ≤ y22



s.t. y32 ≤ y31 , y33 ≤ y32

y11 + y12 + y13 + y21 + y21 + y22 + y23 + y31 + y32 + y33 = 5





yij ∈ {0, 1}, i, j = 1, 2, 3.


For each i, the auxiliary binary variable yij is introduced for possible
value no smaller than "j" of the variable xi , i.e. yij = 1 if xi ≥ j and yij = 0
if xi < j. =⇒ yij are the increments and xi = yi1 + yi2 + yi3 .

Interpretations. In class.
Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 22 / 58
Example 3: Covering All Characteristics

Exercise.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 23 / 58
Solving an IP: Some General Comments

Some general comments on solving IP.

Solving an IP is NOT easier than solving a LP as was thought.
Having a finite number of feasible solutions ensures that it is readily
solvable. Wrong!
Removing some feasible solutions (the non-integer ones) from a LP
makes an IP easier to solve. Wrong!
The fact that an IP is in general more difficult to solve than a LP
(relaxation) can be understood as follows.
The simplex method (solving LP) visits only the corner-point
feasible solutions, which generally have a much smaller number
than the integer feasible solutions.
Most successful IP algorithms incorporate a LP algorithm by
relating an IP to its LP relaxation.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 24 / 58
Solving an IP: Some General Comments (cont.)

A LP relaxation of an IP is to keep all the original program except for

removing the integer-value restrictions on variables.
Consider a maximization problem. The value of a LP relaxation is
always NO SMALLER than that of an IP. Thus solving a
corresponding LP relaxation gives a upper bound for an IP.
It is fortuitous if a LP relaxation’s optimal solution happens to be
integer values, in which case the solution is also optimal to the IP.
However, this is in general not guaranteed.
Nevertheless, in some specific applications, it is fortuitous to have
this solution structure: minimum cost flow problem and its special
cases (transportation problem, assignment problem, shortest-path
problem, maximum flow problem).

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 25 / 58
Solving an IP: Some General Comments (cont.)

One temptation for solving an IP is to first solve its LP relaxation, and

then round the optimal solution to integer values.
This is an approximation method which may not always lead to an
optimal solution. The approximation might be good if the variables’
values are quite large.
But in general, it is not accepted due to the following two pitfalls:
The rounded "optimal" solution may not be feasible.
Even if feasibility is not a problem, the rounded "optimal" solution
may be far from optimal.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 26 / 58
Solving an IP: Some General Comments (cont.)

Pitfall 1. The rounded "optimal" solution may not be feasible.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 27 / 58
Solving an IP: Some General Comments (cont.)

Pitfall 2. The rounded "optimal" solution may be far from optimal.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 28 / 58
Solving an IP: the Branch-and-Bound Algorithm
The Branch-and-Bound Algorithm: a motivating example of Poker.
Objective. Within a random bundle of Poker, find out one card with
the highest value and has a red-heart format.
Tool. Among any bundle of cards, a machine that is able to pick
out the card(*) with the highest value (no ability to read the
format).
An algorithm.
Put the whole bundle into the machine and obtain the card(*) with
the highest value. STOP if the card’s format is red-heart (the whole
bundle is discarded); otherwise branch the bundle into two.
In a general step, for a newly branched bundle, detect it by the
machine for a highest-valued card(*). Several possibilities:
Discard this bundle if this card(*) is red-heart, and call it incumbent if
its value is higher than the current incumbent one;
Discard this bundle if this card(*)’s value is lower than the current
incumbent one;
Otherwise, brunch the bundle for further detection by machine.
STOP when all cards are discarded. Incumbent = optimal value.
Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 29 / 58
Solving an IP: the Branch-and-Bound Algorithm (cont.)
Objective. For a LP problem, find out one feasible solution with the
highest value and take binary values.
Tool. The simplex method to solve for a LP (relaxation).
An algorithm.
Apply the simple method to the LP relaxation to solve for an optimal
solution. STOP if the optimal solution take binary values; otherwise,
brunch the problem into two subproblems by letting x1 = 0 or x1 = 1.
In a general step, for a newly branched subproblem, solve its LP
relaxation by the simplex method. Several possibilities:
Discard (fathom) this subproblem if this optimal solution take integer
values, and call this solution incumbent if its value is higher than the
current incumbent one;
Discard this subproblem if this optimal value is no larger than the
current incumbent one or if it is infeasible;
Otherwise, brunch the subproblem by xj = 0 or xj = 1 (j in order).
STOP when all subproblems are either branched or fathomed.
Incumbent = optimal solution.
Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 30 / 58
Solving an IP: the Branch-and-Bound Algorithm (cont.)

The Branch-and-Bound Algorithm to solving an BIP: an example.

The (whole) problem is

Max Z = 9x1 + 5x2 + 6x3 + 4x4



 6x1 + 3x2 + 5x3 + 2x4 ≤ 10

 x3 + x4 ≤ 1



s.t. −x1 + x3 ≤ 0

−x2 + x4 ≤ 0





xj ∈ {0, 1}, j = 1, 2, 3, 4.


Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 31 / 58
Solving an IP: the Branch-and-Bound Algorithm (cont.)

Branching. Branch the BIP into two by letting x1 = 0 or x1 = 0.

Subproblem 1 (x1 = 0).

Max Z = 5x2 + 6x3 + 4x4



3x2 + 5x3 + 2x4 ≤ 10

 x3 + x4 ≤ 1



s.t. x3 ≤0

−x2 + x4 ≤ 0





xj ∈ {0, 1}, j = 2, 3, 4.


Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 32 / 58
Solving an IP: the Branch-and-Bound Algorithm (cont.)

Branching. Branch the BIP into two by letting x1 = 0 or x1 = 0.

Subproblem 2 (x1 = 1).

Max Z = 9 + 5x2 + 6x3 + 4x4



3x2 + 5x3 + 2x4 ≤ 4

 x3 + x4 ≤ 1



s.t. x3 ≤1

−x2 + x4 ≤ 0





xj ∈ {0, 1}, j = 2, 3, 4.


Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 33 / 58
Solving an IP: the Branch-and-Bound Algorithm (cont.)

Branching. Branch the BIP into two by letting x1 = 0 or x1 = 0. The

branching tree created by the branching for the first iteration.

x1 is called the branching variable.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 34 / 58
Solving an IP: the Branch-and-Bound Algorithm (cont.)
Bounding. This procedure involves solving the LP relaxation of a
subproblem so as to obtain an upper bound on the BIP’s optimal value.
Consider for example the whole problem. Its LP relaxation is

Max Z = 9x1 + 5x2 + 6x3 + 4x4



 6x1 + 3x2 + 5x3 + 2x4 ≤ 10

 x3 + x4 ≤ 1



s.t. −x1 + x3 ≤ 0

−x2 + x4 ≤ 0





0 ≤ xj ≤ 1, j = 1, 2, 3, 4.


Solving it by simplex method: (x1 , x2 , x3 , x4 ) = ( 56 , 1, 0, 1) with Z = 16 12 .

Therefore any feasible solution for BIP has a value ≤ 16 21 .
Coefficients in the objective function are integers, so 16 12 can be
rounded to 16.
Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 35 / 58
Solving an IP: the Branch-and-Bound Algorithm (cont.)

Bounding. Now we use the similar approach to bound the

Subproblem 1 (x1 = 0) and the Subproblem 2 (x1 = 1).

Subproblem 1 relaxed (x1 = 0).

Max Z = 5x2 + 6x3 + 4x4



3x2 + 5x3 + 2x4 ≤ 10

 x3 + x4 ≤ 1



s.t. x3 ≤0

−x2 + x4 ≤ 0





0 ≤ xj ≤ 1, j = 2, 3, 4.


Solving it by simplex method: (x1 , x2 , x3 , x4 ) = (0, 1, 0, 1) with Z = 9.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 36 / 58
Solving an IP: the Branch-and-Bound Algorithm (cont.)

Bounding.

Subproblem 1 relaxed (x1 = 1).

Max Z = 9 + 5x2 + 6x3 + 4x4


3x + 5x3 + 2x4 ≤ 4
 2


 x3 + x4 ≤ 1



s.t. x3 ≤1

−x2 + x4 ≤ 0





0 ≤ xj ≤ 1, j = 2, 3, 4.


Solving it by simplex method: (x1 , x2 , x3 , x4 ) = (1, 54 , 1, 45 ) with Z = 16 15 .

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 37 / 58
Solving an IP: the Branch-and-Bound Algorithm (cont.)

Fathoming.
The Whole Problem: not fathomed because its LP relaxation has
an optimal (feasible) solution which is not binary valued, and there
is no previous incumbent value (think to be Z ∗ − ∞).
The Subproblem 1 (x1 = 0): fathomed because its LP relaxation
has an optimal solution that is binary valued. Moreover, the
solution (0, 1, 0, 1) and its value Z ∗ = 9 are stored as the incumbent
since there is no previous ones.
The Subproblem 2 (x1 = 1): not fathomed because its LP
relaxation has an optimal (feasible) solution which is not binary
valued, and its value Z = 16 is higher than the incumbent value
Z ∗ = 9.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 38 / 58
Solving an IP: the Branch-and-Bound Algorithm (cont.)

Fathoming.

General rules for fathoming test. A (sub)problem is fathomed if it

satisfies one of the following three.
1 Its bound ≤ Z ∗ ;
2 Its LP relaxation has no feasible solution.
3 Its LP relaxation has integer (binary) solution (which is stored as
the incumbent if its value is higher than the current incumbent
one).

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 39 / 58
Solving an IP: the Branch-and-Bound Algorithm (cont.)

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 40 / 58
Solving an IP: the Branch-and-Bound Algorithm (cont.)
Completing the example (after Iteration 1)

Iteration 2. Subproblem 3 (x1 = 1, x2 = 0).

Max Z = 9 + 6x3 + 4x4



5x3 + 2x4 ≤ 10

 x3 + x4 ≤ 1



s.t. x3 ≤1

x4 ≤ 0





xj ∈ {0, 1}, j = 3, 4.


Its LP relaxation is to replace xj ∈ {0, 1} by 0 ≤ xj ≤ 1 for j = 3, 4.

Solving it by simplex method: (x1 , x2 , x3 , x4 ) = (1, 0, 45 , 0), which is
not integer valued, and with Z = 13 45 .
Bound for Subproblem 3: Z ≤ 13, larger than the incumbent Z ∗ = 9
Thus Subproblem 3 is not fathomed.
Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 41 / 58
Solving an IP: the Branch-and-Bound Algorithm (cont.)
Completing the example (after Iteration 1)

Iteration 2. Subproblem 4 (x1 = 1, x2 = 1).

Max Z = 14 + 6x3 + 4x4



5x3 + 2x4 ≤ 1

 x3 + x4 ≤ 1



s.t. x3 ≤1

x4 ≤ 1





xj ∈ {0, 1}, j = 3, 4.


Its LP relaxation is to replace xj ∈ {0, 1} by 0 ≤ xj ≤ 1 for j = 3, 4.

Solving it by simplex method: (x1 , x2 , x3 , x4 ) = (1, 1, 0, 21 , which is not
integer valued, and with Z = 16.
Bound for Subproblem 4: Z ≤ 16, larger than the incumbent Z ∗ = 9
Thus Subproblem 4 is not fathomed.
Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 42 / 58
Solving an IP: the Branch-and-Bound Algorithm (cont.)

Completing the example (after Iteration 2)

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 43 / 58
Solving an IP: the Branch-and-Bound Algorithm (cont.)

Completing the example (...after Iteration 3)

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 44 / 58
Solving an IP: the Branch-and-Bound Algorithm (cont.)

Completing the example (...after Iteration 4)

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 45 / 58
Solving an IP: the Branch-and-Bound Algorithm (cont.)

Completing the example: the Iteration Procedure

Operation 1. Bounding Problem [all], to obtain the solution
Xall = ( 65 , 1, 0, 1) and the bound Zall = 16.
Operation 2. Branching Problem [all] since X0 is not
integer-valued, into Subproblem [0] (x0 = 0) and [1] (x1 = 1).
Operation 2.1. Bounding Subproblem [0], to obtain the solution
X0 = (0, 1, 0, 1) and the bound Z0 = 9. [0] fathomed and set the
incumbent Z ∗ = Z0 = 9 since X0 is the first integer-valued solution so
far.
Operation 2.2. Bounding Subproblem [1], to obtain the solution
X1 = (1, 45 , 0, 54 ) and the bound Z1 = 16. [1] not fathomed because X1
is not integer-valued .

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 46 / 58
Solving an IP: the Branch-and-Bound Algorithm (cont.)

Completing the example: the Iteration Procedure (cont.)

Operation 3. Branching Problem [1] since Z1 = 16 > 9 = Z ∗ , into
Subproblem [10] (x2 = 0) and [11] (x2 = 1).
Operation 3.1. Bounding Subproblem [10], to obtain the solution
X10 = (1, 0, 54 , 0) and the bound Z10 = 13 > 9 = Z ∗ . [10] not fathomed.
Operation 3.2. Bounding Subproblem [11], to obtain the solution
X11 = (1, 1, 0, 21 ) and the bound Z11 = 16 > 9 = Z ∗ . [11] not fathomed.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 47 / 58
Solving an IP: the Branch-and-Bound Algorithm (cont.)

Operation 4. Branching Subproblem [11] before Subproblem [10]

because Z11 = 16 > 13 = Z10 , into Subproblem [110] (x3 = 0) and
[111] (x3 = 1).
Operation 4.1. Bounding Subproblem [110], to obtain the solution
X110 = (1, 1, 0, 12 ) and the bound Z110 = 16 > 9 = Z ∗ .
[110] not fathomed.
Operation 4.2. Bounding Subproblem [111], to obtain no feasible
solution so [111] fathomed.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 48 / 58
Solving an IP: the Branch-and-Bound Algorithm (cont.)
Completing the example: the Iteration Procedure (cont.)
Operation 5. Branching Subproblem [110] rather than Subproblem
[10] because [110] is newly created, into Subproblem [1100]
(x4 = 0) and Subproblem [1101] (x4 = 1).
Operation 5.1. Bounding Subproblem [1100], to obtain the solution
X1100 = (1, 1, 0, 0) and the bound Z1100 = 14.
Subproblem [1100] fathomed and revise the incumbent
Z ∗ = Z1100 = 14 as X1100 is integer-valued and 14 > 9.
Operation 5.2. Bounding Subproblem [1101] to obtain no feasible
solution thus Subproblem [1101] fathomed.
Operation 6. Fathoming Subproblem [10] due to the fact that the
revised incumbent value Z ∗ = 14 now is larger than Z10 = 13.
STOP because all subproblems are either fathomed or
branched. The optimal solution is X ] = X1100 = (1, 1, 0, 0) and the
optimal value is Z ] = Z ∗ = 14.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 49 / 58
Solving a MIP: the Branch-and-Bound Algorithm

A mixed integer programming problem (MIP) in general form:

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 50 / 58
Solving a MIP: the Branch-and-Bound Algorithm
(cont.)
A variation of the Branch-and Bound algorithm solves MIP. The only
differences are:
1 Selecting the branching variable. In the optimal solution for the LP
relaxation, consider the integer-restricted variables having
non-integer values, and pick the first one in natural ordering;
2 Branching into two subproblems. Let xj∗ be the optimal
(non-integer) value of the branching variable in the optimal
solution of the LP relaxation, then the two subproblems
correspond to "xj ≤ [xj∗ ]" and "xj ≥ [xj∗ ] + 1".
3 Bounding without rounding. This is due to the fact that some
variables are not integer-restricted, so the optimal value is in
general not required to be integers.
4 Fathoming conditions. The subproblem is fathomed if the optimal
solution for its LP relaxation have integer values for those
integer-restricted variables.
Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 51 / 58
Solving a MIP: the Branch-and-Bound Algorithm
(cont.)

Solving a MIP with the Branch-and-Bound Algorithm: an example.

See textbook Hillier and Lieberman (10e) Section 12.7.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 52 / 58
Solving an IP: the Cutting Plane Approach
An example of adding "cuts" to obtain an optimal solution with integer
values for an IP.

Below we follow here Operations Research: Applications and

Algorithms by W.L. Winston, Section 9.8, to present the Cutting Plane
Approach for IP.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 53 / 58
Solving an IP: the Cutting Plane Approach (cont.)
Optimality test. At optimum, x1 = 3.75 and x2 = 2.25, both are
non-integer values. We pick an constraint with the RHS
non-integer. For example,

x1 − 1.25s1 + 0.24s2 = 3.75 − − − − (∗)

Looking for a cut.
Now we write each variable’s coefficient and the constraint’s RHS
of Equation (**) in the form x = [x] + f with 0 ≤ f < 1, to obtain:

x1 − 2s1 + 0.5s1 + 0s2 + 0.25s2 = 3 + 0.75.

Group integer parts into the LHS and non-integer parts the RHS:

x1 − 2s1 − 3 = 0.75 − 0.5s1 − 0.25s2 − − − − (∗∗)

The "cut" of this iteration is obtained as "RHS of (**) ≤ 0 ", i.e.

0.75 − 0.5s1 − 0.25s2 ≤ 0 − − − − (∗ ∗ ∗)

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 54 / 58
Solving an IP: the Cutting Plane Approach (cont.)
The above-obtained cut (constraint) has the following two properties:
1 Any feasible point for the IP will satisfy the cut.
2 Current optimal solution to LP relaxation doesn’t satisfy the cut.
Why?

Remark.
A cut "cut off" the current optimal solution to the LP relaxation, but
not any feasible solution to the IP.
When the cut to the LP relaxation is added, it is hoped that the
optimal solution to the LP relaxation with the cut constraint added
will be integer-valued. If it is the case, an optimal solution to the IP
is obtained; if not, we look for a new cut.
Gomory (1958) found that this process will yield an optimal
solution to an IP after a finite number of iterations (cuts).

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 55 / 58
Solving an IP: the Cutting Plane Approach (cont.)
Substituting "s1 = 6 − x1 − x2 " and "s2 = 45 − 9x1 − 5x2 " into the cut
"0.75 − 0.5s1 − 0.25s2 ≤ 0" to obtain the cut constraint ""3x1 + 2x2 ≤ 15"
thus the cutting plane "3x1 + 2x2 = 15".

Illustration of the cut to the LP relaxation.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 56 / 58
Solving an IP: the Cutting Plane Approach (cont.)
We put the obtained cut constraint "3x1 + 2x2 ≤ 15" into the LP
relaxation. Introducing the slack variable s3 into this constraint, we
obtain the following tableau:

Next we use the dual simplex algorithm to solve the above LP

(optimal but not feasible): select s3 as the leaving variable since
−0.75 < 0 and then select s1 as the entering variable since −0.75 < 0
and |1.25 ÷ (−0.75)| < |0.75 ÷ (−0.25)|.
Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 57 / 58
The dual simplex algorithm,
applied to LP satisfying the optimality
positive coefficient in Eq.(0) but not the feasibility condition (positive
RHS), is essentially a set of operations on constraints so as reaching
feasibility condition while keeping optimality condition (see Section 8.1
in Hillier and Lieberman (10e) for more details).

Using "−0.75" (row-s3 , column-s1 ) as the pivot number, we perform the

Gaussian elimination operations on rows to obtain the same as
the
simplex method, to have the column-s1 finally read as (0, 0, 0, 1)T :

At optimum, all variables are integers so it is feasible thus optimal to IP.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 58 / 58