Summer 2011

Optimization I Lecture 6

Simplex Method Implementation

Last time, we gave the following description of an iteration of the simplex method:
1. We start with a basis AB(1) , . . . , AB(m) and an associated bfs x.
2. We compute c̄j = cj − cTB B−1 Aj for every non-basic variable. If c̄j ≥ 0 for all j, we conclude x is
optimal, and the algorithm terminates.

3. Otherwise, choose j such that c̄j < 0. Variable xj enters the basis.
4. Let u = B−1 Aj . If ui ≤ 0 for all i, then the optimal value is −∞, and the algorithm terminates.
5. Otherwise, set θ∗ = mini:ui >0 xB(i) /ui , and let ` be such that θ∗ = xB(`) /u` . Variable xB(`) leaves the
basis.

6. The new basis is obtained by replacing AB(`) by Aj , and the new bfs by setting xnew
j = θ∗ and
new ∗
xB(i) = xB(i) − θ ui for i = 1, . . . , m.
We saw last time that, if we have an initial bfs, and there is no degeneracy, then the Simplex Method
finds an optimal solution or correctly reports that the optimal value is −∞ in finite time.
We will see how to prevent cycling in the case of degeneracy and how to find an initial basic feasible
solution. Before we deal with these questions, we’ll see a more convenient way of doing a simplex iteration.

1 Implementation of a Simplex Iteration

1.1 Revised Simplex Method
Note that in subsequent iterations, the matrix B only changes in one column. Hence, it seems that it should
be possible to update B−1 without having to start from scratch. This is called the revised simplex method.

• Recall how to compute a matrix inverse of an m × m matrix B using Gauss-Jordan elimination: We

append the columns of the identity matrix to obtain [B|Im ] and perform elementary row operations
(which is the same as premultiplying by some m × m matrix Q), so we get Q[B|Im ] = [QB|Q]. The
row operations are chosen, so that the first m columns give the identity, i.e., QB = Im . So Q = B−1
which we can read from the last m columns.
 
3 1 0
Example: B =  1 −1 0 .
0 1 1
1 1
   
3 1 0 | 1 0 0 1 3 0 | 3 0 0
subtract 1/3R1 to R2, divide R1 by 3
 1 −1 0 | 0 1 0  →  0 − 43 0 | − 13 1 0 
0 1 1 | 0 0 1 0 1 1 | 0 0 1
1 1
 
add 1/4R2 to R1, add 3/4R2 to R3, divide R2 by − 43
1 0 0 | 4 4 0
1
→  0 1 0 | 4 − 34 0 .
0 0 1 | − 14 3
4 1
1 1
 
4 4 0
So B−1 =  1
4 − 34 0 .
− 14 3
4 1

1
 • Now, if we update B in the Simplex Method, we just change one column. Suppose we replace column 2
4
with the column  2 , and suppose we do the exact same operations as we did previously. Then only
1
the second column of the 3 × 6-matrix above changes. We also know what the second column  looks

4
like after the last step of the Gauss-Jordan elimination we did above: it will be equal to B−1  2  =
1
 1 1
   3 
4 4 0 4 2
 1 − 3
0   2  =  − 1 .
4 4 2
− 14 3
4 1 1 3
2

So we can reuse our previous computations and find the new inverse as follows:
3 1 1
   
1 2 0 | 4 4 0 add 3R2 to R1, add 3R2 to R3, divide R2 by − 1 1 0 0 | 1 −2 0
 0 −1
2 0 | 1
4 − 3
4 0  → 2
 0 1 0 | − 12 3
2 0 .
0 3
2 1 | − 14 3
4 1 0 0 1 | − 12 − 32 1
 
1 −2 0
So B−1
new =  − 12 3
2 0 .
− 12 − 32 1

• Note that for an m × m matrix, we thus only have to do O(m2 ) operations to update the inverse. Note
that we can omit column 1 and 3 from the 3 × 6 matrix above (since they don’t change anyway).
In general, when we change to a new basis, we replace the `-th column of B, AB(`) , by Aj . This
changes the `-th column of
 the outcome of the Gauss-Jordan elimination computation that computes
B− 1 (i.e., Im | B−1 ) to B−1 Aj = u, so we get

e1 . . . e`1 u e`+1 . . . em | B−1 .

 

We need to do elementary row operations to change this to Im | B−1

 
new (where we note that
columns 1, . . . , ` − 1, ` + 1, . . . , m will not change, so we could just delete them and only do the
elementary row operations to change u | B−1 into e` | B−1
   
new ).

1.2 Tableau Implementation

A second way to implement the Simplex Method is the Tableau Implementation. Rather than just updating
B−1 in a smart way, the tableau allows us to update all information we need in a handy way.
A tableau looks as follows:

−cTB B−1 b cT − cTB B−1 A −cTB xB c̄1 . . . c̄n

associated solution xB =B −1
b xB(1)
⇒ ..
B−1 b B−1 A . B−1 A
xB(m)
The tableau has rows labeled 0 to m, and columns labeled 0 to n.

c̄j 0
The j-th column of the tableau looks like this: . If j = B(i), then this is equal to .
B−1 Aj ei

Given the tableau, we can very conveniently do a simplex iteration:

2
 • c̄j for all j is given in row 0, so we can just pick a column j with c̄j < 0. We call this the pivot column.
• Column j in the tableau also contains B−1 Aj which is the vector u we need.
• Let ` be the index of the row that minimizes xB(i) /ui over all rows with ui > 0 (this is called the
minimum ratio test). We call row ` the pivot row, and (`, j) the pivot element.

• Update the tableau: add/subtract multiples of the `-th row to the other rows, so that the j-th column
has a 1 in position (`, j) and 0 everywhere else.
To see this is correct, recall that we already argued that doing elementary row operations is the same as
premultiplying by some matrix Q. First consider the rows 1, . . . , n of the tableau. Note that the matrix
Q satisfies QB−1 AB(i) = ei for all i 6= ` (since the columns for the variables that
 remain basic stay the
−1
same),
 −1 and QB −1
A =
j  e ` . So QB−1
B new = Im , or QB −1
= B −1
new . Then Q B b | B−1 A =
−1
Bnew b | Bnew A which is exactly what we need.
For row 0, note that it is originally equal to

−cTB B−1 b | cT − cTB B−1 A = 0 | cT − cTB B−1

     
b | A .

Then we add some multiple γ of row `, which is itself equal to b−1,T b | A , where b−1,T
 
i i is the
i-th row of B−1 . So, the new row 0 is

0 | cT − (cTB B−1 − γb−1,T

   
i ) b | A . (1)

Let pT = cTB B−1 − γb−1,T

i . We want to show that pT = cTBnew B−1
new :
We chose γ so that the reduced cost of j becomes 0, and note that we also maintain that the reduced cost
of the other variables that remain basic in the new basis is 0. Since the reduced cost of these variables
in the new tableau is cTBnew − pT ABnew (we get this from (1) by removing the non-basic variables
and the zero-th entry), we thus get that pT = cTBnew A−1 T −1
Bnew = cBnew Bnew .

1.3 An Example of the Tableau Implementation (with Degeneracy)

Example:
min −2x1 −x2
s.t. 3x1 +x2 +x3 =6
x1 −x2 +x4 =2
x2 +x5 =3
x1 , x2 , x3 , x4 , x5 ≥ 0
We first choose x3 , x4 , x5 as the basic variables. Then B is the identity, so B−1 A = A, B−1 b = b. Also,
cTB = [0, 0, 0], so cTB xB = 0 and c̄j = cj − cTB B−1 Aj = cj .
Our first tableau is therefore:

0 -2 -1 0 0 0
6 3 1 1 0 0
2 1 -1 0 1 0
3 0 1 0 0 1
We choose x1 as the entering variable. We have a choice for the pivot row (or leaving variable), since
6
3 = 21 . We choose the second row, which means x4 leaves the basis. We add/subtract copies of the second
row to each other row, so that the first column has 0’s everywhere except in the second row.

3
 4 0 -3 0 2 0
0 0 4 1 -3 0
2 1 -1 0 1 0
3 0 1 0 0 1
Note that this solution is degenerate.
In the next iteration, x2 is our only choice for the entering variable, and the first row is the pivot row
(so x3 is the leaving variable).

4 0 0 3/4 -1/4 0
0 0 1 1/4 -3/4 0
2 1 0 1/4 1/4 0
3 0 0 -1/4 3/4 1
Note that this is the same solution (x1 , x2 , x3 , x4 , x5 ) = (2, 0, 0, 0, 3). Now, we let x4 enter the basis. The
pivot row is the last row.

5 0 0 2/3 0 1/3
3 0 1 0 0 1
1 1 0 1/3 0 -1/3
4 0 0 -1/3 1 4/3
We have found an optimal solution, x(1 , x2 , x3 , x4 , x5 ) = (1, 3, 0, 4, 0) with objective value −5.
Remarks:

• Time per operation is O(mn).

• Note that we had no choice but to do a degenerate pivot in this example, but we did not encounter
cycling – after one degenerate pivot, the next pivot (which was our only choice) led to a different
solution. See http://glossary.computing.society.informs.org/notes/cycling.pdf for an example where
cycling does occur (note: the representation in this paper has our row 0 as its bottom row, and our
column 0 as its rightmost column).