9b Simplex Operations Research
9b Simplex Operations Research
Simplex Method
Here we learn how to solve LPs manually. You may think that it is useless
to know such arcane things: We should use computers, you might say —
especially since the practical LPs are so large that no-one really solves them
manually. This criticism is valid. But, we do not study how to solve LPs
manually in order to solve them manually in practical problems (although it is
not a completely useless skill). We study how to solve them manually in order
to understand them!
This lecture is adapted from [2, Ch. 2] and [4, Ch. 4].
Checking Corners
Theorem 5.2.9 told us that the optimal solution of an LP is in one of the corners
of the feasible region. So, it seems that we have a very simple algorithm for
finding the optimum: Just check all the corners! And, indeed, this naïve
approach works well with such petty examples that we have in this course.
The problem with this naïve approach in practice is the so-called combinatorial
curse, a.k.a. the curse of dimensionality: An LP with n decision variables and
m constraints has
n n!
=
m (n − m)!m!
corners.
Let us consider the curse of dimensionality more closely: Consider an LP
with 30 decision variables and 15 constraints. This LP has
30
= 155,117,520
15
corners. Suppose you have a computer that checks 100 corners per second (this
is pretty fast for today’s computers, and right-out impossible if you program
Towards Simplex Algorithm 69
with Java TM ). Then it would take almost three weeks for the computer to check
all the 155,117,520 corners. You may think this is not a problem: Maybe three
weeks is not such a long time, and a problem with 30 decision variables is way
bigger than anything you would encounter in the real life anyway. Well, think
again! Three weeks is a long time if you need to update your optimal solution
in a changing environment, say, daily, and LPs with at 30 decision variables
are actually rather small. Indeed, let us be a bit more realistic now: Consider a
shop owner who has 200 different products in her stock (a rather small shop).
Suppose the shop owner has 100 constraints (not unreasonable) and a super-
computer that checks 100 million corners per second (very optimistic, even if
one does not program with Java TM ). Then checking all the corners to optimize
the stock would take 6.89 × 1044 years. The author doubts that even the
universe can wait that long!
The bottom line is that checking all the corners will take too much time
even with a fast computer and a good programmer.
Simplex Idea
The general idea of the Simplex algorithm is that you do not check all the
corners. The following list explains the Simplex algorithm in a meta-level. We
call the steps Meta-Steps since they are in such a general level that they are
not immediately useful. In the same way the three Meta-Step algorithm could
be called a Meta-Simplex algorithm.
We shall see later how the Meta-Steps can be implemented in practice.
One hopes that in moving around the corners one hits the optimal corner
pretty soon so that one does not have to check all the corners.
To use the meta-algorithm described above we have to:
Once the points raised above are solved we have a genuine algorithm. This
algorithm is given in the next section. Before that we have to discuss how to
prepare an LP before it can be used in the Simplex algorithm.
Towards Simplex Algorithm 70
Slack Forms
Before we can use the Simplex algorithm we must transform the LP into a
so-called canonical slack form.
We start with the slack form. Here is an informal definition of the slack
form: An LP is in slack form, if
1. It is a maximization problem.
2. The constraints are equalities, rather than inequalities.
3. The Right Hand Side (RHS) of each constraint is non-negative.
6.1.1 Example.
and, since we assumed that the slack s1 was non-negative, we have the sign
constraints
x1 , x2 , s1 ≥ 0 (30 )
The interpretation of the slack variable s1 is that it tells how much of the
resource (1) is unused.
Let us then consider line (2). We see that the LP is not in standard form.
We could change it into a standard form by multiplying the inequality (2) by
−1. But that would make the RHS of (2) negative, which is not good. Instead
we ad — or actually subtract — an auxiliary non-negative excess variable e2
to the inequality. We obtain the equality
x1 + x2 − e2 = 100 (20 )
x1 , x2 , s1 , e2 ≥ 0 (300 ).
Towards Simplex Algorithm 71
The interpretation of the excess is opposite to that of the slack: Excess e2 tells
how much the minimal requirement (2) is, well, excessed.
Now, the LP in 6.1.1 is transformed into a slack form:
max z = c0 x
x
s.t. [A S] = b
s
x, s ≥ 0
max z = c0 x
s.t. Ax ≤ b
x ≥ 0
6.1.3 Remark. The index i of the slack/excess refers to resources. For ex-
ample, if s3 = 2 it means that 2 units of the resource 3 is unused.
6.1.4 Remark. We have two kinds of auxiliary variables: slacks (or surpluses)
and excesses. Mathematically there is no need to differentiate between them:
Excess is just negative slack (or negative surplus). Indeed, in some textbooks
slacks are used for both the surpluses and the excesses. However, making
the sign difference makes the problem, and the solution, easier to interpret,
especially since typically the all the coefficients of an LP are non-negative.
There is still one more concept — or two, or three, depending on how you
count — that we have to discuss before we can present the Simplex algorithm:
That of Basic Variables (BV) and Non-Basic Variables (NBV).
Basic variables (BV) and non-basic variables (NBV) are related to the
corners, or the basic feasible solutions (BFS), of an underdetermined linear
system. So, what we are discussing in this subsection is related to the Meta-
Step 1 of the Meta-Simplex algorithm.
Before going into formal definitions let us consider the following problem:
Second, we have to find the objective function. What is it that Leather Ltd.
wants to optimize? The profit! What is the Leather Ltd.’s profit? Well, each
regular belt contributes =C3 to the profit, and each deluxe belt contributes =C4
to the profit. Since we denoted the number of regular belts produced by x1
and the number of deluxe belts produced by x2 the profit to be maximized is
z = 3x1 + 4x2 .
Finally, we have to find the constraints. So, what are the restrictions Leather
Ltd. has to satisfy in making the belts? There are two restrictions: available
labor and available leather. Let us consider the leather restriction first. There
are only 40 units of leather available, and producing one regular belt requires
1 unit of leather. So does producing one deluxe belt. So, the leather constraint
is
x1 + x2 ≤ 40.
Let us then consider the labor constraint. There are only 60 hours of labor
available. Each regular belt produced consumes 1 hour of labor and each
deluxe belt produced consumes 2 hours of labor. So, the labor constraint is
x1 + 2x2 ≤ 60.
Putting what we have just obtained together we obtain the LP for Leather
Ltd. Here it is:
max z = 3x1 + 4x2
s.t. x 1 + x2 ≤ 40
x1 + 2x2 ≤ 60
x1 , x2 ≥ 0
Following the algorithm given after Definition 6.1.2 we can transform the
LP above into a slack form. Here is what we get:
max z = 3x1 + 4x2
s.t. x1 + x2 + s1 = 40
x1 + 2x2 + s2 = 60
x1 , x2 , s1 , s2 ≥ 0
Let us then solve this slack form by using the method of Brutus Forcius
(108–44 BC), which corresponds to checking all the corners. The Brutus’s
method is based on the following observation, listed here as Remark 6.1.8:
Towards Simplex Algorithm 74
x1 = 20, x2 = 20, s1 = 0, s2 = 0
z = =C140.
So, we have solved Leather Ltd.’s problem. Note that both of the slacks, s1
and s2 , are NBV, i.e. zeros, at the optimal decision. This means that at the
optimal solution the resources, leather and skilled labor, are fully utilized. This
full utilization of the resources is not uncommon in LPs, but it is not always
the case. Sometimes it may be optimal not to use all your resources.
Next picture illustrates the situation. The centers of the red balls are the
candidate BFSs (Pts in the previous table). Note that only the points B , C ,
E , and F are actual BFSs. The optimal BFS is the point E .
Simplex Algorithm 75
x2 60
50
A
40
C
30
E
20
10
F B D
0
0 10 20 30 40 50 60
x1
Simplex Steps
set to be zero, and thus the first Simplex tableau will be solved for
the slacks.
max z = c1 x1 + · · · + cn xn
s.t. a11 x1 + · · · + a1n xn +s1 = b1
a21 x1 + · · · + a2n xn +s2 = b2
.. ..
. .
am1 x1 + · · · + amn xn · · · +sm = bm
x1 , . . . , xn , s1 , . . . , sm ≥ 0
becomes
max z
s.t. z − c1 x1 − · · · − cn xn =0
a11 x1 + · · · + a1n xn +s1 = b1
a21 x1 + · · · + a2n xn +s2 = b2
.. ..
. .
am1 x1 + · · · + amn xn · · · +sm = bm
x1 , . . . , xn , s1 , . . . , sm ≥ 0
Since we have to keep track of the BVs, this form is then represented as
the Simplex tableau
From this tableau one readily reads the BFS related to the BVs
s1 , . . . , sm : [s1 · · · sm ]0 = [b1 · · · bm ]0 ; and [x1 · · · xn ]0 = [0 · · · 0]0 .
obviously increasing the value of any xi will increase the value of the
objective z .
Let us then consider the general case. Suppose that, after some steps, we
have come up with a Simplex tableau with the 0th row
where all the coefficients di are non-negative for all the NBVs. Then
making any NBV a BV would decrease the value of the objective. So,
the criterion for the optimality is: The Simplex tableau is optimal,
if in the 0th row there are no negative coefficients in any NBVs.
If the tableau is optimal the algorithm terminates, and the op-
timal value and decision can be read from the BV and RHS
columns.
RHS of row
.
Coefficient of entering varaible in row
The idea of the ratio test is, that we shall increase the entering variable
as much as possible. At some point the increasing of the entering variable
will force one of the BVs to become zero. This BV will then leave. The
ratio test picks up the row associated to the leaving variable.
Step 5: Find a new BFS Now we have a new system of BVs. Next we have
to solve the Simplex tableau in terms of the new BVs. This can be
done by using the Gauss–Jordan method. Then we have a new Simplex
tableau, and we go back to Step 2.
6.2.1 Remark. The Step 1 above corresponds to the Meta-Step 1. The Step
2 corresponds to the Meta-Step 2. The Steps 3–5 correspond to the Meta-Step
3.
Simplex Algorithm 78
Taking s1 , s2 , s3 , s4 to be our first BVs our first Simplex tableau for Dakota is
Row z x1 x2 x3 s1 s2 s3 s4 BV RHS
0 1 −60 −30 −20 0 0 0 0 z= 0
1 0 8 6 1 1 0 0 0 s1 = 48
2 0 4 2 1.5 0 1 0 0 s2 = 20
3 0 2 1.5 0.5 0 0 1 0 s3 = 8
4 0 0 1 0 0 0 0 1 s4 = 5
Step 2: We check if the current Simplex tableau is optimal. The 0th row
is now
Row z x1 x2 x3 s1 s2 s3 s4 BV RHS
0 1 −60 −30 −20 0 0 0 0 z= 0
We see that there is a NBV x1 with negative coefficient −60. So the first
Simplex tableau is not optimal. (Well, one does not expect to get an optimal
solution by slacking off!)
So, Row 3 wins the ratio test. Since s3 the the BV associated to row 3, s3 is
no longer a BV.
Row z x1 x2 x3 s1 s2 s3 s4 BV RHS
0 1 −60 −30 −20 0 0 0 0 z= 0
1 0 8 6 1 1 0 0 0 s1 = 48
2 0 4 2 1.5 0 1 0 0 s2 = 20
3 0 2 1.5 0.5 0 0 1 0 x1 = 8
4 0 0 1 0 0 0 0 1 s4 = 5
Now we have to solve this Simplex tableau in terms of the BVs. This
means that each row must have coefficient 1 for its BV, and that BV must
have coefficient 0 on the other rows. This can be done with EROs in the
following way:
ERO1: We create a coefficient of 1 for x1 in row 3 by multiplying row 3
by 0.5. Now we have the tableau
Row z x1 x2 x3 s1 s2 s3 s4 BV RHS
0 1 −60 −30 −20 0 0 0 0 z= 0
1 0 8 6 1 1 0 0 0 s1 = 48
2 0 4 2 1.5 0 1 0 0 s2 = 20
3 0 1 0.75 0.25 0 0 0.5 0 x1 = 4
4 0 0 1 0 0 0 0 1 s4 = 5
ERO2: To create a 0 coefficient for x1 in row 0, we replace the row 0 with
60(row 3) + row 0. Now we have the tableau
Row z x1 x2 x3 s1 s2 s3 s4 BV RHS
0 1 0 15 −5 0 0 30 0 z= 240
1 0 8 6 1 1 0 0 0 s1 = 48
2 0 4 2 1.5 0 1 0 0 s2 = 20
3 0 1 0.75 0.25 0 0 0.5 0 x1 = 4
4 0 0 1 0 0 0 0 1 s4 = 5
ERO2: To create a 0 coefficient for x1 in row 1, we replace row 1 with
−8(row3) + row 1. Now we have the tableau
Row z x1 x2 x3 s1 s2 s3 s4 BV RHS
0 1 0 15 −5 0 0 30 0 z= 240
1 0 0 0 −1 1 0 −4 0 s1 = 16
2 0 4 2 1.5 0 1 0 0 s2 = 20
3 0 1 0.75 0.25 0 0 0.5 0 x1 = 4
4 0 0 1 0 0 0 0 1 s4 = 5
ERO2: To create a 0 coefficient for x1 in row 2, we replace row 2 with
−4(row 3) + row 2. Now we have the tableau
Simplex Algorithm 81
Row z x1 x2 x3 s1 s2 s3 s4 BV RHS
0 1 0 15 −5 0 0 30 0 z= 240
1 0 0 0 −1 1 0 −4 0 s1 = 16
2 0 0 −1 0.5 0 1 −2 0 s2 = 4
3 0 1 0.75 0.25 0 0 0.5 0 x1 = 4
4 0 0 1 0 0 0 0 1 s4 = 5
Now we see that this Simplex tableau is solved: Each of the BVs have
coefficient 1 on their own rows and coefficient 0 in other rows. So, we go now
back to Step 2.
Row z x1 x2 x3 s1 s2 s3 s4 BV RHS
0 1 0 5 0 0 10 10 0 z= 280
1 0 0 −2 0 1 2 −8 0 s1 = 24
2 0 0 −2 1 0 2 −4 0 x3 = 8
3 0 1 1.25 0 0 −0.5 1.5 0 x1 = 2
4 0 0 1 0 0 0 0 1 s4 = 5
Now we see that this Simplex tableau is solved: Each of the BVs have
coefficient 1 on their own rows and coefficient 0 in other rows. So, we go now
back to Step 2.
Finally, let us interpret the result: The number of desks, tables, and chairs
Dakota Furniture should manufacture is 2, 0, and 8. With this decision
Dakota’s revenue is =C280. Of the resources: 24 units of lumber is left unused:
s1 = 24. All the other actual resources are fully used: s2 = 0, s3 = 0, but
the market demand for tables is not used at all s5 = 5, since no tables are
manufactured.