Content

Introduction

PART I: Linear Programming

1. Introduction & Standard Form of LP
2. Geometric Methods
3. The Simplex Method

8/4/2024 1
 Introduction
i) Introduction to Optimization

ii) Basic Definitions and Terminologies

iii) Some Examples of Problem Formulations

8/4/2024 2
 Introduction
i) Introduction to Optimization
Linear Programming is a basic course within the subject
area called Mathematical Programming / Operations
Research / Optimization.
Commonly, we use Optimization for the subject area.
What does optimization mean?
 Literally: Among various possibilities, it is to find the
best possible solution of a problem under a set of
constraints.
 In Mathematics: To optimize means –
to maximize or minimize a function f(x), where the
values of x have to satisfy a set of prescribed
mathematical relations.

8/4/2024 3
 Thus, given a real valued function f of several variable, i.e.,
f : R n  R and a set S  Rn
a problem of optimization is either of the following types:

1. Find a point x*S, at which the value of f is maximum

over the set S.
That is, find x*S such that f(x*)  f(x) for all xS .
This problem is written as
max f(x)
s.t x  S
Note: x = (x1, x2, …, xn) in Rn.

2. Find a point x*S, at which the value of f is minimum

over the set S.
That is, find x*S such that f(x*)  f(x) for all xS .
This problem is written as
min f(x)
s.t x  S
8/4/2024 4
 Graphically, consider the followings:

● ●
f(x)
min f(x)
xS
 x * = x2

x0 x1 S x2
●
x * = x2

f(x*) f(x)
max f(x)
xS
 x * = x1 ●
x0 x1 S x2
x* = x1
8/4/2024 5
 Examples: Formulate each of the following problems as
an optimization problem (identify its f and set S)
(Max Volume) Find the length, width and height of a box so
that its surface area is 200 m2, base area is at least 30 m2,
and volume is maximum.
Let x1 = length, x2= width & x3 = height of the box
Restrictions:
Surface area: x3
Base area:
Aim:
x2
Volume:  Max x1
The Problem is
max x1 x2 x3
s.t 2x1 x2 +2x1 x3 + 2x2 x3  200
x1 x2  30
x1 , x2 ,x 3  0
8/4/2024 6
 ii) Basic Definitions and Terminologies

Consider an optimization problem

(P) max (or min) f(x) s.t xS.

The components of variable x, i.e., x1, x2, …, xn, are called

decision variables
The function f(x) is called objective function of (P).
The conditions (inequalities/equations) describing S are called
constraints

The conditions that xj  0 in optimization problem is called non-

negativity constraints

8/4/2024 7
 Example (Production Planning): Suppose that two machines
M1 and M2 are used to produce two containers A and B . In
the production of a unit of the containers, A needs M1 for
2 minutes and M2 for 4 minutes. Similarly, B occupies M1
for 8 minutes and M2 for 4 minutes. The machines can be
used for at most 1 hour per day. The net profit for a
container A is 29 and for B is 45. The problem is to find the
number of daily production of each container that
maximizes the net profit per day.

Let x1:= # of container A, x2:= # of container B

The Problem is
max 29x1  45 x2
s.t 2x1 +8 x2  60
4x1 +4 x2  60
x1 , x2  0
8/4/2024 8
 Feasible and optimal Solution of
(P) max/min f(x)
s.t.. xS

x = (x1, x2, …, xn ) is called a feasible solution (or a feasible

point) of (P), if xS ; i.e., if the values of its components, x1,
x2, …, xn satisfy the constraints.

Note: Optimization problem has usually infinitely many feasible

solutions. In fact, every xS is a feasible solution for (P).

Optimal solution and Optimal Value:

• x‫ ٭‬ S is called optimal solution for a maximization problem if
f(x‫ )٭‬ f(x) for all x  S.
• x‫ ٭‬ S is called optimal solution for a minimization problem if
f(x‫ )٭‬ f(x) for all x  S.
• If x‫٭‬S is optimal solution of an optimization problem (P), then
f(x‫ )٭‬is called its the optimal (objective) value of (P).

8/4/2024 9
 Min problem is equivalent to max problem: i.e.,
x‫ ٭‬is optimal solution of min f(x) on S
if and only if
x‫ ٭‬is optimal solution of max - f(x) on S.

Because, for all xS,

f (x‫ ≤ )٭‬f(x) iff f(x*1)

- f(x*) ≥ - f(x)
f(x)

Hence, it suffices to f(x*2)

consider solution x*1 x*2
procedure only for S
one of them, say, only -f(x*2)
for a maximization
problem.
-f(x)
-f(x*1)

8/4/2024 10
 If the feasible set S =  , then the problem is called infeasible.
Example: Let (P) be max 2 x1  x2
s.t x1  x2  1
 x1  x2  1
x1 , x2  0
There is no value for x1 & x2 that satisfy the constraints. i.e.,

S = ( x1 , x2 )  R 2 | x1  x2  1,  x1  x2  1, x1 , x2  0  
 The problem is called infeasible.
The optimization problem is said to be unbounded, if
• f(x) tends to  on S for the maximization problem
• f(x) tends to –  on S for the minimization problem

Example: max 5x2 ( unbounded

s.t x1  x2  1 problem )
x1 , x2  0

8/4/2024 11
 Linear Programming (LP):
An optimization problem
max (or min) f(x) s.t xS,
is said to be a linear program (LP) if the objective
function f(x) and the functions involving in the
description of S are all linear functions.
That is, if all involved functions (the objective and the
constraints) are linear, then the problem is called a Linear
Programming (LP) or a Linear Optimization

Example: - The Production Planning problem given

above is a linear programming problem.

- The Max Volume problem given above is not

an LP. It is a nonlinear programming problem.

The subject matter of this course is only LP

8/4/2024 12
 A common form of an LP Problem:
max c1 x1  c2 x2  ...  cn xn
s.t. a11 x1  a12 x2  ...  a1n xn  b1
a21 x1  a22 x2  ...  a2 n xn  b2
Constraints

am1 x1  am 2 x2  ...  amn xn  bm

x j  0, j  1...., n Non negativity

Concise form: max cT x

s.t Ax  b
x0
cRn, bRm, and A is an m×n matrix.
( m constraints & n variables. Usually m  n )
8/4/2024 13
 Some Examples of LP Formulations
1. Production Allocation Problem
A company is producing four kind of alloys, called A1, A2, A3 and
A4, by mixing three different metals: iron, copper and zinc. The
following table contains the amount (in gram) of each metals
required to produce a unit of each of the alloys.

Required Products Available quantity

Metals A1 A2 A3 A4 of the metals

Iron 25 30 15 50 165 g
Copper 30 40 50 15 270 g
Zink 55 20 35 35 275 g
Profit /unit 30 20 25 15

The company wants to determine the units of each product

(alloys) to be manufactured in order to maximize profit. Give
a an LP formulation of the problem.

8/4/2024 14
 • Decision variables: xi = units of Ai to be produced,
i = 1, …, 4
• Constraints:
Iron: 25x1  30 x2 +15x3 +50x4  165
Copper: 30x1  40 x2 +50x3 +15x4  270
Zinc: 55x1  20 x2 +35x3 +35x4  270
Non-negativity: x1 ,..., x4  0

• Objective function: 30x1  20 x2 +25x3 +15x4  Max

Thus the LP formulation of the problem is :

max 30x1  20 x2 +25x3 +15x4

s.t 25x1  30 x2 +15x3 +50x4  165
30x1  40 x2 +50x3 +15x4  270
55x1  20 x2 +35x3 +35x4  270
x1 ,..., x4  0
8/4/2024 15
 2. Diet Problem
A patient needs daily at least 5 mg, 20 mg, & 15 mg of
vitamin A , B, & C, respectively. The available sources for
these vitamince are mango, orange and apple. The vitamins
available from
a mango are 0.5 mg of A, 1 mg of B, 1mg of C;
an orange are 2mg of B, 3mg of C; and
an apple are 0.5mg of A, 3 mg of B, 1mg of C,
respectively.
If the cost of a mango, an orange and an apple is 0.75, 0.50
and 1.25, respectively, find the mathematical formulation of
the problem of minimizing the cost of obtaining the fruits so
that the daily requirement of the patient is met.

8/4/2024 16
 Formulation:

The data for the diet problem in table :

Fruit Daily
Vitamins mango Orange Apple Requirement
s of Vitamins
A 0.5 0 0.5 5
B 1 2 3 20
C 1 3 1 15
Cost/unit 0.75 0.50 1.25

• Decision variables: x1 = # of mango to buy

x2 = # of orange to buy
x3 = # of apple to buy

8/4/2024 17
 • Constraints: Daily requirement of Vitamins:

A: 0.5x1 + 0.5x3  5
B: x1  2 x2 +3x3  20
C: x1  3x2 + x3  15
Non-negativity: x1 , x2 , x3  0
• Objective function: 0.75x1  0.50 x2 +1.25x3  Min
Thus the LP formulation of the problem is :

min 0.75x1  0.50 x2 +1.25x3

s.t 0.5x1 + 0.5x3  5
x1  2 x2 + 3x3  20
x1  3x2 + x3  15
x1 , x2 , x3  0
8/4/2024 18
 Linear Programs in Standard Form
A linear program is said to be in standard form if it is
given as
max c1 x1  c2 x2  ...  cn xn
s.t. a11 x1  a12 x2  ...  a1n xn  b1
a21 x1  a22 x2  ...  a2 n xn  b2

am1 x1  am 2 x2  ...  amn xn  bm

x j  0, j  1...., n

Or : max cx cRn, - row vector

s.t Ax  b A - m×n matrix
(Usually m  n )
x0 bRm - column vector
xRn - vector of decision
variables
8/4/2024 19
 In fact, a general LP problem can be

 either a maximization or minimization problem.

 Some of the constraints can be inequalities connected
by either “ ” or “ ” .
 Some of the decision variables can be unrestricted in
sign; i.e., can be negative or positive

However, any LP can be changed to the standard form as

shown below.

1. Minimization problem min cx can change to its

equivalent maximization problem max –cx
2. Inequalities can be changed into equations using slack
and/or surplus variables defined as follows.

8/4/2024 20
 A) Slack Variables:
The “ ” constraint, say
ai1 x1  ai 2 x2 + +ain xn  bi ,
can be transformed into equality by introducing a
variable s, called slack variable, where

s  bi  (ai1 x1  ai 2 x2 + +ain xn )
Observe that, s  0.
Hence, the given inequality is equivalent to
ai1 x1  ai 2 x2 + +ain xn  s  bi , s  0.
For instance, x1  2 x2 +3x3  10
is equivalent to x1  2 x2 +3x3  x4  10,
x4  0.
( x4 is slack variable)

8/4/2024 21
 B) Surplus Variables:
The “ ” constraint, say
ai1 x1  ai 2 x2 + +ain xn  bi ,
can be transformed into equality by introducing a
variable s, called surplus variable, where

s  ai1 x1  ai 2 x2 + +ain xn  bi
Observe that, s  0.
Hence, the given inequality is equivalent to
ai1 x1  ai 2 x2 + +ain xn  s  bi , s  0.
For instance, x1  2 x2 +3x3  10
is equivalent to x1  2 x2 +3x3  x4  10,
x4  0.
( x4 is slack variable)

8/4/2024 22
 3. If a decision variable, say, xj is unrestricted in sign we
can replace it by the difference of two nonnegative
variables
x j : x1 j  x2 j , x1 j , x2 j  0
in order to change the problem into standard
Thus, any LP problem can be written in the standard form.

For instance, the above production allocation problem can

be written in the standard LP form as:

max 30x1  20 x2 +25x3 +15x4

s.t 25x  30 x +15x +50x  x5 =165
1 2 3 4
30x1  40 x2 +50x3 +15x4 +x6  270
55x1  20 x2 +35x3 +35x4 +x7  270
x1 , x2 ..., x7  0
( Here, x5, x6 & x7 are slack variables )

8/4/2024 23
 Exercise: Write the following LP in the standard form.

min  3x1  5 x2  x3
s.t 2 x1  3x2  x3  5
4x1  3x2 +5x3  15
x1  x2 + x3  4
x1 , x2  0, x3  R

8/4/2024 24
 Graphical Solution Methods for LP
In this section we consider the case when an LP problem
involves only two decision variables. Thus, we consider

max/ min f ( x 1 , x2 ), such that ( x1 , x2 )  S  R 2 .

where f (x1, x2) = c1x1 + c2x2

and the feasible set S is a polyhedral given by
a11 x1  a12 x2 ( or  ) b1
a x  a x ( or  )b


21 1 22 2 2

 S
a x  a x ( or  )b
 m1 1 m2 2 m

 x 1 , x2  0


8/4/2024 25
 • The graphical solution method uses level lines of f
• Level line (contour) of f is a set of all points where f has
equal (a constant) value.
That is, a level line of f is a line given by
c1x1 + c2x2 = z , where z is a constant.
• Recall from calculus that the gradient , f = (c1, c2)T , is
orthogonal (perpendicular) to the level line of f.

Recall:
f points toward the direction of maximum rate of
increase of f ; and
–f towards minimum rate of decrease of f .
8/4/2024 26
  Thus, to solve LP, take any point (x1, x2)S,
- determine a level line l of f through this point,
- and use the fact that
f increases if we move from (x1, x2) to any other point in
S in the direction of f (for maximization).
l
• f = z3 , z3  z2
l
f

f = z1 l
f
S f = z2 , z2  z1
(x1,•x2)

 So, the level line l are kept moving parallel to it self in S in

the direction of improving value of f as far as possible.
 The last point of intersection of l and S (where l is tangent
to S) is the optimal solution.
8/4/2024 27
 • Thus, to graphically solve max f (x1, x2) = c1x1 + c2x2 on a
polyhedral set S, follow the following steps.

1. Plot the feasible set, S, on the coordinate plane.

2. Select an X0S and get the value f (X0). Let z = f(x0).

3. Plot the level line of f given by c1x1 + c2x2 = z, through X0.

4. Translate (move) the level line parallel to itself in the

direction of f within S until it is tangent to S at some
point X* on the boundary of S.
This X* is an optimal solution.

Note: The procedure is the same for minimization problem

except the direction of the motion is –f .

8/4/2024 28
 Example: we consider maximizing/minimizing
f (x1, x2) = c1x1 + c2x2 on S, where S is given by the
following system of inequalities.
 2 x1  x 2  3
 x1  2 x 2  9

 x2  6
 3 x  x  21 3x1 +x2 =21
 1 2 x2
 3 x1  x 2  15

 x1 , x 2  0. 7
x2=6 (3,6) (5,6)
6
First draw the feasible (1,5)
5
set S as shown here:
4
3 S: feasible set (6,3)
S is a polyhedral  Set
2
of all points satisfying
1
all linear constraints
x1
1 2 3 4 5 6
8/4/2024 29
 • Now let, f(x1,x2) = x1 + x2 and maximize f on S
 Level line x1 + x2 = z , z a constant.
Optimal
z=11
z=9 x* =(5,6)
z* =11
z=6 7
(3,6) (5,6)
6
(1,5)
z=3 5
z=1 4
3 S (6,3)

z=f(X0)=1 2

X0=(0,1) 1

1 2 3 4 5 6
Increasing
direction:
f=(1,1)
x + x2=1
Berhanu G (Dr) 1 30
8/4/2024
 Next let f(x1,x2) = –5x1 + x2 and minimize f on S.
• level line: –5x1 + x2 = z, and f = ( –5, 1).
Optimal
x2 X*=(6,3)
z=1 z=-9 z= -19 z*= – 27
7
(3,6) (5,6)
6
(1,5) z=-27
5

4
S
3 (6,3)

2
1

-1 1 2 3 4 5 6 x1
decreasing
-5x1 + x2=1 direction: –f = ( 5, –1).

8/4/2024 31
 Note:
• For the above examples, optimal solutions occurred at the
boundary of the polyhedral feasible sets , particularly at a
vertex.
• This is always the case for LP since f is a nonzero
constant vector and f keeps increasing in the direction of
f (or keeps decreasing in the direction of –f ) unless the
progress is stopped by the boundary of the feasible set.
• In fact, the farthest (outermost) boundary point of a
polyhedral in a direction of improvement is a vertex (corner
point) of the polyhedral.
• Hence, both maximum and minimum of a linear objective
function (if exist) occur at a vertex (corner point) of the
feasible polyhedral
• If the polyhedral set is unbounded in the direction of
improvement, then the LP is unbounded.

8/4/2024 32
 Example: Consider max x1  x2
s.t  2 x1  x2  3
 x1  2 x2  3
x1 , x2  0.
• 1st Draw the feasible set S.
• The zigzag at a side of the
polyhedral S indicates the
polyhedral is unbounded in
that side.
• The objective function of
this problem keeps S
increasing in S with no 4
bound. 3
• Hence, this LP is 2
unbounded. 1
X0=(0,1)
1 2 3 4
Increasing
direction: x1 + x2=1
f=(1,1)
8/4/2024 33
 Basic Solutions`

 Consider a system of m linear equations with n variables,

Ax  b.
where A is an mxn matrix bRm, and
rank(A) = rank(A, b) = m
( Otherwise, omit redundant equations)
 A has m linearly independent columns (which forms a
basis of Rm) .
 Any m linearly independent columns of A are called basic
columns, and the remaining n–m columns are called non-
basic columns.
 The m variables attached to the basic columns are called
basic variables (BV), and the remaining n–m variables are
called non-basic variables (NBV).

8/4/2024 34
 The m linearly independent columns of A forms an mxm
invertible sub-matrix, called basic submatrix and denoted by B.
The remaining non-basic columns form an mx(n–m) submatrix
denoted by N.

Now, by rearranging if needed, let us partition A as

A = ( B, N )

  xB , x N 
T
Also, partition x asx

xB is be the m-vector of BVs attached to columns in

B, &
xN is the n-m vector of NBVs attached to columns in
 xB 
N.
Hence, Ax  b   
B, N    b
 xN 
 BxB  NxN  b

8/4/2024 35
 Hence, x is a solution of Ax = b
 x = (xB , xN)T is a solution of BxB +NxN = b

 xB  B 1 NxN  B 1b
 xB  B 1b  B 1 NxN
The last equation is called the basic representation w.r.t B.
Hence, for any choice of values of xN , the values of xB is
uniquely determined so that
x = ( xB , xN )T is a solution of Ax = b.

Thus, if we choose xN = 0 = (0, 0, …, 0), then xB = B–1 b;

and
 xB   B 1b 
x  
 N  0 
x
is the solution of Ax = b, called basic solution wrt B.
8/4/2024 36
 Remarks: Let Ax = b, where A is mxn & rank(A)= m.
1. Any m columns of A can be selected to form an mxm
submatrix B; (It is basic when det(B)  0 )
2. From the n columns, the number of ways to select m
basic columns is at most

n  n!
n
Cm    
 m  m !(n  m)!
3. Hence, the number of basic solutions of the system is
at most equal to nCm ( So, it is finite).

4. Definition: A basic solution x = (xB , 0 )T is called basic

feasible solution (BFS) with respect to B if
xB = B–1 b  0.
Thus, a BFS is a solution of Ax = b, x  0.

8/4/2024 37
 Example 1: Find all basic solutions (BS) of
8 x1  4 x2  6 x3  16 x4  12
3x1  5 x2  4 x3  6 x4  8
(and also identify basic feasible solutions (BFS) )
Solution: The coefficient matrix is A = (A1, A2, A3, A4), where
 4  2 3  8  6
A1    , A2    , A3    , A4    , & b   
3 5  4  6  8 
A has 6 sub-matrices of order 2:
B1   A1 , A2  , B2   A1 , A3  , B3   A1 , A4  ,
B4   A2 , A3  , B5   A2 , A4  , B6   A3 , A4 
and det( B1 )  14, det( B2 )  7, det( B3 )  0
det( B4 )  7, det( B5 )  28, det( B6 )  14
 5 basic sub-matrices: B1, B2, B4, B5, B6
8/4/2024 38
 So, there can be 5 basic solutions w.r.t each of the basic
sub-matrices. In Particular,
 1
• W.r.t 1  1 2  xB  B b     x1  1, x2  1
B  A , A , 1

 1
1 1

 ( x1 , x2 , x3 , x4 )  (1,1,0,0) is a BS, which is also BFS

 0
• W.r.t B2   A1 , A3  , xB  B2 b     x1  0, x3  2
 1

 2
2

 ( x1 , x2 , x3 , x4 )  (0, 0, 2, 0) is a BS, which is also BFS

• Similarly, w.r.t B4, B5 & B6 we obtain

 0   x2  1   x2   2   x3 
xB4    =   , xB5    =   , xB6    =  
 2   x3   1/ 2   x4   0   x4 
Hence, the BS, w.r.t B4, & B6 are also BFS. They give
the same BS: ( x1 , x2 , x3 , x4 )  (0, 0, 2, 0)
However , the BS w.r.t B5, ( x1 , x2 , x3 , x4 )  (0,1,0, 1/ 2)
is not BFS since x4 < 0 .
8/4/2024 39