Lecture 3: Linear Programming Relaxations and Rounding

1 Approximation Algorithms and Linear Relaxations

For the time being, suppose we have a minimization problem. Many times, the problem at hand
can be cast as an integer linear program: minimizing a linear function over integer points in a
polyhedron. Although this by itself doesn’t benefit us, one can remove the integrality constraints
on variables to get a lower bound on the optimum value of the problem. Linear programs can
be solved in polynomial time to give what is called a “fractional solution”. This solution is then
“rounded” to a integer solution, and one normally shows that the rounded solution’s cost can is
within a factor, say ρ, of the fractional solution’s cost. It is clear then that the solution’s cost is
within ρ of the optimum cost.

Example. Let us take the set cover problem we did in the first class. Here is an integer program
for it.
m
X
min c(Sj )xj (1)
j=1
X
subject to xj ≥ 1 ∀i ∈ U (2)
j:i∈Sj

xj ∈ {0, 1} ∀j = 1...m (3)

To get the LP relaxation, we replace the constraint (3) by 0 ≤ xj ≤ 1. Suppose x is an optimal

solution toPthe resulting LP (to stand for linear program henceforth). By the discussion above, we
get opt ≥ m j=1 c(Sj )xj .
Now suppose we are told that each element appears in at most f sets; that is, the frequency of
each element is at most f . For example, in the special case of vertex cover, the frequency of each
element (edge) is 2. What can we say then? Well, let’s look at (2). For each element in U , there
are at most f terms in the corresponding summation, and thus at least one of the xj ’s must be
≥ 1/f . In other words, the algorithm which picks all sets Sj with xj ≥ 1/f , that is rounds their
value up Pto 1, is assured to be a set cover. Furthermore, the cost of this “integral” solution is at
most f · m j=1 c(Sj )xj ≤ f · opt. Thus we get a simple f -approximation to set cover using linear
programs.
Given a particular LP relaxation for an optimization problem, there is a limit on the best
approximation factor obtainable by the process described above. Take an instance I of the problem
(set cover, say), and let lp(I) be the value of the LP relaxation, and opt(I) be the value of the
optimum. Since our solution can’t cost less than opt(I), the best approximation factor we can hope

1
for in this instance is opt(I)
lp(I) . Thus, the best approximation factor one can get for the minimization
problem via this LP relaxation is at most
opt(I)
sup
instances I lp(I)

This quantity is called the integrality gap of the LP relaxation and is very useful in judging the
efficacy of a linear program. What is the integrality gap of the LP relaxation of (1) in terms of f ?

1.1 Some facts regarding linear programs.

A minimization linear program, in its generality, can be denoted as

min c·x (4)

subject to Ax ≥ b

The number of variables is the number of columns of A, the number of constraints is the number
of rows of A. The ith column of A is denoted as Ai , the ith row as ai . Note that constraints of
A could also include constraints of the form xi ≥ 0. These non-negativity constraints are often
considered separately (for a reason). Let’s call the other constraints non-trivial constraints. For
the discussion of this section, let n be the number of variables and let m denote the number of
non-trivial constraints. Thus the number of rows in A are (m + n) and rank(A) = n (since it
contains an In×n as a submatrix).
A solution x is a feasible solution if it satisfies all the constraints. Some constraints may be
satisfied with equality, others as a strict inequality. A feasible solution x is called basic if the
constraints satisfied with equality span the entire space. Let B be the equality rows of A such that
Bx = bB , where bB are the entries of b corresponding to the rows of B. Note that B depends on
x. x is a basic feasible solution if rank(B) = n. The following fact is a cornerstone in the theory
of linear optimization.

Fact 1. There is an optimal solution to every LP which is a basic feasible solution.

Let x be a basic feasible solution, and let supp(x) := {i : xi > 0}. Let B be the basis such
that Bx = bB . Note that B contains at most n − supp(x) non-negativity constraints. Therefore,
at least supp(x) linearly independent non-trivial constraints which are satisfied with equality. This
fact will be used crucially a few classes later.

Fact 2. There is an optimal solution to every LP with at least supp(x) linearly independent non-
trivial constraints satisfied with equality.

A linear program can be solved in polynomial time, polynomial in the number of variables and
constraints, by using the ellipsoid algorithm. Actually more is true. Suppose there is a polynomial
(in n) time algorithm which given a candidate solution x checks if whether Ax ≥ b, and if not
returns a violated inequality ai · x < bi . Such an algorithm is called the separation oracle. If a
separation oracle exists, then LPs can be solved in time polynomial in only the number of variables.
This can possibly be (and will be) used to solve LPs with exponentially (in n) many constraints.
By the way, note that if m is polynomial in n, then clearly there is a polytime separation oracle.

2
Let’s talk a bit more about how the ellipsoid algorithm works. Actually, the ellipsoid method checks
feasibility of a system of inequalities in polynomial time. This is equivalent to solving the LP: one
guesses the optimum M of the LP and adds the constraint c·x ≤ M to the system Ax ≥ b. Suppose
that the system Ax ≥ b is bounded; if not, it can be made bounded by intersecting with a fairly
large cube. In each iteration of the ellipsoid algorithm, a candidate solution xt is generated. (It
is the centre of a carefully constructed ellipsoid containing {x : Ax ≥ b} - hence the name of the
algorithm.) Now, the separation oracle either says that xt is feasible in which case we are done,
or else it returns a constraint at · x ≥ bt (or c · x ≤ M ) which is violated by xt . The nontrivial
theorem regarding the ellipsoid algorithm is that in polynomial (in n) many iterations, we either
get a feasible solution or a proof that the system is infeasible. If the system is infeasible, then our
guess M was too low and we raise it; if the system is feasible then we lower our guess and run
again. By doing a binary search, we can reach the true optimum M in polynomial time.
Consider the run of the ellipsoid algorithm when the guess was M − , “just below” M . That
is, the LP cannot take any value between M and M − . Let (Â, b̂) be the collection of polyno-
mially many constraints/rhs’es returned by the separation oracle in the various iterations, af-
ter which the algorithm guarantees infeasibility. Consider the system Âx ≥ b̂; we claim that
min{c · x : Ax ≥ b} = min{c · x : Âx ≥ b̂}. Surely, the second is lower since we have removed
constraints. However, the ellipsoid algorithm certifies infeasibility of {c · x ≤ M − , Âx ≥ b̂}. So,
min{c · x : Âx ≥ b̂} = M as well. Thus given a cost vector c, the exponentially many constraints of
the LP can be “essentially” reduced to the polynomial sized LP.

We end our discussion with one final point. We claimed that there exists an optimum solution
which is basic feasible. However, the ellipsoid algorithm is not guaranteed to return a basic feasible
solution. Can we get a optimal basic feasible solution in polynomial time? More generally, given a
point x ∈ P := {x : Ax ≥ b} which is guaranteed to minimize c · x over all points of P, can we move
to a basic feasible solution of P of the same value. (We have reverted to the old notation where A
includes the non-negativity constraints as well). Consider the equality submatrix Bx = bB . If B is
full rank, we are at a basic feasible solution. If not, let v be a vector in the null space of B; that
is ai · v = 0 for all ai ∈ B. Such a vector can be found by solving a system of linear equations, for
instance. Suppose c·v ≥ 0. Consider x0 = x−αv where α is a positive scalar such that for all i ∈ / B,
a· x−b
ai · (x − αv) ≥ bi . Since P is bounded, we have that ai · v > 0 for some i. Choose α := min ai ·v
over the i’s ∈/ B with ai · v > 0. What can we say about x0 ? It is feasible. c · x0 ≤ c · x (implying
c · v = 0). Since ai · v > 0 and v is in the null space of B, ai and B form a linearly independent
system. So, we move to a solution with an extra linearly independent constraint in the equality
matrix. Repeat this till one gets a basic feasible solution.

Fact 3. An optimum basic feasible solution to a LP can be found in polynomial time.

Remark 1. A word of caution. Whatever written above about the ellipsoid method should be
taken with a pinch of salt (although the above fact is true). After all, I have not described how
the “carefully constructed” ellipsoids are generated. To make all this work in polynomial time is
more non-trivial and we refer the reader to the book “Geometric Algorithms and Combinatorial
Optimization” by Grötschel, Lovász, and Schrijver.

Let’s get back to approximation algorithms.

3
2 Facility Location
LP Relaxation
X X
min fi yi + c(i, j)xij (xij , yi ≥ 0) (5)
i∈F i∈F,j∈C
X
subject to xij ≥ 1 ∀j ∈ C (6)
i∈F
yi ≥ xij ∀i ∈ F, ∀j ∈ C (7)

P Let (x, y) be a fractional

P solution to the above P LP. We define some notation. Let FLP :=
i∈F fi yi . Let C j := i∈F c(i, j)x ij . Let C LP = j∈C Cj . We now show an algorithm which re-
turns a solution of cost at most 4(FLP + CLP ) ≤ 4opt. The algorithm proceeds in two stages. The
first stage is called filtering which will “take care” of the xij ’s, the second stage is called clustering
which will “take care” of the yi ’s.

Filtering. Given a client j, order the facilities in increasing order of c(i, j). That is, c(1, j) ≤
c(2, j) ≤ · · · ≤ c(n, j). The fractional cost of connecting j to the facilities is Cj ; our goal is to pay
not much more than this cost (say within a factor ρ > 1 which we will figure out what it should
be later). So, we look at the set of facilities Nj (ρ) := {i : c(i, j) ≤ ρ · Cj }. Now it is possible that
xij > 0 for some i ∈ / Nj (ρ). However, we can change the solution so that j is fractionally connected
only to facilities in Nj (ρ). This is called filtering the fractional solution.
ρ
Define xij as follows. xij = ρ−1 · xij for i ∈ Nj (ρ), xij = 0 for i ∈
/ Nj (ρ).

Claim 1. x satisfies (6).

P P P
Proof. Let us consider the sum Cj = i c(i, j)xij = i∈Nj (ρ) c(i, j)xij + i∈N / j (ρ) c(i, j)xij . Since
P 1
c(i, j) > ρ · Cj for all i ∈
/ Nj (ρ), we get i∈N
/ j (ρ) xij < ρ , for otherwise the second term in the RHS
above exceeds Cj . This implies i∈Nj (ρ) xij ≥ (1 − ρ1 ) implying i∈F xij ≥ 1.
P P

Suppose we could guarantee that a client j is always connected to a facility i with xij > 0, then
we could argue that the cost paid to connect this pair is at most ρ · Cj . However, we have to argue
about the facility opening costs as well. This is done using a clustering idea.

Clustering. Suppose we could find sets of disjoint facilities F1 , F2 , . . . such that in each Fk , we
cost facility i∗k in each Fk . Then, our total
P
have i∈Fk yi ≥ 1, and we only P open theP minimum
P
facility opening cost would be k fi∗k ≤ k i∈Fk fi yi ≤ FLP . In fact, if the 1 is replaced by α,
then our facility opening cost is at most FLP /α. We now show how to obtain such a clustering.

The clustering algorithm proceeds in rounds. In round `, the client with minimum Cj in C is
picked. The facility F` is defined as the set of facilities {i : xij` > 0}. All clients j such that there
exists an i ∈ F` with xij > 0 are deleted from C. Let all these clients be denoted by the set N 2 (j` ).
The procedure continues till C becomes empty. Note that Fk ’s are disjoint.
The algorithm opens the cheapest facility i` ∈ F` and connects all clients 2 (j ) to i . Let’s
P j ∈NP ` `
argue about the facility opening cost. Note that for each F` , we have i∈F` yi ≥ i∈F` x(i, j` ) ≥
ρ−1 P ρ−1 ρ
ρ i∈F` xij` ≥ ρ by Claim 1. Thus, the total facility opening cost is at most ρ−1 · FLP .

4
 Consider a client j. Either it is a j` , in which case the assignment cost is at most ρ · Cj .
Otherwise, it is connected to i` ∈ F` for some `. However, there is some i (not necessarily i` ) in
F` such that xij > 0. Now, c(i` , j) ≤ c(i, j) + c(i, j` ) + c(i` , j` ) ≤ ρ · Cj + 2ρ · Cj` ≤ 3ρ · Cj since
Cj ≥ Cj` . The latter’s because the j’s are considered in increasing order of Cj .
ρ
So, alg ≤ ρ−1 FLP + 3ρCLP . If ρ = 4/3, then the above gives alg ≤ 4(FLP + CLP ).

3 Generalized Assignment Problem

In GAP, we are given m items J, and n bins I. Each bin i can take a maximum load of Bi . Each
item j weighs wij in bin i, and gives profit pij when put in it. The goal is to find a max-profit
feasible allocation.

LP Relaxation
X
max pij xij (xij ≥ 0) (8)
i∈I,j∈J
X
subject to wij xij ≤ Bi ∀i ∈ I (9)
j∈J
X
xij ≤ 1 ∀j ∈ J (10)
i∈I
Solve the LP to get a fractional
P solution x. Suppose all the weights wij were Bi , say. Then the
first constraint is equivalent to j∈J xij ≤ 1. Consider the bipartite graph (I, J, E) where we have
an edge (i, j) if xij > 0. Then, the solution x above is a maximum weight fractional matching in
this bipartite graph. This implies that there is an integral matching of the same weight, and thus
there is an allocation equalling the LP value (and hence the optimum value). This is a non-trivial
combinatorial optimization fact and is key to the approximation algorithm below. Let’s come to
this fact later.
Of course wij is not equal to Bi , although we can assume wij ≤ Bi since if not we know xij
has to be 0 for such a pair. (Note this must be put into the P above LP explicitly, that is,Pwe must
set xij = 0 for such pairs as constraints.) Thus, in general, j∈J xij ≥ 1. Let ni := d j∈J xij e.
The algorithm proceeds as follows: it uses x to define a fractional matching on a different bipar-
tite graph, use it to get an integral matching of equal weight in it, and then do a final step of pruning.

New Bipartite Graph. One side of the bipartition is J. The other side is I 0 which consists
of ni copies of each bin i in I. For each bin i ∈ I, consider the items in J in increasing weight
order. That is, suppose, wi1 ≤ wi2 ≤ · · · ≤ wim . Consider the fractions xi1 , xi2 , . . . , xim in this
order. Let j1 , j2 , . . . , jni −1 be the “boundary” items defined as follows. xi1 + · · · + xj1 ≥ 1, and
xi1 + · · · + xj1 −1 < 1; xi1 + · · · + xj2 ≥ 2, and xi1 + · · · + xj1 −2 < 2; and so on. Formally, for each
1 ≤ ` ≤ ni − 1,
j
X̀ ` −1
jX
xij ≥ `; xij < `
j=1 j=1
Recall there are ni copies of the bin i in I 0.
The `th copy, call it i` , has an edge to items j`−1 to j`
(j0 is the item 1). The ni th copy has an edge to items jni−1 −1 to m.

5
New Fractional Matching. x0 is so defined such that for every copy i` , the total fractional
weight incident on it is at most 1 (in fact, it’ll be exactly 1 for all copies but the ni th copy). The
total fractional weight incident on item j is the same as that induced by x. It’s clear how to do it
given the way edges are defined above; formally it’s the mess given below.

x0i` ,j = xij , for j`−1 < j < j`

x0i` ,j`−1 = xi,j`−1 − 0
xi`−1 ,j`−1 (if ` = 1, then the second term is 0).
` −1
jX
x0i` ,j` = 1− x0i` ,j
j=j`−1

Integral Matching and Pruning. Note that x0 defines a fractional matching in (I 0 , J, E 0 ). Thus,
there is an allocation of items to I 0 whose profit is at least the LP profit (and thus at least opt).
The assignment to I 0 implies an assignment to I in the natural way: bin i gets all items allocated
to the ni copies in I 0 . Is this feasible? Not necessarily. But, the total weight allocated to bin i is
not too much larger than Bi .

Claim 2. The total weight of items allocated by the integral matching above is at most Bi + ∆i ,
where ∆i := maxj∈J wij .

Proof. We use the fact that the items (for bin i) were ordered in increasing order of weights. Let
J` be the set of items from j`−1 to j` , and let Jni be the items from j` to m. Since x is a feasible
solution to the LP, we get
 
X ni X
X ni
X X
Bi ≥ wij xij = wij x0i` ,j ≥ wi,j`−1 · x0i` ,j  ≥ wi,j1 + · · · + wi,jni −1
j∈J `=1 j∈J` `=1 j∈J`

The second inequality uses the increasing order of weights, the last uses the fact that x0 forms a
fractional matching.
Now, bin i gets at most one item from each J` (since each copy i` gets at most item from it’s
neighbors in J` .) Note that the heaviest item in J` weighs wi,j` , and ∆i = wi,ni . Thus, the load on
bin i is at most wi,j1 + · · · + wi,ni −1 + wi,ni which is at most Bi + ∆i .

To summarize, we have found an allocation whose profit is at least opt and each bin i has load
at most Bi + ∆i . For each bin i now consider the heaviest item j allocated to it. We know that
wij ≤ Bi . We also know weight of all the other items allocated to it is ≤ Bi . To make the allocation
feasible, keep either j or the rest, whichever gives more profit. In this pruned allocation, each bin
thus gets profit at least half of what it got in the earlier infeasible allocation. Thus, the profit of
this new feasible allocation is at least opt/2. Thus, the above algorithm is a 1/2-approximation.

3.1 Fractional Matching to Integral Matching

Given a fractional matching x, construct the fractional bipartite graph G = (I, J, Ef ) where there
is an edge between i ∈ I and j ∈ J with 0 < xij < 1. We now describe a procedure which takes x
and converts it to x0 such that two things occur: a) the number of edges in the corresponding Ef

6
 wij x0ij . The fact follows.
P P
is strictly less, and b) i,j wij xij ≤ i,j

Procedure Rotate.

1. Pick a path or cycle in G = (I, J, Ef ). Call the edges picked F . Decompose F into two
matchings M1 and M2 .

2. Let  
ε1 := min min xij , min (1 − xij )
(i,j)∈M1 (i,j)∈M2
 
ε2 := min min xij , min (1 − xij )
(i,j)∈M2 (i,j)∈M1

(1) (1)
3. Define x(1) as follows. For each (i, j) ∈ M1 , xij = xij −ε1 , for each (i, j) ∈ M2 , xij = xij +ε1 ,
(1)
for all other edges xij = xij . Define x(2) similarly.

4. Let x0 be the x(1) or x(2) with larger wij x0ij .

P
i,j

It
P is clear0 that the number of edges in Ef decreases in this procedure. Also note that the sum
w x is at least i,j wij xij . This is because the “increase” in weight in x(1) over x is precisely
P
P ij ij
i,j  P 
(2) is ε
P P
ε1 w
(i,j)∈M2 ij − w
(i,j)∈M1 ij , and that in x 2 w
(i,j)∈M1 ij − (i,j)∈M2 ij . One of
w
them is at least 0. The following claims ends the analysis of the procedure.

Claim 3. x0 is a feasible fractional matching.

Proof. If F forms a cycle, then it’s clear that the fractional “load” on any vertex remains unchanged.
If F forms a path, then we need to only concern about end vertices. Let i be such a vertex. Note
that there are no edges (i, j 0 ) with xij 0 = 1 incident on it, and exactly one edge (i, j) ∈ Ef incident
on it. In the end, x0ij ≤ 1.