Introduction to

Approximation
Algorithms

1
NP-completeness

Do your best then.

2
 Coping With NP-Hardness
Brute-force algorithms.
 Develop clever enumeration strategies.
 Guaranteed to find optimal solution.
 No guarantees on running time.

Heuristics.
Develop intuitive algorithms.
Guaranteed to run in polynomial time.
No guarantees on quality of solution.

Approximation algorithms.
• Guaranteed to run in polynomial time.
• Guaranteed to find "high quality" solution, say within 1% of optimum.
Obstacle: need to prove a solution’s value is close to optimum,
without even knowing what optimum value is!
3
 Performance guarantees

An approximation algorithm is bounded by

ρ(n) if, for all input of size n, the cost c of
the solution obtained by the algorithm is
within a factor ρ(n) of the c* of an optimal
solution

4
 Different Approaches

 Special graph classes

e.g. vertex cover in bipartite graphs, perfect graphs.
 Fast exact algorithms, fixed parameter algorithms
find a vertex cover of size k efficiently for small k.
 Average case analysis
find an algorithm which works well on average.
 Approximation algorithms
find an algorithm which return solutions that are
guaranteed to be close to an optimal solution.

5
 Vertex Cover

Vertex cover: a subset of vertices which “covers” every edge.

An edge is covered if one of its endpoint is chosen.

The Minimum Vertex Cover Problem:

Find a vertex cover with minimum number of vertices.

6
 Approximation Algorithms

Key: provably close to optimal.

Let OPT be the value of an optimal solution,

and let SOL be the value of the solution that our algorithm returned.

Constant factor approximation algorithms:

SOL <= cOPT for some constant c.

7
 Vertex Cover: Greedy Algorithm 1

Idea: Keep finding a vertex which covers the maximum number of edges.

Greedy Algorithm 1:
1. Find a vertex v with maximum degree.
2. Add v to the solution and remove v and all its incident edges from the graph.
3. Repeat until all the edges are covered.

How good is this algorithm?

8
Vertex Cover: Greedy Algorithm 1

OPT = 6, all red vertices.

SOL = 11, if we are unlucky in breaking ties.

First we might choose all the green vertices.
Then we might choose all the blue vertices.
And then we might choose all the orange vertices. 9
 Vertex Cover: Greedy Algorithm 1
Not a constant factor
k! vertices of degree k
approximation algorithm!

Generalizing
the example!

k!/k vertices of degree k k!/(k-1) vertices of degree k-1 k! vertices of degree 1

OPT = k!, all top vertices.

SOL = k! (1/k + 1/(k-1) + 1/(k-2) + … + 1) ≈ k! log(k), all bottom vertices. 10

 Vertex Cover: Greedy Algorithm 1
Is the output from this greedy algorithm give a approximation of optimal solution?

Consider Gi : remaining graph after the choice

of ith vertex in the solution
di : maximum degree of any node in Gi-1
vi : vertex in Gi-1 with maximum degree
deg(v,Gi-1) : degree of v in graph Gi-1

Let C* denote the optimal vertex cover of G which contain m number of vertices
|Gi-1| denote the number of edges in the graph Gi-1.

Then

mi=1 d i
 mi=1 |G i-1
| /m  mi=1 |G | /m =|G | = |G| -mi=1 d
m m i
11
 Vertex Cover: Greedy Algorithm 1

In m th iterations, algorithm removes at least half the edges of G

Thus
after m.log |G| iterations
all the edges of G have been removed

Algorithm 1 computes a vertex cover of size O(optimum. log n)

Greedy Algorithm 1 is an O(log n)

approximation algorithm
12
 Vertex Cover: Algorithm 2

Greedy approach does not always lead to the best approximation algorithm

C=
while G has atleast one edge
(u,v) any edge of G
G = G \ {u, v}
C = C  {u, v}
return C

For edge (u, v), at least one of the vertex u or v must be in any
optimal cover

IT FOLLOWS IT IS A 2 APPROXIMATION ALGORITHM

13
 Traveling Salesman

Traveling salesman problem

asks for the shortest Hamiltonian cycle in a weighted undirected graph.

Traveling salesman problem is NP hard

14
 Traveling Salesman
Consider G be an arbitrary undirected graph with n vertices

Length function l(e) = { 1

2
if e is an edge in G
otherwise for Kn

G has a Hamiltonian cycle then

there is an Hamiltonian cycle in Kn whose length is exactly n

Traveling salesman problem is NP hard even if all the edge lengths are 1 or 2
Due to polynomial time reduction from Hamiltonian cycle to this type
of Traveling salesman problem

15
 Traveling Salesman
We can replace the values in length function by any values we like

Length function l(e) = { 1

n
if e is an edge in G
otherwise

G has a Hamiltonian cycle then

there is an Hamiltonian cycle in Kn whose length is exactly n or
has length at least 2n

Thus if we can approximate

the shortest traveling salesman tour within a factor of 2 in polynomial time
we would have a polynomial time algorithm for the Hamiltonian cycle problem
16
 Traveling Salesman

We have the following negative results

For any function f(n) that can be computed in polynomial in n,

there is no polynomial time f(n) approx fo TSP on general
weighted graph unless P=NP.

17
 Traveling Salesman : A Special Case

Edge lengths satisfy triangular inequality

l(u,v)  l(u,w) + l(w,v)

This is true for geometric graph

Compute minimum spanning tree T of the weighted input graph

Depth first traversal of T
Numbering the vertices in order that we first encounter them
Return the cycle obtained by visiting the vertices according to this numbering

18
 Traveling Salesman : A Special Case

Demonstration

Set of points distributed in 2D

19
 Traveling Salesman : A Special Case

Demonstration

Minimum spanning tree

20
 Traveling Salesman : A Special Case

Demonstration

Consider this as
root

Depth first traversal

21
 Traveling Salesman : A Special Case

Demonstration

7
6

1 5

2
3 4

Depth first traversal and

numbering of vertices
22
 Traveling Salesman : A Special Case

Demonstration

7
6

1 5

2
3 4

Traveling salesman tour

23
 Traveling Salesman : A Special Case

Demonstration

7
6

1 5

3
2
4

Traveling salesman tour

with cost 2.MST
24
 Traveling Salesman : A Special Case

Demonstration

7
6

1 5

3
2
4

Traveling salesman tour

with reduced cost  2.MST
25
 Traveling Salesman : A Special Case

Output quality :
Cost of the tour using this algorithm
 2* cost of minimum spanning tree
 2* cost of optimal solution

Conclusion: The algorithm outputs 2 approximation of the minimum

traveling salesman problem

26
Traveling Salesman : A Improved heuristic

Locate odd degree vertices in minimum spanning tree

1 5

2
3 4

Number of odd degree vertices is even

27
Traveling Salesman : A Improved heuristic

Locate odd degree vertices in minimum spanning tree

1
5

Perfect matching of odd degree vertices

28
Traveling Salesman : A Improved heuristic

Locate odd degree vertices in minimum spanning tree

1
5

2
3 4

Merging the perfect edges with MST

29
 Greedy Algorithm

Idea: Keep finding a set which is the most effective in covering remaining elements.

Greedy Algorithm:
1. Find a set S which is most cost-effective.
2. Add S to the solution and remove all the elements it covered from the ground set.
3. Repeat until all the elements are covered.

How good is this algorithm?

30
 Logarithmic Approximation

Theorem. The greedy algorithm is an O(log n) approximation for the set cover problem.

Theorem. Unless P=NP, there is no o(log n) approximation for set cover!

31
 Lower bound and Approximation Algorithm

For NP-complete problem, we can’t compute an optimal solution in polytime.

The key of designing a polytime approximation algorithm is

to obtain a good (lower or upper) bound on the optimal solution.

The general strategy (for a minimization problem) is:

lowerbound OPT SOL

SOL ≤ c · lowerbound  SOL ≤ c · OPT 32

 Linear Programming and Approximation Algorithm

LP lowerbound OPT SOL

Linear programming: a general method to compute a lowerbound in polytime.

To computer an approximate solution,

we need to return an (integral) solution
close to an optimal LP (fractional) solution.

33
 An Example: Vertex Cover

Optimal integer solution.

Integrality gap: = max

Optimal fractional solution.

Over all instances.

In vertex cover, there are instances where this gap is almost 2.

0.5 1

0.5 0.5 1 0

0.5 0.5 1 1 34
Linear Programming Relaxation for Vertex Cover

Theorem: For the vertex cover problem,

every vertex (or basic) solution of the LP
is half-integral, i.e. x(v) = {0, ½, 1}
35
Linear Programming Relaxation for Set Cover

for each element e.

for each subset S.

How to “round” the fractional solutions?

Idea: View the fractional values as probabilities, and do it randomly!

36
 Algorithm

First solve the linear program to obtain the fractional values x*.

Then flip a (biased) coin for each set with probability x*(S) being “head”.

0.3 0.6 0.2 0.7 0.4

sets

elements

Add all the “head” vertices to the set cover. Repeat log(n) rounds.
37
 Performance

Theorem: The randomized rounding gives an O(log(n))-approximation.

Claim 1: The sets picked in each round have an expected cost of at most LP.

Claim 2: Each element is covered with high probability after O(log(n)) rounds.

So, after O(log(n)) rounds, the expected total cost is at most O(log(n)) LP,
and every element is covered with high probability, and hence the theorem.

Remark: It is NP-hard to have a better than O(log(n))-approximation!

38
 Cost

Claim 1: The sets picked in each round have an expected cost of at most LP.

Q.E.D.
39
 Feasibility

Claim 2: Each element is covered with high probability after O(log(n)) rounds.

First consider the probability that an element e is covered after one round.

Let say e is covered by S1, …, Sk which have values x1, …, xk.

By the linear program, x1 + x2 + … + xk >= 1.

Pr[e is not covered in one round] = (1 – x1)(1 – x2)…(1 – xk).

This is maximized when x1=x2=…=xk=1/k, why?

40
Pr[e is not covered in one round] <= (1 – 1/k)k
 Feasibility

Claim 2: Each element is covered with high probability after O(log(n)) rounds.

First consider the probability that an element e is covered after one round.

Pr[e is not covered in one round] <= (1 – 1/k)k

So,

What about after O(log(n)) rounds?

41
 Feasibility

Claim 2: Each element is covered with high probability after O(log(n)) rounds.

So,

So,

42
 Remark

Let say the sets picked have an expected total cost of at most clog(n) LP.

Claim: The total cost is greater than 4clog(n) LP with probability at most ¼.

This follows from the Markov inequality, which says that:

Proof of Markov inequality:

The claim follows by substituting E[X]=clog(n)LP and t=4clog(n)LP 43

 Wrap Up

Theorem: The randomized rounding gives an O(log(n))-approximation.

This is the only known rounding method for set cover.

Randomized rounding has many other applications.

44