Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems

1

Linear Programming and its Usage in
Approximation Algorithms for NP Hard
Optimization Problems
M. Reza Rahimi,
November 2005.

2

Outline

• Introduction
• Linear Programming Overview
• NP Complexity Class
• Approximation Algorithms
• Case Study1: Minimum Weight Vertex Cover
• Case Study2: MAXSAT Problem
• Conclusion

Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems

3

Introduction
• One of the most challenging problem in complexity
theory is problem P vs. NP.
NP
• Research on this open problem leads researchers to
think of new methods and different approaches.
• For example PCP, IP, approximation algorithms,...
• For some problems it is proved that there does not
exist any suitable approximation unless P=NP.


4

• So it seems that research on approximation
algorithms may lead us to good results about P vs.
NP.
• General Method for Approximation Algorithms of
NP Hard Optimization is Greedy Method.
• But we must search for Suitable Framework for
studying NP problems.


5

• Linear Programming was this brake through
method.
• In this talk I focus on LP and its usage in NP
Hard Optimization Problems.


6

Linear Programming Overview
• General formulation of linear programming is
presented as follow:

Maximize C1X1 + C2X2 + … + CdXd
Subject to A1,1X1 + … + A1,dXd ≤ b1
A2,1X1 + … + A2,dXd ≤ b2
…
An,1X1 + … + An,dXd ≤ bn

All calculation and Numbers are on Real Numbers.


7

• There are several Polynomial Algorithms for this
problem such as Ellipsoid and Interior point
method.
method
• I neglect talking about them.
• For these algorithms please refer to optimization
references.


8

NP Complexity Class

• NP Complexity Class is only related to Decision
Problem.
• For example:

SAT = {< Φ >: Φ is a satisfying assignment}.

• So for any optimization problem we consider its
related decision problem.
• This will give us intuition about its hardness.


9

• We know the following definition about NP:

L ∈ NP ⇔ ∃V(.,.) ∈ P, ∃P(.), ∀x ∈ Σ∗ ,
1. x ∈ L ⇒ ∃y, y ≤ P( x ) and V(x, y) accepts.
2. x ∉ L ⇒ ∀y, y ≤ P( x ) and V(x, y) rejects.

• We can look at this process like this:

Y
Find Certificate Y y V(X,Y)

X x
X
x

10

• Example:

SAT = {< Φ >: Φ is a satisfying assignment}.
SAT ∈ NP Because :
1) if Φ 0 ∈ SAT ⇒ We are given True assignment
, and could check it in polynomial time.
2)if Φ 0 ∉ SAT ⇒ Then there is no true assignment.

• There is one another important concept in
complexity which is Reduction.


11

Definition :
L ≤ P L* ⇔ ∃f (Polynomial Time Function) ∀x ∈ L ⇔ f ( x) ∈ L* .

f
L L*
f
L L
L*

Fig1: Graphical Diagram of Reduction Concept


12

Example :
SAT = {< Φ >| Φ is satisfiable assignment}.
3 - CNF = {< Φ >| Φ is 3 - cnf and satisfiable}.
we show that :
SAT ≤ P 3 − CNF .
(3 − CNF is in this form for example :
(x1 ∨ x 2 ∨ x 3 ) ∧ ( x1 ∨ x 2 ∨ x 3 ) ∧ (x1 ∨ x 2 ∨ x 3 )


13

Φ ≡ (¬x1 → x2 ) ↔ ( x3 ∧ x2 )

<->
y1 y2

-> ^

-x1 x2 x3 x4

Φ ≡ ( y1 ↔ (¬x1 → x2 )) ∧ (( x3 ∧ x4 ) ↔ y2 ) ∧ ( y1 ↔ y2 )
Now each paranthesis can be converted into 3 - OR form
using truth table.


14

Example :
INTEGER − PROGRAMMING = {< Am×n , bm×1 >| Am×n and bm×1 are integer matrices,
∃X n×1 ∈ Ζ n , Am×n X n×1 ≤ bm×1}.
Obviously INTEGER − PROGRAMMING ∈ NP.
We will show that :
3 − CNF ≤ P INTEGER − PROGRAMMING.
Proof :
¬X 1 ∨ X 2 ∨ X 3 ↔ (1 − X 1 ) + X 2 + X 3
¬X 1 ∨ X 2 ∨ X 3 ≡ 1 ↔ (1 − X 1 ) + X 2 + X 3 ≥ 1
X i ∈ Ζ, 0 ≤ X i ≤ 1


15

• We conclude this section by the following theorem:
Every language in NP is polynomial time
reducible to SAT language.
• Language like SAT, INTEGER-PROGRAMMING,
Hamiltonian Cycle, and 3-CNF are also the same
and they belong to NP-Complete set.


16

Approximation Algorithms

• The following describes the connection of NP
optimization Complete problem and its related
optimization problem.

Decision Version :
TSP = {< Gn×n , b >| G n×n is complete non negative weighted graph and there
exists Hamiltonian Cycle such that its cost is less than or
equal b}.
Optimization Version :
OP − TSP :
we have complete non negative weighted graph G n×n , find the Hamiltonian Cycle
which its cost is minimun.


17

• Bound of Approximation:
• Suppose that problem P has an optimum solution with cost C-
op. Algorithm X solves problem P
with bound ρ(n) if it finds feasible solution with cost C such
as:
max {c/c-op, c-op/c} ≤ ρ(n).
• Note that with this definition we always have ρ(n) ≥1.
• In the following I try to show that sometimes, finding good
approximation for optimization problems is very hard.


18

• Theorem:
It does not exist any polynomial time approximation
algorithm with Polynomial Bound for OPT-TSP unless
P=NP.
• Proof:
we prove that if such an algorithm exists then we can Solve
Hamiltonian Cycle in Polynomial Time.
For each graph G with n vertex convert
It into weighted complete graph G1 as G has Hamiltonian Cycle
the following procedure: Iff
4) Assign 1 to each of its edge. OPT-TSP(G1)=n
5) Assign nρ(n) to the other edges.


19

• I think that it’s now time to study Randomized
Rounding technique for solving optimization problems.
• I explore it with two examples.


20

Case Study1: Minimum Weight Vertex
Cover
• Now we study randomized rounding method in
approximation algorithms with an example.
• Definition:
• Vertex Cover: In undirected graph G=(V,E), Set A of
vertices is said vertex cover if
for every (u,v) ε E then u ε A or v ε A .
• Minimum Weight Vertex Cover: In undirected graph
G=(V,E) which each vertex has w(v) as positive weight Set
A of vertices is said minimum weight vertex cover if for
every (u,v) ε E then u ε A or v ε A and w(A) is the least.


21

Fig2: Vertex Cover


22

• It is proved that decision version of this
optimization problem is NP Hard.
• At the first step we model it as integer
programming.

x : V → {0,1}
if v ∈ Minimum Weight Vertex Cover Set ⇒ x(v) = 1.
else x(v) = 0.
Integer Programming Method :
min ∑ w(v)x(v)
v∈V
∀u , v ∈ V x(v) + x (u ) ≥ 1
0 ≤ x (v ) ≤ 1


23

• As it was known it is believed that there is no
polynomial time algorithm for integer programming.
• We must think about another method.
• We investigate its related linear programming.

x : V → [0,1]
Linear Programming Method :
min ∑ w(v)x(v)
v∈V
∀u , v ∈ V x(v) + x(u ) ≥ 1
0 ≤ x (v ) ≤ 1


24

• The Last method is called Relaxation to linear
programming.
programming
• It’s now time to state the approximation
algorithm.
Approximation Min Weight Vertex Cover(G, w)
1) C ← Φ
2) Compute x an optimal solution to linear programming.
3) for each v ∈ V
4) do if x (v) ≥ 0.5 / * Rounding Method * /
5) then C ← C ∪ {v}
6) Return C.


25

Proof :
C :: The soulution of algorithm.
C* :: The real optimal set of integer programming.
Z* :: The optimal value of the linear programming.
Obviously we have Z* ≤ w(C* ).
Obviously C is vertex cover.
Z* = ∑ w(v) x(v) ≥ ∑ w(v) x(v)
v∈V v∈V , x ( v ) ≥ 0.5

≥ ∑ w(v)0.5 =∑ w(v)0.5 = 0.5w(C ).
v∈C
v∈V , x ( v ) ≥ 0.5

0.5w(C) ≤ Z* ≤ w(C* ) ⇒ { ρ = 2}


26

• Hastad proved that there is no Polynomial time algorithm for
vertex cover that achieves an approximation better that 1.16
unless P=NP (1997).
• Now we consider another example.


27

Case Study2: MAXSAT Problem

MAXSAT :
Given K - CNF Formula with each Clause C j has weight Wj ≥ 0.
Find assignment to variables that maximizes :
∑w
C j is true
j

Example :
Φ = (x1 + x 2 + x 3 + x 4 )( x1 + x 2 + x 3 + x 4 )(x1 + x 2 + x 3 + x 4 ).
C1 = (x1 + x 2 + x 3 + x 4 ) → w 1
C 2 = ( x1 + x 2 + x 3 + x 4 ) → w 2
C3 = (x1 + x 2 + x 3 + x 4 ) → w 3
Max ∑w
C j is true
j


28

• We state the following randomized algorithm for this
problem. (Johnson 1974).

1) for variables x1 ,...x n randomly assign
to 0 or 1 with probability 0.5.
2) return ∑w
C J istrue
j

Claim :
1 −1
The preceding algorithm has [(1- k
)] approximation bound.
2


29

Proof :
Define the following random varibles :
I j = 1 if clause C j satisfiable if not 0.
∑ w =∑ w I
C j is true
j
Cj
j j

1
p(I j = 1 ) = (1 - ).
2k
1
p(I j = 0 ) = ( ).
2k
1
E{ ∑ w j} =E{∑ w jI j} =∑ w jE{I j} =(1-
C j is true Cj Cj
)∑ w j .
2k C j
1 −1
ρ = [(1 - k
)]
2


30

• We can extend Johnson algorithm as follow:

1) for variables x1 ,...x n randomly and independently assign
to 1 or 0 with probability p i and 1 - p i
2) return ∑w
C J istrue
j

Claim :
E{ ∑ w } = ∑ w (1 − ∏ (1 − p )∏ p )
C j is true
j
Cj
j
if xi
i i
if xi

The proof is just the same.


31

• Now we model it with IP and relax it.
Integer Programming Model for MAXSAT :
Max ∑ w jz j
Relaxation
Cj

∀C j ∑ y + ∑ (1- y ) ≥ z
if x i
i i j
if x i

0 ≤ yi ≤ 1
0 ≤ zj ≤1 yi , z j ∈ Ζ

Relaxation for MAXSAT :
Max ∑ w jz j
Cj

∀C j ∑ y + ∑ (1- y ) ≥ z
if x i
i
if x i
i j

0 ≤ yi ≤ 1
0 ≤ zj ≤1 yi , z j ∈ ℜ


32

Algorithm
1) Solve the LP and find vector (y* , z * ).
2) use extended Johnson algorithm with p i = y*i .
3) return
Lemma :
for any feasible solution (y* , z* ) to LP and for any clause C j with k Literals,
we have
1
1 - ∏ (1- y i )∏ (y i ) ≥ (1 − (1 − ) k ) z j
if x i if x i k
Claim :
The above algorithm has (1- 1/e) -1 approximation ratio.


33

Proof :
∑ w {1 − ∏ (1 − p )∏ ( p )} =
Cj
j
xi
i i
xi

1
∑ w j {1 − ∏ (1 − yi )∏ ( yi )} ≥ (1 − (1 − ) k )∑ w j z j =
* * *

Cj xi xi k Cj
1 k * 1 k *
(1 − (1 − ) ) Z LP ≥ (1 − (1 − ) ) Z IP
k k
[ρ = (1 − 1 / e) −1 ]


34


35

• Algorithm
2. Compute the value from Johnson
algorithm (w1).
3. Compute the value from LP Based
algorithm (w2).
4. Return MAX(w1,w2).


36

Conclusion
• In this talk the randomized rounding method is explored
with two example.
• This new method opens new insight into approximation
algorithms and complexity theory.

The End


Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems

Recommended

More Related Content

What's hot (20)

Similar to Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems (20)

More from Reza Rahimi (20)

Recently uploaded (20)

Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems