Complexity Theory

Polynomial and Non-polynomial problems :

• We can distinguish problems between two distinct
classes. Problems which can be solved by a
polynomial time algorithm and problems for
which no polynomial time algorithm is known.

• An algorithm for a given problem is said to be a

polynomial time algorithm if its worst case time
complexity is O(nk), where k is a fixed integer
and n is size of a problem. For example,
Sequential search : O(n), Binary search : O(logn),
Insertion sort : O(n2), product of two matrices :
O(n3), Quick sort : O(nlogn) etc.
 Unit IV : Complexity Theory
Tractable vs Intractable Problems : OR
Polynomial and Non-polynomial problems contd… :
• The set of all problems that can be solved in polynomial
amount of time are called “Tractable or Polynomial
problems”. These problems can be solved in a
reasonable amount of time for even very large amount of
input data. Their worst case time complexity is O(nk).
• The set of all problems that can not be solved in
polynomial amount of time are called “Intractable or
Non-polynomial Problems”. Their worst case time
complexity is O(kn). These problems require huge
amount of time for even modest input sizes. For example,
0-1 Knapsack problem : O(2n), Traveling Salesperson
Problem : O(n22n)
Tractable vs Intractable OR polynomial vs non-polynomial
Growth rates of few functions w.r.t. the input size „n‟ :
Class of n→ 10 20 30 40
Problems f(n) ↓
1 1 1 1 1
Log(n) 3.3 4.3 4.9 5.3
Tractable n 10 20 30 40
or
Polynomial nlog(n) 33 86 147 212
Problems
n2 100 400 900 1600
n3 1000 8000 27000 64000
Intractable
or non-poly 2n 1024 1 Million 1 Billion 1 Trillion
Problems (1Kilo)
Deterministic vs Non-deterministic Algorithms :
• Deterministic Machines : Conventional Digital
machines are Deterministic in nature.
Serialization of resource access or sequential
execution is the basic concept used in these
machines (Von Neumann Architecture).

• Non-deterministic Machines : These are

hypothetical machines which can do the jobs in
parallel fashion i.e more than one jobs can be
done in one unit of time.
Deterministic vs Non-deterministic Algorithms contd … :
• Deterministic Algorithms : Algorithms in which the
result of any operation is uniquely defined are termed as
Deterministic Algorithms. All algorithms studied so far
are deterministic algorithms. Such algorithms agree with
the way programs are executed on a digital computer i.e.
a deterministic machine.

• Non-deterministic Algorithms : If we remove the

restriction on the outcome of every operation, then
outcomes are not uniquely defined but they are limited to
specified sets of possibilities. There is a termination
condition in such algorithms. Such algorithms are called
as non-deterministic algorithms. Examples
Deterministic vs Non-deterministic Algorithms contd … :
Example-1 : Non-deterministic Linear Search Algorithm:
• Using Deterministic algorithm it takes O(n) time
• Using Non-deterministic algorithm it takes O(1) time.

Algorithm non-det-lin-search
1. j = choice (1:n)
2. If A[ j ] = x
{ write ( j ); success( );}
3. write (0); failure( );
Deterministic vs Non-deterministic Algorithms contd … :
Example-2 : Non-deterministic Sorting Algorithm :
• Using Deterministic algorithm it takes O(nlogn) time
• Using Non-deterministic algorithm it takes O(n) time.
Algorithm Non-det-sort (A, n)
// sorts n positive integers. B[ ] is an auxiliary array.
{ for i = 1 to n do // Initialize aux. array B
B[i] = 0;
for i = 1 to n do // Non-det step decides the rank of the
{ j = choice (1, n); // input number in unit time
if B[ j] ≠ 0 then failure ( );
B [ j] = A[i];
}
for i = 1 to n-1 do // verification
if B[i] > B[i+1] then failure( );
write (B[1:n]);
success ( );
}
Decision Problem & Optimization Problem / Algorithm :

• Decision Problem : Any problem for which answer is

either 0 or 1 is called a decision problem and the
corresponding algorithm is referred as a decision
algorithm. For example : To search a given number
sequentially, Binary search,

• Optimization Problem : Any problem that involves the

identification of an optimal (either min. or max.) value of
a given cost function is known as a optimization problem
and the corresponding algorithm is referred as an
optimization algorithm. For example, Knapsack problem,
Minimum cost spanning tree, TSP, Shortest path etc
P vs NP class problems :

• P class : The class of decision problems that can be

solved in polynomial time using deterministic algorithms
is called the P class or Deterministic Polynomial
problems.
• NP class : The class of decision problems that can be
solved in polynomial time using non-deterministic
algorithms is called the NP class or Non-deterministic
Polynomial problems.
• Any P class problem can be solved using NP class
algorithm. Therefore P is contained in NP class
• Whether NP is contained in P is unknown. Examples
P vs NP class problems contd …:
Examples :

Example P time complexity NP time complexity

Linear Search O(n) O(1)

Sorting O(nlogn) O(n)

Searching an element O(n) O(logn)

in Binary Tree ( by traversing tree) (level by level search)
NP-Complete problems :
• A decision problem D is said to be NP-Complete iff
1. It belongs to NP class
2. Every problem in NP class is polynomially
reducible to D
• If one instance of such problem can be solved using a
polynomial algorithm, the complete class of problems
can be solved using a polynomial algorithm
• Examples : Traveling Salesperson Problem : optimal tour,
Printed circuit board problem,
Bin packing problem,
0-1 knapsack problem,
Vertex (node) cover problem
NP-Hard Problems :

Traveling Salesperson Decision Problem (TSP) :

Let G = (V, E) be a directed graph with edge costs cij for the
edge <i, j> Є E. If there is no edge between i and j then
cij = ∞ . Let |V| = n and assume n > 1. A tour of G is a
directed simple cycle that includes every vertex in V. The
cost of the tour is the sum of the cost of edges on the tour.
TSP is to find a tour of minimum cost.

TSP Decision problem is to determine whether a directed graph

G = (V,E) with edge cost c(u, v) has a tour of cost at most M
 NP-Hard Problems
Traveling Salesperson Problem (TSP) :
Example : For the following directed graph represented as cost adjacency
matrix (length of edges) find the optimal tour with shortest path length.
1 2 3 4
1 | 0 10 15 20 |
2 | 5 0 9 10 |
3 | 6 13 0 12 |
4 | 8 8 9 0 |

5 10
1 2
8 6 8 13

Graph G = (V,E)
V = {1, 2, 3, 4}
20 10 15 9
4 3
12 9
NP-Hard problems :
• A problem L is said to be NP Hard problem if and only if
satisfiability α L

• NP hard problems are basically the optimization versions

of the problems in NP complete class

• NP hard problems are not mere yes / no problems. They

are problems wherein we need to find the optimal
solution

• A problem L is NP complete if and only if L is NP hard

and L Є NP
NP-Hard problems contd … :

• Commonly believed relationship among P, NP,

NP- complete and NP-hard
Diagram

• An example of NP-hard problem not in NP-complete :

Halting Problem : To determine for an arbitrary

deterministic algorithm A and an input I whether
algorithm A with input I ever terminates or enters an
infinite loop. This problem is undecidable. No algorithm
exists to solve this problem.
Time Complexity of a non-deterministic algorithm:

• The time required by a non-deterministic algorithm on any

given input is the minimum number of steps needed to reach
a successful completion if there exists a sequence of choices
leading to such a completion. In case successful completion
is not possible, then the time required is O(1).
• A non-deterministic algorithm is of complexity O(f(n)) if
for all inputs of size n, n>=n0, that results in a successful
completion, the time required is at most c.f(n) for some
constant c and n0.
Example : 0/1 Knapsack Decision Problem
Time Complexity of a non-deterministic algorithm:

Algorithm DKP(p,w,n,m,r,x)
{ W = 0; P = 0;
for i = 1 to n do
{ x[i] = choice (0, 1)
W = W + w[i]*x[i]; P = P + p[i] *x[i];
}
if ((W > m) or (P < r) then failure
else success();
}
Time Complexity of DKP = O(n).
Satisfiability :
• Literal : A literal is either a Boolean variable or its negation, Ex.
a,⌐ a
• Clause : A clause is a literal or a disjunction of literals or a
conjunction of literals. Ex. (p V ⌐q V r), (p ٨ q ٨ r)
• CNF : Boolean formula is in Conjunctive Normal Form (CNF)
iff it is represented as ٨Ci , where Ci are clauses each represented
as V lij
1<= i<= k and 1<= j<= n

For example : (x3 V ⌐ x4) ٨(x1V ⌐ x2)

• DNF : Boolean formula is in Disjunctive Normal Form (DNF)
iff it is represented as VCi , where Ci are clauses each
represented as ٨ lij
1<= i<= k and 1<= j<= n

For example : (x1 ٨ ⌐ x2)V(x3 ٨ ⌐ x4)

Satisfiability contd… :
• The Satisfiability Problem is to determine whether a Boolean
formula is true for some assignments of truth values to the
variables.
• CNF-Satisfiability is the Satisfiability Problem for CNF formula.
Time Complexity (Deterministic algorithm ) for Satisfiability
and CNF-Satisfiability is O(2n), where „n‟ is total number of
Boolean variables in the formula. So it is NP-Hard … (1)
• It is easy to obtain a polynomial time non-deterministic
algorithm that terminates successfully if and only if a given
propositional formula E(x1,x2, …,xn) is satisfiable. Such an
algorithm could proceed by simply choosing (non-
deterministically) one of the (2n) assignments of truth values to
(x1,x2, …,xn) and verifying that E(x1,x2, …,xn) is true for that
assignment.
Non-deterministic algorithm to solve Satisfiability problem
Algorithm Eval(E,n)
// Determine whether the propositional formula E is satisfiable
{ for i = 1 to n do
xi = choice(false, true);
if E(x1,x2, …,xn) then success()
else failure();
}
Analysis : Time Complexity = O(n) i.e. polynomial time. Therefore
“Satisfiability is in NP” ……….. (2)
• From (1) and (2) above Satisfiability is NP-Complete.
Cook‟s Theorem : “Satisfiability is in P
NP-Hard Problems : Reduction of problem
The strategy we adopt to show that a problem L2 is NP-hard is :
1. Pick a problem L1 already known to be NP-hard
2. Show how to obtain (in polynomial deterministic time) an
instance I‟ of L2 from any instance I of L1 such that from the
solution of I‟ we can determine (in polynomial deterministic
time) the solution to instance I of L1.
3. Conclude from step ( 2 ) that L1α L2.
4. Conclude from step (1) and (3) and the transitivity of α that L2 is
NP-hard.

All NP-hard decision problems that we shall discuss here are NP-
Complete and this can be shown by exhibiting a polynomial time
non-deterministic algorithm for L2
NP-Hard Problems : Reduction of problem
Reduction of L1 to L2 :

I of L1 I‟ Solution for I
Poly. Time Algorithm Poly. Time
transform for L2 transform

L1α L2 or L1 ≤ p L2
NP-Complete Problems : Proof by Reduction
Lemma : If L2 is a problem (language) such that L1 α L2 for
some L1 Є NP-Complete (or NPC) then L2 is
NP-Hard. Further if L2 Є NP then L2 Є NPC
The structure of NP-completeness proofs:
CIRCUIT-SAT
First NP-Complete Problem
SAT

CNF-SAT

CLIQUE SUBSET-SUM

VERTEX-COVER

HAM-CYCLE

TSP
NP-Hard Problems :
To prove : 1) CIRCUIT-SAT α Satisfiability or
2) Satisfiability is NP-complete
Proof :
Consider the following electronic digital circuit.
x1 OR x5
x2
OR
x6
NOT x6 x8
OR x9 x10
AND
X3 x4 x7
NOT AND
x4 x7
NP-Hard Problems :
To prove : 1) CIRCUIT-SAT α Satisfiability or
2) Satisfiability is NP-complete
Proof contd… :
Reducing circuit satisfiability to formula satisfiability:

F = x10 ٨ (x4 ↔ ¬ x3))

٨ (x5 ↔ (x1 V x2))
٨ ( x6 ↔ ¬ x4 )
٨ ( x7 ↔ x1 ٨ x2 ٨ x4)
٨ ( x8 ↔ (x5V x6))
٨ ( x9 ↔ (x6 V x7))
٨ ( x10 ↔ (x8 ٨ x9 ٨ x7))

Therefore CIRCUIT-SAT α Satisfiability.

NP-Hard Problems :
To prove : 1) CIRCUIT-SAT α Satisfiability or
2) Satisfiability is NP-complete
Proof contd… :
In general Boolean formula will have n variable (literals) and to
find legal values of n variables for Ф to be true, needs to evaluate
the formula for every permutation of n values which is = O(2n),
Thus it will take super-polynomial time to find all the permutations
for which the formula is true and there is no polynomial time
algorithm exists.

We know that CIRCUIT-SAT is NP-Hard and we have reduced

CIRCUIT-SAT to Satisfiability i.e. CIRCUIT-SAT α Satisfiability .
Therefore Satisfiability is NP-Hard. We know that Satisfiability is
in NP. Hence Satisfiability is NP-Complete as well.
NP-Hard Problems :
To prove : 1) Satisfiability α 3-CNF-SAT or
2) 3-CNF Satisfiability is NP-complete
Proof (1) : We divide the proof in three parts. Let Boolean formula,
Ф = ((x1 → x2) V ¬((¬x1 ↔ x3) V x4)) ٨ ¬x2.
a) First step : Construct a binary “parse” tree for the formula Ф.
y1
٨
y2
V
y3 y4 ¬
¬ x2
→

x1 x2 V y5 Introduce a variable (y) for

y6 the output of each internal node
↔ x4
¬ x1 x3
NP-Hard Problems :
• Rewrite the original formula Ф as the AND of root variable and a
conjunction of clauses describing the operation of each node.
Ф‟ = y1 ٨ ( y1 ↔ (y2 ٨ ¬ x2))
٨ ( y2 ↔ (y3 V y4))
٨ ( y3 ↔ (x1 → x2))
٨ ( y4 ↔ ¬ y5)
٨ ( y5 ↔ (y6 V x4))
٨ ( y6 ↔ (¬ x1 ↔ x3))
• The formula Ф‟ thus obtained is a conjunction of clauses Ф‟i , each of
which has at most 3 literals
b) Second step : Converts each clause Ф‟i into CNF. For this we
construct a truth table for each Ф‟i by evaluating all possible
assignments to its variables. Each row of the truth table consists of a
possible assignments of the variables of the clause, together with the
value of clause under the assignment. For example,
NP-Hard Problems :
• Consider the formula ( y1 ↔ (y2 ٨ ¬ x2))
y1 y2 x2 ( y1 ↔ (y2 ٨ ¬ x2))
-------------------------------------------------
1 1 1 0
1 1 0 1
1 0 1 0
1 0 0 0
0 1 1 1
0 1 0 0
0 0 1 1
0 0 0 1
• For the truth table entries that evaluate to 0 (4 entries in above
table), we build a formula in DNF – an OR of ANDs that is
equivalent to ¬ Ф‟
NP-Hard Problems :
• ¬ Ф‟1 = [(y1 ٨ y2٨ x2) V (y1 ٨ ¬ y2٨ x2) V (y1 ٨ ¬ y2٨ ¬ x2) V
(¬ y1 ٨ y2٨ ¬ x2)]
• Negate ¬ Ф‟1 and apply DeMorgans‟s laws to get Ф‟‟1
DeMorgan‟s Laws : ¬ (a ٨ b) = ¬a V ¬b and
¬ (a V b) = ¬a ٨ ¬b
Ф‟‟1 = (¬ y1 V ¬ y2V ¬x2) ٨ (¬ y1 V y2 V ¬ x2) ٨
(¬ y1 V y2 V x2) ٨ (y1 ٨ ¬y2 ٨ x2) = original Ф‟1
• At this point we have converted each clause Ф‟i of the formula Ф‟ into
a CNF formula Ф‟‟ and thus Ф‟ is equivalent to the CNF formula Ф‟‟
consisting of the conjunction of the Ф‟‟i . Moreover each clause of Ф‟
has at most 3 literals.
c) Third Step : This final step of reduction further transforms the
formula so that each clause has exactly 3 literals. We construct the
final 3-CNF formula Ф‟‟‟ from the clauses of the CNF formula Ф‟‟ .
NP-Hard Problems :
• The formula Ф‟‟‟ also uses two auxiliary variables that we shall call
p and q. For each clause Ci of Ф‟‟ , we include the following clauses
in Ф‟‟‟ :
if Ci has 3 distinct literals, then include Ci as a clause of Ф‟‟‟.
if Ci has 2 distinct literals, then replace
Ci = ( l1 V l2) = ( l1 V l2 V p) ٨ ( l1 V l2 V ¬ p)
 if Ci has 1distinct literals, then replace
Ci = ( l1 ) = (l1 V p V q) ٨ (l1 V p V ¬ q) ٨
(l1 V ¬ p V q) ٨ (l1 V ¬ p V ¬ q)
• Thus 3-CNF formula Ф‟‟‟ is satisfiable if and only if Ф is satisfiable.
• Reduction is computable in polynomial time. Step 1 introduces one
variable in each clause of Ф. Step 2 introduces at most 8 clauses for
each clause of Ф‟ (truth table for each clause has at most 8 rows).
Step 3 introduces at most 4 clauses for each clause of Ф‟‟. Thus the
size of resulting formula Ф‟‟‟ is polynomial in the length of the
original formula.
NP-Hard Problems :
• Thus the instance, I of satisfiability is converted into an
instance I‟ in 3-CNF form in polynomial time.
• Therefore we can say that
Satisfiability α 3-CNF Satisfiability.
Therefore 3-CNF Satisfiability is NP-Hard

Proof (2) :
• But we also know that Satisfiability is NP-Hard, therefore
we can say that 3-CNF Satisfiability is also NP-Hard.
• We can verify in polynomial time the certificate of 3-CNF
satisfiability. Hence 3-CNF Satisfiability is NP as well
• Hence 3-CNF Satisfiability is NP-Complete.
NP-Hard Problems :
Node Cover Decision Problem (NCDP):
A set S, a subset of V, is a node cover for graph G = (V, E) if
and only if all edges in E are incident to at least one vertex
in S. The size |S| of the cover is the number of vertices in S.
Example:
1 2

5 4

S = {2, 4} is a node cover of size 2.

S = {1, 3, 5} is a node cover of size 3.
In the node cover decision problem we are given a graph G and
an integer k. We are required to determine whether G has a node
cover of size at most k (i.e. ≤ k)
 Unit IV : Complexity Theory
NP-Hard Problems :
Node Cover Decision Problem (NCDP):
Theorem : The Clique Decision Problem α the Node Cover Decision
Problem or the vertex cover problem is NP-Complete.
Proof :
• Let G = (V, E) and k define an instance of CDP. Assume that
|V| = n.
• We construct a graph G‟ such that G‟ has a node cover of size at
most n – k if and only if G has a clique of size at least k. Graph
G‟ = (V, E‟) where E‟ = {(u, v) | u, v Є V and (u, v) does not Є
E}. The set G‟ is known as complement of G.
• Now let us show that G has a clique of size at least k if and only
if G‟ has a node cover of size at most n – k.
• Let K be any clique in G. Since there are no edges in E‟
connecting vertices in K, the remaining n - |k| vertices in G‟ must
cover all edges in E‟.
 Unit IV : Complexity Theory
NP-Hard Problems :
Node Cover Decision Problem (NCDP):
Proof contd. :
• Similarly if S is a node cover of G‟, then V – S must form a
complete sub-graph in G.
• Since G‟ can be obtained from G in polynomial time, CDP can be
solved in polynomial deterministic time if we have a polynomial
time deterministic algorithm for NCDP.
• Hence the Clique Decision Problem α the Node Cover Decision
problem.
We know that CNF-satisfiability α CDP, CDP α NCDP and α is
transitive, it follows that CNF-satisfiability α NCDP. So NCDP is
NP-hard. It is also in NP because we can non-deterministically choose
a subset C of V of size k and verify in polynomial time that C is a
cover of G. So NCDP is NP-complete.
Example / Diagram
 Unit IV : Complexity Theory
NP-Hard Problems :
Node Cover Decision Problem (NCDP):
Example : Reducing CLIQUE to VERTEX-COVER
a) An undirected graph G = (V, E) with clique V‟ = {u, v, x, y}

u v

w
z

y x
 Unit IV : Complexity Theory
NP-Hard Problems :
Node Cover Decision Problem (NCDP):
Example : Reducing CLIQUE to VERTEX-COVER
b) An undirected graph G‟ = (V, E‟) with node cover V-V‟ = {w, z}

u v

w
z

y x