Advanced Algorithms

P, NP, and NP-Complete

1
Outline
Algorithm complixity
Problems types
P type
NP type

2
Complexity definitions
Big-Oh
Big-Theta
Big - Omega

3
Big Oh (O)

f(n)= O(g(n)) iff there exist positive constants c and n0

O-notation to give an upper bound on a function

4
Omega Notation

Big oh provides an asymptotic upper bound on a

5
Theta Notation

Theta notation is used when function f can be bounded

6
How bad is exponential
complexity
Fibonacci example – the recursive fib cannot even
compute fib(50)

7
Tractability
Some problems are intractable:
as they grow large, we are unable to solve them in
reasonable time
What constitutes reasonable time?
» Standard working definition: polynomial time
» On an input of size n the worst-case running time is O(nk)
for some constant k
» O(n2), O(n3), O(1), O(n lg n), O(2n), O(nn), O(n!)
» Polynomial time: O(n2), O(n3), O(1), O(n lg n)
» Not in polynomial time: O(2n), O(nn), O(n!)

8
Polynomial-Time Algorithms
Are some problems solvable in polynomial time?
» Of course: many algorithms we’ve studied provide
polynomial-time solutions to some problems
Are all problems solvable in polynomial time?
» No: Turing’s “Halting Problem” is not solvable by any
computer, no matter how much time is given
Most problems that do not yield polynomial-time
algorithms are either optimization or decision
problems.

9
Optimization/Decision
Problems
Optimization Problems
» An optimization problem is one which asks, “What is
the optimal solution to problem X?”
» Examples:
0-1 Knapsack
Fractional Knapsack
Minimum Spanning Tree
Decision Problems
» A decision problem is one with yes/no answer
» Examples:
Does a graph G have a MST of weight  W?

10
Optimization/Decision
Problems
An optimization problem tries to find an optimal solution
A decision problem tries to answer a yes/no question
Many problems will have decision and optimization
versions
» Eg: Traveling salesman problem
optimization: find hamiltonian cycle of minimum weight
decision: is there a hamiltonian cycle of weight  k
Some problems are decidable, but intractable:
as they grow large, we are unable to solve them in
reasonable time
» Is there a polynomial-time algorithm that solves the problem?

11
The Class P
P: the class of decision problems that have polynomial-time
deterministic algorithms.
» That is, they are solvable in O(p(n)), where p(n) is a polynomial on n
» A deterministic algorithm is (essentially) one that always computes the
correct answer

Why polynomial?
» if not, very inefficient
» nice closure properties
the sum and composition of two polynomials are always polynomials too

12
Sample Problems in P
Fractional Knapsack
MST
Sorting
Others?

13
The class NP
NP: the class of decision problems that are solvable in polynomial
time on a nondeterministic machine (or with a nondeterministic
algorithm)
(A determinstic computer is what we know)
A nondeterministic computer is one that can “guess” the right
answer or solution
» Think of a nondeterministic computer as a parallel machine that can
freely spawn an infinite number of processes
Thus NP can also be thought of as the class of problems
» whose solutions can be verified in polynomial time
Note that NP stands for “Nondeterministic Polynomial-time”

14
Sample Problems in NP
Fractional Knapsack
MST
Sorting
Others?
» Hamiltonian Cycle (Traveling Salesman)
» Graph Coloring
» Satisfiability (SAT)
the problem of deciding whether a given
Boolean formula is satisfiable

15
The Satisfiability (SAT)
Problem

16
Example: CNF satisfiability

17
Review: P And NP Summary

18
NP-complete problems

19
Reduction
A problem R can be reduced to another problem Q if any
instance of R can be rephrased to an instance of Q, the
solution to which provides a solution to the instance of R
» This rephrasing is called a transformation
Intuitively: If R reduces in polynomial time to Q, R is
“no harder to solve” than Q
Example: lcm(m, n) = m * n / gcd(m, n),
lcm(m,n) problem is reduced to gcd(m, n) problem

20
NP-Hard and NP-Complete

21
Practical Implication
Given a problem that is known to be NP-Complete
» Try to solve it by designing a polynomial-time algorithm?
Prove P=NP

» Alleviate the intractability of such problems

22