Omputer Cience: 16. Intractability
Omputer Cience: 16. Intractability
S E D G E W I C K / W A Y N E
PA R T I I : A L G O R I T H M S , M A C H I N E S , a n d T H E O R Y
Computer Science
ROBERT SEDGEWICK
K E V I N WAY N E
Section 7.4
http://introcs.cs.princeton.edu
COMPUTER SCIENCE
S E D G E W I C K / W A Y N E
PA R T I I : A L G O R I T H M S , M A C H I N E S , a n d T H E O R Y
16. Intractability
• Reasonable questions
• P and NP
• Poly-time reductions
• NP-completeness
• Living with intractability
CS.16.A.Intractability.Questions
Context
Fundamental questions
• What is a general-purpose computer? ✓
• Are there limits on what we can do with digital computers? ✓
• Are there limits on what we can do with the machines we can build? focus of today's lecture
Kurt Gödel
1909−1994 John Nash Michael Rabin Dana Scott Dick Karp Steve Cook Leonid Levin
Asked the question Asked the question Introduced the critical concept
in a "lost letter" to in a "lost letter" to of nondeterminism Asked THE question Answer still unknown
von Neumann the NSA
3
A difficult problem
Exhaustive search. Try all N ! orderings of the cities to look for a tour of length less than M .
4
How difficult can it be?
If one took the 100 largest cities in the US and wanted to travel them all, what is the distance of the
shortest route? I'm sure there's a simple answer. Anyone wanna help? A quick google revealed nothing.
I don't think there's any substitute for doing it manually. Google the cities, then pull out your map and
get to work. It shouldn't take longer than an hour. Edit: I didn't realize this was a standardized problem.
Writing a program to solve the problem would take 5 or 10 minutes for an average programmer.
However, the amount of time the program would need to run is, well, a LONG LONG LONG time.
My Garmin could probably solve this for you. Edit: probably not.
5
How difficult can it be?
value
quantity
(conservative estimate)
electrons in universe 1079
supercomputer instructions per second 1013
age of universe in seconds 1017
Q. Could the UBERcomputer solve the TSP for 100 cities with the brute force algorithm?
A. Not even close. 100! > 10157 >> 107910131017 = 10109 Would need 1048 UBERcomputers
Model of computation
• Running time: Number of steps as a function of input size N.
• Poly-time: Running time less than aNb for some constants a and b.
• Definitely not poly-time: Running time ~cN for any constant c > 1.
• Specific computer generally not relevant (simulation uses only a polynomial factor).
Def (in the context of this lecture). An algorithm is efficient if it is poly-time for all inputs.
Q. Can we find efficient algorithms for the practical problems that we face?
7
Reasonable questions about problems
Example 1: Sorting. Not intractable. (Insertion sort takes time proportional to N 2.)
Example 2: TSP. ??? (No efficient algorithm known, but no proof that none exists.)
8
Four fundamental problems
Example of an instance A solution
LSOLVE x1 + x2 = 1 x0 = − .25
• Solve simultaneous linear equations. 2x0 + 4x1 − 2x2 = 0.5 x1 = .5
• Variables are real numbers. 3x1 + 15x2 = 9 x2 = .5
ILP x1 + x2 ≥ 1 x0 = 0
• Solve simultaneous linear inequalities. x0 + x2 ≥ 1 x1 = 1
• Variables are 0 or 1. x0 + x1 + x2 ≤ 2 x2 = 1
9
Reasonable questions
LSOLVE, LP, ILP, and SAT are all important problem- LSOLVE
solving models with countless practical applications. • Solve simultaneous linear equations.
• Variables are real numbers.
11
Another profound connection to the real world
=
Implications
• Validates exponential/polynomial divide.
NO
= YES
... ...
12
COMPUTER SCIENCE
S E D G E W I C K / W A Y N E
Image sources
http://en.wikipedia.org/wiki/David_Hilbert#/media/File:Hilbert.jpg
http://www.nobelprize.org/nobel_prizes/economic-sciences/laureates/1994/nash-bio.html
http://en.wikipedia.org/wiki/Michael_O._Rabin#/media/File:Michael_O._Rabin.jpg
http://www.google.com/imgres?imgurl=http://www.cmu.edu/news/image-archive/Dana_Scott.jpg
http://en.wikipedia.org/wiki/Richard_M._Karp#/media/File:Karp_mg_7725-b.cr2.jpg
http://en.wikipedia.org/wiki/Stephen_Cook#/media/File:Prof.Cook.jpg
CS.16.A.Intractability.Questions
COMPUTER SCIENCE
S E D G E W I C K / W A Y N E
PA R T I I : A L G O R I T H M S , M A C H I N E S , a n d T H E O R Y
16. Intractability
• Reasonable questions
• P and NP
• Poly-time reductions
• NP-completeness
• Living with intractability
CS.16.B.Intractability.PandNP
Search problems
Search problem. Any problem for which an efficient algorithm exists to certify solutions.
Example: TSP.
Problem instance:
Set of cities, pairwise distances, and threshold M.
Certify solution by
adding distances and
checking that the total
is less than M
Significance. Problems that scientists, engineers, and applications programmers aspire to solve.
16
Brute force search
Brute-force search. Given a search problem, find a solution by checking all possibilities.
number of
problem description N (size) possibilities
number of
TSP ( S, M ) Find a tour of cities in S of length < M N!
cities
number of
ILP ( A, b ) Find a binary vector x that satisfies Ax ≤ b 2N
variables
number of
SAT ( Φ, b ) Find a boolean vector x that satisfies Ax = b 2N
variables
number of
FACTOR ( x ) Find a nontrivial factor of the integer M 10√N
digits in M
Some problems are more naturally formulated in one regime than another.
The regimes are not technically equivalent, but conclusions that we draw apply to all three.
NP. Class of all search problems, some of which seem solvable only by brute force.
P. Class of search problems solvable in poly-time.
The question: Is P = NP ?
P ≠ NP P = NP
• Intractable search problems exist. • All search problems are tractable.
• Brute force search may be the best • Efficient algorithms exist for IP, SAT,
we can do for some problems. FACTOR ... all problems in NP.
NP
Intractable
P
problems
P = NP
Frustrating situation. Researchers believe that P ≠ NP but no one has been able to prove it ( ! ! )
20
Nondeterminism: another way to view the situation
P ≠ NP P = NP
• Intractable search problems exist. • No intractable search problems exist.
• Nondeterministic machines would • Nondeterministic machines would be
admit efficient algorithms. of no help!
Frustrating situation. No one has been able to prove that nondeterminism would help ( ! ! )
21
Creativity: another way to view the situation
Examples
• Mozart composes a piece of music; the audience appreciates it.
Creative genius
• Wiles proves a deep theorem; a colleague checks it.
• Boeing designs an efficient airfoil; a simulator verifies it.
• Einstein proposes a theory; an experimentalist validates it.
Frustrating situation. No one has been able to prove that creating a solution to a problem
is more difficult than checking that it is correct.
22
COMPUTER SCIENCE
S E D G E W I C K / W A Y N E
Image sources
http://www.imdb.com/name/nm1101562/
http://en.wikipedia.org/wiki/Wolfgang_Amadeus_Mozart#/media/File:Wolfgang-amadeus-mozart_1.jpg
CS.16.B.Intractability.PandNP
COMPUTER SCIENCE
S E D G E W I C K / W A Y N E
PA R T I I : A L G O R I T H M S , M A C H I N E S , a n d T H E O R Y
19. Intractability
• Reasonable questions
• P and NP
• Poly-time reductions
• NP-completeness
• Living with intractability
CS.16.C.Intractability.Reductions
Classifying problems
Possible starting point: Assume that SAT is intractable (and hence P ≠ NP)
• Brute-force algorithm finds solution for any SAT instance.
• No known efficient algorithm does so.
A reasonable assumption.
Note. Many ways to extend. (Example: Use a polynomial number of instances of Y.)
26
Key point: poly-time reduction is transitive
27
Two ways to exploit reduction
• Poly-time reduce X to Y.
The efficient algorithm for Y gives an efficient algorithm for X.
SAT ILP
• Solve simultaneous boolean sums. • Solve simultaneous linear inequalities.
• Variables are true or false • Variables are 0 or 1.
(1 − t0) + t1 + t2 ≥ 1
¬x0 ⋁ x1 ⋁ x2 = true
t0 + (1 − t1) + t2 ≥ 1 ti = 0 iff xi = false
x0 ⋁ ¬x1 ⋁ x2 = true
(1 − t0) + (1 − t1) + (1 − t2) ≥ 1 ti = 1 iff xi = true
¬x0 ⋁ ¬x1 ⋁ ¬x2 = true
(1 − t0) + (1 − t1) + t3 ≥ 1
¬x0 ⋁ ¬x1 ⋁ x3 = true
An instance of SAT x0 = false Poly-time reduction to an instance of ILP t0 = 0
x1 = true
t1 = 1
x2 = true
t2 = 1
x3 = false
t3 = 0
A solution
A solution
SAT
Dick Karp
1985 Turing Award
EXACT COVER CLIQUE HAMILTON CYCLE
PARTITION
30
Still more poly-time reductions from SAT
CS.16.C.Intractability.Reductions
COMPUTER SCIENCE
S E D G E W I C K / W A Y N E
PA R T I I : A L G O R I T H M S , M A C H I N E S , a n d T H E O R Y
19. Intractability
• Reasonable questions
• P and NP
• Poly-time reductions
• NP-completeness
• Living with intractability
CS.16.D.Intractability.NPcomplete
NP-completeness
Nondeterministic SAT
Turing machine instance
Equivalent. Assuming that SAT is intractable is the same as assuming that P ≠ NP.
34
Cook-Levin theorem
SAT
PARTITION
35
Karp + Cook-Levin
SAT
3-COLOR ILP VERTEX COVER Steve Cook Leonid Levin Dick Karp
36
Two possible universes
P ≠ NP P = NP
• Intractable search problems exist. • No intractable search problems exist.
• Nondeterminism would help. • Nondeterminism is no help.
• Computing an answer is more difficult • Finding an answer is just as easy as
than correctly guessing it. correctly guessing an answer.
• Can prove a problem to be intractable by
• Guaranteed poly-time algorithms exist for
poly-time reduction from an NP-complete all problems in NP.
problem.
NP
P NP-complete P = NP = NPC
Frustrating situation. No progress on resolving the question despite 40+ years of research.
37
Summary
NP. Class of all search problems, some of which seem solvable only by brute force.
P. Class of search problems solvable in poly-time.
NP-complete. "Hardest" problems in NP.
Intractable. Search problems not in P (if P ≠ NP).
38
Princeton CS building, west wall
39
Princeton CS building, west wall (closeup)
1
0 1
0 0 0
1 1 0
1 1 0
0 0 1 0
1 1 1 1
0 1 1
0 0 0
1 1 0 0
1 0
1 1 1
40
COMPUTER SCIENCE
S E D G E W I C K / W A Y N E
CS.16.D.Intractability.NPcomplete
COMPUTER SCIENCE
S E D G E W I C K / W A Y N E
PA R T I I : A L G O R I T H M S , M A C H I N E S , a n d T H E O R Y
16. Intractability
• Reasonable questions
• P and NP
• Poly-time reductions
• NP-completeness
• Living with intractability
CS.16.E.Intractability.Living
Living with intractability
43
Living with intractability: Don't try to solve intractable problems
Knows no theory
Knows computability
I can't find an efficient algorithm.
I guess I'm just to dumb.
Knows intractability
45
Living with intractability: look for solutions to real-world problem instances
Observations
• Worst-case inputs may not occur for practical problems.
• Instances that do occur in practice may be easier to solve.
Reasonable approach: relax the condition of guaranteed poly-time algorithms.
SAT
• Chaff solves real-world instances with 10,000+ variables.
• Princeton senior independent work (!) in 2000. TSP solution for 13,509 US cities
TSP
• Concorde routinely solves large real-world instances.
• 85,900-city instance solved in 2006.
ILP
• CPLEX routinely solves large real-world instances.
• Routinely used in scientific and commercial applications.
46
Exploiting intractability: RSA cryptosystem
Multiply (easy)
Factor (difficult)
47
Exploiting intractability: RSA cryptosystem
74037563479561712828046796097429573142593188889231289
Example: Factor this 08493623263897276503402826627689199641962511784399589
212-digit integer 43305021275853701189680982867331732731089309005525051
16877063299072396380786710086096962537934650563796359
48
Fame and fortune through intractability
74037563479561712828046796097429573142593188889231289
Factor this
08493623263897276503402826627689199641962511784399589 $30,000 prize
212-digit integer 43305021275853701189680982867331732731089309005525051 claimed in July, 2012
16877063299072396380786710086096962537934650563796359
Create an e-commerce
company based on the RSA sold to EMC
for $2.1 billion in 2006
difficulty of factoring
$1 million prize
Resolve P vs. NP unclaimed since 2000
49
A final thought
Theorem (Shor, 1994). An N-bit integer can be factored in N 3 steps on a quantum computer.
Image sources
http://people.csail.mit.edu/rivest/
http://www.google.com/imgres?imgurl=http://upload.wikimedia.org/wikipedia/commons/1/1e/Adi_Shamir_at_TU_Darmstadt_(2013).jpg
http://www.google.com/imgres?imgurl=http://upload.wikimedia.org/wikipedia/commons/a/af/Len-mankin-pic.jpg
CS.16.E.Intractability.Living
COMPUTER SCIENCE
S E D G E W I C K / W A Y N E
PA R T I I : A L G O R I T H M S , M A C H I N E S , a n d T H E O R Y
Computer Science
ROBERT SEDGEWICK
K E V I N WAY N E
Section 7.4
http://introcs.cs.princeton.edu