Scribd
Upload
33 views

Formal Languages, Automata and Computability: Read Sections 7.1 - 7.3 of The Book For Next Time

This document discusses time complexity and NP-completeness. It begins by defining time complexity and asymptotic analysis using big-O notation. Non-deterministic Turing machines are introduced, and NP is defined as the set of problems verifiable in polynomial time. Boolean formulas and satisfiability (SAT) are discussed, and SAT is shown to be NP-complete. This implies that if SAT could be solved in polynomial time, then P would equal NP. 3SAT is also proven to be NP-complete. The document concludes by discussing polynomial-time reductions and the implications of NP-complete problems.

Uploaded by

Vinny Sasha
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
33 views

Formal Languages, Automata and Computability: Read Sections 7.1 - 7.3 of The Book For Next Time

This document discusses time complexity and NP-completeness. It begins by defining time complexity and asymptotic analysis using big-O notation. Non-deterministic Turing machines are introduced, and NP is defined as the set of problems verifiable in polynomial time. Boolean formulas and satisfiability (SAT) are discussed, and SAT is shown to be NP-complete. This implies that if SAT could be solved in polynomial time, then P would equal NP. 3SAT is also proven to be NP-complete. The document concludes by discussing polynomial-time reductions and the implications of NP-complete problems.

Uploaded by

Vinny Sasha
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 46

15-453

FORMAL LANGUAGES,
AUTOMATA AND
COMPUTABILITY
Read sections 7.1 7.3 of the book for next time

TIME COMPLEXITY
OF ALGORITHMS
(Chapter 7 in the textbook)

COMPLEXITY THEORY
Studies what can and cant be computed under
limited resources such as time, space, etc.
Today: Time complexity

MEASURING TIME COMPLEXITY


We measure time complexity by counting the
elementary steps required for a machine to halt
Consider the language A = { 0k1k | k 0 }
1. Scan across the tape and reject if the
2k
string is not of the form 0m1n
2k2

2. Repeat the following if both 0s and 1s


remain on the tape:
Scan across the tape, crossing off a
single 0 and a single 1

2k 3. If 0s remain after all 1s have been crossed


off, or vice-versa, reject. Otherwise accept.

The number of steps that an algorithm uses


on a particular input may depend on several
parameters.
For instance, if the input is a graph, then the
number of steps may depend on the number
of nodes, the number of edges, et cetera.
For simplicity, we compute the running time
purely as a function of the length of the input
string and dont consider any other
parameters.

Let M be a TM that halts on all inputs.


Assume we compute the running time purely as
a function of the length of the input string.
Definition: The running time or time-complexity
function of M is the function f : N N such
that f(n) is the maximum number of steps that M
uses on any input of length n.

ASYMPTOTIC ANALYSIS
5n3 + 2n2 + 22n + 6 = O(n3)
Big-O notation has been discussed
in previous classes. We will briefly
review it.

BIG-O
Let f and g be two functions f, g : N R+.
We say that f(n) = O(g(n)) if positive integers
c and n0 exist so that for every integer n n0

f(n) c g(n)
When f(n) = O(g(n)), we say that g(n) is an
asymptotic upper bound for f(n)

5n3 + 2n2 + 22n + 6 = O(n3)


If c = 6 and n0 = 10, then 5n3 + 2n2 + 22n + 6 cn3

2 n4.1 + 200283 n4 + 2 = O(n4.1)


3 n log2 n + 5 n log2log2 n = O(n log2 n)
n log10 n78 = O(n log10 n)
log10 n = log2 n / log2 10
O(n log2 n) = O(n log10 n) = O(n log n)

Definition: TIME(t(n)) is the set of


languages decidable in O(t(n)) time
by a Turing Machine.
{ 0k1k | k 0 } TIME(n2)

A = { 0k1k | k 0 } TIME(n log n)


Cross off every other 0 and every other 1. If the #
of 0s and 1s left on the tape is odd, reject.

00000000000001111111111111
x0x0x0x0x0x0xx1x1x1x1x1x1x
xxx0xxx0xxx0xxxx1xxx1xxx1x
xxxxxxx0xxxxxxxxxxxx1xxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxx

We can prove that a (single-tape) TM


cant decide A faster than O(n log n).
A = { 0k1k | k 0 }

Can A = { 0k1k | k 0 } be decided in


time O(n) with a two-tape TM?
Scan all 0s and copy them to the
second tape.
Scan all 1s, crossing off a 0 from the
second tape for each 1.

Different models of computation


yield different running times for
the same language!

Theorem.
Let t(n) be a function such that t(n) n.
Then every t(n)-time multi-tape TM has
an equivalent O(t(n)2) single-tape TM.

Polynomial Time

P=

TIME(n )
k

kN

NON-DETERMINISTIC
TURING MACHINES
AND NP

NON-DETERMINISTIC TMs
read

write

0 0, R

move

0 0, R

, R
0 0, R

, R
qreject

qaccept

NON-DETERMINISTIC TMs
are just like standard TMs, except:
1. The machine may proceed according to
several possibilities.
2. The machine accepts a string if there
exists a path from start configuration to an
accepting configuration.

Deterministic
Computation

Non-Deterministic
Computation

reject

accept or reject

accept

Definition: A Non-Deterministic TM is a 7-tuple


T = (Q, , , , q0, qaccept, qreject), where:
Q is a finite set of states
is the input alphabet, where
is the tape alphabet, where and

: Q Pow(Q {L,R})
q0 Q is the start state
qaccept Q is the accept state
qreject Q is the reject state, and qreject qaccept

Deterministic
Computation

Non-Deterministic
Computation

reject

accept or reject

accept

Definition: NTIME(t(n)) is the set of languages


decided by a O(t(n))-time non-deterministic
Turing machine.
TIME(t(n)) NTIME(t(n))

BOOLEAN FORMULAS
logical
parentheses
operations
A satisfying assignment is a setting of the
variables that makes the formula true.

= (x y) z

x = 1, y = 1, z = 1 is a satisfying assignment for


variables

BOOLEAN FORMULAS
x = 0, y = 0, z = 1 is a satisfying assignment for:

(x y) (z x)
0

0
1

0
1

A boolean formula is satisfiable if there


exists a satisfying assignment for it.
YES
NO

a b c d
(x y) x

Definition: SAT is the language consisting of


all satisfiable boolean formulas.
SAT = { | is a satisfiable boolean formula }

Conjunctive Normal Form (CNF)


A literal is a variable or the negation of a var.
Example: The variable x is a literal, and
its negation, x, is a literal.
A clause is a disjunction (an OR) of literals.
Example: (x y z) is a clause.
A formula is in Conjunctive Normal Form (CNF)
if it is a conjunction (an AND) of clauses.
Example: (x z) (y z) is in CNF.
A CNF formula is a conjunction of disjunctions
of literals.

Definition: A CNF formula is a 3CNF-formula iff


each clause has exactly 3 literals.
(x1 x2 x3) (x4 x2 x5) ... (x3 x2 x1)
clauses
A literal is a variable or the negation of a var.
A clause is a disjunction (an OR) of literals.
A formula is in Conjunctive Normal Form (CNF)
if it is a conjunction (an AND) of clauses.
A CNF formula is a conjunction of disjunctions
of literals.

Definition: A CNF formula is a 3CNF-formula iff


each clause has exactly 3 literals.
(x1 x2 x3) (x4 x2 x5) ... (x3 x2 x1)
clauses
YES (x1 x2 x1)
NO (x3 x1) (x3 x2 x1)
NO (x1 x2 x3) (x4 x2 x1) (x3 x1 x1)
NO (x1 x2 x3) (x3 x2 x1)
3SAT = { | is a satisfiable 3cnf-formula }

3SAT = { | is a satisfiable 3cnf-formula }


Theorem: 3SAT NTIME(n2)
On input :
1. Check if the formula is in 3cnf.
2. For each variable, non-deterministically
substitute it with 0 or 1.
( x y x )
( 0 y 0 )
( 0 0 0 )

( 1 y 1 )
( 0 1 0 )

3. Test if the assignment satisfies .

Non-deterministic Polynomial Time

NP =

NTIME(n )
k

kN

Theorem: L NP if and only if there exists a


poly-time Turing machine V with
L = { x | y. |y| = poly(|x|) and V(x,y) accepts }.
Proof:
(1) If L = { x | y. |y| = poly(|x|) and V(x,y) accepts }
then L NP.
Because we can guess y and then run V.
(2) If L NP then
L = { x | y. |y| = poly(|x|) and V(x,y) accepts }
Let N be a non-deterministic poly-time TM
that decides L. Define V(x,y) to accept if y is
an accepting computation history of N on x.

3SAT = { | y such that y is a satisfying


assignment to and is in 3cnf }
SAT = { | y such that y is a satisfying
assignment to }

A language is in NP if and only if there


exist polynomial-length certificates for
membership to the language.
SAT is in NP because a satisfying
assignment is a polynomial-length
certificate that a formula is satisfiable.

NP = The set of all the problems for


which you can verify an alleged
solution in polynomial time.

P = NP?

POLY-TIME REDUCIBILITY
f : * * is a polynomial-time computable
function if some poly-time Turing machine M, on
every input w, halts with just f(w) on its tape.
Language A is polynomial time reducible to
language B, written A P B, if there is a polytime computable function f : * * such that:
w A f(w) B
f is called a polynomial-time reduction of A to B.

B
f

Theorem: If A P B and B P, then A P.


Proof: Let MB be a poly-time (deterministic)
TM that decides B and let f be a poly-time
reduction from A to B.
We build a machine MA that decides A as follows:
On input w:
1. Compute f(w)
2. Run MB on f(w)

Definition: A language B is NP-complete iff:


1. B NP
2. Every language in NP is reducible
to B in polynomial time.

If B is NP-Complete and B P then NP = P. Why?

NP

P
B

SAT = { | is a satisfiable boolean formula }


3-SAT = { | is a satisfiable 3cnf-formula }
A 3cnf-formula is of the form:
(x1 x2 x3) (x4 x2 x5) (x3 x2 x1)
clauses

SAT = { | is a satisfiable boolean formula }


3-SAT = { | is a satisfiable 3cnf-formula }

SAT, 3-SAT NP

Theorem (Cook-Levin): SAT is NP-complete.


Proof Outline:
(1) SAT NP
(2) Every language A in NP is polynomial time
reducible to SAT
We build a poly-time reduction from A to SAT
The reduction turns a string w into a 3-cnf
formula such that w A iff 3-SAT.
will simulate the NP machine N for A on w.
Let N be a non-deterministic TM that decides
A in time nk How do we know N exists?

So proof will also show:


3-SAT is NP-Complete

NP
3-SAT

f turns a string w into a 3-cnf formula such that


w A 3-SAT.
will simulate an NP machine N on w, where A = L(N)

You might also like