Complexity Theory

Course Notes

Sebastiaan A. Terwijn
Radboud University Nijmegen
Department of Mathematics
P.O. Box 9010
6500 GL Nijmegen
the Netherlands
[email protected]

Copyright c 2010 by Sebastiaan A. Terwijn

Version: December 2017

ii
Contents

1 Introduction 1
1.1 Complexity theory . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.3 Turing machines . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.4 Big O and small o . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.5 Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.6 Number theory . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Basics 6
2.1 Time and space bounds . . . . . . . . . . . . . . . . . . . . . . 6
2.2 Inclusions between classes . . . . . . . . . . . . . . . . . . . . 7
2.3 Hierarchy theorems . . . . . . . . . . . . . . . . . . . . . . . . 8
2.4 Central complexity classes . . . . . . . . . . . . . . . . . . . . 10
2.5 Problems from logic, algebra, and graph theory . . . . . . . . 11
2.6 The Immerman-Szelepcsényi Theorem . . . . . . . . . . . . . 12
2.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3 Reductions and completeness 16

3.1 Many-one reductions . . . . . . . . . . . . . . . . . . . . . . . 16
3.2 NP-complete problems . . . . . . . . . . . . . . . . . . . . . . 18
3.3 More decision problems from logic . . . . . . . . . . . . . . . . 19
3.4 Completeness of Hamilton path and TSP . . . . . . . . . . . . 22
3.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

4 Relativized computation and the polynomial hierarchy 27

4.1 Relativized computation . . . . . . . . . . . . . . . . . . . . . 27
4.2 The Polynomial Hierarchy . . . . . . . . . . . . . . . . . . . . 28
4.3 Relativization . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

iii
5 Diagonalization 34
5.1 The Halting Problem . . . . . . . . . . . . . . . . . . . . . . . 34
5.2 Intermediate sets . . . . . . . . . . . . . . . . . . . . . . . . . 34
5.3 Oracle separations . . . . . . . . . . . . . . . . . . . . . . . . 36
5.4 Many-one versus Turing reductions . . . . . . . . . . . . . . . 38
5.5 Sparse sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
5.6 The Gap Theorem . . . . . . . . . . . . . . . . . . . . . . . . 40
5.7 The Speed-Up Theorem . . . . . . . . . . . . . . . . . . . . . 41
5.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

6 Randomized computation 45
6.1 Probabilistic classes . . . . . . . . . . . . . . . . . . . . . . . . 45
6.2 More about BPP . . . . . . . . . . . . . . . . . . . . . . . . . 48
6.3 The classes RP and ZPP . . . . . . . . . . . . . . . . . . . . . 50
6.4 Primes again . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
6.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

7 Circuit complexity 55
7.1 Boolean functions . . . . . . . . . . . . . . . . . . . . . . . . . 55
7.2 Circuits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
7.3 The relation with advice classes . . . . . . . . . . . . . . . . . 57
7.4 Small circuits for NP? . . . . . . . . . . . . . . . . . . . . . . 58
7.5 Sparse T-complete sets . . . . . . . . . . . . . . . . . . . . . . 59
7.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

8 Cryptography 62
8.1 Public-key cryptography . . . . . . . . . . . . . . . . . . . . . 62
8.2 Signed messages . . . . . . . . . . . . . . . . . . . . . . . . . . 64
8.3 One-way functions . . . . . . . . . . . . . . . . . . . . . . . . 64
8.4 The class UP . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
8.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

9 Interactive proofs 67
9.1 Interactive protocols and the class IP . . . . . . . . . . . . . . 67
9.2 IP = PSPACE . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
9.3 Zero-knowledge proofs . . . . . . . . . . . . . . . . . . . . . . 72
9.4 Lowness of Graph Isomorphism . . . . . . . . . . . . . . . . . 73
9.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

10 Approximations 75
10.1 Approximating the traveling salesman . . . . . . . . . . . . . . 75
10.2 Approximation algorithms . . . . . . . . . . . . . . . . . . . . 76
10.3 Probabilistically checkable proofs . . . . . . . . . . . . . . . . 77
10.4 PCP theory and nonapproximability . . . . . . . . . . . . . . 78

iv
 10.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

11 Proof complexity 81
11.1 Propositional proof systems and Cook’s program . . . . . . . . 81
11.2 The pigeonhole principle . . . . . . . . . . . . . . . . . . . . . 82
11.3 Resolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
11.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

Further reading 87

Bibliography 88

Index 93

v
Chapter 1

Introduction

1.1 Complexity theory

Complexity theory is concerned with the resources, such as time and space,
needed to solve computational problems. After the success of the general
theory of computability, that gave us a precise definition of the notion of
algorithm and fundamental insights into the notion of mathematical proof
and undecidability, it was only natural to also consider the notion of efficient
computation, and ask why some problems that are computable in principle
resist all attempts to compute them in a reasonable amount of time. There
is an abundance of problems from mathematics and computer science that
are trivially solvable by brute force search of an exponential number of in-
stances, but for which currently no efficient algorithm is known. Complexity
theory is the appropriate setting for the study of such problems. It is also the
home of one of the most fundamental open problems in mathematics, namely
the famous NP versus P problem. Some 40 years after the discovery of this
problem, complexity theory has matured into an extremely rich and fasci-
nating subject, despite the lack of progress on some of its most fundamental
problems.

1.2 Preliminaries
In the following sections we list some preliminaries and notation. Most of the
material treated here can be found in textbooks such as [2], [4], [32], and [35].
The notation used below is mostly standard. Given a set Σ, the set of all
finite strings of elements from Σ is denoted by Σ∗ . (This set is also referred to
as the set of all words over the alphabet Σ.) The ∗-operator is called Kleene
star. We usually work over the set of finite binary strings {0, 1}∗ , that can be
interpreted as binary representations of the natural numbers N. The length of
a string x is denoted by |x|, and {0, 1}n denotes the set of strings of length n.
The concatenation of strings x and y is denoted by xby or simply by xy. The

1
2 Chapter 1. Introduction

i-fold concatenation of x with itself is denoted by xi . Whether xi denotes the

i-fold concatenation of the string x or the i-th power of the natural number
x will always be clear from the context. We let x ⊑ y denote that the string
x is an initial segment of the string y. Throughout, h· , ·i denotes a bijective
pairing function from N × N to N (or equivalently from pairs of strings to
strings) defined by
1 2
 1 2

hx, yi = (x + y + 1) − (x + y + 1) + x = (x + y) + 3x + y .
2 2
Objects such as formulas and Turing machines that can be coded using strings
from {0, 1}∗ are routinely handled as such. Subsets A ⊆ {0, 1}∗ are variously
called set, problem (in case we think of it as a decision problem, namely to
decide for a given element x whether x ∈ A), or language. We often identify
a set A with its characteristic sequence, i.e. we write A(x) = 1 if x ∈ A
and A(x) = 0 otherwise. For a given set A, A denotes the complement of
A, and A↾x denotes the finite set A ∩ {0 . . . x}, or equivalently, the finite
string
 A(0)A(1)
. . . A(x). For a given class C of sets, co-C denotes the class
A : A ∈ C of all sets whose complement is in C. |A| denotes the cardinality
of A.
Each chapter contains a section with exercises. Exercises that are more
challenging or that require a more elaborate solution are marked with ⋆ .

1.3 Turing machines

Preliminaries about Turing machines can be found in [42]. (Note however
that in complexity theory Turing machines are allowed to have any finite
number of tapes.)
Given a Turing machine M and an input x, we use the notation M (x) ↓
to denote that the computation of M on x halts in a finite number of steps,
and we write M (x) ↑ if this is not the case. A set or a function is computable
if there is a Turing machine computing it.1 For notational convenience, we
often identify machines with the sets they compute. A set is computably
enumerable (c.e.) if it is empty or the range of a computable function (cf.
also Exercise 2.7.13.)
Given a computable set A and a machine M that computes A, we also
say that M recognizes A, or that M accepts A.
A Turing machine is nondeterministic if at any computation step, there
is a set of next possible states, rather than a single next state as in a de-
1
In computability theory it is essential to work with partial computable functions, i.e.
functions that need not be defined on all arguments. Partial functions that are everywhere
defined are called total, and a computable function is total by definition. In complexity
theory, because of the presence of time and space bounds, the distinction between total
and partial functions is less important.
1.4. Big O and small o 3

terministic computation. That means that, on any given input, there is a

set of possible computation paths, rather than a single one. By definition, a
nondeterministic Turing machine accepts an input when some computation
path accepts it.
We will often use the existence of universal Turing machines, that is,
the existence of a machine M with two arguments such that for any Turing
machine M ′ there exists i ∈ N (thought of as a code for M ′ ) such that
M (i, x) = M ′ (x) for any x. Here M (i, x) is defined as M (hi, xi). This means
that there is an computable list {Mi }i∈N of all Turing machines. This also
holds for Turing machines with resource bounds, cf. Theorem 2.1.1.

1.4 Big O and small o

We use the following common notation to compare the asymptotic behavior
of functions on N. The phrase “for almost all n ∈ N” means “for all n except
perhaps finitely many”.
f ∈ O(g) if there is a constant c ∈ N such that for almost all n we have
f (n) 6 cg(n).
f ∈ o(g) if
f (n)
lim = 0.
n→∞ g(n)

We can rephrase this as follows: for every real constant r > 0 it holds that
for almost every n, f (n) 6 rg(n).
The Big-O notation is often used to introduce constants without having to
name them: f ∈ O(1) means that f is bounded by a constant. For example,
g(n) 6 nO(1) means that g is at most of polynomial growth.
Big O and small o also have a dual notation, denoted with omegas:
f ∈ ω(g) if for every constant c ∈ N there are infinitely many n such that
f (n) > cg(n).
f ∈ Ω(g) if there is a real constant r > 0 such that for infinitely many n,
f (n) > rg(n).2
We also use the notation ∀∞ xϕ(x) to denote that ϕ(x) holds for almost
all x. Similarly, ∃∞ xϕ(x) denotes that there are infinitely many x such that
ϕ(x) holds.

1.5 Logic
We use the following terminology from logic. We assume the reader is familiar
with the basics of propositional logic. Propositional formulas are also called
2
This is the Hardy-Littlewood definition of Ω, see also Exercise 1.7.1. Knuth uses the
version with “almost every n” instead of “infinitely many n”, which is not equivalent.
4 Chapter 1. Introduction

Boolean formulas. A literal is a Boolean variable or the negation thereof.

For a literal x its negation is denoted by x̄. A clause is a disjunction of
literals. Clauses are often denoted using set notation, so the set of literals
{x1 , . . . , xk } denotes the clause x1 ∨ . . . ∨ xk .  denotes the empty clause, and
stands for false. A Boolean formula is in conjunctive normal form (CNF) if
it is a conjunction of clauses. A formula is in disjunctive normal form (DNF)
if it is a disjunction of conjunctions of literals.

1.6 Number theory

In this section we summarize some preliminaries from number theory that we
will use later. The reader can skip this section and refer back to it later when
needed.
For any integer n, Z/nZ is the additive group of integers modulo n.
(Z/nZ)∗ denotes the multiplicative group of integers x with gcd(x, n) = 1.

Theorem 1.6.1. (Chinese remainder theorem) For all n and m such that
gcd(n, m) = 1 we have (Z/nmZ) ∼
= (Z/nZ) × (Z/mZ).

Proof. Define the isomorphism by x mod nm 7→ (x mod n, x mod m).

The order (cardinality) of (Z/nZ)∗ is denoted by ϕ(n). The function ϕ is

called the Euler phi function. For p prime we have ϕ(p) = p − 1, as is easily
seen from the definition of (Z/pZ)∗ .

Theorem 1.6.2. (Euler) xϕ(n) ≡ 1 mod n for all x and n with gcd(x, n) = 1.

Proof. By definition, ϕ(n) is the order of (Z/nZ)∗ . By Lagrange’s theorem,

the order of every element in the group divides the order of the group.

A special case of Theorem 1.6.2 is Fermat’s little theorem: if p is prime then

ap−1 ≡ 1 mod p for every a with gcd(a, p) = 1.
We will apply the above results later in section 8.1.

Definition 1.6.3. Given a prime p > 2, an element r ∈ (Z/pZ)∗ is a primitive

root modulo p if it has order p − 1, i.e.

(i) rp−1 ≡ 1 mod p, and

(ii) ri 6≡ 1 mod p for every 1 < i < p − 1.

Note that we can replace (ii) in the above definition by

(ii)′ rp−1/q 6≡ 1 mod p for every proper prime factor q < p − 1 of p − 1.

1.7. Exercises 5

Namely, suppose that rp−1 ≡ 1 mod p and that i is the least such that ri ≡
1 mod p, and that i < p − 1. First note that i | p − 1, for otherwise the
remainder a = p − 1 mod i is smaller than i and also satisfies ra ≡ 1 mod p,
contradicting the minimality of i. So p − 1 is a multiple of i, and as i < p − 1
it follows that there is a prime factor q < p − 1 of p − 1 such that p−1 q
is a
multiple of i. Hence rp−1/q ≡ 1 mod p.

Theorem 1.6.4. A number p > 2 is prime precisely when there is a primitive

root modulo p.

Proof. If p is prime, it can be shown that there are ϕ(p − 1) primitive roots
modulo p. A proof of this can be found in Papadimitriou [32, p227].
Conversely, suppose that (ii)′ above holds. We prove that p is prime. If
p−1
r ≡ 1 mod p then the order of r divides p − 1. By (ii)′ it cannot be a
proper divisor, so the order of r is p − 1. Hence (Z/pZ)∗ has order p − 1, and
p is prime.
There is also a notion of primitive root modulo numbers n that are not prime:
r is a primitive root modulo n if ri 6≡ 1 mod n for every proper divisor i | ϕ(n).
We will later use the fact that for every p > 2 prime, there is a primitive root
modulo p2 , cf. Exercise 1.7.3.

1.7 Exercises
Exercise 1.7.1. (a) Show that f ∈ ω(g) if and only if f ∈
/ O(g) (for nonzero
functions).
(b) Show that f ∈ Ω(g) if and only if f ∈
/ o(g).

Exercise 1.7.2. Prove by formula induction that every propositional formula

can be put into conjunctive normal form.

Exercise 1.7.3. Suppose that p > 2 is prime. In this exercise we show that
there exists a primitive root modulo p2 . Let r be a primitive root modulo p.
(a) Show that r is a primitive root modulo p2 if both rp 6≡ 1 mod p2 and
rp−1 6≡ 1 mod p2 .
(b) Show that rp 6≡ 1 mod p2 .
(c) Show that (r + p)p ≡ rp mod p2 . Conclude that also (r + p)p 6≡ 1 mod p2 .
(d) Suppose that r is not a primitive root modulo p2 . Show that in that case
r + p is a primitive root modulo p2 .
Chapter 2

Basics

2.1 Time and space bounds

In the following we will consider computations with bounded resources. It is
customary to measure the complexity of computations in the size of the input,
which is usually its length when represented as a binary string. This means
in particular that the size of a number n ∈ N is of order log n. (Unless stated
otherwise, log is always to the base 2.) We only want to consider resource
bounds that are reasonable in the following sense.

• t : N → N is time constructible if there is a Turing machine which on

every input of length n halts in precisely t(n) steps.

• s : N → N is space constructible if there is a Turing machine that on

every input of length n halts in a configuration with exactly s(n) non-
blank tape cells and that used no other cells during the computation.1

For time constructible bounds it is possible to construct an effective enumer-

ation of all Turing machines that are clocked with the given bound. Thus
we can avoid decidability problems arising when considering arbitrary Tur-
ing machines. From now on we assume that all complexity bounds are time
or space constructible. What may happen in absence of this assumption is
discussed in section 5.6.
Given such constructible resource bounds we define the following classes:

• TIME(t) is the class of all sets that are accepted by a Turing machine
within running time t(n) for all inputs of length n.
1
Note that an input of length n automatically uses n tape cells of the input tape. In
order to let sublinear space bounds such as log n make sense, it is customary in complexity
theory to equip Turing machines with a separate input tape, and to not count the input
tape when counting the space used in a computation.

6
2.2. Inclusions between classes 7

• NTIME(t) is defined as TIME(t), but now with nondeterministic Turing

machines.

• SPACE(s) is the class of all sets that are accepted by a Turing machine
using space at most s(n) for all inputs of length n.

• NSPACE(s) is defined as SPACE(s), but now with nondeterministic

Turing machines.

We will often use the following basic result about universal Turing ma-
chines. A similar result holds for nondeterministic Turing machines.

Theorem 2.1.1. There exists an enumeration {Me }e∈N of all Turing ma-
chines such that the set

he, x, 1t i : M accepts x in 6 t steps

is in TIME(n2 ).

Proof. This follows in the same way as the existence of universal Turing
machines. The bound n2 is necessary for the simulation of Turing machines
with many tapes by a Turing machine with two tapes, though better bounds
are possible. For details see e.g. [21, p292] or [31, p82].

2.2 Inclusions between classes

We have the following inclusions between classes:

• TIME(t) ⊆ NTIME(t), SPACE(s) ⊆ NSPACE(s).

• TIME(t) ⊆ SPACE(t), NTIME(t) ⊆ NSPACE(t). This holds because

a machine cannot use more space than time.

• SPACE(c · s) ⊆ SPACE(s) for any c ∈ N. This result is called the

tape compression theorem. The idea of the proof is to increase the tape
alphabet by introducing a symbol for every block of length c. Idem for
NSPACE. The result means that we can work with O(s) rather than s
in the future.

• TIME(c · t) ⊆ TIME(t) for any c ∈ N and t with n ∈ o(t), i.e. such

that ∀r > 0∀∞ n(n < r · t(n)). This result is called the linear speed-
up theorem. The proof is slightly more complicated than that of the
previous item, but the idea is similar: again use symbols for c-blocks,
and simulate several steps with one by monitoring the actions of the
tape head within a given region.
8 Chapter 2. Basics

• NTIME(t) ⊆ SPACE(t). Reuse space! We only need to keep a counter

of length t for the current path. This gives NTIME(t) ⊆ SPACE(2t) =
SPACE(t).

• NSPACE(s) ⊆ TIME(2O(s) ). This holds because there are only 2s(n)

configurations of length s(n). Starting with the initial configuration,
inductively make a list of all configurations reachable by the nondeter-
ministic machine from the ones already found, and see if there is an
accepting one. This takes at most 2s(n) rounds, and every round has at
most 2s(n) steps. Note that it follows from this and the previous item
that NTIME(t) ⊆ TIME(2O(t) ).

• NSPACE(s) ⊆ SPACE(s2 ). This is Savitch’s Theorem proved below.

Savitch’s Theorem can be proved using that the path problem for graphs is
solvable in space log2 n, cf. Theorem 2.2.1 and Exercise 2.7.1 below. The
problem PATH (also known as reachability or connectivity) is defined by
Input: a directed graph G, and nodes x, y ∈ G,
Question: is there a path from x to y?

Theorem 2.2.1. PATH ∈ SPACE(log2 n).

Proof. Let Path(x, y, i) express that there is a path from x to y of length 6 2i .

Since there are n nodes, we will try to determine whether Path(x, y, ⌈log n⌉).
If i = 0 we can check Path(x, y, i) directly from the input.
If i > 0, check for all z ∈ G whether Path(x, z, i − 1) and Path(z, y, i − 1).
For the second, we can reuse the space of the first!
We have to keep a stack of answers to the recursive calls. The depth of the
recursion is log n, and the current value is of size 3 log n (because we work
with triples (x, y, i)), so the cost is at most 3 log2 n.
Note that in the path problem, the path from x to y can have length n, which
does not fit in space log2 n, so that the algorithm from Theorem 2.2.1 can
really only give a yes/no answer.

2.3 Hierarchy theorems

The following hierarchy theorems were proven by Hartmanis and Stearns in
1965, and can be seen as the start of the field of complexity theory.

Theorem 2.3.1. (Space Hierarchy Theorem) SPACE(s) 6⊇ SPACE(s′ ) for

every s′ ∈ ω(s), i.e. such that ∀c ∃∞ n (s′ (n) > c · s(n)).

Theorem 2.3.2. (Time Hierarchy Theorem) TIME(t′ ) 6⊆ TIME(t) for every

t′ ∈ ω(t log t). In particular, if also t′ > t then TIME(t) ( TIME(t′ ).
2.3. Hierarchy theorems 9

Proof of the hierarchy theorems. We only sketch the proof of the time hier-
archy theorem, since the space version is similar. Given time bounds t and t′
as above, consider an effective enumeration {Mi }i∈N of all Turing machines
working in time t. (Cf. Theorem 2.1.1. Note that we are using the time-
constructibility of t here.) On input w = 1i 0u run Mi (w) for t′ (|w|) steps. If
Mi halts within this time, let M (w) = 1 − Mi (w), and let M (w) = 0 other-
wise. Then M 6= Mi for every i, because t′ is infinitely often large enough
to complete a computation of Mi . The set computed by M is in TIME(t′ );
the extra factor log t is needed to simulate many tapes by a fixed number of
tapes. 
The proof just given is a variant of Cantor’s famous diagonal argument show-
ing that the set of reals is uncountable. The diagonalization method has been
very fruitful in computability and complexity theory. We will come back to
it in Chapter 5.
There are also hierarchy theorems for the nondeterministic classes. The
problem is how to do the diagonalization step M (w) = 1 − Mi (w) from the
proof above. In case of nondeterministic space, this can be arranged in a nice
way with an appeal to Theorem 2.6.1. The following hierarchy theorem for
nondeterministic time was proven by Cook [13].
Theorem 2.3.3. NTIME(n2 ) 6⊆ NTIME(n).
Proof. Let {Mi }i∈N be an enumeration of all nondeterministic Turing ma-
chines working in linear time. Such an enumeration can certainly be con-
structed in time n2 , that is, there is a nondeterministic Turing machine work-
ing in time n2 that, given a code i and input x, simulates Mi (x). We want
to construct a nondeterministic machine M working in time n2 such that for
every i the requirement


Ri : ∃x M (x) 6= Mi (x)
is satisfied. Given i, it seems problematic to satisfy Ri directly, as in the proof
of Theorem 2.3.2. Instead we pick a fresh candidate witness x and ensure the
following:
(i) M (y) = Mi (y + 1) for all y ∈ [x, 2x ),
(ii) M (2x ) 6= Mi (x).
Item (ii) can be easily achieved by direct diagonalization, since in time 2n
there is enough time to simulate all computation paths of Mi (x). To see that
this suffices, suppose that Mi (y) = M (y) for all y ∈ [x, 2x ]. Then in particular
Mi (y) = Mi (y + 1) for every y ∈ [x, 2x ) by (i), and hence Mi (x) = Mi (2x ).
But by (ii) we have Mi (2x ) 6= Mi (x), a contradiction.
To satisfy all the requirements Ri in one construction, all we need to do is
pick the witnesses x in such a way that the intervals [x, 2x ] are disjoint.
10 Chapter 2. Basics

Using a more careful analysis, a somewhat tighter version of the previous

hierarchy theorem can be obtained. Of course, the proof generalizes also to
larger time bounds.

2.4 Central complexity classes

There are many important complexity classes, but any list of them would
certainly include the following.
S
• P = i TIME(ni ) is the class of polynomial time computable sets.
S
• NP = i NTIME(ni ) is the class of nondeterministic polynomial time
computable sets.
S
• PSPACE = i SPACE(ni ) is the class of polynomial space computable
sets.
S i
• EXP = i TIME(2n ) is the class of exponential time computable sets.
Note that by the results of the previous section we have
P ⊆ NP ⊆ PSPACE ⊆ EXP (2.1)
and that by Savitch’s Theorem it holds that PSPACE = NPSPACE (since
(ni )2 is still polynomial). Note also that by the time hierarchy theorem,
P ( EXP, hence some of the inclusions in (2.1) must be strict! Unfortunately,
we have no clue which ones, but the conjecture is that in fact all the inclusions
are strict.
One of the central themes in complexity theory is the difference between
determinism and nondeterminism, and the tradeoff between time and space.
? ?
This translates into the most prominent questions P = NP and P = PSPACE.
P is often identified with the class of problems that are solvable in “reason-
able” time, although it may of course depend on the context what one finds
reasonable. In contrast, NP corresponds to the class of problems of which it
may be hard to find solutions, but of which one can efficiently check whether
a candidate solution really is one. Because it does seem not very useful if
one cannot even check whether a solution is correct, one could with a bit
of exaggeration say that NP is the largest class of problems that one would
be interested in knowing the answers to. Given a nondeterministic Turing
machine M working in time nc , one can view a computation of M as a binary
tree of depth nc , so that every binary string of length nc corresponds with
a computation path, and acceptance means that there is an accepting path,
cf. Figure 2.1. (Cf. also Exercise 2.7.5.) In discussing NP-problems one of-
ten says: “guess” a path and then perform the polynomial computation. An
accepting path is sometimes referred to as a certificate for the acceptance of
the computation.
2.5. Problems from logic, algebra, and graph theory 11

O w
✌✌ ✶✶✶
✌ ✶✶
✌✌✌ ✶✶
✌✌ ✶✶
✌✌ ✶✶
✌✌ ✶✶
✌ ✶✶
✌✌ ✶✶
nc ✌✌✌ ✶✶
✌✌ ✶✶
✌✌ ✶✶
✌✌ ✶✶
✌✌ ✶✶
✌✌ ✶✶
✌✌ ✶✶
✌ ✶✶
 ✌✌
accept/reject

Figure 2.1: A nondeterministic computation

2.5 Problems from logic, algebra, and graph theory

The satisfiability problem SAT is defined by
Input: a Boolean formula ϕ,
Question: is there a satisfying assignment of ϕ ?
It is easy to check that SAT ∈ NP: Given ϕ of length n, ϕ has at most n
propositional variables. Nondeterministically guess an assignment, and check
that it satisfies ϕ. If so, accept, if not, reject.
The problem PRIMES is defined by:
Input: n ∈ N,
Question: is n prime ?
Obviously, PRIMES is in co-NP (guess a decomposition). It was a longstand-
ing open problem whether PRIMES is in P, until this was finally solved in [1].
The proof is too long to include here, but we prove instead the weaker result
that PRIMES is in NP. Note that NP ∩ co-NP is not known to be equal to P.
In section 6.4 we will discuss a randomized algorithm for PRIMES.

Theorem 2.5.1. (Pratt) PRIMES is in NP.

Proof. We use Theorem 1.6.4, namely that p > 2 is prime precisely when
there is a primitive root modulo p, that is, an element 1 < x < p of order
p − 1:
(i) xp−1 ≡ 1 mod p, and
(ii) xp−1/q 6≡ 1 mod p for every prime factor q of p − 1.
Now given p, guess x and verify (i) and (ii). (i) poses no problem. To check
(ii), guess a prime factorization of p − 1 and recursively check that all the
factors q are prime and that (ii) holds.

Since graphs can be used to model so many discrete problems, it should not
12 Chapter 2. Basics

come as a surprise that numerous problems in complexity theory come from

graph theory. Here are two examples.
The HAMILTON PATH problem is defined by:
Input: a graph G,
Question: does G have a Hamilton path, i.e. a path that visits every node
exactly once?
HAMILTON PATH is in NP: simply guess a path of length |G| and check
that it is a Hamilton path.
The traveling salesman problem TSP is defined by:
Input: a complete undirected weighted graph G, that is, every two nodes i
and j have a distance di,j (with di,j = dj,i ), and k ∈ N,
Question: is there a tour of length (i.e. weight) at most k ? (A tour is a
path without repetitions visiting all nodes, starting and finishing in the same
node. So this is the same as a Hamilton circuit, cf. Exercise 3.5.18.)2
The problem TSP is in NP: Guess a permutation of the nodes of G and
check that it defines a tour of length at most k. Note that in contrast to
this, Dijkstra’s algorithm shows that the problem of finding shortest paths in
graphs is in P, cf. Exercise 2.7.8.

2.6 The Immerman-Szelepcsényi Theorem

The following theorem was proven independently in 1987 by Immerman [23]
and Szelepcsényi [41]. It says that the nondeterministic space classes are
closed under complementation. Note that if A ∈ NSPACE(f ), an application
of Savitch’s Theorem would only give that A ∈ NSPACE(f 2 ).

Theorem 2.6.1. Let f be a space constructible function such that f (n) >
log n. Then NSPACE(f ) = co-NSPACE(f ).

Proof. Suppose that machine M accepts A in nondeterministic space f (n).

Note that on a given input x of length n there are 2f (n) possible configurations.
We define a nondeterministic machine M that accepts the complement A, i.e.
x∈ / A if and only if M has a path accepting x. The idea is to cycle through
all possible configurations σ of M , for each one checking whether it is a final
configuration that is reachable from the initial configuration by a computation
of M . In case no such accepting configuration is found we know x ∈ / A so
2
Note that the parameter k is used to phrase the problem as a decision problem. For-
mally, all our complexity classes are classes of sets, that can be thought of as coding
decision problems with yes/no answers. One can also formulate complexity classes of func-
tions rather than sets, but we will not do this here. We also note that, although TSP is
most naturally phrased as an optimization problem (namely, what is the optimal tour?),
the optimization problem can be solved with repeated questions (logarithmically many,
using binary search) to the decision problem. This kind of reduction (a polynomial time
Turing reduction) will be discussed in Chapter 4.
2.6. The Immerman-Szelepcsényi Theorem 13

we let M accept x. The problem is that to see if σ is reachable we can only

guess a path of M .3 If a wrong path was guessed we may falsely think that
σ was unreachable. (Note that we will never think that an unreachable σ is
reachable.) Hence, in doing so for every σ we may underestimate the number
n(x) of reachable σ’s. If we knew the number n(x) in advance, we could
match the number of σ’s found in the above procedure against it, and let M
reject x in case the numbers did not match. In case the numbers matched
we would know we had seen all reachable configurations, and if none of them
were accepting we could safely declare x ∈ / A. If indeed x ∈ / A, for some
path of M all the guessed paths for the σ’s would be correct and we would
correctly declare x ∈/ A.
So it remains to be shown that we can nondeterministically compute the
number n(x) of configurations σ that are reachable by M from the initial con-
figuration. This is done by “inductive counting”. Let m(x, t) be the number
of configurations that are reachable in at most t steps. We can determinis-
tically compute m(x, 1). Suppose that we know m(x, t). Configuration σ is
reachable in at most t + 1 steps if σ is initial or there is a τ reachable in at
most t steps such that σ is reachable from τ in one more step. We compute
m(x, t + 1) from m(x, t) as follows. Cycle trough all configurations σ. For
every σ, cycle through all possible τ , each time guessing a path to reach τ
in at most t steps. If a τ is found to be reachable, and if σ is reachable
from τ in one step, increase the counter m(x, t + 1) by one. If at the end
the number of τ ’s found does not match m(x, t), reject x (since we know
a mistake was made). If there is a match with m(x, t) we know that also
m(x, t + 1) is correct. Finally, n(x) is equal to m(x, t) for the first t such that
m(x, t) = m(x, t + 1).

Theorem 2.6.1 has an important consequence for the theory of formal lan-
guages, namely that the class of languages that can be generated with a
context-sensitive grammar is closed under complements. This had been an
open problem for about 25 years.

Corollary 2.6.2. The class of context-sensitive languages is closed under

complements.

Proof. It is a result of formal language theory that the class of context-

sensitive languages is precisely the class of sets accepted by linearly bounded
automata, that is, NSPACE(n), so the result follows from Theorem 2.6.1.

Note that while Theorem 2.6.1 shows the closure of the nondeterministic space
classes under complementation, the same result for the nondeterministic time
classes remains one of the fundamental open problems in complexity theory.
3
Note that trying all paths is too expensive: with 2f (n) possible configurations there
are simply too many paths, so that a counter would be too long.
14 Chapter 2. Basics

It seems unlikely that a similar result holds in this case, e.g. that NP = co-NP
would hold.

2.7 Exercises
Exercise 2.7.1. Derive Savitch’s Theorem from Theorem 2.2.1. (Hint: Given
a nondeterministic machine we can apply Theorem 2.2.1 to the configuration
graph of the machine on a given input by asking whether an accepting con-
figuration is reachable.)
Exercise 2.7.2. The following problem is related to PATH. Suppose we are
given the adjacency matrix A of a binary relation ≺ on n points. Show that
it is possible to construct the adjacency matrix of the transitive closure of ≺
in polynomial time. (Hint: Ak contains the information whether from node i
we can reach node j in k steps.)
Exercise 2.7.3. By definition, a set is regular if it can be recognized by a one-
way Turing machine (i.e. tape heads can only move right) with a read-only
input.
Given L, define an equivalence relation on Σ∗ by x ≡L y precisely when
for all z, xz ∈ L ↔ yz ∈ L. Prove the Myhill-Nerode Theorem: L is regular
if and only if ≡L has only finitely many equivalence classes.
Exercise 2.7.4.⋆ In this exercise we prove that the class of regular sets equals
SPACE(k), the class of sets recognizable in constant space (for any fixed
constant k). Note that SPACE(k) = SPACE(1) by the tape compression
theorem.
(a) Show that regular sets can be recognized using constant space. (Recall
from page 6 that the space taken by the input does not count.)
(b) (Shepherdson) Prove that two-way machines with a read-only input
and working in constant space can be simulated by one-way machines. (Hint:
For the equivalence defined in Exercise 2.7.3 we have x ≡L y if and only if x
and y induce the same mapping of states of M as follows. Define mappings
fx and gx by fx (q) = p if whenever M enters x at the left in state q it leaves x
at the right in state p. Similar for gx but now from right to left. Acceptance
of xz depends only on fx and gx and not on x, and there are only finitely
many such functions since the number of states is finite.)
Exercise 2.7.5. In general the computation tree of a nondeterministic com-
putation in NTIME(nc ) is k-splitting, where k is the number of states in the
Turing machine. Argue that we may think of the tree as a binary tree, where
every path has length ⌈log k⌉nc .
Exercise 2.7.6. Deciding whether gcd(x, y) = 1 is in P. (Hint: The Eu-
clidean algorithm uses at most a logarithmic number of divisions x = ny + r,
since each time the remainder r is 6 21 x.)
2.7. Exercises 15

Exercise 2.7.7. (a) TSP was defined using tours. What is the total number
of tours in TSP?
(b)⋆ Note that, despite the bound from (a), TSP is solvable in time 2O(n) .

Exercise 2.7.8. (Dijkstra) Given a weighted graph G and x, y ∈ G, show

that the problem of finding a shortest path from x to y is in P. (If you like,
you can first phrase this as a decision problem.)

Exercise 2.7.9. Show that NP is closed under Kleene star. That is, if A is
in NP, then so is the set A∗ consisting of strings obtained by concatenating
a finite number of strings from A.

Exercise 2.7.10. ⋆ In this exercise we show that P is closed under Kleene

star. This is much harder than the previous exercise!
(a) Given L ∈ P and a string w of length n, let w[i, j] denote the substring
from bit i to j. Define a binary relation ≺ on pairs (i, j) by (i, j) ≺ (i′ , j ′ ) if
i′ = j + 1 and w[i, j] and w[i′ , j ′ ] are both in L. Let ≺∗ denote the transitive
closure of ≺. Show that w ∈ L∗ if and only if there exist j and k at most n
such that (1, j) ≺∗ (k, n).
(b) Suppose that L ∈ P. Show that L∗ ∈ P. (Hint: Use Exercise 2.7.2.)

Exercise 2.7.11. Is every subset of a set in NP automatically in NP ?

Exercise 2.7.12. Classify the following KNAPSACK problem: Given a set

{1, . . . , n} of items such that item i has value vi and weight wi for every i.
Given a knapsack that can hold a total weight of at most W , the problem
is to find a subset S ⊆ {1, . . . , n} of maximal
P value that fits into the knap-
sack.
P Put more mathematically: maximize i∈S vi subject to the condition
i∈S wi 6 W .

Exercise 2.7.13. A function g : {0, 1}∗ → {0, 1}∗ is called honest if there is
a polynomial p such that p(|g(x)|) > |x|, that is, g cannot shrink its input
by more than a polynomial factor. This exercise indicates the interest in the
notion of honesty for complexity theory.

(i) By definition, a set is computably enumerable (c.e.) if it is empty or the

range of a computable function. Show that a set A is c.e. if and only
if it is empty
 or the range of a polynomial time computable function g,
i.e. A = y : ∃x g(x) = y .

(ii) Show that a set A is in NP if and only if it is empty or the range of an

honest polynomial time computable function g.
Chapter 3

Reductions and completeness

One of the goals of complexity theory is to classify problems according to their

complexity. The main tool for doing this is to consider effective reductions
between problems. A key insight is that classes such as NP contain hardest
problems. Reductions also allow us to substantiate the idea that various
problems, though differently formulated, are actually the same.

3.1 Many-one reductions

Definition 3.1.1. Given sets A and B, A is many-one reducible or simply
m-reducible to B, written A 6m B, if for some computable function f ,

x ∈ A ⇐⇒ f (x) ∈ B

for every x. If f is in addition polynomial time computable then we say that

p p p
A is p-m-reducible to B,
p
 written pA 6 m B. We write A ≡m B if both A 6m B
and B 6m A. The set B : A ≡m B is called the p-m-degree of A.

The join A ⊕ B of two sets of strings A and B is defined as

 
A ⊕ B = xb0 : x ∈ A ∪ xb1 : x ∈ B .

Viewing strings as numbers, this is the same as putting A on the even numbers
and B on the odd ones. The set A ⊕ B contains precisely all the information
from A and B “zipped” together.

Proposition 3.1.2. (i) 6pm is reflexive and transitive.

(ii) A 6pm B precisely when A 6pm B.

(iii) If A ∈ P then A 6pm B for every set B that is not empty or equal to
{0, 1}∗ .

16
3.1. Many-one reductions 17

(iv) If both A 6pm C and B 6pm C then A ⊕ B 6pm C. Since clearly A,

B 6pm A ⊕ B this shows that on the p-m-degrees ⊕ gives the least upper
bound of A and B.
Proof. Exercise 3.5.1.
The graph problem 3COLORING is defined by
Input: a graph G,
Question: can G be colored with 3 colors, that is, does there exist a 3-
coloring of the vertices such that no pair of adjacent vertices have the same
color?
Proposition 3.1.3. 3COLORING 6pm SAT.
Proof. Suppose the given graph G has k vertices. Introduce variables xi,j ,
i = 1, . . . , k and j = 1, 2, 3, meaning “vertex i has color j”. We define an
instance of SAT by the following set of clauses:

C(i) = xi,1 , xi,2 , xi,3 , i = 1 . . . k, “every vertex has a color”

T (i) = x̄i,1 , x̄i,2 , i = 1 . . . k, “i not both color 1 and 2”

U (i) = x̄i,1 , x̄i,3 ,

V (i) = x̄i,2 , x̄i,3 .

Together these clauses express that every vertex has exactly one color. Next,
for every edge e from u to v and every color j define

D(e, j) = x̄u,j , x̄v,j

that is, u and v should not both have color j. We can write down these
clauses in linear time, and the set of clauses is satisfiable precisely when G is
3-colorable.
Proposition 3.1.3 is an example of the expressivity of SAT. In the following
we show that SAT can express all NP-problems.
Definition 3.1.4. A set A is called hard for a class C if C 6pm A for every
C ∈ C. A is complete for C if in addition A ∈ C.
Note that if A is hard and A 6pm B then also B is hard. Also note that all
the classes P, NP, co-NP, PSPACE are downwards closed under 6pm , that is,
if A 6pm B and B is in the class then also A belongs to it.
The notion of completeness is one of the central notions of our subject.
It allows us to study complexity classes by focussing on certain individual
elements in them. Before we can do so, we first have to show that such
complete elements exist.
18 Chapter 3. Reductions and completeness

3.2 NP-complete problems

One can easily define an NP-complete problem as follows. Let

K = hM, x, 1t i : M is a nondeterministic TM that accepts x in 6 t steps .

It is debatable whether K is a “natural” NP-problem. Of course, K is nat-

ural in our given context of complexity theory. It is the direct analogue of
the halting problem in computability theory. However, sometimes the term
“natural” is reserved for problems that arose in some other context and were
not defined directly using the notions at hand.

Theorem 3.2.1. K is NP-complete.

Proof. K ∈ NP: Given input hM, x, 1t i, guess a computation path of M of

length t and check whether it accepts x. This can be done in time O(n2 ) (cf.
Theorem 2.1.1).
K is NP-complete: Let A be an arbitrary set in NP. We show that
A 6pm K. Let M be a nondeterministic machine accepting A in time p(n).
Let f (x) = hM, x, 1p(|x| i. Then x ∈ A ⇔ f (x) ∈ K.
The following problem has perhaps a better claim to the label “natural”. It
is one of the most famous NP-complete problems, and it was also the first
one.

Theorem 3.2.2. (Cook [12])1 SAT is NP-complete.

Proof. Let L(M ) be a language in NP accepted by a nondeterministic machine

M working in time p. Given x of length n, we can effectively define the
following Boolean formulas. Let α and β denote vectors of Boolean variables
of length p(n). Intuitively, α codes a “snapshot” of a possible computation
of M at a time t, including the position of the tape head, the state of the
machine, and the contents of cells 1 to p(n).

1. Initial(α, x) expresses that α codes an initial configuration of M on input x.

2. Next(α, β) expresses that β is a configuration following α after one possible

computation step of M .

3. Accept(α) expresses that α codes an accepting configuration.

Now combine these formulas to form

p(n)−1
^
Accepted(x) = Initial(α1 , x) ∧ Next(αi , αi+1 ) ∧ Accept(αp(n) ).
i=1

1
Because of similar work by Levin, this is often referred to as the Cook-Levin theorem.
3.3. More decision problems from logic 19

Note that we can write this formula down in polynomial time, and that
it is satisfiable precisely when there are assignments αi encoding a com-
plete accepting computation of M on x. Hence x ∈ L(M ) if and only if
Accepted(x) ∈ SAT.

3.3 More decision problems from logic

In the following we will assume that all Boolean formulas are built with the
connectives ¬, ∨, ∧, and →. In particular we assume that the formulas do
not contain equivalences ↔. The reason for this is that removing these (by
rewriting ϕ ↔ ψ as (ϕ → ψ)∧(ψ → ϕ)) may result in an exponential blow-up
of the formula. Note that we can rewrite ϕ → ψ as ¬ϕ ∨ ψ without problems.
We have already encountered the set SAT of all satisfiable Boolean for-
mulas. Note that our assumption that formulas do not contain ↔ does not
affect the NP-completeness of SAT, as no equivalences were used in the proof
of this. Consider the following variants of SAT.

• SAT-CNF is the subset of SAT consisting of all satisfiable formulas that

are in conjunctive normal form.

• nSAT is the subset of SAT-CNF where every clause has at most n

literals. See Exercise 3.5.16 for some variations of this set.

• QBF is the set of satisfiable quantified Boolean formulas. The set of

q.b.f.’s is defined by

(i) 0,1,x (variable) are q.b.f.’s,

(ii) the set of q.b.f.’s is closed under ∧, ∨, ¬,
(iii) if F is a q.b.f. then also ∃xF and ∀xF are q.b.f.’s.

∃xF is true if F with x = 0 is true or F with x = 1 is true. ∀xF is true

if both F with x = 0 is true and F with x = 1 is true. As for SAT,
a q.b.f. is satisfiable if there is an assignment of the free variables that
makes the formula true. Note that it also makes sense to talk about the
truth of closed q.b.f.’s, i.e. those without free variables. E.g. ∀x(x ∨ ¬x)
is true.

Note that putting a Boolean formula in CNF using the distributive law

(ϕ ∧ ψ) ∨ χ ←→ (ϕ ∨ χ) ∧ (ψ ∨ χ)

can be expensive: it may result in an exponential blow-up of the formula

because of the repetition of χ on the right hand side. For QBF, however, we
have the following result:
20 Chapter 3. Reductions and completeness

Proposition 3.3.1. For every Boolean formula F of size m and containing

variables x1 . . . xm there is an equivalent q.b.f. ∃y1 . . . ∃yk F ′ with variables
x1 . . . xm , y1 . . . yk such that F ′ is in CNF and of size at most c · m2 , where c
is a constant independent of F .

Proof. Without loss of generality ¬ occurs only directly in front of variables

in F . Also, assume that all implications ϕ → ψ are written as ¬ϕ ∨ ψ. The
proof is by formula induction on F . By the previous assumptions, we only
need induction steps for ∨ and ∧.
The base case where F is a literal or a variable is clearly o.k.
F = F1 ∧ F2 . By induction F1 and F2 have the required form. Now just
put their ∃’s in front, after a possible renaming of variables.
F = F1 ∨ F2 . By induction F1 = ∃~y F1′ and F2 = ∃~zF2′ . Introduce a new
variable x to indicate whether F1 or F2 is true: Convert every clause C in F1′
to x∨C and every C in F2′ to ¬x∨C and let F ′ be the conjunction of all these
new clauses. Then ∃x∃~y ∃~zF ′ is equivalent to F . Note that the ∨-step adds a
constant to the size of every clause, hence at most c · m to the formula. Since
there are at most m such steps, the total size of the new formula is bounded
by c · m2 .

Corollary 3.3.2. SAT 6pm SAT-CNF. Hence SAT-CNF is NP-complete.

Proof. Given F , Proposition 3.3.1 gives an equivalent q.b.f. ∃~y F ′ with F ′ in

CNF. Now just drop the ∃’s: F ∈ SAT if and only if F ′ ∈ SAT.

The problem CLIQUE is defined by

Input: a graph G and k ∈ N,
Question: does G have a complete subgraph of size k ?

Theorem 3.3.3. CLIQUE is NP-complete.

Proof. That the problem is in NP is obvious: simply guess a subgraph of

size k and check that it is complete. We show that SAT-CNF6pm CLIQUE.
Given a formula F with literals x1 , . . . , xr and clauses C1 , . . . , Cs , let


V = (xi , Cj ) : xi ∈ Cj


E = (xi , Cj ), (xm , Cn ) : j 6= n ∧ x̄i 6= xm

k=s

We leave it as Exercise 3.5.11 to check that this is a p-m-reduction.

Theorem 3.3.4. 3SAT is NP-complete.

3.3. More decision problems from logic 21

Proof. We show that SAT-CNF6pm 3SAT, so the result follows from Corol-
lary 3.3.2. Given  an instance of SAT-CNF, first note that an assigment sat-
isfies a clause x1 ,  x2 , . . . , xk if and
 only if the same assignment satisfies
the conjunction of x1 , x2 , z and z̄, x3 ,  . . . , xk for a suitable value of z.
By iterating this, we replace every clause x1 , x2 , . . . , xk with more than 3
literals by the k − 2 new clauses
    
x1 , x2 , z1 , x3 , z̄1 , z2 , x4 , z̄2 , z3 , . . . , xk−2 , z̄k−2 , zk−3 , xk−1 , xk , z̄k−3 .

Then the formula obtained in this way is equisatisfiable with the original
one.
In contrast to Theorem 3.3.4 we have
Theorem 3.3.5. 2SAT is in P.
Proof. Exercise 3.5.12.
The proof of the following theorem has a certain similarity with the proof of
Savitch’s theorem.
Theorem 3.3.6. (Meyer and Stockmeyer [24]) QBF is PSPACE-complete.
Proof. First note that QBF ∈ SPACE(n): try all (linear size) assignments,
using a (linear sized) counter.
For the completeness, let M be a given machine with space bound p. A
computation of M on an input of size n has at most 2p(n) configurations, and
is accepting if an accepting configuration is reachable in at most 2p(n) steps
from the initial configuration. So the theorem follows if we can efficiently
construct a q.b.f. Access2m (α, β) expressing that α and β code configurations
of M and β is reachable from α in 6 2m steps. We already saw in the proof of
Theorem 3.2.2 that there are Boolean formulas expressing that α is an (initial
or accepting) configuration of M .
m = 0: We can use the formula Next(α, β) from the proof of Theorem 3.2.2
to express that β is reachable from α in at most one step.
m > 0: Given Access2m−1 (α, β), a bad attempt would be to write


Access2m (α, β) = ∃γ Access2m−1 (α, γ) ∧ Access2m−1 (γ, β)

because then the length of the formula doubles with every step. Instead write

Access2m (α, β) = ∃γ∀α′ , β ′ (α′ = α ∧ β ′ = γ) ∨ (α′ = γ ∧ β ′ = β) →

Access2m−1 (α′ , β ′ ) .

Note that the extra quantification is used so that we have to write the previous
formula Access2m−1 only once instead of twice. This makes the recursion a
linear affair.
22 Chapter 3. Reductions and completeness

3.4 Completeness of Hamilton path and TSP

We can view reductions as a translation from the language of the easier prob-
lem to that of the harder one. As we have seen, these reductions are a useful
tool in proving that certain problems are complete, using problems that we
already know to be complete. The reductions we have seen so far were rather
easy to describe. In this section we give an example of a more elaborate
reduction, showing completeness of HAMILTON PATH and TSP.

Theorem 3.4.1. (Karp [25]) HAMILTON PATH is NP-complete.

Proof. We follow the proof in [32]. That the problem is in NP was already
discussed on page 12. We prove that 3SAT 6pm HAMILTON PATH, so that
the completeness follows from Theorem 3.3.4. Suppose that ϕ is an instance
of 3SAT, with variables x1 , . . . , xn and clauses C1 , . . . , Cm . Without loss of
generality, every clause contains exactly three variables (see Exercise 3.5.16).
We construct an instance f (ϕ) of HAMILTON PATH by piecing together
some basic modules. Truth or falsity of a variable is coded by the truth
module from Figure 3.1. Any Hamilton path has to choose between the left

✟✟ ✻✻✻
✟✟ ✻✻
✟✟ ✻✻
✟✟ ✻✻
✟

Figure 3.1: The truth module Figure 3.2: The clause module

and the right edge. We let the left one stand for “true” and the right one
for “false”. We represent the clauses by triangles, as in Figure 3.2, where
each side of the triangle represents a literal from the clause. We will arrange
things such that only sides for which the corresponding literal is false have to
be traversed. Note that by definition no Hamilton path can traverse all sides
of a triangle. Hence if all literals are false there will be no Hamilton path.
Next consider the module on the left hand side of Figure 3.3. Any Hamil-
ton path that does not start or end in a node of this module has only two
possible ways of routing through it: Either entering in a0 and exiting in a1 or
entering in b0 and exiting in b1 . (Cf. Exercise 3.5.17.) Thus the module acts
as a kind of XOR gate forcing a choice between two “edges”, viz. the upper
3.4. Completeness of Hamilton path and TSP 23

a0 a1 ✤O
✤

✤
L
✤
✤
✤
✤
✤
b0 b1

Figure 3.3: The XOR module

and the lower level. We abbreviate this module by the picture on the right of
Figure 3.3. It has the property that in each Hamilton path, one of the edges
is traversed, and the other is not.
We put all the pieces together as follows. The graph f (ϕ) has n truth
modules, one for each variable x1 , . . . , xn , connected in series, as on the left
side of Figure 3.4. Call the first node 1 and the last one 2. It also has m
triangles, one for each clause. Every side of a triangle represents a literal in
the clause. If the literal is xi , we connect the side with a XOR module to the
“true” side of the truth module of xi , and if the literal is ¬xi we connect it
to the “false” side. Also, we add an extra node 3. We pairwise connect all
3m nodes of the triangles plus nodes 2 and 3. Finally, we add a node 4 only
connected to node 3. An example of the final graph thus obtained is pictured
in Figure 3.4.
We verify that the construction works. Suppose that the graph f (ϕ)
possesses a Hamilton path. Then its two ends must be 1 and 4. Without
loss of generality suppose it starts in 1. The only way to get to node 2 is
to traverse the series of truth modules. This defines a valuation v of the
variables xi . We claim that v satisfies ϕ. The use of the XOR module ensures
that a side of a triangle is not traversed if and only if the corresponding
literal is true under v. Since no Hamilton path can traverse all edges of a
triangle, every triangle has an untraversed edge, which means that v makes
the corresponding literal true. Hence ϕ is satisfied.
Conversely, suppose that there is a valuation v satisfying ϕ. Then f (ϕ)
has a Hamilton path as follows. First traverse the truth modules according
to v, choosing left if v(xi ) = 1 and right if v(xi ) = 0. Since every clause has a
literal that is true under v, for no triangle we have to traverse all edges. Since
all edges between the nodes of the triangles and nodes 2 and 3 are present, we
easily see from this that we can reach 4 visiting all nodes exactly once.

Corollary 3.4.2. TSP is NP-complete.

Proof. We show that HAMILTON PATH 6pm TSP. Given a graph G with n
nodes, construct an instance f (G) of TSP by letting the cities be the nodes
of G. Let the distance between two cities be 1 if the nodes are connected in
24 Chapter 3. Reductions and completeness

1 4

•3
x1 ❩m ❨
✯U
❳ ❱ L ☛
•
☛☛ ✸✸✸
✸
❙ ☛ ✸✸
✰ ❘ ☛☛☛ ✸✸
P ☛ ✸✸
☛
✰ ◆'☛
☛
☛ ❥ ✐ ❤ ❤ ❣✸3 ✸
✸✸
✱ ☛
☛ ❥ ✸✸
L ❧☛☛ ☛
❦ ✸✸
✱ ♠ ☛☛ ✸

♣ ✲
✲
♣ ♦
♥ ♠
• ①
✇; •
q ①
rx
❝ ❵ ✳❪ ❨ ❯ ③ ②
x2 ❤s ❣ ❡ ❞ P
L ④ L
④ ❉
✴
L ⑤ ❁
✴
⑦
✻
⑧ ✴ ✶
✵ ✲

✁
✂
✵
✶
✠✠
•
✠ ✺✺✺ ✭
✯
✺✺ ✬
✄ ✶ ✠✠✠ ✺✺ ✫
☎
✠✠ ✺✺
x3 ✺Z ✠
✷✠ ✪✺
✺✺
✠✠ ✺✺
✽ ✠✠ ✺✺
✠ ✺
✠✠
• •
❂
❇● q
t:
▲
P ❯L
❤ ❦ ♥
❪ ❛ ❞

•2


Figure 3.4: Graph for the formula x1 ∨ ¬x2 ∨ x3 ∧ ¬x1 ∨ x2 ∨ x3 . The
upper triangle is for the first clause, the lower for the second. Also, every
pair of black dots is connected.

G, and 2 otherwise. Then G has a Hamilton path if and only if there is a

tour of length at most (n − 1) · 1 + 2 = n + 1.

3.5 Exercises
Exercise 3.5.1. Prove Proposition 3.1.2.

Exercise 3.5.2. (a) Show that for every L in PSPACE there is an L′ in

SPACE(n) such that L p-m-reduces to L′ . (Hint: define L′ by prefixing the
3.5. Exercises 25

elements of L with strings of polynomial length. This technique is called

padding.)
(b) Suppose that C is closed downwards under p-m-reducibility and that
SPACE(n) ⊆ C ⊆ PSPACE. Show that C = PSPACE.
(c) Show that P 6= SPACE(n) and that NP 6= SPACE(n). (Hint: use (b)
and the space hierarchy theorem.)

Exercise 3.5.3. Use the time hierarchy theorem to show that TIME(nk ) is
not closed downwards under 6pm .

Exercise 3.5.4. Define analogous versions of the set K that are p-m-complete
for co-NP, PSPACE, and EXP.

Exercise 3.5.5. Think about the details of Theorem 3.2.2.

Exercise 3.5.6. Show that P = NP if and only if SAT ∈ P.

Exercise 3.5.7. In a 1956 letter to von Neumann (reprinted in [34]) Gödel

mentioned the following problem: Given a predicate formula ϕ and n ∈ N,
does ϕ have a proof (in a fixed proof system for predicate logic) of length at
most n ? Gödel explicitly mentions the possibility that the problem could
be solvable in quadratic time, as opposed to the exponential bound resulting
from brute force search. Show that the problem is in NP, and that it is
NP-complete by reducing SAT to it.

Exercise 3.5.8. A set A is called self-reducible if A 6pm A via a reduction f

such that f (x) 6= x for all x. Show that SAT is self-reducible.

Exercise 3.5.9. There are two exponential time classes in the literature:
[
E= TIME(2cn )
c
[ c
EXP = TIME(2n ).
c

These are often abbreviated to E = 2linear and EXP = 2polynomial . Show that
EXP is equal to the downward closure of E under 6pm .

Exercise 3.5.10. Let LIN = TIME(n), and let NLIN = NTIME(n). It is

known that LIN 6= NLIN, see e.g. vol. II of [4] for a proof. In this exercise
we prove that P 6= NLIN.

(i) If NLIN ⊆ P then P = NP.

(ii) If P ⊆ NLIN then P 6= NP.

(iii) P 6= NLIN.
26 Chapter 3. Reductions and completeness

Exercise 3.5.11. Finish the proof of Theorem 3.3.3.

Exercise 3.5.12. Prove Theorem 3.3.5, namely that 2SAT is in P. (Hint:
Given ϕ, draw a directed graph of all literals, with an arrow from x to y if
{x̄, y} is a clause of ϕ. Argue that ϕ is satisfiable if and only if there is no
literal x for which there is both a path from x to x̄ and from x̄ to x. Then
use Exercise 2.7.2.)
Exercise 3.5.13. Let kDNF be the set of all satisfiable formulas in disjunc-
tive normal form (i.e. written as a disjunction of conjunctions) where every
conjunct has at most k literals. Show that kDNF is in P for every k.
Exercise 3.5.14. HORNSAT is the subset of SAT-CNF consisting of the
satisfiable Horn formulas, i.e. those where every clause contains at most one
positive literal. Show that HORNSAT is in P. (Hint: Given ϕ, define the
minimal truth assignment T satisfying ϕ. Hence ϕ is satisfiable if and only
if T satisfies ϕ.)
Exercise 3.5.15. Show that one can define the logical constant ⊥ (the for-
mula that is always false) with an nSAT-formula in which every clause con-
tains exactly n variables. Namely, show that there is an nSAT-formula ϕ,
of size polynomial in n, in which every clause contains exactly the variables
x1 , . . . , xn such that ϕ(x1 , . . . , xn ) holds precisely if x̄1 = . . . = x̄n = ⊥.
Exercise 3.5.16. Consider the following variants of nSAT.
nSAT : every clause has at most n literals.
nSAT′ : every clause has exactly n variables.
nSAT′′ : every clause has exactly n literals.
Show that nSAT ≡pm nSAT′ ≡pm nSAT′′ . (Note that by definition clauses are
sets, so that repeating a literal in a clause makes no difference to the number
of elements in it. Instead, use Exercise 3.5.15.)
Exercise 3.5.17. Check the claim from Theorem 3.4.1 that there are only two
possible ways for a Hamilton path to traverse the module of Figure 3.3, given
that the path enters in either a0 or b0 and the internal nodes of the module
(i.e. those different from the end points of the module) are not connected to
any nodes outside the module.
Exercise 3.5.18. Define the problem HAMILTON CIRCUIT by:
Input: a graph G,
Question: does G have a cycle, starting and returning in the same node
and visiting every other node exactly once?
Define the variant H by:
Input: graph G and two nodes x0 , x1 ∈ G,
Question: does G have a Hamilton path starting in x0 and ending in x1 ?
Show that the problems HAMILTON PATH, HAMILTON CIRCUIT, and H
all have the same p-m-degree.
Chapter 4

Relativized computation and the

polynomial hierarchy

4.1 Relativized computation

One of the central notions in computability theory is Turing’s notion of rela-
tive computability, defined using oracle Turing machines. Such a machine has
an extra tape on which the (infinitely many) bits of a oracle set can be writ-
ten, and that can be used during the computation. The information coded
by this oracle set can be thought of as given for free, and the sets computable
in this fashion as being computable relative to the oracle set. The resource
bounded version of this notion is defined as follows. The classical formulation
with oracle tapes is not entirely appropriate in this setting, since running
up and down the oracle tape takes too long. Instead, we extend our Turing
machines with a query tape and query states. The machine can write queries
to the oracle on the query tape, and when the machine is in a query state
the answer to the query is received instantaneously, so that the queries to the
oracle count as a single computation step.

Definition 4.1.1. Given an oracle machine M , we let M B denote the func-

tion computed by M using oracle B. A set A Turing reduces to B in poly-
nomial time, denoted A 6pT B, if A = M B for some machine M working in
polynomial time.

Note that we have that

A 6pm B =⇒ A 6pT B.

Turing reducibility is much more liberal than many-one reducibility: To an-

swer questions of the form “x ∈ A?” we not only can query the oracle B more
than once, but the queries can also be adaptive, i.e. depend on the answer to
previous queries. By diagonalization, one can show that 6pT differs from 6pm

27
28 Chapter 4. Relativized computation and the polynomial hierarchy

on the computable sets, or even EXP (see Theorem 5.4.1). It is open if the
two notions are different on NP. Of course, if P = NP they are the same on
NP. By Exercise 4.4.1, NP is closed under 6pT if and only if NP = co-NP.
Hence if NP 6= co-NP then 6pT and 6pm differ on NP.
The notion of reducibility 6pT satisfies the same properties as those of 6pm
listed in Proposition 3.1.2, except that now we have in addition

A 6pT B =⇒ A 6pT B.

Again, A ⊕ B is a least upper bound operator. We also  have an equivalence

relation ≡pT induced by 6pT , and the equivalence class B : A ≡pT B is called
the p-T-degree of A.
We employ the following notation in the presence of oracles:

• PA = P(A) = B : B 6pT A ,

• NPA = NP(A) = B : B = M A for some nondeterministic M ,
S C
• PC = P(C) = P :C∈C ,
S C
• NPC = NP(C) = NP : C ∈ C .
For any class defined in terms of Turing machines, such as co-NP or PSPACE,
we have similar definitions.
Proposition 4.1.2. NP(PC ) = NP(C).
Proof. (⊇) Obvious since C ⊆ PC .
(⊆) Suppose B = M0A , where M0 is a nondeterministic machine, and A = M1C
for a C ∈ C and a deterministic machine M1 . Nondeterministically compute
B as follows: guess a computation path of M0 . For every query to A, insert
the equivalent computation of M1C . The resulting path still has polynomial
length.

4.2 The Polynomial Hierarchy

The polynomial time hierarchy, or simply the polynomial hierarchy, was in-
troduced by Stockmeyer in 1977, and is the direct analogue of Kleene’s arith-
metical hierarchy from computability theory. In the analogy P corresponds
to the computable sets and NP to the computably enumerable sets. The
analogy only goes so far. For starters, we do not know whether P 6= NP.
Also, a set A is computable precisely if both A and its complement A are c.e.
In contrast with this, it is believed that NP ∩ co-NP 6= P.
Definition 4.2.1. (The Polynomial Hierarchy) We inductively define a hier-
archy as follows.
4.2. The Polynomial Hierarchy 29

..
.

∆pn+1
① ❋❋
①① ❋❋
①① ❋❋
①① ❋
Σpn ● Πpn
●● ✇✇
●● ✇✇
●● ✇
✇✇
∆pn

..
.

∆p3 ❋
①① ❋❋
①① ❋❋
① ❋❋
①①
Σp2 ❋ Πp2
❋❋ ①
❋❋ ①①
❋❋ ①①①
①
∆p2 ❋
①① ❋❋
①① ❋❋
① ❋❋
①①
NP = Σp1 ● Πp1 = co-NP
●● ✇
●● ✇✇
●● ✇✇✇
✇
P

Figure 4.1: The polynomial hierarchy

• Σp0 = Πp0 = P.

• Σpn+1 = NP(Σpn ).

• Πpn+1 = co-Σpn+1 .

• ∆pn+1 = P(Σpn ).
S
• PH = n>0 Σpn .

The inclusions Σpn ⊆ ∆pn+1 ⊆ Σpn+1 and Πpn ⊆ ∆pn+1 ⊆ Πpn+1 are immediate
from the definition. Figure 4.1 is a picture of the levels of this hierarchy and
their inclusions.

Proposition 4.2.2. (i) ∆p1 = P.

(ii) Πpn+1 = (co-NP)(Σpn ).

30 Chapter 4. Relativized computation and the polynomial hierarchy

(iii) Σpn+1 = NP(Πpn ).

(iv) ∆pn+1 = P(Πpn ).

(v) Σpn+1 = NP(∆pn+1 ).

Proof. Exercise 4.4.5.

The levels of the arithmetical hierarchy can be defined using quantification

over previous levels. This is also true for the polynomial time hierarchy. The
analogue of classical quantification is bounded quantification.

Definition 4.2.3. ∃p(n) x R(x) denotes that there is a string x with |x| 6 p(n)
with property R. If no confusion can arise we will simply write ∃p . ∀p(n) x R(x)
of course means that for all x with |x| 6 p(n) property R holds. For a given
class C, ∃C denotes the class of sets of the form

x : ∃p(|x|) y (x, y) ∈ B

where p can be any polynomial and B ∈ C. The class ∀C is defined analo-

gously.

Proposition 4.2.4. (i) ∃P = NP, ∀P = co-NP.

(ii) ∃Σpn = Σpn , ∀Πpn = Πpn , for n > 0.

(iii) ∃Πpn = Σpn+1 , ∀Σpn = Πpn+1 .

Proof. Exercise 4.4.6.

It follows from Proposition 4.2.4 that a set is in Σpn precisely if it can be

defined by a formula with n quantifier alternations, starting with ∃. Similarly
for Πpn , but now the first quantifier is ∀.
In contrast to the situation in computability theory, we do not know
whether the polynomial time hierarchy is proper, since we do not even know
whether P = NP. Of course, when P = NP then the whole hierarchy collapses
to P, as we invite the reader to check. The conjecture is that PH does not
collapse, i.e. that Σpn 6= Σpn+1 for all n. However, it is also possible that
P 6= NP and that the hierarchy collapses to some higher level. Whatever is
in the hierarchy, it takes place within PSPACE:

Proposition 4.2.5. PH ⊆ PSPACE.

Proof. The proof is by induction on the levels Σpn of PH.

n = 0: By definition, Σp0 = P ⊆ PSPACE.
n + 1: By induction hypothesis, Σpn+1 = NP(Σpn ) is included in NP(PSPACE).
But NP(PSPACE) ⊆ PSPACE(PSPACE) = PSPACE.
4.3. Relativization 31

By Proposition 4.2.5 we see that the statement PH = PSPACE expresses

that PH is as large as possible. Amusingly, this implies its collapse, i.e. if PH
is large then it is already large at one of its finite levels:

Proposition 4.2.6. If PH = PSPACE then PH collapses.

Proof. Consider the PSPACE-complete set QBF from Theorem 3.3.6. If

PH = PSPACE then QBF ∈ Σpn for some level n. By completeness of QBF
and Σpn+1 ⊆ PSPACE this implies that all sets Σpn+1 reduce to a set in Σpn .
Since Σpn is downwards closed under p-m-reductions we obtain Σpn+1 ⊆ Σpn .

For every level of PH there are sets that are p-m-complete for that level.
For example, consider the fragment QBFk of QBF consisting of formulas
with k − 1 quantifier alternations, starting with ∃. Stockmeyer and Wrathall
showed that QBFk is Σpk -complete for every k (cf. Exercise 4.4.10.) See [36]
for other examples of sets that are complete for various levels of PH.
There is a whole array of intermediate notions between the two extremes
of p-m-reducibility and p-T-reducibility. There are situations in which 6pm is
too restrictive, and 6pT too liberal. Useful variations such as 1-1, truth-table,
and bounded truth-table reducibility can be found in several of the standard
textbooks.

4.3 Relativization
We have seen that the classes P and NP have relativized versions PA and NPA
for any oracle A. In general we can relativize any class C that is defined in
terms of Turing machines, simply by adding A as an oracle. Given an oracle
A, the relativized class is denoted by C A .
Often a result about complexity classes holds in fact for all the relativized
versions of the classes as well. In this case we say that the result relativizes.
For example, the strict inclusion P ( EXP relativizes, i.e. PA ( EXPA for
any A. In fact, we can obtain this result by using the same proof as before,
just adding the oracle A in the appropriate places. In this case we say that
the proof relativizes.
Sometimes we need to be more specific about what we mean by rela-
tivization. For example, Theorem 3.2.1 relativizes to: K A is NPA -complete
for every A. But what do we mean by NPA -complete? It could be taken to
mean complete under the unrelativized 6pm , or complete under m-reductions
computed by functions in PA . In this case the stronger version, using unrel-
ativized reductions, holds, but in general one has to be specific about this.
We may not know whether P 6= NP, but we know something about a
possible proof: Whether P = NP or P 6= NP, this cannot be proven using a
proof that relativizes. This will follow from the results in section 5.3.
32 Chapter 4. Relativized computation and the polynomial hierarchy

4.4 Exercises
Exercise 4.4.1. NP is closed under 6pT if and only if NP = co-NP.

Exercise 4.4.2. Show that a class C is countable if and only if there is a set
A that is p-T-hard for C.

Exercise 4.4.3. ⋆ Show that the class COMP of all computable sets does not
have p-m-complete sets. Show the same result for p-T-completeness. (You
can either prove this by direct diagonalization, or by using the fact that there
is no universal computable set.)

Exercise 4.4.4. Explain why we did not define a polynomial space hierarchy.

Exercise 4.4.5. Prove Proposition 4.2.2.

Exercise 4.4.6. Prove Proposition 4.2.4. (Hint: Item (ii) follows by taking
two quantifiers of the same kind together using the pairing function h· , ·i. For
(iii), prove Σpn+1 ⊆ ∃Πpn by induction. Separate the positive and the negative
queries.)

Exercise 4.4.7. Let VALID be the set of all valid propositional formulas.

(i) Show that SAT ≡pm VALID.

(ii) For which level of PH is VALID complete?

Exercise 4.4.8. Prove the following:

• If Σpi = Πpi then PH = Σpi = Πpi .

• If Σpi = Σpi+1 then PH = Σpi .

• If Πpi = Πpi+1 then PH = Πpi .

Exercise 4.4.9. Prove that Σpi (NP) = Σpi+1 for every i.

Exercise 4.4.10. Show that QBFk is Σpk -complete.

Exercise 4.4.11. Show that the following results relativize:

(i) All of section 2.2,

(ii) All of section 2.3.

Exercise 4.4.12. Show that K A is complete for NPA under unrelativized

p-m-reductions.

Exercise 4.4.13. Show that

4.4. Exercises 33

(i) PP = P,

(ii) EXPEXP 6= EXP.

(iii) PSPACEPSPACE = PSPACE.

(iv) ∃PSPACE = PSPACE.

Exercise 4.4.14. Let A be an NP-complete set. Show that


(i) Σp1 = B | B 6pm A ,

(ii) ∆p2 = B | B 6pT A .

Exercise 4.4.15. Positive reductions are a generalization of m-reductions

defined as follows: A 6ppos B if A = M B for a deterministic Turing machine
M working in polynomial time such that

C ⊆ D =⇒ M C ⊆ M D .

Show that NP is downwards closed under 6ppos .

Chapter 5

Diagonalization

Diagonalization is a technique for constructing sets in infinitely many stages.

The name derives from Cantors famous diagonal argument showing that the
reals are uncountable. Since Cantor the method has been extended and re-
fined in numerous ways in the field of mathematical logic. In this chapter
we will see some incarnations of this method in the resource bounded set-
ting. Note that we have already made use of the method in the proofs of the
hierarchy theorems in section 2.3.

5.1 The Halting Problem

As a warm-up, let us consider the proof that the Halting Problem is undecid-
able. Let {Me }e∈N be an effective enumeration of all Turing machines. The
halting set is defined as

H = he, xi : Me (x) ↓ .

Now suppose for a contradiction that H is computable. Then we can define

a machine M such that for all e, M (e) ↓ if and only if Me (e) ↑. Now let d be a
code of M , i.e. M = Md . Then we have Md (d) ↓ if and only if Md (d) ↑. From
this contradiction we conclude that H must be noncomputable.

5.2 Intermediate sets

Are there any sets in NP that are neither in P nor NP-complete? Call such
sets of intermediate degree. Of course, in case P = NP then the class of
NP-complete sets coincides with P, so then there are no intermediate sets. A
priori it could be possible that P 6= NP and that still there are no intermedi-
ate sets. In that case NP would split into the sets in P and the NP-complete
sets. The next result shows that this cannot happen. The proof uses a form

34
5.2. Intermediate sets 35

of the diagonalization method introduced by Ladner, called delayed diagonal-

ization, also known as the looking back technique or wait-and-see arguments.
The result itself is the direct analogue of the Friedberg-Muchnik Theorem in
Computability Theory, which states that there exist c.e. sets of intermediate
Turing-degree [31].
Theorem 5.2.1. (Ladner [27]) If P 6= NP then there is a set B ∈ NP that is
neither in P nor NP-complete.
Proof. We will in fact ensure that B is not p-T-complete, so that in particular
it is not p-m-complete. Fix a set A that is p-m-complete for NP, and suppose
that A ∈ / P (i.e. that P 6= NP). Fix a polynomial time enumeration {Me }e∈N
of all polynomial time machines. We construct B in infinitely many stages,
while trying to satisfy for every e the requirements


R2e : ∃y A(y) 6= MeB (y) ,


R2e+1 : ∃y B(y) 6= Me (y) .
Note that the even requirements ensure that B is not complete, and the odd
requirements that B is not in P. Call y as in Rd a witness for the requirement.
The problem is that to see if y is a witness for R2e we need to compute A,
which is expensive. (Even computing Me (y) is expensive, since the polynomial
varies with e.) We know however that we will succeed if we let B look like a
finite set for long enough. If we wait until we are dealing with larger inputs
(and hence we have more computing time) we can simply look back and see a
point where we succeeded. We deal with R2e+1 in a similar way, now letting
B look like A to keep it out of P.
At stage x we have defined B up to input x. We define B(x) as follows.
First, let s be the largest number such that, by recursively running the con-
struction, we can compute B ↾s in |x| steps. Next we use |x| computation
steps to compute for as many y = 0, 1, 2, . . . as we can whether y is a witness
for some Rd with d < x at stage s. (We say that y is a witness for R2e at
stage s if A(y) 6= MeB ↾s (y), and the computation uses only numbers smaller
than s.) We do this by dovetailing all pairs hy, di in their natural order. Note
that since A is hard to compute, this may not get us very far, but that is o.k.1
Let d be the least number such that no witness for Rd is found. If d = 2e is
even that means we are still working on R2e , so we let B(x) = 0. If d = 2e + 1
is odd that means we are still working on R2e+1 , so we let B(x) = A(x). This
ends the construction of B.
Clearly B ∈ NP by construction. We verify that the construction suc-
ceeds in satisfying all requirements. Suppose for a contradiction that some
1
This is reminiscent of Sterne’s Tristram Shandy, who needs a full year to describe
just one day of his life [40, Vol. 2, Chapter XIII]. Slow as this may be, every day would
eventually be described, if he were to live forever.
36 Chapter 5. Diagonalization

requirement is not satisfied, and let Rd be the least such. Note that if a wit-
ness for Rd is never found, it does not exist. If d = 2e is even then B(x) = 0
for almost every x, and hence, because A(y) = MeB (y) for every y, if follows
that A ∈ P, contrary to assumption. If d = 2e + 1 is odd then B(x) = A(x)
for almost every x, and B(y) = Me (y) for every y, hence we have again that
A ∈ P, contrary to assumption.
Since we do not know that P 6= NP, of course we cannot prove at the moment
of any particular set A in NP that it has intermediate degree. However, there
are natural candidates for such sets. The most famous one is the Graph
Isomorphism problem GI defined by:
Input: two graphs G0 and G1 ,
Question: are G0 and G1 isomorphic?
Obviously, this problem is in NP (guess an isomorphism). Up to now it has
resisted any attempt to prove that it is in P or that it is NP-complete. Further
evidence for the fact that GI is not NP-complete is the fact that if it were
NP-complete, then Σp2 = Πp2 (see Theorem 9.4.2), and hence the polynomial
hierarchy would collapse to the second level.
According to Theorem 5.2.1 there are at least three p-m-degrees inside
NP, provided of course that P 6= NP. In fact, in this case the p-m-degrees
are dense (cf. Exercise 5.8.1), so that in particular there are infinitely many
p-m-degrees inside NP. (This is the analog of the Sacks density theorem
from computability theory.) Moreover, by Exercise 5.8.3 these degrees are
not linearly ordered. Much more is known about this world of intermediate
degrees that falls outside of the scope of these notes. In a precise sense, the
structure of intermediate degrees is as complicated as possible.

5.3 Oracle separations

A central theme in computability theory is the study of relativized computa-
tion, that is, the way sets behave in the presence of oracles. We will use the
following basic fact about oracle computations.
Proposition 5.3.1. (Use Principle) Suppose that B is such that for every
query q in the oracle computation M A (x) we have B(q) = A(q). Then
M A (x) = M B (x).
Proof. The oracle B gives the same answers as oracle A, hence the computa-
tions are the same.
Theorem 5.3.2. (Baker, Gill, and Solovay [3]) There exists a computable
set A such that PA = NPA .
Proof. Let A be any set that is p-m-complete for PSPACE, such as QBF.
Then we have
PA ⊆ NPA ⊆ PSPACEA ⊆ PSPACE ⊆ PA .
5.3. Oracle separations 37

Theorem 5.3.3. (Baker, Gill, and Solovay [3]) There exists a computable
6 NPA .
set A such that PA =

Proof. For any set A the set


B = 0n : ∃x ∈ {0, 1}n (x ∈ A)

is in NPA . We want to construct A such that B ∈ / PA . As in a typical

diagonalization construction, we split this global requirement into infinitely
many subrequirements Re . We construct S A in stages s. At stage s we have
defined a finite part As of the set A = s∈N As . Let Me be an enumeration
of all polynomial time p-T-reductions, and suppose that Me works in time pe .
We want to satisfy for every e the requirement
Re : MeA 6= B.

Requirement Re guarantees that the eth p-T-reduction to A does not compute

B, hence all the requirements together guarantee that B is not in PA . We
satisfy Re by ensuring that for some n, MeA (0n ) 6= B(0n ). We use the fact that
the polynomial time computation MeA can only make a polynomial number
of queries to the oracle A. Changing the oracle outside the set of queries does
not change the computation, and this gives us an opportunity to diagonalize.
At stage s = e, pick a fresh number n, that is, a number larger than any
number used so far in the construction, and also ensure that 2n > pe (n).
Case 1. Suppose that MeAs (0n ) = 1. Then we can satisfy Re by keeping
A ∩ {0, 1}n = ∅, so that 0n ∈/ B.
Case 2. Otherwise we have MeAs (0n ) = 0. (W.l.o.g. MeX is total and
0-1-valued for all X.) Now we want that 0n ∈ B, that is, we have to put
something of length n into A. There are 2n strings of length n, and the
computation MeAs (0n ) can make at most pe (n) queries to As , so there must
be a string x ∈ {0, 1}n that is not queried. Hence, adding x to As does not
change the computation. Also, since n was fresh, adding x does not change
any computation from a previous stage by the Use Principle 5.3.1. So in this
case, setting
As+1 = As ∪ {x}

satisfies Re , without destroying the satisfaction of previous requirements Rd

with d < e.

As pointed out in section 4.3, Theorems 5.3.2 and 5.3.3 put methodological
limits on the possible ways we can prove P = NP or P 6= NP.
38 Chapter 5. Diagonalization

5.4 Many-one versus Turing reductions

The definition of p-T-reduction seems much more liberal than that of p-m-
reduction. Let us prove that they are indeed different.

Theorem 5.4.1. (Ladner, Lynch, and Selman [28]) The relations 6pm and
6pT differ on EXP.

Proof. This is in fact an easier diagonalization, exploiting the fact that a p-

m-reduction is only allowed a single query. We want to construct sets A and
B such that A 6pT B but A 66pm B. Clearly the former is satisfied if we let
n ∈ A if and only if either 2n ∈ B or 2n + 1 ∈ B. Let fe be an efficient
enumeration of all polynomial time p-m-reductions. We want to satisfy for
every e the requirement
Re : ∃n A(n) 6= B(fe (n)),
that is, A does not p-m-reduceS to B via fe , Sas witnessed by the counterex-
ample n. We construct A = s As and B = s Bs in stages. At stage s = e
of the construction we are given finite sets As and Bs , and we have already
satisfied Rd for all d < e. To satisfy Re we pick a fresh n, i.e. an n bigger
than any number used so far in the construction, and we compute fe (n). If
fe (n) ∈ Bs then we let n ∈/ A. In this case we should also have 2n ∈ / B and
2n + 1 ∈ / B, which is indeed the case since n is fresh, so Bs does not contain
any elements bigger than n. If fe (n) ∈ / Bs then we let n ∈ As+1 , and we
also choose one of 2n and 2n + 1 that is different from fe (n) and put it into
Bs+1 . This clearly satisfies Re , without affecting the satisfaction of earlier
requirements since n was picked fresh.
To ensure that A and B are in EXP, we can easily modify the construction
by considering at stage s only n such that we can check in time 2n that n
is fresh. Such n exist because the fe work in polynomial time. This extra
condition ensures that A and B are in TIME(2n ).

Do these two fundamental reduction notions differ on NP? To prove this, it

seems one needs an assumption stronger than just P 6= NP. E.g. the answer
is yes if there is a sparse set in NP − P. (See for example [31, p223,254].)

5.5 Sparse sets

Definition 5.5.1. A set A is sparse if there is a polynomial p such that for
all n, |A ∩ {0, 1}6n | 6 p(n). Here {0, 1}6n denotes the set of all strings of
length at most n.

Definition 5.5.2. A polynomial time computable function f : {0, 1}∗ →

{0, 1}∗ is a p-isomorphism if f is bijective and has a polynomial time com-
5.5. Sparse sets 39

putable inverse. Two sets A and B are p-isomorphic if there is a p-isomor-

phism f such that f (A) = B.

Berman-Hartmanis Conjecture All sets that are p-m-complete for NP are

p-isomorphic.

This conjecture is the analog of the fact that all m-complete c.e. sets are
computably isomorphic.2
The Berman-Hartmanis conjecture implies that all NP-complete sets have
the same density. Indeed, the following is known:

Theorem 5.5.3. (Mahaney [29] If there is a sparse p-m-complete set for NP,
then P = NP.

Proof. We give a proof based on the exposition in Odifreddi [31, p206]. Recall
the definition of the NP-complete set K on page 18:

hM, x, 1t i ∈ K ⇐⇒ M accepts x in 6 t steps.

Without loss of generality, we can think of the computation paths of M as

strings y ∈ {0, 1}t .
Suppose that S is a sparse p-m-complete set for NP. We prove that K
is in P, and hence that P = NP. Given hM, x, 1t i, we use S to search for
an accepting path y in polynomially many steps. First define the following
modification of K:

hM, x, 1t , yi ∈ B ⇐⇒ M accepts x in 6 t steps, with an accepting path 6 y.

Note that a string y ∈ {0, 1}t is also a number in [0, 2t ]. Now B ∈ NP, and
hence B 6pm S by the completeness of S. Suppose that

hM, x, 1t , yi ∈ B ⇐⇒ f (M, x, 1t , y) ∈ S

and that f is computable in time q for some polynomial q. In particular

|f (M, x, 1t , y)| 6 q(hM, x, 1t , yi). As S is sparse, there is a polynomial p such
that |S ∩{0, 1}6n | 6 p(n) for every n. So, given hM, x, 1t i, S contains at most
p(q(hM, x, 1t , 2t i)) elements of the form f (M, x, 1t , y) with |y| 6 t. We narrow
down the search for an accepting y as follows. Let m = p(q(hM, x, 1t , 2t i))+1.
Divide [0, 2t ] into m + 1 intervals of equal length:

0 < y1 < . . . < ym < 2t .

2
This statement follows from the following two facts: (i) A c.e. set is m-complete if and
only if it is 1-complete, cf. Soare [38, p43]. (ii) The Myhill Isomorphism Theorem, that
says that two sets have the same 1-degree if and only if they are computably isomorphic,
cf. Soare [38, p24]. This in turn is an effective version of the Cantor-Schröder-Bernstein
theorem from set theory.
40 Chapter 5. Diagonalization

Case 1. There exist i < j such that f (M, x, 1t , yi ) = f (M, x, 1t , yj ). In

this case there is an accepting path 6 yi if and only if there is an accepting
path 6 yj , so we can delete the interval (yi , yj ] from our search.
Case 2. For all i 6= j we have f (M, x, 1t , yi ) 6= f (M, x, 1t , yj ). Then
f (M, x, 1t , yi ) = 0 for some i, as S contains at most m − 1 elements of this
form. So in this case we can delete the interval [0, yi ] from our search.
Now we repeat the procedure, i.e. we divide the set of remaining paths
into m + 1 intervals of equal size by choosing new y1 , . . . , ym , and repeat. At
every step we remove at least one of the m + 1 intervals, i.e. we reduce the
1
number of possible paths y by multiplying with a factor (1 − m+1 ). When we
do this k times, where k is so large that
 k
1
1− · 2t 6 1, (5.1)
m+1

then our search has become trivial, and hence we can check the existence of
an accepting path. A straightforward computation, using ex > 1 + x, shows
that (5.1) is satisfied if k > t(m + 1), which is polynomial in the input size
hM, x, 1t i.
In Exercise 5.8.6 we will prove that there are no sparse p-m-complete sets
for EXP. It follows from the results in sections 7.4 and 7.5 that NP has no
sparse p-T-complete sets, unless PH collapses.

5.6 The Gap Theorem

By the time hierarchy theorem, for time constructible functions f we have
TIME(f ) 6= TIME(2f ). The next result shows that the assumption that f is
time constructible is essential for this result.

Theorem 5.6.1. (Gap Theorem, Trakhtenbrot, Borodin) There exists a

computable function f such that TIME(f (n)) = TIME(2f (n) ).

Proof. We construct f in stages, satisfying for every e the requirement

Re : / (f (n), 2f (n) ] .
∀x |x| = n > e ∧ Me (x) ↓ in t steps =⇒ t ∈

This suffices to prove the theorem: Suppose that A ∈ TIME(2f (n) ) as wit-
nessed by Me . Then for every x, Me (x) ↓ within 2f (n) steps, hence by Re ,
for every x of length > e, Me (x) ↓ within f (n) steps. Since the finitely many
exceptions do not matter, we have A ∈ TIME(f (n)).
At stage n we define f (n) as follows. We consider all computations Me (x)
with e 6 n and |x| = n. Define the sequence k0 = 0, kl+1 = 2kl . Call kl
wrong for a computation Me (x) if Me (x) ↓ in t steps and t ∈ (kl , 2kl ]. Note
that we can compute whether kl is wrong for Me (x) by running Me (x) for 2kl
5.7. The Speed-Up Theorem 41

steps. Note further that there can be at most one kl wrong for Me (x), since
the intervals are disjoint. Since at stage n we consider only finitely many
computations Me (x) it follows that there is a kl that is not wrong for any of
these. Define f (n) = kl for the least such kl .
Now suppose that |x| > e. Then at stage n = |x| the computation Me (x)
is taken into consideration, and hence Re is satisfied.

5.7 The Speed-Up Theorem

We often talk about “the” complexity of a computational problem. We al-
ready know that this use of language is not entirely appropriate because of
the linear speed-up theorem: Any program computing a function can be sped
up by any linear factor on almost all of its arguments. We can welcome this
situation by interpreting it as that we can be sloppy about constants, as we
in fact did. But the situation is in fact much worse than this. In this section
we show that there are computable problems A such that any program for
A can be sped up exponentially. This result challenges the idea that every
problem has a well-defined level of complexity.
The result in fact holds for any measure of complexity, but we will prove it
here for time complexity. Also, the result holds in more generality: Instead of
an exponential speed-up one can use any fast growing computable function,
cf. Exercise 5.8.12. The idea of speed-up goes back to Gödel [17], who proved
a speed-up theorem about the length of proofs in arithmetic.
As before in Chapter 2, fix an enumeration {Me }e∈N of all deterministic
Turing machines. In the following we will identify Me with the language
computed by it. We call e an index for the language Me . Let Φe (x) denote
the number of computation steps of Me (x) if this computation converges,
and let Φe (x) be undefined otherwise. Note that in general we cannot decide
whether Me (x) converges, hence neither whether Φe (x) is defined, but given t
we can decide whether Φe (x) < t by running Me for t steps. We will use this
fact in the proof.

Theorem 5.7.1. (Speed-Up Theorem, M. Blum [6]) There exists a com-

putable set A such that for every index e for A there is another index i for A
such that

∀∞ x Φi (x) 6 log Φe (x) . (5.2)
That is, the program Mi computes A exponentially faster than Me .

Proof. Let g(x) = 2x , and define g (1) (x) = g(x) and g (n+1) (x) = g(g (n) (x))
for n > 1. So g (n) (x) is a stack of iterated exponentials of height n. Also, let
g (n) (x) = 0 if n 6 0. First we define a scale of functions as follows. Let

he (x) = g (x−e) (0).

42 Chapter 5. Diagonalization

Then g(he+1 (x)) = he (x) for all x > e+1, so the functions he form a decreasing
family of functions, with an exponential gap between every two of them.
We construct a set A such that


(I) ∀e Me = A =⇒ ∀∞ x he (x) 6 Φe (x) ,


(II) ∀e∃i Mi = A ∧ ∀∞ x Φi (x) 6 he+1 (x) .
Note that (I) and (II) together imply (5.2):

Φi (x) 6 he+1 (x) 6 log he (x) 6 log Φe (x).

Construction. To obtain (I), we satisfy for every e the requirement

Re : ∃∞ x Φe (x) < he (x) =⇒ Me 6= A.
At stage x of the construction we define A(x) as follows. Say that e requires
attention at stage x if e 6 x, e is not yet cancelled, and Φe (x) < he (x). At
stage x, choose the least e that requires attention, define A(x) = 1 − Me (x),
and declare e cancelled. If there is no such e we give A(x) some default value,
say A(x) = 0. This ends the construction of A.
We verify that the construction satisfies all requirements. Suppose that
the premiss of Re holds, that is, there are infinitely many x such that Φe (x) <
he (x). Since there are only finitely many d < e that can require attention
at a stage x, and every time this happens some d < e is cancelled, there are
only finitely many stages at which e requires attention but is not the least
such number. At the first stage following this where e requires attention
(which exists by assumption), e is the least such number and it is ensured
that A 6= Me . Hence Re is satisfied.
It remains to verify (II). Clearly the construction above gives an algorithm
for computing A. We want to compute A in a more efficient way. Fix u ∈ N.
Suppose we know the finite set

Fu = (e, x, A(x)) : e < u ∧ e cancelled at stage x .

Then we can compute the set A as follows. We run the above construction,
but instead of considering all e 6 x at stage x we only consider e with
u 6 e 6 x. The full information of what happened at stage x in the original
construction can be recovered from the finite set Fu . The point of this is that
we save having to run the functions he (x) for e < u. Instead, we directly
find out which, if any, of these e required attention by consulting Fu . To see
which e with u 6 e 6 x requires attention we only have to compute he (x) for
e > u. Hence the time to compute A(x) is bounded by

x · hu (x) + O(1) 6 hu−1 (x) a.e. x.

This proves (II), because we may take u as large as we want.

5.8. Exercises 43

5.8 Exercises
Exercise 5.8.1. ⋆ Use delayed diagonalization to prove that the p-m-degrees
of the computable sets are dense, that is, if A and C are computable and
C <pm A then there is a computable B such that C <pm B <pm A.
Exercise 5.8.2. ⋆ Same as Exercise 5.8.1, but now for 6pT instead of 6pm .
Exercise 5.8.3. ⋆ Assuming that NP 6= P, show that there are incomparable
p-T-degrees in NP, that is, show that there are sets A and B in NP such
that neither A 6pT B nor B 6pT A. Note that it automatically follows that
A and B are intermediate sets. This result is the analogue of the Friedberg-
Muchnik theorem in computability theory that states that there are Turing-
incomparable c.e. sets.
Exercise 5.8.4. A set A is tally if A ⊆ {0}∗ . Show that for every sparse set
S there is a tally set T such that S 6pT T . Hint: consider

T = 0hn,j,ii : the i-th bit of the j-th string in S =n is 1 .
Exercise 5.8.5. Show that the Berman-Hartmanis conjecture implies that
P 6= NP.
Exercise 5.8.6. Let {fi }i∈N be an effective enumeration of all p-m-reduc-
tions. Call f almost injective if f (x) = f (y) for only finitely many pairs x, y
with x 6= y.
(a) Show that there is a set B ∈ EXP such that for any set A and any i,
if fi is not almost injective then B does not p-m-reduce to A via fi .
(b) Show that there exists B as in (a) such that |B ∩ {0, 1}6n | > 2n .
(c) (Berman) Show that there are no sparse p-m-complete (or even hard)
sets for EXP. (See also Theorem 7.5.2.)
Exercise 5.8.7. Use diagonalization to explicitly construct a computable
function that is not time constructible.
Exercise 5.8.8. Show that the function f from Theorem 5.6.1 is bounded
by a time constructible function.
Exercise 5.8.9. Prove the analog of the Gap Theorem for space complexity.
Exercise 5.8.10. Prove that the gap in Theorem 5.6.1 can be made arbi-
trarily large by using, instead of 2n , an arbitrary computable function.
Exercise 5.8.11. Given any nondecreasing computable function g, with at
least g(x) > x, construct a g-scale, that is, a descending family of functions
{he }e∈N such that
g(he+1 (x)) 6 he (x)
for every x.
44 Chapter 5. Diagonalization

Exercise 5.8.12. Let g be an arbitrary fast growing computable function

with g(x) > x. Formulate and prove a general version of the speed-up theo-
rem, using g instead of 2x .
Chapter 6

Randomized computation

6.1 Probabilistic classes

In this section we consider variants of NP where the acceptance criterion is
given by a probabilistic condition over all paths. Consider nondeterministic
Turing machines M whose computation trees are full binary trees, and in
which every computation path has the same length. Given a set L and a ma-
chine M that is supposed to compute L, and an input x, the error probability
is the ratio of the number of paths giving the wrong answer and the total
number of paths. When thinking of Turing machines in this way we speak
also of probabilistic Turing machines. All the probabilistic classes considered
in this chapter were introduced by Gill [16].
Definition 6.1.1. The class PP, for probabilistic polynomial time, is the class
of sets L that are computed in polynomial time by a probabilistic machine
such that
x ∈ L ⇔ the fraction of accepting paths is > 21 ,
x∈/ L ⇔ the fraction of rejecting paths is > 21 .
By Exercise 6.5.1, we can replace the second clause in the definition by
/ L ⇒ the fraction of rejecting paths is > 21 .
x∈
Example 6.1.2. Let MAJ be the subset of SAT consisting of the Boolean
formulas that are satisfied by more than 21 of all assignments. Then MAJ is
in PP.
Proposition 6.1.3. NP ⊆ PP ⊆ PSPACE.
Proof. PP ⊆ PSPACE: Try all paths, reusing the same space and keep a
counter for the paths used.
NP ⊆ PP: We can convert any NP-computation into a PP-computation
as in Figure 6.1, where on the right is the original NP-computation and on
the left we add a tree of the same size where all paths accept.

45
46 Chapter 6. Randomized computation

❧❙❙
❧❧❧❧❧ ❙❙❙❙❙❙
❧ ❙❙❙
❧❧❧ ❙❙❙
❧ ❧ ❧❧❧ ❙❙❙
❧ ❧ ❧ ❙❙❙
✶
❧ ❧ ❙✶
✌✌ ✶✶ ✌✌ ✶✶✶
✌ ✶✶✶ ✌ ✶✶
✌✌✌ ✶ ✌✌✌ ✶✶
✌✌ ✶✶ ✌✌ ✶✶
✌✌ ✶✶✶ ✌✌ ✶✶
✌✌ ✶✶ ✌✌ ✶✶
✌✌ ✶✶ ✌✌ ✶✶
✌✌ ✶✶ ✌✌ ✶✶
✌✌ ✶✶ ✌✌ ✶✶
✌✌ ✌✌
1111111111111111

Figure 6.1: Converting an NP into a PP computation

It follows from Exercise 6.5.1 that PP is closed under complementation, hence

also co-NP ⊆ PP. The closure of PP under unions and intersections was open
for a long time before it was finally solved in [5].1 By Exercise 6.5.2, PP is
closed under p-m-reductions.
Like NP, the class PP has sets that are complete under p-m-reductions.
Consider the set

SA = hϕ, ii : ϕ is satisfied by > i assignments .

Theorem 6.1.4. The sets MAJ and SA are PP-complete.

Proof. We already noticed in Example 6.1.2 that MAJ is in PP. We first show
that SA 6pm MAJ. Then SA ∈ PP follows from the fact that PP is closed
under p-m-reductions (Exercise 6.5.2). Let hϕ, ii be an instance of SA, where
ϕ has m variables. Note that we must have that 2m > i. Let ψ be a formula
on the same m variables as ϕ with exactly 2m −i satisfying assignments. (One
can effectively construct such a formula, cf. Exercise 6.5.3.) Now define
f (hϕ, ii) = (y ∧ ϕ) ∨ (¬y ∧ ψ),
where y is a new variable. Then ϕ has > i satisfying assignments if and
only if f (hϕ, ii) has > i + 2m − i = 2m satisfying assignments. Note that
2m = 12 2m+1 , and that m + 1 is the number of variables in f (hϕ, ii). Hence f
is a p-m-reduction from SA to MAJ.
To prove completeness of SA, use the same reduction x 7→ ϕx as for the
NP-completeness of SAT (Theorem 3.2.2). Given A ∈ PP computed in time
p(n), x is accepted if and only if > 2p(n)−1 +1 paths accept x. Hence A 6pm SA
via x 7→ hϕx , 2p(n)−1 i.
1
Key to the proof is to approximate the sign of the function #acc − #rej, the difference
of the number of accepting and rejecting paths, by suitable polynomials. This can only be
done for each length separately.
6.1. Probabilistic classes 47

Definition 6.1.5. The class BPP, for bounded probabilistic polynomial time,
is the class of sets L that are recognized in polynomial time by probabilistic
machines with error probability bounded away from 21 , i.e. such that for some
ε > 0 and every x,

x ∈ L ⇔ the fraction of accepting paths is > 12 + ε,

x∈/ L ⇔ the fraction of rejecting paths is > 12 + ε.

Obviously P ⊆ BPP ⊆ PP. Note that the relation between BPP and NP
is not clear. The interest in the class BPP comes from the following result,
which implies that the answers given by BPP-machines can be boosted to
any precision arbitrarily close to complete correctness.

Theorem 6.1.6. A ∈ BPP if and only if for all polynomials p there is a prob-
abilistic Turing machine recognizing A in polynomial time with error proba-
1
bility 6 2p(n) .

Proof. Suppose M is a BPP-machine computing A. We simply iterate the

computations of M a polynomial number of times and take the majority of
the answers. A straightforward computation shows that this will suffice to
obtain correct answers with a high probability.2
Suppose M recognizes A with error probability ε < 12 . Let δ = 1 − ε, and
fix a polynomial p. Let q(n) = cp(n), where c is a constant such that

c 1
4εδ < . (6.1)
2

(Note that εδ < 14 because ( 21 + ρ)( 12 − ρ) < 14 for every ρ > 0.) Now, given
an input x of length n, iterate the nondeterministic computation M (x) m =
2q(n) + 1 times, and accept only when at least q(n) + 1 of the computations
accept. The probability that this procedure gives a wrong answer is the
probability that at most q(n) computations are correct, which is

q(n)  
X m
δ j εm−j . (6.2)
j=0
j

m m m
Since δ > ε and j 6 2
, we have δ j εm−j 6 δ 2 ε 2 so the expression (6.2) is
less or equal to

2
A convenient way to do this is to use Chernoff bounds, see e.g. [32]. The calculation
presented here is more elementary.
48 Chapter 6. Randomized computation

q(n)  
m m
X m m m Pm

m
δ ε 2 2 < δ 2 ε 2 2m (by the binomial formula 2m = j=0 j
)
j=0
j

m2
= 4εδ

cp(n)
6 4εδ ( m2 = cp(n) + 21 )
1
< p(n) (by choice of c)
2

Proposition 6.1.7. BPP is closed under complementation, unions, and in-

tersections.

Proof. Exercise 6.5.4.

6.2 More about BPP

In this section we prove two fundamental results about BPP. We first prove
that BPP has short advice strings, a notion that is related to circuit com-
plexity, cf. Chapter 7. Then we prove that BPP is included in Σp2 , the second
level of the polynomial time hierarchy.

Definition 6.2.1. Let B ∈ P be a fixed set. Given a set A, y is correct advice

for A at length n if
x ∈ A ⇐⇒ hx, yi ∈ B (6.3)
for every x of length n.

One can think of the advice string y as coding a circuit to determine member-
ship in A for all x of a given length. We now prove that the sets in BPP have
(many) short advice strings for each length. Using the circuit terminology
one can phrase this by saying that BPP has small circuits, cf. section 7.3.

Theorem 6.2.2. (Adleman) The following are equivalent.

(i) A ∈ BPP,

(ii) For any polynomial q there exist B ∈ P and a polynomial p such that
for all n, among all strings of length p(n) there are at least
 
p(n) 1
2 1 − q(n) .
2

correct advice strings y for A at length n, as in (6.3).

6.2. More about BPP 49

Proof. (ii)⇒(i) is straightforward: Given B and p and n, randomly pick an

advice string y of length p(n). Then with high probability, A(x) = B(hx, yi).
(i)⇒(ii). Fix A and q. By Theorem 6.1.6 we may assume that there is a

q(n)+n
machine M accepting A with error probability at most 21 . Let p be
p(n)−q(n)−n
the running time of M . Given x of length n, at most 2 paths are
n p(n)−q(n)−n
incorrect. Therefore at most 2 · 2 paths are incorrect for some x,
p(n) p(n)−q(n)
and hence 2 −2 paths are correct for all x, and we can use any of
these as a correct advice string.
Sipser proved that BPP ⊆ PH, which was improved by Gács to the fol-
lowing:
Theorem 6.2.3. BPP ⊆ Σp2 .
Proof. Let L ∈ BPP. By Theorem 6.1.6 we may assume that there is a
machine M accepting L with error probability at most 21n . Let l be the
length of the paths in M . Given a path r ∈ {0, 1}l , denote the outcome of
M along that path by M (x, r), and define

A(x) = r ∈ {0, 1}l : M (x, r) accepts .

We can view {0, 1}l as a vector space of dimension l over the finite  field F2 ,
with addition of vectors modulo 2. For t ∈ {0, 1}l define A(x)+t = r+t : r ∈
A(x) . Thus every t defines a 1-1 map, that we can think of as a translation.
Note that (r + t) + t = r for every r and t. If A(x) is large then there is a
small set of translations of it covering the whole space {0, 1}l :
1


ClaimS1. If |A(x)| > 2l 1 − 2|x| then there exist t1 . . . tl , all of length l, such
l
that i A(x) + ti = {0, 1} .
To prove Claim 1, pick t1 . . . tl at random. (That is, S
we use the probabilistic
method to prove the existence of the t’s.) Let S = i A(x) + ti . For every
r ∈ {0, 1}l we have
l
Y
   
Pr r ∈
/S = Pr r ∈
/ A(x) + ti
i=1
 l
1
6 .
2|x|
(Here Pr refers to the choice of the ti ’s.) Hence
X
Pr [∃r r ∈
/ S] 6 2−|x|l = 2l−|x|l 6 2−|x| ,
r
  1
since l − |x|l 6 −|x| when l, |x| > 2. Hence Pr S = {0, 1}l > 1 − 2|x| . In
particular there exist t1 . . . tl such that S = {0, 1}l . This proves Claim 1.
50 Chapter 6. Randomized computation

1
ClaimS2. If |A(x)| 6 2l 2|x| then there do not exist t1 . . . tl of length l such
l
that i A(x) + ti = {0, 1} .
To prove Claim 2, note that |A(x)+ti | 6 2l−|x| for every i and that l·2l−|x| < 2l
because l is polynomial in |x|. This proves Claim 2.
Now to finish the proof of the theorem, note that it follows from these claims
that we can write
_
x ∈ L ⇔ ∃t1 . . . tl ∈ {0, 1}l ∀r ∈ {0, 1}l M (x, r + ti ) accepts
16i6l

and that this is a Σp2 -formula, with polynomially bounded quantifiers.

Note that it follows immediately from Theorem 6.2.3 that BPP ⊆ Σp2 ∩ Πp2
because BPP is closed under complementation (Proposition 6.1.7). In con-
trast with PP, there are no problems known to be complete for BPP. It has
been conjectured that P = BPP, and of course this statement is equivalent
to the statement that all the problems in P are complete for BPP.

6.3 The classes RP and ZPP

Define the following probabilistic classes:

Definition 6.3.1. The class RP, for randomized polynomial time, is the class
of sets L that are recognized in polynomial time by probabilistic machines
with one-sided error probability 21 , i.e. such that

x ∈ L ⇔ the fraction of accepting paths is > 21 ,

x∈/ L ⇔ the fraction of rejecting paths is 1.

By Exercise 6.5.7 the class RP remains the same if we replace the > 21 in its
definition by > 23 . It follows from this that RP ⊆ BPP. RP-algorithms are
also known as Monte Carlo algorithms. Note that we have P ⊆ RP ⊆ NP
(Exercise 6.5.6).
The class ZPP, for zero-error probabilistic polynomial time, is the class
of sets L that are recognized in polynomial time by probabilistic machines
with three possible outcomes: accept, reject, and ? for “don’t know”. The
requirement is that no path gives a wrong answer, and that the fraction
of paths that outputs ? is at most 13 , so that at least 32 in fact give the
correct answer. ZPP-algorithms are also known as Las Vegas algorithms. By
Exercise 6.5.9 we have ZPP = RP ∩ co-RP. Note that this is a probabilistic
analog of the question whether P = NP ∩ co-NP
Figure 6.2 summarizes some of the relations between complexity classes.
The relation between PP and PH is not clear, but the following is known.
6.4. Primes again 51

P ❖❖ NP∩co-NP NP ▲▲ PH PSPACE
❖❖❖
❖❖❖ ♦♦♦♦♦ ▲▲▲
rrrr ♣♣♣♣
▲▲r▲r ♣
❖❖❖ ♦♦♦ ♣♣
❖❖❖ ♦♦♦♦ rrrrr ▲▲▲▲ ♣♣♣♣
♦♦ r ♣♣
RP BPP PP

Figure 6.2: Relations between complexity classes

In Section 6.1 we discussed closure properties of PP. The ultimate closure

property would be PPP = PP, i.e. the closure of PP under 6pT . If this is the
case then PP in fact contains PH by the following result revealing some of
the strength of randomization.
Theorem 6.3.2. (Toda) PH ⊆ PPP .
Proof. This important theorem can be proved using counting classes, which
is a very interesting topic in itself. A proof can be found in [34].

6.4 Primes again

We saw in section 2.5 that PRIMES is in NP ∩ co-NP. In this section we
show that PRIMES is in co-RP, that is, there is a Monte Carlo algorithm for
testing compositeness. Given a number n, if n is prime this algorithm will
with certainty output the verdict “n is prime”, and if n is composite it will
with high probability output the verdict “n is composite”.
We start with some number-theoretic preliminaries. From now on, p is a
prime larger than 2.
Definition 6.4.1. A number 0 < a < p is called a quadratic residue modulo
p if the equation x2 ≡ a mod p has a solution. The Legendre symbol is defined
as   (
a 1 if a is a quadratic residue modulo p,
=
p −1 otherwise.
  p−1
Theorem 6.4.2. (Euler’s criterion) ap ≡ a 2 mod p.

Proof. Note that since (Z/pZ) is a field, the equation x2 ≡ a can have at
most two solutions modulo p. We claim that there are either two or zero
solutions. By Theorem 1.6.4, pick a primitive root r modulo p. In particular
p−1
there is i < p − 1 such that ri ≡ a mod p. If i = 2j is even then a 2 =
p−1
rj(p−1) ≡ 1 mod p, and x2 ≡ a has the solutions rj and rj+ 2 . Note that if i
varies over the even numbers, we obtain half of all possible residues, and since
each has two square roots, no such roots are left for the odd i. So if i = 2j + 1
p−1 p−1 p−1
is odd, ri has no square roots, and a 2 ≡ rj(p−1) r 2 ≡ r 2 mod p. Since
the latter is a square root of 1 unequal to 1, it must be −1.
52 Chapter 6. Randomized computation

Theorem 6.4.2 allows us to compute the Legendre symbol efficiently. Note

that we can efficiently compute exponentials by repeated squaring, cf. [9] for
more details.
    p−1 q−1
Theorem 6.4.3. (Law of Quadratic Reciprocity) pq · pq = (−1) 2 2 .

Definition 6.4.4. The Jacobi symbol generalizes the Q Legendre symbol as

follows. If n is odd and has prime factorization n = i qi then
m Y m
= .
n i
q i

Lemma 6.4.5. (i) m1nm2 = m1
n
m2
n
.



(ii) m+n
n
= m
n
.
n

m
m−1 n−1
(iii) m · n = (−1) 2 2 .

Proof. The first two items easily follow from the definitions and the third
follows from Theorem 6.4.3.

It follows from Lemma 6.4.5 that also the Jacobi symbol is effectively com-
putable, in a manner similar to that of Euclid’s algorithm for computing the
gcd.

n−1
Lemma 6.4.6. If n is odd and m n
≡ m 2 mod n for all m ∈ (Z/nZ)∗
then n is prime.

Proof. Suppose that the premiss holds but that n is composite. Suppose first
n =p1 . . . pk is the product of distinct primes. Let r ∈ (Z/p1 Z)∗ be such
that 
that pr1 = −1. By the Chinese remainder theorem let m ∈ (Z/nZ)∗ be such
n−1

that m ≡ r mod p1 and m ≡ 1 mod pj for j = 2 . . . k. Then m 2 ≡ m n
≡
n−1
−1 mod n, hence m 2 ≡ −1 mod p2 , contradicting that m ≡ 1 mod p2 .
From the previous contradiction we conclude that n = p2 m for some p > 2
prime. By Exercise 1.7.3,

let r be a primitive root modulo p2 . By assumption
n−1
r
we have r 2 ≡ n ∈ {1, −1} mod n, hence rn−1 ≡ 1 mod n. The latter
implies that n − 1 is a multiple of ϕ(p2 ) = p(p − 1), hence p divides n − 1.
But p also divides n, a contradiction.

Lemma 6.4.7. If n is odd

andn−1
composite, then at least half of the elements
m ∈ (Z/nZ)∗ satisfy m
n
≡
6 m 2 mod n.

Proof. Consider the set


n−1
X = x ∈ (Z/nZ)∗ : x
n
≡x 2 mod n .
6.5. Exercises 53

By Lemma 6.4.5 the elements of X form a subgroup of (Z/nZ)∗ , and by

Lemma 6.4.6 it is not the whole group. If a ∈ (Z/nZ)∗ − X then the left
coset aX has the same cardinality as X, hence X contains at most half the
elements of (Z/nZ)∗ .

Theorem 6.4.8. (Solovay and Strassen [39]) PRIMES is in co-RP.

Proof. Given n odd, the algorithm runs as follows: Randomly pick a number
m with 2 6 m 6 n − 1. If gcd(m, n) 6= 1 then n is composite. (Note that
we can effectively compute
then−1gcd using Euclid’s algorithm.) Otherwise,
∗ m
m ∈ (Z/nZ) . See if n ≡ m 2 mod n. If not, declare n to be composite.
If yes, declare n to be probably prime. Note that if p is prime then by
p−1
m
Theorem 6.4.2, p ≡ m 2 mod p so in the first case we are sure the answer
is correct. If n is composite then by Lemma 6.4.7 the answer is correct with
probability at least 12 .

6.5 Exercises
Exercise 6.5.1. Show that if L ∈ PP then there is a probabilistic machine
/ L ⇒ the fraction of accepting paths is < 12 .
M such that x ∈

Exercise 6.5.2. PP is closed downwards under 6pm . That is, if B ∈ PP and

A 6pm B, then also A ∈ PP.

Exercise 6.5.3. Show that, given a set of n Boolean variables {x1 , . . . , xn }

and i 6 2n , there is a formula in these variables having exactly i satisfying
assignments. Furthermore, the formula is of size a polynomial in n. (Hint:
First do this when i is a power of 2. Then for the general case write i in
binary as a sum of powers of 2 and combine the formulas of the components
by disjunctions, making sure that the assignments do not overlap.)

Exercise 6.5.4. Prove Proposition 6.1.7.

Exercise 6.5.5. Use Theorem 6.1.6 to prove that in Definition 6.1.5 we can
allow ε to be a function of the form 1/p(n), where p is a polynomial and n is
the size of the input.

Exercise 6.5.6. Prove that P ⊆ RP ⊆ NP.

Exercise 6.5.7. Prove that we obtain an equivalent definition of RP if we

replace the > 21 in its definition by > 23 . (Hint: Because the error is one-sided,
we can achieve this by iteration. We cannot use the same argument to prove
that PP ⊆ BPP.) Conclude from this that RP ⊆ BPP.

Exercise 6.5.8. Prove that BPP and RP are downwards closed under 6pm .
54 Chapter 6. Randomized computation

Exercise 6.5.9. Prove that ZPP = RP ∩ co-RP.

Exercise 6.5.10. Show that P ⊆ ZPP ⊆ NP ∩ co-NP.

Exercise 6.5.11. Prove that NP ⊆ BPP if and only if NP = RP. (Hint: One
direction is easy from Exercise 6.5.7. For the nontrivial direction, first try to
find, bit by bit, a certificate for the NP-computation, using the BPP-machine.
Second, use the NP-machine to check whether the certificate is correct.) See
also Exercise 7.6.9.
Chapter 7

Circuit complexity

7.1 Boolean functions

We already discussed in Exercise 1.7.2 that every propositional formula can be
put into conjunctive normal form. As mentioned before (on page 19), putting
a formula in CNF may result in an exponential blow-up. In this chapter we
will consider various ways in which Boolean functions and formulas may be
represented succinctly.
Definition 7.1.1. An n-ary Boolean function is a function f : {0, 1}n →
{0, 1}.
For every set A and every length n, the characteristic function of A∩{0, 1}n is
a Boolean function. Also note that every propositional formula ϕ(x1 , . . . , xn )
of n variables defines an n-ary Boolean function. Conversely, every Boolean
function can be represented by a propositional formula:
Proposition 7.1.2. For every n-ary Boolean function f there is a proposi-
tional formula ϕ such that the function defined by ϕ is equal to f .
Proof. Define 
F = ~z ∈ {0, 1}n : f (~z) = 1 .
For every vector ~z = (z1 , . . . , zn ) ∈ {0, 1}n define the formula ϕ~z as the
conjunction of all variables xi such that zi = 1 and all negated variables ¬xi
such that zi = 0. Finally define the formula ϕ by
_
ϕ= ϕ~z .
~
z ∈F

Then clearly ϕ is true under an assignment ~z of its variables precisely when

f (~z) = 1.
Note that the size of the formula ϕ in Proposition 7.1.2 is in general expo-
nential in n.

55
56 Chapter 7. Circuit complexity

x1 x2 ❖ x3 x4
✹✹ ❖❖❖ ⑧
✹✹ ❖❖❖❖⑧ ⑧
 ✹✹ ⑧⑧⑧ ❖❖'
89:;
?>=<
¬ ❖❖❖ ✹✹⑧⑧ 89:;
?>=<
¬ ❖❖❖
❖❖❖ ⑧⑧✹ ❖❖❖
❖❖❖ ⑧⑧ ✹✹✹ ❖❖❖
❖❖' ⑧⑧ ❖❖' 
89:;
?>=< ✹✹ 89:;
?>=<
∨ ❖❖❖ ✹✹ ∧
❖❖❖ ✹
❖❖❖ ✹✹
❖❖' 
89:;
?>=<
∧
 
89:;
?>=<
∨

z

Figure 7.1

7.2 Circuits
A circuit is a directed acyclic graph (dag for short), the nodes of which are
called gates. The in-degree, or fan-in, of a node is the number of arrows
leading to it, and the out-degree, or fan-out, the number of arrows leaving
it. There are input gates x1 , . . . , xn , output gates z1 , . . . , zm , and Boolean
gates of type not, and, and or. The and-gates and or-gates have in-degree 2
and the not-gates in-degree 1. The fan-out of any gate can be arbitrary. The
input gates act as function inputs that can be 0 or 1. Thus a circuit computes
a function from {0, 1}n to {0, 1}m . In principle n and m can be arbitrary,
but we will mostly consider circuits with one output gate, i.e. with m = 1.
Figure 7.1 pictures an example circuit.

The Boolean

formula corresponding
to this circuit is (¬x1 ∨ x3 ) ∧ x2 ∨ ¬x2 ∧ x4 .
The size of a circuit is the total number of and-gates, or-gates, and not-
gates.1 Thus we can take as a measure of complexity of a Boolean function
f : {0, 1}n → {0, 1} the minimal size of a circuit computing it. Note that by
Proposition 7.1.2 such a circuit always exists.
To extend the notion of circuit complexity to arbitrary languages L ⊆
{0, 1}∗ we consider circuit families, with one circuit for every input length,
as follows.

Definition 7.2.1. Suppose that {Cn }n∈N is a family of circuits such that
for every n, Cn computes a Boolean function of n inputs. We say that such
a family computes a language L ⊆ {0, 1}∗ if for every length n and every
x ∈ {0, 1}n ,
x ∈ L ⇐⇒ Cn (x) = 1.
1
Some authors do not count the not-gates, which makes no essential difference.
7.3. The relation with advice classes 57

For a given function s : N → N, SIZE(s(n)) denotes the class of languages

L that are computed by a circuit family such that the size of Cn is bounded
by s(n).
It is important to note that circuit size complexity as defined here is a nonuni-
form measure of complexity, in the sense that in the definition of SIZE(s(n))
there is no computational restriction on the sequence of circuits {Cn }n∈N .
(See also Exercise 7.6.3.) In some contexts it makes sense to impose ex-
tra computational restrictions, e.g. by requiring that the sequence {Cn }n∈N
is uniformly computable in polynomial time. As for now, we stick to the
nonuniform definition.
The following result shows that we do not need to consider circuits of
more than exponential size.
Proposition 7.2.2. Any language L ⊆ {0, 1}∗ is in SIZE(O(2n )).
Proof. Given L and a length n, we can consider the characteristic function of
L ∩ {0, 1}n . We can describe this Boolean function with a formula in DNF of
size O(2n ) (cf. the proof of Proposition 7.1.2). Finally, we can easily translate
this formula into a circuit of roughly the same size.
Proposition 7.2.3. There are languages that do not have subexponential cir-
cuit size: For any s such that s(n) = 2o(n) there is a language L ∈
/ SIZE(s(n)).
Proof. This follows by counting the circuits of size s. Assuming that s > n,
we arrive at an upper bound as follows. There are s internal gates and n
input gates. To describe the circuit, we need to indicate for every gate the
type (one of three) and where its two inputs are coming from. In total this
costs no more than
s · (2 log(n + s) + 3) 6 s · (2 log 2s + 3) = s · (2 log s + 5)
bits. Thus the total number of circuits of size s is bounded by 2s(2 log s+5) . On
n
the other hand, there are 22 Boolean functions f : {0, 1}n → {0, 1}. Now if
s(n) = 2o(n) then s(n)(2 log s(n) + 5) ∈ o(2n ), because 2o(n) · 2o(n) = 2o(n) ∈
n n
o(2n ). Since 2o(2 ) is asymptotically smaller than 22 , there are not enough
circuits of size s(n) to compute all Boolean functions, if n is large.

7.3 The relation with advice classes

Recall the definition of advice strings (Definition 6.2.1).
Definition 7.3.1. Let C be a complexity class and let F be a class of advice
functions of the form s : N → {0, 1}∗ . We think of s as giving an advice
string for every length n. Then C/F is the class of all sets of the form

x : hx, s(|x|)i ∈ B
58 Chapter 7. Circuit complexity

for some B ∈ C and some s ∈ F. Let poly be the class of polynomially

bounded advice functions, i.e. those s such that |s(n)| 6 nc for some con-
stant c. In particular P/poly is the class of sets that have polynomial advice.
Theorem 7.3.2. P/poly = SIZE(nO(1) ).
Proof. The direction SIZE(nO(1) ) ⊆ P/poly follows because we can evaluate
a circuit in polynomial time. This means that the set

B = hx, Ci : C circuit ∧ C(x) = 1

is in P. So if A ∈ SIZE(nO(1) ) and for every length n, Cn is a polynomial size

circuit for A ∩ {0, 1}n , then x ∈ A ⇔ hx, C|x| i ∈ B, hence A ∈ P/poly.
For the converse direction P/poly ⊆ SIZE(nO(1) ) we have to do a bit
more work. We have to show that computations of a polynomial time Turing
machine on inputs of a fixed length n can be converted into a polynomial size
circuit. We leave this as Exercise 7.6.5.
Note that by Theorem 6.2.2 we have BPP ⊆ P/poly, so from the previous
theorem we obtain:
Theorem 7.3.3. BPP ⊆ SIZE(nO(1) ).

7.4 Small circuits for NP?

In this section we prove that not all sets in NP have polynomial size circuits,
unless the polynomial hierarchy collapses to the second level, that is, if Πp2 ⊆
Σp2 . For this we need to prove that polynomially bounded ∀∃-statements can
be written as ∃∀-statements. This uses an old proof-theoretic trick called
Skolemization. The idea is that if the formula

∀y∃z ϕ(y, z)

holds in N (or in {0, 1}∗ ) then there is an implicit Skolem function f : N → N

such that ∀y ϕ(y, f (y)). That is, f picks for every y a witness z = f (y) such
that ϕ(y, f (y)). Now the statement that such an f exists, i.e.

∃f ∀y ϕ(y, f (y))

has the required Σp2 -form, provided that we have a polynomially bounded
way to describe f . Here we use the assumption that SAT has small circuits.
The next lemma shows that in this case we actually have small circuits that
produce satisfying assignments, instead of just deciding satisfiability.
Lemma 7.4.1. If NP ⊆ SIZE(nO(1) ) then there is a family of polynomial size
circuits that, given (the binary encoding of ) a formula as input, compute a
satisfying assignment for that formula, if such an assignment exists.
7.5. Sparse T-complete sets 59

Proof. Suppose that {Cn }n∈N is a family of polysize circuits computing SAT,
such that for every formula ϕ of size n, Cn (ϕ) = 1 if and only if ϕ ∈ SAT.
Given a formula ϕ = ϕ(x1 , . . . , xk ) of size n, compute a satisfying assignment
as follows. First see if there is a satisfying assignment starting with 1 by
checking if Cn (ϕ(1, x2 , . . . , xk )) = 1. Continue inductively as follows: Given
the first i − 1 bits b1 , . . . , bi−1 of a satisfying assignment, see if we can extend
it with bi = 1 by checking if

Cn (ϕ(b1 , . . . , bi−1 , 1, xi+1 , . . . , xk )) = 1.

By putting these k circuit computations together in series we obtain one big

(but still polynomial) circuit C such that ϕ ∈ SAT if and only if C(ϕ) satisfies
ϕ, for every ϕ of size n.
Theorem 7.4.2. (Karp-Lipton-Sipser [26]) If NP ⊆ SIZE(nO(1) ) then Πp2 ⊆
Σp2 .
Proof. Suppose that L ∈ Πp2 . By the NP-completeness of SAT there is a
polynomial time computable function f such that for all x,

x ∈ L ⇐⇒ ∀p y f (x, y) ∈ SAT.

Without loss of generality we may assume that, for fixed x, the formulas
f (x, y) all have the same polynomial size. By Lemma 7.4.1 we have a polysize
circuit C such that f (x, y) ∈ SAT if and only if the assignment C(f (x, y))
satisfies f (x, y). Hence there is a polynomial q such that


x ∈ L ⇐⇒ ∃q C∀p y C(f (x, y)) satisfies f (x, y)

which shows that L ∈ Σp2 .

In particular, if NP has small circuits then PH collapses, cf. Exercise 7.6.8.
It also follows that it is unlikely that randomization will help in solving NP-
complete problems, cf. Exercise 7.6.9.

7.5 Sparse T-complete sets

S
Theorem 7.5.1. P/poly = P (S) : S sparse .
Proof. Suppose A ∈ P/poly, say B ∈ P and f is a polynomially bounded
advice function such that

x ∈ A ⇐⇒ hx, f (|x|)i ∈ B. (7.1)

We code the advice function into a sparse set S as follows:


S = h0n , zi : z ⊑ f (n) .
60 Chapter 7. Circuit complexity

S is indeed sparse: the strings of a given length m are of the form h0n , zi
with n 6 m and |z| 6 m, and for a given n there are at most m such z,
so in total there are no more than m2 strings of length m in S. Clearly
A 6pT S: Given x, first use S to find f (|x|), and then apply (7.1). This takes
a polynomial number of steps since f is polynomially bounded.
For the converse, suppose that S is sparse and A 6pT S. Suppose that
A = M S and that M runs in time p. Then on input x, M can only query
elements up to length p(|x|). We can code the polynomial many elements
in S ∩ {0, 1}6p(|x|) into an advice string f (|x|) of polynomial length. In the
computation M S (x), instead of using the oracle S we can equivalently use
the advice string f (|x|).

We can now summarize and rephrase some of the earlier results as follows:

• (Theorem 5.5.3) If there is a sparse p-m-complete set for NP, then

P = NP.

• (Theorems 7.4.2 + 7.5.1) If there is a sparse p-T-complete set for NP,

then PH = Σp2 .

We saw in Exercise 5.8.6 that there are no sparse p-m-complete (or hard) sets
for EXP. As a consequence of the previous theorem we have:

Theorem 7.5.2. If there is a sparse p-T-hard set for EXP then

(i) PH collapses, and

(ii) P 6= NP.

Proof. (i) From the assumption and Theorem 7.5.1 we have EXP ⊆ P/poly.
By Exercise 7.6.13 we then have EXP ⊆ Σp2 . In particular, since PH ⊆ EXP,
we have PH = Σp2 .
(ii) If P = NP then P = Σp2 , hence by the proof of (i) we have EXP = P,
contradiction.

7.6 Exercises
Exercise 7.6.1. The Boolean XOR operator is defined as follows: xXORy
is 1 when exactly one of x and y is 1, and it is 0 otherwise.
(a) Show that xXORy is equal to x + y mod 2.
(b) Define XOR in terms of the usual propositional connectives.
(c) Define a circuit computing XOR.
(d) Show that adding XOR-gates to the definition of circuits only changes
the notion of circuit size by a constant factor.
7.6. Exercises 61

Exercise 7.6.2. The n-ary parity function parityn : {0, 1}n → {0, 1} outputs
1 if the input string contains an odd number of 1’s, and 0 otherwise. Note that
the 2-ary parity function is equal to the XOR operator from Exercise 7.6.1.
Show that there is a circuit of size O(n) that computes parityn .

Exercise 7.6.3. As noted above, circuit size is a nonuniform measure of

complexity. Show that indeed the class SIZE(s(n)) contains noncomputable
sets, even if s is constant.

Exercise 7.6.4. Improve on Exercise 7.6.3 by showing that SIZE(1) is un-

countable.

Exercise 7.6.5.⋆ Finish the proof of Theorem 7.3.2. (For a complete descrip-
tion see Vol. I of [4]. Though conceptually straightforward, the full proof is
a significant amount of work.)

Exercise 7.6.6. Show that P and BPP are strictly included in P/poly.

Exercise 7.6.7. Draw a picture of the circuit from Lemma 7.4.1.

Exercise 7.6.8. Show that if Πp2 ⊆ Σp2 then PH = Σp2 .

Exercise 7.6.9. Show that if NP ⊆ BPP then PH collapses. (This shows

that it is unlikely that randomized algorithms can solve all problems in NP.
See also Exercise 6.5.11.)

Exercise 7.6.10. Show that SIZE(nO(1) ) is closed downwards under 6pT .

S
Exercise 7.6.11. Prove that P/poly = P (S) : S tally .

Exercise 7.6.12. Suppose that EXP ⊆ P/poly, i.e. all sets in EXP have
polysize circuits. Then also all functions in EXP have polysize circuits, pro-
vided that the function is polynomially bounded. (We say f is polynomially
bounded if |f (x)| 6 p(|x|) for some polynomial p.)

Exercise 7.6.13.⋆ (Meyer, cf. [26]) Prove that if EXP ⊆ P/poly then EXP ⊆
Σp2 .

Exercise 7.6.14. There is a sparse p-T-hard set for EXP if and only if
EXP ⊆ P/poly.
Chapter 8

Cryptography

We tend to think of complexity as something negative entailing that we cannot

do something. In this chapter we will see that sometimes complexity can be
something positive that we can use to our advantage. Cryptography is the
prime example of such a topic. This is in fact one of the oldest fields of
applied mathematics, that has stirred the imagination since ancient history.
The main problem is to devise a secure and efficient method to transfer secret
messages. To complicate the task, we require that the method should also
work in case the parties that want to exchange messages have never met
before, so that they have not had the chance of agreeing in private on some
key to be used. We will see that such a method indeed exists, based on a
complexity-theoretic assumption.

8.1 Public-key cryptography

In public-key cryptography, every participant has a public key known to ev-
erybody and a private key only known to himself. If participant Alice wants
to send participant Bob a secret message over a, presumably unsafe, channel,
she encrypts the message using Bob’s public key. The protocol is devised in
such a way that the only way to recover the message is to use Bob’s private
key. This ensures that only Bob can decrypt the message, and that a possible
eavesdropper cannot decipher it.
RSA (which stands for the names of its inventors Rivest, Shamir, and
Adleman) is such a public-key encryption protocol. In 1977 the authors of
RSA published a challenge to decipher a message that was encrypted using
their protocol. It took until 1994 until the message was deciphered, in a large
effort using massive parallel computing, coordinated by Arjen Lenstra and
others. The message turned out to be “The Magic Words are Squeamish
Ossifrage”.
While it is easy to multiply numbers, there does not seem to be an efficient
way of factoring a large number into its prime factors. RSA is based on the

62
8.1. Public-key cryptography 63

assumption that there is no polynomial time algorithm for factoring. (In

contrast, note that we can decide in polynomial time whether a number is
prime, see p 11.) The RSA protocol works as follows. As before, ϕ denotes
Euler’s function.
• Choose two large prime numbers p and q.
• Choose an integer e coprime with ϕ(pq).
• Let d be a multiplicative inverse of e modulo ϕ(pq) (which exist by the
previous item), that is, ed ≡ 1 mod ϕ(pq). Note that d can be easily
computed using the Euclidean algorithm, provided we know p and q,
and hence ϕ(pq) = (p − 1)(q − 1).
• Public key: pq and e
• Private key: p, q, and d
• Encryption algorithm: E(e, x) = xe mod pq
• Decryption algorithm: D(d, y) = y d mod pq
Proposition 8.1.1. For any integer x,
xed ≡ x mod pq. (8.1)
Proof. If gcd(x, p) = 1 then by Fermat’s little theorem (cf. section 1.6), for
any k we have
x1+kϕ(pq) ≡ x1+k(p−1)(q−1) ≡ x mod p. (8.2)
On the other hand, (8.2) clearly also holds if x ≡ 0 mod p, hence it holds for
any x. Similarly, x1+kϕ(pq) ≡ x mod q for any x. It follows by the Chinese
remainder theorem that
x1+kϕ(pq) ≡ x mod pq (8.3)
for any integer x. Since ed ≡ 1 mod ϕ(pq), (8.1) follows from this.
From Proposition 8.1.1 we see that the decryption algorithm D is indeed the
inverse of the encryption algorithm E. In particular, since E has an inverse,
it is a bijection on Z/pqZ.
Note further that anybody can encrypt a message, since pq and e are
public, but that one needs d for decryption, where d is obtained from p and q.
Since pq is public, it is essential that there is no efficient way to obtain p and
q from their product pq. It is currently widely believed, but unproven, that
such a method indeed does not exist.
Note using the above protocol we can encrypt messages of length at
most pq. Longer messages need to be chopped up in pieces of this length. In
the original RSA challenge, the primes p and q were 64 and 65 digits long.
64 Chapter 8. Cryptography

8.2 Signed messages

Suppose Alice wants to send Bob a secret message, and that in addition she
wants to prove to Bob that she is the sender. She can use her own pair of
public and private keys eA and dA to do so in the following way. Let eB
and dB denote Bob’s pair of keys. Instead of sending the encrypted message
xeB as before, Alice now sends the message xdA ·eB to Bob. Bob decodes the
message to xdA using his private key dB . Then he uses Alice’s public key eA
to further decode the message to x. At this point Bob not only knows the
message x, but also that the message had to come from Alice, since Alice is
the only person that knows dA .
The dynamics of the interaction between Alice and Bob in this protocol
reveals some of the power of interactive cryptographic protocols that we will
encounter again later in Chapter 9. Note that the introduction of interaction
between various parties that perform computations adds a whole new layer
of complexity to the theory of computation. The computational as well as
the logical aspects of this are highly interesting.

8.3 One-way functions

Informally, a one-way (or trapdoor ) function is a function that is easy to
compute, but hard to invert. Such functions may serve as the basis for a
cryptographic scheme.
Note first that some easily computable functions may be hard to invert
for trivial reasons. For example, if the function is not injective then there is
no inverse at all. We may counter this by requiring that an inverse should
compute some inverse value, that need not be unique. A second trivial reason
that a function may not be easily invertible is that it might map large strings
to small ones. Consider for example f (x) = log x. Since we have f (2n ) = n, to
write down the inverse f −1 (n) we need to write 2n , which is of length n, which
is exponential in log n, the input size of f −1 (n). So f −1 is not polynomial
time computable. Functions that do not map large strings to small ones,
i.e. such that the length of inputs and outputs are polynomially related, are
called honest. So in studying invertability, we should restrict our attention
to such honest functions.
We have the following formal definition. A function f : {0, 1}∗ → {0, 1}∗
is a one-way function if
(i) f is injective. (As indicated above, this requirement is not always im-
posed.)
(ii) f is honest: There is a constant k such that for all x,

|x| 6 |f (x)|k .
8.4. The class UP 65

(iii) f is polynomial time computable.

(iv) f −1 is not polynomial time computable.

Note that the existence of one-way functions implies that P 6= NP. (Because
of the honesty, computing the – unique – inverse of f (x) is a typical NP
-task: we can guess an inverse and use f to check its correctness.) Even if
we assume that P 6= NP, however, the existence of one-way functions is not
known. Their existence is tied to a special complexity class that we discuss
in the next section
The RSA protocol is based on the assumption that multiplication, which
has factoring as an inverse, is a trapdoor function. There are many other such
functions. An example is the discrete exponential function, and its inverse the
discrete logarithm. Given a prime p and an integer x, the discrete exponential
is 2x mod p. This function can be computed in time O(n3 ), where n is the
length of p. Currently no polynomial time algorithm is known for computing
the inverse of this function.

8.4 The class UP

Definition 8.4.1. Call a nondeterministic Turing machine unambiguous if
for every input there is at most one accepting computation. UP is the class
of sets that are accepted by some unambiguous machine in polynomial time.

Obviously we have the inclusions

P ⊆ UP ⊆ NP.

Nothing more than this is known. The following result ties the class UP to
the existence of one-way functions.

Theorem 8.4.2. (Valiant) One-way functions exist precisely when P 6= UP.

Proof. Suppose that f is a one-way function. Define the set


L = (x, y) : ∃z 6 x(f (z) = y) .

Since f is injective, L ∈ UP. Also L ∈ / P: If L ∈ P then we could find an

inverse of f in polynomial time by binary search.
Conversely, if L is a set in UP − P then we can define a one-way function
f as follows. Suppose M is an unambiguous machine accepting L. Given a
computation path y of M on input x, we let f map y to 1bx if y is accepting,
and to 0by if y is not accepting. Note that x can be effectively retrieved
from y. Since accepting paths of M are unique, f is injective. Also, f is
honest since the computations of M are of polynomial length. So to prove
66 Chapter 8. Cryptography

that f is one-way it suffices to show that its inverse cannot be computed in

polynomial time. (Note that f is not surjective, so the inverse is only defined
on the range of f .) Suppose we could do this. Then we could decide L in
polynomial time as follows: Given x, compute f −1 (1bx). If this yields a y
we know that x ∈ L, and if not we know that x ∈ / L. This contradicts the
assumption that L ∈ / P, hence f is one-way.

8.5 Exercises
Exercise 8.5.1. Strictly speaking, in the RSA protocol above we only need
to know ϕ(pq) and not p and q to compute d. Show however that if we know
both pq and ϕ(pq) then we also know p and q, so obtaining ϕ(pq) from pq is
just as hard as to obtain p and q.

Exercise 8.5.2. Show that UP is downwards closed under p-m-reductions.

Chapter 9

Interactive proofs

9.1 Interactive protocols and the class IP

We can view NP as the class of languages with short certificates. Think of this
as an unbounded Prover P convincing a polynomial time bounded Verifier V .
It is not clear how to describe e.g. co-NP in this way. However, the idea still
works if we add rounds between P and V and make V probabilistic.

Example 9.1.1. (Graph Nonisomorphism [18].) Recall the Graph Iso-

morphism problem GI of deciding whether two given graphs are isomorphic.
This is a famous example of an NP-problem not known to be in P or NP-
complete. We describe an interactive protocol for its complement Graph
Nonisomorphism GNI. Given G0 and G1 , how can Prover convince Verifier
that G0 ∼6= G1 ? Note that Prover cannot send a short certificate, since Graph
Nonisomorphism is not known to be in NP. Instead, Verifier probabilistically
tests Prover as follows. Verifier randomly selects G0 or G1 and randomly
permutes its nodes to obtain a graph H, which he sends to Prover with the
question which of G0 , G1 was the source of H. This is repeated for a polyno-
mial, or constant, number of rounds, say 100 times. If indeed G0 ∼ 6= G1 then
Prover can always reconstruct the answer, using his unbounded computing
power. If on the other hand G0 ∼ = G1 he has no better than a chance of 21 to
answer correctly. So after 100 rounds, if all of the answers of Prover are still
correct, G0 ∼ 1
6= G1 with probability at least 1 − 2100 . In summary, if P does his
∼
best to convince V that G0 6= G1 he can always succeed in doing so if indeed
G0 6∼
= G1 , whereas he will fail almost surely if G0 ∼= G1 .

The complexity class IP consists of all languages that allow for a protocol
of the type just described. In the following we make this more precise. The
protocol will consist of several rounds of sending messages m ∈ {0, 1}∗ be-
tween Prover P and Verifier V . At every round there will be a history string
hm1 , . . . , mi i of messages of previous rounds.

67
68 Chapter 9. Interactive proofs

• Verifier V is a polynomial time computable function that is given the

input w of length n, as well as a random string r of length p(n), where
p is a fixed polynomial. Given message history hm1 , . . . , mi i, V either
accepts, rejects, or sends a next message mi+1 .

• Prover P is an unrestricted function that is given the input, and that

outputs a message mi+1 when given history hm1 , . . . , mi i.

• Define the interaction V ↔ P by setting (V ↔ P )(w, r) = 1 if there is

a legal sequence of messages hm1 , . . . , mk i between V and P for some k
such that mk is “accept”.

• We also require that all messages have length at most p(n), and that
the number of rounds k is bounded by p(n).

Define
   
Pr V ↔ P accepts w = Pr (V ↔ P )(w, r) = 1 .
|r|=p(n)

Definition 9.1.2. We define A ∈ IP if there exist a polynomial time com-

putable verifier V such that for all inputs w,
 
w ∈ A =⇒ ∃P Pr V ↔ P accepts w > 32 , (9.1)
 
w∈/ A =⇒ ∀P Pr V ↔ P accepts w 6 31 . (9.2)

(9.1) is called the completeness of the protocol, and (9.2) is called the sound-
ness.

As for BPP, we can make the error exponentially small by iterating the
protocol a polynomial number of times. Note that NP ⊆ IP: P supplies V
with the certificate in just one round. No randomness is needed here, so the
completeness is in fact 1, and the soundness 0. Also BPP ⊆ IP. For the latter
no interaction with a prover is needed; given w and r, V simply performs a
polytime computation and accepts or rejects.
Note that the protocol for Graph Nonisomorphism in Example 9.1.1 above
shows that this problem is in IP, with perfect completeness 1.

Theorem 9.1.3. IP ⊆ PSPACE.

Proof. We show that, given a verifier V , we can compute an optimal prover

P (i.e. one that maximizes the probability of acceptance) in PSPACE. This
is a brute force derandomization. For a given input w, consider the value
 
M = max Pr V ↔ P accepts w .
P
9.2. IP = PSPACE 69

Note that this value is > 23 if w ∈ A and 6 31 if w ∈ / A. We can define

a similar expression starting with an arbitrary partial message history m
~ =
hm1 , . . . , mi i:
 
~ = max Pr V ↔ P accepts w starting with m
M [m] ~ .
P

Now M = M [∅] can be computed in PSPACE by recursively computing

M [m]
~ for all possible partial message histories m ~ of length at most p(n). If
m
~ has the maximal length p(n) then M [m] ~ is either 0 or 1, as we can see
immediately from m ~ by checking if the last message is “accept”. The value
M [hm1 , . . . , mi i] can be computed from the values M [hm1 , . . . , mi+1 i] for all
possible extensions mi+1 . Namely, if mi+1 is P ’s message, then the former
value is simply the maximum of all M [hm1 , . . . , mi+1 i]. If mi+1 is V ’s message,
then we have to consider the various r’s, and take a weighted sum:
X
M [hm1 , . . . , mi i] = Pr[V (w, r, mi ) = mi+1 ] · M [hm1 , . . . , mi+1 i]
r
mi+1

Thus we can compute M [∅] by backward induction, starting with m

~ of length
p(n), and then decreasing to zero.

Note that in the definition of IP, the prover P has no access to the random
string r of the verifier. This version of interactive proof was introduced by
Goldwasser, Micali, and Rackoff [19] and is referred to as the private coin
model. The class AM (introduced independently by Babai) is defined by
giving P access to the random strings r, and is known as the public coin
model. Hence AM is a subset of IP. It is a difficult result (by Goldwasser
and Sipser) that for a polynomial number of rounds this does not make a
difference. (It should be noted that AM usually refers to protocols with
only two rounds.) Note that in Example 9.1.1, it is crucial that the random
strings of V are private, for if they were public then P could always guess
right. However, by the Goldwasser-Sipser result, there still is an AM-protocol
for Graph Nonisomorphism.

9.2 IP = PSPACE
Towards the converse inclusion of Theorem 9.1.3, consider the following vari-
ant of the problem SA from page 46:

#SAT = hϕ, ki : ϕ has exactly k satisfying assignments .

Theorem 9.2.1. #SAT ∈ IP.

70 Chapter 9. Interactive proofs

Proof. (Sketch.) Suppose ϕ = ϕ(x1 , . . . , xm ) is a Boolean formula and let

fi (a1 , . . . , ai ) be the number of satisfying assignments of ϕ where xj = aj for
j 6 i. Hence f0 () is the number of satisfying assignments of ϕ. Then we have

fi (a1 , . . . , ai ) = fi+1 (a1 , . . . , ai , 0) + fi+1 (a1 , . . . , ai , 1).

Consider the following (not yet polynomial) protocol for #SAT. The input is
a pair hϕ, ki, where k ∈ N is supposed to be the number of assignments of ϕ.
Round 0. P sends f0 () to V , and V checks whether f0 () = k, and rejects
if not.
Round 1. P sends f1 (0) and f1 (1) to V . V checks f0 () = f1 (0) + f1 (1)
and rejects if not.
Round 2. P sends f2 (0, 0), f2 (0, 1), f2 (1, 0), f2 (1, 1) to V . V checks
whether

f1 (0) = f2 (0, 0) + f2 (0, 1),

f1 (1) = f2 (1, 0) + f2 (1, 1)

and rejects if not.

Round m. P sends fm (a1 , . . . , am ) for all a1 , . . . , am ∈ {0, 1}. V checks
m
2 equations and rejects if any of them fail.
Round m + 1. If all fm (a1 , . . . , am ) are correct V accepts, and V rejects
otherwise.
This protocol for #SAT is obviously correct, but the problem is that it
is not polynomial: the rounds are exponential because at every next round
the number of equations and the size of the messages doubles. To remedy
this, write the fi ’s as polynomials over a finite field Fq , q > 2n prime, where
n is the size of ϕ. Such polynomials are easily defined from ϕ by writing
¬x = 1 − x, ∧ as multiplication, and x ∨ y as

x ∗ y = 1 − (1 − x)(1 − y). (9.3)

(We could use + instead of ∗, but ∗ has the advantage that it behaves the
same as ∨ on Boolean values.) Now, in Round 1 of the protocol, instead of
f1 (0) and f1 (1), choose a random r ∈ Fq and consider f1 (r). If adversary P
lies about f0 () then it also has to lie about at least one of f1 (0) and f1 (1),
hence about the polynomial f1 (z). Say it sends f˜1 (z) 6= f1 (z). The key point
of the proof then is that for random r, f˜1 (r) 6= f1 (r) with high probability,
because two different polynomials of degree 6 n can agree on at most n points,
for a nonzero polynomial of degree n has at most n roots.1 So working with
random inputs for the fi ’s prevents the blow-up in the above protocol, and
brings it down to polytime.
1
The precise formulation of this fact for multivariate polynomials is called the Schwartz-
Zippel theorem, cf. [43, p45], [34, p29].
9.2. IP = PSPACE 71

The following theorem was proved by Shamir, building on work by Lund,

Fortnow, Karloff, and Nisan.

Theorem 9.2.2. IP = PSPACE.

Proof. One inclusion was already proven in Theorem 9.1.3, so we only have
to prove PSPACE ⊆ IP. We try to repeat the proof of #SAT ∈ IP with the
PSPACE-complete problem QBF instead of #SAT. Without loss of generality
we can work with closed q.b.f.’s (no free variables) since this subset of QBF
is still PSPACE-complete. Given a closed q.b.f. ϕ = Q1 x1 . . . Qm xm ψ, where
Qi ∈ {∀, ∃}, define
(
1 if Qi+1 xi+1 . . . Qm xm ψ is true,
fi (a1 , . . . , ai ) =
0 otherwise.

f0 () is the truth value of ϕ. Again we can find polynomial expressions for the
fi ’s. We now have the identities

Qi+1 = ∀ : fi (a1 , . . . , ai ) = fi+1 (a1 , . . . , ai , 0) · fi+1 (a1 , . . . , ai , 1), (9.4)

Qi+1 = ∃ : fi (a1 , . . . , ai ) = fi+1 (a1 , . . . , ai , 0) ∗ fi+1 (a1 , . . . , ai , 1), (9.5)

where ∗ was defined in (9.3). Now we run the same protocol as in the proof of
Theorem 9.2.1. A new problem is that in the identities (9.4) and (9.5) every
Qi potentially doubles the degree of the polynomial (so that they may grow
exponential), whereas it is essential in the proof of Theorem 9.2.1 that we
work with polynomials of a degree bounded by a fixed n ≈ size(ϕ). To keep
the degree small we perform the following trick. Given a polynomial f (x) in
variable x, note that

fˆ(x) = (1 − x)f (0) + xf (1)

is linear in x and coincides with f on Boolean values of x. If every time (9.4)

or (9.5) is applied we first perform this transformation for every xj , j 6 i (i.e.
those variables whose degree might go up) then the degree is kept bounded
by 2, without changing the result on Boolean values. The obstacle thus being
removed, the rest of the proof is the same as before.
This shows that QBF is in IP. Now PSPACE ⊆ IP follows since IP is
closed under p-m-reductions (Exercise 9.5.2).

A full account of the proof of Theorem 9.2.2 can be found in Sipser [35].
Note that the result implies that IP is closed under complements, which is
not obvious from its definition.
72 Chapter 9. Interactive proofs

9.3 Zero-knowledge proofs

Using public-key cryptography and interactive proofs, it is possible to con-

vince someone that you are in possession of a proof without actually giving
anything away about it. Such proofs (which are actually interactive proce-
dures) are called zero-knowledge proofs.
The following example (due to M. Blum [7]) may serve to illustrate the
idea. Consider the NP-complete problem 3COLORING. Suppose Alice wants
to convince Bob that a given graph G = (V, E) is 3-colorable. She could of
course do this by giving Bob a 3-coloring after which Bob could efficiently
check that it is correct, but Alice wants to keep her 3-coloring secret and still
convince Bob that she has it. This seemingly impossible task can be achieved
by the following interactive protocol.
In every round Alice chooses a random permutation π of the 3 colors,
and for every node i chooses an RSA tuple (pi , qi , ei , di ). For every i she
announces to Bob (pi qi , ei ) and the encoded version of π(χ(i)), the color of i
after applying π. At this point Bob cannot do anything with this information
since he does not have the decoding key di , but Alice has made a commitment,
albeit encrypted, as to what the colors are. Next, it is Bob’s turn to test Alice
by picking at random an edge (i, j) of the graph, after which Alice reveals to
Bob the secret keys di and dj that allow Bob to decode π(χ(i)) and π(χ(j))
and see if they are different, as they should be. This ends one round of the
protocol.
Now if Alice uses indeed a 3-coloring of the graph, Bob will always be
satisfied. If not, in every round there is an edge (i, j) where π(χ(i)) = π(χ(j)).
1
There are |E| edges, so in every round Bob has a chance at least |E| of
discovering a mistake. By iterating a polynomial number of times this chance
becomes exponentially close to 1.
Let us check that Bob has indeed learned nothing about Alice’s 3-coloring
in the course of this procedure. In every round Bob receives a number of keys
and a pair of colors. But these are randomly permuted versions of the original
colors, hence nothing more than a random pair of different colors. Since in
the next round Alice chooses another random permutation, Bob might just as
well flip a coin himself to generate such random pairs, and he learns nothing
from this.
The NP-completeness of 3COLORING can be used to show that all prob-
lems in NP have zero-knowledge proofs, cf. [18].
Note that the IP protocol for Graph Nonisomorphism in Example 9.1.1
is also zero-knowledge: In case the prover fails to convince the verifier that
the graphs are not isomorphic, which only happens when in fact they are
isomorphic, the verifier still has no clue what the isomorphism is.
9.4. Lowness of Graph Isomorphism 73

9.4 Lowness of Graph Isomorphism

In analogy to the high/low hierarchy in Computability Theory (cf. Soare [38]),
Schöning [37] introduced a similar hierarchy in Complexity Theory.

Definition 9.4.1. A set A ∈ NP is lown if Σpn (A) = Σpn .

A set A ∈ lown may not be in P , but it is close to P in the sense that it

does not help as an oracle from level n of the polynomial hierarchy onwards.
Note that the low0 sets coincide with P . Also, it is not difficult to see that a
set is low1 if and only if it is in NP ∩ co-NP (Exercise 9.5.6).
Schöning [37] proved that the Graph Isomorphism problem GI is low2 .
This result was preceded by the result of Boppana, Håstad, and Zachos that

if GI is NP-complete then Σp2 = Πp2 . (9.6)

This may be seen as evidence that GI is not NP-complete, and hence is a

candidate for a problem in NP of intermediate degree.
We note that the fact that GI is low2 implies (9.6): If GI is NP-complete
then Σp3 = Σp2 (GI) = Σp2 , which is equivalent with Σp2 = Πp2 . Logically speak-
ing, the statement (9.6) is strictly weaker, since (9.6) holds for the halting
problem H, but Σp2 (H) 6⊆ Σp2 .

Theorem 9.4.2. ([8]) If GI is NP-complete then Σp2 = Πp2 .

Proof. We have seen that there is an IP protocol for graph nonisomorphism

GNI. By the result of Goldwasser and Sipser quoted above, there is also an
AM protocol, with public coins and only one round between the verifier V
and the prover P . This means V sends a random sequence r, and P replies
with a message a. We may further assume the error probability is bounded by
2−p(n) , for a given polynomial p(n), and that we have perfect completeness.
Now let A be an arbitrary set in Σp2 . Then A can be defined as

z ∈ A ⇐⇒ ∃x ∀y R(z, x, y), (9.7)

where the quantifiers are polynomially bounded in the size of z, and R is

a predicate in P . We want to show that (9.7) can be put in Πp2 -form. By
assumption, GI is NP-complete, hence GNI is co-NP-complete, so we can
define a polytime function f such that z ∈ A if and only if

∃x f (z, x) ∈ GNI. (9.8)

We claim that (9.8) is equivalent to

∀r ∃x ∃a V (f (z, x), r, a) = 1, (9.9)

74 Chapter 9. Interactive proofs

which has the required Πp2 form. Namely, if (9.8) holds, then by perfect
completeness we have (9.9) (even the stronger form with ∃x∀r∃a). Note that
a is polynomially bounded in the size of f (z, x), hence in the size of z. If
(9.8) does not hold, then ∀x f (z, x) ∈
/ GNI, so then for all x there are many
r such that P has no reply a that makes V accept f (z, x). Without loss of
generality, we may assume that the error probability is bounded by 2−p(n) ,
with p(n) the size of x. Hence

#x · 2−p(n) = 2|x| · 2−p(n) < 1,

so we see that there exists r such that V rejects f (z, x) for all x.2 Hence the
negation of (9.9) holds.

9.5 Exercises
Exercise 9.5.1. Show that the class IP does not change if we require that
the prover P is computable in PSPACE instead of unbounded. (Hint: See
the proof of Theorem 9.1.3.) Since we may assume that P is in PSPACE, in
particular we may also assume that it is in EXP.

Exercise 9.5.2. Show that IP is closed downwards under 6pm .

Exercise 9.5.3. Verify the claim made in the proof of Theorem 9.2.2, that
the subset of QBF consisting of closed formulas is still NP-complete.

Exercise 9.5.4. Show that we can always achieve perfect completeness in

IP, that is, we can replace 23 in (9.1) by 1. (Hint: This follows from the proof
of Theorem 9.2.2.)

Exercise 9.5.5. The original proof of Theorem 9.2.2 employed the notion
of a simple formula. A first-order formula ϕ is simple if no occurrence of a
variable is separated by more than one universal quantifier from its point of
quantification. Show that by introducing new variables every ϕ can be effec-
tively transformed to an equivalent simple formula. How many new variables
are needed at most?

Exercise 9.5.6. A is low1 if and only if A ∈ NP ∩ co-NP.

Exercise 9.5.7. Show that there is a set A ∈ low1 that is NP-complete if

and only if NP = co-NP.

2
Note that this is the same trick as from Adleman’s Theorem 6.2.2. Arora and Barak
[2] call this the “probabilistic method basic principle”.
Chapter 10

Approximations

Since there is an abundance of computational problems that we do not know

how to handle directly, it makes sense to ask for approximate solutions. In
this chapter we study the extent to which hard computational problems can
be effectively approximated.

10.1 Approximating the traveling salesman

Recall the problem TSP from section 2.5. In general this problem is NP-
complete (Corollary 3.4.2), so we have currently no feasible way to deal with
the problem in general. Does it become easier when we ask for approximate
solutions? In particular, say we only ask for a polynomial time algorithm
M that, given an instance x of TSP, always returns a solution M (x) that is
within a factor r of an optimal solution S(x), i.e. such that

c(M (x)) 6 r · c(S(x)), (10.1)

where c is the length of the tour and r ∈ [1, ∞) is fixed. One can think
of many situations where having such an algorithm for some not too large
factor r would be quite helpful. The bad news is that no matter how large
we choose r, such an algorithm does not exist, unless we could already solve
the original problem in polynomial time!

Theorem 10.1.1. (Sahni and Gonzalez) Let r > 1 be a constant. There is

no polynomial time algorithm M that approximates TSP to within a factor r,
as in (10.1), unless P = NP.

Proof. Suppose that such an M exists. We show how to solve the NP-
complete problem HAMILTON CIRCUIT in polynomial time. The idea is
very similar to that of Corollary 3.4.2. Given a graph G = (V, E) we use M
to decide whether G has a Hamilton circuit. Define an instance I of TSP

75
76 Chapter 10. Approximations

as follows. The cities are the points in V = {1, . . . , n}, and we define their
distances by (
1 if (i, j) ∈ E,
di,j =
rn otherwise.
Note that if G has a Hamilton circuit then I has a tour of length n, and
that if G does not have a Hamilton circuit then every tour in I has length at
least rn + (n − 1), which is at least rn + 1 assuming that n > 2, which we
may assume since the case n = 1 is trivial. Now apply M to the instance I.
Then G has a Hamilton circuit if and only if the length of the tour M (I) is
at most rn.

10.2 Approximation algorithms

In general, for every instance x of an optimization problem A we have a
possible set of solutions F (x) that we try to minimize or maximize. This
means that every possible solution s ∈ F (x) has a certain cost c(s). In case
of a minimization problem, the optimum cost is then defined as

OPT(x) = min c(s).

s∈F (x)

(In case of a maximization problem of course we take the maximum.)

Definition 10.2.1. An ε-approximation algorithm M for a minimization
problem A is an algorithm that for each instance x always returns a solution
M (x) ∈ F (x) such that
c(M (x)) − OPT(x)
6 ε. (10.2)
c(M (x))
The ratio on the left is called the relative error of M (x). The approximation
threshold of A is the greatest lower bound of all ε > 0 such that there exists a
polynomial time ε-approximation algorithm for A. Note that this threshold
is always a number in the interval [0, 1].
With the above definitions, we can reformulate Theorem 10.1.1 as follows:
Theorem 10.2.2. The approximation threshold of TSP is 1, unless P = NP.
Proof. Suppose that M is a polynomial time ε-approximation algorithm for
TSP. Then by (10.2) we have c(M (x)) − OPT(x) 6 ε · c(M (x)), hence
1
c(M (x)) 6 OPT(x).
1−ε
By Theorem 10.1.1 this cannot hold for any ε < 1.
10.3. Probabilistically checkable proofs 77

Definition 10.2.3. A polynomial time approximation scheme (PTAS) for an

optimization problem is an algorithm that for each ε > 0 and each instance
x outputs a solution with relative error at most ε, as in (10.2), and that runs
in time polynomial in |x|. If the running time is also polynomial in 1ε then
the scheme is called fully polynomial.
Definition 10.2.3 points to an interesting feature, namely that computational
problems may have several parameters, and that algorithms to solve them may
be polynomial in some of parameters but not in others. The field in which
this phenomenon is systematically explored is called parameterized complexity
theory, and the standard textbook on this subject is Downey and Fellows [15].
The reader should note that approximability is not preserved by p-m-
reductions: Some NP-complete problems, such as TSP, do not have a PTAS
(unless P = NP, cf. Theorem 10.2.2), whereas others, such as KNAPSACK,
do have one. It is possible to devise a type of reduction that preserves approx-
imability, and to develop the theory of approximation problems analogous to
the theory of NP-completeness.

10.3 Probabilistically checkable proofs

In this section we discuss the notion of probabilistically checkable proof, and
the main theorem about this notion, the PCP Theorem. Although we will
not prove this theorem, we discuss some of its consequences for the theory of
approximations, where the theorem has had a profound influence.
Suppose that L is a language in NP. By Proposition 4.2.4 there is a
polynomial time predicate R such that

L = x : ∃p(|x|) y (x, y) ∈ R .
This implies the existence of a verifier V for L, that, given any input x and
any certificate y of length at most p(|x|), checks whether y is indeed proof of
the fact that x ∈ L or not. The verification can be done in polynomial time,
and in principle uses all of the bits of y. In the following we will see that we
can greatly improve on the number of bits examined if we are willing to make
V probabilistic.
Definition 10.3.1. A probabilistic verifier is a probabilistic algorithm (in
the sense of section 6.1) that given an input x and a proof π as oracle,
examines the proof and accepts or rejects it with a certain probability. Given
functions r(n) and q(n), a language L is in the class PCP[r(n), q(n)] if there
is a polynomial time probabilistic verifier V that on an input x of length n
uses r(n) random bits and at most q(n) nonadaptive queries to the oracle π
such that
 
x ∈ L ⇔ ∃π Pr V π (x) = 1 = 1,
 
x∈/ L ⇔ ∀π Pr V π (x) = 1 6 21 .
78 Chapter 10. Approximations

Here the probabilities are taken over all choices of the r(n) random bits.
The following theorem is one of the landmarks of the subject. It was the
culmination of a series of papers by a large number of authors, with Sanjeev
Arora as one of the main contributors. For more information about the history
of the result we refer to [33].

S 10.3.2. (PCP Theorem) There exists a constant q ∈ N such that

Theorem
NP = c>1 PCP[c log n, q].
Hence in order to verify with high probability that a given instance x is in
some NP-language, we only need to examine a constant number of bits of the
proof, regardless the length of x. Furthermore, the constant is the same for all
languages in NP. The proof of Theorem 10.3.2 is too long and complicated
to discuss here. By now there is an somewhat easier proof by Irit Dinur
available. We recommend the exposition of Dinur’s proof in [33]. Despite the
various improvements in presentation, the proof remains beyond the scope of
these course notes.

10.4 PCP theory and nonapproximability

Consider the following optimization problem MAX3SAT:
Input: a 3CNF formula ϕ (with three literals per clause),
Question: find an assignment satisfying the largest number of clauses of ϕ.
Since 3SAT is NP-complete by Theorem 3.3.4, MAX3SAT is not solvable in
polynomial time, unless P = NP.
Proposition 10.4.1. The approximation threshold of MAX3SAT is at most
1/2.
Proof. Given a formula in CNF, consider the all-zero and all-one assignments.
Since every clause is satisfied by at least one of these, one of them satisfies at
least half of the clauses, so pick one that does this.
In fact, it is possible to improve Proposition 10.4.1 from 12 to 81 , cf. Exer-
cise 10.5.5.
By the following theorem, the approximation threshold of MAX3SAT is
bigger than 0. In particular there is no PTAS for MAX3SAT, unless P = NP.
Definition 10.4.2. We say that a CNF formula ϕ is ε-far from satisfiable if
for every assignment, at most a fraction 1 − ε of the clauses of ϕ is satisfied.
Theorem 10.4.3. There exists ε > 0 such that for every L ∈ NP there is a
p-m-reduction x 7→ ϕx of L to 3SAT such that
x ∈ L ⇔ ϕx satisfiable,
x∈/ L ⇔ ϕx ε-far from satisfiable. (10.3)
10.4. PCP theory and nonapproximability 79

Proof. By the PCP theorem, let V be a probabilistic verifier witnessing that

L ∈ PCP[O(log n), O(1)]. Let r ∈ {0, 1}O(log |x|) range over all the random
strings for V . For every r, V makes a constant number of queries to the
proof π. Without loss of generality we may assume that the queries are
nonadaptive (since an exponential of a constant number of queries is still
constant). Suppose that the number of queries is ℓ. For each choice of r,
V π (x) depends on a Boolean function fr : {0, 1}ℓ → {0, 1} describing all
possible answers to the ℓ queries. For the translation into a 3CNF-formula,
introduce variables x1 , . . . , xk that stand for all positions of π that can be
queried, i.e. xi stands for π(i). The number k equals the length of the proof
π, which is polynomial in |x|. We can represent fr by a CNF-formula ψr with
2ℓ clauses (cf. Proposition 7.1.2) using the variables xq1 , . . . , xqℓ corresponding
to the queries V makes with r. To convert every clause of length ℓ to a clause
with three literals (as in Theorem 3.3.4) we need for every clause ℓ auxiliary
clauses, so in total ψr has at most ℓ2ℓ clauses. V For every r we use a different
set of auxiliary variables. Finally, we let ϕx = r ψr . Now
 
x ∈ L ⇒ ∃π Pr V π (x) = 1 = 1
⇒ π satisfies all clauses of ϕx .

 
/ L ⇒ ∀π Pr V π (x) = 1 6 12
x∈
⇒ for at least half of the r, π does not satisfy ψr , i.e.
at least one clause of ψr is false,
1 1
⇒ ϕx is ε-far from satisfiable, where ε = .
2 ℓ2ℓ
Theorem 10.4.3 is a consequence of the PCP theorem. Interestingly, the
converse also holds:

Theorem 10.4.4. If L ∈ NP and L has a reduction as in (10.3) then L ∈

PCP[O(log n), O(1)].

Proof. We define a probabilistic verifier V for L as follows. Given x, compute

ϕx as in (10.3), and interpret the proof π as an assignment for ϕx . V randomly
picks c = 1ε ln 2 clauses of ϕx and checks whether π satisfies all of them. If
so, V accepts x, and V rejects x otherwise. To specify an item in a list of n
objects takes log n bits, and since the number of clauses in ϕx is polynomial
in |x| we need c log |x|O(1) = O(log |x|) random bits. Checking these clauses
takes O(1) bits of π. Now

x ∈ L ⇒ ϕx satisfiable
 
⇒ ∃π Pr V π (x) = 1 = 1
80 Chapter 10. Approximations

x∈
/ L ⇒ ∀π at least a fraction ε of the clauses of ϕx is false
 
⇒ ∀π Pr V π (x) = 1 6 (1 − ε)c
1
To finish the proof, we just have to see that (1 − ε)c 6 2
for c = 1ε ln 2, which
easily follows from the inequality 1 − x 6 e−x .

10.5 Exercises
Exercise 10.5.1. PCP[0, O(log n)] = P.

Exercise 10.5.2. PCP[0, poly n] = NP.

Exercise 10.5.3. Prove the easy direction of Theorem 10.3.2, namely that
PCP[O(log n), O(1)] ⊆ NP.

Exercise 10.5.4. PCP[O(log n), 1] = P.

Exercise 10.5.5. Show that the approximation threshold of MAX3SAT is

at most 18 . (Hint: Use the probabilistic method : Assign the value 1 to every
variable x with probability 12 and compute the expected number of satisfied
clauses.)
Chapter 11

Proof complexity

In mathematical logic, the complexity of a theorem can sometimes be mea-

sured by considering the strength of the axioms that are needed to prove it.
There are interesting complexity theoretic analogues of this theme, that we
discuss in this chapter. Gödel’s classic incompleteness theorem tells us that,
no matter how strong our axioms, not every true arithmetical formula is prov-
able. Here we consider the length of proofs, and show that it is unlikely that
every true propositional formula has a short proof. “Short” here of course
means of polynomial size in the length of the formula.

11.1 Propositional proof systems and Cook’s program

Recall that SAT is the set of all satisfiable propositional formulas. Consider
its companion VALID consisting of all valid propositional formulas, that is,
those formulas that are valid under all assignments.
By Exercise 4.4.7, VALID is co-NP-complete. Hence, if VALID ∈ NP
then NP = co-NP, something that is believed to be unlikely. This means
that, if is indeed true that NP 6= co-NP, not every valid formula has a short
certificate that proves its validity. This can be seen as a bounded version of
Gödel’s incompleteness theorem: Not every true propositional formula has a
short proof.

Definition 11.1.1. A propositional proof system is any polynomial time com-

putable function F : {0, 1}∗ → VALID. (We can without loss of generality
require that F is also surjective, cf. Exercise 11.4.2.)

If F (x) = ϕ, then x is a proof that ϕ is valid. The restriction that F be

polynomial time computable is natural: given a proof we certainly want to
be able to check efficiently that it indeed is a proof.
If NP 6= co-NP then the following must hold:
c
∀F ∀c ∃ϕ ∈ VALID ∀x ∈ {0, 1}|ϕ| (F (x) 6= ϕ). (11.1)

81
82 Chapter 11. Proof complexity

Here the quantification is over all propositional proof systems F . This state-
ment expresses that for every such F there exist formulas requiring super-
polynomial proofs. Note that the contrapositive of (11.1) is equivalent to
NP = co-NP (Exercise 11.4.3). So if we ever want to prove that NP 6= co-NP,
we should at least be able to prove (11.1) for any given F . Cook’s program 1 is
to come up with stronger and stronger proof systems, and with formulas that
have superpolynomial proof complexity in those systems. One nice aspect of
this program is that we can measure our progress on such a difficult question
as NP 6= co-NP.
Let us start simple. The most naive proof system for propositional logic is
the method of truth tables. (Truth tables apparently first appeared, in their
modern tabular form, in Wittgenstein [45, 4.31 ff], though there are several
precursors, including Peirce and Post.) Since the truth table of a formula
with n propositional variables has size O(2n ), every proof of validity using
truth tables has exponential size, regardless which formula we are proving.
So we have made our first step in Cook’s program, namely we have proven
an exponential lower bound for the proof system of truth tables.
Of course the problem becomes more difficult for more sophisticated proof
systems F . The problem is open for several of the standard proof systems
for classical propositional logic. In section 11.3 we consider proofs based
on resolution, and show that (11.1) holds for this system. One possibility
for making some progress is to consider proof systems that are weaker than
those for classical logic. An obvious candidate is to consider intuitionistic
logic. A breakthrough result for this logic was obtained by Hrubeš [22]. He
showed that in a proof system for intuitionistic propositional logic, exponen-
tial lower bounds for certain tautologies can indeed be proven. (N.B. The set
of intuitionistic validities is PSPACE-complete, and it is a subset of VALID.)
Hrubeš uses a tautology similar to the pigeonhole principle discussed below,
namely k-colorability: a complete graph on k + 1 nodes is not k-colorable.
This has a short classical proof, but it does not have a short proof in an
intuitionistic Frege system.

11.2 The pigeonhole principle

As discussed above, part of the task in Cook’s program is, given a proof sys-
tem F , to come up with candidate formulas that are supposed to only have
superpolynomial proofs. For some time it was hoped that the combinato-
rial statement called the pigeonhole principle would provide such formulas.
This statement says that if we have m pigeons that we have to put into n
holes, and m > n, then there is a pigeonhole containing more than one pi-
1
The connection between proof systems and NP 6= co-NP was first made in Cook and
Reckhow [14]. For a recent survey see Buss [11].
11.3. Resolution 83

geon. By Exercise 11.4.4, the pigeonhole principle can be formulated as a

Boolean formula PHPm m
n . The hope that the formulas PHPn would only have
superpolynomial proofs was crushed when Buss [10] showed that in fact they
do have polynomial size proofs, in a certain Hilbert style proof system for
classical logic. Such systems, based on modus ponens, are called Frege sys-
tems in this context. The formulas PHPm n can be used, however, to show an
exponential lower bound for resolution, see the proof of Theorem 11.3.4.

11.3 Resolution
In this section we discuss a simple propositional proof system, namely reso-
lution. In resolution there is only one rule, which speeds up the search for
proofs, but the method only works for formulas in CNF. (Recall that we have
seen in section 3.3 that converting formulas to CNF may be expensive. Recall
also that the CNF fragment of SAT is still NP-complete.) Resolution is also
the method underlying the declarative programming language Prolog.
In the following we use the terminology from section 1.5.
Definition 11.3.1. If C1 = {l} ⊔ C1′ and C2 = {¯l} ⊔ C2′ are clauses, where ⊔
denotes that we are taking a union of disjoint sets, then C1′ ∪ C2′ is called a
resolvent of C1 and C2 .
Note that resolution is sound, i.e. preserves satisfiability. That is, if both
of the parent clauses are satisfiable then also their resolvent is satisfiable.
A resolution proof of a clause C from a formula S is a finite sequence
C1 , C2 , . . . , Cn = C of clauses such that each Ci is an element of S or a
resolvent of clauses earlier in the sequence. If such a proof exists we write
S ⊢R C. If S ⊢R  we say that S is refutable.
We can in principle picture resolution proofs as binary trees. For example,
the following is a refutation proof from the set

S = {p, r}, {q, ¬r}, {¬q}, {¬p, t}, {¬s}, {s, ¬t} .

❦  ❚❚❚❚❚
❦❦❦❦❦❦❦ ❚❚❚❚
❚❚❚❚
❦❦❦❦❦ ❚❚
❦
{p} {¬p}❊
❋❋
✇✇
✇ ❋❋ ✈✈✈ ❊❊
❊❊
✇✇ ❋ ✈✈✈
{p, q} {¬q} {¬p, s}❍ {¬s}
●● ❍❍❍
③③
③ ●● ✉✉✉ ❍❍
③③ ● ✉✉✉
{p, r} {q, ¬r} {¬p, t} {s, ¬t}

Note however that the representation of a resolution proof as a tree may be

costly in case a clause is repeated several times in the proof, which may result
in a repetition of parts of the tree.
84 Chapter 11. Proof complexity

Theorem 11.3.2. (Soundness of propositional resolution) S ⊢R  =⇒ S

unsatisfiable.

Proof. Obviously, if V is an assignment satisfying C1 and C2 , then also V |= C

for any resolvent C of C1 and C2 . Hence if S is satisfiable then S 6⊢R .

Resolution is not complete in the sense that for every set of clauses S and
every clause C, whenever S |= C then S ⊢R C. For example, |= p ∨ ¬p but
6⊢R {p, ¬p}. However, resolution is complete in the sense that any inconsistent
S is refutable. This fact is called refutation completeness.

Theorem 11.3.3. (Refutation completeness of propositional resolution) S

unsatisfiable =⇒ S ⊢R .

Proof. Cf. [30].

The first superpolynomial lower bounds for resolution were proven by Tseitin
[44], and the first exponential ones by Haken [20]. The proof of the latter
result uses the following representation of ¬PHPm n as a CNF-formula: For
every i 6 m and j 6 n there is a variable pij with the intended meaning
“pigeon i is in hole j”. For every i we have a clause pi1 ∨ pi2 ∨ . . . ∨ pin
expressing that i is in some hole, and for every k and i 6= j we have a clause
p̄ik ∨ p̄jk expressing that no hole k gets two pigeons i and j. Taken together
these clauses express the formula ¬PHPm n , which is of course false if m > n.

Theorem 11.3.4. (Haken [20]) There are unsatisfiable formulas that only
have exponential size resolution refutations. More specific, for every n > 2,
n
any resolution refutation of ¬PHPnn−1 has size at least 2 20 .

Proof. We follow the presentation in [2]. We view a resolution refutation as

an analysis showing that there is no assignment from the set of all possible
assignments that satisfies all the given clauses. The analysis remains valid if
we restrict attention to a subset of all possible assignments. We will restrict
attention to the following set of assignments: An assignment for ¬PHPnn−1
is i-critical if it assigns n − 1 pigeons in an injective way to n − 1 holes and
leaves pigeon i unassigned.
Restricting attention to the above set of critical assignments has the ad-
vantage that we can make all clauses in the refutation monotone, that is,
containing only positive literals. (Strictly speaking, changing the clauses
makes the refutation no longer a resolution proof, but it is equivalent to it
as far as the assignments under consideration are concerned.) We make the
proof monotone by replacing in every clause every negative literal p̄ij by
_
pkj .
k6=i
11.3. Resolution 85

Note that for critical assignments these are equivalent, since such an assign-
ment assigns a pigeon to every hole.
2
Call a clause in the proof large if it has at least n10 variables, and let L
2
be the number of large clauses. All large clauses together have at least n10 L
occurrences of variables, and there are n2 variables pij , so there must be a
1
variable pij that occurs at least 10 L times in the L large clauses. Setting
pij = 1 and pij ′ = 0 for every j ′ 6= j and pi′ j = 0 for every i′ 6= i leaves at
9
most 10 L large clauses. Since this removes one pigeon and one hole, we now
n−1
have a monotone refutation of ¬PHPn−2 .
9 t
We repeat this t times, where t is so large that ( 10 ) L < 1, i.e. ( 10
9
)t > L,
so we let t = log 10 L (rounded up, plus 1 if necessary). This removes all the
9
n−t
large clauses, and leaves us with a monotone refutation of ¬PHPn−t−1 .
n
Now if L < 2 20 (which would be the case if the original refutation had
n n
size less than 2 20 ) then t = log 10 L < 20 log 10 2, from which it follows that
9 9
2 2 1 2
9
(n − t) > 10 n.
n−t
So we have a monotone refutation of ¬PHPn−t−1 with no clauses with 29 (n−t)2
variables, contradicting Lemma 11.3.5.
Lemma 11.3.5. Every monotone refutation of ¬PHPnn−1 contains a clause
with at least 29 n2 variables.
Proof. For every clause C define

witness(C) = i : there is an i-critical assignment falsifying C ,
and define the complexity comp(C) to be |witness(C)|. If C is a resolvent of
C ′ and C ′′ then comp(C) 6 comp(C ′ ) + comp(C ′′ ) because any assignment
falsifying C must falsify at least one of C ′ and C ′′ . Note that if comp(C) > 23 n
then comp(C ′ ) + comp(C ′′ ) > 23 n, hence comp(C ′ ) > 31 n or comp(C ′′ ) > 31 n.
So if C is the first clause in the proof with comp(C) > 13 n (which exists since
comp() = n) then
1
3
n 6 comp(C) 6 23 n.
We show that C has at least 29 n2 variables, namely, that if comp(C) = x then
it contains at least x(n − x) variables. This suffices since x(n − x) > 92 n2 for
1
3
n 6 x 6 32 n.
Suppose i ∈ witness(C) and σ is an i-critical assignment falsifying C. For
each j ∈ / witness(C), consider the assignment σ ′ obtained by replacing j by i:
if σ maps j to hole k, σ ′ maps i to hole k and leaves j unassigned. Then
σ ′ is j-critical, and since j ∈
/ witness(C) it satisfies C. Since σ ′ satisfies C
and σ does not, this can only be because σ ′ makes pik true, and in particular
C must contain pik . Repeating the argument for every j ∈ / witness(C), and
noting that for every j the assignment σ maps j to a different hole k, we see
that C contains (n − x) distinct variables pik . Since there are x different i in
witness(C), this gives x(n − x) different variables pik in total.
86 Chapter 11. Proof complexity

11.4 Exercises
Exercise 11.4.1. Show that if we drop the requirement in Definition 11.1.1
that F is polynomial time computable then there is a proof system in which
every valid formula has a short proof.

Exercise 11.4.2. Show that in Definition 11.1.1 we can require without loss
of generality that F is surjective.

Exercise 11.4.3. Show that (11.1) is equivalent to NP 6= co-NP.

Exercise 11.4.4. Formulate the pigeonhole principle for m pigeons and n

holes as a formula PHPmn of propositional logic. Introduce variables xi,j with
the meaning “pigeon i is in hole j”.

Exercise 11.4.5. Let PHPm 3

n the formula from Exercise 11.4.4. Prove PHP2
3
using resolution, i.e. write ¬PHP2 in clausal form, and refute it.
Further reading

By now there is a large number of textbooks available for the subject, and
several texts and notes are electronically available. The following is a very
incomplete list:

• M. R. Garey and D. S. Johnson, Computers and Intractability: A Guide

to the Theory of NP-Completeness, W. H. Freeman, 1979.

• J. L. Balcázar, J. Dı́az, and J. Gabarró, Structural Complexity, two

volumes, Springer, 1988 (Vol. I) and 1990 (Vol. II).

• C. H. Papadimitriou, Computational Complexity, Addison Wesley, 1994.

• M. Sipser, Introduction to the Theory of Computation, Course Technol-

ogy, 2006.

• S. Arora and B. Barak, Computational Complexity: A Modern Ap-

proach, Cambridge University Press, 2009.

• L. Trevisan, Lecture Notes on Computational Complexity, available at

web pages of the author.

• H. S. Wilf, Algorithms and complexity, available at web pages of the

author.

87
Bibliography

[1] M. Agrawal, N. Kayal, and N. Saxena, PRIMES is in P, Annals of

Mathematics 160 (2004) 781–793.

[2] S. Arora and B. Barak, Computational Complexity: A Modern Approach,

Cambridge University Press, 2009.

[3] T. Baker, J. Gill, and R. M. Solovay, Relativizations of the P = NP

question, SIAM Journal on Computing 4 (1975) 431–442.

[4] J. L. Balcázar, J. Dı́az, and J. Gabarró, Structural Complexity, two vol-

umes, Springer, 1988 (Vol. I) and 1990 (Vol. II).

[5] R. Beigel, N. Reingold and D. Spielman, PP is closed under intersection,

Journal of Computer and System Sciences 50(2) (1995) 191–202.

[6] M. Blum, A machine-independent theory of the complexity of recursive

functions, Journal of the ACM 14 (1967) 322–336.

[7] M. Blum, How to prove a theorem so no one else can claim it., Proc.
International Congress of Mathematicians (1987) 1444–1451.

[8] R. B. Boppana, J. Håstad, and S. Zachos, Does co-NP have short inter-
active proofs?, Information Processing Letters 25 (1987) 127–132.

[9] J. Buhler and S. Wagon, Basic algorithms in number theory, in: Algo-
rithmic number theory, MSRI Publications 44 (2008) 25–68.

[10] S. R. Buss, Polynomial size proofs of the propositional pigeonhole prin-

ciple, Journal of Symboloc Logic 52 (1987) 916–927.

[11] S. R. Buss, Towards NP-P via proof complexity and search, Annals of
Pure and Applied Logic 163 (7) (2012) 906–917.

[12] S. Cook, The complexity of theorem proving procedures, Proc. 3rd ACM
Symposium on the Theory of Computing (1971) 151–158.

[13] S. A. Cook, A hierarchy for nondeterministic time complexity, Proc. 4th

ACM Symposium on the Theory of Computing (1972) 187–192.

88
Bibliography 89

[14] S. A. Cook and R. A. Reckhow, On the lengths of proofs in the proposi-

tional calculus, Proc. 6th ACM Symposium on the Theory of Computing,
(1974) 135–148.

[15] R. Downey and M. R. Fellows, Parameterized complexity, Springer, 1999.

[16] J. Gill, Computational complexity of probabilistic Turing machines,

SIAM Journal on Computing 6 (1977) 675–695.

[17] K. Gödel, Über die Länge von Beweisen, Ergebnisse eines mathemati-
schen Kolloquiums 7 (1936) 23–24.

[18] O. Goldreich, S. Micali, and A. Wigderson, Proofs that yield nothing but
their validity or all languages in NP have zero-knowledge proof systems,
Journal of the ACM 38 (1991) 690–728.

[19] S. Goldwasser, S. Micali, and C. Rackoff, The knowledge complexity of

interactive proof systems, SIAM Journal on Computing 18 (1989) 186–
208. (Preliminary version in STOC 1985.)

[20] A. Haken, The intractability of resolution, Theoretical Computer Science

39 (1985) 297–308.

[21] J. E. Hopcroft and J. D. Ullman, Introduction to automata theory, lan-

guages, and computation, Addison-Wesley, 1979.

[22] P. Hrubeš, A lower bound for intuitionistic logic, Annals of Pure and
Applied Logic 146 (2007) 72–90.

[23] N. Immerman, Nondeterministic space is closed under complementation,

SIAM Journal on Computing 17 (1988) 935–938.

[24] A. R. Meyer and L. J. Stockmeyer, The equivalence problem for regular

expressions with squaring requires exponential space, Proc. 13th IEEE
Symposium on Switching and Automata Theory (1972) 125-129.

[25] R. M. Karp, Reducibility among combinatorial problems, In: R. E. Miller

and J. W. Thatcher (eds), Complexity of Computer Computations, 1972,
85–103.

[26] R. Karp and R. Lipton, Turing machines that take advice, L’ensignement
mathématique 28 (1982) 191–210.

[27] R. Ladner, On the structure of polynomial time reducibility, Journal of

the ACM 22 (1975) 155–171.
90 Bibliography

[28] R. Ladner, N. Lynch, and A. Selman, Comparison of polynomial-time

reducibilities, Proc. 6th ACM Symposium on the Theory of Computing
(1974) 110–121.

[29] S. R. Mahaney, Sparse complete sets for NP: solution to a conjecture of

Berman and Hartmanis, Journal of Computer and System Sciences 25
(1982) 130–143.

[30] A. Nerode and R. A. Shore, Logic for applications, 2nd edition, Springer
1997.

[31] P. G. Odifreddi, Classical recursion theory, Vol. 2, Studies in logic and

the foundations of mathematics Vol. 143, North-Holland, 1999.

[32] C. H. Papadimitriou, Computational complexity, Addison Wesley, 1994.

[33] J. Radhakrishnan and M. Sudan, On Dinur’s proof of the PCP theorem,

Bulletin of the AMS 44 (2007) 19–61.

[34] S. Rudich and A. Wigderson (eds.), Computational complexity theory,

American Mathematical Society, 2004.

[35] M. Sipser, Introduction to the theory of computation, 2nd edition, Brooks

Cole, 2005.

[36] M. Schaefer and C. Umans, Completeness in the polynomial-time hier-

archy: a compendium in: SIGACT News, 2002.

[37] U. Schöning, Graph Isomorphism is in the low hierarchy, Journal of Com-

puter and System Sciences 37 (1988) 312–323.

[38] R. I. Soare, Recursively Enumerable Sets and Degrees, Springer, 1987.

[39] R. Solovay and V. Strassen, A fast Monte-Carlo test for primality, SIAM
Journal on Computing 6(1) (1977) 84–85.

[40] L. Sterne, The Life and Opinions of Tristram Shandy, Gentleman, 1759.
Everyman’s Library, 1991.

[41] R. Szelepcsényi, The method of forcing for nondeterministic automata,

Bulletin of the EATCS 33 (1987) 96–100.

[42] S. A. Terwijn, Syllabus computability theory, Technical University of Vi-

enna, 2004. Available at web pages author.

[43] L. Trevisan, Lecture Notes on Computational Complexity, available at

web pages of the author.
Bibliography 91

[44] G. S. Tseitin, On the complexity of derivation in propositional calcu-

lus, In: A. O. Slisenko (ed.), Studies in constructive mathematics and
mathematical logic (1970) 115–125.

[45] L. Wittgenstein, Tractatus Logico-Philosophicus, 1922. Routledge 1981.

92 Bibliography
Index

C/F, 57 E, 25
C A , 31 EXP, 10, 25
|A|, cardinality, 2 IP, 67, 68
A, complement, 2 LIN, 25
A↾x, restriction, 2 NLIN, 25
∃C, 30 NP, 10
∃p(n) , 30 P, 10
∀p(n) , 30 PCP[r(n), q(n)], 77
, empty clause, 4 PH, 29, 50
co-C, 2 PP, 45, 50
⊔, disjoint union, 83 PSPACE, 10, 69
⊕, 16 P/poly, 58
K, 18 RP, 50
|x|, length of x, 1 SIZE(s(n)), 57
M (x) ↓, halting computation, 2 UP, 65
M (x) ↑, diverging computation, 2 ZPP, 50
x̄, negated literal, 4 ∆pn , 29
O(f ), 3 Πpn , 29
o(f ), 3 Σpn , 29
ω(f ), 3 3COLORING, 17, 72
Ω(f ), 3 CLIQUE, 20
M B , oracle machine, 27 GI, 36, 67, 73
6pT , 27 GNI, 67
6pm , 16 HAMILTON CIRCUIT, 12, 26
6m , 16 HAMILTON PATH, 12, 22, 26
h· , ·i, pairing function, 2 HORNSAT, 26
PHPm n , pigeonhole principle, 83 KNAPSACK, 15
poly, 58 MAJ, 45
⊢R , resolution provable, 83 MAX3SAT, 78
Σ∗ , 1 PATH, 8
{0, 1}n , 1 PRIMES, 11, 51
xby, concatenation, 1 QBF, 19, 21, 36
x ⊑ y, 2 QBFk , 31, 32
AM, 69 SA, 46
BPP, 47, 48 SAT, 11, 17–19, 25

93
94 Index

SAT-CNF, 19 diagonalization, 9
#SAT, 69 Dijkstra, 12
TSP, 12, 15, 23, 75 Dinur, 78
VALID, 32, 81 disjunctive normal form, 4
kDNF, 26 DNF, 4
nSAT, 26
nSAT, 19 ε-far, 78
error probability, 45
adaptive query, 27 Euler, 4
Adleman, 48 Euler phi function, 4
advice, 48, 57 Euler’s criterion, 51
advice function, 57
fan-in, 56
approximation threshold, 76
fan-out, 56
Arora, 78
Fermat’s little theorem, 4
Baker, 36, 37 Frege system, 83
Berman, 43 Friedberg-Muchnik Theorem, 35, 43
Berman-Hartmanis Conjecture, 39, Gács, 49
43 gap theorem, 40
Blum, 41 Gill, 36, 37
Borodin, 40 Gödel, 25, 41, 81
bounded quantification, 30 Graph Isomorphism, 36, 67, 73
Buss, 82, 83 guess, 10
c.e., 2, 15 Haken, 84
Cantor, 9, 34 Halting Problem, 34
certificate, 10 hard, 17
Chinese remainder theorem, 4 honest function, 15, 64
circuit, 48, 56 Horn formula, 26
circuit size, 56 Hrubeš, 82
clause, 4
monotone, 84 Immerman, 12
CNF, 4, 19 incomparable degrees, 43
collapse, 30 index, 41
complete, 17 intermediate degree, 34
completeness intuitionistic logic, 82
refutation, 84
Jacobi symbol, 52
computable, 2
join operator, 16
conjunctive normal form, 4
context-sensitive, 13 Karp, 59
Cook, 18, 82 Kleene star operator, 1, 15
delayed diagonalization, 35 Ladner, 35, 38
density, 43 language, 2
Index 95

Las Vegas algorithm, 50 quantified Boolean formula, q.b.f.,

law of quadratic reciprocity, 52 19
Legendre symbol, 51 query, 27
Levin, 18
linear speed-up, 7 Reckhow, 82
Lipton, 59 refutable, 83
literal, 4 regular set, 14
lown , 73 relative error, 76
Lynch, 38 relativize, 31
resolution, 83
m-reducible, 16 proof, 83
Mahaney, 39 resolvent, 83
Monte Carlo algorithm, 50 RSA, 62
Myhill Isomorphism Theorem, 39
Savitch, 8, 21
Myhill-Nerode Theorem, 14
scale, 41, 43
NP-complete, 18 Schwartz-Zippel theorem, 70
self-reducible, 25
one-way function, 64 Selman, 38
optimization problem, 76 set, 2
oracle machine, 27 Sipser, 49, 59
Skolem function, 58
p-isomorphic, 39 Skolemization, 58
p-m-degree, 16 Solovay, 36, 37, 53
p-m-reducible, 16 soundness
p-T-degree, 28 resolution, 84
padding, 25 space constructible, 6
parity function, 61 space hierarchy theorem, 8
PCP Theorem, 77 sparse, 38, 43, 59
pigeonhole principle, 82 speed-up theorem, 41
polynomial hierarchy, 28 Sterne, 35
positive reductions, 33 Strassen, 53
Pratt, 11 Szelepcsényi, 12
primitive root, 4, 5
private coin, 69 tally, 43
probabilistic method, 49, 80 tape compression, 7
probabilistic Turing machine, 45 time constructible, 6, 40
probabilistic verifier, 77 time hierarchy theorem, 8, 40
problem, 2 Toda, 51
propositional proof system, 81 tour, 12
prover, 67 Trakhtenbrot, 40
PTAS, 77 traveling salesman problem, 12, 75
truth table, 82
quadratic residue, 51 Tseitin, 84
96 Index

Turing machine, 2
Turing reducible, 27

unambiguous machine, 65
Use Principle, 36

verifier, 67, 77
von Neumann, 25

Wittgenstein, 82
word, 1

XOR, 60

zero-knowledge proof, 72