Cantor’s Theorem

Theorem 7 The set of all subsets of N (2N ) is infinite and

not countable.
• Suppose (2N ) is countable with f : N → 2N being a
bijection.a
• Consider the set B = {k ∈ N : k ∈ f (k)} ⊆ N.
• Suppose B = f (n) for some n ∈ N.
a Note that f (k) is a subset of N.

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 144
 The Proof (concluded)
• If n ∈ f (n) = B, then n ∈ B, but then n ∈ B by B’s
definition.
• If n ∈ f (n) = B, then n ∈ B, but then n ∈ B by B’s
definition.
• Hence B = f (n) for any n.
• f is not a bijection, a contradiction.

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 145
 Georg Cantor (1845–1918)

Kac and Ulam (1968), “[If] one

had to name a single person
whose work has had the most
decisive influence on the present
spirit of mathematics, it would
almost surely be Georg Cantor.”

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 146
 Cantor’s Diagonalization Argument Illustrated
0 1 2 3 4 5

f(0)

f(1)

f(2)

f(3)

f(4)

f(5)

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 147
 A Corollary of Cantor’s Theorem
Corollary 8 For any set T , finite or infinite,

| T | < | 2T |.

• The inequality holds in the finite T case as k < 2k .

• Assume T is infinite now.
• | T | ≤ |2T |.
– Consider f (x) = {x} ∈ 2T .
– f maps a member of T = { a, b, c, . . . } to the
corresponding member of { { a }, { b }, { c }, . . . } ⊆ 2T .

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 148
 The Proof (concluded)
• | T | = |2T |.
– Use the same argument as Cantor’s theorem.

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 149
 A Second Corollary of Cantor’s Theorem
Corollary 9 The set of all functions on N is not countable.
• It suffices to prove it for functions from N to {0, 1}.
• Every function f : N → {0, 1} determines a subset of N:

{n : f (n) = 1} ⊆ N,

and vice versa.

• So the set of functions from N to {0, 1} has cardinality
| 2N |.
• Cantor’s theorem (p. 144) then implies the claim.

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 150
 Existence of Uncomputable Problems
• Every program is a finite sequence of 0s and 1s, thus a
nonnegative integer.a
• Hence every program corresponds to some integer.
• The set of programs is therefore countable.
a Different
binary strings may be mapped to the same integer (e.g.,
“001” and “01”). To prevent it, use the lexicographic order as the map-
ping or simply insert “1” as the most significant bit of the binary string
before the mapping (so “001” becomes “1001”). Contributed by Mr.
Yu-Chih Tung (R98922167) on October 5, 2010.

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 151
 Existence of Uncomputable Problems (concluded)
• A function is a mapping from integers to integers.
• The set of functions is not countable by Corollary 9
(p. 150).
• So there are functions for which no programs exist.a
a Asa nondeterministic program may not compute a function, we
consider only deterministic programs for this sentence. Contributed by
Mr. Patrick Will (A99725101) on October 5, 2010.

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 152
 Universal Turing Machinea
• A universal Turing machine U interprets the input
as the description of a TM M concatenated with the
description of an input to that machine, x.
– Both M and x are over the alphabet of U .
• U simulates M on x so that

U (M ; x) = M (x).

• U is like a modern computer, which executes any valid

machine code, or a Java Virtual machine, which
executes any valid bytecode.
a Turing (1936).

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 153
 The Halting Problem
• Undecidable problems are problems that have no
algorithms.
– Equivalently, they are languages that are not
recursive.
• We knew undecidable problems exist (p. 151).
• We now define a concrete undecidable problem, the
halting problem:

H = {M ; x : M (x) =}.

– Does M halt on input x?

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 154
 H Is Recursively Enumerable
• Use the universal TM U to simulate M on x.
• When M is about to halt, U enters a “yes” state.
• If M (x) diverges, so does U .
• This TM accepts H.

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 155
 H Is Not Recursivea
• Suppose H is recursive.
• Then there is a TM MH that decides H.
• Consider the program D(M ) that calls MH :
1: if MH (M ; M ) = “yes” then
2: ; {Writing an infinite loop is easy.}
3: else
4: “yes”;
5: end if
a Turing (1936).

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 156
 H Is Not Recursive (concluded)
• Consider D(D):
– D(D) =⇒ MH (D; D) = “yes” ⇒ D; D ∈ H ⇒
D(D) =, a contradiction.
– D(D) = “yes” ⇒ MH (D; D) = “no” ⇒ D; D ∈ H ⇒
D(D) =, a contradiction.

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 157
 Comments
• Two levels of interpretations of M :a
– A sequence of 0s and 1s (data).
– An encoding of instructions (programs).
• There are no paradoxes with D(D).
– Concepts should be familiar to computer scientists.
– Feed a C compiler to a C compiler, a Lisp interpreter
to a Lisp interpreter, a sorting program to a sorting
program, etc.
a Eckert and Mauchly (1943); von Neumann (1945); Turing (1946).

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 158
 Cantor’s Paradox (1899a )
• Let T be the set of all sets.b
• Then 2T ⊆ T because 2T is a set.
• But we knowc | 2T | > | T | (p. 148)!
• We got a “contradiction.”
• Are we willing to give up Cantor’s theorem?
• If not, what is a set?
a In
a letter to Richard Dedekind. First published in Russell (1903).
b Recall this ontological argument for the existence of God by

St Anselm (1033–1109) in the 11th century: If something is possible

but is not part of God, then God is not the greatest possible object of
thought, a contradiction.
c Really?

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 159
 Self-Loop Paradoxesa
Russell’s Paradox (1901): Consider R = {A : A ∈ A}.
• If R ∈ R, then R ∈ R by definition.
• If R ∈ R, then R ∈ R also by definition.
• In either case, we have a “contradiction.”b
Eubulides: The Cretan says, “All Cretans are liars.”
Liar’s Paradox: “This sentence is false.”
Hypochondriac: a patient (like Gödel) with imaginary
symptoms and ailments.
a E.g.,
Hofstadter, Gödel, Escher, Bach: An Eternal Golden Braid
(1979) or Quine, The Ways of Paradox and Other Essays (1966).
b Gottlob Frege (1848–1925) to Bertrand Russell in 1902, “Your dis-

covery of the contradiction [. . .] has shaken the basis on which I intended

to build arithmetic.”

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 160
 Self-Loop Paradoxes (continued)
Sharon Stone in The Specialist (1994): “I’m not a
woman you can trust.”
Spin City (1996–2002): “I am not gay, but my boyfriend
is.”
Numbers 12:3, Old Testament: “Moses was the most
humble person in all the world [· · · ]” (attributed to
Moses).

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 161
 Self-Loop Paradoxes (concluded)
The Egyptian Book of the Dead: “ye live in me and I
would live in you.”
John 17:21, New Testament: “just as you are in me and
I am in you.”
Pagan & Christian Creeds (1920): “I was moved to
Odin, myself to myself.”
Soren Kierkegaard in Fear and Trembling (1843):
“to strive against the whole world is a comfort, to strive
with oneself is dreadful.”

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 162
 Bertrand Russell (1872–1970)

Karl Popper (1974), “per-

haps the greatest philoso-
pher since Kant.”

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 163
 Reductions in Proving Undecidability
• Suppose we are asked to prove that L is undecidable.
• Suppose L (such as H) is known to be undecidable.
• Find a computable transformation R (called reduction)
from L to L such thata

∀x {x ∈ L if and only if R(x) ∈ L}.

• Now we can answer “x ∈ L ?” for any x by asking

“R(x) ∈ L?” because they have the same answer.
– L is said to be reduced to L.
a Contributed by Mr. Tai-Dai Chou (J93922005) on May 19, 2005.

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 164
 algorithm for L

x R(x) algorithm yes/no

R
for L

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 165
 Reductions in Proving Undecidability (concluded)
• If L were decidable, “R(x) ∈ L?” becomes computable
and we have an algorithm to decide L , a contradiction!
• So L must be undecidable.
Theorem 10 Suppose language L1 can be reduced to
language L2 . If L1 is not recursive, then L2 is not recursive.

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 166
 More Undecidability
• H ∗ = {M : M halts on all inputs}.
– We will reduce H to H ∗ .
– Given the question “M ; x ∈ H?”, construct the
following machine (this is the reduction):a

Mx (y) {M (x); }

– M halts on x if and only if Mx halts on all inputs.

– In other words, M ; x ∈ H if and only if Mx ∈ H ∗ .
– So if H ∗ were recursive (recall the box for L on p.
165), H would be recursive, a contradiction.
a Simplified
by Mr. Chih-Hung Hsieh (D95922003) on October 5, 2006.
Mx ignores its input y; x is part of Mx ’s code but not Mx ’s input.

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 167
 More Undecidability (concluded)
• {M ; x : there is a y such that M (x) = y}.
• {M ; x : the computation M on input x uses all states of M }.
• {M ; x; y : M (x) = y}.

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 168
 Complements of Recursive Languages
The complement of L, denoted by L̄, is the language
Σ∗ − L.
Lemma 11 If L is recursive, then so is L̄.
• Let L be decided by M , which is deterministic.
• Swap the “yes” state and the “no” state of M .
• The new machine decides L̄.a
a Recall p. 111.

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 169
 Recursive and Recursively Enumerable Languages
Lemma 12 (Kleene’s theorem) L is recursive if and
only if both L and L̄ are recursively enumerable.
• Suppose both L and L̄ are recursively enumerable,
accepted by M and M̄ , respectively.
• Simulate M and M̄ in an interleaved fashion.
• If M accepts, then halt on state “yes” because x ∈ L.
• If M̄ accepts, then halt on state “no” because x ∈ L.
• Note that either M or M̄ (but not both) must accept
the input and halt.

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 170
 A Very Useful Corollary and Its Consequences
Corollary 13 L is recursively enumerable but not recursive,
then L̄ is not recursively enumerable.
• Suppose L̄ is recursively enumerable.
• Then both L and L̄ are recursively enumerable.
• By Lemma 12 (p. 170), L is recursive, a contradiction.
Corollary 14 H̄ is not recursively enumerable.a
a Recall that H̄ = {M ; x : M (x) =}.

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 171
 R, RE, and coRE
RE: The set of all recursively enumerable languages.
coRE: The set of all languages whose complements are
recursively enumerable.
R: The set of all recursive languages.
• Note that coRE is not RE.
– coRE = { L : L ∈ RE } = { L : L ∈ RE }.
– RE = { L : L ∈ RE }.

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 172
 R, RE, and coRE (concluded)
• R = RE ∩ coRE (p. 170).
• There exist languages in RE but not in R and not in
coRE.
– Such as H (p. 155, p. 156, and p. 171).
• There are languages in coRE but not in RE.
– Such as H̄ (p. 171).
• There are languages in neither RE nor coRE.

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 173
 RE coRE
R

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 174
 Undecidability in Logic and Mathematics
• First-order logic is undecidable (answer to Hilbert’s
“Entscheidungsproblem” (1928)).a
• Natural numbers with addition and multiplication is
undecidable.b
• Rational numbers with addition and multiplication is
undecidable.c
a Church (1936).
b Rosser(1937).
c Robinson (1948).

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 175
 Undecidability in Logic and Mathematics (concluded)
• Natural numbers with addition and equality is decidable
and complete.a
• Elementary theory of groups is undecidable.b
a Presburger’sMaster’s thesis (1928), his only work in logic. The
direction was suggested by Tarski. Mojz̄esz Presburger (1904–1943) died
in a concentration camp during World War II.
b Tarski (1949).

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 176
 Julia Hall Bowman Robinson (1919–1985)

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 177
 Alfred Tarski (1901–1983)

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 178
 Boolean Logic

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 179
 It seemed unworthy of a grown man
to spend his time on such trivialities,
but what was I to do? [· · · ]
The whole of the rest of my life might be
consumed in looking at
that blank sheet of paper.
— Bertrand Russell (1872–1970),
Autobiography, Vol. I (1967)

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 180
 Boolean Logica
Boolean variables: x1 , x2 , . . ..

Literals: xi , ¬xi .

Boolean connectives: ∨, ∧, ¬.

Boolean expressions: Boolean variables, ¬φ (negation),

φ1 ∨ φ2 (disjunction), φ1 ∧ φ2 (conjunction).

• n i=1 φi stands for φ1 ∨ φ2 ∨ · · · ∨ φn .

• n i=1 φi stands for φ1 ∧ φ2 ∧ · · · ∧ φn .

Implications: φ1 ⇒ φ2 is a shorthand for ¬φ1 ∨ φ2 .

Biconditionals: φ1 ⇔ φ2 is a shorthand for

(φ1 ⇒ φ2 ) ∧ (φ2 ⇒ φ1 ).
a George Boole (1815–1864) in 1847.

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 181
 Truth Assignments
• A truth assignment T is a mapping from boolean
variables to truth values true and false.
• A truth assignment is appropriate to boolean
expression φ if it defines the truth value for every
variable in φ.
– {x1 = true, x2 = false} is appropriate to x1 ∨ x2 .
– {x2 = true, x3 = false} is not appropriate to
x1 ∨ x2 .

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 182
 Satisfaction
• T |= φ means boolean expression φ is true under T ; in
other words, T satisfies φ.
• φ1 and φ2 are equivalent, written

φ1 ≡ φ2 ,

if for any truth assignment T appropriate to both of

them, T |= φ1 if and only if T |= φ2 .

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 183
 Truth Tables
• Suppose φ has n boolean variables.
• A truth table contains 2n rows.
• Each row corresponds to one truth assignment of the n
variables and records the truth value of φ under that
truth assignment.
• A truth table can be used to prove if two boolean
expressions are equivalent.
– Just check if they give identical truth values under all
appropriate truth assignments.

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 184
 A Truth Table
p q p∧q
0 0 0
0 1 0
1 0 0
1 1 1

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 185
 A Second Truth Table
p q p∨q
0 0 0
0 1 1
1 0 1
1 1 1

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 186
 A Third Truth Table
p ¬p
0 1
1 0

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 187
 De Morgan’s Lawsa
• De Morgan’s laws say that
¬(φ1 ∧ φ2 ) ≡ ¬φ1 ∨ ¬φ2 ,
¬(φ1 ∨ φ2 ) ≡ ¬φ1 ∧ ¬φ2 .

• Here is a proof of the first law:

φ1 φ2 ¬(φ1 ∧ φ2 ) ¬φ1 ∨ ¬φ2
0 0 1 1
0 1 1 1
1 0 1 1
1 1 0 0
a Augustus DeMorgan (1806–1871) or William of Ockham (1288–
1348).

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 188
 Conjunctive Normal Forms
• A boolean expression φ is in conjunctive normal
form (CNF) if
n
φ= Ci ,
i=1
where each clause Ci is the disjunction of zero or more
literals.a
– For example,

(x1 ∨ x2 ) ∧ (x1 ∨ ¬x2 ) ∧ (x2 ∨ x3 ).

• Convention: An empty CNF is satisfiable, but a CNF

containing an empty clause is not.
a Improved by Mr. Aufbu Huang (R95922070) on October 5, 2006.

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 189
 Disjunctive Normal Forms
• A boolean expression φ is in disjunctive normal form
(DNF) if
n
φ= Di ,
i=1
where each implicant Di is the conjunction of zero or
more literals.
– For example,

(x1 ∧ x2 ) ∨ (x1 ∧ ¬x2 ) ∨ (x2 ∧ x3 ).

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 190
 Clauses and Implicants

• The of clauses remains a clause.
– For example,

(x1 ∨ x2 ) ∨ (x1 ∨ ¬x2 ) ∨ (x2 ∨ x3 )

= x1 ∨ x2 ∨ x1 ∨ ¬x2 ∨ x2 ∨ x3 .

• The of implicants remains an implicant.
– For example,

(x1 ∧ x2 ) ∧ (x1 ∧ ¬x2 ) ∧ (x2 ∧ x3 )

= x1 ∧ x2 ∧ x1 ∧ ¬x2 ∧ x2 ∧ x3 .

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 191
 Any Expression φ Can Be Converted into CNFs and DNFs
φ = xj :
• This is trivially true.
φ = ¬φ1 and a CNF is sought:
• Turn φ1 into a DNF.
• Apply de Morgan’s laws to make a CNF for φ.
φ = ¬φ1 and a DNF is sought:
• Turn φ1 into a CNF.
• Apply de Morgan’s laws to make a DNF for φ.

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 192
 Any Expression φ Can Be Converted into CNFs and DNFs
(continued)
φ = φ1 ∨ φ2 and a DNF is sought:
• Make φ1 and φ2 DNFs.
φ = φ1 ∨ φ2 and a CNF is sought:
• Turn φ1 and φ2 into CNFs,a
n1
 n2

φ1 = Ai , φ2 = Bj .
i=1 j=1

• Set
n1 
 n2
φ= (Ai ∨ Bj ).
i=1 j=1
a Corrected by Mr. Chun-Jie Yang (R99922150) on November 9, 2010.

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 193
 Any Expression φ Can Be Converted into CNFs and DNFs
(concluded)
φ = φ1 ∧ φ2 and a CNF is sought:
• Make φ1 and φ2 CNFs.
φ = φ1 ∧ φ2 and a DNF is sought:
• Turn φ1 and φ2 into DNFs,
n1
 n2

φ1 = Ai , φ2 = Bj .
i=1 j=1

• Set
n1 
 n2
φ= (Ai ∧ Bj ).
i=1 j=1

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 194
 An Example: Turn ¬((a ∧ y) ∨ (z ∨ w)) into a DNF

¬((a ∧ y) ∨ (z ∨ w))
¬(CNF∨CNF)
= ¬(((a) ∧ (y)) ∨ ((z ∨ w)))
¬(CNF)
= ¬((a ∨ z ∨ w) ∧ (y ∨ z ∨ w))
de Morgan
= ¬(a ∨ z ∨ w) ∨ ¬(y ∨ z ∨ w)
de Morgan
= (¬a ∧ ¬z ∧ ¬w) ∨ (¬y ∧ ¬z ∧ ¬w).

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 195
 Satisfiability
• A boolean expression φ is satisfiable if there is a truth
assignment T appropriate to it such that T |= φ.
• φ is valid or a tautology,a written |= φ, if T |= φ for all
T appropriate to φ.
• φ is unsatisfiable if and only if φ is false under all
appropriate truth assignments if and only if ¬φ is valid.
a Wittgenstein(1889–1951) in 1922. Wittgenstein is one of the
most important philosophers of all time. “God has arrived,” the great
economist Keynes (1883–1946) said of him on January 18, 1928. “I met
him on the 5:15 train.” Russell (1919), “The importance of ‘tautology’
for a definition of mathematics was pointed out to me by my former
pupil Ludwig Wittgenstein, who was working on the problem. I do not
know whether he has solved it, or even whether he is alive or dead.”

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 196
 Ludwig Wittgenstein (1889–1951)

Wittgenstein (1922), “Whereof one

cannot speak, thereof one must be
silent.”

c
2014 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 197