1 Background

We give a basic overview of the mathematical background required for this

course.
1.1 Set Theory
We introduce some concepts from naive set theory (as opposed to axiomatic
set theory). The rst part covers the basic denitions of set theory, and
is covered by any basic course in discrete mathematics. The second part
covers some results regarding the cardinality of sets, and is necessary for the
arguments that you will encounter in this course.
Set Theory - Basic denitions
A set is informally dened as a collection of objects. The concept of a set in
this setting is not precisely dened. If x is a member of the set A, we say x
belongs to A, and write x A. If, every element of a set A belongs to a set
B, we say that A is a subset of B, written as A B. If both A B and
B A, we say that A and B are equal, written as A = B.
The operations on sets that we typically consider are the following.
1. Union: If A and B are sets, their union, written A B is dened as
A B = x [ x A or x B.
2. Intersection: If A and B are sets, their intersection, written as AB,
is dened as
A B = x [ x A and x B.
3. Relative Complement: If A and B are sets, then the complement of B
relative to A, written as A B, is dened as
A B = x : x A and x / B.
If we have the notion of a universal set, denoted U, the superset of all sets
in the domain of interest, then it is possible to dene the following notion.
5. Absolute Complement: If A is a set, the absolute complement of A,
written A
c
is dened as U A.
1
Theorem 1.1.1 (De Morgans Laws). The following hold for arbitrary sets
A and B.
(A B)
c
= A
c
B
c
.
and
(A B)
c
= A
c
B
c
.
We have the notion of a power set, which is the collection of all subsets
of a set.
6. Power Set: If A is a set, then there is a set, called the power set of A,
written T(A) (or, as 2
A
), is the set of all subsets of A. In symbols,
T(A) = B [ B A.
The next operation that we dene is the Cartesian product of sets. For
this, we have to rst dene what an ordered pair of two objects is. A
sequence of elements
abcd
where we say that the rst element is a, the second b, the third c and
the fourth d, in the specied order is actually a function f : 0, 1, 2, 3
a, b, c, d specied by
f(0) = a, f(1) = b, f(2) = c, f(3) = d.
Since we have not dened functions yet, we cannot use this approach to
dene an ordered pair as a collection (a, b) with the rst element being a
and the second being b. We proceed as follows.
Denition 1.1.2. The ordered pair of a
1
and a
2
with rst coordinate a
1
and the second coordinate a
2
, written (a
1
, a
2
) is dened by
(a
1
, a
2
) = a
1
, a
1
, a
2
.
Though this might appear strange at rst, we have succeeded in dening
an ordered pair of arbitrary elements without appealing either to the concept
of an index (or a number) or a mapping from an index to an element. The
rst coordinate can now be specied using only the set operations thus, as
the unique element of
1

A(a,b)
A = a
1
a
1
, a
2

1
Thanks to Pervez Khan for correcting some errors in a previous version of the deni-
tion.
2
The second co-ordinate may be dened as the unique element of

A[(a,b)\{a
1
}]
A

a
1
.
This strategy may be extended to dene the ordered tuples of arbitrar-
ily many elements. With this denition, it is now possible to dene the
Cartesian Product of sets.
7. Cartesian Product: If A and B are two sets, their cartesian product
of A with B, written AB, is dened as:
AB = (a, b) [ a A and b B.
A relation between two sets A and B is just a subset of the cartesian
product AB. A is called the domain of the relation, and B its range.
Denition 1.1.3. A function f from A to B, written f : A B, is a
relation from A to B with the restriction that for every element a in A,
there is exactly one ordered pair (a, b) f. A is called the domain of f and
B its range. For any ordered pair (a, b) f, we say that b is the image of
a under f, written f(a). The preimage of b B under f is the set of all
elements a A such that f(a) = b.
Example 1.1.4. If A = a, b, c, d and B = 1, 2, 3, 4, and f = (a, 1), (b, 1), (c, 3), (d, 4),
then f(a) = 1, f(b) = 1, and f
1
(1) = a, b.
We can restate the restriction of f as saying that every element in A has
exactly one image in B.
We specify the following classes of functions for later discussion.
A function f : A B is said to be injective, or one-to-one if, the
elements of A have distinct images in B. In symbols,
a
1
, a
2
A f(a
1
) = f(a
2
) implies a
1
= a
2
.
The function f is said to be surjective, or onto if every element in B has a
pre-image. In symbols,
b Ba A f(a) = b.
If f is both one-to-one and onto, then f is said to be bijective.
3
Cardinality
We now discuss questions of cardinality, which will be used to compare the
sizes of sets. For any nite set, we can dene the cardinality as the number
of elements in it. Suppose A and B are nite sets. It is easy to show that
if A and B have equal cardinality, then there is a bijection f : A B, . If
the cardinality of A is at most that of B, then there is a one-to-one function
f : A B. If the cardinality of B is at most that of A, then there is an
onto function f : A B.
When we consider innite sets, however, this approach falters. We have
no way of distinguishing among the cardinalities of innite sets if we use this
concept. We also run into seeming paradoxes: Galileo posed the question,
Which set is bigger - the set of even numbers or the set of numbers?
Obviously, the set of even numbers is a strict subset of the set of numbers.
But, if we multiply a natural number with 2, then we get a unique even
number, thus there is a bijection between these two sets. This example shows
the puzzling nature of innite sets they can be isomorphic to strict subsets
of themselves. A way to resolve the questions of cardinality consistently is
to take the reverse approach of our initial approach to the cardinality of
nite sets: Say, for example, that cardinality of A is less than or equal to
that of B if there is a one-to-one function f : A B. We elaborate this
approach, and study the properties of cardinalities of innite sets, and delve
into some famous examples.
This treatment follows Halmos. If there is a injection f : A B, then
we say that A is dominated by B, and write A B.
This suggests the idea that A and B are equivalent (not equal sets),
written A B if there is a bijection from f : A B. In the case of
comparison operators between numbers m and n, we know that if m n
and n m, then m = n. Can we dene a notion that if A B and A B,
then A B?
This is a very dicult theorem in set theory, known as the Cantor-
Schroder-Bernstein Theorem.
Theorem 1.1.5 (Cantor-Schroder-Bernstein). Let A and B be two arbitrary
sets. If there is a one-to-one function f : A B and a one-to-one function
g : B A, then there is a bijection h : A B.
We omit the proof, and go into the discussion of countability of sets. An
innite set is one, where, even if we remove any nite number of elements,
there are still elements left. Removing an element is a process of assigning a
natural number index to some particular element of the set - the only thing
4
we forbid is repetition, since no element can be removed multiple times.
This gives us the following denition of an innite set.
Denition 1.1.6. A set A is innite if N A.
This gives us an idea of the smallest kind of innite sets. If an innite
set A N, then A N. We call such sets countably innite sets. Count-
ably innite sets have the same cardinality as N. It is customary to call
countable any set that is either nite, or is countably innite.
Denition 1.1.7. A set A is said to be countable if there is an onto function
f : N A.
The set of natural numbers N is an obvious example of such a set. The
set of even numbers is also a countably innite set. We show that the set of
rational numbers (usually written as Q), is also countable, in the following
sequence of theorems.
First, we show that if we take a nite union of countable sets, the result-
ing set is countable.
Theorem 1.1.8. If A
0
, A
1
, . . . , A
n1
are pairwise disjoint countable sets,
then their union is also countable.
Proof. Since each A
i
, i = 0 to n 1, is countable, there are functions
f
i
: N A
i
which are onto. We construct a function g : N
n1
i=0
A
i
to
show that the union of these sets is countable.
Let g(i) = f
i mod n
(i/n) where / denotes integer division.
If each f
i
: N A
i
is onto, we can see that g : N
n
i=1
A
i
is also onto.
For, the k
th
element in the m
th
set will be g(k n + m) (We start counting
from 0).
Intuitively, if we let all A
i
s be countably innite, we can explain as
follows: g enumerates the elements of the union in the form of a two-
dimensional array: there are exactly n columns, the column i enumerating
the elements of A
i
. g goes row-by-row, enumerating the rst elements of all
the A
i
s rst, then the second elements and so on. In this manner, since each
A
i
is countable, we can count all the elements in the union.
Theorem 1.1.9. A union of countably many nite sets is countable.
Intuitively, since each A
i
is nite, we can enumerate the elements of A
0
until they are exhausted, and then start enumerating the elements of A
1
5
and so on.
2
In this way, there is a nite index at which any given element
of a given set will be enumerated.
Proof. Consider
/ =

i=0
A
i
where each A
i
is a nite set. Then, if we construct an onto function g : N
/, it will prove that / is countable.
Let [A
i
[ denote the number of elements in A
i
. Since each A
i
is nite,
there is a bijection f
i
: A
i
0, 1, . . . , [A
i
[ 1. Since
[A
0
[, [A
0
[ +[A
1
[, [A
0
[ +[A
1
[ +[A
2
[, . . .
is an increasing sequence, for any j, there is a least index n such that
j < [A
0
[ +[A
1
[ + +[A
n
[.
Then
g(j) = f
n
(j [[A
0
[ +[A
1
[ + +[A
n1
[]) .
To show that g is onto, we have to show that for any arbitrary i, and m
such that 0 m < [A
i
[, there is some index j such that g(j) = f
i
(m). This
follows routinely from the denition of g.
The above theorem is one of the most useful tools in proving the count-
ability of sets - when we have to prove that some class of sets is countable,
what we have to do essentially is to rethink of the same class as a countable
union of nite sets. A brilliant example is the following theorem.
With this, we now can show that the set of non-negative rational numbers
is countable. We will actually show something more, that NN is countable.
Since every non-negative rational number can be represented as a ratio
p
q
where p is nonnegative and q ,= 0, what we prove amounts to proving that
a bigger set is countable.
Theorem 1.1.10. The cross-product of two countable sets is countable.
The proof uses a technique which is sometimes picturesquely called
dovetailing. The idea can be paraphrased as follows. Consider a rectan-
gular lattice where the x-axis corresponds to the elements of a countable
set A and the y-axis corresponds to a countable set B. (Figure 1).
2
This could not be done in the previous theorem because each Ai was innite, so we
would never have been done enumerating A0 at a nite time.
6
Figure 1: Illustration of dovetailing
Proof. There are onto functions g : N A and h : N B which are given.
We dene an onto function f : N AB that enumerates ordered pairs in
in A B in the following manner. First, (g(0), h(0)) is enumerated. In the
next pass, we enumerate (g(1), h(0)), (g(0), h(1)). In the following pass, we
enumerate (g(2), h(0)), (g(1), h(1)) and (g(0), h(2)).
Thus in pass c, we enumerate pairs (g(i), h(j)) such that i +j = c. This
enables us to distinguish among dierent passes. Within the same pass, we
can distinguish among the pairs
(g(i + j), h(0)), (g(i + j 1), h(1)), . . . , (g(0), h(i + j))
by their second co-ordinates. Thus there are i +j +1 elements in pass i +j.
Let a pair (g(i), h(j)) be given. Then it will be enumerated in pass i +j.
Let us denote i+j by c. The total number of elements in passes 0, 1, . . . , c1
is equal to
T[c 1] =
c1

i=1
(i + 1) =
c(c + 1)
2
.
Now, in pass c, the pair (g(i), h(j)) will be enumerated in position j.
Thus, the onto function f : N AB we dene is
f(T[i + j 1] + j) = (g(i), h(j)).
However, there are sets of higher cardinalities. We prove a particular
case of the general theorem. This proof follows what is usually called a
diagonalization argument. (Please dont confuse this with dovetailing!)
7
Theorem 1.1.11. There is no bijection f : N T(N).
Proof. Suppose there is a bijection f : N T(N). Dene the following set.
S = i : i / f(i).
Then, we prove that there is no j such that f(j) = S, thus contradicting
that every subset of N has been enumerated by f. For, if we consider whether
j is present in S, we nd that if j is in the set f(j), then it is not in S.
Conversely, if j is not in the set f(j), then it is in S. Thus S diers from
the set f(j) in at least one element, hence cannot be the same as f(j).
Since this is true for any j, S is not in the image of f.
We briey describe the characteristic function of a set of numbers. Let
S be a set of numbers. The characteristic function
S
: N 0, 1 is the
function dened by
S
(i) = 1 if i S and
S
(i) = 0 otherwise. Intuitively,
this can be seen as a bit-map of S. Every set of numbers has a unique
characteristic function.
The cardinality of the power set of natural numbers is the same as the
cardinality of the real numbers in [0, 1) (the unit interval with all numbers
r where 0 r < 1).
To see this, we use the binary representation of the real numbers. A
set of numbers can be represented by its characteristic sequence b
0
b
1
b
2
. . .
where b
i
= 1 if s
i
is present in the set and b
i
= 0 otherwise. Let us dene
a mapping from such a sequence to the number

i=1
b
i1
/2
i
. This denes
an onto function from T(N) to [0, 1), the cardinality of [0, 1) is at most as
much as T(N).
To see that the cardinality of [0, 1) is at least that of T(N), we dene an
onto function from [0, 1) to T(N).
It is natural to consider the mapping dened by the binary representation
of reals in the interval. However, wee cannot use the binary representation
of real numbers in the unit interval for this purpose, since dyadic rationals,
that is, rationals of the form i/2
m
, have two binary representations. For
example, 0.5 has the representations 0.1000 . . . and 0.0111 . . . . If we pick
one representation, and dene g(0.5) = 1000 . . . , where the binary sequence
is the characterestic sequence of a set, then we do not have an onto function
The set dened by 0111 . . . does not have an inverse image.
To solve this issue, we adopt the ternary notation of the reals in the
interval. We dene an onto function from [0, 1) to T(N) as follows. If
a ternary expansion of a number does not have the digit 2, that is, the
number can be written in the form

i=1
t
i1
/3
i
where each t
i1
is 0 or 1,
8
then its image is the set represented by the characteristic sequence t
0
t
1
t
2
. . . .
There are numbers in the interval with innitely many 2s in their ternary
expansion - the image of such numbers is dened to be N, merely so that
we have a total function from [0, 1) to T(N). Since this function is onto, we
can conclude that the cardinality of [0, 1) is at least that of T(N).
3
1.2 Computability Theory
In this part, we will briey cover notions of computability. We will rst
dene a Turing machine which accepts/decides a language, and then dene
the notion of a computable function.
Denition 1.2.1. A one-way innite, 2-tape Turing Machine is an 8-tuple
(Q, , , B, , q
0
, q
a
, q
r
) where
Q is the set of states of the Turing machine.
is the nite set of input alphabet symbols.
is the nite set of tape alphabet symbols, . Here, a strict
subset.
B is a reserved tape alphabet symbol called the blank, not part of the
input alphabet. (B .)

: (Qq
a
, q
r
)
2
Q
2
L, R
2
is the transition function of the Turing machine.
q
0
Q is the initial state of the Turing machine.
q
a
Q is the accept state.
q
r
Q is the reject state.
A conguration of the Turing machine consists of the state, the contents
of the 2 tapes, and the position of the tape heads. An input string w is said
to be accepted by a Turing machine if, the computation of M with initial
conguration having w on the rst tape and both heads at the left end of
w, terminates in q
a
. The machine is said to reject the string if the Turing
machine terminates in q
r
. (There is of course, the possibility that the Turing
Machine may not terminate its execution.)
3
Thanks to Arpita Korwar and Rohit Gurjar for pointing this out.
9
A language L is a set of strings. Recall that a Turing machine is said
to accept a language L if every string x in the language is accepted by the
Turing Machine in the above sense, and no other string is accepted. If a
string is not in the language, then it might either be rejected by the Turing
machine, or it may cause the Turing machine to run forever.
Denition 1.2.2. A language is said to be acceptable if there is a Turing
machine which accepts it.
Denition 1.2.3. A language L is said to be decidable if both L and L
c
are acceptable.
Equivalently, we may dene a decidable language L as follows. There
is a Turing machine which accepts every string in the language, and rejects
every other string. (HW 1A)
It is possible to dene a Turing machine that computes a function from
strings to strings. Then we will proceed to dene a computably enumerable
language.
Denition 1.2.4. A one-way innite, 2-tape Turing Machine is said to
compute f :

if the machine, presented with the input string x in

the initial conguration, terminates in state q
a
with f(x)B on the second
tape whenever f(x) is dened, and runs forever if f(x) .
A function f is said to be partial computable if there is a Turing machine
which computes f in the above sense. A partial computable function f is
called total computable if there is a Turing machine computing f that halts
on all inputs.
To dene functions which operate on numbers, it is necessary to pick
a representation of numbers. The standard enumeration of strings is the
enumeration
, 0, 00, 01, 10, 11, 000, . . .
- that is, the enumeration which lists all strings of a shorter length before any
longer string, and within any length, lists strings in lexicographic order. For
any number i, let s
i
denote the i
th
string in this sequence. Then s : N

is
a bijection between numbers and strings, and can be used as a representation
of numbers.
(There is an easy closed form formula for s, that is, given a number
n > 0, nd s
n
. Convert n+1 to binary, and strip the leading 1 in the binary
representation. This will give s
n
. (Please verify!))
10
Denition 1.2.5. A function f : N N is partial computable if there is a
partial computable function h :

such that for every number n 0

f(n) = s
1
h(s
n
).
It is now routine to dene unary partial computable functions which
map any given countably innite domain to any countably innite range.
We will use the following function to represent a pair of strings.
Denition 1.2.6. The bit-doubling function
bd :

is dened by
bd() =
bd(x0) = bd(x)00
bd(x1) = bd(x)11
The bit doubling function of x is a string in which every bit in x is
doubled. For example, the bd(0010) = 00 00 11 00.
Denition 1.2.7. The pairing function
, ) :

is dened by
x, y) = bd(x)01bd(y)01
where x and y are strings.
We observe that the number of times 01 occurs in any encoding, is
exactly 2, and all valid encodings end in 01.
Lemma 1.2.8. The pairing function is a prex code - that is, the encoding
of a pair cannot be the prex of the encoding of another pair.
Proof. Suppose
bd(x)01bd(y)01w
is the encoding of some pair. Since the number of times 01 can occur
in the encoding is 2, 01 is forbidden from w. However, the sux of the
encoding 01w should end in 01. These conditions together imply that
w = .
Thus, the encoding of a pair cannot be the prex of another pair.
11
For ease of description, it is preferrable to have a high-level description
of Turing Machines so that we do not have to give detailed programs for
doing elementary operations every time we specify a Turing machine. For
this, the common approach is to use a high-level language, and prove that
every elementary operation, as well as every programming construct like
loops, can be simulated by a Turing machine. Once we do this, it follows
that every program we write in the high level language corresponds to a
Turing Machine. We will assume that programs written in an ALGOL-like
language can be computed using Turing machines. We will describe our
programs using this toy language.
Now, we can make the following analogy:
partial computable: total computable :: acceptable : decidable.
We will explore this analogy. For this, we need to now dene Turing
machines which enumerates a language. This is the computable analogue
of an onto function f : N L with which we dened countability of L.
Intuitively, Turing machine that enumerates L will keep printing the ele-
ments of L one after another. However, we would like every Turing machine
outputting a string to halt in a nite time, so we model the machine such
that for a given n, it will output the n
th
element in the enumeration.
Denition 1.2.9. A language L is said to be computably enumerable if it
is either nite, or there is a total computable bijection f : N L.
Note that, L can be the empty language.
Theorem 1.2.10. A language is computably enumerable if and only if it is
acceptable.
Proof. We rst prove that if a language is computably enumerable, then
it is acceptable. If a language is nite, then it is acceptable by a Turing
Machine which hardcodes the strings in the language in a nite lookup
table, and accepts exactly those strings. Suppose a computably enumerable
language L is innite. Then there is a computable bijection f : N L.
We describe a Turing machine that accepts L as follows. (This is not the
formal description, but merely the pseudo-code description of the Turing
Machine. There is a Turing Machine which corresponds to this, using the
Church-Turing thesis.)

1. input x.
2. Initialize integer i to 0.
3. while true do
12
4. if f(i) == x then
5. accept x, halt.
6. else increment i to i+1.
7. endif
8. done

The above machine accepts and halts all x in L, and it does not accept
any other string. Hence L is Turing acceptable.
Conversely, we assume that L is an acceptable language. If L is nite,
then by denition, L is computably enumerable. Assume that L is innite.
There is a Turing machine M, that, given any input x accepts if and only if
x L.
Given an input i 0, we will dovetail the execution of M on , 0, 1, . . .
until i strings have been accepted and output the last string that has been
accepted.
We describe the pseudo-code as follows.

1. input number i.
2. Initialize Set and oldSet to ; and m to 0.
3. while [Set[ i do
4. if s
m
is of the form x, s
n
) do
5. Run M(x) for n steps.
6. if M(x) has accepted then
7. oldSet := Set.
8. Set := Set x.
9. endif
10. endif
11. Increment m.
12. end while
13. Output oldSet Set.

Since L is innite, we know that this algorithm will enable us to output

distinct elements in L for every number i, so M computes a one-to-one
function. Also, every x L is accepted in some nite number of steps N,
so the dovetailing will output x when m x, s
N
).
There is also a characterization of decidable languages in this manner.
13
Theorem 1.2.11. A language is decidable if and only if it is computably
enumerable in increasing order. That is, a language L is decidable if and
only if it is nite or there is a total computable bijection f : N L such
that for all numbers n,
f(n) < f(n + 1).
Proof. If L is nite, then it is decidable. Suppose L is not nite, and there
is a bijection f : N L and for all n,
f(n) < f(n + 1).
Then, we can build a Turing Machine which decides L in the following
manner.

1. input x.
2. Initialize i to 0.
3. whilef(i) < x do
4. Increment i by 1.
5. end while
6. if f(i) == x then
7. Accept x and halt.
8. else reject x and halt.

Conversely, suppose L is decidable. If L is nite, then it is computably

enumerable by denition. If L is innite, then there is a total computable
bijection f : N L that enumerates L in increasing order. In particu-
lar, there is a total computable bijection f that enumerates L, hence L is
computably enumerable.
It is now easy to prove the following theorem, which we use later in the
course.
Theorem 1.2.12. Every innite computably enumerable set contains an
innite decidable subset.
Proof.
We use the following fact from computability theory.
Theorem 1.2.13. There is a universal Turing machine.
14
Theorem 1.2.14. (Kleenes normal form theorem) There is a 3-ary partial
computable function C and a 1-ary partial computable function U such that
any 1-ary partial recursive function can be expressed as
f
e
(n) = U(z[C(e, n, z) = 0]).
Here, z denotes the least z such that.
Theorem 1.2.15. There is a partial computable function that is not total
computable.
Proof. We dene the following partial computable function f : N N as
f
e
(n) = U(z[C(n, n, z) = 0]) + 1.
So f(n) = f
n
(n) +1 when the value of f
n
(n) is dened, and is undened
otherwise. f(n) is a partial computable function by construction. But f is
not a total computable function:
Suppose f is total computable. Then there is an index e in the com-
putable enumeration of partial computable functions such that f
e
= f. This
would imply that f(e) = f
e
(e) + 1 > f(e), which is a contradiction.
15