Undecidability

1
Decidability vs. Undecidability
There are two types of TMs (based on halting):
(Recursive)
TMs that always halt, no matter accepting or non-accepting
DECIDABLE PROBLEMS
(Recursively enumerable)
TMs that are guaranteed to halt only on acceptance. If non-
accepting, it may or may not halt (i.e., could loop forever).

Undecidability:
Undecidable problems are those that are not recursive
2
Fall 2004 COMP 335 3
Definition:
A language is recursively enumerable if
some Turing machine accepts it
Fall 2004 COMP 335 4
For string :
Let L be a recursively enumerable language
and M the Turing Machine that accepts it
L w
w
if then M halts in a final state
L w
if then M halts in a non-final state
or loops forever
Fall 2004 COMP 335 5
Definition:
A language is recursive if some Turing machine accepts it
and halts on any input string
In other words:
A language is recursive if there is a membership algorithm
for it
Fall 2004 COMP 335 6
For string :
Let be a recursive language
L
and the Turing Machine that accepts it
M
L w
w
if then halts in a final state
M
L w
if then halts in a non-final state
M
Recursive, RE, Undecidable languages
7
Regular
(DFA)
Context-
free
(PDA)
C
o
n
t
e
x
t

s
e
n
s
i
t
i
v
e

R
e
c
u
r
s
i
v
e

R
e
c
u
r
s
i
v
e
l
y

E
n
u
m
e
r
a
b
l
e

(
R
E
)

Non-RE Languages
(all other languages for which
no TMs can be built)
LBA
TMs that always halt
TMs that may or
may not halt
No TMs exist
Undecidable problems
Decidable problems
Recursive Languages &
Recursively Enumerable (RE) languages
Any TM for a Recursive language is going to look
like this:



Any TM for a Recursively Enumerable (RE)
language is going to look like this:

8
M
w
accept
reject
M
w
accept
Closure Properties of:
- the Recursive language class, and
- the Recursively Enumerable
language class
9
Recursive Languages are closed under
complementation
If L is Recursive, L is also Recursive
10
M
w
accept
reject reject
accept
w
M
Are Recursively Enumerable Languages closed
under complementation? (NO)
If L is RE, L need not be RE
11
M
w
accept
reject
accept
w
M
?
?
Recursive Langs are closed under
Union
Let M
u
= TM for L
1
U L
2

M
u
construction:
1. Make 2-tapes and copy
input w on both tapes
2. Simulate M
1
on tape 1
3. Simulate M
2
on tape 2
4. If either M
1
or M
2

accepts, then M
u
accepts
5. Otherwise, M
u
rejects.
12
w
M
1
M
2
accept
reject
accept
reject
OR
M
u
Recursive Langs are closed under
Intersection
Let M
n
= TM for L
1
L
2

M
n
construction:
1. Make 2-tapes and copy
input w on both tapes
2. Simulate M
1
on tape 1
3. Simulate M
2
on tape 2
4. If either M
1
AND M
2

accepts, then M
n
accepts
5. Otherwise, M
n
rejects.
13
w
M
1
M
2
accept
reject
accept
reject
M
n
AND
AND
Other Closure Property Results
Recursive languages are also closed under:
Concatenation
Kleene closure (star operator)
Homomorphism, and inverse homomorphism
RE languages are closed under:
Union, intersection, concatenation, Kleene closure

RE languages are not closed under:
complementation
14
Fall 2004 COMP 335 15
Non-recursively enumerable
Recursively-enumerable
Recursive
Context-sensitive
Context-free
Regular
The Chomsky Hierarchy
Fall 2004 COMP 335 16
Decidability

Fall 2004 COMP 335 17
Consider problems with answer YES or NO
Examples:
Does Machine have three states ?
M
Is string a binary number?
w
Does DFA accept any input?
M
Fall 2004 COMP 335 18
A problem is decidable if some Turing machine
decides (solves) the problem
Decidable problems:
Does Machine have three states ?
M
Is string a binary number?
w
Does DFA accept any input?
M
Fall 2004 COMP 335 19
Turing Machine
Input
problem
instance
YES
NO
The Turing machine that decides (solves)
a problem answers YES or NO
for each instance of the problem
Fall 2004 COMP 335 20
The machine that decides (solves) a problem:
If the answer is YES
then halts in a yes state
If the answer is NO
then halts in a no state
These states may not be final states
Fall 2004 COMP 335 21
YES states
NO states
Turing Machine that decides a problem
YES and NO states are halting states
Fall 2004 COMP 335 22
Difference between
Recursive Languages and Decidable problems
The YES states may not be final states
For decidable problems:
Fall 2004 COMP 335 23
Some problems are undecidable:
which means:
there is no Turing Machine that
solves all instances of the problem
A simple undecidable problem:
The membership problem
Fall 2004 COMP 335 24
The Membership Problem
Input: Turing Machine
M
String
w
Question:
Does accept ?
M
w
? ) (M L w
Fall 2004 COMP 335 25
Theorem:
The membership problem is undecidable
Proof: Assume for contradiction that
the membership problem is decidable
(there are and for which we cannot
decide whether )
M
w
) (M L w
Fall 2004 COMP 335 26
Thus, there exists a Turing Machine
that solves the membership problem
H
H
M
w
YES
M
accepts
w
NO
M
rejects
w
Fall 2004 COMP 335 27
Let be a recursively enumerable language
L
Let be the Turing Machine that accepts
M L
We will prove that is also recursive:
L
we will describe a Turing machine that
accepts and halts on any input
L
Fall 2004 COMP 335 28
M
accepts ?
w
NO
YES
M
w
H
accept
w
Turing Machine that accepts
and halts on any input
L
reject
w
Fall 2004 COMP 335 29
Therefore, L
is recursive
But there are recursively enumerable
languages which are not recursive
Contradiction!!!!
Since is chosen arbitrarily, every recursively enumerable
language is also recursive
L
Fall 2004 COMP 335 30
Therefore, the membership problem
is undecidable
END OF PROOF
Fall 2004 COMP 335 31
Another famous undecidable problem:
The halting problem
Fall 2004 COMP 335 32
The Halting Problem
Input: Turing Machine
M
String
w
Question:
Does halt on input ?
M
w
Fall 2004 COMP 335 33
Theorem:
The halting problem is undecidable
Proof: Assume for contradiction that
the halting problem is decidable
(there are and for which we cannot
decide whether halts on input )
M
w
M
w
Fall 2004 COMP 335 34
Thus, there exists Turing Machine
that solves the halting problem
H
H
M
w
YES
M
halts on
w
M
doesnt
halt on
w
NO
Fall 2004 COMP 335 35
H
w w
M
0
q
y
q
n
q
Input:
initial tape contents
Encoding
of
M
w
String
YES
NO
Construction of
H
Fall 2004 COMP 335 36
Construct machine :
H

If returns YES then loop forever

Loop forever
YES
Fall 2004 COMP 335 38
H

Construct machine :
Input:
If
M
halts on input
M
w
Then loop forever
Else halt
M
w
(machine )
M
Fall 2004 COMP 335 39
M
w
M M
w w
copy
M
w
H

Fall 2004 COMP 335 40

Run machine with input itself:

H
w

Fall 2004 COMP 335 41
on input
H

H
w

If halts then loops forever
If doesnt halt then it halts
:
H

NONSENSE !!!!!
Fall 2004 COMP 335 42
Therefore, we have contradiction
The halting problem is undecidable
END OF PROOF
Fall 2004 COMP 335 43
Another proof of the same theorem:
If the halting problem was decidable then
every recursively enumerable language
would be recursive
Fall 2004 COMP 335 44
Theorem:
The halting problem is undecidable
Proof: Assume for contradiction that
the halting problem is decidable
Fall 2004 COMP 335 45
There exists Turing Machine
that solves the halting problem
H
H
M
w
YES
M
halts on
w
M
doesnt
halt on
w
NO
Fall 2004 COMP 335 46
Let be a recursively enumerable language
L
Let be the Turing Machine that accepts
M L
We will prove that is also recursive:
L
we will describe a Turing machine that
accepts and halts on any input
L
Fall 2004 COMP 335 47
M
halts on ?
w
YES
NO M
w
Run
with input
M
w
H
reject
w
accept
w
reject
w
Turing Machine that accepts
and halts on any input
L
Halts on final state
Halts on non-final
state
Fall 2004 COMP 335 48
Therefore L
is recursive
But there are recursively enumerable
languages which are not recursive
Contradiction!!!!
Since is chosen arbitrarily, every
recursively enumerable language
is also recursive
L
Fall 2004 COMP 335 49
Therefore, the halting problem is undecidable
END OF PROOF
Languages vs. Problems
A language is a set of strings

Any problem can be expressed as a set of all strings that
are of the form:
<input, output>


==> Every problem also corresponds to a language!!

50
Think of the language for a problem == a verifier for the problem
e.g., Problem (a+b) Language of strings of the form { a#b, a+b }
Costas Busch - RPI 51
We will prove that two particular problems are
unsolvable:
Membership problem
Halting problem
The Halting Problem
An example of a recursive
enumerable problem that is also
undecidable
52
53
Regular
(DFA)
Context-
free
(PDA)
C
o
n
t
e
x
t

s
e
n
s
i
t
i
v
e

R
e
c
u
r
s
i
v
e

R
e
c
u
r
s
i
v
e
l
y

E
n
u
m
e
r
a
b
l
e

(
R
E
)

Non-RE Languages
The Halting Problem
x
What is the Halting Problem?
Definition of the halting problem:

Does a givenTuring Machine M halt on a given
input w?

54
Machine
M
Input w
The Universal Turing Machine
Given: TM M & its input w
Aim: Build another TM called H, that will output:
accept if M accepts w, and
reject otherwise

An algorithm for H:
Simulate M on w


H(<M,w>) =


55
accept, if M accepts w

reject, if M does does not accept w
A Turing Machine simulator
Question: If M does not halt on w, what will happen to H?
Implies: H is in RE
A Claim
Claim: No H that is always guaranteed to
halt, can exist!
Proof: (Alan Turing, 1936)
By contradiction, let us assume H exists



56
H
<M,w>
accept
reject
HP Proof (step 1)
Let us construct a new TM D using H as a subroutine:
On input <M>:
1. Run H on input <M, <M> >; //(i.e., run M on M itself)
2. Output the opposite of what H outputs;
57
H
<M>
accept
reject reject
accept
D
<M, <M> >
Therefore, if H exists D also should exist.
But can such a D exist? (if not, then H also cannot exist)
HP Proof (step 2)
The notion of inputing <M> to M itself
A program can be input to itself (e.g., a compiler is a
program that takes any program as input)
58
accept, if M does not accept <M>

reject, if M accepts <M>
D (<M>) =
accept, if D does not accept <D>

reject, if D accepts <D>
D (<D>) =
Now, what happens if D is input to itself?
A contradiction!!! ==> Neither D nor H can exist.