Turing Machines (TM)

• Generalize the class of CFLs:

Non-Recursively Enumerable Languages

Recursively Enumerable Languages

Recursive Languages

Context-Free Languages

Regular Languages

1
• Another Part of the Hierarchy:

Non-Recursively Enumerable Languages

Recursively Enumerable Languages

Recursive Languages

Context-Sensitive Languages

Context-Free Languages - ε

Regular Languages - ε

2
• Recursively enumerable languages are also known as type 0 languages.

• Context-sensitive languages are also known as type 1 languages.

• Context-free languages are also known as type 2 languages.

• Regular languages are also known as type 3 languages.

3
• TMs model the computing capability of a general purpose computer, which
informally can be described as:
– Effective procedure
• Finitely describable
• Well defined, discrete, “mechanical” steps
• Always terminates
– Computable function
• A function computable by an effective procedure

• TMs formalize the above notion.

• Church-Turing Thesis: There is an effective procedure for solving a problem

if and only if there is a TM that halts for all inputs and solves the problem.
– There are many other computing models, but all are equivalent to or subsumed by
TMs. There is no more powerful machine (Technically cannot be proved).

• DFAs and PDAs do not model all effective procedures or computable

functions, but only a subset.
4
 Deterministic Turing Machine (DTM)
…….. B B 0 1 1 0 0 B B ……..

Finite
Control

• Two-way, infinite tape, broken into cells, each containing one symbol.
• Two-way, read/write tape head.
• An input string is placed on the tape, padded to the left and right infinitely with
blanks, read/write head is positioned at the left end of input string.
• Finite control, i.e., a program, containing the position of the read head, current
symbol being scanned, and the current state.
• In one move, depending on the current state and the current symbol being
scanned, the TM 1) changes state, 2) prints a symbol over the cell being
scanned, and 3) moves its’ tape head one cell left or right.
• Many modifications possible, but Church-Turing declares equivalence of all. 5
 Formal Definition of a DTM

• A DTM is a seven-tuple:

M = (Q, Σ, Γ, δ, q0, B, F)

Q A finite set of states

Σ A finite input alphabet, which is a subset of Γ– {B}
Γ A finite tape alphabet, which is a strict superset of Σ
B A distinguished blank symbol, which is in Γ
q0 The initial/starting state, q0 is in Q
F A set of final/accepting states, which is a subset of Q
δ A next-move function, which is a mapping (i.e., may be undefined) from

Q x Γ –> Q x Γ x {L,R}

Intuitively, δ(q,s) specifies the next state, symbol to be written, and the direction of tape
head movement by M after reading symbol s while in state q. 6
• Example #1: {w | w is in {0,1}* and w ends with a 0}

0
00
10
10110
Not ε

Q = {q0, q1, q2}

Γ = {0, 1, B}
Σ = {0, 1}
F = {q2}
δ:

0 1 B
->q0 (q0, 0, R) (q0, 1, R) (q1, B, L)
q1 (q2, 0, R) - -
q2* - - -

– q0 is the start state and the “scan right” state, until hits B
– q1 is the verify 0 state
– q2 is the final state 7
• Example #2: {0n1n | n ≥ 1}

0 1 X Y B
->q0 (q1, X, R) - - (q3, Y, R)0’s finished -
q1 (q1, 0, R)ignore1 (q2, Y, L) - (q1, Y, R) ignore2 - (more 0’s)
q2 (q2, 0, L) ignore2 - (q0, X, R) (q2, Y, L) ignore1 -
q3 - - (more 1’s) - (q3, Y, R) ignore (q4, B, R)
q 4* - - - - -

• Sample Computation: (on 0011), presume state q looks rightward

q00011BB.. |— Xq1011
|— X0q111
|— Xq20Y1
|— q2X0Y1
|— Xq00Y1
|— XXq1Y1
|— XXYq11
|— XXq2YY
|— Xq2XYY
|— XXq0YY
|— XXYq3Y B…
|— XXYYq3 BB…
|— XXYYBq4
8
Making a TM for {0n1n | n >= 1}

Try n=2 or 3 first.

• q0 is on 0, replaces with the character to X, changes state to q1, moves right
• q1 sees next 0, ignores (both 0’s and X’s) and keeps moving right
• q1 hits a 1, replaces it with Y, state to q2, moves left
• q2 sees a Y or 0, ignores, continues left
• when q2 sees X, moves right, returns to q0 for looping step 1 through 5
• when finished, q0 sees Y (no more 0’s), changes to pre-final state q3
• q3 scans over all Y’s to ensure there is no extra 1 at the end (to crash on seeing any 0 or 1)
• when q3 sees B, all 0’s matched 1’s, done, changes to final state q4
• blank line for final state q4
q00011BB.. |— Xq1011
|— X0q111
Try n=1 next.
|— Xq20Y1
|— q2X0Y1
Make sure unbalanced 0’s and 1’s, or mixture of 0-1’s,
|— Xq00Y1
“crashes” in a state not q4, as it should be
|— XXq1Y1
|— XXYq11
|— XXq2YY
|— Xq2XYY
|— XXq0YY
|— XXYq3Y B…
|— XXYYq3 BB…
|— XXYYBq4 9
• Same Example #2: {0n1n | n ≥ 1}

0 1 X Y B
q0 (q1, X, R) - - (q3, Y, R) -
q1 (q1, 0, R) (q2, Y, L) - (q1, Y, R) -
q2 (q2, 0, L) - (q0, X, R) (q2, Y, L) -
q3 - - - (q3, Y, R) (q4, B, R)
q4 - - - - -

Logic: cross 0’s with X’s, scan right to look for corresponding 1, on finding it cross it with Y, and scan
left to find next leftmost 0, keep iterating until no more 0’s, then scan right looking for B.
– The TM matches up 0’s and 1’s
– q1 is the “scan right” state, looking for 1
– q2 is the “scan left” state, looking for X
– q3 is “scan right”, looking for B
– q4 is the final state

Can you extend the machine to include n=0?

How does the input-tape look like for string epsilon?
• Other Examples:
000111 00
11 001
011
10
 Formal Definitions for DTMs
• Let M = (Q, Σ, Г, δ, q0, B, F) be a TM.

• Definition: An instantaneous description (ID) is a triple α1qα2, where:

– q, the current state, is in Q

– α1α2, is in Г*, and is the current tape contents up to the rightmost non-blank symbol, or the symbol to
the left of the tape head, whichever is rightmost
– The tape head is currently scanning the first symbol of α 2
– At the start of a computation α1= ε
– If α2= ε then a blank is being scanned

• Example: (for TM #1)

q00011 Xq1011 X0q111 Xq20Y1 q2X0Y1

Xq00Y1 XXq1Y1 XXYq11 XXq2YY Xq2XYY

XXq0YY XXYq3Y XXYYq3 XXYYBq4 11

• Suppose the following is the current ID of a DTM

x1x2…xi-1qxixi+1…xn

Case 1) δ(q, xi) = (p, y, L)

(a) if i = 1 then qx1x2…xi-1xixi+1…xn |— pByx2…xi-1xixi+1…xn

(b) else x1x2…xi-1qxixi+1…xn |— x1x2…xi-2pxi-1yxi+1…xn

– If any suffix of xi-1yxi+1…xn is blank then it is deleted.

Case 2) δ(q, xi) = (p, y, R)

x1x2…xi-1qxixi+1…xn |— x1x2…xi-1ypxi+1…xn

– If i>n then the ID increases in length by 1 symbol

x1x2…xnq |— x1x2…xnyp

12
• Definition: Let M = (Q, Σ, Г, δ, q0, B, F) be a TM, and let w be a string in Σ*. Then w is
accepted by M iff

q0w |—* α1pα2

where p is in F and α1 and α2 are in Г*

• Definition: Let M = (Q, Σ, Г, δ, q0, B, F) be a TM. The language accepted by M,

denoted L(M), is the set

{w | w is in Σ* and w is accepted by M}

• Notes:
– In contrast to FA and PDAs, if a TM simply passes through a final state then the
string is accepted.
– Given the above definition, no final state of a TM need to have any transitions.
Henceforth, this is our assumption.
– If x is NOT in L(M) then M may enter an infinite loop, or halt in a non-final
state.
– Some TMs halt on ALL inputs, while others may not. In either case the language
defined by TM is still well defined. 13
• Definition: Let L be a language. Then L is recursively enumerable if there exists a TM
M such that L = L(M).

– If L is r.e. then L = L(M) for some TM M, and

• If x is in L then M halts in a final (accepting) state.
• If x is not in L then M may halt in a non-final (non-accepting) state or no transition is available, or loop
forever.

• Definition: Let L be a language. Then L is recursive if there exists a TM M such that L

= L(M) and M halts on all inputs.

– If L is recursive then L = L(M) for some TM M, and

• If x is in L then M halts in a final (accepting) state.
• If x is not in L then M halts in a non-final (non-accepting) state or no transition is available (does not
go to infinite loop).

Notes:

– The set of all recursive languages is a subset of the set of all recursively enumerable
languages

– Terminology is easy to confuse: A TM is not recursive or recursively enumerable, rather a

language is recursive or recursively enumerable. 14
• Recall the Hierarchy:

Non-Recursively Enumerable Languages

Recursively Enumerable Languages

Recursive Languages

Context-Sensitive Languages

Context-Free Languages - ε

Regular Languages - ε

15
 Modifications of the Basic TM Model

• Other (Extended) TM Models:

– One-way infinite tapes
– Multiple tapes and tape heads
– Non-Deterministic TMs
– Multi-Dimensional TMs (n-dimensional tape)
– Multi-Heads
– Multiple tracks

All of these extensions are equivalent to the basic DTM model

16
 Closure Properties for Recursive and
Recursively Enumerable Languages

•TMs model General Purpose (GP) Computers:

– If a TM can do it, so can a GP computer
– If a GP computer can do it, then so can a TM

If you want to know if a TM can do X, then some equivalent question

are:
– Can a general purpose computer do X?
– Can a C/C++/Java/etc. program be written to do X?

For example, is a language L recursive?

– Can a C/C++/Java/etc. program be written that always halts and accepts L?

17
• TM Block Diagrams:
– If L is a recursive language, then a TM M that accepts L and always halts can be
pictorially represented by a “chip” or “box” that has one input and two outputs.

yes
w M
no

– If L is a recursively enumerable language, then a TM M that accepts L can be

pictorially represented by a “box” that has one output.

yes
w M

– Conceivably, M could be provided with an output for “no,” but this output cannot
be counted on. Consequently, we simply ignore it. 18
• Theorem 1: The recursive languages are closed with respect to complementation, i.e., if L
is a recursive language, then so is L   * L

• Proof: Let M be a TM such that L = L(M) and M always halts. Construct TM M’ as

follows:

M’
yes
yes
w M no
no

• Note That:
– M’ accepts iff M does not
– M’ always halts since M always halts

From this it follows that the complement of L is recursive. 

• Question: How is the construction achieved? Do we simply complement the final states in
the TM? No! A string in L could end up in the complement of L.
– Suppose q5 is an accepting state in M, but q0 is not.
– If we simply complemented the final and non-final states, then q 0 would be an accepting state in
M’ but q5 would not.
– Since q0 is an accepting state, by definition all strings are accepted by M’ 19
 • Theorem 2: The recursive languages are closed with respect to union, i.e., if L1
and L2 are recursive languages, then so is L3  L1  L2

• Proof: Let M1 and M2 be TMs such that L1 = L(M1) and L2 = L(M2) and M1 and
M2 always halts. Construct TM M’ as follows:

M’ yes
yes start
w M1 M2 no
no

• Note That:
– L(M’) = L(M1) L(M )
2

• L(M’) is a subset of L(M1) U L(M2)

• L(M1) U L(M2) is a subset of L(M’)
– M’ always halts since M1 and M2 always halt

It follows from this that L3  L1  L2is recursive.  20

• Theorem 3: The recursive enumerable languages are closed with respect to union, i.e., if L1
and L2 are recursively enumerable languages, then so is L3  L1  L2

• Proof: Let M1 and M2 be TMs such that L1 = L(M1) and L2 = L(M2). Construct M’ as follows:

M’ yes yes
M1

w
yes
M2

• Note That:
– L(M’) = L(M1) U L(M2)
• L(M’) is a subset of L(M1) U L(M2)
• L(M1) U L(M2) is a subset of L(M’)
– M’ halts and accepts iff M1 or M2 halts and accepts

It follows from this that is recursively enumerable. 

L3  L1  L2
• Question: How do you run two TMs in parallel?
21
• Theorem 4: If L and L are both recursively enumerable then L (and therefore L ) is
recursive.

• Proof: Let M1 and M2 be TMs such that L = L(M1) and L= L(M2). Construct M’ as
follows:

M’ yes yes
M1

w
yes
M2 no

• Note That:
– L(M’) = L
• L(M’) is a subset of L
• L is a subset of L(M’)
– M’ is TM for L
– M’ always halts since either M1 or M2 halts for any given string
– M’ shows that L is recursive
It follows from this that L (and therefore its’ complement) is recursive.
So, L is also recursive (we proved it before). 
22
• In terms of the hierarchy: (possibility #1)

Non-Recursively Enumerable Languages

Recursively Enumerable Languages

L L

Recursive Languages

23
• In terms of the hierarchy: (possibility #2)

Non-Recursively Enumerable Languages

L L

Recursively Enumerable Languages

Recursive Languages

24
• In terms of the hierarchy: (possibility #3)

Non-Recursively Enumerable Languages

L L

Recursively Enumerable Languages

Recursive Languages

25
• In terms of the hierarchy: (Impossibility #1)

Non-Recursively Enumerable Languages

L L

Recursively Enumerable Languages

Recursive Languages

26
• In terms of the hierarchy: (Impossibility #2)

Non-Recursively Enumerable Languages

Recursively Enumerable Languages

Recursive Languages

27
• In terms of the hierarchy: (Impossibility #3)

Non-Recursively Enumerable Languages

Recursively Enumerable Languages

Recursive Languages

28
 The Halting Problem - Background
• Definition: A decision problem is a problem having a yes/no answer (that one
presumably wants to solve with a computer). Typically, there is a list of parameters on
which the problem is based.
– Given a list of numbers, is that list sorted?
– Given a number x, is x even?
– Given a C program, does that C program contain any syntax errors?
– Given a TM (or C program), does that TM contain an infinite loop?

From a practical perspective, many decision problems do not seem all that interesting.
However, from a theoretical perspective they are for the following two reasons:
– Decision problems are more convenient/easier to work with when proving complexity results.
– Non-decision counter-parts can always be created & are typically at least as difficult to solve.

• Notes:
– The following terms and phrases are analogous:

Algorithm - A halting TM program

Decision Problem - A language
(un)Decidable - (non)Recursive
29
On 111 111 as a TM encoding

<Quote> It was ambiguous, in my opinion, based on the definition in the Hopcroft book, i.e., the
definition in the Hopcroft book was not clear/precise enought to

account this special case. I don't have the book in front of me right now, but I think this is the example
I used in class: Consider the TM that has exactly one state, but no transitions. Perfectly valid TM, and
it would give us this encoding (111111). In that case the encoded machine would accept sigma*
because the highest numbered state would be q0, the only state, and that would be the final state under
the Hopcroft encoding. Now consider the TM that has exactly two states, but no transitions. Also a
perfectly valid TM, and it would give us the same encoding. In that case the encoded machine would
not accept anything because the final state is q1 (highest numbered state), and there is no way to get to
it. I used it only as a way to raise that issue in class, i.e., the the Hopcroft definition is a bit ambiguous
in this case.

One way to resolve the ambiguity is to require the encoding to specifically specify the final state (at the
end or something). In that case, 111111 isn't even a valid TM, since it doesn't specify the final state.
Another related question is, does a TM even have to have any states at all to be a valid TM? The
encoding would have to be able to isolate that as a unique string also. <End Quote>

Phil Bernhard

30