Automata, Grammars and Languages

Discourse 05

Turing Machines

C SC 473 Automata, Grammars & Languages

 The TM Model current state
q

0 1 2 3  Unbounded blank tape

Fixed # bits b a0 a1 a2 a3 a4 ⊔ ⊔




R/W head
• Model of ``effectively computable procedure” (including all
algorithms)
• Primitive as possible (math. simplicity & constructions)
• Arbitrary definition choice, but standard
• Easy comparison with other models (RAM, etc.)
• Finitely describable (just a formatted string)
• 1 machine = 1 procedure (no stored program -- different
FSA for each TM)
• Computes in discrete steps (moves); each step physically
trivial
• Simple complexity measures (steps=time, cells=space)
C SC 473 Automata, Grammars & Languages 2
The TM Model
0 1 2 3 
b 




Finite-
state
control
• Control unit: FSA with state set Q
 Input: character in cell under R/W head
 Output: new state & overstrike character & head moves L or R
 Initial state, accept state, reject state:
q0 , qaccept , qreject ∈ Q
• Tape unit
 Finite length string of characters with blanks ⊔⊔⊔⊔⋯ to right

⊔ marks R end of input
{⊔}
 Fixed alphabet  ( ⊔  ); input alphabet Σ ⊆ Γ −
 Tape bounded on left & unbounded on right

C SC 473 Automata, Grammars & Languages 3

 TM Examples input and R/W head left adjusted at start
• TM to scan R over non-blanks to halt over first blank
0  0, R ⊔  ⊔, L q a (q,a)
q0 q0 (q0 , 0 .R )
1  1, R
qaccept  q0
0
1 (q0 , 1,R )
q0 B (qaccept , −, −)
• TM to accept odd parity # 1’s
0  0, R
1  1, R 0  0, R
q0 q1
1  1, R

⊔  ⊔, L
⊔  ⊔, L
qreject qaccept

C SC 473 Automata, Grammars & Languages 4

TM Definition (1-tape deterministic TM)
• A 7-tuple M = (Q, Σ, Γ, δ , q0 , qaccept , qreject )
 Q = finite set of states
  = finite tape alphabet; blank ⊔; input alphabet Σ ⊆ Γ −{Ú }
 Transition function δ : Q ×Γ → Q ×Γ×{L , R }
 initial state q0 ∈ Q ; halt (accept/reject) states: qaccpet , qreject ∈ Q
• Configuration C = uaqbv u , v ∈ Γ* , q ∈ Q, a, b ∈ Γ
• (1-step) yields relation between configurations
uaqbv 
uacq ' v ⇔ δ (q, b) = (q ', c, R )


uaq = uaq ⊔ uacq ' ⇔ δ (q,Ú )= (q',c,R )



uaqbv uq ' acv ⇔ δ (q, b) = (q ', c, L )



Note: Below says you can move left from leftmost cell, although you don’t
really move left! This convention makes accept, reject the only halt states.
qbv q ' cv ⇔ δ (q, b) = (q ', c, L )


qbv cq ' v ⇔ δ (q, b) = (q ', c, R )



C SC 473 Automata, Grammars & Languages 5

Acceptance
• Move relation (multi-step):

*
• w* is accepted by M  q0 w uqaccept v

*
• Language recognized by M:
L( M ) = {w ∈ Γ* | (∃u , v) q0 w uqaccept v}

*
• TM-recognizable language – a set S such that
(∃ TM M ) S = L( M )
• Configuration I is halting: (¬ ∃C ') C C'


 Nowhere to move: from definition, only 2 ways to halt:
A rejecting or accepting configuration: uq
reject v, uqaccept v


 A configuration for which no move is defined

• Initial configuration: q0 w w∈ Σ *

Any halting configuration without q is non-accepting
accept
 Can always “force” non-accepting, halting configs to be rejecting
 For undefined δ ( q, a ) just add transitions to q
reject
C SC 473 Automata, Grammars & Languages 6
Turing-decidable vs. Turing-recognizable
• A language L is Turing-recognizable iff L =L(M ) for
some TM M .

M need not, but might, halt rejecting on some strings  L
 Such a TM is an acceptor or recognizer for L
• A language L is Turing-decidable iff
L = L(M ) for some TM M that halts on every input string
(accepts or rejects). Such a TM is called a decider or algorithm.

Recognizer yes Decider yes

w w
for L for L no

“procedure” “algorithm”
Note: ``yes’’ means enters qaccept and halts
``no’’ means enters qreject and halts

C SC 473 Automata, Grammars & Languages 7

Example
BB, L
• Acceptor for L = { anbncn | n ≥1} bb, R
CC, L
aa, R CC, R
a A,R BB, R
q1 q2 q3 cC, L
q4
bB, R

B  B, R A  A, R AA, L
aa, L
q5 bb, L
B  B, R
qreject
C C, R cc, L
⊔ ⊔, L
q6 C C, R
All transitions not shown go to qreject
⊔ ⊔, L (See dotted example)
So halts rejecting unless accepting
qaccept state reached

C SC 473 Automata, Grammars & Languages 8

Acceptance (Cont.)
• Sequence of configurations I 0 I1 I2 K It


is a computation of length (time) t
• Computation I 0 I1 I2 K is in a tight loop iff

 +


∃k I 0 Ik & Ik Ik
*

(a config. repeats)
q0 w = C0 Ca = uqaccept v

*
(1) How does a TM accept w?
(2) How does a TM reject w?
q0 w = C0 C1 C2 K
in a tight loop
 *


•

• q0 w = C0 Cr = uqreject v a halting & rejecting config

computation q0 w = C0 C1 C2 K
 never


reaches a halting config., and no configuration repeats. Call

this a divergent computation.
NOTE: Can make (1) coincide with halting. Can eliminate (a) in
favor of (b). (How?) But there is no algorithm to detect (c )
and eliminate (as we shall see)
C SC 473 Automata, Grammars & Languages 9
Acceptance (Cont.)
• Recognizers and Deciders – summary
 M recognizes S ⇔ S = L( M )
 M decides
S⇔

S = L( M ) recognizes

∀w ∉ S q0 w ê∗ uqrejectv rejects (& halts)
M is called a decider or algotithm
• S is a Turing-recognizable language if there is some TM
that recognizes it
• S is a Decidable language iff there is some TM that decides
it
 NOTE: every TM recognizes some language, but only

“special” TMs—ones that halt for every input—decide

the language they recognize

C SC 473 Automata, Grammars & Languages 10

A Look Ahead
b le sets
ida
c
De C
Tu
∃ Decider to answer " x ∈ C ?" r
Re ing-
Context-free co
gn
iza
∃ PDA rec. B ble
B

Regular A
∃ Finite Automaton rec. A

D

∃ Turing-recognizer for D

ATM

ATM Halting Problem
¬ ∃ recognizer for E !!
E

C SC 473 Automata, Grammars & Languages 11

TM Extensions: Church’s Thesis
• Evidence for Church’s Thesis
• Church’s Thesis (Church-Turing Thesis): the effectively
computable functions are those characterized by one the
“standard formalisms” such as the TM.
• Convenience of having alternate models
• Two models of computation are equivalent 
 whenever a language L is recognized by a machine in one model,
there is an algorithm to construct a machine recognizing L in the
second, and vice versa
 whenever a function f is computed by a machine in one model,
there is an algorithm to construct a machine computing f in the
second, and vice versa
 I.e., Simulation both directions by a “compiling” algorithm

C SC 473 Automata, Grammars & Languages 12

 TM Extensions: k-tape TM
• Tapes simulated by one tape with k tracks & software heads
 a in cell i of tape 1  a on track 1 of cell i

 a in cell i of tape 1 & head 1 scanning cell  â on track 1 of cell

i Γ′ = (Γ ∪ Γˆ ) k
 Tape alphabet is head sweeps L to R until 
B
store k scanned chars. In
a b c d e f B ... states. Sweep R to L & mark B
Mk 0 1 2 3 … changes. Sweep L to R then R to L to move
head marks. End cycle scanning cell 0.
A B C D E F G ...
0 1 2 3 …
δ : (Q −{qa , qr }) ×Γk → Q ×Γk ×{L , R }k a b ĉ d e f B . . .
M1 A B C D E F̂ G . . .

δ : Q′×Γ′ → Q′×Γ′×{L , R }

C SC 473 Automata, Grammars & Languages 13

Nondeterministic TM
• A 7-tuple M = (Q, Σ, Γ, δ , q0 , qaccept , qreject )
 Transition function
• Transition function δ : Q ×Γ → P (Q ×Γ×{L , R })
 For each (q,a),  (q,a) provides a set of choices

δ (q, a) = {(q1 , b1 , D1 ), (q2 , b2 , D2 ),K , (qd , bd , Dd )}

• Acceptance means: sequence of configurations leads to

a configuration with state qaccept .
• No use for reject state (rejection is not by entering a
state)
• Rejection means no possible chain of configurations leads
to one with qaccept .
• No next move is possible: can have δ ( q, a ) = ∅

C SC 473 Automata, Grammars & Languages 14

Nondeterministic TM
• Acceptance:
L( M ) =
{w ∈ Σ* | q0 w ê ∗ αqacceptγ forsom e α ,γ ∈Γ*}

w ∈ L( M ) ⇔ ∃ sequence of configurations
q0 w = I0 I1 I2 K I t = α qaccept γ





 note the existential quantifier: w is accepted if there is some

sequence of “guesses” that drive the initial tape configuration to an
accepting configuration
 A word w is not accepted iff every possible computation starting
with q0 w fails to enter an accepting configuration

C SC 473 Automata, Grammars & Languages 15

Choice Numbers
• Choices for a given state and symbol
δ (q, a ) = {( p1 , b1 , D1 ), ( p2 , b2 , D2 ),K , ( pn , bn , Dn )}
1 2 K n
• For each q and a assign each choice in (q,a) a number

• For each machine, there is some largest number of

choices for some transition–call it b

• Strings over {1, 2,3,K , b} can be interpreted as

deterministic directions for which choice to make from
each configuration I =K qa K

C SC 473 Automata, Grammars & Languages 16

Computation Tree
I0 = q0 w
1 b
2
I1 I2 L Ib
1 b 1 b 1 b
2 2 2
I11 I12 L I1b I 21 I 22L I 2b I b1 I b 2 L I bb
b
1 2

L L I 2b1 I 2b 2 L I 2bb L
Choice sequence
2b1 → I0 I2 I 2b I 2b1





C SC 473 Automata, Grammars & Languages 17

NTM Equivalent to TM
• Theorem 3.16. If L is accepted by an NTM N , there is a
DTM D, constructable by algorithm, that accepts L.
Proof: Simulate all possible computations on N for all possible choice
strings and halt if a sequence of choices is found that leads to an
accepting configuration. D has 3 tapes: read-only input, worktape
simulating that of N, and an enumeration tape. On the latter,
enumerate all choice sequences in lexicographic order
λ ,1,K , b,11,12,K ,1b, 21, 22,K , 2b,K ,111,112,K
Main cycle: generate the next choice seq. c on the enum. Tape. Use
this sequence to drive computation of N for a total of |c| steps. If an
accepting configuration is reached, D accepts and halts. If the
computation halts rejecting, move to the next main cycle after clearing
the worktape. Otherwise move to the next main cycle after |c| steps.
Iterate the main cycle while an accepting configuration has not been
found. If an accepting c exists, D will eventually find it. If none exists,
the input will not be accepted. So L(D) = L(N). QED

C SC 473 Automata, Grammars & Languages 18

The Universal TM
• Any (hardware) TM M can be encoded as a formatted
string (software) M
 Encoding details Ma M below
• The UTM U reads M , w and simulates the action of M
on w
• The UTM U is one, fixed, finite machine, capable of
simulating any TM (an interpreter)
 For example, U reads U, M,w and gives the same result as
for input M , w
• We shall see that, whenever universality exists,
unsolvability is an inevitable consequence

C SC 473 Automata, Grammars & Languages 19

 Canonical Encoding of TM M
• Let M = (Q, Σ, Γ, δ , q0 , qaccept ) Q = {q0 , q1 ,K , qk −1}
• Encode over 9 symbol alphabet ,
{0,1, , { , } ,[, ], (, ) }
object in M encoding in M
qi c(qi) = [bin(i)]
q0 c(q0 ) = [0]
qaccept = q1 c(q1 ) = [1]
δ (qi, aj) = (qk , al, D ); L = 1, R = 2 (c(qi), c(aj), c(qk ), c(al), D )
δ (sequence of , -separated 5-tuples )
M = (Q , Σ, Γ, δ , q0 , qaccept) (sequence of , -separated substrings )
M M −string

C SC 473 Automata, Grammars & Languages 20

Canonical Encoding of TM M (cont.)
• M −string is a word over ,
{0,1, , { , } ,[, ], (, ) }
• Final encoding M is ASCII (binary) of that string
• Notational convention: an encoding M of a TM M ``is”
(the binary representation of) a natural number
• number M is called the Gödel number of M
• Conversely if e is a natural number, M e is the TM with
that Gödel number
• If e is a syntactically invalid code, M e is by definition
a TM that halts and prints 0 on every input

C SC 473 Automata, Grammars & Languages 21

UTM Construction
• Use 4 tapes: input M Ú w program tape, state tape w
& worktape of M being simulated
• Parse input, copying M to program tape. If invalid,
put [0] on worktape and halt. Else copy w to worktape &
put [0] on state tape simulating stateq0
• While the state tape  [1] do {
• If the state tape contains c ( qi ) count over to tuples for that state
in program M Use the character a under scan on the worktape
([0],[1],…) to scan to the correct 5-tuple(c (qi ), c(a ), c( qk ), c (b), D )
• Overwrite the scanned character c(a) on the worktape by c(b) and
move the head direction D on the worktape.
• Copy the new state string c ( qk ) from the program tape to the
state tape (after erasing the previous state string)
}

C SC 473 Automata, Grammars & Languages 22

UTM Construction
• 4 tapes: M Ú w input to U, tape to hold M , tape to hold
state of M and worktape of M
M Úw . . . read-only input
read-only program

U (([0],[1],[0],[0], 2), ([0],[2],[2],[2], 2),K

[ 0 ] [ 1 ] ... [ 0 ] B B B ...

worktape of M state of M

C SC 473 Automata, Grammars & Languages 23

2-State, 3-Tape Symbol UTM
• http://www.wolframscience.com/prizes/tm23/

C SC 473 Automata, Grammars & Languages 24

TM Extensions: Enumerators
• TM that prints out a list of strings, separated by blanks
• Starts with an empty tape
• Defines a set of strings (language)
• Strings need not be unique (repeated strings allowed)
• Strings can be enumerated in no particular order
• Usually does not halt; runs on forever printing strings
• Finite or not, an enumerator E defines a set of strings
Theorem 3.21: A language is Turing-recognizable iff it is
enumerable by some enumerator.
Pf: Must show 2 directions: a language defined by an
enumerator has a Turing recognizer, and any TM-
recognizable language has an enumerator. We will show
there are algorithms for going in both directions.
C SC 473 Automata, Grammars & Languages 25
TM Extensions: Enumerators (cont’d)
• Enumerator  Recognizer. Let A = L(E) where E is an
enumerator. Using the code for E construct a recognizer
M as follows.
• M = “On input w:
 Continue running E and output the next string s. Pause E.
Compare s to w.
 If w = s, halt and accept. Otherwise, continue at step 1.”
 Machine M recognizes just those strings A that E
enumerates.

C SC 473 Automata, Grammars & Languages 26

TM Extensions: Enumerators (cont’d)
• Recognizer  Enumerator. Let A = L(M) where M is a
TM recognizer. Using the code for M construct an
enumerator E as follows.
Let s1 , s2 , s3 , … be a list of all strings in *. (It is
easy to build a routine that generates strings in lexical
order: ε , a, b, aa, ab, ba, bb, aaa, … )
• E = “Ignore the input.
 for i = 1 to  do {
 Run M for i steps on each of the inputs s1 , s2 , … , si .
 If any computation sequence accepts an input s j
print it out.
}“

E (eventually) enumerates every string M accepts
 Technique: “interleaving computations”
C SC 473 Automata, Grammars & Languages 27
What is ``Effectively Computable’’?
 Early 1900s: logicians sought a universal algorithm that
would enable “Automatic Theorem-Proving”
 Worked to find “deciders” to tell:
• whether formulas like((∀y)(P(y) → Q)) ↔ ((∃x)( P x) → Q)
are logically true via a “mechanical” method (Decision
Problem for First Order Logic)
• whether multi-variable polynomial equations like
5x2y − 10xz2 + 20 = 0 have integer solutions
(Hilbert’s 10th Problem)
• Want to decide any such questions by ‘computation’ that is
‘effectively performable’ with a finite number of
computation steps
• Had no idea there were “undecidable” problems
C SC 473 Automata, Grammars & Languages 28
``Effectively Computable’’ (cont’d)
 But what did “effectively computable” mean?
 An intuitive notion: ‘you know it when you see it’
 There were no programming languages or computers
 Each author had to pick a “mechanically performable”
method to illustrate what was computable
 1936: Alan Turing proposed Turing Machines
 1936: Alonzo Church proposed -calculus
 2 remarkable outcomes:
 Both definition methods proved to be equivalent (agreed on what
was a computable set or function)
 Once a method was picked, it was discovered that there must be
undecidable sets;
 Eventually many more methods proved equivalent:

C SC 473 Automata, Grammars & Languages 29

 Computability: The Evidence for Church-Turing’s Thesis

Turing (1936) Gödel & Herbrand (1934)

TM Kleene (1936)
Church(1936)
General Recursive Functions
-Calculus
Markov (1954)
TM-computable Markov Algorithms
Phrase-structure Functions /
(Type 0) TM-recognizable
Grammar(1959)
Sets

Any programming Shepherdson & Sturgis (1963)

language to date Register Machines(RAM)

Post (1943) Elgot & Robinson (1964)

Canonical Systems RASPs
For each pair of representations, there is a uniform algorithm (“compiler”) for
translating one to the other
C SC 473 Automata, Grammars & Languages 30
Church-Turing Thesis
Thesis [OED]
A proposition laid down or stated, esp. as a theme to be
discussed and proved, or to be maintained against attack ;
a statement, assertion, tenet.

Intuitive concept
of algorithm
= Turing-machine
algorithm
Intuitive concept
of algorithm = RAM
algorithm
Intuitive concept
of algorithm
= Markov
algorithm

M
 All methods also agree on what is a “decidable set” and
what is a “recognizable set”
C SC 473 Automata, Grammars & Languages 31
``Effectively Computable’’ (cont’d)
 Both decision problems (Hilberts 10th, 1st Order Predicate
Calculus) eventually shown to be undecidable
 In our terminology, a decision problem is translated into a
language. The problem is decidable iff the language has a
Turing decider.
 None of the following languages have a decider; they are
undecidable languages.
A10 th = { p | p is a multi-variable polynomial such
that p = 0 has an integer solution }
APC = {e| e is a true formula (theorem) of
the 1st order predicate calculus }
A TM = { M , w | M is a TM and w ∈ L( M ) }
 This last is called the Halting or Membership Problem.
C SC 473 Automata, Grammars & Languages 32