Theory of Computation

Turing Machine and Pushdown Automata

1. What is a Turing Machine?

A Turing Machine is an accepting device which accepts the languages (recursively enumerable set)
generated by type 0 grammars. It was invented in 1936 by Alan Turing.

A Turing Machine (TM) is a mathematical model which consists of an infinite length tape divided into cells
on which input is given. It consists of a head which reads the input tape. A state register stores the state
of the Turing machine. After reading an input symbol, it is replaced with another symbol, its internal state
is changed, and it moves from one cell to the right or left. If the TM reaches the final state, the input
string is accepted, otherwise rejected.

A TM can be formally described as a 7-tuple (Q, X, Σ, δ, q0, B, F) where:

 Q is a finite set of states


 X is the tape alphabet

 Σ is the input alphabet

 δ is a transition function; δ : Q × X → Q × X × {Left_shift, Right_shift}.

 q0 is the initial state

 B is the blank symbol

 F is the set of final states

Example of Turing machine:

Turing machine M = (Q, X, Σ, δ, q0, B, F) with

 Q = {q0, q1, q2, qf}


 X = {a, b}

 Σ = {1}

 q0= {q0}

 B = blank symbol

 F = {qf }

δ is given by:

Tape alphabet Present State Present State Present State

symbol ‘q0’ ‘q1’ ‘q2’

a 1Rq1 1Lq0 1Lqf

b 1Lq2 1Rq1 1Rqf

Here the transition 1Rq1 implies that the write symbol is 1, the tape moves right, and the next state is q 1.
Similarly, the transition 1Lq2 implies that the write symbol is 1, the tape moves left, and the next state is
q2.
2. Explain the different types of Turing Machines
A) Multi-tape Turing Machines have multiple tapes where each tape is accessed with
a separate head. Each head can move independently of the other heads. Initially the input is on tape 1
and others are blank. At first, the first tape is occupied by the input and the other tapes are kept blank.
Next, the machine reads consecutive symbols under its heads and the TM prints a symbol on each tape
and moves its heads.

A Multi-tape Turing machine can be formally described as a 6-tuple (Q, X, B, δ, q0, F) where:

 Q is a finite set of states


 X is the tape alphabet

 B is the blank symbol

 δ is a relation on states and symbols where

δ: Q ×Xk →Q× (X× {Left_shift, Right_shift, No_shift })k

where there is k number of tapes

 q0 is the initial state


F is the set of final states
Note: Every Multi-tape Turing machine has an equivalent single-tape Turing machine.

B) Multi-track Turing machines, a specific type of Multi-tape Turing machine, contain multiple tracks
but just one tape head reads and writes on all tracks. Here, a single tape head reads n symbols from n
tracks at one step. It accepts recursively enumerable languages like a normal single-track single-tape
Turing Machine accepts.

A Multi-track Turing machine can be formally described as a 6-tuple (Q, X, Σ, δ, q0, F) where:

 Q is a finite set of states


 X is the tape alphabet

 Σ is the input alphabet

 δ is a relation on states and symbols where

δ(Qi, [a1, a2, a3,....]) = (Qj, [b1, b2, b3,....], Left_shift or Right_shift)

 q0 is the initial state


F is the set of final states

Note: For every single-track Turing Machine S, there is an equivalent multi-track Turing Machine M
such that L(S) = L(M).

C) A non-deterministic Turing machine can be formally defined as a 6-tuple (Q, X, Σ, δ, q0, B, F)

where:

 Q is a finite set of states


  X is the tape alphabet

 Σ is the input alphabet

 δ is a transition function;

δ : Q × X → P(Q × X × {Left_shift, Right_shift}).

 q0 is the initial state


 B is the blank symbol

 F is the set of final states

D) A semi-infinite tape Turing Machine - A Turing Machine with a semi-infinite tape has a left end but
no right end. The left end is limited with an end marker.

It is a two-track tape:

1. Upper track: It represents the cells to the right of the initial head position.

2. Lower track: It represents the cells to the left of the initial head position in reverse order.

The infinite length input string is initially written on the tape in contiguous tape cells.

The machine starts from the initial state q0 and the head scans from the left end marker ‘End’. In each
step, it reads the symbol on the tape under its head. It writes a new symbol on that tape cell and then it
moves the head either into left or right one tape cell. A transition function determines the actions to be
taken.

It has two special states called accept state and reject state. If at any point of time it enters into the
accepted state, the input is accepted and if it enters into the reject state, the input is rejected by the TM.

3. What is an Undecidable Language

In an undecidable language, there is no Turing Machine which accepts the language and
makes a decision for every input string w (TM can make decision for some input string though). A decision
problem P is called “undecidable” if the language L of all yes instances to P is not decidable.

Example:

 The halting problem of Turing machine


 The Post correspondence problem, etc.
4. Explain Turing Machine Halting Problem

Input: A Turing machine and an input string w.

Problem: Does the Turing machine finish computing of the string w in a finite number of steps? The
answer must be either yes or no.

Proof: At first, we will assume that such a Turing machine exists to solve this problem and then we will
show it is contradicting itself. We will call this Turing machine as a Halting machine that produces a ‘yes’
or ‘no’ in a finite amount of time. If the halting machine finishes in a finite amount of time, the output
comes as ‘yes’, otherwise as ‘no’. The following is the block diagram of a Halting machine:

Input Yes (HM halts on input w)

string Halting
Machine
No (HM does not halt on input w)

Now we will design an inverted halting machine (HM)’ as:

 If H returns YES, then loop forever.


 If H returns NO, then halt.

The following is the block diagram of an ‘Inverted halting machine’:

Infinite loop

Yes
Qi Qj
Input Halting

string Machine
No

Further, a machine (HM)2 which input itself is constructed as follows:

 If (HM)2 halts on input, loop forever.


 Else, halt.

Here, we have got a contradiction. Hence, the halting problem is undecidable.

5. Explain Post Correspondence Problem

The Post Correspondence Problem (PCP), introduced by Emil Post in 1946, is an undecidable decision
problem. The PCP problem over an alphabet  is stated as follows:

Given the following two lists, M and N of non-empty strings over :

Example 1
Find whether the lists

M = (abb, aa, aaa) and N = (bba, aaa, aa)

have a Post Correspondence Solution?

Solution

x1 x2 x3

M Abb aa aaa

N Bba aaa aa

Here,

x2 x1x3 = ‘aaabbaaa’

and y2 y1y3 = ‘aaabbaaa’

We can see that

x2 x1x3 = y2y1 y3

Hence, the solution is i=2, j =1, and k=3.

Example 2
Find whether the lists M = (ab, bab, bbaaa) and N = (a, ba, bab) have a Post Correspondence
Solution?

Solution

x1 x2 x3

M ab bab bbaaa

N a ba bab

In this case, there is no solution because:

| x2 x1x3 | ≠ | y2y1 y3 | (Lengths are not same)

Hence, it can be said that this Post Correspondence Problem is undecidable.

6. What is a Grammar? Give Example

A grammar G can be formally written as a 4-tuple (N, T, S, P) where

 N or VN is a set of variables or non-terminal symbols


 T or  is a set of Terminal symbols
S is a special variable called the Start symbol, S ∈ N


P is Production rules for Terminals and Non-terminals.

Example
Grammar G1:
 ({S, A, B}, {a, b}, S, {S →AB, A →a, B →b})

Here,

S, A, and B are Non-terminal symbols;

a and b are Terminal symbols

S is the Start symbol, S ∈ N

Productions, P: S →AB, A →a, B →b

Example:
Grammar G2:

({S, A}, {a, b}, S,{S → aAb, aA →aaAb, A→ε } )

Here,

S and A are Non-terminal symbols.

a and b are Terminal symbols.

ε is an empty string.
S is the Start symbol, S ∈ N

Production P : S → aAb, aA →aaAb, A→ε

7. Explain Chomsky’s Classification of Grammar.

Noam Chomsky classified Grammars into various types. There are four types of grammars:
Type 0, Type 1, Type 2, and Type 3.

Grammar Grammar Language

Automaton
Type Accepted Accepted

Recursively enumerable
Type 0 Unrestricted grammar Turing machine
Language

Context-sensitive Linear-bounded
Type 1 Context-sensitive grammar
Language automaton

Pushdown
Type 2 Context-free grammar Context-free language
automaton

Finite state
Type 3 Regular grammar Regular language
automaton

Type - 3 Grammar
Type-3 grammars generate regular languages. Type-3 grammars must have a single non-terminal on
the left-hand side and a right-hand side consisting of a single terminal or single terminal followed by a
single non-terminal.
The productions must be in the form X → a or X → aY where X, Y ∈ N
(Non terminal)

And a ∈ T (Terminal)

The rule S → ε is allowed if S does not appear on the right side of any rule.

Example

X → ε
X → a | aY
Y →b

Type - 2 Grammar
Type-2 grammars generate context-free languages.

The productions must be in the form A → γ

where A ∈ N (Non terminal)

and γ ∈ (T∪N)* (String of terminals and non-terminals).

These languages generated by these grammars are be recognized by a non-deterministic pushdown

automaton.

Example
S → X a
X → a
X → aX
X → abc
X → ε

Type - 1 Grammar
Type-1 grammars generate context-sensitive languages. The productions must be in the form

αAβ→αγβ

where A ∈ N (Non-terminal)

and α, β, γ ∈ (T ∪ N)* (Strings of terminals and non-terminals)

The strings α and β may be empty, but γ must be non-empty.

The rule S → ε is allowed if S does not appear on the right side of any rule. The languages generated by
these grammars are recognized by a linear bounded automaton.

Example
 AB → AbBc
A → bcA
B → b

Type - 0 Grammar
Type-0 grammars generate recursively enumerable languages. The productions have no restrictions.
They are any phase structure grammar including all formal grammars.

They generate the languages that are recognized by a Turing machine.

The productions can be in the form of α→ β where α is a string of terminals and non-terminals with at
least one non-terminal and α cannot be null. β is a string of terminals and non-terminals.

Example
S → ACaB
Bc → acB
CB → DB
aD → Db

8. What is Pushdown Automata? Explain its basic structure.

A pushdown automaton is a way to implement a context-free grammar in a similar way we design
DFA for a regular grammar.

Basically a pushdown automaton is:

"Finite state machine" + "a stack"

A pushdown automaton has three components:

 an input tape,

 a control unit, and

 a stack with infinite size.
The stack head scans the top symbol of the stack.

A stack does two operations:

 Push: a new symbol is added at the top.


 Pop: the top symbol is read and removed.

A PDA may or may not read an input symbol, but it has to read the top of the stack in every transition.

Takes
Input Finite control
unit Accept or reject

Push or Pop

Input tape

Stack

A PDA can be formally described as a 7-tuple (Q, Σ, S, δ, q0, I, F):

 Q is the finite number of states


 Σ is input alphabet

  S is stack symbols
δ is the transition function: Q × (Σ∪{ε}) × S × Q × S*
q0 is the initial state (q0 ∈ Q)
I is the initial stack top symbol (I ∈ S)
F is a set of accepting states (F ∈ Q)

The following diagram shows a transition in a PDA from a state q 1 to state q2, labeled as a,b → c :

Push
Input Stack top
Symbol
Symbol Symbol

a,b→c
q q

This means at state q1, if we encounter an input string ‘a’ and top symbol of the stack is ‘b’, then we pop
‘b’, push ‘c’ on top of the stack and move to state q2.
9. Define Instantaneous Description
The instantaneous description (ID) of a PDA is represented by a triplet (q, w, s) where

 q is the state

 w is unconsumed input

 s is the stack contents

10. Explain Chomsky’s Normal Form with an Example.

A CFG is in Chomsky Normal Form if the Productions are in the following forms:

 A → a

 A → BC

 S → ϵ

where A, B, and C are non-terminals and a is terminal.

Algorithm to Convert into Chomsky Normal Form:

Step 1 If the start symbol S occurs on some right side, create a new start symbol S’ and a new
production S’ → S.

Step 2 Remove Null productions. (Using the Null production removal algorithm discussed earlier)

Step 3 Remove unit productions. (Using the Unit production removal algorithm discussed earlier)

Step 4 Replace each production A → B1…Bn where n > 2 with A → B1C where C → B2 …Bn. Repeat
this step for all productions having two or more symbols in the right side.

Step 5 If the right side of any production is in the form A→ aB where a is a terminal and A, B are non-
terminal, then the production is replaced by A→ XB and X→ a. Repeat this step for every
production which is in the form A→ aB.

Problem:
Convert the following CFG into CNF

S→ ASA | aB, A → B | S, B→b|ϵ

Solution:

(1) Since S appears in R.H.S, we add a new state S0 and S0→S is added to the production set and it
becomes:

S0→S, S→ ASA | aB, A → B | S, B→b|ϵ

(2) Now we will remove the null productions:

B→ϵ and A→ϵ

After removing B → ϵ, the production set becomes:

S0→S, S→ ASA | aB | a, A → B | S | ϵ, B → b

After removing A → ϵ, the production set becomes:

S0→S, S→ ASA | aB | a | AS | SA | S, A → B | S, B → b
(3) Now we will remove the unit productions. After removing S→ S,

the production set becomes:

S0→S, S→ ASA | aB | a | AS | SA, A → B | S, B → b After removing

S0→ S, the production set becomes:

S0→ ASA | aB | a | AS | SA, S→ ASA | aB | a | AS | SA

A → B | S, B→b

After removing A→ B, the production set becomes:

S0 → ASA | aB | a | AS | SA, S→ ASA | aB | a | AS | SA

A→S|b

B→b

After removing A→ S, the production set becomes:

S0 → ASA | aB | a | AS | SA, S→ ASA | aB | a | AS | SA

A → b |ASA | aB | a | AS | SA, B → b

(4) Now we will find out more than two variables in the R.H.S

Here, S0→ ASA, S → ASA, A→ ASA violates two Non-terminals in R.H.S.

Hence we will apply step 4 and step 5 to get the following final production set which is in
CNF:

S0→ AX | aB | a | AS | SA

S→ AX | aB | a | AS | SA

A → b |AX | aB | a | AS | SA

B→b

X→ SA
(5) We have to change the productions S0→ aB, S→ aB, A→ aB And the final production set becomes:

S0→ AX | YB | a | AS | SA S→ AX | YB | a | AS | SA

A → b |AX | YB | a | AS | SA B → b

X→ SA Y → a

11. Explain Chomsky’s Normal Form with an Example.

A CFG is in Greibach Normal Form if the Productions are in the following forms:

A→b

A → bD1…Dn
 S→ϵ

where A, D1,....,Dn are non-terminals and b is a terminal.

Algorithm to Convert a CFG into Greibach Normal Form:

Step 1 If the start symbol S occurs on some right side, create a new start symbol S’ and a new
production S’ → S.

Step 2 Remove Null productions. (Using the Null production removal algorithm discussed earlier)

Step 3 Remove unit productions. (Using the Unit production removal algorithm discussed earlier)

Step 4 Remove all direct and indirect left-recursion.

Step 5 Do proper substitutions of productions to convert it into the proper form of GNF.

Problem:
Convert the following CFG into CNF

S→ XY | Xn | p

X → mX | m

Y → Xn | o

Solution:

Here, S does not appear on the right side of any production and there are no unit or null productions in
the production rule set. So, we can skip Step 1 to Step 3.

Step 4:

Now after replacing

X in S → XY | Xo | p

with

mX | m
we obtain

S → mXY | mY | mXo | mo | p.

And after replacing

X in Y→ Xn | o

with the right side of

X → mX | m

we obtain

Y→ mXn | mn | o.

Two new productions O→ o and P → p are added to the production set and then we came to the final GNF
as the following:

S → mXY | mY | mXC | mC | p

X→ mX | m
 Y→ mXD | mD | o

O→o

P→p

11. How does PDA correspond to CFG? Give an example.

If a grammar G is context-free, we can build an equivalent nondeterministic PDA which
accepts the language that is produced by the context-free grammar G.

Also, if P is a pushdown automaton, an equivalent context-free grammar G can be constructed where

L (G) = L (P)

Algorithm to find PDA corresponding to a given CFG

Input: A CFG, G= (V, T, P, S)

Output: Equivalent PDA, P= (Q, Σ, S, δ, q0, I, F)

Step 1 Convert the productions of the CFG into GNF.

Step 2 The PDA will have only one state {q}.

Step 3 The start symbol of CFG will be the start symbol in the PDA.

Step 4 All non-terminals of the CFG will be the stack symbols of the PDA and all the terminals of
the CFG will be the input symbols of the PDA.

Step 5 For each production in the form A→ aX where a is terminal and A, X are combination of
terminal and non-terminals, make a transition δ (q, a, A).

Problem
Construct a PDA from the following CFG.

G = ({S, X}, {a, b}, P, S)

where the productions are:

S → XS |  , A → aXb | Ab | ab

Solution

Let the equivalent PDA,

P = ({q}, {a, b}, {a, b, X, S}, δ, q, S)

where δ:

δ (q,  , S) = {(q, XS), (q,  )}

δ δ(q,  , X) = {(q, aXb), (q, Xb), (q, ab)}

δ δ(q, a, a) = {(q,  )}

δ δ(q, 1, 1) = {(q,  )}

Algorithm to find CFG corresponding to a given PDA

Input: A CFG, G= (V, T, P, S)
Output: Equivalent PDA, P = (Q, Σ, S, δ, q0, I, F) such that the non- terminals of the grammar G will
be {Xwx | w,x ∈ Q} and the start state will be Aq0,F.
Step 1 For every w, x, y, z ∈ Q, m ∈ S and a, b ∈ Σ, if δ (w, a, epsilon) contains (y, m) and (z, b,
m) contains (x, epsilon), add the production rule Xwx → a Xyzb in grammar G.
Step 2 For every w, x, y, z ∈ Q, add the production rule Xwx → XwyXyx in grammar G.
Step 3 For w ∈ Q, add the production rule Xww→ epsilon in grammar G.