Automata Theory

About this Tutorial

Automata Theory is a branch of computer science that deals with designing
abstract self-propelled computing devices that follow a predetermined sequence
of operations automatically. An automaton with a finite number of states is
called a Finite Automaton.

This is a brief and concise tutorial that introduces the fundamental concepts of
Finite Automata, Regular Languages, and Pushdown Automata before moving
onto Turing machines and Decidability.

Audience
This tutorial has been prepared for students pursuing a degree in any
information technology or computer science related field. It attempts to help
students grasp the essential concepts involved in automata theory.

Prerequisites
This tutorial has a good balance between theory and mathematical rigor. The
readers are expected to have a basic understanding of discrete mathematical
structures.

Copyright & Disclaimer

 Copyright 2015 by Tutorials Point (I) Pvt. Ltd.

All the content and graphics published in this e-book are the property of
Tutorials Point (I) Pvt. Ltd. The user of this e-book is prohibited to reuse, retain,
copy, distribute or republish any contents or a part of contents of this e-book in
any manner without written consent of the publisher.

We strive to update the contents of our website and tutorials as timely and as
precisely as possible, however, the contents may contain inaccuracies or errors.
Tutorials Point (I) Pvt. Ltd. provides no guarantee regarding the accuracy,
timeliness or completeness of our website or its contents including this tutorial.
If you discover any errors on our website or in this tutorial, please notify us at
[email protected]

i
 Automata Theory

Table of Contents
About this Tutorial ..................................................................................................................................... i

Audience .................................................................................................................................................... i

Prerequisites .............................................................................................................................................. i

Copyright & Disclaimer .............................................................................................................................. i

Table of Contents ...................................................................................................................................... ii

1. INTRODUCTION..................................................................................................................... 1

Automata – What is it? ............................................................................................................................. 1

Related Terminologies .............................................................................................................................. 1

Alphabet ................................................................................................................................................... 1
String ........................................................................................................................................................ 1
Length of a String ..................................................................................................................................... 1
Kleene Star ............................................................................................................................................... 2
Kleene Closure / Plus ................................................................................................................................ 2
Language .................................................................................................................................................. 2

Deterministic and Nondeterministic Finite Automaton ............................................................................. 2

Deterministic Finite Automaton (DFA) ..................................................................................................... 2
Non-deterministic Finite Automaton (NDFA) ........................................................................................... 4

DFA vs NDFA ............................................................................................................................................. 5

Acceptors, Classifiers, and Transducers ..................................................................................................... 6

Acceptability by DFA and NDFA ................................................................................................................ 6

Converting an NDFA to an Equivalent DFA ................................................................................................ 7

DFA Minimization using Myhill-Nerode Theorem ..................................................................................... 9

DFA Minimization using Equivalence Theorem ....................................................................................... 11

Moore and Mealy Machines ................................................................................................................... 13

Mealy Machine ....................................................................................................................................... 13
Moore Machine ...................................................................................................................................... 13
Mealy Machine vs. Moore Machine ....................................................................................................... 14

Moore Machine to Mealy Machine ......................................................................................................... 15

Mealy Machine to Moore Machine ......................................................................................................... 16

ii
 Automata Theory

2. CLASSIFICATION OF GRAMMARS ........................................................................................ 18

Grammar ................................................................................................................................................ 18

Derivations from a Grammar .................................................................................................................. 19

Language Generated by a Grammar ........................................................................................................ 19

Construction of a Grammar Generating a Language ................................................................................ 20

Chomsky Classification of Grammars ...................................................................................................... 22

Type - 3 Grammar................................................................................................................................... 23
Type - 2 Grammar................................................................................................................................... 23
Type - 1 Grammar................................................................................................................................... 24
Type - 0 Grammar................................................................................................................................... 24

3. REGULAR GRAMMARS ........................................................................................................ 26

Regular Expressions ................................................................................................................................ 26

Some RE Examples ................................................................................................................................. 26

Regular Sets ............................................................................................................................................ 27

Properties of Regular Sets ...................................................................................................................... 27

Identities Related to Regular Expressions ............................................................................................... 30

Arden’s Theorem .................................................................................................................................... 30

Construction of an FA from an RE ........................................................................................................... 34

Finite Automata with Null Moves (NFA-ε) ............................................................................................... 36

Removal of Null Moves from Finite Automata ........................................................................................ 36

Pumping Lemma for Regular Languages.................................................................................................. 38

Applications of Pumping Lemma ............................................................................................................ 39

Complement of a DFA ............................................................................................................................. 40

4. CONTEXT-FREE GRAMMARS ............................................................................................... 42

Context-Free Grammar ........................................................................................................................... 42

Generation of Derivation Tree ................................................................................................................ 42

Representation Technique: .................................................................................................................... 42
Derivation or Yield of a Tree ................................................................................................................... 43
Sentential Form and Partial Derivation Tree .......................................................................................... 44
Leftmost and Rightmost Derivation of a String ...................................................................................... 45

Ambiguity in Context-Free Grammars ..................................................................................................... 47

iii
 Automata Theory

Closure Property of CFL ........................................................................................................................... 49

Union ...................................................................................................................................................... 49
Concatenation ........................................................................................................................................ 49
Kleene Star ............................................................................................................................................. 49

Simplification of CFGs ............................................................................................................................. 50

Reduction of CFG .................................................................................................................................... 50
Removal of Unit productions ................................................................................................................. 51
Removal of Null Productions .................................................................................................................. 53

Chomsky Normal Form ........................................................................................................................... 54

Greibach Normal Form ............................................................................................................................ 56

Left and Right Recursive Grammars ........................................................................................................ 57

Pumping Lemma for Context-Free Grammars ......................................................................................... 58

Applications of Pumping Lemma ............................................................................................................ 58

5. PUSHDOWN AUTOMATA .................................................................................................... 59

Basic Structure of PDA ............................................................................................................................ 59

Terminologies Related to PDA ................................................................................................................ 60

Instantaneous Description ..................................................................................................................... 60
Turnstile Notation .................................................................................................................................. 61

Acceptance by PDA ................................................................................................................................. 61

Final State Acceptability ......................................................................................................................... 61
Empty Stack Acceptability ...................................................................................................................... 61

Correspondence between PDA and CFL .................................................................................................. 63

Parsing and PDA...................................................................................................................................... 64

Design of Top-Down Parser .................................................................................................................... 65
Design of a Bottom-Up Parser ................................................................................................................ 65

6. TURING MACHINE ............................................................................................................... 67

Definition ................................................................................................................................................ 67

Accepted Language and Decided Language ............................................................................................. 68

Designing a Turing Machine ................................................................................................................... 68

Multi-tape Turing Machine ..................................................................................................................... 70

Multi-track Turing Machine .................................................................................................................... 71

Non-Deterministic Turing machine ......................................................................................................... 72

iv
 Automata Theory

Turing Machine with Semi-infinite Tape ................................................................................................. 73

Time and Space Complexity of a Turing Machine .................................................................................... 74

Linear Bounded Automata ...................................................................................................................... 74

7. DECIDABILITY ...................................................................................................................... 76

Decidability and Decidable Languages .................................................................................................... 76

Undecidable Languages .......................................................................................................................... 78

TM Halting Problem ................................................................................................................................ 78

Rice Theorem .......................................................................................................................................... 79

Undecidability of Post Correspondence Problem .................................................................................... 80

v
 1. INTRODUCTION Automata Theory

Automata – What is it?

The term "Automata" is derived from the Greek word "αὐτόματα" which means
"self-acting". An automaton (Automata in plural) is an abstract self-propelled
computing device which follows a predetermined sequence of operations
automatically.

An automaton with a finite number of states is called a Finite Automaton (FA)

or Finite State Machine (FSM).

Formal definition of a Finite Automaton

An automaton can be represented by a 5-tuple (Q, Σ, δ, q0, F), where:

 Q is a finite set of states.

 Σ is a finite set of symbols, called the alphabet of the automaton.
 δ is the transition function.
 q0 is the initial state from where any input is processed (q0 ∈ Q).
 F is a set of final state/states of Q (F ⊆ Q).

Related Terminologies

Alphabet
 Definition: An alphabet is any finite set of symbols.

 Example: Σ = {a, b, c, d} is an alphabet set where ‘a’, ‘b’, ‘c’, and ‘d’
are symbols.

String
 Definition: A string is a finite sequence of symbols taken from Σ.

 Example: ‘cabcad’ is a valid string on the alphabet set Σ = {a, b, c, d}

Length of a String
 Definition : It is the number of symbols present in a string. (Denoted by
|S|).
1
 Automata Theory

 Examples:

o If S=‘cabcad’, |S|= 6

o If |S|= 0, it is called an empty string (Denoted by λ or ε)

Kleene Star
 Definition: The Kleene star, Σ*, is a unary operator on a set of symbols
or strings, Σ, that gives the infinite set of all possible strings of all possible
lengths over Σ including λ.

 Representation: Σ* = Σ0 U Σ1 U Σ2 U……. where Σp is the set of all

possible strings of length p.

 Example: If Σ = {a, b}, Σ*= {λ, a, b, aa, ab, ba, bb,………..}

Kleene Closure / Plus

 Definition: The set Σ+ is the infinite set of all possible strings of all
possible lengths over Σ excluding λ.

 Representation: Σ+ = Σ1 U Σ2 U Σ3 U…….
Σ+ = Σ* − { λ }

 Example: If Σ = { a, b } , Σ+ ={ a, b, aa, ab, ba, bb,………..}

Language
 Definition : A language is a subset of Σ* for some alphabet Σ. It can be
finite or infinite.

 Example : If the language takes all possible strings of length 2 over Σ =

{a, b}, then L = { ab, bb, ba, bb}

Deterministic and Nondeterministic Finite Automaton

Finite Automaton can be classified into two types:

 Deterministic Finite Automaton (DFA)

 Non-deterministic Finite Automaton (NDFA / NFA)

2
 Automata Theory

Deterministic Finite Automaton (DFA)

In DFA, for each input symbol, one can determine the state to which the
machine will move. Hence, it is called Deterministic Automaton. As it has a
finite number of states, the machine is called Deterministic Finite Machine or
Deterministic Finite Automaton.

Formal Definition of a DFA

A DFA can be represented by a 5-tuple (Q, Σ, δ, q0, F) where:

 Q is a finite set of states.

 Σ is a finite set of symbols called the alphabet.
 δ is the transition function where δ: Q × Σ → Q
 q0 is the initial state from where any input is processed (q0 ∈ Q).
 F is a set of final state/states of Q (F ⊆ Q).

Graphical Representation of a DFA

A DFA is represented by digraphs called state diagram.

 The vertices represent the states.

 The arcs labeled with an input alphabet show the transitions.
 The initial state is denoted by an empty single incoming arc.
 The final state is indicated by double circles.

Example
Let a deterministic finite automaton be 

 Q = {a, b, c},
 Σ = {0, 1},
 q0={a},
 F={c}, and
 Transition function δ as shown by the following table:

Present State Next State for Next State for

Input 0 Input 1
a a b
b c a
c b c

3
 Automata Theory

Its graphical representation would be as follows:

1 0
a b c
1 0
1
0

DFA – Graphical Representation

Non-deterministic Finite Automaton (NDFA)

In NDFA, for a particular input symbol, the machine can move to any
combination of the states in the machine. In other words, the exact state to
which the machine moves cannot be determined. Hence, it is called Non-
deterministic Automaton. As it has finite number of states, the machine is
called Non-deterministic Finite Machine or Non-deterministic Finite
Automaton.

Formal Definition of an NDFA

An NDFA can be represented by a 5-tuple (Q, Σ, δ, q0, F) where:

 Q is a finite set of states.

 Σ is a finite set of symbols called the alphabets.

 δ is the transition function where δ: Q × Σ → 2Q

(Here the power set of Q (2Q) has been taken because in case of NDFA,
from a state, transition can occur to any combination of Q states)

 q0 is the initial state from where any input is processed (q0 ∈ Q).

 F is a set of final state/states of Q (F ⊆ Q).

Graphical Representation of an NDFA: (same as DFA)

4
 Automata Theory

An NDFA is represented by digraphs called state diagram.

 The vertices represent the states.

 The arcs labeled with an input alphabet show the transitions.
 The initial state is denoted by an empty single incoming arc.
 The final state is indicated by double circles.

Example
Let a non-deterministic finite automaton be 

 Q = {a, b, c}
 Σ = {0, 1}
 q0 = {a}
 F={c}
 The transition function  as shown below:

Next State for Next State for

Present State Input 0 Input 1
a a, b b
b c a, c
c b, c c

Its graphical representation would be as follows:

1 0
a b c
0, 1 0, 1
0 0, 1

NDFA – Graphical Representation

DFA vs NDFA
The following table lists the differences between DFA and NDFA.

DFA NDFA

5
 Automata Theory

The transition from a state is to a single The transition from a state can be to
particular next state for each input multiple next states for each input symbol.
symbol. Hence it is called deterministic. Hence it is called non-deterministic.

Empty string transitions are not seen in

NDFA permits empty string transitions.
DFA.

In NDFA, backtracking is not always

Backtracking is allowed in DFA
possible.

Requires more space. Requires less space.

A string is accepted by a NDFA, if at least
A string is accepted by a DFA, if it transits
one of all possible transitions ends in a
to a final state.
final state.

Acceptors, Classifiers, and Transducers

Acceptor (Recognizer)
An automaton that computes a Boolean function is called an acceptor. All the
states of an acceptor is either accepting or rejecting the inputs given to it.

Classifier
A classifier has more than two final states and it gives a single output when it
terminates.

Transducer
An automaton that produces outputs based on current input and/or previous
state is called a transducer. Transducers can be of two types:

 Mealy Machine The output depends both on the current state and the
current input.

 Moore Machine The output depends only on the current state.

Acceptability by DFA and NDFA

A string is accepted by a DFA/NDFA iff the DFA/NDFA starting at the initial state
ends in an accepting state (any of the final states) after reading the string
wholly.

A string S is accepted by a DFA/NDFA (Q, Σ, δ, q0, F), iff

δ*(q0, S) ∈ F

6
 Automata Theory

The language L accepted by DFA/NDFA is

{S | S ∈ Σ* and δ*(q0, S) ∈ F}

A string S′ is not accepted by a DFA/NDFA (Q, Σ, δ, q0, F), iff

δ*(q0, S′) ∉ F

The language L′ not accepted by DFA/NDFA (Complement of accepted language

L) is {S | S ∈ Σ* and δ*(q0, S) ∉ F}

Example
Let us consider the DFA shown in Figure 1.3. From the DFA, the acceptable
strings can be derived.

0
1
a c
0
1
1
d
0

Acceptability of strings by DFA

Strings accepted by the above DFA: {0, 00, 11, 010, 101, ...........}

Strings not accepted by the above DFA: {1, 011, 111, ........}

Converting an NDFA to an Equivalent DFA

Problem Statement
Let X = (Qx, Σ, δx, q0, Fx) be an NDFA which accepts the language L(X). We
have to design an equivalent DFA Y = (Qy, Σ, δy, q0, Fy) such that L(Y) = L(X).
The following procedure converts the NDFA to its equivalent DFA:

Algorithm 1
Input: An NDFA

Output: An equivalent DFA

Step 1 Create state table from the given NDFA.

7
 Automata Theory

Step 2 Create a blank state table under possible input alphabets for the
equivalent DFA.

Step 3 Mark the start state of the DFA by q0 (Same as the NDFA).

Step 4 Find out the combination of States {Q0, Q1,... , Qn} for each
possible input alphabet.

Step 5 Each time we generate a new DFA state under the input alphabet
columns, we have to apply step 4 again, otherwise go to step 6.

Step 6 The states which contain any of the final states of the NDFA are the
final states of the equivalent DFA.

Example
Let us consider the NDFA shown in the figure below.

q δ(q,0) δ(q,1)
a {a,b,c,d,e} {d,e}
b {c} {e}
c ∅ {b}
d {e} ∅
e ∅ ∅

Using Algorithm 1, we find its equivalent DFA. The state table of the DFA is
shown in below.

q δ(q,0) δ(q,1)
[a] [a,b,c,d,e] [d,e]
[a,b,c,d,e] [a,b,c,d,e] [b,d,e]
[d,e] [ e] ∅

[b,d,e] [c,e] [ e]
[ e] ∅ ∅

[c,e] ∅ [b]
[b] [ c] [ e]
[ c] ∅ [b]

State table of DFA equivalent to NDFA

8
 Automata Theory

The state diagram of the DFA is as follows:

1 0
[a,b,c,d,e] [b,d,e] [c,e]

0 1

1 [b]
[a]

1 1 1
0
0
[d,e] [e] [c]

State diagram of DFA

DFA Minimization using Myhill-Nerode Theorem

Algorithm 2
Input DFA

Output Minimized DFA

Step 1 Draw a table for all pairs of states (Qi, Qj) not necessarily connected
directly [All are unmarked initially]

Step 2 Consider every state pair (Qi, Qj) in the DFA where Qi ∈ F and Qj ∉ F
or vice versa and mark them. [Here F is the set of final states]

Step 3 Repeat this step until we cannot mark anymore states:

9
 Automata Theory

If there is an unmarked pair (Qi, Qj), mark it if the pair {δ(Qi, A),
δ (Qi, A)} is marked for some input alphabet.

Step 4 Combine all the unmarked pair (Qi, Qj) and make them a single
state in the reduced DFA.

Example
Let us use Algorithm 2 to minimize the DFA shown below.

0, 1
1 1
b d f
0
0 0 1
1
a c e
1 0
0

State Diagram of DFA

Step 1 : We draw a table for all pair of states.

a b c d e f
a
b
c
d
e
f
Step 2 : We mark the state pairs:

a b c d e f
a
b
c ✓ ✓
d ✓ ✓
e ✓ ✓
f ✓ ✓ ✓

10
 Automata Theory

Step 3 : We will try to mark the state pairs, with green colored check mark,
transitively. If we input 1 to state ‘a’ and ‘f’, it will go to state ‘c’ and ‘f’
respectively. (c, f) is already marked, hence we will mark pair (a, f). Now, we
input 1 to state ‘b’ and ‘f’; it will go to state ‘d’ and ‘f’ respectively. (d, f) is
already marked, hence we will mark pair (b, f).

a b c d e f
a
b
c ✓ ✓
d ✓ ✓
e ✓ ✓
f ✓ ✓ ✓ ✓ ✓

After step 3, we have got state combinations {a, b} {c, d} {c, e} {d, e} that
are unmarked.

We can recombine {c, d} {c, e} {d, e} into {c, d, e}

Hence we got two combined states as: {a, b} and {c, d, e}

So the final minimized DFA will contain three states {f}, {a, b} and {c, d, e}

0, 1
0
1
(a, b) (c, d, e)

(f)
0, 1

State diagram of reduced DFA

==========

End of ebook preview

If you liked what you saw…

Buy it from our store @ https://store.tutorialspoint.com

11