THEORY OF COMPUTATION

UNIT-I
FINITE AUTOMATA

Introduction to languages,Kleen closures,Finite Automata(FA),Transition Graph,Non

determenistic Automata(NFA),Deterministic Automata(DFA),Equivalence of NFA
and DFA,Construction of DFA from NFA and optimization of Finite Automata,
Applications and Limitations of FA (FAT Tool).

UNIT-I
 FINITE AUTOMATA

AUTOMATA THEORY -INTRODUCTION

Automata theory, also known as the theory of computation, is a fundamental concept

in computer science. It focuses on abstract machines and their computational
capabilities. It has a rich history that shaped modern computing.

Timeline of Automata Theory Evolution

The following table highlights the key developments during the evolution of
Automata Theory −

Era Key Developments

Ancient and Mechanical machines (e.g., Heron's
Medieval Roots devices, Al-Jazari's automatons)
Turing Machine (1936)
1930s - 1940s Lambda calculus
Finite automata concept (1943)
Chomsky's Hierarchy (1956)
1950s - 1960s Regular Expressions
Push-Down Automata (PDA)
NP-completeness (1971)
1970s - 1980s
Applications in compiler design and NLP
AI, machine learning, and quantum
computing applications
Modern Era
Advanced research (e.g., bio-inspired
automata)

Automata Theory with Ancient and Medieval Roots

Automaton or automata (in plural) is nothing but a system to solve a problem

automatically. In other words, it is self-operating device. From the old ages with
human evolvement, humans are trying to make their life easier by making such self-
operable machines. These ideas are older enough, even older than formalization of
mathematics.

 Greek Wonders (c. 270 BC) − Heron of Alexandria developed a self-playing

hydraulic organ and an automated theatre featuring moving figures.
 Islamic Golden Age (7th-13th centuries) − Al-Jazari invented his inventions of
water clocks and automatons for dispensed drinks.

Formal Automata Theory (1930s - 1940s)

 Though there were certain progress in automated machines, but the concept of
automata was developed in later ages. In between 1930s and 1940s, it was a golden
period of mathematicians to design such complex systems.

 Alan Turing (1936) − Invented the Turing Machine (a simple, mathematically-based

computation system). It had the theoretical ability to simulate any other system,
becoming a fundamental concept in computer science.
 Alonzo Church − He coined the concept of lambda calculus, a formal system for
computing using functions, was later proven to be the equivalent to Turing Machines
and a universal computation model.
 Warren McCulloch and Walter Pitts (1943) − Designed the concept of finite
automata, a simplified version of Turing and Church's models. It is proved useful for
real-world problem modeling and laid the groundwork for a branch of computer
science.

Automata Theory: Development in the 1950s and 1960s

From the decades of 1930s and 1940s the development of automata theory was
formulated, but it becomes much more advanced in 50s to 60s decade. Here we have
started to understand and manipulate the complexity of languages used in
computation.

 Noam Chomsky's Hierarchy of Formal Languages (1956) − A revolutionary

classification system categorizes formal languages based on complexity, with four
levels: Regular grammar, Context Free grammar, Context Sensitive grammar and
Recursively Enumerable Grammar.
 Stephen Kleene's Contributions (1950s) − Coined the concept of Regular
expressions, representing patterns in strings using symbols and operators, connecting
to finite automata.
 Push-Down Automata (PDA) and Context-Free Grammars (CFG) − PDA and
CFG are powerful mathematical models capable of handling complex languages with
nested structures and rule-based systems that define valid string formation.

Theoretical Advancements in the 1970s and 1980s

In the latter half of the 20th century, there were many such theoretical advancements
and practical applications in various computing fields associated with automata theory
or the theory of computation.

Stephen Cook's NP-completeness (1971)

Introduced the concept of NP-completeness, which suggests that a problem can be

quickly verified, but solving it directly may be computationally expensive.

Automata Theory with Modern Applications and Advancements

All the newer technologies are being used are relying on the same fundamental
computing and AI, machine learning, bioinformatics, and quantum computing. It
helps design intelligent agents capable of navigating complex environments, analyze
 the complexity of learning algorithms, and design efficient models for tasks like
pattern recognition and sequence analysis.

 Compiler Design −Analyzing code syntax using finite automata and regular
expressions.
 Natural Language Processing (NLP) −Automata theory is being used to understand
and manipulate human language.
 Contributions to Advanced Fields −
1. Creating intelligent agents capable of navigating complex environments
2. Analyzes learning algorithm complexity, and creates efficient models for tasks
like pattern recognition and sequence analysis.

Applications of Automata Theory in Natural Language Processing

Natural language processing (NLP) involves speech recognition, spelling correction,

and information retrieval using computers. NLP provides a great scope for automata
theory since the tasks are deterministic and it has a great grammatical connection.

NLP Tasks − Speech recognition, spelling correction, information retrieval.

Automata Theory's Role − Helps solve NLP problems due to determinism and
grammatical connection.
Finite State Methods − Useful for NLP due to mathematical model and compact data
representation.
1. Finite State Machines − Decide on string acceptance/rejection.
2. FTransducers − Provide output for input.
3. FApplications − Determining if a word belongs to a specific language.

Morphology and Automata

In NLP, there is a concept called Morphology which is the study of the structure and
pattern of word formation, with morphemes being the basic units that contain
meaning.

Morphemes − Basic units with meaning (e.g., "walk," "s").

Free Morphemes − Standalone words (e.g., "walk").
Bound Morphemes − Require attachment (e.g., "-s").
Automata for Morphology −
1.Finite State Automata − Determine if a word belongs to a language.
2.Transducers − Parse and generate words from lexical form.

Compilers and Programming Languages

Automata theory has big role to play in programming languages and their translators,
namely the compilers and interpreters.

 Compilers and Interpreters − Translate high-level code to low-level code.

 Automata Theory's Role − Enables machines to understand high-level languages.
 Finite Automata in Compilers −
 Lexical Analysis − Breaks code into tokens using regular expressions (finite
automata).
  Parsing − Uses context-free grammars (CFG) to build an Abstract Syntax
Tree (AST).
 Top-down Parsing (Recursive Descent) − Simpler but can require
backtracking (no automata).
 Bottom-up Parsing − Uses CFGs to construct automata and build
AST.

Real-World Applications of Automata Theory

We explained the applications of Automata in broad spectrum, but Automata finds its
role in fairly simple real-word applications as well −

 Network Protocols − Define rules, match traffic patterns (intrusion detection,

firewalls).
 Digital Circuits − Model behaviour, analyse sequential circuits.
 Vending Machines − Design control logic, manage states/transitions.
 Bioinformatics − Analyse DNA sequences, identify patterns (gene research).

 Current Research Areas −

 Formal verification
 Probabilistic automata
 Bio-inspired automata, which aim to design novel automata models inspired
by biological systems.

Theory of Automata

Theory of automata is a theoretical branch of computer science and mathematical. It is

the study of abstract machines and the computation problems that can be solved using
these machines. The abstract machine is called the automata. The main motivation
behind developing the automata theory was to develop methods to describe and
analyse the dynamic behaviour of discrete systems.

This automaton consists of states and transitions. The State is represented by circles,
and the Transitions is represented by arrows.

Automata is the kind of machine which takes some string as input and this input
goes through a finite number of states and may enter in the final state.

Symbols:

Symbols are an entity or individual objects, which can be any letter, alphabet or any
picture.

Example:

1, a, b, #

Alphabets:

Alphabets are a finite set of symbols. It is denoted by ∑.

 Examples:

∑ = {a, b}

∑ = {A, B, C, D}

∑ = {0, 1, 2}

∑ = {0, 1, ....., 5]

∑ = {#, β, Δ}

String:

It is a finite collection of symbols from the alphabet. The string is denoted by w.

Example 1:

If ∑ = {a, b}, various string that can be generated from ∑ are {ab, aa, aaa, bb, bbb, ba,
aba.....}.

o A string with zero occurrences of symbols is known as an empty string. It is

represented by ε.
o The number of symbols in a string w is called the length of a string. It is
denoted by |w|.

Example 2:

. w = 010
.
. Number of Sting |w| = 3

Introduction to languages

Language:

A language is a collection of appropriate string. A language which is formed over Σ

can be Finite or Infinite.

In automata theory, language refers to a set of words accepted by an automaton. It is

a subset of an alphabet and can be finite or infinite. Formal language theory is based
on automata theory and involves symbols, alphabets, and words.

Example: 1

L1 = {Set of string of length 2}

= {aa, bb, ba, bb} Finite Language

Example: 2
L2 = {Set of all strings starts with 'a'}

= {a, aa, aaa, abb, abbb, ababb} Infinite Language

Example of Finite Language:
L1 = { set of string of 2 }
L1 = { xy, yx, xx, yy }

Example of Infinite Language:

L1 = { set of all strings starts with 'b' }
L1 = { babb, baa, ba, bbb, baab, ....... }

KLEEN CLOSURES
Kleene Closure, also known as the Kleene Star, is a fundamental concept in the
Theory of Computation (TOC). It is a unary operation on a set of symbols or strings
that results in the infinite set of all possible strings of all possible lengths over the
given set, including the empty string (λ).

Kleene Closure (*) gives an infinite set of all possible strings of any length,
including epsilon (ε), over input values of sigma (Σ). It is also known as Kleene
Star and denoted by Σ*. The following is a mathematical notation of Kleene
Clouser.

Σ* = Σ0 U Σ1 U Σ2 ………. .

Where Σ0 represents epsilon (ε) and gives infinite language.

Kleene Closure is extensively used in regular expressions and finite automata. It

allows the representation of repetitive patterns within strings. For example, the regular
expression (a+b)* represents the set of all strings composed of 'a' and 'b' in any order
and any number of times, including the empty string.
Example
Consider the regular expression (ab+a)*. Using Kleene's Theorem, we can
systematically generate a finite automaton (FA) for this expression. The steps involve
creating smaller finite automata for sub-expressions and combining them using union,
concatenation, and Kleene Star operations.
Union Operation
For two regular expressions S and T, the union operation (S + T) results in a new FA
that accepts the language L(S) ∪ L(T). This involves creating a new start state with
null transitions to the start states of both S and T, and combining their final states into
a single final state.
Concatenation Operation
For two regular expressions S and T, the concatenation operation (S.T) results in a
new FA that accepts the language L(S).L(T). This involves connecting the final state
of S to the start state of T with a null transition.
Example: Union of Regular Expressions S=ab and T=cd
Union of S=a∗ and T=b∗

Example 4: Union of S=(a∣b) and T=c

Concatenation of S=a and T=b

Concatenation of S=a∗ and T=b

Concatenation of S=ab and T=cd

Concatenation of S=(a∣b) and T=c

Kleene Star Operation

For a regular expression S, the Kleene Star operation (S*) results in a new FA that
accepts the language L(S)*. This involves creating a new start state and a new final
state, with null transitions to and from the start and final states of S, allowing for
repetition.

Example: Suppose Σ input value is “(0,1)”, applying Kleene Closure (*) over Σ
input which gives the following result

{0,1}* =( 0∪1)* = {ε,0,1,11,00, 01,10 ,0101,1010, …………., n} // where n is

infinity

Result (0,1)* will give all possible combinations of “0” and “1” of any length,
including epsilon (ε). So, we can say Kleene Closure always gives infinite
language.

Examples of Kleen Closure

The following are various examples of Kleene Closure over input values of sigma (Σ).

Example 01: Kleene Closure applied to the empty set

∅* = {ε}

Example 02: Kleene Closure applied to a single character

{a}* = {ε, a, aa, aaa, aaaa, ……………., } // all combination of “a” with any length

Example 03: Kleene Closure applied to the set of characters

{x, y, z}* = { ε, x, y, z, xx, xy, xz, yx, yy, yz, za, zb, zz, xxx, xxy, …………}.

Example 04: Kleene Closure applied to a single string

{ab}* = {ε, ab ,abab,ababab, abababab, ………….n}

Example 05: Kleene Closure applied to the set of strings

{01,0}* = {ε, “01”, “0”, “0101”, “010”, “001”, “00”, “010101”, “01010”, “01001”,
“0100”, “00101”, “0010”, “0001”, “000”, …}

Example 06: Kleene Closure applied to some Special case

{a*bc*} = {a*+b+ c*} = {b, ab, aabcc, ………… n} // where “b” is compulsory in
each string

Positive Closure

Positive Closure is similar to Keleene Closure except epsilon (ε). It also gives an
infinite set of all possible strings of any length, excluding epsilon (ε) over input
values of sigma (Σ). Σ+ denotes it. The following is the mathematical notation of
Positive Closure.

Σ+ = Σ*– {ε}

Suppose Σ input value is “(0,1)”, applying Positive Closure (+) over Σ input which
gives the following result

{0,1}+ = (0∪1)+ = {0,1,11,00, 01,10 ,0101,1010, …………., n} // where n is

infinity

Result (0,1)+ will give all possible combinations of “0” and “1” of any
length excluding epsilon (ε). So, we can say Positive Closure always gives infinite
language.

Examples of Positive Closure

All examples of positive Closure are similar to Kleene closure except epsilon (ε). The
following are the various examples of Positive Closure over sigma (Σ) input values.
Example 01: Kleene Closure applied to an empty set, the result is an empty set

∅+ = {}

Example 02: Kleene Closure applied to single character

{a}+ = {a, aa, aaa, aaaa, …………….} // all combination of “a” with any length

Difference Between Σ* and Σ+

Σ* contains the epsilon (ε) sting along with other N strings, but Σ+ is just like the
Σ* but does not hold the epsilon (ε) sting.

We can say Σ* = Σ+ + Σ0 . Where Σ0 represent epsilon (ε)

So, Σ* holds the identity(single) element called Absalon, but Σ+ does not have the
identity element called epsilon (ε)

Tips about Positive and Kleene Clouser

Σ* is a Universal Set.

Cardinality: The number of elements in a set, basically |Σ n |, is called cardinality.

So, |Σ3| has eight cardinality.

Kleene Closure is a powerful tool in the Theory of Computation, enabling the

representation and manipulation of repetitive patterns in strings. It is essential for
constructing finite automata and regular expressions, providing a systematic approach
to handle complex languages.

Finite Automata(FA)

Automata – What is it?

The term "Automata" is derived from the Greek word "αὐτόματα" which means "self-
acting". An automaton (Automata in plural) is an abstract self-propelled computing
device which follows a predetermined sequence of operations automatically.

An automaton with a finite number of states is called a Finite Automaton (FA)

or Finite State Machine (FSM).

o Finite automata are used to recognize patterns.

o It takes the string of symbol as input and changes its state accordingly. When
the desired symbol is found, then the transition occurs.
 o At the time of transition, the automata can either move to the next state or stay
in the same state.
o Finite automata have two states, Accept state or Reject state. When the input
string is processed successfully, and the automata reached its final state, then it
will accept.

Formal Definition of FA

A finite automaton is a collection of 5-tuple (Q, ∑, δ, q0, F), where:

. Q: finite set of states

. ∑: finite set of the input symbol
. q0: initial state
. F: final state
. δ: Transition function

Finite Automata Model:

Finite automata can be represented by input tape and finite control.

Input tape: It is a linear tape having some number of cells. Each input symbol is
placed in each cell.

Finite control: The finite control decides the next state on receiving particular input
from input tape. The tape reader reads the cells one by one from left to right, and at a
time only one input symbol is read.

Types of Automata:

There are two types of finite automata:

1. DFA(deterministic finite automata)

2. NFA(non-deterministic finite automata)
1. DFA

DFA refers to deterministic finite automata. Deterministic refers to the uniqueness of

the computation. In the DFA, the machine goes to one state only for a particular input
character. DFA does not accept the null move.

2. NFA

NFA stands for non-deterministic finite automata. It is used to transmit any number of
states for a particular input. It can accept the null move.

Some important points about DFA and NFA:

1. Every DFA is NFA, but NFA is not DFA.

2. There can be multiple final states in both NFA and DFA.
3. DFA is used in Lexical Analysis in Compiler.
4. NFA is more of a theoretical concept.

Transition Graph

A transition diagram or state transition diagram is a directed graph which can be

constructed as follows:

o There is a node for each state in Q, which is represented by the circle.

o There is a directed edge from node q to node p labeled a if δ(q, a) = p.
o In the start state, there is an arrow with no source.
o Accepting states or final states are indicating by a double circle.
 Some notations that are used in the transition diagram:

There is a description of how a DFA operates:

1. In DFA, the input to the automata can be any string. Now, put a pointer to the start
state q and read the input string w from left to right and move the pointer according to
the transition function, δ. We can read one symbol at a time. If the next symbol of
string w is a and the pointer is on state p, move the pointer to δ(p, a). When the end of
the input string w is encountered, then the pointer is on some state F.

2. The string w is said to be accepted by the DFA if r ∈ F that means the input string
w is processed successfully and the automata reached its final state. The string is said
to be rejected by DFA if r ∉ F.

Example 1:

DFA with ∑ = {0, 1} accepts all strings starting with 1.

Solution:
The finite automata can be represented using a transition graph. In the above diagram,
the machine initially is in start state q0 then on receiving input 1 the machine changes
its state to q1. From q0 on receiving 0, the machine changes its state to q2, which is
the dead state. From q1 on receiving input 0, 1 the machine changes its state to q1,
which is the final state. The possible input strings that can be generated are 10, 11,
110, 101, 111......., that means all string starts with 1.

Example 2:

NFA with ∑ = {0, 1} accepts all strings starting with 1.

Solution:

The NFA can be represented using a transition graph. In the above diagram, the
machine initially is in start state q0 then on receiving input 1 the machine changes its
state to q1. From q1 on receiving input 0, 1 the machine changes its state to q1. The
possible input string that can be generated is 10, 11, 110, 101, 111......, that means all
string starts with 1.

Transition Table

The transition table is basically a tabular representation of the transition function. It

takes two arguments (a state and a symbol) and returns a state (the "next state").

A transition table is represented by the following things:

o Columns correspond to input symbols.

o Rows correspond to states.
o Entries correspond to the next state.
o The start state is denoted by an arrow with no source.
o The accept state is denoted by a star.

A transition table is a tabular representation of the transition function that takes two
arguments and returns a state. The column contains the state in which the automaton
will be on the input represented by that column. The row corresponds to the state the
finite control unit can be in. The entry for one row corresponding to state q and the
column corresponds to input a is the state δ(q, a).
Example 1:

Solution:
Transition table of given DFA is as follows:

PrSPresent Next state for Input NtNext State of Input

Statetate 0Input 1
→q0 q1 q2
q1 q0 q2
*q2 q2 q2
Explanation:

o In the above table, the first column indicates all the current states. Under
column 0 and 1, the next states are shown.
o The first row of the transition table can be read as, when the current state is q0,
on input 0 the next state will be q1 and on input 1 the next state will be q2.
o In the second row, when the current state is q1, on input 0, the next state will
be q0, and on 1 input the next state will be q2.
o In the third row, when the current state is q2 on input 0, the next state will be
q2, and on 1 input the next state will be q2.
o The arrow marked to q0 indicates that it is a start state and circle marked to q2
indicates that it is a final state.

Example 2:

Solution:

Transition table of given NFA is as follows:

PrSPresent Next state for Input Next State of Input 1

Statetate 0Input
→q0 q0 q1
 q1 q1, q2 q2
q2 q1 q3
*q3 q2 q2
Explanation:

o The first row of the transition table can be read as, when the current state is q0,
on input 0 the next state will be q0 and on input 1 the next state will be q1.
o In the second row, when the current state is q1, on input 0 the next state will
be either q1 or q2, and on 1 input the next state will be q2.
o In the third row, when the current state is q2 on input 0, the next state will be
q1, and on 1 input the next state will be q3.
o In the fourth row, when the current state is q3 on input 0, the next state will be
q2, and on 1 input the next state will be q2.
For the transition graph in the above figure the transition table would be as follows:

states a b
q0 q1 q2
q1 q1 q3
q2 q2 q3
*q3 q3 q3
In the above table we observe that on giving a as input to DFA in state q0 the DFA
changes state to q1 and on input b the state changes to q2. Similarly by writing this
information for other states we generate the transition graph.

Finite Automaton can be classified into two types −

 Deterministic Finite Automaton (DFA)

 Non-deterministic Finite Automaton (NDFA / NFA)

Deterministic Finite Automaton (DFA)

In DFA, for each input symbol, one can determine the state to which the machine will
move. Hence, it is called Deterministic Automaton. As it has a finite number of
states, the machine is called Deterministic Finite Machine or Deterministic Finite
Automaton.

Formal Definition of a DFA

A DFA can be represented by a 5-tuple (Q, ∑, δ, q0, F) where −

Q is a finite set of states.

∑ is a finite set of symbols called the alphabet.
δ is the transition function where δ: Q × ∑ → Q
 q0 is the initial state from where any input is processed (q0 ∈ Q).
F is a set of final state/states of Q (F ⊆ Q).

Key Properties of a DFA

DFAs have the following defining characteristics:

1.Deterministic Transitions: For every state and every input symbol, there is exactly one
transition to a next state.

2.Finite States: A DFA has a finite number of states that define its possible
configurations.

3.Single Start State: The DFA begins processing from a specific initial state.

4.Acceptance Condition: A DFA accepts an input string if, after processing all input
symbols, it ends in one of its accept states.

Graphical Representation of a DFA

A DFA is represented by digraphs called state diagram.

The vertices represent the states.

The arcs labeled with an input alphabet show the transitions.
The initial state is denoted by an empty single incoming arc.
The final state is indicated by double circles.

Example Let a deterministic finite automaton be →

Q = {a, b, c}, ∑ = {0, 1}, q0 = {a}, F = {c}, and

Transition function δ as shown by the following table −

Present
Next State for Input 0 Next State for Input 1
State
a a b
b c a
c b c

Its graphical representation would be as follows −

Examples of DFA

Example 1:

Design a FA with ∑ = {0, 1} accepts those string which starts with 1 and ends with 0.

Solution:

The FA will have a start state q0 from which only the edge with input 1 will go to the
next state.

In state q1, if we read 1, we will be in state q1, but if we read 0 at state q1, we will
reach to state q2 which is the final state. In state q2, if we read either 0 or 1, we will
go to q2 state or q1 state respectively. Note that if the input ends with 0, it will be in
the final state.

Example 2:

Design a FA with ∑ = {0, 1} accepts the only input 101.

Solution:

In the given solution, we can see that only input 101 will be accepted. Hence, for
input 101, there is no other path shown for other input.

Example 3:

Design FA with ∑ = {0, 1} accepts even number of 0's and even number of 1's.

Solution:

This FA will consider four different stages for input 0 and input 1. The stages could
be:
Here q0 is a start state and the final state also. Note carefully that a symmetry of 0's
and 1's is maintained. We can associate meanings to each state as:

q0: state of even number of 0's and even number of 1's.

q1: state of odd number of 0's and even number of 1's.
q2: state of odd number of 0's and odd number of 1's.
q3: state of even number of 0's and odd number of 1's.

Example 4:

Design FA with ∑ = {0, 1} accepts the set of all strings with three consecutive 0's.

Solution:

The strings that will be generated for this particular languages are 000, 0001, 1000,
10001, .... in which 0 always appears in a clump of 3. The transition graph is as
follows:

Note that the sequence of triple zeros is maintained to reach the final state.

Example 5:

Design a DFA L(M) = {w | w ε {0, 1}*} and W is a string that does not contain
consecutive 1's.

Solution:

When three consecutive 1's occur the DFA will be:

Here two consecutive 1's or single 1 is acceptable, hence

 The stages q0, q1, q2 are the final states. The DFA will generate the strings that do
not contain consecutive 1's like 10, 110, 101,..... etc.

Example 6:

Design a FA with ∑ = {0, 1} accepts the strings with an even number of 0's followed
by single 1.

Solution:

The DFA can be shown by a transition diagram as:

Example: DFA for Strings Containing ‘101’

Let’s create a DFA to recognize binary strings containing the substring ‘101’.

We need to construct a Deterministic Finite Automaton (DFA) to recognize strings

over the alphabet Σ = {0, 1} that contain the substring ‘101’. A string is considered
accepted if it contains this specific pattern anywhere in the input. For example:

 Accepted Strings: '101', '1101', '0101', '1010', '10101'

 Rejected Strings: '1', '0', '100', '110', '111'

The DFA must correctly handle:

1. Start state: Where the computation begins.

2. Transitions: Moves between states based on the input character.
3. Accept state: Where the DFA ends if the substring ‘101’ is found in the input.