HAWASSA UNIVERSITY

Department of Computer Science

Automata and Complexity Theory

(Theory of Computation)

2014 E.C
 Course Contents
1. Theory of Computation 3. Introduction Turing Machine(TM)
 Standard TM
 Automata Theory
 Construction of TM
 Formal languages theory
 TM Decidable and Acceptable
 Complexity Theory
 Undecidable Problems
 Type of Finite Automata  Undecidability
 Conversion of Automata  Turing Decidable
 Turing Acceptable
2. Regular Languages, Expressions
 Regular languages and grammars  Undecidable Problems
 Conversions of Res to FA 4. Computability Theory
 Construction of FA from REs  Recursive Functions
 Context Free Languages, Grammars and  Recursive Languages(RLs)
Simplifications  Recursive Enumerable languages(RELs)


Type 2 Grammars, Closure properties
5. Computational complexity Theory
CFG Simplifications  Big o Notation
 Pushdown Automata(PDA)  Class P vs NP
 Construction of PDA for the given Languages  NP-Complete Problems
 PDA Acceptance  Cook’s Theorem
Automata and Complexity Theory 2
Course Goals
• Theoretically explore the capabilities and limitations of computers
• Complexity of Algorithms
• How much resources are required to run the algorithms?
• Automata theory
• How can we mathematically model computation?
• Formal language Theory
• How a language defined, described, bounded & accepted?
• Complexity theory
• What makes some problems computationally hard and others easy?
• Computability theory
• What problems can be solved by a computer?
• Computational Complexity theory
• How the solution is efficient and effective enough?

Automata and Complexity Theory 3

Theory of Computation: Areas
• Formal Language Theory
• language = set of sentences (strings)
• grammar = rules for generating strings
• production/deduction systems
• capabilities & limitations
• application: programming language specification
• Automata Theory (Abstract Machines)
• models for the computing process with various resources
• characterize ``computable”
• capabilities & limitations
• application: parsing algorithms
• Complexity Theory
• inherent difficulty of problems (upper and lower bounds)
• time/space resources
• intractable or unsolvable problems
• application: algorithm design

Automata and Complexity Theory 4

 Chapter 1
Theory of Computation
Automata Theory

Automata and Complexity Theory 5

Computation
• Computation is a general term for any type of information processing that
can be represented as an algorithm precisely (mathematically).
Examples:
• Adding two numbers in our brains, on a piece of paper or using a calculator.
• Converting a decimal number to its binary presentation or vise versa.
• Finding the greatest common divisors of two numbers.
• …etc.
• Theory of Computation
• A very fundamental and traditional branch of Theory of Computation that
seeks:
1. A more tangible definition for the intuitive notion of algorithm which results in a
more concrete definition for computation.
2. Finding the set of boundaries (limitations) of computation.

Automata and Complexity Theory 6

Complexity Theory
• What make some problems computational hard and others easy?(basic question)
• To solve this problems:
• Understand aspects at the root of difficulty,
• Settle for approximately perfect solution,
• Some are hard in worst case and mostly easy,
• Depending on the application, you may choose slow or quick solutions,
• Other types of computations are randomized that speed up certain tasks.
Algorithm:
• A finite sequence of simple instructions that is guaranteed to halt in a finite amount of
time.
• This is a very abstract definition, since:
• We didn’t specify the nature of this simple instructions.
• For example an instruction can “increment a number by one” or “Calculate the triple integral”
• We didn’t specify the entity which can execute these instructions.
• For example: is this entity a person, a computer, …
• If it is a computer what is the processor type?andHow
Automata muchTheory
Complexity memory does it have? …. ? 7
Complexity Theory and Classes
• Complexity classes provide a way to group languages of similar computational
complexity.
• A complexity class is a set of problems of related complexity. Complexity classes
are defined by the following factors:
• The type of computational problem: The most commonly used problems are decision
problems. However, complexity classes can be defined based on function problems,
counting problems, optimization problems, promise problems, etc.
• The model of computation: The most common model of computation is the deterministic
Turing machine, but many complexity classes are based on non-deterministic Turing
machines, Boolean circuits, quantum Turing machines, monotone circuits, etc.
• The resource (or resources) that is being bounded and the bound: These two properties are
usually stated together, such as "polynomial time", "logarithmic space", "constant depth",
etc.
• The set of decision problems solvable by a deterministic/non deterministic Turing
machine within time f(n). (This complexity class is known as P and NP class with
PTIME(f(n)) and NTIME(f(n)) respectively.)
Automata and Complexity Theory 8
Complexity Theory and Classes …
• Recall that the complexity class P is the collection of all languages L that can
be recognized “efficiently", i.e., by a deterministic polynomial-time(may
be(o(kn)) Turing machine. Whereas the traditional definition of NP associates
the class NP with the collection of languages that can be efficiently recognized
by a non-deterministic Turning machine, we provide an alternative definition,
that in our view better captures the conceptual contents of the class.
• Informally, we view NP as the class of all languages that admit a short
“certificate" for membership in the language. Given this certificate, called a
witness, membership in the language can be verified efficiently, i.e., in
polynomial time(o(n2)).
• Every computable function can be computed by a Turing machine.
• Every efficiently computable function can be efficiently computed by a
Turing machine.
Automata and Complexity Theory 9
Complexity Theory and Classes …
Major Classes

Automata and Complexity Theory 10

Computational Complexity Theory
• Computational complexity theory focuses on classifying computational problems according to
their resource usage, and relating these classes to each other. A computational problem is a task
solved by a computer. A computation problem is solvable by mechanical application of
mathematical steps, such as an algorithm.
• Computational complexity, a measure of the amount of computing resources (time and space)
that a particular algorithm consumes when it runs. For those problems, computer scientists seek
to find heuristic algorithms that can almost solve the problem and run in a reasonable amount of
time.
• Complexity theory helps computer scientists relate and group problems together into complexity
classes. Complexity helps determine the difficulty of a problem, often measured by how much
time and space (memory) it takes to solve a particular problem.
• Central to the development of computational complexity theory is the notion of a decision
problem. Decision problems are typically specified in the form of questions about a class of
mathematical objects whose positive instances determine the set in question

Automata and Complexity Theory 10

Computational Complexity Theory …
• Aim of Complexity Theory :Classes of problems and Languages

• All Languages

Decidable
Languages

“Easy” Problems “Hard” Problems

Automata and Complexity Theory 12

Complexity Theory and Classes … Examples

All
Languages
Decidable
Languages

O (n ) O (n 2 ) O (2n )
O ( n log n )

Automata and Complexity Theory 13

 Classifying Problems
• We have seen that decision problems (and their associated
languages) can be classified into decidable and undecidable.
• This result was obtained by Turing and others in the 1930’ - before the invention of
computers.

• After the invention of computers, it became clear that it would be

useful to classify decidable problems, to distinguish harder problems
from easier problems.
• This led to the development of computational complexity theory
in the 1960’s and 1970’s.

Automata and Complexity Theory 14

 Describing Decison Problems
• Problems described by functions
• functions assign ouputs to inputs
• described by tables, formulae, circuits, math logic, algorithms
Example: TRAVELLING SALESPERSON Problem
Input: n(n-1)/2 distances between n cities
Output: order to visit cities that gives shortest tour (or length of tour)
• Problems described by languages
• ``yes/no’’ problems or decision problems
• a language represents a problem or question
• input strings in the language: answer is ``yes’’; else ``no”
Example: HALTING Problem
Input: a string w and a Turing Machine M
Output: ``yes” if w  L(M;`) ”no” otherwise
• equivalent to asking if string (e, w)  LH
where
LH {(e, w) | Turing Machine M e halts on input w}

Automata and Complexity Theory 15

What is Automata Theory?
• Study of abstract computing devices, or “machines”
• Automaton = an abstract computing device
• Note: A “device” need not even be a physical hardware!
• A fundamental question in computer science:
• Find out what different models of machines can do and cannot do
• The theory of computation
• Computability vs. Complexity

Automata and Complexity Theory 16

Theory of Computation: A Historical Perspective

1930s • Alan Turing studies Turing machines

• Decidability
• Halting problem

1940-1950s • “Finite automata” machines studied

• Noam Chomsky proposes the
“Chomsky Hierarchy” for formal
languages

1969 Cook introduces “intractable” problems

or “NP-Hard” problems

1970- now Modern computer science: compilers,

computational & complexity theory evolve

Automata and Complexity Theory 17

Theory of Computation …
• Models of Computation:
• Finite Automaton(FA): Finite control (set of states). Input is read in one direction. No
memory.
• Pushdown Automaton(PDA): Finite control (set of states). Input is read in one
direction. Memory is restricted to be in the form of a stack (however the stack length is
unbounded).
• Linear Bounded Automata(LBA)- Finite control (set of states). Input is read in two
directions. Memory is restricted to be in the form of double stacks (however the stack
length is unbounded).
• Turing machines(TM):A Turing machine can be thought of as a physical computational
device consisting of a tape and a head.

Automata and Complexity Theory 15

Model of Computations and Applications
Expressive Power of various Automata:
•The Expressive Power of any machine can be determined
from the class or set of Languages accepted by that
particular type of Machine. Here is the increasing
sequence of expressive power of machines :

FA<PDA<LBA<TM

1. Finite Automata (FA)

• For the designing of lexical analysis of a compiler.
• For recognizing the pattern using regular expressions.
• For the designing of the combination and sequential circuits using
Mealy and Moore Machines.
• Used in text editors.
• For the implementation of spell checkers.
FA≡ PDA with finite Stack ≡ TM with finite tape ≡ TM with unidirectional tape ≡ TM with read
only tape

Automata and Complexity Theory 19

Model of Computations and Applications …
2. Push Down Automata (PDA)
• For designing the parsing phase of a compiler (Syntax Analysis).
• For implementation of stack applications.
• For evaluating the arithmetic expressions.
• For solving the Tower of Hanoi Problem.
PDA ≡ Finite Automata with Stack

3. Linear Bounded Automata (LBA)

• For implementation of genetic programming.
• For constructing syntactic parse trees for semantic analysis of the compiler.
LBA ≡ Finite Automata with double Stacks(2 Stacks)

4. Turing Machine (TM) –

• For solving any recursively enumerable problem.
• For understanding complexity theory.
• For implementation of neural networks.
• For implementation of Robotics Applications.
• For implementation of artificial intelligence.
Turing Machine ≡ PDA with additional Stack ≡ FA with 2 Stacks

Automata and Complexity Theory 20

Theory of Computation …
• A Formal Language is an abstraction of the general characteristics of
programming languages.
• A formal language consists of a set of symbols and some rules of formation by
which these symbols can be combined into entities called sentences.
• A formal language is the set of all strings permitted by the rules of formation.
• Although some of the formal languages we study here are simpler than
programming languages, they have many of the same essential features.
• We can learn a great deal about programming languages from formal languages.
• Though computer science is a practical discipline, study of Formal Languages is a
theoretical approach and it can also be viewed as “Theory of Computation”.
• To start with in this chapter we will discuss about the mathematical notations and
basic concepts required to understand about the Theory of Computation.

Automata and Complexity Theory 21

Theory of Automata
• Theory of automata is the study of abstract machines and the computation
problems that can be solved using these machines. The abstract machine is
called the automata. The main motivation behind developing the automata
theory was to develop methods to describe and analyze the dynamic behavior
of discrete systems.
• This automaton consists of states and transitions. The State is represented
by circles, and the Transitions is represented by arrows.
• Automata is the kind of machine which takes some string as input and this
input goes through a finite number of states and may enter in the final state.

Automata and Complexity Theory 22

Theory of Automata …
• At this point, we have only a general understanding of what an automaton is.
• To progress, let us provide more formal and precise definitions, and start to develop rigorous
results.
• We begin with finite acceptors, which are a simple case of the general scheme discussed
already in the previous chapter.
• This type of automaton is characterized by having no temporary storage. Since, an input file
cannot be rewritten; a finite automaton is severely limited in its capacity to “remember” things
during the computation.
• A finite amount of information can be retained in the control unit by placing the unit into a
specific state.
• But since the number of such states is finite, a finite automaton can only deal with situations in
which the information to be stored at any time is strictly bounded.
• The figure given below is an example of Finite Automata, which accepts all legal Pascal
identifiers.

Automata and Complexity Theory 23

Theory of Automata …
• In general, an Automaton is defined as a system where energy, materials and
information are transformed, transmitted and used for performing some
functions without direct participation of man, for example, automatic machine
tools, automatic packing machines, and automatic photo printing machines.
• In Computer Science the term ‘Automaton’ means “Discrete Automaton”.
• An automaton is an abstract model of a digital computer.
• As such, every automaton includes some essential features.
• Automaton mechanism for reading input. It will be assumed that the input is a
string over a given alphabet, written on an input file, which the automaton can
read but not change.
• The input mechanism can read the input file left to right, one symbol at a time.
The input mechanism can also detect the end of the input string (by sensing an
end-of-file condition).
Automata and Complexity Theory 24
Characteristics of Finite Automata
• Inputs - It is a finite set of input symbols( or sequences of symbols, {x1, x2,
x3,... ...
• Outputs - It is a finite set of output symbols, {y1, y2, y3,...ym}, where m is the
number of outputs.
• States - It is a finite set of states, denoted by Q whose definition depends on
the type of automaton.
• There are two main perspectives of Finite state automata- Acceptors that can
tell if a string is in the language and Generator that produces only the strings
in the language. Grammars(G)
Set of Symbols, Alphabets, Strings

Generate
Languages(L) Language L generated by Grammar G
L(M)=L
Accept
Machines(M) Language L Accepted by Machine M

Automata and Complexity Theory 25

 Finite Automata(FA) …
• The automaton can produce output of some form.
• It may have a temporary storage device, consisting of sufficient number of cells, each capable
of holding (not necessarily the same one as the input alphabet).
• The automaton can read and change the contents of the storage cells.
• Finally, automaton has a control unit, which can be in any one of a finite number of internal
states, and which can change state is some defined manner.
• Figure shown below shows a schematic representation of a general automaton:

Automata and Complexity Theory 26

Finite Automata(FA) …
• An automaton is assumed to operate in a discrete time frame.
• At any given time, the control unit is in some internal state, and the input
mechanism is scanning a particular symbol of the input file.
• The internal state of the control unit at the next step is determined by the next-
state or transition function.
• This transition function gives the next state in terms of the current state, the
current input symbol, and the information currently in the temporary storage.
• During the transition from one time interval to the next, output may be produced
or the information in the temporary storage changed.
• The term Configuration will be used to refer to a particular state of the control
unit, input file, and temporary storage.
• The transition of the automaton from one configuration to the next will be called a
move.
Automata and Complexity Theory 27
Basic terminologies Finite Automata
• Alphabet: Any finite set of symbols in automata theory is known as Alphabet.
Represented by the letter∑ the set {a, b, c, d, e,} is called Alphabet set, whereas
the letters of set ‘a’, ‘b’, ‘c’, ‘d’, ‘e’ are called symbols.
• String: In automata, a string is a finite sequence of symbols taken from the
alphabet set ∑, For example, the string S = ‘adeaddadc’ is valid upon the
alphabet set∑ = {a, b, c, d,e,}.
• Length of String: The number of symbols present in the string is known as Length of
string. For the string S = ‘adaada’ length of the string is|S| = 6. If the length of the string
is 0, then it is called an empty string.
• Language: Language is the subset of the Kleene star set∑* for some Alphabet
set Σ. Language can be finite or infinite. For example if a language takes all the
possible strings of length 2 over the set Σ = {a, d}, then L = {aa, ad, da, dd}.

Automata and Complexity Theory 28

Types of Finite Automata(FA)
• There are two types of finite automata:
• DFA(deterministic finite automata)
• NFA(non-deterministic finite automata)

• Deterministic refers to the uniqueness of the • It is used to transmit any number of

computation. In the DFA, the machine goes to states for a particular input. It can
one state only for a particular input character. accept the null move.
DFA does not accept the null move.
• In DFA if corresponding to an input symbol,
there is single resultant state i.e. there is only
one transition.
Automata and Complexity Theory 29
Automata … DFA and NFA
• A Deterministic Automaton is one in which each move is uniquely
determined by the current configuration(Uniqueness of Computation).
• If we know the internal state, the input, and the contents of the temporary
storage, we can predict the future behavior of the automaton exactly.
• In a Nondeterministic automaton, that is not so. At each point, a
Nondeterministic automaton may have several possible moves, so we can only
predict a set of possible actions (Guess of Computations).
• An automaton whose output response is limited to a simple “yes” or “no” is
called an acceptor. Presented with an input string, an acceptor either accepts
(recognizes) the string or rejects it.
• A more general automaton, capable of producing strings of symbols as output,
is called a transducer.

Automata and Complexity Theory 30

Automata … DFA and NFA
• Some interpretation is necessary:
• We assume that initially the automaton is in initial state; we indicate this by drawing an
arrow to this state.
• When the automaton is either accepting or rejecting the input string, it is said to be in
any one of the final states; we indicate this by drawing two concentric circles.
• As always, the string to be examined is read from left to right, one character at
each step; when the first symbol is a letter, the automaton goes into state 2,
after which the rest of the string can be either letter or digit.
• Finally, when the entire string is processed the state 2 “accepts” the string or
represents the “yes” state of the automaton.
• Conversely, if the first symbol is a digit, the automaton will go into state 3, the
“reject” or “no” state, and remain there.
• In our solution we assume that no input other than letters or digits is possible.
Automata and Complexity Theory 31
Deterministic Finite Automata(DFA)
• The first type of automaton we study in detail are Finite Automata that are
deterministic in their operation.
• A DFA is defined by the Quintuple (5-Tuples)
M = (Q, Σ, δ, q0, F)
Where, Q is finite set of internal states
Σ is a finite set of symbols called the input alphabet
δ : QX∑ → Q is a total function called the transition function
q0 ∈ Q is the initial state
F ⊆ Q is a set of Final states

Automata and Complexity Theory 32

Deterministic Finite Automaton (DFA)

Input Tape

String

Output

“Accept”
Finite or
Automaton “Reject”

33
DFA…
• A Deterministic Finite Automata operates in the following manner
At the initial time, it is assumed to be in the initial state q0, with its input mechanism
on the leftmost symbol of the input string.
During each move of the automaton, the input mechanism advances one position to the
right(R), so each move consumes one input symbol.
When the end of the string is reached, the string is accepted if the automaton is in one
of its final states F. Otherwise the string is rejected.
The input mechanism can move only from left to right and reads exactly one symbol on
each step.

Automata and Complexity Theory 34

DFA …
• To visualize and represent finite automata, we use transition graphs, in
which the vertices or nodes represent states and the edges represent
transitions.
• The labels on the vertices are the names of the states, while the labels on the
edges are the current values of the input symbols.
• For example, if q0 and q1 are internal states of some DFA M, then the graph
associated with M will have one vertex labeled q 0 and another labeled q1.
• An edge (q0, q1) labeled ‘a’ represents the transition δ(q0, a) = q 1.
• The initial state will be identified by an incoming unlabeled arrow not
originating at any vertex and final states are identified with double circle.

Automata and Complexity Theory 35

DFA …
• More formally, if M = (Q, Σ, δ, q0, F) is a DFA, then its associated transition graph Gm has
exactly |Q| vertices, each one labeled with a different qi ∈ Q.
• For every transition rule δ(qi, qj) labeled ‘a’.
• The vertex associated with q0 is called the initial vertex, while those labeled with qf ∈ F are the
final vertices.
• The transition graph given below represents the DFA
M = ({q0, q1, q2}, {0, 1}, δ, q0, {q1})
where δ is given by δ(q0, 0) = q0, δ(q0, 1) = q1, δ(q1, 0) = q0, δ(q1, 1) = q2,
δ(q2 , 0) = q2 , δ(q2 , 1) = q1
• This DFA accepts the input string 01. The DFA does not accept the string 00, since, after reading
two consecutive 0’s, it will be in state q0.
• By similar reasoning, we see that the automaton will accept the strings 101, 0111, and 11001, but
not 100 or 1100.

Automata and Complexity Theory 36

DFA …
• It is convenient to introduce the extended transition function δ* : Qx∑* → Q
• The second argument of δ* (i.e. ∑*) is a string, rather than a single symbol, and its value
gives the state the automaton will be in after reading that string.
• For example, if δ(q0,a)=q1, and δ( q1, b)=q2 then δ*(q0,ab) = q2
• Formally, we can define δ* recursively by,
δ *(q, ε)=q Equ. (2.1)
δ *(q, wa)= δ (δ *(q, w),a) Equ (2.2)
for all q ∈ Q, w ∈ Σ*, a ∈ Σ.
• To see why this is appropriate, let us apply these definitions to the simple case above
δ *(q0, ab)= δ (δ *(q0, a),b) Equ. (2.3)
But,
δ *(q0, a)= δ (δ *(q0, ε),a)= δ(q0, a)
δ*(q0, a) =q1 Equ.(2.4)
• Now, substituting Equ (2.4) in EquAutomata
(2.3),andwe get, δ *(q0, ab)=δ (q1, b)=q2 as expected.
Complexity Theory 37
Graphical Representation of a DFA
• A DFA is represented by digraphs called state diagram.
• The vertices represent the states.
• The arcs labeled with an input alphabet show the transitions.
• The initial state is denoted by an empty single incoming arc.
• The final state is indicated by double circles.
• Example 1: Let a deterministic finite automaton be →
Q = {a, b, c},
∑ = {0, 1},
q0 = {a},
F = {c}, and

Automata and Complexity Theory 38

 DFA
• Transition function δ as shown by the following table -
Transition Functions:- (δ(a,0)= δ(a,1,R)=>b, δ(a,1)= δ(b,1,R)=>b, δ(b,0)= δ(b,0,R)=>c etc) −
States State 0 1
Inputs

a a b • Each particular
input have only
b c a one state
• Can’t move via
c b c

• Its graphical representation would be as follows −

Automata and Complexity Theory 39

Languages and DFA
• Having made a precise definition of an automaton, we are now ready to define formally what we
mean by an associated language.
• The association is obvious: the language is the set of all strings recognized by the automaton.
• The language accepted by a DFA M = (Q, Σ, δ, q0, F) is the set of all strings on Σ accepted by M.
• In formal notation, L(M) = {w ∈ Σ* : δ*(q0, w) ∈ F}
• A DFA will process every string in Σ* and either accept it or not accept it.
• Non acceptance means that the DFA stops in a non-final state, so that, L(M) = {w ∈ Σ* : δ*(q0, w) ∉
F}
• Example: Consider the DFA shown in the figure below:

Automata and Complexity Theory 40

Languages and DFA …
Can a machine(i.e., algorithm) recognize a regular language? Yes!
Deterministic Finite Automata
A deterministic finite automaton (DFA) over an alphabet A is a finite digraph (where vertices or nodes are called states) for
which each state emits one labeled edge for each letter of A. One state is designated as the start state and a set of states may
be final states.
Example 2: Either of the following alternatives is acceptable for representing a DFA, where final states are indicated by
double circles.

a b b
1 a, b 1
Start 0 b a Start 0 a
a 2 b 2 a, b

The Execution of DFA for input string w  A* begins at the start state and follows a path whose edges concatenate to w.
The DFA accepts w if the path ends in a final state. Otherwise the DFA rejects w. The language of a DFA is the set of
accepted strings.
Example. The example DFA accepts the strings
a, b, ab, bb, abb, bbb, …, abn, bbn, ….
So the language of the DFA is given by the regular expression (a + b)b*.
Automata and Complexity Theory 41
Languages and DFA …
• In the above figure, two labels were allowed on a single edge. Such multiple
labeled edges are short hand for two or more distinct transitions: the transition
is taken whenever the input symbol matches any of the edge labels.
• For example, the automaton in the above figure remains in its initial state q 0
until the first ‘b’ is encountered.
• If this is also the last symbol of the input, then the string is accepted.
• If not, the DFA goes into state q2, from which it can never escape. Such a state
is called a trap state.
• We can see clearly from the graph that the automaton accepts all strings which
have exactly one b, located at the rightmost position of the string.
• All other input strings are rejected. In set notation, the language accepted by
this automaton is
L = {an b| n ≥ 0}
Automata and Complexity Theory 42
Theorem
• Let M = (Q, Σ, δ, q0, F) be a deterministic finite acceptor, and let Gm be its
associated transition graph. Then for every q i, qj ∈ Q, and w ∈ Σ+, δ*(qi,w)=qj
if and only if there is in Gm a walk with label w from q i to qj.
qi
a
qo v
b

qj

Automata and Complexity Theory 43

Proof of the Theorem
• It can be proved rigorously using an induction on the length of w.
• Assume that the claim is true for all strings v with |v|≤n.
• Consider then any string w of length n+1 and represent it as w = va.
• Suppose now that δ*(qi,v)=qk. Since, |v| = n, there must be a walk in Gm labeled v
from qi to qk.
• But if δ*(qi,w)=qj, then M must have a transition δ(qk,a)=qj, so that by construction
Gm has an edge (qk,qj) with label ‘a’.
• Thus, there is a walk in Gm labeled va= w between qi and qj.
• Since, the result is obviously true for n = 1, we can claim by induction that, for
every w ∈ Σ+, δ*(qi,w)=qj implies that there is a walk in Gm from qi to qj labeled w.
• The argument can be turned around in a straight forward way to show that the
existence of such a path implies the, thus completing the proof.
Automata and Complexity Theory 44
DFA
• Example 3: Find a Deterministic Finite Acceptor that recognizes the set of all strings on
Σ={a,b} starting with the string ab.
• The only issue here is the first two symbols in the string; after they have been read no
further decisions need to be made.
• We can therefore solve the problem with an automaton that has four states; an initial
state, two states for recognizing ‘ab’ ending in a final trap state, and one non final trap
state.
• If the first symbol is an ‘a’ and the second is a ‘b’, the automaton goes to the final trap
state, where it will stay since the rest of the input does not matter.
• On the other hand, if the first symbol is not an ‘a’ or the second one is not a ‘b’, the
automaton enters the non final trap state.

45
Automata and Complexity Theory
Nondeterministic Finite Automata (NFA)
• Non-determinism means a choice of moves for an automaton.
• Rather than prescribing a unique move in each situation, we allow a set of possible moves.
• Formally, we achieve this by defining the transition function so that its range is a set of
possible states.
• A Nondeterministic Finite Automata or NFA is defined by the quintuple.
M = (Q, Σ, δ, q0, F)
where,
Q is a finite set of internal states
Σ is a finite set of symbols called the input alphabet
q0 ∈ Q is the initial state
F ⊆ Q is a set of final states
but,
δ: Q X (Σ ∪ {ε}) → 2Q
Automata and Complexity Theory 46
Graphical Representation of an NFA
• An NDFA is represented by digraphs called state diagram.
• The vertices represent the states.
• The arcs labeled with an input alphabet show the transitions.
• The initial state is denoted by an empty single incoming arc.
• The final state is indicated by double circles.
• Example 1: Let a non-deterministic finite automaton be →
Q = {a, b, c}
∑ = {0, 1}
q0 = {a}
F = {c}

Automata and Complexity Theory 47

 NFA … Example
A. Transition Table: The transition function δ as shown below − Inputs

States State 0 1
-At each state there are many next
states and many outputs(paths) at
a {a, b} {b} same time.

b {c} {a, c}

c {b, c} {c}

B. State Diagram: Its graphical representation would be as follows −

Automata and Complexity Theory 48

NFA … Example …
• Example 2: one or more 0’s followed by one or more 1’s
L={01n ,n>=0} A. State Diagram
0 1 0/1
Q = {q0, q1, q2}
Σ = {0, 1} 0 1
q0 q1 q2
Start state is q0
F = {q2}
B. Transition Table
T. function Inputs
δ: 0 1
q0
{q0, q1} {}

{} {q1, q2}
q1
{q2} {q2}
q2

49
Automata and Complexity Theory
NFA … Example …
• Example 3: The following NFA recognizes the language of a + aa*b + a*b.
A. State Diagram 1 a
L
a b
a
Start 0 2

B. Transition Table representation of NFA

T a b 
start 0 {1, 2}  {1}
1 {1} {2} 
final 2   



Automata and Complexity Theory 50

NFA … Example …
• Example 4: Pair of 0’s or pair of 1’s as substring
Q = {q0, q1, q2 , q3 , q4} 0/1 A. State Diagram 0/1
Σ = {0, 1} 0 0 q4
Start state is q0 q0 q3
F = {q2, q4} 1 0/1
B. Transition Table
δ: 0 1
1
q1 q2
q0 {q0, q3} {q0, q1}

{} {q2}
q1
{q2} {q2}
q2
{q4} {}
q3
{q4} {q4}

q4
Automata and Complexity Theory 51
Differences between NFA and DFA
• Note that there are two major differences between the above NFA definition
and the definition of a DFA.
1. In a NFA, the range of δ is the power set 2Q, so that its value is not a single
element of Q, but a subset of it. This subset defines the set of all possible
states that can be reached by the transition. Example, δ(q1, a) = {q0, q2}.
2. Also we allow ε (null) as the second argument of δ. This means that the
NFA can make a transition without consuming an input symbol.

Automata and Complexity Theory 52

NFA Vs. DFA
• Like DFAs, NFA can also be represented by transition graphs.
• The vertices are determined by Q, while an edge (qi, qj) with label ‘a’ is in the graph if
and only if δ(qi, a) contains qj.
• Note that since ‘a’ may be the empty string, there can be some edges labeled ε.
• A string is accepted by an NFA if there is any sequence of possible moves that will put
the machine in a final state at the end of the string.
• Non-determinism can therefore be viewed as involving “intuitive” insight by which
the best move can be chosen at every stage (assuming that the NFA wants to accept
every string).
• For example consider the transition graph shown in the figure below. It describes a
nondeterministic automaton since there are two transitions labeled ‘a’ out of q0.

Automata and Complexity Theory 53

NFA Vs. DFA …
• Another nondeterministic automaton is shown in the figure below.

• It is nondeterministic not only because several edges with the same label
originate from one vertex, but also it has ε - transition.

Automata and Complexity Theory 54

NFA Vs. DFA …
• Again, the transition function can be extended so its second argument is a string.
• We require of the extended transition function δ* that if δ* (q i, w) = Qj then Qj is
the set of states the automaton may be in, having started in state q all possible
states and having read ‘w’.
• For an NFA, the extended transition function is defined so that δ*(q i, w) contains
qj if and only if there is a walk in the transition graph from q i to qj labeled w.
• This holds for all qi, qj ∈ Q and w ∈ Σ*
• The language L accepted by an NFA M = (Q, Σ, δ, q0, F) is defined as the set of
all strings on Σ accepted by M.
• Formally, L(M) = {w ∈ Σ* : δ*(q0, w) ∩ F ≠ ∅}.
• In words, the language contains all strings ‘w’ for which there is a walk labeled
w from the initial vertex of the transition graph to some final vertex. Where, ∅
denotes the empty set. Automata and Complexity Theory 55
DFA and NFA
Deterministic Finite Automata Non-Deterministic Finite Automata
Each transition leads to exactly one state called as A transition leads to a subset of states i.e. some transitions
deterministic can be non-deterministic.
Accepts input if the last state is in Final Accepts input if one of the last states is in Final.
Backtracking is allowed in DFA. Backtracking is not always possible.
Requires more space. Requires less space.
Empty string transitions are not seen in DFA. Permits empty string transition.
For a given state, on a given input we reach a deterministic For a given state, on a given input we reach more than one
and unique state. state.
DFA is a subset of NFA. Need to convert NFA to DFA in the design of a compiler.
δ:Q×Σ→Q δ : Q × Σ → 2Q
For example − δ(q0,a)={q1} For example − δ(q0,a)={q1,q2}

DFA is understood as one machine. NFA is understood as multiple small machines computing at
the same time.

DFA is more difficult to construct. NFA is easier to construct.

Automata and Complexity Theory 56

Equivalence of NFA and DFA
• Starting now to discuss the fundamental concept – In what sense are DFAs and
NFAs different? Is there any other difference other than the definition?
• To explore this question, we introduce the notion of equivalence between
automata.
• We say that two automata are equivalent if they accept the same language.
• For any given language, we can usually find an unlimited number of equivalent
automata – deterministic or not.
• The DFA shown in Fig.(a) is equivalent to the NFA shown in Fig.(b) since, they
both accept the language {(10)n/n≥0}.

Automata and Complexity Theory 57

Equivalence of NFA and DFA …
• When we compare different classes of automata, the question invariably arises
whether one class is more powerful than the other.
• By more powerful we mean that tan automaton of one kind can achieve
something that cannot be done by any automaton of the other kind.
• Since, a DFA is a restricted kind of NFA, it is clear that any language that is
accepted by a DFA is also accepted by some NFA.
• But the converse is not so obvious.
• This means that we can actually give a way of converting any NFA into an
equivalent DFA.
• The rationale for the construction is the following. After an NFA has read a
string ‘w’, we may not know exactly what state it will be in, but we can say that
it must be in one state of a set of possible states, say {q i, qj, ……, qk}.
• An equivalent DFA after reading the same string must be in some definite state.
Automata and Complexity Theory 58
Equivalence of NFA and DFA …
• How can we make these two situations correspond?
• The solution is to label the states in such a way that, after reading w, the
equivalent DFA will be in a single state labeled {qi, qj, ……, qk}.
• Since for a set of |Q| states there are exactly 2 |Q|subsets, the corresponding
DFA will have a finite number of states.
• We now present an algorithm for constructing from an NFA a DFA that
recognizes the same language.
• This algorithm is often called as “subset construction”, is useful for
simulating an NFA by a computer program.

Automata and Complexity Theory 59

Acceptability by DFA and NDFA
• A string is accepted by a DFA/NDFA iff the DFA/NDFA starting at the initial
state ends in an accepting state (any of the final states) after reading the
string wholly.
• A string S is accepted by a DFA/NDFA (Q, ∑, δ, q0, F), iff
δ*(q0, S) ∈ F
• The language L accepted by DFA/NDFA is
{S | S ∈ ∑* and δ*(q0, S) ∈ F}
• A string S′ is not accepted by a DFA/NDFA (Q, ∑, δ, q0, F), iff
δ*(q0, S′) ∉ F
• The language L′ not accepted by DFA/NDFA (Complement of accepted
language L) is
{S | S ∈ ∑* and δ*(q0, S) ∉ F}

Automata and Complexity Theory 60

Acceptability by DFA and NFA …
• Example: Let us consider the DFA shown in Figure below from the DFA, the
acceptable strings can be derived.

• Strings accepted by the above DFA: {0, 00, 11, 010, 101, ...........}
• Strings not accepted by the above DFA: {1, 011, 111, ........}

Automata and Complexity Theory 61

Regular Languages
• Every Finite automaton accepts some language.
• If we consider all possible finite automata, we get a set of languages associated
with them.
• We will call such a set of languages a family.
• The family of languages that is accepted by deterministic finite automata is
quite limited.
• A language L is called regular if and only if there exists some deterministic
finite accepter M such that
L = L(M)
• Precisely, a language can be claimed as regular if a DFA can be constructed for
it.

Automata and Complexity Theory 62

Regular Languages …
• For example: Show that the language L={awa|w ∈ {a,b}*} is regular.

Automata and Complexity Theory 63

Conversion NFA to DFA
• Let X = (Qx, ∑, δx, q0, Fx) be an NFA which accepts the language L(X). We have to design an
equivalent DFA Y = (Qy, ∑, δy, q0, Fy) such that L(Y) = L(X).
• The following procedure converts the NDFA to its equivalent DFA:
Algorithm
Input − An NDFA
Output − An equivalent DFA
Step 1 − Create state table from the given NDFA.
Step 2 − Create a blank state table under possible input alphabets for the equivalent DFA.
Step 3 − Mark the start state of the DFA by q0 (Same as the NDFA).
Step 4 − Find out the combination of States {Q0, Q1,... , Qn} for each possible input alphabet.
Step 5 − Each time we generate a new DFA state under the input alphabet columns, we have
to apply step 4 again, otherwise go to step 6.
Step 6 − The states which contain any of the final states of the NDFA are the final states of the
equivalent DFA.
Automata and Complexity Theory 64
Conversion NFA to DFA
• Example 1: Let us consider the NDFA shown in the figure below.

q δ(q,0) δ(q,1)
a {a,b,c,d,e} {d,e}
b {c} {e}
c ∅ {b}
d {e} ∅
e ∅ ∅
Automata and Complexity Theory 65
Conversion NFA to DFA
• Using the above algorithm, we find its equivalent DFA. The state table of the DFA is shown in
below.

q δ(q,0) δ(q,1)
[a] [a,b,c,d,e] [d,e]
[a,b,c,d,e] [a,b,c,d,e] [b,d,e]
[d,e] [e] ∅
[b,d,e] [c,e] [e]
[e] ∅ ∅
[c, e] ∅ [b]
[b] [c] [e]
[c] ∅ [b]

Automata and Complexity Theory 66

Conversion NFA to DFA
• The state diagram of the DFA is as follows −

Automata and Complexity Theory 67

Automata and Complexity Theory 68