CSE2002 Theory of Computation and Compiler Design

Module:1
Introduction To Languages and Grammars

Overview of a Computational Model

Languages and grammars
Alphabets
Strings
Operations on languages,
Introduction to Compilers
Analysis of the Source Program
Phases of a Compiler
Sivasankari.S
 Goals and Objectives

-This course is a theoretical study of computing devices.

- Some of the questions we ask and attempt to answer, which
are the following.
-Are there problems that cannot be solved on any computing
device?
-How does one determine if a given problem can or cannot
be computationally solved?
- If we place bounds on the resources (time and space)
available to a computer, then what can be said about which
problems can and which problems cannot be solved on a
computer?
-How does the power of a computer change, if it has access
to random bits? - 2
 The answers of previous questions are given in
the following topics

-Computation models: Finite State Automata, Turing

machines (TM).
-Regular Expressions and Regular Grammars.
-Context-free Grammars and Pushdown Automata.
--Computability Theory and reducibility
-Computational complexity
-Determinism and Non-determinism
-Time and Space hierarchy and
-NP completeness

3
What is computation?

Execution of algorithms (or)

Step by step procedure that produce a result from
given input.(or)
Computation is simply a sequence of steps that can be
performed by computer.
What is Theory?

A theory not only explains known facts; it also allows

scientists to make predictions of what they should
observe if a theory is true.
 Why is it necessary to study Theory of
computation?

understand the way an algorithm is meant to work, and to

actually prove it works through analyzing problems that may
arise with the technique used and finding solutions to these
problems

5
 Purpose of Theory of Computation

 Develop formal mathematical models of computation that

reflect real-world computers.
Models of computation can be classified in three
categories:
Sequential models
Functional models
Concurrent models.
Sequential models include:
FSM,PDA,Random Access Machine,TM
Functional models include:
Lambda Calculus
General Recursive functions
Combinatory logic
Concurrent models include:
Petri nets
DFG
Interaction nets
Actor model

7
 Brief History of Theory of Computation

 At beginning of 19th century, mathematicians

believe:
Axiomatization of mathematics
 That is,
it is possible to decide the truth of any
mathematical statement thru use of
inference rules and axiom systems.

8
 Brief History of Theory of Computation

 In 1931, Kurt Gödel proposed the

incompleteness theorem which says:
the truth of certain mathematical statements
cannot be proved by the use of
mathematical axioms.
 That is,
some mathematical problems cannot be
solved by algorithms.

9
 Brief History of Theory of Computation

 From then, the ideal of axiomatization of

mathematics was broken.
 Then, mathematicians started to
 think the scope of algorithm capabilities,
 design theoretical computational
machines, &
 study their capability limitations. 10
 Brief History of Theory of Computation
-In 1936, Turing proposed the model of universal
algorithm machine,which later was called Turing
machine.
-Some similar ideas were proposed by other scholars,
including Stephen C. Kleene,
Alonzo Church,
Emil Post, etc., in different forms of models.
-In 1943, neural physiologists
Warren S. McCulloch and
Walter Pitts developed finite-state systems to
simulate neural networks in biological systems.
-They are pioneers of automata theory. 11
 Brief History of Theory of Computation

 In late 1940’s ， Von Neumann proposed the idea of

stored program for computer models.
 In 1951, a real computer following the idea of “stored
program” was constructed:
UNIVAC I, the world's first commercially
available computer (manufactured by Eckert-Mauchly
Computer Company)
 In late 1950’s, linguist N. Chomsky proposed a
mathematical model for grammars of natural languages.
12
Brief History of Theory of Computation
 In 1956, Kleene proposed the concept of finite automaton
for simulating neural networks proposed by McCulloch
and Pitts.
 He also proposed regular expressions to describe strings
of symbols, and proved them equivalent to finite
automata.
 In 1959 and 1960, John W. Backus & Peter Naur proposed
sequentially a special expression for computer language
grammars, called later Backus-Naur form (BNF) .
 It was used to describe computer language ALGOL-60,
leading to development of compiling theory.
13
Brief History of Theory of Computation
 In 1969, Stephen A. Cook found that computation
problems may be separated into tractable and intractable
ones.
 Intractable problems are also called NP-hard problems.

 Such problems cannot be solved by computers except

very small instances (with only small-sized inputs).

14
 Field Related to TOC
Web search: theory of pattern matching.
Sequential circuits: theory of finite state automata.
Compilers: theory of context free grammars.
Cryptography: theory of computational complexity.
Neural networks : Automata theory
Computational biology: DNA and proteins are strings.
Artificial intelligence: the undecidability of first-order logic.(FL and
Automata)
Natural languages are mostly context-free. Speech understanding
systems use probabilistic FSMs
Algorithm Analysis:Computational complexity
What is Theory of Computation ?

Branch that deals with how efficiently problems can be

solved on a model of computation, using an algorithm.
Theory of Computation Classified into
 Automata Theory
 Computability Theory
 Computational Complexity Theory

16
 Theory of Computation

Complexity Computability Automata

Theory Theory Theory

Solvable FA
Easy

Hard UnSolvable PDA

TM

17
 Computability Theory
Study of problem solving capabilities of computational
models.
Problem types based on resources
-Impossible problems
-Possible-with-unlimited-resources-but- impossible-with-
limited-resources problems
-Possible-with-limited-resources problems
Problem types based on time
-Undecidable problems
-Intractable problems
-Tractable problems
Advantage -computability help us not to waste time on
“unsolvable problems” 18
 Computational complexity Theory

-Study of “efficiency” of problem solving.

- The main question asked in this area is “What makes some 
problems computationally hard and other problems easy?”

A problem is called “easy”, if it is efficiently solvable.
Examples of “easy” problems are  searching for a name in a 
telephone  directory.

- A problem is called “hard”, if it cannot be solved efficiently, or if 
we don’t know whether it can be solved efficiently.
Examples of “hard” problems are   (i) factoring a 300-digit integer 
into its prime factors. 19
 What is Automata Theory?
Study of dynamic behavior of discrete parameter
information systems in the form of abstract computing
devices, or “machines”

General Concepts Automata Theory

Examples of discrete-parameter information systems
digital systems,
nerve systems,
languages information transmission ,
human-environment interactions, …

20
 temporary memory
f(x)= x 3
z2*24
f (x)  z * 2 
8 input
x2
CPU
output
Program memory
compute xx
compute 2
x  21 2
 Automaton [Representation]
temporary memory

Automaton
input

output
transition

state

CPU + ProgramMem = States + Transitions

22
 Different Kinds of Automata

Automata are distinguished by the

temporary memory
• Finite Automata: no temporary memory

• Pushdown Automata: stack

• Turing Machines: random access memory

Memory affects computational power:
More flexible memory -results to

The solution of more computational

problems
23
 Finite Automaton

temporary memory

input
Finite
Automaton
output
Example: Elevators, Vending Machines,
Lexical Analyzers
(small computing power)
24
 Pushdown Automaton
Temp.
memory Stack Push, Pop

Pushdown input

Automaton
output

Example: Parsers for Programming Languages

(medium computing power)
25
 Turing Machine

Temp.
memory Random Access Memory

input
Turing
Machine
output

Examples: Any Algorithm

(highest known computing power)
 Power of Automata
Simple More complex Hardest
problems problems problems

Finite Pushdown Turing

Automata Machine
Automata

Less power More power

Solve more
computatio
nal problems 27
(A pioneer of automata theory)

Alan Turing (1912-1954)

 Father of Modern Computer
Science
 English mathematician
 Studied abstract machines called
Turing machines

28
29
30