CSC312

Automata Theory

Lecture # 1

Introduction
 Administrative Stuff
• Instructor: Muhammad Usman Akram
[email protected]
Room # C-11
WebLink:
http://usmanlive.com
•Office Hrs: Mon-Thu 11:30 – 13:00 hrs
(or by appointment)
•Prerequisite: CSC102 - Discrete Structures
 Course Objectives:
This course is designed to enable the students to
study and mathematically model various
abstract computing machines that serve as
models for computations and examine the
relationship between these automata and formal
languages.
 Course Outline
Regular expressions, TG, DFA and NFAs. Core
concepts of Regular Languages and Finite
Automata; Moore and Mealy Machines,
Decidability for Regular Languages; Non-
regular Languages; Transducers (automata
with output). Context-free Grammar and
Languages; Decidability for Context-free
Languages; Non-context-free Languages;
Pushdown Automata, If time allows we will also
have a short introduction to Turing machines.
 Course Organization
Text Book:
i) Denial I. A. Cohen Introduction to Computer
Theory, Second Edition, John Wiley & Sons.
Reference Books:
i) J. E. Hopcroft, R. Motwani, & J. D. Ullman
Introduction to Automata Theory,
Languages, and Computation, Third Edition,
Pearson, 2008.
Instruments:
There will be 3~4 assignments, 3~4 quizzes,
Weights: Assignments 10%
Quizzes 15%
S-I 10%
S-II 15%
Final Exam 50%
 Schedule of Lectures
Lect.# Topics/Contents
1 Introduction to Automata theory, Its background, Mathematical Preliminaries,
Sets, Functions, Graphs, Proof Techniques
2 Formal Languages, Introduction to defining languages, alphabet, language,
word, null string, length of a string, reverse of a string, Palindrome, Kleene
closure.
3 Formal definition of Regular Expressions, Defining languages with regular
expressions, Languages associated with regular expressions.
4 Equality of Regular Expressions, Introducing the language EVEN-EVEN.
5 More examples related to regular expressions.
6 Introducing Finite Automata., Defining languages using Finite Automata.
Constructing Finite Automata for different languages.
7 Recognizing the language defined by the given Finite Automata.
8 More examples related to Finite Automata.
9 Transition Graphs with examples, Generalized Transition Graphs, Non-
determinism in case of Transition Graphs.
10 Non-deterministic FA’s. Differences between FA, TG and NFA.
11 Sessional I
 Schedule of Lectures
Lect.# Topics/Contents
12 Discussion on the solution of S-I
13 Kleene’s Theorem, Algorithm for turning TGs into REs
14 Kleene’s Theorem, Algorithm for turning REs into FAs
15 Different Rules for turning Res into FAs
16 Nondeterminism, NFA, converting NFA into FA. Union of two FAs using
NFA.
17 Finite Automata with output, Moore’s machines and Mealy machines with
examples. 1’s Complement machine, Increment machine.
18 Theorems for Converting Moore machines into Mealy machines and vice
versa.
19 Transducers as models of sequential circuits.
20 Regular Languages, Closure properties (i.e. , Concatenation and Kleene
closure) of Regular Languages with examples.
21 Complements and Intersections of Regular Languages, Theorems relating to
regular languages and the related examples.
22 Non-Regular Languages, The pumping Lemma, Examples relating to
Pumping Lemma.
 Schedule of Lectures
23 Sessional-II
24 Decidability, decision procedure, Blue-paint method, Effective decision
procedure to prove whether two given RE’s or FA’s are equivalent. Myhill-
Nerode theorem, Related Examples.
25 Context-Free Grammars, CFG’s for Regular Languages with examples.
CFG’s for non-regular languages.
26 CFG’s of PALINDROME, EQUAL and EVEN-EVEN languages, Backus-
Naur Form.
27 Parse Trees, Examples relating to Parse Trees, Lukasiewicz notation, Prefix
and Postfix notations and their evaluation.
28 Ambiguous and Unambiguous CFG’s, Total language tree. Regular
Grammars, Semi-word, Word, Working String, Converting FA’s into
CFG’s.
29 Regular Grammars, Constructing Transition Graphs from Regular
Grammars. Killing null productions. Killing unit productions,
30 Chomsky Normal form with examples, Left most derivations.
Pushdown Automata, Constructing PDA’s for FA’s, Pushdown stack.
 Some basics
• Automaton = A self-operating machine or
mechanism (Dictionary definition), plural is
Automata.
• Automata = abstract computing devices
• Automata theory = the study of abstract
machines (or more appropriately, abstract
'mathematical' machines or systems), and
the computational problems that can be
solved using these machines.
• Mathematical models of computation
• Finite automata
• Push-down automata
• Turing machines
 History
• 1930s : Alan Turing defined machines more
powerful than any in existence, or even any that
we could imagine – Goal was to establish the
boundary between what was and was not
computable.
• 1940s/1950s : In an attempt to model “Brain
function” researchers defined finite state
machines.
• Late 1950s : Linguist Noam Chomsky began the
study of Formal Grammars.
• 1960s : A convergence of all this into a formal
theory of computer science, with very deep
philosophical implications as well as practical
applications (compilers, web searching,
Computation

CPU memory

11
temporary memory

input memory
CPU
output memory

Program memory

12
 3
Example: f ( x ) x

temporary memory

input memory
CPU
output memory
Program memory
compute xx
2
compute x x
13
 3
f ( x) x

temporary memory
input memory
x 2
CPU
output memory
Program memory
compute xx
2
compute x x
14
 3
temporary memory f ( x) x
z 2 * 2 4
f ( x) z * 2 8
input memory
x 2
CPU
output memory
Program memory
compute xx
2
compute x x
15
 3
temporary memory f ( x) x
z 2 * 2 4
f ( x) z * 2 8
input memory
x 2
CPU
f ( x) 8
Program memory output memory
compute xx
2
compute x x
16
 Automaton
temporary memory

Automaton
input memory
CPU
output memory

Program memory

17
 Different Kinds of Automata
Automata are distinguished by the temporary memo

• Finite Automata: no temporary memory

• Pushdown Automata: stack

• Turing Machines: random access memory

18
 Finite Automaton

temporary memory

input memory
Finite
Automaton
output memory

Example: Automatic Door, Vending Machines

(small computing power)
19
 Pushdown Automaton
Stack Push, Pop

input memory
Pushdown
Automaton
output memory

Example: Compilers for Programming Languages

(medium computing power)
20
 Turing Machine

Random Access
Memory

input memory
Turing
Machine
output memory

Examples: Any Algorithm

(highest computing power)
21
 Power of Automata

Finite Pushdown Turing

Automata Automata Machine

Less power More power

Solve more
computational problems
22
 Mathematical Preliminaries

• Sets
• Functions
• Relations
• Graphs
• Proof Techniques

23
 SETS
A set is a collection of elements
A {1, 2, 3}
B {train, bus, bicycle, airplane}

We write
1 A
ship  B
24
 Set Representations

C = { a, b, c, d, e, f, g, h, i, j, k }

C = { a, b, …, k } finite set

S = { 2, 4, 6, … } infinite set

S = { j : j > 0, and j = 2k for some

k>0 }

S = { j : j is nonnegative and even }

25
 A = { 1, 2, 3, 4, 5 }
U
6 A
2 3 8
1
7 4 5
9
10

Universal Set: all possible elements

U = { 1 , … , 10 }
26
 Set Operations
A = { 1, 2, 3 } B = { 2, 3, 4, 5}
A B
• Union
2 4
1
3
A U B = { 1, 2, 3, 4, 5 } 5

• Intersection
U 2
A B = { 2, 3 } 3

• Difference
1
A-B={1}

B - A = { 4, 5 } Venn diagrams 5
27
• Complement
Universal set = {1, …, 7}
A = { 1, 2, 3 } A = { 4, 5, 6, 7}

4
A
A 3 6
1
2
5 7

A=A
28
 { even integers } = { odd
integers }

Integers

1 odd
even 5
2 6
0
4
3 7

29
DeMorgan’s Laws

U
AUB=A B

U
A B=AUB

30
 Empty, Null Set:
={}

SU =S
U
S = = Universal Set

S- =S

-S=

31
 Subset
A = { 1, 2, 3} B = { 1, 2, 3, 4, 5 }
A B

U
Proper Subset: A B

U
B
A

32
 Disjoint Sets
A = { 1, 2, 3 } B = { 5, 6}

U
A B=

A B

33
 Set Cardinality
• For finite sets
A = { 2, 5, 7 }

|A| = 3

(set size)

34
 Powersets
A powerset is a set of sets

S = { a, b, c }

Powerset of S = the set of all the subsets of S

={ , {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c

Observation: | 2S | = 2|S| ( 8 = 23 )

35
 Cartesian Product
A = { 2, 4 } B = { 2, 3, 5 }

A X B = { (2, 2), (2, 3), (2, 5), ( 4, 2), (4, 3), (4, 5) }

|A X B| = |A| |B|

Generalizes to more than two sets

AXBX…XZ
36
 FUNCTIONS
domain range
4 A B
f(1) = a a
1
2 b
3 c
5
f : A -> B
If A = domain
then f is a total function
otherwise f is a partial function
37
 RELATIONS
Let A & B be sets. A binary relation “R” from A to B
R = {(x1, y1), (x2, y2), (x3, y3), …}
Where xi  Aand yi  B
R⊆AxB

xi R yi to denote ( a, b)  R

e. g. if R = ‘>’: 2 > 1, 3 > 2, 3 > 1

38
 GRAPHS
A directed graph
e
b
node
a d
ed g e c
Nodes (Vertices)
V = { a, b, c, d, e }
Edges
E = { (a,b), (b,c), (b,e),(c,a), (c,e), (d,c), (e,b), (e,d)
39
 Labeled Graph
2
6 e
b 2
1 3
a 6 d
5
c

40
 Walk

e
b
a d

Walk is a sequence of adjacent edges

(e, d), (d, c), (c, a)

41
 Path

e
b
a d

Path is a walk where no edge is repeated

Simple path: no node is repeated

42
 Cycle

base e
b
3
a 1 d
2
c

Cycle: a walk from a node (base) to itself

Simple cycle: only the base node is repeated

43
 Euler Tour
8 base
7 e
b 1
4 6
a 5 2 d
3
c

A cycle that contains each edge once

44
 Hamiltonian Cycle
5 base
e
b 1
4
a 2 d
3
c

A simple cycle that contains all nodes

45
 Trees
root

parent

leaf

child

Trees have no cycles

46
 root
Level 0

Level 1
leaf Height 3

Level 2

Level 3

47
Binary Trees

48