By

Dr.Hamed Alrjoub

THEORY OF AUTOMATA
MATH'S- OVERVIEW
 Theory Of Automata - Maths

TEXT AND REFERENCE MATERIAL

1. Introduction to Computer Theory, by Daniel

I. Cohen, John Wiley and Sons, Inc., 1991,
Second Edition

2. Introduction to Languages and Theory of

Computation, by J. C. Martin, McGraw Hill
Book Co., 1997, Second Edition

2
 Theory Of Automata - Maths

BACKGROUND
 We shall form several mathematical models that will
describe with varying degrees of accuracy parts of
computers, types of computers, and similar machines.

 In particular, the way we shall be studying about computers

is to build mathematical models, called machines, and then
to study their limitations by analyzing the types of inputs on
which they can operate successfully.

 The collection of these successful inputs is called the

language of the machine.

3
 Theory Of Automata - Maths

 Every time we introduce a new machine, we will learn its

language; and every time we develop a new language, we will
try to find a machine that corresponds to it.

 This interplay between languages and machines will be our

way of investigating problems and their potential solutions by
automatic procedures, which we call algorithms.

 We will study different types of theoretical machines that are

mathematical models for actual physical processes. By
considering the possible inputs on which these machines can
work, we can analyze their various strengths and weaknesses.

4
 Theory Of Automata - Maths

What is mathematical about the

models?
we are obliged to prove the truth about whatever we
discover.

 So consider only question of whether certain tasks can be

done at all. Conclusion are of the form “this can be done” or
“this can never be done”.

 When reach conclusions of the second type, we mean not

just that techniques for performing these tasks are
unknown at the present time, but that such techniques will
never exist in the future.
5
 Theory Of Automata - Maths

 We will arrive at what we may believe to be the most

powerful machine possible. When we do, we will be
surprised to find tasks that even such machine cannot
perform.

 Our ultimate result is that no matter what machine we build,

there will always be questions that are simple to state and
that the machine can not answer.

6
 Theory Of Automata - Maths

WHAT DOES AUTOMATA MEAN?

It is the plural of automaton, and it means

“something that works automatically”

7
 Theory Of Automata - Maths

LANGUAGES
 In English, we distinguish 3 different entities: letters, words,
and sentences.

 Groups of letters make up words and groups of words make

up sentences.

 However, not all collections of letters form valid words, and

not all collections of words form valid sentences.

 This situation also exists with computer languages.

 Certain (but not all) strings of characters are recognizable

words (e.g., IF, ELSE, FOR, WHILE …); and certain (but not all)
strings of words are recognizable commands.

8
 Theory Of Automata - Maths

 To construct a general theory of formal languages, we need

to have a definition of a language structure, in which the
decision of whether a given string of units constitutes a
valid larger unit is not a matter of guesswork, but is based
on explicitly stated rules.

 In this model, language will be considered as symbols with

formal rules, and not as expressions of ideas in the minds
of humans.

 The term “formal” emphasizes that it is the form of the

string of symbols that we are interested in, not the
meaning.

9
 Theory Of Automata - Maths

Basic Definitions
 A finite non-empty set of symbols (letters), is
called an
alphabet. It is denoted by Σ ( Greek letter
sigma).

Example:
Σ={a,b}
Σ={0,1} //important as this is the language
//which the computer understands.
Σ={i,j,k}

10
 Theory Of Automata - Maths

STRINGS

Concatenation of finite symbols from the

alphabet is called a string.

 Example:
If Σ= {a,b} then
a, abab, aaabb, ababababababababab

11
 Theory Of Automata - Maths

WORDS
 Words are strings belonging to some language.

Example:
If Σ= {x} then a language L can be defined as
L={xn : n=1,2,3,…..} or L={x,xx,xxx,….}
Here x,xx,… are the words of L

 All words are strings, but not all strings are words.

12
 Theory Of Automata - Maths

EMPTY STRING or NULL STRING

 We shall allow a string to have no letters. We call this empty
string or null string, and denote it by the symbol Λ.
 For all languages, the null word, if it is a word in the
language, is the word that has no letters. We also denote
the null word by Λ.

 Two words are considered the same if all their letters are
the same and in the same order.

 For clarity, we usually do not allow the symbol Λ to be part

of the alphabet of any language.

13
 Theory Of Automata - Maths

Discussion of null
 The language that has no words is denoted by the standard
symbol for null set, ø.

 It is not true that Λ is a word in the language ø since this

language has no words at all.

 If a certain language L does not contain the word Λ and we

wish to add it to L, we use the operation “+” to form L + {Λ}.
This language is not the same as L.

 However, the language L + ø is the same as L since no new

words have been added.

14
 Theory Of Automata - Maths

INTRODUCTION TO DEFINING LANGUAGES

 The rules for defining a language can be of two kinds:

 They can tell us how to test if a string of alphabet letters is a valid

word, or

 They can tell us how to construct all the words in the language by
some clear procedures.

15
 Theory Of Automata - Maths

DEFINING LANGUAGES

 Example: Consider this alphabet with only one letter

∑={x}

 We can define a language by saying that any nonempty

string of alphabet letters is a word
L1 = { x, xx, xxx, xxxx, … } or
L1 = { xn for n = 1, 2, 3, … }

Note that because of the way we have defined it, the language
L1 does not include the null word Λ.

16
 Theory Of Automata - Maths

EXAMPLE:
The language L of strings of odd length,
defined over Σ={a}, can be written as
L={a, aaa, aaaaa,…..}
 Example:
The language L of strings that does not start
with a, defined over Σ={a,b,c}, can be written as

L={b, c, ba, bb, bc, ca, cb, cc, …}

17
 Theory Of Automata - Maths

EXAMPLE:
The language L of strings of length 2, defined over
Σ={0,1,2}, can be written as
L={00, 01, 02,10, 11,12,20,21,22}

 Example:
The language L of strings ending in 0, defined over
Σ ={0,1}, can be written as
L={0,00,10,000,010,100,110,…}

18
 Theory Of Automata - Maths

EXAMPLE:
 The language EQUAL, of strings with number of
a’s equal to number of b’s, defined over
Σ={a,b}, can be written as
{Λ ,ab,aabb,abab,baba,abba,…}

 The language EVEN-EVEN, of strings with even

number of a’s and even number of b’s, defined
over Σ={a,b}, can be written as
{Λ, aa, bb, aaaa,aabb,abab, abba, baab, baba,
bbaa, bbbb,…}

19
 Theory Of Automata - Maths

CONCATENATION
 Let us define an operation, concatenation, in which two strings are
written down side by side to form a new longer string.
xxx concatenated with xx is the word xxxxx
xn concatenated with xm is the word xn+m

 For convenience, we may label a word in a given language by a new

symbol. For example,
xxx is called a, and xx is called b
 Then to denote the word formed by concatenating a and b, we can write
ab = xxxxx
 It is not true that when two words are concatenated, they produce
another word. For example, if the language is
L2 = {x, xxx, xxxxx, …} = {x2n+1 for n = 0, 1, 2, …}
then a = xxx and b = xxxxx are both words in L2, but their concatenation
ab = xxxxxxxx is not in L2

20
 Theory Of Automata - Maths

CONCATENATION MAKES NEW WORDS?

 Note that in this simple example, we have:
ab = ba
But in general, this relationship does NOT hold for all
languages (e.g., houseboat and boathouse are two different
words in English).
 Example: Consider another language by beginning with the
alphabet
∑ = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 }
Define the language
L3 = { any finite string of alphabet letters that does not start
with the letter zero }

21
 Theory Of Automata - Maths

 This language L3 looks like the set of positive integers:

L3 = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, … }
 If we want to define L3 so that it includes the string (word)
0, we could say
L3 = { any finite string of alphabet letters that, if it starts
with a 0, has no more letters after the first}

22
 Theory Of Automata - Maths

DEFINITION: LENGTH
 We define the function length of a string to be the number
of letters in the string.
 Example:
 If a = xxxx in the language L1, then length(a) = 4
 If c = 428 in the language L3, then length(c) = 3
 If d = 0 in the language L3, then length(d) = 1
 In any language that includes the null word Λ, then length(Λ) =
0

 For any word w in any language, if length(w) = 0 then w = Λ.

23
 Theory Of Automata - Maths

 Recall that the language L1 does not contain the null string
Λ. Let us define a language like L1 but that does contain Λ:
L4 = { Λ, x, xx, xxx, xxxx, … }
= { xn for n = 0, 1, 2, 3, … }
 Here we have defined that
x0 = Λ (NOT x0 = 1 as in algebra)
 In this way, xn always means the string of n alphabet letters
x’s.
 Remember that even Λ is a word in the language, it is not a
letter in the alphabet.

24
 Theory Of Automata - Maths

DEFINITION: REVERSE

 If a is a word in some language L, then reverse(a) is the

same string of letters spelled backward, even if this
backward string is not a word in L.
 Example:
 reverse(xxx) = xxx
 reverse(145) = 541
 Note that 140 is a word in L3, but reverse(140) = 041 is NOT a word
in L3

25
 Theory Of Automata - Maths

DEFINITION: PALINDROME

 Let us define a new language called Palindrome over the

alphabet
∑ = { a, b }
PALINDROME = { Λ, and all strings x such that reverse(x)
=x}
 If we want to list the elements in PALINDROME, we find
PALINDROME = { Λ, a, b, aa, bb, aaa, aba, bab, bbb, aaaa,
abba, … }

26
 Theory Of Automata - Maths

PALINDROME

 Sometimes two words in PALINDROME when concatenated

will produce a word in PALINDROME
 abba concatenated with abbaabba gives abbaabbaabba (in
PALINDROME)

 But more often, the concatenation is not a word in

PALINDROME

 aa concatenated with aba gives aaaba (NOT in PALINDROME)

 The language PALINDROME has interesting properties that

we shall examine later.

27
 Theory Of Automata - Maths

TASK

 Q) Prove that there are as many palindromes of

length 2n, defined over Σ = {a,b,c}, as there are
of length 2n-1, n = 1,2,3… . Determine the
number of palindromes of length 2n defined
over the same alphabet as well.

28
 Theory Of Automata - Maths

KLEENE CLOSURE

 Definition: Given an alphabet ∑, we define a language in which

any string of letters from ∑ is a word, even the null string Λ. We
call this language the closure of the alphabet ∑, and denote
this language by ∑*.

 Examples:
If ∑ = { x } then ∑* = { Λ, x, xx, xxx, … }
If ∑ = { 0, 1 } then ∑* = { Λ, 0, 1, 00, 01, 10, 11,
000, 001, … }
If ∑ = { a, b, c } then ∑* = { Λ, a, b, c, aa, ab, ac,
ba, bb, bc, ca, cb, cc, aaa, … }

29
 Theory Of Automata - Maths

LEXICOGRAPHIC ORDER

 Notice that we listed the words in a language in size order

(i.e., words of shortest length first), and then listed all the
words of the same length alphabetically.
 This ordering is called lexicographic order, which we will
usually follow.
 The star in the closure notation is known as the Kleene
star.
 We can think of the Kleene star as an operation that
makes, out of an alphabet, an infinite language (i.e.,
infinitely many words, each of finite length).

30
 Theory Of Automata - Maths

KLEENE CLOSURE

 Let us now generalize the use of the Kleene star oprator to

sets of words, not just sets of alphabet letters.

 Definition: If S is a set of words, then S* is the set of all

finite strings formed by concatenating words from S, where
any word may be used as often as we like, and where the
null string Λ is also included.

31
 Theory Of Automata - Maths

KLEENE CLOSURE

 Example: If S = { aa, b } then

S* = { Λ plus any word composed of factors of aa and b },
or
S* = { Λ plus any strings of a’s and b’s in which the a’s
occur in even clumps }, or
S* = { Λ, b, aa, bb, aab, baa, bbb, aaaa, aabb, baab, bbaa,
bbbb, aaaab, aabaa, aabbb, baaaa, baabb, bbaab, bbbaa,
bbbbb, … }
Note that the string aabaaab is not in S* because it has a
clump of a’s of length 3.

32
 Theory Of Automata - Maths

KLEENE CLOSURE

 Example: Let S = { a, ab }. Then

S* = { Λ plus any word composed of factors of a and ab }, or

S* = { Λ plus all strings of a’s and b’s except those that

start with b and those that contain a double b }, or
S* = { Λ, a, aa, ab, aaa, aab, aba, aaaa, aaab, abaa, abab,
aaaaa, aaaab, aaaba, aabaa, aabab, abaaa, abaab, ababa,
…}
 Note that for each word in S*, every b must have an a
immediately to its left, so the double b, that is bb, is not
possible; neither any string starting with b.

33
 Theory Of Automata - Maths

How to prove a certain word is in

the closure language S*
 We must show how it can be written as a concatenation of
words from the base set S.

 In the previous example, to show that abaab is in S*, we

can factor it as follows:
abaab = (ab)(a)(ab)
 These three factors are all in the set S, therefore their
concatenation is in S*.
 Note that the parentheses, ( ), are used for the sole purpose
of demarcating the ends of factors.

34
 Theory Of Automata - Maths

 Observe that if the alphabet has no letters, then its closure

is the language with the null string as its only word; that is
if ∑ = ø (the empty set), then ∑* = { Λ }
 Also, observe that if the set S has the null string as its only
word, then the closure language S* also has the null string
as its only word; that is
if S = { Λ }, then S* = { Λ }
because ΛΛ = Λ.
 Hence, the Kleene closure always produces an infinite
language unless the underlying set is one of the two cases
above.

35
 Theory Of Automata - Maths

POSITIVE CLOSURE
 If we wish to modify the concept of closure to refer only
the concatenation of some (not zero) strings from a set S,
we use the notation + instead of *.
 This “plus operation” is called positive closure.
 Example: if ∑ = { x } then ∑+ = { x, xx, xxx, … }
 Observe that:

1. If S is a language that does not contain Λ, then S+ is the

language S* without the null word Λ.

2. If S is a language that does contain Λ, then S+ = S*

3. Likewise, if ∑ is an alphabet, then ∑+ is ∑* without the

word Λ.

36
 Theory Of Automata - Maths

EXAMPLE
 Defining language of EVEN
Step 1:
2 is in EVEN.
Step 2:
If x is in EVEN then x+2 and x-2 are also in
EVEN.
Step 3:
No strings except those constructed in above,
are allowed to be in EVEN.

37
 Theory Of Automata - Maths

EXAMPLE

 Defining the language factorial

Step 1:
As 0!=1, so 1 is in factorial.
Step 2:
n!=n*(n-1)! is in factorial.
Step 3:
No strings except those constructed in
above, are allowed to be in factorial.
38
 Theory Of Automata - Maths

 Defining the language PALINDROME, defined over

Σ = {a,b}
Step 1:
a and b are in PALINDROME
Step 2:
if x is palindrome, then s(x)Rev(s) and xx will also
be palindrome, where s belongs to Σ*
Step 3:
No strings except those constructed in above,
are allowed to be in palindrome

39
 Theory Of Automata - Maths

 Defining the language {anbn }, n=1,2,3,… , of

strings defined over Σ={a,b}
Step 1:
ab is in {anbn}
Step 2:
if x is in {anbn}, then axb is in {anbn}
Step 3:
No strings except those constructed in
above, are allowed to be in {anbn}

40
 Theory Of Automata - Maths

 Defining the language L, of strings ending in a ,

defined over Σ={a,b}
Step 1:
a is in L
Step 2:
if x is in L then s(x) is also in L, where s belongs to
Σ*
Step 3:
No strings except those constructed in above,
are allowed to be in L

41
 Theory Of Automata - Maths

 Defining the language L, of strings beginning and

ending in same letters , defined over Σ={a, b}
Step 1:
a and b are in L
Step 2:
(a)s(a) and (b)s(b) are also in L, where s belongs to
Σ*
Step 3:
No strings except those constructed in above,
are allowed to be in L

42
 Theory Of Automata - Maths

 Defining the language L, of strings containing aa

or bb , defined over Σ={a, b}
Step 1:
aa and bb are in L
Step 2:
s(aa)s and s(bb)s are also in L, where s belongs to
Σ*
Step 3:
No strings except those constructed in above,
are allowed to be in L

43
 Theory Of Automata - Maths

 Defining the language L, of strings containing

exactly aa, defined over
Σ={a, b}
Step 1:
aa is in L
Step 2:
s(aa)s is also in L, where s belongs to b*
Step 3:
No strings except those constructed in above,
are allowed to be in L

44