MODULE-1 What is TOC? In theoretical computer science, the theory of computation is the b whether and how efficiently problems can be solved on a model of algorithm. The field is divided into three major branches: automata and computational complexity theory. In order to perform a rigorous study of computation, computer scientists work with a ‘mathematical abstraction of computers called a model of computation. There are several models in use, but the most commonly examined is the Turing machine. Automata theory In theoretical computer science, automata theory is the study of abstract machines (or more. appropriately, abstract ‘mathematical’ machines or systems) and the computational problems that, can be solved using these machines. These abstract machines are called automata. ‘This automaton consists of © states (represented in the figure by circles), * and transitions (represented by arrows). As the automaton sees a symbol of input, it makes a transition (or jump) to another state, according to its transition function (which takes the current state and the recent symbol as its inputs). Uses of Automata: compiler design and pursing. _ computability theoryAdditive inverse: a+(-a)=0 Multiplicative inverse: a* I/a=1 Universal set U={1,2,3,4,5} Subset A=(1,3) A’ =(24,5) ‘Absorption law: AU(A MB) = A, AM(AUB) = A . De Morgan’s Law: (AUBY' =A". (anBy =A’ UB? Double compliment (ay ANA’=0 Logic relations: a>b=>7aUb TaNb)=7a U 7 Relations: Let a and b be two sets a relation R contains aXb, Relations used in TOC: Reflexive: a= Symmetric: aRb => bRa Transition: aRb, bRe = > aRe If a given relation is reflexive, symmentric and transitive then the relation is called equivalence relation, Deductive proof: Consists of sequence of statements whose truth lead us from some initial | statement called the hypothesis or the give statement to a conclusion statement. The theorem that is proved when we go from a hypothesis H to a conclusion 4 Cisthe statement “if 7 then C.” We say that C is deduced from H.Proof by contrapositive: ___ The contrapositive of the statement “if H' then ©” is “if not C then not H.” A statement and its contrapositive are either both true or both false, so we can prove either to prove the other. Theorem 1.10: RU(SQT)=(RUS)N (RUT). [|__| Statement Justification I. | zis in RU(SNT) Given 2. | wisin Ror z isin SAT | (1) and definition of union 3. | visin Ror zisin (2) and definition of intersection both § and T 4.| visinRUS (3) and definition of union 5. | visin RUT (3) and definition of union 6. | x isin (RUS) (RUT) | (4), (6), and definition of intersection Figure 1.5: Steps in the “if” part of Theorem 1.10 | Statement | Justification isin (RUS)N(RUT) | Given risin RUS (1) and definition of intersection . | cisin RUT (1) and definition of intersection | vis in Ror zis in (2), (3), and reasoning oth S and T about unions 8 (4) and definition of intersection (5) and definition of union Steps in the “only-if” part of Theorem 1.10 “if H then C” and “if not C then not H” are logically equivalent, first observe that there are four cases to consider:1. H and C both true. 2. H true and C false. 3. C true and H false. 4. H and C both false. ‘ ‘Hand not C implies falsehood. ‘esis Hand the negation of the ething known to be of is called proof by ‘That is, start by assuming both the hypoth conclusion C. Complete the proof by showing that som false follows logically from H and not C. This form of pros contradiction. Tt often is easier to prove that a statement is not a theorem than to Prove it is a theorem. As we mentioned, if $ is any statement, then the statement ig ie not a theorem” is itself a statement without parameters, and thus can Be regarded as an observation than a theorem. 18: All primes are odd. (More formally, we might say: Alleged Theorem 1. then x is odd.) if integer x is a prime, pisPROOF: The integer 2 is a prime, anb = @ and cis a subset of b the prove that ae =D c subset b but 2iseven. O ets a,b,c if xe {.e,, the assumption is wrong)Proof by mathematical Induction: Suppose we are given a statement $(n), about an integer n, to prove. One ‘common approach is to prove two things: 1. The basis, where we show S(i) for a particular integer i. Usually, i = 0 or i= 1, but there are examples where we want to start at some higher i, perhaps because the statement S is false for a few small integers. e basis integer, 2. The inductive step, where we assume n 2 1, where i is the wwe show that “if S(n) then S(n + 1).” © The Induction Principle: If we prove S(i) and we prove that for all n > i, ‘S(m) implies $(n + 1), then we may conclude $(n) for all n 2 i. Languages : nis an abstraction of natural languages. That is, ‘The languages we consider for our discussio 1 definitions. Programming our focus here is on formal languages that need precise and formal languages belong to this category. Symbols : symbols are the atoms ‘Symbols are indivisible objects or entity that cannot be defined, That is, 44,0,1,4% of the world of languages, A symbol is any single object such as begin, or do. Alphabets : ‘Analphabet isa finite, nonempty set of symbols, The alphabet ofa language is normally denoted by 2. When more than one alphabets are considered for discussion, then peused (e.g. =1r22etc) or sometimes other symbol like G may also be Strings or Wor AA string or word over an alphabet is a finite sequence of concatenated symbols ofExample: 0110, 11, 001 are three strings over the binary alphabet { 0, 1 ) ‘aab, abc, b, cc are four strings over the alphabet { a,b, ¢} It is not the case that a string over some alphabet should contain all the symbols from the alpha- ‘bet. For example, the string cc over the alphabet { a, b, ¢ } does not contain the symbols a and b. “Hence, itis true that a string over an alphabet is also a string over any superset of that alphabet Length of a string : ‘The number of symbols ina string w is called its length, denoted by (| sxample :|011|=4, |11|=2, |b|=1 Convention : We will use small case letters towards the beginning of the English alphabet to denote symbols of an alphabet and small case letters towards the end to denote strings over an alphabet. That is, a . 4,8,¢€ (gymbolsyand % %¥ =) are strings. ‘Some String Operations : Let ¥ 7 22s © Geang ¥ ~ Bbc € Pepe two strings. The concatenation of x and y ayaa ~ avhbss ~ That is, the concatenation of x and y denoted by xy, is the string ‘copy of y without any intervening denoted by xy is the string that has a copy of x followed by space between them, Example : Consider the string 011 over the binary alphabet, All the prefixes suffixes and substrings of this string are listed below. Prefixes: 4 0, 01, 011 Suffixes: ¢ 1, 11, 011. Substrings: 6 0, 1, 01, 11, O11. {suffix or substring) to x, for any string x and eis a prefix (suffix or of string y if x is a prefix (sutfix) of y and x 4 y. spt O11 are proper prefixes. m0, weuse *to denote the string2 LetL={"0"\nisan integer}. Then L*= { T0"\n is an integer }. Language concatenation : The concatenation of languages Lrand “is defined as Le | xy) 7© bang VEX), Example: (a, ab}{ b, ba} = (ab, aba, ab, abba} Note that , dt Lay in general, =o 3, He) = 2 = (e) Iterated concatenation of languages : Since we can concatenate two languages, we also repeat this to concatenate any number of languages, Or we can concatenate a language with itself any number of times, The operation denotes the concatenation of L with itself'n times. This is defined formally as follows: 4 -{0) fm Lie Example : Let L = { a, ab }, Then according to the definition, we have = ( L, = Le) = L = (a, ab) L = LL- (2, a8) (a, ab) = (aa, aad, aba, abad) L “LL, ~ (a, ab) (aa, aab, aba, abab) = (aaa, aaab, aaba, aabab, abaa, abaab, ababa, ababab) + The Kleene star operation on a language L, denoted as is ‘zero of more strings from L }Thus “is the set of all strings derivable by any number ofeoneatenations of strings in L Its also useful to define c = + iesall strings derivable by one or more concatenations of strings in L. That is (Unionn in Nandn>0) 2 sfutuby Example: Let L= {a,b }, Then we have, EEE 8 . fe) U(a,ab) V (aa, aab, aba, abab) U PT potuty. = (a, ab) U (aa, aab, aba, abab) W... Note: eis in 2, for every language L, including ‘The previously introduced definition of "is an instance of Kleene star. (Generates) (Recognizes) Grammar ————+ Language <——— Avtomata Automata: A algorithm or program that automatically recognizes if a particular sting belongs to guage or not, by checking the grammar of the tring. stract computing device (or machine). There are different varities of such 0 called models of computation) which can be defined mathematically, assumed to be a sequence of symbols over a given, ttape(or writen on an input file). The simpler auton 31a a time from left to right but not change. Poverful right or right to let) and change the input.+ The automaton can produce output of some form. If the output in response to an is string is binary (say, aceept or reject, then it sealed an accepter. Ifit produces an cot, ‘put sequence in response to an input then itis called a 2 ee - ‘sequence, transducer(or automato: automaton may have a temporary storage, consisting of an unlimited number of ‘of holding a symbol from an elphabet ( whcih may be different from et). The automaton can both read and change the contents of storage. The accusing capzbility of this ‘feature of the automaton is its control unit, e number of interval states at any point, It can. dd manner determined by a transition function. input tape 4 inte z [Ganado |<———+foe| Teeorary Tne Figure 1: The figure above shows a diagrammatic representation of a tion generic automa- Operation of the automation is defined as follows. At any point of time the automaton is in some integral state and a particular symbol from the input tape by using the mechanism for reading inp ton then moves to some other integral (or remain in the same state) as defined by the transition on. based on the current state, input symbol read, and the content 3 rage may be changed and the input, can be modeled by finite and hence a restricted model of computation.Finite Automata oe (Singular ; automation) are a particularly simple, but useful, model of compu ‘ation. They were initially proposed! as a simple model for the behavior of neurons. States, Transitions and Finite-State Transition System : Let us first give some intuitive idea about a stare ofa an em and state transitions before describing finite automata, Informa ly, @ state of a system is an instantaneous deseription of that system which gives all relevant information necessary to determine how the system can evolve from that point on, Transitions are changes of states that can occur spontaneously oF itt states, Though transitions usually take time, we assume that state (which is an abstraction) ponse to inputs to the sitions are instantaneous, Some examples of state transition systems are: di al systems, vending machines, ete, A system containing only a finite number of states and transitions among them is called 8 finite-state transition system. Finite-state transition systems can be modeled abstractly by a mathematical model called finite automation Deterministic Finite (-state) Automata Informally, a DFA (Deterministic Finite State Automaton) is a simple machine that reads an in put string -- one symbol at a time ~- and then, after the input has been completely rend, decid whether to accept or reject the input, As the symbols are read from the tape, the automaton ean change its state, to reflect how it reacts to what it has seen so far, A machine for which a deter= istic code can be formulated, and if there is only one unique way to formulate the code, then. is called deterministic finite automata, lly consists of 3 parts: -a symbol fom tor reading symbols trot 3. A control , which itself consists of 3 things: © finite number of states that the machine i are designated as accept or final © Weurrent state, initially set tow© astate transit for changing the current state, ‘An automaton processes a string on the tape by repeating the following actions until the tape head has traversed the entire string: 1 ‘The tape head reads the current tape cell and sends the symbol s found there to the control, Then the tape head moves to the next cell, RE contol takes sand the current state and consult the state transition funetion to get the next state, which becomes the new current state. Once the entire string has been processed, the state in which the automation enters is examined itis an accept state, the input string is accepted; otherwise, the string is rejected . Summariz~ ing all the above we can formulate the following formal definition: Deterministic Finite State Automaton : A Deterministic Finite State Automaton (DFA) is aS-tuple: M-(Q. 2.4, a, F) * Qisa finite set of states. sa finite set of input symbols or alphabet + FOE > Gig the “nent state” transi function that tells which state to move ition function (which is total ). Intuitively, is 6 a to in response to an input, ie, if M is in state qand sees input a, it moves wo ste (7-4) + %E0 is the start state . FoQ, is the set of accept or final states. Acceptance of Strings : A DFA accepts a string “4:42 rif there is a sequence of states 40+ 4 such that Zain) 1, is the start state 2, BGidie) = tutor all 0 2 which is function on symbols) &:Qxr 30 F ’ That is, © (%™*) is the state the automation reaches when from the state q and finish processing the string w. Formally, we can give an inductive definition as follows: The language of the DFA M is the set of strings that can take the start state to one of the accepting states i.e. L(M) = { ¥€2" | M accepts w} wed] (dome Example 1: M~(Q,3, 5,47) 2- (40.0) ois the start state P-{a) S(40.0)=40 — 9(%.0)- a SGGol)- (a1) = It is a formal description of a DFA. But it is hard to comprehend. For ex. The language of the DFA is any string over { 0, 1} having at least one | We can describe the same DFA by transition table or state transition diagram as follow- ing: ‘Transition Table : On [sa ]alaExpl We cannot reach find state %w/0 or in the ip string. There ean be any no. ‘of O's at the beginni (The selt-loop at % on label 0 indicates it). Similarly there ccan be any no. of 0's & 1's in any order atthe end of the string, ‘Transition table Itis basically a tabular representation of the transition function that takes two arguments (a state and a symbol) and returns a value (the “next state”). + Rows correspond to states, + Columns correspond to input symbols, + Entries correspond to next states ‘The start state is marked with an arrow ‘The accept states are marked with a star (*). atone] Safe a [a Ta [a k (Stat Transition diagram ‘A state transition diagram or simply a transition diagram is a directed graph which ean be constructed as follows: 1. For each state in Q there is a node, 2, There isa directed edge from node qo node plabeled ait (9 4)" P _ apnere are several input symbols that cause a transition, the edge is labeled by the list of these symbols.) 3. There is an arrow with no source into the start state 4. Accepting states are indicated by double circle, a va. & Here isan informal description how a DFA operates. An input to @ DFA can be any String EE Puta pointer othe stat state q. Read the input string w from let to right, one symbol ata time, moving the pointer according to the transition function, &. Ifthe next symbol of w is andthe pointers on sat p, move the Pointer to $CP- 4), When the end of ie input sing is encoutered the poner on Some state, r. The string is sid to be accepted by the DEA if reP and rejected if 7 €# . Note that there is no formal mechanism for moving the pointer. 7. Alanguage LEE" js said tobe regular if L ~L(M) for some DFA M. Regular Expressions: Formal Definition, ‘We construct REs from primitive constituents (basic elements) by repeatedly applying certain recursive rules as given below. (In the definition) Definition : Let S be an alphabet. The regular expressions are defined recursively as follows. Basis i) jisaRE " il) isaRE € saisRE, s primitive regular expression i. Primitive Constituents_— wy Use of Nea. i 'S is helpful in the case i.e. we Construct NFAs for every REs which ‘epresented by transition diagram only. ~ £ Basis : = then £0) 6 pe DO + Case (i): 7 #. Then “(")~*. Then £(")"# and the following NFA N recognizes L(r). Formally ¥~(. (@). 5, 8a F. #) where @= (4) and [email protected])=#V acs, Fp —O. © 9s L(r)-(e) N-(@), 5, and the following NFA N accepts Lie). Formally 8.4.) yyere 50.2) =6 Vaed Since the start state is also the accept step, and there is no any transition defined, it will accept the only string ©and nothing else. = Case (il) : r=a for some 4© =, Then L(r) = (a), and the following NFA N’ accepts L(". Formally, V~((7.2). 2 &.7-(2)) were 2(4)" (a), 86. 2) =(O for $4 Por dea Induction‘Assume that the start ofthe theorem is tue for REs “tand ”2. Hence we can assume that we have automata “and M4 that accepts languages denoted by REs "!and respectively .e, “(4)= (5) ang 24)" L(5)_ the FAs are represented ae o-6) - ° Each has an initial state and a final state. There are four cases to consider. + Case (i): Consider the RE > “i * denoting the language 1) Y E>). We construct FA “5, from #iand 2to accept the language denoted by RE "?as follows : My Create a new (initial) start state 7 and give €- transition to the Ee tial state of 4 and 4 This is the intial state of 4, Create a final state J” and give ¢-ransition from the two final state of “and My "is the only final state of “and final state of “and *4 will be ordinary states in “4 Allthe state of “and #4 are also state of 44,yy Allthe moves of Miand Mzare also moves of 4. { Formal Construction) is easy to prove that A4) = Lt) Proot: To show that 24) = £5) we must show that = Le) UL) FOG) E00) 5 folowing wanstion ot Starts at intial state 7 and enters the start state of either “or "folhvoing the transition ie. without consuming any input, WLOG, assume that, it enters the stant state of #6. From this point onward it has to follow only the transition of ito enter the final state of “1, because this isthe only way to enter the final state of Mf by following the e- transition.(Which is the last transition & no input is taken at hte transition). Henice the Whole input w is considered while traversing from the start state of “to the final state of 4 Therefore Mimust accept say, ” £()og » € L(0s) WLOG, say 7 €2(44) Therefore when ““ipracess the string w , it starts at the intial state and enters the final state when w consumed totally, by following its transition. Then “also accepts 1, by starting at state % and taking ©-transition enters the start state of “follows the moves. of Mito enter the final state of “/iconsuming input w thus takes €-transition to J” Hence proved * Gase(i) : Consider the RE "3 ~ "denoting the language “DE(G) we consiruct FA from Mia to accept “(5).as follows :Create a new start state @ and a new final state 4, Add €- transition from 2 tothe start state of 4 2 Vt final state of “to the start state of 2, Allthe states of “are also the states of 2, “Shas 2 more states than that of Minamely 7 and 3, Allthe moves of Miare also included in By the transition of ype (o), Sean accept © By the transton of type (a), "can enters te inital state of any input and then follow all kinds moves of “to enter the fina state of “and then following €-transition can enter 1”. Hence if any Is accepted by “thon wis also accepted by 5. By the transition of type (b), strings accepted by "ican be repeated by any no of times & thus accepted by #4, Hence “accepts “and any string accepted by ‘repeated (ie. £046) = (L(A) “(LOY = 7 concatenated) any no of times. Hence Caseliv) : Let *=(")). Then the FA iis also the FA for ('), since the use of parentheses does not change the language denoted by the expression Non-Deterministic Finite Automata Nondeterminism is an important abstraction in computer science. Importance of rnondeterminism is found in the design of algorithms. For examples, there are many problems with efficient nondeterministc solutions but no known efficient deterministic Solutions. (Travelling salesman, Hamitonean cycle, clique, etc). Behaviour of a process is in a distributed system Is also a good example of nondeterministic situation. Because=_— ie cae nas 1 2° 2C°eP! state athernse, the automation rejects tony sonfiternatively, we can say that, ® is accepted i there exists a path with label © from vahich easate fo Some accep state. Since there is no mechanism for determanin: Which state to start in or which of the Possible next moves to take (including the 2 - som ons). response to an input symbol we can think that he automaton having Some “guessing” Power to chose the correct one in case the input is accented Example 1 : Consider the language L -(@ © {0, 1}* | The 3rd symbol from the right is 1). The following four-state automation accep! 8 L. ‘The m/cis not deterministic since there are two transitions from state Son input t and Ro transition (zero transition) from %*on both 0 & 1 For any string ® whose rd symbol trom the right Tenens leading from the start state, tothe accept state %¥. But fr any string © where 3rd symbol from the rights 0, there is no possible sequence ot legal Depp's leading ftom ?:and #*, Hence mic accepts. How does it accept any sting or) tis @ 1, there exists a sequence of legal Formal definition of NFA M=(Q5.5,4,,F Fomaly, an nea sautuple Mf = (22 .8,40-F) sere o£, 2 and bear the same meaning as or a DFA, but 5, he transtiontuncion is rodefined se tolonn, 6 Qx(EU{e)) > P( where P(Q) is the power set of Qi.e. 2 The Langauge of an NFA From the discussion of the acceptance by an NFA, we can give the formal definition of a language accepted by an NFA as follows (2.2 .2.40-? is an NEA, then the langauge avcepted by Nis writtten as LON) is ~{0| 8(a,0) nF = aiven by 209 ~{21 (4.2) 08-4} Thats, L(W) is the set ofall strings w in Z°such that £(o®) contains atleast one accepting state.the power of an NFA. That is, any €- NFA\ NFA with transition) we can always construct an equivalent NEA witnoNt €-transitions. The equivalent NFA must keep track where the © NFA goes at every step during computation. This can be done by adding extra transitions for removal of every €- transitions from the €~ NFA as follows. ttwe removed the €= transition °C?®) “trom the €= NFA then we need to moves from state p to all the state Yon input symbol # © which are reachable from state q (in trom state p10 atsame input symbol g. This will alow the modified NFA to move fort Ue e pio allstates on some input symbols which were possible in case -NFAon the same input symbol. This process is ‘stated formally in the following theories. Theorem if Lis accepted by an €- NFA N,, then there is some equivalent NFA N’yithout €transitions accepting the same language / Proot: tet ¥~(0:940-*) be the given €-FA with Mecorerictde a 10) 5'(g.a)= @ whore, 2%) {2[p© $2) 9 9 and #¢Fand = (ZU (a)) if 5(ao.€) MF * 9 otherwise. Other elements of N’and N We can show that “(= 4("”)i.¢, nvand Nare equivalent, We need to prove that Y= = weL(W) iff weZ(I) jo ywed F(qo,w)eF’ itt S(q)eF We will show something more, that is, wwe 5 (ao,w) = 5(q0.¥) — —_— =em en sme Met pn 2265 su Moet -Sle Dey cerans reir ee wn EM, Insets yon essere Ho) Hom) we[Fiwo) oslo) sy eetnanctensrin ct ay rans, unig Hud Rebee RO Sy ostasone ane #Hte?) Tocemgthe tne ane me ase vitan He oe iTopics - Finite State Machine (FSM) + What are FSM's «Why / When to use FSM 2.3 Finite State Machine (FSM) - Implementing of Finite State Machines Concept and Implementation “Home Work Assignment{(part 2) What Is A Finite State MachineM Examples in Daily Live What Is A Finite State Machine - Vending Machines - Areactive system whose response to a particular stimulus - Traffic Lights (a signal, or a piece of input) is not the same on every Elevaton occasion, depending on its current “state”. + Alarm Clock A Misowae, + For example, in the case of a parking ticket machine, will not print a ticket when you press the button unless you have already inserted some money. Thus the response to the print button depends on the previous history of the use Each of these devices can be thought of as a reactive system - that of the system. is because each of them work by reacting to signals or inputs from the external world. + Cash Registers ‘etd Unneay (t2a 2 onjon S06 cto 80 More Precisely (Formally) + £m, 1) inserting money, requesting ticket, + AFinite State Machine is defined by (Z,S,30,5,F), where: ee ccna is the input alphabet (a finite, non-empty set of symbols). + S (1, 2): unpaid, paid a finite, non-empty set of states. + S9(1): an initial state, an element of S. + 59 is an initial state, an element of S. + & (shown below) : transition function: 5: $x E—+ is the state-transition function: 5: Sx => S 5 ; + F: ompi + Fis the set of final states, a (possibly empty) subset ofS. Manette. ccimer aint + Ois the set (possibly empty) of outputsseptors and Transducers Acceptors: no output, have final states + Transducers: non-empty set of output Transducer / To Implement an FSM ite State Machine class keeps track of the current and the list of valid state transitions. u define each transition by specifying : ~~ the starting state for this transition a callable which when it retums True means this on is valid callable function which is invoked when this a aera as Deterministic and Non-Deterministic + Non-deterministic: Competing “Transitions” Leaving Same State We only concern ourselves with Deterministic FSM in this class Simplest FSM Prossiclick “b” Pressiclick “a”Why Finite State Machines For Robot * Response to an event is dependent on the “state” of the robot a e Home Work #2-2: “Cleaner” (Push Out “Trash”) + Trash: small white boxes, ‘about same size as robot, very light + No other obstacles inside boundary except trash Za SEE : Two Robot Examples + Obstacle Avoidance Example - "Escape" ExampleLL OOOOOOOEOEOOETEEe——————— HW Specification , «Push out trash to outside of boundary (black tape) — at least half of the “box’ is outside of boundary + Indicate (with sound or light) that track has been pushed out + Quit (success condition) after pushing 3 pieces of trash out + Assumptions: + No other object inside boundary except trash + Trash are small white boxes about the same size as the robot