Theory of Computation

CSE ( CET2008B)

TY BTech

Unit V
Basic Introduction to Complexity
 Unit V
Basic Introduction of Complexity
 Course Objective & Course Outcomes
• Course Objectives:
1. To Study computing machines by describing, classifying and comparing
different types of computational models.
2. Encourage students to study Theory of Computability and Complexity.
• Course Outcomes:
After successful completion of this course students will be able to:
1. Construct finite state machines to solve problems in computing
2. Write mathematical expressions for the formal languages
3. Apply well defined rules for syntax verification
4. Construct and analyze Push Down, Post and Turing Machine for formal
languages
5. Express the understanding of the decidability and Undecidability problems
6. Express the understanding of computational complexity.
 Text Books & Reference Books
• Text Books
1. Michael Sipser “Introduction to the Theory of Computation” CENGAGE
Learning, 3rd Edition ISBN-13:978-81-315-2529-6
2. Vivek Kulkarni, “Theory of Computation”, Oxford University Press,
ISBN-13: 978-0-19-808458-7

• Reference Books
1. Hopcroft Ulman, “Introduction To Automata Theory, Languages And
Computations”, Pearson Education Asia, 2nd Edition
2. Daniel. A. Cohen, “Introduction to Computer Theory” Wiley-India,
ISBN:978-81-265-1334-5
3. K.L.P Mishra ,N. Chandrasekaran ,“Theory Of Computer Science (Automata,
Languages and Computation)”, Prentice Hall India,2nd Edition
4. John C. Martin, “Introduction to Language and Theory of Computation”, TMH,
3rd Edition ISBN: 978-0-07-066048-9
5. Kavi Mahesh, “Theory of Computation: A Problem Solving Approach”,
Wiley-India, ISBN: 978-81-265-3311-4
 Turing Machine
A Turing Machine is represented by a 7-tuple
T = (Q, Σ, Γ, δ, q0, qaccept, qreject):
Q is a finite set of states
Σ is the input alphabet, where Z0∉ Σ
Γ is the tape alphabet, a superset of Σ; Z0 ∈
Γ

q0 ∈ Q is the start
state
qaccept ∈ Q is the accept
state
 Solvable and Semi-Solvable Problems
🙣 Solvable problem: TM when applied to such a problem, always
eventually terminates with the correct “yes” or “no” answer
🙣 A class of all such problems is called as Recursive language
🙣 The mathematical functions that denote these type of problems are called as
Total Recursive Functions
🙣 Simple Examples - multiplication, addition, concatenation and many other

🙣 Semi-solvable problem: TM when applied to such a problem, always eventually

terminates with correct answer when answer is “yes” and may or may not
terminate when the correct answer is “no”
🙣 A class of all such problems is called as Recursively Enumerable language
🙣 The mathematical functions that denote these type of problems are called as
Partial Recursive Functions
🙣 Simple Examples - division, factorial and many other
 Halting Problem and Unsolvablity
• The halting problem is the problem of determining, from a description of an arbitrary
computer program and an input, whether the program will finish running or continue to run
forever.

• the halting problem is undecidable over Turing machines. It is one of the first examples of a
decision problem.

• For example, in pseudo code, the program:

– while (true) continue
does not halt; rather, it goes on forever in an infinite loop. On the other hand, the program
– print "Hello, world!"
 Halting Problem and Unsolvablity
🙣 For a given input for any general TM two
cases arise,
🙣 The machine may halt after a finite
number of steps
🙣 The machine may not ever halt no
matter how long it runs
🙣 Given any TM, problem of algorithmically
determining whether it ever halts or not, is
called as the Halting Problem
🙣 The halting problem is unsolvable
 Proof :-Halting problem is undecidable

• Proof by contradiction
• There is a Turing machine T that will decide the halting
problem.<M> this is the description of Turing machine M
and string W . T write "accept" when M halts on w, and
reject If M does not halts on W
 Proof :-Halting problem is undecidable

We build a Turing machine Tm and here we

standardizing T so when T write yes and halt then Tm
will goes into loop forever
 Proof :-Halting problem is undecidable

we build Tc with the help of TM. Here we take Tc as input which

is the description of Turing machine M and we write it in this way
d(M) now we copies it to obtain the string d(M)*d(M), where * is
a symbol that break up the two copies of d(M) and then provide
d(M)*d(M) to the Turing machine Tm .
 Proof :-Halting problem is undecidable

What Turing machine Tc does when a string given to it which

describe Tc itself
d(Tc) is given as input to Tc it make copy of it and build the
string d(Tc)*d(Tc) and allot to standardized T.so the altered T
is specified a description of Tc and string d(Tc)
 Post Correspondence Problem
• First introduced by Emil Post in 1946, the Post
Correspondence Problem (𝑃𝐶𝑃)
• It is a well-known example of an undecidable decision
problem.
• The goal of the problem is to determine whether or not a
given set of dominos has a match, which means that
dominos from the set can be arranged in a way (with
repetition) such that the top and the bottom strings are
identical.
Post Correspondence Problem
 Post Correspondence Problem
Input: Two sets of strings

There is a Post Correspondence Solution

if there is a sequence such that:

PC-solution:
Post Correspondence Problem
Post Correspondence Problem
 Post Correspondence Problem

Because total length of strings from B

is smaller than total length of strings from A
 Turing Machine
• Invented by Alan Turing in 1936

• A simple mathematical model of a general purpose

computer

• It is capable of performing any calculation which can be

performed by any computing machine
 Turing Machine
• Invented by Alan Turing in 1936

• A simple mathematical model of a general purpose

computer

• It is capable of performing any calculation which can be

performed by any computing machine
 Turing Machine
• Types of Problems
• Decidable
• Undecidable

• Decidable : Problems that can be solvable in a

reasonable(polynomial) time.
• Undecidable : Some problems are intractable, as they grow
large, we are unable to solve them in reasonable time
 Complexity of TM

• For a Turing machine, the time complexity refers

to the measure of the number of times the tape
moves when the machine is initialized for some
input symbols.

• The time-complexity of a NTM is defined as the

maximum number of steps it uses on any
branch on any input of length n.
 Turing Machine
• What constitutes reasonable time?
– Standard working definition: polynomial time
– On an input of size n the worst-case running time is O(nk) for
some constant k
– O(n2), O(n3), O(1), O(n lg n), O(2n), O(nn), O(n!)
– Polynomial time: O(n2), O(n3), O(1), O(n lg n)
– Not in polynomial time: O(2n), O(nn), O(n!)
• Are all problems solvable in polynomial time?
– No: Turing’s “Halting Problem” is not solvable by any computer,
no matter how much time is given.
Time complexity graph
P, NP, NP-Hard, NP-Complete
 The Class P (Polynomial-time)
P: the class of problems that have polynomial-time deterministic
algorithms.
– That is, they are solvable in O(p(n)), where p(n) is a polynomial on n
– A deterministic algorithm is (essentially) one that always
computes the correct answer
-They are simple to solve, easy to verify and take computationally
acceptable time for solving any instance of the problem. Such problems
are also known as “tractable”.
-Why polynomial?
– efficient
– nice closure properties
•the sum and composition of two polynomials are always polynomials too
 The Class P (Polynomial-time)
•Searching an element in the array (O(n)),
•inserting an element at the end of a linked list (O(n)),
• sorting data using selection sort(O(n2)),
•finding the height of the tree (O(log2n)),
•sort data using merge sort(O(nlog2n)),
•matrix multiplication O(n3).
•MST
 The Class NP(Nondeterministic Polynomial-time)
NP: the class of decision problems that are solvable in polynomial time on a
nondeterministic machine (or with a nondeterministic algorithm)
– (A determinstic computer is what we know)
– A nondeterministic computer is one that can “guess” the right answer or solution
• Think of a nondeterministic computer as a parallel machine that can freely spawn an
infinite number of processes
• Thus NP can also be thought of as the class of problems “whose solutions can be
verified in polynomial time”
• Note that NP stands for “Nondeterministic Polynomial-time”
 The Class NP (Non-Polynomial-time)
•Travelling salesman problem (O(n!)),
•Hamiltonian cycle (O(n!))
•Tower of Hanoi (O(2n – 1)),
•Knapsack problem (O(2n)),
•Clique problem.
•Vertex cover problem.
 TSP
Given a set of cities and the distance between every pair of
cities, the problem is to find the shortest possible route that
visits every city exactly once and returns to the starting
point.
Note the difference between Hamiltonian Cycle and TSP.
The Hamiltonian cycle problem is to find if there exists
a tour that visits every city exactly once.
Here we know that Hamiltonian Tour exists (because the
graph is complete) and in fact, many such tours exist, the
problem is to find a minimum weight Hamiltonian Cycle.
 Knapsack Problem
Given a set of items, holding weights and profit values, one
must determine the subset of the items to be added in a
knapsack such that, the total weight of the items must not
exceed the limit of the knapsack and its total profit value is
maximum.

Hamiltonian cycle
 Difference between N and NP

• P problems are set of problems • NP problems are problems which can

which can be solved in polynomial be solved in nondeterministic
time by deterministic algorithms. polynomial time.
• P Problems can be solved and • The solution to NP problems
verified in polynomial time cannot be obtained in polynomial
time, but if the solution is given, it
can be verified in polynomial time.
• P problems are a subset of NP
• NP Problems are a superset of P
problems
problems
• All P problems are deterministic in
• All the NP problems are
nature
non-deterministic in nature
• Example: Selection sort, Linear
• Example: TSP, Knapsack problem
search
 Difference between P and NP
• P problems are set of problems • NP problems are problems which can
which can be solved in polynomial be solved in nondeterministic
time by deterministic algorithms. polynomial time.
• P Problems can be solved and • The solution to NP problems
verified in polynomial time cannot be obtained in polynomial
time, but if the solution is given, it
can be verified in polynomial time.
• P problems are a subset of NP
• NP Problems are a superset of P
problems
problems
• All P problems are deterministic in
• All the NP problems are
nature
non-deterministic in nature
• Example: Selection sort, Linear
• Example: TSP, Knapsack problem
search
 The Class NP Hard
What does NP-hard mean?
– A lot of times you can solve a problem by reducing it to a different problem.
I can reduce Problem B to Problem A if, given a solution to Problem A, I can
easily construct a solution to Problem B. (In this case, "easily" means "in
polynomial time”).

• A problem is NP-hard if all problems in NP are polynomial time reducible to

it, ... Ex- Hamiltonian Cycle

• Every problem in NP is reducible to HC in polynomial time. Ex:- TSP is

reducible to HC.
B A
Example: lcm(m, n) = m * n / gcd(m, n),
• Example: TSP (the optimization problem of finding the least-cost cyclic
route through all nodes of a weighted graph)
• the Knapsack Problem, and the Integer Programming Problem
 The Class NP Complete
• A problem is NP-complete if the problem is
both
– NP-hard, and
– NP
• Examples:
• the traveling salesman problem
• satisfiability problems
• graph-covering problems
 NP hard and NP Complete
• If R is polynomial-time reducible to Q,
• we denote this R <=p Q
• Definition of NP-Hard and NP-Complete:
– If all problems R  NP are polynomial-time reducible to Q,
then Q is NP-Hard
– We say Q is NP-Complete if Q is NP-Hard and Q  NP
• If R <=p Q and R is NP-Hard, Q is also NP-Hard
 Summary
• P is set of problems that can be solved by a deterministic Turing machine
in Polynomial time.
• NP is set of problems that can be solved by a Non-deterministic Turing
Machine in Polynomial time. P is subset of NP (any problem that can be
solved by deterministic machine in polynomial time can also be solved by
non-deterministic machine in polynomial time) but P≠NP.
• There are some problems that every single problem in NP can be
translated into, and a fast solution to such a problem would automatically
give us a fast solution to every problem in NP. This group of problems are
known as NP-Complete. Ex:- Clique
• A problem is NP-hard if an algorithm for solving it can be translated into
one for solving any NP-problem (nondeterministic polynomial time)
problem. NP-hard therefore means "at least as hard as any NP-problem,"
although it might, infact, be harder
The End!!