Unit 5

Introduction to
Computational Complexity
 Unit 5

Properties of Recursive and Recursively enumerable

Introduction to Computational Complexity
languages

Time and Complexity

Space classes: NP NP
complexity Class P, hardness Completeness
of TMs Class NP
Undecidability
Decidable Languages
• A language is decidable,
• if there is a Turing machine (decider)
• that accepts the language and
• halts on every input string
Decidability
Example
Theorem
Theorem
Theorem
Theorem
Undecidable Languages
• undecidable language = not decidable language
• There is no decider:
• there is no Turing Machine which accepts the language and makes a
decision (halts) for every input string
• (machine may make decision for some input strings)
• there is no Turing Machine (Algorithm) that gives an answer (yes or
no) for every input instance
• (answer may be given for some input instances)
Undecidable problem
Halting Problem
Turing Machines are countable
 Turing machines and computability

Recap :
1. Definition of Turing machines: high level and low-level descriptions

2. Variants of Turing machines

3. Decidable and Turing recognizable languages

4. Church-Turing Hypothesis UNDECIDABILITY

5. Undecidability and a proof technique by diagonalization

Unit 5 / SRMIST / KTR 20

 Undecidable problems

• is empty?

• is regular?

• has size 2?

This can be generalized to all non-trivial properties of Turing-acceptable languages

Unit 5 / SRMIST / KTR 21

 Rice’s theorem (1953)

1. Any non-trivial property of R.E languages is undecidable!

2. Property P ≡ set of languages satisfying P
3. Property of r.e languages: membership of M in P depends only on
the language of M.
If L(M) = L(M ’), then <M> ∈ P if <M’> ∈ P
4. Non-trivial: It holds for some but not all TMs.

Unit 5 / SRMIST / KTR 22

Properties of RE Languages

Unit 5 / SRMIST / KTR 23

Property : Set of language
Just a subset of r.e. languages. Thus, L satisfies a property P if L ∈ P.

Examples:
1. Set of regular languages I
2. Set of context-free languages I
3. Set of all languages I
4. {∅} I
5. ∅

• So decidability of property P is decidability of language L P .

Unit 5 / SRMIST / KTR 24

Type of Property

Unit 5 / SRMIST / KTR 25

Non-trivial property

Unit 5 / SRMIST / KTR 26

 Non-trivial property:

A property possessed by some Turing-acceptable languages but not

all
Example: : is empty?
YES
NO

NO

Unit 5 / SRMIST / KTR 27

More examples of non-trivial properties

Unit 5 / SRMIST / KTR 28

Trivial property:
A property possessed by ALL
Turing-acceptable languages

: has size at least 0?

True for all languages

: is accepted by some
Turing machine?
True for all
Turing-acceptable languages
Unit 5 / SRMIST / KTR 29
We can describe a property as the set
of languages that possess the property

If language has property then

Example: : is empty?
YES
NO

NO
Unit 5 / SRMIST / KTR 30
Example: Suppose alphabet is

: has size 1?
NO
YES
NO
NO

Unit 5 / SRMIST / KTR 31

 Non-trivial property problem
Input: Turing Machine

Question: Does have the non-trivial

property ?

Corresponding language:

Unit 5 / SRMIST / KTR 32

 Can u apply rice theorem ?

For each of No answers above, is the language decidable?

What do you do when Rice’s theorem does not apply? Fall back on reductions!

Unit 5 / SRMIST / KTR 33

Rice Theorem ( Part 1 )

Unit 5 / SRMIST / KTR 34

Rice Theorem – The reduction

Unit 5 / SRMIST / KTR 35

Rice Theorem ( Part 2 )

Unit 5 / SRMIST / KTR 36

Rice Theorem (Part 2) – The reduction

Unit 5 / SRMIST / KTR 37

POST’s CORRESPONDENCE PROBLEM
Post Correspondence Problem

• The Post Correspondence Problem (PCP), introduced by Emil Post in

1946, is an undecidable decision problem.
• The PCP problem over an alphabet ∑ is stated as follows −
• Given the following two lists, M and N of non-empty strings over ∑ −
• M = (x1, x2, x3,………, xn)
• N = (y1, y2, y3,………, yn)
• We can say that there is a Post Correspondence Solution, if for some
i1,i2,………… ik, where 1 ≤ ij ≤ n, the condition xi1 …….xik =
yi1 …….yik satisfies.
Example 1

Find whether the lists

M = (abb, aa, aaa) and N = (bba, aaa, aa)
have a Post Correspondence Solution?
Solution for Example 1

Here,

x2x1x3 = ‘aaabbaaa’

and y2y1y3 = ‘aaabbaaa’

We can see that

x2x1x3 = y2y1y3

Hence, the solution is i = 2, j = 1, and k = 3.

Example 2

• Find whether the lists M = (ab, bab, bbaaa) and N = (a, ba, bab) have
a Post Correspondence Solution?
Solution for Example 2
• General Objective:

– to understand the concept of PCP

• Specific Objectives

– State the objective of PCP

– Define PCP problem

– Check whether the PCP instance have a solution or

not.

– Define MPCP

– Illustrate the conversion of TM to MPCP with an

example
 Post Correspondence Problem
• It involves with strings

• Goal:
– To prove this problem about strings
to be undecidable
 Post Correspondence Problem
FORMAL DEFINITION
• lists of strings A and B ( equal length)
Given two

A = w1, w2, …, wk B = x1, x2, …, xk

The problem is to determine if there is a sequence of

one or more integers i1, i2, …, im such that:

wi1wi2…wim = xi1xi2…xim

(wi, xi) is called a corresponding pair.

 Example
A B
i wi xi
1 1 111
2 10111 10
3 10 0
This PCP instance has a solution: 2, 1, 1, 3:
w2w1w1w3 = x2x1x1x3 = 101111110
 Does this PCP instance have a solution?
A B
i wi xi
1 110 110110
2 0011 00
3 0110 110
This PCP instance has a solution: 2,3,1 w2w3w1 = x2x3x1 =
00110110110

One more solution:

2,1,1,3,2,1,1,3
 Modified Post Correspondence Problem (MPCP)

Definition:
First pair in the A and B lists must be the first pair in the solution, i.e., the problem
is to determine if there is a sequence of zero or more integers i1, i2, …, im such
that:
w1wi1wi2…wim = x1xi1xi2…xim
Modified Post Correspondence Problem (MPCP)
List A List B
i wi xi
1 10 10
2 110 11
3 11 011

This MPCP instance has a solution: 1,2,3

w1 w2 w3 = x1 x2 x3
10 110 11 = 10 11 011
 Undecidability of PCP
To show that PCP is undecidable, we will reduce the universal
language problem (Lu) to MPCP and then to PCP:

Universal
A mapping MPCP A mapping PCP
Language
Problem (Lu )

If PCP can be solved, Lu can also be solved. Lu is undecidable,

so PCP must also be undecidable.
 Reducing MPCP to PCP
• This can be done by inserting a special symbol (*) to the
strings in list A and B of to make sure that the first pair
will always go first in any solution.
• List A : * follows the symbols of ∑
• List B : * precedes the symbols of ∑
• w
k+1
= $ ; x
k+1
= *$
 Mapping MPCP to PCP
Suppose the original MPCP
instance is:
A B
i wi xi
1 1 111
2 10111 10
3 10 0
 Mapping MPCP to PCP
The mapped PCP instance will be:
A B
i wi xi
0 *1* *1*1*1
1 1* *1*1*1
2 1*0*1*1*1* *1*0
3 1*0* *0
4 $ *$
 Mapping Lu to MPCP

• Turing machine M and an input w, we want to determine if

M will accept w.
• the mapped MPCP instance should have a solution
if and only if M accepts w.
 Mapping Lu to MPCP
Lu instance MPCP instance
Construct an
MPCP instance
Given: Two lists:
(T,w) A and B

If T accepts w, the two lists can be matched. Otherwise, the two lists cannot be
matched.
 Rules of Reducing Lu to MPCP

• We summarize the mapping as follows. Given T and w, there

are five types of strings in list A and B:
• Starting string (first pair):
List A List B
# #q0w#
where q0 is the starting state of T.
 • Strings for copying:
List A List B
X X
# #

where X is any tape symbol (including the blank).

# is a separator can be appended to both the lists
• Strings from the transition function δ:

List A List B

qX Yp from δ(q,X)=(p,Y,R)
ZqX pZY from δ(q,X)=(p,Y,L)
q# Yp# from δ(q,#)=(p,Y,R)
Zq# pZY# from δ(q,#)=(p,Y,L)

where Z is any tape symbol except the blank.

 • Strings for consuming the tape symbols at the end:

List A List B

Xq q
qY q
Xq q
Y
where q is an accepting state, and each X and Y is any
tape symbol except the blank.
 • Ending string:
List A List
q## B #

where q is an accepting state.

• Using this mapping, we can show that the original Lu
instance has a solution if and only if the mapped MPCP
instance has a solution.
 PCP is undecidable
• Theorem: Post’s Correspondence Problem is undecidable.

• We have seen the reduction of MPCP to PCP

• Now we see how to reduce Lu to MPCP.

– M accepts w if and only if the constructed MPCP instance has a solution.

– As Lu is undecidable, MPCP is also undecidable.

Recursive and Recursively Enumerable Language
- Properties
 Recursive Enumerable (RE) or Type -0 Language

• RE languages or type-0 languages are generated by type-0 grammars.

• An RE language can be accepted or recognized by Turing machine which means it
will enter into final state for the strings of language
• May or may not enter into rejecting state for the strings which are not part of the
language.
• It means TM can loop forever for the strings which are not a part of the language.
• RE languages are also called as Turing recognizable languages.
 Recursive Enumerable (RE) or Type -0 Language

• RE languages or type-0 languages are generated by type-0 grammars.

• An RE language can be accepted or recognized by Turing machine which
means it will enter into final state for the strings of language
• May or may not enter into rejecting state for the strings which are not part of
the language.
• It means TM can loop forever for the strings which are not a part of the
language.
• RE languages are also called as Turing recognizable languages.
Recursive Language (REC)

• A recursive language (subset of RE)

• Decided by Turing machine
• It will enter into final state for the strings of language and
rejecting state for the strings which are not part of the language.
• e.g.; L= {anbncn|n>=1} is recursive because we can construct a turing
machine which will move to final state if the string is of the form anbncn
• else move to non-final state.
• So the TM will always halt in this case.
• REC languages are also called as Turing decidable languages.
Closure Properties of Recursive Languages
Introduction to Computational Complexity
• Computational Problems that can be
modelled and solved by
computer
• Complexity how much of some
resource (time, space) a
problem takes up when being
solved
• The amount of resources required for executing a particular
(computation or) algorithm is the computational complexity of that
algorithm.

• In general, when we talk about complexity we are talking about time

complexity and space complexity.
Turing Machines - the Role
• Basic tool for complexity theory.
• An abstract model of computation machine: conceptually simple .
• can execute any computation possible on “known computers”
• resources consumption models well “real” computers
TIME COMPLEXITY-Turing Machine

• How long computation takes to execute

• For a Turing machine, the time complexity refers to the measure of
the number of times the tape moves when the machine is initialized
for some input symbols.
• i.e Number of machine cycles
• Time complexity of Turing Machine M on an input word w, denoted as
T(M,w) is the number of steps done by the machine before it halts. If it
does not, we set T(M,w) = ∞

• We can also define time complexity of a machine itself as follows:

• Function f : N → N is time complexity of M iff ∀n∈N :

• f (n) = max{T(M,w) : w ∈ Σ n} (assuming it halts)
SPACE COMPLEXITY-Turing Machine
• How much storage is required for computation.
• In Turing machine It is number of cells used.
• No. of bytes used
• Space used by TM M on input word w, denoted as S(M,w) is defined as the
number of tape cells that were visited by the head before M halted.

• If M does not halt, it is not defined. We say that f : N → N is space

complexity of M (with stop property) iff ∀n∈N : f (n) = max{S(M,w) : w ∈ Σ n}
EXAMPLE
0 0 0 1 1 1
Length n

Time complexity
x€0n1n

• Each scan takes n Steps

• The overall running time is O(n2)
• Optimal solution for one tape Turing Machine O(nlogn)
EXAMPLE Read only Tape

0 0 0 1 1 1
Length n

Space complexity
x€0n1n
• Make the input tape as read only tape
• How much space it require for computation?
• If it uses tape as a stack it requires O(n) cells
• Add Binary counter logic to the TM so If n=64,it takes
log2(n)=7 bits in binary to keep the count

• So we can solve it using O(logn) space.

NP HARD AND NP COMPLETE

-
P and NP problems
• P:P problems refer to problems where an algorithm would take a polynomial
amount of time to solve, or where Big-O is a polynomial (i.e. O(1), O(n), O(n²),
etc).
• These are problems that would be considered ‘easy’ to solve, and thus do not
generally have immense run times.
• Examples:
• a. Linear Search
• b. Binary Search
• c. Merge Sort
• d. Quick sort etc.,
P and NP problems
• Can you guess the approximate time complexity of the given problem.
NP-Hard and Complete-Need

• Np Hard and complete problems gives a base for a research topics.

• Most of the algorithms are categorized into polynomial time (linear time),
exponential time (2^n)algorithms.

• Reason for looking other algorithms when we already have an algorithm

• Reducing the run time is the major challenge!

• Improve Efficiency until we have constant time Algorithm ie.O(1)

Polynomial Vs Exponential Algorithms
Polynomial Algorithms Exponential Algorithms

Linear Search - n 0/1 Knapsack Problem -2n

Binary Search -log n Travelling Salesman Problem-2n

Insertion sort -n 2 Sum of Subsets-2n

Merge Sort -n log n Hamiltonian cycle-2n

Exponential Algorithms are much more time consuming than Polynomial Algorithms.
The framework for converting Exponential to Polynomial Algorithms is called NP
Hard and Np Complete problems.
P and NP problems
• Non deterministic Polynomial time solving.
• Problem which can't be solved in polynomial time like TSP( travelling salesman problem)
• An easy example of this is subset sum: given a set of numbers, does there exist a subset whose sum is
zero?.
• but NP problems are checkable in polynomial time means that given a solution of a problem , we can check
that whether the solution is correct or not in polynomial time.
NP and P Problems
• P -Deterministic polynomial time Algorithms
• NP-Non Deterministic Polynomial Algorithms

For eg.Merge Sort from Non deterministic to Deterministic Algorithm

NP
P
NP Hard and NP Complete
• Before knowing about NP-Hard and NP-Complete we need to know
about reducibility:
• Reduction:
• Consider there are two problems, A and B, and we know that the
problem A is NP class problem and problem B is a P class problem.
• If problem A can be reduced, or converted to problem B, and this
reduction takes a polynomial amount of time, then we can say that A
is also a P class problem (A is reducible to B).
NP Hard and NP Complete
• A problem is classified as NP-Hard when an algorithm for solving it
can be translated to solve any NP problem.
• Then we can say, this problem is at least as hard as any NP problem,
but it could be much harder or more complex.
• NP-Complete problems are problems that live in both the NP and NP-
Hard classes.
• This means that NP-Complete problems can be verified in polynomial
time and that any NP problem can be reduced to this problem in
polynomial time.
How to convert Exponential Algorithms to Polynomial
Algorithms?
This can be done in two ways
• By finding Deterministic and Non Deterministic Polynomial Algorithms
• Deterministic –finite solution for every run of the program
• Non Deterministic-random solution for every run of the program(every state has
n number of possible next states)

Or

• By solving a base problem(which includes all Exponential problems)by finding out

the relationship among them
Non Deterministic Algorithm(Magic into procedure in
future)
Problem Statement : Search an element x on A[1:n] where n>=1, on successful search return j if a[j] is equals to x otherwise return 0.
• Non-deterministic Algorithm for this problem :
1.j= choice(a, n)------------------------O(1)
2.if(A[j]==x) then
{
write(j);
success();-----------------------------O(1)
}
3.write(0);
failure();------------------------------------O(1)
Non Deterministic Methods
choice(X) : chooses any value randomly from the set X.
failure() : denotes the unsuccessful solution.
success() : Solution is successful and current thread terminates.
Satisfiability Problem(SAT)
• Definition. A formula A is satisfiable if there exists an assignment of values to its
variables that makes A true.
A=(¬x ∨ y) ∧(~y V z)^(x V ~z V y)-Three variables hence we have 23 input so this is similar to other
exponential Algorithms
It take values from 000 to 111 (different combination of binary input for each variable x,y,z)
We have to try 8 values ie.23. For n variables it will be 2n
State space tree x=1 x=0

y=1 y=0
1 0

z=1 0 1 0 1 0 1 0

If this problem is solved all other exponential algorithms can be solved in polynomial time!
Np Hard-Reduction
• SAT is a Np-Hard Problem

• SAT 0/1 Knapsack Problem(Reduction!)

• Then 0/1 knapsack is also a NP Hard Problem if the reduction is in polynomial

time

• SAT L1,then L1 is also Np-Hard

• SAT L1,L1 L2 then L2 is also NP Hard hence transitive property holds true.
 Np Complete
If a problem comes under Np Hard and if it has a non deterministic algorithm then it is
also called as a Np Complete problem

SAT L Non Deterministic NP Complete Problem

Cook’s Theorem
• Cook’s Theorem states that

• Any NP problem can be converted to SAT in polynomial time

• SAT will come into P if and only if P=NP, it will become deterministic.
Clique to Vertex Cover
Summary

• Polynomial and Exponential Algorithms

• Framework/Guidelines for working on Exponential Algorithms

• NP-Hard

• Np-Complete

• Cook’s Theorem
Hope all our Non Deterministic Algorithms will turn into Polynomial Deterministic
Algorithm i.e.P=NP☺