Complexity Theory

(CoSc-4132)

Chapter One
Introduction to complexity theory and
preliminaries on Turing Machine

Samuel Gudeta (MSc.)

Department of Computer Science
Dilla University
October 2019
 Objective
At the end of this session students will be able to:
 Understand the basic concepts of complexity Theory

 Discuss the general overview of complexity Theory, System and complex system.

 Describe what a Turing Machine is

 Discuss the formal Definition of TMs

 Discuss the construction Techniques of Turing machine

 Turing machine Vs real machines

 History and computability of Turing machine

 Describe the big O notation

 Overview of Complexity Theory
• Complexity Theory is concerned with the study of the intrinsic complexity of
computational tasks.
• Its “final” goals include the determination of the complexity of any well-
defined task.
• Moreover, its goal includes obtaining an understanding of the relations
between various computational phenomena (e.g., relating one fact
regarding computational complexity to another).
• Indeed, we may say that the former type of goals is concerned with absolute
answers regarding specific computational phenomena, whereas the latter type
is concerned with questions regarding the relation between computational
phenomena.
 Cont.…
• Complexity theory has failed to determine the intrinsic complexity of tasks
such as finding a satisfying assignment to a given (satisfiable) propositional
formula or finding a 3-coloring of a given (3-colorable) graph.

• But it has established that these two seemingly different computational tasks
are in some sense the same (or, more precisely, are computationally
equivalent).
 Cont.…
• In general, complexity theory is concerned with problems the solutions of
which are implicit in the problem's statement.
• That is, the problem contains all necessary information, and one merely
needs to process this information in order to supply the answer.
• Thus, complexity theory is concerned with manipulation of information, and its
transformation from one representation (in which the information is given) to
another representation (which is the one desired).
• Indeed, a solution to a computational problem is merely a different
representation of the information given; that is, a representation in which the
answer is explicit rather than implicit.
• For example, the answer to the question of whether or not a given Boolean
formula is satisfiable is implicit in the formula itself (but the task is to make the
answer explicit).
 Cont.…
• Thus, complexity theory clarifies a central issue regarding representation;
that is, the distinction between what is explicit and what is implicit in a
representation. Furthermore, it even suggests a quantification of the level of
non-explicitness.
• In general, complexity theory provides new viewpoints on various
phenomena that were considered also by past thinkers.
• Examples include the aforementioned concepts of proofs and representation
as well as concepts like randomness, knowledge, interaction, secrecy and
learning. We next discuss some of these concepts and the perspective offered
by complexity theory.

5
 Cont.…
• Complexity theory is the body of knowledge concerning fundamental
principles of computation. Its beginnings can be traced way back in history to
the use of asymptotic complexity and reducibility by the Babylonians.
• Modern complexity theory is the result of research activities in many different
fields:
• biologists studying models for neuron nets or evolution, electrical engineers
developing switching theory as a tool to hardware design, mathematicians
working on the foundations of logic and arithmetic’s, linguists investigating
grammars for natural languages, physicists studying the implications of
building Quantum computers, and last but not least, computer scientists
searching for efficient algorithms for hard problems.
Cont.…
• complexity theory is related to
• Randomness
• Recursive function
• Complexity classes L,P,NP, Pspace
• zero-knowledge interactions
• Cryptography
• probabilistically checkable proofs
• Approximation
• etc.
 summary
• Complexity theory is concerned with manipulation of information
• A solution to a computational problem is a different representation of the information
• A representation in which the answer is explicit rather than implicit
• The problem contains all necessary information
• Process the information in order to supply the answer
• E.g. the answer to “is a given Boolean formula satisfiable” is implicit in the formula itself and the
task is to make the answer explicit
• Failed to establish lower bounds on resources
• Showed that many problems computationally equivalent
• All of them have efficient algorithms or all of them do not
• E.g. failed to determine complexity of finding satisfying assignment of boolean formula
(SAT) or 3COL
• In contrast, established that these problems are computationally equivalent
• Study the resources required to solve computational tasks
• time, space(memory)
• Understanding relations between complexity phenomena
• Provides new perspective on various concepts
 Turing Machine(TM)
Turing Machine
• TMs are an abstract model of computation. They provide a precise formal definition
of what it means for a function to be computable.
• Turing describes a machine that has an infinitely long tape upon which it writes,
reads and alters symbols.
• He further shows that a machine with the correct minimal set of operations can
calculate anything that is computable, no matter the complexity.

• While it may seem as if the tape is merely the input and output of the machine, it is
not! Nor is the tape just the memory of the machine. In a way the tape is the
computer.
• As the symbols on the tape are manipulated by simple rules, the computing
happens. The output is really more of an artifact of the machine using the tape as the
computer.
10
 Cont..
• The heart of the Turing machine is the read-write head. The read-write head
transports the tape and positions cells of the tape appropriately.
• It can read a cell determining what, if any, symbol is written there.
• The machine works on, and knows about, only one cell at a time.
• The Turing machine is not intended to be a functional computing technology;
instead, it is intended as a hypothetical machine that represents a computing
machine.

11
 Cont.…
• Turing machines mathematically model a device that mechanically runs using a tape. This
tape includes symbols, which the machine can write and read, one after the other, with the
help of a tape head.
• More specifically, a Turing machine includes the following:

• Tape: A tape that is split into cells, one beside the other. Every cell includes a symbol
from a certain finite alphabet. The alphabet includes a unique blank symbol as well as
one or more other symbols. The volume of tape required for the computation is always
included in the Turing machine.
• Head: A head that is able to write and read symbols on the tape. In certain models, the
head moves while the tape is fixed.
• State register: A state register to store the Turing machine's state. There is a special
start state through which the state register is initialized.
• Finite table: A finite table (sometimes referred to as a transition function or an action
table) of instructions, which are generally quintuples, but occasionally quadruples.
Complex T GS CS 15
Complex T GS CS 16
 Turing Machine: Formal Definition
Definition (Turing Machine)
A Turing machine M for a seven-tuple model is given by M=(Q, Γ, B, Σ, q0, F, δ) where

Q is a finite, non-empty set of states

Γ is a finite, non-empty set of the tape alphabet/symbols
B belongs to Γ is the blank symbol (the only symbol allowed to occur on the tape
infinitely often at any step during the computation)
Σ is a subset of or equal to Γ\{b}, is the set of input symbols
q0 belongs to Q, is the initial state
F is a subset or equal to Q, is the set of final or accepting states.
δ is the transition function, δ: Q\F × Σ → Γ ×{L,N,R} × Q\F
 construction Techniques of Turing machine
• Deals with the rules of operations
1. Start in state q0, head over its first symbol
2. At each step of the computation
✓Read the current symbol
✓Update (write) the same symbol
✓Move exactly one cell exactly either to the left or right
3. Stop as soon as qF, left on tape: out put
Rules of operation- 2
• Control is related with a sort of FSM or PDA
• Initial state
• Final state can either be accept state or reject state
• Computation can either be
1. HALT and ACCEPT
2. HALT and REJECT
3. LOOP (the machine fails to HALT)
Turing machine Vs real machines
• TMs are able to execute any operation that a real program can.
• A real machine can only be in finitely many configuration. On the
other hand, Turing machines are equivalent to machines that have an
unlimited amount of storage space for their computations.
• TMs are intended to model computation itself
History
• They were described in 1936 by Alan Turing.
• Robin Gandy (1919–1995)—a student of Alan Turing traces the
lineage of the notion of “calculating machine” and proposes
“Babbage's Thesis":
• Gandy's analysis of Babbage's Analytical Engine describes the
following five operations
• The arithmetic functions +, −, × where − indicates "proper" subtraction x − y = 0 if y ≥ x
• Any sequence of operations is an operation
• Iteration of an operation (repeating n times an operation P)
• Conditional iteration (repeating n times an operation P conditional on the "success" of test T)
• Conditional transfer (i.e. conditional "goto")
 Church-Turing thesis
• The Church-Turing thesis states the equivalence between the mathematical
concepts of algorithm or computation and Turing-Machine.

• It asserts that if some calculation is effectively carried out by an algorithm, then

there exists a Turing machines which will compute that calculation.

• The notion of algorithm, computation, a step-by-step procedure or a defined method

to perform calculations has been used informally and intuitively in mathematics for
centuries.
• However, attempts to formalize the concept only begun in the beginning of the 20th
century.
 Cont.…
• However, it has gain widely acceptance in the mathematical and
philosophical community.
• The thesis has been wrongly attributed to many controversial claims in
philosophy, that although related are not implied in the original thesis.
Some examples are:
• The universe is equivalent to a Turing Machine and non-computable
functions are physically impossible.
• The universe isn’t equivalent to a Turing Machine and incomputable.
• The human mind is a Turing Machine, the human mind and/or
consciousness are equivalent to and can be instantiated by a computer.
• The human mind isn’t a Turing Machine, the human mind and/or
consciousness emerge due to the existence of incomputable process,
such as microtubules performing quantum process in the brain.
 Summery on The Church-Turing thesis
• The Church-Turing thesis concerns the notion of an effective or mechanical method
in logic and mathematics.
• 'Effective' and its synonym 'mechanical' are terms of art in these disciplines: they do
not carry their everyday meaning. A method, or procedure, M, for achieving some
desired result is called 'effective' or 'mechanical' just in case M is set out in terms of
a finite number of exact instructions (each instruction being expressed by means of
a finite number of symbols).
• M will, if carried out without error, produce the desired result in a finite number of
steps.
• M can (in practice or in principle) be carried out by a human being unaided by any
machinery save paper and pencil.
• M demands no insight or ingenuity on the part of the human being carrying it out
Computability of Turing machine
• Universal Turing Machine
• Alternating Turing Machine
• Quantum Turing machine (QTM)
• Read-only Turing Machine
• Read-only Right Turing Machine
• Multi-tape Turing Machine
 Big-O Notation
• Big-O Notation gives an upper bound to the growth of a function as n increases -
ignoring any coefficient or constants etc. Essentially the same as when you're
taking the limit of a function to infinity, you can disregard any “small" terms, we
do the same here.
• We use it to determine the order of the worst case runtime of an algorithm in terms
of the number of basic steps required and with respect to the size of the input,
denoted n.
• We also have big-omega, big-theta, and little-w notation which mean similar but
slightly different things.
• Common orders include:
• O(k)(constant) - e.g. determining if binary number is odd
• O(log n) (logarithmic) - e.g. binary search
• O(n) (linear) - e.g. finding an item in an unsorted list
• O(n log n) (linearithmic or quasilinear) - e.g. quick sort
• O(nk)(polynomial) - e.g. bubble and shuffle sort
• O(kn) (exponential) - e.g. solving TSP (but brute-force method is factorial!)
 (Big–O, O ()) VS (Little–o, o ()):
• Big-O (O ()) is one of five standard asymptotic notations. In practice, Big-O is used
as a tight upper-bound on the growth of an algorithm’s effort (this effort is
described by the function f (n)), even though, as written, it can also be a loose
upper-bound.
• To make its role as a tight upper-bound more clear, “Little-o” (o ()) notation is used
to describe an upper-bound that cannot be tight.
• Definition (Big–O, O ()): Let f (n) and g (n) be functions that map positive integers
to positive real numbers. We say that f(n) is O(g(n)) (or f(n) ∈ O(g(n))) if there exists
a real constant c > 0 and there exists an integer constant n0= 1 such that f(n) = c *
g(n) for every integer n = n.
• Definition (Little–o, o ()): Let f (n) and g (n) be functions that map positive integers
to positive real 0 numbers. We say that f(n) is o(g(n)) (or f(n) ∈ o(g(n))) if for any real
constant c > 0, there exists an integer constant n0= 1 such that f(n) < c * g(n) for
every integer n = n0.
• On the other side of f (n), it is convenient to define parallels to O () and o () that
provide tight and loose lower bounds on the growth of f (n).
Complex T GS CS 20
 Cont.…
• Floors and ceilings
For any real number x, the greatest integer less than or equal to x is denoted by x .
For any real number x, the least integer greater than or equal to x is denoted by x .
For all real numbers x,
x 1< x x x < x+1.