Some Pre-requisites and

UNIT 2 SOME PRE-REQUISITES AND Asymptotic Bounds

ASYMPTOTIC BOUNDS
Structure Page Nos.
2.0 Introduction 41
2.1 Objectives 41
2.2 Some Useful Mathematical Functions &Notations 42
2.2.1 Functions & Notations
2.2.2 Modular Arithmetic/Mod Function
2.3 Mathematical Expectation 49
2.4 Principle of Mathematical Induction 50
2.5 Concept of Efficiency of an Algorithm 52
2.6 Well Known Asymptotic Functions & Notations 56
2.6.1 Enumerate the Five Well-Known Approximation Functions
and How These are Pronounced
2.6.2 The Notation O
2.6.3 The Ω Notation
2.6.4 The Notation Θ
2.6.5 The Notation o
2.6.6 The Notation ω
2.7 Summary 66
2.8 Solutions/Answers 67
2.9 Further Readings 70

2.0 INTRODUCTION
We have already mentioned that there may be more than one algorithms, that solve a
given problem. In Section 3.3, we shall discuss eight algorithms to sort a given list of
numbers, each algorithm having its own merits and demerits. Analysis of algorithms,
The understanding of the
the basics of which we study in Unit 3, is an essential tool for making well-informed theory of a routine may be
decision in order to choose the most suitable algorithm, out of the available ones if greatly aided by providing,
any, for the problem or application under consideration. at the time of construction
one or two statements
concerning the state of the
A number of mathematical and statistical tools, techniques and notations form an machine at well chose
essential part of the baggage for the analysis of algorithms. We discuss some of these points…In the extreme form
tools and techniques and introduce some notations in Section 2.2. However, for of the theoretical method a
detailed discussion of some of these topics, one should refer to the course material of watertight mathematical
MCS-013. proof is provided for the
assertions. In the extreme
form of the experimental
Also, in this unit, we will study a number of well-known approximation functions. method the routine is tried
These approximation functions which calculate approximate values of quantities out one the machine with a
under consideration, prove quite useful in many situations, where some of the variety of initial conditions
and is pronounced fit if the
involved quantities are calculated just for comparison with each other. And the assertions hold in each case.
correct result of comparisons of the quantities can be obtained even with approximate Both methods have their
values of the involved quantities. In such situations, the advantage is that the weaknesses.
approximate values may be calculated much more efficiently than can the actual
A.M. Turing
values. Ferranti Mark 1
Programming Manual (1950)

2.1 OBJECTIVES
After going through this Unit, you should be able to:

• use a number of mathematical notations e.g.,

∑, ∏ , ⎣ ⎦, ⎡ ⎤, mod, log, e etc.
41
Introduction to
Algorithmics
• define and use a number of concepts like function, 1-1 function, onto function,
monotonic function, floor and ceiling functions, mod function, exponentiation,
logarithm functions etc
• define and use Mathematical Expectation
• Use of Principle of Mathematical Induction in establishing truth of infinitely
many statements and

• define and use the five asymptotic function viz.,

(i) O: (O (n2) is pronounced as ‘big-oh of n2’ or sometimes just as oh of n2)

(ii) Ω: (Ω (n2 ) is pronounced as ‘big-omega of n2 or sometimes just as

omega of n2’)

(iii) Θ: (Θ (n2) is pronounced as ‘theta of n2’)

(iv) o: (o (n2) is pronounced as ‘little-oh of n2’)

(v) ω: (ω (n2) is pronounced as ‘little-omega of n2’).

2.2 SOME USEFUL MATHEMATICAL

FUNCTIONS & NOTATIONS
Let us start with some mathematical definitions.

2.2.1 Functions & Notations

Just to put the subject matter in proper context, we recall the following notations and
definitions.

Unless mentioned otherwise, we use the letters N, I and R in the following sense:

N = {1, 2, 3, …}
I = {…, ─ 2, ─, 0, 1, 2, ….}
R = set of Real numbers.

Notation 2.2.1.1: If a1, a2…an are n real variables/numbers

then

(i) Summation:

The expression
a1 + a2+ …+ai+…+an
may be denoted in shorthand as
n

∑a
i =1
i

(ii) Product

The expression
a1 × a2 × … × ai × … × an
may be denoted in shorthand as
n

∏a
i =1
i

42
Definition 2.2.1.2: Some Pre-requisites and
Asymptotic Bounds
Function:
For two given sets A and B (which need not be distinct, i.e., A may be the same as B)
a rule f which associates with each element of A, a unique element of B, is called a
function from A to B. If f is a function from a set A to a set B then we denote the fact
by f: A → B. Also, for x ε A, f(x) is called image of x in B. Then, A is called the
domain of f and B is called the Codomain of f.

Example 2.2.1.3:
Let f: I → I be defined such that
f(x) = x2 for all x ε I
Then
f maps ─ 4 to 16
f maps 0 to 0
f map 5 to 25

Remark 2.2.1.4:
We may note the following:

(i) if f: x → y is a function, then there may be more than one elements, say x1
and x2 such that
f(x1) = f(x2)
For example, in the Example 2.2.1.3
f(2) = f(─2) = 4

By putting restriction that f(x) ≠ f(y) if x ≠ y, we get special functions, called

1-1 or injective functions and shall be defined soon.

(ii) Though for each element x ε X, there must be at least one element y ε Y
s.t f(x) = y. However, it is not necessary that for each element y ε Y,
there must be an element x ε X such that f(x) = y. For example, for
y = ─ 3 ε Y there is no x ε X s.t f(x) = x2 = ─ 3.

By putting the restriction on a function f, that for each y ε Y, there must be at least
one element x of X s.t f(x) = y, we get special functions called onto or surjective
functions and shall be defined soon.

Next, we discuss some important functions.

Definition 2.2.1.5:

1-1 or Injective Function: A function f: A → B is said to 1-1 ∗ or injective if

for x, y ε A, if f(x) = f(y) then x = y

We have already seen that the function defined in Example 2.2.1.3 is not 1-1.
However, by changing the domain, through defined by the same rule, f becomes a
1-1 function.

Example 2.2.1.2:
In this particular case, if we change the domain from I to N = {1,2,3…} then we can
easily check that function

∗
Some authors write 1-to-1 in stead of 1-1. However, other authors call a function 1-to-1 if
f is both 1-1 and onto (to be defined 0 in a short while). 43
Introduction to f:N→I defined as
Algorithmics f(x) = x2, for all x ε N,

is 1-1.

Because, in this case, for each x ε N its negative ─ x ∉ N. Hence for f(x) = f(y)
implies x = y. For example, If f(x) = 4 then there is only one value of x, viz, x = 2 s.t
f(2) = 4.

Definition 2.2.1.7:

Onto/Surjective function: A function f: X → Y is said to onto, or surjective if to

every element of Y, the codomain of f, there is an element x ε X s.t f(x) = y.

We have already seen that the function defined in Example 2.2.1.3 is not onto.

However, in this case either, by changing the codomain Y or changing the rule, (or
both) we can make f as Onto.

Example 2.2.1.8: (Changing the domain)

Let X = I = {… ─ 3, ─ 2, ─1, 0, 1, 2, 3, …}, but, we change Y as
Y = {0, 1, 4, 9, …} = {y⏐ y = n2 for n ε X}
then it can be seen that
f: X→ Y defined by
f(x) = x2 for all x ε X is Onto

Example 2.2.1.9: (Changing the rule)

Here, we change the rule so that X = Y = {… ─ 3, ─2, ─1, 0, 1,2, 3…}
But f: X → Y is defined as
F(x) = x + 3 for x ε X.

Then we apply the definition to show that f is onto.

If y ε Y, then, by definition, for f to be onto, there must exist an x ε X such that

f(x) = y. So the problem is to find out x ε X s.t f(x) = y (the given element of the
codomain y).

Let us assume that x ε X exists such that

f(x) = y
i.e., x+3=y
i.e., x=y─3

But, as y is given, x is known through the above equation. Hence f is onto.

Definition 2.2.1.10:
Monotonic Functions: For the definition of monotonic functions, we consider
only functions
f: R → R
where, R is the set of real numbers ∗ .

∗
Monotonic functions
f : X → Y,
may be defined even when each of X and Y, in stead of being R, may be any ordered sets.
44 But, such general definition is not required for our purpose.
A function f: R → R is said to be monotonically increasing if for x, y ε R and x ≤ y Some Pre-requisites and
Asymptotic Bounds
we have f(x) ≤ f(y).

In other words, as x increases, the value of its image f(x) also increases for a
monotonically increasing function.

Further, f is said to be strictly monotonically increasing, if x < y then f(x) < f(y)

Example 2.2.1.11:

Let f: R → R be defined as f(x) = x + 3, for x ε R

Then, for x1, x2 ε R, the domain, if x1 ≥ x2 then x1 + 3 ≥ x2 + 3, (by using monotone

property of addition), which implies f(x1) ≥ f(x2). Hence, f is monotonically
increasing.

We will discuss after a short while, useful functions called Floor and Ceiling
functions which are monotonic but not strictly monotonic.

A function f: R → R is said to be monotonically decreasing, if, for x, y ε R and x ≤ y

then f(x) ≥ f(y).
In other words, as x increases, value of its image decreases.

Further, f is said to be strictly monotonically decreasing, if x < y then f(x) > f(y).

Example 2.2.1.12:

Let f: R → R be defined as
F(x) = ─ x + 3
if x1 ≥ x2 then ─ x1 ≤ ─ x2 implying ─ x1+3 ≤ ─ x2 + 3,
which further implies f(x1) ≤ f(x2)
Hence, f is monotonically decreasing.

Next, we define Floor and Ceiling functions which map every real number to an
integer.

Definition 2.2.1.13:
Floor Function: maps each real number x to the integer, which is the greatest of all
integers less than or equal to x. Then the image of x is denoted by ⎣ x ⎦.

Instead of ⎣ x ⎦, the notation [x] is also used.

For example: ⎣ 2.5 ⎦ = 2, ⎣ ─ 2.5 ⎦ = ─ 3, ⎣ 6 ⎦ = 6.

Definition 2.2.1.14:
Ceiling Function: maps each real number x to the integer, which is the least of all
integers greater than or equal to x. Then the image of x is denoted by ⎡x ⎤.

For example: ⎡2.5⎤ = 3, ⎡─2.5 ⎤ = ─ 2, ⎡6 ⎤ = 6

Next, we state a useful result, without proof.

Result 2.1.1.10: For every real number x, we have

x ─ 1 < ⎣ x ⎦ ≤ x ≤ ⎡x ⎤ < x + 1.

45
Introduction to Example 2.2.1.15:
Algorithmics
Each of the floor function and ceiling function is a monotonically increasing function
but not strictly monotonically increasing function. Because, for real numbers x and y,
if x ≤ y then y = x + k for some k ≥ 0.

⎣ y⎦ = ⎣ x + k⎦ = integral part of (x + k) ≥ integral part of x = ⎣ x ⎦

Similarly
⎡y⎤ = ⎡x + k⎤ = least integer greater than or equal to x + k ≥ least integer
greater than or equal to x = ⎡x⎤.
But, each of floor and ceiling function is not strictly increasing, because

⎣ 2.5⎦ = ⎣ 2.7⎦ = ⎣ 2.9⎦ = 2

and
⎡2.5⎤ = ⎡2.7⎤ = ⎡2.9⎤ = 3
2.2.2 Modular Arithmetic/Mod Function
We have been implicitly performing modular arithmetic in the following situations:
(i) If, we are following 12-hour clock, (which usually we do) and if it
11 O’clock now then after 3 hours, it will be 2 O’clock and not 14 O’clock
(whenever the number of o’clock exceeds 12, we subtract n = 12 from the number)

(ii) If, it is 5th day (i.e., Friday) of a week, after 4 days, it will be 2nd day
(i.e., Tuesday) and not 9th day, of course of another, week (whenever the number of
the day exceeds 7, we subtract n = 7 from the number, we are taking here Sunday as 7th day, in
stead of 0th day)

(iii) If, it is 6th month (i.e., June) of a year, then after 8 months, it will be 2nd month
(i.e., February) of, of course another, year ( whenever, the number of the month exceeds
12, we subtract n = 12)

In general, with minor modification, we have

Definition 2.2.2.1:
b mod n: if n is a given positive integer and b is any integer, then

b mod n = r where 0≤r<n

and b=k*n+r

In other words, r is obtained by subtracting multiples of n from b so that the remainder

r lies between 0 and (n ─ 1).

For example: if b = 42 and n = 11 then

b mod n = 42 mod 11 = 9.

If b = ─ 42 and n = 11 then
b mod n = ─ 42 mod 11 = 2 (Θ ─ 42 = (─ 4) × 11 + 2)

Mod function can also be expressed in terms of the floor function as follows:

b (mod n) = b ─ ⎣ b/n⎦ × n

Definition 2.2.2.2:
Factorial: For N = {1,2,3,…}, the factorial function
factorial: N ∪ {0} → N∪ {0}
46 given by
 factorial (n) = n × factorial (n ─ 1) Some Pre-requisites and
Asymptotic Bounds
has already been discussed in detail in Section 1.6.3.2.

Definition 2.2.2.3:
Exponentiation Function Exp: is a function of two variables x and n where x is any
non-negative real number and n is an integer (though n can be taken as non-integer
also, but we restrict to integers only)

Exp (x, n) denoted by xn, is defined recursively as follows:

For n = 0

Exp (x, 0) = x0 = 1
For n > 0
Exp (x, n) = x × Exp (x, n ─ 1)
i.e
xn = x × xn-1
For n < 0, let n = ─ m for m > 0
1
xn = x-m =
xm
In xn, n is also called the exponent/power of x.
For example: if x = 1.5, n = 3, then

Also, Exp (1.5, 3) = (1.5)3 = (1.5) × [(1.5)2] = (1.5) [1.5 × (1.5)1]

= 1.5 [ (1.5 × (1.5 × (1.5)0))]
= 1.5[(1.5 × (1.5 × 1))] = 1.5 [(1.5 × 1.5)]
= 1.5 [2.25] = 3.375
1 1
Exp (1.5, ─ 3) = (1.5)─3 = =
(1.5) 3
3.375
Further, the following rules apply to exponential function.

For two integers m and n and a real number b the following identities hold:

((b) ) m n
= bmn
(b ) m n
= (b )n m

bm . bn = bm+n

For b ≥ 1 and for all n, the function bn is monotonically increasing in n. In other

words, if n1 ≥ n2 then b n1 ≥ b n 2 if b ≥ 1.

Definition 2.2.2.4:
Polynomial: A polynomial in n of degree k, where k is a non-negative integer, over
R, the set of real numbers, denoted by P(n), is of the form

Pk(n) = ak nk + ak-1 nk-1 + …+ a1 n + a0,

where ak ≠ 0 and ai ε R, i = 0, 1, …, k.
Using the summation notation
k
Pk(n) = ∑a
i+0
i ni ak ≠ 0, ai ε R
47
Introduction to Each of (ai ni) is called a term.
Algorithmics
Generally the suffix k in Pk(n) is dropped and in stead of Pk(n) we write P(n) only

We may note that P(n) = nk = 1.nk for any k, is a single-term polynomial. If k ≥ 0 then
P(n) = nk is monotonically increasing. Further, if k ≤ 0 then p(n) = nk is
monotonically decreasing.

Notation: Though 0o is not defined, yet, unless otherwise mentioned, we will

o
take 0 = 1. The following is a very useful result relating the exponentials and
polynomials

Result 2.2.2.5: For any constants b and c with b > 1

nc
Lim n = 0
n →∞ b

The result, in non-mathematical terms, states that for any given constants b and c, but
1c 2 c 3c kc
with b > 1, the terms in the sequence 1 , 2 , 3 , ..., k , .... gradually decrease and
b b b b
approaches zero. Which further means that for constants b and c, and integer
variable n, the exponential term bn, for b > 1, increases at a much faster rate than
the polynomial term nc.

Definition 2.2.2.6:
The letter e is used to denote the quantity

1 1 1
1+ + + + ......,
1! 2! 3!
and is taken as the base of natural logarithm function, then for all real numbers x,

we define the exponential function

x2 x3 ∞
xi
ex = 1 + x + +
2! 3!
+ .... = ∑
i = 0 i!

For, all real numbers, we have ex ≥ 1 + x

Further, if ⏐ x⏐ ≤ 1 then 1+x ≤ ex ≤ 1 + x+ x2

The following is another useful result, which we state without proof:

n
⎛ x⎞
Result 2.2.2.7: Lim ⎜1 + ⎟ = e x
n →∞ ⎝ n ⎠

Definition 2.2.2.8:
Logarithm: The concept of logarithm is defined indirectly through the definition of
Exponential defined earlier. If a > 0, b > 0 and c > 0 are three real numbers, such that

c = ab

Then b = loga c (read as log of c to the base a)

Then a is called the base of the algorithm.
For example: if 26 = 64, the log264 = 6,

48 i.e., 2 raised to power 6 gives 64.

Generally two bases viz 2 and e are very common in scientific and computing Some Pre-requisites and
Asymptotic Bounds
fields and hence, the following specials notations for these bases are used:

(i) lg n denotes log2n (base 2)

(ii) ln n denotes logen (base e);
where the letter l in ln denotes logarithm and the letter n in ln denotes natural.

The following important properties of logarithms can be derived from the properties
of exponents. However, we just state the properties without proof.

Result 2.2.2.9:
For n, a natural number and real numbers a, b and c all greater than 0, the following
identities are true:
(i) loga (bc) = loga b+loga c
n
(ii) loga (b ) = n logab
(iii) logba = logab
(iv) loga (1/b) = ─ logba
1
(v) logab =
log b a

(vi) a log b c = c logb a

2.3 MATHEMATICAL EXPECTATION

In average-case analysis of algorithms, to be discussed in Unit 3, we need the concept

of Mathematical expectation. In order to understand the concept better, let us first
consider an example.

Example 2.1: Suppose, the students of MCA, who completed all the courses in the
year 2005, had the following distribution of marks.

Range of marks Percentage of students

who scored in the range

0% to 20% 08
20% to 40% 20
40% to 60% 57
60% to 80% 09
80% to 100% 06

If a student is picked up randomly from the set of students under consideration, what
is the % of marks expected of such a student? After scanning the table given above,
we intuitively expect the student to score around the 40% to 60% class, because, more
than half of the students have scored marks in and around this class.

Assuming that marks within a class are uniformly scored by the students in the class,
the above table may be approximated by the following more concise table:

49
Introduction to % marks Percentage of students scoring the marks
Algorithmics
10 ∗ 08
30 20
50 57
70 09
90 06

As explained earlier, we expect a student picked up randomly, to score around 50%

because more than half of the students have scored marks around 50%.
This informal idea of expectation may be formalized by giving to each percentage of
marks, weight in proportion to the number of students scoring the particular
percentage of marks in the above table.

Thus, we assign weight (8/100) to the score 10% (Θ 8, out of 100 students, score on
the average 10% marks); (20/100) to the score 30% and so on.

Thus
8 20 57 9 6
Expected % of marks = 10 × + 30 × + 50 × + 70 × + 90 × = 47
100 100 100 100 100

The final calculation of expected marks of 47 is roughly equal to our intuition of the
expected marks, according to our intuition, to be around 50.

We generalize and formalize these ideas in the form of the following definition.

Mathematical Expectation
For a given set S of items, let to each item, one of the n values, say, v1, v2,…,vn, be
associated. Let the probability of the occurrence of an item with value vi be pi. If an
item is picked up at random, then its expected value E(v) is given by

n
E(v) = ∑pv
i −1
i i = p1. v1 + p 2 . v 2 + ........ p n . vn

2.4 PRINCIPLE OF MATHEMATICAL

INDUCTION
Frequently, in establishing the truth of a set of statements, where each of the
statement is a function of a natural number n, we will be using the Principle of
Mathematical Induction. In view of the significance of the principle, we discuss it
here briefly.

The principle is quite useful in establishing the correctness of a countably many

infinite number of statements, through only finite number of steps.

The method consists of the following three major steps:

1. Verifying/establishing the correctness of the Base Case
2. Assumption of Induction Hypothesis for the given set of statements
3. Verifying/Establishing the Induction Step

We explain the involved ideas through the following example:

50 ∗
10 is the average of the class boundaries 0 and 20.
Let us consider the following sequence in which nth term S(n) is the sum of first Some Pre-requisites and
Asymptotic Bounds
(n─1) powers of 2, e.g.,

S(1) = 20 =2─1
S(2) = 20 + 21 = 22 ─ 1
S(3) = 20 + 21 + 22 = 23 ─ 1

We (intuitively) feel that

S(n) = 20 + 21 +….+ 2n-1 should be 2n ─ 1 for all n ≥ 1.

We may establish the correctness of the intuition, i.e., correctness of all the infinite
number of statements

S(n) = 2n─1 for all n ≥ 1,

through only the following three steps:
(i) Base Case: (In this example, n = 1 ) we need to show that
S(1) = 21 ─ 1 = 1

But, by definition S(1) = 20 = 1 = 2 ─ 1 = 21─ 1 is correct

(ii) Induction Hypothesis: Assume, for some k > base-value (=1, in this case)
that
S(k) = 2k ─ 1.

(iii) Induction Step: Using (i) & (ii) establish that (in this case)

S(k+1) = 2k+1 ─ 1

In order to establish
S(k+1) = 2k+1 ─ 1, (A)
we use the definition of S(n) and Steps (i) and (ii) above

By definition

S(k+1) = 20 + 21+…+2k+1-1
= (20 + 21+…+2k-1) + 2k (B)

But by definition
20 + 21+…+2k-1 = S(k). (C)

Using (C) in (B) we get

S(k+1) = S(k) + 2k (D)

and by Step (ii) above

S(k) = 2k ─ 1 (E)

Using (E) in (D), we get

S(k+1) = (2k ─ 1) + 2k
∴ S(k+1) = 2.2k ─ 1 = 2k+1 ─ 1
which establishes (A).

Ex.1) By using Principle of Mathematical Induction, show that 6 divides n3 ─ n,

where n is a non-negative integer.

Ex.2) Let us assume that we have unlimited supply of postage stamps of Rs. 5 and
Rs. 6 then
51
Introduction to (i) through, direct calculations, find what amounts can be realized in terms
Algorithmics of only these stamps.
(ii) Prove, using Principle of Mathematical Induction, the result of your
efforts in part (i) above.

2.5 CONCEPT OF EFFICIENCY OF AN

ALGORITHM
If a problem is algorithmically solvable then it may have more than one algorithmic
solutions. In order to choose the best out of the available solutions, there are criteria
for making such a choice. The complexity/efficiency measures or criteria are based
on requirement of computer resources by each of the available solutions. The solution
which takes least resources is taken as the best solution and is generally chosen for
solving the problem. However, it is very difficult to even enumerate all possible
computer resources e.g., time taken by the designer of the solution and the time taken
by the programmer to encode the algorithm.

Mainly the two computer resources taken into consideration for efficiency measures,
are time and space requirements for executing the program corresponding to the
solution/algorithm. Until it is mentioned otherwise, we will restrict to only time
complexities of algorithms of the problems.

In order to understand the complexity/efficiency of an algorithm, it is very important

to understand the notion of the size of an instance of the problem under consideration
and the role of size in determining complexity of the solution.

It is easy to realize that given an algorithm for multiplying two n × n matrices, the
time required by the algorithm for finding the product of two 2 × 2 matrices, is
expected to take much less time than the time taken by the same algorithm for
multiplying say two 100 × 100 matrices. This explains intuitively the notion of the
size of an instance of a problem and also the role of size in determining the (time)
complexity of an algorithm. If the size (to be later considered formally) of general
instance is n then time complexity of the algorithm solving the problem (not just
the instance) under consideration is some function of n.

In view of the above explanation, the notion of size of an instance of a problem plays
an important role in determining the complexity of an algorithm for solving the
problem under consideration. However, it is difficult to define precisely the concept
of size in general, for all problems that may be attempted for algorithmic solutions.

Formally, one of the definitions of the size of an instance of a problem may be taken
as the number of bits required in representing the instance.

However, for all types of problems, this does not serve properly the purpose for which
the notion of size is taken into consideration. Hence different measures of size of an
instance of a problem, are used for different types of problem. For example,

(i) In sorting and searching problems, the number of elements, which are to be
sorted or are considered for searching, is taken as the size of the instance of
the problem of sorting/searching.
(ii) In the case of solving polynomial equations or while dealing with the algebra
of polynomials, the degrees of polynomial instances, may be taken as the
sizes of the corresponding instances.

There are two approaches for determining complexity (or time required) for executing
an algorithm, viz.,
52
(i) empirical (or a posteriori) and Some Pre-requisites and
Asymptotic Bounds
(ii) theoretical (or a priori).

In the empirical approach (the programmed) algorithm is actually executed on

various instances of the problem and the size (s) and time (t) of execution for each
instance is noted. And then by some numerical or other technique, t is determined as
a function of s. This function then, is taken as complexity of the algorithm under
consideration.

In the theoretical approach, we mathematically determine the time needed by the

algorithm, for a general instance of size, say, n of the problem under consideration.
In this approach, generally, each of the basic instructions like assignment, read and
write and each of the basic operations like ‘+’, comparison of pair of integers etc., is
assumed to take one or more, but some constant number of, (basic) units of time for
execution. Time for execution for each structuring rule, is assumed to be some
function of the times required for the constituents of the structure.
Thus starting from the basic instructions and operations and using structuring rules,
one can calculate the time complexity of a program or an algorithm.

The theoretical approach has a number of advantages over the empirical approach
including the ones enumerated below:
(i) The approach does not depend on the programming language in which the
algorithm is coded and on how it is coded in the language,
(ii) The approach does not depend on the computer system used for executing (a
programmed version of) the algorithm.
(iii) In case of a comparatively inefficient algorithm, which ultimately is to be
rejected, the computer resources and programming efforts which otherwise
would have been required and wasted, will be saved.
(iv) In stead of applying the algorithm to many different-sized instances, the
approach can be applied for a general size say n of an arbitrary instance of the
problem under consideration. In the case of theoretical approach, the size n
may be arbitrarily large. However, in empirical approach, because of
practical considerations, only the instances of moderate sizes may be
considered.

Remark 2.5.1:
In view of the advantages of the theoretical approach, we are going to use it as
the only approach for computing complexities of algorithms. As mentioned earlier,
in the approach, no particular computer is taken into consideration for calculating time
complexity. But different computers have different execution speeds. However, the
speed of one computer is generally some constant multiple of the speed of the other.
Therefore, this fact of differences in the speeds of computers by constant
multiples is taken care of, in the complexity functions t for general instance sizes
n, by writing the complexity function as c.t(n) where c is an arbitrary constant.

An important consequence of the above discussion is that if the time taken by one
machine in executing a solution of a problem is a polynomial (or exponential)
function in the size of the problem, then time taken by every machine is a polynomial
(or exponential) function respectively, in the size of the problem. Thus, functions
differing from each other by constant factors, when treated as time complexities
should not be treated as different, i.e., should be treated as complexity-wise
equivalent.

53
Introduction to Remark 2.5.2:
Algorithmics
Asymptotic Considerations:
Computers are generally used to solve problems involving complex solutions. The
complexity of solutions may be either because of the large number of involved
computational steps and/or because of large size of input data. The plausibility of the
claim apparently follows from the fact that, when required, computers are used
generally not to find the product of two 2 × 2 matrices but to find the product of two
n × n matrices for large n, running into hundreds or even thousands.
Similarly, computers, when required, are generally used not only to find roots of
quadratic equations but for finding roots of complex equations including polynomial
equations of degrees more than hundreds or sometimes even thousands.

The above discussion leads to the conclusion that when considering time complexities
f1(n) and f2(n) of (computer) solutions of a problem of size n, we need to consider and
compare the behaviours of the two functions only for large values of n. If the relative
behaviours of two functions for smaller values conflict with the relative behaviours
for larger values, then we may ignore the conflicting behaviour for smaller values.
For example, if the earlier considered two functions

f1(n) = 1000 n2 and

f2(n) = 5n4

represent time complexities of two solutions of a problem of size n, then despite the
fact that
f1 (n) ≥ f2 (n) for n ≤ 14,
we would still prefer the solution having f1 (n) as time complexity because

f1(n) ≤ f2 (n) for all n ≥ 15.

This explains the reason for the presence of the phrase ‘n ≥ k’ in the definitions
of the various measures of complexities and approximation functions, discussed
below:

Remark 2.5.3:
Comparative Efficiencies of Algorithms: Linear, Quadratic, Polynomial
Exponential
Suppose, for a given problem P, we have two algorithms say A1 and A2 which solve
the given problem P. Further, assume that we also know time-complexities T1(n) and
T2 (n) of the two algorithms for problem size n. How do we know which of the two
algorithms A1 and A2 is better?

The difficulty in answering the question arises from the difficulty in comparing time
complexities T1(n) and T2(n).

For example, let T1(n) = 1000n2 and T2(n) = 5n4

Then, for all n, neither T1(n) ≤ T2(n) nor T2(n) ≤ T1 (n).

More explicitly

T1 (n) ≥ T2 (n) for n ≤ 14 and

T1 (n) ≤ T2 (n) for n ≥ 15.

The issue will be discussed in more detail in Unit 3. However, here we may mention
that, in view of the fact that we generally use computers to solve problems of large
54
sizes, in the above case, the algorithms A1 with time-complexity T1 (n) = 1000n2 is Some Pre-requisites and
Asymptotic Bounds
preferred over the algorithm A2 with time-complexity T2 (n) = 5n4, because
T1 (n) ≤ T2(n) for all n ≥ 15.

In general if a problem is solved by two algorithms say B1 and B2 with time-B

complexities BT1(N) and BT2(n) respectively, then

(i) if BT1 (n) is a polynomial in n , i.e,

BT ( n ) = a k n k + a k −1 + ..... + a i n i + ... + a 1 n + a 0
1
for some k ≥ 0 with a ' s as real numbers and a k > 0, and
i

BT2 (n) is an exponential function of n, i.e., BT2 (n) is of the form

BT2 (n) = c an where c and a are some real numbers with a > 1,
then generally, for large values of n, BT1 (n ) ≤ BT2 (n ).
Hence, algorithm B1 with polynomial time complexity is assumed to the more
efficient and is preferred over algorithm B2 with exponential time complexity.

(ii) If, again a problem is solved by two algorithms D1 and D2 with respectively
polynomial time complexities DT1 and DT2 then if

degree (DT1) < degree (DT2),

then the algorithm D1 is assumed to be more efficient and is preferred over D2.

Certain complexity functions occur so frequently that special names commensurate

with their usage may be given to such functions. For example, complexity function
c n is called linear time complexity and corresponding algorithm, is called as linear
algorithm.

Similarly, the terms ‘quadratic’ and ‘polynomial time’ complexity functions and
algorithms are used when the involved complexity functions are respectively of the
forms c n2 and c1 nk +…….+ck.

In the next section we find examples of linear and quadratic algorithms.

Remark 2.5.4:
For all practical purposes, the use of c, in (c t(n)) as time complexity measure, offsets
properly the effect of differences in the speeds of computers. However, we need to be
on the guard, because in some rarely occurring situations, neglecting the effect of c
may be misleading.

For example, if two algorithms A1 and A2 respectively take n2 days and n3 secs for
execution of an instance of size n of a particular problem. But a ‘day’ is a constant
multiple of a ‘second’. Therefore, as per our conventions we may take the two
complexities as of C2 n2 and C3 n3 for some constants C2 and C3. As, we will discuss
later, the algorithm A1 taking C2 n2 time is theoretically preferred over the algorithm
A2 with time complexity C3 n3. The preference is based on asymptotic behaviour of
complexity functions of the algorithms. However in this case, only for instances
requiring millions of years, the algorithm A1 requiring C2 n2 time outperforms
algorithms A2 requiring C3 n3.

Remark 2.2.5:
Unit of Size for Space Complexity: Though most of the literature discusses the
complexity of an algorithm only in terms of expected time of execution, generally
55
Introduction to neglecting the space complexity. However, space complexity has one big advantage
Algorithmics over time complexity.

In the case of space complexity, unit of measurement of space requirement can be

well defined as a bit. But in the case of time complexity such obvious choice is not
available. The problem lies in the fact that unlike ‘bit’ for space, there is no standard
time unit such as ‘second’ since, we do not have any standard computer to which
single time measuring unit is applicable.

Ex.3) For a given problem P, two algorithms A1 and A2 have respectively time
complexities T1 (n) and T2 (n) in terms of size n, where

T1 (n) = 4n5 + 3n and

T2 (n) = 2500n3 + 4n

Find the range for n, the size of an instance of the given problem, for whichA1 is more
efficient than A2.

2.6 WELL KNOWN ASYMPTOTIC FUNCTIONS &

NOTATIONS
We often want to know a quantity only approximately, and not necessarily exactly,
just to compare with another quantity. And, in many situations, correct comparison
may be possible even with approximate values of the quantities. The advantage of the
possibility of correct comparisons through even approximate values of quantities, is
that the time required to find approximate values may be much less than the times
required to find exact values.
We will introduce five approximation functions and their notations.
The purpose of these asymptotic growth rate functions to be introduced, is to facilitate
the recognition of essential character of a complexity function through some simpler
functions delivered by these notations. For example, a complexity function f(n) =
5004 n3 + 83 n2 + 19 n + 408, has essentially the same behaviour as that of g(n) = n3
as the problem size n becomes larger and larger. But g(n) = n3 is much more
comprehensible and its value easier to compute than the function f(n).

2.6.1 Enumerate the five well-known approximation functions and how

these are pronounced

(i) O: (O (n2) is pronounced as ‘big-oh of n2’ or sometimes just as oh of n2)

(ii) Ω: (Ω (n2 ) is pronounced as ‘big-omega of n2 or sometimes just as

omega of n2’)

(iii) Θ: (Θ (n2) is pronounced as ‘theta of n2’)

(iv) o: (o (n2) is pronounced as ‘little-oh of n2’)

(v) ω: (ω (n2) is pronounced as ‘little-omega of n2’).

The O-notation was introduced by Paul Bachman in his book Analytische

Zahlentheorie (1894)
These approximations denote relations from functions to functions.
For example, if functions
56
 f, g: N→N are given by Some Pre-requisites and
Asymptotic Bounds
f(n) = n2 – 5n and
g(n) = n2
then
O(f(n)) = g(n) or O(n2 – 5n) = n2
To be more precise, each of these notations is a mapping that associates a set of
functions to each function under consideration. For example, if f (n) is a polynomial
of degree k then the set O (f (n)) includes all polynomials of degree less than or equal
to k.
Remark 2.6.1.1:
In the discussion of any one of the five notations, generally two functions say f and g
are involved. The functions have their domains and codomains as N, the set of natural
numbers, i.e.,

f: N→N
g: N→N

These functions may also be considered as having domain and codomain as R.

2.6.2 The Notation O

Provides asymptotic upper bound for a given function. Let f(x) and g(x) be two
functions each from the set of natural numbers or set of positive real numbers to
positive real numbers.

Then f (x) is said to be O (g(x)) (pronounced as big-oh of g of x) if there exist two

positive integer/real number constants C and k such that

f (x) ≤ C g(x) for all x≥ k (A)

(The restriction of being positive on integers/reals is justified as all complexities are

positive numbers).

Example 2.6.2.1: For the function defined by

f(x) = 2x3 + 3x2 + 1

show that

(i) f(x) = O (x3)

(ii) f(x) = O (x4)
(iii) x3 = O (f(x))
(iv) x4 ≠ O (f(x))
(v) f(x) ≠ O ( x2)

Solution:

Part (i)
Consider

f(x) = 2x3 +3x2 +1

≤ 2x3 +3x3 +1 x3 = 6x3 for all x ≥ 1

(by replacing each term xi by the highest degree term x3)

57
Introduction to ∴ there exist C = 6 and k = 1 such that
Algorithmics
f(x) ≤ C. x3 for all x≥k

Thus we have found the required constants C and k. Hence f(x) is O(x3).

Part (ii)
As above, we can show that

f(x) ≤ 6 x4 for all x ≥ 1.

However, we may also, by computing some values of f(x) and x4, find C and k as
follows:

f(1) = 2+3+1 = 6 ; (1)4 = 1

f(2) = 2.23 + 3.22 + 1 = 29 ; (2)4 = 16
f(3) = 2.33 + 3.32 + 1 = 82 ; (3)4 = 81

for C = 2 and k = 3 we have

f(x) ≤ 2. x4 for all x ≥ k

Hence f(x) is O(x4).

Part (iii)
for C = 1 and k = 1 we get
x3 ≤ C (2x3 + 3x2 +1) for all x ≥ k

Part (iv)
We prove the result by contradiction. Let there exist positive constants C and k
such that

x4 ≤ C (2x3 + 3x2 +1) for all x ≥ k

∴x4 ≤ C (2x3 +3x3+x3) = 6Cx3 for x≥k
∴ x4 ≤ 6 C x3 for all x ≥ k.

implying x ≤ 6C for all x≥ k

But for x = max of { 6 C + 1, k}, the previous statement is not true.
Hence the proof.

Part (v)
Again we establish the result by contradiction.

Let O (2 x3+3x2+1) = x2

Then for some positive numbers C and k

2x3 + 3x2+1 ≤C x2 for all x ≥k,

implying

x3≤C x2 for all x≥k (Θ x3 ≤ 2x3+3x2+1 for all x ≥1)

implying

x≤C for x ≥ k
58
Again for x = max {C + 1, k } Some Pre-requisites and
Asymptotic Bounds

The last inequality does not hold. Hence the result.

Example 2.6.2.2:
The big-oh notation can be used to estimate Sn, the sum of first n positive integers

Hint: Sn=1+2+3+……….+n ≤ n+n +…………+ n = n2

Therefore, Sn = O (n2).

Remark 2.6.2.2:
It can be easily seen that for given functions f(x) and g(x), if there exists one pair of C
and k with f(x) ≤ C.g (x) for all x ≥ k, then there exist infinitely many pairs (Ci, ki)
which satisfy

f(x) ≤ Ci g(x) for all x ≥ ki.

Because for any Ci ≥ C and any ki ≥ k, the above inequality is true, if f(x)≤ c.g(x) for
all x ≥ k.

2.6.3 The Ω Notation

The Ω Notation provides an asymptotic lower bound for a given function.
Let f(x) and g(x) be two functions, each from the set of natural numbers or set of
positive real numbers to positive real numbers.

Then f (x) is said to be Ω (g(x)) (pronounced as big-omega of g of x) if there exist two

positive integer/real number Constants C and k such that

f(x) ≥ C (g(x)) whenever x ≥ k

Example 2.6.3.1:
For the functions

f(x) = 2x3 + 3x2 + 1 and h (x) = 2x3─3x2+2

show that

(i) f(x) = Ω (x3)

(ii) h(x)= Ω (x3)

(iii) h(x)= Ω (x2)

(iv) x3 = Ω (h(x))

(v) x2 ≠ Ω (h(x))

Solutions:

Part (i)

For C =1, we have

f(x) ≥ C x3 for all x ≥ 1

59
Introduction to Part (ii)
Algorithmics
h(x) = 2x3─3x2+2
Let C and k > 0 be such that
2x3─3x2+2 ≥ C x3 for all x ≥ k
3 2
i.e., (2─C) x ─ 3x +2 ≥ 0 for all x ≥ k

Then C = 1 and k≥ 3 satisfy the last inequality.

Part (iii)

2x3─3x2+2 = Ω (x2)

Let the above equation be true.

Then there exists positive numbers C and k

s.t.
2x3─ 3x2+2 ≥ C x2 for all x ≥ k
3 2
2 x ─ (3 + C) x + 2 ≥ 0

It can be easily seen that lesser the value of C, better the chances of the above
inequality being true. So, to begin with, let us take C = 1 and try to find a value of k
s.t
2x3─ 4x2+2 ≥ 0.
For x ≥ 2, the above inequality holds

∴ k=2 is such that

2x3─ 4x2+2 ≥ 0 for all x ≥ k

Part (iv)
Let the equality

x3 = Ω (2x3─3x2+2)

be true. Therefore, let C>0 and k > 0 be such that

x3 ≥ C(2(x3─3/2 x2 +1))

For C = ½ and k = 1, the above inequality is true.

Part (v)
We prove the result by contradiction.

Let x2 = Ω (3x3─ 2x2+2)

Then, there exist positive constants C and k such that

x2 ≥ C (3x3 – 2x2 + 2) for all x ≥ k

i.e., (2C +1) x2 ≥ 3C x3 + 2 ≥ C x3 for all x ≥ k

2C +1
≥ x for all x ≥ k
C

60
 (2 C + 1) Some Pre-requisites and
But for any x ≥ 2 , Asymptotic Bounds
C

The above inequality can not hold. Hence contradiction.

2.6.4 The Notation Θ

Provides simultaneously both asymptotic lower bound and asymptotic upper bound
for a given function.

Let f(x) and g(x) be two functions, each from the set of natural numbers or positive
real numbers to positive real numbers. Then f(x) said to be Θ (g(x)) (pronounced as
big-theta of g of x) if, there exist positive constants C1, C2 and k such that
C2 g(x) ≤ f(x) ≤ C1 g(x) for all x ≥ k.

(Note the last inequalities represent two conditions to be satisfied simultaneously viz.,
C2 g(x) ≤ f(x) and f(x) ≤ C1 g(x))

We state the following theorem without proof, which relates the three functions
O, Ω, Θ.

Theorem: For any two functions f(x) and g(x), f(x) = Θ (g(x)) if and only if
f(x) = O (g(x)) and f(x) = Ω (g(x)).

Examples 2.6.4.1: For the function

f(x) = 2 x3 + 3x2 + 1,
show that

(i) f(x) = Θ (x3)

(ii) f(x) ≠ Θ (x2)

(iii) f(x) ≠ Θ (x4)

Solutions

Part (i)
for C1 = 3, C2 = 1 and k = 4

1. C2 x3 ≤ f(x) ≤ C1 x3 for all x ≥ k

Part (ii)
We can show by contradiction that no C1 exists.

Let, if possible for some positive integers k and C1, we have 2x3+3x2+1≤C1. x2 for all
x≥k
Then

x3≤ C1 x2 for all x≥k

i.e.,

x≤ C1 for all x≥k

But for
61
Introduction to x= max {C1 + 1, k }
Algorithmics

The last inequality is not true.

Part (iii)

f(x) ≠ Θ (x4)

We can show by contradiction that there does not exist C2

s.t

C2 x4 ≤ (2x3 + 3x2 + 1)

If such a C2 exists for some k then C2 x4 ≤ 2x3 + 3x2 + 1 ≤ 6x3 for all x ≥ k≥1,

implying

C2 x ≤ 6 for all x ≥ k

⎛ 6 ⎞
But for x = ⎜⎜ + 1⎟⎟
⎝ C2 ⎠

the above inequality is false. Hence, proof of the claim by contradiction.

2.6.5 The Notation o

The asymptotic upper bound provided by big-oh notation may or may not be
tight in the sense that if f(x) = 2x3 + 3x2 +1

Then for f (x) = O (x3), though there exist C and k such that

f(x) ≤ C (x3) for all x ≥ k

yet there may also be some values for which the following equality also holds

f(x) = C (x3) for x ≥ k

However, if we consider

f(x) = O (x4)

then there can not exist positive integer C s.t

f (x) = C x4 for all x ≥ k

The case of f(x) = O (x4), provides an example for the next notation of small-oh.

The Notation o
Let f(x) and g(x) be two functions, each from the set of natural numbers or positive
real numbers to positive real numbers.

Further, let C > 0 be any number, then f(x) = o(g(x)) (pronounced as little oh of
g of x) if there exists natural number k satisfying

f(x) < C g(x) for all x ≥ k≥1 (B)

62
Here we may note the following points Some Pre-requisites and
Asymptotic Bounds
(i) In the case of little-oh the constant C does not depend on the two functions f (x)
and g (x). Rather, we can arbitrarily choose C >0

(ii) The inequality (B) is strict whereas the inequality (A) of big-oh is not
necessarily strict.

Example 2.6.5.1: For f(x) = 2x3 + 3x2 + 1, we have

(i) f(x) = o (xn) for any n ≥ 4.

(ii) f(x) ≠ o(xn) for n≤ 3

Solutions:

Part (i)

Let C > 0 be given and to find out k satisfying the requirement of little-oh.
Consider

2x3 + 3x2 + 1 < C xn

3 1
= 2 + + 3 < C x n-3
x x

Case when n = 4

Then above inequality becomes

3 1
2+ + <C x
x x3

⎧7 ⎫
if we take k = max ⎨ ,1⎬
⎩C ⎭
then

2x3 + 3x2 + 1 < C x4 for x ≥ k.

In general, as xn > x4 for n ≥ 4,

therefore

2x3 + 3x2 + 1 < C xn for n ≥ 4

for all x ≥ k
⎧7 ⎫
with k = max ⎨ , 1⎬
⎩c ⎭
Part (ii)

We prove the result by contradiction. Let, if possible, f(x) = 0(xn) for n≤3.

Then there exist positive constants C and k such that 2x3+3x2+1< C xn

for all x≥ k.

Dividing by x3 throughout, we get

3 1
2+ + < C xn-3
x x2 63
Introduction to n ≤ 3 and x ≥ k
Algorithmics
As C is arbitrary, we take

C = 1, then the above inequality reduces to

3 1
2+ + 2 < C. xn-3 for n ≤ 3 and x ≥ k ≥ 1.
x x
Also, it can be easily seen that

xn-3 ≤ 1 for n ≤ 3 and x ≥ k ≥ 1.

3 1
∴ 2+ + ≤1 for n ≤ 3
x x2

However, the last inequality is not true. Therefore, the proof by contradiction.

Generalising the above example, we get the

Example 2.6.5.3: If f(x) is a polynomial of degree m and g(x) is a polynomial of

degree n. Then

f(x) = o(g(x)) if and only if n>m.

We state (without proof) below two results which can be useful in finding small-oh
upper bound for a given function.

More generally, we have

Theorem 2.6.5.3: Let f(x) and g(x) be functions in definition of small-oh notation.

Then f(x) = o(g(x) if and only if

f (x)
Lim =0
x →∞ g( x )

Next, we introduce the last asymptotic notation, namely, small-omega. The relation of
small-omega to big-omega is similar to what is the relation of small-oh to big-oh.

2.6.6 The Notation ω

Again the asymptotic lower bound Ω may or may not be tight. However, the
asymptotic bound ω cannot be tight. The formal definition of ω is follows:
Let f(x) and g(x) be two functions each from the set of natural numbers or the set of
positive real numbers to set of positive real numbers.

Further

Let C > 0 be any number, then

f(x) = ω (g(x))

if there exist a positive integer k s.t

f(x) > C g(x) for all x ≥ k

Example 2.6.6.1:
64
 If f(x) = 2x3 + 3x2 + 1 Some Pre-requisites and
Asymptotic Bounds
then
f(x) = ω (x)
and also
f(x) = ω (x2)
Solution:
Let C be any positive constant.

Consider
2x3 + 3x2 + 1 > C x

To find out k≥ 1 satisfying the conditions of the bound ω.

1
2x2 + 3x + > C (dividing throughout by x)
x
Let k be integer with k≥C+1

Then for all x ≥ k

1
2x2 + 3x + ≥ 2x2 + 3x >2k2 +3k > 2C2 +3C > C. (Θ k ≥ C+1)
x
∴ f(x) = ω (x)

Again, consider, for any C > 0,

2x3 + 3x2 + 1 > C x2

then
1
2x + 3 + >C Let k be integer with k ≥ C+1
x2
Then for x ≥ k we have
1
2x + 3 + 2 ≥ 2x + 3 > 2k + 3 > 2C + 3 >C
x
Hence
f(x) = ω (x2)

In general, we have the following two theorems (stated without proof).

Theorem 2.6.6.2: If f(x) is a polynomial of degree n, and g(x) is a polynomial of

degree n, then

f(x) = ω (g(x)) if and only if m > n.

More generally

Theorem 2.6.6.3: Let f(x) and g(x) be functions in the definitions of little-omega
Then f(x) = ω (g(x)) if and only if

f (x )
Lim =∞
x →∞ g(x )

g(x )
Lim =0
x →∞ f (x )

Ex.4) Show that n! = O(nn).

Ex.5) Show that n2 + 3logn = O(n2).
65
Introduction to Ex.6) Show that 2n = O(5n).
Algorithmics

2.7 SUMMARY
In this unit, first of all, a number of mathematical concepts are defined. We defined
the concepts of function, 1-1 function, onto function, ceiling and floor functions, mod
function, exponentiation function and log function. Also, we introduced some
mathematical notations.

In Section 2.3, the concept of mathematical expectation is introduced, which is useful

in average-case analysis. Formally, mathematical expectation is defined as follows:
Mathematical Expectation
For a given set S of items, let to each item, one of the n values, say, v1, v2,…,vn, be
associated. Let the probability of the occurrence of an item with value vi be pi. If an
item is picked up at random, then its expected value E(v) is given by

n
E(v) = ∑pv
i −1
i i = p1. v1 + p 2 . v 2 + ........ p n . vn

Also, the Principle of Mathematical Induction is discussed, which is useful in

establishing truth of infinitely many statements.
The method of Mathematical Induction consists of the following three major
steps:
4. Verifying/establishing the correctness of the Base Case
5. Assumption of Induction Hypothesis for the given set of statements
6. Verifying/Establishing the Induction Step
Complexity of an algorithm: There are two approaches to discussing and computing
complexity of an algorithm viz
(i) empirical
(ii) theoretical.
These approaches are briefly discussed in Section 2.5. However, in the rest of the
course only theoretical approach is used for the analysis and computations of
complexity of algorithms. Also, it is mentioned that analysis and computation of
complexity of an algorithm may be considered in terms of
(i) time expected to be taken or
(ii) space expected to be required for executing the algorithm.
Again, we will throughout consider only the time complexity. Next, the concepts of
linear, quadratic, polynomial and exponential time complexities and algorithms, are
discussed.

Next, five Well Known Asymptotic Growth Rate functions are defined and
corresponding notations are introduced. Some important results involving these
are stated and/or proved

The notation O provides asymptotic upper bound for a given function.

Let f(x) and g(x) be two functions each from the set of natural numbers or set of
positive real numbers to positive real numbers.
Then f (x) is said to be O (g(x)) (pronounced as big-oh of g of x) if there exist two
positive integer/real number Constants C and k such that
66
 Some Pre-requisites and
Asymptotic Bounds
f (x) ≤ C g(x) for all x≥ k
The Ω notation provides an asymptolic lower bound for a given function

Let f(x) and g(x) be two functions, each from the set of natural numbers or set of
positive real numbers to positive real numbers.

Then f (x) is said to be Ω (g(x)) (pronounced as big-omega of g of x) if there exist two

positive integer/real number Constants C and k such that

f(x) ≥ C (g(x)) whenever x ≥ k

The Notation Θ
Provides simultaneously both asymptotic lower bound and asymptotic upper bound
for a given function.

Let f(x) and g(x) be two functions, each from the set of natural numbers or positive
real numbers to positive real numbers. Then f(x) said to be Θ (g(x)) (pronounced as
big-theta of g of x) if, there exist positive constants C1, C2 and k such that
C2 g(x) ≤ f(x) ≤ C1 g(x) for all x ≥ k.

The Notation o
Let f(x) and g(x) be two functions, each from the set of natural numbers or positive
real numbers to positive real numbers.

Further, let C > 0 be any number, then f(x) = o(g(x)) (pronounced as little oh of
g of x) if there exists natural number k satisfying

f(x) < C g(x) for all x ≥ k≥1

The Notation ω

Again the asymptotic lower bound Ω may or may not be tight. However, the
asymptotic bound ω cannot be tight. The formal definition of ω is as follows:
Let f(x) and g(x) be two functions each from the set of natural numbers or the set of
positive real numbers to set of positive real numbers.

Further

Let C > 0 be any number, then

f(x) = ω (g(x))

if there exist a positive integer k s.t

f(x) > C g(x) for all x ≥ k

2.8 SOLUTIONS/ANSWERS
Ex. 1) We follow the three-step method explained earlier.
Let S(n) be the statement: 6 divides n3 ─ n

Base Case: For n = 0, we establish S(0); i.e., we establish that 6

divides 03 ─ 0 = 0

67
Introduction to But 0 = 6 × 0. Therefore, 6 divides 0. Hence S(0) is
Algorithmics correct.

Induction Hypothesis: For any positive integer k assume that S(k) is

correct, i.e., assume that 6 divides (k3 ─ k).

Induction Step: Using the conclusions/assumptions of earlier steps, to

show the correctness of S(k+1), i.e., to show that 6 divides (k+1)3 ─
(k+1).

Consider (k+1)3 ─ (k+1) = (k3 + 3k2 + 3k +1) ─ (k +1)

= k3 + 3k2 + 2k = k(k+1) (k+2)

If we show that k (k+1) (k+2) is divisible by 6 than by Principle of

Mathematical Induction the result follows.

Next, we prove that 6 divides k (k+1) (k+2) for all non-negative

integers.

As k, (k+1) and (k+2) are three consecutive integers, therefore, at least

one of these is even, i.e., divisible by 2. Hence it remains to show that
k(k+1) (k+2) is divisible by 3. This we establish through the following
case analysis:

(i) If k is divisible by 3, then, of course, k(k+1) (k+2) is also divisible

by 3.
(ii) If on division of k, remainder is 1, i.e., if k = 3t + 1 for some integer
t then k (k+1) (k+2) = (3t+1) (3t+2) (3t+3) = 3 (3t+1) (3t+2) (t + 1)
is divisible by 3.
(iii) If on division of k, remainder is 2, i.e., if k = 3t + 2 then
k(k+1) (k+2) = (3t + 2) (3t + 3) (3t + 4) = 3(3t + 2) (t +1) (3t + 4) is
again divisible by 3.

Ex. 2) Part (i): With stamps of Rs. 5 and Rs. 6, we can make the following
the following amounts

5 = 1 × 5 + 0× 6
6 = 0 × 5 + 1× 6 using 2 stamps

10 = 2 × 5 + 0× 6
11 = 1 × 5 + 1× 6 using 2 stamps
12 = 0 × 5 + 2× 6

15 = 3× 5 + 0× 6
16 = 2× 5 + 1× 6
17 = 1 × 5 + 2× 6 using 3 stamps
18 = 0 × 5 + 3× 6

19 is not possible
20 = 4× 5 + 0× 6
21 = 3 × 5 + 1× 6
22 = 2 × 5 + 2× 6 using 4 stamps
23 = 1 × 5 + 3× 6
24 = 0× 5 + 4× 6
68
 25 = 5 × 5 + 0× 6 Some Pre-requisites and
26 = 4× 5 + 1× 6 Asymptotic Bounds
27 = 3× 5 + 2× 6
28 = 2× 5 + 3× 6 using 5 stamps
29 = 1 × 5 + 4× 6
30 = 0 × 5 + 5× 6

It appears that for any amount A ≥ 20, it can be realized through stamps
of only Rs. 5 and Rs. 6.

Part (ii): We attempt to show using Principle of Mathematical

Induction, that any amount of Rs. 20 or more can be realized
through using a number of stamps of Rs. 5 and Rs. 6.

Base Step: For amount = 20, we have shown earlier that

20 = 4.5 + 0.6.

Hence, Rs. 20 can be realized in terms of stamps of Rs. 5 and

Rs. 6.

Induction Hypothesis: Let for some k ≥ 20, amount k can be realized in

terms of some stamps of Rs. 5 and some stamps of
Rs. 6.
Induction Step: Next, the case of the amount k+1 is analysed as
follows:
Case (i): If at least one Rs. 5 stamp is used in making an
amount of Rs. k, then we replace one Rs. 5 stamp by
one Rs. 6 stamp to get an amount of Rs. (k+1).
Hence the result in this case.
Case (ii): If all the ‘Stamps’ in realizing Rs. k ≥ 20 is through
only Rs. 6 stamps, then k must be at least 24 and
hence at least 4 stamps must be used in realzing k.
Replace 4 of Rs. 6 stamps by 5 of Rs. 5 stamps. So
that out of amount k, 6 × 4 = 24 are reduced through
removing of 4 stamps of Rs. 6 and an amount of Rs.
5 × 5 = 25 is added. Thus we get an amount of
k ─ 24 + 25 = k+1. This completes the proof.

Ex. 3) Algorithm A1 is more efficient than A2 for those values of n for which
4n5 + 3n = T1 (n) ≤ T2 (n) = 2500 n3 + 4n
i.e.,
4n4 + 3 ≤ 2500 n2 + 4
i.e.,

4n2 ─ 2500 ≤ 1/n2 (i)

Consider
4n2 ─ 2500 = 0
then n = 50/2 = 25

for n ≤ 25

4n2 ─ 2500 ≤ (25)2 ─ 2500 ≤ 0 ≤ 1/n2

Hence (i) is satisfy

Next, consider n ≥ 26
69
Introduction to 4n2 ─ 2500 ≥ 4(26)2 ─ 2500 = 2704 ─ 2500
Algorithmics 1 1
= 204 > 1 > ≥ 2
(26) n
2

Therefore, for n ≥ 26, (i)is not satisfied

Conclusion: For problem sizes n, with 1 ≤ n ≤ 25, A1 is more
efficient than A2. However, for n ≥ 25, A2 is more
efficient than A1
Ex. 4)
n!/nn = (n/n) ((n─1)/n) ((n─2)/n) ((n─3)/n)…(2/n)(1/n)
= 1(1─(1/n)) (1─(2/n)) (1─(3/n))…(2/n)(1/n)

Each factor on the right hand side is less than equal to 1 for all value of
n. Hence, The right hand side expression is always less than one.
Therefore, n!/nn ≤ 1
or, n! ≤ nn
Therefore, n! =O( nn)

Ex. 5)

For large value of n, 3logn < < n2

Therefore, 3logn/ n2< < 1

(n2 + 3logn)/ n2 =1 + 3logn/ n2

or, (n2 + 3logn)/ n2 <2

or, n2 + 3logn = O(n2).

Ex.6) We have, 2n/5n < 1

or, 2n <5n
Therefore, 2n = O(5n).

2.9 FURTHER READINGS

1. Discrete Mathematics and Its Applications (Fifth Edition) K.N. Rosen: Tata
McGraw-Hill (2003).

2. Introduction to Alogrithms (Second Edition), T.H. Coremen, C.E. Leiserson &

C. Stein: Prentice – Hall of India (2002).

70