Asymptotic Analysis-1

Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/1

 Asymptotic Analysis-1
Topics
 Objectives

 Asymptotic Notation

 Definitions ( Big-Oh, Big-Omega and Theta)

 Asymptotic Notation Theorems

Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/2

 Asymptotic Analysis
Objectives
The purpose of asymptotic analysis is to examine the behavior of an algorithm for large
input size. More specifically, if T(n) is the running time for an input of size n , we would
want to know the behavior or growth rate of T(n) for very large values of n. An analysis of
algorithm for large input is referred to as asymptotic analysis.

The asymptotic behavior of an algorithm is often compared to some standard

mathematical function, such as n2, n lg n etc The relationship or similarity of behavior is
often expressed by a special notation which is called asymptotic notation.

The standard asymptotic notations commonly used in the analysis of algorithms are known
as O (Big Oh), Ω (Big Omega), and θ(Theta).

Sometimes, additional notations o( small-oh) and ω( small-omega) are also used to show
the growth rates of algorithms

Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/3

Asymptotic Notation

Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/4

 O-Notation
Definition
If f(n) is running time of an algorithm, and g(n) is some standard growth function such
that for some positive constants c and integer n0 ,

f(n) ≤ c.g(n) for all n ≥ n0

then f(n) = O(g(n)) (Read f(n) is Big-Oh of g(n) )

The behavior of f(n) and g(n) is portrayed in the diagram. It follows that for n<n0, ,f(n) may
lie above or below g(n), but for all n ≥ n0, f(n) falls consistently below g(n)..

Trend of running time

Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/5
 O-Notation
Asymptotic Upper Bound
If f(n) = O(g(n)), then the function g(n) is called asymptotic upper bound of f(n)

Since the worst-case running time of an algorithm is the maximum running time for any
input, it would follow that g(n) provides an upper bound on the worst running time

The notation O( g(n) ) does not imply that g(n) is the worse running time; it simply means
that worst running time would never exceed upper limit determined by g(n).

Asymptotic Upper Bound

Asymptotic Upper Bound of f(n)

Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/6
 O-Notation
Basic Method
Example(1): Using basic definition, we show that 3n2 + 10n = O(n2)

Consider, 10 ≤ n for n ≥ 10 ( obvious !)

10n ≤ n2 for n ≥ 10( Multiplying both sides with n ) 3n2+10n ≤ 3n2

+ n2 for n ≥ 10( Adding 3n2 to both sides )

=4n2
(Simplifying )

3n2 + 10 n ≤ c.n2 for n ≥ n0 where c =4 and n0= 10 (Solution)

Therefore, it follows from the basic definition that

3n2 +10n = O(n2)

The choice of constant c is not unique. However, for each different c there is a
corresponding value of n0 which satisfies the basic relation .This behavior is illustrated
by the next example
Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/7
 O-Notation
Basic Method
Example(2): In the preceding example it was shown that 3n2 + 10n ≤ c.n2 for c=4, n0=10. We
now show that the relation holds true for a different value of c and corresponding n0.

Consider n ≤ n2 for n ≥ 1 ( Obvious)

10n ≤ 10n2 for n≥ 1 (Multiplying both sides with 10 )

3n2 +10 n ≤ 3n2 + 10n2 for n≥ 1 ( Adding 3n2 on both

sides)

3n2 +10n ≤ 13n2 for n ≥ 1 ( Simplifying )

Or, 3n2 +10n ≤ c.n2, for n ≥ n0 where c=13, n0=1 ( Solution )

Therefore, by basic definition,

3n2 + 10n = O(n2)

Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/8

 O-Notation Example
Comparison of Growth Rates
The results of analysis in the preceding examples are plotted in the diagram below.
It can be seen that the function cg(n) =4n2 ( c= 4) overshoots the function f(n)= 3n2 + 10n for
n0=10. Also, the function cg(n) =13n2 ( c =13 ) grows faster than f(n) = 3n2 + 10n for n0=1

cg(n)=4n2

f(n)=3n2 + 10 n
cg(n)= 13n2

n0=1 n0=10
Growth of functions 13n2 and 4n2 versus the function 3n2 + 10 n

Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/9

 Set Builder O-Notation
Definition
Consider the functions , say f1(n), f2(n), f3(n), ..fk(n) for which g(n) is the asymptotic
upper bound. By definition,
f1(n) ≤ c1 g(n) for n ≥ n1
f2(n) ≤ c2 g(n) for n ≥ n2

f3(n) ≤ c3 g(n) for n ≥

n3
……………………
fk(n)

≤ ck g(n) for ≥ nk

where c1, c2, c3 ,.. ck are constants and n1 ,n2 ,n3,.. nk are positive integers. The functions f1(n),
f2(n), f3(n), .. fk(n) are said to belong to the class O( g(n)). In set notation, the relation is
denoted by
O( g(n) )= { f1(n) , f2(n), f3(n)…,fk(n)}

then,
 Alternatively,f(n)  O(g(n))notation, if
using set-builder
Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/100
 Ω-Notation
Definition
If f(n) is running time of an algorithm, and g(n) is some standard growth function such
that for some positive constants c, positive integer n0 ,

c.g(n) ≤ f(n) for all n ≥ n0

then f(n) = Ω(g(n)) (Read f(n) is Big-Omega of g(n) )

The behavior of f(n) and g(n) is portrayed in the graph. It follows that for n < n0, f(n) may
lie above or below g(n), but for all n ≥ n0, f(n) falls consistently above g(n). It also implies that
g(n) grows slower than f(n)

Trend of running time

Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/111
 Ω-Notation
Asymptotic Lower Bound
If f(n) = Ω(g(n)), then the function g(n) is called asymptotic lower bound of f(n)

Since the best-case running time of an algorithm is the minimum running time for any input,
it would follow that g(n) provides a lower bound on best running time

As before, the notation Ω( g(n) ) does not imply that g(n) is the best running time; it simply
means that best running time would never be lower than g(n).

Asymptotic Lower Bound

Asymptotic lower bound of f(n)

Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/122
 Ω-Notation
Basic Method
Example(1) : Using basic definition, we show that n2 -10n = Ω(n2).

For, n ≥ n / 2 for n ≥ 0
( Obvious !)
n - 10 ≥ n / (2 x 10) for n ≥ 10 ( Dividing right side by 10, to maintain the relation )

= n / 20

n2 – 10n ≥ n2 / 20 for n ≥ 10 ( Multiplying both sides with n )

n2 – 10n ≥ c. n2 for n ≥ n0, where c=1 / 20 and n0=10 ( Solution )

Therefore, by definition ,

n2 -10n = Ω(n2).

Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/133

 Ω-Notation Example-1
Comparison of Growth Rates
The behavior of functions n2-10n and n2 / 20 is shown in the graph. For n > 10, the
function cg(n)= n2 / 20 (c=1/20) consistently fall below the function f(n) = n2-10n

f(n)= n2 - 10n

Asymptotic Lower bound

cg(n)=n /20 n0
2

n
Growth of function n2 /20 versus the function
f(n)= n2 - 10 n
Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/144
 Ω-Notation
Basic Method
Example(2): Next we show that 3n2 -25n = Ω (n2 ).

For, n ≥ n/2 for n ≥ 0 ( Obvious !)

n- 25/3 ≥ 3 n / (2 x 25) for n ≥ 9 ( Dividing right side by 25/3 ≈ 8.3 to

maintain
the inequality, since n >8.3

ie n≥9 )
3n2 – 25n ≥ 9n2 / 50 for n ≥ ( Multiplying both sides with 3n to obtain
9 = 3 n / 50 the for
desired
n ≥ 9function on the )left hand side)
(Simplifying

3n2 – 25 n≥ c. n2 for n ≥ n0, where c= 9 / 50 , n0=9

(Solution)

Therefore, by definition,

3n2 -25n = Ω(n2).

Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/155

 Ω-Notation Example-2
Comparison of Growth Rates
The behavior of functions 3n2-25n and 9n2 / 50 is shown in the graph. Observe that for
n≥9, the function 9n2 / 50 falls below the function 3n2-25n

f(n)=3n2 -25n

n0=9
Asymptotic Lower bound
cg(n)=9.n2/50

n
Growth of function 9 n /50 versus the function f(n)=3 n2 -25 n
2

Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/166

 Set Builder Ω-Notation
Definition
Consider the functions , say f1(n), f2(n), f3(n), ..fk(n) for which g(n) is the asymptotic
lower bound. By definition,
f1(n) ≥ c1 g(n) for n ≥ n1
f2(n) ≥ c2 g(n) for n ≥
n2

f3(n) ≥ c3 g(n) for n ≥

n3
……………………
fk(n)

≥ ck g(n) for ≥ nk

where c1, c2, c3 ,.. ck are constants and n1 ,n2 ,n3,.. nk are positive integers. The functions f1(n),
f2(n), f3(n), .. fk(n) are said to belong to the class Ω(g(n)). In set notation, the relation is
denoted by
then Ω( g(n)Ω(g(n))
)= { f1(n) , f2(n), f3(n)…,fk(n)}
f(n)  Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/177
 Alternatively, using set-builder notation , if
 θ-Notation
Definition
If f(n) is running time of an algorithm, and g(n) is some standard growth function
such that for some positive constants c1 , c2 and positive integer n0 ,

0 < c2.g(n) ≤ f(n) ≤ c1.g(n) for all n ≥ n0

then f(n) = θ(g(n)) (Read f(n) is theta of g(n) )

The behavior of f(n) and g(n) is portrayed in the graph. It follows that for n < n0, f(n) may
be above or below g(n), but for all n ≥ n0, f(n) falls consistently between c1.g(n) and c2.g(n). It
also implies that g(n) grows as fast as f(n) The function g(n) is said to be the asymptotic tight
bound for f(n)
Asymptotic Upper Bound

Asymptotic Lower Bound

Asymptotic tight bound of f(n)

Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/188
 Set Builder θ-Notation
Definition
There can be a several functions for which the g(n) is asymptotic tight bound. All such
functions are said to belong to the class Ө(g(n))

 Using set builder notation,, if

Ө(g(n)) = { f(n): there exit positive constants c1,c2 and positive integer n0 such that

c1 g(n) ≤ f(n) ≤ c2 g(n) }

then f(n)  θ(g(n))

Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/199

 θ-Notation
Basic Method
Example: We show that 5n2 -19n = θ(n2).
First consider the upper bound . As explained before,
5n2 – 19n ≤ 5n2 for n ≥ 0

5n2 – 19 n ≤ c1. n2 for n ≥ n1, where

c =5 and n =0 ………..(1)
Next,1 consider 1the lower bound,
n ≥ n / 2 for n ≥ 0 ( Obvious !)

n- 19/5 ≥ 5 n / (2 x 19) for n ≥ 4 ( Dividing right side by 19/5= 3.8)

= 5 n / 38 for n ≥ 4

5n2 – 19n ≥ 25n2 / 38 for n ≥ ( Multiplying both sides with 5n )

4
5n2 – 19 n≥ c2. n2 for n ≥ n2, where c2=25 / 38 and n2=4 …….
(2)
From (1) and (2) it follows ,
0 < c2.n2 ≤ 5n2 - 19n ≤ c1.n2 for n ≥ n0 , where n0=4, c1=5 and c2=25/38. (Solution)

Therefore, 5n2 -19n = θ(n2)

Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/20
 θ-Notation Example
Comparison of Growth Rates
The relation 5n2 -19n = θ(n2) is illustrated in the figure below. The graph shows the
asymptotic upper and lower bounds for the function f(n)= 5n2-19 n

Asymptotic Upper bound

cg(n)=5n2

f(n)=5n2 -19n

cg(n)=25n2/38
n0=4 AsymptoticLower bound

n
Growth of functions 5 n2 and 25n2/38 versus the function f(n)=5 n2 -19 n
Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/ 221
 Asymptotic Notation
Constant Running Time
If running time T(n)=c is a constant, i. e independent of input size, then by convention,
the asymptotic behavior is denoted by the notation
O(c) = O(1) , θ (c) = θ(1), Ω(c) = Ω(1)
The convention implies that the running time of an algorithm ,which does not depend
on the size of input, can be expressed in any of the above ways.

 If c is constant then using basic definition it can be shown that

O( c.f(n) ) = O( f(n) )
θ( c.f(n) ) = θ( f(n) )
Ω( c.f(n) ) = Ω( f(n ))

Example: (i) O(1000n) = O(n),

(ii) θ(7lgn ) = θ (lg n),
(ii) Ω(100 n!) = Ω(n!)

The above results imply that in asymptotic notation the multiplier constants in an
expression for the running time can be dropped

Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/22

 Asymptotic Notations
Relationships
Theorem: If f(n) = θ( g(n) ) then

f(n) = Ω( g(n) ), and O( g(n) )

f(n)=
and f(n) = O( g(n) )
Conversely, if f(n) = Ω( g(n) )
then f(n) = θ( g(n) )
 The above relations can be established by using basic definitions

Example(1): Since, n(n-1)/2 = θ(n2 ), therefore it follows that

n(n-1)/2 = Ω(n2 )
n(n-1)/2 = O(n2 )

Example(2): It can be shown that

5n2+1 = Ω(n2 )
and 5n2+1 = O(n2 )
Therefore, 5n2 + 1 = θ(n2 )
Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/23
 Asymptotic Set Notations
Relationship
We have seen that if f(n) = O( g(n) ) and f(n) = Ω(g(n)) then f(n) = Ө( g(n) )

Using set notation we can express the relationship as follows:

if f(n) O( g(n) ),

then
and f(n) f(n)
Ω (g(n)),
 Ө( g(n) ) ,
where Ө(g(n)) = O( g(n) ) ∩ Ω (g(n) )

 The above property is illustrated by the following Venn diagram

O(g(n)) Ө(g(n)) Ω(g(n))

Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/24

 Asymptotic Set Notation
Example
The relationship among the O ,Ω , and θ notations can be expressed using set notation
Consider, for example, the following sets of growth functions

O(n2) = {√n, n+5, lg n+4n, n1.5+n, √n+5n2, n2+5n, lg n+4n2, n1.5+3n2 }

Ω(n2) = { √n+5n2, n2+5n, lg n+4n2, n1.5+3n2 , 5n2+n3, n3 +n2+n, lg n+4n4, nlg n+3n4 }

θ(n2) = { √n+5n2 , n2+5n, lg n+4n2, n1.5+3n2 }

 It follows that θ(n2) = O(n2) ∩ Ω(n2)

Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/25

 Asymptotic Set Notation
Example
The following Venn diagram represents the preceding set relations

√n √n+5n2
√n+5n2 5n2+n3
n+5 n2+5n
n2+5n n3 +n2+n
lg n+4n2
lg n+4n 1.5
n +3n2 lg n+4n2 lg n+4n4
n1.5+n n1.5+3n2 nlg n+3n4

O(n2)
Ω(n2)

√n √n+5n2 5n2+n3
n+5 n +5n
2
n3 +n2+n
lg n+4n lg n+4n2 lg n+4n4
n1.5+3n2 nlg n+3n4
n1.5+n

θ(n2)

Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/26

 Asymptotic Notation
Order Theorem
Theorem: If f1(n) = O( g1(n) ) and f2(n) = O( g2(n) ) then
f1(n) + f2(n)= O( max( g1(n) , g2(n) )

Proof: By definition,
f1(n) ≤ c1. g1(n) for n ≥ n1
f2(n) ≤ c2. g2(n) for n ≥ n2
Let n0 = max( n1, n2)
c3=max(c1, c2) f1(n)≤ c3. g1(n)
f2(n) ≤ c3 g2(n) for n ≥ n0
for n ≥ n0
f1(n) + f2(n) c3.g1(n) + c3. g2(n) ) for n ≥ n0
≤Let h(n) = max( g1(n) , g2(n) )
f1(n) + f2(n) ≤ 2c3.h(n) = c. h(n) where c=2c3 for n ≥ n0
f1(n) + f2(n) c. h(n) = c. max( g1(n) , g2(n) )
for n ≥ n0
≤
Therefore,
f1(n) + f2(n) = O( max( g1(n) , g2(n) )
 The theorem also applies to θ and Ω notations
Asymptotic Analysis-1/IIU 2008/Dr.A.Sattar/27
 Asymptotic Notations
General
then
Theorem
Theorem: If f1(n)= O(g1(n)) , f2(n) =O(g2(n)), f3(n)=O(g3(n))…., fk(n)=O(gk(n))

f1(n) + f2(n) + f3(n) ….+ fk(n) = O( max (g1 (n), g2 (n), g3 (n)…, gk(n) ) )
The theorem can be proved by using basic definition of Big-Oh
where max means fastest growing function
It follows from the theorem that in an expression consisting of sum of several functions,
the comparatively slower growing functions can be discarded in favor of the fastest
growing function to obtain the Big oh notation for the whole expression This also true for
the Ө and Ω notations

Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/28

 Asymptotic Notations
Using Order Theorem
Example: Consider the summation f(n) consisting of basic functions :

f(n) = n +√n + n1.5+ lg n + n lg n n2

We have seen that

lg n < √n < n <n lg n <n1.5 < n2

which means that the function n2 grows faster than all other functions in the expression

Thus,

f(n) = n + √n + n1.5+ lg n + n lg n + n2

= O( max (n , √n , n1.5, lg n , n lg n , n2 ) )

= O(n2)

Asymptotic Analysis-1/KICSIT/Mr. Fahim Khan/29