Ω-Notation Using Limit

Definition
If f(n) is running time of an algorithm and g(n) is some standard growth function such
that
f(n)
lim —— = c , where c is a constant, such that 0 < c ≤ ∞
n → ∞ g(n)

then f(n) = Ω( g(n) ).

 Note that zero value for the constant c is excluded from the permissible range, but
infinity is included.
 Ω-Notation
Examples
Example(1): 7n + 12n+8= Ω(n )
2 2

lim 7n2 + 14n + 8

n→∞
n2

= 7+ 14 / n +8 / n2
=7+0+0
=7
(Non-zero constant)
Therefore, 7n2 + 14n+ 8 = Ω(n2)
Example(2): 10n3 + 5n+ 2 = Ω(n2)

lim 10n3 + 5n + 2
n→∞ 2 n

=10 n+ 5 / n + 2 / n2
=∞+0+0
=∞
Therefore, 10n3 + 5n+ 2 = Ω
(n2)
 Ө-Notation Using Limit
Definition
If f(n) is running time of an algorithm and g(n) is some standard growth function such
that
f(n)
lim —— = c , where c is a constant, such that 0 < c < ∞
n → ∞ g(n)

then f(n) = Ө ( g(n) ).

Note that zero value and infinity are excluded from the permissible values for the
range of constant c
 Θ-Notation
Examples
Example(1): 45n3 - 3n2 - 5n+ 20 = θ(n3)
lim 45n3 -3n2 - 5n + 20
n→∞ 3 n

= 45-3 / n -5 / n2 +20/n3
= 45 - 0 – 0 + 0
=45 (non-zero
Therefore, 45n3 -constant)
3n2 - 5n+ 20 = θ(n3)

Example(2): n lg n + n + n2 = θ(n2)

lim n lg n + n + n2
n→∞ n2
= lg n / n +1/n +1
= 0 +0 + 1
=1
(non-zero constant)
Thus, n lg n + n + n2 = θ(n2)
 Θ-Notation
Examples
Example(3): 45n3 - 3n2 - 5n+ 20 θ(n4)
≠ 45n3 -3n2 - 5n + 20
lim
n→∞ 4
n

= 45 / n - 3 / n2 - 5 / n3 + 20 / n4
=0-0-0+0
= 0 (zero is excluded from the permissible range for Ө-
Therefore, 45n3 -notation)
3n2 - 5n+ 20 ≠
θ(n3)

Example(4): n lg n + n ≠ θ(n2)
lim n lg n + n
n→∞
n2
= lg n / n + 1/n ( Using the L’Hopital Rule to evaluate the limit of lgn /n)
= 0 +0
=0
Thus, n lg n + n ≠ θ(n2)
 o-Notation Using Limit
Definition
If f(n) is running time and g(n) is some standard growth function such that

f(n)
lim —— = 0 ( zero )
n→∞ g(n)

then f(n) = o( g(n) ) (Read f (n) is small-oh of g(n))

 o( g(n) ) is referred to as the loose upper bound for f(n)

 o-Notation
Examples
Example(1): 5n+ 20 = o(n2)
lim 5n + 20
n→∞
n2
= 5 / n + 20 / n2
= 0

+ 0
Example(2): 10n2 + 25n+ 7 =2 o(n3)
Therefore, 5n+ 20 = o(n )
lim 10n2 + 25n + 7
n→∞ n3

=10 / n+ 25 / n2 + 7 / n3
= 0+0+0

=0
Therefore, 10n2 + 25n + 7 = o(n3)
 o-Notation
Examples
Example(3): lg n = o (n)

lim lg n ∞
= ( Need to use L’Hopital Rule)
n→∞ n ∞

lim ln n
( Converting to natural log ,lg n = log2 n e n / log e2 )
n→∞ n ln 2 =log

lim 1
( Differentiating numerator and denominator )
n→∞ n ln2

=0

Therefore, lg n = o (n)
 ω-Notation
Definition
If f(n) is running time and g(n) is some standard growth function such that
f(n)
lim —— = ∞
n→∞ g(n)

then f(n) = ω(g(n)) (Read f(n) is small-omega of g(n))

 ω(g(n)) is referred to as the loose lower bound for f(n)

 ω-Notation
Examples
Example(1): 5n3+ 20n2+n+10 = ω(n2)

lim 5n3 + 20n2+n+10

n→∞
n2
=5n +20 + 1/ n+10 / n2
= ∞ + 20 + 0+ 0
=∞
Therefore, by definition, 5n3+ 20n2+n+10 = ω(n2)

Example(2): 10n2 + 25n+ 7 = ω(n)

lim 10n2 + 25n + 7

n→∞ n
= n+ 25 + 7 / n
= ∞ +25 + 0
=∞
Therefore, 10n2 + 25n+ 7 = ω(n)
Asymptotic Analysis-2/IIU 2008/Dr.A.Sattar/10
 ω-Notation
Examples
Example(3): n! =ω(2n)

lim n!
= ∞ / ∞ (Need to use the LHpoital Rule
n→∞ 2n
The function n! cannot be differentiated directly. We first use Stirling’s approximation:

n! = √ 2πn ( n /e ) n

lim for large n n!

n→∞ 2n

√ 2πn ( n / e ) n
=
2n
= √ 2πn ( n /2 e ) n

=∞

Thus, n! = ω( 2n )
Asymptotic Analysis-2/IIU 2008/Dr.A.Sattar/11
 Asymptotic Notation
Summary
f(n)
——— = α
lim
Let f(n) be time complexity and g(n) standard function, such that n→∞ g(n)
Table below summarizes the asymptotic behavior of f(n) in terms of g(n)

Notation Using Basic Definition Using Limits Asymptotic Bound

f(n)=O( g(n) ) f(n) ≤ c.g(n) for some c>0, and n ≥ n0 0≤ α <∞ upper tight

f(n)=o( g(n) ) f(n) <c.g(n) for all c>0, and n ≥ n0 α=0 upper loose

f(n)=Ω( g(n) ) f(n) ≥ c.g(n) for some c>0 and n ≥ n0 0< α ≤∞ lower tight

f(n)=ω( g(n) ) f(n) > c.g(n) for all c>0 and n ≥ n0 α=∞ lower loose

f(n)=θ( g(n) ) c1.g ≤ f(n) ≤ c2.g(n) for some c1>0, c2>0 0<α<∞ tight
and n ≥ n0

Asymptotic Analysis-2/IIU 2008/Dr.A.Sattar/12

 Asymptotic Set Notations
Definitions
 The asymptotic notation can also be expressed in Set notations by using the limits
f(n)
——— = c
 Let f(n) be time complexity and g(n) standard function, such that lim
n→∞ g(n)
where c is zero, positive constant or infinity.

 The set notations for the asymptotic behavior is are defined as

follows

(i) O(g(n) ) = { f(n): 0≤c<∞}

(ii) Ω( g(n) ) = { f(n) : 0 < c ≤ ∞ }

(iii) Ө( g(n) ) = { f(n): 0 < c < ∞ }

(iv) o( g(n) ) = { f(n):

c = 0 } (v)

ω( g(n) ) = { f(n):
Asymptotic Analysis-2/IIU 2008/Dr.A.Sattar/13
 Asymptotic Set Notations
Examples
Example (1): Let f(n) O ( g(n) ), f(n)  Ω( f(n) ) . Then

O(g(n) ) = { f(n): 0 ≤ c < ∞ } (definition of O-notation)

Ω( g(n) ) = { f(n) : 0 < c ≤ ∞ } (definition of Ω-notation )

O(g(n) ) ∩ Ω ( g(n) ) = { f(n) : 0 < c < ∞ } ( Performing Set intersection operation

)

= Ө (g(n)) (definition of Ө-notation }

Thus, O(g(n) ) ∩ Ω ( g(n) ) = Ө (g(n))

Example (2):Suppose f(n)

o(g(n)), and f(n)ω(g(n)). Then
o(g(n)) = { f(n): c = 0 } (definition of o-notation)

Ω(g(n))= { f(n): c = ∞ } (definition of ω-notation)

Therefore, o(g(n)) ∩ ω( g(n)) =φ ( Empty set )

Asymptotic Analysis-2/IIU 2008/Dr.A.Sattar/14
Analysis of Summations

Asymptotic Analysis-2/IIU 2008/Dr.A.Sattar/15

 Arithmetic Summation
Asymptotic Behavior
 The sum of first of n terms of arithmetic series is :
1 + 2 + 3………..+n = n(n+1)/2

Let f(n)= n(n+1)/2

and g(n)= n2
f(n) n(n+1)/2 n2 /2 + n/2
lim = = = 1 /2 + 1/2n =1/2 + 0=1/2
n→∞ n2 n2
g(n)
 Since the limit is non-zero and finite it follows

f(n) = θ( g(n)) = θ(n2)

Or, 1 + 2 + ………..+ n = θ(n2)

Asymptotic Analysis-2/IIU 2008/Dr.A.Sattar/16

 Geometric Summation
Asymptotic Behavior
 The asymptotic behavior of geometric series
1 +r + r2+……..+rn
depends on geometric ratio r. Three cases
need to be considered

Case r > 1: It can be shown that sum f(n) of

n+1 - 1
f(n) n terms
= 1 +ris+asr2follows r
+……..+rn = —————
first
r- 1

Case r = 1: This is trivial

f(n)= 1 + 1 + 1+ ……+1 = n =Ө(n)

Case r < 1: It can be shown that

1 - rn+1
f(n) = 1 +r + r2 +……..+rn = —————
1-r
 The asymptotic behavior in first case and third case is explored by computing limits.

Asymptotic Analysis-2/IIU 2008/Dr.A.Sattar/17

 Geometric Summation
Case r >
1
rn+1
- 1
Let f(n) = 1 +r + r2+……..+rn = —————
r- 1
Let g(n)= rn

Consider, the limit

f(n) - 1 r - 1/rn
lim rn+1
= =
n→∞ g(n) (r - 1).rn (r - 1)

Since r>1 , 1/rn →0 as n→ ∞

f(n)
lim r
= >0, since r > 1
n→∞ g(n) (r - 1)
Therefore, f(n) = θ(rn) for
r>1
Or, 1 +r + r2+……..+rn = θ(rn) for r > 1

Asymptotic Analysis-2/IIU 2008/Dr.A.Sattar/18

 Geometric Summation
Case r <
Consider 1 +r + r2+……..+r n 1= —————
rn+1
- 1
r- 1
Let g(n)=c where c is some positive constant
Taking the limit
f(n) - 1
lim rn+1 1- rn+1
= =
n→∞ g(n) (r - 1).c (1- r )c
Since r<1 , rn+1 →0 as n→ ∞

f(n)
lim 1
=
n→∞ g(n) >0, since r < 1
(1 - r). c

Therefore, f(n)= θ(g(n))=θ(c) = θ(1) for r <1

Or, 1 +r + r2+……..+rn =θ(1) for r <1

Asymptotic Analysis-2/IIU 2008/Dr.A.Sattar/19

 Geometric Summation
Asymptotic Behavior
From the preceding analysis it follows that the asymptotic behavior of geometric summation is

Ө(rn) when r>1

S(n) =1+r+r2+.........+ rn =
Ө(1) when r<1

When r <1, the largest term in the geometric summation would be 1 ( the first term) , and
S(n) =Ө(1) . On the other hand if r>1, the largest term would be rn ( the last term), and
S(n)=Ө(rn) In the light of this observation it can said that asymptotic behavior of geometric
summation is determined by the largest term i.e S(n)=Ө (largest term).
Example(1): The geometric series
1+21+ 22 +….+ 2n
has geometric ratio r=2>1 ,
Thus, 1+21+ 22 +….+ 2n =Ө(2n) , ► Ө(largest term)
Example(2): The geometric series
1+(2/3)1+ (2/3)2 +….+ (2/3)n
has geometric ratio r=2/3 <1 ,
Thus, 1+(2/3)1+ (2/3)2 +….+ (2/3)n =Ө(1) , ►Ө(largest term)
Asymptotic Analysis-2/IIU 2008/Dr.A.Sattar/20
 Logarithm Summation
Asymptotic Behavior
 The logarithmic series has the summation
lg(1)+lg(2)+ lg(3)+….. +lg(n)
Let f(n) = lg(2)+ lg(3)+….+.lg(n) = lg(2.3……..n) =
lg( n!)
and g(n) = n lg n
lim f(n) lg n! lg ( √(2πn)(n/e)n ) (Using Stirling’s approximation)
n→∞ g(n) = n lg n = n lg n
Now, lg ( √(2πn)(n/e)n ) = (1+lg π + lg n)/2 + n lg n - n lg e , therefore

lim lg ( √(2πn)(n/e)n )
= ( 1+lg π + lg n ) /(2 nlg n) + 1 - lg e / lg n = (0+ 1- 0)=1
n→∞ n lg n
Since limit is non-zero and finite, it follows
f(n)= θ(g(n))

Or, lg(1)+lg(2)+ lg(3)+….. +lg(n) = θ(n lg n)

…
Asymptotic Analysis-2/IIU 2008/Dr.A.Sattar/21
 Harmonic Summation
Asymptotic Behavior
The sum of first n terms of Harmonic series is
1+ 1/2 + 1/3+…..+1/n
Let f(n) = 1+ 1/2 + 1/3+…..+1/n
and g(n) = lg n

f(n) 1+1/2+1/3+ …..+1/n

lim
n→∞ g(n) = lg n
It can be shown that 1+ 1/2+ 1/3+…+1/n = lg (n) + γ + 1/2n – 1/12n2+.. where γ ≈ 0.5772

f(n) lg n + γ + 1/2n – 1/12n2 +…

lim = 1+0+0-0+0+…=1
n→∞ g(n) = lg n
Since limit is finite and non-zero, it follows
f(n) = θ( g(n))
Or, 1+ 1/2 + 1/3+…..+1/n = θ (lg n )

Asymptotic Analysis-2/IIU 2008/Dr.A.Sattar/22