Module 1

By ~ Tapaswini Dash
 Asymptotic Notation
• Asymptotic Notation: Is the mathematical notation used to
describe the running time of an algorithm when i/p tends
towards a particular value.
• They are 3 asymptotic notations are mostly used to represent
time complexity of algorithm.

i. Big oh (O)notation
ii. Big omega (Ω) notation
iii. Theta(Θ) notation
iv. Little oh notation
v. Little omega(Ω) notation
 Growth Rate
• Growth Rate of various algorithm: the growth rate of an
algorithm is the arte at which the running time (cost) of an
algorithm grows as the size of the i/p grow.
• An algorithm’s proportional time requirement is known as
growth rate.
• We can compare the efficiency by comparing their growth
rate.
Growth rate graph
 Classification of growth
• Growing with the same rate
• Growing with he slower rate
• Growing with the faster rate

Will learns about notation in coming slides .

 Big Oh(O) –Notation
• The Big-oh(O) notation is the most commonly
used notation to express an algorithm’s
performance .
• O-notation express the upper bound of a function
within a constant factor.
• If g(n) is a upper bound of a function f(n), then
f(n) = O C.g(n), if and only if there exist two
positive constant c and n0 |f(n)| ≤ C|g(n)| ∀ n ≤ n0
 For example:-

The function to calculate O notation is ,F(n) ≤ OC.g(n),

let’s assume c= 4;
n=1

Extra Question –

1. Find the O-notation for function ?

2. Find the O-notation for function ?
3. Is
 Big Omega(Ω) notation
• The Ω notation provides an asymptotic lower
bound on a function that means , we can run
the program in any machine . It takes the
minimum time g(n).
• If g(n) in an lower bound f(n) , then
f(n) = ΩC.g(n), if and only if there exists two
positive constant c and n0 such that ,
|f(n)| ≥ C|g(n)| ∀ n≥ n0
For example
 Big theta (θ) Notation
• The θ notation provides an asymptotic average
bound on a function that means. It takes the
average time g(n).
• By using the θ notation we can calculate the
average time taken by the algorithm, that’s why
it is called average case time complexity.
• If g(n) in an lower bound f(n) , then
f(n) = θ(g(n)) , if and only if there exists some
positive constant c1 and c2 and n such that ,
• c1 g(n)<=f(n)<=c2 g(n)
 Solving Recurrence
• A recurrence relation is an equation which is
defined in terms of itself .
• Let’s consider the problems of finding the nth
Fibbonacci number or computing the factorial
of a number .
• To solve a recurrence relation means to obtain
a function defined on the Natural numbers that
satisfy the recurrence .
• There are basically five methods for solving
recurrence .
1. Substitution method
2. Iteration Method
3. Recursion tree method
4. Master’s method
5. Changing variable method
 Substitution Method
• To solve substitution method there is only two
ways;
• Guess the solution and,
• Mathematical induction to find the constant
and show that the solution is correct.
• For ex- here we have constants is “n” and
varies = n/2, we have to show that it is
asymptotically bound by O(logn) for for some
constant C, so
• put this in given recurrence relation equation
•
•
•
• Thus ,T(n) = O(logn)
 Solving recurrence using substitution
method

• Lets assume
• is eq 2

• Now substitute eq 2 in eq 1

• When we solve the termination sequence

• =
• So the recurrence sequence is
• Taking n is common for sequences
 Iteration Method
• It means to expand the recurrence and express
it as a summation of terms of n and initial
condition.
• Is substitution method and iteration method is
same?
• Yes, iteration and substitution are both
methods used to solve recurrence relations, but
they can involve different approaches.
 Iteration Vs Substitution
Iteration Substitution
1. Also known as the iterative method or 1. This method involves isolating one
backwards substitution, this method variable and substituting it with an
involves substituting the value of the expression from another equation.
recurrent part of an equation until a
pattern is found.

2. This method is sometimes called a 2. This method is sometimes called the

"brute force" method. "guess-and-check" method, where you
What is brute force? make a guess about the runtime and then
- The brute force technique is a straight prove it by induction.
forward technique which solve the
problem with the computability power
without any common sense

3. It doesn't require guessing the answer,

but it may require more algebra than the
substitution method.
 Example
• T (n) = 1 if n=1
= 2T (n-1) if n>1

Repeat the procedure for i times

=

Time Complexity Analysis

• The function grows exponentially with nnn, meaning the time complexity is:
• O(2^n)
Conclusion
• The recurrence represents an exponential time complexity O(2^n),
indicating that the function grows very rapidly as n increases. This suggests
that solving this recurrence directly for large n is computationally expensive.
For example :-
• T (n) = T (n-1) +1 and T (1) = θ (1).
• Sol-

= Where k = n-1 = n-k =0,

= T (n) = θ (1) + (n-1)

= 1+n-1
= n= θ (n).
 Practice questions
• Solve using both substitution and
iteration method,
 Recursion Tree Method
• Recursion is a fundamental concept in
computer science and mathematics that allows
functions to call themselves, enabling the
solution of complex problems through iterative
steps.
• One visual representation commonly used to
understand and analyze the execution of
recursive functions is a recursion tree.
 Recursion Tree

• A recursion tree is a graphical representation that

illustrates the execution flow of a recursive function.
• It provides a visual breakdown of recursive calls,
showcasing the progression of the algorithm as it
branches out and eventually reaches a base case.
• The tree structure helps in analyzing the time
complexity and understanding the recursive process
involved.
 Type of Recursion
• There are two different Recursion as follows-
• Linear Recursion - A function that calls itself just once each
time it executes is said to be linearly recursive.
• A nice illustration of linear recursion is the factorial function.
The name "linear recursion" refers to the fact that a linearly
recursive function takes a linear amount of time to execute.
• Tree Recursion - When you make a recursive call in your
recursive case more than once, it is referred to as tree
recursion.
• An effective illustration of Tree recursion is the fibonacci
sequence. Tree recursive functions operate in exponential
time; they are not linear in their temporal complexity.
 Practice Question
Q.1
Q.2 a) Distinguish between Algorithm and Pseudocode.
b) Define i) Profiling ii) Time Complexity iii) Space Complexity
Q.3 . a) Explain the Asymptotic notations with example.
b) Prove 3n^3 + 2n2 = O(n^3) ; 3n !=O(2n ).
Q.4 How the performance can be analyzed? Explain with the example.
Q.5 Explain recursive functions algorithm analysis with an example

Q.6
 For example
1.
• Here c – order of n
• Cn –> cost required to solve T(n) = 2T(n/2)
Let’s assume n = 16
= n/bk=1
= Solving for k:
= k=log⁡bn
So, we will get (logn)
we use log n because we need to know how many
times we can divide n before reaching 1.
• So the solution for this is O(n log n)

Since logarithmic follow the rule

The depth of the recursion tree is

Since, logb is a constant , it gets absorbed into O-

Notation .
=

• i.e. b is disappear in asymptotic complexity

 Master’s Method
• The master’s method provides a fast method for
solving the recurrence of the form
•
• In Master’s method , one has to remember three
cases, and then the recurrence can be solved very
easily.
• Let

• Case1 :
• Case2:
• Case3 :
Q.1
Q.2