B.

TECH V SEM CSE

ACADEMIC YEAR: 2023-2024

Course Name: Design and Analysis of Algorithm

Topic: Asymptotic Notation- Analysis of Algorithms

Course code : CS 3102

Credits : 4
Mode of delivery : Hybrid (Power point presentation)
Faculty : Mr. Satpal Singh Kushwaha
Email-id : [email protected]
CS3102 (DAA), Department of 1
CSE
Assignment
quiz Assessment
Mid term examination criteria’s
End term Examination

CS3102 (DAA), Department of 2

CSE
Asymptotic Notation- Analysis of Algorithms

CS3102 (DAA), Department of 3

CSE
 Course Information
• Course Handout
• Communicate through eMail
• Office hours
• To be communicated
• Grading policy
• Will be communicated as per university guidelines

CS3102 (DAA), Department of

CSE 4
 Syllabus
• Introduction: Fundamentals of Algorithms, Important Problem Types, Analysis
of algorithm efficiency. Analysis Framework: Asymptotic Notations and Basic
Efficiency Classes. Mathematical Analysis of Nonrecursive and Recursive
Algorithms: Brute force Techniques, Divide and Conquer. Decrease and
Conquer: Insertion Sort, Depth First Search, Breadth First Search, Topological
Sorting. Transform and Conquer: Presorting, BST, Heapsort. Space and Time
tradeoffs: Input Enhancement in String Matching. Dynamic Programming:
Warshall's and Floyd's Algorithms, The Knapsack Problem. Greedy Techniques:
Prim's, Kruskal's and Dijkstra's Algorithm, Huffman Trees. Coping with
limitations of algorithmic power. Backtracking: nQueens problem, Hamiltonian
Circuit Problem, Subset Sum Problem. Branch and Bound: Assignment
Problem, Knapsack Problem, TSP. P, NP, and NP-complete Problems.

CS3102 (DAA), Department of

CSE 5
 More Information

• Textbook
•Introduction to Algorithms 3rd ,Cormen,
Leiserson, Rivest and Stein, The MIT Press,
• Fundamentals of Computer Algorithms,
2nd, Sartaj Sahni, Ellis Horowitz,
Sanguthevar Rajasekaran

• Others
• Introduction to Design & Analysis Computer Algorithm 3rd,
Sara Baase, Allen Van Gelder, Adison-Wesley, 2000.
• Algorithms, Richard Johnsonbaugh, Marcus Schaefer, Prentice
Hall, 2004.
• Introduction to The Design and Analysis of Algorithms 2nd
Edition, Anany Levitin, Adison-Wesley, 2007.

CS3102 (DAA), Department of

CSE 6
 Course Objectives
• CS1501.1 Analyse the running times of algorithms using asymptotic analysis.
• CS1501.2 Demonstrate and Design algorithms using divide-and-conquer
paradigm to solve business problems hence enhance skills.

• CS1501.3 Illustrate the concept of greedy and dynamic-programming approach

to solve real life problems to enhance entrepreneurship capabilities.
• CS1501.4 Demonstrate the concept of backtracking and branch & bound
algorithms.
• CS1501.5 Synthesize and analyse various advanced algorithms concept such as
graphs, string matching, approximation algorithms and complexity classes to
enhance employability.

CS3102 (DAA), Department of

CSE 7
 Asymptotic Notation
• By now you should have an intuitive feel
for asymptotic (big-O) notation:
• What does O(n) running time mean?
O(n2)?
O(n lg n)?
• How does asymptotic running time relate to
asymptotic memory usage?
• Our first task is to define this notation
more formally and completely
CS3102 (DAA), Department of
CSE 8
 Asymptotic Notation
➢ Asymptotic notations are the mathematical notations used to describe the
running time of an algorithm when the input tends towards a particular
value or a limiting value.
➢ For example: In bubble sort, when the input array is already sorted, the
time taken by the algorithm is linear i.e. the best case.
➢ But, when the input array is in reverse condition, the algorithm takes the
maximum time (quadratic) to sort the elements i.e. the worst case.
➢ When the input array is neither sorted nor in reverse order, then it takes
average time. These durations are denoted using asymptotic notations.
➢ There are mainly three asymptotic notations: Theta notation, Omega
notation and Big-O notation.

CS3102 (DAA), Department of

CSE 9
 Big-O Notation (O-notation)
Big-O notation represents the upper bound of the running time of an algorithm.
Thus, it gives the worst-case complexity of an algorithm.

O(g(n)) = { f(n): there exist positive constants c and n0

such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0 }
The above expression can be described as a function f(n) belongs to the set
O(g(n)) if there exists a positive constant c such that it lies between 0 and cg(n),
for sufficiently large n.
For any value of n, the running time of an algorithm does not cross time provided
by O(g(n)).
Since it gives the worst-case running time of an algorithm, it is widely used to
analyze an algorithm as we are always interested in the worst-case scenario.
CS3102 (DAA), Department of
CSE 10
Asymptotic Notation

CS3102 (DAA), Department of

CSE 11
Big-O Visualization
O(g(n)) is the set of
functions with smaller
or same order of
growth as g(n)

CS3102 (DAA), Department of

CSE
12
The Gist of Big-Oh
Take functions f(n) & g(n), consider only
the most significant term and remove
constant multipliers:
▪ 5n+3 → n
▪ 7n+.5n2+2000 → n2
▪ 300n+12+nlogn → n log n

Then compare the functions; if f(n) ≤ g(n),

then f(n) is in O(g(n))

CS3102 (DAA), Department of CSE 13

 Example: n2 + n = O(n 3 )

Proof:
• Here, we have f (n) = n 2 + n, and g(n) = n 3
• Notice that if n ≥ 1, n ≤ n 3 is clear.
2 3
• Also, notice that if n ≥ 1, n ≤ n is clear.
a b
• Side Note: In general, if a ≤ b, then n ≤ n
whenever n ≥ 1. This fact is used often in these
types of proofs.
• Therefore,
n 2 + n ≤ n 3 + n 3 = 2n 3

• We have just shown that

n 2 + n ≤ 2n 3 for all n ≥ 1

• Thus, we have shown that n 2 + n = O ( n 3 )

(by definition of Big-O, with n 0 = 1, and c = 2.)

CS3102 (DAA), 14
 More Examples
• Show that 30n+8 is O(n).
– Show  c,n0: 30n+8  cn, n>n0 .
• Let c=31, n =8.
0 Assume n>n0=8. Then
cn = 31n = 30n + n > 30n+8, so 30n+8 < cn.

CS3102 (DAA), Department of 15

 Big-O example, graphically
• Note 30n+8 isn’t
less than n
anywhere (n>0).
cn =
• It isn’t even
31n 30n+8

Value of function →
less than 31n
everywhere.
• But it is less than
31n everywhere to
30n+8
the right of n=8.
n
O(n)
n>n0=8 →
Increasing n →

CS3102 (DAA), Department of 16

CS3102 (DAA), Department of 17
CS3102 (DAA), Department of 18
CS3102 (DAA), Department of 19
 Examples

– 2n2 = O(n3): 2n2 ≤ cn3  2 ≤ cn  c = 1 and n0= 2

– n2 = O(n2): n2 ≤ cn2  c ≥ 1  c = 1 and n0= 1

– 1000n2+1000n = O(n2):

1000n2+1000n ≤ 1000n2+ n2 =1001n2 c=1001 and n0 = 1000

– n = O(n ):
2 n ≤ cn 2  cn ≥ 1  c = 1 and n = 1
0

CS3102 (DAA), Department of 20

 No Uniqueness
• There is no unique set of values for n0 and c in proving the
asymptotic bounds

• Prove that 100n + 5 = O(n2)

– 100n + 5 ≤ 100n + n = 101n ≤ 101n2

for all n ≥ 5

n0= 5 and c = 101 is a solution

– 100n + 5 ≤ 100n + 5n = 105n ≤ 105n2

for all n ≥ 1

n0= 1 and c = 105 is also a solution

Must find SOME constants c and

CS3102 n0 that
(DAA), satisfy the
Department of asymp221
t2otic notation relation
Examples

▪ True or false?
1. 4+3n is O(n) True
2. n+2 logn is O(log n) False
3. logn+2 is O(1) False
4. n50 is O(1.1n) True

CS3102 (DAA), Department of CSE 22

Examples (cont.)
For f(n)=4n & g(n)=n2, prove f(n) is in O(g(n))
A valid proof is to find valid c and n0
When n=4, f=16 and g=16, so this is the
crossing over point
We can then chose n0 = 4, and c=1

We also have infinitely many others choices for

c and n0, such as n0 = 78, and c=42

CS3102 (DAA), Department of CSE 23

Big Oh: Common Categories
From fastest to slowest
O(1) constant (or O(k) for constant k)
O(log n) logarithmic
O(n) linear
O(n log n) "n log n”
O(n2) quadratic
O(n3) cubic
O(nk) polynomial (where is k is constant)
O(kn) exponential (where constant k > 1)

CS3102 (DAA), Department of CSE 24

Comment on Notation
▪ We say (3n2+17) is in O(n2)
▪ We may also say/write is as
▪ (3n2+17) is O(n2)
▪ (3n2+17) = O(n2)
▪ (3n2+17) ∈ O(n2)

▪ But it’s not ‘=‘ as in ‘equality’:

▪ We would never say O(n2) = (3n2+17)

CS3102 (DAA), Department of CSE 25

 Omega Notation (Ω-notation)

Omega notation represents the lower bound of the running time of an algorithm.
Thus, it provides best case complexity of an algorithm.

Ω(g(n)) = { f(n): there exist positive constants c and n0

such that 0 ≤ cg(n) ≤ f(n) for all n ≥ n0 }
The above expression can be described as a function f(n) belongs to the set
Ω(g(n)) if there exists a positive constant c such that it lies above cg(n), for
sufficiently large n.
For any value of n, the minimum time required by the algorithm is given by
Omega Ω(g(n))
CS3102 (DAA), Department of
CSE 26
Asymptotic Notation

CS3102 (DAA), Department of

CSE 27
CS3102 (DAA), Department of
CSE 28
CS3102 (DAA), Department of
CSE 29
 Examples
– 5n2 = (n)
 c, n0 such that: 0  cn  5n2  cn  5n2  c = 1 and n0 = 1

– 100n + 5 ≠ (n2)
 c, n0 such that: 0  cn2  100n + 5
100n + 5  100n + 5n ( n  1) = 105n
cn2  105n  n(cn – 105)  0
Since n is positive  cn – 105  0  n  105/c
 contradiction: n cannot be smaller than a constant

– n = (2n), n3 = (n2), n = (logn)

CS3102 (DAA), Department of 30
 Asymptotic Notation
Theta Notation (Θ-notation)
Theta notation encloses the function from above and below. Since it represents
the upper and the lower bound of the running time of an algorithm, it is used for
analyzing the average case complexity of an algorithm.

For a function f(n), Θ(g(n)) is given by the relation:

Θ(g(n)) = { f(n): there exist positive constants c1, c2 and n0
such that 0 ≤ c1g(n) ≤ f(n) ≤ c2g(n) for all n ≥ n0 }
The above expression can be described as a function f(n) belongs to the set
Θ(g(n)) if there exist positive constants c1 and c2 such that it can be sandwiched
between c1g(n) and c2g(n), for sufficiently large n.
If a function f(n) lies anywhere in between c1g(n) and c2g(n) for all n ≥ n0, then
f(n) is said to be asymptotically tight bound.

CS3102 (DAA), Department of

CSE 31
Asymptotic Notation

CS3102 (DAA), Department of

CSE 32
CS3102 (DAA), Department of
CSE 33
CS3102 (DAA), Department of
CSE 34
CS3102 (DAA), Department of
CSE 35
 Examples
– n2/2 –n/2 = (n2)

•½n 2 - ½ n ≤ ½ n2 n ≥ 0  c2= ½

• ½ n2- ½ n ≥ ½ n2 - ½ n * ½ n ( n ≥ 2 ) = ¼ n2

 c1= ¼

– n ≠ (n2): c1 n2 ≤ n ≤ c2 n2

 only holds for: n ≤ 1/c1

CS3102 (DAA), Department of 36
 Examples
– 6n3 ≠ (n2): c1 n2 ≤ 6n3 ≤ c2 n2

 only holds for: n ≤ c2 /6

– n ≠ (logn): c1 logn ≤ n ≤ c2 logn

 c2 ≥ n/logn,  n≥ n0 – impossible

CS3102 (DAA), Department of 37