B.

TECH V SEM CSE

ACADEMIC YEAR: 2022-2023

Course Name: Design and Analysis of Algorithm

Topic: Introduction to Algorithms, Specification of Algorithm

Course code : CS 3102

Credits : 4
Mode of delivery : Physical
Faculty : Mr. Tarun Jain
Email-id : [email protected]
1
 Assignment
quiz Assessment
Mid term examination
criteria’s
End term Examination

11/01/2022 CS3102 (DAA), Dept. of CSE 2

 Mathematical Analysis of Recursive techniques

11/01/2022 CS3102 (DAA), Dept. of CSE 3

 Course Information
• Course Handout
• Communicate through eMail
• Office hours
• To be communicated
• Grading policy
• Will be communicated as per university guidelines

CS3102 (DAA), Dept. of CSE

11/01/2022 4
 Syllabus
• Introduction: Fundamentals of Algorithms, Important Problem Types,
Analysis of algorithm efficiency. Analysis Framework: Asymptotic Notations
and Basic Efficiency Classes. Mathematical Analysis of Nonrecursive and
Recursive Algorithms: Brute force Techniques, Divide and Conquer. Decrease
and Conquer: Insertion Sort, Depth First Search, Breadth First Search,
Topological Sorting. Transform and Conquer: Presorting, BST, Heapsort. Space
and Time tradeoffs: Input Enhancement in String Matching. Dynamic
Programming: Warshall's and Floyd's Algorithms, The Knapsack Problem.
Greedy Techniques: Prim's, Kruskal's and Dijkstra's Algorithm, Huffman Trees.
Coping with limitations of algorithmic power. Backtracking: nQueens problem,
Hamiltonian Circuit Problem, Subset Sum Problem. Branch and Bound:
Assignment Problem, Knapsack Problem, TSP. P, NP, and NP-complete
Problems.

CS3102 (DAA), Dept. of CSE

11/01/2022 5
 More Information
• Textbook
• Introduction to Algorithms 3rd ,Cormen,
Leiserson, Rivest and Stein, The MIT Press,
• Fundamentals of Computer Algorithms,
2nd, Sartaj Sahni, Ellis Horowitz,
Sanguthevar Rajasekaran
• Others
• Introduction to Design & Analysis
Computer Algorithm 3rd,
Sara Baase, Allen Van Gelder, Adison-
Wesley, 2000.
• Algorithms, Richard Johnsonbaugh, Marcus Schaefer, Prentice
Hall, 2004.
• Introduction to The Design and Analysis of Algorithms 2nd
Edition, Anany Levitin,
11/01/2022
Adison-Wesley,
CS3102 2007.
(DAA), Dept. of CSE
6
 Course Objectives
• CS1501.1 Analyse the running times of algorithms using asymptotic analysis.
• CS1501.2 Demonstrate and Design algorithms using divide-and-
conquer paradigm to solve business problems hence enhance skills.
• CS1501.3 Illustrate the concept of greedy and dynamic-programming approach
to solve real life problems to enhance entrepreneurship capabilities.
• CS1501.4 Demonstrate the concept of backtracking and branch &
bound algorithms.
• CS1501.5 Synthesize and analyse various advanced algorithms concept such as
graphs, string matching, approximation algorithms and complexity classes to
enhance employability.

CS3102 (DAA), Dept. of CSE

11/01/2022 7
Definition
⚫ A recurrence relation, is a recursive function of integer
T(n), variable n.
⚫ Like all recursive functions, it has both recursive case and base
case.
⚫ Example:

⚫ The portion of the definition that does not contain T is called the
base case of the recurrence relation; the portion that contains T is
called the recurrent or recursive case.

⚫ Recurrence relations are useful for expressing the running times

(i.e., the number of basic operations executed) of recursive
8
algorithms
Example: Fibonacci Sequence
⚫The Fibonacci is defined by the
sequence recurrence
relation
⚫ fn = fn-1 + fn-2, n>=3
and initial conditions
⚫f 1 = 1, f2 = 1.

⚫Fibonacci Sequence: 1, 1, 2, 3, 5, 8, 13, 21,…….

9
Forming Recurrence Relation
⚫ Example 1: Write the recurrence relation for the following method.

void f(int n) {
if (n > 0)
{
cout<<n;
f(n-1);
} }

⚫ The base case is reached when n == 0. The method performs one

comparison. Thus, the number of operations when n == 0, T(0), is some
constant a.
⚫ When n > 0, the method performs two basic operations and then calls itself,
using ONE recursive call, with a parameter n – 1.
⚫ Therefore the recurrence relation is:
T(0) = a where a is constant
T(n) = b + T(n-1) where b is constant, 10
n>0
Forming Recurrence Relation
⚫ Example 2: Write the recurrence relation for the following
method.
int g(int n) {
if (n == 1)
return 2;
else
return g(n /
2) + g( n
/ 2) + 5;
⚫ The }base case is reached when n == 1. The method performs
one comparison and one return statement. Therefore, T(1), is constant
c.
⚫ When n>1, the method performs TWO recursive calls, each with the
parameter n/2, and some constant # of basic operations.
⚫ Hence, the recurrence relation is:

11
 Solving Recurrences
• Iteration Method
• Recursion Tree Method
• Master Method
• Change of Variable Method
 Iteration method
⚫ In this method the recurrence is expanded as summation of
terms and finally the summation provides the solution
⚫ Inevaluating the summation one or more of
the following summation formulae may be used:
⚫ Arithmetic series:

•Special Cases of Geometric Series:

⚫ Geometric Series:

13
 Iteration method
 Harmonic Series:

 Others:

14
 The Method of Iteration
• Let {ai} be the sequence defined by:

ak = ak−1 + 2 with a0 = 1.
• Plugging values of k into the relation, we get:
a1 = a0 + 2 = 1 + 2

a2 = a1 + 2 = 1 + 2 + 2 = 1 + 2(2)

a3 = a2 + 2 = 1 + 2 + 2 + 2 = 1 + 3(2)

a4 = a3 + 2 = 1 + 2 + 2 + 2 + 2 = 1 + 4(2)
• Continuing in this fashion reinforces the apparent pattern that
an = 1 + n(2) = 1 + 2n.
• This brute force technique is the Method of Iteration.
 The Iteration Method
T(n) = c + T(n/2)

T(n) = c + T(n/2) T(n/2) = c + T(n/4)

= c + c + T(n/4)
T(n/4) = c + T(n/8)
= c + c + c + T(n/8) +---+ T(n/2i)

In ith step n/2i = 1

so n = 2i
by the property of log i = log2n

T(n) = c + c + … + c + T(1)
i times
= c*i + T(1)
= clgn + T(1)
16
= Θ(lgn)
Iteration Method – Example
Solve using iteration method T(n) = n + 2T(n/2)

T(n/2) = n/2 + 2T(n/4)

T(n) = n + 2T(n/2)
= n + 2(n/2 + 2T(n/4))
= n + n + 4T(n/4)
= n + n + 4(n/4 + 2T(n/8))
= n + n + n + 8T(n/8) +---+ 2iT(n/2i)
= i*n + 2iT(n/2i)
In ith step n/2i = 1
so n = 2i
by the property of i = log2n

= in + 2iT(1)
= nlgn + nT(1) = Θ(nlgn) 17
Iteration Method
Iteration Method cont……
 Recursion-tree method
• A recursion tree models the costs (time) of a
recursive execution of an algorithm.
• The recursion tree method is good for
generating guesses for the substitution
method.
• Convert the recurrence into a tree:
• – Each node represents the cost incurred at
various levels of recursion
• – Sum up the costs of all levels
Solving Recurrences using Recursion Tree Method

• Here while solving recurrences, we divide the problem into

subproblems of equal size.

• For e.g., T(n) = a T(n/b) + f(n) where a > 1 ,b > 1 and f(n) is a given
function .

F(n) is the cost of splitting or combining the sub problems.

T n/b T n/b
 Example 1
W(n) = 2W(n/2) + n2

• Subproblem size at level i is: n/2i

• Subproblem size hits 1 when 1 = n/2i  i = lgn
• Cost of the problem at level i = (n/2i)2 No. of nodes at level i = 2i
• Total cost: lg n1
2 lg n1 i

i 1
n 1 1
W (n)   2 W (1)  n     n  n     O(n) n
lg n 2 2 2
 O(n)  2n 2
i
i 0 2 i 0  2  i 0  2  1 1 2
 W(n) = O(n2) 10
 Example 2
T(n) = 3T(n/4) + cn2

• Subproblem size at level i is: n/4i

• Subproblem size hits 1 when 1 = n/4i  i = log4n
• Cost of a node at level i = c(n/4i)2
• Number of nodes at level i = 3i  last level has 3log n = nlog 3 nodes
4 4

• Total cost:
T (n) 
log 4 n1 i
 3  cn   logn 3
2

   163 cn i 2 
  
cn  2 n  2

 16 4 log 4 3
n
log 4 3 1 3
 i0
=  i0 1 O(n )
  16 11
 2
 Example 3 (simpler proof)
W(n) = W(n/3) + W(2n/3) + n

• The longest path from the root to a

leaf is: n
 (2/3)n  (2/3)2 n  …  1
• Subproblem size hits 1 when 1
=
(2/3)in  i=log3/2n
•
Cost of the problem at level i = n
•
Total cost:
lg n
W (n)  n  n  ...  n(log
3/ n)  n
2
 O(n lg
3
n) lg 2
34
 W(n) = O(nlgn)
 Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:

L2.35
 Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
T(n)

L2.36
 Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2
T(n/4) T(n/2)

L2.37
 Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2
(n/4)2 (n/2)2

T(n/16) T(n/8) T(n/8) T(n/4)

L2.38
 Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2
(n/4)2 (n/2)2

(n/16)2 (n/8)2 (n/8)2 (n/4)2

…

Q(1)

L2.39
 Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2
(n/4)2 (n/2)2

(n/16)2 (n/8)2 (n/8)2 (n/4)2

…

…
…

…
…

Q(1) Q(1) Q(1) Q(1) Q(1) Q(1) Q(1) Q(1)

 Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2
(n/4)2 (n/2)2

(n/16)2 (n/8)2 (n/8)2 (n/4)2

…

…
…

…
…

Q(1) Q(1) Q(1) Q(1) Q(1) Q(1) Q(1) Q(1)

 Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2
(n/4)2 (n/2)2

(n/16)2 (n/8)2 (n/8)2 (n/4)2

…

…
…

…
…

Q(1) Q(1) Q(1) Q(1) Q(1) Q(1) Q(1) Q(1)

 Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2
(n/4)2 (n/2)2

(n/16)2 (n/8)2 (n/8)2 (n/4)2

…
…

…
…

…
…

Q(1) Q(1) Q(1) Q(1) Q(1) Q(1) Q(1) Q(1)

 Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2
(n/4)2 (n/2)2

(n/16)2 (n/8)2 (n/8)2 (n/4)2

…
…

…
…

…
…

Q(1) Q(1) Q(1) Q(1) Q(1) Q(1) Q(1) Q(1) )

 Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2
(n/4)2 (n/2)2

(n/16)2 (n/8)2 (n/8)2 (n/4)2

…
…

…
…

…
…

Q(1) Q(1) Q(1) Q(1) Q(1) Q(1) Q(1) Q(1) )

 Example of recursion tree

Solve T(n) = T(n/4) + T(n/2) + n2:

n2
(n/4)2 (n/2)2

(n/16)2 (n/8)2 (n/8)2 (n/4)2

…
…

…
…

…
…

Q(1) Q(1) Q(1) Q(1) Q(1) Q(1) Q(1) Q(1) )

n/2i = 1 => n = 2i => i = log2n
Example of recursion tree
+ 2i *T(1)

)i + 2log2n

For simplicity assume it as an infinity series

)i + nlog22

So T(n) = n2 (1 / (1-5/16)) + n

T(n) = n2 (11/16) + n

So T(n) = Q(n2)
Appendix: geometric series

for x ¹ 1

for |x| < 1

1) T(n) = 2T(n/2) + n

The recursion tree for this recurrence is :

n
n
T n/2 T n/2
n/2 n/2 n

log2 n n/22 n/22 n/22 n/22

:
:
:

1 1 1 1 1 1 1
When we add the values across the levels of the recursion tree, we get a
value of n for every level.

We have n + n + n + …… log n times

= n (1 + 1 + 1 + …… log n times)
= n (log2 n)
= Ɵ (n log n)

T(n) = Ɵ (n log n)
II.
Given : T(n) = 2T(n/2) + 1
Solution : The recursion tree for the above recurrence is

T(n/2) T(n/2)
II.
Given : T(n) = 2T(n/2) + 1
Solution : The recursion tree for the above recurrence is

1 1

T(n/4) T(n/4) T(n/4) T(n/4)

II.
Given : T(n) = 2T(n/2) + 1
Solution : The recursion tree for the above recurrence is

1 1

1 1 2

log n 1 1 1 1 4

:
:
:
 Now we add up the costs over all levels of the recursion tree, to determine
the cost for the entire tree :

We get series like

1 + 2 + 22 + 23 + …… log n times which is a G.P.

[ So, using the formula for sum of terms in a G.P. :

a + ar + ar2 + ar3 + …… + ar n – 1 = a( r n – 1 )
r – 1 ]
= 1 (2log n – 1)
2–1
= n–1
= Ɵ (n – 1) (neglecting the lower order terms)
= Ɵ (n)
 Example (using change the variable method)
Domain transformations can sometimes be used to substitute a function for the
argument of the relation and make it easier to solve.

Consider the following recurrence and find out the upper bound
T(n) = 2∗T(√n) + logn
Sol: So we are given T(n) = 2∗T(√n) + logn
To simplify this let n = 2m (or m = log n)
⇒ T(2m) = 2∗T(√ 2m) + log(2m)
⇒ T(2m) = 2∗T(2m/2) + m
now to simplify more let S(m) = T(2m)
⇒ S(m) = 2∗S(m/2) + m
now we can guess the solution very easily so S(m) = O( m log m)
now changing back from S(m) to T(n) we will obtain
T(n) = T(2m) = S(m) = O (m log m)
so T(n) = O(log n log log n)
MATHEMATICAL ANALYSIS OF NON-RECURSIVE
ALGORITHMS

General Plan for Analysis

Decide on parameter n indicating input size
Identify algorithm’s basic operation
Determine worst, average, and best cases for input of size n
Set up a sum for the number of times the basic operation is
executed
Simplify the sum using standard formulas and rules
Example 1: Maximum element

C(n) є Θ(n)
Example 2: Element uniqueness problem

C(n) є Θ(n2)
Example 3: Matrix multiplication

C(n) є Θ(n3)