Data Structures and Algorithms Lecture 4: Analyzing

Soheila Ashkezari-T. Recursive Code

@SAshkezari

CSE 373 24WI 1

Modeling Recursive Code
Summations
Designing Recursive functions
Example

Data Structures and Algorithms – Ashkezari 2

Repetition in programming
● One way to describe repetition within a computer
program is the use of loops, such as Java’s while-
loop and for-loop constructs.
● An entirely different way to achieve repetition is
through a process known as recursion.
○ In computing, recursion provides an elegant and
powerful alternative for performing repetitive tasks.
○ Modeling and analyzing recursive code is all about
finding patterns in how the input changes between
calls and how much work is done within each call.

Data Structures and Algorithms – Ashkezari 3

The Factorial Function

● recursive definition:
base cases

Recursive cases

● the overall number of operations for

computing factorial(n) is O(n), as there
are n+1 activations, each of which
accounts for O(1) operations.
Data Structures and Algorithms – Ashkezari 4
Linear sum

it will take O(n) time, because it spends a constant amount of time performing the nonrecursive part of each call. Moreover, we can also
see that the memory space used by the algorithm (in addition to the array) is also O(n), as we use a constant amount of memory space
for each of the n+1 frames in the trace at the time we make the final recursive call (with n = 0).

Data Structures and Algorithms – Ashkezari 5

 Reversing a Sequence with Recursion

We note that whenever a recursive call is made, there will be two fewer elements in the relevant portion of the array. (See Figure 5.11.)
Eventually a base case is reached when the condition low < high fails, either because low == high in the case that n is odd, or because low ==
high + 1 in the case that n is even. The above argument implies that the recursive algorithm of Code Fragment 5.7 is guaranteed to terminate
𝑛
after a total of 1 + recursive calls. Because each call involves a constant amount of work, the entire process runs in O(n) time
2

Data Structures and Algorithms – Ashkezari 6

Binary Search (non-recursive)
class BinarySearch {
// Returns index of x if it is present in arr[].
int binarySearch(int arr[], int x)
{
int l = 0, r = arr.length - 1;
while (l <= r) { Is it appropriate for an ordered/sorted
int m = l + (r - l) / 2; list or an unordered one?
// Check if x is present at mid
if (arr[m] == x)
return m;
// If x greater, ignore left half
if (arr[m] < x)
l = m + 1;
// If x is smaller, ignore right half
else
r = m - 1;
}
// If we reach here, then element was
// not present
return -1;
}

Data Structures and Algorithms – Ashkezari 7

Binary Search
public int binarySearch(int[] arr, int toFind, int lo, int hi) {
if( hi < lo ) {
return -1;
}
if(hi == lo) {
if(arr[hi] == toFind) {
return hi;
}
return -1;
}
int mid = (lo+hi) / 2;
if(arr[mid] == toFind) {
return mid;
} else if(arr[mid] < toFind) {
return binarySearch(arr, toFind, mid+1, hi);
} else {
return binarySearch(arr, toFind, lo, mid-1);
}
}

Data Structures and Algorithms – Ashkezari 8

Binary Search Runtime
binary search: Locates a target value in a sorted array or list by successively eliminating half
of the array from consideration.
● Example: Searching the array below for the value 42:
index 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
value -4 2 7 10 15 20 22 25 30 36 42 50 56 68 85 92 103

min min mid

mid min max
mid max

How many elements will be examined?

● What is the best case?
element found at index 8, 1 item examined, O(1)
● What is the worst case?
Take a guess! What is the tight Big-
element not found, ½ elements examined, then ½ of that… O of worst case binary search?
Pattern #1 – Halving the input
Data Structures and Algorithms – Ashkezari 9
Binary Search runtime
For an array of size N, it eliminates ½ until 1
element remains.
N, N/2, N/4, N/8, ..., 4, 2, 1
○ How many divisions does it take?
○ N/2x = 1 ➔ 2x = N

Think of it from the other direction:

○ How many times do I have to multiply by 2 to reach N?
1, 2, 4, 8, ..., N/4, N/2, N
○ Call this number of multiplications "x".

2x =N
x = log2 N

Binary search is in the logarithmic

complexity class.
Data Structures and Algorithms – Ashkezari 10
Binary Search Code Model
Let F(n) represent the worst case runtime for binary search
public int binarySearch(int[] arr, int toFind, int lo, int hi) {
if( hi < lo ) { +1
return -1; +1
} if(hi == lo) { +1 base case Approach
if(arr[hi] == toFind) { +2 - Model the base case first (non recursive)
best case: +2 - Model any non-recursive work in the recursive case
return hi;
} +1 worst case: +5 - Write a recursive math function for recursive work
return -1; +1
}
int mid = (lo+hi) / 2; +3
if(arr[mid] == toFind) { +2
return mid; +1
} else if(arr[mid] < toFind) { +2 +? recursive case
return binarySearch(arr, toFind, mid+1, hi); +? best case: +6
} else { +? worst case: +7 + ???
return binarySearch(arr, toFind, lo, mid-1); +?
}
}

Data Structures and Algorithms – Ashkezari 11

Meet the Recurrence
A recurrence relation is an equation that defines a sequence based on a rule
that gives the next term as a function of the previous term(s)
It’s a lot like recursive code:
● At least one base case and at least one recursive case
● Each case should include the values for n to which it corresponds
● The recursive case should reduce the input size in a way that eventually triggers the
base case
● The cases of your recurrence usually correspond exactly to the cases of the code

Data Structures and Algorithms – Ashkezari 12

Example: Write a Recurrence
public int recursiveFunction(int n){
if(n < 3) { +1
return 3; +1 base case: +2
}
for(int int i=0; i < n; i++) {
System.out.println(i); +1 *n
}
non-recursive work: n+2
int val1 = recursiveFunction(n/3);
int val2 = recursiveFunction(n/3); recursive work: 2F(n/3)

return val1 * val2; +2

}

Data Structures and Algorithms – Ashkezari 13

Recurrence to Big Θ Techniques
A recurrence is a mathematical function that includes itself in its definition
This makes it very difficult to find the dominating term that will dictate the asymptotic growth
Solving the recurrence or “finding the closed form” is the process of eliminating the recursive
definition. So far, we’ve seen three methods to do so: Master Theorem

1. Apply Master Theorem

○ Pro: Plug and chug convenience
○ Con: only works for recurrences of a certain format If then

2. Unrolling
then
If
then
If
○ Pro: Least complicated setup
○ Con: requires intuitive pattern matching T(1) = d
T(2) = 2T(2-1) + c = 2(d) + c
3. Tree Method T(3) = 2T(3-1) + c = 2(2(d) + c) + c = 4d + 3c
○ Pro: Plug and chug
f(4)

T(4) = 2T(4-1) + c = 2(4d + 3c) + c = 8d + 7c

○ Con: Complex setup f(3) f(3)

T(5) = 2T(5-1) + c = 2(8d + 7c) + c = 16d +25c

f(2) f(2) f(2) f(2)

f(1) f(1) f(1) f(1) f(1) f(1) f(1) f(1)

Data Structures and Algorithms – Ashkezari 14

Recurrence to Big-Θ
It’s still really hard to tell what the big-O is just by looking at it.
But fancy mathematicians have a formula for us to use!

Master Theorem

If then
If then
If then

Data Structures and Algorithms – Ashkezari 15

Recurrence to Big-Θ
Binary Search algorithm:

Master Theorem
a=1, b=2, c=0

𝑙𝑜𝑔21 = 0 == c
If then
Therefore
If then
If then 𝐹 𝑛 = Θ 𝑛0 log 𝑛 = Θ log 𝑛

Data Structures and Algorithms – Ashkezari 16

Recurrence to Big-Θ
Binary Search algorithm:

Compute it using Unrolling method?

Data Structures and Algorithms – Ashkezari 17

Understanding Master Theorem
Master Theorem The log of a < c case
○ Recursive case does a lot of non recursive work in
comparison to how quickly it divides the input size
○ Most work happens in beginning of call stack
○ Non recursive work in recursive case dominates growth,
nc term

If then
The log of a = c
If then ○ Recursive case evenly splits work between non recursive
If then work and passing along inputs to subsequent recursive
calls
○ Work is distributed across call stack
● a measures how many recursive calls are
triggered by each method instance The log of a > c case
○ Recursive case breaks inputs apart quickly and doesn’t do
● b measures the rate of change for input much non recursive work
● c measures the dominating term of the non
○ Most work happens near bottom of call stack
recursive work within the recursive method
● d measures the work done in the base case
Data Structures and Algorithms – Ashkezari 18
 Understanding Master Theorem
More general form:

Master Theorem

If then
If then
If then

Data Structures and Algorithms – Ashkezari 19

Questions?

Data Structures and Algorithms – Ashkezari 20

Recursive Patterns
●Pattern #1: Halving the Input
Binary Search Θ(logn)
●Pattern #2: Constant size input and doing work
Merge Sort
●Pattern #3: Doubling the Input

Data Structures and Algorithms – Ashkezari 21

 0 1 2 3 4
Merge Sort 8 2 57 91 22
0 1 0 1 2
8 2 57 91 22
mergeSort(input) {
if (input.length == 1) +2
return +1
base case: +3 0 0 0 0 1
else n/2 2 57 91 22
8
smallerHalf = mergeSort(input [0, ..., mid])
largerHalf = mergeSort(input [mid + 1, ...]) n/2
return merge(smallerHalf, largerHalf) +n recursive work: 2F(n/2)
0 0
} 91 22
non-recursive work: n

0 1
22 91

0 1 0 1 2
Pattern #2 – Constant size input and doing work 2 8 22 57 91

0 1 2 3 4
2 8 22 57 91

Data Structures and Algorithms – Ashkezari 22

Merge Sort Recurrence to Big-Θ

Master Theorem

If then
If then
If then

Data Structures and Algorithms – Ashkezari 23

Merge Sort Recurrence to Big-Θ

Compute it using Unrolling method?

Data Structures and Algorithms – Ashkezari 24

Questions?

Data Structures and Algorithms – Ashkezari 25

Recursive Patterns
●Pattern #1: Halving the Input
Binary Search - Θ(logn)
●Pattern #2: Constant size input and doing work
Merge Sort - Θ(nlogn)

●Pattern #3: Doubling the Input

Calculating Fibonacci

Data Structures and Algorithms – Ashkezari 26

 Calculating Fibonacci
f(5)

public int fib(int n) {

if (n <= 1) {
f(4) f(3)
return 1;
}
return fib(n-1) + fib(n-2); f(3) f(2) f(2) f(1)
}

● Each call creates 2 more calls f(2) f(1) f(1) f(0) f(1) f(0)
● Each new call has a copy of the
input,
● Almost doubling the input at f(1) f(0)

each call
Pattern #3 – Doubling the Input

Data Structures and Algorithms – Ashkezari 27

Calculating Fibonacci Recurrence to Big-Θ
public int fib(int n) {
if (n <= 1) {
return 1; C0
}
return fib(n-1) + fib(n-2); 2T(n-C1) + C2
}
Can we intuit a pattern? (“unrolling”)
T(1) = d
T(2) = 2T(2-1) + c = 2(d) + c
Apply Master Theorem… T(3) = 2T(3-1) + c = 2(2(d) + c) + c = 4d + 3c
Master Theorem
T(4) = 2T(4-1) + c = 2(4d + 3c) + c = 8d + 7c
T(5) = 2T(5-1) + c = 2(8d + 7c) + c = 16d +25c
Uh oh, our model
Looks like something’s happening but it’s tough
If then doesn’t match
If then
that format… Maybe geometry can help!
then
If
Data Structures and Algorithms – Ashkezari 28
 Calculating Fibonacci Recurrence to Big-Θ
How many layers in the function call tree?
How many layers will it take to transform
“n” to the base case of “1” by subtracting 1 f(5)
For our example, 4 -> Height = n
f(4) f(3)

How many function calls per layer?

f(3) f(2) f(2) f(1)
Layer Function calls
How many function calls on layer k?
1 1
2k-1 f(2) f(1) f(1) f(0) f(1) f(0)
2 2
How many function calls TOTAL
3 4 for a tree of k layers?
4 8 f(1) f(0)
1 + 2 + 4 + 8 +… + 2k-1

Data Structures and Algorithms – Ashkezari 29

 Calculating Fibonacci Recurrence to Big-Θ
Patterns found:
How many layers in the function call tree? n
How many function calls on layer k? 2k-1
How many function calls TOTAL for a tree of k layers?
1 + 2 + 4 + 8 + … + 2k-1
Total runtime = (total function calls) x (runtime of each function call)
Total runtime = (1 + 2 + 4 + 8 + … + 2k-1) x (constant work)
Summation Identity
Finite Geometric Series
1 + 2 + 4 + 8 + … + 2k-1 =

Data Structures and Algorithms – Ashkezari 30

 Recursive Patterns

Pattern #1: Dividing the Input Pattern #2: Constant input + work Pattern #3: Exponential Input
Binary Search Θ(log2n) Merge Sort Θ(nlogn) Calculating Fibonacci Θ(2n)

Data Structures and Algorithms – Ashkezari 31

Questions?

Data Structures and Algorithms – Ashkezari 32

Modeling Recursive Code
Summations
Designing Recursive functions
Example

Data Structures and Algorithms – Ashkezari 33

Modeling Complex Loops
Write a mathematical model of the following code

for (int i = 0; i < n; i++) {

for (int j = 0; j < i; j++) { f(n) = n2
System.out.println(“Hello!”); +1 n n
}
}

Keep an eye on loop

bounds!

Data Structures and Algorithms – Ashkezari 34

Modeling Complex Loops
for (int i = 0; i < n; i++) {
for (int j = 0; j < i; j++) {
System.out.print(“Hello! ”); +1
}
System.out.println();
}

T(n) = What is the Big O?

Definition: Summation

= f(a) + f(a + 1) + f(a + 2) + … + f(b-2) + f(b-1) + f(b)

Data Structures and Algorithms – Ashkezari 35

Simplifying Summations
for (int i = 0; i < n; i++) {
for (int j = 0; j < i; j++) { simplified
System.out.println(“Hello!”); tight big O
}
}
𝒊

= σ𝑛−1
𝑖=0 (1 + 1 + … + 1)

Summation of a constant Factoring out a constant Gauss’s Identity

Data Structures and Algorithms – Ashkezari 36

Modeling Recursive Code
Summations
Designing Recursive functions
Example

Data Structures and Algorithms – Ashkezari 37

Bad Implementation of Fibonacci

O(2𝑛 )

Data Structures and Algorithms – Ashkezari 38

Good Implementation of Fibonacci

O(n)

Data Structures and Algorithms – Ashkezari 39

Example: Element Uniqueness

Data Structures and Algorithms – Ashkezari 40

Example: Element Uniqueness

O(n log n)
O(n log n)
O(n)

Data Structures and Algorithms – Ashkezari 41

Example: Element Uniqueness

T(n) = 2T(n-1)+c ➔ O(2𝑛 )

As a base case, when n = 1, the elements are trivially unique. For n ≥ 2, the elements are unique if and
only if the first n−1 elements are unique, the last n−1 items are unique, and the first and last elements are
different (as that is the only pair that was not already checked as a subcase).

Data Structures and Algorithms – Ashkezari 42

Quiz

Write a better recursive function for Element Uniqueness example

Data Structures and Algorithms – Ashkezari 43

Modeling Recursive Code
Summations
Designing Recursive functions
Example
Using Recursion Tree for complexity analysis

Data Structures and Algorithms – Ashkezari 45

Using Recursion Tree for complexity analysis

Question 1: T(n) = 2T(n/2) + c

Step 1: Draw a recursive tree

Step 2: Calculate the work done or cost at each level and count total no of levels in recursion tree
Choose the longest path from root node to leaf node : n/20 -→ n/21 -→ n/22 -→ ……… -→ n/2k
At last level size of problem becomes 1 ➔ n/2k= 1
2k = n ➔ k = log2(n) ‫ تعداد‬:‫ارتفاع درخت‬
‫یالها در طوالنی ترین‬
‫مسیر‬

‫عمق (سطح) درخت‬

Total no of levels in recursive tree = k +1 = log2(n) + 1

Data Structures and Algorithms – Ashkezari 46

Using Recursion Tree for complexity analysis

•Step 3: Count total number of nodes in the last level and calculate cost of last level
No. of nodes at level 0 = 20 = 1
No. of nodes at level 1 = 21 = 2
………
No. of nodes at level log2(n) (last level) = 2log2(n) = nlog2(2) = n

Cost of sub problems at level log2(n) (last level) = nxT(1) = nx1 = n

•Step 4: Sum up the cost all the levels in recursive tree

T(n) = c + 2c + 4c + ... + c(no. of levels-1) times + last level cost
𝑛
= c + 2c + 4c + ... + c 2𝑙𝑜𝑔2 + Θ(n)
𝑛
= c(1 + 2 + 4 + ... +c 2𝑙𝑜𝑔2 ) + Θ(n)
𝑛
➔ 20 + 21 + 22 + ... + 2𝑙𝑜𝑔2 times ➔ Geometric Progression(G.P.)
𝑛
2𝑙𝑜𝑔2 −1
= 2−1 c + Θ(n)

Thus, T(n) = Θ(n) Data Structures and Algorithms – Ashkezari 47

Using Recursion Tree for complexity analysis

• Question 2: T(n) = T(n/10) + T(9n/10) + n

Step 1: Draw a recursive tree

Step 2: Calculate the work done or cost at each level and count total no of levels in recursion tree
Choose the longest path from root node to leaf node :
(9/10)0n –> (9/10)1n –> (9/10)2n –> ……… –> (9/10)kn
Size of problem at last level = (9/10)kn
‫ارتفاع درخت‬
At last level size of problem becomes 1
(9/10)kn = 1 ➔ (9/10)k = 1/n ➔ k = log10/9(n)
‫عمق (سطح) درخت‬
Total no of levels in recursive tree = k +1 = log10/9(n) + 1
Data Structures and Algorithms – Ashkezari 48
Using Recursion Tree for complexity analysis
T(n) = T(n/10) + T(9n/10) + n
•Step 3: Count total number of nodes in the last level and calculate cost of last level
No. of nodes at level 0 = 20 = 1
No. of nodes at level 1 = 21 = 2
……………………
No. of nodes at level log10/9(n) =
Cost of sub problems at last level
note that it will be skewed tree as we have left child having n/10 value and right child
having 9n/10 value (division factor for the left child is much higher). So the last level
would have constant number of nodes or Θ(1) and there work done at the last level is
Θ(1) * Θ(1) = Θ(1)

•Step 4: Sum up the cost all the levels in recursive tree

•note that we consider a perfectly full tree till the second last level here
for simplicity and calculate an upper bound
T(n) < n + n + n + … + (no. of levels – 1) times + last level cost
< n + n + n + … log10/9(n) times + Θ(1)
< nlog10/9(n) + Θ(1)

Thus, T(n) = O(nlog10/9(n)) Θ(nlog10/9(n))

Data Structures and Algorithms – Ashkezari 49