CSE 2215: Data Structures and Algorithms I

Sherajul Arifin
Lecturer, Department of Computer Science and Engineering,
United International University
Time Complexity Analysis

Let T(n) denote its worst-case running time on input instances of size
n. Thus the running time,T(n) satisfies the following recurrence
relation.

T(n) ≤ 2T(n / 2) + O(n)

→T(n) ≤ 2T(n / 2) + cn
Here, c is some arbitrary constant.

Now, the base case for this recurrence?

T(2) ≤ c.
2
Solving Recurrence Relations

Figure: Unrolling the recurrence T(n) ≤ 2T(n/2) + O(n).

3
Solving Recurrence Relations
Method-1: Unrolling the Recursive Tree
Step -1: Analyze the first few steps:
o At the first level of recursion, we have a single problem of
size n, which takes time at most cn plus the time spent in
all subsequent recursive calls.
o At the next level, we have two problems each of size n/2.
Each of these takes time at most cn/2, for a total of at
most cn, again plus the time in subsequent recursive calls.
o At the third level, we have four problems each of size n/4,
each taking time at most cn/4, for a total of at most cn.
4
Solving Recurrence Relations

Step -2: Identifying a pattern:

What’s going on in general?

o At level j of the recursion, the number of subproblems has doubled
j times, so there are now a total of 2j subproblems.
o Each has correspondingly shrunk in size by a factor of two j times,
and so each has size n/2j, and hence each takes time at most
cn/2j .
o Thus level j contributes a total of at most 2j (cn/2j) = cn to the total
running time.

5
Solving Recurrence Relations
Step -3: Summing over all levels of recursion:

We know, the recursion will ‘bottom out’ when the value of input n
reaches n = 1. We assume the recursion will level down x times. Each
time the input size is halved, after x times it will be n/2x

n/2x = 1 Each level contributes = cn

→n = 2x Total run time = # of levels * level

contribution

→x = log2 n = log2 n * cn

= cn log2 n = O(n log n)

6
Solving Recurrences by Master Theorem
Method-2: Use the magic formula
Master Theorem:

Let us consider the following general recurrence

T(n) = aT(n/b) + f(n)

Where, f(n) is a function of the form f(n) = O(nk)

1. If a > bk, T(n) = O(nlogb a)

2. If a = bk, T(n) = O(nlogb a log n)
3. If a < bk, T(n) = O(nk)
7
Solving Recurrences by Master Theorem
For Merge Sort –

T(n) = 2T(n / 2) + O(n)

a = 2, b = 2, k = 1, so, a = bk

Hence, T(n) = O(nlogb a log2 n)

= O(nlog2 2 log2 n)

= O(n1 log n)

= O(n log n)

8
Divide & Conquer (revisited)

9
Divide-and-Conquer Technique
• Divide-and-Conquer is a general algorithm design
paradigm:
o Divide the problem into a number of subproblems that
are smaller
o instances of the same problem
o Conquer the subproblems by solving them recursively
o Combine the solutions to the subproblems into the
solution for the original problem
• The base case for the recursion are subproblems of
constant size
• Analysis can be done using recurrence equations

10
Quick Sort

11
Divide-and-Conquer

12
Merge Sort and Quick Sort
•Two well-known sorting algorithms adopt this divide-and-conquer
strategy

•Merge sort
o Divide step is trivial – just split the list into two equal parts.
o Work is carried out in the conquer step by merging two sorted
lists.

•Quick sort
o Work is carried out in the divide step using a pivot element.
o Conquer step is trivial.

13
Quick Sort
Another divide-and-conquer algorithm
■The array A[p..r] is partitioned into two non-empty subarrays A[p..q] and
A[q+1..r]

p q q+1 r

8 15 4 30 25 7 18 12 8 4 7 12 15 30 25 18

Invariant: All elements in A[p..q] are less than all elements in A[q+1..r]
■The subarrays are recursively sorted by calls to quicksort
■Unlike merge sort, no combining step: two subarrays form an already-sorted
array

14
Quick Sort

15
16
Quicksort -Simulation

17
Quicksort -Simulation

18
Quicksort -Simulation

19
Quicksort -Simulation

20
Quicksort -Simulation

21
Quicksort -Simulation

22
Quicksort -Simulation

23
Quicksort -Simulation

24
Quicksort -Simulation

25
Quicksort -Simulation

26
Quick Sort (Partition)
Clearly, all the actions take place in the partition()function
■Rearranges the subarrays in place
■End result:
Two subarrays
All values in first subarray | pivot | all values in the second
subarray
■Returns the index of the “pivot” element separating the two
subarrays

27
Quick Sort (Analysis)
What will be the worst case for the algorithm?
■Partition is always unbalanced
What will be the best case for the algorithm?
■Partition is perfectly balanced

Which is more likely?

■The partition is almost balanced …
Will any particular input elicit the worst case?
■Yes: Already-sorted input

28
Quick Sort: Worst case running time
The recurrence for the worst-case running time T(n) is [Partition is
always unbalanced]

29
Quick Sort: Best case running time
The recurrence for the best-case running time T(n) is [Partition is
always balanced]

30
Quick Sort: Running Time

31
Quick Sort: Analysis
The real liability of quicksort is that it runs in O(n2) on already-
sorted input Two solutions:
■Randomize the input array, OR
■Pick a random pivot element

How will these solve the problem?

■By ensuring that no particular input can be chosen to make quick-
sort run in O(n2) time
■Assuming random input, average-case running time is much closer
to O(nlog n) than O(n2)

32
Thank you!