Lecture 2

Measuring Efficiency of an
Algorithm
Time Complexity Analysis using
Random Access Machine
(RAM)
By
Dr. Bonface Ngari
By the end of this lesson the learners should
be able to:
1. Describe the Tools for Comparing Algorithms
Objectively
2. Measure the efficiency of an algorithm using RAM
Method
3. Measure the efficiency of an algorithm using
Exact computational Method
4. Differentiate computational analysis and
asymptotic analysis of algorithms
Introduction
How do we measure the Efficiency of an
Algorithm?

1. There are often many approaches (algorithms)

to solve a problem. How do we choose
between them?
2. At the heart of computer program design are
two (sometimes conflicting) goals:
a) To design an algorithm that is easy to
understand, code, debug.
b) To design an algorithm that makes efficient
use of the computer’s resources.
Introduction(cont)

• Goal (1)  To design an algorithm that is easy to

understand, code, debug, is the concern of Software
Engineering.

• Goal (2)  To design an algorithm that makes efficient

use of the computer’s resources, is the concern of
data structures and algorithm analysis.

• When goal (2) is important,

how do we measure an algorithm’s cost?
How to Measure Efficiency?

1. Empirical comparison (run programs)

2. Asymptotic Algorithm Analysis
Critical resources 
Factors affecting running time:

• For most algorithms, running time depends on “size” of the

input.
• Running time is expressed as T(n) for some function T on input
size n.
• Tools for Comparing Algorithms Objectively:
• Exact Computational Complexity Analysis
• Worse-case, Best-case and Average-Case Behaviors
• The Big-Oh Notation
• Worse-case Analysis of Algorithms
• Space Complexity
• Growth Rates of Order Classes
• Why Logarithms are Important in Algorithm Design.
 Example: Bubble sort running time
• The bubble step take (n2-n)/2 “steps”

• Assume the bubble sort takes the following number

of steps on specific CPUs:
• Intel Pentium IV CPU: 58*(n2-n)/2
• Motorola CPU: 84.4*(n2-2n)/2
• Intel Pentium V CPU: 44*(n2-n)/2

• Notice that each has an n2 term

• As n increases, the other terms will drop out

7
 Bubble sort running time (cont.)

• This leaves us with:

• Intel Pentium IV CPU: 29n2
• Motorola CPU: 42.2n2
• Intel Pentium V CPU: 22n2

• As processors change, the constants will always change

• The exponent on n will not

• Thus, we can’t care about the constants

8
 Comparing Algorithms: an Analytical Approach
(Asymptotic Algorithm Analysis)
• How to solve “which algorithm” problems without machines or test data
 Primitives
• Primitive operations
• x=4 assignment
• ... x + 5 ... arithmetic
• if (x < y) ... comparison
• x[4] index an array
• *x dereference (C)
• x.foo( ) calling a method
• Others
• new/malloc memory usage
 How many Bestoos?
for (j = 1; j <= N; ++j) {
for (k = 1; k <= M; ++k) {
besto( );
}
}

N M
ΣΣ 1 = NM
j = 1k =
1
 How many Bestoos?
for (j = 1; j <= N; ++j) {
for (k = 1; k <= j; ++k) {
besto( );
}
}

N j N N (N + 1)
ΣΣ 1 = Σ j =
2
j = 1k = j=1
1
 How many bestoos?
for (j = 0; j < N; ++j) {
for (k = 0; k < j; ++k) {
besto( ); N(N + 1)/2
}
}
for (j = 0; j < N; ++j) {
for (k = 0; k < M; ++k) {
besto( );
} NM
}
 How many Bestoos?
void foo(int N) { T(0) = T(1) = T(2) = 1
if(N <= 2) T(n) = 1 + T(n/2) if n > 2
return;
besto(N / 2); T(n) = 1 + (1 + T(n/4))
} = 2 + T(n/4)
= 2 + (1 + T(n/8))
= 3 + T(n/8)
= 3 + (1 + T(n/16))
= 4 + T(n/16)
…
≈ log2 n
The trick

ank + bnk-1 + … + yn + z

ank

nk
Complexity Analysis

Tools for Comparing Algorithms Objectively

• Computational and Asymptotic Complexity

• The “Big-Oh” Notation
Complexity Analysis

Tools for Comparing Algorithms Objectively

• Computational and Asymptotic Complexity

• The “Big-Oh” Notation
Complexity Analysis:
Computational and Asymptotic Complexity

Algorithm #1
Problem solve Algorithm #2  Efficiency
…
Algorithm #n

Grow
proportionally with


 
the amount of data

Complexity Analysis:
Computational and Asymptotic Complexity

Algorithm #1
Algorithm #2  Efficiency
…
Algorithm #n
Compare?

Computational Complexity :

Measure the degree of difficulty of an algorithm

Complexity Analysis:
Computational and Asymptotic Complexity

How much effort ?

Computational
Complexity : How costly ?

Various ways of measuring

Our concern  efficiency criteria of time an

Focus
How can we measure time and space?
 Exact Computational Complexity
Analysis
• Exact Time Complexity (unit: seconds)
• measure of an algorithm’s total time taken.
• Example: Two algorithms A and B, solving a problem instance of P, can take 3 and
100 millisecs.
• Useful
• as an initial gauge of performance.
• for profiling and finding bottlenecks.
• Therefore, A and B would have Time Complexities of:
• 100 * 4 (ns) and
• 1000 * 4 (ns)
• We had assumed
• that each work step consumed the same number of cycles.
• input data model is the same for both algorithms.
• features of the execution platform were the same.

Exact Computational Complexity is the total discrete

amount of work done by the algorithm.

Exact Time Complexity is some constant multiple of

the Exact Computational Complexity.
Exact Space Complexity
(unit: bytes)
•Measure of an algorithm’s total memory
requirements during runtime.
• Data structures,
• Temporary variables.
Complexity Analysis:
Computational and Asymptotic Complexity

Exact Time Complexity

? NO!
Real-time units (sec, msec,…)
How to evaluate
an algorithm’s
Exact Computational Complexity
efficiency? Logical units (work) ? YES!

Remember:
Exact Computational Complexity is the total discrete
amount of work done by the algorithm.

Exact Time Complexity is some constant multiple of

the Exact Computational Complexity.
Complexity Analysis:
Computational and Asymptotic Complexity

Exact Time Complexity

NO!
Real-time units (sec, msec,…)
How to evaluate
an algorithm’s
Exact Computational Complexity
efficiency? Logical units (work) YES!

Array or file of size n X time t to process the data

t1  cn1 t1  log 2 n1
n2  5n1  t 2  5t1 n2  2n1  t 2  log 2 2n1 
Complexity Analysis:
Computational and Asymptotic Complexity

Function expressing relationship between n and t

• Complex
• Important (large bodies of data)
• Basic recipe  Eliminate non-contributing terms

Resulting function  APPROXIMATE MEASURE

Asymptotic Complexity
Complexity Analysis: Example
Computational and Asymptotic Complexity

f n   n 2  100n  log 10 n  1000

Complexity Analysis:
Computational and Asymptotic Complexity
Growth Rates of the Time Complexities of Algorithms
with respect to increasing problem sizes.

On a machine running at 1 GHz (assuming 1

cycle per work unit).
For small values of n:

What conclusions can we draw?

Which algorithms remain useful as problem size increases?
Complexity Analysis:
Computational and Asymptotic Complexity
Growth Rates of the Time Complexities of Algorithms
with respect to increasing problem sizes.

For larger values of n:

What conclusions can we draw?

Which algorithms remain useful as problem size increases?
 Tractable  log n  n  n log n  n 2

n
Exponential  2
Factorial  n!
Intractable!
Complexity Analysis

Tools for Comparing Algorithms Objectively

• Computational and Asymptotic Complexity

• The “Big-Oh” Notation
Complexity Analysis:
Formal Definition:
Big-O Notation
Let f and g be two functions such that

f n : N  R and g n : N  R

f  the numerical function that represents the

best, average, and worst behavior (it can be
any of the three)

g  the function for you to discover

Complexity Analysis:
Formal Definition:
Big-O Notation
Let f and g be two functions such that

f n : N  R and g n : N  R

if there exists positive constants c and n0 such th
f n   cg n , for all n  n0
or if
f n 
0  lim c
n  g n 

then we write
f n  Og n
So g(n) is an asymptotic upper-bound for f(n) as n increases
(g(n) bounds f(n) from above)
Complexity Analysis:
Informal Definition:
Big-O Notation
f(n) is O(g(n))
if f grows at most as fast as g.

Grow refers to the value of f(n) as n increases

cg(n) is an approximation to f(n), bounding from above

“Big-Oh” Examples

7n  2 O(n)
By definition, we need to find:

• a real constant c > 0

• an integer constant n0 >= 1

Such that 7n - 2 <= cn,

for every integer n >= n0

“Big-Oh” Examples

7n  2 O(n)
Possible choice is:

• c=7
• n0 = 1

7n - 2 <= 7n, for n >= 1

 20n  10n log n  5
3

3
O(n )
20n 10n log n  5  35n , for n  1
3 3
 3 log n  log log n

O(log n)
3 log n  log log n  4 log n, for n  2
 100
2

O(1)
100
2  2 .1, for n  1
100
 5n

O(1 n)
5 n  5(1 n), for n  1
Mergesort and Quicksort
Sorting algorithms

• Insertion, selection and bubble sort have quadratic worst-case

performance
• The faster comparison based algorithm ?
O(nlogn)

• Mergesort and Quicksort

Merge Sort

• Apply divide-and-conquer to sorting problem

• Problem: Given n elements, sort elements into non-
decreasing order
• Divide-and-Conquer:
• If n=1 terminate (every one-element list is already sorted)
• If n>1, partition elements into two or more sub-
collections; sort each; combine into a single sorted list
• How do we partition?
Partitioning - Choice 1
• First n-1 elements into set A, last element set B
• Sort A using this partitioning scheme recursively
• B already sorted
• Combine A and B using method Insert() (= insertion
into sorted array)
• Leads to recursive version of InsertionSort()
• Number of comparisons: O(n2)
• Best case = n-1

• Worst case = n
n(n  1)
c i 
i 2 2
 Partitioning - Choice 2
• Put element with largest key in B, remaining elements
in A
• Sort A recursively
• To combine sorted A and B, append B to sorted A
• Use Max() to find largest element  recursive
SelectionSort()
• Use bubbling process to find and move largest element to
right-most position  recursive BubbleSort()
• All O(n2)
Partitioning - Choice 3

• Let’s try to achieve balanced partitioning

• A gets n/2 elements, B gets rest half
• Sort A and B recursively
• Combine sorted A and B using a process called merge, which
combines two sorted lists into one
• How? We will see soon
 Example
• Partition into lists of size n/2

[10, 4, 6, 3, 8, 2, 5, 7]

[10, 4, 6, 3] [8, 2, 5, 7]

[10, 4] [6, 3] [8, 2] [5, 7]

[4] [10] [3][6] [2][8] [5][7]

Example Cont’d

• Merge

[2, 3, 4, 5, 6, 7, 8, 10 ]

[3, 4, 6, 10] [2, 5, 7, 8]

[4, 10] [3, 6] [2, 8] [5, 7]

[4] [10] [3][6] [2][8] [5][7]

 Static Method mergeSort()
Public static void mergeSort(Comparable []a, int left,
int right)
{
// sort a[left:right]
if (left < right)
{// at least two elements
int mid = (left+right)/2; //midpoint
mergeSort(a, left, mid);
mergeSort(a, mid + 1, right);
merge(a, b, left, mid, right); //merge from a to b
copy(b, a, left, right); //copy result back to a
}
}
Merge Function
Evaluation

• Recurrence equation:
• Assume n is a power of 2
c1 if n=1
T(n) =
2T(n/2) + c2n if n>1, n=2k
 Solution
By Substitution:
T(n) = 2T(n/2) + c2n
T(n/2) = 2T(n/4) + c2n/2

T(n) = 4T(n/4) + 2 c2n

T(n) = 8T(n/8) + 3 c2n

T(n) = 2iT(n/2i) + ic2n

Assuming n = 2k, expansion halts when we get T(1) on right side;

this happens when i=k T(n) = 2kT(1) + kc2n
Since 2k=n, we know k=logn; since T(1) = c1, we get
T(n) = c1n + c2nlogn;
thus an upper bound for TmergeSort(n) is O(nlogn)
Quicksort Algorithm

Given an array of n elements (e.g., integers):

• If array only contains one element, return
• Else
• pick one element to use as pivot.
• Partition elements into two sub-arrays:
• Elements less than or equal to pivot
• Elements greater than pivot
• Quicksort two sub-arrays
• Return results
Example
We are given array of n integers to sort:

40 20 10 80 60 50 7 30 100
Pick Pivot Element
There are a number of ways to pick the pivot element. In this
example, we will use the first element in the array:

40 20 10 80 60 50 7 30 100
Partitioning Array

Given a pivot, partition the elements of the array

such that the resulting array consists of:
1. One sub-array that contains elements >= pivot
2. Another sub-array that contains elements < pivot

The sub-arrays are stored in the original data array.

Partitioning loops through, swapping elements

below/above pivot.
pivot_index = 0 40 20 10 80 60 50 7 30 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index

pivot_index = 0 40 20 10 80 60 50 7 30 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index

pivot_index = 0 40 20 10 80 60 50 7 30 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index

pivot_index = 0 40 20 10 80 60 50 7 30 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index

pivot_index = 0 40 20 10 80 60 50 7 30 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index

pivot_index = 0 40 20 10 80 60 50 7 30 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]

pivot_index = 0 40 20 10 80 60 50 7 30 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]

pivot_index = 0 40 20 10 30 60 50 7 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 60 50 7 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 60 50 7 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 60 50 7 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 60 50 7 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 60 50 7 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 60 50 7 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
5. Swap data[too_small_index] and data[pivot_index]

pivot_index = 0 40 20 10 30 7 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
5. Swap data[too_small_index] and data[pivot_index]

pivot_index = 4 7 20 10 30 40 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
Partition Result

7 20 10 30 40 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

<= data[pivot] > data[pivot]

Recursion: Quicksort Sub-arrays

7 20 10 30 40 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

<= data[pivot] > data[pivot]

Quicksort Analysis

• Assume that keys are random, uniformly distributed.

• What is best case running time?
Quicksort Analysis

• Assume that keys are random, uniformly distributed.

• What is best case running time?
• Recursion:
1. Partition splits array in two sub-arrays of size n/2
2. Quicksort each sub-array
Quicksort Analysis

• Assume that keys are random, uniformly distributed.

• What is best case running time?
• Recursion:
1. Partition splits array in two sub-arrays of size n/2
2. Quicksort each sub-array
• Depth of recursion tree?
Quicksort Analysis

• Assume that keys are random, uniformly distributed.

• What is best case running time?
• Recursion:
1. Partition splits array in two sub-arrays of size n/2
2. Quicksort each sub-array
• Depth of recursion tree? O(log2n)
Quicksort Analysis

• Assume that keys are random, uniformly distributed.

• What is best case running time?
• Recursion:
1. Partition splits array in two sub-arrays of size n/2
2. Quicksort each sub-array
• Depth of recursion tree? O(log2n)
• Number of accesses in partition?
Quicksort Analysis

• Assume that keys are random, uniformly distributed.

• What is best case running time?
• Recursion:
1. Partition splits array in two sub-arrays of size n/2
2. Quicksort each sub-array
• Depth of recursion tree? O(log2n)
• Number of accesses in partition? O(n)
Quicksort Analysis

• Assume that keys are random, uniformly distributed.

• Best case running time: O(n log2n)
Quicksort Analysis

• Assume that keys are random, uniformly distributed.

• Best case running time: O(n log2n)
• Worst case running time?
Quicksort: Worst Case

• Assume first element is chosen as pivot.

• Assume we get array that is already in order:

pivot_index = 0 2 4 10 12 13 50 57 63 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
5. Swap data[too_small_index] and data[pivot_index]

pivot_index = 0 2 4 10 12 13 50 57 63 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
5. Swap data[too_small_index] and data[pivot_index]

pivot_index = 0 2 4 10 12 13 50 57 63 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
5. Swap data[too_small_index] and data[pivot_index]

pivot_index = 0 2 4 10 12 13 50 57 63 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
5. Swap data[too_small_index] and data[pivot_index]

pivot_index = 0 2 4 10 12 13 50 57 63 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
5. Swap data[too_small_index] and data[pivot_index]

pivot_index = 0 2 4 10 12 13 50 57 63 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
5. Swap data[too_small_index] and data[pivot_index]

pivot_index = 0 2 4 10 12 13 50 57 63 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
5. Swap data[too_small_index] and data[pivot_index]

pivot_index = 0 2 4 10 12 13 50 57 63 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

<= data[pivot] > data[pivot]

Quicksort Analysis

• Assume that keys are random, uniformly

distributed.
• Best case running time: O(n log2n)
• Worst case running time?
• Recursion:
1. Partition splits array in two sub-arrays:
• one sub-array of size 0
• the other sub-array of size n-1
2. Quicksort each sub-array
• Depth of recursion tree?
Quicksort Analysis

• Assume that keys are random, uniformly

distributed.
• Best case running time: O(n log2n)
• Worst case running time?
• Recursion:
1. Partition splits array in two sub-arrays:
• one sub-array of size 0
• the other sub-array of size n-1
2. Quicksort each sub-array
• Depth of recursion tree? O(n)
Quicksort Analysis

• Assume that keys are random, uniformly

distributed.
• Best case running time: O(n log2n)
• Worst case running time?
• Recursion:
1. Partition splits array in two sub-arrays:
• one sub-array of size 0
• the other sub-array of size n-1
2. Quicksort each sub-array
• Depth of recursion tree? O(n)
• Number of accesses per partition?
Quicksort Analysis

• Assume that keys are random, uniformly

distributed.
• Best case running time: O(n log2n)
• Worst case running time?
• Recursion:
1. Partition splits array in two sub-arrays:
• one sub-array of size 0
• the other sub-array of size n-1
2. Quicksort each sub-array
• Depth of recursion tree? O(n)
• Number of accesses per partition? O(n)
Quicksort Analysis

• Assume that keys are random, uniformly

distributed.
• Best case running time: O(n log2n)
• Worst case running time: O(n2)!!!
Quicksort Analysis

• Assume that keys are random, uniformly

distributed.
• Best case running time: O(n log2n)
• Worst case running time: O(n2)!!!
• What can we do to avoid worst case?
Improved Pivot Selection

Pick median value of three elements from data array:

data[0], data[n/2], and data[n-1].

Use this median value as pivot.

Improving Performance of Quicksort

• Improved selection of pivot.

• For sub-arrays of size 3 or less, apply brute force search:
• Sub-array of size 1: trivial
• Sub-array of size 2:
• if(data[first] > data[second]) swap them
• Sub-array of size 3: left as an exercise.