Today’s Material

• Divide & Conquer (Recursive) Sorting Algorithms

– QuickSort

• External Sorting

1
 Divide and Conquer Without Extra Space?
• Mergesort requires temporary array for
merging = O(N) extra space
• Can we do in place sorting without extra space?
• Want a divide and conquer strategy that does not
use the O(N) extra space

• Quicksort – Idea:
• Partition the array such that Elements in left sub-
array < elements in right sub-array.
• Recursively sort left and right sub-arrays

2
 How do we Partition the Array?
• Choose an element from the array as the pivot
• Move all elements < pivot into left sub-array
and all elements > pivot into right sub-array
• E.g.,
7 18 2 15 9 11
• Suppose pivot = 7
• Left subarray = 2
• Right sub-array = 18 15 9 11

3
 Quicksort
• Quicksort Algorithm:
1. Partition array into left and right sub-arrays such
that: Elements in left sub-array < elements in right
sub-array
2. Recursively sort left and right sub-arrays
3. Concatenate left and right sub-arrays with pivot in
middle
• How to Partition the Array:
• Choose an element from the array as the pivot
• Move all elements < pivot into left sub-array and all
elements > pivot into right sub-array
• Pivot?
• One choice: use first element in array 4
 Quicksort Example
• Sort the array containing:

9 16 4 15 2 5 17 1
Pivot
Partition 4251 < 9 < 16 15 17

Partition 21 4 5 15 16 17

Concatenate 1 2 5 15 17

1 2 4 5 15 16 17
Concatenate

Concatenate 1 2 4 5 9 15 16 17

5
 Partitioning In Place
• Seems like we need an extra array for partitioning and
concatenating left/right sub-arrays
• No!
• Algorithm for In Place Partitioning:
1. pivot=A[0]
2. Set pointers i and j to the beginning and the end of the array
3. Increment i until you hit an element A[i] > pivot
4. Decrement j until you hit an element A[j] < pivot
5. Swap A[i] and A[j]
6. Repeat until i and j cross (i exceeds or equals j)
7. Restore the pivot by swapping A[0] with A[j]

• Example: Partition in place:

• 9 16 4 15 2 5 17 1 (pivot = A[0] = 9)
6
 Partitioning In Place
Swap

Swap A[i] and A[j] 1 4 15 2 5 17 16

9 16 1

i j

Swap
Swap A[i] and A[j] 9 1 5 2 15
4 15 5 17 16

i i j j

Swap

Swap A[j] and pivot] 9

2 1 4 5 9
2 15 17 16
ij i

Partitioning Complete 2 1 4 5 9 15 17 16
<=9 >=9 7
 Partition Algorithm
int Partition(int A[], int N){
if (N<=1) return 0;
int pivot = A[0]; // Pivot is the first element
int i=1, j=N-1;

while (1){
while (A[j]>pivot) j--; // Move j
while (A[i]<pivot && i<j) i++; // Move i
if (i>=j) break;
Swap(&A[i], &A[j]);
i++; j--;
} //end-while

Swap(&A[j], &A[0]); // Restore the pivot

return j; // return the index of the pivot

} //end-Partition

8
 Pivotal Role of Pivots
• How do we pick the pivot for each partition?
– Pivot choice can make a big difference in run time

• 1st Idea: Pick the first element in (sub)array as pivot

– What if it is the smallest or largest?
– What if the array is sorted? How many recursive calls does
quicksort make?
24689
24689
24689
24689
24689

Total: O(N) recursive calls!

9
 Choosing the Right Pivot
• 2nd Idea: Pick a random element 9 16 4 15 2

– Gets rid of asymmetry in left/right sizes 9 4 2 15 16

– But…requires calls to pseudo-random
number generator – expensive/error prone

• 3rd idea: Pick median (N/2 largest

element)
• Ideal but hard to compute without
sorting!

• Compromise: Pick median of three

elements

10
 Median-of-Three Pivots
• Find the median of the first, middle and last element
2 4 9 15 16 5 4 2 15 16

9 5

• Takes only O(1) time and not error-prone like the

pseudorandom pivot choice

• Less chance of poor performance as compared to

looking at only 1 element

• For sorted inputs, splits array nicely in half each

recursion
• Good performance
11
Partition with MedianOf3 Heuristic
42 71 10 14 95 63 38 27 81 56 Before Partition

Pivot = MedianOf3(42, 95, 56) = 56

42 71 10 14 81 63 38 27 56 95 After MedianOf3

i j
42 27
71 10 14 81 63 38 71
27 56 95 Need to swap 71 & 27

i j
42 27 10 14 38
81 63 81
38 71 56 95 Need to swap 81 & 38

i j
42 27 10 14 38 56
63 81 71 63
56 95 Swap A[i]=63 & pivot=56

j i
42 27 10 14 38 56 81 71 63 95 After Partition 12

<= 56 >= 56
 MedianOf3 Algorithm
// Assumes N>=3
int MedianOf3(int A[], int N){
int left = 0;
int right = N-1;
int center = (left+right)/2;

// Sort A[left], A[right], A[center]

if (A[left] > A[center]) Swap(&A[left], &A[center]);
if (A[left] > A[right]) Swap(&A[left], &A[right]);
if (A[center] > A[right]) Swap(&A[center], &A[right]);

Swap(&A[center], &A[right-1]); // Hide the pivot

return A[right-1]; // Return the pivot

} //end-MedianOf3

13
 Partition with MedianOf3
// Assumes N>=3
int Partition(int A[], int N){
int pivot = MedianOf3(A, 0, N-1);

int i = 0; int j = N-1;

while (1){
while (A[++i] < pivot);
while (A[--j] > pivot);
if (i < j) Swap(&A[i], &A[j]);
else break;
} //end-while

Swap(&A[i], &A[N-2]); // Restore the pivot

return i;
} //end-Partition

14
 QuickSort Performance Analysis
• Best Case Performance: Algorithm always
chooses best pivot and keeps splitting sub-
arrays in half at each recursion
– T(0) = T(1) = O(1) (constant time if 0 or 1 element)
– For N > 1, 2 recursive calls + linear time for
partitioning
– Recurrence Relation for T(N) = 2T(N/2) + O(N)
– Big-Oh function for T(N) = O(N logN)

15
 QuickSort Performance Analysis
• Worst Case Performance: Algorithm keeps
picking the worst pivot – one sub-array empty
at each recursion
– T(0) = T(1) = O(1)
– T(N) = T(N-1) + O(N)
– T(N-2) + O(N-1) + O(N) = …
– T(0) + O(1) + … + O(N)
– T(N) = O(N2 )

• Fortunately, average case performance is

O(N log N)
16
 External Sorting
• What if the data to be sorted does not fit into
memory?
– You have a machine with 1GB memory, but have 8GB
of data to be sorted stored on a disk?

• We need to divide the data to several pieces,

sort each piece and then merge the results
8GB unsorted data: 1GB 1GB 1GB 1GB 1GB 1GB 1GB 1GB

Sort each 1GB piece separately & store on disk

1GB sorted data: 1GB 1GB 1GB 1GB 1GB 1GB 1GB 1GB
total 8 pieces

8-Way Merging
17
8GB sorted data
 External Sorting Running Time
• Assume you have N bytes to sort
• Further assume that your memory is M bytes

• There are K = N/M pieces

• Sorting 1 piece takes: MlogM
• Sorting K pieces: K*MlogM
• Merging K pieces: K*N
• Total: O(K*MlogM + K*N) = O(NlogM + N2/M)

• It is possible to perform K-way merging in

N*logK time using a binary heap: Project2 18