Design and Analysis of Algorithms

Lecture# 03: Sorting Algorithms

1
Sorting Algorithm
• A Sorting Algorithm is used to rearrange a given
array or list elements according to a comparison
operator on the elements

2
Types of Sorting Algorithms
• A recursive sorting algorithm calls on itself to sort a smaller
part of the array, then combining the partially sorted results.
Quick-sort is an example
• A non-recursive algorithm does the sorting all at once, without
calling itself. Bubble-sort is an example of a non-recursive
algorithm
• Non-comparison sorting algorithms sorts using key-value,
Radix sort, which examines individual bits of keys, and Bucket
Sort which examines bits of keys

3
Types of Sorting Algorithms
• Non-recursive/Incremental comparison sorting
• Selection sort
• Bubble sort
• Insertion sort
• Recursive comparison sorting
• Merge sort
• Quick sort
• Heap sort
• Non-comparison linear sorting
• Count sort
• Radix sort
• Bucket sort 4
Selection Sort
• Idea:
• Find the smallest element in the array
• Exchange it with the element in the first position
• Find the second smallest element and exchange it with the
element in the second position
• Continue until the array is sorted

8 4 6 9 2 3 1 1 2 3 4 9 6 8

1 4 6 9 2 3 8 1 2 3 4 6 9 8

1 2 6 9 4 3 8 1 2 3 4 6 8 9
5
1 2 3 9 4 6 8 1 2 3 4 6 8 9
Selection Sort Example
Number of elements = 5
5 2 1 4 3
Pass 1:
1 2 5 4 3
Pass 2: for 2nd element
1 2 5 4 3
Pass 3: for 3rd element
1 2 3 4 5
Pass 4: for 4th element
1 2 3 4 5
Number of passes = n-1 6
Algorithm
1. Selection Sort (A)
2. for (i=0; i<n-1; i++){
3. int min = i;
4. for(j=i+1; j<n; j++){
5. if(a[j] <a[min])
6. min = j;}
7. If (min != i){
8. swap(a[i], a[min]) }

7
Bubble Sort
Given a list of N elements, repeat the following steps N-1 times:
• For each pair of adjacent numbers, if number on the left is greater than the
number on the right, swap them.
• “Bubble” the largest value to the end using pair-wise comparisons and swapping.

12
6 12
6 22
14 22
14
8 8
17
22 17
22

6 12 14
8 14
8 17 22

6 12
8 12
8 14 17 22
8

6 8 12 14 17 22
Bubble Sort Analysis
BubbleSort(A)
1. n = Length[A];
2. for i = 0; i < n-1; i++
3. for j = 0; j < n-1; j++
4. if A[j] > A[j+1]
5. swap(A[j],A[j+1])
6. return A;

9
Insertion Sort
• Idea: like sorting a hand of playing cards
• Start with empty left hand and cards face down on the table.
• Remove one card at a time from the table, and insert it into the
correct position in the left hand
• compare it with each card already in the hand, from right to left
• The cards held in the left hand are sorted
• these cards were originally the top cards of the pile on the table

10
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1 {
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key) {
5. A[j+1] = A[j]
6. j = j – 1
7. }
8. A[j+1] = key
9. }

Array index 0 1 2 3 4 5 6 7 8 9
Value 2.78 7.42 0.56 1.12 1.17 0.32 6.21 4.42 3.14 7.71

11

Iteration 0: step 0.
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9
Value 2.78 7.42 0.56 1.12 1.17 0.32 6.21 4.42 3.14 7.71

12

Iteration 1: step 0.
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9
Value 2.78 0.56
7.42 7.42
0.56 1.12 1.17 0.32 6.21 4.42 3.14 7.71

13

Iteration 2: step 0.
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9
Value 2.78 2.78
0.56 0.56 7.42 1.12 1.17 0.32 6.21 4.42 3.14 7.71

14

Iteration 2: step 1.
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9
Value 2.78 2.78
0.56 0.56 7.42 1.12 1.17 0.32 6.21 4.42 3.14 7.71

15

Iteration 2: step 2.
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9
Value 0.56 2.78 1.12
7.42 7.42
1.12 1.17 0.32 6.21 4.42 3.14 7.71

16

Iteration 3: step 0.
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9
Value 0.56 1.12
2.78 2.78
1.12 7.42 1.17 0.32 6.21 4.42 3.14 7.71

17

Iteration 3: step 1.
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9
Value 0.56 1.12
2.78 2.78
1.12 7.42 1.17 0.32 6.21 4.42 3.14 7.71

18

Iteration 3: step 2.
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9
Value 0.56 1.12 2.78 1.17
7.42 7.42
1.17 0.32 6.21 4.42 3.14 7.71

19

Iteration 4: step 0.
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9
Value 0.56 1.12 1.17
2.78 2.78
1.17 7.42 0.32 6.21 4.42 3.14 7.71

20

Iteration 4: step 1.
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9
Value 0.56 1.12 1.17 2.78 7.42 0.32 6.21 4.42 3.14 7.71

21

Iteration 4: step 2.
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9
Value 0.56 1.12 1.17 2.78 0.32
7.42 7.42
0.32 6.21 4.42 3.14 7.71

22

Iteration 5: step 0.
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9
Value 0.56 1.12 1.17 0.32
2.78 2.78
0.32 7.42 6.21 4.42 3.14 7.71

23

Iteration 5: step 1.
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9
Value 0.56 1.12 0.32
1.17 1.17
0.32 2.78 7.42 6.21 4.42 3.14 7.71

24

Iteration 5: step 2.
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9
Value 0.56 0.32
1.12 1.12
0.32 1.17 2.78 7.42 6.21 4.42 3.14 7.71

25

Iteration 5: step 3.
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9
Value 0.56 0.56
0.32 0.32 1.12 1.17 2.78 7.42 6.21 4.42 3.14 7.71

26

Iteration 5: step 4.
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9
Value 0.56 0.56
0.32 0.32 1.12 1.17 2.78 7.42 6.21 4.42 3.14 7.71

27

Iteration 5: step 5.
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9
Value 0.32 0.56 1.12 1.17 2.78 6.21
7.42 7.42
6.21 4.42 3.14 7.71

28

Iteration 6: step 0.
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9
Value 0.32 0.56 1.12 1.17 2.78 6.21
7.42 7.42
6.21 4.42 3.14 7.71

29

Iteration 6: step 1.
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9
Value 0.32 0.56 1.12 1.17 2.78 6.21 4.42
7.42 7.42
4.42 3.14 7.71

30

Iteration 7: step 0.
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9
Value 0.32 0.56 1.12 1.17 2.78 4.42
6.21 6.21
4.42 7.42 3.14 7.71

31

Iteration 7: step 1.
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9
Value 0.32 0.56 1.12 1.17 2.78 4.42
6.21 6.21
4.42 7.42 3.14 7.71

32

Iteration 7: step 2.
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9
Value 0.32 0.56 1.12 1.17 2.78 4.42 6.21 3.14
7.42 7.42
3.14 7.71

33

Iteration 8: step 0.
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9
Value 0.32 0.56 1.12 1.17 2.78 4.42 3.14
6.21 6.21
3.14 7.42 7.71

34

Iteration 8: step 1.
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9
Value 0.32 0.56 1.12 1.17 2.78 3.14
4.42 4.42
3.14 6.21 7.42 7.71

35

Iteration 8: step 2.
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9
Value 0.32 0.56 1.12 1.17 2.78 3.14
4.42 4.42
3.14 6.21 7.42 7.71

36

Iteration 8: step 3.
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9
Value 0.32 0.56 1.12 1.17 2.78 3.14 4.42 6.21 7.42 7.71

37

Iteration 9: step 0.
Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9
Value 0.32 0.56 1.12 1.17 2.78 3.14 4.42 6.21 7.42 7.71

38

Iteration 10: DONE!.

Example

5 4 10 1 6 2• .

39
Types of Sorting Algorithms
• Non-recursive/incremental comparison sorting
• Selection sort
• Bubble sort
• Insertion sort
• Recursive comparison sorting
• Merge sort
• Quick sort
• Heap sort
• Non-comparison linear sorting
• Count sort
• Radix sort
• Bucket sort 40
Recursive Comparison Sorting
There are many ways to design algorithms:

Insertion/Bubble/Selection sort uses an incremental approach

• Having sorted to array A[i..j]
• We insert the single element A[j+1] into its proper place, to get a sorted
array A[i..j+1]

An alternative design approach is “Divide and Conquer”

• Divide the problems into a number of sub-problems
• Conquer the sub-problems by solving them recursively. If sub-problem
sizes are small enough, just solve them in a straight forward manner.
• Combine the solutions to the sub-problems into the solution for the
original problem. 41
Merge Sort
• Idea: Order a list of values by recursively dividing the list in half until
each sub-list has one element, then recombining

• More specifically:
• Divide the n element sequence to be sorted into two subsequences
of n/2 elements each.
• Conquer: Sort the two subsequences to produce the sorted answer.
• Combine: Merge the two sorted sub sequences to produce the
sorted answer.

mergeSort(0, n/2-1) mergeSort(n/2, n-1)

42
sort merge(0, n/2, n-1) sort
Merge Sort Pseudocode

43
 Merge Sort
Base Case: When the sequences to be sorted has length 1.

Unsorted
Sorted 66
12 108
14 56
34 14
56 89
66 12
89 108
34

Divide
Merge Divide
Merge

66
14 108
56 56
66 108
14 89
12 12
34 34
89

Divide
Merge Divide
Merge Divide
Merge Divide
BCase
Merge

66
66 108
108 56
14 14
56 89
12 12
89 34

Merge
Divide
BCase Divide
BCase
Merge Divide
BCase
Merge Divide
BCase
Merge
44
66 108 56 14 89 12
Merge Function
Given two sorted arrays, merge operation produces a sorted array
with all the elements of the two arrays

8 1 4 3 2
Divide the array in half and conquer

...
1 4 8 2 3
Merge the halves

tempArray: 1 2 3 4 8
copy

Original Array: 1 2 3 4 8
45
Merge Sort Analysis
Level 0
Merge n items:
n O(n)

n/2 n/2
Level 1 Merge two n/2 items:
O(n)

n/4 n/4 n/4 n/4 Level 2

Each level requires

O(n) operations

1 1 1 ... 1 1 1 Tree Height : log2n

Each level O(n) operations & O(log2n) levels  O(n*log2n)

46
Merge Sort Analysis
• Worst case: O(n * log2n).

• Advantage:
• Merge sort is an extremely fast algorithm.

• Disadvantage:
• Merge sort requires a second array as large as the original
array.

47
Quick Sort
• Quick Sort: orders a list of values by partitioning the list around
one element called a pivot, then sorting each partition. It is based
on divide and conquer approach.

• Key Points: Given an array S to be sorted

• Pick any element v in array S as the pivot (i.e. partition element)
• Partition the remaining elements in S into two groups
• S1 = {all elements in S-{v} that are smaller than v}
• S2 = {all elements in S-{v} that are larger than v}
• apply the quick sort algorithm (recursively) to both partitions

• Trick lies in handling the partitioning

• i.e. picking a good pivot is essential to performance 48
Quick Sort Illustrated
pick a pivot

40 37 2
10 18 32 35
12 17 6

partition
6 40
10 12 2 17 18
32 35 37

quicksort quicksort
2 6 10 12 17 18 32 35 37 40
combine 49
2 6 10 12 17 18 32 35 37 40
Quick Sort Pseudocode

50
Picking the Pivot
• How would you pick one?

• Strategy 1: Pick the first element in S

• Works only if input is random

• What if input S is sorted, or even mostly sorted?

• All the remaining elements would go into either S1 or S2!
• Terrible performance!

• Why worry about sorted input?

• Remember  Quicksort is recursive, so sub-problems could be sorted
• Plus mostly sorted input is quite frequent

51
Picking the Pivot (contd.)
• Strategy 2: Pick the pivot randomly
• Would usually work well, even for mostly sorted input
• Unless the random number generator is not quite random!
• Plus random number generation is an expensive operation

• Strategy 3: Median-of-three Partitioning

• Ideally, the pivot should be the median of input array S
• Median = element in the middle of the sorted sequence

• Would divide the input into two almost equal partitions

• Unfortunately, harder to calculate median quickly, without sorting first!
• So find the approximate median
• Pivot = median of the left-most, right-most and center elements of array S
52
• Solves the problem of sorted input
Picking the Pivot (contd.)
• Example: Median-of-three Partitioning

• Let input S = {6, 1, 4, 9, 0, 3, 5, 2, 7, 8}

• Left = 0 and S[left] = 6

• Right = 9 and S[right] = 8

• center = (left + right)/2 = 4 and S[center] = 0

• Pivot
• = Median of S[left], S[right], and S[center]
• = median of 6, 8, and 0
• = S[left] = 6
53
Quick Sort: Final Comments
• What happens when the array contains many duplicate elements?
• Not stable

• What happens when the size of the array is small?

• For small arrays (N ≤ 50), Insertion sort is faster than quick sort due to
recursive function call overheads of quick sort. Use hybrid algorithm;
quick sort followed by insertion sort when N ≤ 50

• However, Quicksort is usually O(n log2n)

• Constants are so good that it is generally the fastest algorithm known
• Most real-world sorting is done by Quicksort
• For optimum efficiency, the pivot must be chosen carefully
• “Median of three” is a good technique for choosing the pivot
54
Heap Sort
Uses Heap data structure to manage data during algorithm execution

A (Max/Min)-Heap is a balanced, left-justified binary tree in which no

node has a value greater/lesser than the value in its parent
• A[parent[i]]  A[i] (max-heap)
• A[parent[i]]  A[i] (min-heap)

Largest/Smallest element is stored at the root.

Subtree rooted at a node contains values no larger/smaller than that

at the node.

55
Heap Property
20
Larger than
its parent
15 8

4 10 7 9 Not a Heap

25

Not left-justified 10 12

9 5 11 7
56

1 6 3 8
Array Representation of Heaps
1 2 3 4 5 6 7 8 9 10
26 24 20 18 17 19 13 12 14 11

Max-heap as an array

26

24 20

18 17 19 13

Max-heap as a binary tree

12 14 11 57

Node Insertion: Insert in last row from left to right

Node Deletion: Delete from last row from right to left
 Example: A: 4 1 3 2 16 9 10 14 8 7

i=5 i=4 i=3

1 1 1

4 4 4
2 3 2 3 2 3

1 3 1 3 1 3
4 5 6 7 4 5 6 7 4 5 6 7

8
2 9 10
16 9 10 8
2 9 10
16 9 10 8
14 9 10
16 9 10
14 8 7 14 8 7 2 8 7

i=2 i=1
1 1 1

4 4 16
2 3 2 3 2 3

1 10 16 10 14 10
4 5 6 7 4 5 6 7 4 5 6 7

8
14 9 10
16 9 3 8
14 9 10
7 9 3 8
8 9 10
7 9 3
2 8 7 2 8 1 2 4 1
58
How Fast Can We Sort?
• Selection Sort, Bubble Sort, Insertion Sort  O(n2)
• Merge Sort, Quick Sort, Heap Sort  O(n log n)

59
Types of Sorting Algorithms
• Non-recursive/incremental comparison sorting
• Selection sort
• Bubble sort
• Insertion sort
• Recursive comparison sorting
• Merge sort
• Quick sort
• Heap sort
• Non-comparison linear sorting
• Count sort
• Radix sort
• Bucket sort 60
 Counting Sort
• Assumptions:
• n integers which are in the range [0,k]
• k is in the order of n, that is, k=O(n).

• Idea:
• For each element x, find the number of elements ≤ x
• Place x into its correct position in the output array

61
https://www.cs.usfca.edu/~galles/visualization/CountingSort.html
Counting Sort
1 2 3 4 5 6 7 8

A 2 5 3 0 2 3 0 3 Input Array
0 1 2 3 4 5

C 2 0 2 3 0 1 Auxiliary Array
0 1 2 3 4 5

C 2 2 4 7 7 8

1 2 3 4 5 6 7 8

B 0 0 2 2 3 3 3 5 Output Array
0 1 2 3 4 5
62
C 0
2 2 2
1 3 7
4 4
6 7 7
5 8
Counting Sort
Counting-Sort(A, B, k)
for i = 1 to k
do C[i] = 0  (k)
for j = 1 to length[A]
do C[A[j]] = C[A[j]] + 1  (n)
// C[i] now contains the number of elements = i
for i = 2 to k
do C[i] = C[i] + C[i-1]  (k)
// C[i] now contains the number of elements ≤ i
for j = length[A] downto 1
do B[C[A[j]]] = A[j]
C[A[j]] = C[A[j]] - 1  (n)

63
Running time is (n+k) (or (n) if k = O(n))!
Counting Sort
• Is counting sort stable
• The input order is maintained among items with equal keys

• Cool! Why don’t we always use counting sort?

• Because it depends on range k of elements

• Could we use counting sort to sort 32 bit integers? Why or why not?
• Answer: no, k too large (232 = 4,294,967,296)

64
Radix Sort
• Represents keys as d-digit numbers in some base-k
• key = x1x2...xd where 0≤xi≤k-1

• Example: key=15
• key10 = 15, d=2, k=10 where 0≤xi≤9
• key2 = 1111, d=4, k=2 where 0≤xi≤1

• Assumptions: d=O(1) and k =O(n)

• Sorting looks at one column at a time
• For a d digit number, sort the least significant digit first
• Continue sorting on the next least significant digit, until all
digits have been sorted
65
• Requires only d passes through the list
https://www.cs.usfca.edu/~galles/visualization/RadixSort.html
Radix Sort
RadixSort(A, d)
for i=1 to d
d times
StableSort(A) on digit i  (n+k)

Running time is (dn+dk) (or (n) if k = O(n) and d = O(1))

66

(stable sort: preserves order of identical elements)

Radix Sort
• Problem: sort 1 million 64-bit numbers
• Treat as four-digit radix 216 numbers
• Can sort in just four passes with radix sort!

67
Bucket Sort
• Assumption:
• Input numbers are uniformly distributed in [0,1).
• Suppose input size is n.

• Idea:
• Divide [0,1) into n equal-sized buckets (k= (n))
• Distribute the n input values into the buckets
• Sort each bucket (insertion sort as default).
• Go through the buckets in order, listing elements in each one

• Input: A[1 . . n], where 0 ≤ A[i] < 1 for all i

• Output: elements A[i] sorted 68
 Bucket Sort
Step 1: Distribute into Buckets
.78 0

.17 1 .12 .17

.39 2 .21 .23 .26
.26 3 .39
.72 4
Step 2: Sort within each Bucket
.94 5
.21 6 .68
.12 7 .72 .78
.23 8
.68 9 .94 69
Step 3: Concatenate all Buckets

.12 .17 .21 .23 .26 .39 .68 .72 .78 .94 /
Bucket Sort
BUCKET-SORT(A, n)
for i ← 0 to n-1
insert A[i] into list B[⌊n A[i]⌋] O(n)
for i ← 0 to n-1
sort list B[i] with insertion sort O(ni2)
concatenate the lists B[0], B[1], . . ., B[n - 1] in order O(n)

Where ni is the size of bucket B[i].

Thus T(n) =  (n) +
=  (n) + (n) =  (n)
70
Summary: Sorting Algorithms
Insertion Sort: Suitable only for small n ≤ 50 or nearly sorted inputs
Merge Sort: Guaranteed to be fast even in its worst case; stable
Quick Sort: Most useful general-purpose sorting for very little memory
requirement and fastest average time. (Choose the median of
three elements as pivot in practice
Heap Sort: Requiring minimum memory and guaranteed to run fast;
average and maximum time both roughly twice the average
time of quick sort. Useful for time-critical applications

Counting Sort: Very useful when the keys have small range; stable;
memory space for counters and for 2n records.
Radix Sort: Appropriate for keys either rather short or with an
lexicographic collating sequence. 71

Bucket Sort: Assuming keys to have uniform distribution.

Quiz#02
• Suppose we have an array ‘A’, containing the values shown
below;
• {6,2,9,11,3,7,12}
• Perform the sorting of an array in ascending order using
Bubble sort. Show the complete number of passes for sorting.
Understanding of question is the part of quiz

72