UNIT 4

Sorting

© Oxford University Press 2014. All rights reserved.

 Insertion Sort
• Insertion sort is a very simple sorting algorithm, in which the sorted array (or list)
is built one element at a time.
• Insertion sort works as follows.
• The array of values to be sorted is divided into two sets. One that stores sorted
values and the other contains unsorted values.
• The sorting algorithm will proceed until there are elements in the unsorted set.
• Suppose there are n elements in the array. Initially the element with index 0
(assuming LB, Lower Bound = 0) is in the sorted set, rest all the elements are in
the unsorted set
• The first element of the unsorted partition has array index 1 (if LB = 0)
• During each iteration of the algorithm, the first element in the unsorted set is
picked up and inserted into the correct position in the sorted set.

© Oxford University Press 2014. All rights reserved.

 Insertion Sort
Example: Consider an array of integers given below. Sort the values in
the array using insertion sort.

39 9 45 63 18 81 108 54 72 36

39 9 45 63 18 81 108 54 72 36

© Oxford University Press 2014. All rights reserved.

 Insertion Sort
9 39 45 63 18 81 108 54 72 36

9 39 45 63 18 81 108 54 72 36

9 39 45 63 18 81 108 54 72 36

9 18 39 45 63 81 108 54 72 36

9 18 39 45 63 81 108 54 72 36

9 18 39 45 63 81 108 54 72 36

9 18 39 45 54 63 81 108 72 36

9 18 39 45 54 63 72 81 108 36

9 18 36 39 45 54 63 72 81 108

© Oxford University Press 2014. All rights reserved.

 Insertion Sort
Note : In Pass 1, A[0] is the only element in the sorted set.
In Pass 2, A[1] will be placed either before or after A[0], so that the array A is sorted
In Pass 3, A[2] will be placed either before A[0], in-between A[0] and A[1] or after
A[1], so that the array is sorted.
In Pass 4, A[4] will be placed in its proper place so that the array A is sorted.

In Pass N, A[N-1] will be placed in its proper place so that the array A is sorted.
Therefore, we conclude to insert the element A[K] is in the sorted list A[0], A[1], ….
A[K-1], we need to compare A[K] with A[K-1], then with A[K-2], then with A[K-3]
until we meet an element A[J] such that A[J] <= A[K].
In order to insert A[K] in its correct position, we need to move each element A[K-1],
A[K-2], …., A[J] by one position and then A[K] is inserted at the (J+1)th location.
© Oxford University Press 2014. All rights reserved.
 Insertion Sort

Insertion sort (ARR, N) where ARR is an array of N elements

Step 1: Repeat Steps 2 to 5 for K = 1 to N

Step 2: SET TEMP = ARR[K]
Step 3: SET J = K – 1
Step 4: Repeat while TEMP <= ARR[J]
SET ARR[J + 1] = ARR[J]
SET J = J – 1
[END OF INNER LOOP]
Step 5: SET ARR[J + 1] = TEMP
[END OF LOOP]
Step 6: EXIT

© Oxford University Press 2014. All rights reserved.

Complexity of Insertion Sort Algorithm
• For an insertion sort, the best case occurs when the array is already sorted. In
this case the running time of the algorithm has a linear running time (i.e., O(n)).
This is because, during each iteration, the first element from unsorted set is
compared only with the last element of the sorted set of the array.

• Similarly, the worst case of the insertion sort algorithm occurs when the array is
sorted in reverse order. In the worst case, the first element of the unsorted set
has to be compared with almost every element in the sorted set. Furthermore,
every iteration of the inner loop will have to shift the elements of the sorted set
of the array before inserting the next element. Therefore, in the worst case,
insertion sort has a quadratic running time (i.e., O(n2)).

• Even in the average case, the insertion sort algorithm will have to make at least
(K-1)/2 comparisons. Thus, the average case also has a quadratic running time.

© Oxford University Press 2014. All rights reserved.

 Merge Sort
• Merge sort is a sorting algorithm that uses the divide, conquer and combine algorithmic paradigm. Where,

• Divide means partitioning the n-element array to be sorted into two sub-arrays of n/2 elements in each
sub-array. (If A is an array containing zero or one element, then it is already sorted. However, if there are
more elements in the array, divide A into two sub-arrays, A1 and A2, each containing about half of the
elements of A).

• Conquer means sorting the two sub-arrays recursively using merge sort.

• Combine means merging the two sorted sub-arrays of size n/2 each to produce the sorted array of n
elements.

© Oxford University Press 2014. All rights reserved.

 Merge Sort
• Merge sort algorithms focuses on two main concepts to improve its performance
(running time):
• A smaller list takes few steps and thus less time to sort than a large list.
• Less steps, thus less time is needed to create a sorted list from two sorted lists rather
than creating it using two unsorted lists.

The basic steps of a merge sort algorithm are as follows:

• If the array is of length 0 or 1, then it is already sorted. Otherwise:
• (Conceptually) divide the unsorted array into two sub- arrays of about half the size.
• Use merge sort algorithm recursively to sort each sub-array
• Merge the two sub-arrays to form a single sorted list

© Oxford University Press 2014. All rights reserved.

 Merge Sort
Example: Sort the array given below using merge sort
9 9 81 45 90 27 72 18

Divide and conquer the array

39 9 81 45 90 27 72 18

39 9 81 45
90 27 72 18

90 27 72 18
39 9 81 45

90 27 72 18
39 9 81 45

39 9 81 45 00 27 18 72

9 39 45 81 27 90 18 72

9 39 45 81 18 27 72 90

9 18 39 45 72 81 90

Combine the elements to form a sorted array

© Oxford University Press 2014. All rights reserved.
 Merge Sort
• To understand the merge algorithm, consider figure 14.4 which
shows how we merge two lists to form one list. For the sake of
understanding we have taken two sub-lists each containing four
elements. The same concept can be utilized to merge 4 sub-lists
containing two elements, and eight sub-lists having just one
element.
9 39 45 81 18 27 72 90

BEG, I MID J END

TEMP

INDEX

9 39 45 81 18 27 72 90

BEG I mID J END

© Oxford University Press 2014. All rights reserved.

 Merge Sort
TEMP

9 18

INDEX
9 39 45 81 18 27 72 90
BEG I Mid J END
TEMP

9 18 27

INDEX

9 39 45 81 18 27 72 90
BEG I Mid J END
TEMP

9 18 27 39

INDEX

© Oxford University Press 2014. All rights reserved.

 Merge Sort
9 39 45 81 18 27 72 90

BEG I Mid J END

9 18 27 39 45

INDEX

9 39 45 81 18 27 72 90

BEG I,Mid J END

9 18 27 39 45 72

INDEX

When I is greater than MID copy the remaining elements of the right sub-array in TEMP

9 18 27 39 45 72 72 81 90
INDEX

© Oxford University Press 2014. All rights reserved.

 Merge Sort
MERGE (ARR, BEG, MID, END)

Step 1: [Initialize] SET I = BEG, J = MID + 1, INDEX = 0

Step 2: Repeat while (I <= MID) AND (J<=END)
IF ARR[I] < ARR[J], then
SET TEMP[INDEX] = ARR[I]
SET I = I + 1
ELSE
SET TEMP[INDEX] = ARR[J]
SET J = J + 1
[END OF IF]
SET INDEX = INDEX + 1
[END OF LOOP]
Step 3: [ Copy the remaining elements of right sub-array, if any] IF I > MID, then
Repeat while J <= END
SET TEMP[INDEX] = ARR[J]
SET INDEX = INDEX + 1, SET J = J + 1
[END OF LOOP]
[Copy the remaining elements of left sub-array, if any] Else
Repeat while I <= MID
SET TEMP[INDEX] = ARR[I]
SET INDEX = INDEX + 1, SET I = I + 1
[END OF LOOP]
[END OF IF]
Step 4: [Copy the contents of TEMP back to ARR] SET K=0
Step 5: Repeat while K < INDEX
a. SET ARR[K] = TEMP[K]
b. SET K = K + 1
[END OF LOOP]
Step 6: END

© Oxford University Press 2014. All rights reserved.

 Merge Sort

MERGE_SORT( ARR, BEG, END)

Step 1: IF BEG < END, then

SET MID = (BEG + END)/2
CALL MERGE_SORT( ARR, BEG, MID)
CALL MERGE_SORT (ARR, MID + 1, END)
MERGE (ARR, BEG, MID, END)
[END OF IF]
Step 2: END

© Oxford University Press 2014. All rights reserved.

Complexity of Merge Sort Algorithm
• The running time of the merge sort algorithm in average case and worst
case can be given as O(n logn). Although algorithm merge sort has an
optimal time complexity but a major drawback of this algorithm is that
it needs an additional space of O(n) for the temporary array TEMP

© Oxford University Press 2014. All rights reserved.

 Quick Sort
• Quicksort is a widely used sorting algorithm developed by C. A. R. Hoare that makes
O(nlogn) comparisons in average case to sort an array of n elements. However, in the
worst case, quick sort algorithm has quadratic running time given as O(n2). Basically, the
quick sort algorithm is faster than other O(nlogn) algorithms, because efficient
implementation of the algorithm can minimize the probability of requiring quadratic
time.
The quick sort algorithm works as follows:
• Select an element pivot from the array elements
• Re-arrange the elements in the array in such a way that all elements that are less than
the pivot appear before the pivot and all elements greater than the pivot element come
after it (equal values can go either way). After such a partitioning, the pivot is placed in its
final position. This is called the partition operation.
• Recursively sort the two sub-arrays thus obtained. (One with sub-list of lesser value than
that of the pivot element and the other having higher value elements).

© Oxford University Press 2014. All rights reserved.

 Quick Sort
• The main task in the quick sort algorithm is to find the pivot element, which will partition the
array into two halves. To understand how we find the pivot element follow the steps given
below. (we will take the first element in the array as pivot)

• Set the index of the first element in the array to loc and left variables. Also set the index of
the last element of the array to the right variable.

• That is, loc =0, left = 0 and right = n-1 (where n in the number of elements in the array)

• Start from the element pointed by right and scan the array from right to left, comparing each
element on the way with the element pointed by variable loc.

• That is, a[loc]should be less than a[right]. If that is the case then simply continue comparing
until right becomes equal to loc. Because once right = loc, then it means the pivot has been
placed in its correct position.

© Oxford University Press 2014. All rights reserved.

 Quick Sort
• However, if at any point we have a[loc]>a[right] then, interchange the two values and jump to step
3.

• Set loc = right

• Start from the element pointed by left and scan the array from left to right, comparing each element on
the way with the element pointed by variable loc.

• That is, a[loc]should be greater than a[left].

• If that is the case then simply continue comparing until left becomes equal to loc. Because once left
= loc, then it means the pivot has been placed in its correct position.

• However, if at any point we have a[loc]<a[left] then, interchange the two values and jump to step 2
• Set loc = left.

© Oxford University Press 2014. All rights reserved.

 Quick Sort
Example: Consider the array given below and sort the elements using quick sort
algorithm.
27 10 36 18 25 45

We choose the first element as the pivot. Set loc = 0, left = 0,

right = 5 as,
27 10 36 18 25 45

Loc Right
Left

Scan from right to left. Since a[loc] < a[right], decrease the value of right.
27 10 36 18 25 45

Loc Right
Left

© Oxford University Press 2014. All rights reserved.

 Quick Sort

Since, a[loc] > a[right], interchange the two values and set loc = right.

25 10 36 18 27 45

Left Right, Loc

25 10 36 18 27 45

Left
Right, Loc

© Oxford University Press 2014. All rights reserved.

 Quick Sort
Start scanning from left to right. Since, a[loc] > a[right], increment
the value of left.
25 10 36 18 27 45

Left
Right, Loc

Now since, a[loc] < a[right], interchange the values and set loc = left
25 10 27 18 36 45

Left, Loc
Right,

© Oxford University Press 2014. All rights reserved.

 Quick Sort
Scan from right to left. Since a[loc] < a[right], decrement the value of
right.
25 10 27 18 36 45

Left, Loc Right,

Since, a[loc] > a[right], interchange the two values and set loc =
right. 25 10 18 27 36 45

Left Right, Loc

Start scanning from left to right. Since, a[loc] > a[right], increment
the value of left. 25 10 18 27 36 45

Left, Right, Loc

© Oxford University Press 2014. All rights reserved.

 Quick Sort
PARTITION ( ARR, BEG, END, LOC)

Step 1: [Initialize] SET LEFT = BEG, RIGHT = END, LOC = BEG, FLAG = 0
Step 2: Repeat Steps 3 to while FLAG = 0
Step 3: Repeat while ARR[LOC] <= ARR[RIGHT] AND LOC != RIGHT
SET RIGHT = RIGHT – 1
[END OF LOOP]
Step 4: IF LOC == RIGHT, then
SET FLAG = 1
ELSE IF ARR[LOC] > ARR[RIGHT], then
SWAP ARR[LOC] with ARR[RIGHT]
SET LOC = RIGHT
[END OF IF]
Step 5: IF FLAG = 0, then
Repeat while ARR[LOC] >= ARR[LEFT] AND LOC != LEFT
SET LEFT = LEFT + 1
[END OF LOOP]
Step 6: IF LOC == LEFT, then
SET FLAG = 1
ELSE IF ARR[LOC] < ARR[LEFT], then
SWAP ARR[LOC] with ARR[LEFT]
SET LOC = LEFT
[END OF IF]
[END OF IF]
Step 7: [END OF LOOP]
Step 8: END

© Oxford University Press 2014. All rights reserved.

 Quick Sort

QUICK_SORT ( ARR, BEG, END)

Step 1: IF (BEG < END), then

CALL PARTITION ( ARR, BEG, END, LOC)
CALL QUICKSORT(ARR, BEG, LOC – 1)
CALL QUICKSORT(ARR, LOC + 1, END)
[END OF IF]
Step 2: END

© Oxford University Press 2014. All rights reserved.

Complexity of Quick Sort Algorithm
• In the average case, the running time of the quick sort algorithm can be given
as, O(nlogn). The partitioning of the array which simply loops over the elements
of the array once, uses O(n) time.
• In the best case, every time we partition the array, we divide the list into two
nearly equal pieces. That is, recursive call processes a sub-array of half the size.
Since, at the most only logn nested calls can be made before we reach a sub-
array of size 1. This means that the depth of the call tree is O(logn). And
because at each level there can only be O(n), the resultant time is given as
O(nlogn) time.
• Practically, the efficiency of the quick sort algorithm very much depends on the
element is chosen as the pivot. The worst-case efficiency of the quick sort is
given as O(n2). The worst case occurs when the array is already sorted (either
in ascending or descending order) and the left-most element is chosen as the
pivot.
• However, many implementations randomly choose the pivot element. The
randomized version of the quick sort algorithm always has an algorithmic
complexity of O(n log n).
© Oxford University Press 2014. All rights reserved.