cosc 460

Department of Computer Science

University of Canterbury

Supervisor: Dr. A. Moffat

John Wright
October 1986
CONTENTS

Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.0 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.0 Preliminary Testing ................................ 5

2.1 Sortedness Ratio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 Inversions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3 Number of Ascending Sequences .................. 7
2.4 Longest Ascending Sequences ..................... 7
2.5 Number of Exchanges .......................... 8
2.6 Results ................................... 10
2. 7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3.0 The Implementation of Five Algorithms ................. 12

3.1 Linear Insertion Sort .......................... 13
3.2 Ysort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.3 Cksort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.4 Natural Mergesort ............................ 17
3.5 Smoothsort ................................ 19
3.6 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

4.0 Improvements, Modifications and Other Ideas ............. 26

4.1 Improvements to Y sort ........................ 26
4.2 Improvements to Cksort ....................... 27
4.3 Alterations to Smoothsort ...................... 29
4.4 Alterations to Linear Insertion Sort ................ 30

5.0 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Appendix A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Appendix B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Appendix C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

page2
 ABSTRACT

The following five algorithms for sorting in situ are examined: linear
insertion sort, cksort, natural mergesort, ysort and smoothsort. Quicksort and
heapsort are also considered although they are not discussed in detail. The
algorithms have been implemented and compared, with particular emphasis
being placed on the algorithms' efficiency when the lists are nearly sorted.
Five measures of sortedness are investigated. These are sortedness ratio,
number of inversions, longest ascending sequence, number of ascending
sequences, and number of exchanges. The sortedness ratio is chosen as a
basis for comparison between these measures and is used in experiments on
the above algorithms.
Improvements to cksort and ysort are suggested, while modifications to
smoothsort and linear insertion sort failed to improve the efficiency of these
algorithms.

page3
1.0 INTRODUCTION

One of the most widely studied problems in computer science is sorting. Algorithms for
sorting were developed early and have received considerable attention from mathematicians and
computer scientists. A large number of algorithms were developed, but no one algorithm is best
for all situations and in many cases the files to be sorted are just too large and need to be sorted
externally using discs or tapes as the storage medium. In recent years the interest in sorting has
concentrated on algorithms that exploit the degree of sortedness of the input.
Quicksort is an internal sorting algorithm that was first proposed by C.A.R Hoare [1] in
1962 and has been studied extensively. It has been widely accepted as the most efficient internal
sorting algorithm and achieves a lower bound of O(n log n), but has a worst case complexity of
O(n2) and the algorithm does not attempt to exploit the degree of sortedness of the input. Many
modifications to quicksort have occurred and there exists a large number of algorithms based on
the original theme. Other algorithms such as heapsort and mergesort achieve an O(n log n) lower
bound and yet do not have quicksort's worst case complexity, but these algorithms also do not
take into account the sortedness of the input.

Considered here are five algorithms for sorting in situ. Each algorithm is claimed to have
good behaviour on nearly sorted lists. In particular linear insertion sort, natural mergesort,
ysort, cksort and smoothsort have been studied. Both ysort and cksort have utilized the
quicksort algorithm or one of its descendants. Natural mergesort is an extension of the merge
sorting technique, but uses natural runs, ascending or descending sequences, in the data.
Smoothsort is a new algorithm, proposed by Dijkstra [8] that uses a sophisticated data structure
based on heaps and was designed specifically to sort nearly sorted lists in linear time and have a
worst case complexity of O(n log n). Finally insertion sort has been included as its performance
on nearly sorted lists is widely known although it has a worst case complexity of O(n2).
All of the algorithms attempt to utilize the sortedness of the data in some way and each is
claimed to have O(n) complexity on nearly sorted lists.

page4
2.0 PRELIMINARY TESTING

In dealing with sorting algorithms the phrases " ... on a nearly sorted list ... " and " ... on a
randomly sorted list ... " arise frequently, but a list may be "nearly" sorted according to one
terminology and "randomly" sorted according to another. Intuitively a list is nearly sorted if only
a few of its elements are out of order. The problem is to define an easily computable measure
that coincides with intuition.
A list will be considered sorted if all its elements form a non decreasing sequence and a list is
reverse sorted if all its elements form a non ascending sequence. A list of length n can be sorted
in O(n) time if the number of operations on all the elements of the list is proportional ton by
some constant factor. For example the list,

5 6 7 8 4 3 2 1,

can be sorted in O(n) since n/2 comparisons and n/2 swaps are needed to sort the list, provided
the list is sorted by merging sequences from opposite ends.
This chapter discusses and relates five measures of sortedness, the sortedness ratio,
inversions, number of ascending sequences, longest ascending sequence and the number of
exchanges. The first measure discussed [2] provides a basis from which to compare the other
four measures. Results have been graphed and also included in Appendix A.

2.1 Sortedness ratio

Cook and Kim [2] defined the sortedness ratio of a list of length n as,\

sortedness ratio = kin,

where k is the minimum number of elements that need to be removed to leave the list sorted. For
a sorted list this ratio is O since all the elements are in their correct positions and for a list in
reverse sorted order this ratio approaches 1. For example,

21354

has sortedness 2/5 since the removal of 5 or 4 and 1 or 2 will leave the list sorted.

page5
 This measure of sortedness is by no means perfect. Consider the following lists:

87654321
21436587

The first list has a sortedness ratio of 7/8, the maximum possible sortedness ratio for a list, yet it
can be sorted in O(n) time by reversing the list. The second list contains local disorder and has a
sortedness ratio of 4/8, indicating a high degree of unsortedness. Yet it too can be sorted in O(n)
time by an algorithm that will exploit local disorder.

2.2 Inversions

For a list on n elements; x 1, x2 , x3, ... xn, the number of inversions is defined to be,\

number of inversions = Ii=l,n-l Ij=i+l,n inv,

where inv = { 0 if xi ~ xj,

{ 1 if xi > xj.

If the list is in reverse order there are n(n - 1)/2 inversions. The number of inversions is
bounded below by 0, for the sorted list, and above by n(n - 1)/2, for the reverse sorted list. For
example in the list,

423865

there are 5 inversions, namely (4,2), (4,3), (8,6), (8,5) and (6,5).
This measure of sortedness indicates a list in reverse order would be less sorted than a list of
elements chosen from a random distribution. For example, in

10987654321
87531492610

the first list has 45 inversions, indicating a high degree of unsortedness but certainly can be
sorted in O(n) time. The second list contains 22 inversions but has no obvious properties that
allow it to be sorted in O(n) time.

page6
2.3 Number of ~scending Sequences
t '>,

The number of ascending sequences in a list of length n is defined as,

number of ascending sequences = I,i=l,n-l (run) + 1,

where run= { 1 if xi+l <xi,

{ 0 if xi+l :::: xi.

For example in the list,

5 4 9 2 6 7 8 3 10 11 15,

there are 4 ascending sequences partitioned as follows,

(5) (4 9) (2 6 7 8) (3 10 11 15).

For a list in sorted order there is just 1 ascending sequence and for a list in reverse order
there are n ascending sequences. This method has its disadvantages in lists which have a high
degree of local disorder. For example, lists such as,

2 1 4 3 6 5 8 7 ..... ,

have a large number of ascending sequences but certainly have properties that allow linear time
sorting since each element is only one position from its correct position in the sorted list.

2.4 Longest Ascending Sequence

The longest ascending sequence of a list can be seen from the following example,

1541236798

which is partitioned into the following ascending sequences,

(1 5) (4) (12367 9) (8),

with the longest ascending sequence of length 6.

page 7
 A sorted list has a single ascending sequence of length n and a reverse sorted list has n
ascending sequences of length 1 and generally the greater the number of ascending sequences
the less sorted the list. But consider the following lists:

21436587109
10987654321
10563582174

Both the 1st and 2nd lists have immediate properties that allow O(n) time for sorting. In the first
list each element is 1 position from its sorted position, even though the longest ascending
sequence is of length 1. The 3rd list has no obvious properties yet its longest ascending
sequence is of length 3, and according to this measure it is more sorted than the first two lists.

2.5 Number of Exchanges

The number of exchanges in a list is the smallest number of exchanges of elements needed to
bring the list into a sorted order. For example the list,

13879546,

requires 3 exchanges to move the elements into their correct position:

1 3 .a
7 9 5 .4 6
1 3 4 1 9 5. 8 6
1 3 4 5 .2. 7 8 .Q
1 3 4 5 6 7 8 9
.. ·-

A sorted list clearly requires O exchanges, on the other hand a reverse sorted list requires [n/2J
exchanges.
Since each exchange will move at least 1 element into its correct place there will be at most
O(n) exchanges required to construct the sorted list. For example in the list,

5 1 2 3 4,

n-1 or 4 exchanges are required to sort the list as follows.

page8
 5. 1 2 3 4
1 5. 2 3 4
125.:14
1 2 3 5. ~
1 2 3 4 5

One of the disadvantages of this measure can be seen in the above example which required the
maximum number of exchanges to sort the list, indicating a high degree of unsortedness, but the
list can certainly be sorted in O(n) time by swapping the first element into the last position of the
list.
The main disadvantage of this measure is illustrated in figure 4 and shows that the number of
exchanges does not correlate well with the sortedness ratio and other sorting measures
compared. Consider a sorted list of length n. If just one element is removed and inserted at a
random position elsewhere in the list then the average distance between the element's new
position and its correct position will be 1/3n. O(n) or l/3n exchanges will be required to move
the element to its correct position, so_ a sortedness ratio of 1/n will have l/3n or O(n) exchanges.
As n increases 1/n becomes a smaller fraction but will still result in O(n) exchanges, whereas a
list with a smaller sortedness ratio should require less effort to sort.

page9
2.6 Results

Experiments were run on lists of length 10, 100 and 1000. The lists were generated by first
taking a sorted list and removing k elements, from random positions within the list, and then
inserting these k elements at new random positions within the list. If the inserted elements
formed an ascending sequence with its two adjacent elements then it was shifted either left or
right to ensure the list had a sortedness ratio of kin. The sortedness ratio was varied between 0%
and 30% and for each list generated the number of inversions, ascending sequences, exchanges
and the longest ascending sequence was determined. Results obtained were graphed, the point
graphed being the average of 10 trials. Results have also been included in Appendix A along
with the maximum and minimum values for each trial.
Of particular interest was the relationship with other measures of sortedness when the list
was nearly sorted. Further analysis showed that as the sortedness ratio became higher than
30%, the first two measures considered, number of inversions and number of ascending
sequences, began to approach an upper bound asymptotically. This can be seen in figures 1 and
2 where a linear line has approximated the points plotted, but as the sortedness ratio has
increased the points have fallen below the line plotted.
It can be seen from the graphs that the first three measures considered correlated successfully
with the sortedness ratio. The number of inversions, figure 1, increases quadratically as the
sortedness ratio increases, but a linear relationship has been obtained when the number of
inversions have been scaled by l/n2. The number of ascending sequences, figure 2, gives a
good linear approximation especially when the sortedness ratio is small, but as the sortedness
ratio approaches 25% this relationship begins to deteriorate. The inverse of the longest
ascending sequence, figure 3, has the closest relationship with the sortedness ratio and the
deterioration in the linear correlation was not apparent as the sortedness ratio was varied from
0% to 30%. The last measure considered, number of exchanges, shows a poor correlation with
the sortedness ratio for reasons explained in 2.5. As n increases the number of exchanges more
rapidly approaches the upper bound of n-1 exchanges.

page 10
C,0

-·-·-·-·-·
---.-----...
· . -.-_-::..-:

::;~/-::~

n ~ the length of the list

Q.12

0,11

0.10

0.09

0.08
C\lc

~ om
.Q
VJ
tJ C).06
>
c
4-- 0.05
0
L
1s o.04
E
::J
Z a.OJ

0.1 _Q,2

.Soc.tedness Ratio

)-',.,.,,:-.
Sortedness Ratio and the Number of Inversions.
 n=the length of the list

Sortedness Ratio

f".: .•·
I
Sortedness Ratio and the Number of Ascending Sequences.
I
 n=the length of the list

(... .::·.-"..·
I ..

l\:i'IOO

0.1

Sortedness Ratio t. ·..

,fiigure 3. Sortedness Ratio and the Largest Ascending Sequence.

C,ORMACI ( n n p PE n (t r

l fl
{ +
l]IT
I
~, I
Hi!1:
Ilj ! $
I : ',I
'lj l IIt
l
:j :r J
+

I111I
Ii 11
ltl . i=r,
l"l
,I ij
ii 11! Iji I
Ij1
:c

I
CL
I' 1, "1
I
I, I 1! I

Il il I~
1

r
1 :I i! ~-

~ ·~
t
+

11 11 ' t 1
,_
!l ?
I11 I,'I
JI 1
11
I,-++ tt
ill li
," lj n=the length of the list
I t'
I
II Ii! Iii
11! I
t r

I! T Ill
1,11
It r
1I
I111
I~! 1.0
I,

+
+

o.9
1111 ,
I .

1 I
o.S I. -
It''\I I
i
I===··<:
1,
,II
!11
1

I I ,,i,:
It
I

i
J

c
o .1

--- 0.6
11:
iii I•: U)
I''
"
11 \ (lJ
en
c
iit!!! ! !
ro
..c ().$
u
1,11 I x
II,
II !t w
OA !

~i ~ !

l....
I (lJ
..0 0.3 I

I
' :t
E:
:0 '
, I
z O.?,'
,
I '\:

v
I
:11 :! 1:.1
'I '0.1
1Ht
~I, 11:i
rl,
1••
+I-

r
I I '!I J
'lj!

j I
i1l

Ift
I''
~:
t t
,++
~ -0

• t
0.1

Sortedness Ratio
I ' t t l_

I '
' !
j:
-t-

IJ
1J1
111

'if
II
Ji
I
11 +
1 t
j

I \ Fjigu~~ 4. Sortedness Ratio and the Nurn\Jer of Exchanges.

~ +

I t 'Ji
Ii
jl !1H
El ,
t r* .,
+

Ir f i t
I
!
d
/irJ r· ! I
+
+I ~+
t

il
,-:. , H:,
'
1rt. ::It
l
2.7 Conclusions

The problem still remains to find a measure of sortedness that returns appropriate values for
all lists that have properties that allow O(n) sorting. For the measures discussed here, a list may
be nearly sorted by one measure and very unsorted by another.
Algorithms that have been claimed to give good performance on nearly sorted lists have
typically been tried on lists in which the measure of sortedness has been the measure that the
algorithm best exploits. For example natural mergesort utilizes the natural runs, ascending or
descending sequences, and compared with alternative algorithms, gives good results on lists
which contain a high percentage of runs. Insertion sort performs well on list that have a high
degree of local disorder, for example sequences of the form,

2, 1, 4, 3, 6, 5 ........

This type of sequence has a low number of inversions but is almost totally unsorted according to
the other measures discussed.

page 11
3.0 THE IMPLEMENTATION OF FIVE ALGORITHMS

Five algorithms, linear insertion sort, ysort, cksort, natural mergesort and smoothsort, were
chosen and implemented in Pascal and run on a Prime 750 at the University of Canterbury.
These algorithms have been chosen as their authors have claimed they have good performance
on nearly sorted lists, although in many cases a precise definition of a nearly sorted list has not
been given, and justification for the algorithm's efficiency has only been provided by way of
worked examples or a general discussion of the algorithms behaviour.
In addition to these five algorithms, quicksort and heapsort have been implemented.
Quicksort is generally accepted as the best internal sorting algorithm and is often implemented as
a system sort. It has been implemented here as a hybrid algorithm with linear insertion sort and
uses the middle element as the partition element [3]. Heapsort was included as its worst case and
average case complexity is O(n log n), unlike quicksort which has an O(n2) worst case
complexity. For this reason heapsort is often implemented in situations where the sorting time of
an algorithm is critical, since its O(n log n) behaviour can be guaranteed.

page 12
3.1 Linear Insertion Sort

The simplest of the algorithms implemented was linear insertion sort [4]:

begin
i := 2;

while i ::;;; n do
begin
j := i;
while j > 1 cand A[j] < A[j-1] do
begin
swap(A[j],A[j-1]);
j := j - 1;
end;
i := i + 1;
end;

end

The simplicity of this algorithm makes it very useful and for small values of n it is frequently
implemented, even if the structure of the list is unknown.
The algorithm has an O(n2) worst case complexity, but on lists in which each element is no
more thank positions from its final position it has an O(kn) complexity. This fact makes it very
useful for a sorting algorithm on nearly sorted lists and also makes it very useful in hybrid
algorithms. For example, Sedgewick [3] used insertion sort to very good effect in a hybrid
algorithm with quicksort.

page 13
3.2 Ysort

Ysort [5] is a variation on the quicksort algorithm. Its main difference from quicksort occurs
in the construction of the sublists as shown in the following description of the algorithm.

Unsorted array is in A[l .. r], initially l is 1 and r is n.

begin
i, j, T := 1, r, partition element

while i <= j do
begin
while A[i] < T do
begin
i := i + 1
get position of minimum and maximum values of left subfile, A[l .. i]
end;
while A[j] > T do
begin
j := j - 1;
get position of minimum and maximum values of right subfile, A[j .. r]
end
i, j, A[i], A[j] := i+l, j-1, A[j], A[i)
get positions of minimum and ~aximum values for left and right subfiles
end

swap minimum and maximum values into position, if necessary

if left subfile not sorted then

sort the sublist A[l+l] .. A[j-1)
if right subfile not sorted then
sort the sublist A[i+l] .. A[r-1)

end

As the sublists are constructed the location of the minimum and maximum elements of each
sublist are recorded. When the partitioning step is completed the minimum and maximum
elements are exchanged with the left and right elements of each sublist respectively. Ysort is
then used again to sort the left and right sublist. The new sublists to be sorted exclude the end
elements as these are the minimum and maximum elements and are now in their correct position.
After a partitioning step on an unsorted array, A[l..r], the array will have the following
properties.

page 14
 j i r

11 ::::; T III ~T 11

l subfile to l>e sorted

m. iD.im. 'U.Ill. Thlue

ll
m.~ 'U.Ill. ruv.e
,.i,m, to )t " " ' '
m.i.D.im. 'U.Ill. ruv.e m.~ 'U.Ill.
l
ruv.e
of Mt subfile of ~ft subfile of right subfile of right subfile

T :is the value of the parti.tion element.

The advantage with this algorithm is that it also keeps track in the construction of each sublist
whether the new element placed in the sublist is the new maximum element for the left sublist or
the new minimum element for the right sublist. If for the left sublist each new element placed in
the list becomes the new maximum element then the list will be sorted and needs not be
partitioned any further. A similar argument applies for the right sublist. For a nearly sorted list
ysort needs only to partition until it finds a sorted sublist. Since there will be many sorted
sublists in a nearly sorted list few partitioning steps will be required to sort the list completely.
The cost of this algorithm compared with quicksort is the additional number of comparisons
it needs to make to keep track of the maximum and minimum elements of each sublist.

page 15
3.3 Cksort

Cksort as proposed by Cook and Kim [2] is essentially a hybrid sorting technique based on
three sorting algorithms: quicksort, linear insertion sort and merging. Like all hybrid algorithms
it attempts to exploit the advantages of each of its composite algorithms.

A is the unsorted list, Bis used for processing.

begin
first, second, place, upto := 1, 2, 1,1
while second<= n do
begin
if A[first] > A[second] then
begin
B[place], B[place+l], place := A[first], A[second], place+2
first, second:= previous element compared, next uncompared element
end
else
begin
A[upto], A[upto+l] := A[first], A[second]
first, second, upto := upto+l, second+l, upto+2
end
end
if place -1 > 30 then - number of elements in list B
quicksort B[l] .. B[place-1]
else
insertion sort B[l] .. B[place-1]
merge list A[l] .. A[upto-1] and B[l] .. B[place-1]
end

The first pass of cksort scans the list and removes all unordered pairs of elements and places
them in a separate list. After a pair of unordered elements has been removed, the next pair
compared are the elements immediately preceding and immediately following the pair just
removed. Upon completion of the first pass the original list will be sorted since it contains only
ordered pairs. The second list, of unordered pairs, is then sorted by either quicksort if there is
more than 30 elements or by inse1iion sort, otherwise. Finally the two sorted lists are merged.
For a nearly sorted list there will be few unordered pairs removed, which are then sorted
efficiently by either quick sort or linear insertion sort and merged with the first list. The removal
of unordered pairs in a nearly sorted list will take O(n) time, the sorting of the small number of
unordered pairs will be efficiently perf01med by quicksort or linear insertion sort, and finally the
merging with the first list will take O(n) time, giving the algorithm a complexity of O(n) on a
nearly sorted list. For an unsorted list, the list will consist of mostly unordered pairs which will
be placed in the second list to be sorted by quicksort. In this case the algorithm deteriorates to
the average case complexity of quicksort, namely O(n log n).

page 16
3.4 Natural Mergesort

Natural mergesort is described by Knuth [6] and takes advantage of the runs, ascending and
descending sequences, within a list. The algorithm merges runs from opposite ends of the
unsorted list, the merged sequences being placed at alternate ends of a separate list. When all
runs from the first list have been merged the separate list will contain half the number of runs as
the first. The separate list is then processed in a similar manner to the first until eventually a list
contains only one non decreasing sequence.

listl contains the unsorted list

list2 is the list used in processing

begin

number of runs := n - some arbitrary value> 1

lista, listb := listl, list2
while number of ascending runs> 1 do
begin
1, r := 1, n
while l <= r do
begin
merge ascending sequence starting at l with the ascending
sequence (right to left) starting at r
put merged sequence in opposite end of listb
l := next ascending sequence from left
r := next ascending sequence from right
end
listb, lista := lista, listb
end

if listb = list2 then - copy unsorted array to sorted array

listl := listb

end

If the list is nearly sorted there will be few passes, but on a random list there will be about
1/2n runs, resulting in O(log n) passes, since each pass halves the number of runs. For each
pass there will be O(n) comparisons giving the algorithm an average case and worst case
complexity of O(n log n).

page 17
 In the following example the arrows indicate the natural runs in the list. Each pass merges
natural runs from opposite ends placing the merged sequences at alternate ends of the new list.

- - - - -E-- -E-- ~ -
503 087 512 061 908 170 897 275 653 426 154 509 612 677 765 703
--;>----> ----> --------3> ----';),
<;--- ~----------- ~ - - - - - - - - -
503 703 765 061 612 908 154 275 426 653 897 509 170 677 512 087
--------,> ------';), --- ----:>

-E-- ~---------------------
087 503 512 677 703 765 154 275 426 653 908 897 612 509 170 061
-----------------------------> ------------------ >
<-------------
061 087 170 503 509 512 612 677 703 765 897 908 653 426 275 154
-----------------------------~---~------ a>

~-
061 087 154 170 275 426 503 509 512 612 653 677 703 765 897 908
~--------------------------------------------------------------------------------->

page 18
3.5 Smoothsort

Smoothsort is a sorting algorithm proposed by Dijkstra [8][9]. It attempts to address the

problem of sorting nearly sorted lists with O(n) complexity while retaining a worst case
complexity of only O(n log n), and to provide a smooth transition between the two. The
implementation of the algorithm has also been described by Hertel [10], with a slight variation
on Dijkstra's proposal. However the asymptotic behaviour of the algorithm remains the same.
Hertel's description is easier to analyse and implement and was chosen as the algorithm to be
coded and explained.

begin

while elements still remaining do

begin
k := llog2(number of remaining elements) I
Make a complete heap of the 2k-1 remaining elements.
Swap the new root leftwards until the
roots form a non decreasing sequence.
Sift the root into its correct place in the heap.
end

m := n
while m > 1 do
begin
if size of last heap is 1 then
Remove the heap from consideration,
element is in its correct position.
else
begin
Remove the root form consideration, the root element is in its
correct positionand split the remainder of the heap into two
heaps of equal size.
Swap new roots leftwards so that the
roots form a non decreasing sequence.
Sift the new roots into their correct place within their heaps.
end
m := m - 1 - one more element in its correct place
end

end

The algorithm uses a sophisticated data structure based on heaps and consists of two passes.
The first pass builds a forest of complete heaps. Each of the heaps constructed has 2k-1
elements, k ~ 0, so each element of the heap has either O or 2 sons. The first heap is as large as
possible and will contain at least half the elements of the list. Successive heaps are of decreasing
size. The exception to this rule occurs for the last two heaps constructed which may be of the

page 19
 same size. The root of each heap is at the right most position of the elements of the heap. Each
heap is constructed over A[i ..j] so that a root at position j has sons at position j-1 and at position
_ i + (j-1) div 2 - 1.
For example, in an array of size 28 the size of the heaps constructed will be 15,7,3 and 3, the
roots of each heap being located at positions 15, 22, 25, and 28 respectively. The last two heaps
constructed are of size 3. The sons of each element are indicated by arrows.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

I I I I I I I I I I I I I I ~ I I I I I l%a I l%a I ~
e:iJte:iJVJlC:-!JlC:-!JVf.!JC:-!Jl C:-!JUJ t UJ t UJ

In addition to the above properties the forest of heaps is constructed so that the roots of the
heaps form a non decreasing sequence. The last heap then constructed will have a root at
position n and A[n] will be the largest element of all the roots and therefore all the heaps.
A heap of size k can be constructed in O(k:) time [4] and since the sum of sizes of all heaps
constructed is n then the construction of the forest of heaps will take O(n) time. To keep the
roots of all the heaps in non decreasing order will take 0(2 log n) time since the root can be
swapped left at most log n times and then sifted to its maximum depth of log k, where k is less
than or equal to n, in one of the heaps. The first pass then takes time
O((log n)2 + n) = O(n).
For a list already in sorted order the forest of heaps exists already and no swapping is required
thus only the construction of the heaps is required, which will take O(n) time.

An example of the first pass of smoothsort on list A consisting of 11 elements follows. The
roots of the current heaps being considered and the heaps constructed are boldfaced.

7 45 2 54 5 32 45 5 23 53 4

The size of the first heap is 7, with the root of the heap at position 7. Sons of the root are located
at positions 3 and 6. Firstly two heaps are constructed in A[l..3] and A[4 .. 6].

7 2 45 32 5 54 45 5 23 53 4

A heap is constructed over A[l..7] by combining the two heaps in A[l..3] and A[4 .. 6] and the
new root A[7].

7 2 45 32 5 45 54 5 23 53 4

page 20
The root of the next heap is at position 10, sons of the root are at positions 8 and 9. A heap is
constructed in A[8 .. 10].

7 2 45 32 5 45 54 5 23 53 4
The new root 53 is swapped with roots to the left so that the roots form a nondecreasing
sequence. The new root is then sifted, if necessary, into its correct place within the heap.

7 2 45 32 5 45 53 5 23 54 4

The root of the last heap is at position 11 and contains no sons. The root is swapped left until
the roots form an ascending sequence.

7 2 45 32 5 45 4 5 23 53 54
The element 4 is sifted to its correct place within the heap and the first pass is completed.

4 2 7 32 5 45 45 5 23 53 54

The second pass of smoothsort passes from right to left in the list and consists of removing
the root of the last heap being considered (since this root is the largest current element) and
dividing the remainder of the heap into two heaps of equal size. Suppose the heap being
considered is in the list A[i..j] then the sons will be at positions i + (j - i) div 2 - 1 and j-1 and
will be roots of the heaps over A[i .. i + (j - i) div 2 -1] and A[i + (j - i) div 2 .. j-1]. The
structure is then rebuilt so that the roots of all the heaps form a non decreasing sequence. Each
step in the second pass removes one element from consideration as it will be in its correct place
and after all n elements have been removed the list will be sorted.
For a nearly sorted list each step in the second pass will remove one element from
consideration, but the new roots will need to be swapped leftwards infrequently as they are
usually in their correct position within the forest of heaps. Thus there will be O(n) steps for the
second pass giving the complete algorithm a complexity of O(n) on sorted or nearly sorted lists.
If the list is unsorted then the second pass may need to swap each new root O(log n) roots
leftwards and sift the new root to its maximum depth of O(log n) in its new heap. Since there
may be O(n) new roots constructed in the second pass, the second pass will have a complexity
of O(n (2 log n)) = O(n log n). Thus the complete algorithm has a complexity of O(n log n) for
lists that are unsorted.

An example of the second pass of smoothsort follows. The forest of heaps is constructed in the
example for the first pass of smoothsort. Roots of the heaps are at positions 7, 10 and 11.

4 2 7 32 5 45 45 5 23 53 54
The last element is removed from consideration, since it is in its correct position, and because it
is a heap of only one element it cannot be divided further.

4 2 7 32 5 45 45 5 23 53 54

page 21
The last element, 53, is removed and its heap divided into two smaller heaps of equal size.

4 2 7 32 5 45 45 5 23 53 54
The structure is then rebuilt by swapping the roots of the two new heaps leftwards so that the
roots form an ascending sequence and then sifting the elements into their correct position within
their new heaps.

4 2 7 5 5 23 32 45 45 53 54
The last two elements are removed from consideration as they are both heaps of size one.

4 2 7 5 5 23 32 45 45 53 54
The root of the last heap is removed, dividing the remaining elements into two heaps. The new
roots form an ascending sequence, as required.

4 2 7 5 5 23 32 45 45 53 54
The last root, 23, is removed and its sons divided into two heaps of size 1. The roots are then
sorted so that they form a non decreasing sequence.

4 2 5 5 7 23 32 45 45 53 54
The last two elements are removed since they are heaps of size one. The root of the last
remaining heap is removed and its two sons form the roots of two more heaps which are then
sorted.

2 4 5 5 7 23 32 45 45 53 54
The last two elements are removed as they are heaps of size one.

2 4 5 5 7 23 32 45 45 53 54

The list is sorted.

page22
3.6 Results

The main measure of the algorithms efficiency has been taken to be the number of
comparisons required to sort the list. Also measured were the number of array accesses and the
CPU time required to sort each list. The CPU time required to sort each list does not include the
time taken to collect statistics on the number of comparisons or array accesses.
The number of comparisons has been plotted against the sortedness ratio on lists of length
64, 256, 1024 and 4096 and the sortedness ratio was varied between 0% and 25% and between
0% and 25% on a reverse ordered list. Results were gathered on reverse ordered lists as the
decision of a list being sorted when its elements form a non decreasing sequence is arbitrary.
The number of comparisons were also plotted against lists with a completely random order
which highlighted some algorithms which had good performance on nearly sorted lists but bad
performance on lists in which the elements were in a random order. For the lists of length 4096
the CPU time was also plotted against the sortedness ratio. The CPU time was not considered
the main criterion for evaluation of the algorithm's efficiency as the time may depend on
machine features such as system overhead, disc accesses, time slicing etc, but the CPU time
should reflect the algorithms complexity and relationships of CPU time and the number of
comparisons should give similar curves when plotted against the sortedness ratio.
Each point on the graphs represent an average of 10 trials. Lines in red indicate lists with
reverse sortedness and have only been plotted when the forward and reverse sortedness differs
significantly, as is the case with cksort and smoothsort. Appendix B contains a complete list of
the results for lists of size 64 and 4096.

Linear insertion sort performed well on lists that were very close to being sorted, and for a
list of length 64 linear insertion sort was one of the better sorts compared. As the length of the
list increased the efficiency deteriorated compared with the other sorting algorithms tested. This
is mainly due to the average distance of elements from their final position, in the sorted list,
increasing as the length of the list increases.
Unfortunately insertion sort's efficiency on a reverse sorted list and lists that have sortedness
ratio in reverse represents the worst possible case and the algorithm deteriorates to its known
complexity of O(n2). For this reason it was impractical to collect statistics when the list was
sorted or nearly sorted in reverse.
Ysort and natural mergesort were not the best sorting algorithms analysed but they have
several features that make them worth considering. Both sorting methods are symmetrical in that
they handle reverse sorted or nearly sorted lists in an ascending manner equally well, making
these two sorting methods more robust than some of the other methods analysed. This is an
important feature since in practice a list may be nearly sorted in reverse as frequently as it is

page23
nearly sorted in an ascending manner and it makes sense to exploit this feature in a sorting
algorithm.
Both ysort and mergesort lose their O(n) efficiency characteristics quickly compared with the
sortedness ratio and the longer the list the faster the algorithms approached their worst case
complexities of O(n log n). For a list of size 4096 these algorithms were close to their worst
case complexities for a sortedness ratio of about 6%.
Smoothsort has a greater number of initial comparisons than all the other sorting methods
compared but it soon becomes more efficient when compared with other sorting methods such
as mergesort, heapsort and ysort. However, the performance of the algorithm does not support
Dijkstra's claim of a smooth transition from O(n) to O(n log n) complexity as the list becomes
unsorted. The worst case for smoothsort occurs on a reverse list which it is not capable of
handling in O(n) time as the heap is built with its leaves at the left of the list and the roots of each
heap, which are the greatest elements of each heap, at the right of the list.
Cksort is the best sorting technique analysed. It had the smoothest transition from an O(n)
complexity on a sorted list to an O(n log n) complexity on an unsorted list. For a list with
varying kin ratio in an ascending order cksort was unsurpassed. The descending sequences
proved a problem for cksort with both ysort and mergesort being more efficient initially and
while the list was close to sorted (kin ratio in the range 0% to 5%). The bad performance on
reverse lists is due to the design decision to extract elements from the initial list if they are
descending pairs. For a reverse list most of the elements will be descending pairs and will be
extracted and then sorted by quicksort resulting in an O(n log n) complexity.
The graph of the CPU time against the sortedness ratio, figure 9, confirms the complexity of
the algorithms, as the curves for most algorithms reflect the curves obtained for the number of
comparisons graphed against the sortedness ratio. Mergesort appears to be a more efficient
algorithm when its CPU time is drawn against other algorithms, but this is due to the recursive
nature of most of the algorithms and the iterative nature of the mergesort algorithm. Quicksort is
the most efficient, in terms of CPU time, when the list is totally unsorted.

page 24
C'OF!MAC.

r,

'r
,f
µ_

,,Iffll!T
,L

,~~ t
-:°'·•'..·.

+i, ··.·
.S f , ~
~
1'/1«~ "~so-\
· ~~
. ~r:,o<t Si~
J 9 ·~ ~rt
~ ~Q.)\~ :,..':.::.-~-:
H
,,_
-H
µ

ff
Ill

I U)
c
0
U)
.c:::
ro
0.
I
I
E
II I 0
LJ
' Ol
CTl
_g 7

·.·
t '·-"
",

l f
I
I'

j '
'

:t
_, '---.....µ...----1-------+- - - - - + - - - , - - - ----........ .. . .. x
-c 6.25 fQ\\ct011\
I l
t
~
I
:.i- Sortedness Ratio 0/o
+

t 11

,Figure 5. Sortedness Ratio and the Number of Comparisons.

~
 l<~1fS'r\
,~~~~-:::~#2~.....,....;~+-,+....,--~~===-'7'1"""-="==;:::;.;=:;:;::::;::::··,-ia;~<\
~~y\ ....·~ \
l(C \<.r.prt
- - - ~-, S~~\
----~~,.......... . . . . . ~·=-"·-~-~~cw,c(\
11

1· >..

~ ----------~----------~----------~----------.-+
6.'25 18.75 25.0
....... ..... x
rorrurn.

Sortedness Ratio 0/o

Figure 6. Sortedness Ratio and the Number of Comparisons.

 H

n=1024

.-.·.·.-.·
·.·-.·.·

I I
I
I
i
j.

l
i1L
1ll!1
1 (/)
11 11
c
1A
111
1 1
I 0
"' 1· (/)
i11 1 '
L
ro
0..
E
0
LJ
~
_g

. . _ _ - - - - - - - - - - - - - - - - - - - -- ~ · . . . . . . . . . . • l(

6.25 12 .5 ,s.75 1.':>.0 rol\dom

Sortedness Ratio 0/o

IFigure
, 7. Sortedness Ratio and the Number of Comparisons.
:-·-:-:,.·.
[
;1t l-vJ~.;,,<r
),( lta:f'.P' ~

x~t

I V)
c
'o
1.1)
!-
(_
ro
Cl. I

'E
0
LJ
C\l
en
_Q

I I

.... .. .. . . .. ,.
f'Ol)ODIT\

Sortedness Ratio 0/o

Sortedness Ratio and the Number of Comparisons.

 .,..s\"X:<l\"6:icl :.::->::~:
'< '(:-.cd
5~ 1

U)
·~ ~('~

~H "'~"
tJ
c .~ .·-·
(D
u
aJ .><C~t
~

.~
aJ

-+- · tJer~ X~_r'cOI'\

.J(Q.\lt:\l,.<;Ofl
=>
n.. , 0.1J1c\i.8:;r.\
LJ ~C\<.'i!::.'1'\
N
cr,
..9

',t~.::.;
r..- .,· ,~..
I~-. I

~ "'"rr'-~~c--- t - -~ ~ ~- - l - -~ I '"?"?" .•..• ' ... . . . )\ '

- - ; - -- ----:-7'---:'--+-
\8.1 5 2s.o ran~6m

Sortedness Ratio 0/o

9 Sortedness Ratio and the CPU time.

3.7 Conclusions

From the graphs of the results it can be seen that if the structure of the list is unknown
quicksort remains the best internal sorting algorithm, since it out performs all other algorithms
when the list is unsorted. When the sortedness ratio of the list is greater than 5% quicksort is
only outperformed by cksort, except for when the list is very short, when linear insertion so1t is
also more efficient. For very short lists linear insertion sort is the best algorithm to implement
due to its short simple code even though it may not be the most efficient in terms of the number
of comparisons. For lists that are known to be nearly sorted in an ascending manner cksort is
the best algorithm to implement, but the bad performance of this algorithm on lists that are
nearly sorted in reverse and on random lists make quicksort the more preferable algorithm under
these conditions.
Of the other algorithms compared, several of them, ysort and mergesort, handled lists that
were nearly sorted in reverse as efficiently as list that were nearly sorted in an ascending
manner, but the transition from O(n) to O(n log n) complexity was rapid making quicksort the
more preferable algorithm for all but very nearly sorted lists. Smoothsort was easily
outperformed by cksort and in most cases quicksort as well. Smoothsort could not efficiently
sort lists with reverse sortedness and the algorithm is also the most complicated. For these
reasons it is not a suitable algorithm to implement under any situations.
The ideal algorithm should provide the best performance under all situations. Lists with
ascending and descending sortedness could be handled efficiently and there would be a smooth
transition from O(n) to O(n log n) complexity as the list became more unsorted. The graphs
highlighted several deficiencies in each of the algorithms analysed and none of the algorithms
considered gave ideal performance under all conditions.

page25
4.0 IMPROVEMENTS, MODIFICATIONS AND OTHER IDEAS

The results in the previous sections showed many of the deficiencies in the algorithms
implemented and this motivated changes to the algorithms in the hope of decreasing the number
of comparisons required to sort a list. Attempts to modify the algorithms to cater for lists with
reverse sortedness and random order were made. In some cases simple changes resulted in
dramatic increases in performance and more elaborate changes resulted in a degradation of
performance. Particular attention was paid to cksort as it had the best performance for lists with
an ascending sortedness ratio and had the potential to be the best all round algorithm analysed.
Where improvements in an algorithm's performance have occurred results have been graphed
against the unoptimized version on lists of size 4096. Appendix B contains a complete list of the
results for the optimized versions for lists of length 64 and 4096.

4.1 Improvements to Ysort

Although ysort has good performance on sorted and nearly sorted lists (kin in the range 0%
to 5%), its performance deteriorates rapidly to its O(n log n) behaviour. The major reason for
this deterioration is the expense in finding the maximum and minimum values of the left and
right subfiles. If the subfiles are unsorted then most of the elements will require two
comparisons to determine if they are the new maximum or new minimum element.
Ysort's biggest improvement over quicksort is in determining whether or not the subfiles are
sorted and not by calculating the maximum and minimum values of the subfiles. A more
efficient algorithm could then determine whether the list is sorted by taking only one comparison
per element. Also, without any additional comparisons the maximum or minimum element of
each subfile could be determined.
The ysort algorithm was modified with the following code placed after a subfile had been
partitioned:

{ for the left subfile}

max:= 1
fork:= (1 + 1) to j do
if A[max] <= A[k] then
max := k
else
lsorted := false
swap(A[max], A[j])

and a similar piece of code to determine the minimum element of the right subfile and whether

page26
 ;.:: · _···.

Lil
I l
c
0 ..
Lil
L
ro 15
0..
E
0
LJ
(\1
0,
_g

14

II

'I
'1

1I

r{ '=.·.·<'..·.:
13'r I. ..
f

!::;::::;:
.o.o
Sortedness Ratio 0/o

++ Figure 10. Sortedness Ratio and the Number of Comparisons.

the right subfile is sorted.
As can be seen, each element in each subfile now requires only 1 comparison to determine
whether the subfile is sorted and in addition the maximum value of the left subfile and the
minimum value of the right subfile are swapped into their correct place.
To sort the subfiles the last element of the left subfile and the first element of the right subfile
do not need to be considered as they are in their correct places. In the unoptimized algorithm
both the minimum and maximum element of each subfile are in their correct place after a file has
been partitioned.
The code to control sorting of the subfiles will then be as follows, where lsorted and rsorted
indicate whether the left and right subfiles are sorted and 1 and r are the leftmost and right most
positions of the unsorted file:

if not lsorted then

sort the subfile A[l .. j-1]

if not rsorted then

sort the subfile A[i+l .. r]

The above changes resulted in significant improvements for lists of length 64, 256, 1024,
and 4096. Figure 10 shows the improvement for lists of length 4096 and Appendix B contains a
complete listing of all results for lists of length 64 and 4096. In all cases the biggest
improvement occurred when the lists was totally unsorted, but improvements occurred for all
degrees of sortedness.

4.2 hnproven1ents to Cksort

The algorithm for cksort is very effective and is the most efficient sorting algorithm analysed.
Its major deficiency is that it cannot process lists with reverse sortedness as efficiently as those
with forward sortedness. As can be seen from the graphs in the previous chapter, cksort is close
to its O(n log n) behaviour on lists with reverse sortedness.
An extension to the algorithm uses a third list C. Decreasing pairs are extracted from list A
and placed in list B as before, and increasing pairs are then extracted from list B and placed in
list C. This results in two sorted lists; A, which contains non decreasing elements, and B, which
contains non ascending elements. List C contains all the unordered pairs that were extracted
from list B and is sorted by quicksort if there are greater than 30 elements, and by linear
insertion sort otherwise. Finally all three list are merged into the original list A. Lists that are
sorted or nearly sorted in reverse will contain mostly decreasing pairs which remain in list B,

page27
C,Or~MAC'I< C, lA H

I ,,t+
I
I

I I, i
I
t #1
I

f1f
+ I
I l
' ' 1 ;,1C~O!\\
l'' f'
I
J ,16 ~h,,1~(1-'.t>rl
I '
:t J: ;

r
t,
:t:

T
'
'

'\\.1I
tP
'

15'
t I+
t

I V
1 t
+ I ,,
t

' II
+I
-.-, lf *
14
·1

i- f.

+
~'
i: 1

l~
r1
I t
t3
I

I ~' -+

II
E
t ji
II '
I
t [:,)::
I 1·- ---,
+12,,
I
I I I J. .
! ,.
I I

1 n: -l-

I .0.0 , .6.25 12.'5 18.75

I I
I
a~ Sorte_do_ess Ratio 0/o
1I f~ :,

tt; if
I, I I
1,J

Fig Ure 11. Sortedness Ratio and the Number of Comparisons.

11lt
'
I 1i
1:f: II '

~ I
f
J

CL
 resulting in a small number of unordered pairs in list C.
A further improvement results from partitioning until the subfiles are smaller than some
_ critical value and using linear insertion sort to complete the sorting on the partitioned subfiles.
The threshold value at which to stop partioning was selected as 10 [3]. This is a reasonably
simple improvement to make since the algorithm for linear insertion sort is contained within the
algorithm for cksort.
As can be seen from figure 5 to figure 9, cksort performed badly when the list was unsorted
and quicksort easily outperforms it in these situations. This is a result of a bad choice of
partition element. The ideal partition element is close to the median element of the the list being
sorted. Choosing the middle element as the partition element can result, and frequently does, in
an element that is typically small or large compared with the other elements of the list being
sorted. This is because the unsorted list, list C in the improved algorithm, consists of pairs of
increasing elements. By choosing the mean of the two middle elements, a value that better
approximated the median was obtained resulting in fewer partition steps and therefore fewer
comparisons. For a random list of size 4096 this one simple modification resulted in almost a
10% reduction in the number of comparisons. The optimized algorithm is as follows.

list A - the unsorted array

list B - contains the descending sequences.
list C - contains the unordered pairs

Extract all decreasing pairs from list A (as before) and place in list B.
Array A now contains a sorted non decreasing list.

Extract all increasing pairs from list Band place in list C.

Array B now contains a sorted non ascending list.

if number of elements in list C is less than 30 then

Sort list C by insertion sort.
else
Sort list C by quicksort, using a cutoff and insertion sort,
the partition element is the mean of the two middle elements.

Merge lists A, Band C into A.

The major modification to cater for reverse lists resulted in an O(n) complexity when the list
was sorted in reverse and a slight degradation in efficiency for lists with ascending sortedness.
Both of the other two improvements resulted in an increase in efficiency for lists with both
forward and reverse sortedness, see figure 11. The biggest increase in efficiency as a result of
using a cutoff occurs when the list is unsorted as then a large portion of the original list will be
present in list C and require sorting by quicksort.

page 28
4.3 Alterations to Smoothsort

The sophisticated data structure of smoothsort makes it difficult to change the structure of the
program and almost impossible to improve the algorithm to cater for lists that are nearly sorted
in reverse. One of the simplest changes to the algorithm is to alter the branching factor of the
heaps. Previous results were obtained by using a binary heap. By using a ternary heap it was
hoped that the number of comparisons needed to sort a list would be reduced.
For a list of size n the upper bound for the number of binary heaps in a forest of complete
heaps is,

r
N2 = log2n l,
and the upper bound for the number of ternary heaps is,

r
N3 = 1.2618 log2n + 0.2618 l,
(see Appendix C). Experiments performed on lists of size 1 to 1000 showed that there was
typically a greater number of ternary heaps than binary heaps.
Sorting using a single ternary heap has been shown to have a better performance than a
single binary heap [7] due to fewer comparisons being required to sift an element to its correct
place within the heap. Results on lists of size 64, 256, 1024 and 4096 showed that smoothsort
was inferior in just about all cases and this can be attributed to the greater number of average
heaps required. For an unsorted list in the second pass of smoothsort a greater number of heaps
results in a greater number of swaps, and therefore comparisons, to move an element to its
correct heap and this overcomes the advantage of fewer comparisons in sifting an element to its
correct place within a heap when using a ternary heap. For lists of length 4096, with the
sortedness ratio varied in an ascending manner the number of comparisons to sort a list are as
follows:

kin% binaiy heaps ternary heaps

0.00 16344.0 16334.0
6.25 60098.5 75755.4
12.50 69598.1 89833.7
18.75 80558.0 104173.2
25.00 83115.0 105701.5

The number of comparisons for lists of length 64, 256 and 1024 gave similar results with a
forest of binary heaps having a better performance in almost all cases. A forest of ternary heaps
gave almost the same number of comparisons when the list was completely sorted. However as

page29
the list became unsorted a forest of binary heaps always had a better performance. For a
sortedness ratio of 25% and a list of length 64 a forest of binary heaps had 12% fewer
comparisons and for lists of length 256, 1024 and 4096 a forest of binary heaps had at least
20% fewer comparisons.

4.4 Alterations to Linear Insertion Sort

Although linear insertion sort has a very bad worst case performance it gives the best results,
along with cksort, when the list is sorted in an ascending manner. Modifications to insertion sort
to improve its performance when the list is not sorted are worthy, since the code is short and
simple and the algorithm is frequently implemented by itself, and as a hybrid algorithm.
The biggest disadvantage with linear insertion sort is a result of the O(n2) comparisons it
performs when the list is unsorted. To overcome this problem and to preserve the algorithm's
best case performance the algorithm was modified.
During the sorting of a list of n elements the following situation exists:

1 2 3 k n

I I I I· . . II II
sorted list
l
element to sod
unsorted list

The implemented algorithm performs a linear search from right to left to locate the correct
position for the unsorted element. However, the list it searches is in fact sorted and an
improvement would be to use a binary search to locate the position for the unsorted element.
This would eliminate insertion sort's best case performance since it would not necessarily
compare the unsorted element with the element immediately to the left. To overcome this the
sorted list is first searched using a divergent binary search to isolate the region for the new
element and then using a conventional binary search to isolate the correct position for the
element. For example, consider the list above being sorted by this algorithm when the next
element to be sorted is at position k. The element is compared with elements at positions k-1,
k-3, k-7, k-15, ... In this way the correct region for the element can be located in O(log2 k)
comparisons. The actual position for the element is then isolated using a conventional binary
search on the region determined and will take at most O(log2 k/2) comparisons. In this way the
conect position for an element can be located with

page30
 O(log2k) + O(log2k/2) = O(log2k)
s; O(log 2n),

comparisons and yet only one comparison is required if the list is sorted. This may seem an
ideal solution to the problem but once a position is located there may be O(n) swaps to move the
unsorted element into position giving the algorithm a worst case time complexity of O(n2) as
before.
In an attempt to overcome the possible O(n 2) complexity when sorting a list the following
data structure that incorporated pointers was used:

sorted order

1 2 3 4 5 n

linked lists
in sorted
order

To locate the correct position for an element within the so1ied list the array is searched using
first a divergent binary search and the a convergent binary search as before. For each
comparison the element at the head of the list is compared with the unsorted element. O(n log n)
comparisons are required to locate the correct linked list to search. The correct position within
the list is then located by using a linear search and the unsorted element is inserted into the list
without causing any swaps.
Unfortunately this algorithm had an inferior performance compared to straight linear insertion
sort. The best case performance was preserved for completely sorted lists, but as the sortedness
ratio increased straight linear insertion sort easily outperformed the new algorithm. With a
sortedness ratio of 6.25% and a list of length 64 the new algorithm had more then 3 times as
many comparisons and for higher sortedness ratio or with longer length lists there was even
more comparisons required for the new algorithm.
The problem with this algorithm can be seen when sorting the list,

10 1 2 3 4 5 6 7 8 9,

page 31
 which has a sortedness ratio of 0.1 and is therefore nearly sorted. The algorithm inserts each
- element at the head of the first linked list and the advantage of using binary search to locate the
correct position has been lost, since the current sorted list consists of only one linked list.
Suppose the first 4 elements have been sorted then the situation is as follows.

1 2 3 4

To insert the next element, 4, the complete chain has to be traversed to locate the correct
position, and the algorithm is equivalent to straight linear insertion sort with respect to the
number of comparisons. Also the new algorithm had a considerable slower execution time due
to pointer manipulation.

page 32
5.0 CONCLUSIONS

Of all the algorithms compared the optimized cksort had the best performance in all cases,
except for when the list was very unsorted, and then quicksort only outperformed cksort by a
linear factor. However quicksort is a considerably easier algorithm to implement and if the
structure of the list is unknown, or if it is known that the list is most likely to be totally unsorted
then quicksort is probably the more preferable algorithm to implement.
As mentioned previously, cksort, and the other algorithms, have been analysed using a
measure of sortedness that cksort exploits efficiently, whereas in practice the sortedness ratio
may not best capture the sortedness properties of a nearly sorted list. For example, consider the
list,

n/2 + 1, n/2 + 2, ...... , n, n/2, n/2 - 1, ..... 1,

which consists of an ascending and descending sequence and has a sortedness ratio of 1/2.
Cksort would require all the elements to be sorted by quicksort and the optimized algorithm
would require n-2 of the elements to be sorted by quicksort, in both cases the algorithms would
have a complexity of O(n log n) on this list. On the the other hand, sorting the list by natural
mergesort would require only two passes and therefore have a complexity of O(n) on a list with
this structure.
The development of faster algorithms for nearly sorted lists requires measures of sortedness
that better capture the properties of a list that allow O(n) time sorting. These measures should
also return appropriate values for nearly sorted lists that occur in practice and not be confined to
a theoretical definition of a nearly sorted list.

page 33
 APPENDIX A

- A summary of the results obtained for the preliminary tests.

The average has been taken over 10 trials.

Table 1

Number of Ascending Sequences

k n average minimum maximum

0 10 1.0 1 1
1 10 2.0 2 2
2 10 3.0 3 3
3 10 4.0 4 4

0 100 1.0 1 1
5 100 5.9 5 6
10 100 1.1 10 11
15 100 14.9 13 16
20 100 19.1 18 21
25 100 23.5 22 25
30 100 27.1 25 30

0 1000 1.0 1 1
50 1000 49.8 49 51
100 1000 96.8 93 99
150 1000 139.3 135 143
200 1000 180.9 172 183
250 1000 219.4 209 227
300 1000 256.1 248 267

Table 2

1 I Largest Ascending Sequence

k n average minimum maximum

0 10 0.1 0.1 0.1

1 10 0.14 0.11 0.16
2 10 0.19 0.14 0.25
3 10 0.25 0.17 0.33

0 100 0.01 0.01 0.01

5 100 0 .026 0.016 0.040
10 100 0.036 0.025 0.046
15 100 0.052 0.042 0.063
20 100 0.063 0.044 0.100

page 34
 0 1000 0.001 0.001 0.001
50 1000 0.012 0.007 0.018
100 1000 0.020 0.014 0 .029
150 1000 0.029 0.021 0.039
200 1000 0.040 0.029 0.050
250 1000 0.047 0.036 0.059
300 1000 0.056 0.040 0.067

Table 3

Number of Inversions
k n average minimum maximum '~·~-.,--
.
- -
---

0 10 0.0 0 0
1 10 3.6 1 7
2 10 6.7 2 12
3 10 9.4 3 17

0 100 o.o 0 0
5 100 186.4 89 239
10 100 302.1 218 382
15 100 528.9 429 658
20 100 666.8 471 795
25 100 821. 9 759 919
30 100 905.6 793 997

0 1000 0.0 0 0
50 1000 16214.5 14865 17833
100 1000 32619. 4 28923 35162
150 1000 48892.9 43281 52770
200 1000 63572.4 55146 72136
250 1000 78816.0 75795 82604
300 1000 93154.2 88144 99383

Table 4

Number of Exchanges
k n average minimum maximum

0 10 o.o 0 0
1 10 3.6 1 7
2 10 4.9 2 8
3 10 4.6 2 7

0 100 o.o 0 0
5 100 62. 6 38 83
10 100 71. 5 60 83
15 100 85.5 70 94
20 100 85.8 75 95

page35
 0 1000 0.0 0 0
50 1000 903.7 856 939
100 1000 924.4 841 969
150 1000 933.5 871 970
200 1000 957.4 908 991
250 1000 965.6 942 982
300 1000 959.4 931 984

page36
 APPENDIX B
A listing of the results obtained for the sorting algorithms. Natural mergesort is refen-ed to
_ here as mergesort and linear insertion sort is refen-ed to here as just insertion sort.
Entries that have been marked by a'-' indicate situations in which the time to compute the
required statistics is unnecessarily large. Results have been included for lists of length 64 and
4096 only and in which the sortedness ratio has been varied in both an ascending and
descending manner. The direction of sortedness for each list is indicated. Each value represents
an average of 10 trials.

Comuarisons

Table 1 n= 64, Forward sortedness

kin% Quicksort Heapsort Insertion sort Smoothsort Ysort Ck sort Mergesort
0.00 258.0 593.0 63.0 234.0 128.0 63.0 126. 0
6.25 307.8 593.4 135.6 345.1 487.7 137.7 471. 6
12.50 356.0 581.8 242.7 435.4 574.4 176 .3 527.1
18.75 338.8 588.2 287.8 453.4 563.6 219.4 582.9
25.00 387.1 582.1 384.3 518.6 614.0 299.5 604.0
random 455.6 572.3 1014.7 713.6 728 .o 566.9 651.6

Table 2 n=4096, Forward sortedness

kin% Quicksort Heapsort Insertion sort Smoothsort Ysort Cksort Mergesort

o.oo 40968.0 88835.0 4097.0 16352.0 8196.0 4097.0 8194.0
6.25 54263.0 87563.9 337013.7 63884.7 106560.5 13794.0 72889.4
12.50 54850.9 86532.7 687817.3 69598.1 112811. 5 20198.3 80473.6
18.75 55981. 4 86382.5 1005505.1 80558.0 117022.2 26942.4 88205.9
25.00 55642.1 86382.5 1322238.3 83115. 0 119404. 0 33637.6 87958.4
random 63271.1 85833.7 126138.1 149505.5 75164.7 89458.6

Table 3 n=64, Reverse sortedness

kin% Quicksort Heapsort Insertion sort Smoothsort Ysort Cksort Mergesort
0.00 259.0 525.0 758.0 128.0 367.0 126 .o
6.25 293.8 532.6 760.9 455.6 472.0 471. 6
12.50 317.2 533.7 752.7 531.2 495.5 527.1
18.75 344.1 541. 6 753.9 555.7 528.9 582.9
25.00 368.2 544.7 748.5 613.4 539.6 604.0

Table 4 n=4096, Reverse sortedness

kin% Quick sort Heapsort Insertion sort Smoothsort Ysort Ck sort Mergesort
0.00 40975.0 82304.0 109203.0 8192.0 47131.0 8190.0
6.25 52633.9 82390.7 112681. 4 105879.0 64036.3 72860.5
12.50 53751.3 82651.2 115656. 7 111699. 8 64877.5 80473.6
18.75 54670.8 83066.9 117525.9 115309.8 66717.0 88205.9
25.00 55863.6 83355.7 120199.9 118868.9 67016.1 87958.4

page37
Accesses
Table 5 n=64, Forward sortedness
k/n % Quicksort Heap sort Insertion sort Smoothsort Ysort Cksort Mergesort
0.00 28.0 1448.0 0.0 0.0 o.o 0.0 320.0
6.25 157.8 1432.8 193.0 217.6 169.6 248.0 678. 4
12.50 243.8 1407.6 460.4 402.8 206.8 359.4 742.4
18.75 249.0 1416.4 553.6 446.0 229 .2 451.2 806.4
25.00 335.4 1392.8 751. 4 592.8 252.0 541. 4 832.0
random 474.8 1322.4 2028.0 1123 .2 342. 4 705.7 896.0

Table 6 n=4096, Forward sortedness

kin% Quicksort Heapsort Insertion sort Smoothsort Ysort Cksort Mergesort

0.00 2044.0 193440.0 0.0 0.0 o.o 0.0 20480.0
6.25 31169.8 189874.0 684016.2 76088.4 22188.8 20639.1 86016.0
12.50 30202.2 187368.8 1375597.6 91896.0 23484.0 25312.3 94208.0
18.75 34807.0 185750.8 2010978.2 113356. 4 26668.8 30574.1 102400.0
25.00 36199.4 186473.2 2644445.0 119770. 0 28080.8 35215.9 102400.0
random 52810.0 182683.2 207862.4 43194.4 69274.8 104857.6

Table 7 n=64, Reverse sortedness

kin% Quicksort Heapsort Insertion sort Smoothsort Ysort Cksort Mergesort
0.00 152.0 1152. 0 1312.0 128.0 419.0 320.0
6.25 223.2 1169. 6 1301. 6 246.4 545.2 678.4
12. 50 292.4 1186. 0 1273.2 295.2 574.7 742. 4
18.75 348.4 1195. 6 1281.2 308.8 646.6 806.4
25.00 359.2 1209.6 1257.6 317.2 642.4 832.0

Table 8 n=4096, Reverse sortedness

kin% Quicksort Heapsort Insertion sort Smoothsort Ysort Ck sort Mergesort
0.00 10232.0 170120.0 178728.0 8192.0 26627.0 20480.0
6.25 36123.2 170740.4 185080.4 28266.8 52099.4 86016.0
12. 50 37992.2 171579 .2 190944.0 29838.0 54345.4 94208.0
18.75 40624.6 172835.6 195164.4 32512.8 57426.6 102400.0
25.00 42002.0 173777.2 199878.0 33726. 8 59320.4 102400.0

CPU Time (seconds)

Table 9 n=64, Forward sortedness

kin% Quick sort Heapsort Insertion sort Smoothsort Ysort Cksort Mergesort
0.00 0.00 0.02 0.00 0.02 0.00 0.00 0.00
6.25 0.01 0.02 0.00 0.03 0.01 0.00 0.01
12.50 0.01 0.02 0.01 0.03 0.02 0.01 0.01
18.75 0.01 0.02 0.01 0.03 0.02 0.01 0.01
25.00 0.01 0.02 0.01 0.04 0.02 0.01 0.01
random 0.01 0.02 0.02 0.03 0.02 0.02 0.01

page 38
Table 10 n=4096, Forward sortedness
kin% Quicksort Heapsort Insertion sort Smoothsort Ysort Cksort Mergesort
0.00 0.49 3.00 0.10 1. 04 0.22 0.12 0.15
6.25 0. 74 2.87 6.84 2.59 2.67 0.32 0.80
12.50 0.74 2.82 13.76 2.95 2.76 0.45 0.87
18.75 0.76 2.80 20.10 3.33 2.87 0.59 0.95
25.00 0.77 2.82 26. 44 3.44 2.93 0. 71 0.95
random 0.89 2.90 4.35 3.84 1.50 0.95

Table 11 n=64, Reverse sortedness

kin% Quicksort Heapsort Insertion sort Smoothsort Ysort Cksort Mergesort

0.00 0.01 0.02 0.05 0.01 0.1 0.00
6.25 0.01 0.02 0.06 0.02 0.01 0.01
12.50 0.01 0.02 0.05 0.02 0.01 0.01
18.75 0.01 0.02 0.05 0.02 0.02 0.01
25.00 0.01 0.02 0.05 0.02 0.02 0.01

Table 12 n=4096, Reverse sortedness

kin% Quicksort Heap sort Insertion sort Smoothsort Ysort Cksort Mergesort
0.00 0.52 2.66 4.35 0.30 0.78 0.15
6.25 0.74 2.66 4.49 2.69 1.26 0.78
12.50 0. 76 2.68 4.59 2.80 1.28 0.85
18.75 o. 79 2.69 4.66 2.90 1.32 0.93
25.00 0.80 2.69 4.77 2.98 1.34 0.93

OPTIMIZED ALGORITHMS
Comparisons

Table 13 n = 64, Forward sortedness

kin% Optimized Y sort Optimized Cksort

0.00 126.0 63.0
6.25 451. 9 143.0
12.50 511.4 184.6
18.75 504.9 230.6
25.00 538.9 304.1
random 624 .1 564.4

Table 14 n = 4096, Forward sortedness

kin% Optimized Y sort Optimized Cksort

0.00 8190.0 4095.0
6.25 84557.0 13701.1
12.50 86830.9 20084.9
18.75 90123.1 26452.3
25.00 89951. 6 32571. 8
random 105606.9 70675.1

page 39
Table 15 n = 64, Reverse sortedness

kin% Optimized Y sort Optimized Cksort

o.oo 126. 0 96.0
6.25 423 .6 232.7
12.50 481. 6 317.0
18.75 497.1 432.0
25.00 518.4 434.6

Table 16 n = 4096, Reverse sortedness

kin% Optimized Y sort Optimized Cksort

o.oo 8190.0 6144.0
6.25 83384.5 22002.7
12.50 85932.7 29020.4
18.75 88080.7 35578.3
25.00 90498.5 41278.0

Accesses

Table 17 n = 64, Forward sortedness

kin% Optimized Y sort Optimized Cksort
o.oo 5.0 2.0
6.25 210.4 269.4
12.50 250.4 400.4
18.75 277.1 510.0
25.00 304.7 683.6
random 393.9 1092.5

Table 18 n = 4096, Forward sortedness

kin% Optimized Y sort Optimized Cksort
0.00 5.0 2.0
6.25 25040.3 23755.9
12.50 26756.1 31746.0
18.75 30301. 9 39622.7
25.00 31886.0 46800.6
random 47780.6 98330.3

Table 19 n = 64, Reverse sortedness

kin% Optimized Ysort Optimized Cksort

0.00 129 .o 636. 0
6.25 284.5 713.8
12.50 339. 7 856.4
18.75 357.5 1096.4
25.00 369 .5 1022.4

page40
Table 20 n = 4096, Reverse sortedness

kin% Optimized Y sort Optimized Cksort

0.00 8193.0 40956.0
6.25 30969.7 50499.2
12.50 33020.5 57473.3
18.75 36019.8 64331.1
25.00 37698.2 70065.2

CPU Time (seconds)

Table 21 n = 64, Forward sortedness

kin% Optimized Y sort Optimized Cksort

0.00 0.00 0.00
6.25 0.01 o.oo
12.50 0.01 0.00
18.75 0.01 0.01
25.00 0.01 0.01
random 0.01 0.01

Table 22 n = 4096, Forward sortedness

kin% Optimized Y sort Optimized Cksort

0.00 0.08 0.10
6.25 1.04 0.31
12.50 1.07 0.42
18.75 1.12 0.54
25.00 1.13 0.64
random 1.37 1.35

Table 23 n =64, Reverse sortedness

kin% Optimized Y sort Optimized Cksort

0.00 0.00 0.00
6.25 0.01 0.01
12.50 0.01 0.01
18.75 0.01 0.01
25.00 0.01 0.01

Table 24 n = 4096, Reverse sortedness

kin% Optimized Y sort Optimized Cksort

0.00 0.12 0.26
6.25 1.07 0.52
12.50 1.10 0.65
18.75 1.14 0.76
25.00 1.17 0.87

page 41
APPENDIX C

The appendix gives a proof for the upper bound for the number complete ternary and binary
heaps in a list of size n.

For a forest of complete binary heaps the relationship between the length of the list, n, and
the upper bound of the number of heaps, k is,
n k
1 =1
2 =1+1 2
5 =3+1+1 3
12 = 7 + 3 + 1 + 1 4
27 = 15 + 7 + 3 + 1 + 1 5
58 = 31 + 15 + 7 + 3 + 1 + 1 6
etc,

and a precise mathematical description of this relation is

n = 1 + 1:i=l,k-1 (2i - 1)

= 1 - (k-1) + 1:i=l,k-1(2i)

= 1:i=l,k-1 (2i) - k + 2
= (2 + 4 + 8 + ... + 2k-l) - k + 2
= 2(1 - 2k-l) I (1 - 2) - k + 2
= 2 - (2 - 2k) - k
= 2k-k

Thus the number of complete binary heaps is given by,

k = log2(n + k),

and since log2(n + k) ~ log2n as n ~ oo then k ""log2n,

Hence an upper bound for the number of complete binary heaps is,

For a forest of complete ternary heaps the relationship between the upper bound of the
number of heaps and the length of the list is,
n k x
1 1 0
3=2+1 3 1
11=8+2+1 5 2
37 = 26 + 8 + 2 + 1 7 3
117 = 80 + 26 + 8 + 2 + 1 9 4
etc,

page42
where k is the number of heaps and x == (k - 1) I 2. A precise mathematical definition is as
follows:

n == f(x) == 1 + :Ei=l,/3i - 1)

== 1 - x + :Ei= 1_x<3i)
= 1 - x + (3 + 9 + 27 + ... 3x)
= 1 - x + 3(1 - 3x) I ( 1 -3)

f(x) = (3x+l I 2)- x - 1/2

n = f(k) = 3(k+l)l2 ; 2 - k/2
2n + k = 3(k+l)/2
k = 2 log 3(2n + k) - 1

Since log 3 (2n + k) ~ log3(2n) as n ~ oo then

k "" 2 (log32n) - 1
= 1.2618 log 22n - 1
= 1.2618 log2n + 0.2618
Hence an upper bound for the number of.complete ternary heaps is

r 1.2618 Iog2n + 0.2618 l.

page43
 References

- [1] Hoare, C.A.R. Algorithm 63: Partition. Algorithm 64: Quicksort.

Commun. ACM 4, 7(July 1961), 321.

[2] Cook, C.R. and Kim, D.J. Best Sorting Algorithm for Nearly Sorted Lists.
Commun. ACM 23, ll(November 1980), 620-624.

[3] Sedgewick, R. Implementing Quicksort Programs.

Commun. ACM 21, lO(October 1978), 847-857.

[4] Aho, A.V., Hopcroft, J.E. and Ullman, J.D.

The Design and Analysis of Computer Algorithms, (Addison-Wesley, Reading, MA,
1975), 90-91.
[5] Wainwright, R.L. A Class of Sorting Algorithms Based on Quicksort.
Commun. ACM 28, 4(April 1985), 396-402.

[6] Knuth, D.E The Art of Computer Programming, Volume 3, Sorting and Searching,
(Addison-Wesley, Reading, MA, 1973), 161-163.

[7] Knuth, D.E The Art of Computer Programming, Volume 3, Sorting and Searching,
(Addison-Wesley, Reading, MA, 1973), 619

[8] Dijkstra, E.W. An Alternative for Sorting in Situ.

Science of Computer Programming 1, 1982, 223-233.

[9] Dijkstra, E.W. Errata. The Science of Computer Programming 2, 1982, 85.

[10] Hertel, S. Smoothsort's Behavior on Presorted Sequences.

Information Processing Letters 16, 1983, 165-170.

page44