Introduction to heapsort

K K Singh, RGUKT, Nuzvid

K K Singh 1 12/15/2021
 Sorting Revisited

● we’ve talked about the sorting algorithms like

● Merge sort & Quick Sort
● O(n lg n) worst-case running time
● Insertion sort
● sorts in place
● When array “nearly sorted”, runs fast in practice
● Next on the agenda: Heapsort
● Combines advantages of both previous algorithms

K K Singh 1 12/15/2021
 Heaps

● A heap can be seen as a nearly complete binary

tree: 16

14 10

8 7 9 3

2 4 1

● What makes a binary tree complete?

● Is the example above complete?
K K Singh 1 12/15/2021
 Heaps

● A heap can be seen as a complete binary tree:

16

14 10

8 7 9 3

2 4 1 1 1 1 1 1

● The book calls them “nearly complete” binary

trees; can think of unfilled slots as null pointers

K K Singh 1 12/15/2021
 Heaps

● In practice, heaps are usually implemented as

arrays: 1
6
1 1
4 0

A = 16 14 10 8 7 9 3 2 4 1 = 8 7 9 3

2 4 1

The root node is A[1]

Node i is A[i]
The parent of node i is A[i/2] (note: integer divide)
The left child of node i is A[2i]
The right child of node i is A[2i + 1]

K K Singh 1 12/15/2021
 Referencing Heap Elements

● So…
Parent(i) { return ⎣i/2⎦; }
Left(i) { return 2*i; }
right(i) { return 2*i + 1; }
● An aside: How would you implement this
most efficiently?

K K Singh 1 12/15/2021
 The Heap Property

● Heaps also satisfy the heap property:

A[Parent(i)] ≥ A[i] for all nodes i > 1
● In other words, the value of a node is at most the
value of its parent
● Where is the largest element in a heap stored?

K K Singh 1 12/15/2021
 Heap Height

● Definitions:
● The height of a node in the tree = the number of
edges on the longest downward path to a leaf
● The height of a tree = the height of its root
● What is the height of an n-element heap? Why?
● This is nice: basic heap operations take at most
time proportional to the height of the heap

K K Singh 1 12/15/2021
 Heap Operations: Heapify()

● Heapify(): maintain the heap property

● Given: a node i in the heap with children l and r
● Given: two subtrees rooted at l and r, assumed to
be heaps
● Problem: The subtree rooted at i may violate the
heap property (How?)
● Action: let the value of the parent node “float
down” so subtree at i satisfies the heap property
● What do you suppose will be the basic operation
between i, l, and r?

K K Singh 1 12/15/2021
 Heap Operations: Heapify()
Heapify(A, i)
{
l = Left(i); r = Right(i);
if (l <= heap_size(A) && A[l] > A[i])
largest = l;
else
largest = i;
if (r <= heap_size(A) && A[r] > A[largest])
largest = r;
if (largest != i)
Swap(A, i, largest);
Heapify(A, largest);
}

K K Singh 1 12/15/2021
 Heapify() Example

1
6
1
4
0
1
7 9 3
4
2 8 1

A = 16 4 10 14 7 9 3 2 8 1

K K Singh 1 12/15/2021
 Heapify() Example

1
6
1
4
0
1
7 9 3
4
2 8 1

A = 16 4 10 14 7 9 3 2 8 1

K K Singh 1 12/15/2021
 Heapify() Example

1
6
1
4
0
1
7 9 3
4
2 8 1

A = 16 4 10 14 7 9 3 2 8 1

K K Singh 1 12/15/2021
 Heapify() Example

1
6
1 1
4 0
4 7 9 3

2 8 1

A = 16 14 10 4 7 9 3 2 8 1

K K Singh 1 12/15/2021
 Heapify() Example

1
6
1 1
4 0
4 7 9 3

2 8 1

A = 16 14 10 4 7 9 3 2 8 1

K K Singh 1 12/15/2021
 Heapify() Example

1
6
1 1
4 0
4 7 9 3

2 8 1

A = 16 14 10 4 7 9 3 2 8 1

K K Singh 1 12/15/2021
 Heapify() Example

1
6
1 1
4 0
8 7 9 3

2 4 1

A = 16 14 10 8 7 9 3 2 4 1

K K Singh 1 12/15/2021
 Heapify() Example

1
6
1 1
4 0
8 7 9 3

2 4 1

A = 16 14 10 8 7 9 3 2 4 1

K K Singh 1 12/15/2021
 Heapify() Example

1
6
1 1
4 0
8 7 9 3

2 4 1

A = 16 14 10 8 7 9 3 2 4 1

K K Singh 1 12/15/2021
 Analyzing Heapify(): Informal

● Aside from the recursive call, what is the

running time of Heapify()?
● How many times can Heapify() recursively
call itself?
● What is the worst-case running time of
Heapify() on a heap of size n?

K K Singh 1 12/15/2021
 Analyzing Heapify(): Formal

● Fixing up relationships between i, l, and r

takes Θ(1) time
● If the heap at i has n elements, how many
elements can the subtrees at l or r have?
● Draw it
● Answer: 2n/3 (worst case: bottom row 1/2 full)
● So time taken by Heapify() is given by
T(n) ≤ T(2n/3) + Θ(1)

K K Singh 1 12/15/2021
 Analyzing Heapify(): Formal

● So we have
T(n) ≤ T(2n/3) + Θ(1)
T(n) = O(lg n)
● Thus, Heapify() takes logarithmic time

K K Singh 1 12/15/2021
 Heap Operations: BuildHeap()

● We can build a heap in a bottom-up manner by

running Heapify() on successive subarrays
● Fact: for array of length n, all elements in range
A[⎣n/2⎦ + 1 .. n] are heaps (Why?)
● So:
● Walk backwards through the array from n/2 to 1, calling
Heapify() on each node.
● Order of processing guarantees that the children of node
i are heaps when i is processed

K K Singh 1 12/15/2021
 BuildHeap()
// given an unsorted array A, make A a heap
BuildHeap(A)
{
heap_size(A) = length(A);
for (i = ⎣length[A]/2⎦ downto 1)
Heapify(A, i);
}

K K Singh 1 12/15/2021
 BuildHeap() Example

● Work through example

A = {4, 1, 3, 2, 16, 9, 10, 14, 8, 7}

1 3
1 1
2 9
6 0
1
8 7
4

K K Singh 1 12/15/2021
 Analyzing BuildHeap()

● Each call to Heapify() takes O(lg n) time

● There are O(n) such calls (specifically, ⎣n/2⎦)
● Thus the running time is O(n lg n)
● Is this a correct asymptotic upper bound?
● Is this an asymptotically tight bound?
● A tighter bound is O(n)
● How can this be? Is there a flaw in the above
reasoning?

K K Singh 1 12/15/2021
 Analyzing BuildHeap(): Tight

● To Heapify() a subtree takes O(h) time

where h is the height of the subtree
● h = O(lg m), m = # nodes in subtree
● The height of most subtrees is small
● Fact: an n-element heap has at most ⎡n/2h+1⎤
nodes of height h
● prove that BuildHeap() takes O(n) time

K K Singh 1 12/15/2021
 Heapsort

● Given BuildHeap(), an in-place sorting

algorithm is easily constructed:
● Maximum element is at A[1]
● Discard by swapping with element at A[n]
● Decrement heap_size[A]
● A[n] now contains correct value
● Restore heap property at A[1] by calling
Heapify()
● Repeat, always swapping A[1] for A[heap_size(A)]

K K Singh 1 12/15/2021
 Heapsort
Heapsort(A)
{
BuildHeap(A);
for (i = length(A) downto 2)
{
Swap(A[1], A[i]);
heap_size(A) -= 1;
Heapify(A, 1);
}
}

K K Singh 1 12/15/2021
 Analyzing Heapsort

● The call to BuildHeap() takes O(n) time

● Each of the n - 1 calls to Heapify() takes
O(lg n) time
● Thus the total time taken by HeapSort()
= O(n) + (n - 1) O(lg n)
= O(n) + O(n lg n)
= O(n lg n)

K K Singh 1 12/15/2021
 Priority Queues

● Heapsort is a nice algorithm, but in practice

Quicksort (coming up) usually wins
● But the heap data structure is incredibly useful
for implementing priority queues
● A data structure for maintaining a set S of
elements, each with an associated value or key
● Supports the operations Insert(),
Maximum(), and ExtractMax()
● What might a priority queue be useful for?

K K Singh 1 12/15/2021
 Priority Queue Operations

● Insert(S, x) inserts the element x into set S

● Maximum(S) returns the element of S with the
maximum key
● ExtractMax(S) removes and returns the
element of S with the maximum key
● We cam implement these operations using a
heap

K K Singh 1 12/15/2021
Any Query??