Design and Analysis of Algorithms

(DAA)
Heap Sort - Heap
• A heap is a specialized binary tree-based data structure that satisfies the
heap property.
• It can be used in algorithms like heap sort and is also the core component of
priority queues.
• A heap is particularly useful because it allows efficient access to the minimum
or maximum element in a set of data.
Heap Sort - Heap
• Key Properties of a Heap
1. Binary Tree:
• A heap is a complete binary tree. This means that all levels of the tree are fully
filled, except possibly for the last level, which is filled from left to right.
• In a binary tree, each node has at most two children: a left child and a right child.
2. Heap Property
• In a max-heap, for every node i, the value of i is greater than or equal to the values
of its children. This ensures that the maximum element is always at the root of the
heap.
• In a min-heap, for every node i, the value of i is less than or equal to the values of
its children. This ensures that the minimum element is always at the root of the
heap.

• Heap property examples:

•Max-Heap: A[parent(i)] ≥ A[i] for all nodes i
•Min-Heap: A[parent(i)] ≤ A[i] for all nodes i
Heap Sort Algorithm
• HEAP-SORT(A):
BUILD-MAX-HEAP(A)
for i = length[A] downto 2
exchange A[1] with A[i]
heap-size[A] = heap-size[A] – 1
MAX-HEAPIFY(A, 1)

Heap Sort Steps:

1. Build-Max-Heap-
• The first step is to transform the unsorted array into a max-heap.
• A max-heap is a binary tree where each parent node is greater than or equal to
its children.
• The array representation of the heap starts at index 0, with the left child of node i
at index 2i + 1 and the right child at 2i + 2.
• The BUILD-MAX-HEAP procedure works by calling the MAX-HEAPIFY function
from the last non-leaf node to the root node, ensuring that each subtree of the
heap maintains the max-heap property.
Heap Sort
MAX-HEAPIFY(A, i):
• This function assumes that the binary trees rooted at the left and right children of
i are max-heaps but that the value at A[i] may violate the heap property.
• To restore the max-heap property, the function compares the node with its
children and swaps it with the largest of its children if necessary.
• The function is then recursively called on the affected subtree to ensure the
entire structure is a max-heap.

BUILD-MAX-HEAP(A,n)
A.heap-size = n
for i = ⌊ n/2 ⌋ downto 1
MAX-HEAPIFY (A, i )
Heap Sort
MAX-HEAPIFY(A, i):
l = LEFT(i)
r = RIGHT(i)
if l <= heap-size[A] and A[l] > A[i]:
largest = l
else:
largest = i
if r <= heap-size[A] and A[r] > A[largest]:
largest = r
if largest != i:
exchange A[i] with A[largest]
MAX-HEAPIFY(A, largest)
Heap Sort - Steps

1. After building the max-heap, swap the root (the largest element) with the
last element in the array.
2. Reduce the size of the heap by one since the last element is now in its
correct position.
3. Call Max-Heapify to restore the heap property.
4. Repeat this process until the heap size is reduced to 1.
Example
A = [4, 10, 3, 5, 1]
A = [nil, 4, 10, 3, 5, 1] (nil is a placeholder for index 0)
Step 1: BUILD-MAX-HEAP(A)
In 1-based indexing, A.length = 5, so we start at ⌊n/2⌋ = 2 (the index of the last
non-leaf node).
1.1 Call MAX-HEAPIFY(A, 2)
•At index 2, the value is 10.
•Left child is at index 2 * 2 = 4 (A[4] = 5).
•Right child is at index 2 * 2 + 1 = 5 (A[5] = 1).
•The value 10 is greater than both children, so no changes are made.
Array after MAX-HEAPIFY(2):
A = [nil, 4, 10, 3, 5, 1]
Example
1.2 Call MAX-HEAPIFY(A, 1)
•At index 1, the value is 4.
•Left child is at index 2 * 1 = 2 (A[2] = 10).
•Right child is at index 2 * 1 + 1 = 3 (A[3] = 3).
•The left child (10) is larger than 4, so we swap A[1] with A[2].
After the swap:
•A = [nil, 10, 4, 3, 5, 1]
•Now, we need to call MAX-HEAPIFY(A, 2) to restore the heap property at index 2.
1.3 Call MAX-HEAPIFY(A, 2) (again after the swap)
•At index 2, the value is 4.
•Left child is at index 4 (A[4] = 5).
•Right child is at index 5 (A[5] = 1).
•The left child (5) is larger than 4, so we swap A[2] with A[4].
After the swap:
A = [nil, 10, 5, 3, 4, 1]
Example
Now, A[4] has no children, so no further MAX-HEAPIFY calls are needed.
At this point, we have successfully built the max-heap.
Max-Heap after BUILD-MAX-HEAP(A):
A = [nil, 10, 5, 3, 4, 1]
Step 2: HEAPSORT(A)
We now proceed with sorting the array by repeatedly extracting the maximum
element (root of the heap)and moving it to the end of the array,
while maintaining the heap structure.
2.1: Swap A[1] with A[5]
•Swap the root (10) with the last element in the heap (1).

After the swap: A = [nil, 1, 5, 3, 4, 10]

Example

Now, reduce the heap size (A.heap_size = 4) and call MAX-HEAPIFY(A, 1)

to restore the heap property.
2.2: Call MAX-HEAPIFY(A, 1)
•At index 1, the value is 1.
•Left child is at index 2 (A[2] = 5).
•Right child is at index 3 (A[3] = 3).
•The left child (5) is larger than 1, so we swap A[1] with A[2].

After the swap:

A = [nil, 5, 1, 3, 4, 10]
Now, call MAX-HEAPIFY(A, 2) to restore the heap property at index 2.
Example
2.3 Call MAX-HEAPIFY(A, 2)
•At index 2, the value is 1.
•Left child is at index 4 (A[4] = 4).
•The left child (4) is larger than 1, so we swap A[2] with A[4].
After the swap:
A = [nil, 5, 4, 3, 1, 10]
A[4] has no children, so no further heapification is needed.

2.4: Swap A[1] with A[4]

•Swap the root (5) with the last element in the heap (1).
After the swap:
A = [nil, 1, 4, 3, 5, 10]
Reduce the heap size (A.heap_size = 3) and call MAX-HEAPIFY(A, 1).
Example
2.5: Call MAX-HEAPIFY(A, 1)
•At index 1, the value is 1.
•Left child is at index 2 (A[2] = 4).
•Right child is at index 3 (A[3] = 3).
•The left child (4) is larger than 1, so we swap A[1] with A[2].
After the swap:
A = [nil, 4, 1, 3, 5, 10]
No need to call MAX-HEAPIFY further because A[2] has no children.
2.6: Swap A[1] with A[3]
•Swap the root (4) with the last element in the heap (3).
•After the swap:
•A = [nil, 3, 1, 4, 5, 10]
Reduce the heap size (A.heap_size = 2) and call MAX-HEAPIFY(A, 1).
Example
2.7: Call MAX-HEAPIFY(A, 1)
•At index 1, the value is 3.
•Left child is at index 2 (A[2] = 1).
•Since 3 is greater than its child, no swaps are needed.
Array remains: A = [nil, 3, 1, 4, 5, 10]
2.8: Swap A[1] with A[2]
•Swap the root (3) with the last element in the heap (1).
•After the swap:
•A = [nil, 1, 3, 4, 5, 10]

Now the heap has size 1, so sorting is complete.

Final Sorted Array:
Removing the nil placeholder, the final sorted array is:
Sorted Array: [1, 3, 4, 5, 10]
Shell Sort
The Shell Sort algorithm is an extension to the insertion sort method, where it
sorts subsets of elements that are distant from each other rather than adjacent.
Here, an array is made h-sorted for a large value of h.
The value of h is kept reducing until it becomes 1.

This algorithm uses insertion sort on a widely spread elements, first to sort them
and then sorts the less widely spaced elements.
This spacing is termed as interval.

Algorithm
shellSort(array, size)
for interval i <- size/2n down to 1
for each interval "i" in array sort all the elements at interval "i"
end shellSort
Shell Sort
The performance of the shell sort depends on the type of sequence used for a given
input array.
Some of the optimal sequences that can be used in the shell sort algorithm are:
• Shell's original sequence: N/2 , N/4 , …, 1
• Knuth's increments: 1, 4, 13, …, (3k – 1) / 2
• Sedgewick's increments: 1, 8, 23, 77, 281, 1073, 4193, 16577...4j+1+ 3·2j+ 1
• Hibbard's increments: 1, 3, 7, 15, 31, 63, 127, 255, 511…
• Papernov & Stasevich increment: 1, 3, 5, 9, 17, 33, 65,...
• Pratt: 1, 2, 3, 4, 6, 9, 8, 12, 18, 27, 16, 24, 36, 54, 81....
Shell Sort Algorithm - Example
A = [22, 7, 2, 17, 3, 5, 10, 15]

We will sort the array using the gap sequence [4, 2, 1].

Step 1: Gap = 4
In this step, we consider elements that are 4 positions apart.
1.Compare A[0] = 22 with A[4] = 3. Since 3 < 22, swap them.

Array after swap:

A = [3, 7, 2, 17, 22, 5, 10, 15]
2. Compare A[1] = 7 with A[5] = 5. Since 5 < 7, swap them.
Array after swap:
A = [3, 5, 2, 17, 22, 7, 10, 15]

3. Compare A[2] = 2 with A[6] = 10. No swap is needed, as 2 < 10.

4. Compare A[3] = 17 with A[7] = 15. Since 15 < 17, swap them.
Array after swap:
A = [3, 5, 2, 15, 22, 7, 10, 17]
Shell Sort Algorithm - Example
After completing the gap = 4 phase, the array looks like:
A = [3, 5, 2, 15, 22, 7, 10, 17]

Step 2: Gap = 2
Now we reduce the gap to 2. We compare elements that are 2 positions apart.
1.Compare A[0] = 3 with A[2] = 2. Since 2 < 3, swap them.
•Array after swap:

A = [2, 5, 3, 15, 22, 7, 10, 17]

2. Compare A[1] = 5 with A[3] = 15. No swap needed, as 5 < 15.
3. Compare A[2] = 3 with A[4] = 22. No swap needed, as 3 < 22.
4. Compare A[3] = 15 with A[5] = 7. Since 7 < 15, swap them.
Array after swap:
A = [2, 5, 3, 7, 22, 15, 10, 17]
5. Compare A[4] = 22 with A[6] = 10. Since 10 < 22, swap them.
Shell Sort Algorithm - Example
Array after swap
A = [2, 5, 3, 7, 10, 15, 22, 17]
Compare A[5] = 15 with A[7] = 17. No swap is needed, as 15 < 17.
After completing the gap = 2 phase, the array looks like:
A = [2, 5, 3, 7, 10, 15, 22, 17]

Step 3: Gap = 1 (Insertion Sort Phase)

Finally, we reduce the gap to 1. Now the array will be sorted using insertion sort,
where adjacent elements are compared and swapped if necessary.
1.Compare A[1] = 5 with A[0] = 2. No swap needed.
2.Compare A[2] = 3 with A[1] = 5. Since 3 < 5, swap them.
Array after swap:
A = [2, 3, 5, 7, 10, 15, 22, 17]
Shell Sort Algorithm - Example
3. Compare A[3] = 7 with A[2] = 5. No swap needed.
4. Compare A[4] = 10 with A[3] = 7. No swap needed.
5. Compare A[5] = 15 with A[4] = 10. No swap needed.
6. Compare A[6] = 22 with A[5] = 15. No swap needed.
7. Compare A[7] = 17 with A[6] = 22. Since 17 < 22, swap them.
Array after swap:

A = [2, 3, 5, 7, 10, 15, 17, 22]

After completing the gap = 1 phase, the array is now sorted:
Final Sorted Array:
A = [2, 3, 5, 7, 10, 15, 17, 22]
Linear Time Sorting
Counting Sort

• Counting Sort is a non-comparison-based sorting algorithm.

• It is particularly efficient when the range of input values is small compared to the
number of elements to be sorted.
• The basic idea behind Counting Sort is to count the frequency of each distinct
element in the input array and use that information to place the elements in their
correct sorted positions.
Linear Time Sorting
Counting Sort

COUNTING-SORT(A, B, k) k is the maximum value of the array

let C[0..k] be a new array
for i = 0 to k
C[i] = 0
for j = 1 to length[A]
C[A[j]] = C[A[j]] + 1
for i = 1 to k
C[i] = C[i] + C[i - 1]
for j = length[A] downto 1
B[C[A[j]]] = A[j]
C[A[j]] = C[A[j]] – 1
Return B
Linear Time Sorting
Counting Sort - Example

A = [2, 5, 3, 0, 2, 3, 0, 3]
We are given the array A. The elements in the array range from 0 to 5, so k = 5
(maximum element is 5).
Step 1: Initialize the Count Array
• We create a count array C of size k + 1 = 6, initialized to zeros. This array will
keep track of the frequency of each element in the input array.
• C = [0, 0, 0, 0, 0, 0]
Linear Time Sorting
Counting Sort - Example
Step 2: Count the Frequencies
• Now, we iterate through the array A and increment the value of C[A[i]] for each
element in A.

• For each element in A:

• A[0] = 2 → increment C[2], so C = [0, 0, 1, 0, 0, 0]
• A[1] = 5 → increment C[5], so C = [0, 0, 1, 0, 0, 1]
• A[2] = 3 → increment C[3], so C = [0, 0, 1, 1, 0, 1]
• A[3] = 0 → increment C[0], so C = [1, 0, 1, 1, 0, 1]
• A[4] = 2 → increment C[2], so C = [1, 0, 2, 1, 0, 1]
• A[5] = 3 → increment C[3], so C = [1, 0, 2, 2, 0, 1]
• A[6] = 0 → increment C[0], so C = [2, 0, 2, 2, 0, 1]
• A[7] = 3 → increment C[3], so C = [2, 0, 2, 3, 0, 1]
Linear Time Sorting
Counting Sort - Example
Step 2: Count the Frequencies
After this step, C[i] contains the number of occurrences of each element i in A.

C = [2, 0, 2, 3, 0, 1]
• Step 3: Modify the Count Array
• We now modify the count array C so that each element C[i] contains the
cumulative count of elements less than or equal to i. This will give us the position
of each element in the sorted array.
• For each element in C:
• C[1] = C[1] + C[0] → C = [2, 2, 2, 3, 0, 1]
• C[2] = C[2] + C[1] → C = [2, 2, 4, 3, 0, 1]
• C[3] = C[3] + C[2] → C = [2, 2, 4, 7, 0, 1]
• C[4] = C[4] + C[3] → C = [2, 2, 4, 7, 7, 1]
• C[5] = C[5] + C[4] → C = [2, 2, 4, 7, 7, 8]
Linear Time Sorting
Counting Sort - Example
Now, C[i] tells us how many elements are less than or equal to i.
C = [2, 2, 4, 7, 7, 8]
• Step 4: Build the Output Array
• Create an output array B of size n = 8, initially empty.
• We now iterate through the input array A in reverse order (to maintain stability)
and place each element in its correct position in B, using the count array C.
• After placing each element, decrement its corresponding value in C.
For each element in A (iterate from right to left):
1. A[7] = 3 → C[3] = 7 → place 3 at index 6 in B, decrement C[3]
B = [_, _, _, _, _, _, 3, _], C = [2, 2, 4, 6, 7, 8]
2. A[6] = 0 → C[0] = 2 → place 0 at index 1 in B, decrement C[0]:
B = [_, 0, _, _, _, _, 3, _], C = [1, 2, 4, 6, 7, 8]
Linear Time Sorting
Counting Sort - Example
3. A[5] = 3 → C[3] = 6 → place 3 at index 5 in B, decrement C[3]:
B = [_, 0, _, _, _, 3, 3, _], C = [1, 2, 4, 5, 7, 8]
4. A[4] = 2 → C[2] = 4 → place 2 at index 3 in B, decrement C[2]:
B = [_, 0, _, 2, _, 3, 3, _], C = [1, 2, 3, 5, 7, 8]
5. A[3] = 0 → C[0] = 1 → place 0 at index 0 in B, decrement C[0]:
B = [0, 0, _, 2, _, 3, 3, _], C = [0, 2, 3, 5, 7, 8]
6. A[2] = 3 → C[3] = 5 → place 3 at index 4 in B, decrement C[3]:
B = [0, 0, _, 2, 3, 3, 3, _], C = [0, 2, 3, 4, 7, 8]
7. A[1] = 5 → C[5] = 8 → place 5 at index 7 in B, decrement C[5]:
B = [0, 0, _, 2, 3, 3, 3, 5], C = [0, 2, 3, 4, 7, 7]
8. A[0] = 2 → C[2] = 3 → place 2 at index 2 in B, decrement C[2]:
• Step 5: Output
• After processing all the elements, the final sorted array is:
B = [0, 0, 2, 2, 3, 3, 3, 5]
Linear Time Sorting
Counting Sort - Example
Radix Sort
• Radix sort is the linear sorting algorithm that is used for integers.
• In Radix sort, there is digit by digit sorting is performed that is started from the
least significant digit to the most significant digit.
• Radix Sort can be performed using different variations, such as Least Significant
Digit (LSD) Radix Sort or Most Significant Digit (MSD) Radix Sort.

Algorithm
radixSort(arr)
max = largest element in the given array
d = number of digits in the largest element (or, max)
Now, create d buckets of size 0 - 9
for i -> 0 to d
sort the array elements using counting sort (or any stable sort) according to the digi
ts at the ith place
Radix Sort
Steps:
• Choose a Radix (Base):
• The radix is the number of unique digits (or values) a number can take. For
base 10 numbers (decimal system), the radix is 10 because the digits range
from 0 to 9.
• Start with the Least Significant Digit (LSD):
• Group or sort the numbers based on their least significant digit. Numbers are
placed in "buckets" according to the value of this digit.
• Move to the Next Significant Digit:
• After sorting by the least significant digit, proceed to the next more significant
digit (ones → tens → hundreds → etc.). Re-arrange numbers within the
previous grouping, now considering the current digit.
• Repeat Until All Digits Are Processed:
• Continue the process until you’ve sorted using the most significant digit. At this
point, the list is sorted.
Radix Sort

Here, we start from comparing of least significant digit to most significant digit one
bye one.
Bucket Sort
• Bucket Sort is an algorithm that sorts elements by distributing them into several
"buckets" and then sorting the individual buckets.
• Bucket sort is a sorting technique that involves dividing elements into various
groups, or buckets.
• These buckets are formed by uniformly distributing the elements.
• Once the elements are divided into buckets, they can be sorted using any other
sorting algorithm.
• Finally, the sorted elements are gathered together in an ordered fashion.
Bucket Sort
BUCKET-SORT(A)
1. n = length[A]
2. let B[0..n−1] be a new array of empty lists
3. for i = 1 to n
4. do insert A[i] into list B[⌊n * A[i]⌋]
5. for i = 0 to n − 1
6. do sort list B[i] with insertion sort
7. concatenate the lists B[0], B[1], ..., B[n − 1] together in order
8. return the concatenated list
Bucket Sort
Example
A = [0.78, 0.17, 0.39, 0.72, 0.94, 0.21, 0.12, 0.23, 0.68, 0.84].
Step 1: Initialization: Create 10 empty buckets: B[0], B[1], ..., B[9].
Step 2: Distribute Elements: Using the formula floor(n * A[i]), where n = 10, we
distribute elements into the corresponding buckets:
• 0.78 goes to bucket B[7].
• 0.17 goes to bucket B[1].
• 0.39 goes to bucket B[3].
• 0.72 goes to bucket B[7].
• 0.94 goes to bucket B[9].
• 0.21 goes to bucket B[2].
• 0.12 goes to bucket B[1].
• 0.23 goes to bucket B[2].
• 0.68 goes to bucket B[6].
• 0.84 goes to bucket B[8].
Bucket Sort
Step 3: Sort Each Bucket: After distributing the elements, sort each bucket
individually:
•Bucket B[0]: [ ]
•Bucket B[1]: [0.12, 0.17]
•Bucket B[2]: [0.21, 0.23]
•Bucket B[3]: [0.39]
•Bucket B[6]: [0.68]
•Bucket B[7]: [0.72, 0.78]
•Bucket B[8]: [0.84]
•Bucket B[9]: [0.94]

Step 4: Concatenate Buckets: Finally, concatenate the buckets: [0.12, 0.17, 0.21,
0.23, 0.39, 0.68, 0.72, 0.78, 0.84, 0.94].
Bucket Sort