Design and Analysis of Algorithms

(BTCO13403)

Chapter 1
INTRODUCTION TO ALGORITHM
AND ANALYSIS

Lecture Notes by,

Prof. Bhumika Patel,
IT Department,
SCET, Surat

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA
 Reference Books
1. Introduction to Algorithms, Thomas H. Cormen, Charles E.
Leiserson, Ronald L. Rivest and Clifford Stein, PHI.

2. Fundamental of Algorithms by Gills Brassard, Paul Bratley, PHI.

3. Fundamentals of Computer Algorithms by Ellis Horowitz, Sartaj

Sahni, Pearson

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 2
 Algorithm

•Definition 1: An algorithm is any well-defined computational

procedure that takes some value, or set of values as input and
produces some value, or set of values as output.

•Definition 2: An algorithm is a set of rules for carrying out

some calculation, either by hand or, more usually on a
machine.

•Definition 3: A well-specified set of instructions to be

followed in order to solve a problem.

•The study of algorithms is called “Algorithmics”.

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 3
 Algorithm
• Example: To sort a sequence of numbers into non-decreasing order.

• Input: A sequence of n numbers <a1, a2, ….,an>.

• Output: A permutation (reordering) <a’1, a’2, ….,a’n> of the input
sequence such that a’1 <= a’2 <=…. <= a’n.

• Input sequence: <31, 41, 59, 26, 41, 58>

• A sorting algorithm returns as output the sequence <26, 31, 41, 41, 58,
59>.
• Such an input sequence is called an instance of the sorting problem.

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 4
 Algorithmic Approaches
• Divide and Conquer

• Greedy approach

• Dynamic Programming

• Backtracking

• Branch and Bound

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 5
 Sorting Algorithms
• Bubble sort
• Selection sort
• Insertion sort
• Heap sort

• Sorting in linear time

▫ Bucket sort
▫ Radix sort and
▫ Counting sort

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 6
 Sorting
• Sorting takes an unordered collection and makes it an
ordered one.

1 2 3 4 5 6

77 42 35 12 101 5

1 2 3 4 5 6

5 12 35 42 77 101

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 7
 BUBBLE SORT

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA
 Bubble Sort
• “Bubbling up” the Largest element

• Traverse a collection of elements

▫ Move from the front to the end
▫ “Bubble” the largest value to the end using pair-wise comparisons
and swapping

1 2 3 4 5 6

77 42 35 12 101 5

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 9
 "Bubbling Up" the Largest Element

• Traverse a collection of elements

▫ Move from the front to the end
▫ “Bubble” the largest value to the end using pair-wise comparisons
and swapping

1 2 3 4 5 6

42 Swap4277
77 35 12 101 5

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 10
 "Bubbling Up" the Largest Element

• Traverse a collection of elements

▫ Move from the front to the end
▫ “Bubble” the largest value to the end using pair-wise comparisons
and swapping

1 2 3 4 5 6

42 7735 Swap35
77 12 101 5

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 11
 "Bubbling Up" the Largest Element

• Traverse a collection of elements

▫ Move from the front to the end
▫ “Bubble” the largest value to the end using pair-wise comparisons
and swapping

1 2 3 4 5 6

42 35 7712 Swap 12
77 101 5

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 12
 "Bubbling Up" the Largest Element

• Traverse a collection of elements

▫ Move from the front to the end
▫ “Bubble” the largest value to the end using pair-wise comparisons
and swapping

1 2 3 4 5 6

42 35 12 77 101 5

No need to swap

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 13
 "Bubbling Up" the Largest Element

• Traverse a collection of elements

▫ Move from the front to the end
▫ “Bubble” the largest value to the end using pair-wise comparisons
and swapping

1 2 3 4 5 6

42 35 12 77 1015 Swap 101

5

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 14
 "Bubbling Up" the Largest Element

• Traverse a collection of elements

▫ Move from the front to the end
▫ “Bubble” the largest value to the end using pair-wise comparisons
and swapping

1 2 3 4 5 6

42 35 12 77 5 101

Largest value correctly placed

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 15
 Bubble Sort Algorithm
index <- 1
last_compare_at <- n – 1

loop
exitif(index > last_compare_at)
if(A[index] > A[index + 1]) then
Swap(A[index], A[index + 1])
endif
index <- index + 1
endloop

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 16
 Swap isn’t built in

Procedure Swap(a, b isoftype Num)

t isoftype Num
t <- a
a <- b
b <- t
endprocedure // Swap

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 17
 Items of Interest

• Notice that only the largest value is correctly placed

• All other values are still out of order
• So we need to repeat this process

1 2 3 4 5 6

42 35 12 77 5 101

Largest value correctly placed

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 18
 Repeat “Bubble Up” How Many Times?

• If we have N elements…

• And if each time we bubble an element, we place it in

its correct location…

• Then we repeat the “bubble up” process N – 1 times.

• This guarantees we’ll correctly place all N elements.

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 19
 “Bubbling” All the Elements
1 2 3 4 5 6
42 35 12 77 5 101

1 2 3 4 5 6
35 12 42 5 77 101

1 2 3 4 5 6
N-1

12 35 5 42 77 101

1 2 3 4 5 6
12 5 35 42 77 101

1 2 3 4 5 6
5 12 35 42 77 101

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 20
 Reducing the Number of Comparisons
• On the Nth “bubble up”, we only need to do MAX-N comparisons.

• For example:
▫ This is the 4th “bubble up”
▫ MAX is 6
▫ Thus we have 6-4= 2 comparisons to do

1 2 3 4 5 6
12 35 5 42 77 101

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 21
 Reducing the Number of Comparisons
1 2 3 4 5 6
77 42 35 12 101 5

1 2 3 4 5 6
42 35 12 77 5 101

1 2 3 4 5 6
35 12 42 5 77 101

1 2 3 4 5 6
12 35 5 42 77 101

1 2 3 4 5 6
12 5 35 42 77 101

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 22
 procedure Bubblesort(A isoftype Arr_Type)
to_do, index isoftype Num
to_do <- N – 1

loop
exitif(to_do = 0)
index <- 1
loop
exitif(index > to_do)

Inner loop

Outer loop
if(A[index] > A[index + 1]) then
Swap(A[index], A[index + 1])
endif
index <- index + 1
endloop
to_do <- to_do - 1
endloop
endprocedure // Bubblesort

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 23
 N is … // Size of Array

Arr_Type defines Array[1..N] of Num

Procedure Swap(n1, n2 isoftype Num)

temp isoftype Num
temp <- n1
n1 <- n2
n2 <- temp
endprocedure // Swap

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 24
 Already Sorted Collections?
• What if the collection was already sorted?
• What if only a few elements were out of place and after a couple of
“bubble ups,” the collection was sorted?

• We want to be able to detect this and “stop early”!

1 2 3 4 5 6
5 12 35 42 77 101

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 25
 Using a Boolean “Flag”

• We can use a boolean variable to determine if any swapping occurred

during the “bubble up.”

• If no swapping occurred, then we know that the collection is already

sorted!

• This boolean “flag” needs to be reset after each “bubble up.”

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 26
 did_swap isoftype Boolean
did_swap <- true

loop
exitif ((to_do = 0) OR NOT(did_swap))
index <- 1
did_swap <- false
loop
exitif(index > to_do)
if(A[index] > A[index + 1]) then
Swap(A[index], A[index + 1])
did_swap <- true
endif
index <- index + 1
endloop
to_do <- to_do - 1
endloop

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 27
 An Animated Example

N 8 did_swap true

to_do 7

index

98 23 45 14 6 67 33 42

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 28
 An Animated Example
N 8 did_swap false

to_do 7

index 1

98 23 45 14 6 67 33 42

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 29
 An Animated Example

N 8 did_swap false

to_do 7

index 1

Swap

98 23 45 14 6 67 33 42

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 30
 An Animated Example

N 8 did_swap true

to_do 7

index 1

Swap

23 98 45 14 6 67 33 42

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 31
 An Animated Example
N 8 did_swap true

to_do 7

index 2

23 98 45 14 6 67 33 42

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 32
 An Animated Example

N 8 did_swap true

to_do 7

index 2

Swap

23 98 45 14 6 67 33 42

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 33
 An Animated Example

N 8 did_swap true

to_do 7

index 2

Swap

23 45 98 14 6 67 33 42

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 34
 An Animated Example

N 8 did_swap true

to_do 7

index 3

23 45 98 14 6 67 33 42

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 35
 An Animated Example

N 8 did_swap true

to_do 7

index 3

Swap

23 45 98 14 6 67 33 42

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 36
 An Animated Example

N 8 did_swap true

to_do 7

index 3

Swap

23 45 14 98 6 67 33 42

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 37
 An Animated Example

N 8 did_swap true

to_do 7

index 4

23 45 14 98 6 67 33 42

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 38
 An Animated Example

N 8 did_swap true

to_do 7

index 4

Swap

23 45 14 98 6 67 33 42

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 39
 An Animated Example

N 8 did_swap true

to_do 7

index 4

Swap

23 45 14 6 98 67 33 42

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 40
 An Animated Example

N 8 did_swap true

to_do 7

index 5

23 45 14 6 98 67 33 42

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 41
 An Animated Example

N 8 did_swap true

to_do 7

index 5

Swap

23 45 14 6 98 67 33 42

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 42
 An Animated Example

N 8 did_swap true

to_do 7

index 5

Swap

23 45 14 6 67 98 33 42

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 43
 An Animated Example

N 8 did_swap true

to_do 7

index 6

23 45 14 6 67 98 33 42

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 44
 An Animated Example

N 8 did_swap true

to_do 7

index 6

Swap

23 45 14 6 67 98 33 42

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 45
 An Animated Example

N 8 did_swap true

to_do 7

index 6

Swap

23 45 14 6 67 33 98 42

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 46
 An Animated Example

N 8 did_swap true

to_do 7

index 7

23 45 14 6 67 33 98 42

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 47
 An Animated Example

N 8 did_swap true

to_do 7

index 7

Swap

23 45 14 6 67 33 98 42

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 48
 An Animated Example

N 8 did_swap true

to_do 7

index 7

Swap

23 45 14 6 67 33 42 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 49
 After First Pass of Outer Loop

N 8 did_swap true

to_do 7

index 8 Finished first “Bubble Up”

23 45 14 6 67 33 42 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 50
 The Second “Bubble Up”

N 8 did_swap false

to_do 6

index 1

23 45 14 6 67 33 42 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 51
 The Second “Bubble Up”

N 8 did_swap false

to_do 6

index 1

No Swap

23 45 14 6 67 33 42 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 52
 The Second “Bubble Up”

N 8 did_swap false

to_do 6

index 2

23 45 14 6 67 33 42 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 53
 The Second “Bubble Up”

N 8 did_swap false

to_do 6

index 2

Swap

23 45 14 6 67 33 42 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 54
 The Second “Bubble Up”

N 8 did_swap true

to_do 6

index 2

Swap

23 14 45 6 67 33 42 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 55
 The Second “Bubble Up”

N 8 did_swap true

to_do 6

index 3

23 14 45 6 67 33 42 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 56
 The Second “Bubble Up”

N 8 did_swap true

to_do 6

index 3

Swap

23 14 45 6 67 33 42 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 57
 The Second “Bubble Up”

N 8 did_swap true

to_do 6

index 3

Swap

23 14 6 45 67 33 42 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 58
 The Second “Bubble Up”

N 8 did_swap true

to_do 6

index 4

23 14 6 45 67 33 42 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 59
 The Second “Bubble Up”

N 8 did_swap true

to_do 6

index 4

No Swap

23 14 6 45 67 33 42 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 60
 The Second “Bubble Up”

N 8 did_swap true

to_do 6

index 5

23 14 6 45 67 33 42 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 61
 The Second “Bubble Up”

N 8 did_swap true

to_do 6

index 5

Swap

23 14 6 45 67 33 42 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 62
 The Second “Bubble Up”

N 8 did_swap true

to_do 6

index 5

Swap

23 14 6 45 33 67 42 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 63
 The Second “Bubble Up”

N 8 did_swap true

to_do 6

index 6

23 14 6 45 33 67 42 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 64
 The Second “Bubble Up”

N 8 did_swap true

to_do 6

index 6

Swap

23 14 6 45 33 67 42 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 65
 The Second “Bubble Up”

N 8 did_swap true

to_do 6

index 6

Swap

23 14 6 45 33 42 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 66
 After Second Pass of Outer Loop

N 8 did_swap true

to_do 6

index 7 Finished second “Bubble Up”

23 14 6 45 33 42 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 67
 The Third “Bubble Up”

N 8 did_swap false

to_do 5

index 1

23 14 6 45 33 42 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 68
 The Third “Bubble Up”

N 8 did_swap false

to_do 5

index 1

Swap

23 14 6 45 33 42 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 69
 The Third “Bubble Up”

N 8 did_swap true

to_do 5

index 1

Swap

14 23 6 45 33 42 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 70
 The Third “Bubble Up”

N 8 did_swap true

to_do 5

index 2

14 23 6 45 33 42 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 71
 The Third “Bubble Up”

N 8 did_swap true

to_do 5

index 2

Swap

14 23 6 45 33 42 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 72
 The Third “Bubble Up”

N 8 did_swap true

to_do 5

index 2

Swap

14 6 23 45 33 42 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 73
 The Third “Bubble Up”

N 8 did_swap true

to_do 5

index 3

14 6 23 45 33 42 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 74
 The Third “Bubble Up”

N 8 did_swap true

to_do 5

index 3

No Swap

14 6 23 45 33 42 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 75
 The Third “Bubble Up”

N 8 did_swap true

to_do 5

index 4

14 6 23 45 33 42 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 76
 The Third “Bubble Up”

N 8 did_swap true

to_do 5

index 4

Swap

14 6 23 45 33 42 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 77
 The Third “Bubble Up”

N 8 did_swap true

to_do 5

index 4

Swap

14 6 23 33 45 42 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 78
 The Third “Bubble Up”

N 8 did_swap true

to_do 5

index 5

14 6 23 33 45 42 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 79
 The Third “Bubble Up”

N 8 did_swap true

to_do 5

index 5

Swap

14 6 23 33 45 42 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 80
 The Third “Bubble Up”

N 8 did_swap true

to_do 5

index 5

Swap

14 6 23 33 42 45 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 81
 After Third Pass of Outer Loop

N 8 did_swap true

to_do 5

index 6 Finished third “Bubble Up”

14 6 23 33 42 45 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 82
 The Fourth “Bubble Up”

N 8 did_swap false

to_do 4

index 1

14 6 23 33 42 45 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 83
 The Fourth “Bubble Up”

N 8 did_swap false

to_do 4

index 1

Swap

14 6 23 33 42 45 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 84
 The Fourth “Bubble Up”

N 8 did_swap true

to_do 4

index 1

Swap

6 14 23 33 42 45 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 85
 The Fourth “Bubble Up”

N 8 did_swap true

to_do 4

index 2

6 14 23 33 42 45 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 86
 The Fourth “Bubble Up”

N 8 did_swap true

to_do 4

index 2

No Swap

6 14 23 33 42 45 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 87
 The Fourth “Bubble Up”

N 8 did_swap true

to_do 4

index 3

6 14 23 33 42 45 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 88
 The Fourth “Bubble Up”

N 8 did_swap true

to_do 4

index 3

No Swap

6 14 23 33 42 45 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 89
 The Fourth “Bubble Up”

N 8 did_swap true

to_do 4

index 4

6 14 23 33 42 45 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 90
 The Fourth “Bubble Up”

N 8 did_swap true

to_do 4

index 4

No Swap

6 14 23 33 42 45 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 91
 After Fourth Pass of Outer Loop

N 8 did_swap true

to_do 4

index 5 Finished fourth “Bubble Up”

6 14 23 33 42 45 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 92
 The Fifth “Bubble Up”

N 8 did_swap false

to_do 3

index 1

6 14 23 33 42 45 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 93
 The Fifth “Bubble Up”

N 8 did_swap false

to_do 3

index 1

No Swap

6 14 23 33 42 45 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 94
 The Fifth “Bubble Up”

N 8 did_swap false

to_do 3

index 2

6 14 23 33 42 45 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 95
 The Fifth “Bubble Up”

N 8 did_swap false

to_do 3

index 2

No Swap

6 14 23 33 42 45 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 96
 The Fifth “Bubble Up”

N 8 did_swap false

to_do 3

index 3

6 14 23 33 42 45 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 97
 The Fifth “Bubble Up”

N 8 did_swap false

to_do 3

index 3

No Swap

6 14 23 33 42 45 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 98
 After Fifth Pass of Outer Loop

N 8 did_swap false

to_do 3

index 4 Finished fifth “Bubble Up”

6 14 23 33 42 45 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 99
 Finished “Early”

N 8 did_swap false

to_do 3
We didn’t do any swapping,
index 4 so all of the other elements
must be correctly placed.

We can “skip” the last two

passes of the outer loop.

6 14 23 33 42 45 67 98

1 2 3 4 5 6 7 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 100
 Summary
• Bubble Sort algorithm will move largest value to its correct
location (to the right)

• Repeat “Bubble Up” until all elements are correctly placed:

▫ Maximum of N-1 times
▫ Can finish early if no swapping occurs

• Nobody ever uses bubble sort because it is extremely inefficient.

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 101
 Bubble Sort Analysis
• Bubble sort has worst-case and average complexity both О(n2),
where n is the number of items being sorted.

• When the list is already sorted (best-case), the complexity of

bubble sort is only O(n).

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 102
 SELECTION SORT

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA
 Selection Sort
• Find the smallest element in the array.

• Exchange it with the element in the first position.

• Find the second smallest element and exchange it with

the element in the second position.

• Continue until the array is sorted.

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 104
 Example

8 4 6 9 2 3 1 1 2 3 4 9 6 8

1 4 6 9 2 3 8 1 2 3 4 6 9 8

1 2 6 9 4 3 8 1 2 3 4 6 8 9

1 2 3 9 4 6 8 1 2 3 4 6 8 9

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 105
 Selection Sort 8 4 6 9 2 3 1

SELECTION-SORT(A)
*Index of smallest value is stored in
n ← length[A] variable smallest
for j ← 1 to n - 1
do smallest ← j
for i ← j + 1 to n
do if A[i] < A[smallest]
then smallest ← i
end for
exchange A[j] ↔ A[smallest]
end for

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 106
 Selection Sort Tracing 8 4 6 9 2 3 1

• Iteration 1:
▫ j=1
▫ Smallest=1
▫ for i=2 to 7
 i=2 -> if(A[2] < A[1]) Yes
 Smallest=2
 i=3 -> if(A[3] < A[2]) No
 i=4 -> if(A[4] < A[2]) No
 i=5 -> if(A[5] < A[2]) Yes
 Smallest=5
 i=6 -> if(A[6] < A[5]) No
 i=7 -> if(A[7] < A[5]) Yes
 Smallest=7
▫ Exchange A[1] and A[7]
1 4 6 9 2 3 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 107
 Selection Sort Tracing 1 4 6 9 2 3 8

• Iteration 2:
▫ j=2
▫ Smallest=2
▫ for i=3 to 7
 i=3 -> if(A[3] < A[2]) No
 i=4 -> if(A[4] < A[2]) No
 i=5 -> if(A[5] < A[2]) Yes
 Smallest=5
 i=6 -> if(A[6] < A[5]) No
 i=7 -> if(A[7] < A[5]) No
▫ Exchange A[2] and A[5]
1 2 6 9 4 3 8

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 108
 Selection Sort Analysis
• Selecting the lowest element requires scanning
all n elements (this takes n − 1 comparisons) and then
swapping it into the first position.

• Finding the next lowest element requires scanning the

remaining n − 1 elements and so on, for
(n − 1) + (n − 2) + ... + 2 + 1 = n(n - 1)/2 ∈ Θ(n2)
comparisons.

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 109
 INSERTION SORT

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA
 Insertion Sort
• Idea is like sorting a hand of playing cards

• Remove one card at a time from the table, and insert it

into the correct position in the left hand
▫ Compare it with each of the cards already in the hand, from
right to left

• The cards held in the left hand are sorted

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 111
 Insertion Sort

To insert 12, we need to make room

for it by moving first 36 and then 24.

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 112
 Insertion Sort

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 113
 Insertion Sort

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 114
 Insertion Sort

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 115
 Insertion Sort
input array

5 2 4 6 1 3

At each iteration, the array is divided in two sub-arrays:

left sub-array right sub-array

sorted unsorted

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 116
 Insertion Sort

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 117
 INSERTION SORT
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j] //shift one position right
j = j - 1
} *First element of right sub array is a key

A[j+1] = key
}
}

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 118
 INSERTION SORT: Example

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 119
 INSERTION SORT: Example 120

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 120
 INSERTION SORT: Example 121

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 121
 INSERTION SORT: Example 122

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 122
 INSERTION SORT: Example 123

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 123
 INSERTION SORT: Example 124

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 124
 INSERTION SORT: Example 125

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 125
 INSERTION SORT: Example 126

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 126
 INSERTION SORT: Example 127

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 127
 INSERTION SORT: Example 128

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 128
 INSERTION SORT: Example 129

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 129
 INSERTION SORT: Example 130

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 130
 INSERTION SORT: Example 131

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 131
 INSERTION SORT: Example 132

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 132
 INSERTION SORT: Example 133

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 133
 INSERTION SORT: Example 134

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 134
 INSERTION SORT: Example 135

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 135
 INSERTION SORT: Example 136

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 136
 INSERTION SORT: Example 137

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 137
 INSERTION SORT: Example 138

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 138
 INSERTION SORT: Example 139

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 139
 INSERTION SORT: Example 140

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 140
 INSERTION SORT: Example 141

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 141
 INSERTION SORT: Example 142

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 142
 INSERTION SORT: Example 143

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 143
 INSERTION SORT: Example 144

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 144
 INSERTION SORT: Example 145

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 145
 INSERTION SORT: Example 146

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 146
 INSERTION SORT: Example 147

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 147
 Insertion Sort Example
• Example: Sorting the sequence {3, 7, 4, 9, 5, 2, 6, 1}. In each step,
the key under consideration is underlined. The key that was
moved (or left in place because it was biggest yet considered) in
the previous step is shown in bold.

3 7 4 9 5 2 6 1
3 7 4 9 5 2 6 1
3 7 4 9 5 2 6 1
3 4 7 9 5 2 6 1
3 4 7 9 5 2 6 1
3 4 5 7 9 2 6 1
2 3 4 5 7 9 6 1
2 3 4 5 6 7 9 1
1 2 3 4 5 6 7 9

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 148
 Insertion Sort Analysis
• The best case input is an array that is already sorted. In this case
insertion sort has a linear running time (i.e., O(n)). During each
iteration, the first remaining element of the input (key) is only
compared with the right-most element of the sorted subsection of
the array.

• The simplest worst case input is an array sorted in reverse

order. In these cases every iteration of the inner loop will scan
and shift the entire sorted subsection of the array before inserting
the next element. This gives insertion sort a quadratic running
time (i.e., O(n2)).

• The average case is also quadratic (i.e., O(n2)), which makes

insertion sort impractical for sorting large arrays.

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 149
 HEAP SORT

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA
 HEAP SORT
• A heap can be seen as a complete binary tree.
• heaps are usually implemented as arrays.
• To represent a complete binary tree as an array:
▫ The root node is A[1] and Node i is A[i]
▫ Parent(i) { return i/2; }
▫ Left(i) { return 2*i; }
▫ right(i) { return 2*i + 1; }
16

14 10

8 7 9 3

2 4 1

A= 16 14 10 8 7 9 3 2 4 1
Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 151
 HEAP Property
• A[Parent(i)]  A[i] for all nodes i > 1

• In other words, the value of a node is at most the value of

its parent.
• That is, the children nodes have always lower values than
their parent.

• Where is the largest element in a heap stored?

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 152
 HEAP Height
• The height of a node in the tree = the number of edges on
the longest downward path to a leaf

• The height of a tree = the height of its root

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 153
 HEAP Operations
• Heapify(): maintain the heap property
▫ Given: a node i in the heap with children l and r.
▫ Given: two subtrees rooted at l and r, assumed to be
heaps.

▫ Problem: The subtree rooted at i may violate the heap

property.

▫ Action: let the value of the parent node “float down” so

subtree at i satisfies the heap property.

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 154
 Heap Operations: Heapify()
Heapify(A, i)
{
l = Left(i); r = Right(i);
if (l <= heap_size(A) && A[l] > A[i])
largest = l;
else
largest = i;
if (r <= heap_size(A) && A[r] > A[largest])
largest = r;
if (largest != i)
Swap(A, i, largest);
Heapify(A, largest);
}

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 155
 Heapify() Example
16

4 10

14 7 9 3

2 8 1

A= 16 4 10 14 7 9 3 2 8 1

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 156
 Heapify() Example
16

4 10

14 7 9 3

2 8 1

A= 16 4 10 14 7 9 3 2 8 1

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 157
 Heapify() Example
16

4 10

14 7 9 3

2 8 1

A= 16 4 10 14 7 9 3 2 8 1

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 158
 Heapify() Example
16

14 10

4 7 9 3

2 8 1

A= 16 14 10 4 7 9 3 2 8 1

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 159
 Heapify() Example
16

14 10

4 7 9 3

2 8 1

A= 16 14 10 4 7 9 3 2 8 1

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 160
 Heapify() Example
16

14 10

4 7 9 3

2 8 1

A= 16 14 10 4 7 9 3 2 8 1

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 161
 Heapify() Example
16

14 10

8 7 9 3

2 4 1

A= 16 14 10 8 7 9 3 2 4 1

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 162
 Heapify() Example
16

14 10

8 7 9 3

2 4 1

A= 16 14 10 8 7 9 3 2 4 1

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 163
 Heapify() Example
16

14 10

8 7 9 3

2 4 1

A= 16 14 10 8 7 9 3 2 4 1

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 164
 Heap Operations: BuildHeap()
• We can build a heap in a bottom-up manner by running
Heapify() on successive subarrays
▫ Walk backwards through the array from n/2 to 1, calling
Heapify() on each node.

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 165
 BuildHeap()

// given an unsorted array A, make A a heap

BuildHeap(A)
{
heap_size(A) = length(A);
for (i = length[A]/2 downto 1)
Heapify(A, i);
}

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 166
 BuildHeap() Example
• Work through example
A = {4, 1, 3, 2, 16, 9, 10, 14, 8, 7}

1 3

2 16 9 10

14 8 7

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 167
 Heapsort
• Given BuildHeap(), an in-place sorting algorithm is easily
constructed:
▫ Maximum element is at A[1]
▫ Discard by swapping with element at A[n]
 Decrement heap_size[A]
 A[n] now contains correct value
▫ Restore heap property at A[1] by calling Heapify()
▫ Repeat, always swapping A[1] for A[heap_size(A)]

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 168
 Heapsort

Heapsort(A)
{
BuildHeap(A);
for (i = length(A) downto 2)
{
Swap(A[1], A[i]);
heap_size(A) -= 1;
Heapify(A, 1);
}
}

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 169
 HEAP SORT

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 170
 HEAP SORT
Heap Newly added Swap
• 1. Build the heap element elements
Null 6
6 5
6, 5 3
6, 5, 3 1
6, 5, 3, 1 8
6, 5, 3, 1, 8 5,8
6, 8, 3, 1, 5 6,8
8, 6, 3, 1, 5 7
8, 6, 3, 1, 5, 7 3,7
8, 6, 7, 1, 5, 3 2
8, 6, 7, 1, 5, 3, 2 4
8, 6, 7, 1, 5, 3, 2, 4 1,4
8, 6, 7, 4, 5, 3, 2, 1

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 171
 HEAP SORT
• 2. Sorting
Heap Swap Delete Element Sorted array
elements
8, 6, 7, 4, 5, 3, 2, 1 8,1
1, 6, 7, 4, 5, 3, 2, 8 8
1, 6, 7, 4, 5, 3, 2 1, 7 8
7, 6, 1, 4, 5, 3, 2 1,3 8
7, 6, 3, 4, 5, 1, 2 7,2 8
2, 6, 3, 4, 5, 1, 7 7 8
2, 6, 3, 4, 5, 1 2,6 7,8
6, 2, 3, 4, 5, 1 2,5 7,8
6, 5, 3, 4, 2, 1 6,1 7,8
1, 5, 3, 4, 2, 6 6 7,8
1, 5, 3, 4, 2 1,5 6,7,8
5, 1, 3, 4, 2 1,4 6,7,8 (Conti…)
Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 172
 Heap Swap Delete Element Sorted array
HEAP SORT elements
6,7,8
5, 1, 3, 4, 2 1,4
2.1,Sorting
5, •4, 3, 2 5,2 6,7,8
2, 4, 3, 1, 5 5 6,7,8
2, 4, 3, 1 2,4 5,6,7,8
4, 2, 3, 1 4,1 5,6,7,8
1, 2, 3, 4 4 5,6,7,8
1,2,3 1,3 4,5,6,7,8
3,2,1 3,1 4,5,6,7,8
1,2,3 3 4,5,6,7,8
1,2 1,2 3,4,5,6,7,8
2,1 2,1 3,4,5,6,7,8
2 3,4,5,6,7,8
1,2

1 1 2,3,4,5,6,7,8
1,2,3,4,5,6,7,8
Completed
Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 173
 HEAP SORT Algorithm
BuildHeap(A)
Heapify(A, i)
{
{
heap_size(A) = length(A);
l = Left(i); r = Right(i);
if (l <= heap_size(A) && for (i = length[A]/2
A[l] > A[i]) downto 1)
largest = l; Heapify(A, i);
else }
largest = i; Heapsort(A)
{
if (r <= heap_size(A) && BuildHeap(A);
A[r] > A[largest]) for (i = length(A) downto 2)
largest = r; {
Swap(A[1], A[i]);
if (largest != i) heap_size(A) -= 1;
Swap(A, i, largest); Heapify(A, 1);
}
Heapify(A, largest) }
}
Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 174
 HEAP SORT Analysis
• The performance of this algorithm is O(n + n log n) =
O(n log n).

• Height of the binary tree: log n

• Heapify takes O(log n) time.
• We apply Heapify n times.
• Total time: O(n log n)

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 175
 Sorting in Linear Time
• There are sorting algorithms that run faster than O(n lg n)
time but they require special assumptions about the input
sequence to be sorted.

• Counting sort
• Bucket sort
• Radix sort

• Counting sort and Radix sort assume that the input consists of
integers in a small range.

• Whereas, Bucket sort assumes that the input is generated by a

random process that distributes elements uniformly over the
interval.

• Non-comparision based sorting

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 176
 COUNTING SORT

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA
 Counting Sort
• Assumptions:
▫ n records
▫ Each record contains keys and data
▫ All keys are in the range of 1 to k

• Space
▫ The unsorted list is stored in A, the sorted list will be stored in
an array B
▫ Uses an additional array C of size k

• Input: A[1..n], where A[j]  {1, 2, 3, …, k}

• Output: B[1..n], sorted
• Also: Array C[1..k] for auxiliary storage
Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 178
 Counting Sort
CountingSort(A, B, k)
for i=1 to k
C[i]= 0;
for j=1 to n
C[A[j]] += 1;
for i=2 to k
C[i] = C[i] + C[i-1];
for j=n downto 1
B[C[A[j]]] = A[j];
C[A[j]] -= 1;
end

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 179
 Counting Sort: Example

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 180
 Counting Sort: Example

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 181
 Counting Sort: Another Example

(c)-(e): three iterations of last loop

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 182
 Counting Sort Analysis
• The time for the whole algorithm is O(n + k)

• Because it uses arrays of length k + 1 and n, the total space

usage of the algorithm is also O(n + k).

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 183
 BUCKET SORT

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA
 Bucket Sort
• Whereas counting sort assumes that the input consists of integers in a
small range, bucket sort assumes that the input is generated by a
random process that distributes elements uniformly and independently
over the interval [0, 1).

• Bucket sort divides the interval [0, 1) into n equal-sized subintervals, or

buckets, and then distributes the n input numbers into the buckets.

• To produce the output, we simply sort the numbers in each bucket and
then go through the buckets in order, listing the elements in each.

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 185
 Bucket Sort
• Assume that the input is an n-element array A and that each element
A[i] in the array satisfies 0 <= A[i] < 1.
• The code requires an auxiliary array B[0 ... n-1] of linked lists (buckets).

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 186
 Bucket Sort
A B

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 187
 Bucket Sort
BUCKET-SORT(A)
Let B[0 ... n-1] be a new array
n = A.length
for i = 0 to n-1
Make bucket B[i] an empty list
for i = 1 to n
insert A[i] into list(bucket) B[⎿n*A[i]⏌]
for i = 0 to n-1
sort list B[i] with insertion sort or any other
sort, concatenate the lists B[0], B[1], .... , B[n-
1] together in order

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 188
 Bucket Sort Analysis
• For bucket sort to be O(n) on average, the number of buckets n must be
equal to the length of the array being sorted, and the input array must be
uniformly distributed across the range of possible bucket values.

• Worst-case performance O(n2)

• Best-case performance Ω(n+k)
• Average performance Θ(n+k)
• Worst-case space complexity O(nk)

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 189
 RADIX SORT

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA
 RADIX Sort
• Main idea
▫ Break key into “digit” representation
▫ key = id, id-1, …, i2, i1
▫ "digit" can be a number in any base, a character, etc.

• Radix sort:
for i= 1 to d
sort “digit” i using a stable sort

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 191
 RADIX Sort

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 192
 RADIX Sort
RADIX-SORT(A, d)
//Assume that each element in the n-element array A
has d digits, where digit 1 is the lowest-order
digit and digit d is the highest-order digit.

for i = 1 to d
use a stable sort to sort array A on
digit i

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 193
 RADIX Sort Analysis
• Let there be d digits in input integers. Radix Sort takes
O(d*(n+b)) time where b is the base for representing
numbers, for example, for decimal system, b is 10.

• If k is the maximum possible value, then d would be

O(logb(k)). So overall time complexity is O((n+b) * logb(k)).

• If we set b as n, we get the time complexity as O(n).

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA 194
 END

Prof. Bhumika Patel Sarvajanik College of Eng. & Tech., Surat DAA