UNIT-2: ANALYSIS OF ALGORITHMS

Mr. Prayag Patel

 Topics to be covered
• Analysis of Algorithm
• The efficient algorithm
• Average, Best and Worst case analysis
• Asymptotic Notations
• Analyzing control statement
• Sorting Algorithms and analysis: Bubble sort, Selection sort,
Insertion sort, Shell sort and Heap sort
• Sorting in linear time : Bucket sort, Radix sort and Counting sort
• Amortized analysis
 Analysis of Algorithm
 What is the analysis of an algorithm?
• Analyzing an algorithm means calculating/predicting the
resources that the algorithm requires.
• Two most important resources are computing time (time
complexity) and storage space (space complexity).
 Why analysis is required?
• By analyzing some of the candidate algorithms for a problem, the
most efficient one can be easily identified.
 Efficiency of Algorithm
The efficiency of algorithm can be specified using time
complexity and space complexity.
1) Time Complexity: The amount of time taken by an
algorithm to run. [Means analyze that an algorithm is
slow or fast]
2) Space Complexity: The amount of space (Memory)
taken by an algorithm. [Means analyze that an
algorithm requires more or less space]
 Efficiency of Algorithm
 Frequency Count: The frequency count is a count that
denotes how many times particular statement is
executed (for calculating time complexity)
Example:
int a = 0; 1 [executed 1 time]
for (i=1; i<=n; i++) n+1 [executed n time + 1 wrong condition]
{
a=a+i; n [executed n time]
}
print (“%d”, a); 1 [executed 1 time]

T(n) = 1+n+1+n+1 = 2n+3 =O(n)

 Linear Search
• Suppose, you are given a jar containing
some business cards.
• You are asked to determine whether the
name “Mukesh Ambani" is in the jar.
• To do this, you decide to simply go through
all the cards one by one.
• How long this takes?
• Can be determined by how many cards
are in the jar, i.e., Size of Input.
 Linear Search
• Linear search is a method for finding a particular value
from the given list.
• The algorithm checks each element, one at a time and in
sequence, until the desired element is found.
• Linear search is the simplest search algorithm.
• It is a special case of brute-force search.
 Linear Search Example

Comparing value of ith index with the given element one by one, until we get the
required element or end of the array
Step 1: i=0 Step 3: i=2

i i

Step 2: i=1 Step 4: i=3

i i
Element found at ith index, i=3
 Linear Search Algorithm
# Input : Array A, element
# Output: First index of element in A
or -1 if not found

Algorithm: Linear_Search(A[1..n],key)
for i = 1 to n do
if (A[i]=key) then
return i
return -1
 Linear Search
• The required element in the given array can be found at,
1. E.g. 2: It is at the first position
Best Case: minimum comparison is required
2. E.g. 3 or 1: Anywhere after the first position
Average Case: average number of comparison is required
3. E.g. 8 or 7: Last position or does not found at all
Worst Case: maximum comparison is required
Worst Case

2 9 3 1 8

Best Case

Average Case
 Analysis of Algorithm
Best case Average case Worst case
Resource usage is Resource usage is Resource usage is
minimum average maximum
Algorithm’s behavior Algorithm’s behavior Algorithm’s behavior
under optimal under random under the worst
condition condition condition
Minimum number of Average number of Maximum number of
steps or operations steps or operations steps or operations
Lower bound on Average bound on Upper bound on
running time running time running time
Generally do not Average and worst-case performances are
occur in real the most used in algorithm analysis.
C Best = 1 C Avg = (n + 1) / 2 C Worst = n
 Number Sorting
• Suppose you are sorting numbers in Ascending / Increasing order.
• The initial arrangement of given numbers can be in any of the
following three orders.
1. Numbers are already in required order, i.e., Ascending order
No change is required – Best Case
2. Numbers are randomly arranged initially.
Some numbers will change their position – Average Case
3. Numbers are initially arranged in Descending or Decreasing
order.
All numbers will change their position – Worst Case

5 9 12 23 32 41
9 5 12 32 23 41 5 9 12 23 32 41
41 32 23 12 9 5
 Asymptotic Notations
 Asymptotic Notations (Big O, θ - Theta and Ω - Omega) allow us to
analyze an algorithm’s running time.
 Asymptotic notation is a shorthand way to represent the time
complexity.
 This is also known as an algorithm’s growth rate.
 Asymptotic Notations are used,
1. To characterize the complexity of an algorithm.
2. To compare the performance of two or more algorithms solving
the same problem.
 Bounds on Running Time
 Lower Bound: A lower bound L(n) of an algorithm
defines the minimum time required. It is not possible to
have any other algorithm (for the same problem) whose
time complexity is less than L(n).

 Upper Bound: An upper bound U(n) of an algorithm

defines the maximum time required, we can always
solve the problem in at most U(n) time. Time taken by a
known algorithm to solve a problem with worst case
input gives the upper bound.
1. O-Notation (Big O notation) (Upper Bound)
The Big oh notation is denoted by “ O ”. It is a method of representing
the upper bound of algorithm’s running time.

The function f (n) = O (g (n)) [read as "f of n is big-oh of g of n"] if and

only if exist positive constant c and n0such that
f(n) ⩽ C g(n) , n ≥ n0
2. Ω-Notation (Omega notation) (Lower Bound)
The Omega notation is denoted by “Ω ”. It is a method of
representing the lower bound of algorithm’s running time.

The function f(n) = Ω(g(n)), if there is a constant c > 0 and a natural

number n0 such that

c*g(n) ≤ f(n) for all n ≥ n0

3. θ-Notation (Theta notation)(Same order)
The Theta notation is denoted by “θ”. By this method the running
time is between upper bound and lower bound.

The function f(n)= Θ(g(n)), if there are constants c1, c2 > 0 and a
natural number n0 such that
c1* g(n) ≤ f(n) ≤ c2 * g(n) for all n ≥ n0
 Asymptotic Notations - Example

Algo. 1 Algo. 2 Algo. 1 Algo. 2

running time running time running time running time

1 1 1 1 1 1
2 4 2 2 2 4
3 9 3 3 3 9
4 16 4 4 4 16
5 25 5 5 5 25
Asymptotic Notations - Example

Algo. 1 Algo. 2
running time running time

1 1 2
2 4 4
3 9 8
4 16 16
5 25 32
6 36 64
7 49 128
 Asymptotic Notations - Example

g (n)=n2+1

Value of function 
f(n)=30n+8

Increasing n 
Common Orders of Magnitude (Order of growth)

4 2 8 16 64 16 24
16 4 64 256 4096 65536 2.09 x 1013
64 6 384 4096 262144 1.84 × 1019 1.26 x 1029
256 8 2048 65536 16777216 1.15 × 1077
1024 10 10240 1048576 1.07 × 109 1.79 × 10308
4096 12 49152 16777216 6.87 × 1010 101233
 Growth of Function
 Arrange the given notations in the increasing order of their values.
1
2n, n log n, n2, 1, n, log n, n!, n3
Solution
1, log n, n, n log n, n2, n3, 2n, n!
2
n3, 1, n2, n log n, n2 log n, log n, n0.5
Solution
1, log n, n0.5, n log n, n2, n2 log n, n3
3
n!, 2n , (n+1)!, 22n, nn, nlog n
Solution
nlog n , 2n , n!, (n+1)!, 22n, nn
Growth of Function

Negligible
 Analyzing Control Statement
While analyzing the given fragment of code various programming
constructs appear Those are:
1) Sequencing
2) For loops
3) Recursive calls
4) While & Repeat loops
1) Sequencing:
Sequencing means putting one instruction after another.
Example:
i = 10; 1
printf (“ %d” , i); 1
i = 10 + 5; 1
Here each instruction executes only once. So frequency count
= 1+1+1 = 3
 Sum of n Elements of Array
2) For Loops
• Algorithm
//Input: int A[n], array of n integers
//Output: Sum of all numbers in array A
int Sum(int A[], int n)
{
n+1
int s=0;
for (int i=0; i<n; i++)
1 s = s + A[i];
return s;
}
n
Running Time of Algorithm
Running Time of Algorithm
 Analyzing Control Statement
Example 1: Example 3:

Statement is executed once only

Analysis

Example 2:

Total time is denoted as,

 Analyzing Control Statement
Example 4: Example 6:

Example 5:
 Analyzing Control Statement
3) Recursive calls:
A recursive function is a function that calls itself during its execution.
Example: Algorithm to find factorial of given number
Algorithm factorial(n)
//Problem Description: Algorithm to find factorial of given number.
//Input: Number n
//Output: Returns factorial value of n.
{ If n=3 then
if (n == 0) then
return 1; 3 * fact (2)
else 3*2* fact (1)
return n * factorial(n - 1); 3*2*1*fact (0)
} 3*2*1*1
Here Recursive function can be formulated as: f(n) = n * f(n-1)
We can write recursive relation as: T(n) = T(n-1) + 1
Multiplication for compute f(n-1) To multiply factorial(n-1) by n
 Analyzing Control Statement
4) While or Repeat loops:
Analyzing While or Repeat loop we need to find out how many times the loop is repeated.
Example: Algorithm to find maximum value from given Array.
Algorithm max_element(A[0….n-1])
//Problem Description: Algorithm to find maximum value from given Array.
//Input: Array A[0….n-1]
//Output: Returns largest element from the array.
{ j = n-1
max_value A[0];
i 1;
T (n) = 1
i= 1
j n-1;
While (i <= j) do =(n-1)-1+1 =(n – 1)
{
if (A[i] > max_value) then = θ(n)
max_value A[i] ;
}
return max_value;
}
 Sorting Algorithms

Mr. Prayag Patel

Types of Sorting Algorithms
1. Bubble Sort.
2. Selection Sort.
3. Insertion Sort.
4. Heap Sort.
5. Shell Sort.
6. Radix Sort.
7. Bucket Sort.
8. Counting Sort.
9. Merge Sort.
10.Quick Sort.
 Stable Sorting algorithm
Stable sorting algorithms maintain the relative order of records
with equal keys (i.e. values).
Stable Sorting algorithm

Example: Bubble sort, Insertion Sort, Merge sort, Counting sort.

 In place Sorting algorithm
In-place sort algorithm : A sort algorithm in which the sorted
items occupy the same storage as the original ones.
In-place means that the algorithm does not use extra space
for manipulating the input.
 In place Sorting algorithm

Example: Bubble sort, Selection sort, Insertion Sort, Heap sort, Shell sort, Quick sort.
Equation used in analysis
1) 1 = 1 + 1 + 1 +……+ 1 = n

2) 1 = 1 + 1 + 1 +……+ 1 = n – 1

3) i = 1 + 2 + 3 +……+ n = n(n+1) /2

4) i = 1 + 2 + 3 +……+ (n – 1) = n(n -1) /2

 Bubble Sort

Mr. Prayag Patel

 Bubble Sort
 It is a simple sorting algorithm that works by
• Comparing each pair of adjacent items and swapping them if they are in
the wrong order.
• The pass through the list is repeated until no swaps are needed, which indicates
that the list is sorted.
 As it only uses comparisons to operate on elements, it is a
comparison sort.
 Although the algorithm is simple, it is too slow for practical use.
 It is stable sorting algorithm.
 It is In place sorting algorithm.
 Bubble Sort
Sort the following array in Ascending order
45 34 56 23 12

Pass 1 :

34
45 34 34 34

swap
34
45 45 45 45
56 56 56
23 23

swap
23 23 23
56 56
12

swap
12 12 12 12
56

if(A[j] > A[j+1])

swap(A[j],A[j+1])
 Bubble Sort
Pass 2 : Pass 3 : Pass 4 :

34 34 34 23
34 23 12
23

swap
swap
45 45
23 23 23
34 34
12 12
23

swap
swap
23 23
45 45
12 12 12
34 34

swap
12 12 12
45 45 45 45
56 56 56 56 56 56

if(A[j] > A[j+1])

swap(A[j],A[j+1])
 Bubble Sort Algorithm
# Input: Array A n=5
# Output: Sorted array A 0 34
i = 1,2,3,4 1 45
Algorithm: Bubble_Sort(A) 2 23
j = 0,1,2,3
3 12
for i ← 1 to n-1 do 4 56
for j ← 1 to n-i do
if A[j] > A[j+1] then
thenswap(A[j],A[j+1])
temp← ←A[j]
temp A[j]
A[j]← ←A[j+1]
A[j] A[j+1]
A[j+1]← ←temp
A[j+1] temp

The time complexity of bubble sort is

 Bubble Sort Analysis
Algorithm: Bubble_Sort(A) Cost Times
for i ← 1 to n-1 do
for j ← 1 to n-i do

if A[j] > A[j+1]

then temp ← A[j]

A[j] ← A[j+1]
A[j+1] ← temp
 Bubble Sort Analysis

T(n) = C1n+ C2[ (n+1) - i ] + C3 [ n - i ] + C4 [ n - i]

= C1n+C2[(n+1) 1 - i ] + C3[n 1- i ] +C4[n 1- i]

= C1n + C2 (n+1) (n-1) - C2 n(n-1)/2 + C3 n(n-1) - C3n(n-1)/2 + C4 n(n-1) -

C4 n(n-1)/2
= C1n + C2 n2- C2 - + C2 n2/2 + C2n/2 + C3 n2- C3 n - C3 n2/2 + C3n/2 + C4 n2-
C4 n - C4 n2/2 + C4n/2
= n2(C2 – C2/2 + C3 – C3/2 + C4 – C4 /2 ) + n(C1 + C2/2 – C3 + C3/2 – C4
+ C4 /2) + (– C2)
= an2 + bn + c = O (n2) For Worst case and Average case
Bubble Sort: Best Case Analysis
Pass 1 : i=1

int flag=1; 12 j=0

for(i = 1; i < n; i++) 23 j=1
34
for(j = 0; j < =n-1-i; j++) j=2
45
if(A[j] > A[j+1]) Condition never 59 j=3
becomes true
flag=0;
swap(A[j],A[j+1])
if(flag == 1)
cout<<"already sorted"<<endl
break;

Only one iteration takes place from 0 to n-2 (n-1-1), i.e. Pass 1
 Selection Sort

Mr. Prayag Patel

 Selection Sort
• Selection sort divides the array or list into two parts,
1. The sorted part at the left end
2. and the unsorted part at the right end.
• Initially, the sorted part is empty and the unsorted part is the entire list.
• The smallest element is selected from the unsorted array and swapped with the
leftmost element, and that element becomes a part of the sorted array.
• Then it finds the second smallest element and exchanges it with the element in the
second leftmost position.
• This process continues until the entire array is sorted.
 Selection Sort
Sort the following elements in Ascending order
5 1 12 -5 16 2 12 14

Step 1 :
Unsorted Array
5 1 12 -5 16 2 12 14
1 2 3 4 5 6 7 8

Step 2 :
Unsorted Array (elements 2 to 8) i = 1, Value A[i] = 5
Find the smallest value
-5
5 1 12 -5
5 16 2 12 14 from the entire Unsorted
1 2 3 4 5 6 7 8 array
Min = 4, value A[min] = -5
Swap
 Selection Sort
Step 3 : i = 2, Value A[i] = 1
Unsorted Array (elements 3 to 8)
Find min value from
-5 1 12 5 16 2 12 14 Remaining unsorted array
1 2 3 4 5 6 7 8 min = 2, value A[min] = 1

No Swapping as min value is already at right place

Step 4 :
Unsorted Array
(elements 4 to 8) i = 3, Value A[i] = 12

Find min value from

-5 1 12
2 5 16 12
2 12 14 Remaining unsorted array
1 2 3 4 5 6 7 8 min = 6, value A[min] = 2

Swap
 Selection Sort
Step 5 :
Unsorted Array i = 4, value A[i] = 5
(elements 5 to 8)
Find min value from
-5 1 2 5 16 12 12 14 Remaining unsorted array
min= 4, value A[min] = 5
1 2 3 4 5 6 7 8

No Swapping as min value is already at right place

Step 6 :
Unsorted Array
(elements 6 to 8)
i = 5, value A[i] = 16

-5 1 2 5 12
16 16
12 12 14 Find min value from
1 2 3 4 5 6 7 8 Remaining unsorted array
min = 6, value A[min] = 12
Swap
 Selection Sort
Step 7 :
Unsorted Array i = 6, value A[i] = 16
(elements 7 to 8)
Find min value from
-5 1 2 5 12 12
16 16
12 14 Remaining unsorted array
min = 7, value A[min] = 12
1 2 3 4 5 6 7 8

Swap

Step 8 :
Unsorted Array
(element 8)
i = 7, Value A[i] = 16

-5 1 2 5 12 12 14
16 16
14 Find min value from
1 2 3 4 5 6 7 8 Remaining unsorted array
min = 8, value A[min]= 14
Swap
 Selection Sort Analysis
# Input: Array A
# Output: Sorted array A
Algorithm: Selection_Sort(A[])
for i ← 1 to n-1 do
{
min ← i;
for j ← i + 1 to n do
{
if (A[j] < A[min]) then
min ← j;
}
temp ← A[i];
A[i] ← A[min];
A[min] ← temp;
}
 Selection Sort Algorithm
Algorithm: Selection_Sort(A)
Pass 1 :
for i ← 1 to n-1 do
min ← i; i = 1
for j ← i + 1 to n do
Min ← 12
if A[j] < A[min] then
min ← j ; A[min]← 34
45
temp ← A[i];
A[i] ← A[min]; j = 2 3
A[min] ← temp; No Change

Sort in Ascending order

45 34 56 23 12
1 2 3 4 5
 Selection Sort Algorithm
Algorithm: Selection_Sort(A) Pass 1 :
for i ← 1 to n-1 do
min ← i; i = 1
for j ← i + 1 to n do Min ← 51
2
4
if A[j] < A[min] then
min ← j ; A[min] ← 12
34
23
45
temp ← A[i];
j = 2 3 4 5
A[i] ← A[min];
A[min] ← temp;

Sort in Ascending order Unsorted Array

45 34 56 23 12 45
12 34 56 23 12
45
1 2 3 4 5 1 2 3 4 5
45
12 23
34 34
56 34
23
45
56 45
56
 Selection Sort Analysis
# Input: Array A
# Output: Sorted array A
Algorithm: Selection_Sort(A)
for i ← 1 to n-1 do
{
min ← i;
for j ← i + 1 to n do
{
if (A[j] < A[min]) then
min ← j;
}
temp ← A[i];
A[i] ← A[min];
A[min] ← temp;
}
The time complexity of Selection sort is
 Selection Sort Analysis
Algorithm: Selection_Sort(A) Cost Times
for i ← 1 to n-1 do
min ← i ;
for j ← i + 1 to n do

if A[j] < A[min] then

C3

min ← j ;

temp ← A[i]; C4

A[i] ← A[min]; C5

A[min] ← temp; C6
 Selection Sort Analysis
T(n) = C1n+ C2(n-1)+ C3 n(n+1)/2+ C4(n-1) + C5(n-1 ) + C6(n-1)

= C1n+ C2n- C2+ C3 n2/2 + C3 n/2 + C4n- C4 + C5n- C5+C6n- C6

= n2 C3/2 + n(C1+ C2+ C3/2 + C4 + C5 + C6) + (-C2 –C4 –C6)

= an2 + bn + c

= O (n2)

Time Complexity of Selection sort algorithm is O (n2)

 Insertion Sort

Mr. Prayag Patel

 Insertion Sort
Sort the following elements in Ascending order
5 1 12 -5 16 2 12 14

Step 1 :
Unsorted Array
5 1 12 -5 16 2 12 14
1 2 3 4 5 6 7 8
Step 2 :

15 1 12 -5 16 2 12 14
1 2 3 4 5 6 7 8

Shift down
 Insertion Sort
Step 3 :

1 5 12 -5 16 2 12 14
1 2 3 4 5 6 7 8

No Shift will take place

Step 4 :

1
-5 5 12 -5 16 2 12 14
1 2 3 4 5 6 7 8
Shift down
Shift down
Shift down
 Insertion Sort
Step 5 :

-5 1 5 12 16 2 12 14
1 2 3 4 5 6 7 8

No Shift will take place

Step 6 :

-5 1 25 12 16 2 12 14
1 2 3 4 5 6 7 8

Shift down Shift down

Shift down
 Insertion Sort
Step 7 :

-5 1 2 12 12 14
5 12 16
1 2 3 4 5 6 7 8
Shift down

Step 8 :

-5 1 2 14 14
5 12 12 16
1 2 3 4 5 6 7 8

Shift down

Sorted List -5 1 2 5 12 12 14 16
Insertion Sort Algorithm
Insertion Sort Analysis
 HEAP SORT

Mr. Prayag Patel

 Heap Sort
 A heap data structure is a binary tree with the following two
properties.
1. It is a complete binary tree: Each level of the tree is completely filled,
except possibly the bottom level. At this level it is filled from left to right.

a a

b c b c

d e f d e f

Binary Tree Complete Binary Tree - Heap

 Heap Sort
2. It satisfies the heap order property: the data item stored in each node
is greater than or equal to the data item stored in its children node.

9 9

6 7 6 7

2 4 8 2 4 1

Not a Heap Heap

 Types of Heap Sort
1. Max-Heap − Where the value of the root 9
node is greater than or equal to either of
its children.
6 7

2 4 1
1
2. Min-Heap − Where the value of the root
node is less than or equal to either of its
2 4 children.

6 7 9
Array Representation of Heap
heap
16

14 10

8 7 9 3

2 4 1
Array representation of heap

16 14 10 8 7 9 3 2 4 1
 Steps of Heap Sort
1. Build the complete binary tree using given elements.
2. Create Max-heap to sort in ascending order.
3. Once the heap is created, swap the last node with the
root node and delete the last node from the heap.
4. Repeat step 2 and 3 until the heap is empty.
 Heap Sort Example
 Sort given element in ascending order using heap sort.
19, 7, 16, 1, 14, 17
1 2 3 4 5 6
19 7 16 1 14 17
Step 1: Create complete binary tree

1
19

2 3 16
7

4 1 5 14 17
6
 Heap Sort Example
 Sort given element in ascending order using heap sort. 19, 7, 16,
1, 14, 17 1 2 3 4 5 6
19 7 16 1 14 17

Step 2: Create Max-heap Step 2: Create Max-heap

Step-3
1 1
19 19
Step-1
Step-2
2 7 3 16 3 17
2 14

4 1 5 14 17 1 5 7 16
6 4 6
 Heap Sort Example
Step 3: swap the last node with the root node and delete the
last node from the heap

1 2 3 4 5 6
19 14 17 1 7 16

1 19

Swap &
remove
2 14 3 17 the last
element
6
4 1 5 7 16
 Heap Sort Example
Step 2: Create Max-heap

1 2 3 4 5 6
1
16 14 17 1 7 19 16

2 14 17
3

4 5
1 7
 Heap Sort Example
Step 3: swap the last node with the Step 2: Create Max-heap
root node and delete the last node
from the heap

1 2 3 4 5 6 1 2 3 4 5 6
17 14 16 1 7 19 7 14 16 1 17 19

1 17 1 7
Swap &
remove
2 14 3 16 the last 14 3
16
2
element

4 1 5 7 4 1
 Heap Sort Example
Step 3: swap the last node with the Step 2: Create Max-heap
root node and delete the last node
from the heap

1 2 3 4 5 6 1 2 3 4 5 6
16 14 7 1 17 19 1 14 7 16 17 19

1 16 1 1
Swap &
remove
14 3 7 14 3 7
2 the last 2
element

4 1
 Heap Sort Example
Step 3: swap the last node with the Step 2: Create Max-heap
root node and delete the last node
from the heap

1 2 3 4 5 6 1 2 3 4 5 6
14 1 7 16 17 19 7 1 14 16 17 19

1 1 7
14 Swap &
remove
the last
element 2 1
2 1 3 7

Already a Max-heap
 Heap Sort Example
Step 3: swap the last node with the Step 2: Create Max-heap
root node and delete the last node
from the heap 1 2 3 4 5 6
1 7 14 16 17 19
1 2 3 4 5 6
1
7 1 14 16 17 19 1

1 7 Swap &
Remove the last element
remove
the last
element The final array is sorted
2 1

1 2 3 4 5 6
1 7 14 16 17 19
Heap Sort Algorithm
Algorithm: BUILD-MAX-HEAP(A)
heap-size[A] ← length[A]
for i ← ⌊length[A]/2⌋ downto 1
do MAX-HEAPIFY(A, i)
Max-Heapify - Algorithm

1
 SHELL SORT

Mr. Prayag Patel

Shell Sort
Shell Sort
 RADIX SORT

Mr. Prayag Patel

 Radix Sort

Algorithm: RADIX-SORT(A, d)
for i ← 1 to d
do use a stable sort to sort array A
on digit i
 Radix Sort
3 6 3
7 2 9
3 2 9
8 7 3
6 9 1
5 2 1 521 873 329
691 363 435 297 729
4 3 5 0 1 2 3 4 5 6 7 8 9
2 9 7

Sort on column 1
Sorted list after sort on column 1 = 691, 521, 363, 873, 435, 297, 729, 329
 Radix Sort
6 9 1
5 2 1
3 6 3
8 7 3
4 3 5
329
2 9 7 729 297
521 435 363 873 691
7 2 9
0 1 2 3 4 5 6 7 8 9
3 2 9

Sort on column 2
Sorted list after sort on column 2 = 521, 729, 329, 435, 363, 873, 691, 297
 Radix Sort
5 2 1
7 2 9
3 2 9
4 3 5
3 6 3
8 7 3 363
297 329 435 521 691 729 873
6 9 1 0 1 2 3 4 5 6 7 8 9
2 9 7

Sort on column 3
Sorted list after sort on column 3 = 297, 329, 363, 435, 521, 691, 729, 873
 BUCKET SORT

Mr. Prayag Patel

 Bucket Sort
• Sort the following elements in ascending order using bucket sort.
45 96 29 30 27 12 39 61 91

•Consider total bucket =10 and range like 0-9, 10-19, 20-29….
 Bucket Sort Example
45 96 29 30 27 12 39 61 91

45 96 29 30 27 12 39 61 91

29
27 39 91
96

12 27
29 30 45 61 96
91

0 1 2 3 4 5 6 7 8 9

Sort each bucket queue with insertion sort

Merge all bucket queues together in order

12 27 29 30 39 45 61 91 96
Bucket Sort Example-2

Now Sort each bucket individually

using insertion sort.
0.72 0.78
 Bucket Sort Algorithm
# Input: Array A
# Output: Sorted array A
Algorithm: Bucket-Sort(A[1,…,n])
n ← length[A]
for i ← 1 to n do
insert A[i] into bucket B[⌊ n A[i] ⌋]
for i ← 0 to n – 1 do
sort bucket B[i] with insertion sort
concatenate the buckets B[0], B[1], . . ., B[n - 1] together
in order.
COUNTING SORT
Counting Sort Analysis
 Counting Sort Algorithm
# Input: Array A
# Output: Sorted array A
Algorithm: Counting-Sort(A[1,…,n], B[1,…,n], k)
for i ← 1 to k do
c[i] ← 0
for j ← 1 to n do
c[A[j]] ← c[A[j]] + 1
for i ← 2 to k do
c[i] ← c[i] + c[i-1]
for j ← n downto 1 do
B[c[A[j]]] ← A[j]
c[A[j]] ← c[A[j]] - 1
 AMORTIZED ANALYSIS

Mr. Prayag Patel

 Amortized Analysis
 Amortized analysis considers not just one operation, but a
sequence of operations on a given data structure.
 The time required to perform a sequence of data
structure
operations is averaged over all operations performed.
 Amortized analysis can be used to show that the average
cost of an operation is small even though a single
operation might be expensive.
 Amortized analysis means finding average running time
per operation over a worst case sequence of operation.
Amortized Analysis
Amortized Analysis - Example
Amortized Analysis - Example
Counter value [7] [6] [5] [4] [3] [2] [1] [0] Incre. cost Total cost

0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 1 1 1
+
2 0 0 0 0 0 0 1 0 2 3
3 0 0 0 0 0 0 1 1 1 4
4 0 0 0 0 0 1 0 0 3 7
5 0 0 0 0 0 1 0 1 1 8
6 0 0 0 0 0 1 1 0 2 10
7 0 0 0 0 0 1 1 1 1 11
8 0 0 0 0 1 0 0 0 4 15
9 0 0 0 0 1 0 0 1 1 16
10 0 0 0 0 1 0 1 0 2 18
11 0 0 0 0 1 0 1 1 1 19
1. Aggregate Method
1. Aggregate Method

Counter Number of
value bit flips
0 0000 0
1 0001 1
2 0010 2
3 0011 1
4 0100 3
5 0101 1
6 0110 2
7 0111 1
8 1000 4
15
1. Aggregate Method
 2. Accounting Method

Amortized cost = Actual cost + Credit

Actual cost Amortized cost
1 0 =1 1 0 =0
0 1 =1 0 1 =2
Amortized Analysis - Example
Counter value [7] [6] [5] [4] [3] [2] [1] [0] Incre. cost Total cost

0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 1 1 1
2 0 0 0 0 0 0 1 0 2 3
3 0 0 0 0 0 0 1 1 1 4
4 0 0 0 0 0 1 0 0 3 7
5 0 0 0 0 0 1 0 1 1 8
6 0 0 0 0 0 1 1 0 2 10
7 0 0 0 0 0 1 1 1 1 11
8 0 0 0 0 1 0 0 0 4 15
9 0 0 0 0 1 0 0 1 1 16
10 0 0 0 0 1 0 1 0 2 18
11 0 0 0 0 1 0 1 1 1 19
 3. Potential Method
The potential method is similar to Accounting method in which the
concept of prepaid is used , there will be “potential energy,” or just
“potential,” which can be released to pay for future operations.
We associate the potential with the data structure as a whole rather
than with specific objects within the data structure.
Initial data structure D0.
For n operations D0, D1, D2,……, Dn will be the data structure and C1, C2,
… …, Cn denotes the actual cost.
For each i = 1, 2, ……, n, we let ci be the actual cost of the ith operation
and Di be the data structure that results after applying the i th operation
to data structure Di-1.
Let is a potential function which is a real number.
 ((Di ) is called potential of Di .
 3. Potential Method
The amortized cost of the i th operation is:

Amortized cost

Actual cost Potential Change