2150703

Analysis and Design of

Algorithms (ADA)

Unit-2
Analysis of
Algorithms

Dr. Gopi Sanghani

9825621471
[email protected]
Topics to be covered
 Analysis of Algorithm
 The efficient algorithm
 Average, Best and Worst case analysis
 Asymptotic Notations
 Analyzing control statement
 Sorting Algorithms and analysis: Bubble sort, Selection sort,
Insertion sort, Shell sort and Heap sort
 Sorting in linear time : Bucket sort, Radix sort and Counting sort
 Amortized analysis

Analysis of Algorithms 2 Darshan Institute of Engineering & Technology

Analysis of Algorithm

3
Analysis of Algorithm
 What is the analysis of an algorithm?
• Analyzing an algorithm means calculating/predicting the resources that
the algorithm requires.
• Two most important resources are computing time (time complexity) and
storage space (space complexity).
 Why analysis is required?
• By analyzing some of the candidate algorithms for a problem, the most
efficient one can be easily identified.

Analysis of Algorithms 4 Darshan Institute of Engineering & Technology

How Analysis is Done?
1. Empirical (posteriori) 2. Theoretical (priori)
approach approach
• Programming different competing • Determining mathematically the
techniques & running them on resources needed by each
various inputs using computer. algorithm.
• Implementation of different  Uses the algorithm instead of an
techniques may be difficult. implementation.
 The same hardware and software  The speed of an algorithm can be
environments must be used for determined independent of the
comparing two algorithms. hardware/software environment.
 Results may not be indicative of  Characterizes running time as a
the running time on other inputs function of the input size
not included in the experiment. considers all possible values.

5
Analysis of Algorithms 5 Darshan Institute of Engineering & Technology
Problem & Instance
 Instance: An Instance of a problem consists of the input needed to
compute the solution to the problem.
 Example:
Problem: to multiply two positive numbers
Instance:
 Instance size: Any integer (generally ) that in some way measures
the number of components in an instance.
 Examples:
1. Sorting problem: Instance size is number of elements to be sorted.
2. Graph problem: Instance size is number of nodes or edges or both.

Analysis of Algorithms 6 Darshan Institute of Engineering & Technology

Efficiency of Algorithm
 The efficiency of an algorithm is a measure of the amount of
resources consumed in solving a problem of size .
 The important resource is time, i.e., time complexity
 To measure the efficiency of an algorithm requires to measure its
time complexity using any of the following approaches:
Empirical
1. To run it and measure how much processor time is needed. Approach
2. Mathematically computing how much time is needed as a function of input
size.
Theoretical
Approach

Analysis of Algorithms 7 Darshan Institute of Engineering & Technology

Efficiency of Algorithm
 Running time of an algorithm depends upon,
1. Input Size
2. Nature of Input
 Generally time grows with the size of input, for example, sorting
100 numbers will take less time than sorting of 10,000 numbers.
 So, running time of an algorithm is usually measured as a function
of input size.
 In theoretical computation of time complexity, Running time is
measured in terms of number of steps/primitive operations
performed.

Analysis of Algorithms 8 Darshan Institute of Engineering & Technology

Linear Search
 Suppose, you are given a jar containing
some business cards.
 You are asked to determine whether the
name “Mukesh Ambani" is in the jar. 
 To do this, you decide to simply go
through all the cards one by one.
 How long this takes?
 Can be determined by how many cards
are in the jar, i.e., Size of Input.

Analysis of Algorithms 9 Darshan Institute of Engineering & Technology

 Linear Search
 Linear search is a method for finding a particular value from the
given list.
 The algorithm checks each element, one at a time and in
sequence, until the desired element is found.
 Linear search is the simplest search algorithm.
 It is a special case of brute-force search.

Analysis of Algorithms 10 Darshan Institute of Engineering & Technology

 Linear Search - Example
Search for in given array

Comparing value of ith index with the given element one by one, until we get the required
element or end of the array
Step 1: i=0 Step 3: i=2

i i
2 ≠1 3≠1
Step 2: i=1 Step 4: i=3

𝟏
i i
9≠1 Element found at ith index, i=3
Analysis of Algorithms 11 Darshan Institute of Engineering & Technology
 Linear Search Algorithm
# Input : Array A, element
# Output : First index of element in A or
-1 if not found

Algorithm: Linear_Search
for i = 1 to last index of A:
if A[i] equals element:
return i
return -1

Analysis of Algorithms 12 Darshan Institute of Engineering & Technology

 Linear Search
 The required element in the given array can be found at,
1. E.g. 2: It is at the first position
Best Case: minimum comparison is required
2. E.g. 3 or 1: Anywhere after the first position
Average Case: average number of comparison is required
3. E.g. 8 or 7: Last position or does not found at all
Worst Case: maximum comparison is required
Worst Case

2
2 9 3 1 8

Best Case

Average Case

Analysis of Algorithms 13 Darshan Institute of Engineering & Technology

 Analysis of Algorithm
Best case Average case Worst case
Resource usage is Resource usage is Resource usage is
minimum average maximum
Algorithm’s behavior Algorithm’s behavior Algorithm’s behavior
under optimal under random under the worst
condition condition condition
Minimum number of Average number of Maximum number of
steps or operations steps or operations steps or operations
Lower bound on Average bound on Upper bound on
running time running time running time
Generally do not Average and worst-case performances are
occur in real the most used in algorithm analysis.

Analysis of Algorithms 14 Darshan Institute of Engineering & Technology

Asymptotic Notations

15
 Book Finder
 Suppose, you are writing a program to
find a book from the shelf.
 For any required book, it will start
checking books one by one from the
bottom.
 If you wanted Harry Potter 3, it would
only take 3 actions (or tries) because it’s
the third book in the sequence.
 If Harry Potter 7 — it’s the last book so
it would have to check all 7 books.
 What if there are total 10 books? How
about 10,00,000 books? It would take 1
million tries.

Analysis of Algorithms 16 Darshan Institute of Engineering & Technology

 Number Sorting
 Suppose you are sorting numbers in Ascending / Increasing order.
 The initial arrangement of given numbers can be in any of the
following three orders.
1. Numbers are already in required order, i.e., Ascending order
No change is required – Best Case
2. Numbers are randomly arranged initially.
Some numbers will change their position – Average Case
3. Numbers are initially arranged in Descending or Decreasing order.
All numbers will change their position – Worst Case
5 9 12 23 32 41
9 5 12 32 23 41 5 9 12 23 32 41
41 32 23 12 9 5
Analysis of Algorithms 17 Darshan Institute of Engineering & Technology
 Best, Average, & Worst Case

Problem Best Case Average Case Worst Case

Element at the Element in any Element at last

Linear Search first position of the middle position or not
positions present
The first book Any book in- The last book
Book Finder between
Already sorted Randomly Sorted in
arranged reverse order
Sorting

Analysis of Algorithms 18 Darshan Institute of Engineering & Technology

 Bounds on Running Time
 Lower Bound: A lower bound L(n) of an algorithm defines the
minimum time required, it is not possible to have any other
algorithm (for the same problem) whose time complexity is less
than L(n) for random input.
 Upper Bound: An upper bound U(n) of an algorithm defines the
maximum time required, we can always solve the problem in at
most U(n) time. Time taken by a known algorithm to solve a
problem with worse case input gives the upper bound.

Analysis of Algorithms 19 Darshan Institute of Engineering & Technology

 Asymptotic Notations
 Asymptotic Notations (Big O, θ - Theta and Ω - Omega) allow us
to analyze an algorithm’s running time.
 This is also known as an algorithm’s growth rate.
 Asymptotic Notations are used,
1. To characterize the complexity of an algorithm.
2. To compare the performance of two or more algorithms solving
the same problem.

Analysis of Algorithms 20 Darshan Institute of Engineering & Technology

 1. O-Notation (Big O notation) (Upper Bound)

For a given function , we denote by the set of functions,

= { : there exist positive constants and such that
for all }

 is an asymptotically upper bound for

.
 implies:

Analysis of Algorithms 21 Darshan Institute of Engineering & Technology

 2. Ω-Notation (Omega notation) (Lower Bound)

For a given function , we denote by Ωthe set of functions,

= { : there exist positive constants and such that
for all}

 𝑔(𝑛) is an asymptotically lower

bound for 𝑓(𝑛).
 implies:

Analysis of Algorithms 22 Darshan Institute of Engineering & Technology

 3. θ-Notation (Theta notation) (Same order)

For a given function , we denote by the set of functions,

= { : there exist positive constants , and such that for all}

 is a set, we can write to indicate

that is a member of
 is an asymptotically tight bound for
 implies:

Analysis of Algorithms 23 Darshan Institute of Engineering & Technology

 Asymptotic Notations
1. O-Notation (Big O notation) (Upper Bound)
= { : there exist positive constants and such thatfor all}

2. Ω-Notation (Omega notation) (Lower Bound)

= { : there exist positive constants and such that
for all}
3. θ-Notation (Theta notation) (Same order)
= { : there exist positive constants , and such that for all}

Analysis of Algorithms 24 Darshan Institute of Engineering & Technology

 Asymptotic Notations
𝑓 (𝑛)=𝑂 (𝑔 (𝑛))
= { : there exist positive constants and such thatfor all}

𝑓 (𝑛)=Ω(𝑔 (𝑛))
= { : there exist positive constants and such that
for all}

𝑓 (𝑛)=𝜃(𝑔(𝑛))
= { : there exist positive constants , and such that for all}

Analysis of Algorithms 25 Darshan Institute of Engineering & Technology

 Asymptotic Notations - Example
 Example 1:  Example 2:
and
and

Algo. 1 Algo. 2 Algo. 1 Algo. 2

running time running time running time running time

𝑓 (𝑛 ) ≥ 𝑔 (𝑛 ) 𝑓 (𝑛 )=Ω(𝑔(𝑛)) 𝑓 (𝑛 ) ≤ 𝑔 (𝑛 ) 𝑓 (𝑛 )=O(𝑔(𝑛))

1 1 1 1 1 1
2 4 2 2 2 4
3 9 3 3 3 9
4 16 4 4 4 16
5 25 5 5 5 25

Analysis of Algorithms 26 Darshan Institute of Engineering & Technology

 Asymptotic Notations - Example
 Example: Let and 𝒇 ( 𝒏 )=O ( 𝒈 ( 𝒏) )
Algo. 1 Algo. 2
running time running time

   
1 1 2
2 4 4
3 9 8
4 16 16
5
Here for ,
25 32
6 36 64
7 49 128

Analysis of Algorithms 27 Darshan Institute of Engineering & Technology

 Common Orders of Magnitude

4 2 8 16 64 16 24
16 4 64 256 4096 65536 2.09 x 1013
64 6 384 4096 262144 1.84 × 1019 1.26 x 1029
256 8 2048 65536 16777216 1.15 × 1077
1024 10 10240 1048576 1.07 × 109 1.79 × 10308
4096 12 49152 16777216 6.87 × 1010 101233

Analysis of Algorithms 29 Darshan Institute of Engineering & Technology

 Growth of Function (HW)
 Arrange the given notations in the increasing order of their values.

Analysis of Algorithms 30 Darshan Institute of Engineering & Technology

 Exercises
For each of the following pairs of functions, either is , is , or .
Determine which relationship is correct.

Analysis of Algorithms 31 Darshan Institute of Engineering & Technology

 Asymptotic Notations in Equations
 Maximum Rule: Let, the max rule says that:

1. - is
 The low order terms in a function are relatively insignificant for large

 Consider the example of buying elephants and goldfish:

Cost = cost_of_elephants + cost_of_goldfish Negligible

Cost cost_of_elephants (approximation)

 and have the same rate of growth.

Analysis of Algorithms 32 Darshan Institute of Engineering & Technology

 Exercises
 Express the function in terms of θ notation.

𝜃 (𝑛3)
 Express in terms of O notation.

 Express in terms of O notation. 𝑂(𝑛3)

𝑂(𝑛log ⁡𝑛)

Analysis of Algorithms 33 Darshan Institute of Engineering & Technology

 Home Work
1. Prove (i) Is 2n+1 = O(2n) ? (ii) Is 22n = O(2n)?
2. Check the correctness for the following equality.

3. Find θ notation for the following function

4. Find O notation for the following function

5. Find Ω notation for the following function

Analysis of Algorithms 34 Darshan Institute of Engineering & Technology

Analyzing Control
Statements

35
 Sum of n Elements of Array
 Algorithm
//Input: int A[n], array of n integers
//Output: Sum of all numbers in array A
int Sum(int A[], int n)
{
n+1
int s=0;
for (int i=0; i<n; i++)
1 s = s + A[i];
return s; Total:
} The time complexity of
the algorithm is :
n
Analysis of Algorithms 36 Darshan Institute of Engineering & Technology
 Running Time of Algorithm
 Estimated running time for different values of :
The time complexity of the
algorithm is :

𝑛= 10 steps

𝑛=100 steps

𝑛=1000 steps

𝑛=10000 steps

 As grows, the number of steps grow in linear proportion to for

the given algorithm Sum.

Analysis of Algorithms 37 Darshan Institute of Engineering & Technology

 Running Time of Algorithm
 The dominating term in the function of time complexity is :
The time complexity of the
algorithm is :

𝑛= 10 steps

𝑛=100 steps

𝑛=1000 steps

𝑛=10000 steps

• As gets large, the becomes insignificant.

• The time is linear in proportion to .

Analysis of Algorithms 38 Darshan Institute of Engineering & Technology

 Analyzing Control Statement
Example 1: 𝑠𝑢𝑚=𝑎+𝑏 𝑐 Example 3:
𝑐 1 ( 𝑛+1 )
Statement is executed once only 𝑐 2 𝑛 ( 𝑛+1 )  
The execution time is some 𝑐 3 ∗ 𝑛 ∗𝑛
constant
Analysis
𝑇 (𝑛)=𝑐 1(𝑛+1)+𝑐 2 𝑛(𝑛+1)+𝑐 3 𝑛 (𝑛)
Example 2: 𝑇 (𝑛)=𝑐 1 𝑛+𝑐 1+ 𝑐2 𝑛 2+𝑐2 𝑛+𝑐 3 𝑛 2
𝑐
∗ ( 𝑛+11) 𝑇 (𝑛)=𝑛 2(𝑐 2+ 𝑐3 )+𝑛(𝑐 1+𝑐 2 )+𝑐 1
∗ ( 𝑛 )2
𝑐 𝑇 (𝑛)=𝑎𝑛 2+ 𝑏𝑛+𝑐
𝑻 (𝒏)=𝑶 ( 𝒏 )
𝟐
Total time is denoted as,

Analysis of Algorithms 39 Darshan Institute of Engineering & Technology

 Analyzing Control Statement
Example 4: Example 6:

𝜃 ( 𝑛2 )

𝜃 (𝑛 )
𝑡 𝑛 =𝜃 𝑛 )
( 3
( )
𝜃 (1)
Example 5: printf(“sum is now %d”,)

𝑡 ( 𝑛 )=𝜃 ( 𝑛 6 )
Analysis of Algorithms 40 Darshan Institute of Engineering & Technology
 Home Work
Example 7: Example 8:

sum = 0
while ()

While
sum = sum +

Example 9:

Analysis of Algorithms 41 Darshan Institute of Engineering & Technology

Sorting Algorithms

42
 Applications of Sorting
1. Phone Bill: the calls made are date wise sorted.
2. Bank statement or Credit card Bill: transactions made are date
wise sorted.
3. Filling forms online: “select country” drop down box will have
the name of countries sorted in Alphabetical order.
4. Online shopping: the items can be sorted price wise, date wise or
relevance wise.
5. Files or folders on your desktop are sorted date wise.

Analysis of Algorithms 43 Darshan Institute of Engineering & Technology

 Bubble Sort
Sort the following array in Ascending order
45 34 56 23 12

Pass 1 :

34
45 34 34 34
swap

34
45 45 45 45
56 56 56
23 23
23 23 23
56 swap 56
12

swap
12 12 12 12
56

if(A[j] > A[j+1])

swap(A[j],A[j+1])
Analysis of Algorithms 44 Darshan Institute of Engineering & Technology
 Bubble Sort
Pass 2 : Pass 3 : Pass 4 :

34 34 34 23
34 23 12
23

swap
swap
45 45
23 23 23
34 34
12 12
23

swap
swap
23 23
45 45
12 12 12
34 34

swap
12 12 12
45 45 45 45
56 56 56 56 56 56

if(A[j] > A[j+1])

swap(A[j],A[j+1])
Analysis of Algorithms 45 Darshan Institute of Engineering & Technology
 Bubble Sort
 It is a simple sorting algorithm that works by
• Comparing each pair of adjacent items and swapping them if they are in
the wrong order.
• The pass through the list is repeated until no swaps are needed, which
indicates that the list is sorted.
 As it only uses comparisons to operate on elements, it is a
comparison sort.
 Although the algorithm is simple, it is too slow for practical use.

Analysis of Algorithms 46 Darshan Institute of Engineering & Technology

 Bubble Sort - Algorithm
# Input: Array A
# Output: Sorted array A

Algorithm: Bubble_Sort(A)

for i ← 1 to n do 𝜽 (𝒏)
for j ← 1 to n-i do
if A[j] > A[j+1] then
thenswap(A[j],A[j+1])
temp ←← A[j]
temp A[j] 𝜽𝒏 )
( 𝟐

A[j] ←← A[j+1]
A[j] A[j+1]
A[j+1] ←← temp
A[j+1] temp

𝜽 (𝒏 )
𝟐
The time complexity of bubble sort is

Analysis of Algorithms 47 Darshan Institute of Engineering & Technology

 Bubble Sort - Analysis
Algorithm: Bubble_Sort(A) Cost Times
for i ← 1 to n do
for j ← 1 to n-i do

if A[j] > A[j+1]

then temp ← A[j]

A[j] ← A[j+1]
A[j+1] ← temp

𝑻 ( 𝒏 )=𝜽 ( 𝒏𝟐 )
Analysis of Algorithms 48 Darshan Institute of Engineering & Technology
 Bubble Sort: Best Case Analysis
Best case time complexity = Pass 1 : i = 1

int flag=1; 12 j = 1
for(i = 1; i < n; i++) 23 j = 2
for(j = 1; j < n-i; j++)
34
j = 3
45
if(A[j] > A[j+1]) Condition never 59 j = 4
becomes true
flag=0;
swap(A[j],A[j+1])
if(flag == 1)
cout<<"already sorted"<<endl
break;

Only one iteration takes place from to , i.e., Pass 1

Analysis of Algorithms 50 Darshan Institute of Engineering & Technology
 Selection Sort
Sort the following elements in Ascending order
5 1 12 -5 16 2 12 14

Step 1 :
Unsorted Array
5 1 12 -5 16 2 12 14
1 2 3 4 5 6 7 8

Step 2 :
Unsorted Array (elements 2 to 8) Minj = 1, Minx = 5
Find the smallest value
-5
5 1 12 -5
5 16 2 12 14 from the entire Unsorted
1 2 3 4 5 6 7 8 array

Swap
Index = 4, value = -5

Analysis of Algorithms 51 Darshan Institute of Engineering & Technology

 Selection Sort
Step 3 : Minj = 2, Minx = 1
Unsorted Array (elements 3 to 8)
Find min value from
-5 1 12 5 16 2 12 14 Remaining unsorted array
1 2 3 4 5 6 7 8 Index = 2, value = 1

No Swapping as min value is already at right place

Step 4 :
Unsorted Array
(elements 4 to 8) Minj = 3, Minx = 12

Find min value from

-5 1 12
2 5 16 12
2 12 14 Remaining unsorted array
1 2 3 4 5 6 7 8 Index = 6, value = 2

Swap

Analysis of Algorithms 52 Darshan Institute of Engineering & Technology

 Selection Sort
Step 5 :
Unsorted Array Minj = 4, Minx = 5
(elements 5 to 8)
Find min value from
-5 1 2 5 16 12 12 14 Remaining unsorted array
Index = 4, value = 5
1 2 3 4 5 6 7 8

No Swapping as min value is already at right place

Step 6 :
Unsorted Array
(elements 6 to 8)
Minj = 5, Minx = 16

-5 1 2 5 12
16 16
12 12 14 Find min value from
1 2 3 4 5 6 7 8 Remaining unsorted array
Index = 6, value = 12
Swap
Analysis of Algorithms 53 Darshan Institute of Engineering & Technology
 Selection Sort
Step 7 :
Unsorted Array Minj = 6, Minx = 16
(elements 7 to 8)
Find min value from
-5 1 2 5 12 12
16 16
12 14 Remaining unsorted array
Index = 7, value = 12
1 2 3 4 5 6 7 8

Swap

Step 8 :
Unsorted Array
(element 8)
Minj = 7, Minx = 16

-5 1 2 5 12 12 14
16 16
14 Find min value from
1 2 3 4 5 6 7 8 Remaining unsorted array
Index = 8, value = 14
Swap
Analysis of Algorithms 54 Darshan Institute of Engineering & Technology
 Selection Sort
 Selection sort divides the array or list into two parts,
1. The sorted part at the left end
2. and the unsorted part at the right end.
 Initially, the sorted part is empty and the unsorted part is the
entire list.
 The smallest element is selected from the unsorted array and
swapped with the leftmost element, and that element becomes a
part of the sorted array.
 Then it finds the second smallest element and exchanges it with
the element in the second leftmost position.
 This process continues until the entire array is sorted.

Analysis of Algorithms 55 Darshan Institute of Engineering & Technology

 Selection Sort - Algorithm
# Input: Array A
# Output: Sorted array A
Algorithm: Selection_Sort(A)
for i ← 1 to n-1 do
minj ← i;
minx ← A[i];
for j ← i + 1 to n do
if A[j] < minx then
minj ← j;
minx ← A[j];
A[minj] ← A[i];
A[i] ← minx;
Analysis of Algorithms 56 Darshan Institute of Engineering & Technology
 Selection Sort - Example
Algorithm: Selection_Sort(A) Pass 1 :
for i ← 1 to n-1 do
minj ← i; minx ← A[i];
i = 1
for j ← i + 1 to n do minj ← 12
if A[j] < minx then minx ← 34
45
minj ← j ; minx ← A[j];
A[minj] ← A[i]; j = 2 3
A[i] ← minx; No Change

Sort in Ascending order

45
45 34 56 23 12
1 2 3 4 5

Analysis of Algorithms 57 Darshan Institute of Engineering & Technology

 Selection Sort Example
Algorithm: Selection_Sort(A) Pass 1 :
for i ← 1 to n-1 do
minj ← i ; minx ← A[i];
i = 1
for j ← i + 1 to n do minj ← 51
2
4
if A[j] < minx then 34
12
23
minx ← 45
minj ← j ; minx ← A[j];
A[minj] ← A[i]; j = 2 3 4 5
A[i] ← minx;

Sort in Ascending order Unsorted Array

45
45 34 56 23 12 45
12 34 56 23 12
45
1 2 3 4 5 1 2 3 4 5
45
12 34
23 56
34 23
34
45
56 45
56
Analysis of Algorithms 58 Darshan Institute of Engineering & Technology
 Selection Sort - Analysis
# Input: Array A
# Output: Sorted array A
Algorithm: Selection_Sort(A)
for i ← 1 to n-1 do
minj ← i;
minx ← A[i]; 𝜃 (𝑛 )
for j ← i + 1 to n do
if A[j] < minx then
minj ← j;
minx ← A[j];
𝜃 (𝑛 )2

A[minj] ← A[i];
A[i] ← minx;
𝜽 (𝒏 )
𝟐
The time complexity of Selection sort is
Analysis of Algorithms 60 Darshan Institute of Engineering & Technology
 Selection Sort - Analysis
Algorithm: Selection_Sort(A) Cost Times
for i ← 1 to n-1 do
minj ← i ; minx ← A[i];
for j ← i + 1 to n do

if A[j] < minx then

minj ← j ; minx = A[j]

A[minj] = A[i];
A[i] = minx;

Analysis of Algorithms 61 Darshan Institute of Engineering & Technology

 Selection Sort - Analysis

The time complexity of Selection sort algorithm is

Analysis of Algorithms 62 Darshan Institute of Engineering & Technology

 Insertion Sort - Example
Sort the following elements in Ascending order
5 1 12 -5 16 2 12 14

Step 1 :
Unsorted Array
5 1 12 -5 16 2 12 14
1 2 3 4 5 6 7 8

Step 2 :

𝒋 𝒊=𝟐 , 𝒙=𝟏 𝒋=𝒊 – 𝟏 𝒂𝒏𝒅 𝒋 >𝟎

15 1 12 -5 16 2 12 14 while do
1 2 3 4 5 6 7 8

Shift down

Analysis of Algorithms 63 Darshan Institute of Engineering & Technology

 Insertion Sort - Example
Step 3 : 𝒊=𝟑, 𝒙=𝟏𝟐 𝒋=𝒊 – 𝟏 𝒂𝒏𝒅 𝒋 >𝟎
𝒋
while do
1 5 12 -5 16 2 12 14
1 2 3 4 5 6 7 8

No Shift will take place

Step 4 :

𝒋 𝒋 𝒊=𝟒, 𝒙=− 𝟓 𝒋=𝒊 –𝟏 𝒂𝒏𝒅 𝒋 >𝟎

while do
1
-5 5 12 -5 16 2 12 14
1 2 3 4 5 6 7 8
Shift down
Shift down
Shift down

Analysis of Algorithms 64 Darshan Institute of Engineering & Technology

 Insertion Sort - Example
Step 5 : 𝒊=𝟓, 𝒙=𝟏𝟔 𝒋=𝒊 – 𝟏 𝒂𝒏𝒅 𝒋 >𝟎
𝒋
while do
-5 1 5 12 16 2 12 14
1 2 3 4 5 6 7 8

No Shift will take place

Step 6 :

𝒋 𝒊=𝟔 , 𝒙=𝟐 𝒋=𝒊 – 𝟏 𝒂𝒏𝒅 𝒋 >𝟎

𝒋
while do
-5 1 52 12 16 2 12 14
1 2 3 4 5 6 7 8

Shift down Shift down

Shift down
Analysis of Algorithms 65 Darshan Institute of Engineering & Technology
 Insertion Sort - Example
Step 7 : 𝒊=𝟕, 𝒙=𝟏𝟐 𝒋=𝒊 – 𝟏 𝒂𝒏𝒅 𝒋 >𝟎
𝒋
while do
-5 1 2 12 12 14
5 12 16
1 2 3 4 5 6 7 8

Shift down

Step 8 :

𝒋 𝒊=𝟖, 𝒙=𝟏𝟒 𝒋=𝒊 – 𝟏 𝒂𝒏𝒅 𝒋 >𝟎

-5 1 2 14 14
5 12 12 16 while do
1 2 3 4 5 6 7 8

Shift down
Analysis of Algorithms 66 Darshan Institute of Engineering & Technology
 Insertion Sort - Algorithm
# Input: Array T
# Output: Sorted array T
Algorithm: Insertion_Sort(T[1,…,n])
for i ← 2 to n do
x ← T[i];
j ← i – 1;
while x < T[j] and j > 0 do
T[j+1] ← T[j];
j ← j – 1;
T[j+1] ← x;

Analysis of Algorithms 67 Darshan Institute of Engineering & Technology

 Insertion Sort - Analysis
# Input: Array T
# Output: Sorted array T
Algorithm: Insertion_Sort(T[1,…,n])
for i ← 2 to n do
x ← T[i];
j ← i – 1;
𝜃 (𝑛 )
while x < T[j] and j > 0 do
T[j+1] ← T[j];
j ← j – 1; 𝜃𝑛 2
( )
T[j+1] ← x;

𝜽 (𝒏 )
𝟐
The time complexity of Insertion sort is
Analysis of Algorithms 68 Darshan Institute of Engineering & Technology
 Insertion Sort – Best Case Analysis
Pass 1 :
# Input: Array T
12
# Output: Sorted array T
23 i=2 T[j]=12
Algorithm: Insertion_Sort(T[1,…,n]) 34 i=3 T[j]=23
for i ← 2 to n do 45 i=4 T[j]=34
x ← T[i]; 59 i=5 T[j]=45

j ← i - 1;
while x < T[j] and j > 0 do
T[j+1] ← T[j];
j ← j – 1;
T[j+1] ← x;
The best case time complexity of Insertion sort is 𝜽 (𝒏)
Analysis of Algorithms 69 Darshan Institute of Engineering & Technology
 Insertion Sort - Analysis
Algorithm: Insertion_Sort(T) Cost Times
for i←2 to n do
x←T[i]
j←i-1
while x<T[j] and j>0 do

T[j+1] ← T[j]

j = j - 1

T[j+1] = x;

Analysis of Algorithms 70 Darshan Institute of Engineering & Technology

 Insertion Sort – Time Complexity

+ c4 +c5+ c6 c4 c5 c6+
=(-C4- C5- C6(C1 +C2 +C3 +C7 + C4+ C5+ C6)- 1(C2 +C3 +C7)
=
The time complexity of Insertion sort algorithm is

Analysis of Algorithms 71 Darshan Institute of Engineering & Technology

 Insertion Sort - Example
 Sort given elements in descending order using insertion sort: 3, 9,
6, 5, 23, 14
Step 1 : Step 5:
3 9 6 5 23 14 23
9 6 5 53 23 14

Step 2 :
Step 6 :
39 93 6 5 23 14
23 9
14 6 5 3 14
Step 3 :
39 36 63 5 23 14

Step 4 :
39 36 35 53 23 14

Analysis of Algorithms 72 Darshan Institute of Engineering & Technology

 Tutorial Questions
1. Explain selection sort algorithm with its time complexity. Sort the
given elements using your algorithm.
2. Discuss insertion sort algorithm with its time complexity. Support
your answer by sorting given elements.

Analysis of Algorithms 73 Darshan Institute of Engineering & Technology

 Introduction to Heap
 A heap data structure is a binary tree with the following two
properties.
1. It is a complete binary tree: Each level of the tree is completely filled,
except possibly the bottom level. At this level it is filled from left to right.

a a

b c b c

d e f d e f

Binary Tree Complete Binary Tree - Heap

Analysis of Algorithms 74 Darshan Institute of Engineering & Technology
 Introduction to Heap
2. It satisfies the heap order property: the data item stored in each node
is greater than or equal to the data item stored in its children node.

9 9

6 7 6 7

2 4 8 2 4 1

Not a Heap Heap

Analysis of Algorithms 75 Darshan Institute of Engineering & Technology
 Array Representation of Heap
 Heap can be implemented using an Array.
 An array that represents a heap is an object with two attributes:
1. which is the number of elements in the array, and
2. , the number of elements in the heap stored within array .

Heap
16

14 10

8 7 9 3
Array representation of heap

2 4 1
16 14 10 8 7 9 3 2 4 1

Analysis of Algorithms 76 Darshan Institute of Engineering & Technology

 Array Representation of Heap
 In the array , that represents a heap
1. length= heap-size
2. For any node the parent node is
3. For any node , its left child is and right child is
1
16 For node , parent node is
2 3
14 10
For node ,
4 5 6 7 Left child node is node
8 7 3
Right child is node
9
8 9 10
2 4 1
16 14 10 8 7 9 3 2 4 1

Analysis of Algorithms 77 Darshan Institute of Engineering & Technology

 Types of Heap
1. Max-Heap − Where the value of the root 9
node is greater than or equal to either of
its children. 6 7

2 4 1

1 2. Min-Heap − Where the value of the root

node is less than or equal to either of its
children.
2 4

6 7 9
Analysis of Algorithms 78 Darshan Institute of Engineering & Technology
 Heap Sort
1. Build the complete binary tree using given elements.
2. Create Max-heap to sort in ascending order.
3. Once the heap is created, swap the last node with the root node
and delete the last node from the heap.
4. Repeat step 2 and 3 until the heap is empty.

Analysis of Algorithms 79 Darshan Institute of Engineering & Technology

 Heap Sort – Example 1
Sort the following elements in Ascending order
4 10 3 5 1

4 10 3 5 1

Step 1 : Build Heap

10

Analysis of Algorithms 80 Darshan Institute of Engineering & Technology

 Heap Sort – Example 1
Sort the following elements in Ascending order
4 10 3 5 1

4 10 3 5 1

Step 1 : Build Heap

10 3

Analysis of Algorithms 81 Darshan Institute of Engineering & Technology

 Heap Sort – Example 1
Sort the following elements in Ascending order
4 10 3 5 1

4 10 3 5 1

Step 1 : Build Heap

10 3

Analysis of Algorithms 82 Darshan Institute of Engineering & Technology

 Heap Sort – Example 1
Sort the following elements in Ascending order
4 10 3 5 1

4 10 3 5 1

Step 1 : Build Heap

Now, a binary 4
tree is created
and we have to
10 3
convert it into a
Heap.
5 1

Analysis of Algorithms 83 Darshan Institute of Engineering & Technology

 Heap Sort – Example 1
Sort the following elements in Ascending order
4 10 3 5 1

4 10 3 5 1

Step 1 : Build Heap

Create Max Heap
In a Max Heap, 4
parent node is
always greater
10 3
than or equal
to the child
nodes. 5 1

Analysis of Algorithms 84 Darshan Institute of Engineering & Technology

 Heap Sort – Example 1
Sort the following elements in Ascending order
4 10 3 5 1

4 10 3 5 1

Step 1 : Build Heap

Create Max Heap
In a Max Heap, 4
parent node is 10 is greater than 4
always greater
10 3
than or equal So, swap 10 & 4
to the child
nodes. 5 1

Analysis of Algorithms 85 Darshan Institute of Engineering & Technology

 Heap Sort – Example 1
Sort the following elements in Ascending order
4 10 3 5 1

4 10 3 5 1

Step 1 : Build Heap Swap

Create Max Heap
In a Max Heap, 10
parent node is 10 is greater than 4
always greater
4 3
than or equal So, swap 10 & 4
to the child
nodes. 5 1

Analysis of Algorithms 86 Darshan Institute of Engineering & Technology

 Heap Sort – Example 1
Sort the following elements in Ascending order
4 10 3 5 1

10 4 3 5 1

Step 1 : Build Heap Swap

Create Max Heap
In a Max Heap, 10
parent node is 5 is greater than 4
always greater
4 3
than or equal So, swap 5 & 4
to the child
nodes. 5 1

Analysis of Algorithms 87 Darshan Institute of Engineering & Technology

 Heap Sort – Example 1
Sort the following elements in Ascending order
4 10 3 5 1

10 5 3 4 1

Step 2 : Heap Sort

Max Heap is build
Swap the first 10
and the last
nodes and
5 3
delete the last
node.
4 1

Analysis of Algorithms 88 Darshan Institute of Engineering & Technology

 Heap Sort – Example 1
Sort the following elements in Ascending order
4 10 3 5 1

10 5 3 4 1

Step 2 : Heap Sort Swap

Max Heap is build
Swap the first 10
and the last
nodes and
5 3
delete the last
node.
4 1

Analysis of Algorithms 89 Darshan Institute of Engineering & Technology

 Heap Sort – Example 1
Sort the following elements in Ascending order
4 10 3 5 1

1 5 3 4 10

Step 2 : Heap Sort

Max Heap is build
Swap the first 1
and the last
nodes and
5 3
delete the last
node.
4 10

Analysis of Algorithms 90 Darshan Institute of Engineering & Technology

 Heap Sort – Example 1
Sort the following elements in Ascending order
4 10 3 5 1

1 5 3 4 10

Step 2 : Heap Sort

Create Max Heap
In a Max Heap, 1
parent node is
always greater
5 3
than or equal
to the child
nodes. 4

Analysis of Algorithms 91 Darshan Institute of Engineering & Technology

 Heap Sort – Example 1
Sort the following elements in Ascending order
4 10 3 5 1

5 1 3 4 10

Step 2 : Heap Sort

Create Max Heap
In a Max Heap, 5
parent node is
always greater
1 3
than or equal
to the child
nodes. 4

Analysis of Algorithms 92 Darshan Institute of Engineering & Technology

 Heap Sort – Example 1
Sort the following elements in Ascending order
4 10 3 5 1

5 4 3 1 10

Step 2 : Heap Sort

Max Heap is build
Swap the first 5
and the last
nodes and
4 3
delete the last
node.
1

Analysis of Algorithms 93 Darshan Institute of Engineering & Technology

 Heap Sort – Example 1
Sort the following elements in Ascending order
4 10 3 5 1

1 4 3 5 10

Step 2 : Heap Sort

Max Heap is build
Swap the first 1
and the last
nodes and
4 3
delete the last
node.
5

Analysis of Algorithms 94 Darshan Institute of Engineering & Technology

 Heap Sort – Example 1
Sort the following elements in Ascending order
4 10 3 5 1

1 4 3 5 10

Step 2 : Heap Sort

Create Max Heap
In a Max Heap, 1
parent node is
always greater
4 3
than or equal
to the child
nodes.

Analysis of Algorithms 95 Darshan Institute of Engineering & Technology

 Heap Sort – Example 1
Sort the following elements in Ascending order
4 10 3 5 1

4 1 3 5 10

Step 2 : Heap Sort

Max Heap is build
Swap the first 4
and the last
nodes and
1 3
delete the last
node.

Analysis of Algorithms 96 Darshan Institute of Engineering & Technology

 Heap Sort – Example 1
Sort the following elements in Ascending order
4 10 3 5 1

3 1 4 5 10

Step 2 : Heap Sort

Max Heap is build
Swap the first 3
and the last
nodes and
1 4
delete the last
node.

Analysis of Algorithms 97 Darshan Institute of Engineering & Technology

 Heap Sort – Example 1
Sort the following elements in Ascending order
4 10 3 5 1

3 1 4 5 10

Step 2 : Heap Sort

Max Heap is build
Swap the first 3
and the last
nodes and
1
delete the last
node.

Analysis of Algorithms 98 Darshan Institute of Engineering & Technology

 Heap Sort – Example 1
Sort the following elements in Ascending order
4 10 3 5 1

1 3 4 5 10

Step 2 : Heap Sort

Max Heap is build
Swap the first 1
and the last
nodes and
3
delete the last
node.
Remove the last element from heap and the sorting is over.
Analysis of Algorithms 99 Darshan Institute of Engineering & Technology
 Heap Sort – Example 2
 Sort given element in ascending order using heap sort. 19, 7, 16,
1, 14, 17
19 7 16 1 14 17

Step 1: Create binary tree Step 2: Create Max-heap

19 19

7 16 14 17

1 14 17 1 7 16

Analysis of Algorithms 100 Darshan Institute of Engineering & Technology

 Heap Sort – Example 2

19 14 17 1 7 16 16 14 17 1 7 19

19 16
Swap &
remove
14 17 14 17
the last
element

1 7 16 1 7

Create Max-heap

Analysis of Algorithms 101 Darshan Institute of Engineering & Technology

 Heap Sort – Example 2

17 14 16 1 7 19 7 14 16 1 17 19

17 7
Swap &
remove
14 16 14 16
the last
element

1 7 1

Create Max-heap

Analysis of Algorithms 102 Darshan Institute of Engineering & Technology

 Heap Sort – Example 2

16 14 7 1 17 19 1 14 7 16 17 19

16 1
Swap &
remove
14 7 14 7
the last
element

Create Max-heap

Analysis of Algorithms 103 Darshan Institute of Engineering & Technology

 Heap Sort – Example 2

14 1 7 16 17 19 7 1 14 16 17 19

14 Swap &
7
remove
the last
element
1 7 1

Already a Max-heap
Swap & remove the last
element

Analysis of Algorithms 104 Darshan Institute of Engineering & Technology

 Heap Sort – Example 2

1 7 14 16 17 19 1 7 14 16 17 19

The final array is sorted

1

Remove the last

element

Analysis of Algorithms 105 Darshan Institute of Engineering & Technology

 Heap Sort - Algorithm
 # Input: Array A
 # Output: Sorted array A
 Algorithm: Heap_Sort(A[1,…,n])
BUILD-MAX-HEAP(A)
for i length[A] downto 2
do exchange A[1] A[i]
heap-size[A] heap-size[A] - 1
MAX-HEAPIFY(A, 1, n)

Analysis of Algorithms 106 Darshan Institute of Engineering & Technology

 Build-Max-Heap - Algorithm
Algorithm: BUILD-MAX-HEAP(A) 4 1 3 2 9 7
heap-size[A] ← length[A] 1
4
for i ← ⌊length[A]/2⌋ downto 1 2 3
do MAX-HEAPIFY(A, i) 1 3
4 5 6
heap-size[A] = 6 2 9 7

4 1 7 2 9 3 4 9 7 2 1 3 9 4 7 2 1 3
i=3 1 i=2 1 i=1 1
4 4 94
2 3 2 3 2 3
1 37 19 7 49 7
4 5 6 4 5 6 4 5 6
2 9 3
7 2 1
9 3 2 1 3

Analysis of Algorithms 107 Darshan Institute of Engineering & Technology

 Heap Sort - Algorithm
 # Input: Array A 9 4 7 2 1 3

 # Output: Sorted array A 1

9
 Algorithm: Heap_Sort(A[1,…,n]) 2 3
4 7
BUILD-MAX-HEAP(A) 6
4 5
for i length[A] downto 2 2 1 3

do exchange A[1] A[i]

heap-size[A] heap-size[A] - 1
MAX-HEAPIFY(A, 1, n)

Analysis of Algorithms 108 Darshan Institute of Engineering & Technology

 Heap Sort - Algorithm
 # Input: Array A 3 4 7 2 1 9

 # Output: Sorted array A 1

3
 Algorithm: Heap_Sort(A[1,…,n]) 2 3
4 7
BUILD-MAX-HEAP(A) 6
4 5
for i length[A] downto 2 2 1 9

do exchange A[1] A[i]

heap-size[A] heap-size[A] - 1
MAX-HEAPIFY(A, 1, n)

Analysis of Algorithms 109 Darshan Institute of Engineering & Technology

 Max-Heapify - Algorithm
Algorithm: Max-heapify(A, i, n) 1
3
l LEFT(i) l 2 1 2 3
r RIGHT(i) r 3 4 7
if l ≤ n and A[l] > A[i] Yes 4 5
then largest l largest 2 2 1
else largest i
if r ≤ n and A[r] > A[largest] Yes
then largest r largest 3
if largest i Yes
then exchange A[i] A[largest]
MAX-HEAPIFY(A, largest, n)

Analysis of Algorithms 110 Darshan Institute of Engineering & Technology

 Heap Sort - Algorithm
 # Input: Array A 1
7
 # Output: Sorted array A
2 3
 Algorithm: Heap_Sort(A[1,…,n]) 4 3

4 5
BUILD-MAX-HEAP(A) 2 1

for i length[A] downto 2

do exchange A[1] A[i]
O(𝑛 −1)
heap-size[A] heap-size[A] - 1
MAX-HEAPIFY(A, 1, n)

O ( log𝑛 )

Analysis of Algorithms 111 Darshan Institute of Engineering & Technology

 Heap Sort - Analysis
 # Input: Array A
 # Output: Sorted array A
 Algorithm: Heap_Sort(A[1,…,n])
BUILD-MAX-HEAP(A) O(𝑛log ⁡𝑛)
for i length[A] downto 2 𝑛−1
do exchange A[1] A[i]
heap-size[A] heap-size[A] - 1
MAX-HEAPIFY(A, 1, n) O(log ⁡𝑛)
Running time of heap sort algorithm is:

Analysis of Algorithms 112 Darshan Institute of Engineering & Technology

 Exercises (HW)
1. Sort given element in ascending order using heap sort.
4 1 3 2 16 9 10 14 8 7

2. Sort given element in descending order using heap sort.

65, 77, 5, 25, 32, 45, 99, 83, 69, 81

3. Sort the given elements with Heap Sort Method: 20, 50, 30, 75,
90, 60, 25, 10, 40.

Analysis of Algorithms 113 Darshan Institute of Engineering & Technology

 Shell Sort - Example
 Sort the following elements in ascending order using shell sort.
80 93 60 12 42 30 68 85 10

Step 1: Divide input array into segments, where Initial Segmenting

Gap = 4 (n÷2)

80 93 60 12 42 30 68 85 10

10 93 60 12 42 30 68 85 80

Analysis of Algorithms 114 Darshan Institute of Engineering & Technology

 Shell Sort - Example
Step 1: Initial Segmenting Gap = 4 (n÷2)

10 30 60 12 42 93 68 85 80

10 30 60 12 42 93 68 85 80

10 30 60 12 42 93 68 85 80

Analysis of Algorithms 115 Darshan Institute of Engineering & Technology

 Shell Sort - Example
Step 2: Now, the Segmenting Gap = 2 (4 ÷2)

10 30 60 12 42 93 68 85 80

10 30 42 12 60 93 68 85 80

10 12 42 30 60 85 68 93 80

Analysis of Algorithms 116 Darshan Institute of Engineering & Technology

 Shell Sort - Example
Step 3: Segmenting Gap = 1 (2 ÷2)

10 12 42 30 60 85 68 93 80

10 12 30 42 60 68 80 85 93

The final array is sorted

Analysis of Algorithms 117 Darshan Institute of Engineering & Technology

 Shell Sort - Procedure
1. Divide the array into several smaller non contiguous segments.
2. The distance between successive elements in one segment is
called a gap.
3. Each segment is sorted within itself using insertion sort.
4. Then re-segment into larger segments (smaller gap) and repeat
sorting process.
5. Continue until only one segment is left, i.e., gap = 1.
6. Final sorting finishes the entire array sorting.

Time complexity of shell sort is

Analysis of Algorithms 118 Darshan Institute of Engineering & Technology

 Shell Sort - Exercise
 Sort the following array in descending order using Shell Sort.

45 96 29 30 27 12 39 61 91

Analysis of Algorithms 119 Darshan Institute of Engineering & Technology

 Radix Sort
 Radix Sort puts the elements in order by comparing the digits of
the numbers.
 Each element in the -element array has digits, where digit is the
lowest-order digit and digit is the highest order digit.

Algorithm: RADIX-SORT(A, d)
for i ← 1 to d
do use a stable sort to sort array A
on digit i
 Sort following elements in ascending order using radix sort.
363, 729, 329, 873, 691, 521, 435, 297

Analysis of Algorithms 120 Darshan Institute of Engineering & Technology

 Radix Sort - Example
3 6 3
7 2 9
3 2 9
8 7 3
6 9 1
5 2 1
4 3 5
2 9 7

Sort on column 1

Analysis of Algorithms 121 Darshan Institute of Engineering & Technology

 Radix Sort - Example
6 9 1
5 2 1
3 6 3
8 7 3
4 3 5
2 9 7
7 2 9
3 2 2

Sort on column 2

Analysis of Algorithms 122 Darshan Institute of Engineering & Technology

 Radix Sort - Example
5 2 1
7 2 9
3 2 9
4 3 5
3 6 3
8 7 3
6 9 1
2 9 7

Sort on column 3 The final array is sorted

Analysis of Algorithms 123 Darshan Institute of Engineering & Technology
 Radix Sort - Analysis
 Radix sort time complexity
• Let there be digits in input integers. Radix Sort takes time where is the
base for representing numbers. For decimal system, is .

Analysis of Algorithms 124 Darshan Institute of Engineering & Technology

 Bucket Sort - Example
 Sort the following elements in ascending order using bucket sort.

45 96 29 30 27 12 39 61 91

1. Create empty buckets.

2. Add each input element to appropriate bucket as,
a. Bucket holds values in the half-open interval,

3. Sort each bucket queue with insertion sort.

4. Merge all bucket queues together in order.
 Expected running time is O(n + N), with n = size of original
sequence. If N is O(n) then sorting algorithm in O(n).

Analysis of Algorithms 125 Darshan Institute of Engineering & Technology

 Bucket Sort - Example
45 96 29 30 27 12 39 61 91

45 96 29 30 27 12 39 61 91

29
27 39 91
96

12 27
29 30 45 61 96
91

0 1 2 3 4 5 6 7 8 9

Sort each bucket queue with insertion sort

Merge all bucket queues together in order

12 27 29 30 39 45 61 91 96

Analysis of Algorithms 126 Darshan Institute of Engineering & Technology

 Bucket Sort - Algorithm
# Input: Array A
# Output: Sorted array A
Algorithm: Bucket-Sort(A[1,…,n])
n ← length[A]
for i ← 1 to n do
insert A[i] into bucket B[⌊A[i] ÷ n ⌋]
for i ← 0 to n – 1 do
sort bucket B[i] with insertion sort
concatenate the buckets B[0], B[1], . . ., B[n - 1]
together in order.
Analysis of Algorithms 127 Darshan Institute of Engineering & Technology
 Exercises – Home Work
1. Sort following numbers in ascending order using Shell sort.
18 32 12 5 38 33 16 2
2. Sort the following numbers in descending order using Bucket
sort.
.74, .17, .26, .72, .39, .21
3. Sort following numbers in descending order using Radix sort.
493, 812, 715, 710, 195, 437, 582, 340, 385

Analysis of Algorithms 128 Darshan Institute of Engineering & Technology

 Counting Sort - Example
 Sort the following elements in an ascending order.
3 6 4 1 3 4 1 4 2

Step 1 Given elements are stored in an input array

Index
Elements 3 6 4 1 3 4 1 4 2

Step 2 Define a temporary array . The size of an arrayis equal to the

maximum element in array . Initialize .
Index
Elements
0 0 0 0 0 0

Analysis of Algorithms 129 Darshan Institute of Engineering & Technology

 Counting Sort - Example
 Input array
3 6 4 1 3 4 1 4 2

Step 3 Update an array C with the occurrences of each value of array

Index
2 1 2 3 0 1
Elements

+ +
Step 4 In array from index add the value with previous element

Index
Elements 2 3 5 8 8 9

Analysis of Algorithms 130 Darshan Institute of Engineering & Technology

 Counting Sort - Example
 Create an output array . Start positioning elements of Array as shown
below.

Input Array A 3 6 4 1 3 4 1 4 2

Temporary Array C 2 32 5 87 8 9

Output Array B 1 1 2 3 3 4 4 4 6

Analysis of Algorithms 131 Darshan Institute of Engineering & Technology

 Counting Sort - Example
 Create an output array . Start positioning elements of Array shown as
follows.

Input Array A 3 6 4 1 3 4 1 4 2

Temporary Array C 21 3 5 68 8 9
2 7

Output Array B 1 1 2 3 3 4 4 4 6

Analysis of Algorithms 132 Darshan Institute of Engineering & Technology

 Counting Sort - Example
 Create output an array . Start positioning elements of Array shown as
follows.

Input Array A 3 6 4 1 3 4 1 4 2

Temporary Array C 21 3 53
0 2 4 658 8 98

Output Array B 1 1 2 3 3 4 4 4 6
The final array is sorted
Analysis of Algorithms 133 Darshan Institute of Engineering & Technology
 Counting Sort - Procedure
 Counting sort assumes that each of the input elements is an
integer in the range , for some integer .
 When , the counting sort runs in time.
 The basic idea of counting sort is to determine, for each input
element , the number of elements less than .
 This information can be used to place element directly into its
position in the output array.

Analysis of Algorithms 134 Darshan Institute of Engineering & Technology

 Counting Sort - Algorithm
# Input: Array A When the sort runs in
# Output: Sorted array A time
Algorithm: Counting-Sort(A[1,…,n], B[1,…,n], k)
for i ← 1 to k do
    c[i] ← 0
for j ← 1 to n do
    c[A[j]] ← c[A[j]] + 1
for i ← 2 to k do
    c[i] ← c[i] + c[i-1]
for j ← n downto 1 do
    B[c[A[j]]] ← A[j]
    c[A[j]] ← c[A[j]] - 1
Analysis of Algorithms 135 Darshan Institute of Engineering & Technology
 Counting Sort - Exercise
 Sort following numbers in ascending order using counting sort.
i. 2, 5, 3, 0, 2, 3, 0, 3
ii. 5, 2, 9, 5, 2, 3, 5, 7

 Write counting sort algorithm to sort given elements in

descending order.

Analysis of Algorithms 136 Darshan Institute of Engineering & Technology

Amortized Analysis

137
 Amortized Analysis
 Amortized analysis considers not just one operation, but a
sequence of operations on a given data structure.
 The time required to perform a sequence of data structure
operations is averaged over all operations performed.
 Amortized analysis can be used to show that the average cost of
an operation is small even though a single operation might be
expensive.

Analysis of Algorithms 138 Darshan Institute of Engineering & Technology

 Amortized Analysis
 There are three most common techniques of amortized analysis,
1. The aggregate method
 A sequence of operation takes worst case time
 Amortized cost per operation is
2. The accounting method
 Assign each type of operation an (different) amortized cost
 Overcharge some operations
 Store the overcharge as credit on specific objects
 Then use the credit for compensation for some later operations
3. The potential method
 Same as accounting method
 But store the credit as “potential energy” and as a whole.

Analysis of Algorithms 139 Darshan Institute of Engineering & Technology

 Amortized Analysis - Example
 Incrementing a Binary Counter
1. Implementing a -bit binary counter that counts upward from .
2. Use array of bits as the counter where,
.
3. is the least significant bit.
4. is the most significant bit.

Analysis of Algorithms 140 Darshan Institute of Engineering & Technology

 Amortized Analysis - Example

Counter value [7] [6] [5] [4] [3] [2] [1] [0] Incre. cost Total cost

0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 1 1 1
+
2 0 0 0 0 0 0 1 0 2 3
3 0 0 0 0 0 0 1 1 1 4
4 0 0 0 0 0 1 0 0 3 7
5 0 0 0 0 0 1 0 1 1 8
6 0 0 0 0 0 1 1 0 2 10
7 0 0 0 0 0 1 1 1 1 11
8 0 0 0 0 1 0 0 0 4 15
9 0 0 0 0 1 0 0 1 1 16
10 0 0 0 0 1 0 1 0 2 18
11 0 0 0 0 1 0 1 1 1 19

Analysis of Algorithms 141 Darshan Institute of Engineering & Technology

 1. Aggregate Method
 The running time of an increment operation is proportional to the
number of bits flipped.
 However, all bits are not flipped at each INCREMENT.
 flips at each increment operation;
 flips at alternate increment operations;
 flips only once for 4 successive increment operations;
 In general, bit flips times in a sequence of INCREMENTs.

Analysis of Algorithms 142 Darshan Institute of Engineering & Technology

 1. Aggregate Method
 For (no. of bits) and (counter value) total number of flips of bit can
be given as,
Counter Number of
value bit flips
0 0000 0
1 0001 1
2 0010 2
3 0011 1
4 0100 3
5 0101 1
6 0110 2
7 0111 1
8 1000 4

15
Analysis of Algorithms 143 Darshan Institute of Engineering & Technology
 1. Aggregate Method
 Therefore, the total number of flips in the sequence is,

K-1

The amortized cost of each operation is

Analysis of Algorithms 144 Darshan Institute of Engineering & Technology

 2. Accounting Method
 Running time of an increment operation is proportional to the
number of bits flipped.
 If we charge an amortized cost of to set a bit to .
 When a bit is set we use to pay for the actual setting of the bit
and we place the other on the bit as a credit.
 At any time point, every in the counter has a of credit on it.
 So, no need to charge anything to reset a bit to
 As we can pay for the reset with the on it.
 At most one bit is set to 1, in an increment operation.

Analysis of Algorithms 145 Darshan Institute of Engineering & Technology

 Amortized Analysis - Example

Counter value [7] [6] [5] [4] [3] [2] [1] [0] Incre. cost Total cost

0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 1 1 1
2 0 0 0 0 0 0 1 0 2 3
3 0 0 0 0 0 0 1 1 1 4
4 0 0 0 0 0 1 0 0 3 7
5 0 0 0 0 0 1 0 1 1 8
6 0 0 0 0 0 1 1 0 2 10
7 0 0 0 0 0 1 1 1 1 11
8 0 0 0 0 1 0 0 0 4 15
9 0 0 0 0 1 0 0 1 1 16
10 0 0 0 0 1 0 1 0 2 18
11 0 0 0 0 1 0 1 1 1 19

Analysis of Algorithms 146 Darshan Institute of Engineering & Technology

 2. Accounting Method
 Running time of an increment operation is proportional to the
number of bits flipped.
 If we charge an amortized cost of to set a bit to .
 When a bit is set we use to pay for the actual setting of the bit
and we place the other on the bit as a credit.
 At any time point, every in the counter has a of credit on it.
 So, no need to charge anything to reset a bit to
 As we can pay for the reset with the on it.
 At most one bit is set to 1, in an increment operation.
 The amortized cost of an increment operation is at most .
 For increment operations, the total amortized cost is

Analysis of Algorithms 147 Darshan Institute of Engineering & Technology

 3. Potential Method
 Initial data structure
 operations, resulting in with costs .
 A potential function is
 is called the potential of
 Amortized cost of the operation is:

(actual cost + potential change)

 If is the total amortized cost of performing operations on data
structure then,

Analysis of Algorithms 149 Darshan Institute of Engineering & Technology

 3. Potential Method
 Total actual cost of operations can be bounded as,

 Total actual cost is bounded by total amortized cost plus

the net drop in potential over an entire sequence of
operations.
Analysis of Algorithms 150 Darshan Institute of Engineering & Technology
Thank
you!