Analysis and Design of

Algorithms (ADA)
GTU # 3150703

Unit-2:
Analysis of
Algorithm

Rupesh Vaishnav
Computer Engineering
Department
Darshan Institute of Engineering & Technology, Rajkot
[email protected]
9428037452
 Looping
Outline
 Analysis of Algorithm
 The efficient algorithm
 Average, Best and Worst case analysis
 Asymptotic Notations
 Analyzing control statement
 Loop invariant and the correctness of the algorithm
 Sorting Algorithms and analysis: Bubble sort,
Selection sort, Insertion sort, Shell sort and Heap
sort
 Sorting in linear time : Bucket sort, Radix sort and
Counting sort
 Amortized analysis
Analysis of Algorithm
Introduction

What is Analysis of an Algorithm?

 Analyzing an algorithm means calculating/predicting the
resources that the algorithm requires.
 Analysis provides theoretical estimation for the required
resources of an algorithm to solve a specific computational
problem.
 Two most important resources are computing time (time
complexity) and storage space (space complexity).
Why Analysis is required?
 By analyzing some of the candidate algorithms for a problem,
the most efficient one can be easily identified.

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 4

Efficiency of Algorithm

consumed in solving a problem of size 𝑛.

 The efficiency of an algorithm is a measure of the amount of resources

 An algorithm must be analyzed to determine its resource usage.

 Two major computational resources are execution time and memory
space.
 Memory Space requirement can not be compared directly, so the
important resource is computational time required by an algorithm.
 To measure the efficiency of an algorithm requires to measure its
execution time using any of the following approaches:
1. Empirical Approach: To run it and measure how much processor time is needed.
2. Theoretical Approach: Mathematically computing how much time is needed as a
function of input size.

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 5

How Analysis is Done?
Empirical (posteriori) Theoretical (priori)
approach approach
 Programming different  Determining
competing techniques & mathematically the
running them on various resources needed by each
inputs using computer. algorithm.
 Implementation of different  Uses the algorithm instead
techniques may be difficult. of an implementation.
 The same hardware and  The speed of an algorithm
software environments can be determined
must be used for comparing independent of the
two algorithms. hardware/software
 Results may not be environment.
indicative of the running  Characterizes running time
time on other inputs not as a function of the input
included in the experiment. size considers all possible
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 6
Time Complexity
 Time complexity of an algorithm quantifies the amount of time taken by
an algorithm to run as a function of the length of the input.
 Running time of an algorithm depends upon,
1. Input Size
2. Nature of Input
 Generally time grows with the size of input, for example, sorting 100
numbers will take less time than sorting of 10,000 numbers.
 So, running time of an algorithm is usually measured as a function of input
size.
 Instead of measuring actual time required in executing each statement in
the code, we consider how many times each statement is executed.
 So, in theoretical computation of time complexity, running time is
measured in terms of number of steps/primitive operations performed.

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 7

Problem & Instance
 Instance: An Instance of a problem consists of the input needed to
compute the solution to the problem.
 Example:
Problem: to multiply two positive integers
Instance:
 Instance size: Any integer (generally 𝒏) that in some way measures the
number of components in an instance.
 Examples:
1. Sorting problem: Instance size is number of elements to be sorted.
2. Graph problem: Instance size is number of nodes or edges or both

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 8

Linear Search
 Suppose, you are given a jar containing some business cards.
 You are asked to determine whether the name “Bill Gates" is in the jar.
 To do this, you decide to simply go through all the cards one by one.
 How long this takes?
 Can be determined by how many cards are in the jar, i.e., Size of Input.
 Linear search is a method for finding a particular value from the given list.
 The algorithm checks each element, one at a time and in sequence, until
the desired element is found.
 Linear search is the simplest search algorithm.
 It is a special case of brute-force search.

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 9

Linear Search – Example

Search for in given array

Comparing value of ith index with the given element one by one, until
we get the required element or end of the array
Step 1: i=1 Step 3: i=3

i i
𝟐≠𝟏 𝟑≠𝟏
Step 2: i=2 Step 4: i=4

𝟏
i i
𝟗≠𝟏 Element found at ith index, i=4
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 10
Linear Search - Algorithm
# Input : Array A, element x
# Output : First index of element x in A or -1 if not found

Algorithm: Linear_Search
for i = 1 to last index of A
if A[i] equals element x
return i
return -1

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 11

Linear Search - Analysis
 The required element in the given array can be found at,

Case 1: element 2 Case 2: element 3 Case 3: element 7 at

which is at the first anywhere after the last position or
position so minimum first position so, an element does not
comparison is required average number of found at all, maximum
comparison is required comparison is required
Best Case Average Case Worst Case
Worst
Case

𝟑
Search 𝟕
for

Best
Case Average
Case
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 13
Analysis of Algorithm

Best Case Average Case Worst Case

Resource usage is Resource usage is Resource usage is
minimum average maximum
Algorithm’s behavior Algorithm’s behavior Algorithm’s behavior
under optimal under random under the worst
condition
Minimum number of condition
Average number of condition
Maximum number of
steps or operations steps or operations steps or operations
Lower bound on Average bound on Upper bound on
running time running time running time
Generally do not Average and worst-case performances are the
occur in real most used in algorithm analysis.

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 14

Book Finder Example  Suppose, you are writing a program
to find a book from the shelf.
 For any required book, it will start
checking books one by one from the
bottom.
 If you wanted Harry Potter 3, it
would only take 3 actions (or tries)
because it’s the third book in the
sequence.
 If Harry Potter 7 — it’s the last book
so it would have to check all 7
books.
 What if there are total 10 books?
How about 10,00,000 books? It
would take 1 million tries.

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 15

Number Sorting - Example
 Suppose you are sorting numbers in Ascending / Increasing order.
 The initial arrangement of given numbers can be in any of the following
three orders.
Case 1: Numbers are Case 2: Numbers are Case 3: Numbers are
already in required randomly arranged initially arranged in
order, i.e., Ascending initially. Some numbers Descending order so,
order will change their all numbers will
No change is required position change their position
Best Case Average Case Worst Case

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 17

Best, Average, & Worst Case

Problem Best Case Average Case Worst Case

Linear Element at the Element in any of Element at last
Search first position the middle position or not
Book Finder The first book positions
Any book in- present
The last book
between
Sorting Already sorted Randomly Sorted in reverse
arranged order

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 18

Asymptotic Notations
Introduction
 The theoretical (priori) approach of analyzing an algorithm to measure the
efficiency does not depend on the implementation of the algorithm.
 In this approach, the running time of an algorithm is describes as
Asymptotic Notations.
 Computing the running time of algorithm’s operations in mathematical
units of computation and defining the mathematical formula of its run-
time performance is referred to as Asymptotic Analysis.
 An algorithm may not have the same performance for different types of
inputs. With the increase in the input size, the performance will change.
 Asymptotic analysis accomplishes the study of change in performance of
the algorithm with the change in the order of the input size.
 Using Asymptotic analysis, we can very well define the best case, average
case, and worst case scenario of an algorithm.

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 20

Asymptotic Notations
 Asymptotic notations are mathematical notations used to represent the
time complexity of algorithms for Asymptotic analysis.
 Following are the commonly used asymptotic notations to calculate the
running time complexity of an algorithm.
1. Notation
2. Notation
3. Notation
 This is also known as an algorithm’s growth rate.
 Asymptotic Notations are used,
1. To characterize the complexity of an algorithm.
2. To compare the performance of two or more algorithms solving the same problem.

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 21

1. -Notation (Big notation) (Upper Bound)
 The notation is the formal way to express the upper bound of an
algorithm's running time.
 It measures the worst case time complexity or the longest amount of time
an algorithm can possibly take to complete.
 For a given function , we denote by the set of functions,
= { : there exist positive constants and such that for all }

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 22

Big() Notation  is an asymptotically upper bound
for .

 implies:
𝒄 . 𝒈(𝒏)

 An upper bound of an algorithm

𝒇 (𝒏)
defines the maximum time
required, we can always solve the
problem in at most time.

 Time taken by a known algorithm to

solve a problem with worse case
𝒏 𝟎 𝒇 (𝒏)=𝑶 (𝒈 (𝒏)) 𝒏 input gives the upper bound.

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 23

2. -Notation (Omega notation) (Lower Bound)
 Big Omega notation () is used to define the lower bound of any algorithm
or we can say the best case of any algorithm.
 This always indicates the minimum time required for any algorithm for all
input values, therefore the best case of any algorithm.
 When a time complexity for any algorithm is represented in the form of
big-Ω, it means that the algorithm will take at least this much time to
complete it's execution. It can definitely take more time than this too.
 For a given function 𝑔(𝑛), we denote by 𝑔(𝑛)) the set of functions,

there exist positive constants and such that for all

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 24

2. -Notation (Omega notation) (Lower Bound)
 For a given function 𝑔(𝑛), we denote by 𝑔(𝑛)) the set of functions,

there exist positive constants and such that for all

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 25

Big() Notation  is an asymptotically lower bound
for

𝒇 (𝒏)  implies:

𝒄 . 𝒈(𝒏)  A lower bound of an algorithm

defines the minimum time required,
it is not possible to have any other
algorithm (for the same problem)
whose time complexity is less than
for random input.

𝒏 𝟎 𝒇 (𝒏)=𝜴 (𝒈 (𝒏)) 𝒏

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 26

3. -Notation (Theta notation) (Same order)
 The notation is the formal way to enclose both the lower bound and the
upper bound of an algorithm's running time.
 Since it represents the upper and the lower bound of the running time of
an algorithm, it is used for analyzing the average case complexity of an
algorithm.
 The time complexity represented by the Big- notation is the range within
which the actual running time of the algorithm will be.
 So, it defines the exact Asymptotic behavior of an algorithm.
 For a given function 𝑔(𝑛), we denote by θ(𝑔(𝑛)) the set of functions,
there exist positive constants

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 27

-Notation  is a set, we can write to indicate
𝒄 𝟐 .𝒈(𝒏) that is a member of

 is an asymptotically tight bound for

𝒇 (𝒏)
 implies:
𝒄 𝟏 .𝒈(𝒏)

𝒏𝟎
𝒏
𝒇 (𝒏)=𝜽(𝒈 (𝒏))

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 28

Asymptotic Notations
1. O-Notation (Big O notation) (Upper Bound)

= { : there exist positive constants and such thatfor

all}
𝐟 (𝐧 )=𝐎 ( 𝐠 ( 𝐧 ))

2. Ω-Notation (Omega notation) (Lower Bound)

= { : there exist positive constants and such that for

all }
𝐟 ( 𝐧 ) =Ω(𝐠 ( 𝐧 ))

3. θ-Notation (Theta notation) (Same order)

= { : there exist positive constants , andsuch that for

all }
𝐟 (𝐧 )=𝛉 (𝐠 ( 𝐧))

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 29

Asymptotic Notations – Examples
 Example 1:  Example 2:
and and
Algo. 1 Algo. 2 Algo. 1 Algo. 2
running running running running
time time time time

𝑓 ( 𝑛 ) ≥ 𝑔 ( 𝑛 ) ⟹ 𝑓 ( 𝑛 ) =Ω(𝑔 (𝑛)) 𝑓 ( 𝑛 ) ≤ 𝑔 ( 𝑛 ) ⟹ 𝑓 ( 𝑛 ) =O(𝑔 (𝑛))

1 1 1 1 1 1
2 4 2 2 2 4
3 9 3 3 3 9
4 1 4 4 4 1
5 6
2 5 5 5 6
2
5 5
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 30
Asymptotic Notations – Examples
 Example 3: and

𝑓 ( 𝑛 ) ≤ 𝑔 ( 𝑛 ) ⟹ 𝑓 ( 𝑛 ) =O(𝑔 (𝑛))

1 1 2
2 4 4
3 9 8
4 1 1
6 6 Here for ,
5 2 3
5 2
6 3 6
6 4
7 4 12
9 8

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 31

Asymptotic Notations –  Example 4:
is in the order of , or
Examples is order , or

𝒇 (𝒏)=𝑶 (𝒈(𝒏))
g (n)=n +1
2
Value of function 

f(n)=30n+8
 In general, any function is faster-
growing than any function.

Base value
Increasing n 

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 32

Common Orders of Magnitude

4 2 8 16 64 16 24
16 4 64 256 4096 65536 2.09 x
1013
64 6 384 4096 262144 1.84 × 1.26 x
1019 1029
256 8 2048 65536 16777216 1.15 ×
1077
102 10 10240 1048576 1.07 × 109 1.79 ×
4 10308
409 12 49152 1677721 6.87 × 101233
6 Vaishnav
Rupesh #3150703 6 1010of Algorithm
(ADA)  Unit 2 – Analysis 33
Growth of Function

 Arrange the given notations in the increasing order of their values.

1.
2.

 For each of the following pairs of functions, either is , is , or . Determine

which relationship is correct.

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 34

Asymptotic Notations in Equations
 Consider an example of buying elephants and goldfish:
Cost = cost_of_elephants + cost_of_goldfish
Negligib
Cost ≈ cost_of_elephants (approximation) le

𝑂( 𝑓(𝑛)
 Maximum Rule: Let, the max rule says that:

+𝑔(𝑛))=𝑂(max⁡(𝑓(𝑛),𝑔(𝑛)))

1. - is

 The low order terms in a function are relatively insignificant for large 𝒏

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 35

Asymptotic Notations
1. O-Notation (Big O notation) (Upper Bound)

= { : there exist positive constants and such thatfor

all}
𝐟 (𝐧 )=𝐎 ( 𝐠 ( 𝐧 ))

2. Ω-Notation (Omega notation) (Lower Bound)

= { : there exist positive constants and such that for

all }
𝐟 ( 𝐧 ) =Ω(𝐠 ( 𝐧 ))

3. θ-Notation (Theta notation) (Same order)

= { : there exist positive constants , andsuch that for

all }
𝐟 (𝐧 )=𝛉 (𝐠 ( 𝐧))

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 36

Expressing Function in terms of 𝑂, , θ notation
 Find Big O notation for

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 37

Expressing Function in terms of 𝑂, , θ notation
 Find Big O, Omega Theta θ notation for

Big O Thus,
Omega Theta θ Notation:
Notation: Notation:

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 38

Exercises
1. Express the function in terms of θ notation.
2. Express in terms of O notation.
3. Express in terms of O notation.
4. Prove or disprove (i) Is 2n+1 = O(2n) (ii) Is 22n = O(2n)
5. Check the correctness for the following equality,
6. Find θ notation for the following function

7. Find O notation for the following function

8. Find Ω notation for the following function

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 39

Analyzing Control
Statements
For Loop
# Input : int A[n], array of n integers
# Output : Sum of all numbers in array A

Algorithm: int Sum(int A[], int n)

{
int s=0; n+1
for (int i=0; i<n; i++)
1 s = s + A[i];
return s;
} n

Total time taken =

n+1+n+2 = 2n+3
Time Complexity f(n) =
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 41
Running Time of Algorithm
 The time complexity of the algorithm is :
 Estimated running time for different values of 𝑛 :

𝒏=𝟏𝟎 steps

𝒏=𝟏𝟎𝟎 steps

𝒏=𝟏𝟎𝟎𝟎 steps

𝒏=𝟏𝟎𝟎𝟎𝟎 steps

 As 𝑛 grows, the number of steps grow in linear proportion to 𝑛 for the

given algorithm Sum.
 The dominating term in the function of time complexity is : As gets large,
the becomes insignificant.
 The time is linear in proportion to .
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 42
Analyzing Control Statements
Exampl Exampl
e 1: e 3:
𝐜 𝒄 𝟏 ( 𝒏 +𝟏 )
 Statement is executed once 𝒄 𝟐 𝒏 ( 𝒏 +𝟏 )
only
 So, The execution time is 𝒄 𝟑 ∗ 𝒏∗𝒏
some constant  Analysis
Exampl
e 2:
𝐜
∗ (𝒏+𝟏 ) 𝟏

𝐜
∗ (𝒏 ) 𝟐
 Total time is denoted as,

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 43

Analyzing Control Statements
Exampl Exampl
e 4: e 6:
𝜽 ( 𝒏𝟐 )

𝜽 (𝒏)

𝒕 ( 𝒏 )=𝜽 ( 𝒏𝟑 )
𝜽 ( 𝟏)
printf(“sum is now %d”,)
Exampl
e 5:

𝒕 ( 𝒏 )=𝜽 ( 𝒏𝟔 )
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 44
Sorting Algorithms
Bubble Sort, Selection Sort, Insertion Sort
Introduction
 Sorting is any process of arranging items systematically or arranging
items in a sequence ordered by some criterion.
 Applications of Sorting
1. Phone Bill: the calls made are date wise sorted.
2. Bank statement or Credit card Bill: transactions made are date wise sorted.
3. Filling forms online: “select country” drop down box will have the name of countries
sorted in Alphabetical order.
4. Online shopping: the items can be sorted price wise, date wise or relevance wise.
5. Files or folders on your desktop are sorted date wise.

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 46

Bubble Sort – Example

Sort the following array in Ascending order

45 34 56 23 12

Pass 1 :
3
4 swap
3 3 3
5
4
3 4 4 4
4
5 5 5
2 5
2

swap
6
2 6
2 6
3
2
5 3
5
1

swap
3
1 3
1 3
6
1 6
2
1
5
2 2 2 2
6

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 47

Bubble Sort – Example
Pass 2 : Pass 3 : Pass 4 :
3 3 3 2
3 2 1
2

swap
swap
4 4
2 4
2 4
3
2 3
1 3
2
1

swap
swap
5
2 5
3
2
4 3
4
1 3
4
1 4
2
1
3 2
3

swap
3
1 3
5
1 5
2
1
4 2
4 2
4 4
2
5 2
5 2
5 5 5 5
6 6 6 6 6 6

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 48

Bubble Sort - Algorithm
# Input: Array A
# Output: Sorted array A

Algorithm: Bubble_Sort(A)
for i ← 1 to n-1 do 𝛉 (𝐧)
for j ← 1 to n-i do
if A[j] > A[j+1] then
temp ← A[j] A[j+1])
swap(A[j], 𝛉 ( 𝐧 𝟐)
A[j] ← A[j+1]
A[j+1] ← temp

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 49

Bubble Sort
 It is a simple sorting algorithm that works by comparing each pair of
adjacent items and swapping them if they are in the wrong order.
 The pass through the list is repeated until no swaps are needed, which
indicates that the list is sorted.
 As it only uses comparisons to operate on elements, it is a comparison
sort.
 Although the algorithm is simple, it is too slow for practical use.
 The time complexity of bubble sort is

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 50

Bubble Sort Algorithm – Best Case Analysis
# Input: Array A Pass 1 : i = 1
# Output: Sorted array A 1 j = 1
Algorithm: Bubble_Sort(A) 2 j = 2
int flag=1; 3
Condition j = 3
never 4
for i ← 1 to n-1 do j = 4
becomes 5
for j ← 1 to n-i do true 9
if A[j] > A[j+1] then Best case time
flag = 0; complexity =
swap(A[j],A[j+1])
if(flag == 1)
cout<<"already sorted"<<endl
break;
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 51
Selection Sort – Example 1
Sort the following elements in
Ascending order5 1 12 -5 16 2 12 14

Step 1 :
Unsorted Array
5 1 12 -5 16 2 12 14
1 2 3 4 5 6 7 8

Step 2 :  Minj denotes the current index and Minx is

Unsorted Array (elements the value stored at current index.
2 to 8)  So, Minj = 1, Minx = 5
1 1 1 1
-5
5 1 -5
5 2  Assume that currently Minx is the smallest
2 6 2 4 value.
1 2 3 4 5 6 7 8  Now find the smallest value from the remaining
Swap Index = 4, entire Unsorted array.
value = -5
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 54
Selection Sort – Example 1

Step 3 :
Unsorted Array  Now Minj = 2, Minx = 1
(elements 3 to 8)  Find min value from
1 1 1 1
-5 1 5 2 remaining unsorted array
2 6 2 4
1 2 3 4 5 6 7 8
Index = 2,
value = 1
No Swapping as min value is already at right place
Step 4 :
Unsorted Array  Minj = 3, Minx = 12
(elements 4 to 8)  Find min value from
remaining unsorted array
1 1 1 1 1
-5 1 2 5 2 Index = 6,
2 6 2 2 4
1 2 3 4 5 6 7 8 value = 2

Swap

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 55

Selection Sort – Example 1

Step 5 : Unsorted Array  Now Minj = 4, Minx = 5

(elements 5 to 8)  Find min value from
1 1 1 1 remaining unsorted array
-5 1 2 5
6 2 2 4 Index = 4,
1 2 3 4 5 6 7 8
value = 5
No Swapping as min value is already at right place
Step 6 :
 Minj = 5, Minx = 16
 Find min value from
Unsorted Array remaining unsorted array
(elements 6 to 8)

1 1 1 1 Index = 6,
-5 1 2 5 value = 12
6
2 2
6 2 4
1 2 3 4 5 6 7 8

Swap
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 56
Selection Sort – Example 1

Step 7 :
Unsorted Array  Now Minj = 6, Minx = 16
(elements 7 to 8)  Find min value from
1 1 1 1 remaining unsorted array
-5 1 2 5
2 6
2 2
6 4 Index = 7,
1 2 3 4 5 6 7 8
value = 12
Swap

Step 8 :
Unsorted Array  Minj = 7, Minx = 16
(element 8)  Find min value from
remaining unsorted array
1 1 1 1
-5 1 2 5
2 2 6
4 4
6 Index = 8,
1 2 3 4 5 6 7 8 value = 14
Swap
The entire array is
sorted now.
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 57
Selection Sort
 Selection sort divides the array or list into two parts,
1. The sorted part at the left end
2. and the unsorted part at the right end.
 Initially, the sorted part is empty and the unsorted part is the entire list.
 The smallest element is selected from the unsorted array and swapped
with the leftmost element, and that element becomes a part of the sorted
array.
 Then it finds the second smallest element and exchanges it with the
element in the second leftmost position.
 This process continues until the entire array is sorted.
 The time complexity of selection sort is

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 58

Selection Sort - Algorithm
# Input: Array A
# Output: Sorted array A

Algorithm: Selection_Sort(A)
for i ← 1 to n-1 do 𝛉 (𝐧)
minj ← i;
minx ← A[i];
for j ← i + 1 to n do
if A[j] < minx then
minj ← j; 𝛉 ( 𝐧 𝟐
)
minx ← A[j];
A[minj] ← A[i];
A[i] ← minx;

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 59

Selection Sort – Example 2
Algorithm: Selection_Sort(A)
Pass 1 :
for i ← 1 to n-1 do
minj ← i; minx ← A[i]; i = 1
for j ← i + 1 to n do minj ← 21
if A[j] < minx then
minx ← 34
45 No
minj ← j ; minx ← A[j]; Change
A[minj] ← A[i]; j = 2 3
A[i] ← minx; A[j] = 5
34 6
Sort in Ascending
order
45 34 56 23 12
1 2 3 4 5

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 60

Selection Sort – Example 2
Algorithm: Selection_Sort(A)
Pass 1 :
for i ← 1 to n-1 do
minj ← i; minx ← A[i]; i = 1
for j ← i + 1 to n do 42
minj ← 5
if A[j] < minx then
minx ← 12
34
23
minj ← j ; minx ← A[j];
A[minj] ← A[i]; j = 2 3 4 5
A[i] ← minx; A[j] = 1
23 2
Sort in Ascending Unsorted Array
order
45 34 56 23 12 45 34 56 23 45
12 12
1 2 3 4 5 1 2 3 4 5

Swap
45
12 34
23 56
34 23
34
45
56 45
56
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 61
Insertion Sort – Example
Sort the following elements in
Ascending order5 1 12 -5 16 2 12 14

Step 1 :
Unsorted Array
5 1 12 -5 16 2 12 14
1 2 3 4 5 6 7 8

Step 2 :

𝒋 𝒊=𝟐 , 𝒙=𝟏 𝒋=𝒊 – 𝟏 𝒂𝒏𝒅 𝒋>𝟎

5
1 1 1 -5 1 2 1 1 while do
1 2 2
3 4 6
5 6 2
7 4
8

Shift
down
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 63
Insertion Sort – Example

Step 3 :
𝒋 𝒊=𝟑 , 𝒙=𝟏𝟐 𝒋=𝒊 – 𝟏 𝒂𝒏𝒅 𝒋>𝟎
1 5 1 -5 1 2 1 1 while do
1 2 2
3 4 6
5 6 2
7 4
8

No Shift will take

place
Step 4 :
𝒊=𝟒 , 𝒙=− 𝟓 𝒋=𝒊 – 𝟏 𝒂𝒏𝒅 𝒋>𝟎
while do
𝒋 𝒋
-5
1 5 1 -5 1 2 1 1
1 2 2
3 4 6
5 6 2
7 4
8
ShiftShiftShift
downdown down
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 64
Insertion Sort – Example

Step 5 :
𝒋 𝒊=𝟓 , 𝒙=𝟏𝟔 𝒋=𝒊 – 𝟏 𝒂𝒏𝒅 𝒋>𝟎
-5 1 5 1 1 2 1 1 while do
1 2 3 2
4 6
5 6 2
7 4
8

No Shift will take

place
Step 6 :
𝒊=𝟔 , 𝒙=𝟐 𝒋=𝒊 – 𝟏 𝒂𝒏𝒅 𝒋>𝟎
𝒋 𝒋 while do

-5 1 5
2 1 1 2 1 1
1 2 3 2
4 6
5 6 2
7 4
8

ShiftShiftShift
downdown down
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 65
Insertion Sort – Example

Step 7 :
𝒋 𝒊=𝟕 , 𝒙=𝟏𝟐 𝒋=𝒊 – 𝟏 𝒂𝒏𝒅 𝒋>𝟎
-5 1 2 5 1 1 1 1 while do
1 2 3 4 2
5 2
6
6 2
7 4
8

Shift
down
Step 8 :
𝒊=𝟖 , 𝒙=𝟏𝟒 𝒋=𝒊 – 𝟏 𝒂𝒏𝒅 𝒋>𝟎
𝒋 while do

-5 1 2 5 1 1 1 1
1 2 3 4 2
5 2
6 4
6
7 4
8
The entire array is
Shift
down sorted now.
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 66
Insertion Sort - Algorithm
# Input: Array T
# Output: Sorted array T

Algorithm: Insertion_Sort(T[1,…,n])
for i ← 2 to n do
x ← T[i];
𝛉 (𝐧)
j ← i – 1;
while x < T[j] and j > 0 do
T[j+1] ← T[j];
𝛉 ( 𝐧 𝟐)
j ← j – 1;
T[j+1] ← x;

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 67

Insertion Sort Algorithm – Best Case Analysis
# Input: Array T Pass 1 :
# Output: Sorted array T 1
2
i=2 x=23 T[j]=1
Algorithm: Insertion_Sort(T[1,…,n]) 3 2
i=3 x=34 T[j]=23
for i ← 2 to n do 4
x ← T[i];
𝛉 (𝐧) 5
i=4 x=45 T[j]=34
i=5 x=59 T[j]=45
j ← i – 1; 9
while x < T[j] and j > 0 do The best case time
T[j+1] ← T[j]; complexity of Insertion
j ← j – 1; sort is
T[j+1] ← x; The average and worst
case time complexity of
Insertion sort is

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 68

Heap & Heap Sort
Algorithm
Introduction
 A heap data structure is a binary tree with the following two properties.
1. It is a complete binary tree: Each level of the tree is completely filled, except
possibly the bottom level. At this level it is filled from left to right.
2. It satisfies the heap order property: the data item stored in each node is
greater than or equal to the data item stored in its children node.

a a 9 9

b c b c 6 7 6 7

d e f d e f 2 4 8 2 4 1

Binary Tree but not a Complete Binary Tree Not a Heap

Heap - Heap Heap

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 70

Array Representation of Heap
 Heap can be implemented using an Array.
 An array 𝐴 that represents a heap is an object with two attributes:
1. 𝑙𝑒𝑛𝑔𝑡ℎ[𝐴], which is the number of elements in the array, and
2. ℎ𝑒𝑎𝑝−𝑠𝑖𝑧𝑒[𝐴], the number of elements in the heap stored within array 𝐴

1
6

1 1
4 0

Array representation
8 7 9 3
of heap

2 4 1 1 1 1 8 7 9 3 2 4 1
Hea
6 4 0
p
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 71
Array Representation of Heap
 In the array 𝐴, that represents a heap
1. length[𝐴] = heap-size[𝐴]
2. For any node 𝒊 the parent node is 𝒊/𝟐
3. For any node 𝒋, its left child is 𝟐𝒋 and right child is 𝟐𝒋+ 𝟏

𝟏 1 For node , parent node is

6
𝟐 𝟑 For node ,
1 1 Left child node is node
4 0 Right child is node
𝟒 𝟓 𝟔 𝟕
8 7 9 3

𝟖 𝟗 𝟏𝟎 1
2 4 1 1 1 1 8 7 9 3 2 4 1
Hea 4
6 4 0
p
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 72
Types of Heap
1. Max-Heap − Where the value of the 9
root node is greater than or equal to
either of its children.
6 7

2 4 1

1 2. Min-Heap − Where the value of the root

node is less than or equal to either of its
children.
2 4

6 7 9
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 73
Introduction to Heap Sort
1. Build the complete binary tree using given elements.
2. Create Max-heap to sort in ascending order.
3. Once the heap is created, swap the last node with the root node and
delete the last node from the heap.
4. Repeat step 2 and 3 until the heap is empty.

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 74

 Heap Sort – Example 1
Sort the following elements in
Ascending 4order
10 3 5 1

Step 1 : Create Complete Binary Tree

4

4 10 3 5 1 1
0
3

Now, a binary tree is 5 1

created and we have
to convert it into a
Heap.

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 75

Heap Sort – Example 1
Sort the following elements in
Ascending 4order
10 3 5 1

Step 2 : Create Max Heap

1
4 10 is greater
0 than 4
4
1 10
4 3 5 1 So, swap 10 & 4
1
0 4
0
3
Swa
p

In a Max Heap, 5 1
parent node is
always greater than
or equal to the child
nodes.
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 76
Heap Sort – Example 1
Sort the following elements in
Ascending 4order
10 3 5 1

Step 2 : Create Max Heap

1
5 is greater
0
than 4
10 4
5 3 5
4 1 So, swap 5 & 4
5
4 3
Swa
p

In a Max Heap, 4
5 1
parent node is
always greater than Max Heap is
or equal to the child created
nodes.
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 77
Heap Sort – Example 1
Sort the following elements in
Ascending 4order
10 3 5 1

Step 3 : Apply Heap Sort

1
1
0

10
1 5 3 4 1
0 5 3
Swa
p
1. Swap the first and 4 1
the last nodes and 0
2. Delete the last
node.

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 78

Heap Sort – Example 1
Sort the following elements in
Ascending 4order
10 3 5 1

Step 3 : Apply Heap Sort

5
1 Max Heap
Property is
1
5 5
1
4 3 4
1 10 violated so,
1
4
5 3 create a Max
Swa Heap again.
p

1
4

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 79

Heap Sort – Example 1
Sort the following elements in
Ascending 4order
10 3 5 1

Step 3 : Apply Heap Sort

1
5 Max Heap is
created
5
1 4 3 1
5 10
4 3
Swa
p
1. Swap the first and 5
1
the last nodes and
2. Delete the last
node.

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 80

Heap Sort – Example 1
Sort the following elements in
Ascending 4order
10 3 5 1

Step 3 : Apply Heap Sort

3
1
4 Create Max
Heap again
1
4
3 4
1 3
4 5 10 Max Heap is
1
4 3
4 created
Swa
p

1. Swap the first and

the last nodes and
2. Delete the last
node.

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 81

Heap Sort – Example 1
Sort the following elements in
Ascending 4order
10 3 5 1

Step 3 : Apply Heap Sort

1
3 Already a Max
Heap
3
1 1
3 4 5 10
3
1
Swa
p

1. Swap the first and

the last nodes and
2. Delete the last
node.

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 82

Heap Sort – Example 1
Sort the following elements in
Ascending 4order
10 3 5 1

Step 3 : Apply Heap Sort

1 Already a Max
Heap
1 3 4 5 10

Remove the last

element from heap
and the sorting is
over.

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 83

Heap Sort – Example 2
 Sort given element in ascending order using heap sort. 19, 7, 16, 1, 14,
17
19 7
1 16
1 1 14
7 17
1
4 7 6

Step 1: Create binary Step 2: Create

tree Max-heap
1 1
9 9

1 1 1
7 7
6 4 6
7

1 1 1 1
1 1 7
4 7 4 7
6

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 84

Heap Sort – Example 2
Step 3 Step 4

19
1 14 17 1 7 16
1 16
1 14 17
1 1 7 19
6 9 7 6
1 1 Create Max-
9
6 6
7
Swap heap
1 1 & 1
1
4 7 remo 7
6
4
ve
the
1
1 7 last 1 7
6
9
eleme
nt

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 85

Heap Sort – Example 2
Step 5 Step 6

17
7 14 16 1 7
1 19 7
1 14 16
7 1 17 19
7 6
1 1 Create Max-
7 7
7 6
Swap heap
1 1 & 1
1 7
4 6 remo 6
4
ve
the
1
1 7 last 1
7
eleme
nt

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 86

Heap Sort – Example 2
Step 7 Step 8

16
1 14 7 1 17 19 1 14
1 7 16 17 19
6 4
1 1 Create Max-
1 1
6 4
Swap heap
1 &
7 1 7
4 remo 1
4
ve
the
1
1 last
6
eleme
nt

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 87

Heap Sort – Example 2
Step 9 Step 10

14
7 1 7
1 16 17 19 7
1 1
7 14 16 17 19
4
1 Already a
7 Swap & 7
1
4
remove Max-heap
Swap & remove
1 the last the last element
1 7 element 1
7
4

Step 11
1 7 14 16 17 19

Remove
1
The entire array
theis last
element
sorted now.
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 88
Exercises
 Sort the following elements using Heap Sort Method.
1. 34, 18, 65, 32, 51, 21
2. 20, 50, 30, 75, 90, 65, 25, 10, 40

 Sort the following elements in Descending order using Hear Sort

Algorithm.
1. 65, 77, 5, 23, 32, 45, 99, 83, 69, 81

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 89

Binary Tree Analysis

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 90

Heap Sort – Algorithm
# Input: Array A
# Output: Sorted array A

Algorithm: Heap_Sort(A[1,…,n])
BUILD-MAX-HEAP(A)
for i length[A] downto 2
do exchange A[1] A[i]
heap-size[A] heap-size[A] – 1
MAX-HEAPIFY(A, 1, n)

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 91

Heap Sort – Algorithm
Algorithm: BUILD-MAX-HEAP(A) 4 1 3 2 9 7
heap-size[A] ← length[A] 1
4
for i ← ⌊length[A]/2⌋ downto
1 2 3
1 3
do MAX-HEAPIFY(A, i)
4 5 6
heap- 2 9 7
size[A] = 6
4 1 7 2 9 3 4 9 7 2 1 3 9 4 7 2 1 3
i= 1 i= 1 i= 1
4 4 4
9
3 2 1
2 3 2 3 2 3
1 3
7 9
1 7 9
4 7
4 5 6 4 5 6 4 5 6
2 9 3
7 2 9
1 3 2 1 3

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 92

Heap Sort – Algorithm
# Input: Array A
# Output: Sorted array A 3
9 4 7 2 1 9
3

1
Algorithm: Heap_Sort(A[1,…,n]) 9
BUILD-MAX-HEAP(A) 2 3
for i length[A] downto 2 4 7
6
do exchange A[1] A[i] 4 5
heap-size[A] heap-size[A] – 1 2 1 3
MAX-HEAPIFY(A, 1, n)

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 93

Heap Sort – Algorithm
Algorithm: Max-heapify(A, i, n)
l LEFT(i) l 2 1 3 4 7 2 1 9
r RIGHT(i) r 3
1
if l ≤ n and A[l] > A[i] Yes 3
2 3
then largest l largest 2
4 7
else largest i 4 5
if r ≤ n and A[r] > A[largest] Yes 2 1
then largest r largest 3

if largest i Yes
then exchange A[i] A[largest]
MAX-HEAPIFY(A, largest, n)

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 94

Heap Sort – Algorithm
# Input: Array A
# Output: Sorted array A 3 4 7 2 1 9

1
Algorithm: Heap_Sort(A[1,…,n]) 7
BUILD-MAX-HEAP(A) 2 3
for i length[A] downto 2 4 3
do exchange A[1] A[i] 4 5
heap-size[A] heap-size[A] – 1 2 1
MAX-HEAPIFY(A, 1, n)

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 95

Heap Sort Algorithm – Analysis
# Input: Array A
# Output: Sorted array A
heap-size[A] ← length[A]
for i ← ⌊length[A]/2⌋ downto𝒏1/ 𝟐
Algorithm: Heap_Sort(A[1,…,n])
𝐎 (𝒍𝒐𝒈 ⁡𝒏)
do MAX-HEAPIFY(A, i)
BUILD-MAX-HEAP(A)𝐎 (𝒏 𝒍𝒐𝒈 ⁡𝒏 )
for i length[A] downto 2
𝒏− 𝟏
do exchange A[1] A[i]
heap-size[A] heap-size[A] – 1
MAX-HEAPIFY(A, 1, n) 𝑶 (𝒏−𝟏)(𝒍𝒐𝒈𝒏)

Running time of heap sort algorithm is:

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 96

Sorting Algorithms
Shell Sort, Radix Sort, Bucket Sort, Counting Sort
Shell Sort - Example
 Sort the following elements in ascending order using This
shell sort. avoids large
algorithm
shifts as in case of
80 93 60 12 42 30 68 85 10 insertion sort, if the
smaller value is to the far
right and has to be moved
Step 1: Divide input array into segments, where
to the far left. Initial
Segmenting Gap = 4 (n/2)

80 93 60 12 42 30 68 85 10
Each segment
is sorted
within itself
using insertion
10 93 60 12 42 30 68 85 80
sort

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 98

Shell Sort - Example
 Sort the following elements in ascending order using shell sort.

80 93 60 12 42 30 68 85 10

Step 1: Divide input array into segments, where Initial

Segmenting Gap = 4 (n/2)
10 30 60 12 42 93 68 85 80

Each segment
10 30 60 12 42 93 68 85 80 is sorted
within itself
using insertion
sort
10 30 60 12 42 93 68 85 80
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 99
Shell Sort - Example
 Sort the following elements in ascending order using shell sort.

80 93 60 12 42 30 68 85 10

Step 2: Now, the Segmenting Gap = 2 (4/2)

10 30 60 12 42 93 68 85 80

Each segment
10 30 42 12 60 93 68 85 80 is sorted
within itself
using insertion
sort
10 12 42 30 60 85 68 93 80
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 100
Shell Sort - Example
 Sort the following elements in ascending order using shell sort.

80 93 60 12 42 30 68 85 10

Step 3: Now, the Segmenting Gap = 1 (2/2)

10 12 42 30 60 85 68 93 80

Each segment
is sorted
10 12 30 42 60 68 80 85 93 within itself
using insertion
sort
The entire array is
sorted now.
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 101
Shell Sort - Procedure
 Divide the array into several smaller non contiguous segments.
 The distance between successive elements in one segment is called a
gap.
 Each segment is sorted within itself using insertion sort.
 Then re-segment into larger segments (smaller gap) and repeat sorting
process.
 Continue until only one segment is left, i.e., gap = 1.
 Final sorting finishes the entire array sorting.
 Time complexity of shell sort is

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 102

Radix Sort
 Radix Sort puts the elements in order by comparing the digits of the
numbers.
 Each element in the 𝒏-element array 𝑨 has 𝒅 digits, where digit 1 is the
lowest-order digit and digit 𝑑 is the highest order digit.
Algorithm: RADIX-SORT(A, d)
for i ← 1 to d
do use a stable sort to sort array A on
digit i
 Sort following elements in Ascending order using radix sort.
363, 729, 329, 873, 691, 521, 435, 297

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 103

Radix Sort - Example
3 6 6 9 5 2
7 32 5 12 7 12
3 92 3 16 3 92
8 97 8 37 4 93
6 39 4 33 3 56
5 12 2 59 8 37
4 13 7 72 6 39
2 59 3 92 2 19
7 9 7
The entire array is
Sort on Sort on Sort on
column 1 column 2 sorted now. 3
column
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 104
Bucket Sort – Introduction
 Sort the following elements in Ascending order using bucket sort.

45 96 29 30 27 12 39 61 91

1. Create empty buckets.

2. Add each input element to appropriate bucket as,
a. Bucket holds values in the half-open interval,

3. Sort each bucket queue with insertion sort.

4. Merge all bucket queues together in order.

 Expected running time is with = size of original sequence. If is then

sorting algorithm in
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 105
Bucket Sort – Example

45 96 29 30 27 12 39 61 91

4 9 2 3 2 1 3 6 9
5 6 9 0 7 2 9 1 1

2 3 9
9
7 9 6
1
1 2 3 4 6 9
2 7
9 0 5 1 1
6
0 1 2 3 4 5 6 7 8 9

Sort each bucket queue with

insertion
Merge all sort
bucket queues together in
order
12 27 29 30 39 45 61 91 96

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 106

Bucket Sort - Algorithm
# Input: Array A
# Output: Sorted array A
Algorithm: Bucket-Sort(A[1,…,n])
n ← length[A]
for i ← 1 to n do
insert A[i] into bucket B[⌊A[i] ÷ n ⌋]
for i ← 0 to n – 1 do
sort bucket B[i] with insertion sort
concatenate the buckets B[0], B[1], . . ., B[n - 1]
together in order.

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 107

Counting Sort – Example
 Sort the following elements in Ascending order using counting sort.

3 6 4 1 3 4 1 4 2

Step 1 Given elements are stored in an input array

Index
Elements 3 6 4 1 3 4 1 4 2

Step 2 Define a temporary array . The size of an arrayis

equal to the maximum element in array . Initialize .
Index
Elements 0 0 0 0 0 0

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 108

Counting Sort – Example
 Sort the following elements in Ascending order using counting sort.

Input array 3 6 4 1 3 4 1 4 2

value of array 𝐴
Step 3 Update an array C with the occurrences of each

Index
Elements 02 01 0
2 03 0
0 0
1

+ +
Step 4 In array 𝐶, from index 2 to 𝑛 add the value with
previous element
Index
Elements 2 3 5 8 8 9

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 109

Counting Sort – Example
 Create an output array 𝐵[1…9]. Start positioning elements of Array 𝐴
𝑡𝑜 𝐵 as shown below.

Input array 3 6 4 1 3 4 1 4 2

1
Temporary Array C 2
0 2
3 4
3
5 6
7
5
8 8 8
9

Output Array B 1 1 2 3 3 4 4 4 6

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 110

Counting Sort - Procedure
 Counting sort assumes that each of the 𝑛 input elements is an integer in
the range 0 to 𝑘, for some integer 𝑘.
 When 𝒌=𝑶(𝒏), the counting sort runs in 𝜽(𝒏) time.
 The basic idea of counting sort is to determine, for each input element 𝑥,
the number of elements less than 𝑥.
 This information can be used to place element 𝑥 directly into its position
in the output array.

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 111

Counting Sort - Algorithm
# Input: Array A
# Output: Sorted array A
Algorithm: Counting-Sort(A[1,…,n], B[1,…,n], k)
for i ← 1 to k do
c[i] ← 0
for j ← 1 to n do
c[A[j]] ← c[A[j]] + 1
for i ← 2 to k do
c[i] ← c[i] + c[i-1]
for j ← n downto 1 do
B[c[A[j]]] ← A[j]
c[A[j]] ← c[A[j]] - 1

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 112

Amortized Analysis
Introduction
 Amortized analysis considers not just one operation, but a sequence of
operations on a given data structure or a database.
 Amortized Analysis is used for algorithms where an occasional operation is
very slow, but most of the other operations are faster.
 The time required to perform a sequence of data structure operations is
averaged over all operations performed.
 In Amortized Analysis, we analyze a sequence of operations and
guarantee a worst case average time which is lower than the worst case
time of a particular expensive operation.
 So, Amortized analysis can be used to show that the average cost of an
operation is small even though a single operation might be expensive.

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 114

Amortized Analysis Techniques
 There are three most common techniques of amortized analysis,
1. The aggregate method
 A sequence of 𝑛 operation takes worst case time 𝑇(𝑛)
 Amortized cost per operation is 𝑇(𝑛)/𝑛
2. The accounting method
 Assign each type of operation an (different) amortized cost
 Overcharge some operations
 Store the overcharge as credit on specific objects
 Then use the credit for compensation for some later operations
3. The potential method
 Same as accounting method
 But store the credit as “potential energy” and as a whole.

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 115

Amortized Analysis -
Incrementing a Binary Counter
Example
Count Increm Tot
 Implementing a -bit binary counter
er [7][6][5][4][3][2][1][0] ent al that counts upward from .
value cost cos
0t  Use array of bits as the counter
0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 1 1 1 where,
+
2 0 0 0 0 0 0 1 0 2 3
3 0 0 0 0 0 0 1 1 1 4  is the least significant bit.
4 0 0 0 0 0 1 0 0 3 7  is the most significant bit.
5 0 0 0 0 0 1 0 1 1 8
6 0 0 0 0 0 1 1 0 2 10
7 0 0 0 0 0 1 1 1 1 11
8 0 0 0 0 1 0 0 0 4 15
9 0 0 0 0 1 0 0 1 1 16
10 0 0 0 0 1 0 1 0 2 18
11 0 0 0 0 1 0 1 1 1 19

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 116

Amortized Analysis -
Aggregate Method
Example
Count Increm Tot  The running time of an increment
er [7][6][5][4][3][2][1][0] ent al operation is proportional to the
value cost cos
0 0 0 0 0 0 0 0 0 0t number of bits flipped.
1 0 0 0 0 0 0 0 1 1 1  However, all bits are not flipped
2 0 0 0 0 0 0 1 0 2 3 at each INCREMENT.
3 0 0 0 0 0 0 1 1 1 4  flips at each increment operation;
4 0 0 0 0 0 1 0 0 3 7  flips at alternate increment
5 0 0 0 0 0 1 0 1 1 8
6 0 0 0 0 0 1 1 0 2 10 operations;
7 0 0 0 0 0 1 1 1 1 11  flips only once for 4 successive
8 0 0 0 0 1 0 0 0 4 15 increment operations;
9 0 0 0 0 1 0 0 1 1 16  In general, bit flips times in a
10 0 0 0 0 1 0 1 0 2 18
11 0 0 0 0 1 0 1 1 1 19
sequence of INCREMENTs.

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 117

Aggregate Method
 For (no. of bits) and (counter value) total number of flips of bit can be
given as,
8 8 8 8
𝐴= 0 + 1 + 2 + 3 Counte Number of
2 2 2 2 r value bit flips
0 0000 0

𝐴=8+4+2+1=𝟏𝟓 1
2
0001
0010
1
2
3 0011 1
 s can be 4 0100 3
𝒌 −𝟏 5 0101 1
𝒏
𝑨= ∑ 𝒊
6 0110 2

𝒊=𝟎 𝟐 7
8
0111
1000
1
4

Total Flips = 15

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 118

Aggregate Method
 Therefore, the total number of flips in the sequence is,

K-1

 Total time
 The amortized cost of each operation is

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 119

Amortized Analysis -
Accounting Method
Example
Count Increm Tot  If we charge an amortized cost of to
er [7][6][5][4][3][2][1][0] ent al
value cost cos set a bit to .
0 0 0 0 0 0 0 0 0 0t  When a bit is set we use to pay for the
1 0 0 0 0 0 0 0 1 1 1 actual setting of the bit and we place
2 0 0 0 0 0 0 1 0 2 3 the other on the bit as a credit.
3 0 0 0 0 0 0 1 1 1 4  At any time point, every in the counter
4 0 0 0 0 0 1 0 0 3 7 has a of credit on it.
5 0 0 0 0 0 1 0 1 1 8  So, no need to charge anything to reset
6 0 0 0 0 0 1 1 0 2 10
a bit to
7 0 0 0 0 0 1 1 1 1 11
 As we can pay for the reset with the
8 0 0 0 0 1 0 0 0 4 15
9 0 0 0 0 1 0 0 1 1 16 on it.
10 0 0 0 0 1 0 1 0 2 18  At most one bit is set to 1, in an
11 0 0 0 0 1 0 1 1 1 19 increment operation.
 The amortized cost of an increment
Running time of an increment operation is operation is at most .
proportional to the number of bits flipped.  For increment operations, the total
Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 121
Potential Method
 Potential method represents the prepaid work as potential energy.
 Potential energy can be released to pay for the future operations.
 Initial data structure
 operations, resulting in with costs .
 A potential function is
 is called the potential of
 Amortized cost of the operation is:
(actual cost + potential change)

 If is the total amortized cost of performing operations on data structure

then,

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 122

Potential Method
 Total actual cost of operations can be bounded as,

 Total actual cost is bounded by total

′ amortized cost plus the net drop in
𝑻 = 𝐓 +𝝋 − 𝝋
potential over an entire sequence of operations.
𝟎 𝒏

Rupesh Vaishnav #3150703 (ADA)  Unit 2 – Analysis of Algorithm 123

Thank You!