[DESIGN AND ANALYSIS OF ALGORITHMS (21CS42] Search Creators..

Module – 1
INTRODUCTION
1.1 Notion of Algorithm
1.2 Review of Asymptotic Notation
1.3 Mathematical Analysis of Non-Recursive and Recursive Algorithms
1.4 Brute Force Approaches: Introduction
1.5 Selection Sort and Bubble Sort
1.6 Sequential Search and Brute Force String Matching.

1.1 Notion of Algorithm

Need for studying algorithms:
The study of algorithms is the cornerstone of computer science. It can be recognized as the
core of computer science. Computer programs would not exist without algorithms. With computers
becoming an essential part of our professional & personal life‘s, studying algorithms becomes a
necessity, more so for computer science engineers. Another reason for studying algorithms is that if we
know a standard set of important algorithms, They further our analytical skills & help us in developing
new algorithms for required applications
Algorithm
An algorithm is finite set of instructions that is followed, accomplishes a particular task. In
addition, all algorithms must satisfy the following criteria:
1. Input. Zero or more quantities are externally supplied.
2. Output. At least one quantity is produced.
3. Definiteness. Each instruction is clear and produced.
4. Finiteness. If we trace out the instruction of an algorithm, then for all cases, the
algorithm terminates after a finite number of steps.
5. Effectiveness. Every instruction must be very basic so that it can be carried out,
in principal, by a person using only pencil and paper. It is not enough that each
operation be definite as in criterion 3; it also must be feasible.

Search Educational Page 1

 [DESIGN AND ANALYSIS OF ALGORITHMS (21CS42] Search Creators..

An algorithm is composed of a finite set of steps, each of which may require one or more op-
erations. The possibility of a computer carrying out these operations necessitates that certain
constraints be placed on the type of operations an algorithm can include. The fourth criterion
for algorithms we assume in this book is that they terminate after a finite number of opera-
tions.

Criterion 5 requires that each operation be effective; each step must be such that it can, at least
in principal, be done by a person using pencil and paper in a finite amount of time. Performing
arithmetic on integers is an example of effective operation, but arithmetic with real numbers is
not, since some values may be expressible only by infinitely long decimal expansion. Adding
two such numbers would violet the effectiveness property.

• Algorithms that are definite and effective are also called computational procedures.
• The same algorithm can be represented in same algorithm can be represented in several ways
• Several algorithms to solve the same problem
• Different ideas different speed
Example:
Problem:GCD of Two numbers m,n
Input specifiastion :Two inputs,nonnegative,not both zero
Euclids algorithm
-gcd(m,n)=gcd(n,m mod n)
Untill m mod n =0,since gcd(m,0) =m

Search Educational Page 2

 [DESIGN AND ANALYSIS OF ALGORITHMS (21CS42] Search Creators..
Another way of representation of the same algorithm
Euclids algorithm

Search Educational Page 3

 [DESIGN AND ANALYSIS OF ALGORITHMS (21CS42] Search Creators..

Step1:if n=0 return val of m & stop else proceed step 2

Step 2:Divide m by n & assign the value of remainder to r
Step 3:Assign the value of n to m,r to n,Go to step1.
Another algorithm to solve the same problem
Euclids algorithm
Step1:Assign the value of min(m,n) to t
Step 2:Divide m by t.if remainder is 0,go to step3 else goto step4
Step 3: Divide n by t.if the remainder is 0,return the value of t as the answer and
stop,otherwise proceed to step4
Step4 :Decrease the value of t by 1. go to step 2

1.1 Review of Asymptotic Notation

Fundamentals of the analysis of algorithm efficiency
• Analysis of algorithms means to investigate an algorithm‘s efficiency with
respect to resources:
• running time ( time efficiency )
• memory space ( space efficiency )
Time being more critical than space, we concentrate on Time efficiency of algorithms.
The theory developed, holds good for space complexity also.
Experimental Studies: requires writing a program implementing the algorithm and
running the program with inputs of varying size and composition. It uses a function, like
the built-in clock() function, to get an accurate measure of the actual running time, then
analysis is done by plotting the results.
Limitations of Experiments
• It is necessary to implement the algorithm, which may be difficult
• Results may not be indicative of the running time on other inputs not included in
the experiment.
• In order to compare two algorithms, the same hardware and software
environments must be used
Theoretical Analysis: It uses a high-level description of the algorithm instead of an

Search Educational Page 4

 [DESIGN AND ANALYSIS OF ALGORITHMS (21CS42] Search Creators..
implementation. Analysis characterizes running time as a function of the input size, n,and
takes into account all possible inputs. This allows us to evaluate the speed of an

Search Educational Page 5

 [DESIGN AND ANALYSIS OF ALGORITHMS (21CS42] Search Creators..

algorithm independent of the hardware/software environment. Therefore theoretical

analysis can be used for analyzing any algorithm

Framework for Analysis

We use a hypothetical model with following assumptions
• Total time taken by the algorithm is given as a function on its input size
• Logical units are identified as one step
• Every step require ONE unit of time
• Total time taken = Total Num. of steps executed
Input’s size: Time required by an algorithm is proportional to size of the problem
instance. For e.g., more time is required to sort 20 elements than what is required to sort 10 elements.
Units for Measuring Running Time: Count the number of times an algorithm‘s basic operation is
executed. (Basic operation: The most important operation of the algorithm, the operation contributing
the most to the total running time.) For e.g., The basic operation is usually the most time-
consuming operation in the algorithm‘s innermost loop.
Consider the following example:

ALGORITHM sum_of_numbers ( A[0… n-1] )

// Functionality : Finds the Sum
// Input : Array of n numbers
// Output : Sum of „n‟numbers
i 0
sum 0
while i < n
sum sum + A[i] n
i i+1
return sum

Total number of steps for basic operation execution, C (n) = n

NOTE:
Constant of fastest growing term is insignificant: Complexity theory is an Approximation

Search Educational Page 6

 [DESIGN AND ANALYSIS OF ALGORITHMS (21CS42] Search Creators..
theory. We are not interested in exact time required by an algorithm to solve the problem. Rather we
are interested in order of growth. i.e How much faster will algorithm run on computer that is twice as

Search Educational Page 7

 [DESIGN AND ANALYSIS OF ALGORITHMS (21CS42] Search Creators..

fast? How much longer does it take to solve problem of double input size? We can crudely estimate
running time by
T (n) ≈ Cop �C(n)
Where,
T (n): running time as a function of n.
Cop : running time of a single operation.
C (n): number of basic operations as a function of n.
Order of Growth: For order of growth, consider only the leading term of a formula and ignore the
constant coefficient. The following is the table of values of several functions important for analysis of
algorithms.

Worst-case, Best-case, Average case efficiencies

Algorithm efficiency depends on the input size n. And for some algorithms efficiency depends
on type of input. We have best, worst & average case efficiencies.

• Worst-case efficiency: Efficiency (number of times the basic operation will be executed) for
the worst case input of size n. i.e. The algorithm runs the longest among all possible inputs of
size n.

• Best-case efficiency: Efficiency (number of times the basic operation will be executed) for the
best case input of size n. i.e. The algorithm runs the fastest among all possible inputs of size n.

• Average-case efficiency: Average time taken (number of times the basic operation will be
executed) to solve all the possible instances (random) of the input. NOTE: NOT the average of
worst and best case

Search Educational Page 8

 [DESIGN AND ANALYSIS OF ALGORITHMS (21CS42] Search Creators..

Asymptotic Notations
Asymptotic notation is a way of comparing functions that ignores constant factors and small input
sizes. Three notations used to compare orders of growth of an algorithm‘s basic operation count are:
O, Ω, Θ notations
Big Oh- O notation
Definition:
A function t(n) is said to be in O(g(n)), denoted t(n)=O(g(n)), if t(n) is bounded above by some
constant multiple of g(n) for all large n, i.e., if there exist some positive constant c and some
nonnegative integer n0 such that
t(n) ≤ cg(n) for all n ≥ n0

Big Omega- Ω notation

Definition:
A function t (n) i s said to be in Ω (g(n)), denoted t(n) = Ω (g (n)), if t (n) is bounded below bysome
constant multiple of g (n) for all large n, i.e., if there exist some positive constant c and some
nonnegative integer n0 such that

t(n) ≥ cg(n) for all n ≥ n0

Search Educational Page 9

 [DESIGN AND ANALYSIS OF ALGORITHMS (21CS42] Search Creators..

Big Theta- Θ notation

Definition:
A function t (n) is said to be in Θ(g (n)), denoted t(n) = Θ(g (n)), if t (n) is bounded bot h above and
below by some constant multiple of g (n) for all large n, i.e., if there exist some positive constant c1 and
c2 and some nonnegative integer n0 such that

c2 g (n) ≤ t (n) ≤ c1 g (n) for all n ≥ n0

Search Educational Page 10

 [DESIGN AND ANALYSIS OF ALGORITHMS (21CS42] Search Creators..

Basic Efficiency classes

The time efficiencies of a large number of algorithms fall into only a few classes.

High time efficiency

1 constant
fast
log n logarithmic
n linear

n log n n log n

n2 quadratic

n3 cubic

2n exponential
n! factorial low time efficiency
slow

1.2 Mathematical Analysis of Non-Recursive and Recursive Algorithms

Mathematical analysis (Time Efficiency) of Non-recursive Algorithms
General plan for analyzing efficiency of non-recursive algorithms:
1. Decide on parameter n indicating input size
2. Identify algorithm‘s basic operation
3. Check whether the number of times the basic operation is executed depends only on the input
size n. If it also depends on the type of input, investigate worst, average, and best case
efficiency separately.
4. Set up summation for C(n) reflecting the number oftimes the algorithm‘s basic operation is
executed.
5. Simplify summation using standard formulas
Example: Finding the largest element in a given array
ALOGORITHM MaxElement(A[0..n-1])
//Determines the value of largest element in a given array
//nput: An array A[0..n-1] of real numbers
//Output: The value of the largest element in A
currentMax ← A[0]

Search Educational Page 11

 [DESIGN AND ANALYSIS OF ALGORITHMS (21CS42] Search Creators..

for i ← 1 to n - 1 do
if A[i] > currentMax

Search Educational Page 12

 [DESIGN AND ANALYSIS OF ALGORITHMS (21CS42] Search Creators..

currentMax ← A[i]
return currentMax
Analysis:
1. Input size: number of elements = n (size of the array)
2. Basic operation:
a) Comparison
b) Assignment
3. NO best, worst, average cases.
4.
Let C (n) denotes number of comparisons: Algorithm makes one comparison on
each execution of the loop, which is repeated for each value of the loop‘s variable
i within the bound between 1 and n – 1.

Example: Element uniqueness problem

Algorithm UniqueElements (A[0..n-1])
//Checks whether all the elements in a given array are distinct
//Input: An array A[0..n-1]
//Output: Returns true if all the elements in A are distinct and false otherwise
for i 0 to n - 2 do
for j i + 1 to n – 1 do
if A[i] = = A[j]
return false
return true

Search Educational Page 13

 [DESIGN AND ANALYSIS OF ALGORITHMS (21CS42] Search Creators..

Analysis
1. Input size: number of elements = n (size of the array)
2. Basic operation: Comparison
3. Best, worst, average cases EXISTS.
Worst case input is an array giving largest comparisons.
• Array with no equal elements
• Array with last two elements are the only pair of equal elements
4. Let C (n) denotes number of comparisons in worst case: Algorithm makes one
comparison for each repetition of the innermost loop i.e., for each value of the
loop‘s variable j between its limits i + 1 and n – 1; and this is repeated for each value of the outer
loop i.e, for each value of the loop‘s variable i between its
limits 0 and n – 2

Search Educational Page 14

 [DESIGN AND ANALYSIS OF ALGORITHMS (21CS42] Search Creators..

Mathematical analysis (Time Efficiency) of recursive Algorithms

General plan for analyzing efficiency of recursive algorithms:
1. Decide on parameter n indicating input size
2. Identify algorithm‘s basic operation
3. Check whether the number of times the basic operation is executed depends only
on the input size n. If it also depends on the type of input, investigate worst,
average, and best case efficiency separately.
4. Set up recurrence relation, with an appropriate initial condition, for the number
of times the algorithm‘s basic operation is executed.
5. Solve the recurrence.

Example: Factorial function

ALGORITHM Factorial (n)
//Computes n! recursively
//Input: A nonnegative integer n

Search Educational Page 15

 [DESIGN AND ANALYSIS OF ALGORITHMS (21CS42] Search Creators..
//Output: The value of n!
if n = = 0

Search Educational Page 16

 [DESIGN AND ANALYSIS OF ALGORITHMS (21CS42] Search Creators..

return 1
else
return Factorial (n – 1) * n
Analysis:
1. Input size: given number = n
2. Basic operation: multiplication
3. NO best, worst, averagecases.
4. Let M (n) denotes number of multiplications.
M (n) = M (n – 1) + 1 for n > 0
M (0) = 0 initial condition
Where: M (n – 1) : to compute Factorial (n – 1)
1 :to multiply Factorial (n – 1) by n
5. Solve the recurrence: Solving using “Backward substitution method”:
M (n) = M (n – 1) + 1
= [ M (n – 2) + 1 ] + 1
= M (n – 2) + 2
= [ M (n – 3) + 1 ] + 3
= M (n – 3) + 3
…
In the ith recursion, we have
= M (n – i ) + i
When i = n, we have
= M (n – n ) + n = M (0 ) + n
Since M (0) = 0
=n

M (n) = Θ (n)

Search Educational Page 17

 [DESIGN AND ANALYSIS OF ALGORITHMS (21CS42] Search Creators..

Example: Find the number of binary digits in the binary representation of a positive
decimal integer
ALGORITHM BinRec (n)
//Input: A positive decimal integer n
//Output: The number of binary digits in n‟sbinary representation
if n = = 1
return 1
else
return BinRec (└ n/2 ┘) + 1

Analysis:
1. Input size: given number = n
2. Basic operation: addition
3. NO best, worst, average cases.
4. Let A (n) denotes number of additions.
A (n) = A (└ n/2 ┘) + 1 for n > 1
A (1) = 0 initial condition
Where: A (└ n/2 ┘) : to compute BinRec (└ n/2 ┘)
1 : to increase the returned value by 1
5. Solve the recurrence:
A (n) = A (└ n/2 ┘) + 1 for n > 1
Assume n = 2k (smoothness rule)
A (2k) = A (2k-1) + 1 for k > 0; A (20) = 0
Solving using “Backward substitution method”:
A (2k) = A (2k-1) + 1
= [A (2k-2) + 1] + 1
= A (2k-2) + 2
= [A (2k-3) + 1] + 2
= A (2k-3) + 3
…
In the ith recursion, we have
Search Educational Page 18
 [DESIGN AND ANALYSIS OF ALGORITHMS (21CS42] Search Creators..

= A (2k-i) + i

Search Educational Page 19

 [DESIGN AND ANALYSIS OF ALGORITHMS (21CS42] Search Creators..

When i = k, we have
= A (2k-k) + k = A (20) + k
Since A (20) = 0
A (2k) = k
Since n = 2k, HENCE k = log2 n
A (n) = log2 n
A (n) = Θ ( log n)

1. 3 Sequential Search.
Sequential Search
ALGORITHM SequentialSearch2(A [0..n], K)
//The algorithm implements sequential search with a search key as a sentinel
//Input: An array A of n elements and a search key K
//Output: The position of the first element in A[0..n - 1] whose value is
// equal to K or -1 if no such element is found
A[n]=K
i=0
while A[i] = K do
i=i + 1
if i < n return i
else return

Search Educational Page 20