B.

TECH V SEM CSE

ACADEMIC YEAR: 2023-2024

Course Name: Design and Analysis of Algorithm

Topic: Time & Space Complexity

Course code : CS 3102

Credits : 4
Mode of delivery : Physical
Faculty : Mr. Tarun Jain
Email-id : [email protected]
1
 Analysis of Algorithms
• Analysis is performed with respect to a
computational model
• We will usually use a generic uni-
processor random-access machine (RAM)
• All memory equally expensive to access
• No concurrent operations
• All reasonable instructions take unit time
• Except, of course, function calls
• Constant word size
• Unless we are explicitly manipulating bits
CS3102 (DAA), Dept. of CSE
 Analysis of Algorithms
 Analysis of efficiency of an algorithm can be performed at two different
stages, before implementation and after implementation, as

 A priori analysis − This is defined as theoretical analysis of an

algorithm. Efficiency of algorithm is measured by assuming that all
other factors e.g. speed of processor, are constant and have no
effect on implementation.

• A posterior analysis − This is defined as empirical analysis of an

algorithm. The chosen algorithm is implemented using
programming language. Next the chosen algorithm is executed on
target computer machine. In this analysis, actual statistics like
running time and space needed are collected.

 Algorithm analysis is dealt with the execution or running time of various

operations involved. Running time of an operation can be defined as
number of computer instructions executed per operation.

CS3102 (DAA), Dept. of CSE

 Input Size
• Time and space complexity
• This is generally a function of the input size
• E.g., sorting
• How we characterize input size depends:
• Sorting: number of input items
• Graph algorithms: number of nodes & edges
• Etc

CS3102 (DAA), Dept. of CSE

 Running Time
• Number of primitive steps that are
executed
• Except for time of executing a function call
most statements roughly require the
same amount of time
• y=m*x+b
• c = 5 / 9 * (t - 32 )
• z = f(x) + g(y)
• We can be more exact if need be
CS3102 (DAA), Dept. of CSE
 Analysis
• Worst case
• Provides an upper bound on running time
• An absolute guarantee
• Average case
• Provides the expected running time
• Very useful, but treat with care: what is
“average”?
• Random (equally likely) inputs
• Real-life inputs

CS3102 (DAA), Dept. of CSE

 Analysis of Algorithms
• How good is the algorithm?
• Correctness
• Time efficiency
• Space efficiency

CS3102 (DAA), Dept. of CSE

 What is an algorithm?
• Recipe, process, method, technique,
procedure, routine,… with following
requirements:
1. Finiteness
terminates after a finite number of steps
2. Definiteness
rigorously and unambiguously specified
3. Input
valid inputs are clearly specified
4. Output
can be proved to produce the correct output given a
valid input
5. Effectiveness
CS3102 (DAA), Dept. of CSE
steps are sufficiently simple and basic
Algorithm Design and Analysis
Process
Understand the problem

Decide on : algorithm
design techniques etc.

Design an algorithm

CS3102 (DAA), Dept. of CSE

Algorithm Design and Analysis
Process
Understand the problem

Decide on : algorithm
design
designtechniques
techniques
etc.
Design an algorithm

Prove correctness

Analyze efficiency etc.

CS3102 (DAA), Dept. of CSE

Algorithm Design and Analysis
Process
Understand the problem

Decide on : algorithm
design
designtechniques
techniques
etc.
Design an algorithm

Prove correctness

Analyze efficiency etc

Code the algorithm correctness

CS3102 (DAA), Dept. of CSE
efficiency
 Complexity
 Suppose X is treated as an algorithm and N is treated as the size of input
data, the time and space implemented by the Algorithm X are the two main
factors which determine the efficiency of X.

 Time Factor − The time is calculated or measured by counting the number

of key operations such as comparisons in sorting algorithm.
 Space Factor − The space is calculated or measured by counting the
maximum memory space required by the algorithm.

 The complexity of an algorithm f(N) provides the running time and / or storage
space needed by the algorithm with respect of N as the size of input data.

CS3102 (DAA), Dept. of CSE

 Complexity

Time complexity:
– How much time it takes to compute
– Measured by a function T(N)

Space complexity:
– How much memory it takes to compute
– Measured by a function S(N)

CS3102 (DAA), Dept. of CSE

 Time complexity
• N = Size of the input
• T(N) = Time complexity function
• Order of magnitude:
– How rapidly T(N) grows when N grows
– For example: O(N) O(logN) O(N²)
O(2N)

CS3102 (DAA), Dept. of CSE

Asymptotic analysis

CS3102 (DAA), Dept. of CSE

 Space Complexity
• Space complexity = The amount of memory required
by an algorithm to run to completion
• [Core dumps = the most often encountered cause is
“memory leaks” – the amount of memory required
larger than the memory available on a given system]
• Some algorithms may be more efficient if data
completely loaded into memory
• Need to look also at system limitations
• E.g. Classify 2GB of text in various categories [politics,
tourism, sport, natural disasters, etc.] – can I afford to
load the entire collection?

CS3102 (DAA), Dept. of CSE

 Space Complexity (cont’d)
• S(P) = c + S(instance characteristics)
• c = constant
• Example:
SUM(P, Q)
Step 1 - START
Step 2 - R ← P + Q + 10
Step 3 – Stop

Here we have three variables P, Q and R and one constant.

Hence S(p) = 1+3. Now space is dependent on data types of
given constant types and variables and it will be multiplied
accordingly.
CS3102 (DAA), Dept. of CSE
 Analysis of Algorithms
• Efficiency of an algorithm can be
measured in terms of:
• Execution time (time complexity)
• The amount of memory required (space
complexity)
• Which measure is more important?
• Answer often depends on the limitations of
the technology available at time of
analysis CS3102 (DAA), Dept. of CSE
 Time Complexity
• For most of the algorithms associated
with this course, time complexity
comparisons are more interesting than
space complexity comparisons
• Time complexity: A measure of the
amount of time required to execute an
algorithm

CS3102 (DAA), Dept. of CSE

 Time Complexity
• Factors that should not affect time
complexity analysis:
• The programming language chosen to
implement the algorithm
• The quality of the compiler
• The speed of the computer on which the
algorithm is to be executed

CS3102 (DAA), Dept. of CSE

 Time Complexity
• Time complexity analysis for an
algorithm is independent of
programming language,machine used
• Objectives of time complexity
analysis:
• To determine the feasibility of an algorithm
by estimating an upper bound on the
amount of work performed
• To compare different algorithms before
deciding onCS3102
which one to implement
(DAA), Dept. of CSE
 Time Complexity
• Analysis is based on the amount of
work done by the algorithm
• Time complexity expresses the
relationship between the size of the
input and the run time for the algorithm
• Usually expressed as a proportionality,
rather than an exact function

CS3102 (DAA), Dept. of CSE

 Time Complexity
• To simplify analysis, we sometimes
ignore work that takes a constant
amount of time, independent of the
problem input size
• When comparing two algorithms that
perform the same task, we often just
concentrate on the differences between
algorithms
CS3102 (DAA), Dept. of CSE
 Time Complexity
• Simplified analysis can be based on:
• Number of arithmetic operations performed
• Number of comparisons made
• Number of times through a critical loop
• Number of array elements accessed
• etc

CS3102 (DAA), Dept. of CSE

Worst Case vs. Average Case
• Worst case analysis is used to find an
upper bound on algorithm performance
for large problems (large n)
• Average case analysis determines the
average (or expected) performance
• Worst case time complexity is usually
simpler to work out

CS3102 (DAA), Dept. of CSE

 Big-O Analysis in General
• With independent nested loops: The
number of iterations of the inner loop is
independent of the number of
iterations of the outer loop
• Example:
Outer loop executes n/2
int x = 0;
times. For each of those
for ( int j = 1; j <= n/2; j++ ) times, inner loop executes n2
times, so the body of the
for ( int k = 1; k <= n*n; k++ )
inner loop is executed
x = x + j + k; (n/2)*n2 = n3/2 times. The
algorithm
CS3102 (DAA), Dept. of CSE is O(n3) .
 Big-O Analysis in General
• With independent nested loops: The
number of iterations of the inner loop is
independent of the number of
iterations of the outer Ifloop
you notice, j keeps doubling till it is

• Example: less than or equal to n. Number of times,

we can double a number till it is less than
n would be log(n).
int i, j, k = 0; Let’s take the examples here.
for (i = n / 2; i <= n; i++) { for n = 16, j = 2, 4, 8, 16
for (j = 2; j <= n; j = j * 2) { for n = 32, j = 2, 4, 8, 16, 32
So, j would run for O(log n) steps.
k = k + n / 2;
i runs for n/2 steps.
} So, total steps = O(n/ 2 * log (n)) =
} O(n*logn)
CS3102 (DAA), Dept. of CSE
 Big-O Analysis in General
• Example:
int a = 0, i = N;
while (i > 0) {
a += i;
i /= 2;
}

Complexity = O(logN)

CS3102 (DAA), Dept. of CSE

 Big-O Analysis in General
• With dependent nested loops: Number
of iterations of the inner loop depends
on a value from the outer loop
• Example:
When j is 1, inner loop executes 3
int x = 0; times; when j is 2, inner loop executes
for ( int j = 1; j <= n; j++ ) 3*2 times; … when j is n, inner loop
executes 3*n times. In all the inner
for ( int k = 1; k < 3*j; k+ loop executes 3+6+9+…+3n =
+ ) x = x + j; 3(1+2+3+…+n) = 3n2/2 + 3n/2 times.
The algorithm is O(n2).
CS3102 (DAA), Dept. of CSE
 Big-O Analysis in General
• With dependent nested loops: Number
of iterations of the inner loop depends
on a value from the outer loop
• Example:
int a = 0; The above code runs total no of times
for (i = 0; i < N; i++) { = N + (N – 1) + (N – 2) + … 1 + 0
for (j = N; j > i; j--) { = N * (N + 1) / 2
a = a + i + j; = 1/2 * N^2 + 1/2 * N
} O(N^2) times.
}

CS3102 (DAA), Dept. of CSE

 Big-O Analysis in General
Assume that a computer executes a million instructions a second.
This chart summarizes the amount of time required to execute
f(n) instructions on this machine for various values of n.

f(n) n=103 n=105 n=106

log2(n) 10-5 sec 1.7 * 10-5 sec 2 * 10-5 sec
n 10-3 sec 0.1 sec 1 sec
n*log2(n) 0.01 sec 1.7 sec 20 sec
n2 1 sec 3 hr 12 days
n3 17 min 32 yr 317 centuries
2n 10285 centuries 1010000 years 10100000 years

CS3102 (DAA), Dept. of CSE

 Big-O Analysis in General
• Examples of dominant terms:
n dominates log2(n)
n*log2(n) dominates n
n2 dominates n*log2(n)
nm dominates nk when m > k
an dominates nm for any a > 1 and m >= 0
• That is, log2(n) < n < n*log2(n) < n2 < …
< nm < an for a >= 1 and m > 2
CS3102 (DAA), Dept. of CSE
 Intractable problems
• A problem is said to be intractable if
solving it by computer is
impractical
• Example: Algorithms with time
complexity O(2n) take too long to solve
even for moderate values of n; a
machine that executes 100 million
instructions per second can execute 260
instructions in about
CS3102 365
(DAA), Dept. of CSEyears
 Constant Time Complexity
• Algorithms whose solutions are independent
of the size of the problem’s inputs are said
to have constant time complexity

• Constant time complexity is denoted as

O(1)

CS3102 (DAA), Dept. of CSE

Basic Search Algorithms and
their Complexity Analysis

CS3102 (DAA), Dept. of CSE

 Linear Search: Example 1
• The problem: Search an array a of size n to
determine whether the array contains the
value key; return index if found, -1 if not found

Set k to 0.
While (k < n) and (a[k] is not
key) Add 1 to k.
If k == n Return –
1. Return k.

CS3102 (DAA), Dept. of CSE

 Analysis of Linear Search
• Total amount of work done:
• Before loop: a constant amount a
• Each time through loop: 2 comparisons, an
and operation, and an addition: a constant
amount of work b
• After loop: a constant amount c
• In worst case, we examine all n array
locations, so T(n) = a +b*n + c = b*n + d,
where d = a+c, and time complexity is O(n)

CS3102 (DAA), Dept. of CSE

 Analysis of Linear Search
• Simpler (less formal) analysis:
• Note that work done before and after loop is
independent of n, and work done during a
single execution of loop is independent of n
• In worst case, loop will be executed n times,
so amount of work done is proportional to
n, and algorithm is O(n)

CS3102 (DAA), Dept. of CSE

 Binary Search
(on sorted arrays)

• General case: search a sorted array a

of size n looking for the value key
• Divide and conquer approach:
• Compute the middle index mid of the
array
• If key is found at mid, we’re done
• Otherwise repeat the approach on the half
of the array that might still contain key
• etc… CS3102 (DAA), Dept. of CSE
Example: Binary Search For Ordered List
int binarySearch(int low, int high, int key)
{
while(low<=high)
{
int mid=(low + high) / 2;
if(a[mid]<key)
{
low=mid+1;
}
else if(a[mid]>key)
{
high=mid-1;
}
else
{
return mid;
}
}
return -1; //key not found
}
CS3102 (DAA), Dept. of CSE
 Analysis of Binary Search
• The amount of work done before and
after the loop is a constant, and
independent of n
• The amount of work done during a
single execution of the loop is constant
• Time complexity will therefore be
proportional to number of times the loop
is executed, so that’s what we’ll
analyze
CS3102 (DAA), Dept. of CSE
 Analysis of Binary Search
• Worst case: key is not found in the array
• Each time through the loop, at least half
of the remaining locations are rejected:
• After first time through, <= n/2 remain
• After second time through, <= n/4 remain
• After third time through, <= n/8 remain
• After kth time through, <= n/2k remain

CS3102 (DAA), Dept. of CSE

Analysis of Binary Search
In general, we can write the size
of the problem as n/2i after doing
i comparisons. Thus, for the base
case,

n/2i =1

Þ n= 2i

=> i=log2n

Also, the comparison will reach

the base case only in the worst
case and thus, binary search is a
O(logn) algorithm.
CS3102 (DAA), Dept. of CSE