CS 232

Data Structures
Lecture-04

Department of Computer Science & IT

Sarhad University of Science and Information
Technology, Peshawar
Last
Last Lecture
Lecture Summary
Summary
• Algorithm Specification
• Algorithm Analysis
• Algorithm Performance Calculation
– Empirical Method
– Analytical Method
– Visualization (From Last Lecture)

CS 232: Data Structures 2

Quiz:
Quiz: Find
Find the
the Time
Time Complexity
Complexity
int Sum = 0;
int j;
for (j = 0; j < N; j++)
Sum++;
cout << Sum;
N--;

Solution:
Here, int Sum = 0; taking one operation, for (j = 0; j < N; j++)
& Sum++; taking 3N + 2 operations, cout << Sum; taking 1
operation. Then N--; takes one operation.

So a total of 1 + (3N + 2) + 1 + 1 = 3N + 5.
CS 232: Data Structures 3
Today’s
Today’s Lecture
Lecture
• Algorithm Performance Calculation
– Empirical Method (Discussed Previously)
– Analytical Method (Discussed Previously)
– Visualization
• Running Time Classification
– Best Case
– Worst Case
– Average Case
• Space Efficiency (Space Complexity)
• Algorithm Correctness

CS 232: Data Structures 4

Visualization
Visualization
• For algorithm analysis, visualization techniques are
used to:
– analyze the inner working of an algorithm by tracing the
basic operations.
– observe algorithm steps with animations.
– examine algorithm performance with interactive inputs.
– count primitive operations for analysis.
– make simulations with a variety of data sets.
– report the performance.

CS 232: Data Structures 5

Running
Running Time
Time Classification
Classification
• The running time of an algorithm depends on the input size.
– However, the running time may also be affected
• due to the order in which the data items are input to the algorithm.
• Following are the various running time classification/cases
– Worst cases
– Best cases
– Average Cases
• Let j is the possible orderings of all input data of size n,
t1(n), t2(n),..tj(n) be the running times for each instance.
• The best, worst and average running times are represented as
tbest(n), tworst(n), taverage(n) respectively.

CS 232: Data Structures 6

Running
Running Time
Time Classification
Classification –– Cont…
Cont…
• Notations
– Big Oh Notation (O): Upper bound (Worst Case) i.e.,
maximum time an algorithm may take.
• In computer science, big O notation is used to classify algorithms
by how they respond (e.g., in their processing time or working
space requirements) to changes in input size.
– Omega (Ω): Lower bound (Best Case) i.e., the minimum
time an algorithm may take
– Theta Notation (Θ) : Tighter bound (Average Case)

CS 232: Data Structures 7

Running
Running Time
Time Classification
Classification –– Cont…
Cont…
• Best Case
– The minimum running time of an algorithm i.e., the
minimum number of steps that can be executed for the
given parameters.
– tbest(n) = minimum(t1,t2,..tj); known as an optimistic time.
• Worst Case
– The maximum running time of an algorithm i.e., The
maximum possible number of steps that can be taken.
– tworst(n) = maximum(t1,t2,..tj); known as a pessimistic time.
• Average Case
– The average of running times for all possible ordering of
inputs of the same size.
– taverage(n) = (t1 + t2 + … + tj) / j
CS 232: Data Structures 8
Running
Running Time
Time Classification
Classification –– Cont…
Cont…

• The graph above shows the best, worst, and average running times of an
algorithm.
– Running times for 20 inputs (data items) of same size, but in different order.
• The algorithm takes
– minimum time to process Input #13 , the best time.
– maximum time to process Input #5 , the worst time.
– average time of all the running times for 20 inputs is also shown in the graph.

CS 232: Data Structures 9

Space
Space Efficiency
Efficiency
• Space Efficiency
– Space efficiency determines
• the total memory (RAM and disk storage) required for an
algorithm to run with given input.
– The space requirement consists of
• the amount of real storage needed to run the algorithm code and
• the storage to hold the application data.
– The algorithm code
• occupies a fixed amount of space.
• independent of the input size.
– The storage requirement for the application data
• depends on the nature of data structure used to provide faster and
flexible access to stored information.

CS 232: Data Structures 10

Space
Space Efficiency
Efficiency –– Cont…
Cont…
• For data structures, like an arrays and linked lists
– the space requirement is directly proportional to the input
size.

• Some algorithms require extra storage

– to hold results of intermediate computations, for example,
• Merge sort and
• Dynamic programming techniques

CS 232: Data Structures 11

Space
Space Efficiency
Efficiency –– Cont…
Cont…
• Space complexity is the total amount of memory space used by an
algorithm / program including the space of input values for
execution.

• For space complexity, it is enough to calculate the space occupied

by the variables used in an algorithm / program.

Space Complexity = Auxiliary space + Space use by input values

• Auxiliary space is a temporary or extra space used by the

algorithm during its execution.

CS 232: Data Structures 12

Space
Space Efficiency
Efficiency –– Cont…
Cont…
Example #1: To calculate Space Complexity of an Algorithm:
{
int a = 5, b = 5, c;
c = a + b;
cout<<"sum = "<<c<<endl;
return 0;
}
Output:
• In the above program, 3 integer variables are used. The size of the integer data
type is 2 or 4 bytes which depends on the compiler.
• Now, let’s assume the size as 4 bytes.
• So, the total space occupied by the above-given program is 4 * 3 = 12 bytes.
• Since no additional variables are used, no extra space is required.
• Hence, space complexity for the above-given program is O(1), or constant.

CS 232: Data Structures 13

Space
Space Efficiency
Efficiency –– Cont…
Cont…
• Example #2 Output:
{
int n, i, sum = 0; • In the above-given code, the array
cout<<"Enter array size = "; consists of n integer elements.
cin>>n; • So, the space occupied by the
cout<<"------------------"<<endl; array is 4 * n.
int arr[n]; • Also we have integer variables
for(i = 0; i < n; i++) such as n, i and sum.
{ • Assuming 4 bytes for each
cout<<"Input no. "<<i+1 <<" = "; variable, the total space
cin>>arr[i]; occupied by the program is 4n
sum = sum + arr[i]; + 12 bytes.
} • Since the highest order of n in the
cout<<"------------------"<<endl; equation 4n + 12 is n.
cout<<"sum = "<<sum<<endl;
• So, the space complexity is O(n)
return 0;
or linear.
}

CS 232: Data Structures 14

Algorithm
Algorithm Correctness
Correctness
• Algorithm Correctness - Verifying an algorithm
– A completely new algorithm when designed, its thorough
analysis w.r.t. correctness and efficiency is important.
– Correctness is one of the important aspect of any algorithm
– Correctness reflect that the algorithm is correct. i.e.,
• it always produces the expected output for the range of inputs and
• it eventually terminates.
– The difficult task is to prove that an algorithm is correct.
– No standard methods exist for proving the correctness of a
given algorithm.
– Although, empirical analysis is often used by Programmers
to find errors in an algorithm
• But empirical analysis, unfortunately, is not feasible.

CS 232: Data Structures 15

Algorithm
Algorithm Correctness
Correctness –– Cont…
Cont…
An algorithm / C++ code that finds the maximum value in a list of numbers.

// Example 1 // Example 2
// Expected maxNum = 24 // Expected maxNum = –4
int numbers[4] = {13, 4, 24, int numbers[4] ={-13, -4, -24,
7}; -7};
int maxNum1 = -1; int maxNum2 = -1;
for (int i=0; i<4; i++) { for (int i=0; i<4; i++) {
if (numbers[i] > if (numbers[i] > maxNum2)
maxNum1) maxNum2 =
maxNum1 = numbers[i]; }
numbers[i];//}Example 3: Expected maxNum = -4
cout<<"Max="<<maxNum2;
int numbers[4] ={-13, -4, -24, -7};
cout<<"Max="<<maxNum1
; int maxNum3 = numbers[0];
for (int i=0; i<4; i++) {
if (numbers[i] > maxNum3)
maxNum3 = numbers[i]; }
cout<<"Max="<<maxNum3;
CS 232: Data Structures 16
Summary
Summary
• Algorithm Performance Calculation
– Empirical Method (Discussed Previously)
– Analytical Method (Discussed Previously)
– Visualization
• Running Time Classification
– Best Case
– Worst Case
– Average Case
• Space Efficiency (Space Complexity)
• Algorithm Correctness

CS 232: Data Structures 17

THANK
THANK YOU
YOU

CS 232: Data Structures 18