Data Structures &

Algorithms
Adil M. Khan

Professor of Computer Science

Innopolis University

Lecture 2
Which Phone Will You
Buy, and Why?
Which Car Will You Buy,
and Why?
 Which Sorting Algorithm
Will You Choose and
• Insertion Sort Why?

• Selection Sort

• Merge Sort

• Quick Sort
 Today’s Objectives
• Learn and be able to describe what is ―Algorithm
Analysis‖

• Learn and be able to describe the importance of

analyzing algorithms

• Learn and practice the method to analyze algorithms

• Learn and be able to explain the methods of

expressing an algorithm’s complexity
 Recap
• What is an ―Algorithm‖?

• What is a ‖data structures‖?

• Why is it important to study them?

What is ―Algorithm
Analysis‖?
 Algorithm Analysis
• Analyzing how resource requirements of an
algorithm will scale when increasing the input size
Why analyze
algorithms?
 Two Characteristics of
Algorithmic Problems
• They have practical applications

• They have many candidate solutions

 Algorithm Analysis
• Allows us to:

• Compare the merits of two

alternative approaches to a
problem we need to solve

• Determine whether a
proposed solution will meet
required resource
constraints before we invest
money and time coding

Performed before coding!

How to analyze
algorithms?
 Time Complexity
• As discussed in the last week, we will focus on time
complexity

• That is, we will analyze how

much time does
an algorithm take to run to its completion

• There are Two ways with which we can do this!

 Experimental Analysis
• Write a program that implements
the algorithm

• Run it with inputs of varying size

and composition

• Measure the actual running time

• Plot the results

What is wrong with this approach?

 Pseudocode
• A high-level description of an
algorithm

• More structured than English

prose

• Less detailed than a

program
Theoretical Analysis
 Wait a Second, Please!!
• Even for inputs of the same size, the time consumed
can be very different

Example: an algorithm that finds the first prime

number in an array by scanning it left to right

Can different situations affect the running time of

this algorithm?
 Measuring Time Complexity
• Analyze running time for the

• best case: usually useless

• average case: very difficult to determine

• worst case: a safer choice

Why is the worst case a safer choice?

Choosing a Roommate

You can see their past (to judge their behavior) but you have to choose

(1) best case: a day when everything was great

(2) average case: An average day (but it is hard to determine such a day)
(3) worst case: a day when everything was horrible
 Thus, we analyze worst
case Time Complexity T(n)
of algorithms
 Execution Time
• Tied to the underline machine and compiler

• To simplify this, we use the RAM model

1. Each simple operation (+, *, -, =, if, call) takes

exactly one step
2. Loops and subroutines are not considered
simple operations
3. Each memory access takes exactly one time
stamp
 Measuring Time Complexity

• Total time taken by each statement is approximately

the product of execution time
(represented as constants) and the frequency
count
Example
 Example (cont.)
You already know
 Example (cont.)
We know that

Thus
 Example (cont.)
This can be expressed as

Complexity of Insertion Sort as a function of its

input, independent of underlying platform,
hardware, programming language and so on.

But what is it telling us?

 Time Complexity
• Time complexities of algorithms when expressed in the
form of numerical functions over the size of the input are
difficult to work with:

 Have too many bumps

 Require too much detail to specify

• Thus, to make analysis easier, we talk about upper and

lower bounds of these functions – Big O Analysis

• This helps us to ignore details that do not impact our

comparison of algorithms
 Russian Winters
• Knowing that

 the whole world

knows that how cold
Russia can be in the
winters,

 allows us to express
severity of winters of
any place in terms of
Russian winters
Big O Analysis
Big Oh Analysis
 Big Oh Analysis

• So for our example:

• But we just learned that constant terms and lower

order terms don’t matter

• Thus, under Big Oh analysis, we can express it as

Big Oh Notations
What do you think?
 Properties
• Transitivity

• Reflexivity

• Symmetry

• Transpose Symmetry
Growth Rates of Common
Functions
Growth Rates of Common
Functions
 Space Complexity
• Determine how much space an algorithm requires by
analyzing its storage requirements as a function of the
input size

• Example:

• Let’s say, our algorithm reads a stream of n

characters

• But always stores a constant number of them

• then, its space complexity is O(1)

 Space Complexity
• Another Example:

• Let’s say, our algorithm reads a stream of n

characters

• and stores all of them

• then, its space complexity is O(n)

 Space Complexity
• Exercise:

• Let’s say, our algorithm reads a stream of n

characters

• and stores all of them, and each record results in

the creation of a constant number of other records

• then, its space complexity is ?

 Space Complexity
• Another Exercise:

• Let’s say, our algorithm reads a stream of n

characters

• and stores all of them, and each record results in

the creation of a number of new records — the
number is proportional to the size of the data

• then, its space complexity is ?

 Time-Space Tradeoff
• Generally, decreasing the time complexity of an
algorithm results in increasing its space complexity
— and vice versa

• This is called the time-space tradeoff

• Example: Storing a sparse matrix as a two-

dimensional linked list vs. a two-dimensional array
 Did we Achieve Today’s
Objectives?
• Learn and be able to describe what is ―Algorithm
Analysis‖

• Learn and be able to describe the importance of

analyzing algorithms

• Learn and practice the method to analyze algorithms

• Learn and be able to explain the methods of

expressing an algorithm’s complexity