Design and Analysis of

Algorithms (CS-306)

Lecture No. 2

Dr. Javed Iqbal

 Analysis of Algorithms
◼ An algorithm is a finite set of precise instructions for
performing a computation or for solving a problem.
◼ What is the goal of analysis of algorithms?
◼ To compare algorithms mainly in terms of running time but
also in terms of other factors (e.g., memory requirements,
programmer's effort etc.)
◼ What do we mean by running time analysis?
◼ Determine how running time increases as the size of the
problem increases.

2
 Algorithm Representation

◼ Generally, SW developers represent them in one of three

forms:
◼ Pseudo code
◼ Flowcharts
◼ Actual code

3
 Pseudo Code
◼ Language that is typically used for writing algorithms

◼ Similar to a programming language, but not as rigid

◼ The method of expression most suitable for a given situation

is used:
◼ At times, plain English
◼ At others, a programming language like syntax

4
 Flowchart
◼ A graphical representation of a process (e.g. an algorithm),
in which graphic objects are used to indicate the steps &
decisions that are taken as the process moves along from
start to finish

◼ Individual steps are represented by boxes and other shapes

on the flowchart, with arrows between those shapes
indicating the order in which the steps are taken

5
 Start or stop

Process

Input or output
Flowchart
Elements Decision

Flow line
Connector

Off-page connector
6
 Data Types & Data Structures
◼ Applications/programs read data, store data temporarily,
process it and finally output results.
◼ What is data? Numbers, Characters, etc.

Data Application/ Data

Program

7
 Data Types & Data Structures
◼ Data is classified into data types. e.g. char, float, int,
etc.
◼ A data type is (i) a domain of allowed values and (ii) a
set of operations on these values.
◼ Compiler signals an error if wrong operation is
performed on data of a certain type. For example, char
x,y,z; z = x*y is not allowed.

8
 Data Structures

A data structure is a scheme for

organizing data in the memory of
a computer.

Some of the more commonly

used data structures include lists,
arrays, stacks, queues, heaps,
trees, and graphs.
Binary Tree

9
 Data Structures

The way in which the data is

organized affects the
performance of a program for
different tasks.

Computer programmers decide

which data structures to use
based on the nature of the data
and the processes that need to Binary Tree
be performed on that data.

10
 Efficiency

A solution is said to be efficient if it solves the

problem within its resource constraints.
◼ Space
◼ Time
◼ The cost of a solution is the amount of resources
that the solution consumes.

11
 Costs and Benefits
◼ A data structure requires a certain amount of:
◼ space for each data item it stores
◼ time to perform a single basic operation
◼ programming effort.

12
 Selecting a Data Structure

Select a data structure as follows:

1. Analyze the problem to determine the resource
constraints a solution must meet.
2. Determine the basic operations that must be
supported. Quantify the resource constraints
for each operation.
3. Select the data structure that best meets these
requirements.

13
 Data Structures

Simple data structures that can be

used to organize data, or to create
other data structures:
• Lists
• Arrays

14
 Lists

A list is an ordered set of data. It is often used to store

objects that are to be processed sequentially.

A list can be used

to create a queue.

15
 Arrays

An array is an indexed set of variables, such as

dancer[1], dancer[2], dancer[3],… It is like a set of
boxes that hold things.

A list is a set of items.

An array is a set of
variables that each
store an item.

16
 Arrays and Lists

You can see the difference between arrays and

lists when you delete items.

17
 Arrays and Lists

In a list, the missing spot is filled in when

something is deleted.

18
 Arrays and Lists

In an array, an empty variable is left behind

when something is deleted.

19
 Example: A Queue
A queue is an example of commonly used simple data structure.
A queue has beginning and end, called the front and back of the
queue.

Data enters the queue at one end and leaves at the other.
Because of this, data exits the queue in the same order in which
it enters the queue, like people in a checkout line at a
supermarket.
20
 Example: A Binary Tree

A binary tree is another

commonly used data structure.
It is organized like an upside
down tree.

Each spot on the tree, called a

node, holds an item of data
along with a left pointer and a
right pointer. Binary Tree

21
 Example: A Binary Tree

The pointers are lined up so

that the structure forms the
upside down tree, with a
single node at the top, called
the root node, and branches
increasing on the left and
right as you go down the tree.

Binary Tree

22
 Choosing Data Structures

By comparing the queue with the

binary tree, you can see how the
structure of the data affects what
can be done efficiently with the
data.

23
 Choosing Data Structures

A queue is a good data structure to

use for storing things that need to be
kept in order, such as a set of
documents waiting to be printed on a
network printer.

24
 Choosing Data Structures

The jobs will be printed in the order

in which they are received.

Most network print servers maintain

such a print queue.

25
 Choosing Data Structures

A binary tree is a good data structure

to use for searching sorted data.

The middle item from the list is

stored in the root node, with lesser
items to the left and greater items to
the right.

26
 Choosing Data Structures

A search begins at the root. The

computer either find the data, or
moves left or right, depending on the
value for which you are searching.

Each move down the tree cuts the

remaining data in half.

27
 Choosing Data Structures

Items can be located very quickly in

a tree.
Telephone directory assistance
information is stored in a tree, so that
a name and phone number can be
found quickly.

28
 Choosing Data Structures

For some applications, a queue is the

best data structure to use.
For others, a binary tree is better.
Programmers choose from among
many data structures based on how
the data will be used by the program.

29
 What kinds of problems are solved by
algorithms?
◼ Sorting is by no means the only computational problem for which algorithms have
been developed.
◼ The Human Genome Project has the goals of identifying all the 100,000 genes in
human DNA, determining the sequences of the 3 billion chemical base pairs that
make up human DNA, storing this information in databases, and developing tools
for data analysis. Each of these steps requires sophisticated algorithms.
◼ The Internet enables people all around the world to quickly access and retrieve
large amounts of information. In order to do so, clever algorithms are employed to
manage and manipulate this large volume of data.
◼ Electronic commerce enables goods and services to be negotiated and exchanged
electronically. The ability to keep information such as credit card numbers,
passwords, and bank statements private is essential. Public-key cryptography and
digital signatures are core technologies used and are based on numerical
algorithms and number theory

30
 What kinds of problems are solved by
algorithms?
◼ Searching (Linear and Non-Linear)
◼ Sorting ( Elementary and Advanced)
◼ String Processing ( Pattern Matching, Parsing, Compression,
Cryptography)
◼ Optimization Problem ( Shortest routs, Minimum costs)
◼ Image Processing( Compression, Conversion, Matching)
◼ Data Mining Algorithms ( Clustering, Cleansing, Rules
Mining)
◼ Mathematical Algorithms ( Random Number generator,
Matrix Operation etc. )

31
 What does “size of the input” mean?
◼ If we are searching an array, the “size” of the input
could be the size of the array
◼ If we are merging two arrays, the “size” could be the
sum of the two array sizes
◼ If we are computing the nth Fibonacci number, or the
nth factorial, the “size” is n
◼ We choose the “size” to be the parameter that most
influences the actual time/space required
◼ It is usually obvious what this parameter is
◼ Sometimes we need two or more parameters

32
 Analysis of Algorithms
◼ An algorithm is a finite set of precise instructions for
performing a computation or for solving a problem.
◼ What is the goal of analysis of algorithms?
◼ To compare algorithms mainly in terms of running time but
also in terms of other factors (e.g., memory requirements,
programmer's effort etc.)
◼ What do we mean by running time analysis?
◼ Determine how running time increases as the size of the
problem increases.

33
 Analysis of algorithms
◼ Issues:
◼ correctness
◼ time efficiency
◼ space efficiency
◼ optimality

◼ Approaches:
◼ theoretical analysis
◼ empirical analysis
 Input Size
◼ Input size (number of elements in the input)
◼ size of an array

◼ polynomial degree

◼ # of elements in a matrix

◼ # of bits in the binary representation of the input

◼ vertices and edges in a graph

35
 Theoretical analysis of time efficiency
Time efficiency is analyzed by determining the number
of repetitions of the basic operation as a function of
input size
◼ Basic operation: the operation that contributes the

most towards the running time of the algorithm

input size
T(n) ≈ copC(n)

running time execution time Number of times

for basic operation basic operation is
or cost executed

Note: Different basic operations may cost

 Input size and basic operation examples

Problem Input size measure Basic operation

Searching for key in a list

Number of list’s items, i.e. n Key comparison
of n items

Multiplication of two Matrix dimensions or total Multiplication of two

matrices number of elements numbers

Checking primality of a n’size = number of digits (in

Division
given integer n binary representation)

Visiting a vertex or
Typical graph problem #vertices and/or edges
traversing an edge
 Types of Analysis
◼ Worst case
◼ Provides an upper bound on running time
◼ An absolute guarantee that the algorithm would not run longer, no
matter what the inputs are
◼ Best case
◼ Provides a lower bound on running time
◼ Input is the one for which the algorithm runs the fastest

Lower Bound  Running Time  Upper Bound

◼ Average case
◼ Provides a prediction about the running time
◼ Assumes that the input is random

38
Analyzing Complexity
The End

(for now)

41