SUBJECT NAME :Design and Analysis of Algorithm

SUBJECT CODE : CCS53

SEMESTER : V

UNIT –I

UNIT –I: ALGORITHM AND ANALYSIS

1.Elementary Data Structures

1.1 Stack
1.2Queues
1.3Priority Queue
1.4Trees
1.5Graphs
2.What is an Algorithm?
3. Algorithm Specification
4.Performance Analysis:
4.1 Time Complexity
4.2 Space Complexity
4.3 Asymptotic Notation
4.3.1. Properties of Asymptotic Notation
4.4 Randomized Algorithms.
 1. Elementary of data structure

In this chapter, we examine the representation of dynamic sets by simple data

structures that use pointers. Although many complex data structures can be fashioned
using pointers, we present only the rudimentary ones: stacks, queues, linked lists, and
rooted trees. We also discuss a method by which objects and pointers can be
synthesized from arrays.

Stacks and queues are dynamic sets in which the element removed from the set by
the DELETE operation is prespecified. In a stack, the element deleted from the set is the
one most recently inserted: the stack implements a last-in, first-out, or LIFO, policy.
Similarly, in a queue , the element deleted is always the one that has been in the set
for the longest time: the queue implements a first-in, first-out, or FIFO, policy. There
are several efficient ways to implement stacks and queues on a computer. In this
section we show how to use a simple array to implement each.

1.1Stacks
The INSERT operation on a stack is often called PUSH, and the DELETE operation, which
does not take an element argument, is often called POP. These names are allusions to
physical stacks, such as the spring-loaded stacks of plates used in cafeterias. The order
in which plates are popped from the stack is the reverse of the order in which they
were pushed onto the stack, since only the top plate is accessible.

As shown in Figure 11.1, we can implement a stack of at most n elements with an

array S [1..n]. The array has an attribute top[S] that indexes the most recently inserted
element. The stack consists of elements S[1..top[S]], where S[1] is the element at the
bottom of the stack and S[top[S]] is the element at the top.

When top [S] = 0, the stack contains no elements and is empty. The stack can be tested
for emptiness by the query operation STACK-EMPTY. If an empty stack is popped, we
say the stack underflows, which is normally an error. If top[S] exceeds n, the
stack overflows. (In our pseudocode implementation, we don't worry about stack
overflow.)
1.2Queues
We call the INSERT operation on a queue ENQUEUE, and we call the
DELETE operation DEQUEUE; like the stack operation POP, DEQUEUE takes no element
argument. The FIFO property of a queue causes it to operate like a line of people in
the registrar's office. The queue has a head and a tail. When an element is enqueued,
it takes its place at the tail of the queue, 1ike a newly arriving student takes a place at
the end of the line. The element dequeued is always the one at the head of the queue,
like the student at the head of the line who has waited the longest. (Fortunately, we
don't have to worry about computational elements cutting into line.)

1.3 Priority Queue:

A priority queue is a special type of queue in which each element is associated with a priority value.
And, elements are served on the basis of their priority. That is, higher priority elements are served first.
However, if elements with the same priority occur, they are served according to their order in the queue.

A queue is termed as a priority queue if it has the following characteristics:

 Each item has some priority associated with it.
 An item with the highest priority is moved at the front and deleted first.
 If two elements share the same priority value, then the priority queue follows
the first-in-first-out principle for de queue operation.

A priority queue is of two types:

 Ascending Order Priority Queue

 Descending Order Priority Queue

Ascending Order Priority Queue

An ascending order priority queue gives the highest priority to the lower
number in that queue. For example, you have six numbers in the priority
queue that are 4, 8, 12, 45, 35, 20. Firstly, you will arrange these numbers in
ascending order. The new list is as follows: 4, 8, 12, 20. 35, 45. In this list, 4
is the smallest number. Hence, the ascending order priority queue treats
number 4 as the highest priority.
4 8 12 20 35 45

In the above table, 4 has the highest priority, and 45 has the lowest priority.

Descending Order Priority Queue

A descending order priority queue gives the highest priority to the highest
number in that queue. For example, you have six numbers in the priority
queue that are 4, 8, 12, 45, 35, 20. Firstly, you will arrange these numbers in
ascending order. The new list is as follows: 45, 35, 20, 12, 8, 4. In this list, 45
is the highest number. Hence, the descending order priority queue treats
number 45 as the highest priority.
45 35 20 12 8 4

In the above table, 4 has the lowest priority, and 45 has the highest priority.
Representation of priority queue
Now, we will see how to represent the priority queue through a one-way list.

We will create the priority queue by using the list given below in which INFO list contains the
data elements, PRN list contains the priority numbers of each data element available in
the INFO list, and LINK basically contains the address of the next node.

Let's create the priority queue step by step.

In the case of priority queue, lower priority number is considered the higher priority,
i.e., lower priority number = higher priority.

Step 1: In the list, lower priority number is 1, whose data value is 333, so it will be inserted in
the list as shown in the below diagram:

Step 2: After inserting 333, priority number 2 is having a higher priority, and data values
associated with this priority are 222 and 111. So, this data will be inserted based on the FIFO
principle; therefore 222 will be added first and then 111.
Step 3: After inserting the elements of priority 2, the next higher priority number is 4 and data
elements associated with 4 priority numbers are 444, 555, 777. In this case, elements would be
inserted based on the FIFO principle; therefore, 444 will be added first, then 555, and then 777.

Step 4: After inserting the elements of priority 4, the next higher priority number is 5, and the
value associated with priority 5 is 666, so it will be inserted at the end of the queue.

Implementation of Priority Queue

The priority queue can be implemented in four ways that include arrays, linked list, heap data
structure and binary search tree. The heap data structure is the most efficient way of
implementing the priority queue, so we will implement the priority queue using a heap data
structure in this topic. Now, first we understand the reason why heap is the most efficient way
among all the other data structures.

Analysis of complexities using different implementations

Implementation add Remove peek

Linked list O(1) O(n) O(n)

Binary heap O(logn) O(logn) O(1)

Binary search tree O(logn) O(logn) O(1)

1.4 Trees

Representing rooted trees

The methods for representing lists given in the previous section extend to any
homogeneous data structure. In this section, we look specifically at the problem of
representing rooted trees by linked data structures. We first look at binary trees, and
then we present a method for rooted trees in which nodes can have an arbitrary
number of children.

We represent each node of a tree by an object. As with linked lists, we assume that
each node contains a key field. The remaining fields of interest are pointers to other
nodes, and they vary according to the type of tree.

Binary trees
As shown in Figure 11.9, we use the fields p, left, and right to store pointers to the
parent, left child, and right child of each node in a binary tree T. If p[x] = NIL, then x is
the root. If node x has no left child, then left[x] = NIL, and similarly for the right child.
The root of the entire tree T is pointed to by the attribute root[T]. If root [T] = NIL,
then the tree is empty.
Figure 11.9 The representation of a binary tree T. Each node x has the fields p[x]
(top), left[x] (lower left), and right[x] (lower right). The key fields are not shown.

Rooted trees with unbounded branching

The scheme for representing a binary tree can be extended to any class of trees in
which the number of children of each node is at most some constant k: we replace
the left and right fields by child1, child2, . . . , childk. This scheme no longer works
when the number of children of a node is unbounded, since we do not know how
many fields (arrays in the multiple-array representation) to allocate in advance.
Moreover, even if the number of children k is bounded by a large constant but most
nodes have a small number of children, we may waste a lot of memory.

Fortunately, there is a clever scheme for using binary trees to represent trees with
arbitrary numbers of children. It has the advantage of using only O(n) space for any n-
node rooted tree. The left-child, right-sibling representation is shown in Figure
11.10. As before, each node contains a parent pointer p, and root[T] points to the root
of tree T. Instead of having a pointer to each of its children, however, each node x has
only two pointers:

1. left-child[x] points to the leftmost child of node x, and

2. right-sibling[x] points to the sibling of x immediately to the right.

If node x has no children, then left-child[x] = NIL, and if node x is the rightmost child
of its parent, then right-sibling[x] = NIL.

Other tree representations

We sometimes represent rooted trees in other ways. In Chapter 7, for example, we
represented a heap, which is based on a complete binary tree, by a single array plus an
index. The trees that appear in Chapter 22 are only traversed toward the root, so only
the parent pointers are present; there are no pointers to children. Many other schemes
are possible. Which scheme is best depends on the application.
1.5 Graph:
A graph is a pictorial representation of a set of objects where some pairs of objects are connected
by links. The interconnected objects are represented by points termed as vertices, and the links
that connect the vertices are called edges.
Formally, a graph is a pair of sets (V, E), where V is the set of vertices and E is the set of edges,
connecting the pairs of vertices. Take a look at the following graph −

In the above graph,

V = {a, b, c, d, e}
E = {ab, ac, bd, cd, de}

Graph Data Structure

Mathematical graphs can be represented in data structure. We can represent a graph using an
array of vertices and a two-dimensional array of edges. Before we proceed further, let's
familiarize ourselves with some important terms −
 Vertex − Each node of the graph is represented as a vertex. In the following example, the
labeled circle represents vertices. Thus, A to G are vertices. We can represent them using
an array as shown in the following image. Here A can be identified by index 0. B can be
identified using index 1 and so on.
  Edge − Edge represents a path between two vertices or a line between two vertices. In the
following example, the lines from A to B, B to C, and so on represents edges. We can use
a two-dimensional array to represent an array as shown in the following image. Here AB
can be represented as 1 at row 0, column 1, BC as 1 at row 1, column 2 and so on,
keeping other combinations as 0.
 Adjacency − Two node or vertices are adjacent if they are connected to each other
through an edge. In the following example, B is adjacent to A, C is adjacent to B, and so
on.
 Path − Path represents a sequence of edges between the two vertices. In the following
example, ABCD represents a path from A to D.

Basic Operations
Following are basic primary operations of a Graph −
 Add Vertex − Adds a vertex to the graph.
 Add Edge − Adds an edge between the two vertices of the graph.
 Display Vertex − Displays a vertex of the graph.

2.What is an algorithm:

An algorithm is a set of steps of operations to solve a problem performing calculation, data

processing, and automated reasoning tasks. An algorithm is an efficient method that can be
expressed within finite amount of time and space.
An algorithm is the best way to represent the solution of a particular problem in a very simple
and efficient way. If we have an algorithm for a specific problem, then we can implement it in
any programming language, meaning that the algorithm is independent from any
programming languages.

Characteristics of an Algorithm
Not all procedures can be called an algorithm. An algorithm should have the following
characteristics −
 Unambiguous − Algorithm should be clear and unambiguous. Each of its steps (or
phases), and their inputs/outputs should be clear and must lead to only one meaning.
 Input − An algorithm should have 0 or more well-defined inputs.
  Output − An algorithm should have 1 or more well-defined outputs, and should match
the desired output.
 Finiteness − Algorithms must terminate after a finite number of steps.
 Feasibility − Should be feasible with the available resources.
 Independent − An algorithm should have step-by-step directions, which should be
independent of any programming code.

3.ALGORITHM SPECIFICATION:
4.PERFORMANCE ANALYSIS

4.1 Time Complexity

Time complexity of an algorithm represents the amount of time required by the algorithm to run
to completion. Time requirements can be defined as a numerical function T(n), where T(n) can
be measured as the number of steps, provided each step consumes constant time.

For example, addition of two n-bit integers takes n steps.

Consequently, the total computational time is T(n) = c ∗ n,
where c is the time taken for the addition of two bits. Here,
we observe that T(n) grows linearly as the input size
increases exity
Suppose X is an algorithm and n is the size of input data, the time and space used by the
algorithm X are the two main factors, which decide the efficiency of X.
 Time Factor − Time is measured by counting the number of key operations such as
comparisons in the sorting algorithm.
 Space Factor − Space is measured by counting the maximum memory space required by
the algorithm.
The complexity of an algorithm f(n) gives the running time and/or the storage space required by
the algorithm in terms of n as the size of input data.

In this chapter, we will discuss the complexity of computational problems with respect to the
amount of space an algorithm requires.
Space complexity shares many of the features of time complexity and serves as a further way of
classifying problems according to their computational difficulties.
What are the Different Types of Time complexity
Notation Used?
As we have seen, Time complexity is given by time as a function of the length of the input.
And, there exists a relation between the input data size (n) and the number of operations
performed (N) with respect to time. This relation is denoted as Order of growth in Time
complexity and given notation O[n] where O is the order of growth and n is the length of the
input. It is also called as ‘Big O Notation’

Big O Notation expresses the run time of an algorithm in terms of how quickly it grows
relative to the input ‘n’ by defining the N number of operations that are done on it. Thus, the
time complexity of an algorithm is denoted by the combination of all O[n] assigned for each
line of function.

There are different types of time complexities used, let’s see one by one:

1. Constant time – O (1)

2. Linear time – O (n)

3. Logarithmic time – O (log n)

4. Quadratic time – O (n^2)

5. Cubic time – O (n^3)

and many more complex notations like Exponential time, Quasilinear time, factorial time,
etc. are used based on the type of functions defined.
4.2 Space Complexity
Space complexity is a function describing the amount of memory (space) an algorithm takes in
terms of the amount of input to the algorithm.
We often speak of extra memory needed, not counting the memory needed to store the input
itself. Again, we use natural (but fixed-length) units to measure this.
We can use bytes, but it's easier to use, say, the number of integers used, the number of fixed-
sized structures, etc.

In the end, the function we come up with will be independent of the actual number of bytes
needed to represent the unit.
Space complexity is sometimes ignored because the space used is minimal and/or obvious,
however sometimes it becomes as important issue as time complexity

Space complexity is the total amount of memory space used by an

algorithm/program including the space of input values for execution. So
to find space-complexity, it is enough to calculate the space occupied by the
variables used in an algorithm/program.

But often, people confuse Space-complexity with Auxiliary space. Auxiliary

space is just a temporary or extra space and it is not the same as space-
complexity. In simpler terms,
Space Complexity = Auxiliary space + Space use by input values

Important Note: The best algorithm/program should have the lease space-
complexity. The lesser the space used, the faster it executes.

How to calculate Space Complexity of an Algorithm?

Let us understand the Space-Complexity calculation through examples.

Example #1

#include<stdio.h>
int main()
{
int a = 5, b = 5, c;
c = a + b;
printf("%d", c);
}

In the above program, 3 integer variables are used. The size of the integer
data type is 2 or 4 bytes which depends on the compiler. Now, lets assume
the size as 4 bytes. So, the total space occupied by the above-given program
is 4 * 3 = 12 bytes. Since no additional variables are used, no extra space is
required.

4.3 Asymptotic natations:

Asymptotic notations are the mathematical notations used to describe the

running time of an algorithm when the input tends towards a particular value
or a limiting value. For example: In bubble sort, when the input array is already
sorted, the time taken by the algorithm is linear i.e. the best case.

The following 3 asymptotic notations are mostly used to represent the time complexity
of algorithms:
1. Big-O Notation (O-notation)
2. Omega Notation (Ω-notation)
3. Theta Notation (Θ-notation)

1) Θ Notation:
Theta notation encloses the function from above and below. Since it represents the
upper and the lower bound of the running time of an algorithm, it is used for analyzing
the average-case complexity of an algorithm.
Let g and f be the function from the set of natural numbers to itself. The function f is
said to be Θ(g), if there are constants c1, c2 > 0 and a natural number n0 such that c1*
g(n) ≤ f(n) ≤ c2 * g(n) for all n ≥ n0

Theta notation

Mathematical Representation of Theta notation:

Θ (g(n)) = {f(n): there exist positive constants c1, c2 and n0 such that 0 ≤ c1 * g(n) ≤ f(n) ≤ c2
* g(n) for all n ≥ n0}
Note: Θ(g) is a set
The above definition means, if f(n) is theta of g(n), then the value f(n) is always
between c1 * g(n) and c2 * g(n) for large values of n (n ≥ n0). The definition of theta
also requires that f(n) must be non-negative for values of n greater than n0.
A simple way to get the Theta notation of an expression is to drop low-order terms and
ignore leading constants.

2) Big O Notation:
Big-O notation represents the upper bound of the running time of an algorithm.
Therefore, it gives the worst-case complexity of an algorithm.
If f(n) describes the running time of an algorithm; f(n) is O(g(n)) if there exist positive
constant C and n0 such that, 0 ≤ f(n) ≤ cg(n) for all n ≥ n0

Mathematical Representation of Big O Notation:

O(g(n)) = { f(n): there exist positive constants c and n0
such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0 }
For example, Consider the case of Insertion Sort. It takes linear time in the best case
and quadratic time in the worst case. We can safely say that the time complexity of the
Insertion sort is O(n 2). Note that O(n 2) also covers linear time.
If we use Θ notation to represent the time complexity of Insertion sort, we have to use
two statements for best and worst cases:
1. The worst-case time complexity of Insertion Sort is Θ(n 2).
2. The best case time complexity of Insertion Sort is Θ(n).
The Big O notation is useful when we only have an upper bound on the time complexity
of an algorithm. Many times we easily find an upper bound by simply looking at the
algorithm.
3) Ω Notation:
Omega notation represents the lower bound of the running time of an algorithm. Thus,
it provides the best case complexity of an algorithm.
Let g and f be the function from the set of natural numbers to itself. The function f is
said to be Ω(g), if there is a constant c > 0 and a natural number n0 such that c*g(n) ≤
f(n) for all n ≥ n0

Mathematical Representation of Omega notation :

Ω(g(n)) = { f(n): there exist positive constants c and n0
such that 0 ≤ cg(n) ≤ f(n) for all n ≥ n0 }
Let us consider the same Insertion sort example here. The time complexity of Insertion
Sort can be written as Ω(n), but it is not very useful information about insertion sort, as
we are generally interested in worst-case and sometimes in the average case.

4.3.1 Properties of Asymptotic Notations :

As we have gone through the definition of these three notations let’s now discuss some
important properties of those notations.
1. General Properties:
If f(n) is O(g(n)) then a*f(n) is also O(g(n)), where a is a constant.
Example:
f(n) = 2n²+5 is O(n²)
then, 7*f(n) = 7(2n²+5) = 14n²+35 is also O(n²).
Similarly, this property satisfies both Θ and Ω notation.
We can say,
If f(n) is Θ(g(n)) then a*f(n) is also Θ(g(n)), where a is a constant.
If f(n) is Ω (g(n)) then a*f(n) is also Ω (g(n)), where a is a constant.
2. Transitive Properties:
If f(n) is O(g(n)) and g(n) is O(h(n)) then f(n) = O(h(n)).
Example:
If f(n) = n, g(n) = n² and h(n)=n³
n is O(n²) and n² is O(n³) then, n is O(n³)
Similarly, this property satisfies both Θ and Ω notation.
We can say,
If f(n) is Θ(g(n)) and g(n) is Θ(h(n)) then f(n) = Θ(h(n)) .
If f(n) is Ω (g(n)) and g(n) is Ω (h(n)) then f(n) = Ω (h(n))
3. Reflexive Properties:
Reflexive properties are always easy to understand after transitive.
If f(n) is given then f(n) is O(f(n)). Since MAXIMUM VALUE OF f(n) will be f(n)
ITSELF!
Hence x = f(n) and y = O(f(n) tie themselves in reflexive relation always.
Example:
f(n) = n² ; O(n²) i.e O(f(n))
Similarly, this property satisfies both Θ and Ω notation.
We can say that,
If f(n) is given then f(n) is Θ(f(n)).
If f(n) is given then f(n) is Ω (f(n)).
4. Symmetric Properties:
If f(n) is Θ(g(n)) then g(n) is Θ(f(n)).
Example:
If(n) = n² and g(n) = n²
then, f(n) = Θ(n²) and g(n) = Θ(n²)
This property only satisfies for Θ notation.
5. Transpose Symmetric Properties:
If f(n) is O(g(n)) then g(n) is Ω (f(n)).
Example:
If(n) = n , g(n) = n²
then n is O(n²) and n² is Ω (n)
This property only satisfies O and Ω notations.
6. Some More Properties:
1) If f(n) = O(g(n)) and f(n) = Ω(g(n)) then f(n) = Θ(g(n))
2) If f(n) = O(g(n)) and d(n)=O(e(n))
then f(n) + d(n) = O( max( g(n), e(n) ))
Example:
f(n) = n i.e O(n)
d(n) = n² i.e O(n²)
then f(n) + d(n) = n + n² i.e O(n²)
3) If f(n)=O(g(n)) and d(n)=O(e(n))
then f(n) * d(n) = O( g(n) * e(n) )
Example:
f(n) = n i.e O(n)
d(n) = n² i.e O(n²)
then f(n) * d(n) = n * n² = n³ i.e O(n³)
4.4 Randomized Algorithm

A Randomized Algorithm is an algorithm that employs a degree of randomness as

part of its logic. The algorithm typically uses uniformly random bits as an auxiliary input
to guide its behavior, in the hope of achieving good performance in the "average case"
over all possible choices of random bits.
Algorithm Complexity
 A fixed part that is a space required to store certain data and variables, that are
independent of the size of the problem. For example, simple variables and constants used,
program size, etc.
 A variable part is a space required by variables, whose size depends on the size of the
problem. For example, dynamic memory allocation, recursion stack space, etc.
Space complexity S(P) of any algorithm P is S(P) = C + SP(I), where C is the fixed part and S(I)
is the variable part of the algorithm, which depends on instance characteristic I. Following is a
simple example that tries to explain the concept −
Algorithm: SUM(A, B)
Step 1 - START
Step 2 - C ← A + B + 10
Step 3 - Stop
Here we have three variables A, B, and C and one constant. Hence S(P) = 1 + 3. Now, space
depends on data types of given variables and constant types and it will be multiplied accordingly.