What is an Algorithm?

An algorithm is a process or a set of rules required to perform calculations or

some other problem-solving operations especially by a computer. The formal
definition of an algorithm is that it contains the finite set of instructions
which are being carried in a specific order to perform the specific task. It is
not the complete program or code; it is just a solution (logic) of a problem,
which can be represented either as an informal description using a Flowchart
or Pseudocode.

Characteristics of an Algorithm
The following are the characteristics of an algorithm:

o Input: An algorithm has some input values. We can pass 0 or some input
value to an algorithm.
o Output: We will get 1 or more output at the end of an algorithm.
o Unambiguity: An algorithm should be unambiguous which means that the
instructions in an algorithm should be clear and simple.
o Finiteness: An algorithm should have finiteness. Here, finiteness means that
the algorithm should contain a limited number of instructions, i.e., the
instructions should be countable.
o Effectiveness: An algorithm should be effective as each instruction in an
algorithm affects the overall process.
o Language independent: An algorithm must be language-independent so
that the instructions in an algorithm can be implemented in any of the
languages with the same output.

Dataflow of an Algorithm

o Problem: A problem can be a real-world problem or any instance from the

real-world problem for which we need to create a program or the set of
instructions. The set of instructions is known as an algorithm.
o Algorithm: An algorithm will be designed for a problem which is a step by
step procedure.
 o Input: After designing an algorithm, the required and the desired inputs are
provided to the algorithm.
o Processing unit: The input will be given to the processing unit, and the
processing unit will produce the desired output.
o Output: The output is the outcome or the result of the program.

Why do we need Algorithms?

We need algorithms because of the following reasons:

o Scalability: It helps us to understand the scalability. When we have a big

real-world problem, we need to scale it down into small-small steps to easily
analyze the problem.
o Performance: The real-world is not easily broken down into smaller steps. If
the problem can be easily broken into smaller steps means that the problem
is feasible.

Let's understand the algorithm through a real-world example. Suppose we

want to make a lemon juice, so following are the steps required to make a
lemon juice:

Step 1: First, we will cut the lemon into half.

Step 2: Squeeze the lemon as much you can and take out its juice in a
container.

Step 3: Add two tablespoon sugar in it.

Step 4: Stir the container until the sugar gets dissolved.

Step 5: When sugar gets dissolved, add some water and ice in it.

Step 6: Store the juice in a fridge for 5 to minutes.

Step 7: Now, it's ready to drink.

The above real-world can be directly compared to the definition of the

algorithm. We cannot perform the step 3 before the step 2, we need to follow
the specific order to make lemon juice. An algorithm also says that each and
every instruction should be followed in a specific order to perform a specific
task.

Now we will look an example of an algorithm in programming.

We will write an algorithm to add two numbers entered by the user.

The following are the steps required to add two numbers entered by
the user:

Step 1: Start

Step 2: Declare three variables a, b, and sum.

Step 3: Enter the values of a and b.

Step 4: Add the values of a and b and store the result in the sum variable,
i.e., sum=a+b.

Step 5: Print sum

Step 6: Stop

Factors of an Algorithm
The following are the factors that we need to consider for designing
an algorithm:

o Modularity: If any problem is given and we can break that problem into
small-small modules or small-small steps, which is a basic definition of an
algorithm, it means that this feature has been perfectly designed for the
algorithm.
o Correctness: The correctness of an algorithm is defined as when the given
inputs produce the desired output, which means that the algorithm has been
designed algorithm. The analysis of an algorithm has been done correctly.
o Maintainability: Here, maintainability means that the algorithm should be
designed in a very simple structured way so that when we redefine the
algorithm, no major change will be done in the algorithm.
o Functionality: It considers various logical steps to solve the real-world
problem.
 o Robustness: Robustness means that how an algorithm can clearly define
our problem.
o User-friendly: If the algorithm is not user-friendly, then the designer will not
be able to explain it to the programmer.
o Simplicity: If the algorithm is simple then it is easy to understand.
o Extensibility: If any other algorithm designer or programmer wants to use
your algorithm then it should be extensible.

Importance of Algorithms

1. Theoretical importance: When any real-world problem is given to us and

we break the problem into small-small modules. To break down the problem,
we should know all the theoretical aspects.
2. Practical importance: As we know that theory cannot be completed
without the practical implementation. So, the importance of algorithm can be
considered as both theoretical and practical.

Issues of Algorithms
The following are the issues that come while designing an
algorithm:

o How to design algorithms: As we know that an algorithm is a step-by-step

procedure so we must follow some steps to design an algorithm.
o How to analyze algorithm efficiency

Approaches of Algorithm
The following are the approaches used after considering both the
theoretical and practical importance of designing an algorithm:

o Brute force algorithm: The general logic structure is applied to design an

algorithm. It is also known as an exhaustive search algorithm that searches
all the possibilities to provide the required solution. Such algorithms are of
two types:
 1. Optimizing: Finding all the solutions of a problem and then take out
the best solution or if the value of the best solution is known then it will
terminate if the best solution is known.
2. Sacrificing: As soon as the best solution is found, then it will stop.
o Divide and conquer: It is a very implementation of an algorithm. It allows
you to design an algorithm in a step-by-step variation. It breaks down the
algorithm to solve the problem in different methods. It allows you to break
down the problem into different methods, and valid output is produced for the
valid input. This valid output is passed to some other function.
o Greedy algorithm: It is an algorithm paradigm that makes an optimal
choice on each iteration with the hope of getting the best solution. It is easy
to implement and has a faster execution time. But, there are very rare cases
in which it provides the optimal solution.
o Dynamic programming: It makes the algorithm more efficient by storing
the intermediate results. It follows five different steps to find the optimal
solution for the problem:
1. It breaks down the problem into a subproblem to find the optimal
solution.
2. After breaking down the problem, it finds the optimal solution out of
these subproblems.
3. Stores the result of the subproblems is known as memorization.
4. Reuse the result so that it cannot be recomputed for the same
subproblems.
5. Finally, it computes the result of the complex program.
o Branch and Bound Algorithm: The branch and bound algorithm can be
applied to only integer programming problems. This approach divides all the
sets of feasible solutions into smaller subsets. These subsets are further
evaluated to find the best solution.
o Randomized Algorithm: As we have seen in a regular algorithm, we have
predefined input and required output. Those algorithms that have some
defined set of inputs and required output, and follow some described steps
are known as deterministic algorithms. What happens that when the random
 variable is introduced in the randomized algorithm?. In a randomized
algorithm, some random bits are introduced by the algorithm and added in
the input to produce the output, which is random in nature. Randomized
algorithms are simpler and efficient than the deterministic algorithm.
o Backtracking: Backtracking is an algorithmic technique that solves the
problem recursively and removes the solution if it does not satisfy the
constraints of a problem.

The major categories of algorithms are given below:

o Sort: Algorithm developed for sorting the items in a certain order.

o Search: Algorithm developed for searching the items inside a data structure.
o Delete: Algorithm developed for deleting the existing element from the data
structure.
o Insert: Algorithm developed for inserting an item inside a data structure.
o Update: Algorithm developed for updating the existing element inside a data
structure.

Algorithm Analysis
The algorithm can be analyzed in two levels, i.e., first is before creating the
algorithm, and second is after creating the algorithm. The following are the
two analysis of an algorithm:

o Priori Analysis: Here, priori analysis is the theoretical analysis of an algorithm

which is done before implementing the algorithm. Various factors can be
considered before implementing the algorithm like processor speed, which
has no effect on the implementation part.
o Posterior Analysis: Here, posterior analysis is a practical analysis of an
algorithm. The practical analysis is achieved by implementing the algorithm
using any programming language. This analysis basically evaluate that how
much running time and space taken by the algorithm.

Algorithm Complexity
The performance of the algorithm can be measured in two factors:
 o Time complexity: The time complexity of an algorithm is the amount of
time required to complete the execution. The time complexity of an algorithm
is denoted by the big O notation. Here, big O notation is the asymptotic
notation to represent the time complexity. The time complexity is mainly
calculated by counting the number of steps to finish the execution. Let's
understand the time complexity through an example.

1. sum=0;
2. // Suppose we have to calculate the sum of n numbers.
3. for i=1 to n
4. sum=sum+i;
5. // when the loop ends then sum holds the sum of the n numbers
6. return sum;

In the above code, the time complexity of the loop statement will be atleast
n, and if the value of n increases, then the time complexity also increases.
While the complexity of the code, i.e., return sum will be constant as its
value is not dependent on the value of n and will provide the result in one
step only. We generally consider the worst-time complexity as it is the
maximum time taken for any given input size.

o Space complexity: An algorithm's space complexity is the amount of space

required to solve a problem and produce an output. Similar to the time
complexity, space complexity is also expressed in big O notation.

For an algorithm, the space is required for the following purposes:

1. To store program instructions

2. To store constant values
3. To store variable values
4. To track the function calls, jumping statements, etc.

Auxiliary space: The extra space required by the algorithm, excluding the
input size, is known as an auxiliary space. The space complexity considers
both the spaces, i.e., auxiliary space, and space used by the input.

So,

Space complexity = Auxiliary space + Input size.

Types of Algorithms
The following are the types of algorithm:

o Search Algorithm
o Sort Algorithm

Search Algorithm

On each day, we search for something in our day to day life. Similarly, with
the case of computer, huge data is stored in a computer that whenever the
user asks for any data then the computer searches for that data in the
memory and provides that data to the user. There are mainly two techniques
available to search the data in an array:

o Linear search
o Binary search

Linear Search

Linear search is a very simple algorithm that starts searching for an element
or a value from the beginning of an array until the required element is not
found. It compares the element to be searched with all the elements in an
array, if the match is found, then it returns the index of the element else it
returns -1. This algorithm can be implemented on the unsorted list.

Binary Search

A Binary algorithm is the simplest algorithm that searches the element very
quickly. It is used to search the element from the sorted list. The elements
must be stored in sequential order or the sorted manner to implement the
binary algorithm. Binary search cannot be implemented if the elements are
stored in a random manner. It is used to find the middle element of the list.

Sorting Algorithms
Sorting algorithms are used to rearrange the elements in an array or a given
data structure either in an ascending or descending order. The comparison
operator decides the new order of the elements.

Why do we need a sorting algorithm?

 o An efficient sorting algorithm is required for optimizing the efficiency of other
algorithms like binary search algorithm as a binary search algorithm requires
an array to be sorted in a particular order, mainly in ascending order.
o It produces information in a sorted order, which is a human-readable format.
o Searching a particular element in a sorted list is faster than the unsorted list.

Asymptotic Analysis
As we know that data structure is a way of organizing the data efficiently and
that efficiency is measured either in terms of time or space. So, the ideal
data structure is a structure that occupies the least possible time to perform
all its operation and the memory space. Our focus would be on finding the
time complexity rather than space complexity, and by finding the time
complexity, we can decide which data structure is the best for an algorithm.

The main question arises in our mind that on what basis should we compare
the time complexity of data structures?. The time complexity can be
compared based on operations performed on them. Let's consider a simple
example.

Suppose we have an array of 100 elements, and we want to insert a new

element at the beginning of the array. This becomes a very tedious task as
we first need to shift the elements towards the right, and we will add new
element at the starting of the array.

Suppose we consider the linked list as a data structure to add the element at
the beginning. The linked list contains two parts, i.e., data and address of the
next node. We simply add the address of the first node in the new node, and
head pointer will now point to the newly added node. Therefore, we conclude
that adding the data at the beginning of the linked list is faster than the
arrays. In this way, we can compare the data structures and select the best
possible data structure for performing the operations.

How to find the Time Complexity or running time for performing

the operations?
The measuring of the actual running time is not practical at all. The running
time to perform any operation depends on the size of the input. Let's
understand this statement through a simple example.

Suppose we have an array of five elements, and we want to add a new

element at the beginning of the array. To achieve this, we need to shift each
element towards right, and suppose each element takes one unit of time.
There are five elements, so five units of time would be taken. Suppose there
are 1000 elements in an array, then it takes 1000 units of time to shift. It
concludes that time complexity depends upon the input size.

Therefore, if the input size is n, then f(n) is a function of n that denotes the
time complexity.

How to calculate f(n)?

Calculating the value of f(n) for smaller programs is easy but for bigger
programs, it's not that easy. We can compare the data structures by
comparing their f(n) values. We can compare the data structures by
comparing their f(n) values. We will find the growth rate of f(n) because
there might be a possibility that one data structure for a smaller input size is
better than the other one but not for the larger sizes. Now, how to find f(n).

Let's look at a simple example.

f(n) = 5n2 + 6n + 12

where n is the number of instructions executed, and it depends on the size of

the input.

When n=1

% of running time due to 5n2 = * 100 = 21.74%

% of running time due to 6n = * 100 = 26.09%

% of running time due to 12 = * 100 = 52.17%

From the above calculation, it is observed that most of the time is taken by
12. But, we have to find the growth rate of f(n), we cannot say that the
maximum amount of time is taken by 12. Let's assume the different values
of n to find the growth rate of f(n).

n 5n2 6n 12

1 21.74% 26.09% 52.17%

 10 87.41% 10.49% 2.09%

100 98.79% 1.19% 0.02%

1000 99.88% 0.12% 0.0002%

As we can observe in the above table that with the increase in the value of n,
the running time of 5n2 increases while the running time of 6n and 12 also
decreases. Therefore, it is observed that for larger values of n, the squared
term consumes almost 99% of the time. As the n 2 term is contributing most
of the time, so we can eliminate the rest two terms.

Therefore,

f(n) = 5n2

Here, we are getting the approximate time complexity whose result is very
close to the actual result. And this approximate measure of time complexity
is known as an Asymptotic complexity. Here, we are not calculating the
exact running time, we are eliminating the unnecessary terms, and we are
just considering the term which is taking most of the time

In mathematical analysis, asymptotic analysis of algorithm is a method of

defining the mathematical boundation of its run-time performance. Using the
asymptotic analysis, we can easily conclude the average-case, best-case and
worst-case scenario of an algorithm.

It is used to mathematically calculate the running time of any operation

inside an algorithm.

Example: Running time of one operation is x(n) and for another operation, it
is calculated as f(n2). It refers to running time will increase linearly with an
increase in 'n' for the first operation, and running time will increase
exponentially for the second operation. Similarly, the running time of both
operations will be the same if n is significantly small.

Usually, the time required by an algorithm comes under three types:

Worst case: It defines the input for which the algorithm takes a huge time.

Average case: It takes average time for the program execution.

Best case: It defines the input for which the algorithm takes the lowest time

Asymptotic Notations
The commonly used asymptotic notations used for calculating the running
time complexity of an algorithm is given below:

o Big oh Notation (O)

o Omega Notation (Ω)
o Theta Notation (θ)

Big oh Notation (O)

o Big O notation is an asymptotic notation that measures the performance of

an algorithm by simply providing the order of growth of the function.
o This notation provides an upper bound on a function which ensures that the
function never grows faster than the upper bound. So, it gives the least upper
bound on a function so that the function never grows faster than this upper
bound.

It is the formal way to express the upper boundary of an algorithm running

time. It measures the worst case of time complexity or the algorithm's
longest amount of time to complete its operation. It is represented as shown
below:
For example:

If f(n) and g(n) are the two functions defined for positive integers,

then f(n) = O(g(n)) as f(n) is big oh of g(n) or f(n) is on the order of g(n))
if there exists constants c and no such that:

f(n)≤c.g(n) for all n≥no

This implies that f(n) does not grow faster than g(n), or g(n) is an upper
bound on the function f(n). In this case, we are calculating the growth rate of
the function which eventually calculates the worst time complexity of a
function, i.e., how worst an algorithm can perform.

Let's understand through examples

Example 1: f(n)=2n+3 , g(n)=n

Now, we have to find Is f(n)=O(g(n))?

To check f(n)=O(g(n)), it must satisfy the given condition:

f(n)<=c.g(n)

First, we will replace f(n) by 2n+3 and g(n) by n.

2n+3 <= c.n

Let's assume c=5, n=1 then

2*1+3<=5*1
5<=5

For n=1, the above condition is true.

If n=2

2*2+3<=5*2

7<=10

For n=2, the above condition is true.

We know that for any value of n, it will satisfy the above condition, i.e.,
2n+3<=c.n. If the value of c is equal to 5, then it will satisfy the condition
2n+3<=c.n. We can take any value of n starting from 1, it will always satisfy.
Therefore, we can say that for some constants c and for some constants n 0, it
will always satisfy 2n+3<=c.n. As it is satisfying the above condition, so f(n)
is big oh of g(n) or we can say that f(n) grows linearly. Therefore, it
concludes that c.g(n) is the upper bound of the f(n). It can be represented
graphically as:

The idea of using big o notation is to give an upper bound of a particular

function, and eventually it leads to give a worst-time complexity. It provides
an assurance that a particular function does not behave suddenly as a
quadratic or a cubic fashion, it just behaves in a linear manner in a worst-
case.

Omega Notation (Ω)

o It basically describes the best-case scenario which is opposite to the big o

notation.
o It is the formal way to represent the lower bound of an algorithm's running
time. It measures the best amount of time an algorithm can possibly take to
complete or the best-case time complexity.
o It determines what is the fastest time that an algorithm can run.

If we required that an algorithm takes at least certain amount of time

without using an upper bound, we use big- Ω notation i.e. the Greek letter
"omega". It is used to bound the growth of running time for large input size.

If f(n) and g(n) are the two functions defined for positive integers,

then f(n) = Ω (g(n)) as f(n) is Omega of g(n) or f(n) is on the order of

g(n)) if there exists constants c and no such that:

f(n)>=c.g(n) for all n≥no and c>0

Let's consider a simple example.

If f(n) = 2n+3, g(n) = n,

Is f(n)= Ω (g(n))?

It must satisfy the condition:

f(n)>=c.g(n)

To check the above condition, we first replace f(n) by 2n+3 and g(n) by n.

2n+3>=c*n

Suppose c=1

2n+3>=n (This equation will be true for any value of n starting from 1).

Therefore, it is proved that g(n) is big omega of 2n+3 function.

As we can see in the above figure that g(n) function is the lower bound of the
f(n) function when the value of c is equal to 1. Therefore, this notation gives
the fastest running time. But, we are not more interested in finding the
fastest running time, we are interested in calculating the worst-case
scenarios because we want to check our algorithm for larger input that what
is the worst time that it will take so that we can take further decision in the
further process.

Theta Notation (θ)

o The theta notation mainly describes the average case scenarios.

o It represents the realistic time complexity of an algorithm. Every time, an
algorithm does not perform worst or best, in real-world problems, algorithms
mainly fluctuate between the worst-case and best-case, and this gives us the
average case of the algorithm.
o Big theta is mainly used when the value of worst-case and the best-case is
same.
o It is the formal way to express both the upper bound and lower bound of an
algorithm running time.
Let's understand the big theta notation mathematically:

Let f(n) and g(n) be the functions of n where n is the steps required to
execute the program then:

f(n)= θg(n)

The above condition is satisfied only if when

c1.g(n)<=f(n)<=c2.g(n)

where the function is bounded by two limits, i.e., upper and lower limit, and
f(n) comes in between. The condition f(n)= θg(n) will be true if and only if
c1.g(n) is less than or equal to f(n) and c2.g(n) is greater than or equal to
f(n). The graphical representation of theta notation is given below:

Let's consider the same example where

f(n)=2n+3
g(n)=n

As c1.g(n) should be less than f(n) so c1 has to be 1 whereas c2.g(n) should

be greater than f(n) so c2 is equal to 5. The c1.g(n) is the lower limit of the of
the f(n) while c2.g(n) is the upper limit of the f(n).
c1.g(n)<=f(n)<=c2.g(n)

Replace g(n) by n and f(n) by 2n+3

c1.n <=2n+3<=c2.n

if c1=1, c2=2, n=1

1*1 <=2*1+3 <=2*1

1 <= 5 <= 2 // for n=1, it satisfies the condition c1.g(n)<=f(n)<=c2.g(n)

If n=2

1*2<=2*2+3<=2*2

2<=7<=4 // for n=2, it satisfies the condition c1.g(n)<=f(n)<=c2.g(n)

Therefore, we can say that for any value of n, it satisfies the condition
c1.g(n)<=f(n)<=c2.g(n). Hence, it is proved that f(n) is big theta of g(n). So,
this is the average-case scenario which provides the realistic time
complexity.

Why we have three different asymptotic analysis?

As we know that big omega is for the best case, big oh is for the worst case
while big theta is for the average case. Now, we will find out the average,
worst and the best case of the linear search algorithm.

Suppose we have an array of n numbers, and we want to find the particular

element in an array using the linear search. In the linear search, every
element is compared with the searched element on each iteration. Suppose,
if the match is found in a first iteration only, then the best case would be
Ω(1), if the element matches with the last element, i.e., nth element of the
array then the worst case would be O(n). The average case is the mid of the
best and the worst-case, so it becomes θ(n/1). The constant terms can
be ignored in the time complexity so average case would be θ(n).

So, three different analysis provide the proper bounding between the actual
functions. Here, bounding means that we have upper as well as lower limit
which assures that the algorithm will behave between these limits only, i.e.,
it will not go beyond these limits.

Common Asymptotic Notations

constant - ?(1)

linear - ?(n)

logarithmic - ?(log n)

n log n - ?(n log n)

exponential - 2?(n)

cubic - ?(n3)

polynomial - n?(1)

quadratic - ?(n2)