UNIT 1: INTRODUCTION

The design and analysis of algorithms is a field in computer science that focuses on the creation,
understanding, and evaluation of algorithms.

Problem: Making a Peanut Butter and Jelly Sandwich

Algorithm Design:
Gather Ingredients: Collect bread, peanut butter, jelly, and a butter knife.
Spread Peanut Butter: Take two slices of bread.
Use the butter knife to spread peanut butter on one side of each slice.
Spread Jelly: On one of the slices, spread jelly on the other side.
Combine Slices: Press the peanut butter side of one slice against the jelly side of the other slice,
creating a sandwich.
Cut (Optional): If desired, cut the sandwich into halves or quarters.
Enjoy: Your PB&J sandwich is ready to be enjoyed!

Analysis of the Algorithm:

Time Complexity: The time it takes to make the sandwich depends on the number of steps.
Each step is a constant time operation, so the time complexity is O(1).
Space Complexity: The algorithm doesn't require any additional space other than the physical
ingredients. Its space complexity is O(1), constant space.

Algorithm:
A set of rules for solving a problem in a finite number of steps.
A set of instructions may or may not involve mathematics.
Ex: A Recipe
 ● For a computer to be able to aid in the efficiency of solving your problem, the
algorithm must be comprised of a series of tasks written in a way that a computer can
understand.

● This involves a translation between the language of humans and the high-level
language of humans.
● Computer programmers work to create instructions through coding that are born
from human ideas but translated for a computer to understand.

Designing Algorithm involves

● Instructions must be unambiguous.

● Must have defined inputs and outputs.
● Must have a defined endpoint.
● Steps should be sequenced in correct order.
● Must interpret computer’s results.

Specification of an Algorithm
There are different ways by which an algorithm can be specified.
Natural language: It is very simple to specify an algorithm using natural language.
For example, to add two numbers

● Read the first number ‘A’

● Read the second number ‘B’

● Add A and B and store the result in the third variable.

● Display the result
Pseudocode: Pseudocode is an informal way of programming description that does not require
any strict programming language syntax.
For example, to add two numbers
1. Start
2. Input firstNumber
3. Input secondNumber
4. result = firstNumber + secondNumber
5. Output result
6. End
Flowchart:
A flowchart is a diagrammatic representation of an algorithm.
A graphical representation of steps.
For example, to add two numbers

Types of Algorithms
1. Sequential Algorithm:
1. Executes one operation at a time.
2. Follows a linear sequence of steps.
3. Most traditional algorithms fall into this category.
Example: Linear Search of an element in an array
2. Parallel Algorithm:
1. Exploits parallel processing capabilities of computers.
2. Performs multiple operations simultaneously.
3. Suitable for systems with multiple processors or cores.
Example: Matrix multiplication involves multiplying two matrices to produce a third
matrix.
Notion of Algorithm

Problem:
This represents the task or challenge that needs to be addressed or solved. In the context of
algorithm design, a problem could be anything from sorting a list of numbers to finding the
shortest path in a network.
Algorithm:
The algorithm is a sequence of unambiguous instructions designed to solve the specified
problem. It outlines the steps to be followed to transform the given input into the desired output.
Input:
This is the information or data that is provided to the algorithm as it begins its execution. The
algorithm processes this input to produce the required output.
Computer:
In this context, "computer" refers to something capable of understanding and executing the
instructions of the algorithm. Originally, it meant a human performing numeric calculations,
but nowadays it commonly refers to electronic devices like computers.
Output:
The result or solution produced by the algorithm is based on the provided input. It represents
the desired outcome of the algorithmic process.
The diagram, therefore, illustrates the relationships and flow between these key elements:

• The problem motivates the need for an algorithm.

• The algorithm takes input based on the problem and processes it.
• The "computer" (human or electronic) executes the algorithmic instructions.
• The output is the result generated by the algorithm in response to the provided input.

Example Problem: Greatest Common Divisor (GCD) Calculation

1. Euclid's Algorithm for GCD (Greatest Common Divisor)
1. Step 1: If n=0, return the value of m as the answer and stop; otherwise, proceed
to Step 2.
2. Step 2: Divide m by n and assign the remainder to r.
3. Step 3: Assign the value of n to m and the value of r to n. Go to Step 1.
4. The algorithm repeatedly applies the equation gcd(m,n)=gcd(n,mmodn) until m
mod n becomes zero.
Example: Let's find the GCD of 60 and 24 using Euclid's Algorithm.
Steps:
1. gcd(60,24) - Proceed to Step 2.
2. gcd(24,12) - Proceed to Step 2.
3. gcd(12,0) - The remainder is 0, so the answer is 12.

2. Consecutive Integer Checking Algorithm for GCD

Step 1: Assign the value of min(m,n) to t.
Step 2: Divide m by t. If the remainder is 0, go to Step 3; otherwise, go to Step 4.
Step 3: Divide n by t. If the remainder is 0, return t as the answer and stop; otherwise,
proceed to Step 4.
Step 4: Decrease the value of t by 1. Go to Step 2.
Example: Find the GCD of 60 and 24 using the consecutive integer checking
algorithm.
Steps:
1. Start with t =min(60,24)=24.
2. 60 mod 24=12 so t decreases to 23.
 3. 60 mod 23 ≠ 0, t decreases to 22.
4. Continue this process until t becomes 12.
5. 60 mod 12=0, so the answer is 12.

3. Sieve of Eratosthenes
1. Implement a Sieve of Eratosthenes to find all primes up to min (m,n).
2. Use the primes to find the GCD using the middle-school procedure.
Example: Find the GCD of 60 and 24 using Sieve of Eratosthenes

Steps:
1. Use Sieve to find primes up to min (60,24)=24:
2,3,5,7,11,13,17,19,232,3,5,7,11,13,17,19,23.
2. Use the primes to find GCD using the middle-school procedure: 2×2×3=12.

Fundamentals of Algorithmic Problem Solving

1. Understanding the Problem:
1. Read the problem description carefully.
2. Ask questions to clarify doubts.
3. Work through small examples and consider special cases.
2. Ascertaining the Capabilities of the Computational Device:
1. Understand the computational device's capabilities, typically a computer.
2. Recognize whether the algorithm will be executed sequentially (on a von
Neumann machine) or in parallel.
3. Choosing between Exact and Approximate Problem Solving:
1. Decide whether the problem should be solved exactly or approximately.
2. Exact algorithms aim for precise solutions, while approximation algorithms
provide approximate solutions.
4. Algorithm Design Techniques:
1. Learn and apply general algorithm design techniques or paradigms.
2. Understand various design strategies applicable to different problems.
5. Designing an Algorithm and Data Structures:
1. Choose appropriate algorithm design techniques.
2. Pay attention to selecting data structures suitable for the algorithm's operations.
6. Methods of Specifying an Algorithm:
1. Specify the algorithm using natural language or pseudocode.
2. Natural language may be intuitive, but pseudocode provides a more precise
description.
7. Proving an Algorithm’s Correctness:

1. Prove that the algorithm produces the correct result for all legitimate inputs.
2. Mathematical induction is often used for correctness proofs.
8. Analyzing an Algorithm:
1. Evaluate the algorithm's efficiency in terms of time and space.
2. Consider simplicity as an additional characteristic; simpler algorithms are easier
to understand.
 9. Coding an Algorithm:
1. Implement the algorithm as a computer program.
2. Test and debug the program thoroughly.
10. Iterative Improvement:
1. Continuously work on improving the algorithm.
2. Consider algorithmic tweaks and optimizations for better performance.
11. Conclusion and Reflection:
1. Acknowledge that a good algorithm often results from iterative efforts and
rework.
2. Recognize that achieving perfection is a balance between competing goals and
available resources.

Important Problem Types

1. Sorting:

• Definition: Rearrange items of a list in non-decreasing order.

• Examples: Sorting numbers, characters, strings, or records (e.g., student records).

• Key Concept: Presorting facilitates tasks like searching, and ranking, and improves
efficiency in various applications.
2. Searching:

• Definition: Find a given value (search key) in a set or multiset. Ex: Searching contacts
on mobile phone in an unsorted list.

• Algorithms: Range from sequential search to binary search and specialized algorithms
for large databases.

• Considerations: No one-size-fits-all algorithm; trade-offs between speed and memory

usage.
3. String Processing:

• Focus: Handling algorithms for non-numerical data, particularly string manipulation.

• Examples: Text strings, bit strings, and gene sequences.

• Significance: String matching (searching for a word in text) is highlighted.
 4. Graph Problems:

• Definition: A graph is a collection of vertices connected by edges.

• Applications: Modeling transportation, communication, social networks, etc.

• Algorithms: Graph traversal, shortest-path, and topological sorting.

• Challenges: Some graph problems are computationally challenging, e.g., traveling
salesman and graph coloring.
5. Combinatorial Problems:
• Nature: Involves finding combinatorial objects satisfying constraints.

• Complexity: Among the most challenging problems in computing.

• Examples: Traveling salesman and graph coloring.
6. Geometric Problems:

• Focus: Deals with geometric objects like points, lines, polygons.

• Applications: Computer graphics, robotics, tomography.

• Problems: Closest-pair and convex-hull problems are discussed.
7. Numerical Problems:

• Nature: Involves mathematical objects of continuous nature.

• Challenges: Requires approximation due to continuous nature; potential accumulation

of round-off errors.

• Significance: Historically important, but recent focus has shifted towards business
applications.

The Analysis Framework

1. Efficiency Analysis Framework
• Two types of efficiency: time efficiency and space efficiency.

• Time efficiency (time complexity) refers to how fast an algorithm runs.

• Space efficiency (space complexity) refers to the amount of memory units required.
• Traditionally, more focus is given to time efficiency.

• Input size (parameter "n") is crucial for analyzing efficiency.

2. Measuring Input Size:
1. Input size is often denoted by "n."
2. The choice of the parameter depends on the nature of the problem.
3. For some problems, measuring in bits (binary representation) is preferable.
3. Units for Measuring Running Time:
1. Using standard time units (seconds, milliseconds) has drawbacks.
2. Counting the number of times the algorithm's basic operation is executed is a
more robust approach.
3. Identifying the basic operation (usually in the innermost loop) is crucial.
Running Time:
Running time refers to the total time it takes for an algorithm to complete its
execution, from the beginning to the end. It is a crucial metric used to evaluate the
efficiency and performance of algorithms.
Why Measure Running Time:
Performance Evaluation:
Running time provides a quantitative measure of an algorithm's efficiency. It helps
compare different algorithms solving the same problem.
Algorithmic Analysis:
Understanding the running time helps in analyzing and predicting how an algorithm's
performance scales with the size of the input.

Units for Measuring Running Time:

❖ Using standard time units like seconds or milliseconds has drawbacks:
➢ Dependence on the speed of a particular computer.
➢ Dependence on the quality of the program implementation and compiler.
➢ Difficulty in accurately measuring the actual running time.
❖ Counting Basic Operations:
➢ Instead of relying on external factors, algorithms are often analyzed by counting the
number of times a fundamental operation (basic operation) is executed.
➢ Identifying the basic operation is critical, usually found in the innermost loop where the
majority of computation occurs.
4. Orders of Growth:
how the algorithm's performance scales with the size of the input

1. Analyzing efficiency focuses on the growth rate of the basic operation count for
large inputs.
2. Logarithmic functions grow slowly, while exponential and factorial functions
grow rapidly.
3. Emphasis is on the order of growth rather than specific constants for large
inputs.
5. Worst-Case, Best-Case, and Average-Case Efficiencies:

1. Worst-case efficiency considers the maximum running time for any input of size
"n."
2. Best-case efficiency considers the minimum running time for any input of size
"n."
3. Average-case efficiency analyzes efficiency across a range of inputs,
considering probabilities.
Example: Sequential Search

● Sequential search, also known as linear search, is a straightforward algorithm used to

find a specific element in a list. The algorithm sequentially checks each element of the
list until a match is found or the entire list has been searched. It is a simple and intuitive
method but may not be the most efficient for large datasets.

➢ Illustrates how to analyze worst-case, best-case, and average-case efficiencies for a

sequential search algorithm.
➢ Worst-case occurs when the item is not in the list or is at the end.
➢ Best-case occurs when the item is at the beginning of the list.
➢ Average-case involves probabilities of successful and unsuccessful searches.

● Algorithm:
1. Start from the beginning of the list.
2. Compare the target element with each element in the list.
3. If a match is found, return the index of the matching element.
4. If the end of the list is reached without finding a match, return a special value (e.g., -1)
to indicate that the element is not present.
ALGORITHM LinearSearch(list, target)
// Searches for the target element in the list using linear search
// Input: A list and a target element
// Output: The index of the target element if found, otherwise -1

for each element at index i in the list

if element == target
return i
return -1 // target not found

Scenario:
Let's assume our list has n elements.
Worst-Case:

● The worst-case scenario is when the target element is at the end of the list or not present.
● The algorithm will have to traverse the entire list.
● Worst-case time complexity: O(n)
Best-Case:

• The best-case scenario is when the target element is at the beginning of the list.
• The algorithm finds the target at the first comparison.
• Best-case time complexity: O(1)
Average-Case:

• To analyze the average case, let's assume the target element is equally likely to be at
any position in the list.

• In a successful search, the average position of the target is around n/2 (assuming
uniform distribution).

• The average number of comparisons in a successful search is approximately n/2.

• In an unsuccessful search, the algorithm checks all n elements.

• So, the average-case time complexity is approximately (n/2 + n)/2 = 3n/4.

• Average-case time complexity: O(n)
 • When you calculate the average-case time complexity, you consider both cases
(successful and unsuccessful searches). The average number of comparisons can be
calculated as the sum of the cases divided by the total number of cases.
In this case:
Number of comparisons in successful search + unsuccessful search
Average case time complexity =
Total number of cases

Now, when we say the average-case time complexity is O(n), means that as the input size (n)
grows, the complexity increases linearly. The constant factor of 3/4 is dropped in big O
notation.
In summary, stating the average-case time complexity as O(n) is a way of expressing that the
algorithm's performance grows linearly with the input size, and constant factors are not
considered in this notation.
Example:
Suppose you have a list [3, 1, 4, 1, 5, 9, 2, 6, 5, 3] and you want to find the index of the element
5. The sequential search would iterate through the list and find the first occurrence of 5 at index
4.

Characteristics:
❖ Time Complexity: O(n) - Linear time complexity, where 'n' is the size of the list.
❖ Space Complexity: O(1) - Constant space, as it only requires a few variables to store indices
and comparisons.

Considerations:
❖ Suitable for small lists or unordered lists.
❖ Inefficient for large datasets compared to more advanced search algorithms like binary
search.
❖ Simple to implement and understand.

Note: However, it's important to note that when analyzing algorithms, the focus is often on the
worst-case scenario, as it provides an upper bound on the running time for any input size. In
practical terms, knowing the best-case scenario can be useful, but it's the worst-case scenario
that is typically considered in algorithmic analysis.
Asymptotic Notations and Basic Efficiency Classes
These notations are used in the efficiency analysis of algorithms to describe the growth
rate of an algorithm's basic operation count.

● The concept of order of growth is a way to describe how the performance or resource
usage of an algorithm scales with the size of the input.

● It helps us understand how the efficiency of an algorithm changes as the input size
becomes very large.
Notations:

• Three common notations used are O (Big-Oh), Ω (Big-Omega), and Θ (Big-Theta).

• O describes an upper bound on the growth rate.

• Ω describes a lower bound on the growth rate.

• Θ describes both upper and lower bounds, indicating a tight bound on the growth rate.
O-notation

● f(n) ∈ O(g(n)) if f(n) is bounded above by some constant multiple of g(n) for all large
enough n.

● Mathematically, there exist positive constants c and n0 such that f(n) ≤ c*g(n) for all n≥
n0

● f(n): This represents the function that describes the running time or the growth rate of
an algorithm.

● g(n): This represents a simpler function, often an upper bound or a reference function,
that serves as a comparison for the growth rate of f(n).
Example: Suppose we have an algorithm that performs a linear search on an array of size
n to find a specific element.

● f(n): The running time of the algorithm (actual time taken to perform the linear search).
● g(n): A function that bounds the running time from above.

● For simplicity, let's say g(n)=5n represents a linear upper bound on the running time,
meaning that the algorithm should take no more than five times the array size to
complete.

● Now, we want to show that the running time of our algorithm f(n) is within the
bounds of O(g(n)), i.e., it grows at most linearly with a constant multiple of g(n).
 In this graph:
The horizontal axis represents the size of the array (n).
The vertical axis represents the running time (f(n)).
The line g(n)=5n serves as an upper bound on the running time.
If our algorithm has a linear time complexity, the actual running time curve (f(n)) should
stay below or grow linearly with a constant multiple of g(n) (the linear upper bound). This
relationship is captured by f(n)∈O(g(n)).

Let's take n=10 as an example:

● g (10) =5×10=50: This represents the linear upper bound, indicating that the algorithm's
running time should not exceed 50 units for an array of size 10.

● Now, let's say the actual running time of the algorithm (t (10)) is observed to be 30 units
for an array of size 10.
In this case, t (10) is well within the linear upper bound g (10) =5×10=50.

• The actual running time is significantly less than the upper bound, demonstrating that
the algorithm performs efficiently for this particular input size.
• This relationship holds for larger values of n as well.
• If the running time curve stays within or grows linearly with a constant multiple of g(n)
for sufficiently large n, then f(n) is said to be in O(g(n)), indicating a linear time
complexity.

So, when we say f(n) ≤ c ⋅ g(n) for all n≥n0

c: Constant Multiple
It is a positive constant that represents the factor by which g(n) is multiplied to create an upper
bound.
n0: Threshold Value

● It is a non-negative integer that represents the point beyond which the upper bound
holds true.
● Before n0, the upper bound might not be applicable or accurate.
 Given values:

● f(n)=30 (actual running time for an array of size n=10)

● g(n)=5n (linear upper bound)

● c=5 (constant multiple)

● n0=2 (threshold value)

The condition f(n) ≤ c ⋅ g(n) is as follows: 30 ≤ 5⋅(5n) for all n≥2

Let us simplify the inequality

● 30 ≤ 25n for all n≥2

Now, let's verify if this condition holds true for the given values of n and f(n):
● For n= 2: 30 ≤ 25⋅2 (True)
● For n= 3: 30 ≤ 25⋅3 (True)
● For n= 4: 30≤ 25⋅4 (True)
● .
● .
● .
● (and so on for all n≥2)

Since the condition f(n)≤c⋅g(n) is satisfied for all n≥10 with the given values of c, g(n), and n0,
we can conclude that f(n) is indeed in O(g(n)) for the sequential search algorithm.
Note: The value c=5 was used in the explanation as an illustrative example. In practice, the
choice of c depends on the specific analysis and characteristics of the algorithm, hardware, or
other factors.
The choice of n0 = 2 as the threshold is somewhat arbitrary and depends on the characteristics
of the algorithm and the nature of the input data.

Ω-notation
In the context of asymptotic notation:

● f(n) ∈ Ω(g(n)) if f(n) is bounded below by some constant multiple of g(n) for all large
enough n.
 ● Mathematically, there exist positive constants c and n0 such that f(n) ≥ c*g(n) for all n≥
n0

So, when we say f(n)∈(g(n)), it means that f(n) grows at least as fast as g(n), possibly faster
but not slower, for sufficiently large values of n.
Example: Given: f(n)=2n+5 and g(n)=n, c = 1.
f(n) ≥ c*g(n)
2n+5≥n
= n+5≥0
= n≥−5
So, the inequality holds true for all n ≥−5. However since n is typically a nonnegative integer,
you can choose n0 = 0 as the smallest nonnegative integer for which the inequality holds true.
In this context, n0 is the threshold value, indicating that the relationship between 2n+5 and n
starts to satisfy the condition for
n ≥ n0=0.
So, while 2n+5 grows at a constant rate, it's not the fastest possible growth rate, but it's still a
reasonable and efficient growth rate for many practical purposes.
Time Complexity:

• In this expression, the dominant term is 2n, and the constant term 5 is not significant as
n becomes large.

• Therefore, the time complexity can be expressed as O(n), indicating linear time
complexity.
Expression:
1. The expression 2n+5 has a linear form, with the dominant term being 2n as n
becomes large.
Order of Growth Analysis:
1. Linear growth (O(n)) indicates that the running time increases linearly with the
size of the input (n).
2. In this case, as n increases, the running time grows at a constant rate
proportional to n.
3. For example, if n doubles, the running time also approximately doubles.
2. Visual Representation:
1. A graph of the function 2n+5 would show a straight line, confirming its linear
growth.
Ө-notation
● A function f(n) is in Θ(g(n)), denoted f(n) ∈ Θ(g(n)), if f(n) is both an upper bound and
a lower bound for g(n) up to a constant factor for all sufficiently large n.
● Mathematically, there exist positive constants c1, c2 and n0 such that
c2g(n) ≤ f(n) ≤ c1g(n) for all n≥n0
Upper Bound (≤):
● f(n) is not growing faster than a constant multiple of g(n).
● There exist constants c1 and n0 such that f(n) is always less than or equal to c1 g(n) for
n≥n0.
Lower Bound (≥):
● f(n) is not growing slower than a constant multiple of g(n).
● There exist constants c2 and n0 such that f(n) is always greater than or equal to c2 g(n)
for n≥n0.
Conclusion:
● f(n) is sandwiched between c2g(n) and c1g(n) for sufficiently large n.
● This implies that f(n) grows at the same rate as g(n), neither faster nor slower.
1
Example: 𝑛(𝑛 − 1) ∈ 𝜃(𝑛2)
2

Upper Bound (≤):

● Choose c1=1 and n0=0.
1
● Prove: 2 𝑛(𝑛 − 1) ≤ 𝑐1𝑛2 for n≥ 0

Lower Bound (≥):

● Choose c2=1/4 and n0=2.
1
● Prove: 2 𝑛(𝑛 − 1) ≥ 𝑐2𝑛2 for n≥ 2
 Conclusion:
1
● Since 2 𝑛(𝑛 − 1) is both an upper and lower bound for 𝑛2 we conclude that
1
𝑛(𝑛 − 1) ∈ 𝜃(𝑛2)
2

Mathematical Analysis of Non-recursive Algorithms

● Non-recursive algorithms typically use iteration (loops) or other control structures to
achieve their goals without relying on recursive calls.
● They are often more straightforward and may be preferred in situations where recursion
might lead to unnecessary complexity or where iterative solutions are more efficient.
Example:
ALGORITHM SumOfNaturalNumbers(n)
// Determines the sum of the first n natural numbers
// Input: A positive integer n
// Output: The sum of the first n natural numbers
sum = 0
for i from 1 to n do
sum = sum + i
return sum

General Plan for Analyzing the Time Efficiency of Nonrecursive Algorithms

1. Decide on a parameter indicating an input’s size.
Parameter: n, the number of natural numbers to sum.
2. Identify the algorithm’s basic operation.
Basic Operation: The basic operation is the addition operation inside the loop (sum=sum+i).
3. Check whether the number of times the basic operation is executed depends only on the size
of an input.
Dependency: The number of times the basic operation is executed depends only on n as it is
determined by the loop running n times.
4. Set up a sum expressing the number of times the algorithm’s basic operation is executed.
Sum Expression: The sum S(n) expressing the number of additions is set up as follows:
 𝑛
𝑛(𝑛 + 1)
𝑆(𝑛) = ∑ 𝑖 =
2
𝑖=1

5. Using standard formulas and rules of sum manipulation, either find a closed-form formula
for the count or establish its order of growth.
𝑛
𝑛(𝑛 + 1)
𝑆(𝑛) = ∑ 𝑖 = ∈ 𝑂(𝑛2)
2
𝑖=1

Order of Growth: The order of growth is O(n2) since the dominant term is n.

This indicates that the algorithm's time complexity grows quadratically with the size of the
input n.
In summary, S(n) quantifies the number of times the addition operation is executed in the
algorithm for a given input size n, and the formula provides a concise representation of this
cumulative sum.

Mathematical Analysis of Recursive Algorithms

● Recursion is a technique where a function calls itself to solve a smaller instance of the
same problem.
● Algorithms can be broadly categorized into recursive and non-recursive based on
whether they use recursion or not.

Problem Statement: Compute the factorial function F(n) = n! for a nonnegative integer n using
the recursive algorithm.
General Plan for Analyzing the Time Efficiency of Recursive Algorithms:
1. Parameter for Input Size:
1. Decide on a parameter indicating the input size. In this case, we consider the
input size as n, the nonnegative integer for which we want to compute n!
2. Identify Basic Operation:
1. Identify the algorithm's basic operation. For this recursive algorithm, the basic
operation is multiplication, specifically the multiplication F(n−1) × n.
3. Variability of Basic Operation:
1. Check whether the number of times the basic operation is executed can vary for
different inputs of the same size. In this case, the basic operation involves
multiplication, and it is executed n times to compute n!
 4. Recurrence Relation:
1. Set up a recurrence relation for the number of times the basic operation is
executed. The recurrence relation is derived from the algorithm's structure.
M(n)=M(n−1) +1 with the initial condition M (0) =0.
5. Solve the Recurrence:
• Solve the recurrence relation or ascertain the order of growth of its solution.
• In this example, the recurrence relation M(n)=M(n−1) +1 is solved using the
method of backward substitution:
M(n)=M(n−i) +i With the initial condition M(0)=0, it is derived that
M(n)=n.
*The worst-case and average-case efficiencies are O(n), and the best-case efficiency is O(1) in
this specific example.

Empirical Analysis of Algorithms

● Empirical analysis, in the context of algorithms, refers to the process of evaluating and
understanding the performance of an algorithm through practical experiments and
observations rather than relying solely on theoretical or mathematical analyses.
● It involves running the algorithm on various inputs, collecting data on its behavior, and
analyzing the obtained results.
Example:
Understand the Experiment's Purpose:
• Goal: Compare the efficiency of Bubble Sort and Merge Sort to determine which
performs better on various inputs.
Choose Efficiency Metric and Measurement Unit:
• Metric: Measure the running time of the algorithms.
• Unit: Use milliseconds as the measurement unit for time.
Define Input Sample Characteristics:
• Range: Sorting arrays of integers ranging from 100 to 1000 elements.
• Size: Vary the array sizes, e.g., 100, 200, 300, ..., 1000.
• Pattern: Use both random arrays and arrays sorted in different orders.
Implement the Algorithm:
• Write programs for Bubble Sort and Merge Sort in a programming language like
Python or Java.
Generate Input Sample:
• Create random arrays of integers for the random input.
• Generate sorted and reverse-sorted arrays for different orderings.
Run the Algorithm and Record Data:
• Execute Bubble Sort and Merge Sort on each input.
• Measure the running time using system timers.
• Record the time taken for each algorithm on every input.
Data Analysis:
• Compute the average running time for each algorithm at each input size.
• Create a table or scatterplot to visualize the data.
• Calculate ratios of running times for different input sizes.
Interpret Scatterplots:
• Observe the shape of scatterplots.
• A linear increase suggests a linear algorithm, while a slower increase might
indicate a more efficient algorithm.
Predict Algorithm Performance:
• Use observed patterns to predict how each algorithm might perform on larger
input sizes not included in the experiment.
Consider Mathematical vs. Empirical Analysis:
• Understand that mathematical analysis may provide theoretical insights, but empirical
analysis provides practical insights based on real-world performance.

Algorithm Visualization
1. Define the Purpose:
1. Clearly define the goal, whether for research or education.
2. Specify if the aim is to understand algorithm features or aid in learning.
2. Choose Visualization Type:
1. Decide between static and dynamic visualization.
2. Static shows a sequence of still images; dynamic, or animation, provides
continuous visual representation.
3. Sorting Algorithms Example:
1. Sorting algorithms, due to their visual nature, are commonly used.
 2. Bars or sticks of different lengths represent elements to be sorted.
4. Applications:
• Algorithm visualization serves research and education.
• Research benefits include uncovering unknown algorithm features.
• Education aims to enhance student learning, though effectiveness varies.