ANALYSIS AND DESIGN OF ALGORITHM

INTRODUCTION: Algorithm, pseudo code for expressing algorithms, performance

analysis-space complexity, time complexity, asymptotic notation- big (O) notation, omega
notation, theta notation and little (o) notation, recurrences, probabilistic analysis, disjoint
set operations, union and find algorithms.

1. Introduction to Algorithms

An algorithm is a step-by-step procedure or formula for solving a problem. It is a finite set

of instructions that, when followed, accomplishes a specific task. Algorithms are designed
to be both correct (produce the expected output) and efficient (use minimal resources).

2. Pseudocode for Expressing Algorithms

Pseudocode is a simplified, human-readable representation of an algorithm. It bridges the

gap between plain language and programming code, allowing the logic of an algorithm to
be communicated without being tied to a specific programming syntax.

Example:

Algorithm: Find the maximum number in a list

1. Input: A list of numbers
2. Initialize max to the first number in the list
3. For each number in the list:
If the number is greater than max:
Update max to this number
4. Output max

3. Performance Analysis

Performance analysis evaluates how efficient an algorithm is regarding two main

resources:

1. Time Complexity: How the runtime of an algorithm scales with the size of the input.
2. Space Complexity: How much extra memory the algorithm uses.
Time Complexity
Time complexity measures how the runtime of an algorithm grows relative to the size of its
input n. It is expressed as a function T(n), which describes the number of basic operations
or steps performed.

Why Measure Time Complexity?

1. Predict Performance: To estimate how an algorithm will perform for large inputs.
2. Algorithm Comparison: To compare multiple algorithms and choose the most
efficient one.

Types of Time Complexity

1. Best Case: The minimum time an algorithm takes (e.g., searching in the first
position of a list).
2. Worst Case: The maximum time an algorithm takes (e.g., searching for a missing
element).
3. Average Case: The expected time considering all possible inputs.

Common Time Complexities

Time Complexity Description Example

Constant time (does not depend on
O(1) Accessing an array element
n)
O(log⁡ n) Logarithmic time Binary Search
O(n) Linear time Linear Search
O(nlog⁡n) Line arithmic Merge Sort, Quick Sort (avg)
O(n^2) Quadratic time Bubble Sort, Selection Sort
O(2^n) Exponential Solving the Tower of Hanoi
Traveling Salesman
O(n!) Factorial time
Problem

Steps to Analyze Time Complexity

1. Count Basic Operations: Measure operations independent of the machine (e.g.,

comparisons, additions).
 2. Express in Terms of nn: Relate the number of operations to the input size.
3. Asymptotic Simplification: Focus on the largest growing term and ignore
constants.
Space Complexity
Space complexity measures the amount of memory an algorithm uses, including:

1. Fixed Memory: Space required for constants, variables, and instructions.

2. Variable Memory: Space required for dynamic data structures (e.g., arrays, linked
lists) or recursion stack.

Components of Space Complexity

1. Input Space: Space for storing the input.

2. Auxiliary Space: Additional memory used by the algorithm (e.g., temporary
variables).
3. Recursion Stack Space: Memory used by recursive calls.

Types of analysis :
1. Experimental/ apostrium/ relative analysis

2. Apriori/ independent/ absolute analysis

Asymptotic Notation
Asymptotic notation describes the growth of an algorithm's runtime or space requirements
as the input size n approaches infinity. It simplifies the complexity by ignoring constants
and lower-order terms.

Types of Asymptotic Notations

1. Big-O Notation (O)

• Represents the upper bound on the growth rate of an algorithm.

• Focuses on the worst-case performance.
• Ignores constant factors and lower-order terms.

Example: For T(n) = 3n^2 + 5n + 10:

• Dominant term: n^2.

• Big-O: O(n^2).

2. Omega Notation (Ω\Omega)

• Represents the lower bound on the growth rate.

• Describes the best-case scenario.

Example: For T(n) = 3n^2 + 5n + 10:

• Best-case growth is n^2.

• Omega: Ω(n^2).

3. Theta Notation (Θ\Theta)

• Represents the tight bound, meaning the algorithm grows at the same rate in both
the best and worst cases.

Example: If T(n) = 3n^2 + 5n + 10, and all inputs grow at the same rate:

• Theta: Θ(n^2).
4. Little-o Notation (oo)

• Represents a non-tight upper bound, meaning the algorithm grows slower than the
bound.

Example: If T(n) = n, then:

• o(n^2): True because n grows slower than n^2.

Recurrences

A recurrence relation is an equation that defines a sequence or function T(n) in terms of its
value(s) for smaller input size(s). Recurrences often arise when analyzing recursive
algorithms, as they describe the runtime or behavior of the algorithm.

Why Use Recurrences?

1. Analyze Recursive Algorithms: Understand their time complexity.

2. Divide-and-Conquer Algorithms: These algorithms naturally lend themselves to
recurrence relations (e.g., merge sort, quicksort).
3. Mathematical Modeling: Recurrences help define iterative relationships.

Solving Recurrences

There are several methods to solve recurrence relations:

Master Theorem
6. Probabilistic Analysis

Probabilistic analysis is the study of the expected performance of an algorithm by

analyzing the likelihood of different inputs or scenarios occurring. It is particularly useful
when the input data is random or when an algorithm’s behavior depends on randomness.

Key Concepts in Probabilistic Analysis

1. Average Case Analysis:

a. Instead of focusing on the worst-case or best-case scenarios, probabilistic
analysis evaluates the expected performance across all possible inputs.
b. This requires knowledge of the probability distribution of inputs.
2. Randomized Algorithms:
a. Some algorithms intentionally incorporate randomness into their logic.
Probabilistic analysis is essential to study their expected performance.
3. Expected Value:
a. Central to probabilistic analysis is the concept of expected value, which
represents the average outcome of an experiment over a large number of
trials.
4. Complex Inputs:
a. For complex input distributions, deterministic analysis may be infeasible,
and probabilistic techniques can help.

Steps in Probabilistic Analysis

1. Define the Input Distribution:

a. Identify or assume the probability distribution of inputs.
2. Define the Random Variable:
a. Define a variable (e.g., runtime or memory usage) whose behavior depends
on the input.
3. Compute the Expected Value:
a. Use probability and statistics to compute the expected runtime or resource
usage.
4. Interpret the Results:
a. Analyze the expected performance and compare it with worst-case or best-
case analysis.
Applications of Probabilistic Analysis

1. Data Structures
2. Algorithms
3. Machine Learning
4. Cryptography
5. Network Protocols

7. Disjoint Set Operations

A Disjoint Set (or Union-Find data structure) is a fundamental data structure used to
manage a collection of non-overlapping sets. It is commonly used in graph algorithms like
Kruskal’s Minimum Spanning Tree (MST) and connected component analysis.

1. Set Representation:
a. Each set is represented by a unique representative or leader (often the root
of a tree).

Operations

1. MakeSet(x)

• Creates a new set containing the single element x.

• Each element starts as its own parent (self-loop) with a rank of 0.
• Time Complexity: O(1).

2. Find(x)

• Finds the representative (or root) of the set containing x.

• Uses path compression to make subsequent operations faster.

Path Compression:

• As the algorithm traverses up the tree to find the root, it sets the parent of each
node directly to the root.
Time Complexity: O(α(n)), where α(n) is the inverse Ackermann function (extremely
slow-growing, nearly constant in practical scenarios).

3. Union(x, y)

• Merges the sets containing x and y.

• Uses union by rank to keep the tree as flat as possible.

Union by Rank:

• Attach the smaller tree under the larger tree based on rank.
• If ranks are equal, arbitrarily choose one root and increment its rank.

Time Complexity: O(α(n)).

Time Complexity of Operations

1. MakeSet: O(1)
2. Find: O(α(n))
3. Union: O(α(n))
• Overall Complexity: For m operations on n elements, the total time complexity is
O(mα(n)).

Applications

1. Dynamic Connectivity:
a. Determining if two elements are in the same set.
2. Clustering:
a. Grouping elements into disjoint subsets.
3. Network Analysis:
a. Tracking connected devices in a network.
4. Game Theory:
a. Simulating merges in dynamic systems.

UNIT 2

DIVIDE AND CONQUER: General method, applications-analysis of binary search, quick

sort, merge sort, AND OR Graphs. GREEDY METHOD: General method, Applications-job
sequencing with deadlines, Fractional knapsack problem, minimum cost spanning trees,
Single source shortest path problem.

Divide and Conquer

General Method:

1. Definition: Divide and Conquer is a paradigm that solves a problem by recursively

breaking it into smaller sub-problems, solving the sub-problems independently,
and combining their solutions to solve the original problem.
2. Steps:
a. Divide: Break the problem into smaller sub-problems of the same type.
b. Conquer: Solve each sub-problem recursively.
c. Combine: Merge the solutions of the sub-problems to form the solution to
the original problem.

Applications and Examples

1. Binary Search:

Problem: Search for an element in a sorted array.

Example:

• Array: [2, 4, 6, 8, 10, 12, 14], Target: 10.

Steps:

1. Divide: Compare the target with the middle element (8).

2. Conquer: Since 10 > 8, search in the right half ([10, 12, 14]).
3. Combine: Continue until the target (10) is found.

Result:

• Target found at index 4.

• Time Complexity: O(log⁡n).
2. Merge Sort:

Problem: Sort an array.

Example:

• Array: [38,27,43,3,9,82,10]

Steps:

1. Divide: Split the array into halves until single elements remain:
a. [38,27,43,3,9,82,10] → [38, 27, 43], [3, 9, 82, 10].
2. Conquer: Sort each half recursively:
a. [38, 27, 43] → [27, 38, 43],
b. [3, 9, 82, 10] → [3, 9, 10, 82].
3. Combine: Merge sorted halves:
a. [27, 38, 43] and [3, 9, 10, 82] → [3, 9, 10, 27, 38, 43, 82].

Result:

• Sorted Array: [3, 9, 10, 27, 38, 43, 82].

• Time Complexity: O(nlog⁡n).

3. Quick Sort:

Problem: Sort an array using a pivot.

Example:

• Array: [10, 80, 30, 90, 40, 50, 70] .

Steps:

1. Divide: Choose a pivot (70) and partition:

a. Left: [10, 30, 40, 50], Pivot: 70, Right: [80, 90].
2. Conquer: Recursively sort left and right sub-arrays:
a. [10, 30, 40, 50] → [10, 30, 40, 50],
b. [80, 90] → [80, 90].
3. Combine: Combine sorted arrays with the pivot:
a. [10, 30, 40, 50] + 70 + [80, 90].
Result:

• Sorted Array: [10, 30, 40, 50, 70, 80, 90] .

• Time Complexity: Average O(nlog⁡n), Worst O(n^2).

4. AND-OR Graphs: A graph representation where nodes represent problems, and

edges represent actions. AND nodes require all child nodes to succeed, while OR
nodes require any one child to succeed.
a. Application: Used in game trees, AI planning, and decision-making systems.

Greedy Method

General Method: The Greedy Method is an algorithmic approach where decisions are
made step-by-step, choosing the best option at each stage to achieve the global optimum.
This method works well for optimization problems that exhibit the greedy-choice property
(a locally optimal choice leads to a globally optimal solution) and optimal substructure
(optimal solutions to sub-problems contribute to the overall optimal solution).

1. Steps:
a. Selection: Choose the best option available based on a specific criterion.
b. Feasibility: Check if the choice is feasible within constraints.
c. Solution Building: Add the choice to the solution and repeat.

Applications:

1. Job Sequencing with Deadlines

Problem:

You have n jobs, each with a deadline and profit. Each job takes one unit of time, and only
one job can be scheduled at a time. The goal is to schedule jobs to maximize total profit
while meeting their deadlines.
Example:

• Jobs: J1,J2,J3,J4
• Deadlines: 2,1,2,1
• Profits: 100,19,27,25

Steps:

1. Sort jobs by profit in descending order: J1(100),J3(27),J4(25),J2(19).

2. Create a timeline with the maximum deadline = 2.
3. Schedule jobs:
a. J1: Slot 2 (Profit = 100).
b. J3: Slot 1 (Profit = 27).
c. Remaining jobs cannot be scheduled.

Result:

• Scheduled Jobs: J3,J1

• Total Profit: 100+27=127.

2. Fractional Knapsack Problem

Problem:

You are given n items, each with a weight and value. You must maximize the total value of
items in a knapsack with capacity W, allowing fractions of items.

Example:

• Items: I1,I2,I3
• Weights: 10,20,30
• Values: 60,100,120
• Knapsack Capacity: W=50.

Steps:

1. Compute value-to-weight ratio: I1=6,I2=5,I3=4.

 2. Sort items by ratio: I1,I2,I3.
3. Fill the knapsack:
a. Take I1 fully (10 kg, 60).
b. Take I2 fully (20 kg, 100).
c. Take 20/30 of I3 (20 kg, 80).

Result:

• Total Value: 60+100+80=240.

3. Minimum Cost Spanning Tree (MST)

Problem:

Connect all vertices of a graph with the minimum total edge weight.

Example (Prim's Algorithm):

• Graph: V={A,B,C,D}, Edges:

o A−B(1),A−C(3),B−C(2),C−D(4).

Steps:

1. Start at A. Add A−B(1).

2. Add B−C(2), as it’s the smallest edge connecting the tree to C.
3. Add C−D(4).

Result:

• MST: Edges A−B,B−C,C−D.

• Total Weight: 1+2+4=7.
4. Single Source Shortest Path Problem

Problem:

Find the shortest path from a source vertex to all others in a graph with positive edge
weights.

Example (Dijkstra's Algorithm):

• Graph: V={A,B,C,D} Edges:

o A−B(1), A−C(4), B−C(2), B−D(6), C−D(3).
• Source: A.

Steps:

1. Initialize distances: A=0,B=∞,C=∞,D=∞.

2. Start with A. Update distances: B=1,C=4.
3. Pick B (smallest distance). Update C=3,D=7.
4. Pick C. Update D=6.

Result:

• Shortest Distances: A=0,B=1,C=3,D=6.