Design and Analysis of Algorithms

Introduction

What is an Algorithm?

An algorithm is a step-by-step procedure for solving a problem. It's like a recipe for a
computer, providing a sequence of instructions to achieve a desired outcome.

Role of Algorithms in Computing

Algorithms are the backbone of computer science. They are essential for:

• Problem-solving: Algorithms provide systematic methods to tackle various computational

tasks.
• Efficiency: They help optimize resource usage, making computers faster and more efficient.
• Automation: Algorithms can automate repetitive tasks, saving time and effort.
• Innovation: They drive advancements in technology and create new possibilities.

Example:

• A simple algorithm to find the largest number in a list:

1. Start with the first number as the largest.
2. Compare each subsequent number with the current largest.
3. If the new number is larger, update the largest.
4. Repeat steps 2 and 3 until all numbers are checked.
5. The final largest number is the result.

Analysis of Algorithms

Nature of Input

The nature of the input data significantly impacts the algorithm's performance. Consider these
factors:

• Data Type: The type of data (e.g., numbers, strings, structures) affects the operations involved.
• Distribution: The distribution of values within the input can influence the algorithm's
efficiency.
• Constraints: Any constraints on the input (e.g., range, uniqueness) can be exploited for
optimization.

Size of Input

The size of the input data is a crucial factor in determining the algorithm's running time. Larger
inputs generally require more computational resources.

• Input Size: The number of elements or the amount of data in the input.
• Growth Rate: How the algorithm's running time increases with the input size.

Example:

• The simple algorithm to find the largest number has a linear growth rate. Its running time
increases linearly with the number of elements in the list.
Key Considerations:

• Worst-Case Analysis: Analyzing the algorithm's performance under the worst-possible input
conditions.
• Average-Case Analysis: Analyzing the performance for typical or average input cases.
• Best-Case Analysis: Analyzing the performance under the best-possible input conditions.

Summary

• Big O: Upper bound (worst-case).

• Big Ω: Lower bound (best-case).
• Big Θ: Exact growth rate.
• Little o: Grows slower.
• Little ω: Grows faster.

Sorting Algorithm Analysis

Sorting algorithms are evaluated based on several criteria:

1. Time Complexity

• Definition: The amount of time an algorithm takes to sort as the input size grows.
• Common Classes:
o O(n^2): Example: Bubble Sort, Selection Sort
o O(n log n): Example: Merge Sort, Quick Sort
o O(n): Example: Counting Sort (under certain conditions)

2. Space Complexity

• Definition: The amount of extra memory an algorithm uses.

• Types:
o In-place: Uses a constant amount of space (e.g., Quick Sort).
o Not in-place: Uses extra arrays or structures (e.g., Merge Sort).

3. Stability

• Definition: A stable sorting algorithm maintains the relative order of equal elements.
• Stable Algorithms: Merge Sort, Bubble Sort
• Unstable Algorithms: Quick Sort, Selection Sort

Loop Invariants

A loop invariant is a property that holds true before and after each iteration of a loop. It helps
in proving the correctness of an algorithm.

1. Definition

• A loop invariant is a condition that remains unchanged throughout the execution of the loop.
2. Importance

• Correctness: It helps show that the algorithm works as intended.

• Termination: It can help prove that the loop will eventually finish.

3. Example

Insertion Sort:

• Invariant: At the start of each iteration, the left side of the array is sorted.
• How it Works: Each new element is inserted into its correct position in the sorted portion,
maintaining the invariant.

Summary

• Sorting Analysis: Focuses on time complexity, space complexity, and stability.

• Loop Invariants: Help prove the correctness of algorithms by ensuring certain conditions
remain true throughout execution.

Recursion
1. Definition

• Recursion is a programming technique where a function calls itself to solve smaller instances
of the same problem.

2. Components

• Base Case: The condition under which the recursion stops. It prevents infinite loops.
• Recursive Case: The part of the function where the recursion occurs, breaking the problem into
smaller subproblems.

3. Example

• Factorial Function:
#include <iostream>

unsigned long long factorial(int n) {

if (n == 0) // Base case
return 1;
else // Recursive case
return n * factorial(n - 1);
}

int main() {
int number;
std::cout << "Enter a positive integer: ";
std::cin >> number;

if (number < 0) {
std::cout << "Factorial is not defined for negative numbers." << std::endl;
} else {
std::cout << "Factorial of " << number << " is " << factorial(number) << std::endl;
}

return 0;
}

Recurrence Relations
1. Definition

• Recurrence Relations are equations that define a sequence based on previous terms. They are
used to analyze the time complexity of recursive algorithms.

2. Formulation

• A recurrence relation expresses a term as a function of its predecessors. For example:

o T(n) = T(n-1) + O(1): This means the time to solve a problem of size n is the time to
solve size n-1 plus a constant amount of work.

3. Solving Recurrence Relations

• There are several methods to solve recurrence relations:

o Substitution Method: Guess the form of the solution and prove it.
o Master Theorem: Provides a way to analyze the complexity of divide-and-conquer
algorithms.
o Recursion Tree: Visualize the recursive calls and their costs.

4. Example

• For the Merge Sort algorithm:

o Recurrence Relation: T(n) = 2T(n/2) + O(n)
 ▪ This means it divides the problem into two halves (T(n/2)), and combines the
results in linear time (O(n)).

Summary

• Recursion: A method where a function calls itself to solve smaller problems.

• Recurrence Relations: Equations that express terms based on previous terms, used to analyze
the performance of recursive algorithms.

Algorithm Design Techniques

1. Divide and Conquer

o Breaks problems into smaller subproblems, solves each independently, and
combines their results.
2. Dynamic Programming
o Solves problems by storing results of overlapping subproblems to avoid
redundant calculations.
3. Greedy Algorithms
o Makes the best choice at each step with the hope of finding an overall optimal
solution.
4. Backtracking
o Builds solutions incrementally and abandons paths that don’t meet the
problem’s constraints.
5. Brute Force
o Tries all possible solutions to find the optimal one, often simple but inefficient.
6. Randomized Algorithms
o Uses random numbers in the process to simplify the algorithm and improve
performance.

These techniques provide various strategies for approaching and solving algorithmic problems
effectively.

Brute Force Approach

• Definition: Tries all possible solutions to find the best one.

• Example: Searching through all combinations or permutations.
• Pros: Simple to implement.
• Cons: Often inefficient for large problems.

Divide and Conquer Approach

• Definition: Breaks a problem into smaller, manageable subproblems, solves them

independently, and combines their results.
• Example: Used in algorithms like Merge Sort and Quick Sort.
• Pros: Efficient for many problems.
• Cons: Requires extra memory for recursion.
Merge Sort

• Definition: A sorting algorithm that divides an array into two halves, sorts each half, and
merges them.
• Steps:
1. Divide the array.
2. Recursively sort each half.
3. Merge the sorted halves.
• Time Complexity: O(n log n), stable.

Quick Sort

• Definition: A sorting algorithm that selects a "pivot" and partitions the array into elements less
than and greater than the pivot.
• Steps:
1. Choose a pivot.
2. Partition the array around the pivot.
3. Recursively sort the sub-arrays.
• Time Complexity: Average O(n log n), worst O(n²), not stable.

Greedy Approach

• Definition: Makes the best choice at each step, hoping to find the overall best solution.
• Example: Coin change problem or activity selection.
• Pros: Often simple and fast.
• Cons: Does not guarantee an optimal solution for all problems.

Dynamic Programming

• Definition: A method for solving complex problems by breaking them down into simpler
subproblems, solving each subproblem just once, and storing their solutions.

Key Elements of Dynamic Programming

1. Overlapping Subproblems
o Definition: The problem can be broken down into smaller, recurring subproblems.
o Example: In calculating the Fibonacci sequence, the same Fibonacci numbers are
calculated multiple times.
2. Optimal Substructure
o Definition: An optimal solution to the problem can be constructed from optimal
solutions of its subproblems.
o Example: In the shortest path problem, the shortest path to a destination can be found
by combining the shortest paths to intermediate points.
3. Memoization
o Definition: A top-down approach where results of subproblems are stored (cached) to
avoid redundant calculations.
o Example: Storing Fibonacci numbers once calculated.
4. Tabulation
o Definition: A bottom-up approach where solutions to subproblems are computed and
stored in a table.
o Example: Filling in a table for Fibonacci numbers starting from the base cases up to
the desired number.
 Search Trees

• Definition: Data structures that store data in a hierarchical manner, allowing for efficient
searching, insertion, and deletion.
• Types:
o Binary Search Tree (BST): Each node has at most two children; left child is less than
the parent, and right child is greater.
o Balanced Trees: Such as AVL trees or Red-Black trees, which maintain balance to
ensure efficient operations.
• Operations: Average time complexity for search, insert, and delete is O(log n).

Heaps

• Definition: A special tree-based structure that satisfies the heap property, which can be a max-
heap or min-heap.
• Properties:
o Max-Heap: The value of each parent node is greater than or equal to its children.
o Min-Heap: The value of each parent node is less than or equal to its children.
• Uses: Efficiently implements priority queues and is used in algorithms like Heap Sort.
• Operations: Insertion and deletion of the root node take O(log n) time.

Hashing

• Definition: A technique that converts data into a fixed-size value (hash code) to enable fast
data retrieval.
• Hash Table: A data structure that uses a hash function to map keys to values.
• Collision Handling:
o Chaining: Storing multiple items in the same bucket using a linked list.
o Open Addressing: Finding another open slot in the array for the colliding item.
• Time Complexity: Average case for search, insert, and delete is O(1), but can degrade to O(n)
in the worst case due to collisions.

Summary

• Search Trees: Hierarchical structures for efficient searching and sorting.

• Heaps: Specialized trees for priority queues with specific order properties.
• Hashing: Fast data retrieval method using hash codes, with mechanisms for handling
collisions.

Graph Algorithms

• Definition: Algorithms designed to solve problems related to graph structures (nodes and
edges).
• Key Concepts: Includes traversal (DFS, BFS), searching for shortest paths, and detecting
cycles.

Shortest Paths

• Definition: Algorithms that find the shortest path between nodes in a graph.
• Common Algorithms:
 o Dijkstra's Algorithm: Finds the shortest path from a source to all other nodes in a
weighted graph with non-negative weights. Time complexity: O((V + E) log V) with a
priority queue.
o Bellman-Ford Algorithm: Handles graphs with negative weights and finds the
shortest path from a single source. Time complexity: O(V * E).
o Floyd-Warshall Algorithm: Computes shortest paths between all pairs of nodes. Time
complexity: O(V³).

Sparse Graphs

• Definition: Graphs with relatively few edges compared to the number of vertices (E << V²).
• Properties: Efficient storage using adjacency lists rather than adjacency matrices.
• Implication: Many graph algorithms perform better on sparse graphs, as they can avoid
unnecessary computations associated with the large number of potential edges.

String Matching

• Definition: Algorithms used to find occurrences of a substring (pattern) within a larger string
(text).
• Common Algorithms:
o Naive Approach: Checks all possible positions for the pattern in the text. Time
complexity: O(n * m).
o Knuth-Morris-Pratt (KMP) Algorithm: Preprocesses the pattern to skip unnecessary
comparisons, improving efficiency. Time complexity: O(n + m).
o Rabin-Karp Algorithm: Uses hashing to find any one of a set of patterns in the text.
Time complexity: O(n + m) on average.

Summary

• Graph Algorithms: Solve problems related to graph structures, including traversal and
pathfinding.
• Shortest Paths: Focus on finding the minimal distance between nodes, using algorithms like
Dijkstra’s and Bellman-Ford.
• Sparse Graphs: Characterized by fewer edges, allowing for more efficient algorithms and data
storage.
• String Matching: Techniques to locate substrings within larger strings efficiently.

Introduction to Complexity Classes

Complexity classes are categories used to classify computational problems based on the
resources required to solve them, primarily focusing on time and space. Understanding these
classes helps in analyzing the efficiency of algorithms and determining the feasibility of
solving problems.

Key Complexity Classes

1. P (Polynomial Time)
o Definition: The class of decision problems that can be solved by a deterministic Turing
machine in polynomial time.
o Example: Sorting algorithms (like Merge Sort and Quick Sort) and finding the greatest
common divisor.
2. NP (Nondeterministic Polynomial Time)
o Definition: The class of decision problems for which a solution can be verified in
polynomial time by a deterministic Turing machine.
o Example: The Boolean satisfiability problem (SAT) and the Hamiltonian path
problem.
3. NP-Complete
o Definition: A subset of NP problems that are at least as hard as the hardest problems in
NP. If any NP-complete problem can be solved in polynomial time, then all problems
in NP can be solved in polynomial time.
o Example: Traveling salesman problem, 3-SAT, and the clique problem.
4. NP-Hard
o Definition: Problems that are at least as hard as the hardest problems in NP. They may
not be decision problems and do not have to belong to NP.
o Example: The halting problem and the optimization version of NP-complete problems.
5. BPP (Bounded-error Probabilistic Polynomial Time)
o Definition: The class of decision problems that can be solved by a probabilistic Turing
machine in polynomial time, with a bounded error probability.
o Example: Randomized algorithms like the Miller-Rabin primality test.
PAST PAPERS QUESTIONS
Solution

Question 2:

Part-a:

• The for loop iterates from i = 0 to i = n - 1.

• In each iteration, P is incremented by i.
• The loop terminates when P becomes greater than n.
• Therefore, the statement P = P + i; is executed n times.

Part-b:

• The outer for loop iterates from i = 0 to i = n - 1, executing n times.

• The inner for loop iterates from j = 1 to j = P - 1, executing P - 1 times in each
iteration of the outer loop.
• The value of P increases in each iteration of the outer loop.
• To calculate the total number of iterations of the inner loop, we can sum P - 1 for P
ranging from 1 to n:
o (1 - 1) + (2 - 1) + (3 - 1) + ... + (n - 1)
o This simplifies to (n - 1) * n / 2.
• Therefore, the statement in the second loop is executed (n - 1) * n / 2 times.

Question 3:

Part-a:

• Formal definition: f(n) = O(g(n)) if there exist positive constants c and n₀ such that
f(n) ≤ cg(n) for all n ≥ n₀.
• Examples:
o f(n) = 2n² and g(n) = n²: f(n) = O(g(n)) because 2n² ≤ 2n² for all n ≥
1.
o f(n) = n log n and g(n) = n²: f(n) = O(g(n)) because n log n ≤ n² for
all n ≥ 1.

Part-b:

• Formal definition: f(n) = Ω(g(n)) if there exist positive constants c and n₀ such that
f(n) ≥ cg(n) for all n ≥ n₀.
• Examples:
o f(n) = n² and g(n) = n: f(n) = Ω(g(n)) because n² ≥ n for all n ≥ 1.
o f(n) = 2^n and g(n) = n²: f(n) = Ω(g(n)) because 2^n ≥ n² for all n ≥
4.
Question 4:

Part-I:

a) T(n) = T(n - 1) + n:

• Expanding the recurrence: T(n) = T(n - 1) + n = T(n - 2) + (n - 1) + n =

... = T(0) + 1 + 2 + ... + n
• The sum 1 + 2 + ... + n is equal to n * (n + 1) / 2.
• Therefore, T(n) = 1 + n * (n + 1) / 2 = (n² + n + 2) / 2.

b) T(n) = 2T(n - 1) + 1:

• Expanding the recurrence: T(n) = 2T(n - 1) + 1 = 2(2T(n - 2) + 1) + 1 =

... = 2^n * T(0) + 2^(n - 1) + 2^(n - 2) + ... + 1
• The sum 2^(n - 1) + 2^(n - 2) + ... + 1 is equal to 2^n - 1.
• Therefore, T(n) = 2^n * T(0) + 2^n - 1 = 2^n + 2^n - 1 = 2^(n + 1) - 1.

Part-II:

a) T(n) = T(n/2) + n²:

• This recurrence falls under Case 2 of the Master theorem.

• a = 1, b = 2, f(n) = n², and log_b(a) = log_2(1) = 0.
• Since f(n) = n² is not polynomially smaller than n^0, we have T(n) = Θ(n² log n).

b) T(n) = 4T(n/2) + n:

• This recurrence falls under Case 3 of the Master theorem.

• a = 4, b = 2, f(n) = n, and log_b(a) = log_2(4) = 2.
• Since f(n) = n is polynomially smaller than n^2, we have T(n) = Θ(n^2).

Question 5:

Part-a:

• Directed graph:

Opens in a new window

www.geeksforgeeks.org
 directed graph based on the given adjacency list

• Adjacency matrix:
o [Matrix representation of the directed graph]

• BFS starting from Vertex-3:

o [BFS traversal steps and resulting visited vertices]

Part-b:

Bellman-Ford Algorithm

Pseudocode:

function BellmanFord(graph, source)

dist[source] = 0
for all other vertices v:
dist[v] = infinity

for i from 1 to (V - 1):

for every edge (u, v) in graph:
if dist[v] > dist[u] + weight(u, v):
dist[v] = dist[u] + weight(u, v)

for every edge (u, v) in graph:

if dist[v] > dist[u] + weight(u, v):
print("Negative cycle detected")
return

Explanation:

1. Initialization:
o Set the distance to the source vertex to 0.
o Set the distance to all other vertices to infinity.
2. Relaxation:
o Iterate over the edges of the graph V - 1 times (where V is the number of vertices).
o For each edge (u, v), check if the distance to v can be shortened by going through u.
If so, update the distance to v.
3. Negative Cycle Detection:
o After the main loop, iterate over the edges again to check for negative cycles. If a
distance can still be shortened, it indicates a negative cycle.

Applying Bellman-Ford Algorithm to Find Shortest Paths from Vertex S:

1. Input:
o The given graph (adjacency list or matrix).
o The source vertex S.
2. Run the algorithm:
o Call the BellmanFord function with the graph and S as input.
3. Output:
o If no negative cycle is detected, the dist array will contain the shortest distances from
vertex S to all other vertices.
 o If a negative cycle is detected, the algorithm will print a message indicating this.

Example:

Consider the following graph:

A -- 1 --> B -- 2 --> C
| |
| v
| D
| ^
| |
v |
E -- 3 --> F

To find the shortest paths from vertex A, we can apply the Bellman-Ford algorithm:

1. Initialize: dist[A] = 0, dist[B] = dist[C] = dist[D] = dist[E] = dist[F] =

infinity.
2. Relaxation:
o Iterate V - 1 = 5 times.
o Update distances as needed based on the edges.
3. Negative cycle detection:
o No negative cycle is found.

The final distances will be:

• dist[A] = 0
• dist[B] = 1
• dist[C] = 3
• dist[D] = 2
• dist[E] = 4
• dist[F] = 7

These values represent the shortest distances from vertex A to each other vertex in the graph.