Design and Analysis of Algorithm

KDU-SEN 204
 Outlines
• Basic of an Algorithm
• Methodology of Analysis
• Divide and Conquer
• Greedy Algorithm
• Approximation Algorithms
• Graph Theory
• Heap Algorithms
• Sorting Techniques
• Searching Techniques
Basic of an Algorithm

Week 1
 BASIC ALGORITHM CONCEPTS
1.0 Introduction
2.0 Objectives
3.0 What is an Algorithm?
3.1 Structure of an Algorithm
3.2 Characteristics of an Algorithm
3.2.1 Advantages of Algorithms
3.2.2 Disadvantages of Algorithms
3.3 How to write an Algorithms
3.4 Need of Algorithm
3.5 Measures need to be defined when comparing Algorithm
3.6 Reason for Analyzing Algorithms
3.7 Pseudocode
3.7.1 Advantages of Pseudocode
3.7.2 Disadvantages of Pseudocode
3.7.3 Differences between Algorithm and Pseudocode
3.7.4 Problem Case/ Example
 INTRODUCTION
• Algorithm and complexity analysis is a very fundamental topic of
computer science today.
• An algorithm can be defined as a finite set of steps, which has to
be followed while carrying out a particular problem. It is
nothing but a process of executing actions step by step.
• Every computer science student should know how to define an
algorithm and they should also know how to measure the
usefulness of that Algorithm.
• More precisely, an algorithm is correct, if, for each input instance,
it gets the correct output and gets terminated.
• The concept of an algorithm originated as a means of recording
procedures for solving mathematical problems such as finding
the common divisor of two numbers or multiplying two
numbers.
 OBJECTIVES
By the end of this unit, you should be able to:
• Define and describe what an algorithm is
• Enumerate the different characteristics of an algorithm
• Examine some of the advantages of algorithms
• Identify some shortcomings or disadvantages of
algorithms
• Look at the concept of a pseudocode
• Examine some benefits and shortcomings of a pseudocode
• Make a comparison between and algorithm and a
pseudocode
• Look at the various reasons why an algorithm is needed.
 Algorithm
• An algorithm is a step-by-step procedure or a set of
rules/instructions designed to solve a specific problem or
accomplish a task. It's a fundamental concept in computer
science and various other fields, providing a structured
approach to problem-solving.
• From the definition we have two (2) keywords:
1) Step-by-step instructions:
– Algorithms are essentially a list of instructions or rules that, when
followed in a specific order, lead to a desired outcome.
2) Problem-solving tool:
– They are used to solve problems or perform computations in a
systematic way
• They are used by both human beings and computers.
Examples of Algorithms for Humans
• Common examples of algorithms for humans
include:
 Examples in Computer Science:
• Algorithms are used in a wide variety
of applications, including:
–Searching,
–Sorting,
–Data analysis,
–Machine learning, and more
 Structure of an Algorithm
An algorithm has the following structure
1. Input Step
2. Assignment Step
3. Decision Step
4. Repetitive Step
5. Output Step
 Characteristics/Properties of an Algorithm
• An algorithm should have the following
characteristics or properties −
– Input: Algorithms typically take one or more inputs as data.
– Output: They produce a specific output or result based on
the input and the steps in the algorithm.
– Definiteness: Each step in an algorithm must be clearly and
unambiguously defined.
– Effectiveness: The algorithm should be able to produce the
correct output for the given input.
– Finiteness: The algorithm must terminate after a finite
number of steps.
– It should be written in simple English.
• Generality: An algorithm must be generic enough to solve
all problems of a particular class.
• Feasibility − Should be feasible with the available resources.
• Flexibility: It must be flexible enough to carry out desired
changes with no efforts.
• Efficient: The term efficiency is measured in terms of time
and space required by an algorithm to implement. Thus, an
algorithm must ensure that it takes little time and less
memory space meeting the acceptable limit of
development time.
• Independent: An algorithm must be language independent,
which means that it should mainly focus on the input and
the procedure required to derive the output instead of
depending upon the language.
 Advantages of Algorithms
• Effective Communication: Since it is written in a natural
language like English, it becomes easy to understand the
step-by step delineation of a solution to any particular
problem.
• Easy Debugging: A well-designed algorithm facilitates
easy debugging to detect the logical errors that occurred
inside the program.
• Easy and Efficient Coding: An algorithm is nothing but a
blueprint of a program that helps develop a program.
• Independent of Programming Language: Since it is a
language-independent, it can be easily coded by
incorporating any high-level language.
 Disadvantages of Algorithms
• Developing algorithms for complex problems
would be time consuming and difficult to
understand.
• It is a challenging task to understand complex
logic through algorithms.
 Common Algorithm Types:
1) Searching Algorithms: Used to find specific
data within a collection of data, such as linear
search or binary search.
2) Sorting Algorithms: Used to arrange data in a
specific order, such as bubble sort, insertion
sort, or merge sort.
3) Divide and Conquer: This approach breaks
down a problem into smaller subproblems,
solves them independently, and then
combines the solutions.
4) Dynamic Programming: This technique
solves complex problems by breaking them
into smaller, overlapping subproblems and
storing the solutions to avoid redundant
calculations.
5) Greedy Algorithms: These algorithms make
locally optimal choices at each step with the
hope of finding a globally optimal solution.
6) Brute Force Algorithms: These algorithms
try all possible solutions to find the best
one.
 Algorithm Design Considerations:
• Efficiency: An algorithm's performance,
measured by its time and space complexity, is
a crucial factor.
• Accuracy: The algorithm should produce the
correct output for all valid inputs.
• Readability: An algorithm should be easy to
understand and maintain.
• Scalability: The algorithm should be able to
handle increasing amounts of data or
complexity.
 How to Write an Algorithm
• Example 1:
 Example 2 of an Algorithm
• Design an Algorithm to find the area of
triangle.
• Steps
1. START
2. READ BASE LENGTH, HEIGHT
3. AREA = ½ * Base length * Height
4. PRINT BASE LENGTH, HEIGHT, AREA
5. STOP
• Example3: Write an algorithm to compute the
area and circumference of a cycle given the
diameter d. Use the formular A= πr^2 and C= 2πr

• Solution
Step 1: Start
Step 2: Get the diameter d
Step 3: Compute r=(d/2)
Step 4: Compute A= πr^2
Step 5: Compute C= 2πr
Step 6: Display the results
Step 7: Stop
• Example 4: Write an algorithm that tells you
how to wash dishes

• Solution
Step 1: start
Step 2: scrape food off dishes
Steps 3: wash the dishes with soap and water
Step 4: Rinse the dishes
Step 5: Dry them
Step 6: Stop
• Example 5: Write an algorithm to evaluate the
equation y = a(b-c)^2/d+2
• Solution
Step 1: start
Step 2: Input the value of a, b, c, d
Step 3: The value of y is to be calculated
Step 4: Calculate the value of b-c and denote f
• Step 5: Calculate the square of f
Step 6: Multiply f by a and denote g
Step 7: Calculate the value of d+2 and denote h
Step 8: divide h by g
Step 9: We get the value of y
Step 10: Print y
Step 11: Stop
 Algorithm Cont.
• From the data structure point of view, following
are some important categories of algorithms −
– Search − Algorithm to search an item in a data
structure.
– Sort − Algorithm to sort items in a certain order.
– Insert − Algorithm to insert item in a data structure.
– Update − Algorithm to update an existing item in a
data structure.
– Delete − Algorithm to delete an existing item from a
data structure.
 Need of Algorithms (Why do we need Algorithms?)
1. To understand the basic idea of the problem.
2. To find an approach to solve the problem.
3. To improve the efficiency of existing techniques.
4. To understand the basic principles of designing the
algorithms.
5. To compare the performance of the algorithm with respect
to other techniques.
6. It is the best method of description without describing the
implementation detail.
7. The Algorithm gives a clear description of requirements and
goal of the problem to the designer.
8. A good design can produce a good solution.
9. To understand the flow of the problem.
10. To measure the behavior (or performance) of the
methods in all cases (best cases, worst cases, average cases)
11. With the help of an algorithm, we can also identify the
resources (memory, input-output) cycles required by the
algorithm.
12. With the help of algorithm, we convert art into a science.
13. To understand the principle of designing.
14. We can measure and analyze the complexity (time and
space) of the problems concerning input size without
implementing and running it; it will reduce the cost of design.
 Reasons for analyzing algorithms:
1. To predict the resources that the algorithm
requires
– Computational Time (CPU consumption).
– Memory Space (RAM consumption).
– Communication bandwidth consumption.
2. To predict the running time of an algorithm
– Total number of primitive operations executed.
 Summary
• In summary, algorithms are the
foundation of problem-solving in
computer science and other
fields, providing a structured and
systematic approach to achieving
desired outcomes.
 Pseudocode
• In programming, pseudocode is a simplified,
informal way to outline the logic of an algorithm
or program using plain English, acting as a
blueprint before actual code is written.
• Purpose:
• Pseudocode helps programmers plan and design
algorithms and programs by focusing on the
logic and structure without needing to worry
about specific syntax or a particular
programming language.
 Characteristics of Pseudocode:
• It's written in a natural language (like English)
rather than a formal programming language.
• It's not meant to be compiled or executed by a
computer; it's for human readability and
understanding.
• It uses keywords and phrases that are commonly
understood by programmers, such as "INPUT,"
"OUTPUT," "WHILE," "FOR," "IF," "THEN," "ELSE".
• It's a flexible tool, allowing for easy modification
and refinement of the algorithm's logic.
 Benefits of Pseudocode :
• Improved Communication: Pseudocode facilitates clear
communication of algorithm logic among programmers, even
those who may not be familiar with the same programming
language.
• Enhanced Planning: It helps in the initial planning and design of
algorithms, allowing programmers to think through the
problem and its solution before writing any actual code.
• Debugging: By focusing on the logic first, pseudocode can help
identify potential problems or errors in the algorithm's design
before the code is written.
• Learning: Pseudocode is a great tool for learning programming
concepts and algorithms, as it allows learners to focus on the
underlying logic without getting bogged down in syntax.
Example of a Pseudocode
 Advantages of Pseudocode
• It can be quickly transformed into an actual
programming language than a flowchart since it
is similar to a programming language.
• The layman or user can easily understand it.
• It can be easily modified as compared to
flowcharts.
• Its implementation is beneficial for structured,
designed elements.
• It can easily detect an error before transforming
it into a code.
 Disadvantages of Pseudocode
• Since it does not incorporate any standardized
style or format, it can vary from one user or
programmer to another.
• Error possibility is higher while transforming into
a code.
• It may require a tool for extracting out the
Pseudocode and facilitate the drawing
flowcharts.
• It does not depict/show the design.
 Difference between Algorithm and Pseudocode

• An algorithm is simply a problem-solving process,

which is used not only in computer science to write a
program but also in our day to day life. It is nothing
but a series of instructions to solve a problem or get
to the problem's solution. It not only helps in
simplifying the problem but also to have a better
understanding of it.
• However, Pseudocode is a way of writing an
algorithm. Programmers can use informal, simple
language to write pseudocode without following any
strict syntax. It encompasses semi-mathematical
statements.
Problem Case/Example:
 SUMMARY
• In this unit we have considered an overview of
algorithms and their basic characteristics. In
addition, we looked at some of the benefits and
shortcomings of algorithms and also examined
the concept of a pseudocode as well as some of
its benefits and shortcomings. We also made a
brief comparison between a pseudocode and an
algorithm and finally looked at some of the
reasons why an algorithm is needed
 Self-Assessment Exercise
1. What is an algorithm?
2. Differentiate between an algorithm and a
pseudocode
3. Highlight some of the basic reasons why algorithms
are needed?
4. How is an algorithm similar to and different from a
program?
5. Why must every good computer programmer
understand an algorithm first?
6. State an algorithm for adding three numbers A, B,
and C
 TUTOR MARKED ASSIGNMENT
1. Explain the following terms; (a) Algorithms (b)
Pseudocode (c) Computer Programs
2. State five properties or features of an algorithm.
3. State some basic differences between an
algorithm and a pseudocode and also between
an algorithm and a computer program
4. Give four benefits each of an algorithm and a
pseudocode
Methodology of Analysis

Week 2
How to Measure an Algorithm's Resource Consumption

•To measure an algorithm's resource

consumption, you can analyze its time
complexity (how execution time scales
with input size) and space complexity
(how memory usage scales with input
size), often using techniques like
asymptotic analysis and empirical testing
with profiling tools.
 Here's a more detailed explanation:
1. Understanding Resource Consumption:
a) Time Complexity: This focuses on how the execution time of an
algorithm grows as the input size increases.
• Asymptotic Analysis: This is a common method to analyze time
complexity, focusing on the algorithm's behavior as the input size
becomes very large.
• Big O Notation: A mathematical notation used to describe the upper
bound of an algorithm's runtime, e.g., O(n), O(n^2), O(log n).
b) Space Complexity: This focuses on how the amount of memory an
algorithm uses grows as the input size increases.
• Factors to consider: The memory required to hold the code, data
structures, and variables used by the algorithm.
• Empirical Testing: Running the algorithm with different input sizes
and measuring the memory usage using profiling tools.
2. Measurement Techniques:
a) Asymptotic Analysis:
• Worst-case, average-case, and best-case scenarios: Analyzing
the algorithm's performance in different scenarios.
• Focus on the dominant term: Ignoring constant factors and
focusing on the term that grows fastest as the input size
increases.
b) Empirical Testing:
• Real-world data: Running the algorithm on actual data and
measuring its performance.
• Profiling tools: Using tools to measure the execution time
and memory usage of the algorithm.
• Examples of tools: Performance Monitor (Windows), JMeter,
LoadRunner, Gatling, New Relic.
 Other resources:
• Network usage: Measuring the amount of data
transmitted or received by the algorithm.
• Disk usage: Measuring the amount of temporary
or long-term disk space used by the algorithm.
• Power consumption: Measuring the energy used
by the algorithm.
3. Tools and Techniques:
• Profiling Tools:
– Built-in tools: Many programming languages and
environments have built-in profiling tools.
– Third-party tools: Tools like JMeter, LoadRunner, and New
Relic can be used for performance testing and resource
monitoring.
• Resource Utilization Metrics:
– CPU utilization: The percentage of time the CPU is busy
executing the algorithm.
– Memory usage: The amount of RAM used by the algorithm.
– Disk I/O: The amount of data read from or written to the
disk by the algorithm.
– Network traffic: The amount of data transmitted or
received by the algorithm over the network.
4. Example:
• Sorting Algorithm:
– Time Complexity: A sorting algorithm's time
complexity can be measured by how quickly it sorts a
list of varying lengths.
– Space Complexity: The amount of memory used by
the sorting algorithm to store temporary data
structures or the sorted list.
 Methodology of Analysis
•To measure resource consumption of an
algorithm, different strategies are used as discussed
in this chapter.
•Asymptotic Analysis
•Solving Recurrence Equations
•Amortized Analysis
1. Asymptotic Analysis
•Asymptotic analysis is a method used in
mathematics and computer science to understand
the behavior of algorithms and functions as their
input size grows very large, focusing on their
growth rate or time complexity.
•It uses asymptotic notations like Big O, Big Theta,
and little o to describe these growth patterns.
 Types of Asymptotic Notations:
• Big O (O): The notation Ο(n) is the formal method of
expressing the upper bound of an algorithm's running
time. It measures the worst case time complexity or the
longest amount of time an algorithm can possibly take
to complete.
• Big Theta (Θ): Describes the average-case or tight bound
of an algorithm's growth rate. The notation θ(n) is the
formal way of expressing both the lower bound and the
upper bound of an algorithm's running time.
• Little o (o): Describes how two functions behave as their
input approaches infinity, specifically when one function
grows slower than another.
 Benefits of Asymptotic Analysis :
• Algorithm Comparison: Allows us to compare
the efficiency of different algorithms based on
their growth rates.
• Scalability Analysis: Helps us understand how
well an algorithm will perform with increasing
input sizes.
• Resource Prediction: Provides a simplified
model of resource consumption (time or space)
 Applications of Asymptotic Analysis :
• Computer Science: Used extensively in
algorithm design, analysis, and data structures.
• Mathematics: Used in the study of functions and
their limiting behavior.
 Another Example
• In computer science in the analysis of algorithms,
considering the performance of algorithms when
applied to very large input datasets.
• The simplest example is a function ƒ (n) = n2+3n,
the term 3n becomes insignificant compared to
n2 when n is very large. The function "ƒ (n) is said
to be asymptotically equivalent to n2 as n → ∞",
and here is written symbolically as
• ƒ (n) ~ n2.
 2. Solving Recurrence Equations
•A recurrence relation is an equation that defines a sequence
based on a rule that gives the next term as a function of the
previous term(s). It helps in finding the subsequent term (next
term) dependent upon the preceding term (previous term). If we
know the previous term in a given series, then we can easily
determine the next term.

 A recurrence relation can be solved using the following methods:

 Substitution Method
• Recursion Tree Method
• Master’s Theorem
 Substitution Method
• The substitution method is a condensed
way of proving an asymptotic bound on
a recurrence by induction. (i.e In this
method, we guess a bound and using mathematical
induction we prove that our assumption was correct.
• In the substitution method, instead of trying
to find an exact closed-form solution, we
only try to find a closed-form bound on the
recurrence.
• For example, if there are two equations
x+y=7 and x-y=8, then from the first
equation we can find that x=7-y.
Example of Substitution Method
 Recursion Tree Method
• The Recursion Tree Method is a way of
solving recurrence relations.
• In this method, a recurrence relation is converted into
recursive trees. Each node represents the cost incurred
at various levels of recursion. To find the total cost,
costs of all levels are summed up.
• Steps to solve recurrence relation using recursion
tree method:
1. Draw a recursive tree for given recurrence relation
2. Calculate the cost at each level and count the total no
of levels in the recursion tree.
3. Count the total number of nodes in the last level and
calculate the cost of the last level
4. Sum up the cost of all the levels in the recursive tree
Example of Recursion Tree Method