DAA Unit –I

Algorithms and
Problem Solving
Outline

❏ Algorithm: The Role of Algorithms in Computing -

What are algorithms
❏ Algorithms as technology,
❏ Evolution of Algorithms,
❏ Design of Algorithm,
❏ Need of Correctness of Algorithm,
❏ Confirming correctness of Algorithm – sample
examples,
❏ Iterative algorithm design issues.
❏ Problem solving Principles: Classification of
Algorithm: The Role of Algorithms in
Computing - What are algorithms

● The term Algorithm was coined by the

Persian mathematician al-Khowarizmi in
the ninth century.
● The algorithm is a set of rules which are
used to solve real life problems.
● The algorithm provides a loose form of a
solution in a pseudo programming
language.
● Given the algorithm, it is easy to program
What are algorithms
We can treat an algorithm as a set of finite instructions which solves a
particular problem when applied it is applied to that problem with legal
inputs.
Algorithm:-
● The Algorithm is set of rules defined in specific order to do certain
computation and carry out some predefined task. It is a step
procedure to solve the problem.
● f the algorithm is correct, then the program should produce correct
output on valid input, otherwise, it should generate an appropriate
error message.
● For example, to find the division A/B, correctly written program
would return value of A/B for B >0, and it would show the error
message like "Invalid divisor" for B = 0.
 Algorithms
Properties/Characteristics of algorithms:

●Input from a specified set,

●Output from a specified set (solution),
●Definiteness of every step in the computation,
●Correctness of output for every possible input,
●Finiteness of the number of calculation steps,
●Effectiveness of each calculation step

5
Algorithm
s■A tool for solving a well-specified
computational problem

Input Algorithm Output

■Algorithms must be:

❑ Correct: For each input produce an appropriate output
❑ Efficient: run as quickly as possible, and use as little
memory as possible – more about this later

6
Algorithms Cont.
■A well-defined computational procedure that
takes some value, or set of values, as input and
produces some value, or set of values, as output.

■Written in a pseudo code which can be

implemented in the language of programmer’s
choice.

7
Correct and incorrect algorithms
■ Algorithm is correct if, for every input instance, it ends
with the correct output. We say that a correct algorithm
solves the given computational problem.

■ An incorrect algorithm might not end at all on some

input instances, or it might end with an answer other than
the desired one.

■ We shall be concerned only with correct algorithms.

8
Evaluation of Algorithm
Evaluation of Algorithm

● Though some people credit Babylonians with the

development of the first algorithms, it was the
unknown Indian mathematicians who developed
and used the concept of zero and decimal
positional number system. This allowed the
development of basic algorithms for arithmetic
operations, square roots, cube roots, and the
like.
● The famous sanskrit grammarian Panini gave
data structures like Maheshwar Sutra, which
Evaluation of Algorithm
● during 1940s and 50s, the emphasis was on
building the hardware, developing programming
systems so that the computers can be used in
commercial, scientific, and engineering problems.
● Though Alan Turing had given the idea of
effective procedure in 1936,
● Soon it was realized that systematic methods of
coding are required. This led to concepts like
structured programming.
● The next logical step was seeking the proof of
correctness of an algorithm, as many applications
were identified where proven correctness was
absolutely essential.
●
Evaluation of Algorithm
● Complexity theory classes was developed based
on Tractable and Intractable problems
● Another fact which emerged was that
randomness can be an aid in solving many
● difficult problems.
● A variety of algorithms came up which used
randomness as a basic component, for example,
genetic algorithms, simulated annealing, and
statistically defined algorithms. This area is also
of current interest.
Algorithm as a
technology
Algorithm as a Technology
Parameter Faster Computer A Slower Computer B
Speed Executes 10^10 Instructions Executes 10^7
per second instructions per second
Faster/ A is 1000 time faster than B B is 1000 time slower than
Slower A
Sorting Insertion sort Merge Sort
algorithms
run
Time c1 * n^2 c2 * n * logn
complexity
of Algorithm
Value of c1 is 2 c2 is 50
Constant in
time
complexity
Iterative Algorithm
Design Issues
Iterative Algorithm Design Issues

● Characteristics of most of the common and real life

algorithms, that is, they have at least one iterative
component or a loop.
● the idea is that a part of the algorithm statements will
be executed repeatedly thousands or even millions of
times.
● Even a small amount of excess time spent within the
loop can lead to a major loss in the efficiency of the
algorithm.
● The situation can even be worse if there are loops
within loops or nested iterations.
● There are algorithms in which the depth of nesting itself
is controlled by an outer loop!
Problem Solving
What is Problem Solving?

●Problem solving is the application

of ideas, skills, or factual
information to achieve the solution
to a problem or to reach a desired
outcome. Let's talk about different
types of problems and different
types of solutions.
Problem solving Strategies
1. Brute force is a straightforward approach to solve a problem based on the
problem’s statement and definitions of the concepts involved.
2. Greedy Algorithms The solution is constructed through a sequence of steps,
each expanding a partially constructed solution obtained so far. At each step
the choice must be locally optimal – this is the central point of this technique.
3. Divide-and-Conquer Given an instance of the problem to be solved, split this
into several smaller sub-instances (of the same problem), independently
solve each of the sub-instances and then combine the sub-instance solutions
so as to yield a solution for the original instance.
4. Dynamic Programming is a Bottom-Up Technique in which the smallest sub-
instances are explicitly solved first and the results of these used to construct
solutions to progressively larger sub-instances.
5. Backtracking and branch-and-bound: generate and test methods The
method is used for state-space search problems. The solving process solution
is based on the construction of a state-space tree, whose nodes represent
states, the root represents the initial state, and one or more leaves are goal
states.
Problem Solving Principal
Steps to solve Problem

●Identify a problem
●Understand the problem
●Identify alternative ways to solve a
problem
●Select beat way to solve a problem
from the list of alternative solutions
●Evaluate the solution
Time Complexity

of Algorithms
Algorithm Analysis

● Why we should analyze algorithms?

○Predict the resources that the algorithm requires
■Computational time (CPU consumption)
■Memory space (RAM consumption)
■Communication bandwidth consumption
○The running time of an algorithm is:
■The total number of operations executed
■Also known as algorithm complexity
25
Time Complexity
● Time complexity of an algorithm signifies the total time
required by the program to run to completion.
● Time complexity of an algorithm is measured by its rate
of growth relative to the standard function.
● Cases of time complexity are:
● Worst-case
○An upper bound on the running time for any input of
given size
● Average-case
○Assume all inputs of a given size are equally likely
● Best-case
○The lower bound on the running time 26
Time Complexity – Example
●Sequential search in a list of size n
○ Worst-case:
■n comparisons
○ Best-case:
■1 comparison
○ Average-case:
■n/2 comparisons
●The algorithm runs in linear time
○ Linear number of operations

27
Classification of Time Complexity
Notatio Complexit
Description Example
n y
O(1) Constant Simple statement Addition
O(log(n)) Logarithmic Divide in half Binary search
O(n) Linear loop Linear search
O(n*log(
Linearithmic Divide & Conquer Merge sort
n))
O(n2) Quadratic Double loop Check all pairs
Check all
O(n3) Cubic Triple loop
triples
Exhaustive Check all
O(2n) Exponential
search subsets
Recursive
O(n!) Factorial Factorial
function

28
Correctness of
Algorithm
Correctness of Algorithm

Correctness
of Algorithm

By
Using Loop
Mathematica
Invariant
l Induction
 What does an algorithm ?
•An algorithm is described by:
•Input data
•Output data
• Preconditions: specifies restrictions on input data
• Postconditions: specifies what is the result
•Example: Binary Search
• Input data: a:array of integer; x:integer;
• Output data: found:boolean;
• Precondition: a is sorted in ascending order
• Postcondition: found is true if x is ina, and
found
is false otherwise
 Correct algorithms
•An algorithm is correct if:
• for any correct input data:
• it stops and
• it produces correct output.

•Correct input data: satisfies

precondition
•Correct output data: satisfies
 Proving
correctness
• An algorithm = a list of actions
• Proving that an algorithm is totally
correct:
1. Proving that it will terminate
2. Proving that the list of actions
applied to the precondition
imply the postcondition
• This is easy to prove for simple
sequential algorithms
• This can be complicated to prove for
Example – a sequential
algorithm
Swap1(x,y):
aux := x
x := y
y := au
x
Precondition:
x = a and y = b
Postcondition:
x = b and y = a
Example – a repetitive algorithm
Algorithm Sum_of_N_numbers

Input: a, an array of N numbers Output: s,

the sum of the N numbers in a

s:=0;
k:=0; We use techniques
based on loop
While (k<N)
invariants and
do k:=k+1; induction
s:=s+a[k];
end
 Loop
invariants

•A loop invariant is a logical predicate

such that: if it is satisfied before
entering any single iteration of the
loop then it is also satisfied after the
iteration
Example: Loop invariant for
Sum of n numbers
Algorithm Sum_of_N_numbers

Input: a, an array of N numbers

Output: s, the sum of the N numbers in a

s:=0; Loop invariant =

k:=0; induction hypothesis:
While (k<N) do At step k, S holds the
k:=k+1; sum of the first k
numbers
s:=s+a[k];
end
Using loop
invariants in proofs
We must show the following 3
things about a loop invariant:

1. Initialization: It is true prior to the first

iteration of the loop.

2. Maintenance: If it is true before an

iteration of the loop, it remains true before
the next iteration.

3. Termination: When the loop terminates,

Example: Proving the
correctness of theSum
algorithm (1)
Induction hypothesis: S= sum of the
first k numbers

1. Initialization: The hypothesis is true at

the beginning of the loop:
Before the first iteration: k=0, S=0. The first 0
numbers have sum zero (there are no
Example: Proving the
correctness of theSum
algorithm (2)
Induction hypothesis: S= sum of
the first k numbers
2. Maintenance: If hypothesis is true before
step k, then it will be true before step k+1
(immediately after step k is finished)
We assume that it is true at beginning of step k: “S is the sum
of the first k numbers”
We have to prove that after executing step k, at the beginning
of step k+1:
“S is the sum of the first k+1 numbers”
We calculate the value of S at the end of this step
Example: Proving the
correctness of theSum
algorithm (3)
• Induction hypothesis: S= sum of the first
k numbers

3. Termination: When the loop

terminates,
the hypothesis implies the
correctness of the algorithm
 Loop invariants
and induction
• Proving loop invariants is similar to
mathematical induction:
• showing that the invariant holds before the
first iteration corresponds to the base case,
and showing that the invariant holds from
iteration to iteration corresponds to the
inductive step.
Mathematical induction -
Review
• Let T be a theorem that we want to
prove. T includes a natural
parameter n.
• Proving that T holds for all natural
values of n is done by proving
following two conditions:
1. T holds for n=1
1= Base case
2.2= Inductive
For every Step
n>1 if T holds for n-1,
then T holds for n
 Mathematical induction -
Review
• Strong Induction: a variant of
induction where the inductive step
builds up on all the smaller values
• Proving that T holds for all natural
values of n is done by proving
following two conditions:
1. T holds for n=1
2. For every n>1 if T holds for all
Mathematical induction review –
Example1
• Theorem:The sum of the first n natural
numbers is n*(n+1)/2
• Proof: by induction on n
1. Base case: If n=1, s(1)=1=1*(1+1)/2
2. Inductive step: We assume that
s(n)=n*(n+1)/2,
and prove that this implies
s(n+1)=(n+1)*(n+2)/2 , for all n>=1
Mathematical induction review –
Example2
•Theorem: Every amount of postage that is
at least 12 cents can be made from 4-cent
and 5-cent stamps.
•Proof: by induction on the amount of
postage
•Postage (p) = m * 4 + n * 5
•Base case:
•Postage(12) = 3 * 4 + 0 * 5
•Postage(13) = 2 * 4 + 1 * 5
Mathematical induction review –
Example2 Continue
•Inductive step: We assume that we can
construct postage for every value from 12 up
to k. We need to show how to construct k + 1
cents of postage. Since we have proved base
cases up to 15 cents, we can assume that k +
1 ≥ 16.

•Since k+1 ≥ 16, (k+1)−4 ≥ 12. So by the

inductive hypothesis, we can construct
postage for (k + 1) − 4 cents: (k + 1) − 4 = m
Correctness of
algorithms
•Induction can be used for proving
the correctness of repetitive
algorithms:
• Iterative algorithms:
• Loop invariants
• Induction hypothesis = loop invariant =
relationships between the variables during loop
execution
• Recursive algorithms
• Direct induction
 Example: Correctness proof
for Decimal to Binary
Conversion-
Algorithm Decimal_to_Binary
Input: n, a positive integer
Output: b, an array of bits, the bin repr. of n, starting
t:=n; with the least significant bits
k:=0;
It is a repetitive (iterative)
While (t>0)
do
algorithm, thus we use
k:=k+1; loop invariants and proof
b[k]:=t mod by induction
2;
t:=t div 2;
end
 Example: Correctness proof
for Decimal to Binary
Conversion-
Algorithm Decimal_to_Binary
Input: n, a positive integer
Output: b, an array of bits, the bin repr. of n, starting
t:=n; with the least significant bits
At step k, b holds the k least
k:=0;
significant bits of n,and the
While (t>0)
do value of t, when shifted by k,
k:=k+1; corresponds to the rest of
b[k]:=t mod the bits
2;
t:=t div 2;
end
 Example: Correctness proof
for Decimal to Binary
Conversion-
Algorithm Decimal_to_Binary
Input: n, a positive integer
Output: b, an array of bits, the bin repr. of n, starting
t:=n; with the least significant bits
Loop invariant: If m is the
k:=0;
integer represented by array
While (t>0)
do b[1..k], then n=t*2k+m
k:=k+1;
b[k]:=t mod
2;
t:=t div 2;
end
 Example: Proving the
correctness of the
conversion algorithm
• Induction hypothesis=Loop
Invariant: If m is the integer
represented by array b[1..k], then
n=t*2^k+m
• To prove the correctness of the
algorithm, we have to prove the
3 conditions:
1. Initialization: The hypothesis is
true at the beginning of the loop
2. Maintenance: If hypothesis is true for
Example: Proving the
correctness of the
conversion algorithm (1)

• Induction hypothesis: If m is the

integer represented by array b[1..k],
then n=t*2^k+m

1. The hypothesis is true at the

beginning of the loop:
k=0, t=n, m=0(array is empty)
Example: Proving the
correctness of the
conversion algorithm (2)
• Induction hypothesis: If m is the integer
represented by array b[1..k], then
n=t*2^k+m
2. If hypothesis is true for step k, then it will
be true for step k+1
At the start of step k: assume that n=t*2^k+m,
calculate the values at the end of this step
If t=even then: t mod 2==0, m unchanged, t=t
/ 2, k=k+1=> (t / 2) * 2 ^ (k+1) + m =
t*2^k+m=n
If t=odd then: t mod 2 ==1, b[k+1] is set to 1,
Example: Proving the
correctness of the
conversion algorithm (3)
• Induction hypothesis: If m is the
integer represented by array b[1..k],
then n=t*2^k+m
3. When the loop terminates, the
hypothesis implies the correctness
of the algorithm
The loop terminates when t=0
=> n=0*2^k+m=m
n==m, proved
Proof of Correctness for
Recursive Algorithms
• In order to prove recursive algorithms,
we have to:
1. Prove the partial correctness (the fact that
the program behaves correctly)
• we assume that all recursive calls with
arguments that satisfy the preconditions behave
as described by the specification, and use it to
show that the algorithm behaves as specified
Example - Merge Sort
MERGE-SORT(A,p,r) p q r

MERGE-SORT(A,p,q)
MERGE-SORT(A,q+1,r)
MERGE(A,p,q,r)

Precondition:

Postcondition:
Correctness proofs for
Recursive Algorithm
n , n , … n are some values smaller than n but
1 2 r

bigger than small_value

• Base Case: Prove that RECURSIVE works for n=

small_value

•Inductive Hypothesis:
• Assume that RECURSIVE works correctly for
n=small_value, ..., k

•Inductive Step:
•Proving that an algorithm is totally
correct means:
1.Proving that it will terminate
2.Proving that the list of actions applied
to the
precondition imply the postcondition

• How to prove repetitive algorithms:

• Iterative algorithms: use Loop
invariants, Induction