DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING

UNIT – I
Analysis of Algorithm:

I N T R O D U C T I O N – A N A LY Z I N G CONTROL STRUCTURES-
AV E R A G E C A S E A N A LY S I S -S O LV I N G RECURRENCES.

ALGORITHM

Informal Definition:
An Algorithm is any well-defined computational procedure that takes some value or set of
values as Input and produces a set of values or some value as output. Thus algorithm is a sequence of
computational steps that transforms the i/p into the o/p.

Formal Definition:
An Algorithm is a finite set of instructions that, if followed, accomplishes a particular task.
In addition, all algorithms should satisfy the following criteria.

1. INPUT à Zero or more quantities are externally supplied.

2. OUTPUT à At least one quantity is produced.
3. DEFINITENESS à Each instruction is clear and unambiguous.
4. FINITENESS à If we trace out the instructions of an algorithm, then for all cases, the algorithm
terminates after a finite number of steps.
5. EFFECTIVENESS à Every instruction must very basic so that it can be carried out, in principle,
by a person using only pencil & paper.

Issues or study of Algorithm:

 How to device or design an algorithm à creating and algorithm.

 How to express an algorithm à definiteness.
 How to analysis an algorithm à time and space complexity.
 How to validate an algorithm à fitness.
 Testing the algorithm à checking for error.

Algorithm Specification:

Algorithm can be described in three ways.

0 Natural language like English:

When this way is chooses care should be taken, we should ensure that each & every statement is
definite.

2. Graphic representation called flowchart:

This method will work well when the algorithm is small& simple.

3. Pseudo-code Method:
In this method, we should typically describe algorithms as program, which
resembles language like Pascal & algol.

B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND

TECHNOLOGY
DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING
Pseudo-Code Conventions:

1. Comments begin with // and continue until the end of line.

2. Blocks are indicated with matching braces {and}.

3. An identifier begins with a letter. The data types of variables are not explicitly declared.

4. Compound data types can be formed with records. Here is an example,

Node. Record
{
data type – 1 data-1;
.
.
.
data type – n data – n;
node * link;
}

Here link is a pointer to the record type node. Individual data items of a record can be
accessed with à and period.

5. Assignment of values to variables is done using the assignment statement.

<Variable>:= <expression>;

6. There are two Boolean values TRUE and FALSE.

à Logical Operators AND, OR, NOT

àRelational Operators <, <=,>,>=, =, !=

7. The following looping statements are employed.

For, while and repeat-until

While Loop:
While < condition > do
{
<statement-1>
.
.
.

<statement-n>
}

For Loop:
For variable: = value-1 to value-2 step step do

{
<statement-1>
B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND
TECHNOLOGY
DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING
.
.
.
<statement-n>
}
repeat-until:

repeat
<statement-1>
.
.
.
<statement-n>
until<condition>

8. A conditional statement has the following forms.

à If <condition> then <statement>

à If <condition> then <statement-1>
Else <statement-1>

Case statement:

Case
{
: <condition-1> : <statement-1>
.
.
.
: <condition-n> : <statement-n>
: else : <statement-n+1>
}

9. Input and output are done using the instructions read & write.

10. There is only one type of procedure:

Algorithm, the heading takes the form,

Algorithm Name (Parameter lists)

à As an example, the following algorithm fields & returns the maximum of ‘n’ given numbers:

1. algorithm Max(A,n)
2. // A is an array of size n
3. {
4. Result := A[1];
5. for I:= 2 to n do
6. if A[I] > Result then
7. Result :=A[I];
B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND
TECHNOLOGY
DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING
8. return Result;
9. }

In this algorithm (named Max), A & n are procedure parameters. Result & I are Local variables.

à Next we present 2 examples to illustrate the process of translation problem into an algorithm.

Selection Sort:

 Suppose we Must devise an algorithm that sorts a collection of n>=1 elements of arbitrary
type.

 A Simple solution given by the following.

 ( From those elements that are currently unsorted ,find the smallest & place it next in the
sorted list.)

Algorithm:

1. For i:= 1 to n do
2. {
3. Examine a[I] to a[n] and suppose the smallest element is at a[j];
4. Interchange a[I] and a[j];
5. }

à Finding the smallest element (sat a[j]) and interchanging it with a[ i ]

 We can solve the latter problem using the code,

t := a[i];
a[i]:=a[j];
a[j]:=t;

 The first subtask can be solved by assuming the minimum is a[ I ];checking a[I] with
a[I+1],a[I+2]…….,and whenever a smaller element is found, regarding it as the new
minimum. a[n] is compared with the current minimum.

 Putting all these observations together, we get the algorithm Selection sort.

Theorem:
Algorithm selection sort(a,n) correctly sorts a set of n>=1 elements .The result remains is a
a[1:n] such that a[1] <= a[2] ….<=a[n].

Selection Sort:
Selection Sort begins by finding the least element in the list. This element is moved to the
front. Then the least element among the remaining element is found out and put into second position.
This procedure is repeated till the entire list has been studied.

B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND

TECHNOLOGY
DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING

Example:

LIST L = 3,5,4,1,2

1 is selected , à 1,5,4,3,2
2 is selected, à1,2,4,3,5
3 is selected, à1,2,3,4,5
4 is selected, à1,2,3,4,5

Proof:
 We first note that any I, say I=q, following the execution of lines 6 to 9,it is the case that
a[q] Þ a[r],q<r<=n.
 Also observe that when ‘i’ becomes greater than q, a[1:q] is unchanged. Hence, following
the last execution of these lines (i.e. I=n).We have a[1] <= a[2] <=……a[n].
 We observe this point that the upper limit of the for loop in the line 4 can be changed to n-
1 without damaging the correctness of the algorithm.

Algorithm:

1. Algorithm selection sort (a,n)

2. // Sort the array a[1:n] into non-decreasing order.
3.{
4. for I:=1 to n do
5. {
6. j:=I;
7. for k:=i+1 to n do
8. if (a[k]<a[j])
9. t:=a[I];
10. a[I]:=a[j];
11. a[j]:=t;
12. }
13. }

Recursive Algorithms:

 A Recursive function is a function that is defined in terms of itself.

 Similarly, an algorithm is said to be recursive if the same algorithm is invoked in the body.
 An algorithm that calls itself is Direct Recursive.
 Algorithm ‘A’ is said to be Indirect Recursive if it calls another algorithm which in turns
calls ‘A’.
 The Recursive mechanism, are externally powerful, but even more importantly, many
times they can express an otherwise complex process very clearly. Or these reasons we
introduce recursion here.
 The following 2 examples show how to develop a recursive algorithms.

B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND

TECHNOLOGY
DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING
à In the first, we consider the Towers of Hanoi problem, and in the second, we
generate all possible permutations of a list of characters.

1. Towers of Hanoi:

.
.
.

Tower A Tower B Tower C

 It is Fashioned after the ancient tower of Brahma ritual.

 According to legend, at the time the world was created, there was a diamond tower (labeled A)
with 64 golden disks.
 The disks were of decreasing size and were stacked on the tower in decreasing order of size
bottom to top.
 Besides these tower there were two other diamond towers(labeled B & C)
 Since the time of creation, Brehman priests have been attempting to move the disks from tower A
to tower B using tower C, for intermediate storage.
 As the disks are very heavy, they can be moved only one at a time.
 In addition, at no time can a disk be on top of a smaller disk.
 According to legend, the world will come to an end when the priest have
completed this task.
 A very elegant solution results from the use of recursion.
 Assume that the number of disks is ‘n’.
 To get the largest disk to the bottom of tower B, we move the remaining ‘n-1’
disks to tower C and then move the largest to tower B.
 Now we are left with the tasks of moving the disks from tower C to B.
 To do this, we have tower A and B available.
 The fact, that towers B has a disk on it can be ignored as the disks larger than the
disks being moved from tower C and so any disk scan be placed on top of it.

Algorithm:

1. Algorithm TowersofHanoi(n,x,y,z)
2. //Move the top ‘n’ disks from tower x to tower y.
3. {

.
.
B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND
TECHNOLOGY
DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING
.

4.if(n>=1) then
5. {
6. TowersofHanoi(n-1,x,z,y);
7. Write(“move top disk from tower “ X ,”to top of tower “ ,Y);
8. Towersofhanoi(n-1,z,y,x);
9. }
10. }

2. Permutation Generator:

 Given a set of n>=1elements, the problem is to print all possible permutations of this set.
 For example, if the set is {a,b,c} ,then the set of permutation is,

{ (a,b,c),(a,c,b),(b,a,c),(b,c,a),(c,a,b),(c,b,a)}
 It is easy to see that given ‘n’ elements there are n! different permutations.
 A simple algorithm can be obtained by looking at the case of 4 statement(a,b,c,d)
 The Answer can be constructed by writing

1. a followed by all the permutations of (b,c,d)

2. b followed by all the permutations of(a,c,d)
3. c followed by all the permutations of (a,b,d)
4. d followed by all the permutations of (a,b,c)

Algorithm:

Algorithm perm(a,k,n)
{
if(k=n) then write (a[1:n]); // output permutation
else //a[k:n] ahs more than one permutation
// Generate this recursively.
for I:=k to n do
{
t:=a[k];
a[k]:=a[I];
a[I]:=t;
perm(a,k+1,n);
//all permutation of a[k+1:n]
t:=a[k];
a[k]:=a[I];
a[I]:=t;
}
}

Performance Analysis:

1. Space Complexity:

B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND

TECHNOLOGY
DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING
The space complexity of an algorithm is the amount of money it needs to run to
compilation.

2. Time Complexity:
The time complexity of an algorithm is the amount of computer time it needs to run to
compilation.

Space Complexity:

Space Complexity Example:

Algorithm abc(a,b,c)
{
return a+b++*c+(a+b-c)/(a+b) +4.0;
}

à The Space needed by each of these algorithms is seen to be the sum of the following component.

1.A fixed part that is independent of the characteristics (eg:number,size)of the inputs and outputs.
The part typically includes the instruction space (ie. Space for the code), space for simple
variable and fixed-size component variables (also called aggregate) space for constants, and so on.

2. A variable part that consists of the space needed by component variables whose size is dependent
on the particular problem instance being solved, the space needed by referenced variables (to the
extent that is depends on instance characteristics), and the recursion stack space.

 The space requirement s(p) of any algorithm p may therefore be written as,
S(P) = c+ Sp(Instance characteristics)
Where ‘c’ is a constant.

Example 2:

Algorithm sum(a,n)
{
s=0.0;
for I=1 to n do
s= s+a[I];
return s;
}

 The problem instances for this algorithm are characterized by n,the number of
elements to be summed. The space needed d by ‘n’ is one word, since it is of type
integer.
 The space needed by ‘a’a is the space needed by variables of tyepe array of floating
point numbers.
 This is atleast ‘n’ words, since ‘a’ must be large enough to hold the ‘n’ elements to be
summed.
 So,we obtain Ssum(n)>=(n+s)
[ n for a[],one each for n,I a& s]

B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND

TECHNOLOGY
DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING

Time Complexity:

The time T(p) taken by a program P is the sum of the compile time and the run
time(execution time)

àThe compile time does not depend on the instance characteristics. Also we may assume that a
compiled program will be run several times without recompilation .This rum time is denoted
by tp(instance characteristics).

à The number of steps any problem statemn t is assigned depends on the kind of statement.

For example, comments à 0 steps.

Assignment statements à 1 steps.
[Which does not involve any calls to other algorithms]

Interactive statement such as for, while & repeat-untilà Control part of the statement.

1. We introduce a variable, count into the program statement to increment count with initial value
0.Statement to increment count by the appropriate amount are introduced into the program.
This is done so that each time a statement in the original program is executes count is
incremented by the step count of that statement.

Algorithm:

Algorithm sum(a,n)
{
s= 0.0;
count = count+1;
for I=1 to n do
{
count =count+1;
s=s+a[I];
count=count+1;
}
count=count+1;
count=count+1;
return s;
}
à If the count is zero to start with, then it will be 2n+3 on termination. So each invocation of sum
execute a total of 2n+3 steps.
2. The second method to determine the step count of an algorithm is to build a table in which we
list the total number of steps contributes by each statement.
àFirst determine the number of steps per execution (s/e) of the statement and the total number of
times (ie., frequency) each statement is executed.
àBy combining these two quantities, the total contribution of all statements, the step count for the
entire algorithm is obtained.

B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND

TECHNOLOGY
DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING

Statement S/e Frequency Total

1. Algorithm Sum(a,n) 0 - 0
2.{ 0 - 0
3. S=0.0; 1 1 1
4. for I=1 to n do 1 n+1 n+1
5. s=s+a[I]; 1 n n
6. return s; 1 1 1
7. } 0 - 0

2n+3
Total

AVERAGE –CASE ANALYSIS

 Most of the time, average-case analysis are performed under the more or less realistic
assumption that all instances of any given size are equally likely.
 For sorting problems, it is simple to assume also that all the elements to be sorted are distinct.
 Suppose we have ‘n’ distinct elements to sort by insertion and all n! permutation of these
elements are equally likely.
 To determine the time taken on a average by the algorithm ,we could add the times required to
sort each of the possible permutations ,and then divide by n! the answer thus obtained.
 An alternative approach, easier in this case is to analyze directly the time required by the
algorithm, reasoning probabilistically as we proceed.
 For any I,2¿ I¿ n, consider the sub array, T[1….i].
 The partial rank of T[I] is defined as the position it would occupy if the sub array were sorted.
 For Example, the partial rank of T[4] in [3,6,2,5,1,7,4] in 3 because T[1….4] once sorted is
[2,3,5,6].
 Clearly the partial rank of T[I] does not depend on the order of the element in
 Sub array T[1…I-1].

Analysis

Best case:
This analysis constrains on the input, other than size. Resulting in the fasters possible run time

Worst case:
This analysis constrains on the input, other than size. Resulting in the fasters possible run
time

Average case:
This type of analysis results in average running time over every type of input.

Complexity:
Complexity refers to the rate at which the storage time grows as a function of the problem
size

B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND

TECHNOLOGY
DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING
Asymptotic analysis:
Expressing the complexity in term of its relationship to know function. This type
analysis is called asymptotic analysis.

Asymptotic notation:

Big ‘oh’: the function f(n)=O(g(n)) iff there exist positive constants c and no such that f(n)≤c*g(n) for all
n, n ≥ no.

Omega: the function f(n)=Ω(g(n)) iff there exist positive constants c and no such that f(n) ≥ c*g(n) for all
n, n ≥ no.

Theta: the function f(n)=ө(g(n)) iff there exist positive constants c1,c2 and no such that c1 g(n) ≤ f(n) ≤
c2 g(n) for all n, n ≥ no.

Recursion:
Recursion may have the following definitions:
-The nested repetition of identical algorithm is recursion.
-It is a technique of defining an object/process by itself.
-Recursion is a process by which a function calls itself repeatedly until some specified condition has been
satisfied.

When to use recursion:

Recursion can be used for repetitive computations in which each action is stated in terms of
previous result. There are two conditions that must be satisfied by any recursive procedure.
1. Each time a function calls itself it should get nearer to the solution.
2. There must be a decision criterion for stopping the process.
In making the decision about whether to write an algorithm in recursive or non-recursive form, it is
always advisable to consider a tree structure for the problem. If the structure is simple then use non-
recursive form. If the tree appears quite bushy, with little duplication of tasks, then recursion is suitable.

The recursion algorithm for finding the factorial of a number is given below,

Algorithm : factorial-recursion
Input : n, the number whose factorial is to be found.
Output : f, the factorial of n
Method : if(n=0)
f=1
else
f=factorial(n-1) * n
if end
algorithm ends.

The general procedure for any recursive algorithm is as follows,

1. Save the parameters, local variables and return addresses.
2. If the termination criterion is reached perform final computation and goto step 3 otherwise
perform final computations and goto step 1

B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND

TECHNOLOGY
DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING

3. Restore the most recently saved parameters, local variable and return address and goto the latest
return address.

Iteration v/s Recursion:

Demerits of recursive algorithms:

1. Many programming languages do not support recursion; hence, recursive mathematical function
is implemented using iterative methods.
2. Even though mathematical functions can be easily implemented using recursion it is always at the
cost of execution time and memory space. For example, the recursion tree for generating 6
numbers in a Fibonacci series generation is given in fig 2.5. A Fibonacci series is of the form
0,1,1,2,3,5,8,13,…etc, where the third number is the sum of preceding two numbers and so on. It
can be noticed from the fig 2.5 that, f(n-2) is computed twice, f(n-3) is computed thrice, f(n-4) is
computed 5 times.
3. A recursive procedure can be called from within or outside itself and to ensure its proper
functioning it has to save in some order the return addresses so that, a return to the proper location
will result when the return to a calling statement is made.
4. The recursive programs needs considerably more storage and will take more time.

Demerits of iterative methods :

 Mathematical functions such as factorial and Fibonacci series generation can be easily
implemented using recursion than iteration.
 In iterative techniques looping of statement is very much necessary.
Recursion is a top down approach to problem solving. It divides the problem into pieces or selects out
one key step, postponing the rest.
Iteration is more of a bottom up approach. It begins with what is known and from this constructs the
solution step by step. The iterative function obviously uses time that is O(n) where as recursive function
has an exponential time complexity.
It is always true that recursion can be replaced by iteration and stacks. It is also true that stack can be
replaced by a recursive program with no stack.

B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND

TECHNOLOGY
DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING

SOLVING RECURRENCES :-( Happen again (or) repeatedly)

 The indispensable last step when analyzing an algorithm is often to solve a recurrence equation.
 With a little experience and intention, most recurrence can be solved by intelligent guesswork.
 However, there exists a powerful technique that can be used to solve certain classes of recurrence
almost automatically.
 This is a main topic of this section the technique of the characteristic equation.

1. Intelligent guess work:

This approach generally proceeds in 4 stages.

1. Calculate the first few values of the recurrence

2. Look for regularity.
3. Guess a suitable general form.
4. And finally prove by mathematical induction(perhaps constructive induction).

Then this form is correct.

Consider the following recurrence,

0 if n=0
T(n) = 3T(n ÷ 2)+n otherwise

 First step is to replace n ÷ 2 by n/2

 It is tempting to restrict ‘n’ to being ever since in that case n÷2 = n/2, but recursively dividing an
even no. by 2, may produce an odd no. larger than 1.
 Therefore, it is a better idea to restrict ‘n’ to being an exact power of 2.
 First, we tabulate the value of the recurrence on the first few powers of 2.

n 1 2 4 8 16 32
T(n) 1 5 19 65 211 665

* For instance, T(16) = 3 * T(8) +16

B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND
TECHNOLOGY
DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING
= 3 * 65 +16
= 211.

* Instead of writing T(2) = 5, it is more

useful to write T(2) = 3 * 1 +2.

Then,
T(A) = 3 * T(2) +4
= 3 * (3 * 1 +2) +4
= (32 * 1) + (3 * 2) +4

* We continue in this way, writing ‘n’ as an explicit power of 2.

n T(n)

1 1
2 3 * 1 +2
22 32 * 1 + 3 * 2 + 22
23 33 * 1 + 32 * 2 + 3 * 22 + 23
24 34 * 1 + 33 * 2 + 32 * 22 + 3 * 23 + 24
25 35 * 1 + 34 * 2 + 33 * 22 + 32 * 23 + 3 * 24 + 25

 The pattern is now obvious.

T(2k ) = 3k20 + 3k-121 + 3k-222+…+312k-1 + 302k.

= ∑ 3k-i 2i

= 3k ∑ (2/3)i

= 3k * [(1 – (2/3)k + 1) / (1 – (2/3)]

= 3k+1 – 2k+1

Proposition: (Geometric Series)

Let Sn be the sum of the first n terms of the geometric series a, ar, ar2….Then
Sn = a(1-rn)/(1-r), except in the special case when r = 1; when Sn = an.

= 3k * [ (1 – (2/3) k+1) / (1 – (2/3))]

= 3k * [((3 k+1 – 2 k+1)/ 3 k+1) / ((3 – 2) / 3)]

3 k+1 – 2k+1 3
k
= 3 * ----------------- * ----
3 k+1 1
k+1 k+
3 –2 1
= 3k * -----------------
3k+1-1
B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND
TECHNOLOGY
DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING

= 3k+1 – 2k+1

* It is easy to check this formula against our earlier tabulation.

EG : 2
0 n=0
tn = 5 n=1
3tn-1 + 4tn-2, otherwise

tn = 3tn-1 – 4tn-2 = 0 à General function

Characteristics Polynomial, x2 – 3x – 4 = 0
(x – 4)(x + 1) = 0

Roots r1 = 4, r2 = -1

General Solution, fn = C1r1n + C2 r2n à (A)

n=0 è C1 + C2 = 0 à (1)
n=1 è C1r1 + C2r2 = 5 à (2)

Eqn 1 è C1 = -C2

sub C1 value in Eqn (2)

-C2r1 + C2r2 = 5
C2(r2 – r1) = 5
5
C2 = -------
r2 – r1
5
= ------
-1 + 4

= 5 / (-5) = -1

C2 = -1 , C1 = 1

Sub C1, C2, r1 & r2 value in equation à (A)

fn = 1. 4n + (-1) . (-1)n
fn = 4 n + 1 n

DIVIDE AND CONQUER

B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND

TECHNOLOGY
 DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING
GENERAL METHOD:

 Given a function to compute on ‘n’ inputs the divide-and-conquer strategy suggests splitting the inputs into ‘k’ distinct subsets,
1<k<=n, yielding ‘k’ sub problems.

 These sub problems must be solved, and then a method must be found to combine sub solutions into a solution of the whole.

 If the sub problems are still relatively large, then the divide-and-conquer strategy can possibly be reapplied.

 Often the sub problems resulting from a divide-and-conquer design are of the same type as the original problem.

 For those cases the re application of the divide-and-conquer principle is naturally expressed by a recursive algorithm.

 D And C(Algorithm) is initially invoked as D and C(P), where ‘p’ is the problem to be solved.

 Small(P) is a Boolean-valued function that determines whether the i/p size is small enough that the answer can be computed
without splitting.

 If this so, the function ‘S’ is invoked.

 Otherwise, the problem P is divided into smaller sub problems.

 These sub problems P1, P2 …Pk are solved by recursive application of D And C.

 Combine is a function that determines the solution to p using the solutions to the ‘k’ sub problems.

 If the size of ‘p’ is n and the sizes of the ‘k’ sub problems are n1, n2 ….nk, respectively, then the computing time of D And C
is described by the recurrence relation.

T(n)= { g(n) n small

T(n1)+T(n2)+……………+T(nk)+f(n); otherwise.

Where T(n) à is the time for D And C on any I/p of size ‘n’.
g(n) à is the time of compute the answer directly for small I/ps.
f(n) à is the time for dividing P & combining the solution to
sub problems.

1. Algorithm D And C(P)

2. {
3. if small(P) then return S(P);
4. else
5. {
6. divide P into smaller instances
P1, P2… Pk, k>=1;
7. Apply D And C to each of these sub problems;
8. return combine (D And C(P1), D And C(P2),…….,D And C(Pk));
9. }
10. }

 The complexity of many divide-and-conquer algorithms is given by recurrences

of the form
T(n) = { T(1) n=1
AT(n/b)+f(n) n>1
à Where a & b are known constants.
à We assume that T(1) is known & ‘n’ is a power of b(i.e., n=b^k)
 One of the methods for solving any such recurrence relation is called the substitution method.

B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND

TECHNOLOGY
 DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING
 This method repeatedly makes substitution for each occurrence of the function. T is the Right-hand side until all such
occurrences disappear.

Example:
1) Consider the case in which a=2 and b=2. Let T(1)=2 & f(n)=n.
We have,
T(n) = 2T(n/2)+n
= 2[2T(n/2/2)+n/2]+n
= [4T(n/4)+n]+n
= 4T(n/4)+2n
= 4[2T(n/4/2)+n/4]+2n
= 4[2T(n/8)+n/4]+2n
= 8T(n/8)+n+2n
= 8T(n/8)+3n
*
*
*

 In general, we see that T(n)=2^iT(n/2^i )+in., for any log n >=I>=1.

à T(n) =2^log n T(n/2^log n) + n log n

àCorresponding to the choice of i=log n

 Thus, T(n) = 2^log n T(n/2^log n) + n log n

= n. T(n/n) + n log n
= n. T(1) + n log n [since, log 1=0, 2^0=1]
= 2n + n log n

BINARY SEARCH

1. Algorithm Bin search(a,n,x)

2. // Given an array a[1:n] of elements in non-decreasing
3. //order, n>=0,determine whether ‘x’ is present and
4. // if so, return ‘j’ such that x=a[j]; else return 0.
5. {
6. low:=1; high:=n;
7. while (low<=high) do
8. {
9. mid:=[(low+high)/2];
10. if (x<a[mid]) then high;
11. else if(x>a[mid]) then
low=mid+1;
12. else return mid;
13. }
14. return 0;
15. }

 Algorithm, describes this binary search method, where Binsrch has 4I/ps a[], I , l & x.
 It is initially invoked as Binsrch (a,1,n,x)
 A non-recursive version of Binsrch is given below.
 This Binsearch has 3 i/ps a,n, & x.
 The while loop continues processing as long as there are more elements left to check.
 At the conclusion of the procedure 0 is returned if x is not present, or ‘j’ is returned, such that a[j]=x.

B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND

TECHNOLOGY
 DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING
 We observe that low & high are integer Variables such that each time through the loop either x is found or low is increased by
at least one or high is decreased at least one.

 Thus we have 2 sequences of integers approaching each other and eventually low becomes > than high & causes termination
in a finite no. of steps if ‘x’ is not present.

Example:
1) Let us select the 14 entries.
-15,-6,0,7,9,23,54,82,101,112,125,131,142,151.
à Place them in a[1:14], and simulate the steps Binsearch goes through as it searches for different values of ‘x’.
à Only the variables, low, high & mid need to be traced as we simulate the algorithm.
à We try the following values for x: 151, -14 and 9.
for 2 successful searches &
1 unsuccessful search.

 Table. Shows the traces of Bin search on these 3 steps.

X=151 low high mid

1 14 7
8 14 11
12 14 13
14 14 14
Found

x=-14 low high mid

1 14 7
1 6 3
1 2 1
2 2 2
2 1 Not found

x=9 low high mid

1 14 7
1 6 3
4 6 5
Found

Theorem: Algorithm Binsearch(a,n,x) works correctly.

Proof:
We assume that all statements work as expected and that comparisons such as x>a[mid] are appropriately carried out.

 Initially low =1, high= n,n>=0, and a[1]<=a[2]<=……..<=a[n].

 If n=0, the while loop is not entered and is returned.

 Otherwise we observe that each time thro’ the loop the possible elements to be checked of or equality with x and a[low],
a[low+1],……..,a[mid],……a[high].
 If x=a[mid], then the algorithm terminates successfully.
 Otherwise, the range is narrowed by either increasing low to (mid+1) or decreasing high to (mid-1).
 Clearly, this narrowing of the range does not affect the outcome of the search.
 If low becomes > than high, then ‘x’ is not present & hence the loop is exited.

Maximum and Minimum:

B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND

TECHNOLOGY
 DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING
 Let us consider another simple problem that can be solved by the divide-and-conquer technique.

 The problem is to find the maximum and minimum items in a set of ‘n’ elements.

 In analyzing the time complexity of this algorithm, we once again concentrate on the no. of element comparisons.

 More importantly, when the elements in a[1:n] are polynomials, vectors, very large numbers, or strings of character, the cost of
an element comparison is much higher than the cost of the other operations.

 Hence, the time is determined mainly by the total cost of the element comparison.

1. Algorithm straight MaxMin(a,n,max,min)

2. // set max to the maximum & min to the minimum of a[1:n]
3. {
4. max:=min:=a[1];
5. for I:=2 to n do
6. {
7. if(a[I]>max) then max:=a[I];
8. if(a[I]<min) then min:=a[I];
9. }
10. }

Algorithm: Straight forward Maximum & Minimum

 Straight MaxMin requires 2(n-1) element comparison in the best, average & worst cases.

 An immediate improvement is possible by realizing that the comparison a[I]<min is necessary only when a[I]>max is false.

 Hence we can replace the contents of the for loop by,

If(a[I]>max) then max:=a[I];
Else if (a[I]<min) then min:=a[I];

 Now the best case occurs when the elements are in increasing order.
à The no. of element comparison is (n-1).

 The worst case occurs when the elements are in decreasing order.
à The no. of elements comparison is 2(n-1)

 The average no. of element comparison is < than 2(n-1)

 On the average a[I] is > than max half the time, and so, the avg. no. of comparison is 3n/2-1.

 A divide- and conquer algorithm for this problem would proceed as follows:

à Let P=(n, a[I] ,……,a[j]) denote an arbitrary instance of the problem.

à Here ‘n’ is the no. of elements in the list (a[I],….,a[j]) and we are interested in finding the maximum and minimum of the
list.

 If the list has more than 2 elements, P has to be divided into smaller instances.

 For example , we might divide ‘P’ into the 2 instances, P1=([n/2],a[1],……..a[n/2]) & P2= (n-[n/2],a[[n/2]+1],…..,a[n])

 After having divided ‘P’ into 2 smaller sub problems, we can solve them by recursively invoking the same divide-and-conquer
algorithm.

Algorithm: Recursively Finding the Maximum & Minimum

B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND

TECHNOLOGY
 DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING
1. Algorithm MaxMin (I,j,max,min)
2. //a[1:n] is a global array, parameters I & j
3. //are integers, 1<=I<=j<=n.The effect is to
4. //set max & min to the largest & smallest value
5. //in a[I:j], respectively.
6. {
7. if(I=j) then max:= min:= a[I];
8. else if (I=j-1) then // Another case of small(p)
9. {
10. if (a[I]<a[j]) then
11. {
12. max:=a[j];
13. min:=a[I];
14. }
15. else
16. {
17. max:=a[I];
18. min:=a[j];
19. }
20. }
21. else
22. {
23. // if P is not small, divide P into subproblems.
24. // find where to split the set mid:=[(I+j)/2];
25. //solve the subproblems
26. MaxMin(I,mid,max.min);
27. MaxMin(mid+1,j,max1,min1);
28. //combine the solution
29. if (max<max1) then max=max1;
30. if(min>min1) then min = min1;
31. }
32. }

 The procedure is initially invoked by the statement,

MaxMin(1,n,x,y)
 Suppose we simulate MaxMin on the following 9 elements

A: [1] [2] [3] [4] [5] [6] [7] [8] [9]

22 13 -5 -8 15 60 17 31 47
 A good way of keeping track of recursive calls is to build a tree by adding a node each time a new call is made.
 For this Algorithm, each node has 4 items of information: I, j, max & imin.
 Examining fig: we see that the root node contains 1 & 9 as the values of I &j corresponding to the initial call to MaxMin.
 This execution produces 2 new calls to MaxMin, where I & j have the values 1, 5 & 6, 9 respectively & thus split the set into 2
subsets of approximately the same size.
 From the tree, we can immediately see the maximum depth of recursion is 4. (including the 1 st call)
 The include no.s in the upper left corner of each node represent the order in which max & min are assigned values.

No. of element Comparison:

 If T(n) represents this no., then the resulting recurrence relations is

T(n)={ T([n/2]+T[n/2]+2 n>2

n=2
0 n=1

à When ‘n’ is a power of 2, n=2^k for some +ve integer ‘k’, then
T(n) = 2T(n/2) +2
= 2(2T(n/4)+2)+2
= 4T(n/4)+4+2
B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND
TECHNOLOGY
 DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING
*
*
= 2^k-1T(2)+
= 2^k-1+2^k-2
= 2^k/2+2^k-2
= n/2+n-2
= (n+2n)/2)-2

T(N)=(3N/2)-2

*Note that (3n/3)-3 is the best-average, and worst-case no. of comparisons when ‘n’ is a power of 2.

MERGE SORT
 As another example divide-and-conquer, we investigate a sorting algorithm that has the nice property that is the worst case its
complexity is O(n log n)
 This algorithm is called merge sort
 We assume throughout that the elements are to be sorted in non-decreasing order.
 Given a sequence of ‘n’ elements a[1],…,a[n] the general idea is to imagine then split into 2 sets a[1],…..,a[n/2] and
a[[n/2]+1],….a[n].
 Each set is individually sorted, and the resulting sorted sequences are merged to produce a single sorted sequence of ‘n’
elements.
 Thus, we have another ideal example of the divide-and-conquer strategy in which the splitting is into 2 equal-sized sets & the
combining operation is the merging of 2 sorted sets into one.

Algorithm For Merge Sort:

1. Algorithm MergeSort(low,high)
2. //a[low:high] is a global array to be sorted
3. //Small(P) is true if there is only one element
4. //to sort. In this case the list is already sorted.
5. {
6. if (low<high) then //if there are more than one element
7. {
8. //Divide P into subproblems
9. //find where to split the set
10. mid = [(low+high)/2];
11. //solve the subproblems.
12. mergesort (low,mid);
13. mergesort(mid+1,high);
14. //combine the solutions .
15. merge(low,mid,high);
16. }
17. }

Algorithm: Merging 2 sorted subarrays using auxiliary storage.

1. Algorithm merge(low,mid,high)
2. //a[low:high] is a global array containing
3. //two sorted subsets in a[low:mid]
4. //and in a[mid+1:high].The goal is to merge these 2 sets into
5. //a single set residing in a[low:high].b[] is an auxiliary global array.
6. {
7. h=low; I=low; j=mid+1;
8. while ((h<=mid) and (j<=high)) do
9. {
10. if (a[h]<=a[j]) then
B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND
TECHNOLOGY
 DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING
11. {
12. b[I]=a[h];
13. h = h+1;
14. }
15. else
16. {
17. b[I]= a[j];
18. j=j+1;
19. }
20. I=I+1;
21. }
22. if (h>mid) then
23. for k=j to high do
24. {
25. b[I]=a[k];
26. I=I+1;
27. }
28. else
29. for k=h to mid do
30. {
31. b[I]=a[k];
32. I=I+1;
33. }
34. for k=low to high do a[k] = b[k];
35. }

 Consider the array of 10 elements a[1:10] =(310, 285, 179, 652, 351, 423, 861, 254, 450, 520)

 Algorithm Mergesort begins by splitting a[] into 2 sub arrays each of size five (a[1:5] and a[6:10]).
 The elements in a[1:5] are then split into 2 sub arrays of size 3 (a[1:3] ) and 2(a[4:5])
 Then the items in a a[1:3] are split into sub arrays of size 2 a[1:2] & one(a[3:3])
 The 2 values in a[1:2} are split to find time into one-element sub arrays, and now the merging begins.

(310| 285| 179| 652, 351| 423, 861, 254, 450, 520)

à Where vertical bars indicate the boundaries of sub arrays.

àElements a[I] and a[2] are merged to yield,

(285, 310|179|652, 351| 423, 861, 254, 450, 520)

à Then a[3] is merged with a[1:2] and

(179, 285, 310| 652, 351| 423, 861, 254, 450, 520)

à Next, elements a[4] & a[5] are merged.

(179, 285, 310| 351, 652 | 423, 861, 254, 450, 520)

à And then a[1:3] & a[4:5]

(179, 285, 310, 351, 652| 423, 861, 254, 450, 520)

à Repeated recursive calls are invoked producing the following sub arrays.
(179, 285, 310, 351, 652| 423| 861| 254| 450, 520)

à Elements a[6] &a[7] are merged.

àThen a[8] is merged with a[6:7]

(179, 285, 310, 351, 652| 254,423, 861| 450, 520)

à Next a[9] &a[10] are merged, and then a[6:8] & a[9:10]
B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND
TECHNOLOGY
 DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING
(179, 285, 310, 351, 652| 254, 423, 450, 520, 861 )

à At this point there are 2 sorted sub arrays & the final merge produces the fully sorted
result.
(179, 254, 285, 310, 351, 423, 450, 520, 652, 861)

 IF THE TIME FOR THE MERGING OPERATIONS IS PROPORTIONAL TO ‘N’, THEN THE COMPUTING
TIME FOR MERGE SORT IS DESCRIBED BY THE RECURRENCE RELATION.

T(N) = { A N=1,’A’ A CONSTANT

2T(N/2)+CN N>1,’C’ A CONSTANT.

à When ‘n’ is a power of 2, n= 2^k, we can solve this equation by successive substitution.

T(n) =2(2T(n/4) +cn/2) +cn

= 4T(n/4)+2cn
= 4(2T(n/8)+cn/4)+2cn
*
*
= 2^k T(1)+kCn.
= an + cn log n.

à It is easy to see that if s^k<n<=2^k+1, then T(n)<=T(2^k+1). Therefore,

T(n)=O(n log n)

QUICK SORT

 The divide-and-conquer approach can be used to arrive at an efficient sorting method different from merge sort.

 In merge sort, the file a[1:n] was divided at its midpoint into sub arrays which were independently sorted & later merged.

 In Quick sort, the division into 2 sub arrays is made so that the sorted sub arrays do not need to be merged later.

 This is accomplished by rearranging the elements in a[1:n] such that a[I]<=a[j] for all I between 1 & n and all j between (m+1)
& n for some m, 1<=m<=n.

 Thus the elements in a[1:m] & a[m+1:n] can be independently sorted.

 No merge is needed. This rearranging is referred to as partitioning.

 Function partition of Algorithm accomplishes an in-place partitioning of the elements of a[m:p-1]

 It is assumed that a[p]>=a[m] and that a[m] is the partitioning element. If m=1 & p-1=n, then a[n+1] must be defined and must
be greater than or equal to all elements in a[1:n]

 The assumption that a[m] is the partition element is merely for convenience, other choices for the partitioning element than the
first item in the set are better in practice.

B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND

TECHNOLOGY
 DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING
 The function interchange (a,I,j) exchanges a[I] with a[j].

Algorithm: Partition the array a[m:p-1] about a[m]

1. Algorithm Partition(a,m,p)
2. //within a[m],a[m+1],…..,a[p-1] the elements
3. // are rearranged in such a manner that if
4. //initially t=a[m],then after completion
5. //a[q]=t for some q between m and
6. //p-1,a[k]<=t for m<=k<q, and
7. //a[k]>=t for q<k<p. q is returned
8. //Set a[p]=infinite.
9. {
10. v=a[m];I=m;j=p;
11. repeat
12. {
13. repeat
14. I=I+1;
15. until(a[I]>=v);
16. repeat
17. j=j-1;
18. until(a[j]<=v);
19. if (I<j) then interchange(a,i.j);
20. }until(I>=j);
21. a[m]=a[j]; a[j]=v;
22. retun j;
23. }

1. Algorithm Interchange(a,I,j)
2. //Exchange a[I] with a[j]
3. {
4. p=a[I];
5. a[I]=a[j];
6. a[j]=p;
7. }

Algorithm: Sorting by Partitioning

1. Algorithm Quicksort(p,q)
2. //Sort the elements a[p],….a[q] which resides
3. //is the global array a[1:n] into ascending
4. //order; a[n+1] is considered to be defined
5. // and must be >= all the elements in a[1:n]
6. {
7. if(p<q) then // If there are more than one element
8. {
9. // divide p into 2 subproblems
10. j=partition(a,p,q+1);
11. //’j’ is the position of the partitioning element.
12. //solve the subproblems.
13. quicksort(p,j-1);
14. quicksort(j+1,q);
15. //There is no need for combining solution.
16. }
17. }

B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND

TECHNOLOGY
DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING
Record Program: Quick Sort
#include <stdio.h>
#include <conio.h>
int a[20];
main()
{
int n,I;
clrscr();
printf(“QUICK SORT”);
printf(“\n Enter the no. of elements “);
scanf(“%d”,&n);
printf(“\nEnter the array elements”);
for(I=0;I<n;I++)
scanf(“%d”,&a[I]);
quicksort(0,n-1);
printf(“\nThe array elements are”);
for(I=0;I<n;I++)
printf(“\n%d”,a[I]);
getch();
}
quicksort(int p, int q)
{
int j;
if(p,q)
{
j=partition(p,q+1);
quicksort(p,j-1);
quicksort(j+1,q);
}
}

Partition(int m, int p)
{
int v,I,j;
v=a[m];
i=m;
j=p;
do
{
do
i=i+1;
while(a[i]<v);
if (i<j)
interchange(I,j);
} while (I<j);
a[m]=a[j];
a[j]=v;
return j;
}

Interchange(int I, int j)
{
int p;
p= a[I];
a[I]=a[j];
a[j]=p;
}
B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND
TECHNOLOGY
 DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING

Output:
Enter the no. of elements 5
Enter the array elements
3
8
1
5
2
The sorted elements are,
1
2
3
5
8

STRASSON’S MATRIX MULTIPLICAION

 Let A and B be the 2 n*n Matrix. The product matrix C=AB is calculated by using the formula,

C (i ,j )= A(i,k) B(k,j) for all ‘i’ and and j between 1 and n.

 The time complexity for the matrix Multiplication is O(n^3).

 Divide and conquer method suggest another way to compute the product of n*n matrix.

 We assume that N is a power of 2 .In the case N is not a power of 2 ,then enough rows and columns of zero can be added to
both A and B .SO that the resulting dimension are the powers of two.

 If n=2 then the following formula as a computed using a matrix multiplication operation for the elements of A & B.

 If n>2,Then the elements are partitioned into sub matrix n/2*n/2..since ‘n’ is a power of 2 these product can be recursively
computed using the same formula .This Algorithm will continue applying itself to smaller sub matrix until ‘N” become
suitable small(n=2) so that the product is computed directly .
 The formula are

A11 A12 B11 B12 C11 C12

* =
A21 A21 B21 B22 C21 C22

C11 = A11 B11 + A12 B21

C12 = A11 B12 + A12 B22
C21 = A21 B11 + A22 B21
C22 = A21 B12 + A22 B22

For EX:
2222 1 1 11
4*4= 2222 1 1 11
2222 * 1 1 11
2222 1 11 1

B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND

TECHNOLOGY
 DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING

The Divide and conquer method

2 2 2 2 1 1 1 1 4 4 4 4
2 2 2 2 * 1 1 1 1 = 4 4 4 4
2 2 2 2 1 1 1 1 4 4 4 4
2 2 2 2 1 1 1 1 4 4 4 4

 To compute AB using the equation we need to perform 8 multiplication of n/2*n/2 matrix and from 4 addition of n/2*n/2
matrix.
 Ci,j are computed using the formula in equation à4
 As can be sum P, Q, R, S, T, U, and V can be computed using 7 Matrix multiplication and 10 addition or subtraction.
 The Cij are required addition 8 addition or subtraction.

T(n)= b n<=2 a &b are

7T(n/2)+an^2 n>2 constant

Finally we get T(n) =O( n ^log27)

Example

4 4 4 4
*
4 4 4 4

P=(4*4)+(4+4)=64
Q=(4+4)4=32
R=4(4-4)=0
S=4(4-4)=0
T=(4+4)4=32
U=(4-4)(4+4)=0
V=(4-4)(4+4)=0
C11=(64+0-32+0)=32
C12=0+32=32
C21=32+0=32
C22=64+0-32+0=32

So the answer c(i,j) is 32 32

32 32

since n/2n/2 &matrix can be can be added in Cn for some constant C, The overall computing time T(n) of the resulting divide
and conquer algorithm is given by the sequence.

T(n)= b n<=2 a &b are

8T(n/2)+cn^2 n>2 constant

That is T(n)=O(n^3)

* Matrix multiplication are more expensive then the matrix addition O(n^3).We can attempt to reformulate the equation for Cij
so as to have fewer multiplication and possibly more addition .

 Stressen has discovered a way to compute the Cij of equation (2) using only 7 multiplication and 18 addition or subtraction.
 Strassen’s formula are
B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND
TECHNOLOGY
DESIGN AND ANALYSIS OF ALGORITHMS COMPUTER SCIENCE AND ENGINEERING

P= (A11+A12)(B11+B22)
Q= (A12+A22)B11
R= A11(B12-B22)
S= A22(B21-B11)
T= (A11+A12)B22
U= (A21-A11)(B11+B12)
V= (A12-A22)(B21+B22)

C11=P+S-T+V
C!2=R+t
C21=Q+T
C22=P+R-Q+V

B.Balakonda Reddy KKC INSTITUTE OF SCIENCE AND

TECHNOLOGY