Daa notes 2

Design and Analysis of
Algorithms
Prepared By:
Smruti Smaraki Sarangi
Asst. Professor
IMS Unison University, Dehradun
Module - 2

Randomization:
 We call an algorithm randomized, if its behavior is
determined not only by its input but also by values
produced by a random-number generator. We shall assume
that we have at our disposal a random-number generator
RANDOM.
 A call to RANDOM(a, b) returns an integer between „a‟ and
„b‟ inclusive with each such integer being equally likely.
 E.g.: RANDOM(0, 1) produces 0 with probability ½ and it
produces 1 with probability 1/2. A call to RANDOM(3, 7)
returns either 3, 4, 5, 6 or 7, each with probability 1/5. Each
integer returned by RANDOM is independent of the
integers returned on previous calls.
 The Randomize Algorithm is classified into 2 categories.
i.e.: Las Vegas Algorithm and Monte Carlo Algorithm

1) Las Vegas Algorithm:
 It always be correct. Here, expected running time is
probably faster. E.g.: Randomized Quick sort.
i) Randomized Quick sort:
 We have seen that, if we assume all permutations of the
number to be equally likely when selecting the “divide”.
T(n) = θ(nlogn). We get the average case. We have to
enforce the permutations, if we can not assume all
permutations to be equally likely.
 In Randomized version, all permutations of the number
are not necessary equally alike depends upon the
random-number-generator, RANDOM(a, b).
 Randomized version will show worst case or average
case. RANDOM(a, b) returns an integer x, a ≤ x ≤ b. It is
implemented with a pseudorandom generator.

 The Randomized Quick sort algorithm uses
RANDOM(p, r). So as to obtain a new pivot element.
Randomized Quick sort Algorithm:
Randomized_Quicksort(Q, p, r)
1. if p < r then
2. q ← Randomized_Partition(Q, p, r)
3. Randomized_Quicksort(Q, p, q)
4. Randomized_Quicksort(Q, q + 1, r)
Algorithm for Randomized_Partition:
Randomized_Partition(Q, p, r)
1. if i ← call to Random(p, r)
2. set Q[p] ↔ Q[i]
3. return Partition(Q, p, r)

Analysis of Randomized Quick sort:
 The recurrence will generate the worst case running time,
the partitioning of the list will be in θ(n) time, thus the
running time of the algorithm is:
 T(n) = max[T(q) + T(n – q) + θ(n)], where 1 ≤ q ≤ n – 1
and q = some partitioned point produced by random-
number generator.
 By substitution method, we can guess T(n) ≤ cn2,
T(n) ≤ max(cq2 + c(n – q)2) + θ(n) {1 ≤ q ≤ n – 1}.
 We can show that,
q2 + (n – q)2 = q2 + n2 – 2nq + q2 = n2 – 2nq + 2q2 = f(q)
fˈ(q) = 4q – 3n = 0 => q = n/2.
max(q2 + (n – q)2) ≤ (12 + (n – 1)2)
= n2 – 2(n – q) {1 ≤ q ≤ n – 1}.
T(n) ≤ cn2 – 2c(n – 1) + θ(n) ≤ cn2
{2c(n – 1) and θ(n) cancel out }.

 If „c‟ is large enough then, T(n) = θ(n2).
 The value that the partition returns depends upon the rank
of „x‟, we first swap for arbitrary value „x‟.
So rank(n) = i, i = 1, … n is equally probable or
P{rank(x) = i} = 1/n
P{low side i} = {2/n of i = 1 or 1/n if i = 2 … n – 1}
=> T(n) = total cost of Randomized Quick sort.
=> T(1) = θ(1)
 „q‟ is the value returned by partition, then
T(n) = 1/2[T(1) + T(n – 1) + 𝑇 𝑞 + 𝑇(𝑛 − 𝑞)]𝑛 −1
𝑞 −1 +
θ(n).
But, T(1) = θ(1) {worst case = θ(n2)}.
So, 1/n[T(1) + T(n – 1)] = 1/n[θ(1) + θ(n2)] = θ(n)
T(n) = 1/n (𝑇 𝑞 + 𝑇 𝑛 − 𝑞 )𝑛 −1
𝑞 −1 + θ(n)
= 2n 𝑇 𝑞𝑛 −1
𝑞 −1 + θ(n).

 Let us assume, T(n) ≤ anlogn + b
T(n) = 2/n (𝑎𝑞𝑙𝑜𝑔𝑞 + 𝑏)𝑛 −1
𝑞 −1 + θ(n).
= 2a/n (𝑞𝑙𝑜𝑔𝑞)𝑛 −1
𝑞 −1 + 2b/n * (n – 1) + θ(n) …. (i)
 From the above we can get the summation as:
(𝑞𝑙𝑜𝑔𝑞)𝑛 −1
𝑞 −1 = (𝑞𝑙𝑜𝑔𝑞)
⌈𝑛/2⌉ −1
𝑞 −1 + (𝑞𝑙𝑜𝑔𝑞)𝑛 −1
𝑞=⌈𝑛/2⌉
… (ii)
 Upper bound of second summation gives logn, so
equation (ii) becomes, (𝑞𝑙𝑜𝑔𝑞)𝑛 −1
𝑞 −1 ≤ 1/2n2logn – 1/8n2
 T(n) ≤ 2a/n(n2/2logn – n2/8) + 2b(n – 1)/n + θ(n)
≤ anlogn – a/4n + 2b + θ(n)
= anlogn + b(θ(n) + b – an/4)
≤ anlogn + b {a/4 big enough an/4 ≥ q(n) + b}
Thus the average case running time is θ(nlogn).

2) Monte Carlo Algorithm:
 It is mostly correct. That is, probably correct. It gives
guarantee about the running time. E.g.: Primality Testing
i) Primality Test:
 Primality Test for Prime no:
„n‟ is prime, iff n/i → n = 1 or n = i.
 Algorithm:
n ≥ 2 [the time complexity = O(n)]
if n = 2 then return true
if n is even then return false
for(i = 1 to √n/2) do
If 2i + 1divides „n‟ then return false
return true.

Randomize method for prime number:
 Polynomial Time Complexity in the length of the input
i. If the answer is not prime, then „n‟ is not prime.
ii. If the answer is prime, then probability that „n‟ is not
prime is at most p > 0.
 For „k‟ iteration, probability that „n‟ is not prime, at most
„pk‟ means each odd prime number „p‟ divides 2p – 1 – 1.
 Algorithm for Randomize Prime no:
1. calculate z = 2n – 1 mod n
2. if z = 1
then „n‟ is possibly prime
else
n = definitely not prime
Time Complexity = O(1)
E.g.: no. = 7, z = 27 – 1 mod 7 = 63 mod 7 = 1. So, „7‟ is
prime.

Format’s Little Theorem (For Prime no. test):
 If p = prime and 0 < a < p, then ap – 1 mod p = 1.
E.g.: p = 5, a = 4, ap – 1 mod p = 45 – 1 mod 5 = 44 mod 5
= 256 mod 5 = 1. So „5‟ is prime.
Randomize Algorithm for Prime no. Primality Test:
Choose a ← Random[2, n – 1]
Calculate
z = an – 1 mod n
if (z = 1)
then „n‟ is possibly prime.
else
„n‟ is definitely not prime
E.g.: n = 11, a = 8. z = 211 – 1 mod 11 = 59049 mod 11 = 1.
So 11 is prime.

Q): Prove that ‘n’ is not a prime no. but an – 1 % n = 1.
Let n = 6, a = 5.
an – 1 mod n = 1
=> an – 1 mod n = 55 mod 6 = 3/26 mod 6 ≠ 1.
So, to prove an – 1 % n = 1, we consider some theorem
given below.
Carmichael Number Theorem:
„n‟ is a Carmichael no, if an – 1 mod n = 1, for all „a‟ with
GCD(a, n) = 1.
i) Amicable Pair number:
 Let A → a no.
 B → Sum of factors of „A‟ => A = B and B = A.
ii) Happy number:
E.g.: number = 7, => 72 = 49 = 42 + 92 = 97
= 92 + 72 = 130 = 12 + 32 + 02 = 10 = 12 + 02 = 1

iii. Perfect Number:
 Factors add.
 Let no. = 6, its factors = 1, 2, 3. So 6 = 1 + 2 + 3
iv. Strong Number:
Factorial of each digit sum
E.g.: 145 = 1! + 4! + 5! = 145
v. Armstrong Number:
 Digits of cube is added
 Let no. = 153 = 13 + 53 + 33 = 153.

Dynamic Programming:
 It is the most powerful design technique for optimization
problems, was invented by Richard Bellman, a prominent
U.S. Mathematician, in the 1950s.
 The solutions for the Dynamic Programming decisions on
a few common elements.
 The word „Programming‟ does not symbolize computer
programming by planning.
 The dynamic programming is closely related to divide and
conquer technique, where the problem breaks down into a
smaller sub-problems and each sun-problem is solved
recursively.
 The dynamic programming differs from divide and
conquer, on a way that instead of solving problem
recursively.

 It solves each of the sub-problem only once and stores the
solutions to the sub-problem in a table, later on the solution
to the main problem is obtain by those sub-problem
solutions.
 The steps for achieving the Dynamic Programming are:
i. Divide the sub-problem
ii. Storage in table
iii. Combine by bottom up computation method.
1) Divide the sub-problem:
 The main problem is divided into several smaller sub-
problems. In this, the solution of the main problem is
expressed in terms of the solution for the smaller sub-
problems.

2) Storage in table:
 The solution for each sub-problem is stored in a table.
So that it can be used many times whenever required.
3) Combine by Bottom-up computation method:
 The solution to main problem is obtained by combining
the solution of smaller sub-problems.
 The development of a dynamic programming algorithm
can be broken into a sequence of 4 steps. That is:
i. Characterize the structure of an optimal solution
ii. Recursively define the value of an optimal solution
iii. Compute the value of an optimal solution in a bottom
up manner.
iv. Construct an optimal solution from computed
information.

Solutions/Elements of
Dynamic Programming:
 For the problem to be solved through dynamic
programming must satisfy the following conditions. That
is:
 Principle of Optimality
 Polynomial Break up
1. Principle of Optimality:
 The principle of optimality states that for solving the
problem optimality, its sub-problem should be solved
optimally.
 It should be noted that all the times, each sub-problem
is solved optimally, so in that case we should go for
the optimal majority.

2. Polynomial Break up:
 For solving the main problem, the problem is divided
into several smaller sub-problems and for efficient
performance of dynamic programming, the total number
of sub-problem to be solved should be at-most a
polynomial number.
 E.g.: Fibonacci Series:
n = 1, n = 2, n = 3, n = 4, ….
1 1 2 3 5 8 13 21 34.
So, Fib(4) = Fib(3) + Fib(2) = 2 + 1 = 3. So the algorithm
or code is:
int Fibo(int n){
if (n == 1 || n ==2)
return 1;
else
return Fibo(n – 1) + Fibo(n – 2) }

 In divide and conquer method, we are repeating the same
steps 2 times. So, to avoid repetition, we apply Dynamic
Programming method here.
F(5)
F(4)
5
F(3)+
F(4)
3
F(3)+ F(4)
2
F(3)+
F(2)
2
F(1)+
(1) (1)
(1) (1)

 That is:
int Fibo(int n)
{
f =1, S = 1, T
If(n == 1 || n == 2)
return 1;
else
for(i = 1to n = 2)
{
T = f + S;
f = S;
S = T;
}
return T;
}
i T f S n
int i 1 1 “
1 2 1 2
2 3 2 3
3 5 3 5
4 8 4 8

Longest Common Sub-
sequence (LCS):
 The longest common sub-sequence (LCS) problem is that
with given two sequences „X‟ and „Y‟, find a common sub-
sequences which is of maximum length or longest length.
 Theorem:
 Let „X‟ and „Y‟ are 2 sequences. That is:
X = {x1, x2, … xn}, Y = {y1, y2, … yn}
 Let „Z‟ is a sequence. That is: Z = {z1, z2, … zk} be any
longest common subsequence of „X‟ and „Y‟.
 Then,
i. if Xm = Yn, then Zk – 1 is an LCS of Xm – 1 and Yn – 1
ii. If Xm ≠ Yn, then „Z‟ must be the LCS of Xm – 1 and Y.

 The LCS of „x‟ and Yn – 1, whichever is longest and it is
represented with a rule given below. That is: x = m, y = n.
c(i, j) matrix = (m + 1) * (n + 1), where „1‟ is the base case
value for „x‟ and „y‟. So,
c(i, j) = i) 0, if i = 0 || j = 0
ii) c[i – 1, j – 1] + 1, if i, j > 0 and xi = yj
iii) max(c[i, j – 1], c[i – 1, j]), if i, j > 0 and xi ≠ yj
 E.g.: if x = (A, B, C, B, D, A, B), y = (B, D, C, A, B, A)
 The sequence <B, C, A> is a common subsequence of both
„x‟ and „y‟. The sequence <B, C, A> is not a longest
common subsequence (LCS) of „x‟ and „y‟.
 The sequence <B, C, A> has length „3‟ and a sequence <B,
C, B, A> which is also common to both x and y and has
length 4. So, the sequence <B, C, B, A> is an LCS of „x‟
and „y‟, as is the sequence <B, D, A, B>. Since there is no
common sequence of length 5 is greater.

1. Algorithm for LCS:
Algo LCS(X, Y)
1. m ← length[x]
2. n ← length[y]
3. for i ← 0 to m
4. do c[i, 0] ← 0
5. for j ← 0 to n
6. do c[0, j] ← 0
7. for i ← 1 to m
8. do for j ← 1 to n
9. do if xi = yj
10. then c[i, j] ← c[i – 1, j – 1] + 1
11. Previous[i, j] ← „ ‟
12. else if c[i – 1, j] ≥ c[i, j – 1]

13. then c[i, j] ← c[i – 1, j]
14. Previous[i, j] ← „↑‟
15. else c[i, j] ← c[i, j – 1]
16. Previous[i, j] ← „←‟
17. return „c‟ and „Previous‟.
2. Algorithm for LCS:
Output_Print_ LCS(A, Previous, i, j)
1. if i = 0 or j = 0
2. then return
3. if Previous[i, j] = „ ‟
4. then Output_Print_LCS(A, Previous, i – 1, j – 1)
5. Print ai.
6. else if Previous[i, j] = „↑‟ then
7. Output_Print_LCS(A, Previous, i – 1, j)
8. else Output_Print_LCS(A, Previous, i, j – 1)

3. Time Complexity of LCS:
 For Length of LCS: The running time of length of
LCS procedure is O(mn), since each table entry take
O(1) time to compute.
 For Print the LCS: The running time of printing LCS
procedure is O(m + n). Since at least one of „i‟ and „j‟
is decremented in each stage of recursion.
Examples:
 1. x = bacad, y = accbadcb
y = 0 a c c b a d c b
x = 0 0 0 0 0 0 0 0 0 0
b 0 ↑0 ↑0 ↑0 1 ←1 ←1 ←1 1
a 0 1 ←1 ←1 ←1 2 ←2 ←2 ←2
c 0 ↑1 2 2 ←2 ↑2 ↑2 3 ←3
a 0 1 ↑2 ↑2 ↑2 3 ←3 ↑3 ↑3
d 0 ↑1 ↑2 ↑2 ↑2 ↑3 4 ←4 ←4

 LCS = a c a d and length = 4
 2. x = babbabab, y = bbabbaaab
 LCS = <b, a, b, b, a, a, b> and length = 7
y = 0 b b a b b a a a b
x = 0 0 0 0 0 0 0 0 0 0 0
b 0 1 1 ←1 1 1 ←1 ←1 ←1 1
a 0 ↑1 ↑1 2 ←2 ←2 2 2 2 ←2
b 0 1 2 ↑2 3 3 ←3 ←3 ←3 3
b 0 1 2 ↑2 3 4 ←4 ←4 ←4 4
a 0 ↑1 ↑2 3 ↑3 ↑4 5 5 5 ←5
b 0 1 2 ↑3 4 4 ↑5 ↑5 ↑5 6
a 0 ↑1 ↑2 3 ↑4 ↑4 5 6 6 ↑6
b 0 1 2 ↑3 4 5 ↑5 ↑6 ↑6 7

 3. x = babbabab, y = bbabbaaab
 LCS = <p, r, i, d, e, n> and length = 6
y = 0 p r e s i d e n t
x = 0 0 0 0 0 0 0 0 0 0 0
p 0 1 ←1 ←1 ←1 ←1 ←1 ←1 ←1 ←1
r 0 ↑1 2 ←2 ←2 ←2 ←2 ←2 ←2 ←2
o 0 ↑1 ↑2 ↑2 ↑2 ↑2 ↑2 ↑2 ↑2 ↑2
v 0 ↑1 ↑2 ↑2 ↑2 ↑2 ↑2 ↑2 ↑2 ↑2
i 0 ↑1 ↑2 ↑2 ↑2 3 ←3 ←3 ←3 ←3
d 0 ↑1 ↑2 ↑2 ↑2 ↑3 4 ←4 ←4 ←4
e 0 ↑1 ↑2 3 ←3 ↑3 ↑4 5 ←5 ←5
n 0 ↑1 ↑2 ↑3 ↑3 ↑3 ↑4 ↑5 6 ←6
c 0 ↑1 ↑2 ↑3 ↑3 ↑3 ↑4 ↑5 ↑6 ↑6
e 0 ↑1 ↑2 3 ↑3 ↑3 ↑4 5 ↑6 ↑6

Matrix Chain
Multiplication:
 Let a given sequence or chain <A1, A2, … An> of „n‟
matrices to be multiplied and the product computed is: A1 x
A2 x … x An.
 This expression can be evaluated by using the standard
algorithm for multiplying pairs of matrices as a subroutine
once we have parenthesized it to resolve all ambiguities in
how matrices are multiplied together.
 A product of matrices is fully parenthesized, if it is either a
single matrix or the product of 2 fully parenthesized matrix
products, surrounded by parenthesis.
 Matrix multiplication is associative and so all
parenthesization yield the same product.

 E.g.: if the chain of matrices is <A1, A2, A3, A4>, the
product A1A2A3A4 can be fully parenthesized in 5 distinct
ways. That is:
i. (A1(A2(A3A4)))
ii. (A1((A2A3)A4))
iii. ((A1A2)(A3A4))
iv. ((A1(A2A3))A4)
v. (((A1A2)A3)A4)
 The product is: A1A2…An. Let we divided it into 2 parts.
That is: i) A1A2…Ak, the sequence is: <A1, A2, … Ak>
ii) Ak +1Ak + 2…An, the sequence is:
< Ak +1,Ak + 2,…An>.
Let these 2 parts are the optimal, then A1 x A2 x … x An
must be optimal.

 Let A1 x A2 x … x Ak = Pk and Ak +1 xAk + 2 x … x An = Pk +1
=> Pk x Pk +1 = A1 x A2 x … x An.
 So, the total cost for this matrix multiplication is: cost of
(A1 x A2 x … x Ak) + cost of (Ak +1 xAk + 2 x … x An) + cost
of multiplication (A1 x A2 x … x Ak) * (Ak +1 xAk + 2 x … x
An).
 For getting the optimal solution in matric chain
multiplication, we consider the formula is:
i. When i = j, then M[i, j] = 0, where i = row index and j
= column index.
ii. When i ≠ j, then M[i, j] = min[M[i, k] + M[k + 1, j] +
Pi – 1 * Pk * Pj], where i ≤ k ≤ j.

1) Algorithm for Matrix-Chain Multiplication of
order ‘P’:
Matrix_Chain_Multiplication_Order(P[], n)
/*P is sequence matrix = P0, P1, … , Pn and n is the length
of matrix*/
1. n ← length [P] – 1
2. for(i = 1 to n)
3. do M[i, i] ← 0
4. for(len = 2 to n)
5. do for(i = 1 to n – len + 1)
6. do j ← i + len – 1
7. M[i, j] ← ∞
8. for(k = i to j – 1)
9. do q = m[i, k] + m[k + 1, j] + Pi – 1PkPj
10. if q < M[i, j]

11. then M[i, j] ← q
12. S[i, j] ← k
13. end of k loop
14. end of i loop
15. end of len loop
16. return M and S.
2) Algorithm for Printing Optimal Parenthesis:
Print_Optimal_Parenthesis(S, i, j)
1. if i = j
2. then Print „Ai‟
3. else Print ˈ(ˈ
4. Print_Optimal_Parenthesis(S, i, S[i, j])
5. Print_Optimal_Parenthesis(S, S[i, j] + 1, j)
6. Print „)‟
7. end of Print_Optimal_Parenthesis.

3) Time Complexity (Running time) of
Matrix_Chain Order:
1. The nested loop structure of matrix_chain_order yields
running time of O(n3) for the algorithm.
2. The loops for matrix_chain are nested 3 „deep‟ and
each loop index(l, i & k)
3. The algorithm requires θ(n2) space to store the „M‟ and
„S‟ table.
Example:
 A1 = 2 x 4, A2 = 4 x 2, A3 = 2 x 3, A4 = 3 x 1, A5 = 1 x 4.
Here, P0 = 2, P1 = 4, P2 = 2, P3 = 3, P4 = 1 and P5 = 1.
Length = 5 = length(P) – 1 = 6 – 1
 M[1, 1] = M[2, 2] = M[3, 3] = M[4, 4] = M[5, 5] = 0

 For len = 2 to 5
a. For len = 2, i = 1 to n – len + 1 = 1 to 5 – 2 + 1 = 1 to 4
So for len = 2, i = 1, j = i + len – 1 = 1 to 2 – 1 = 2.
M[i, j] = M[1, 2] = ∞
For k = i to j – 1 = 1 to 2 – 1 = 1 to 1
q = M[i, k] + M[k + 1, j] + Pi – 1PkPj
=> q = M[1, 1] + M[2, 2] + P0P1P2
= 0 + 0 + 2 x 4 x 2 = 16
i↓ j → 1 2 3 4 5
1 0
2 0
3 0
4 0
5 0

Here q < M[i, j] => 16 < M[1, 2] => 16 < ∞.
So, M[i, j] ← q => M[1, 2] = 16 and S[1, 2] = k = 1
b. For len = 2, i = 2, j = 3 => M[i, j] = M[2, 3] = ∞
For k = i to j – 1 = 2 to 3 – 1 = 2 to 2
q = M[i, k] + M[k + 1, j] + Pi – 1PkPj
=> q = M[2, 2] + M[3, 3] + P1P2P3
= 0 + 0 + 4 x 2 x 3 = 24
Here q < M[2, 3] => M[2, 2] = 24 and S[2, 3] = k = 2.
c. For len = 2, i = 3, j = 4 => M[i, j] = M[3, 4] = ∞
For k = i to j – 1 = 3 to 4 – 1 = 3 to 3
q = M[i, k] + M[k + 1, j] + Pi – 1PkPj
=> q = M[3, 3] + M[4, 4] + P2P3P4
= 0 + 0 + 2 x 3 x 1 = 6
Here q < M[3, 4] => M[3, 4] = 6 and S[3, 4] = k = 3.

d. For len = 2, i = 4, j = 5 => M[i, j] = M[4, 5] = ∞
For k = i to j – 1 = 4 to 5 – 1 = 4 to 4
q = M[i, k] + M[k + 1, j] + Pi – 1PkPj
=> q = M[4, 4] + M[5, 5] + P3P4P5
= 0 + 0 + 3 x 1 x 4 = 12
Here q < M[4, 5] => M[4, 5] = 12 and S[4, 5] = k = 4.
(M-table) (S-table)
i↓ j → 1 2 3 4 5
1 0 16
2 0 24
3 0 6
4 0 12
5 0
i↓ j → 1 2 3 4 5
1 0 1
2 0 2
3 0 3
4 0 4
5 0

 For len = 3, i = 1 to n – len + 1 = 1 to 5 – 3 + 1 = 1 to 3
a. For len = 3, i = 1, j = i + len – 1 = 1 + 3 – 1 = 3
=> M[i, j] = M[1, 3] = ∞
For k = i to j – 1 = 1 to 2
i. For k = 1, q = M[i, k] + M[k + 1, j] + Pi – 1PkPj
=> q = M[1, 1] + M[2, 3] + P0P1P3
= 0 + 24 + 2 x 4 x 3 = 24 + 24 = 48. Here q < M[1, 3]
=> M[1, 3] = q = 48 and S[1, 3] = k = 1
ii. For k = 2, q = M[i, k] + M[k + 1, j] + Pi – 1PkPj
=> q = M[1, 2] + M[3, 3] + P0P2P3
= 16 + 0 + 2 x 2 x 3 = 16 + 12 = 28.
Here q < M[1, 3] => 28 < 48 =>M[1, 3] = 28 and
S[1, 3] = k = 2.

b. For len = 3, i = 2, j = i + len – 1 = 2 + 3 – 1 = 4
=> M[i, j] = M[2, 4] = ∞
For k = i to j – 1 = 2 to 3
=> q = M[2, 2] + M[3, 4] + P1P2P4
= 0 + 6 + 4 x 2 x 1 = 6 + 6 = 14. Here q < M[2, 4]
=> M[2, 4] = q = 14 and S[2, 4] = k = 2
=> q = M[2, 3] + M[4, 4] + P1P3P4
= 24 + 0 + 4 x 3 x 1 = 24 + 12 = 36.
Here q < M[2, 4] => 36 < 14 (false) =>M[2, 4] = 14
and S[2, 4] = k = 2.

c. For len = 3, i = 3, j = i + len – 1 = 3 + 3 – 1 = 5
=> M[i, j] = M[3, 5] = ∞
For k = i to j – 1 = 3 to 4
=> q = M[3, 3] + M[4, 5] + P2P3P5
= 0 + 12 + 2 x 3 x 4 = 12 + 24 = 36. Here q < M[3, 5]
=> M[3, 5] = q = 36 and S[3, 5] = k = 3
=> q = M[3, 4] + M[5, 5] + P2P4P5
= 6 + 0 + 2 x 1 x 4 = 6 + 8 = 14.
Here q < M[3, 5] => 14 < 36 =>M[3, 5] = 14 and
S[3, 5] = k = 4.

(M-table) (S-table)
a. For len = 4, i = 1, j = i + len – 1 = 1 + 4 – 1 = 4
=> M[i, j] = M[1, 4] = ∞
For k = i to j – 1 = 1 to 3
=> q = M[1, 1] + M[2, 3] + P0P1P4
i↓ j → 1 2 3 4 5
1 0 16 28
2 0 24 14
3 0 6 14
4 0 12
5 0
i↓ j → 1 2 3 4 5
1 0 1 2
2 0 2 2
3 0 3 4
4 0 4
5 0

= 0 + 14 + 2 x 4 x 1 = 14 + 8 = 22. Here q < M[1, 4]
=> M[1, 4] = q = 22 and S[1, 4] = k = 1
=> q = M[1, 2] + M[3, 4] + P0P2P4
= 16 + 6 + 2 x 2 x 1 = 16 + 6 + 4 = 26.
Here q < M[1, 4] => 26 < 22 (false) =>M[1, 4] = 22
and S[1, 4] = k = 1.
iii. For k = 3, q = M[i, k] + M[k + 1, j] + Pi – 1PkPj
=> q = M[1, 3] + M[4, 4] + P0P3P4
= 28 + 0 + 2 x 3 x 1 = 28 + 6 = 34.
Here q < M[1, 4] => 34 < 22 (false) =>M[1, 4] = 22
and S[1, 4] = k = 1.

b. For len = 4, i = 2, j = i + len – 1 = 2 + 4 – 1 = 5
=> M[i, j] = M[2, 5] = ∞
For k = i to j – 1 = 2 to 4
=> q = M[2, 2] + M[3, 5] + P1P2P5
= 0 + 14 + 4 x 2 x 4 = 14 + 32 = 46. Here q < M[2, 5]
=> M[2, 5] = q = 46 and S[2, 5] = k = 2
=> q = M[2, 3] + M[4, 5] + P1P3P5
= 24 + 12 + 4 x 3 x 4 = 24 + 12 + 48 = 84.
Here q < M[2, 5] => 84 < 48 (false) =>M[2, 5] = 46
and S[2, 5] = k = 2.
=> q = M[2, 4] + M[5, 5] + P1P4P5
= 14 + 0 + 4 x 1 x 4 = 14 + 16 = 30.

Here q < M[2, 5] => 30 < 46 =>M[2, 5] = 30 and
S[2, 5] = k = 4.
(M-table) (S-table)
j = i + len – 1 = 1 + 5 – 1 = 5 => M[i, j] = M[1, 5] = ∞.
For k = i to j – 1 = 1 to 4
i↓ j → 1 2 3 4 5
1 0 16 28 22
2 0 24 14 30
3 0 6 14
4 0 12
5 0
i↓ j → 1 2 3 4 5
1 0 1 2 1
2 0 2 2 4
3 0 3 4
4 0 4
5 0

=> q = M[1, 1] + M[2, 5] + P0P1P5
= 0 + 30 + 2 x 4 x 4 = 30 + 32 = 62. Here q < M[1, 5]
=> M[1, 5] = q = 62 and S[1, 5] = k = 1
=> q = M[1, 2] + M[3, 5] + P0P2P5
= 16 + 14 + 2 x 2 x 4 = 46. Here q < M[1, 5]
=> 46 < 62 =>M[1, 5] = 46 and S[1, 5] = k = 2.
=> q = M[1, 3] + M[4, 5] + P0P3P5
= 28 + 12 + 2 x 3 x 4 = 28 + 12 + 24 = 64.
Here, q < M[1, 5] => 64 < 46 (false). So M[1, 5] = 46
and S[1, 5] = 2.
iv. For k = 4, q = M[i, k] + M[k + 1, j] + Pi – 1PkPj
=> q = M[1, 4] + M[5, 5] + P0P4P5

= 22 + 0 + 2 x 1 x 4 = 28 + 8 = 30.
Here, q < M[1, 5] => 30 < 46. So M[1, 5] = 30 and
S[1, 5] = k = 4.
(M-table) (S-table)
Now, the optimal solution of A1 x A2 x A3 x A4 x A5 = 30.
So, the optimal parenthesis is: ((A1(A2(A3A4)))A5)
i↓ j → 1 2 3 4 5
1 0 16 28 22 30
2 0 24 14 30
3 0 6 14
4 0 12
5 0
i↓ j → 1 2 3 4 5
1 0 1 2 1 4
2 0 2 2 4
3 0 3 4
4 0 4
5 0

 The binary tree is:
(S, 1, 5)
(S, 5, 5)(S, 1, 4)
(S, 2, 4)(S, 1, 1)
(S, 3, 4)(S, 2, 2)
(S, 4, 4)(S, 3, 3)
1. ( 13. )
2. ( 11. )
4. ( 10. )
6. ( 9. )
3. A1
5. A2
7. A3 8. A4
12. A5

Disjoint Set Data
Structure:
 A disjoint set data structure maintains a collection of set
S = {s1, s2, … , sk} of disjoint dynamic set. Each set is
identified by a representative, which is some member of
set.
 Disjoint Set Operations:
 If „x‟ is given then,
i. Our task is to find out which set it belongs
ii. Union of 2 sets (if they are disjoint)
iii. Let S = {s1, s2, … , sk} be a set where Si ∩ Sj = ϕ
 If x є Si is any element, we call „x‟ as an object of the set
Si, but to find an element x є Si. We are searching the
element which represents the set Si.

 Disjoint set has 3 operations. If „x‟ is an element of a set
with only one element „x‟ representing to itself, then the
operations are:
i. Make_set(x): It creates a new set with only one
element „x‟ representing to itself.
ii. Union(x, y): The set containing „x‟ is appended to
the set containing „y‟. The representative of Union(x,
y) is either the representative of the 1st or 2nd set.
iii. Find_set(x): It returns a pointer to the representative
of the set containing „x‟.
 Application of Disjoint Set Data Structure:
 One of the application of disjoint set data structure is to
determine the connected components of an undirected
graph.

Test the correctness of
the Undirected Graph:
 Procedure:
I. CONNECTED_COMPONENTS(G)
i. for each vertex v є V[G]
ii. do Make_set(v)
iii. for each edge (u, v) є E[G]
iv. do if Find_set(u) ≠ Find_set(v)
v. then Union(u, v)
II. SAME_COMPONENTS(u, v)
i. if Find_set(u) = Find_set(v)
ii. then return true
iii. else return false

 Example:
1. Suppose that connected components is run on the
undirected graph G = (V, E), where V = {a, b, c … j}
and the edges „E‟ are processed in the following order:
(b, d), (e, g), (a, c), (h, i), (a, b), (e, f), (b, c). List the
vertices in each connected components and draw the
graph.
Edge Processed Correctness of Disjoint Set
Initial set {a}, {b}, {c}, {d}, {e}, {f}, {g}, {h}, {i}, {j}
(b, d) {a}, {b, d}, {c}, {e}, {f}, {g}, {h}, {i}, {j}
(e, g) {a}, {b, d}, {c}, {e, g}, {f}, {h}, {i}, {j}
(a, c) {a, c}, {b, d}, {e, g}, {f}, {h}, {i}, {j}
(h, i) {a, c}, {b, d}, {e, g}, {f}, {h, i}, {j}
(a, b) {a, c, b, d}, {e, g}, {f}, {h, i}, {j}
(e, f) {a, c, b, d}, {e, g, f}, {h, i}, {j}
(b, c) {a, c, b, d}, {e, g, f}, {h, i}, {j}

 The graph for correctness of disjoint sets are:
ba
c d
fe
g
ih
j
(1)
(2)
(3)
(4)

2. V = {a, b, c, d, e, f, g, h, I, j, k}
E = (d, i), (f, k), (g, i), (b, g), (a, h), (i, j), (d, k), (b, j),
(d, f), (g, j), (a, e), (i, d)
Edge Processed Correctness of Disjoint Set
Initial set {a}, {b}, {c}, {d}, {e}, {f}, {g}, {h}, {i}, {j}, {k}
(d, i) {a}, {b}, {c}, {d, i}, {e}, {f}, {g}, {h}, {j}, {k}
(f, k) {a}, {b}, {c}, {d, i}, {e}, {f, k}, {g}, {h}, {j}
(g, i) {a}, {b}, {c}, {d, i, g}, {e}, {f, k}, {h}, {j}
(b, g) {a}, {b, d, i, g}, {c}, {e}, {f, k}, {h}, {j}
(a, h) {a, h}, {b, d, i, g}, {c}, {e}, {f, k}, {j}
(i, j) {a, h}, {b, d, i, g, j}, {c}, {e}, {f, k}
(d, k) {a, h}, {b, d, i, g, j, f, k}, {c}, {e}
(b, j) {a, h}, {b, d, i, g, j, f, k}, {c}, {e}
(d, f) {a, h}, {b, d, i, g, j, f, k}, {c}, {e}
(g, j) {a, h}, {b, d, i, g, j, f, k}, {c}, {e}
(a, e) {a, h, e}, {b, d, i, g, j, f, k}, {c}
(i, d) {a, h, e}, {b, d, i, g, j, f, k}, {c}

 The graph for correctness of disjoint sets are:
ha
e
(1)
c
(3)
(2)
db
g j f
i
k

Linked List Representation
of Disjoint Set:
 Each set is represented by linked list.
 The first object in every linked list is the representative of
the set. Each object in the linked list has 3 components.
That is:
i. Set member
ii. A pointer to the object containing the next member
iii. Another pointer back to the representative.
 Each list maintains pointers head to the representative and
tail to the last object of the list.

 E.g.: i)
ii)
 Implement the Union of (g, c) into linked list:
c h be
head tail
f g d
head tail
f g d c h e b
head tail

Disjoint Set Forest:
 Each set is represented by a tree. Each node contains pointer
to its parent. The root of each tree contains the
representative and also is its own parent.
 It has 3 operations. That is:
i. Make_set(x)
ii. Find_set(x)
iii. Union(x, y)
1. Make_set(x):
 It creates a tree with a single node. The algorithm for this
is:
Make_set(x)
1. P[x] ← x
2. rank[x] ← 0
Here, P[x] is a parameter which returns a pointer to root.

2. Find_set(x):
 Recursively, it can find the root of the tree which
contains the node „x‟. The algorithm for this is:
Find-set(x)
1. if x ≠ P[x]
2. then P[x] ← Find_set(P[x])
3. return P[x]
 The Find_set procedure is a 2-pass method, it makes
one pass of the find path to find the root and it makes a
2nd pass break down the find path to update each node,
so that it points directly to the root.
 Each call of Find_set(x) returns P(x) is step-3. If „x‟ is
the root then the step-2 is not executed and P(x) = x is
returned.

3. Union(x, y):
Union(x, y)
1. Link(Find_set(x), Find_set(y))
Link(x, y)
1. if rank[x] > rank[y]
2. then P[y] ← x
3. else P[x] ← y
4. if rank[x] = rank[y]
5. then rank[y] ← rank[y] + 1

Heuristics to improve the
running time for UNION
operation:
 A sequence „n – 1‟ union operations may create a tree that is
just a linear chain of „n‟ nodes.
 By using 2 heuristics, we can achieve a running time that is
almost linear in the total no. of „m‟ operations. That are:
i. Union by Rank (1st heuristic)
ii. Path compression (2nd heuristic)
1. Union by Rank (1st heuristic):
 It is similar to the weighted union heuristic, we used with
the linked list representation. The idea is to make the root
of the tree with fewer nodes point to the root of the tree
with more nodes.

 Rather than explicitly keeping size of the sub-tree rooted
at each node, we shall use the rank.
 For each node, we maintain the rank. i.e.: An upper
bound on the height of the node.
 In Union by rank, the root will smaller rank is made to
point to the root with larger rank during a union
operation.
2. Path Compression:
 We use it during Find_set operation to make each node
on the find path point directly to the root.
 Path compression does not check any rank.

Greedy Method:
 The greedy method is not as powerful as dynamic
programming technique and also that is not applicable to a
wider area like Dynamic Programming.
 By Greedy Algorithms, we can solve some of the well
known problems effectively, in which some function (called
the optimization for objective function) is to be optimized
(minimum or maximum) subject to be some constants.
 A greedy algorithm always makes the choice that looks best
at the moment. i.e.: It makes a locally optimal choice in the
hope that this choice will lead to a globally optimal
solution.
 Greedy algorithms don‟t always yield optimal solutions, but
for many problems they do. For designing, a greedy
method, we need to follow 2 elements.

 Configuration: Different choices, collections or values
to find.
 Objective Function: A score assigned to configurations
which we want to either maximize or minimize.
Common Rules:
 Some Problem by making sequential decision.
 Decision are made one by one in some order.
 Each decision is made using a greedy method.
 A decision once made is not changed later (usually).
Application of Greedy Method:
1. Assembly line scheduling
2. Activity Selection Problem
3. Huffman Coding (data compression)
4. Task Scheduling

5. Graph Algorithms:
i. Breadth first search
ii. Depth first search,
iii. minimum cost spanning tree(kruskal‟s and prim‟s),
iv. single source shortest path (dijkstra‟s, bellman-
ford‟s and topological sorting),
v. All pair shortest path(Floyd-Warshall‟s)
6. Knapsack Problems
Steps for achieving Greedy Algorithm:
 Feasible: A feasible solution is a solution that satisfies
the constants. Here we check whether it satisfies all the
possible problem constraints or not. So, as to obtain at
least one solution to our problem.

 Local optimal choice: An optimal solution is a feasible
solution that optimizes the objective or optimization
function. The greedy method does not always give
optimal solution, but for many problems it does. In this,
choice, should be the optimum which is selected from the
current available feasible choices.
 Unalterable: Once the choice is made at any subsequent
step that choice is not altered.

Assembly Line Scheduling:
 In assembly line scheduling, there are two assembly lines,
each with „n‟ stations numbered j = 1, 2, … n. Station „j‟
on line is denoted at Si, j, where i = assembly line 1 or
assembly line 2.
 The stations were built at different times and with different
technologies, however the time required at each station
varies, even between stations at the same position on the
two different lines. The Assembly time required at station
Si, j by ai, j.
 Line „i‟ has entry time „ei‟ and an exit time xi and time to
switch lines after station Si, j is ti, j, where i = 1, 2 and j = 1,
2, … n – 1, because after the nth session, assembly is
complete.

 Here the question arises that, which stations should be
chosen to minimize assembly time for one product. For this
let consider an example that:
 There are 3 stations on each line, the time to get from the
entry to line 1 is e1 and the time to do station 1 on line 1
is a11 etc., and the time to transfer from line 1 to line 2
between stations 1 and 2 is t11 etc. and the time to get
from station 3 to exit is x2. Similarly it is done for line 2.
entry
a11 a13a12
a21 a23a22
exit
e1
e2
x1
t11 t12
t21
t22
S1, 1 S1, 2 S1, 3
x2
S2, 1 S2, 2 S2, 3

 The Brute force way of minimizing the time through the
factory is infeasible when there are many stations.
 If we are given a list of which stations to use in line 1 and
which to use in line 2, it is easy to compute in θ(n) time,
how long it takes a product to pass through the factory.
 There are 2n possible way to choose stations, which we see
by viewing the set of stations used in line 1 as a subset of
{1, 2, … n} and noting that there are 2n such subsets.
 Thus determining the fastest way through the factory by
enumerating all possible ways and computing how long
each takes would require Ω(2n) time, which is infeasible
when „n‟ is large.

1. Structure of Fastest way:
 For assembly line scheduling problem, we can perform the
following steps.
 Let‟s consider the fastest possible way for a product to get
from the starting point through station Si, j. If j = 1, there
is only one way.
 For j = 2, 3, … n, there are two choices. i.e.: if j > 1 ,
fastest possible way is the minimum of :
i. Fastest way to S1, j – 1 then to S1, j
ii. Fastest way to S2, j – 1 then to S1, j
 Fastest way through S1, j includes fastest path through
previous station. Similarly fastest path through S2, j includes
fastest path through previous station.
 We can say that, for assembly line scheduling, an optimal
solution to a problem:

 finding the fastest way through station Si, j contains within
it an optimal solution of sub-problems: finding the fastest
way through either S1, j – 1 or S2, j – 1, which is the optimal
sub-structure property.
 For assembly line scheduling, the fastest way through
station S1, j must go through station j – 1 on either line 1 or
line 2. So, the fastest way through station S1, j is either:
 The fastest way through station S1, j – 1 and then directly
through station S1, j or
 The fastest way through station S2, j – 1, a transfer from
line 2 to line 1 and then through station S1, j.
 Using symmetric reasoning, the fastest way through station
S2, j is either
 The fastest way through station S2, j – 1 and then directly
through station S2, j or

 The fastest way through station S1, j – 1 a transfer from
line 1 to line 2 and then through station S2, j.
 To solve the problem of finding the fastest way through
station „j‟ of either line, we solve the sub-problems of
finding the fastest ways through station j – 1 on both lines.
2. A Recursive Solution:
 A second step of the dynamic programming paradigm is to
define the value of an optimal solution recursively in terms
of the optimal solutions to sub-problems.
 For assembly line scheduling problem, we pick as our sub-
problems the problems of finding the fastest way through
station „j‟ on both lines, for j = 1, 2, … n.
 Let fi[j] denote the fastest possible time to get a product
from the starting point through station Si, j.

 Our ultimate goal is to determine the fastest time to get a
product, all the way through the factory, denoted by „f*‟,
which is determined as:
f* = min(f1[n] + x1, f2[n] + x2) --- (1)
 It is also easy to reason about f1[1] and f2[1]. To get
through station 1 on either line, a product just goes directly
to station. So,
f1[1] = e1 + a1, 1 --- (2) and
f2[1] = e2 + a2, 1 --- (3)
 To compute fi[j] for j = 2, 3, … n and i = 1, 2. We recall the
fastest way through station S1, j is either the fastest way
through station S1, j – 1 and then directly through station S1, j
or the fastest way through station S2, j -1, a transfer from line
2 to line 1 and then through station S1, j.

 In the first case, we have f1[j] = f1 [j – 1] + a1, j and in the
later case, f1[j] = f2[j – 1] + t2, j – 1 + a1, j. Thus,
f1[j] = min(f1[j – 1] + a1, j, f2[j – 1] + t2, j – 1 + a1, j --- (4),
for j = 2, 3, … n.
 Symmetrically we have,
f2[j] = min(f2[j – 1] + a2, j, f1[j – 1] + t1, j – 1 + a2, j --- (5),
for j = 2, 3, … n.
 Combining equation 2 to 5, we obtain the recursive
equations,
f1[j] = i) e1 + a1, 1, if j = 1
ii) min(f1[j – 1] + a1, j, f2 [j – 1] + t2, j – 1 + a1, j), if j ≥ 2
--- (6)
f2[j] = i) e2 + a2, 1, if j = 1
ii) min(f2[j – 1] + a2, j, f1 [j – 1] + t1, j – 1 + a2, j), if j ≥ 2
--- (7)

 The fi[j] values give the values of optimal solutions to sub-
problem.
 To keep track of how to construct an optimal solution. Let
li[j] is the line number 1 or 2, whose station j – 1 is used in
a fastest way through station Si, j.
 Here, i = 1, 2 and j = 2, 3, … n and l* be the line whose
station „n‟ is used in a fastest way through the entire
factory.
 li[j] values is used to trace a fastest way. Using the values
of l* and li[j], we trace a fastest way through the factory.
3. Computing the fastest times:
 A recursive algorithm based on equation (1) and the
recurrences equation 6 and 7 to compute the fastest way
through the factory. There is a problem with such a
recursive algorithm: its running time is exponential in „n‟.

 Let ri(j) be the number of references made to fi[j] in a
recursive algorithm. From equation (1), we have:
r1(n) = r2(n) = 1 --- (8)
 From the recurrence equation (6) and (7), we have:
r1(j) = r2(j) = r1(j + 1) + r2(j + 1) --- (9)
for j = 1, 2, … n – 1
 If we compute fi[j] values in a different order from the
recursive way, observe that for j ≥ 2, each value of fi[j]
depends only on the values of f1[j – 1] and f2[j – 1].
 By computing the fi[j] values in order of increasing station
numbers „j‟ -: left to right. The Fastest_way procedure takes
as input the values ai, j, ti, j, ei and xi, as well as n, the
number of stations in each assembly line.

Algorithm:
Fastest_way(a, t, e, x, n)
1. f1[1] ← e1 + a1, 1
2. f2[1] ← e2 + a2, 1
3. for j ← 2 to n
4. do if f1[j – 1] + a1, j ≤ f2[j – 1] + t2, j – 1 + a1, j
5. then f1[j] ← f1[j – 1] + a1, j
6. l1[j] ← 1
7. else f1[j] ← f2[j – 1] + t2, j – 1 + a1, j
8. l1[j] ← 2
9. if f2[j – 1] + a2, j ≤ f1[j – 1] + t1, j – 1 + a2, j
10. then f2[j] ← f2[j – 1] + a2, j
11. l2[j] ← 2
12. else f2[j] ← f1[j – 1] + t1, j – 1 + a2, j
13. l2[j] ← 1

14. If f1[n] + x1 ≤ f2[n] + x2
15. then f* = f1[n] + x1
16. l* = 1
17. else f* = f2[n] + x2
18. l* = 2
 Here, lines 1 – 2 compute f1[1] and f2[1] using equations
(2) and (3). Then the loop of lines 3 – 13 computes fi[j] and
li[j] for i = 1, 2 and j = 2, 3, … n.
 Line 4 – 8 compute f1[j] and l1[j] using equation (4) and
lines 9 – 13 compute f2[j] and l2[j] using equation (5).
Finally lines 14 – 18 computes f* and l* using equation (1).
 Because lines 1 – 2 and 14 – 18 take constant time and each
of the n – 1 iterations of the for loop of lines 3 – 13 takes
constant time, the entire procedure takes θ(n) time.

4. Constructing the fastest way through the
factory:
 Having computed the values of fi[j], f*, li[j] and l*, we need
to construct the sequence of stations used in the fastest way
through the factory.
 The algorithm for this is:
Print_Stations(l, n)
1. i ← l*
2. print “line” i, “station” n.
3. for j ← n down to 2
4. do i ← li[j]
5. print “line” i, “station” j – 1.

Activity Selection
Problem:
 Suppose, we have a set S = {a1, a2, … an} of „n‟ proposed
activities that wish to use a resource, such as a lecture hall,
which can be used by only one activity, at a time.
 Each activity „ai‟ has a start time „si‟ and a finish time „fi‟,
where 0 ≤ si < fi < ∞. If selected activity „ai‟ takes place
during half-open time interval [si, fi).
 Activities „ai‟ and „aj‟ are compatible if the intervals [si, fi)
and [sj, fj) don‟t overlap. i.e.: „ai‟ and „aj‟ are compatible if
si ≥ fj or sj ≥ fi.
 The activity selection problem is to select a maximum size
subset of mutually compatible activities.

Algorithm for Recursive_Activity_Selector:
Recursive_Activity_Selector(s, f, i, j)
1. m ← i + 1
2. while m < j and sm < fi /* find the 1st activity in si, j*/
3. do m ← m + 1
4. if m < j
5. then return {am} U Recursive_Activity_Selector(s, f, m, j)
6. else return ϕ
Running time of Recursive_Activity_Selector:
 Assuming that the activities have already been stored by
finish time, the running time of the calling the procedure
Recursive_Activity_Selector [R.A.S]: (s, f, 0, n + 1) is:
θ(n).

Greedy Algorithm for Activity Selector:
Greedy_Activity_Selector(s, f)
1. n ← length[s]
2. A ← {a1}
3. i ← 1
4. for m ← 2 to m
5. do if sm ≥ fi
6. then A ← A U {am}
7. i ← m
8. return A
Time Complexity for Greedy_Activity_Selector:
 Like recursive version greedy selector schedules a set of „n‟
activity in θ(n) time, assuming that the activities were
already sorted initially by their finish time.

Example:
 Consider the following set „S‟ of activities, which we have
sorted in monotonically increasing order of finish time:
 For this, the subset {a3, a9, a11} consists of mutually
compatible activities. It is not a maximal subset. However
since the subset {a1, a4, a8, a11} is large.
 In fact {a1, a4, a8, a11} is a largest subset of mutually
compatible activities; another largest subset is: {a2, a4, a9,
a11}.
i 1 2 3 4 5 6 7 8 9 10 11
si 1 3 0 5 3 5 6 8 8 2 12
fi 4 5 6 7 8 9 10 11 12 13 14

i si fi
1 1 4
2 3 5
3 0 6
4 5 7
5 3 8
6 5 9
7 6 10
8 8 11
9 8 12
10 2 13
11 12 14
a1(1, 4)
a2(3, 5)
a3(0, 6)
a4(5, 7)
a5(3, 8)
a6(5, 9)
a7(6, 10)
a8(8, 11)
a9(8, 12)
a10(8, 11)
a11(12,14)
0 1 2 3 54 6 7 8 9 1110 1312 14

Huffman Code:
 We are using the Huffman codes for compressing the data,
we consider the data to be a sequence of characters.
 Huffman‟s algorithm uses a table of the frequencies of
occurrence of the characters to build of an optimal way of
representing each character as a binary string.
 There are many ways to represent a file of information. We
consider the problem of designing a binary character code,
where in each character is represented by a unique binary
string.
 If we use a each length code, we need 3 bits to represent 6
characters. That is:
a = 000, b = 001, c = 010, d = 011, e = 100 and f = 101

 A variable length code can do considerably better than a
fixed length code by giving frequent characters. i.e.: short
code words and infrequent characters i.e.: long code words.
Algorithm for Huffman Code:
Huffman(C)
1. n ← |C|
2. Q ← C
3. for i ← 1 to n – 1
4. do allocate a new node „z‟
5. Left |z| ← x ← Extract_min(Q)
6. Right |z| ← y ← Extract_min(Q)
7. f[z] ← f[x] +f[y]
8. Insert(Q, z)
9. return Extract_min(Q)
10. return the root of the tree

Example:
 Then its Huffman code is given below.
Data a b c d e f
Frequency 45 13 12 16 9 5
(i) a d b c e f
45 16 13 12 9 5
14
(ii) a d ef b c
45 16 14 13 12
25
(iii) a bc d ef
45 25 16 14
30
(iv) a def bc
45 30 25
55
(v) defbc a
55 45
100

 The Huffman code is:
13
5
45
16
d
100
55
25 30
12 14
9
a
c b
0
fe
1
0 1
0 1 0 1
0 1

 Code:
Prefix Code:
 The codes in which no code word present, which is also a
prefix of some other code word is known as prefix code.
 It is possible to show that the optimal data compression
achievable by a character code can always be achieve with
a prefix code, so there is no loss of generality in restricting
attention of prefix code.
d111 16
a0 45
f1100 5
e1101 9
c100 12
b101 13
1
0
1
0
25
14
30
1
0
0
1
55
0
1
100

Task Scheduling:
 In case of task scheduling „T‟, will be defined as set of „n‟
tasks having a start time „si‟ and a finishing time „fi‟ and
si < fi always.
 Goal: Complete all the tasks without conflict
(overlapping) the schedules with minimum numbers of
machines.
T4
T2
T1
1 2 3 4 5 6 7 8
T4(3, 6)
T3 T2(1, 2), T3(2, 5)
T1(1, 4)m1
m2
m3
m

Algorithm for Task Scheduling:
Algo_TaskScheduling (T)
Input: set „T‟ of tasks with start time „si‟ and finish time „fi‟.
where si and fi > 0 and si < fi
Output: Non-conflict scheduling with minimum number of
machines
m ← 0 (no. of machines)
while(T ≠ ϕ)
remove task „i‟ with smallest „si‟;
if a machine „j‟ for „i‟ then
schedule i to j on machine „j‟.
else
m ← m + 1;
schedule „i‟ to machine „m‟
end.

Example:
T5[1, 3], T3[1, 4], T1[2, 5], T2[3, 7], T4[4, 7], T7[6, 9] and
T6[7, 8].
T [si, fi]
T1 [2, 5]
T2 [3, 7]
T3 [1, 4]
T4 [4, 7]
T5 [1, 3]
T6 [7, 8]
T7 [6, 9]
T2 T6
T1
T3
T5
1 2 3 4 5 6 7 8
T4
m1
m2
m3
9
T7
0

Graph Algorithms:
Graph:
 It is a diagram which contains set of vertices and set of
edges which is denoted as G = (V, E)
Representation of Graph:
 There are 2 standard ways to represent graph G = (V, E) as a
collection of adjacency list or as an adjacency matrix. Either
way is applicable to both directed and undirected graphs.
 The adjacency list representation is usually preferred
because it provides a compact way to represent sparse
graphs those for which |E| is much less than |V|2.
 An adjacency matrix representation may be preferred
however, when the graph is dense; |E| is close to |V|2, or
when we need to be able to tell quickly if there is an edge
connecting two given vertices.

 The adjacency list representation of a graph G = (V, E)
consists of an array Adj of |V| lists, one for each vertex in V.
For each u є V, the adjacency list Adj[u] contains (u, v) є E.
i.e.: Adj[u] consists of all the vertices to „u‟ in „G‟.
 If „G‟ is directed graph, the sum of the length of all the
adjacency list is |E|, since an edge of the form (u, v) is
represented by having „v‟ appear in Adj[u].
 If „G‟ is an undirected graph, the sum of length of all the
adjacency list in 2|E|. Since, if (u, v) is an undirected edge,
the „u‟ appears in v‟s adjacency list and vice versa.
 Adjacency lists can readily be adapted to represented
weighted graphs i.e.: graphs for which each edge has
associated weight typically given by a weight function that
is: w: E → R.

 E.g.: Let G = (V, E) be a weighted graph with weight
function „w‟. The weight w(u, v) of the edge (u, v) є E is
stored in vertex „v‟ in u‟s adjacency list.
 A disadvantage of adjacency list representation is that there
in no quicker way to determine if a given edge (u, v) is
present in the graph than to search for „v‟ in adjacency list
Adj[u].
 For adjacency matrix representation of a Graph, G = (V, E)
where vertices are numbered 1, 2, … |V| in some arbitrary
manner. Then the adjacency matrix representation of a
graph „G‟ consists of |V| x |V| matrix A = aij, such that:
aij = 1, if (i, j) є E and 0, otherwise

Breadth First Search
(BFS):
 Breadth First Search (BFS) is one of the simplest algorithms
for searching a graph. Given a graph G = (V, E) and a
distinguished source vertex „s‟. BFS systematically explores
the edges of „G‟ to discover every vertex that is reachable
from „s‟.
 It computes the distance from „s‟ to each reachable vertex. It
also produces a breadth-first tree with root „s‟, that contains
all reachable vertices.
 For any vertex „v‟, reachable from „s‟, the path in the
breadth first tree from „s‟ to „v‟, corresponds to a shortcut
path from „s‟ to „v‟ in „G‟. That is a path containing a
smallest number of edges. This algorithm works on both
directed and undirected graph.

Algorithm for BFS:
BFS(G, s)
1. for each vertex u є V[G] – {s}
2. do color[u] ← white
3. d[u] ← ∞
4. π[u] ← ∞
5. color[s] ← gray
6. d[s] ← 0
7. π[s] ← 0
8. Q ← ϕ
9. Enqueue(Q, s)
10. while Q ≠ ϕ
11. do u ← Dequeue(Q)
12. for each vertex v є Adj[u]

13. do if color[v] = white
14. then color[u] ← gray
15. d[v] ← d[u] + 1
16. π[v] ← u
17. Enqueue[Q, v]
18. color[u] ← black
Time Complexity of BFS:
 The operation of enqueuing and dequeuing take O(1) time.
So the total time devoted to queue operations in O(v),
because the adjacency list of each vertex is scanned only,
when the vertex dequeued. Each adjacency list is scanned at
most one.
 Since the sum of the lengths of all the adjacency list θ(E),
the total time spend in scanning adjacency list in each O(E).
The overhead for initialization is O(v). Thus, the total
running time of BFS is O(V + E).

Example:
i)
ii)
iii)
∞ 0
∞
∞ ∞
∞∞ ∞
r s t u
yxwv
Q s
0
1 0
∞
∞ ∞
∞1 ∞
r s t u
yxwv
Q w
1
r
1
1
1
1 0
∞
2 ∞
∞1 2
r s t u
yxwv
Q r
1
t
2
1
1
x
2
2
2

iv)
v)
vi)
1 0
2
2 ∞
∞1 2
r s t u
yxwv
Q t
2
x
2
1
1
v
2
2
2
2
1 0
2
2 3
∞1 2
r s t u
yxwv
Q x
2
v
2
1
1
u
3
2
2
2
3
1 0
2
2 3
31 2
r s t u
yxwv
Q v
2
u
3
1
1
y
3
2
2
2
3
3

vii)
viii)
ix)
1 0
2
2 3
31 2
r s t u
yxwv
Q u
3
1
1
y
3
2
2
2
3
3
1 0
2
2 3
31 2
r s t u
yxwv
Q1
1
y
3
2
2
2
3
3
1 0
2
2 3
31 2
r s t u
yxwv
Q = ϕ1
1
2
2
2
3
3

Depth First Search (DFS):
 In DFS, edges are explored out of the most recently
discovered vertex „v‟, which still has unexplored edges
leaving it.
 When all the v‟s edges have been explored the search
backtracks to explores edges leaving the vertex from which
„v‟ was discovered.
 This process continues until we have discovered all the
vertices that are reachable from the original source vertex.
If any undiscovered vertices remain, then one of them is
selected as a new source and the search is repeated from
that source.
 This entire process is repeated until all vertices are
discovered.

 In this algorithm, we are using 3 colors i.e.: each vertex is
initially white, is grayed when it is discovered in the search
and blackens, when it is finished that is when its adjacency
list has been examined completely.
 Each vertex „v‟ has 2 time stamps. That is:
 1st time stamp d[v] records, when „v‟ is 1st discovered
and grayed.
 The 2nd time stamp f[v] records, when the search finishes
examining v‟s adjacency list and blacken „v‟.
 The procedure DFS records, when it discovered vertex „u‟
in the variable d[u] and it finishes vertex „u‟ in the variable
f[u]. For every vertex „u‟, d[u] < f[u].

Algorithm for DFS:
DFS(G)
1. for each vertex u є V[G]
2. do color[u] ← white
3. π[v] ← nil
4. time ← 0
5. for each vertex u є V[G]
6. do if color[u] = white
7. then DFS_VISIT(u)
DFS_VISIT(u)
1. color(u) ← gray
2. time ← time + 1
3. d[u] ← time
4. for each v є adj[u]

5. do if color[v] = white
6. then π[v] ← u
7. DFS_VISIT(v)
8. color[u] ← Black
9. f[u] ← time ← time + 1
 {u, v, w, x, y, z} = u є V, do color[u] = white, π[u] = nil,
time ← 0
G
u v w
zyx
2
1
0
u v w
zyx
2/7
1/s
9/12
10/11
3/64/5

Time Complexity of DFS:
 Step 1 – 3 and step 5 – 7 of DFS takes time θ(v).
 The procedure DFS_VISIT is called exactly once for each
vertex v є V. Since, DFS_VISIT is invoked only on white
vertices and the 1st thing it does as it paint the vertex Gray.
 During an execution of DFS_VISIT(v), the loop on step 4
to 7 executed |adj[v]| times.
 The total cost of executing step 4 to 7 of DFS_VISIT is
θ(E). Thus the running time of DFS is therefore θ(V + E).

Minimum Spanning Tree:
 A spanning tree is a sub-graph of a given graph, which
includes all vertices without making any cycle and having
only one path.
 Let G = (V, E), where „G‟ is an undirected graph, „V‟ is the
set of vertices and „E‟ is set of edges. We have a weight
w(u, v) specifying the cost to connect „u‟ and „v‟.
 Our aim is to be find acyclic subset T is the subset of E,
that connects all of the vertices and whose total weight is:
w(T) = Σw(u, v) is minimized (u, v) є T.
 Since, „T‟ is acyclic and connects all of the vertices, it
must form a tree, which is known as spanning tree, since it
spans the graph „G‟.

Growing a Minimum Spanning Tree:
 Let „G‟ be a connected undirected graph, where the graph
G = (V, E) with a weight function „w‟ and we have to
find the minimum spanning tree for „G‟.
 We grow the minimum spanning tree, one edge at a time.
The algorithm manages a set of edges „A‟, that is always
a subset of some minimum spanning tree.
 At each step, we determine an edge (u, v) that can be
added to „A‟, without violating this invariant, in the sense
that A U {(u, v)} is also a subset of minimum spanning
tree, we call such an edge, „a‟ safe edge for „A‟, since it
can be safety added to „A‟, while maintaining the
invariant.
 A minimum spanning tree problem has 2 methods. i.e.:
i. Kruskal‟s Algorithm for MST and
ii. Prim‟s Algorithm for MST

Procedure/Algorithm for Minimum Spanning
Tree (MST):
Generic_MST(G, w)
1. A ← ϕ
2. while „A‟ does not from a spanning tree
3. do find an edge (u, v) that is safe for „A‟.
4. A ← A U {(u, v)}
5. return A.
1. Kruskal's Algorithm for MST:
 In Kruskal‟s algorithm, the set „A‟ is a forest.
 The safe edge added to „A‟ is always a least-weight edge
in the graph that connects 2 distinct components.
 It is based directly on the generic minimum spanning tree
algorithm.

 It finds a safe edge to add to the growing forest by
binding of all the edges that connects any 2 trees in the
forest, an edge (u, v) of least weight.
 It uses disjoint set data structure to maintain several
disjoint sets of elements. Each set contains the vertices in
a tree of the current forest.
 The operation Find_set(u) returns a representative
element from the set that contains „u‟ and „v‟ belong to
the same tree by testing whether Find_set(u) equals to
Find_set(v).
Time complexity for Kruskal’s Algorithm:
 The running time for Kruskal‟s algorithm for the graph
G = (V, E), depends on the implementation of disjoint set
data structure.
 Initialisation takes O(v) time.

 Time to start the edges take O(ElgE) time implies there
are O(E) operation on the disjoint set forest which takes
O(ElgE) times.
 So, the total running time of Kruskal's algorithm is:
O(ElgE).
Algorithm:
MST_KRUSKAL
1. A ← ϕ
2. for each vertex v є V[G]
3. do Make_set(v)
4. sort the edges of „E‟ into non-decreasing order by
weight
5. for each edge (u, v) є E, taken in non-decreasing order
by weight

6. do if Find_set(u) ≠ Find_set(v)
7. then A ← A U {(u, v)}
8. Union(u, v)
9. return „A‟
Example:
 Here A = Spanning tree = ϕ, E = set of edges, V = set of
vertices of graph „G‟, v = vertex.
 E = {gh, gf, ci, cf, ab, gi, cd, hi, bc, ah, dc, fe, bh, df}
a
b c d
e
f
i
h g
21
6
2
7
7
8
8
4
4
9
10
11 14

a
b c d
e
f
i
h g
21
6
2
7
7
8
8
4
4
9
10
11 14
E W
gh 1 √
gf 2 √
ci 2 √
cf 4 √
ab 4 √
gi 6 x
cd 7 √
hi 7 x
ah 8 √
bc 8 x
dc 9 √
fe 10 x
bh 11 x
df 14 x
Now, A = {gh, gf, ci, cf, ab, cd, ah, dc}
Here V = 9 element, A = V – 1 = 8 elements
Total cost = 4 + 8 + 1 + 2 + 2+ 4 + 7 + 9=37.
So, the required spanning tree is:
a
b c d
e
f
i
h g
21
2
7
8
4
4
9

2. Prim's Algorithm for MST:
 Prim‟s algorithm is a special case of the generic
minimum spanning tree algorithm. It has the property
that the edges in the set „A‟ always form a single tree.
Hence, the tree starts with an arbitrary root vertex „r‟ and
grows until the tree spans all the vertices in „V‟.
 At each step, a light edge connecting a vertex in „A‟ to a
vertex in „V – A‟ is added to the tree. Let „G‟ be a
connected graph and „r‟ is the root of the spanning tree.
All the vertices, that are not in the tree reside in a priority
queue „Q‟, based on a key field.
 For each vertex „v‟, key[v] is the minimum weight of any
edge connecting „v‟ to a vertex in the tree. If there is no
edge key[v] = ∞, π[v] denotes the parent of „v‟.

Performance of Prim’s Algorithm:
 The performance of prim‟s algorithm depends on how we
implement the min priority queue „Q‟. If we implement
priority queue through binary min heap. We can use the
Build_mean_heap procedure to perform the initialization.
 Step 1 to 5 take O(v) times. The body of the while loop is
executed |V| times and since each extract min operation
like O(lgV) time implies total time for all calls to extract
min is O(VlgV).
 The for loop in step number 8 – 11 executed O(E) times
and decreases key operation takes O(lgV) times implies it
will take time O(ElgV) time. So the total running time for
Prim‟s algorithm is equal to O(VlgV + ElgV) = O(ElgV).
 If we implement priority queue using Fibonacci heap,
then extract min procedure take O(lgV) time. But
decrease key take O(1) time. So the total time is equal to
O(E + VlgV).

Algorithm:
MST_PRIM(G, w, r)
1. for each u є V[G]
2. do key[u] ← ∞
3. π[u] ← Nil
4. key[r] ← 0
5. Q ← V[G]
6. while Q ≠ ϕ
7. do u ← Extract_min(Q)
8. for each v є adj[u]
9. do if v є Q and w(u, v) < key[v]
10. then π[v] ← u
11. Key[v] ← w(u, v)

Example:
 Here key[a] = ∞, π[a] = Nil
 We start from the node a → ab, ah = {4, 8}. Here, ab is
selected from the list, then from b → bc, bh is added to
the list and the list becomes bc, bh, ah = {8, 11, 8}. Here,
bc is selected.
 From c → ci, cf, cd is added to the list. So the list
became ci, cf, cd, bh, ah = {2, 4, 7, 11, 8}. From which ci
is selected. From i → ih, ig is added to the list. So the list
becomes cf, cd, bh, ah, ig, ih = {4, 7, 11, 8, 6, 7}.
a
b c d
e
f
i
h g
21
6
2
7
7
8
8
4
4
9
10
11 14

 From the list cf has minimum value. So „cf‟ is selected
first and from f → fg, fd, fe is added to the list. So the list
becomes fg, fd, fe, ig, ih, cd, bh, ah = {2, 14, 10, 6, 7, 7,
11, 8}. From the list fg is selected, which has minimum
value.
 From g → gh is added to the list. So the list becomes gh,
ig, fd, fe, ih, cd, bh, ah = {1, 6, 14, 10, 7, 7, 11, 8}. Here
gh has minimum value. So it is selected and from h → ah
is added to the list.
 So the list becomes ah, bh, ig, fd, fe, ih, cd = {8, 11, 6,
14, 10, 7, 7}. Here ig = 6, but when we consider it, it
forms a cycle. Then we found ih and cd whose value is 7
but ih forms a cycle. So cd is considered.

 Now from d → de is added to the list. So the list
becomes de, fd, ah, bh, ig, fe, ih = {9, 14, 18, 11, 6, 10,
7}. Here ig and ih is already rejected. Now ah = 8, but
this forms a cycle. So it is rejected and de is considered.
 From e, the list becomes fe, fd, ah, bh, ig, ih = {10, 14, 8,
11, 6, 7}. Now here fe = 10, but it creates a cycle. So bh
= 11 is considered but it creates a cycle and then fd = 14
also creates a cycle. So fe, bh and fd are rejected.
a
b c d
e
f
i
h g
21
6
2
7
7
8
8
4
4
9
10
11 14

 So the required spanning tree is:
 So A = {ab, bc, ci, cf, cd, fg, gh, de}
V[G] = {a, b, c, i, f, d, g, h}
Total cost = 4 + 8 + 2 + 4 + 7 + 2 + 1 + 9 = 37.
a
b c d
e
f
i
h g
21
2
78
4
4
9

Single Source Shortest
Path (SSSP):
 Single source shortest path (SSSP) can be defined as the
shortest path weight from „u‟ to „v‟ by the expression. In a
shortest path problems, we are given a weighted directed
graph G = (V, E), with weight function w: E → R mapping
edges to real-valued weights.
 The weight of path P = (v0, v1, … vk) is the sum of weights
of its constituent edges: w(P) = 𝑤(𝑣𝑖 − 1,
𝑣𝑖)𝑘
𝑖=1 .
 We define the shortest path weight from u to v by:
δ(u, v) = min[w(P)]: u → vi, if there is a path from u to v
∞

 A shortest path from vertex u to v is then defined as any
path „p‟ with weight w(P) = δ(u, v). The single source
shortest path has 3 algorithms. That is:
i. Dijkstra‟s Algorithm
ii. Bellman Ford‟s Algorithm
iii. Topological Sorting using DAG
 Before considering the algorithm, we have know two other
algorithm, which is helpful for the above algorithm. That
is:
i. Algorithm for Initialize single source
ii. Algorithm for Relax

Algorithm for Initialize Single Source:
Initialise_single_source
1. for each vertex v є V[G]
2. do d[v] ← ∞
3. π[v] ← Nil
4. d[s] ← 0
 After initialization, π[v] = Nil, for all v є V, d[s] = 0and
d[v] = ∞ and for v є V – {s}. The relaxation step may
decrease the value of the shortest path estimate d[v] and
updated v‟s predecessor field π[v].
Algorithm for Relaxation:
Relax(u, v, w)
1. if d[v] > d[u] + w(u, v)
2. then d[v] ← d[u] + w(u, v)
3. π[v] ← u

 This algorithm use the technique of relaxation. For each
vertex v є V, we maintain an attribute d[v], which is an
upper bound weight of a shortest path from source „s‟ to
„v‟, d[v] is a shortest-path estimate.
Dijkstra’s Algorithm:
Dijkstra(G, w, s)
1. Initialise_single_source_graph(G, s)
2. s ← ϕ
3. Q ← V[G]
4. while(Q ≠ ϕ)
5. do u ← Extract_min(Q)
6. S ← S U {u}
7. for each vertex v є adj[u]
8. do Relax(u, v, w)

Example:
Ans: Iteration 1:
Q ≠ ϕ => Q = {s, t, x, y, z}
u → Extract_Min(Q) => u = {Q – s} => u = s
=> Q = Q – s = {t, x, y, z}, S = S U {u} => S = s.
For, each vertex v є adj[u] => adj[s] = t, y
Relax(u, v, w) = Relax(s, t, 10)
if d[v] > d[u] + w(u, v) => d[t] > d[s] + w(s, t)
=> ∞ > 0 + 10 => ∞ > 10
0
∞ ∞
∞ ∞
z
1
9
xt
y
s
2 3
5 7
10
64
2
Q *s t x y z
d[v] 0 ∞ ∞ ∞ ∞

then d[v] ← d[u] + w(u, v) => d[t] ← d[s] + w(s, t)
=> d[t] ← 0 + 10 => d[t] = 10
Again, Relax(s, y, 5)
d[y] > d[s] + w(s, y) => ∞ > 0 + 5 => ∞ > 5
then d[y] ← d[s] + w(s, y) => d[y] ← 0 + 5 => d[y] = 5.
Now, the iteration table is:
Now,
Q *s t x *y z
d[v] 0 10 ∞ 5 ∞
0
10 ∞
5 ∞
z
1
9
xt
y
s
2 3
5 7
10
64
2

Iteration 2:
Q = {t, x, y, z}
u → Extract_Min(Q) => u = {Q – y} = {t, x, z}
S = S U {u} => S = {s, y}
adj[u] = adj[y] = {t, z, x}
Relax(y, t, w)
d[t] > d[y] + w(y, t) => 10 > 5 + 3 => 10 > 8.
So d[t] ← d[y] + w(y, t) => d[t] ← 5 + 3 = 8 => d[t] = 8
Again, Relax(y, z, w)
d[z] > d[y] + w(y, z) => ∞ > 5 + 2 => ∞ > 7.
So d[z] ← d[y] + w(y, z) => d[z] ← 5 + 2 = 7 => d[z] = 7
Again, Relax(y, x, w)
d[x] > d[y] + w(y, x) => ∞ > 5 + 9 => ∞ > 14.
So d[x] ← d[y] + w(y, x) => d[z] ← 5 + 9 = 14 => d[z] = 14

So, the iteration table is:
Now,
Iteration 3:
Q = {t, z, x}
u → Extract_Min(Q) => u = {Q – z} = {t, x}
S = S U {u} => S = {s, y, z}
adj[u] = adj[z] = {s, x}
Q *s *y t *z x
d[v] 0 5 8 7 14
0
8 14
5 7
z
1
9
xt
y
s
2 3
5 7
10
64
2

Relax(z, s, w)
d[s] > d[z] + w(z, s) => 0 > 7 + 7 => 0 > 14 (false).
So d[t] ← d[y] + w(y, t) => d[t] ← 5 + 3 = 8 => d[t] = 8
Again, Relax(z, x, w)
d[x] > d[z] + w(z, x) => 14 > 7 + 6 => 14 > 13.
So d[x] ← d[z] + w(z, x) => d[x] ← 7 + 6 = 13 => d[x] = 13
So, the iteration table is:
Now,
Q *s *y *z *t x
d[v] 0 5 7 8 13
0
8 13
5 7
z
1
9
xt
y
s
2 3
5 7
10
64
2

Iteration 4:
Q = {t, x}, u → Extract_Min(Q) => u = t => Q = {x}
S = S U{u} => S = {s, y, z, t}, adj[t] = (y, x)
Relax(t, y, w) => d[y] > d[t] + w(t, y) => 5 > 8 + 2
=> 5 > 10 (false). So d[y] = 5
Again, Relax(t, x, w)
d[x] > d[t] + w(t, x) => 12 > 8 + 1 => 12 > 9. So d[x] = 9
So iteration table is:
Now,
Q s y z t x
d[v] 0 5 7 8 9
0
8 9
5 7
z
1
9
xt
y
s
2 3
5 7
10
64
2

Bellman_Ford’s Algorithm:
 The Bellman_Ford Algorithm solves the single source
shortest path in the general case, in which edge weights
may be negative.
 Given a weighted directed graph G = (V, E) with source
„s‟ and weight function „w‟.
 The Bellman_Ford whether or not there is a negative
weight cycle that is reachable from the source.
 If there is such a cycle, the algorithm indicates that no
solution exist. If there is not such cycle, the algorithm
produces the shortest paths and their weights.
 Here, in the given algorithm, the for loop of step – 2 to 4
consists of relaxing each edge of the graph once. Step 5 to
8 check for a negative weight cycle.

 The algorithm is given below:
Bellman_Ford(G, w, s)
1. Initialise_single_source_graph(G, s)
2. for i ← 1 to |V[G]| – 1
3. do for each edge (u, v) є E[G]
4. do Relax(u, v, w)
5. for each edge(u, v) є E[G]
6. do if d[v] > d[u] + w(u, v)
7. then return false
8. return true
Time Complexity of Bellman_Ford’s Algorithm:
 The Bellman_Ford runs in time O(V, E). Since the
initialization in step – 1 takes time θ(V), each of the |v| – 1
passes over the edges in step 2 to 4 take θ(E) time and the
for loop of step 5 to 7 take O(E) times.

Example:
t
0
∞ ∞
∞ ∞ z
5
-4
x
y
s
-38
7 2
6
7
-2
9
(i) t
0
6 ∞
7 ∞ z
5
-4
x
y
s
-38
7 2
6
7
-2
9
(ii)
t
0
6 4
7 2 z
5
-4
x
y
s
-38
7 2
6
7
-2
9
(iii) t
0
6 4
7 2 z
5
-4
x
y
s
-38
7 2
6
7
-2
9
(iv)
t
0
6 4
7 -2 z
5
-4
x
y
s
-38
7 2
6
7
-2
9
(v)

Topological Sorting using Directed Acyclic Graph
(DAG):
 Directed Acyclic Graph (DAG) is a directed graph, in
which there must be a vertex whose in-degree = 0
A B
C
d = 2
indeg = 0
outdeg = 2
d = 2
d = 2
indeg = 2
outdeg = 0
indeg = 1
outdeg = 1

Algorithm for Topological Sort:
Topological_Sort(G)
while V ≠ ϕ
Remove a vertex „u‟ of in-degree zero and all its
outgoing edges.
Insert „u‟ at the tail of the linked list of vertices
T1 = ϕ set
T2 =
A B
C
D F
E
G
H
K
A B C G D
HFE K x

Algorithm for Single Source Shortest Path for
DAG:
DAG_SSSP(G, s)
Topological_Sort(G)
Initialise_single_source_graph(G, s)
for each vertex „u‟ taken in topological sorted
order
do
for each vertex „v‟, adjacent to „u‟
do
Relax(u, v, w)

All Pair Shortest Path:
 It is based upon the dynamic programming techniques in
which the key part is to reduce a large program or problem
into smaller problems. That is: to solve the all pairs shortest
path problem on a directed graph G = (V, E) and it runs in
θ(V3) time.
 This can be done by limiting the number of edges of the
path, but it does not yield the faster method. Instead of
limiting the edges, this algorithm limits the set of vertices
through which the path is allowed to pass. The main key of
the algorithm is to determine the best method for the
shortest path problem.
 It can be observed that, if a path consists of single edge
then there will be no intermediate vertices, but for a path
P = (v1, v2, … vk), we can say that there exists v1, v2, ……,
vk – 1intermediate vertices.

Method:
 Let‟s consider for the graph „G‟, the vertices set V = {1, 2,
… n} and its subset is {1, 2, … k} of vertices for some
constant „k‟. For any pair of vertices i, j є V.
 Consider all paths from „i‟ to „j‟ whose intermediate
vertices are all drawn from {1, 2, … k}. The shortest path
from „i‟ to „j‟ is denoted as: Di, j(k).
 Let P = Minimum weight path from „i‟ to „j‟ which either
consists of single edge (i, j) or consists of intermediate
vertices such that there vertices are selected from the set
{1, 2, … k} and the path is free to visit any subset of
vertices set in any order.
 This algorithm exploits a relationship between path „P‟ and
shortest paths from „i‟ to „j‟ with all intermediate vertices in
the set {1, 2, … k – 1}. The relationship depends on
whether or not „k‟ is an intermediate vertex of path „P‟.

 If „k‟ is not an intermediate vertex of path „P‟ then all the
intermediate vertices of path „P‟ are in the set {1, 2, …
k – 1}. So a shortest path from vertex „i‟ to vertex „j‟ with
all intermediate vertices in the set {1, 2, …., k – 1} is
also a shortest path from „i‟ to „j‟ with all intermediate
vertices in the set {1, 2, …, k} and the length of the
shortest path is Di, j(k).
 If „k‟ is an intermediate vertex of path „P‟ then we break
„P‟ into P1 and P2 and P1 for „i‟ to „k‟ and P2 for „k‟ to „j‟.
The path P1 and P2 from „i‟ to „k‟ and „k‟ to „j‟ must be
the shortest path will all intermediate vertices in the set
{1, 2, … k – 1}. So the length of the path is:
Di, j(k – 1) + Dk, j(k – 1)

 So, the main equation or recursive definition of Floyd-
Warshall‟s Algorithm is:
 Di, j(k) = i) wi, j, if k = 0
ii) min(Di, j
(k – 1), Di, k
(k – 1) + Dk, j
(k – 1), if k ≥ 1
where wi, j = i) 0, if i = j
ii) w(i, j), if i ≠ j and exist a direct path
from „i‟ to „j‟ (i → j)
iii) ∞, if i ≠ j and no existence of a path
i → j
 It has a π – table and the π – table is also called predecessor
matrix, which constructed from „D‟ matrix. It can be
implemented to run in O(n3) time. Given the predecessor
matrix π, the print_all_pair_shortest_path procedure can be
used to print the vertices on a given shortest path.

 Here we compute a sequence of matrices π(0), π(1), π(2), …
π(n), where π = π(n), and πi, j
(k) is defined to be the
predecessor of the vertices „j‟ on a shortest path from vertex
„i‟, with all intermediate vertices in the set {1, 2, … k}.
 A recursive formulation of πi, j
(k) is given as:
 For k = 0, a shortest path for „i‟ to „j‟ has no intermediate
vertices at all. So,
πi, j
(k) = i) Nil, if i = j or wi, j = ∞
ii) i, if i ≠ j or wi, j < ∞
◦ For k ≥ 1, if we take path „i‟ to „k‟ and „k‟ to „j‟ where
k ≠ j, then the predecessor of „j‟ on a shortest path from
„k‟ with all intermediate vertices in the set {1, 2, … k}.
We choose the same predecessor of „j‟ that we choose on a
shortest path from „i‟, with all intermediate vertices in the
set {1, 2, … k – 1}. So,

πi, j
(k) = i) πi, j
(k – 1) , if Di, j
(k – 1) ≤ Di, k
(k – 1) + Dk, j
(k – 1)
ii) πk, j
(k – 1) , if Di, j
(k – 1) > Di, k
(k – 1) + Dk, j
(k – 1)
Example:
1
2
3
5 4
3 4
8
6
-4
1
7 2
-5
D0 =
1 2 3 4 5
1 0 3 8 ∞ -4
2 ∞ 0 ∞ 1 7
3 ∞ 4 0 ∞ ∞
4 2 ∞ -5 0 ∞
5 ∞ ∞ ∞ 6 0

π0 =
1 2 3 4 5
1 N 1 1 N 1
2 N N N 2 2
3 N 3 N N N
4 4 N 4 N N
5 N N N 5 N
D1 =
1 2 3 4 5
1 0 3 8 ∞ -4
2 ∞ 0 ∞ 1 7
3 ∞ 4 0 ∞ ∞
4 2 5 -5 0 -2
5 ∞ ∞ ∞ 6 0
π1 =
1 2 3 4 5
1 N 1 1 N 1
2 N N N 2 2
3 N 3 N N N
4 4 1 4 N 1
5 N N N 5 N

D2 =
1 2 3 4 5
1 0 3 8 4 -4
2 ∞ 0 ∞ 1 7
3 ∞ 4 0 ∞ ∞
4 2 5 -5 0 -2
5 ∞ ∞ ∞ 6 0
π2 =
1 2 3 4 5
1 N 1 1 2 1
2 N N N 2 2
3 N 3 N N N
4 4 1 4 N 1
5 N N N 5 N
D3 =
1 2 3 4 5
1 0 3 8 4 -4
2 ∞ 0 ∞ 1 7
3 ∞ 4 0 5 11
4 2 -1 -5 0 -2
5 ∞ ∞ ∞ 6 0
π3 =
1 2 3 4 5
1 N 1 1 2 1
2 N N N 2 2
3 N 3 N 2 2
4 4 3 4 N 1
5 N N N 5 N

D4 =
1 2 3 4 5
1 0 3 -1 4 -4
2 3 0 -4 1 -1
3 7 4 0 5 3
4 2 -1 -5 0 -2
5 8 5 1 6 0
π4 =
1 2 3 4 5
1 N 1 1 2 1
2 N N N 2 1
3 4 3 N 2 1
4 4 3 4 N 1
5 4 3 4 5 N
D5 =
1 2 3 4 5
1 0 1 3 2 -4
2 3 0 -4 1 -1
3 7 4 0 5 3
4 2 -1 -5 0 -2
5 8 5 1 6 0
π5 =
1 2 3 4 5
1 N 3 4 5 1
2 4 N 4 2 1
3 4 3 N 2 1
4 4 3 4 N 1
5 4 3 4 5 N

Knapsack Problem:
 Knapsack problem is of following types. That are:
i. 0 – 1 knapsack problem
ii. Fractional knapsack problem
iii. Backtracking knapsack problem
iv. Branch and bound knapsack problem.
1. 0 – 1 knapsack problem:
 If there are „n‟ items, ith item is worth „vi‟ rupees and
weight „wi‟, where „vi‟ and „wi‟ are integers. If we want to
take valuable a lead as possible but we can carry at most
„w‟ kg.
 This is must either be taken or left behind and we can
take fractional among of an item or take one item more
than ones.

2. Fractional knapsack problem:
 In fractional knapsack problem, the set up is some but we
take the fractional item rather than having to make a
binary (0, 1) choice for each item.
 E.g.: 0 – 1
 In fractional, i) vi/wi = 60/10 = 6, ii) 100/20 = 5 and
iii) 120/30 = 4
 Here we take 10 kg from 1st, 20 kg from 2nd and from 3rd
out of 30 we take 20 kg.
10 kg 20 kg 30 kg
Rs. 60/- Rs. 100/- Rs. 120/-
20
: 50 kg cost 220/-
30
20 60
20 100
10 80
240/-

2. Algorithm for Fractional knapsack problem:
1. Sort vi/wi in non-increasing order and let the sorted
value be v1/w1 = v2/w2 = … = vi/wi.
2. for i ← 1 to n
3. do xi ← 0
4. weight ← 0
5. i ← 1
6. while(weight + wi ≤ w)
7. xi ← 1
8. weight ← weight + wi
9. i ← i + 1
10. x ← w – weight/wi
11. return wi

 E.g.: x ← (x1, x2, … xn). Here xi = frequency of ith item
and „xi‟ lies between 0 and 1. i.e.: 0 ≤ xi ≤ 1. vi = value of
ith item , wi = weight of ith item and w = total capacity of
knapsack.
Running time for Fractional knapsack:
 Here, step – 1 takes θ(nlgn).
 The for loop i.e.: step – 2 and step – 3 takes θ(n) time.
 Step – 4 and step – 5 takes θ(1) time.
 Step – 6 to step – 9 takes constant time.
 So the time complexity of the fractional knapsack is:
max(θ(nlgn) + 2θ(n) + θ(1)) takes θ(nlgn).

Back Tracking:
 The backtracking algorithm is applicable to the wide range
of problems and it is a simple method. It is mainly used for
NP – complete problems, because not all the times, it
provides the efficient solution for the problem.
 The key point for back tracking algorithm is binary choice,
which means „yes‟ or „no‟.
 Whenever the back tracking have choice „no‟, that means
the algorithm has encountered a dead end and it back tracks
one step and tries a different path for „yes‟.
 The back tracking resembles a depth first search tree in a
directed graph, where graph is either a tree or at least it
does not have any either a tree or at least it does not have
any cycles. The graph exists only implicitly whatever be its
structure.

 The solution for the problems according to back tracking
can be represented as implicit graph on which back tracking
performs an “intelligent” depth first search, so as to provide
one or all possible solution to the given problem.
 Back tracking concept includes the following problems.
That is:
 Back tracking Knapsack Problem
 Hamiltonian Circuit Problem
 Subset-sum Problem
 Eight Queen Problem
1. Back Tracking Knapsack Problem:
 Here, we have to pack the knapsack in such a manner
than the total weight of items should not be greater than
the knapsack weight „W‟ and it should yield the
maximum value.

 Here, instead of considering the number of items, we
consider that we have „n‟ types of items, and that proper
number of items of each type is available.
 Let item of type Ti = 1, 2, … n, wi and vi indicate its
weight and value respectively. In this problem, we are not
allowed to take a fractional of item. The algorithm for the
knapsack problem according to back tracking is given
below.
Algorithm for Back Tracking Knapsack Problem:
Function_Back_Knapsack(T, w)
The above function computes the maximum value of
the item of types „T‟, in such a manner that the total
weight of the items manner that the total weight of the
items does not exceed the knapsack capacity „W‟.
Step 1: Initialization
Set b ← 0

Step 2: Check the item, loop
for i ← 1 to n
if (wi ≤ W) then
{
set b ← max(b, vi ← back_knapsack(i, W – wi))
}
Step 3: Return at the point of call
return b.
Example:
Given 3 types of items with following weights and values.
T = <T1, T2, T3>, w = <2, 3, 4>, v = <3, 4, 5> and
knapsack capacity W = 5.
Applying Backtracking technique we get the implicit tree
for the problem.

Initially,
We have the following node structure:
T w v
W = 5
T1 2 3
T2 3 4
T3 4 5
; , 0, 0
; , 0, 0
2; 3, 2 3; 4, 3 4; 5, 4
2, 3; 7, 5 2, 4; 8, 6
Dead end Dead end
Backtrack
Backtrack
Backtrack

 Here, left hand side of semicolon corresponds to the
weight chosen and the first entry to the right hand side of
the semicolon corresponds to the total value and the
second entry corresponds to the total value and the
second entry corresponds to the total weight with respect
to left side of semicolon.
 Such a node structure corresponds to a partial solution.
Initially our partial solution is empty. At every move
down to the children we select the type of the item to be
added in the knapsack. For more generality, we consider
the weights in increasing order, so that the search for the
tree is decreased.
 For example, if we once visit node (2, 3; 7, 5) then next
time we do not visit node (3, 2; 7, 5). The first node
visited is (2; 3, 2) the next is (2, 3; 7, 5). It can be seen
that as each new node is visited the partial solution is also
extended.

 After visiting these two nodes the dead end comes as
node (2, 3; 7, 5) has no unvisited successors, since
adding more items to this partial solution violates the
knapsack capacity constraint.
 This partial solution produces the optimum solution so
far, thus we memorize it. Next, we backtrack one step
and find that new addition (2, 4; 8, 6) will also violate the
capacity constraint.
 In the same manner, we backtrack one step and proceed
the search and it will continue until we get the optimum
solution.
 Exploring the whole tree in this way find node (2, 3; 7, 5)
to be optimum as no further improvement is noticed
before the search terminate. This is the optimal solution
for our example, having maximum value = 7.

2. Hamiltonian Circuit Problem:
 Given a graph G = (V, E), we have to find the
Hamiltonian circuit. Using Backtracking approach, we
start our search from any arbitrary vertex, say „a‟. This
vertex „a‟ becomes the root of our implicit tree.
 The first element of our partial solution is the first
intermediate vertex of the Hamiltonian cycle, that is to be
constructed.
 The next adjacent vertex is selected on the basis of
alphabetical or numerical order. If at any stage an
arbitrary vertex say „x‟ makes a cycle with any vertex
other than vertex „a‟, then we say that dead end is
reached.
 In this case, we backtrack one step and again the search
begins by selecting another vertex.

 It should be noted that after backtracking the element
from the partial solution must be removed. The search
using backtracking is successful if a Hamiltonian cycle is
obtained.
Example:
 Consider a graph G = (V, E), where we have to find a
Hamiltonian circuit using backtracking method.
 Initially, we start our search with vertex „a‟, this vertex
„a‟ becomes the root of our implicit tree. i.e.: a <= root
a
b
ec
d

 Next, we choose vertex „b‟ adjacent to „a‟, as it comes
first in lexicographical order (b, c, d)
 Next vertex „c‟ is selected which is adjacent to „b‟ and
which comes first in lexicographical order (c, e).
 Next we select vertex „d‟ adjacent to „c‟ which comes
first in lexicographical order (d, e)
b
a
dc
<= root
b
a
dc
<= root
ec

 Next vertex selected that is adjacent to „d‟ is vertex „e‟. If
choose vertex „a‟, then we do not get the Hamiltonian
cycle.
b
a
dc
<= root
ec
ed
b
a
dc
<= root
ec
ed
e

 The vertex adjacent to „e‟ are b, c, d, but they have
already visited. Thus, we get the dead end. So, we
backtrack one step and remove the vertex „e‟ from our
partial solution.
 The vertex adjacent to „d‟ are e, c and a from which
vertex „e‟ has already been checked and we are left with
vertex „a‟, but by choosing this vertex we do not get the
Hamiltonian cycle. So, again we backtrack one step and
we select the vertex „e‟ adjacent to „c‟.
b
a
dc
<= root
ec
ed
e Dead end
Backtrack

 The vertex adjacent to „e‟ are (b, c, d). So vertex „d‟ is
selected.
b
a
dc
<= root
ec
ed
e Dead end
Backtrack
Backtrack
b
a
dc
<= root
ec
ed
e
Dead end
Backtrack
Backtrack
d

 The vertex adjacent to „d‟ is (a, c, e). So, vertex „a‟ is
selected. Here we get the Hamiltonian cycle as all the
vertex other than the start vertex „a‟ is visited only once
that is: a – b – c – e – d – a.
 The final implicit tree for the Hamiltonian circuit is given
as below.
b
a
dc
<= root
ec
ed
e
Dead end
Backtrack
Backtrack
d
a Solution

 It is noticeable that for a complete graph, there can be
1/2(n – 1)! Hamiltonian circuits.
3. Subset – Sum Problem:
 In a subset-sum problem, we have to find a subset‟s of
given set S = <S1, S2, … Sn> where the elements of set
„S‟ and „n‟ positive integers in such a manner that S' є S
and sum of the elements of subset „S'‟ is equal to some
positive integer „X‟.
b
a
dc
<= root
ec
ed
e
Dead end
Backtrack
Backtrack
d
a Solution
1
0
2
5
4
3
6

 For instance, if given a set S<1, 2, 3, 4> and X = 5, then
there exists a set S' = <3, 2> or S' = <1, 4> whose sum is
equal to X. It can also be noted that some instance of the
problem does not have any solution.
 For instance, if given a set S = <1, 3, 5> and X = 7, then
no subset occurs for which the sum is equal to X.
 The subset – sum problem can be solved by using the
backtracking approach. In this implicit tree is created,
which is a binary tree.
 The root of the tree is selected in such a way that it
represents that no decision is yet taken on any input. We
assume that the elements of the given set are arranged in
an increasing order : S1 ≤ S2 ≤ S3 … Sn.
 The left child of the root node indicates that we have to
include „S1‟ from set „S‟ and right child of the root node
indicates that, we have to exclude „S1‟.

 Preceding to next level, starting from root‟s left child
indicates inclusion of „S2‟ and right child indicated
exclusion of „S2‟.
 Each node stores the sum of the partial solution elements.
If at any stage the number equal to „X‟, then the search is
successful and terminates.
 The dead end in the tree occurs only when either of the
two inequalities exist. That are:
i. The sum of S' is too large: S' + Si + 1 > x
ii. The sum of S' is too small: S' + Sj
n
j−i+1 < x
 Thus, we backtrack one step and continue the search. If
our partial solution elements sum is equal to the positive
integer „X‟ then at that time search will terminate, or it
continues if all the possible solution need to be obtained.

Example:
 Given, a set S = <1, 3, 4, 5> and X = 8. We have to find
subset-sum using backtracking approach. Initially,
S = <1, 3, 4, 5> and X = 8 and S' = <ϕ>. The implicit
binary tree for subset-sum problem is:
Implicit Binary Tree for Subset sum Problem
IN - 1 EX - 1
EX - 3
1
0
0
4 1 3 0
8 4 5 1 7 3
8 3
IN - 3 IN - 3
IN - 4IN - 4 IN - 4
IN - 5Soln 4 + 5 > 8
(DE)
EX - 3
EX - 5
EX - 4EX - 4EX - 4
5 + 5 > 8
(DE)
7 + 5 > 8
(DE)
Soln

 Here the number inside a node is the sum of the partial
solution elements at a particular level (elements of a
subset).
4. Eight Queens Problem:
 The problem is to place n-queens in such a manner on an
n – by – n chess board that no two queen s attach each
other by being in the same row, column or diagonal.
 It can be seen that for n = 1, the problem has a trivial
solution and no solution exists for n = 2 and n = 3. So,
first we will consider the 4 queens problem and then
generalize it to 8-queens problem and an n-queens
problem.
 Given a 4 – by – 4 chessboard, now let us number the
rows and columns of the chessboard 1 through 4.

 Since, we have to place 4 queens on a chessboard, such
that no two queens attach each other, we number these as
q1, q2, q3 and q4.
 It can be observed that for such a condition, each queen
must be placed on a different row, so we place queen „i‟
on row „i‟ without any loss of generality.
1 2 3 4
1
2
3
4
Row
Column
[4 – by – 4 chessboard]

 Initially, we have an empty chessboard, now first we
place queen „q1‟, in the very first acceptable position,
which is row1 and column1 (1, 1).
 Next, we place queen „q2‟ and search for its best position,
so that both these queens do not attach each other. Thus,
we find that if we place „q2‟ in column 1 and 2 then the
dead end is encountered.
 Thus the first acceptable position for q2 is (2, 3). But later
this acceptable position proves to be dead end; as no
position is left for placing queen „q3‟ safely. So we
backtrack one step and place the queen „q2‟ in (2, 4) next
best possible solution.
 We obtain the position for placing „q3‟, which is (3, 2).
But later this position also leads to dead end as no place
is found where „q4‟ can be placed safely.

 Then we have to backtrack till „q1‟ and place it to (1, 2), the
next possible position for it.
 After this, all other queens are placed safely by moving q2
to (2, 4), q3 to (3, 1) and q4 to (4, 3) in such a manner that
no two queens can attach each other by being placed in the
same row, column or diagonal.
 Each node describes its partial solution.
 Finally, we get the solutions <2, 4, 1, 3> that means the one
possible solution for 4 – queen problems is to place q1 in
column 2, q2 is column 4, q3 in column 1 and q4 in column
3.
 The implicit tree for 4 queens problem solution <2, 4, 1, 3>,
the numbers above the node specify in which order nodes
are produced.

0
1
q1
q2
q3
q4
5
q1
x x
2 3
q1
q2
q3
x x
q1
x x x
q1
q2
q3
x x x x
q1
q2
x x x
q1
q2
x x xx
4
q1
q2
x xx
6
7
8
Dead
end
Backtrack
Backtrack
Backtrack
Backtrack
Backtrack
Backtrack
Dead
end
Dead
end
Solution

 The other solution is:
 It can be seen that all the solutions to the 4 queens
problem can be represented as 4-tuples (t1, t2, t3, t4),
where „ti‟ represents the column on which queen „qi‟ is
placed.
 The explicit constraints for this are: Si = <1, 2, 3, 4>
where 1 ≤ i ≤ 4.
q1
q2
q4
q3
1 2 3 4
1
2
3
4
Row
Column
4 – by – 4 chessboard solution:
<3, 1, 4, 2>

 The implicit constraint is that no queen can be placed in
same column thus no two ti‟s can be same and no two
queens can be placed on the same diagonal.
 The permutation tree so obtained by considering the
constraints which signifies 4! Leaf nodes. The edge
labels indicate the possible value of „ti‟ and moving down
from level „i‟ to level „i + 1‟edges are labeled with „ti‟.
 The diagram of the implicit tree is given as follows. The
n-queens problem, we have an n-by-n chess board where
we have to place n-queens in such a manner that no two
queens can attack each other by being on the same row,
column or diagonal.
 So the solution space for this consists of n! permutation
of n-tuple (t1 … tn), which shows queen „qi‟ placed
where.

 Now, we can formulate solution to 8-queens problem,
which need 8-tuples for its representation (t1 – t8), where
ti represents the column on which queen „qi‟ is placed.
 The solution space consists of all 8! permutations. One
possible solution for 8 queens problem. That is:
Solution for 8 – queens problem:
<4, 6, 8, 2, 7, 1, 3, 5>
Row
Column
1
2
3
4
q1
q4
q2
q3
q6
q7
q5
q8
1 2 3 4 5 6 7 8
5
6
7
8

Branch and Bound:
 The branch and bound technique like back tracking
explores the implicit graph. (A graph can be acyclic or
tree) and deals with the optimal solution to a given
problem.
 In this technique, at each stage we calculate the bound will
be able to give the solution or not that means we calculate
how far we are from the solution in the graph.
 If we find that at any node the solution so obtained is
appropriate but the remaining solution is leading to a
worse case, then we leave this part of the graph without
exploring.
 It can be seen that optimal solution is the feasible solution
in an implicit graph, where we get the best value for the
objective function.

 Depth-first search or Breadth-first search is used for
calculating the bound. By these calculations we are able to
select to which path we have to go and explore. For Branch
and Bound technique, we have two additional requirements.
That are:
1. Lower Bound or Upper Bound:
 At every stage, we have for each node a bound – lower
bound for minimization problem and upper bound for
maximization problem.
 This bound is on the possible value of the objective
function for obtaining a solution by adding components
to partial solution.
2. Previous Checking:
 For each node, bound is calculated by the means of
partial solution.

 The calculated bound for the node is checked with
previous best result and if found that new partial
solution result leads to worse case, then bound with the
best solution so far, is selected and we leave this part
without exploring it further.
 Otherwise the checking is done with the previous best
result obtained so far, every partial solution while
exploring.
1. Assignment Problem:
 The first problem we are dealing with is the assignment
problem in which „n‟ people are assigned „n‟ tasks.
 We have to find the maximum total cost of assignment
where each person has exactly one task to perform. For
solving this problem, we are n-by-n cost matrix „C‟.

 Thus, we can state the problem as given cost matrix
„C‟, where Ci, j refers to the minimum cost per person
„Pi‟ to perform task „tj‟, 1 ≤ i ≤ n and 1 ≤ j ≤ n, then the
problem is to assign person to tasks so as to minimize
the total cost of performing the „n‟ tasks.
 In other words, we have to choose the element from the
matrix, in such a manner that no two elements are in
the same column and the total cost should be minimum.
Example:
 Given cost matrix for the three persons P1, P2, P3 which
are assigned 3 tasks.
T1 T2 T3
5 8 4
3 7 2
4 10 5
P1
P2
P3
C =

 If „P1‟ is assigned task T3, P2 is assigned task T1 and P3 is
assigned task T2, then our total cost will be 4 + 3 + 10 = 17
and if, we assign T2 to P1, T1 to P2 and T3 to P3, then the
total cost will be 8 + 3 + 5 = 16.
 But it can seen that the optimal assignment for the above
matrix is tasks T2 to P1, T1 to P2 and T3 to P3 which will
give the total cost 8 + 2 + 4 = 14.
 It is noticeable that, the cost of any solutions, including the
optimal one, can not be smaller than the sum of the
minimum elements present in each row of the cost matrix.
For above matrix, the sum should not be less than 4 + 2 + 4
= 10. This sum becomes the lower bound for our problem.
 In best first branch and bound, the node currently having
the best possible solution is selected and explored.

 Instead of exploring a single child for the last
promising node as we did in backtracking, we explore
all the possible children for the node having current
best solution result among non-terminated leaves in the
current trees.
 We select the node for exploring by comparing the
lower bounds. The node with the minimum lower
bound is selected and explored.
 We have calculated the lowest bound, denoted by „lb‟,
which is 10. This lower bound corresponds to root. We
start from the root which signifies that no assignment is
yet done.
 The structure of the node is follows node is divided
into two parts. The first part gives the information
about the assignments and the second part signifies
possible lower bound for instance.

 In the above node „P1‟ is assigned task „T2‟ and
possible lower bound is 14.
 The first level of the tree corresponds to 3 tasks. Let us
assume that these tasks are assigned to person „P1‟.
Then we calculate possible lower bound. So, we select
the node where task „T3‟ is assigned to „P1‟.
 We know that its children can not have lower bound
less than 11. Now, we explore this node and can see
that „P2‟ can be assigned to „T1‟ or „T2‟.
 It can be seen that lower bounds of these nodes exceed,
the lower bound of node labeled „1‟. So we explore it
and calculate the optimal lower bound.
P1 → T1 lb = 14

 After examining each of the leaves of the trees (which
can give the best result). We obtain the node labeled „g‟
as be having the optimal solution, „P1‟ is assigned „T2‟,
„P2‟ is assigned „T3‟ and „P3‟ is assigned „T2‟, thus total
cost is 8 + 2 + 4 = 14.
 It can be observed that initially node P1 → T3 seems to
be most promising but the optimal solution does not
come from this node. The nodes marked as „*‟ are unable
to give the optimal solution.

2. Knapsack Problem:
 By using branch and bound technique, we have a
bound that none of the items can have total sum more
than knapsack capacity „W‟ and must give the
maximum possible value.
 The implicit tree for this problem is constructed as a
binary tree, where left branch specifies the inclusion of
the item and right branch signifies exclusion.
 Let us assume „xi‟ be 1, if we select the item, otherwise
its value is „0‟. Thus we require to maximize.
xivi
𝑛
𝑖=1 subject to xiwi
𝑛
𝑖=1 ≤ W
 The node structure is as follows:
i. Node is divided into 3 parts, the first part indicates
the total weight of item, the second part indicates
the value of the current item, and the third part
indicates the upper bound for the node.

ii. The upper bound of the node can be computed as:
Ub = v + (W – w)(vi + 1/wi + 1)
Example:
 Given cost matrix for the three persons P1, P2, P3 which
are assigned 3 tasks.
 Also, given a knapsack of capacity W = 3, we have to pack
this knapsack using branch and bound technique.
w v
Ub
#ITEMS w v
I1 1 2
I2 2 3
I3 3 4

 So, as to given the maximum possible value while
considering all constraints. First we calculate value per
weight ratio and arrange the table according to its
descending values.
 Now, we start with the root node, the upper bound for
root node can be computed as: Ub = 0 + (3 – 0) * 2 = 6
where v = 0, w = 0, W = 3 and v1/w1 = 2.
(root node)
 We include item 1, which is indicated by the left
branch and exclude 1, which is indicated by right
branch.
Items wi vi vi/wi
I1 1 2 2
I2 2 3 1.5
I3 3 4 1.3
w = 0 v = 0
Ub = 6

 At every node, we compute the upper bound and explore
the node while selecting the item. Finally, the node with
maximum upper bound is selected as an optimum
solution.
 Here node with item 1 and 2 gives the optimum solution.
That is maximum value of 5 of 5 to given knapsack
problem. The number above the nodes indicates the order
in which the node are generated.

Daa notes 2

More Related Content

What's hot (20)

Similar to Daa notes 2 (20)

More from smruti sarangi (7)

Recently uploaded (20)

Daa notes 2