DCIT 204

Data Structures and

Algorithms 1

Sessions 9 and 10 - Dynamic Programming

Course Writer: Dr Kofi Sarpong Adu-Manu

Contact Information: [email protected]

College of Education
School of Continuing and Distance Education
2020/2021 – 2022/2023
 Dynamic Programming
Dynamic Programming is a general algorithm design technique
for solving problems defined by recurrences with overlapping
subproblems

• Invented by American mathematician Richard Bellman in the

1950s to solve optimization problems and later assimilated by CS

• “Programming” here means “planning”

• Main idea:
- set up a recurrence relating a solution to a larger instance to
solutions of some smaller instances
- solve smaller instances once
- record solutions in a table
- extract solution to the initial instance from that table
 Example 1: Fibonacci numbers
• Recall definition of Fibonacci numbers:

F(n) = F(n-1) + F(n-2)

F(0) = 0
F(1) = 1

• Computing the nth Fibonacci number recursively (top-

down):

F(n)

F(n-1) + F(n-2)

F(n-2) + F(n-3) F(n-3) + F(n-4)

 Example 1: Fibonacci numbers (cont.)

Computing the nth Fibonacci number using bottom-up iteration and

recording results:
F(0) = 0
F(1) = 1
F(2) = 1+0 = 1
…
F(n-2) =
F(n-1) =
F(n) = F(n-1) + F(n-2)
0 1 1 . . . F(n-2) F(n-1) F(n)
Efficiency:
- time
- space
 Example 2: Coin-row problem
There is a row of n coins whose values are some positive integers c₁,
c₂,...,cn, not necessarily distinct. The goal is to pick up the maximum
amount of money subject to the constraint that no two coins adjacent
in the initial row can be picked up.

E.g.: 5, 1, 2, 10, 6, 2. What is the best selection?

 DP solution to the coin-row problem
Let F(n) be the maximum amount that can be picked up from the
row of n coins. To derive a recurrence for F(n), we partition all
the allowed coin selections into two groups:

those without last coin – the max amount is ?

those with the last coin -- the max amount is ?

Thus we have the following recurrence

F(n) = max{cn + F(n-2), F(n-1)} for n > 1,

F(0) = 0, F(1)=c₁
DP solution to the coin-row problem (cont.)
F(n) = max{cn + F(n-2), F(n-1)} for n > 1,

F(0) = 0, F(1)=c₁
index 0 1 2 3 4 5 6
coins -- 5 1 2 10 6 2
F( )

Max amount:
Coins of optimal solution:
Time efficiency:
Space efficiency:
Note: All smaller instances were solved.
 Example 3: Path counting
1
Consider the problem of
A
counting the number of shortest
paths from point A to point B in
a city with perfectly horizontal
streets and vertical avenues

B
 Example 4: Coin-collecting by robot
Several coins are placed in cells of an n×m board. A robot, located in
the upper left cell of the board, needs to collect as many of the coins as
possible and bring them to the bottom right cell. On each step, the
robot can move either one cell to the right or one cell down from its
current location. 1 2 3 4 5 6

5
 Solution to the coin-collecting problem
Let F(i,j) be the largest number of coins the robot can collect and
bring to cell (i,j) in the ith row and jth column.

The largest number of coins that can be brought to cell (i,j):

from the left neighbor ?

from the neighbor above?

The recurrence:
F(i, j) = max{F(i-1, j), F(i, j-1)} + cij for 1 ≤ i ≤ n, 1 ≤ j ≤ m
where cij = 1 if there is a coin in cell (i,j), and cij = 0 otherwise

F(0, j) = 0 for 1 ≤ j ≤ m and F(i, 0) = 0 for 1 ≤ i ≤ n.

Solution to the coin-collecting problem (cont.)
F(i, j) = max{F(i-1, j), F(i, j-1)} + cij for 1 ≤ i ≤ n, 1 ≤ j ≤ m
where cij = 1 if there is a coin in cell (i,j), and cij = 0 otherwise
F(0, j) = 0 for 1 ≤ j ≤ m and F(i, 0) = 0 for 1 ≤ i ≤ n.
1 2 3 4 5 6

5
 Other examples of DP algorithms
• Computing a binomial coefficient (# 9, Exercises 8.1)

• General case of the change making problem (Sec. 8.1)

• Some difficult discrete optimization problems:

- knapsack (Sec. 8.2)
- traveling salesman

• Constructing an optimal binary search tree (Sec. 8.3)

• Warshall’s algorithm for transitive closure (Sec. 8.4)

• Floyd’s algorithm for all-pairs shortest paths (Sec. 8.4)
 Knapsack Problem by DP
Given n items of
integer weights: w1 w2 … wn
values: v1 v2 … vn
a knapsack of integer capacity W
find most valuable subset of the items that fit into the knapsack

Consider instance defined by first i items and capacity j (j  W).

Let V[i,j] be optimal value of such instance. Then
max {V[i-1,j], vi + V[i-1,j- wi]} if j- wi  0
V[i,j] =
V[i-1,j] if j- wi < 0
Initial conditions: V[0,j] = 0 and V[i,0] = 0
 Knapsack Problem by DP (example)
Example: Knapsack of capacity W = 5
item weight value
1 2 $12
2 1 $10
3 3 $20
4 2 $15 capacity j
0 1 2 3 4 5
0
w1 = 2, v1= 12 1
w2 = 1, v2= 10 2
w3 = 3, v3= 20 3
w4 = 2, v4= 15 4 ?
 Optimal Binary Search Trees
Problem: Given n keys a1 < …< an and probabilities p1 ≤ … ≤ pn
searching for them, find a BST with a minimum
average number of comparisons in successful search.
Since total number of BSTs with n nodes is given by
C(2n,n)/(n+1), which grows exponentially, brute force is hopeless.

Example: What is an optimal BST for keys A, B, C, and D with

search probabilities 0.1, 0.2, 0.4, and 0.3, respectively?
 DP for Optimal BST Problem
Let C[i,j] be minimum average number of comparisons made in
T[i,j], optimal BST for keys ai < …< aj , where 1 ≤ i ≤ j ≤ n.
Consider optimal BST among all BSTs with some ak (i ≤ k ≤ j )
as their root; T[i,j] is the best among them.
ak C[i,j] =
min {pk · 1 +
i≤k≤j

k-1
∑ ps (level as in T[i,k-1] +1) +
Optimal Optimal s=i
BST for BST for
a i , ..., ak-1 a k+1 , ..., aj
j
∑ ps (level as in T[k+1,j] +1)}
s =k+1
 DP for Optimal BST Problem (cont.)
After simplifications, we obtain the recurrence for C[i,j]:
j
C[i,j] = min {C[i,k-1] + C[k+1,j]} + ∑ ps for 1 ≤ i ≤ j ≤ n
i≤k≤j s=i
C[i,i] = pi for 1 ≤ i ≤ j ≤ n
 Example: key A B C D
probability 0.1 0.2 0.4 0.3

The tables below are filled diagonal by diagonal: the left one is filled
using the recurrence j
C[i,j] = min {C[i,k-1] + C[k+1,j]} + ∑ ps , C[i,i] = pi ;
i≤k≤j s=i

the right one, for trees’ roots, records k’s values giving the minima
j 0 1 2 3 4 j 0 1 2 3 4
i i
1 0 .1 .4 1.1 1.7 1 1 2 3 3 C

2 0 .2 .8 1.4 2 2 3 3 B D

3 0 .4 1.0 3 3 3
A
4 0 .3 4 4
optimal BST
5 0 5
Optimal Binary Search Trees
Analysis DP for Optimal BST Problem
Time efficiency: Θ(n3) but can be reduced to Θ(n2) by taking
advantage of monotonicity of entries in the
root table, i.e., R[i,j] is always in the range
between R[i,j-1] and R[i+1,j]
Space efficiency: Θ(n2)

Method can be expended to include unsuccessful searches

Warshall’s Algorithm: Transitive Closure
• Computes the transitive closure of a relation

• Alternatively: existence of all nontrivial paths in a digraph

• Example of transitive closure:

3 3
1 1

2 4 0 0 1 0 2 4 0 0 1 0
1 0 0 1 1 1 1 1
0 0 0 0 0 0 0 0
0 1 0 0 1 1 1 1
 Warshall’s Algorithm
Constructs transitive closure T as the last matrix in the sequence
of n-by-n matrices R(0), … , R(k), … , R(n) where
R(k)[i,j] = 1 iff there is nontrivial path from i to j with only first k
vertices allowed as intermediate
Note that R(0) = A (adjacency matrix), R(n) = T (transitive closure)
3 3 3 3 3
1 1 1 1 1

4 4 4 2 4 4
2 2 2 2

R(0) R(1) R(2) R(3) R(4)

0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 0
1 0 0 1 1 0 1 1 1 0 1 1 1 0 1 1 1 1 1 1
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 1 0 0 0 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1
 Warshall’s Algorithm (recurrence)
On the k-th iteration, the algorithm determines for every pair of
vertices i, j if a path exists from i and j with just vertices 1,…,k
allowed as intermediate

{ R(k-1)[i,j]
R(k)[i,j] = or
(path using just 1 ,…,k-1)

R(k-1)[i,k] and R(k-1)[k,j] (path from i to k

and from k to i
k using just 1 ,…,k-1)
i

j
 Warshall’s Algorithm (matrix generation)

Recurrence relating elements R(k) to elements of R(k-1) is:

R(k)[i,j] = R(k-1)[i,j] or (R(k-1)[i,k] and R(k-1)[k,j])

It implies the following rules for generating R(k) from R(k-1):

Rule 1 If an element in row i and column j is 1 in R(k-1),

it remains 1 in R(k)

Rule 2 If an element in row i and column j is 0 in R(k-1),

it has to be changed to 1 in R(k) if and only if
the element in its row i and column k and the element
in its column j and row k are both 1’s in R(k-1)
 Warshall’s Algorithm (example)
3
1 0 0 1 0 0 0 1 0
1 0 0 1 1 0 1 1
R(0) = 0 0 0 0 R(1) = 0 0 0 0
2 4
0 1 0 0 0 1 0 0

0 0 1 0 0 0 1 0 0 0 1 0
1 0 1 1 1 0 1 1 1 1 1 1
R(2) = 0 0 0 0 R(3) = 0 0 0 0 R(4) = 0 0 0 0
1 1 1 1 1 1 1 1 1 1 1 1
 Warshall’s Algorithm (pseudocode and analysis)

Time efficiency: Θ(n3)

Space efficiency: Matrices can be written over their predecessors
Floyd’s Algorithm: All pairs shortest paths
Problem: In a weighted (di)graph, find shortest paths between
every pair of vertices

Same idea: construct solution through series of matrices D(0), …,

D (n) using increasing subsets of the vertices allowed
as intermediate

Example: 4 3
1
1
6
1 5

2 4
3
 Floyd’s Algorithm (matrix generation)
On the k-th iteration, the algorithm determines shortest paths
between every pair of vertices i, j that use only vertices among 1,
…,k as intermediate

D(k)[i,j] = min {D(k-1)[i,j], D(k-1)[i,k] + D(k-1)[k,j]}

D(k-1)[i,k]
k
i
D(k-1)[k,j]
D(k-1)[i,j]
j
 Floyd’s Algorithm (example)

1
2 2 0 ∞ 3 ∞ 0 ∞ 3 ∞
3 6 7 2 0 ∞ ∞ 2 0 5 ∞
D(0) = ∞ 7 0 1 D(1) = ∞ 7 0 1
3
1
4 6 ∞ ∞ 0 6 ∞ 9 0

0 ∞ 3 ∞ 0 10 3 4 0 10 3 4
2 0 5 ∞ 2 0 5 6 2 0 5 6
D(2) = 9 7 0 1 D(3) = 9 7 0 1 D(4) = 7 7 0 1
6 ∞ 9 0 6 16 9 0 6 16 9 0
 Floyd’s Algorithm (pseudocode and analysis)

Time efficiency: Θ(n3)

Space efficiency: Matrices can be written over their predecessors
Note: Shortest paths themselves can be found, too (Problem 10)
 Reference
Levitin, A. (2012). Introduction to the Design and
Analysis of Algorithms ( 3rd Edition). Harlow: Addison
Wesley.
 Acknowledgement

Pearson Education, Inc. Upper Saddle River, NJ.

All Rights Reserved.