Applied Algorithms and Structures week999

CSC 421: Applied
Algorithms and Structures
Week 6
“Programming isn't about what you know; it's about what you can figure out.” – Chris Pine
“An algorithm must be seen to be believed.” - Donald Knuth

Fibonacci pseudocode
Naïve recursive version (Θ 2𝑛 )
Fibo(n)
if n = 0 then return 1
return Fibo(n-1)+Fibo(n-2)
DP smart recursive version (Θ 𝑛 )
Fibo(n, f)
if f[n] exists then return f[n]
if n = 0 or n = 1 then
f[n] = 1
return 1
sum = Fibo(n-1) + Fibo(n-2)
f[n] = sum
return sum

Fibonacci pseudocode
DP iterative version (Θ 𝑛 )
Fibo(n)
f[0] = f[1] = 1
for i = 2 to n
f[i] = f[i-1] + f[i-2]
return f[n]

Dynamic programming
Calculating Fibonacci numbers
• The usual recurrence:
𝐹𝑖𝑏𝑜 𝑛 =
1 𝑖𝑓 𝑛 = 0 ∨ 𝑛 = 1
𝐹𝑖𝑏𝑜 𝑛 − 1 + 𝐹𝑖𝑏𝑜(𝑛 − 2) 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
• Algorithm that calculates using this recurrence
• Runtime recurrence: 𝑇 𝑛 = 𝑇 𝑛 − 1 + 𝑇 𝑛 − 2 + 1
• Memoization (table of previously calculated values)
• Best way: Only keep track of last two values

Fibonacci (memoized) call tree

Catalan numbers
• How many different strings of 𝑛 pairs of balanced parentheses?
• Recurrence formula:
𝐶 𝑛 =
1 𝑖𝑓 𝑛 = 0
𝑖=1
𝑛
𝐶𝑖−1𝐶𝑛−𝑖 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
• 𝐶 0 = 1
• 𝐶 1 = 𝐶 0 𝐶 0 = 1
• 𝐶 2 = 𝐶 0 𝐶 1 + 𝐶 1 𝐶 0 = 2
• 𝐶 3 = 𝐶 0 𝐶 2 + 𝐶 1 𝐶 1 + 𝐶 2 𝐶 0 = 5
• 𝐶 4 = 𝐶 0 𝐶 3 + 𝐶 1 𝐶 2 + 𝐶 2 𝐶 1 + 𝐶 3 𝐶 0 = 14
• 𝐶 5 = 𝐶 0 𝐶 4 + 𝐶 1 𝐶 3 + 𝐶 2 𝐶 2 + 𝐶 3 𝐶 1 + 𝐶 4 𝐶 0 = 42
• First few values of the sequence: 1, 1, 2, 5, 14, 42, 132, ….

Catalan numbers
• Naïve pseudocode:
CATALAN(n)
if n = 0 then return 1
sum = 0
for i = 1 to n
sum += CATALAN(i-1) * CATALAN(n-i)
return sum
• Top-down approach
• Bottom-up approach using memoization

Catalan numbers
• Dynamic programming pseudocode:
Catalan(n)
table[0] = 1
for i = 1 to n+1
sum = 0
for j = 0 to i-1
sum += table[j]*table[i-j-1]
table[i] = sum
return table[n]

Catalan numbers
• Other formulas:
𝐶 𝑛 =
1
𝑛 + 1
2𝑛
𝑛
=
2𝑛 !
𝑛 + 1 ! 𝑛!
𝐶 𝑛 =
2 2𝑛 − 1
𝑛 + 1
𝐶(𝑛 − 1)

Chessboard traversal
Given an 𝑛 × 𝑛 chessboard 𝑝 where each square has a profit associated with it. The
problem is to find the path from the top row to the bottom row that maximizes the
total profit. A move from a square may only go to a square in the next row down that
it touches, that is, one down and one to the left, one straight down, or one down and
one to the right.
See example on the whiteboard.

This is a maximization optimization problem. You are given an 𝑛 × 𝑛 table 𝑝 of profits.
Here is top down recursive definition of maximal profit:
𝑞(𝑖, 𝑗) =
0 𝑗 < 1 or 𝑗 > 𝑛
𝑝[𝑖, 𝑗] if 𝑖 = 1
𝑝 𝑖, 𝑗 + max{𝑞(𝑖 − 1, 𝑗 − 1), 𝑞(𝑖 − 1, 𝑗), 𝑞(𝑖 − 1, 𝑗 + 1)} otherwise
q(i,j)
if j < 1 or j > n then return 0
else if i = 1 then return p[i,j]
else return p[i,j] + max(q(i-1,j-1), q(i-1,j), q(i-1,j+1))
This algorithm would take Θ(2𝑛
) time. Show this with a recursion tree.

Instead of beginning by trying to compute the maximal profit path of length 𝑛, we will
compute maximal profits for paths of length 1, then paths of length 2, and so forth.
We’ll use a 2-dimensional table called 𝑞.
The value of 𝑞[𝑖, 𝑗] is the maximum profit one can earn for every path that ends at
square 𝑖, 𝑗.
Represent the computation of 𝑞 with an 𝑛 × 𝑛 table. Fill the table row-by-row. Begin
by filling row 1 with the values from the profit table (𝑝).

Developing a dynamic programming
solution
a) Formulate the problem recursively.
a) Specification.
b) Solution.
b) Build solutions to your recurrence from the bottom up.
a) Identify the sub-problems.
b) Choose a memoization data structure.
c) Identify dependencies.
d) Find a good evaluation order.
e) Analyze space and running time.
f) Write down the algorithm.

Edit distance
• This is a minimization optimization problem.
• Note the representation showing both strings and the edit operations, for example:
𝐴 𝐿 𝐺 𝑂 𝑅 𝐼 𝑇 𝐻 𝑀
𝐴 𝐿 𝑇 𝑅 𝑈 𝐼 𝑆 𝑇 𝐼 𝐶
𝑑 𝑠 𝑖 𝑖 𝑠 𝑠
• Look at the steps in creating a dynamic programming solution to a problem.
• Specify those for the edit distance problem.

Rod cutting problem
Given a rod of length 𝑛 and a price chart for segments of length up to 𝑛. Determine
which cut maximizes the total price.
Given a rod of length 4. What is the optimal set of cuts?

Rod cutting problem
We can simplify the optimization by realizing that we need to know where the leftmost
cut will be in an optimal cutting of the rod. Given that, we get the following recurrence
relation for calculating 𝑟[𝑘], the maximum profit obtainable from a rod of length 𝑘:
𝑟 𝑘 =
0 if 𝑘 = 0
max
1≤𝑖≤𝑘
{𝑝 𝑖 + 𝑟 𝑘 − 𝑖 } otherwise
The index 𝑖 in the maximum calculation is the location of the leftmost cut.
Calculating this recurrence from the top-down is done by the pseudocode on the next
slide.

Analysis of that solution
The recurrence relation is:
𝑇 𝑛 = 1 +
𝑗=0
𝑛−1
𝑇(𝑗)
This is Θ(2𝑛
).
Need a better way. Let’s solve the problem bottom-up.

Analysis of that solution
Two nested loops:
𝑗=1
𝑛
𝑖=1
𝑗
1 =
𝑗=1
𝑛
𝑗 =
𝑛 𝑛 + 1
2
= Θ(𝑛2)

How does the bottom-up approach work?
The array element 𝑝[𝑖] holds the price for a segment of length 𝑖. The array element
𝑟[𝑖] holds the optimal price for a rod of length 𝑖.
Compute the optimal solution for a rod of length 1.
Compute the optimal solution for a rod of length 2.
…
Computing the optimal solution for a rod of length 𝑗 means trying each of the possible
𝑗 − 1 cuts. We think of each cut as being the first one (leftmost).

Longest common subsequence
In the longest-common-subsequence problem, we are given two sequences:
𝑋 = 〈𝑥1, 𝑥2, … , 𝑥𝑚〉, 𝑌 = 〈𝑦1, 𝑦2, … , 𝑦𝑛〉
We wish to find a maximum-length common subsequence 𝑍 of 𝑋 and 𝑌.

LCS: definition of subsequence
Given a sequence
𝑋 = 〈 𝑥1, 𝑥2, … , 𝑥𝑚〉
Another sequence
𝑍 = 〈𝑧1, 𝑧2, … , 𝑧𝑘〉
is a subsequence of 𝑋 if there exists a strictly increasing sequence 〈𝑖1, 𝑖2, … , 𝑖𝑘〉 of
indices of 𝑋 such that for all 𝑗 = 1, 2, … , 𝑘, we have 𝑥𝑖𝑗
= 𝑧𝑗.
Notation: 𝑋𝑗 is the first 𝑗 characters of sequence 𝑋.
Also, assume that 𝑋 has 𝑚 characters, 𝑌 has 𝑛 characters, and 𝑍 has 𝑘 characters.

LCS: optimal substructure
Given sequences 𝑋 and 𝑌 and longest common subsequence 𝑍:
• If 𝑥𝑚 = 𝑦𝑛, then 𝑧𝑘 = 𝑥𝑚 = 𝑦𝑛 and 𝑍𝑘−1 is an LCS of 𝑋𝑚−1 and 𝑌𝑛−1.
• If 𝑥𝑚 ≠ 𝑦𝑛, then 𝑧𝑘 ≠ 𝑥𝑚 implies that 𝑍 is an LCS of 𝑋𝑚−1 and 𝑌.
• If 𝑥𝑚 ≠ 𝑦𝑛, then 𝑧𝑘 ≠ 𝑦𝑛 implies that 𝑍 is an LCS of 𝑋 and 𝑌𝑛−1.
The LCS problem has optimal substructure.

LCS: recursive solution
We can use the optimal substructure formulas to construct a recurrence relation for
𝑐[𝑖, 𝑗], which is the length of the LCS for the pair 𝑋𝑖 and 𝑌
𝑗:
𝑐 𝑖, 𝑗 =
0 𝑖𝑓 𝑖 = 0 𝑜𝑟 𝑗 = 0
𝑐 𝑖 − 1, 𝑗 − 1 + 1 𝑖𝑓 𝑖, 𝑗 > 0 𝑎𝑛𝑑 𝑥𝑖 = 𝑦𝑗
max{𝑐 𝑖, 𝑗 − 1 , 𝑐 𝑖 − 1, 𝑗 } 𝑖𝑓 𝑖, 𝑗 > 0 𝑎𝑛𝑑 𝑥𝑖 ≠ 𝑦𝑗
There are many possible sub-problems but the formula only considers a few.

LCS: computing using DP
Note that the 𝑐 values may be placed into a table of size 𝑚 × 𝑛.
Index the rows from the top to bottom 0 to 𝑚, index the columns from left to right 0 to
𝑛. Note that computing 𝑐[𝑖, 𝑗] requires table values to its left, above it, or left and
above.
So fill the table row-by-row starting at row 0.
On the board: An example with the sequences 𝑋 = 〈𝐴, 𝐵, 𝐶, 𝐵, 𝐷, 𝐴, 𝐵〉 and 𝑌 =
〈𝐵, 𝐷, 𝐶, 𝐴, 𝐵, 𝐴〉.

LCS algorithm: running time
Nested loops, the outside one executing 𝑚 iterations, the inside one executing 𝑛
iterations: Θ(𝑚𝑛).

Assembly line traversal
There are two assembly lines
We want to minimize the cost of going from the entry point to the exit point. That is,
we want to minimize the total of the costs associated with each station and each
transition from line to the other.

With 𝑛 stations, there are 2𝑛 possible paths (once again, the number of 𝑛-bit binary
strings comes in handy).
Here’s a recurrence that defines an optimal solution for a station on the first line:
𝑓1 𝑗 =
𝑒1 + 𝑎1,1 𝑖𝑓 𝑗 = 1
min{𝑓1 𝑗 − 1 + 𝑎1,𝑗, 𝑓2 𝑗 − 1 + 𝑡2,𝑗−1 + 𝑎1,𝑗} 𝑖𝑓 𝑗 > 1
The first term in the min expression is when the optimal path comes to station 𝑆1,𝑗
from station 𝑆1,𝑗−1; the second term when it comes from station 𝑆2,𝑗−1.

Treating the recurrence relation as a way to define values in a pair of tables, fill each
table bottom-up, that is, starting at the table entry at position 1, rather than filling it by
starting at position 𝑛.
Let 𝑓1 1 = 𝑒1 + 𝑎1,1 and 𝑓2 1 = 𝑒2 + 𝑎2,1.
Have a for-loop indexing 𝑗 from 2 to 𝑛. Inside the for-loop compute 𝑓1[𝑗] and 𝑓2[𝑗].
From the recurrence, each of these relies only values in the table to the left, that is,
values that have already been computed.

34
Here’s an example with actual values:

Applied Algorithms and Structures week999

Recommended

More Related Content

Similar to Applied Algorithms and Structures week999 (20)

Recently uploaded (20)

Applied Algorithms and Structures week999