MINOR PROJECT MS-64

ALGORITHM DESIGN AND ANALYSIS

G.G.D.S.D. COLLEGE
2024-2026

Optimal Binary Search Tree

Submitted To: Submitted By:

Dr. Himani Mittal Shruti Arora
M.Sc.IT (I)
Roll No. 2441751
MINOR PROJECT MS-64 // SHRUTI ARORA

02
ACKNOWLEDGEMENT

I would like to express my sincere gratitude to everyone who has supported me

throughout the completion of this project. First and foremost, I would like to
thank my Professor, Dr. Himani Mittal, for their guidance, valuable insights,
constant encouragement. Their expertise and constructive feedback were
instrumental in shaping the direction of this work.

I am also grateful to my family and friends for their unwavering support,

patience, and understanding during this time. Their encouragement and belief
in me kept me motivated and focused.

Lastly, I would like to acknowledge the resources and facilities provided by

GGDSD College, which helped make this project possible.

Thanks to everyone who contributed to the successful completion of this

project.
MINOR PROJECT MS-64 // SHRUTI ARORA

03
CONTENTS

Sr.No. Topic Page No. Remarks

1 Introduction 4-5

2 Example 6

3 Algorithm 7-8

4 Program Code 9-10

5 Code Snippets 11-12

6 References 13
 MINOR PROJECT MS-64 // SHRUTI ARORA

04
INTRODUCTION

An Optimal Binary Search Tree (OBST), also known as a Weighted Binary Search Tree, is a binary
search tree that minimizes the expected search cost.
It is a classic problem in computer science that belongs to the domain of Dynamic Programming
and deals with efficient searching.

How is OBST different from Binary Search Tree?

A Binary Search Tree (BST) is a fundamental data structure used to store sorted data, allowing for
efficient search, insertion, and deletion operations. However, in many practical applications,
different keys may be searched with varying frequencies. In scenarios where keys are searched
with different frequencies, a regular binary search tree (BST) may not yield the most efficient
results. On the other hand, an OBST is a special type of binary search tree that minimizes the
expected cost of searches by considering the frequency of access to each key. It is a binary search
tree that provides the shortest possible search time or expected search time.

Problem Definition:
Given:
A set of n distinct keys K = {k1, k2, ..., kn} where the keys are sorted.
An array of probabilities P = {p1, p2, ..., pn}, where pi represents the probability of
searching for key ki.
Optionally, we may also be given a set of dummy keys, which correspond to unsuccessful
searches between the actual keys.

Objective: The goal is to construct a binary search tree that minimizes the expected search
cost, where the cost of searching a key is proportional to its depth in the tree.

Approach: Dynamic Programming

To solve the OBST problem optimally, we use dynamic programming by breaking it down into
subproblems and solving them efficiently using a bottom-up approach.
Steps to Solve the OBST Problem:
1. Define the Cost Function:
Let cost(i, j) represent the minimum cost of constructing an optimal BST for the keys ki to
kj.
For each key kr (where i <= r <= j), if kr is chosen as the root, the cost of the tree rooted at
kr is: cost(i,j)=cost(i,r−1)+cost(r+1,j)+sum(i,j)
where sum(i, j) is the sum of the probabilities of all keys from ki to kj.
 MINOR PROJECT MS-64 // SHRUTI ARORA

05
2. Recursive Structure:
The recursive relation captures the optimal cost for a given range of keys by trying all
possible roots within the range. Each subproblem is solved optimally, and the results are
used to build larger solutions.
The base case occurs when i > j, which represents an empty subtree, and its cost is zero.
3. Compute Optimal Solutions for Subproblems:
For each possible subtree, compute the cost for all ranges of keys i to j and store the
results in a table.
Start by solving smaller subproblems (with only one or two keys), then move on to larger
subproblems until the entire range 1 to n is solved.
4. Reconstruct the Optimal Tree:
The dynamic programming table not only gives the minimal search cost but also helps in
reconstructing the structure of the OBST by keeping track of the roots chosen for each
subtree.

Complexity:
Time Complexity: The time complexity of the dynamic programming approach for OBST is
O(n^3) because we compute the cost for all possible subtrees of the keys (a total of O(n^2)
subproblems), and for each subproblem, we evaluate all possible roots (O(n)).
Space Complexity: The space complexity is O(n^2) since we store the computed cost of each
subtree in a table.
 MINOR PROJECT MS-64 // SHRUTI ARORA

06
EXAMPLE

Example: Let n=4 and (a1, a2, a3, a4) = (do, if, int, while). In sorted order as this is a BST.

Let p[1:4] = (3, 3, 1, 1) and q[0:4] = (2, 3, 1, 1, 1)

i=
0 1 2 3 4
distance

w00 =2 w11 =3 w22 =1 w33=1 w44 =1

0 c00=0 c11=0 c22=0 c33=0 c44=0
r00=0 r11=0 r22=0 r33=0 r44=0

w12 = P2 + Q2 + w11= 3+1+2 =7 w23 = P3 + Q3 + w22 = w34= P4 + Q4 + w33 =

w01 = P1 + Q1 + w00= 3+3+2 =8
c01={k=2, c11+c22+w12 1+1+1 =3 1+1+1 =3
c01={k=1, c00+c11+w01
1 =0+0+8=8} =0+0+7=7} c23={k=3, c22+c33+w23 c34={k=4, c33+c44+w34
r12=2 =0+0+3=3} =0+0+3=3}
r01=1
r23=3 r34=4

w02 = P2 + Q2 + w01= 3+1+8 =12 w13 = P3 + Q3 + w12= 1+1+7 =12 w24 = P4 + Q4 + w23
c02= min c13= min = 1+1+3 =5
{k=1, c00+c12+w02 = 19 {k=2, c11+c23+w13 = 12 c24= min
2 k=2, c01+c22+w02 = 20} k=3, c12+c33+w13 = 16} {k=3, c22+c34+w24 = 8
r02=1 r13=2 k=3, c23+c44+w24 = 8}
both are same so take k=3
r24=3

w14 = P4 + Q4 + w13 Formulas used:

w03 = P3 + Q3 + w02= 1+1+12 =14
c03= min = 1+1+9=11 w[i,j] = Pj + Qj + w[i,j-1]
{k=1, c00+c13+w03 = 26 c14= min c[i,j] = min { c[i,k-1] + c[k,j] } + w[i,j]
{k=2, c11+c24+w14 = 19
3 k=2, c01+c23+w03 = 25 i<k j
k=3, c02+c33+w03 = 33} k=3, c12+c34+w14 = 21
r03=2 k=4, c13+c44+w14 = 23}
r14=2

The rank r04 is the root of the calculated optimal tree. It has
node 1 (r02) as the left child and node 3 (r24) as the right child.
w04 = P4 + Q4 + w03= 1+1+14 =16
Node 3 has right child node 4 (r34) has shown below.
c04= min
{k=1, c00+c14+w04 = 35 if
4 k=2, c01+c24+w04 = 32
k=3, c02+c34+w04 = 38
k=4, c03+c44+wo4= 41}
do int
r03=2

while
 MINOR PROJECT MS-64 // SHRUTI ARORA

07
ALGORITHM

Algorithm OBST(p, q, n)
// Given n distinct identifiers a1 < a2 < ... < an and probabilities
// p[i], 1 <= i <= n, and q[i], 0 <= i <= n, this algorithm computes
// the cost c[i, j] of optimal binary search trees tij for identifiers
// ai, ..., aj. It also computes r[i, j], the root of tij.
// w[i, j] is the weight of tij.

{
// Step 1: Initialize matrices w, c, and r for empty and single-node subtrees
for i := 0 to n do
{
w[i, i] := q[i];
r[i, i] := 0;
c[i, i] := 0.0;

if i < n then
{
// Initialize single-node subtrees
w[i, i+1] := q[i] + q[i+1] + p[i+1];
r[i, i+1] := i + 1;
c[i, i+1] := w[i, i+1];
}
}

// Initialize the last element for completeness

w[n, n] := q[n];
r[n, n] := 0;
c[n, n] := 0.0;

// Step 2: Build optimal trees for subtrees of increasing length m

for m := 2 to n do
for i := 0 to n - m do
{
j := i + m;
w[i, j] := w[i, j-1] + p[j] + q[j];
 MINOR PROJECT MS-64 // SHRUTI ARORA

08
ALGORITHM

// Step 3: Use Knuth's optimization to find the optimal root in the range [r[i, j-1], r[i+1, j]]
k := Find(c, r, i, j); // Find the optimal root using restricted range
c[i, j] := w[i, j] + c[i, k-1] + c[k, j];
r[i, j] := k;
}

// Step 4: Output the minimum cost and root matrices

write (c[0, n], w[0, n], r[0, n]);
}

Algorithm Find(c, r, i, j)
// Finds the root k in the range [r[i, j-1], r[i+1, j]] that minimizes the cost c[i, j]

{
min := ∞;
l := r[i, j-1]; // Start from the lower bound of the range
for m := r[i, j-1] to r[i+1, j] do
if (c[i, m-1] + c[m, j] < min) then
{
min := c[i, m-1] + c[m, j];
l := m;
}
return l; // Return the root that minimizes the cost
}
 MINOR PROJECT MS-64 // SHRUTI ARORA

09
CODE

#include <stdio.h>
#include <limits.h> // For INT_MAX

// Define maximum number of nodes (keys)

#define MAX 100

void optimal_bst(float p[], float q[], int n, float c[MAX][MAX], float w[MAX][MAX],
int r[MAX][MAX]) {
// Step 1: Initialize matrices w, c, and r for empty and single-node subtrees
for (int i = 0; i <= n; i++) {
w[i][i] = q[i];
c[i][i] = 0.0;
r[i][i] = 0;

if (i < n) {
// Initialize single-node subtrees
w[i][i + 1] = q[i] + q[i + 1] + p[i + 1];
c[i][i + 1] = w[i][i + 1];
r[i][i + 1] = i + 1;
}
}

// Initialize the last element for completeness

w[n][n] = q[n];
c[n][n] = 0.0;
r[n][n] = 0;

// Step 2: Build optimal trees for subtrees of increasing length m

for (int m = 2; m <= n; m++) {
for (int i = 0; i <= n - m; i++) {
int j = i + m;
w[i][j] = w[i][j - 1] + p[j] + q[j];

// Step 3: Use Knuth's optimization to find the optimal root in the range
[r[i, j-1], r[i+1, j]]
int k = find(c, r, i, j); // Find the optimal root using restricted range
c[i][j] = w[i][j] + c[i][k - 1] + c[k][j];
r[i][j] = k;
}
}
 MINOR PROJECT MS-64 // SHRUTI ARORA

10
CODE

// Step 4: Output the minimum cost

printf("Minimum cost of the Optimal Binary Search Tree is: %f\n", c[0][n]);
printf("\nCost matrix (c):\n");
for (int i = 0; i <= n; i++) {
for (int j = 0; j <= n; j++) {
printf("%8.2f ", c[i][j]);
}
printf("\n");
}

printf("\nWeight matrix (w):\n");

for (int i = 0; i <= n; i++) {
for (int j = 0; j <= n; j++) {
printf("%8.2f ", w[i][j]);
}
printf("\n");
}

printf("\nRoot matrix (r):\n");

for (int i = 0; i <= n; i++) {
for (int j = 0; j <= n; j++) {
printf("%8d ", r[i][j]);
}
printf("\n");
}
}

// Find function to determine the root that minimizes cost

int find(float c[MAX][MAX], int r[MAX][MAX], int i, int j) {
float min = INT_MAX;
int optimal_root = r[i][j - 1]; // Start from the lower bound of the range

// Loop from r[i][j-1] to r[i+1][j] to find the optimal root

for (int m = r[i][j - 1]; m <= r[i + 1][j]; m++) {
float temp_cost = c[i][m - 1] + c[m][j];
if (temp_cost < min) {
min = temp_cost;
optimal_root = m;
}
}

return optimal_root;
}
 MINOR PROJECT MS-64 // SHRUTI ARORA

11
CODE

int main() {
int n = 4; // Number of keys
float p[] = {0, 0.15, 0.10, 0.05, 0.10}; // Probabilities for successful searches
(p[1] to p[n])
float q[] = {0.05, 0.10, 0.05, 0.05, 0.05}; // Probabilities for unsuccessful
searches (q[0] to q[n])

float c[MAX][MAX] = {0}; // Cost matrix

float w[MAX][MAX] = {0}; // Weight matrix
int r[MAX][MAX] = {0}; // Root matrix

// Calculate the Optimal BST

optimal_bst(p, q, n, c, w, r);

return 0;
}
MINOR PROJECT MS-64 // SHRUTI ARORA

12
SNIPPETS
MINOR PROJECT MS-64 // SHRUTI ARORA

13
SNIPPETS
MINOR PROJECT MS-64 // SHRUTI ARORA

14
REFERENCES

www.geeksforgeeks.org
www.javapoint.com
Algorithm Design and Analysis Simplified - Dr. Himani Mittal
Fundamentals of Computer Algorithms - Sartaj Sahani
ChatGpt
www.google.com