CS-622 : Advanced Analysis of

Algorithms

Binary Search Tree

Week # 5
Saima Gul
Sets and Dynamic Sets
 Sets are fundamental to many operations and problems in
computer science.
 The kind of sets we are used to from mathematics courses are
static sets.
 However, the sets that we need in algorithmic problems are
usually dynamic sets.
 In other words, most of the algorithms require to insert new
elements to and remove elements from the sets.
 Hence, the sets we will be handling will change over time.
 Furthermore, we will usually be using multisets.

2
Typical operations of dynamic sets

 Membership test: Search(S,x)

 Adding a new element: Insert(S,x)
 Removing an element: Delete(S,x)
 Finding the smallest element: Minimum(S)
 Finding the greatest element: Maximum(S)
 Finding the smallest number greater than a given number:
Successor(S,x)
 Finding the greatest number smaller than a given number:
Predecessor(S,x)

3
Data structures for representing
dynamic sets
 Several data structures can be used for representing
dynamic sets (queues, linked lists, hash tables, rooted
trees, etc.)
 Depending on the data structure, the implementation of
the basic operations of dynamic sets may have different
asymptotic running times.
 It is important to be aware of various data types for
dynamic sets, to be able to make an informed decision to
pick a suitable data structure for a given application
area.

4
Binary Search Trees

 A Binary Search Tree (BST) is a data structure for

representing dynamic sets.
 BST can support all the basic operations of dynamic sets
efficiently.
 I.e., using BST, we can implement the basic dynamic set
operations efficiently.
 For complete binary search trees, these operations will all
have Θ(lg n) time.
 However, for some binary search trees, we will see that
they can take upto linear time.

5
What is a Binary Search Tree (BST)?

 As the name suggests, it is organized in the form of a

binary tree.
 Therefore, each node will have at most two children.
 A node with no child is called a leaf node.
 All the nodes (except the root) will have exactly one
parent node.
 Each node of a BST is labeled by a data called the key.

6
 How to implement a BST?
implementation
of a node
parent implementation view

abstract view x1
left right
x1 6
6
key
x2 x5

x2 2 x5 8 2 8

x3 x4 x6

1 x3 x4 3 x6 9 1 3 9

red: keys of the nodes xi: name of the node key(xi): key value at node

7
Shorthand notations used for BST visualization

Sometimes, we will use a shorthand notation to denote the subtrees.

x1 6 x1 6

x2 2 x5 8

1 x3 x4 3 x6 9

left subtree of x1 right subtree of x1

8
 BST Property
 A BST is not just an arbitray binary tree.
 We require the so called BST property:

Example:
x k
6

  2 8

 nodes x' in  : key( x' )  key( x)

1 3 9
 nodes x' in  : key ( x)  key ( x' )

9
Repeated keys may exist
 The keys at nodes need not be distinct.

 We may have multiple nodes with the same key to

represent multisets.
Example:
6

1 6

1 3 9

10
Inorder walks over a BST

 Due to BST property, an inorder walk over a BST will

visit the keys in increasing order:

InOrderWalk (x) { // x is a pointer to a node in the tree

if (x != NULL) {
InOrderWalk(left[ x ]); // walk into left subtree
print key[ x ]; // visit the current node
InOrderWalk(right[ x ]); // walk into right subtree
}
}

11
Inorder walks over a BST
Example:
InOrderWalk (x) { 6
if (x != NULL) {
InOrderWalk(left[ x ]);
print key[ x ]; 2 8
InOrderWalk(right[ x ]);
}
}
1 3 9

running time: Θ(n)

where n is the number of keys, 1,2,3,6,8,9
or the number of nodes in BST

12
 Searching in a BST

 Let us consider our first dynamic set operation:

membership test…
 Given a BST whose root is pointed by x, and a number
k, does k appear as a key in one of the nodes in this
tree?
if k=key[x], then we found it

if k < key[x], then we should if k > key[x], then we should

search it in the left subtree   search it in the right subtree

13
Searching in a BST
BST_Search (x,k) {
if (x == NULL)
return NULL; // to denote that k is not found
if (k==key[ x ])
return x; // i.e. return the pointer to the node with key k
if (k < key[ x ]) // k cannot be in the right subtree
return BST_Search(left[ x ], k);
else // k cannot be in the left subtree
return BST_Search(right[ x ], k);
}

Running time: The worst case appears when the number is not in
BST, and the search proceeds along the longest path from the root to
a leaf.
Hence, worst case running time is O(h), where h is the height (the size
of the longest path from the root to a leaf) of the tree.

14
Recursion vs. Iteration
Although it does not affect the asymptotic run time behavior, in
practice, iterative form of an algorithm runs faster than its recursive
implementation.

A tail recursive algorithm can easily be converted into iterative form.

BST_Search_Iterative (x,k) {
while (x != NULL and k != key[ x ]) {
if (k < key[ x ]) // k cannot be in the right subtree
x = left[ x ]; // continue using the left subtree
else // k cannot be in the left subtree
x = right[ x ]; // continue using the right subtree
}
return x;
}

15
Minimum Operation

 Suppose we want to find the minimum key stored in a

given BST.
root We know that (it is guaranteed by
the BST property), k’ ≤ k.
k

However, BST property also

k’ k’’ guarantees that, any node in the
colored subtree, has a key which is
smaller than or equal to k’.

Can we say “The left most leaf

has the minimum key”?

16
Minimum Operation

 Actually, we cannot:

root

x
k Note that, x is not a leaf node…

However, k is the minimum key in the BST.

17
Minimum Operation
 Therefore, the minimum key stored in a BST can be
found by following the left child links (starting from the
root) as long as we can. The first node which does not
have a left child will have the minimum key in the BST.
BST_Minimum (x) {
while (left[ x ] != NULL) // if there exists a left child, its key is
x = left[ x ]; // smaller or equal to the current node
return x;
}

In the worst case, we can traverse the longest path in the BST.
running time: O(h)

18
Maximum Operation

 Suppose we want to find the maximum key stored in a

given BST.
root We know that (it is guaranteed by
the BST property), k ≤ k’’.
k

However, BST property also

k’ k’’ guarantees that, any node in the
red subtree, has a key which is
greater than or equal to k’’.

Can we say “The right most leaf

has the maximum key”?

19
Maximum Operation

 Actually, we cannot:

root

x
k Note that, x is not a leaf node…

However, k is the maximum key in the BST.

20
Maximum Operation
 Therefore, the maximum key stored in a BST can be
found by following the right child links (starting from the
root) as long as we can. The first node which does not
have a right child will have the maximum key in the BST.
BST_Maximum (x) {
while (right[ x ] != NULL) // if there exists a right child, its key is
x = right[ x ]; // greater or equal to the current node
return x;
}

In the worst case, we can traverse the longest path in the BST.
running time: O(h)

21
Successor Operation

 We will also be interested in the successor of a given

node in a BST.
 The successor of a node x is defined as the next node x’
that would follow x in the inorder traversal of the BST.
 Given a node x and its successor node y, key[ y ] is the
smallest key value in the BST that is greater than or
equal to key[ x ].

22
Successor Operation

 Consider a node x: What is the next node visited after x

in an inorder traversal?
When the inorder traversal procedure
x might be the left or reaches to x,
right child of its parent. - it will first visit all the nodes in the left
subtree rooted at xL.
or - then x will be visited,
x - then all the nodes in the right subtree
rooted at xR will be visited.
xL xR What is the first node that will be visited
in the right subtree?
BST_Minimum(xR)

23
Successor Operation

 Consider a node x which does not have a right child:

When the inorder traversal procedure

reaches to x,
x might be the left or
right child of its parent. - it will first visit all the nodes in the left
subtree rooted at xL.
or - then x will be visited,
x - then ???

xL What is the next node that will be visited?

Not immediately clear…

24
Successor Operation
 Suppose x is the left child of its parent:
y
x

xL Then, x’s parent will follow x.

 Suppose x is the right child of its parent:

y Then, x’s parent will precede x.
x

xL The same can be argued if y is the left or the right child of

its parent.

25
Successor Operation
 Successor of x will be the closest ancestor y of x for
which x is in the left subtree of y (if exists).
BST_Successor (x) {
if (right[ x ] != NULL) // simple case
y return (BST_Minimum(x));
// find the closest ancestor of x
// for which x is in the left subtree
y = parent[ x ]; t = x;
while (y != NULL and t = right[ y ]) {
t = y;
x
y = parent[ t ];
}
return y;
} running time: O(h)

26
Insert Operation

 Given a BST T, and a number k, how can we insert k into

T and form a new BST T’ which includes all the keys in T
and k?
 Insertion, unlike the previous operations we have seen on
BSTs, will change the structure of the BST.
 We need to guarantee that T’ also satisfies the BST
property.
 Deletion will similarly modify the BST.
 As we will see, we can preserve BST property easily for
insertion, but for deletion it will be a little bit harder.

27
Insert Operation

 In order to preserve BST property easily, and implement

Insert operation efficiently, we will always insert a new
node as a leaf node.

 All we have to do is to find an appropriate parent node x

(unless the tree is empty) in the given BST, such that
when the given new node is inserted as a child of x, the
BST property is preserved.

28
Insert Operation

 Simplest case: Inserting a new node into an empty BST.

into empty result
Insert 6 6
BST

 A little harder:

into result
Insert 8 6 6

29
Insert Operation
 General case:

into result
Insert 7 6 6

2 8 2 8

1 3 9 1 3 7 9

30
Insert Operation

 General case:

into result
Insert 3 6 6

2 8 2 8

1 3 7 9 1 3 7 9

31
Pseudo code for Insert Operation
BST_Insert (T, x) { // T: the tree, x: a new node to be inserted
if (root[ T ] == NULL)
root[ T ] = x; // inserting into empty BST
else {
y = root[ T ]; // y follows a path to find the correct place for x
z = NULL; // z will simply follow y (value of y in previous iteration)
while (y != NULL) { // as long as we can go deeper in the BST
z = y; // make y follow z, and go into the correct subtree
y = (key[ x ] < key[ y ]) ? left[ y ] : right[ y ];
}
parent[ x ] = z; // z keeps the correct parent for x
if (key[ x ] < key[ z ]) // decide whether x will be a left or right child
left[ z ] = x;
else
right[ z ] = x;
}
}
running time: O(h)

32
Sorting using BSTs

 The insert operation combined with an inorder traversal

of a BST can be used for sorting:
SortUsingBST (A) { // A: the array of numbers to be sorted
root[ T ] = NULL; // create an empty BST
-
for (i=1; i <= length[ A ]; i++) { // for each number
- A[ i ] -
newNode = create a new node with key A[ i ];
BST_Insert(T, newNode); // insert the key (number) into BST
}
InOrderWalk ( root[ T ] ); // will output the keys in sorted order
}

running time: O(nh)+O(n)=O(nh)

33
Delete Operation
 Delete operation will remove a given node from a BST.
 Since this operation will also change the BST, we need
to make sure that after the removal of the given node,
the remaining tree satisfies the BST property.
 Simplest case: Removing a leaf…
6

2 8

1 3 7 9

34
Pseudo code for leaf node deletion
BST_Delete_LeafNode (T, x) { // T: the tree, x: the leaf to be deleted
if (parent[ x ] == NULL) { // x is the only node in the tree
// the tree will become empty
root[ T ] = NULL;
}
else {
if (x == left[parent[ x ]]) // x is the left child of its parent
left[parent[ x ]] = NULL;
else // x is the right child of its parent
right[parent[ x ]] = NULL;
}
}

running time: Θ(1)

35
Delete Operation
 A little bit harder: Removing a node with a single child…
y y y y

x x x x

z z z z

All the keys at z and below z are All the keys at z and below z are
smaller than equal to key[ y ] greater than equal to key[ y ]

Since the BST will satisfy the BST property before the removal, we
can simply make the only child of x the substitute child of parent of x.

36
Delete Operation

 Example for removing a node with a single child…

6 6

2 8 2

1 3 9 1 3 9

37
Delete Operation

 What if that single child node to be deleted is the root

node?..
6
2

1 3

1 3

38
Pseudo code for single child node deletion
BST_Delete_SingleChildNode (T, x) {
// T: the tree, x: the node to be deleted and it has a single child
child = (left[ x ] != NULL) ? left[ x ] : right[ x ]; // the only child of x
if (parent[ x ] == NULL) { // x is the root
// make that single child the new root
root[ T ] = child;
}
else { // x is not the root
if (x == left[parent[ x ]]) // x is the left child of its parent
left[parent[ x ]] = child;
else // x is the right child of its parent
right[parent[ x ]] = child;
}
}

running time: Θ(1)

39
Delete Operation
 Most complex case: Removing a node with two children.
After removing x, there will be two subtrees dangling
y
around. So, we cannot play the trick we used while
removing a node with a single child.
x
In both cases on the left, a reconfiguration within the
subtrees will be sufficient.
 
Suppose we find the maximum key value in α (let’s
y say node z is the max node in α) and we assign
key[ x ] = key[ z ]

x Now, key[x] is deleted but there is one extra key[ z ]

in the tree.
  To fix this, remove the node z.

40
Delete Operation
 Most complex case: Removing a node with two children.

y Note that, BST_Maximum can be used to find the

maximum valued node z in α (in O(h) time).
x

  key[ x ] = key[ z ]  takes constant time

y
Removing the maximum node z takes constant time,
x since z is a node whose right child is missing (it has
at most one child – previous cases apply).

 
41
 Delete Operation
 Example for removing a node with two children.
node to be
deleted 10 10 10

6 18 5 18 5 18

2 7 29 2 7 29 2 7 29

5 5

4 z: max node 4 4
in the left
subtree

42
Pseudo code for complete Delete Operation
BST_Delete (T, x) { // T: the tree, x: the node to be deleted
if (left[ x ] == NULL and right[ x ] == NULL) // x is a leaf node
BST_Delete_LeafNode(T, x);
elseif (left[ x ] == NULL or right[ x ] == NULL) // a single child node
BST_Delete_SingleChildNode(T, x);
else { // x is a node with two children
// find max in the left subtree
z = BST_Maximum(left[ x ]); // needs O(h) time
key[ x ] = key[ z ]; // copy the max value
// no need to copy key[ x ] to max node as it
// will be immediately deleted
BST_Delete(T, z); // z : either leaf or has single child
}
} running time: O(h)

43
All dynamic operations run in O(h) time

 We have shown that all the dynamic set operations can be

implemented in O(h) time using BSTs.
 Recall that, running time of an algorithm must be
described as a function of a measure of the size of the
input.
 Is the heigth of a tree the best measure for its size?

2h-1 h
h elements
elements

 Seems not…

44
All dynamic operations run in O(h) time

 What is the relationship between h (the height of the

BST) and the number of nodes in the tree?

In the best case, BST will be a In the worst case, BST will be a
complete binary tree: Every nonleaf linked list: Every nonleaf node has
node has exactly two children and exactly one child:
all the leaves are at the same level:

h = O(lg n) h = O(n)

45
All dynamic operations run in O(h) time

 Therefore, the best case running time of these

algorithms are logarithmic in the number of nodes of the
BST.
 However, the worst case running time of these
algorithms are linear in the number of nodes of the BST.
 Note that, SortUsingBST will run in O(n2) and O(n lg n)
time for the worst case and for the best case
respectively.
 Only if we had a way to keep the tree balanced…

46