lec-07-rb
lec-07-rb
IITB India
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 1
Topic 7.1
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 2
Maintain balance
BST may have a large height, which is bad for the algorithms.
We call BST imbalanced when the difference between the left and right subtree height is large.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 3
Balancing height by rotation
max(h1 , h2 + 1, h3 + 1) + 1 max(h1 + 1, h2 + 1, h3 ) + 1
(A, B)-rotate
A B
h1 B A h3
T1 T3
(B, A)-rotate
h2 h3 h1 h2
T2 T3 T1 T2
For example, if h3 > h2 = h1 and we rotate the BST, we will get a valid and more balanced BST
with less height.
Rotation may be applied in the reverse direction, where A is the left child of B. We define the
symmetric rotation in both directions.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 4
Example: rotation
Example 7.1
In the following BST, we can rotate 8-17 edge.
8 17
(8, 17) rotate
5 17 8 21
11 21 5 11 18 30
18 30
Commentary: This tree operation is important. Please carefully observe the destination of red,blue, and brown subtrees.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 5
When to rotate? Can only rotation fix imbalance?
We need a definition of balance such that rotations operations should be able to achieve the
balance.
Design principle:
We minimize the number of rotations while allowing some imbalance.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 6
Topic 7.2
Red-black tree
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 7
Null leaves
To describe a red-black tree, we replace the null pointers for absent children with dummy null
nodes.
Example 7.2
The following tiny nodes are the dummy null nodes.
5 17
21
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 8
Red-black tree
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 9
Exercise: Identify red-black trees
Exercise 7.1
Which of the following are red-black trees?
17 17
5 5 17
16 16 5 17
Observations:
▶ Red nodes are not counted in the imbalance. We need them only when there is an imbalance.
▶ There cannot be too many red nodes. (Why?)
▶ Red nodes can be at every level except the root.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 10
Black height
Definition 7.2
The black height (bh) for each node is defined as follows.
0
n is a null leaf
bh(n) = max(bh(right(n)), bh(left(n))) + 1 n is a black node
max(bh(right(n)), bh(left(n))) n is a red node
Until it is stated otherwise, all heights mentioned in this lecture are black heights.
Exercise 7.2
Prove that for each node n in a red-black tree, bh(right(n)) == bh(left(n)).
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 11
Example: black height
Example 7.4
The black height of the following red-black tree is 2.
Exercise 7.3
Can we change the color of some nodes without breaking the conditions of a red-black tree?
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 12
Bound on the height of a red-black tree
▶ n is the smallest when all nodes are black. Therefore, the tree is a complete binary tree.
Therefore, n = 2h − 1.
▶ n is largest when the alternate levels of the tree are red. The height of the tree is 2h.
Therefore, n = 22h − 1.
Exercise 7.4
Define red-black tree inductively and write the proof of the above bounds using induction.
Commentary: Our definition of red-black tree is not inductive therefore the above proof does not read so formal. However, an inductive definition would make proof formal.
Can you write an inductive definition of red-black tree and prove the above statements?
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 13
Search, Maximum/Minimum, and Successor/Predecessor
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 14
Topic 7.3
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 15
BST insertion in red-black tree
1. Follow the usual algorithm of insertion in the BST, which inserts the new node n as a leaf.
▶ Note that there are dummy nodes. n is inserted as the parent of a dummy node.
2. We color n red.
After insertion, we may have a red-red violation, where a red node has a red child.
Commentary: We have null nodes in this setting. We need to add nulls as children to the new node.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 16
Example: insert in red-black tree
Example 7.5
Inserting 20 in the following tree.
8 8
Insert 20
5 17 5 17
21 21
20
The insertion results in violation of the condition of the red-black tree, which says red nodes can
only have black children.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 17
Iteratively remove red-red violation
A red-red violation starts at a leaf. However, an attempt to remove the violation may push up the
violation to the parent.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 18
Commentary: Since the new node is always added at
the leaf, one may think that h must be zero, which
Pattern around red-red violation may not be the case. As noted in the previous slide,
the violation may move upwards.
If n has a red parent, we correct the error by rotation and re-coloring operations.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 19
Case 1: The uncle is red
h+1 g h+1 g
re-color
h u p h+1 u p
T1 T1
n h n h
T4 T4
h h h h
T2 T3 T2 T3
In the subtree of g , there is no change in the black height and no red-red violation.
Now g is red. We have three possibilities: the parent of g is black, the parent of g is red, and g is
the root.
Commentary: Possibility 1: Nothing. Possibility 2:
Exercise 7.6 We have a red-red violation a level up and need to
apply the transformations there. Possibility 3: turn g
What do we do in each of the possibilities? back to black.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 20
Case 2: The uncle is not red and the path to the grandparent is not straight
straight means left 2 (parent 2 (n)) = n
or right 2 (parent 2 (n)) = n
h p h n
T1 T1
n h h p
T4 T2
h h h h
T2 T3 T3 T4
This transformation does not resolve the violation but converts the violation to case 3.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 21
Case 3: The uncle is not red and the path to the grandparent is straight
h+1 g h+1 p
h p g n
(g , p) rotate
T1
n swap colors
h h h h h
T2 T1 T2 T3 T4
h h
T3 T4
Exercise 7.7
a. Why are the roots of T2 , T3 , and T4 not red?
b. Show that if the root of T1 is red then the above operation does not work.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 22
Example: red-red correction case 1
Example 7.6
We just inserted 20 in the following tree. We need to apply case 1 to obtain a red-black tree.
8 8
re-color
5 17 5 17
12 21 12 21
20 20
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 23
Example: red-red correction case 2
Example 7.7
Consider the following example. We are attempting to insert 20. We apply case 2 to move towards
a red-black tree.
8 (20,21)-rotate 8
5 17 5 17
21 20
20 21
The above is not a red-black tree. Furthermore, we need to apply case 3 to obtain the red-black
tree.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 24
Example: red-red correction case 3 (continued)
8 (17,20)-rotate 8
20 17 21
21
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 25
Summary of insertion
Insert like BST, make the new node red, and run the following algorithm.
Commentary: What is the intuitive difference between cases 1 and 3? Case 1 reduces the total number of red nodes in a subtree. Case 3 balances if there are too many red
nodes on one side of the subtree.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 26
Topic 7.4
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 27
BST deletion in red-black tree
▶ Recall: In the BST deletion we always delete a node n with at most one non-null child.
p Remove n p
n x
T2 T2
x T1
T1
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 28
What can go wrong with a red-black tree?
Commentary: The or in the second bullet point is not xor. Both violations are possible at the same time.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 29
Violation removal procedure
To remove the violation at x, we may recolor and rotate around x, which may push the violation to
the parent. Therefore, we consider that the violation is in the middle of the tree.
Orange means that we need to consider all possible colors of the nodes.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 30
Violation pattern
After deletion, we may need to consider the following five nodes around x.
p p
re-color x
h to h − 1 x h s h x h s
T1 T1
c1 c2 c1 c2
T4 T4
T2 T3 T2 T3
Violation solved!
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 32
Case 2: x is not red and root.
Do nothing.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 33
Case 3: x is not red and the sibling of x is red
p s
(p, s) rotate
and swap colors p
h to h − 1 x h s h c2
T1 T4
c1 c2 h−1 x h c1
T4 T1
T2 T3 T2 T3
The transformation does not solve the height violation at the parent of x but changes the sibling of
x from red to black.
Exercise 7.9
Why must p, c1 , and c2 be non-null and black?
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 34
Case 4.1: x is not red, the sibling of x is black, and the right nephew is
red (Assuming x is the left child)
The color of p and c1 does not matter.
p s
(p, s) rotate
swap the color of p and s
h to h − 1 x h s and c2 is turned black h p h c2
T1 T4
h − 1 c1 h − 1 c2 h−1 x h − 1 c1
T4 T1
T2 T3 T2 T3
Exercise 7.10
Write the above case if x is the right child of p.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 35
Case 4.2: x is not red and the sibling is black, and the left and right
nephews are red and not red respectively (Assuming x is the left child)
p p
(s, c1 ) rotate
and swap colors
h to h − 1 x h s x h−1 h c1
T1 T1
c1 c2 s
T2
T4
c2
T2 T3 T3
T4
The above transformation does not solve the height violation. It changes the right child of the
sibling from not red to red, which is the case 4.1.
Exercise 7.11
a. Why can case 4.1 transformation not be applied to case 4.2?
b. Write the above case if x is the right child of p.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 36
Case 4.3: x is not red, the sibling of x is black, and the nephews of x are
not red
p p
color s red
color p black
h to h − 1 x h s h−1 x h−1 s
T1 T1
c1 c2 c1 c2
T4 T4
T2 T3 T2 T3
The above transformation reduces bh(p) by 1, if p was black, and there may be a potential
violation at p, which is at the lower level.
We apply the case analysis again at node p. The only case that kicks the can upstairs!! All cases
are covered.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 37
Structure among cases
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 38
Summary of deletion
1. Delete like BST. There may be a black height violation at the child of the deleted node.
2. While case 4.3 occurs, re-color nodes and move up the black height violation.
3. For all the other cases, we rotate or re-color, and the violation is finished.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 39
Topic 7.5
Tutorial problems
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 40
Exercise: validity of rotation
Exercise 7.12
Prove that after rotation the resulting tree is a binary search tree.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 41
Exercise: sorted insert
Exercise 7.13
Insert sorted numbers 1,2,3,..., 10 into an empty red-black tree. Show all intermediate red-black
trees.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 42
Insert and delete
Exercise 7.14
Consider the tree below. Can it be colored and turned into a red-black tree? If we wish to store
the set 1, . . . , 22, label each node with the correct number. Now add 23 to the set and then
delete 1. Also, do the same in the reverse order. Are the answers the same? When will the answers
be the same?
A
B C
D E F G
H I J K L M N
O P Q R S T U
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 43
Exercise: Hash table vs red-black tree
Exercise 7.15
a. Give running time complexities of delete, insert, and search in the red-black tree.
b. What are the advantages of the red-black tree compared to the Hash table, where every
operation (search, insert, delete) is almost constant time?
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 44
Topic 7.6
Problems
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 45
Exercise: consistent hashing** (midsem 2024)
Exercise 7.16
Let us suppose we want to store n keys on m servers. The servers have IDs. We may use a hash function h
to map the keys to servers. Both keys and IDs of the servers are hashed. The computed hash values are
placed on a ring, like a clock. Some of the points on the ring are the servers and some are keys. A key is
stored on the closest server in the clockwise direction. Each time a server is added/removed the keys are
moved according to the above rule.
8 9
Example: Let h(x) = 1 + x%11. We want to store keys 21, 60, and 90 on 7
10 141
three server with IDs 141, 201, and 738. The hash values of the keys are 60 6
11, 6, 3 and the hash values of the servers are 10, 4, 2. The keys 21, 60, 11 21
and 90 will be stored in servers 738, 141, and 201 respectively. 5
1
201 4
3 2
a. Give an efficient algorithm for adding/removing keys on the servers. 738
90
b. Give an efficient algorithm for implementing add/remove of servers.
Commentary: Solution: To implement the above scheme. We store the server hashes in a red black tree R.
findNext(k,s,s’){ if( s=Null) return s’;if(key(s) < k) return findNext(k,key(s),s’); else return findNext(k,left(s),s);}
findServer(k){s = findNext(k,R.root,Null) if(s == Null) s = minimum(R.root); if(s == Null ) Return Null; Return value(s)}
AddKey(k){ s = findServer(h(k));storeInServer(k,s)} RemoveKey(k){ s = findServer(h(k),R.root); removeInServer(k,s)}
AddServer(s){ s’ := h(s) R.Insert( (s’,s) ); t = nextServer(s’); for k in GreaterKeysOnServer(s’,t) {removeInServer(k,t);storeInServer(k,s)} }
RemoveServer(s){ s’ := h(s) R.Delete(s’); t = nextServer(s’); for k in KeysOnServer(s) { removeInserver(k,s);storeInServer(k,t); } }
Assume red-black Insert takes (key,value) pair as input and no two servers have collision, which can be easily avoided by renaming the IDs. Keys are again stored as
Red-black tree on the servers such that we can efficiently implement GreaterKeysOnServer. removeInServer are storeInServer are trivial calls to insert/delete of RB Tree.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 46
Excercise: Is BST red-black colorable?**
Exercise 7.17
Given a BST, can we check if there exists a colouring which makes it a valid RB-Tree?
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 47
Topic 7.7
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 48
AVL (Adelson, Velsky, and Landis) tree
Definition 7.3
An AVL tree is a binary search tree such that for each node n
|height(right(n)) − height(left(n))| ≤ 1.
Example 7.8 8
An example of an AVL tree.
5 17
7 11 21
10 14
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 49
Exercise: Identify the AVL trees
Exercise 7.18
Which of the following are AVL trees?
17
17 21 17 17
5 20 5 21 5 21
3 18 11 5 11
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 50
Topic 7.8
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 51
AVL tree height
Theorem 7.1
The height of an AVL tree T having n nodes is O(log n).
Proof.
Let n(h) be the minimum number of nodes for height h.
Base case:
n(1) = 1 and n(2) = 2.
Induction step:
Consider an AVL tree with height h ≥ 3. In the minimum case, one child will have a height of
h − 1 and the other child will have a height of h − 2. (Why?)
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 52
AVL tree height(2)
Proof(continued.)
Since n(h − 1) > n(h − 2),
n(h) > 2n(h − 2).
Therefore,
n(h) > 2i n(h − 2i).
For i = h/2 − 1(Why?), Commentary: Here is the explanation of the last step.
Consider an AVL tree with m nodes and h height. By
h/2−1 h/2
n(h) > 2 n(2) = 2 . definition, h(n) ≤ m. Since h < 2 log n(h), h <
2 log m. Therefore, h is O(log m).
Therefore,
h < 2 log n(h).
Theorem 7.2
Let T be an AVL tree. Let the level of the closest leaf to the root of T is k.
height(T ) ≤ 2k − 1
Proof.
.
Let D be the closest leaf of the tree.
2k − 1 r
▶ The height of right(C ) cannot be more than 2. (Why?)
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 54
A part of AVL is a complete tree
Theorem 7.3
Let T be an AVL tree. Let the level of the closest leaf to the root of T is k. Upto level k − 2 all
nodes have two children.
Proof.
A node at level k − 2 − i cannot be a leaf. (Why?)
Exercise 7.19
Show T has at least 2k−1 nodes.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 55
Another proof of tree height bound
Therefore,
n ≥ 2k−1 ≥ 2(h−1)/2
Exercise 7.20
What is the maximum number of nodes given height h?
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 56
Problem: A sharper bound for the AVL tree
Exercise 7.21
a. Find largest c such that c k−2 + c k−1 ≥ c k
b. Recall n(h) = 1 + n(h − 1) + n(h − 2). Let c0 be the largest c. Show that n(h) ≥ c h−1 .
c. Prove that the above bound is a sharper bound than our earlier proof.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 57
Topic 7.9
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 58
Insert and delete
We have to repair height imbalances by rotations around the deepest imbalanced node.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 59
Rebalancing AVL trees
Let x be the deepest imbalanced node. Let t be the taller child. Let g be the grandchild via t that
is not on the straight path from x.
h+3 x
h h+2 t
T1
g
T4
T2 T3
Three cases:
1. Case 1: Height of g is h + 1 and T4 is h + 1.
2. Case 2: Height of g is h and T4 is h + 1.
3. Case 3: Height of g is h + 1 and T4 is h.
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 60
Case 1: Both grandchildren via t have height h + 1
T2 T3 T2 T3
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 61
Case 2: Right-left grandchild has height h
T2 T3 T2 T3
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 62
Case 3: Right right grandchild has height h
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 63
Complexity of insertion/deletion
Exercise 7.22
a. What is the bound on the number of rotations for a single insert/delete?
b. Compare the bounds with RB trees insertion/deletion.
c. Which definition is more strict RB or AVL? Or, are they incomparable?
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 64
End of Lecture 7
cbna CS213/293 Data Structure and Algorithms 2024 Instructor: Ashutosh Gupta IITB India 65