Recurrence Analysis
Recurrence Analysis
In (2.1), n is a nonnegative integer variable, and n0 and T0 are nonnegative integer con-
stants. Functions s(·), f (·) and w(·) are nondecreasing, nonnegative integer functions of n
(as a consequence, T (·) is also a nondecreasing and nonnegative integer function). Finally,
f (n) < n for any n > n0 .
Equation (2.1) is often useful in the analysis of divide-and-conquer algorithms, where
a problem instance of size at most n0 is solved directly, while an instance of size n > n0 is
solved by
(i) decomposing the instance into s(n) instances of the same problem of size (at most1 )
f (n) < n each;
(iii) combining the solutions to the s(n) instances of size (at most) f (n) into a solution
to the instance of size n.
1
Needless to say, whenever the quantities featured in the recurrence are upper bounds, the resulting
solution T (n) will be an upper bound to the running time, while exact values yield the exact running time
of the resulting algorithm.
1
Here, w(n) is (an upper bound to) the overall running time of the decomposition and the
combination procedures. Also, T0 is (an upper bound to) the running time of the algorithm
on instances of size n ≤ n0 . With the given interpretation of n0 , T0 , s(·), f (·), and w(·),
Equation (2.1) uniquely defines a function T (n), which represents (an upper bound to) the
running time complexity of the given algorithm for any problem instance of size n.
The following notation is useful to formulate the general solution of Equation (2.1).
We let f (0) (n) = n, and for i > 0, f (i+1) (n) = f (f (i) (n)). We also denote by f ? (n, n0 ) the
largest k such that f (k) (n) > n0 . Note that, if n ≤ n0 , f ? (n, n0 ) would not be defined.
Conventionally, we set f ? (n, n0 ) = −1 for n ≤ n0 .
With the above notation, f (`) (n) is the size of a single problem instance at the `-th
level of recursion, where ` = 0 corresponds to the initial call. Level ` = f ? (n, n0 ) is the
last for which f (`) (n) > n0 and hence it is the last level for which Equation (2.1.a) applies.
At level f ? (n, n0 ) + 1, Equation (2.1.b) applies instead.
Thus, for 0 ≤ ` ≤ f ? (n, n0 ), the work spent on a single problem instance at level ` is
w(f (`) (n)). For ` = f ? (n, n0 ) + 1, the work per problem instance is T0 .
The instance at level 0 generates s(n) instances at level 1, each of which generates
s(f (n)) instances at level 2, each of which generates s(f (2) (n)) instances at level 3, . . .,
each of which generates s(f (`−1) (n)) instances at level `. Therefore, the total number of
instances at level ` is
`−1
s(n) · s(f (n)) · s(f (2) (n)) · . . . · s(f (`−1) (n)) = s(f (j) (n)),
Y
j=0
f ? (n,n0 ) f ? (n,n0 )
`−1
s(f (j) (n)) w(f (`) (n)) + s(f (j) (n)) T0 ,
X Y Y
T (n) =
`=0 j=0 j=0
where, for f ? (n, n0 ) = −1, the value of the summation in the above expression is conven-
tionally assumed to be 0.
The correctness of the above derivation can be proved by induction on n as follows. Let
us start with the base case(s) n ≤ n0 and recall that, conventionally, we set f ? (n, n0 ) = −1
for n ≤ n0 . Then, the closed formula correctlt yields T0 , since the summation and the
product within evaluate to 0 and 1, respectively.
Assume now that the formula yields the correct value of T (k), for k < n and n > n0 .
2
We have that T (n) = s(n)T (f (n)) + w(n), and, by the inductive hypothesis,
f ? (f (n),n0 ) f ? (f (n),n0 )
`−1
s(f (j+1) (n)) w(f (`+1) (n)) + s(f (j+1) (n)) T0 .
X Y Y
T (f (n)) =
`=0 j=0 j=0
Observe that, by the definition of f ? , in case f (n) ≤ n0 , then f ? (f (n), n0 ) = −1, while
f ? (n, n0 ) = 0. Otherwise, the maximum index k for which f (k) (f (n)) > n0 is clearly one
less than the maximum index k for which f (k) (n) > n0 , hence, in all cases, f ? (f (n), n0 ) =
f ? (n, n0 ) − 1. We have:
s(n)T (f (n))
? ?
f (n,n
X0 )−1 `−1 f (n,n0 )−1
s(f (0) (n)) s(f (j+1) (n)) w(f (`+1) (n)) + s(f (j+1) (n)) T0
Y Y
=
`=0 j=0 j=0
? ?
f (n,n
X0 )−1 `
0
f (n,n0 )
0
s(f (0) (n)) s(f (j ) (n)) w(f (`+1) (n)) + s(f (j ) (n)) T0
Y Y
=
0 0
`=0 j =1 j =1
0
(by substituting j = j + 1 in the two products)
f ? (n,n0 )−1 f ? (n,n0 )
`
(j 0 ) (j 0 )
(n)) w(f (`+1) (n)) +
X Y Y
= s(f s(f (n)) T0
`=0 j 0 =0 j 0 =0
3
Exercise 2.1 Determine f ∗ (n, n0 ) for the following values of f (n) and n0 : (a) (f (n) =
n − 1, n0 = 0), (b) (f (n) = n/2, n0 = 1), (c) (f (n) = n1/2 , n0 = 2) and (d) (f (n) = n1/2 ,
n0 = 1).
Answer: Observe that function iteration is well defined also for real-valued functions.
Therefore, for the sake of generality, we will consider f (n) to be real-valued, with domain
< in cases (a) and (b), and [0, ∞) in cases (c) and (d)
(a) Note that f (1) (n) = n − 1, f (2) (n) = f (1) (n) − 1 = n − 2, and, in general, f (i) (n) =
∗
f (i−1) (n) − 1 = n − i. If n ≤ 0, then f (n, n0 ) = −1. Otherwise, letting
we get i < n. Thus, the largest i ≥ 0 such that f (i) (n) = n − i > 0 is dne − 1. Hence,
dne − 1 if n > 0,
f ∗ (n, 0) =
−1 otherwise.
(b) Note that f (1) (n) = n/2, f (2) (n) = f (1) (n)/2 = n/22 , and, in general, f (i) (n) =
f (i−1) (n)/2 = n/2i . If n ≤ 1, then f ∗ (n, 1) = −1. Otherwise, letting
we get n > 2i , whence i < log2 n. Thus, the largest i ≥ 0 such that f (i) (n) = n/2i > 1
is dlog2 ne − 1. Hence,
dlog2 ne − 1 if n > 1,
f ∗ (n, 1) =
−1 otherwise.
(c) Note that f (1) (n) = n1/2 , f (2) (n) = (f (1) (n))1/2 = n1/4 , and, in general, f (i) (n) =
i
(f (i−1) (n))1/2 = n1/2 . If n ≤ 2, then f ∗ (n, 2) = −1. Otherwise, letting
−i
f (i) (n) = n2 > 2,
i
we get log2 n1/2 = (log2 n)/2i > log2 2 = 1. Therefore, we have 2i < log2 n, whence
−i
i < log2 log2 n. Thus, the largest i ≥ 0 such that f (i) (n) = n2 > 2 is dlog2 log2 ne−1.
4
Hence,
dlog2 log2 ne − 1 if n > 2,
f ∗ (n, 2) =
−1 otherwise.
i
f (i) (n) = n1/2 > 1,
for all i. Thus, f ∗ (n, 1) is undefined, since there infinitely many values of i satisfying
f (i) (n) > 1. Hence,
undefined if n > 1,
f ∗ (n, 1) =
−1 otherwise.
Observe that the degeneracy in the latter case is due to the fact that we are assuming
that f (n) is a real-valued function which may return noninteger values. In fact, no such
phenomenon can be observed for integer-valued functions, where the value of f ∗ (n, n0 )
cannot be larger than n − n0 , since each iteration of f must decrease the value of its
argument by at least one. 2
Exercise 2.2 Consider the recurrence T (n) = 2T ( n2 )+w(n), with T (1) = T0 , an arbitrary
constant. Write the general solution. Specialize your formula in the following cases:
Answer:
n
T (n) = 2T ( ) + w(n)
2
k−1
= 2k T (n/2k ) + 2i w(n/2i )
X
k≥1
i=0
log2 n−1
= 2log2 n T (1) + 2i w(n/2i )
X
i=0
log2 n−1
2i w(n/2i ).
X
= nT0 +
i=0
5
(a) w(n) = a (a constant).
i=0 i=0
= a 2log2 n − 1
= a(n − 1)
Thus,
T (n) = nT0 + a(n − 1) = (a + T0 )n − a.
i=0 i=0 2i
= a[log2 n + 2(log2 n − 1) + 22 (log2 n − 2) + . . .
+ 2log2 n−2 ·2 + 2log2 n−1 ·1]
= · · · + 2| log2 n−2 +
{z
2log2 n−2} + |2log{z
2 n−1
}
2 1
log2 n−1 log2 n−2 log2 n−3 1 0
2i + 2i + 2i + . . . + 2i + 2i
X X X X X
= a
i=0 i=0 i=0 i=0 i=0
log2 n log2 n−1 2
= a[(2 − 1) + (2 − 1) + · · · + (2 − 1) + (2 − 1)]
log2 n
2i − log2 n
X
= a
i=1
h i
= a 2log2 n+1 − 2 − log2 n
= a[2n − log2 n − 2].
Thus,
T (n) = nT0 + a(2n − log2 n − 2) = (T0 + 2a)n − alog2 n − 2a.
6
(c) w(n) = a log22 n.
i=0 i=0 2i
= a[log22 n + 21 (log2 n − 1)2 + 22 (log2 n − 2)2 + . . . + 2log2 n−1 ·12 ]
= · · · + 2| log2 n−2 +
{z
2log2 n−2} + |2log{z
2 n−1
}
22 1
Observing that
log2 n−i
(log2 n − i)2 =
X
(2k − 1),
k=1
Now,
7
= 22 (2n − log2 n − 2) − 2(n − 1)
(see Point (b))
= 6n − 4log2 n − 6. (2.3)
log2 n−1
2i w(n/2i ) = a(6n − 4log2 n − 6 − log22 n).
X
i=0
Therefore,
therefore,
d
S(1) = x S(0).
dx
In general,
d
S(k + 1) = x S(k) k ≥ 0.
dx
Hence, starting with (2.4), one can successively evaluate S(1), S(2), . . ., using this equation.
i=0 i=0 2
= anlog2 n.
8
Thus,
T (n) = anlog2 n + nT0 .
i=0 2i
1
= an2 ·2(1 − )
n
= 2an(n − 1).
Thus,
T (n) = nT0 + 2an2 − 2an = 2an2 + (T0 − 2a)n.
(f ) w(n) = n/log2 n.
since
Z log n+1 log2 n Z log n
2 1 X 1 2 1
loge (log2 n + 1) = dx ≤ ≤1+ dx = 1 + loge log2 n.
1 x i=1 i 1 x
Thus,
T (n) = nT0 + n ln log2 n + O(n) = n ln log2 n + O(n).
Exercise 2.3 Solve the following recurrence when the parameter n is an integral power
9
of 3:
T (n) = 6T ( n3 ) + n(n − 1), n > 1,
T (1) = 4.
Answer: The following table summarizes all the relevant information obtained from the
recursion tree:
10
Exercise 2.4 Solve the following recurrence when the parameter n is a power of two:
n 3
T (n) = T + n2 + 2 log n − 1, n > 1,
2 4
T (1) = 1.
n
T (n) = T + f (n)
2
n n
= T +f + f (n)
4 2
..
.
i−1
n n
X
= T i
+ f j , for 1 ≤ i ≤ log n.
2 j=0 2
Therefore we have:
T (n) = n2 + log2 n. (2.5)
Exercise 2.5 (a) Solve the following recurrence when the parameter n is a power of two
and c and d are positive constants:
n √
√
T (n) = 2T + 2 − 1 c n, n > n0 ,
√2
T (n0 ) = dn0 n0 .
11
Answer:
n
√ √
T (n) = 2T + 2−1 c n
2
2 n
√ √ √
= 2T + 1 + 2 2 − 1 c n
22
n
√ √ 2 √
√
= 23 T + 1 + 2 + 2 2 − 1 c n
23
..
.
√ √ X i−1 √
n k
= 2i T i
+ 2 − 1 c n 2 .
2 k=0
√ √ √ log(n/n
X0 )−1 √ k
T (n) = dn n0 + 2−1 c n 2
k=0
√
s !
√ n
= dn n0 + c −1 n
n0
√
!
√ 1
= d n0 + c √ n−c n
n0
√
= ncoeff(n0 ) − c n.
√
(b) By taking the partial derivative of T (n) with respect to n0 we obtain
δT (n) δcoeff(n0 ) c
√ = √ =d− ,
δ n0 δ n0 n0
whence
δT (n) c
√ ≥ 0 iff n0 ≥ ,
δ n0 d
for any value of the parameter n. Since n0 has to be an integral power of two, the solution
is minimized for either
n o n o
n00 = max 1, 2blog c/dc or n000 = max 1, 2dlog c/de ,
12
Exercise 2.6 Solve the following recurrence when the parameter n is a power of four:
n
T (n) = 16 T + 2 n2 , n > 1,
4
T (1) = 0.
Answer: The above recurrence can be solved in a number of standard ways. Here, we
choose to illustrate a trick (also applicable in other cases) that simplifies the recurrence.
Letting T (n) = n2 Q(n), we obtain T (n/4) = (n2 /16)Q(n/4), and T (1) = Q(1) = 0.
Therefore, the recurrence for T (n) can be rewritten for Q(n) as follows:
n
Q(n) = Q + 2, n > 1,
4
Q(1) = 0.
Exercise 2.7 Solve the following recurrence when the parameter n is a power of two:
n
T (n) = (log n)T + 1, n > 1,
2
T (1) = 1.
Answer:
13
for 2 ≤ i ≤ log n. For i = log n, T (n/2i ) = T (1) = 1, and we get
logY
n−1 logX
n−2 j
Y
T (n) = (log n − j) + 1 + (log n − k)
j=0 j=0 k=0
logX
n−2
(log n)!
= (log n)! + 1 +
j=0 (log n − j − 1)!
(log n)! (log n)! logXn−1
(log n)!
= + +
0! (log n)! k=1 k!
log
Xn 1
= (log n)!
k=0 k!
2
Plog n P∞
Since k=0 1/k! < k=0 1/k! = e we have T (n) = O((log n)!).
Exercise 2.8
(a) Solve the following recurrence when the parameter n is a double power of two (i.e.,
i
n = 22 , for some i ≥ 0).
√ √ √
T (n) = nT ( n) + n − 1, n > 2
T (2) = 1.
(b) Design a divide-and-conquer algorithm for the problem of finding the maximum of a
i
set of n = 22 numbers that performs a number of comparisons obeying to the above
recurrence.
Answer:
i
1
2
(a) Let us compute T (n) for small values of n = 22 , e.g., n = 2, 4 = 22 , 16 = 22 ,
3
256 = 22 .
T (2) = 1
T (4) = 2 · T (2) + 2 − 1 = 2 · 1 + 2 − 1 = 3
T (16) = 4 · T (4) + 4 − 1 = 4 · 3 + 4 − 1 = 15
T (256) = 16 · T (16) + 16 − 1 = 16 · 15 + 16 − 1 = 255
Based on the above values, we guess that T (n) = n−1. Let us prove
0
our guess by induction
2i 2
on i ≥ 0, where n = 2 . The base of the induction holds, since T 2 = T (2) = 1 = 2−1.
14
k
k
Let us now assume that T 22 = 22 − 1 for all values k < i. For k = i, we have:
2i 1/2 2i 1/2
1/2
2i i
T 2 = 2 ·T 2 + 22 −1
i−1
i−1
i−1
= 22 · T 22 + 22 −1
i−1
i−1
i−1
= 22 · 22 − 1 + 22 −1
(inductive hypothesis)
i−1 i−1
= 22 · 22 −1
2i
= 2 − 1.
i
(b) Let A[1..n] be an array of n = 22 numbers. A recursive algorithm SQRT MAX
performing a number of comparisons obeying to the above recurrence is the following:
SQRT MAX(A)
n ← length(A)
if n = 2
then if A[1] ≥ A[2]
then return A[1]
else
√ return A[2]
for i ← 1 to n √ √
do T M P [i] ← SQRT MAX (A [ (i − 1) ∗ n + 1 .. i ∗ n ])
max ← T M P√ [1]
for i ← 2 to n
do if max < T M P [i]
then max ← T M P [i]
return max
For n > 2, the above algorithm recursively determines the maxima for the sub-arrays
h √ √ i √
A (i − 1) ∗ n + 1 .. i ∗ n , 1≤i≤ n,
√
and then determines the overall maximum by performing n − 1 comparisons among these
maxima. The correctness of the algorithm follows from the fact that the above sub-arrays
√ √
induce a partition of the n indices of the original array. Since i∗ n−((i − 1) ∗ n + 1)+1 =
√ √
n, for any 1 ≤ i ≤ n, the number T (n) of comparisons performed by SQRT MAX(A)
when length(A) = n is clearly given by the recurrence solved in Part (a). Therefore
T (n) = n − 1. 2
15
i
Exercise 2.9 Solve the following recurrence when the parameter n is 23 , for some i ≥ 0:
T (n) = n2/3 T n1/3 + n2/3 − 1, n > 2,
T (2) = 1.
Answer: We have:
T (8) = 4 · T (2) + 4 − 1 = 4 · (1 + 1) − 1 = 7,
T (512) = 64 · T (8) + 64 − 1 = 64 · (7 + 1) − 1 = 511.
i
i
As in the previous exercise, we guess that T 23 = 23 − 1, for any i ≥ 0 and prove our
0
0
guess by induction on i. The basis is clearly true, since T 23 = 1 = 23 − 1. For i > 0
we have:
3i 2/3 3i 1/3
2/3
3i i
T 2 = 2 ·T 2 + 23 −1
i−1
i−1
i−1
= 22·3 T 23 + 22·3 −1
2·3i−1 3i−1 2·3i−1
= 2 2 −1 +2 −1
i−1
i−1
= 22·3 · 23 −1+1 −1
2·3i−1 3i−1
= 2 ·2 −1
(2+1)·3i−1
= 2 −1
i
= 23 − 1,
Exercise 2.10 Solve the following recurrence when the parameter n is a power of two:
T 2 (n/2)
T (n) = + n, n > 1,
n
T (1) = 2.
Answer: Let us compute T (n), for small values of the parameter n, by “manually”
unfolding the recursion, so to get an idea of the form of the solution.
n = 1 : T (1) = 2;
n = 2 : T (2) = T 2 (1)/2 + 2 = 4;
n = 4 : T (4) = T 2 (2)/2 + 4 = 8;
16
n = 8 : T (8) = T 2 (4)/2 + 8 = 16.
is a plausible guess. Let us now try to confirm our guess by using induction. Since
T (1) = 2 = 2 · 1, Relation (2.6) holds for the base. Assume now that T (n0 ) = 2n0 , for any
power of two n0 < n. We have:
T 2 (n/2)
T (n) = +n
2
(2 · n/2)2
= +n
n
n2
= + n = 2n,
n
therefore (2.6) holds for any power of two. 2
MAXMIN(S)
Let S = {a1 , a2 , . . . , an }
if n = 2
then if a1 ≥ a2
then return (a1 , a2 )
else return (a2 , a1 )
S1 ← {a1 , a2 , . . . , an/2 }
S2 ← {an/2+1 , an/2+2 , . . . , an }
(max1 , min1 ) ←MAXMIN(S1 )
(max2 , min2 ) ←MAXMIN(S2 )
return (MAX(max1 , max2 ), MIN(min1 , min2 ))
In the above algorithm, when the sequence S has two elements, say S = (a1 , a2 ), we simply
compare a1 and a2 to obtain max{S} and min{S}, thus requiring only one comparison. If
17
|S| > 2, the number of comparisons required to yield max{S} and min{S}, given max{S1 },
min{S1 }, max{S2 } and min{S2 } is 2 (one to compute MAX(max1 , max2 ) and one to
compute MIN(min1 , min2 )). Hence,
n
2T 2
+ 2, if n > 2,
T (n) =
1, if n = 2.
k
n
T (n) = 2k T 2i , k ≥ 1
X
+
2k i=1
n
= T (2) + 2(2log2 n−1 − 1)
2
n n
= + 2( − 1)
2 2
3n
= − 2.
2
Diagram:
This diagram depicts a com-
parison between values a and
a - x - b. The outputs x and y de-
b (a, b) y
- - note the maximum and mini-
mum value, respectively.
The following diagram shows the comparisons performed by the algorithm on input S =
(7, 4, 5, 2, 1, 6, 3, 8).
7 - 7 - 7
4 - (7,4) 4 (7,5)
A
A
A
A
5 - 5 AU - max 8
-
2 - (5,2) 2 - (4,2) 2 (7,8)
A
A
A
A
1 - 6 - 8 AU
6 - (1,6) 1 (6,8) - (2,1) min 1
-
A
A
A
A
3 - 8 AU
8 - (3,8) 3 - (1,3) 1
18
Observe that T (8) = 3·82
− 2 = 10. After n/2 comparisons, there are n/2 − 1 comparisons
organized in a tree of minimum computations, and n/2 − 1 comparisons organized in a tree
of maximum computations. 2
a a
log2 7
T (N ) = +1 N 2 − N.
3 3
(b) Find (an approximation to) N0 such that T (N0 ) = TDEF (N0 ) when a = 15. Recall
that TDEF (N ) = 2N 3/2 − N.
Answer:
(a) To verify the given solution, we can simply plug it into the recurrence equations:
a a
1
T (1) = 1 2 log 7+1 −1
3 3
a a
= + 1 − = 1.
3 3
21 log 7
N a N aN
T = +1 −
4 3 4 3 4
1 1
log 7
a N2 aN a N 2 log 7 aN
= +1 1 − = +1 − .
3 4 2 log 7 12 3 7 12
1
N aN a N 2 log 7 aN aN
T (N ) = 7T + = 7 +1 − +
4 4 3 7 12 4
a 7aN 3aN a aN
1 1
= + 1 N 2 log 7 − + = + 1 N 2 log 7 − .
3 12 12 3 3
However, we can also derive this solution from the recurrence solution given in class:
∗ ∗
f (N,N0 ) `−1 f (N,N0 )
s(f (j) (N )))w(f (`) (N )) + s(f (`) (N ))T0 .
X Y Y
T (N ) = (
l=0 j=0 `=0
19
For this problem, s(N ) = 7, f (N ) = N4 , T0 = 1, w(N ) = aN
4
, and N0 = 1. Thus,
∗ ∗ 1
f (N, N0 ) = f (N, 1) = log4 N − 1 = 2 log N − 1. Plugging these into the formula, we
get:
1 1
log N −1 log N −1
`−1
2 X Y aN 2 Y
T (N ) = 7 + 7
`=0 j=0 4 4` `=0
1
log N −1 `
aN 2
7 1
+ 7 2 log N
X
=
4 `=0 4
1
aN 4 7 2 log N 1
= − 1 + 7 2 log N
4 3 N
a a
1
= 7 2 log N +1 − N
3 3
a a
1
= N 2 log 7 + 1 − N.
3 3
(b) Write the recurrence relation for the number T (n) of arithmetic operations that your
algorithm performs on n × n repetitive matrices.
20
Answer:
(a) Let M1 and M2 be the two n × n repetitive matrices to be multiplied, and let
A A C C
M1 = and M2 = .
B A D C
We have
A×C +A×D 2·A×C
M1 × M2 = , (2.7)
A×D+B×C A×C +B×C
where × and + denote, respectively, row-by-column multiplication and matrix sum, and ·
denotes multiplication by a scalar.
Since A, B, C, D are in turn repetitive matrices of size n/2 × n/2, Equation (2.7)
implies that three (recursive) row-by-column products of n/2 × n/2 repetitive matrices,
three sums of two n/2 × n/2 (general) matrices and one multiplication of an n/2 × n/2
(general) matrix by a scalar suffice to compute a row-by-column product of two n × n
repetitive matrices. Let SUM(X, Y ) be an algorithm that returns the sum of two n × n
matrices X and Y , and let SC PROD(c, X) be an algorithm that returns the scalar product
c · X. Clearly, we have TSUM (n) = TSC PROD (n) = n2 The algorithm for multiplying two
repetitive matrices is the following:
(b) To multiply two repetitive n × n matrices, we perform three recursive calls on n/2 ×
n/2 matrices and then combine the results of the three calls using three sums of n/2 × n/2
21
matrices and one scalar product. Therefore, the total work is w(n) = 4 · (n2 /4) = n2 , and
we obtain the following recurrence:
T (n) = 3T (n/2) + 3TSUM (n/2) + TSC PROD (n/2)
= 3T (n/2) + n2 , n > 1, (2.8)
T (1) = 1.
(c) In Recurrence (2.8), we have s(n) = 3, f (i) (n) = n/2i , w(n) = n2 , T0 = 1 and N0 = 1.
Therefore,
log2 n−1
3` n2 /4` + 3log2 n
X
T (n) =
`=0
log2 n−1
2
(3/4)` + nlog2 3
X
= n
`=0
2
= 4n 1 − (3/4)log2 n + nlog2 3
= 4n2 1 − nlog2 3−2 + nlog2 3
= 4n2 − 3nlog2 3 ,
whence T (n) = Θ(n2 ). Let us prove that the above formula is correct. We have T (1) =
1 = 4 · 1 − 3 · 1log2 3 . If the formula holds for values of the parameter less than n, then
T (n) = 3T (n/2) + n2
= 3 4(n/2)2 − 3(n/2)log2 3 + n2
= 3n2 − 9nlog2 3 /3 + n2
= 4n2 − 3nlog2 3 .
Observe that the sum of two repetitive matrices is a repetitive matrix and therefore
contains many repeated entries, which need to be computed only once. Indeed, to sum
two n × n repetitive matrices, we only have to compute two sums of n/2 × n/2 repetitive
matrices, with no extra arithmetic operations required for combining the results (we just
have to make repeated copies of matrix entries). Hence, when summing two repetitive
matrices,
T
SUM (n) = 2TSUM (n/2) , n > 1,
TSUM (1) = 1.
(An identical recurrence, with the same motivation, holds for TSC PROD (n)). By unfolding
22
the above recurrence for n = 2k , we have
TSUM (n) (resp., TSC PROD (n)) = 2k · TSUM (1) (resp., TSC PROD (1)) = n.
However, we cannot substitute these new running times for TSUM (n) and TSC PROD (n) in
Recurrence (2.8), since we really need to sum (or take scalar products of) general matrices
in the combining phase of REP MAT MULT. This is due to the fact that the product of
two repetitive matrices is not necessarily repetitive, therefore, matrix A × C and matrix
A×D, for instance, may contain an arbitrary number of distinct elements. Summing A×C
and A × D may entail as many as (n/2)2 distinct scalar sums. 2
√
Exercise 2.14 Let n be an even power of two, and let Π be the problem of merging n
√
sorted sequences, each containing n elements, into a single sorted sequence of n elements.
(a) Design and analyze an algorithm for Π that performs at most (n/2) log n comparisons.
(b) Use the algorithm for Π developed in Part (a) to sort a sequence of n elements in at
most n log n comparisons.
(c) Knowing that a lower bound for sorting is n log n−γn comparisons, for a fixed constant
γ > 0, determine a lower bound on the number of comparisons needed to solve Π.
Answer:
√
(a) Consider an input vector A[1..n] containing the concatenation of the n sorted se-
quences. We can use the standard MERGESORT algorithm, halting the recursion when
√
the subproblem size is s = n. Algorithm SQRT SORT below assumes a global knowledge
of vector A:
SQRT SORT(i, q
j)
if j − i + 1 = length(A)
then return
middle ← b(i + j)/2c
SQRT SORT(i, middle)
SQRT SORT(middle + 1, j)
MERGE(A, i, middle, j)
return
The outer call is clearly SQRT SORT(1, length(A)). Recall that MERGE(i, middle, j)
performs at most j − i + 1 comparisons (one for each element of the resulting sorted
23
√
sub-array). Let n = length(A) and s = n. Then, the recurrence on the number of
comparisons TSS (n) performed by SQRT SORT(1, length(A)) is:
√ √
(b) Given n elements, we first group them into n sets of size n and sort each group
using MERGESORT. Then we use SQRT SORT to obtain the sorted sequence.
SORT(A)
n ← length(A)
√
for i ← 1 to n do √ √
MERGESORT(A, (i − 1) · n + 1, i · n)
SQRT SORT(1, n)
return
Essentially, SORT(A) coincides with MERGESORT, with the only difference that the
24
activity performed by MERGESORT is viewed as the cascade of two phases.
√
(c) In Part (b), we showed how to sort n elements by first sorting n sequences of size
√
n, and then solving Π. Let TAΠ (n) be the running time of any algorithm solving Π. Since
√
we can separately sort the n sequences in (n/2) log n comparisons (calling MERGESORT
√
n times) it must be
(n/2) log n + TAΠ (n) ≥ n log n − γ,
or we would obtain an algorithm for sorting which beats the lower bound. Therefore
Exercise 2.15
(b) Write the recurrence associated with the algorithm of Part (a).
Answer:
(a) Let H be the input vector containing N = 2n − 1 elements. Our algorithm works
directly on H as follows. We first (recursively) create two subheaps H1 and H2 (note
that |H1 | = |H2 | = 2n−1 − 1) rooted at the two children of the root. These heaps are
stored according to the typical heap indexing scheme (i.e., if a node is stored at index i,
its children are found at indices 2i and 2i + 1) in the locations of H starting from index
2 for H1 , and index 3 for H2 . Then we “meld” H1 and H2 into a single heap by calling
HEAPIFY(H, 1) (see Cormen, Leiserson and Rivest’s book, page 143) to extend the heap
property to the whole vector.
Algorithm R BUILD HEAP(i) implements the above strategy when the root of the
(sub)-heap is stored in H[i]. Clearly, the entire heap is built by calling R BUILD HEAP(1).
25
R BUILD HEAP(i)
if 2i ≥ (N + 1)/2 then HEAPIFY(H, i)
{ take care of last two levels }
return
R BUILD HEAP(2i)
R BUILD HEAP(2i + 1)
HEAPIFY(H, i)
return
The correctness of the above algorithm is easily shown by induction.
(b) Let S(i) be the set of array indices storing the subheap rooted at index i, for any
i ≥ 0. We have S(i) = {i} ∪ S(2i) ∪ S(2i + 1), and S(2i) ∩ S(2i + 1) = ∅. Moreover
|S(2i)| = |S(2i + 1)|, whence |S(2i)| = |S(2i + 1)| = (|S(i)| − 1)/2. As a consequence, the
recursive calls will work on a set of indices whose size is less than half the size of the one
associated with the outer call. This property is essential to guarantee that the algorithm
is efficient. Also, note the size of any subheap H built by our recursive algorithm is a
power of two minus one, therfore we can write the recurrence using the exponent as the
parameter. Recalling that HEAPIFY takes time ck on a heap with 2k − 1 nodes, we have:
(c) We have:
T (k) = 2T (k − 1) + ck
= 2(2T (k − 2) + c(k − 1)) + ck
..
.
i−1
= 2i T (k − i) + c 2j (k − j)
X
j=0
..
.
k−2
= 2k−1 + c 2j (k − j)
X
j=0
We have:
k−2
2j (k − j) = 3 · 2k−1 − k − 2,
X
j=0
26
therefore, for any k ≥ 1,
T (k) = (3c + 1)2k−1 − c(k + 2).
Note that T (k) = O(2k ), therefore the algorithm is linear in the number of elements of the
heap. 2
Exercise 2.16 Consider an array storing the preorder sequence of a binary search tree
of distinct elements.
(a) Describe an efficient recursive algorithm that determines if a key x belongs to the
array.
(b) Evaluate the running time of the algorithm as a function of the number of nodes n
and the depth d of the tree.
Answer:
(a) We devise an algorithm that “simulates” binary search on a binary search tree:
TREE SEARCH(T, x)
if T = nil then return “not found”
if root(T ) = x then return “found”
if x < root(T )
then return TREE SEARCH(left(T ), x)
else return TREE SEARCH(right(T ), x)
However, we are only given the preorder sequence p(T ) of T , stored in a vector A, therefore
we have to find a way of identifying left and right subtrees as subsequences of p(T ).
It can be easily shown that for any tree T , p(T ) has the following recursive structure:
(note that p(left(T )) (resp., p(right(T ))) is the empty string if left(T ) (resp., right(T )) is
empty). Therefore the preorder sequence of a non empty left subtree starts soon after the
root, which is the first element of the sequence. In contrast, the preorder sequence of a
non empty right subtree terminates the sequence. In order to completely identify the two
sequences, we only have to determine the starting point in the array of p(right(T )). This
can be accomplished efficiently by performing a binary search on the vector, using the root
as the key:
27
FIND RIGHT(h, k, root)
{ h and k are the lower and upper bounds of the
subvector where the search takes place}
if h > k then return h
middle ← (h + k)div2
if root < A[middle]
then return FIND RIGHT(h, middle − 1, root)
else return FIND RIGHT(middle + 1, k, root)
The correctness of the above procedure is due to the fact that all the elements in p(left(T ))
(resp., p(right(T ))) are smaller (resp., larger) than root(T ).
We are now ready to give the required search procedure
ARRAY SEARCH(i, j, x)
if i > j then return “not found” { empty subtree}
if A[i] = x then return “found”
right ← FIND RIGHT(i + 1, j, A[i])
if x < A[i]
then return ARRAY SEARCH(i + 1, right − 1, x)
else return ARRAY SEARCH(right, j, x)
(b) If d is the depth of the tree, ARRAY SEARCH will perform at most d calls of
procedure FIND RIGHT. Each of such calls takes time O(log |h−k|) = O(log n). Therefore
the overall running time is O(d log n). Such
algorithm is more efficient than a trivial scan
n
of the array for depths as large as O log N . 2
Answer: The above recurrence could stem from a divide-and-conquer algorithm that, on
input an instance of size n, splits the instance into three subinstances of size n/2, n/4,
and n/4, respectively, and recombines the results performing 3n/2 work. We note that
the sum of the sizes of the subinstances is n, and that the work is linear in such sum.
This scenario is reminiscent of recurrences where the input instance is partitioned into a
constant number k of subinstances of equal size n/k and the split/combination work is
28
Θ(n), i.e.,
n
T 0 (n) = kT 0 + Θ(n). (2.10)
k
By applying the Master Theorem, we know that the above recurrence yields T 0 (n) =
Θ(n log n). Pursuing the analogy between our recurrence and Recurrence (2.10), let us try
to prove by induction that there exists a constant a > 0 such that T (n) ≤ an log n. We
have two base cases to cover, namely,
T (1) = 0
≤ a · 1 · log 1 = 0,
T (2) = 2
≤ a · 2 · log 2 = 2a,
which is true if and only if a ≥ 1. Assume that our statement holds for powers of four
which are less than n. We then have:
n n 3
T (n) = T + 2T + n
2 4 2
n n n n 3
≤ a log + 2a log + n
2 2 4 4 2
n n 3
= a (log n − 1) + a (log n − 2) + n
2 2 2
3 3
= an log n − an + n.
2 2
The thesis will follow if we make sure that
3 3
− an + n ≤ 0,
2 2
for which again it suffices to choose a ≥ 1. In conclusion, by picking a = 1, we have proved
that
T (n) ≤ n log n. (2.11)
By conducting a completely symmetric reasoning to the above, we can show that T (n) ≥
bn log n for any constant b ≤ 1. By picking b = 1 we then have:
29
By combining (2.11) and (2.12) we can then conclude that
T (n) = n log n,
for any value of n ≥ 1. Note that we obtained the exact solution to our recurrence by
providing perfectly matching lower and upper bounds. Clearly, this is not always the case,
and for other recurrences in this class the best we can do is to determine the order of
magnitude of their solution. 2
Exercise 2.18
(a) Let
√
M√be a square matrix with N = 4k elements. Prove that, if A, B and C are
2
N
× 2N , the following relation holds between M and its inverse M −1 :
A B A−1 −A−1 BC −1
M = M −1 = .
0 C 0 C −1
(c) Write the recurrence relation for T (N ), defined as the number of scalar multiplica-
tions performed by the algorithm of Part (b). Assume that the multiplication of
square matrices is done according to the definition.
(a) Evaluate the running time of an algorithm A2 based on the divide and conquer
strategy, under the assumption that T2 (1) = 0.
(b) Consider an algorithm A using the divide-and-conquer strategy for n > n0 and calling
instead A1 for n ≤ n0 . Determine the running time T (n) of A (note that T (n) will
be a function of n0 ).
(c) Find the value of n0 that minimizes T (n) (note that n0 will be a function of a).
30
?
Exercise 2.20 Let f (n) = log n, and define, for any n ≥ 2, log? n = f (n, 1). Show that
log? n = o(log log n).
Exercise 2.21 Solve the following recurrence when the parameter n is a double power of
i
two (i.e., n = 22 , for some i ≥ 0).
√
T (n) = nT ( n) n > 2
T (2) = 4.
31