Recursive Algorithms

Recursion
A recursive algorithm is one in which objects are defined in terms
of other objects of the same type.
Slides by Christopher M. Bourke
Instructor: Berthe Y. Choueiry Advantages:

I Simplicity of code
I Easy to understand
Fall 2007 Disadvantages:

I Memory
I Speed
Computer Science & Engineering 235
Introduction to Discrete Mathematics
I Possibly redundant work
Sections 7.1 - 7.2 of Rosen
Tail recursion offers a solution to the memory problem, but really,
[email protected]
do we need recursion?

Recursive Algorithms Motivating Example

Analysis Factorial
Recall the factorial function.

1 if n = 1
n! =
n · (n − 1)! if n > 1

Consider the following (recursive) algorithm for computing n!:

We’ve already seen how to analyze the running time of algorithms.
However, to analyze recursive algorithms, we require more Algorithm (Factorial)
sophisticated techniques.
Input :n∈N
Specifically, we study how to define & solve recurrence relations.
Output : n!
1 if n = 1 then
2 return 1
3 end
4 else
5 return Factorial(n − 1) × n
6 end

Motivating Example Recurrence Relations I

Factorial - Analysis? Definition

How many multiplications M (n) does Factorial perform?

Definition
I When n = 1 we don’t perform any.
I Otherwise we perform 1. A recurrence relation for a sequence {an } is an equation that
expresses an in terms of one or more of the previous terms in the
I Plus how ever many multiplications we perform in the
sequence,
recursive call, Factorial(n − 1).
a0 , a1 , . . . , an−1
I This can be expressed as a formula (similar to the definition of
n!. for all integers n ≥ n0 where n0 is a nonnegative integer.
M (0) = 0
A sequence is called a solution of a recurrence relation if its terms
M (n) = 1 + M (n − 1)
satisfy the recurrence relation.
I This is known as a recurrence relation.
Recurrence Relations II Recurrence Relations III
Definition Definition

Example
Consider the recurrence relation: an = 2an−1 − an−2 .
The Fibonacci numbers are defined by the recurrence,
It has the following sequences an as solutions:
F (n) = F (n − 1) + F (n − 2)
1. an = 3n, F (1) = 1
2. an = n + 1, and F (0) = 1
3. an = 5.
The solution to the Fibonacci recurrence is
Initial conditions + recurrence relation uniquely determine the √ !n √ !n
sequence. 1 1+ 5 1 1− 5
fn = √ −√
5 2 5 2

(your book derives this solution).

Recurrence Relations IV Recurrence Relations V

Definition Definition

The initial conditions specify the value of the first few necessary
More generally, recurrences can have the form terms in the sequence. In the Fibonacci numbers we needed two
initial conditions, F (0) = F (1) = 1 since F (n) was defined by the
T (n) = αT (n − β) + f (n), T (δ) = c two previous terms in the sequence.
Initial conditions are also known as boundary conditions (as
or  
n opposed to the general conditions).
T (n) = αT + f (n), T (δ) = c
β From now on, we will use the subscript notation, so the Fibonacci
numbers are
Note that it may be necessary to define several T (δ), initial fn = fn−1 + fn−2
conditions. f1 = 1
f0 = 1

Recurrence Relations VI Solving Recurrences

Definition

Recurrence relations have two parts: recursive terms and

non-recursive terms.
There are several methods for solving recurrences.
2
T (n) = 2T (n − 2) + n − 10}
| {z } | {z I Characteristic Equations
recursive non-recrusive
I Forward Substitution
Recursive terms come from when an algorithm calls itself. I Backward Substitution
I Recurrence Trees
Non-recursive terms correspond to the “non-recursive” cost of the
algorithm—work the algorithm performs within a function. I Maple!

We’ll see some examples later. First, we need to know how to

solve recurrences.
Linear Homogeneous Recurrences Linear Homogeneous Recurrences
Examples

Definition

A linear homogeneous recurrence relation of degree k with Examples

constant coefficients is a recurrence relation of the form
The Fibonacci sequence is a linear homogeneous recurrence
an = c1 an−1 + c2 an−2 + · · · + ck an−k relation. As are the following.

with c1 , . . . , ck ∈ R, ck 6= 0. an = 4an−1 + 5an−2 + 7an−3

an = 2an−2 + 4an−4 + 8an−8

I Linear: RHS is a sum of multiples of previous terms of the
How many initial conditions do we need to specify for these? As
sequence (linear combination of previous terms). The
many as the degree, k = 3, 8 respectively.
coefficients are all constants (not functions depending on n).
I Homogeneous: no terms occur that are not multiples of the
So, how do we solve linear homogeneous recurrences?
aj ’s.
I Degree k: an is expressed in terms of k terms of the sequence.

Solving Linear Homogeneous Recurrences I Solving Linear Homogeneous Recurrences II

We want a solution of the form an = rn where r is some (real)

constant.
We observe that an = rn is a solution to a linear homogeneous rk − c1 rk−1 − c2 rk−2 − · · · − ck−1 r − ck = 0
recurrence if and only if
This is called the characteristic equation of the recurrence relation.
rn = c1 rn−1 + c2 rn−2 + · · · + ck rn−k
The roots of this polynomial are called the characteristic roots of
the recurrence relation. They can be used to find solutions (if they
We can now divide both sides by rn−k , collect terms, and we get a exist) to the recurrence relation. We will consider several cases.
k-degree polynomial.

rk − c1 rk−1 − c2 rk−2 − · · · − ck−1 r − ck = 0

Second Order Linear Homogeneous Recurrences Second Order Linear Homogeneous Recurrences
Example
A second order linear homogeneous recurrence is a recurrence of
the form Example
an = c1 an−1 + c2 an−2
Find a solution to
an = 5an−1 − 6an−2
Theorem (Theorem 1, p462)
with initial conditions a0 = 1, a1 = 4
r2
Let c1 , c2 ∈ R and suppose that − c1 r − c2 = 0 is the
characteristic polynomial of a 2nd order linear homogeneous I The characteristic polynomial is
recurrence which has two distinct1 roots, r1 , r2 .
r2 − 5r + 6
Then {an } is a solution if and only if
I Using the quadratic formula (or common sense), the root can
an = α1 r1n + α2 r2n be found;
r2 − 5r + 6 = (r − 2)(r − 3)
for n = 0, 1, 2, . . . where α1 , α2 are constants dependent upon the
initial conditions. so r1 = 2, r2 = 3
1
we discuss how to handle this situation later.
Second Order Linear Homogeneous Recurrences Second Order Linear Homogeneous Recurrences
Example Continued Example Continued

I Using the 2nd-order theorem, we have a solution,

I Solving for α1 = (1 − α2 ) in (1), we can plug it into the
second.
an = α1 (2n ) + α2 (3n ) 4 = 2α1 + 3α2
4 = 2(1 − α2 ) + 3α2
I Now we can plug in the two initial conditions to get a system 4 = 2 − 2α2 + 3α2
of linear equations. 2 = α2
I Substituting back into (1), we get
a0 = α1 (2)0 + α2 (3)0
a1 = α1 (2)1 + α2 (3)1 α1 = −1

I Putting it all back together, we have

1 = α1 + α2 (1)
4 = 2α1 + 3α2 (2) an = α1 (2n ) + α2 (3n )
= −1 · 2n + 2 · 3n

Second Order Linear Homogeneous Recurrences Single Root Case

Another Example
Recall that we can only apply the first theorem if the roots are
distinct, i.e. r1 6= r2 .
Example
If the roots are not distinct (r1 = r2 ), we say that one
Solve the recurrence characteristic root has multiplicity two. In this case we have to
apply a different theorem.
an = −2an−1 + 15an−2
Theorem (Theorem 2, p464)
with initial conditions a0 = 0, a1 = 1.
Let c1 , c2 ∈ R with c2 6= 0. Suppose that r2 − c1 r − c2 = 0 has
If we did it right, we have only one distinct root, r0 . Then {an } is a solution to
an = c1 an−1 + c2 an−2 if and only if
1 1
an = (3)n − (−5)n
8 8 an = α1 r0n + α2 nr0n
How can we check ourselves?
for n = 0, 1, 2, . . . where α1 , α2 are constants depending upon the
initial conditions.

Single Root Case Single Root Case

Example Example

Example
I By Theorem 2, we have that the solution is of the form
What is the solution to the recurrence relation
an = α1 4n + α2 n4n
an = 8an−1 − 16an−2
I Using the initial conditions, we get a system of equations;
with initial conditions a0 = 1, a1 = 7?
a0 = 1 = α1
a1 = 7 = 4α1 + 4α2
I The characteristic polynomial is
3
I Solving the second, we get that α2 = 4
r2 − 8r + 16 I And so the solution is
I Factoring gives us 3
an = 4n + n4n
4
r2 − 8r + 16 = (r − 4)(r − 4)
I We should check ourselves. . .
so r0 = 4
General Linear Homogeneous Recurrences General Linear Homogeneous Recurrences I
Distinct Roots

Theorem (Theorem 3, p465)

There is a straightforward generalization of these cases to higher Let c1 , . . . , ck ∈ R. Suppose that the characteristic equation
order linear homogeneous recurrences.
rk − c1 rk−1 − · · · − ck−1 r − ck = 0
Essentially, we simply define higher degree polynomials.
The roots of these polynomials lead to a general solution. has k distinct roots, r1 , . . . , rk . Then a sequence {an } is a solution
of the recurrence relation
The general solution contains coefficients that depend only on the
initial conditions. an = c1 an−1 + c2 an−2 + · · · + ck an−k
In the general case, however, the coefficients form a system of if and only if
linear equalities.
an = α1 r1n + α2 r2n + · · · + αk rkn

for n = 0, 1, 2, . . ., where α1 , α2 , . . . , αk are constants.

General Linear Homogeneous Recurrences General Linear Homogeneous Recurrences

Any Multiplicity Any Multiplicity

Theorem (Continued)

Then a sequence {an } is a solution of the recurrence relation

Theorem (Theorem 4, p466)
an = c1 an−1 + c2 an−2 + · · · + ck an−k
Let c1 , . . . , ck ∈ R. Suppose that the characteristic equation
if and only if
rk − c1 rk−1 − · · · − ck−1 r − ck = 0 an = (α1,0 + α1,1 n + · · · + α1,m1 −1 nm1 −1 )r1n +
(α2,0 + α2,1 n + · · · + α2,m2 −1 nm2 −1 )r2n +
has t distinct roots, r1 , . . . , rt with multiplicities m1 , . . . , mt . ..
.
(αt,0 + αt,1 n + · · · + αt,mt −1 nmt −1 )rtn +

for n = 0, 1, 2, . . ., where αi,j are constants for 1 ≤ i ≤ t and

0 ≤ j ≤ mi − 1.

Linear Nonhomogeneous Recurrences Linear Nonhomogeneous Recurrences

For recursive algorithms, cost functions are often not homogenous Here, f (n) represents a non-recursive cost. If we chop it off, we
because there is usually a non-recursive cost depending on the are left with
input size.
an = c1 an−1 + c2 an−2 + · · · + ck an−k
Such a recurrence relation is called a linear nonhomogeneous
recurrence relation. which is the associated homogenous recurrence relation.
Such functions are of the form Every solution of a linear nonhomogeneous recurrence relation is
the sum of a particular solution and a solution to the associated
an = c1 an−1 + c2 an−2 + · · · + ck an−k + f (n)
linear homogeneous recurrence relation.
Linear Nonhomogeneous Recurrences Linear Nonhomogeneous Recurrences

Theorem (Theorem 5, p468)

(p)
If {an } is a particular solution of the nonhomogeneous linear There is no general method for solving such relations. However, we
recurrence relation with constant coefficients can solve them for special cases.
an = c1 an−1 + c2 an−2 + · · · + ck an−k + f (n) In particular, if f (n) is a polynomial or exponential function (or
more precisely, when f (n) is the product of a polynomial and
(p) (h) (h)
then every solution is of the form +{an an },
where {an } is a exponential function), then there is a general solution.
solution of the associated homogenous recurrence relation

an = c1 an−1 + c2 an−2 + · · · + ck an−k

Linear Nonhomogeneous Recurrences Linear Nonhomogeneous Recurrences

Theorem (Continued)
Theorem (Theorem 6, p469)
When s is not a root of the characteristic equation of the
Suppose that {an } satisfies the linear nonhomogeneous recurrence associated linear homogeneous recurrence relation, there is a
relation particular solution of the form
an = c1 an−1 + c2 an−2 + · · · + ck an−k + f (n) (pt nt + pt−1 nt−1 + · · · + p1 n + p0 ) · sn
where c1 , . . . , ck ∈ R and
When s is a root of this characteristic equation and its multiplicity
f (n) = (bt nt + bt−1 nt−1 + · · · + b1 n + b0 ) · sn is m, there is a particular solution of the form

where b0 , . . . , bn , s ∈ R. nm (pt nt + pt−1 nt−1 + · · · + p1 n + p0 ) · sn

Linear Nonhomogeneous Recurrences Other Methods

When analyzing algorithms, linear homogenous recurrences of

order greater than 2 hardly ever arise in practice.

The examples in the text are quite good (see pp467–470) and We briefly describe two “unfolding” methods that work for a lot of
illustrate how to solve simple nonhomogeneous relations. cases.

We may go over more examples if you wish. Backward substitution – this works exactly as its name implies:
starting from the equation itself, work backwards, substituting
Also read up on generating functions in section 7.4 (though we values of the function for previous ones.
may return to this subject).
Recurrence Trees – just as powerful but perhaps more intuitive,
However, there are alternate, more intuitive methods. this method involves mapping out the recurrence tree for an
equation. Starting from the equation, you unfold each recursive
call to the function and calculate the non-recursive cost at each
level of the tree. You then find a general formula for each level and
take a summation over all such levels.
Backward Substitution Backward Substitution
Example Example – Continued
Example If we continue to do this, we get the following.
Give a solution to T (n) = T (n − 2) + 2(n − 1) + 2n
= T (n − 3) + 2(n − 2) + 2(n − 1) + 2n
T (n) = T (n − 1) + 2n
= T (n − 4) + 2(n − 3) + 2(n − 2) + 2(n − 1) + 2n
..
where T (1) = 5. .
= T (n − i) + i−1
P
j=0 2(n − j)
We begin by unfolding the recursion by a simple substitution of the
function values.
Observe that I.e. this is the function’s value at the i-th iteration. Solving the
sum, we get
T (n − 1) = T ((n − 1) − 1) + 2(n − 1) = T (n − 2) + 2(n − 1)
(i − 1)(i − 1 + 1)
T (n) = T (n − i) + 2n(i − 1) − 2 + 2n
2
Substituting this into the original equation gives us
T (n) = T (n − 2) + 2(n − 1) + 2n

Backward Substitution Recurrence Trees

Example – Continued

When using recurrence trees, we graphically represent the

We want to get rid of the recursive term. To do this, we need to recursion.
know at what iteration we reach our base case; i.e. for what value Each node in the tree is an instance of the function. As we
of i can we use the initial condition, T (1) = 5? progress downward, the size of the input decreases.
We can easily see that when i = n − 1, we get the base case. The contribution of each level to the function is equivalent to the
Substituting this into the equation above, we get number of nodes at that level times the non-recursive cost on the
size of the input at that level.
T (n) = T (n − i) + 2n(i − 1) − i2 + i + 2n
The tree ends at the depth at which we reach the base case.
= T (1) + 2n(n − 1 − 1) − (n − 1)2 + (n − 1) + 2n
= 5 + 2n(n − 2) − (n2 − 2n + 1) + (n − 1) + 2n As an example, we consider a recursive function of the form
= n2 + n + 3  
n
T (n) = αT + f (n), T (δ) = c
β

Recurrence Trees Recurrence Trees

Example

Iteration Cost The total value of the function is the summation over all levels of
0 T (n) f (n)
the tree:
logβ n  
X n
T (n) = αi · f
βi
“ ”
T (n/β) ··· α ··· T (n/β) α·f n
1 β i=0

2
„
n
« We consider the following concrete example.
2 T (n/β 2 ) · · · α · · · T (n/β 2 ) T (n/β 2 ) · · · α · · · T (n/β 2 ) α · f
β2

. .
.
.
.
.
Example
„ «
i n
i α ·f
βi

. . n
. .
. . T (n) = 2T + n, T (1) = 4
logβ n α
logβ n
· T (δ)
2
Recurrence Trees Recurrence Trees
Example – Continued Example – Continued

Iteration Cost
0 T (n) n

1 T (n/2) T (n/2) n + n
2 2
The value of the function then, is the summation of the value of
all levels. We treat the last level as a special case since its
non-recursive cost is different.
n
„ «
2 T (n/4) T (n/4) T (n/4) T (n/4) 4·
4

(log2 n)−1
X n
T (n/8) T (n/8) T (n/8) T (n/8) T (n/8) T (n/8) T (n/8) T (n/8) 8·
„
n
«
T (n) = 4n + 2i = n(log n) + 4n
3 8
2i
i=0
. .
. .
. .
0 1

i 2i · @ n A
2i

. .
. .
. .

log2 n 2log2 n · T (1)

Smoothness Rule I How To Cheat With Maple I

In the previous example we make the following assumption: that n Maple and other math tools are great resources. However, they are
was a power of two; n = 2k . This was necessary to get a nice not substitutes for knowing how to solve recurrences yourself.
depth of log n and a full tree. As such, you should only use Maple to check your answers.
We can restrict consideration to certain powers because of the Recurrence relations can be solved using the rsolve command
smoothness rule, which is not studied in this course. For more and giving Maple the proper parameters.
information about the smoothness rule, please consult pages The arguments are essentially a comma-delimited list of equations:
481–483 in the textbook “The Design & Analysis of Algorthims” general and boundary conditions, followed by the “name” and
by Anany Levitin. variable of the function.

How To Cheat With Maple II

> rsolve({T(n) = T(n-1) + 2*n, T(1) = 5}, T(n));

 
1
1 + 2(n + 1) n + 1 − 2n
2

You can clean up Maple’s answer a bit by encapsulating it in the

simplify command:
> simplify(rsolve({T(n) = T(n-1) + 2*n, T(1) = 5},
T(n)));
3 + n2 + n