Master theorem 1

Master theorem
In the analysis of algorithms, the master theorem provides a cookbook solution in asymptotic terms (using Big O
notation) for recurrence relations of types that occur in the analysis of many divide and conquer algorithms. It was
popularized by the canonical algorithms textbook Introduction to Algorithms by Cormen, Leiserson, Rivest, and
Stein, which introduces and proves it in sections 4.3 and 4.4, respectively. Nevertheless, not all recurrence relations
can be solved with the use of the master theorem; its generalizations include the Akra–Bazzi method.

Introduction
Consider a problem that can be solved using a recursive algorithm such as the following:

procedure T( n : size of problem ) defined as:

if n < 1 then exit

Do work of amount f(n)

T(n/b)
T(n/b)
...repeat for a total of a times...
T(n/b)
end procedure

In the above algorithm we are dividing the problem into a number of sub problems recursively, each sub problem
being of size n/b. This can be visualized as building a call tree with each node of the tree as an instance of one
recursive call and its child nodes being instances of subsequent calls. In the above example, each node would have a
number of child nodes. Each node does an amount of work that corresponds to the size of the sub problem n passed
to that instance of the recursive call and given by . For example, if each recursive call is doing a comparison
sort, then the amount of work done by each node in the tree is at least O(n log n). The total amount of work done by
the entire tree is the sum of the work performed by all the nodes in the tree.

Algorithm such as above can be represented as recurrence relationship . This

recursive relationship can be successively substituted in to itself and expanded to obtain expression for total amount
of work done.[1]
The original Master theorem allows to easily calculate run time of such a recursive algorithm in Big O notation
without doing expansion of above recursive relationship. A generalized form of Master Theorem by Akra and Bazzi
introduced in 1998 is applicable on wide number of cases that occur in practice.
Master theorem 2

Generic form
The master theorem concerns recurrence relations of the form:

In the application to the analysis of a recursive algorithm, the constants and function take on the following
significance:
• n is the size of the problem.
• a is the number of subproblems in the recursion.
• n/b is the size of each subproblem. (Here it is assumed that all subproblems are essentially the same size.)
• f (n) is the cost of the work done outside the recursive calls, which includes the cost of dividing the problem and
the cost of merging the solutions to the subproblems.
It is possible to determine an asymptotic tight bound in these three cases:

Case 1

Generic form
If for some constant (using Big O notation)
it follows that:

Example

As one can see in the formula above, the variables get the following values:
, , ,
Now we have to check that the following equation holds:

If we choose = 1, we get:

Since this equation holds, the first case of the master theorem applies to the given recurrence relation, thus resulting
in the conclusion:

If we insert the values from above, we finally get:

Thus the given recurrence relation T(n) was in Θ(n3).

(This result is confirmed by the exact solution of the recurrence relation, which is ,
assuming ).
Master theorem 3

Case 2

Generic form
If it is true, for some constant k ≥ 0, that:

it follows that:

Example

As we can see in the formula above the variables get the following values:
, , , ,
Now we have to check that the following equation holds (in this case k=0):

If we insert the values from above, we get:

Since this equation holds, the second case of the master theorem applies to the given recurrence relation, thus
resulting in the conclusion:

If we insert the values from above, we finally get:

Thus the given recurrence relation T(n) was in Θ(n log n).
(This result is confirmed by the exact solution of the recurrence relation, which is ,
assuming .)

Case 3

Generic form
If it is true that:

for some constant

and if it is also true that:

for some constant and sufficiently large n

it follows that:
Master theorem 4

Example

As we can see in the formula above the variables get the following values:

Now we have to check that the following equation holds:

If we insert the values from above, and choose = 1, we get:

Since this equation holds, we have to check the second condition, namely if it is true that:

If we insert once more the values from above, we get the number :

If we choose , it is true that:

So it follows:

If we insert once more the necessary values, we get:

Thus the given recurrence relation T(n) was in Θ(n2), that complies with the f (n) of the original formula.
(This result is confirmed by the exact solution of the recurrence relation, which is , assuming
.)

Inadmissible equations
The following equations cannot be solved using the master theorem:[2]

a is not a constant

•
(See Below)
non-polynomial difference between f(n) and

a<1 cannot have less than one sub problem

f(n) is not positive

•
Master theorem 5

case 3 but regularity violation.

In the second inadmissible example above, the difference between and can be expressed with the

ratio . It is clear that for any constant . Therefore,

the difference is not polynomial and the Master Theorem does not apply.

Application to common algorithms

Algorithm Recurrence Relationship Run time Comment

Binary search [3]

Apply Master theorem where

Binary tree traversal [3]

Apply Master theorem where

Optimal Sorted Matrix Apply Akra-Bazzi theorem for and to

Search get
Merge Sort

Notes
[1] Duke University, "Big-Oh for Recursive Functions: Recurrence Relations", http:/ / www. cs. duke. edu/ ~ola/ ap/ recurrence. html
[2] Massachusetts Institute of Technology (MIT), "Master Theorem: Practice Problems and Solutions", http:/ / www. csail. mit. edu/ ~thies/ 6.
046-web/ master. pdf
[3] Dartmouth College, http:/ / www. math. dartmouth. edu/ archive/ m19w03/ public_html/ Section5-2. pdf

References
• Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. Introduction to Algorithms,
Second Edition. MIT Press and McGraw-Hill, 2001. ISBN 0-262-03293-7. Sections 4.3 (The master method) and
4.4 (Proof of the master theorem), pp. 73–90.
• Michael T. Goodrich and Roberto Tamassia. Algorithm Design: Foundation, Analysis, and Internet Examples.
Wiley, 2002. ISBN 0-471-38365-1. The master theorem (including the version of Case 2 included here, which is
stronger than the one from CLRS) is on pp. 268–270.
Article Sources and Contributors 6

Article Sources and Contributors

Master theorem  Source: http://en.wikipedia.org/w/index.php?oldid=435709000  Contributors: Alberto da Calvairate, AySz88, BeteNoir, Charles Matthews, Chris-gore, Cybercobra, David
Eppstein, Dcoetzee, Dead3y3, Diegusjaimes, Eric119, FauxFaux, Giftlite, Hairy Dude, Hariva, Igorpak, Jleedev, Jogloran, Justin W Smith, Kainaw, Kaisenl, Karl Dickman, Kenyon, Krappie,
LOL, Magioladitis, Mathiasl26, Michael Hardy, Mobius, Mpercy, Njsg, Oliphaunt, Qutezuce, Ratfox, Regnaron, Rspeer, Seftembr, Sligocki, Snowolf, Svick, Sytelus, Tachikoma's All Memory,
TakuyaMurata, TedPavlic, Unreal128, Vedant, Wazow, WillNess, Xvani, Yuval madar, Zahd, Zeroin23a, 124 anonymous edits

License
Creative Commons Attribution-Share Alike 3.0 Unported
http:/ / creativecommons. org/ licenses/ by-sa/ 3. 0/