HG2NLA Numerical Linear Algebra 6–1

6 The power method

6.1 Description of the method

6.1.1 Introduction
The standard method for finding the eigenvalues of a matrix A is to solve for the roots
λ of
|A − λI| = 0.
This however is totally impracticable when A is large. Even evaluating the determinant
of an n × n matrix is a huge task when n is large, and on top of that we would have to
solve the resulting nth -degree polynomial equation for λ.
The power method provides a very straightforward way of computing one eigenvalue
(usually the largest) λ, and corresponding eigenvector e, of a matrix. It is an iter-
ative method, in which we start with an initial guess x0 and generate a sequence of
approximations xk which hopefully converges as k → ∞.

6.1.2 The algorithm

The algorithm for the power method is very simple.

• Choose initial guess x0 .

• Do k = 1 to m.

• Set y k = Axk−1 .
• Set αk = the largest element of y k (in absolute value).
• Set xk = y k /αk .

• End do.

Here the iterative step is performed a total of m times. At each step we premultiply the
previous approximation xk−1 by A. The next approximation xk is obtained by dividing
the resulting vector y k by its absolutely largest element, which we label αk . (Thus for
each k > 0, xk has the property that its largest element is equal to one.) If the method
is working, then xk → e and αk → λ as k → ∞, where Ae = λe.
This algorithm is particularly easy to implement in FORTRAN90 using the built-in
matrix multiplication functions.
6–2 School of Mathematical Sciences University of Nottingham

6.2 Convergence criteria

Suppose for the moment that A has a complete set of eigenvectors e1 , . . . , en , with
corresponding eigenvalues λ1 , . . . , λn . Without loss of generality we can arrange these
in decreasing order of size, that is

|λ1 | ≥ |λ2 | ≥ . . . ≥ |λn |.

Furthermore we choose to scale the eigenvectors ei such that the absolutely largest
element of each is equal to one. (To do this we simply divide each ei by its largest
element.)
Now we decompose the initial guess x0 into a linear combination of the eigenvectors
of A:
X n
x0 = ci ei .
i=1
Applying the power method algorithm to this vector,
n
X
y 1 = Ax0 = ci λi ei ,
i=1

we obtain
n
1 X
x1 = y 1 /α1 = ci λi ei .
α1
i=1
After k iterations, we have
n
1 X
xk = ci λki ei
α1 α2 . . . αk
i=1
n
(   )
k
c1 λ1 X ci λi k
= e1 + ei . (6.1)
α1 α2 . . . αk c1 λ1
i=2

Now the idea is that since λi is the largest eigenvalue, all the terms apart from
the first in the expression in braces tend to zero as k → ∞. Thus the eigenvector e1
dominates the behaviour of xk for large k. Notice that this will fail if there is another
eigenvalue of the same size as λ1 , that is if |λ2 | = |λ1 |. Therefore we assume that λ1 is
bigger than any other eigenvalue of A:

|λ1 | > |λi | for all i = 2, 3, . . . , n.

Then we can assert that  k

λi
→ 0 as k → ∞,
λ1
for all i > 1, and thus
c1 λk1
xk ∼ e1 as k → ∞.
α1 α2 . . . αk
HG2NLA Numerical Linear Algebra 6–3

But we know that xk has the property that its largest element is always equal to one,
as does e1 . Therefore the constant of proportionality between xk and e1 must approach
unity for large k, that is
xk → e1 as k → ∞,

and
c1 λk1
→ 1 as k → ∞.
α1 α2 . . . αk
This can only occur if
αk → λ1 as k → ∞.

In practice, a good test for convergence is the difference between successive values
of αk , since these get closer together as αk → λ1 . Thus one would typically continue
iterating until |αk − αk−1 | < δ where δ is some prescribed small number (“accuracy
goal”).

6.3 When can the power method fail?

We have shown that, in principle, the approximations xk approach the eigenvector e1
while the scaling factors αk tend to the corresponding eigenvalue λ1 as the number of
iterations k increases, where λ1 is the absolutely largest eigenvalue of A. As pointed
out previously, this can fail if there is not a single largest eigenvalue: if |λ2 | = |λ1 | then
in general the power method alogorithm does not converge at all. Since we do not know
the eigenvalues of A in advance — otherwise there would be no point in using the power
method! — we cannot know in advance whether the power method will work or not.
Recall that the eigenvalues of a real matrix A are in general complex, and occur
in conjugate pairs (which necessarily have the same absolute value). So if the largest
eigenvalue of A is not real (and there is no reason why it should be) then the power
method will certainly fail. For this reason it is sensible to apply the power method only
to matrices whose eigenvalues are known to be real, for example symmetric matrices.
For such matrices we could consider ourselves unlucky if the two largest eigenvalues
happened to have the same absolute value, i.e. if λ2 = ±λ1 .
On a similar note, from (6.1) we see that when the power method does converge, the
rate at which it does so is determined by the ratio |λ2 /λ1 |. Thus if |λ2 | is only slightly
smaller than |λ1 |, then the method converges very slowly, so that a large number of
iterations will be required to obtain a decent accuracy.
Another point worth making is that it is not strictly true that the power method
necessarily converges to the largest eigenvalue λ1 . This would fail in the exceptional
case c1 = 0, that is if the initial guess x0 contains no component of the eigenvector
e1 corresponding to the largest eigenvalue λ1 . In this case, the method will converge
to the largest eigenvalue whose eigenvector is contained in the decomposition of x0 . If
there are two such eigenvalues with the same absolute value, then the method will not
converge at all.
6–4 School of Mathematical Sciences University of Nottingham

Of course it is extremely unlikely in general for c1 to be zero. In practice we can

help to avoid this unfortunate possibility by filling x0 with random entries rather than
something obvious like x0 = (1, . . . , 1)T .

6.4 Extensions to the power method

We have shown how to use the power method to find a single eigenvalue (generically the
largest) and corresponding eigenvector of a matrix A. In this section we will examine
methods whereby more than one eigenvalue of A can be found using variations on the
power method.

6.4.1 The inverse power method

This method operates with A−1 rather than A. Thus we replace the step y k = Axk−1
with y k = A−1 xk−1 or Ay k = xk−1 . Since the eigenvalues of A−1 are 1/λn , . . . , 1/λ1
[exercise], the inverse power method should converge to 1/λn — the largest eigenvalue
of A−1 . Thus this method gives a way of finding the smallest (in absolute value)
eigenvalue of a matrix.

6.4.2 Spectral shift

This method uses the fact that the eigenvalues of A − aI are λ1 − a, . . . , λn − a
[exercise]. Thus having computed one eigenvalue λ1 we start the method over again
using the shifted matrix A − λ1 I. This reduces the eigenvalue we’ve already determined
to zero, and the method now converges to the largest in absolute value of λ2 − λ1 , . . . ,
λn − λ 1 .

6.4.3 Deflation
A spectral shift always enables us to find at least one more eigenvalue than the largest
λ1 . However it is not clear how it could be implemented in general to find all the
eigenvalues of an n × n matrix A. The method of deflation works only for symmetric
matrices, but is somewhat more subtle since it shifts only one eigenvalue at a time to
zero, leaving the others unchanged.
Suppose we have used the power method to determine one eigenvalue λ1 and eigen-
vector e1 of a symmetric matrix A. Now consider the matrix

e1 eT1
A 1 = A − λ1 .
eT1 e1

For any eigenvector ei of A, A1 satisfies

eT1 ei
 
A1 ei = Aei − λ1 e1 .
eT1 e1
HG2NLA Numerical Linear Algebra 6–5

When i = 1 we therefore have

A1 e1 = Ae1 − λ1 e1 = 0,

while for i > 1, since the eigenvectors of a symmetric matrix are orthogonal (i.e. eT1 ei =
0),
A1 ei = Aei = λi ei .
Thus the eigenvalues of A1 are λ2 , . . . , λn , 0, and the power method will pick out the
second largest (in absolute value) eigenvalue λ2 .
Clearly this method can be continued by setting

e2 eT2 ei eTi
A2 = A1 − λ2 , ... Ai = Ai−1 − λi .
eT2 e2 eTi ei

In each case we shift the largest eigenvalue λi discovered thus far to zero, and therefore
the power method applied to Ai produces the largest remaining eigenvalue λi+1 . Thus we
can generate the eigenvalues of A one at a time in reverse order of absolute magnitude.
As for the standard power method, the built-in matrix manipulation subroutines in
FORTRAN90 are ideal for carrying out deflation.

Exercises
6.1. (a) Describe, e.g. by means of a flow chart or a list of instructions, how the power
method may be used to find one eigenvalue and corresponding eigenvector of
a matrix M .
(b) Show that, if the power method converges, in general it does so to the eigen-
value with largest magnitude.
Under what circumstances does the power method not converge?
Why is this unlikely to occur if M is symmetric?
(c) Suppose an n × n symmetric matrix M has eigenvalues λ1 , . . . , λn and cor-
responding eigenvectors e1 , . . . , en . Find the eigenvalues and eigenvectors of
the modified matrix
e1 eT
M 1 = M − λ1 T 1 .
e1 e1

(d) Thus describe how the method of deflation may be used in conjunction with
the power method to find all the eigenvalues and eigenvectors of a symmetric
matrix.

6.2. [1998/99 Question 3]

(a) Describe briefly the power method for determining an eigenvalue and corre-
sponding eigenvector of a matrix M .
6–6 School of Mathematical Sciences University of Nottingham

(b) Find algebraically the eigenvalues of the matrix

 
2 −2
M= .
0 1

Compute four steps of the power method on M starting with the initial guess
x = (1, 1)T for the eigenvector.
Which of the eigenvalues is the power method approaching, and why?
(c) Explain how a spectral shift can be used to obtain further eigenvalues using
the power method.
Apply an appropriate spectral shift to the matrix M and then use the power
method to find the second eigenvalue of M .

6.3. Write a computer program in FORTRAN90 to find the eigenvalues of a real sym-
metric matrix using the power method and deflation.
Test your program on the matrix
 
3 2 1
A =  2 3 2 .
1 2 3