Contents 22

Eigenvalues and
Eigenvectors
22.1 Basic Concepts 2

22.2 Applications of Eigenvalues and Eigenvectors 18

22.3 Repeated Eigenvalues and Symmetric Matrices 30

22.4 Numerical Determination of Eigenvalues and Eigenvectors 46

Learning outcomes
In this Workbook you will learn about the matrix eigenvalue problem AX = kX where A
is a square matrix and k is a scalar (number). You will learn how to determine the
eigenvalues (k) and corresponding eigenvectors (X) for a given matrix A. You will learn
of some of the applications of eigenvalues and eigenvectors. Finally you will learn how
eigenvalues and eigenvectors may be determined numerically.
  

Basic Concepts 22.1 

Introduction
From an applications viewpoint, eigenvalue problems are probably the most important problems that
arise in connection with matrix analysis. In this Section we discuss the basic concepts. We shall see
that eigenvalues and eigenvectors are associated with square matrices of order n × n. If n is small
(2 or 3), determining eigenvalues is a fairly straightforward process (requiring the solutiuon of a low
order polynomial equation). Obtaining eigenvectors is a little strange initially and it will help if you
read this preliminary Section first.

#
• have a knowledge of determinants and
matrices
Prerequisites
• have a knowledge of linear first order
Before starting this Section you should . . .
differential equations
"
# !
• obtain eigenvalues and eigenvectors of 2 × 2
and 3 × 3 matrices
Learning Outcomes
• state basic properties of eigenvalues and
On completion you should be able to . . .
eigenvectors
" !

2 HELM (2008):
Workbook 22: Eigenvalues and Eigenvectors
 ®

1. Basic concepts

Determinants
A square matrix possesses an associated determinant. Unlike a matrix, which is an array of numbers,
a determinant has a single value.
 
c11 c12
A two by two matrix C = has an associated determinant
c21 c22
 
 c11 c12 
det (C) =   = c11 c22 − c21 c12
c21 c22 
(Note square or round brackets denote a matrix, straight vertical lines denote a determinant.)
A three by three matrix has an associated determinant
 
 c11 c12 c13 
 
det(C) =  c21 c22 c23 
 c31 c32 c33 

Among other ways this determinant can be evaluated by an “expansion about the top row”:
     
 c22 c23   c21 c23   c21 c22 
det(C) = c11 
  − c12 
  + c13 
 
c32 c33  c31 c33  c31 c32 
Note the minus sign in the second term.

Task
Evaluate the determinants
 
     6 5 4 
 4 6   4 8   
det(A) =   det(B) =   det(C) =  2 −1 7
 
3 1   1 2 
 −3

2 0 

Your solution

Answer
det A = 4 × 1 − 6 × 3 = −14 det B = 4 × 2 − 8 × 1 = 0
     
 −1 7   2 7   2 −1 
det C = 6 
  − 5
  + 4  = 6 × (−14) − 5(21) + 4(4 − 3) = −185
2 0  −3 0  −3 2 
 
4 8
A matrix such as B = in the previous task which has zero determinant is called a singular
1 2
matrix. The other two matrices A and C are non-singular. The key factor to be aware of is as
follows:

HELM (2008): 3
Section 22.1: Basic Concepts
 Key Point 1
Any non-singular n × n matrix C, for which det(C) 6= 0, possesses an inverse C −1 i.e.
CC −1 = C −1 C = I where I denotes the n × n identity matrix
A singular matrix does not possess an inverse.

Systems of linear equations

We first recall some basic results in linear (matrix) algebra. Consider a system of n equations in n
unknowns x1 , x2 , . . . , xn :
c11 x1 + c12 x2 + ... + c1n xn = k1
c21 x1 + c22 x2 + ... + c2n xn = k2
.. .. .. .
. + . + ... + . = ..
cn1 x1 + cn2 x2 + ... + cnn xn = kn
We can write such a system in matrix form:
    
c11 c12 . . . c1n x1 k1
 c21 c22 . . . c2n   x2   k2 
   
..   ..  =  ..  , or equivalently CX = K.
 
 .. ..
 . . ... .   .   .
cn1 cn2 . . . cnn xn kn
We see that C is an n × n matrix (called the coefficient matrix), X = {x1 , x2 , . . . , xn }T is the n × 1
column vector of unknowns and K = {k1 , k2 , . . . , kn }T is an n × 1 column vector of given constants.
The zero matrix will be denoted by O.
If K 6= O the system is called inhomogeneous; if K = O the system is called homogeneous.

Basic results in linear algebra

Consider the system of equations CX = K.
We are concerned with the nature of the solutions (if any) of this system. We shall see that this
system only exhibits three solution types:
• The system is consistent and has a unique solution for X
• The system is consistent and has an infinite number of solutions for X
• The system is inconsistent and has no solution for X

4 HELM (2008):
Workbook 22: Eigenvalues and Eigenvectors
 ®

There are two basic cases to consider:

det(C) 6= 0 or det(C) = 0
Case 1: det(C) 6= 0
In this case C −1 exists and the unique solution to CX = K is
X = C −1 K
Case 2: det(C) = 0
In this case C −1 does not exist.

(a) If K 6= O the system CA = K has no solutions.

(b) If K = O the system CX = O has an infinite number of solutions.

We note that a homogeneous system

CX = O
has a unique solution X = O if det(C) 6= 0 (this is called the trivial solution) or an infinite number
of solutions if det(C) = 0.

Example 1
(Case 1) Solve the inhomogeneous system of equations
x1 + x2 = 1 2x1 + x2 = 2
which can be expressed as CX = K where
     
1 1 x1 1
C= X= K=
2 1 x2 2

Solution
Here det(C) = −1 6= 0.
   
x1 1
The system of equations has the unique solution: X= = .
x2 0

HELM (2008): 5
Section 22.1: Basic Concepts
 Example 2
(Case 2a) Examine the following inhomogeneous system for solutions
x1 + 2x2 = 1
3x1 + 6x2 = 0

Solution
 
1 2
Here det (C) =   = 0. In this case there are no solutions.
3 6
To see this we see the first equation of the system states x1 + 2x2 = 1 whereas the second equation
(after dividing through by 3) states x1 + 2x2 = 0, a contradiction.

Example 3
(Case 2b) Solve the homogeneous system
x1 + x2 = 0
2x1 + 2x2 = 0

Solution
 
 1 1 
Here det(C) =   = 0. The solutions are any pairs of numbers {x1 , x2 } such that x1 = −x2 ,
 2 2


α
i.e. X= where α is arbitrary.
−α
There are an infinite number of solutions.

A simple eigenvalue problem

We shall be interested in simultaneous equations of the form:
AX = λX,
where A is an n × n matrix, X is an n × 1 column vector and λ is a scalar (a constant) and, in the
first instance, we examine some simple examples to gain experience of solving problems of this type.

6 HELM (2008):
Workbook 22: Eigenvalues and Eigenvectors
 ®

Example 4
Consider the following system with n = 2:

2x + 3y = λx
3x + 2y = λy

so that
   
2 3 x
A= and X= .
3 2 y
It appears that there are three unknowns x, y, λ. The obvious questions to ask
are: can we find x, y? what is λ?

Solution
To solve this problem we firstly re-arrange the equations (take all unknowns onto one side);
(2 − λ)x + 3y = 0 (1)
3x + (2 − λ)y = 0 (2)
Therefore, from equation (2):
(2 − λ)
x=− y. (3)
3
Then when we substitute this into (1)
(2 − λ)2
− y + 3y = 0 which simplifies to [−(2 − λ)2 + 9] y = 0.
3
We conclude that either y = 0 or 9 = (2 − λ)2 . There are thus two cases to consider:
Case 1
If y = 0 then x = 0 (from (3)) and we get the trivial solution. (We could have guessed this
solution at the outset.)
Case 2
9 = (2 − λ)2
which gives, on taking square roots:
±3 = 2 − λ giving λ = 2 ± 3 so λ=5 or λ = −1.

Now, from equation (3), if λ = 5 then x = +y and if λ = −1 then x = −y.

We have now completed the analysis. We have found values for λ but we also see that we cannot
obtain unique values for x and y: all we can find is the ratio between these quantities. This behaviour
is typical, as we shall now see, of an eigenvalue problem.

HELM (2008): 7
Section 22.1: Basic Concepts
2. General eigenvalue problems
Consider a given square matrix A. If X is a column vector and λ is a scalar (a number) then the
relation.
AX = λX (4)
is called an eigenvalue problem. Our purpose is to carry out an analysis of this equation in a
manner similar to the example above. However, we will attempt a more general approach which will
apply to all problems of this kind.
Firstly, we can spot an obvious solution (for X) to these equations. The solution X = 0 is a
possibility (for then both sides are zero). We will not be interested in these trivial solutions of the
eigenvalue problem. Our main interest will be in the occurrence of non-trivial solutions for X.
These may exist for special values of λ, called the eigenvalues of the matrix A. We proceed as in
the previous example:
take all unknowns to one side:
(A − λI)X = 0 (5)
where I is a unit matrix with the same dimensions as A. (Note that AX − λX = 0 does not
simplify to (A − λ)X = 0 as you cannot subtract a scalar λ from a matrix A). This equation (5)
is a homogeneous system of equations. In the notation of the earlier discussion C ≡ A − λI and
K ≡ 0. For such a system we know that non-trivial solutions will only exist if the determinant of the
coefficient matrix is zero:
det(A − λI) = 0 (6)
Equation (6) is called the characteristic equation of the eigenvalue problem. We see that the
characteristic equation only involves one unknown λ. The characteristic equation is generally a
polynomial in λ, with degree being the same as the order of A (so if A is 2 × 2 the characteristic
equation is a quadratic, if A is a 3 × 3 it is a cubic equation, and so on). For each value of λ that is
obtained the corresponding value of X is obtained by solving the original equations (4). These X’s
are called eigenvectors.
N.B. We shall see that eigenvectors are only unique up to a multiplicative factor: i.e. if X satisfies
AX = λX then so does kX when k is any constant.

8 HELM (2008):
Workbook 22: Eigenvalues and Eigenvectors
 ®

Example 5  
1 0
Find the eigenvalues and eigenvectors of the matrix A=
1 2

Solution
The eigenvalues and eigenvectors are found by solving the eigenvalue probelm
 
x
AX = λX X= i.e. (A − λI)X = 0.
y
Non-trivial solutions will exist if det (A − λI) = 0
     
1 0 1 0  1−λ 0 
that is, det −λ = 0, ∴   = 0,
1 2 0 1  1 2−λ 
expanding this determinant: (1 − λ)(2 − λ) = 0. Hence the solutions for λ are: λ = 1 and λ = 2.
So we have found two values of λ for this 2 × 2 matrix A. Since these are unequal they are said to
be distinct eigenvalues.
To each value of λ there corresponds an eigenvector. We now proceed to find the eigenvectors.

Case 1
λ = 1 (smaller eigenvalue). Then our original eigenvalue problem becomes: AX = X. In full this
is

x = x
x + 2y = y

Simplifying

x = x (a)
x+y = 0 (b)
 
x
All we can deduce here is that x = −y ∴ X= for any x 6= 0
−x
(We specify x 6= 0 as, otherwise, we would have the trivial solution.)
   
1 2
So the eigenvectors corresponding to eigenvalue λ = 1 are all proportional to , e.g. ,
  −1 −2
−1
etc.
1
Sometimes we write the eigenvector in normalised form that is, with modulus or magnitude 1.
Here, the normalised form of X is
 
1 1
√ which is unique.
2 −1

HELM (2008): 9
Section 22.1: Basic Concepts
 Solution (contd.)
Case 2 Now we consider the larger eigenvalue λ = 2. Our original eigenvalue problem AX = λX
becomes AX = 2X which gives the following equations:
    
1 0 x x
=2
1 2 y y
i.e.

x = 2x
x + 2y = 2y

These equations imply that x = 0 whilst the variable y may take any value whatsoever (except zero
as this gives the trivial solution).
     
0 0 0
Thus the eigenvector corresponding to eigenvalue λ = 2 has the form , e.g. , etc.
  y 1 2
0
The normalised eigenvector here is .
1
 
1 0
In conclusion: the matrix A = has two eigenvalues and two associated normalised eigen-
1 2
vectors:
λ1 = 1, λ2 = 2
   
1 1 0
X1 = √ X2 =
2 −1 1

Example 6
Find the eigenvalues and eigenvectors of the 3 × 3 matrix
 
2 −1 0
A =  −1 2 −1 
0 −1 2

Solution
The eigenvalues and eigenvectors are found by solving the eigenvalue problem
 
x
AX = λX X= y 
z
Proceeding as in Example 5:
(A − λI)X = 0 and non-trivial solutions for X will exist if det (A − λI) = 0

10 HELM (2008):
Workbook 22: Eigenvalues and Eigenvectors
 ®

Solution (contd.)
that is,
   
 2 −1 0 1 0 0 
det  −1 2 −1  − λ  0 1 0  = 0
0 −1 2 0 0 1
 

 
 2 − λ −1 0 
 
i.e.  −1 2 − λ −1  = 0.
 
 0 −1 2 − λ 
Expanding this determinant we find:
   
 2 − λ −1   −1 −1 
(2 − λ)  + =0
−1 2 − λ   0 2 − λ 
that is,
(2 − λ) {(2 − λ)2 − 1} − (2 − λ) = 0
Taking out the common factor (2 − λ):
(2 − λ) {4 − 4λ + λ2 − 1 − 1}
which gives: (2 − λ) [λ2 − 4λ + 2] = 0.
√
4± 16 − 8 √
This is easily solved to give: λ = 2 or λ = = 2 ± 2.
2
So (typically) we have found three possible values of λ for this 3 × 3 matrix A.
To each value of λ there corresponds an eigenvector.
√
Case 1: λ = 2 − 2 (lowest eigenvalue)
√
Then AX = (2 − 2)X implies
√
2x − y = (2 − 2)x
√
−x + 2y − z = (2 − 2)y
√
−y + 2z = (2 − 2)z

Simplifying
√
2x − y = 0 (a)
√
−x + 2y − z = 0 (b)
√
−y + 2z = 0 (c)

We conclude the following:

√ √
(c) ⇒ y = 2z (a) ⇒ y = 2x
∴ these two relations give x = z then (b) ⇒ −x + 2x − x = 0
The last equation gives us no information; it simply states that 0 = 0.

HELM (2008): 11
Section 22.1: Basic Concepts
 Solution (contd.)
 
√ x
∴ X =  2x  for any x 6= 0 (otherwise we would have the trivial solution). So the
x
 
√ √1
eigenvectors corresponding to eigenvalue λ = 2 − 2 are all proportional to  2 .
1
 
1
1 √ 
In normalised form we have an eigenvector 2 .
2
1
Case 2: λ = 2
    
2 −1 0 x x
Here AX = 2X implies  −1 2 −1   y  = 2  y 
0 −1 2 z z
i.e.

2x − y = 2x
−x + 2y − z = 2y
−y + 2z = 2z

After simplifying the equations become:

−y = 0 (a)
−x − z = 0 (b)
−y = 0 (c)

(a), (c) imply y = 0: (b) implies x = −z

 
x
∴ eigenvector has the form  0  for any x 6= 0.
−x
 
1
That is, eigenvectors corresponding to λ = 2 are all proportional to  0 .
−1
 
1
1
In normalised form we have an eigenvector √  0  .
2 −1

12 HELM (2008):
Workbook 22: Eigenvalues and Eigenvectors
 ®

Solution (contd.)
√
Case 3: λ = 2 + 2 (largest eigenvalue)
Proceeding along similar lines to cases
 1,2 above
 we find that the eigenvectors
 corresponding
 to
√ 1
√ 1
√
1
λ = 2 + 2 are each proportional to  − 2  with normalised eigenvector  − 2  .
2
1 1
In conclusion the matrix A has three distinct eigenvalues:
√ √
λ1 = 2 − 2, λ2 = 2 λ3 = 2 + 2
and three corresponding normalised eigenvectors:
     
1 1 1
1 √ 1 1 √
X1 =  2  , X2 = √  0  , X3 =  − 2 
2 2 −1 2
1 1

Exercise
Find the eigenvalues and eigenvectors of each of the following matrices A:
   
    2 0 −2 10 −2 4
4 −2 1 2
(a) (b) (c)  0 4 0 (d) −20 4 −10
1 1 −8 11
−2 0 5 −30 6 −13
Answer (eigenvectors are written in normalised form)
 √   √ 
2/√5 1/√2
(a) 3 and 2; and
1/ 5 1/ 2
   
1 1 1 1
(b) 3 and 9; √ and √
2 1 17 4
     
2 0 1
1     1  
(c) 1, 4 and 6; √ 0 ; 1 ; √ 0
5 1 0 5 −2
     
1 0 1
1   1   1  
(d) 0, −1 and 2; √ 5 ; √ 2 ; √ 0
26 0 5 1 5 −2

HELM (2008): 13
Section 22.1: Basic Concepts
3. Properties of eigenvalues and eigenvectors
There are a number of general properties of eigenvalues and eigenvectors which you should be familiar
with. You will be able to use them as a check on some of your calculations.
Property 1: Sum of eigenvalues
For any square matrix A:

sum of eigenvalues = sum of diagonal terms of A (called the trace of A)

n
X
Formally, for an n × n matrix A: λi = trace(A)
i=1

(Repeated eigenvalues must be counted according to their multiplicity.)

3
X
Thus if λ1 = 4, λ2 = 4, λ3 = 1 then λi = 9).
i=1

Property 2: Product of eigenvalues

For any square matrix A:

product of eigenvalues = determinant of A

n
Y
Formally: λ 1 λ 2 λ3 · · · λ n = λi = det(A)
i=1
Q P
The symbol simply denotes multiplication, as denotes summation.

Example 7
Verify Properties 1 and 2 for the 3 × 3 matrix:
 
2 −1 0
A =  −1 2 −1 
0 −1 2
whose eigenvalues were found earlier.

Solution
The three eigenvalues of this matrix are:
√ √
λ1 = 2 − 2, λ2 = 2, λ3 = 2 + 2
Therefore
√ √
λ1 + λ2 + λ3 = (2 − 2) + 2 + (2 + 2) = 6 = trace(A)
√ √
whilst λ1 λ2 λ3 = (2 − 2)(2)(2 + 2) = 4 = det(A)

14 HELM (2008):
Workbook 22: Eigenvalues and Eigenvectors
 ®

Property 3: Linear independence of eigenvectors

Eigenvectors of a matrix A corresponding to distinct eigenvalues are linearly independent i.e. one
eigenvector cannot be written as a linear sum of the other eigenvectors. The proof of this result is
omitted but we illustrate this property with two examples.
We saw earlier that the matrix
 
1 0
A=
1 2
has distinct eigenvalues λ1 = 1 λ2 = 2 with associated eigenvectors
   
1 1 0
X (1) = √ X (2) =
2 −1 1
respectively.
Clearly X (1) is not a constant multiple of X (2) and these eigenvectors are linearly independent.
We also saw that the 3 × 3 matrix
 
2 −1 0
A =  −1 2 −1 
0 −1 2
√ √
had the following distinct eigenvalues λ1 = 2 − 2, λ2 = 2, λ3 = 2 + 2 with corresponding
eigenvectors of the form shown:
     
√ 1 1 1
√
X (1) =  2  , X (2) =  0  , X (3) =  − 2 
1 −1 1
Clearly none of these eigenvectors is a constant multiple of any other. Nor is any one obtainable as
a linear combination of the other two. The three eigenvectors are linearly independent.
Property 4: Eigenvalues of diagonal matrices
A 2 × 2 diagonal matrix D has the form
 
a 0
D=
0 d
The characteristic equation
 
 a−λ 0 
|D − λI| = 0 is  =0
0 d−λ 

i.e. (a − λ)(d − λ) = 0
So the eigenvalues are simply the diagonal elements a and d.
Similarly a 3 × 3 diagonal matrix has the form
 
a 0 0
D= 0 b 0 
0 0 c
having characteristic equation

HELM (2008): 15
Section 22.1: Basic Concepts
 |D − λI| = (a − λ)(b − λ)(c − λ) = 0
so again the diagonal elements are the eigenvalues.
We can see that a diagonal matrix is a particularly simple matrix to work with. In addition to the
eigenvalues being obtainable immediately by inspection it is exceptionally easy to multiply diagonal
matrices.

Task
Obtain the products D1 D2 and D2 D1 of the diagonal matrices
   
a 0 0 e 0 0
D1 =  0 b 0  D2 =  0 f 0 
0 0 c 0 0 g

Your solution

Answer  
ae 0 0
D1 D2 = D2 D1 =  0 bf 0 
0 0 cg
which of course is also a diagonal matrix.

Exercise
If λ1 , λ2 , . . . λn are the eigenvalues of a matrix A, prove the following:

(a) AT has eigenvalues λ1 , λ2 , . . . λn .

(b) If A is upper triangular, then its eigenvalues are exactly the main diagonal entries.
1 1 1
(c) The inverse matrix A−1 has eigenvalues , ,... .
λ1 λ2 λn
(d) The matrix A − kI has eigenvalues λ1 − k, λ2 − k, . . . λn − k.
(e) (Harder) The matrix A2 has eigenvalues λ21 , λ22 , . . . λ2n .
(f) (Harder) The matrix Ak (k a non-negative integer) has eigenvalues λk1 , λk2 , . . . λkn .

Verify the above results for any 2 × 2 matrix and any 3 × 3 matrix found in the previous Exercises
on page 13.
N.B. Some of these results are useful in the numerical calculation of eigenvalues which we shall
consider later.

16 HELM (2008):
Workbook 22: Eigenvalues and Eigenvectors
 ®

Answer

(a) Using the property that for any square matrix A, det(A) = det(AT ) we see that if

det(A − λI) = 0 then det(A − λI)T = 0

This immediately tells us that det(AT −λI) = 0 which shows that λ is also an eigenvalue
of AT .

(b) Here simply write down a typical upper triangular matrix U which has terms on the leading
diagonal u11 , u22 , . . . , unn and above it. Then construct (U − λI). Finally imagine how
you would then obtain det(U −λI) = 0. You should see that the determinant is obtained
by multiplying together those terms on the leading diagonal. Here the characteristic
equation is:

(u11 − λ)(u22 − λ) . . . (unn − λ) = 0

This polynomial has the obvious roots λ1 = u11 , λ2 = u22 , . . . , λn = unn .

(c) Here we begin with the usual eigenvalue problem AX = λX. If A has an inverse A−1
we can multiply both sides by A−1 on the left to give

A−1 (AX) = A−1 λX which gives X = λA−1 X

or, dividing through by the scalar λ we get

1
A−1 X = X which shows that if λ and X are respectively eigenvalue and eigen-
λ
vector of A then λ−1 and X are respectively eigenvalue and eigenvector of A−1 .
 
2 3
As an example consider A = . This matrix has eigenvalues λ1 = −1, λ2 = 5
3 2    
1 1
with corresponding eigenvectors X1 = and X2 = . The reader should
−1   1
1 2 −3 1
verify (by direct multiplication) that A−1 = − has eigenvalues −1 and
  5 −3   2 5
1 1
with respective eigenvectors X1 = and X2 = .
−1 1

(d) (e) and (f) are proved in similar way to the proof outlined in (c).

HELM (2008): 17
Section 22.1: Basic Concepts