EIGENVALUES AND EIGENVECTORS

Dr. Gabriel Obed Fosu

Department of Mathematics
Kwame Nkrumah University of Science and Technology
Google Scholar: https://scholar.google.com/citations?user=ZJfCMyQAAAAJ&hl=en&oi=ao
ResearchGate ID: https://www.researchgate.net/profile/Gabriel_Fosu2

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 1 / 45

Lecture Outline

1 Finding Eigenvalues and Eigenvectors

Definition
The Case of Repeated Roots

2 Diagonalization
Some Properties
Power of Matrices

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 2 / 45

 Finding Eigenvalues and Eigenvectors Definition

Introduction

Computing eigenvalues and eigenvectors is one of the most important problems in

numerical linear algebra.
1 Let A be an n × n matrix. For a large number of problems in engineering and science,
it is necessary to find vector v such that Av is a multiple of λ of v
2 In other words,
Av = λv
Av is parallel to v , and λ either stretches or shrinks v .
3 The value λ is an eigenvalue
4 v is an eigenvector associated with λ.

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 4 / 45

 Finding Eigenvalues and Eigenvectors Definition

Definition
1 If A is an n × n matrix, in order to find eigenvalue λ and an associated eigenvector v ,
it must be the case that Av = λv , and this is equivalent to the homogeneous system

(A − λI )v = 0 (1)

2 From (the homogeneous system Ax = 0 has a nontrivial solution if and only if det A = 0)
we deduce that
d et (A − λI ) = 0 (2)
in order that this system to have a nonzero solution.
3 The determinant of A − λI is a polynomial of degree n , so Equation (2) is a problem of
finding roots of the polynomial

p(λ) = d et (A − λI ) (3)

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 5 / 45

 Finding Eigenvalues and Eigenvectors Definition

Definition

1 The polynomial
p(λ) = d et (A − λI ) (4)
is called the characteristic polynomial of A
2 The equation
p(λ) = 0 (5)
is termed the characteristic equation.
3 If λ is a root of p , it is termed an eigenvalue of A ,
4 If v is a nonzero column vector satisfying Av = λv , it is an eigenvector of A .
We say that v is an eigenvector corresponding to the eigenvalue λ.

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 6 / 45

 Finding Eigenvalues and Eigenvectors Definition

Example
1 Let
· ¸ · ¸
−0.4707 0.7481 0.2898
A= , ⃗
v=
1.7481 1.4707 0.9571

2 Then · ¸
0.5796
A⃗
v=
1.9142
3 Moreover · ¸
0.5796
A⃗
v= v
= 2⃗ (6)
1.9142
Thus ⃗
v is an eigenvector of A corresponding to
eigenvalue 2.

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 7 / 45

 Finding Eigenvalues and Eigenvectors Definition

Summary Steps To Finding Eigenvalues and Eigenvectors

Given an n × n matrix A
1 Find the characteristic polynomial

p(λ) = d et (A − λI ) (7)

2 Compute the n roots of

p(λ) = 0 (8)
These are the eigenvalues λ1 , λ2 , · · · , λn of A .
3 For each distinct λi , find an eigenvector x i such that Ax i = λi x by solving

(A − λI )v = 0 (9)

If λi is a multiple root, there may be only one associated eigenvector. If not, compute the
distinct eigenvectors.
Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 8 / 45
 Finding Eigenvalues and Eigenvectors Definition

Definition
A polynomial
p(λ) = a n λn + a n−1 λn−1 + · · · + a 2 λ2 + a 1 λ + a 0
of degree n has exactly n roots; either being, real distinct roots, real repeated roots, or
complex roots. Any complex roots occur in conjugate pairs.

Definition
Vectors v1 , v2 , · · · , vn in R n are said to be linearly dependent if there exist scalars k1 , k2 , · · · , kn
not all zero, such that
k 1 v1 + k 2 v2 + · · · + k n vn = 0 (10)
The vectors v1 , v2 , · · · , vn are called linearly independent if they are not linearly dependent.

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 9 / 45

 Finding Eigenvalues and Eigenvectors Definition

Example
Find the eigenvalues and all the eigenvectors
· of the
¸ matrix
2 1
A=
1 2

First find the characteristic polynomial µ· ¸ · ¸¶

2 1 1 0
p(λ) = det(A − λI ) = det −λ (11)
1 2 0 1
λ 0
µ· ¸ · ¸¶
2 1
= det − (12)
1 2 0 λ
2−λ
µ· ¸¶
1
= det (13)
1 2−λ
= (2 − λ)2 − 1 (14)
2
= λ − 4λ + 3 (15)

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 10 / 45

 Finding Eigenvalues and Eigenvectors Definition

1 Next, compute the n roots of p(λ) = 0. That is

λ2 − 4λ + 3 = 0 (16)
(λ − 1)(λ − 3) = 0 (17)

Thus the eigenvalues are

λ = 1 or λ = 3

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 11 / 45

 Finding Eigenvalues and Eigenvectors Definition

1 To find the eigenvector when λ = 1, we solve

(A − λI )⃗
v =0 (18)
Let ⃗
v = [x, y]T
2 Thus we solve the homogeneous system
2−λ
· ¸· ¸ · ¸
1 x 0
= (19)
1 2−λ y 0
· ¸· ¸ · ¸
2−1 1 x 0
= (20)
1 2−1 y 0
· ¸· ¸ · ¸
1 1 x 0
= (21)
1 1 y 0

3 Solving (21) using Gaussian elimination, the system reduce to

· ¸· ¸ · ¸
1 1 x 0
= (22)
0 0 y 0

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 12 / 45

 Finding Eigenvalues and Eigenvectors Definition

1 This implies that

x + y = 0 =⇒ x = −y

2 Let y ∈ R but y ̸= 0 then the eigenvectors corresponding to λ = 1 are the vectors of the
form · ¸ · ¸ · ¸
x −y −1
= =y
y y 1

3 Choose y = 1 to obtain a specific eigenvector

· ¸
−1
(23)
1

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 13 / 45

 Finding Eigenvalues and Eigenvectors Definition

1 To find the eigenvector when λ = 3, we solve

(A − λI )⃗
v =0 (24)
Let ⃗
v = [x, y]T
2 Thus we solve the homogeneous system
2−λ
· ¸· ¸ · ¸
1 x 0
= (25)
1 2−λ y 0
· ¸· ¸ · ¸
2−3 1 x 0
= (26)
1 2−3 y 0
· ¸· ¸ · ¸
−1 1 x 0
= (27)
1 −1 y 0

3 Solving (27) using Gaussian elimination, the system reduce to

· ¸· ¸ · ¸
−1 1 x 0
= (28)
0 0 y 0

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 14 / 45

 Finding Eigenvalues and Eigenvectors Definition

1 This implies that

−x + y = 0 =⇒ x = y

2 Let y ∈ R but y ̸= 0 then the eigenvectors corresponding to λ = 3 are the vectors of the
form · ¸ · ¸ · ¸
x y 1
= =y
y y 1
3 Choose y = 1 to obtain a specific eigenvector
· ¸
1
(29)
1

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 15 / 45

 Finding Eigenvalues and Eigenvectors Definition

Example
Find the eigenvalues and eigenvectors of the following matrix

4 8 3
 

A = 0 −1 0
0 −2 2

1 The characteristic polynomial is

4−λ 8 3
 

det(A − λI ) = det  0 −1 − λ 0  (30)

0 −2 2−λ
= (−1 − λ)(4 − λ)(2 − λ) (31)

That is expanding by the second row.

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 16 / 45

 Finding Eigenvalues and Eigenvectors Definition

1 Now p(λ) = 0
(−1 − λ)(4 − λ)(2 − λ) = 0 (32)
The roots of the characteristic polynomial are

λ1 = 4, λ2 = −1, λ3 = 2

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 17 / 45

 Finding Eigenvalues and Eigenvectors Definition

λ=4

1 Now the eigenvector when λ = 4. We solve the homogeneous system (A − λI )⃗

v =0

4−λ 8 3 x 0
    
 0 −1 − λ 0   y  = 0
 (33)
0 −2 2−λ z 0
4−4 8 3 x 0
    
 0 −1 − 4 0   y  = 0
 (34)
0 −2 2−4 z 0
0 8 3 x 0
    
0 −5 0   y  = 0 (35)
0 −2 −2 z 0

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 18 / 45

 Finding Eigenvalues and Eigenvectors Definition

1 Solving using Gaussian elimination, reduces the system to

0 8 3 x 0
    
0 −5 0   y  = 0 (36)
0 0 −2 z 0

2 Thus
y = 0, z =0

3 The first row specifies that

(0)x + 8(0) + 3(0) = 0 (37)
The component x is not constrained. Any value of x will work. So we choose x = 1 to
obtain the eigenvector
x 1
   
 y  = 0 (38)
z 0

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 19 / 45

 Finding Eigenvalues and Eigenvectors Definition

λ = −1

1 We solve the homogeneous system (A − λI )⃗

v =0

4−λ 8 3 x 0
    
 0 −1 − λ 0   y = 0
  (39)
0 −2 2−λ z 0
4+1 8 3 x 0
    
 0 −1 + 1 0   y = 0
  (40)
0 −2 2+1 z 0
5 8 3 x 0
    
0 0 0  y  = 0 (41)
0 −2 3 z 0

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 20 / 45

 Finding Eigenvalues and Eigenvectors Definition

1 Solving using Gaussian elimination (interchange row 2 and 3), reduces the system to

5 8 3 x 0
    
0 −2 3  y  = 0 (42)
0 0 0 z 0

2 Thus
3z 8(3z/2) + 3z
y= , x =− = −3z
2 5
3 This gives a general eigenvector

x
       
−3z −3 −3
 y  = 3z/2 = z 3/2 = 3/2 (43)
z z 1 1

Note that z chosen as z = 1

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 21 / 45

 Finding Eigenvalues and Eigenvectors Definition

λ=2

1 We solve the homogeneous system (A − λI )⃗

v =0

4−λ 8 3 x 0
    
 0 −1 − λ 0   y = 0
  (44)
0 −2 2−λ z 0
4−2 8 3 x 0
    
 0 −1 − 2 0   y = 0
  (45)
0 −2 2−2 z 0
2 8 3 x 0
    
0 −3 0  y  = 0 (46)
0 −2 0 z 0

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 22 / 45

 Finding Eigenvalues and Eigenvectors Definition

1 Solving using Gaussian elimination reduces the system to

2 8 3 x 0
    
0 −3 0  y  = 0 (47)
0 0 0 z 0
2 Thus y = 0, x = −3z/2
3 This gives a general eigenvector
x
     
−3z/2 −3/2
y  =  0  = z  0  (48)
z z 1
4 If we choose z = 1, then the eigenvector is
x
   
−3/2
y  =  0  (49)
z 1

Note Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 23 / 45

 Finding Eigenvalues and Eigenvectors The Case of Repeated Roots

Example
Find the eigenvalues and eigenvectors of the following matrix

6 12 19
 

A = −9 −20 −33

4 9 15

1 The characteristic polynomial is

6−λ 12 19
 

det(A − λI ) = det  −9 −20 − λ −33  (50)

4 9 15 − λ
= (λ + 1)(λ − 1)2 (51)

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 24 / 45

 Finding Eigenvalues and Eigenvectors The Case of Repeated Roots

1 The roots of the characteristic polynomial are

λ1 = 1, λ2 = −1

λ = 1 is a multiple root.

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 25 / 45

 Finding Eigenvalues and Eigenvectors The Case of Repeated Roots

λ=1
1 We solve the homogeneous system

6−λ 12 19 x 0
    
 −9 −20 − λ −33   y  = 0 (52)
4 9 15 − λ z 0
5 12 19 x 0
    
−9 −21 −33  y  = 0 (53)
4 9 14 z 0

2 Solving using Gaussian elimination reduces the system to

5 12 19 x 0
    
0 3/5 6/5  y  = 0 (54)
0 0 0 z 0

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 26 / 45

 Finding Eigenvalues and Eigenvectors The Case of Repeated Roots

1 Thus
x = z, y = −2z
x 1
   
 y  = −2 (55)
z 1
2 There is only one linearly independent eigenvector associated with λ = 1
3 The eigenvector when λ = −1 is left as an exercise.

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 27 / 45

 Finding Eigenvalues and Eigenvectors The Case of Repeated Roots

Note
There are cases where an eigenvalue of multiplicity k does produce k linearly independent
eigenvectors.

Symmetric Matrix
Whenever an n × n real matrix is symmetric, it has n linearly independent eigenvectors,
even if its characteristic equation has roots of multiplicity 2 or more.

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 28 / 45

 Finding Eigenvalues and Eigenvectors The Case of Repeated Roots

Example
Find the eigenvalues and eigenvectors of the following matrix

1 1 1
 

A = 1 1 1
1 1 1

1 The characteristic polynomial is

1−λ 1 1
 

det(A − λI ) = det  1 1−λ 1  (56)

1 1 1−λ
= (λ − 3)λ2 (57)

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 29 / 45

 Finding Eigenvalues and Eigenvectors The Case of Repeated Roots

1 The roots of the characteristic polynomial are

λ1 = 0, λ2 = 3

λ = 0 is an eigenvalue of multiplicity 2.

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 30 / 45

 Finding Eigenvalues and Eigenvectors The Case of Repeated Roots

λ=0
1 We solve the homogeneous system

1−λ 1 1 x 0
    
 1 1−λ 1   y = 0
  (58)
1 1 1−λ z 0
1 1 1 x 0
    
1 1 1 y = 0
    (59)
1 1 1 z 0

2 Solving using Gaussian elimination reduces the system to

1 1 1 x 0
    
0 0 0  y  = 0 (60)
0 0 0 z 0

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 31 / 45

 Finding Eigenvalues and Eigenvectors The Case of Repeated Roots

1 Thus x = −y − z
2 When we choose z = 0, then we have
x
     
−y −1
y  =  y  = y  1  (61)
z 0 0
3 Similarly, when we choose y = 0
x
     
−z −1
y  =  0  = z  0  (62)
z z 1
4 Thus we have two linearly independent eigenvectors
   
−1 −1
1 and 0
0 1
5 The eigenvector when λ = 3 is left as an exercise.
Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 32 / 45
 Finding Eigenvalues and Eigenvectors The Case of Repeated Roots

Some Properties of Eigenvalues and Eigenvectors

Singular
An n × n matrix A is singular if and only if it has a 0 eigenvalue.

Eigenspace
The eigenvectors corresponding to λ form a subspace called an eigenspace.

Determinant
If A is an n × n matrix, then
n
λi
Y
det A =
i =1

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 33 / 45

 Diagonalization

Diagonalization

Definition (Diagonalization)
Diagonalization involves how we can use the eigenvectors of a matrix A to transform A into
a diagonal matrix of eigenvalues.

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 35 / 45

 Diagonalization

Diagonalization
The procedure for diagonalizing a matrix A can be done with the following steps:
1 Form the characteristic polynomial p(λ) = det(A − λI ) of A .
2 Find the roots of p ; the eigenvalues and its corresponding eigenvectors.
3 Form the matrix
v 1 , v⃗2 , · · · v n−1
X = [⃗ ⃗ , v⃗n ]

whose columns are eigenvectors of A corresponding to eigenvalues

λ1 , λ2 , · · · , λn−1 , λn .
4 Then,
D = X −1 AX (63)
where D is the diagonal matrix with λ1 , λ2 , · · · , λn−1 , λn on its diagonal. Then the matrix
A and D are called similar matrices.

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 36 / 45

 Diagonalization

Definition (Similar Matrices)

The matrix B is similar to matrix A if there exists a nonsingular matrix X such that

B = X −1 AX

Thus, two similar matrices have the same eigenvalues.

Example
Show that the following matrix can be diagonalize

4 8 3
 

A = 0 −1 0
0 −2 2

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 37 / 45

 Diagonalization

From the previous example, we obtained the eigenvalues

λ1 = 4, λ2 = −1, λ3 = 2 (64)

These have the corresponding eigenvectors

1
     
−3 −3/2
v 1 = 0 v 2 = 3/2 , v3 =  0  (65)
0 1 1

The matrix of eigenvectors is given as

1 −3 −3/2
 

v 1 , v⃗2 , v⃗3 ] = 0 3/2

X = [⃗ 0  (66)
0 1 1

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 38 / 45

 Diagonalization

1 Then we can find D = X −1 AX

2 Clearly
1 1 3/2
 

X −1 = 0 2/3 0  (67)
0 −2/3 1
Then
1 1 3/2 4 8 3 4 4 6
    

X −1 A = 0 2/3 0  0 −1 0 = 0 −2/3 0 (68)

0 −2/3 1 0 −2 2 0 −4/3 2
and
4 4 6 1 −3 −3/2 4 0 0
    

D = X −1 AX = 0 −2/3 0 0 3/2 0  = 0 −1 0 (69)

0 −4/3 2 0 1 1 0 0 2

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 39 / 45

 Diagonalization

Note
1 If an n × n matrix A has distinct eigenvalues, then it can be diagonalized.
2 Again if the matrix have n linearly independent eigenvectors, it can be diagonalized.
3 If there are complex roots, the matrix cannot be diagonalized in R n×n .

Example
The matrix
6 12 19
 

A = −9 −20 −33

4 9 15
cannot be diagonalized, because we found that there is only one linearly independent
eigenvector associated with λ = 1.

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 40 / 45

 Diagonalization Some Properties

If A is a real symmetric matrix, then any two eigenvectors corresponding to distinct eigenvalues are
orthogonal.

Example
3 1 −1
 

Let A be the symmetric matrix A =  1 3 −1, solving the eigenvalues are

−1 −1 5
λ1 = 2, λ2 = 3, λ3 = 6 (70)
The corresponding eigenvectors are
1 1
     
−1
−1 , 1 , −1 (71)
0 1 2

These three eigenvectors are mutually orthogonal, and are linearly independent.

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 41 / 45

 Diagonalization Some Properties

The matrix of eigenvectors is given as

1 1 1
 

X = −1 1 −1 (72)

0 1 2

Thus
2 0 0
 

X −1 AX = D = 0 3 0 (73)
0 0 6
D is the diagonal matrix with the eigenvalues of A on the diagonal.

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 42 / 45

 Diagonalization Power of Matrices

Power of Matrices

If a matrix A can be diagonalized, then is simple to compute A n . That is

1

If D = X −1 AX , =⇒ A = X D X −1 (74)
then
A 2 = (X D X −1 )(X D X −1 ) = (X D)I (D X −1 ) = X D 2 X −1 (75)
again
A 3 = A 2 A = (X D 2 X −1 )(X D X −1 ) = X D 3 X −1 (76)
This can be generalized as
A n = X D n X −1 (77)

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 43 / 45

 Diagonalization Power of Matrices

Exercises
1 Find the eigenvalues and associated eigenvectors for the matrix

1 2 1

6 −1 0
−1 −2 −1
· ¸ · ¸
1 4 −1 7
2 Let A = and B = . Verify that AB and B A have the same eigenvalues.
2 3 1 −4
3 Diagonalize

1/2 1/2 0


A = 1/4 1/4 1/2

1/4 1/4 1/2

0 1

−1
4 Obtain an orthogonal matrix out of the following symmetric matrix A =  1 0 1
−1 1 0

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 44 / 45

 END OF LECTURE
THANK YOU

Dr. Gabby (KNUST-Maths) Eigenvalues and Eigenvectors 45 / 45