CHAPTER 1 : MOTIVATION (AND REVISION!

This course is supposed to be an INTENSIFICATION of the linear algebra course you

have already taken. That means: many of the topics concern things you have already seen.

But what you may not realise is that Linear Algebra is actually a HUGE subject, which

can be (and is) thought about in many different ways. I want to show you a completely

new way of thinking about things you already know, a way related to geometry. (There are

still other, equally respectable ways; see for example MA2101S.) The ultimate objective is

to try to stop you from thinking that there is only one way to do mathematics!

In this first chapter, I want to give a SURVEY, without too much abstraction, of

the main topics in linear algebra. The objective is to understand what linear algebra is

REALLY about and to begin to answer the question: “Why should we care about linear

algebra?” (This is called MOTIVATION, and it’s both incredibly important and, alas,

incredibly neglected.)

The next chapters will be VERY different — there we will ask the deep questions,

emphasising ABSTRACTION. For most of you, maybe all, some of this material will look

extremely unfamiliar. Stay calm and carry on. Unfamiliar 6= difficult.

1
 BTW this is a pure maths course; I will be a bit sloppy in this chapter devoted to MO-

TIVATION, but everything we do AFTER this Chapter is COMPLETELY RIGOROUS.

Important point: COMPLETELY RIGOROUS 6= “not bothering to explain what things

really mean”. Also, COMPLETELY RIGOROUS 6= THEOREMLEMMAPROOFTHEO-

REMLEMMAPROOF......

Then we go back to studying matrices and the special forms you can get them to

take. We will discuss a big Theorem (“Jordan Canonical Form”) which will lead to a

huge result: a COMPLETE CLASSIFICATION of all linear transformations on Euclidean

space! After that we will study BILINEAR FORMS, which are just a generalisation of the

familiar scalar product of vectors, very important in geometry. And then we will confront

the most mysterious object in Linear Algebra, the dreaded DETERMINANT, and explain

what that thing is REALLY about. Expect the unexpected. (Part of the purpose of this

first chapter, however, is just REVISION!)

So relax and cast aside your preconceptions as to what linear algebra is, and (try to)

enjoy the ride.....(NB: because this Chapter is partly revision, I will go fast. Things slow

down in Chapter 2 and beyond.)

1.1 WHAT IS A LINEAR TRANSFORMATION?

2
You know what a function is - it’s a RULE which turns NUMBERS INTO OTHER NUM-

BERS: f (x) = x2 means “please turn 3 into 9, 12 into 144 and so on”.

Similarly a TRANSFORMATION is a rule which turns VECTORS into other VECTORS.

For example, “please rotate all 3-dimensional vectors through an angle of 90◦ clockwise

around the z-axis”. A LINEAR TRANSFORMATION T is one that ALSO satisfies these

* *
rules: if c is any scalar, and u and v are vectors, then

* * * * * *
T (c u ) = cT ( u ) and T ( u + v ) = T ( u ) + T ( v ).

It is called a LINEAR transformation because it maps straight lines to straight lines.

* *
Recall that a straight line is a mapping that sends a real scalar t to the vector u + t v ,

* *
where u and v are given vectors. When we let a linear transformation T act on this, we

* *
get (by the above rules) T u + t T v . But this is just another straight line. REMEMBER

THIS: a linear transformation is a mapping that has a definite GEOMETRIC meaning:

it turns straight lines into other straight lines! So in a sense, linear algebra is a branch

of geometry! (Note: the converse is true: any mapping of three-dimensional space that

always sends straight lines to straight lines is a linear transformation, possibly plus a shift

of the whole space. See Tutorials.)

* * *
EXAMPLE: Let I be the rule I u = u for all u . You can check that I is linear! Called

IDENTITY Linear Transformation.

3
 * * *
EXAMPLE : Let D be the rule D u = 2 u for all u .

* * * *
D(c u ) = 2(c u ) = c(2 u ) = cD u

* * * * * * * *
D( u + v ) = 2( u + v ) = 2 u + 2 v = D u + D v → LINEAR!

* *
Note: Usually we write D( u ) as just D u .

*
EXAMPLE: Let u be a unit vector in any number of dimensions, let α be any real

α
number, and let S* be the rule
u

α * *
* * *
S* : v → v + (α − 1) u · v u ,
u

*
for any vector v . Here the dot, ·, denotes the dot product, which (using either definition)

works in any number of dimensions.

You can readily verify that this is a linear transformation. If you let it act on any

*
vector perpendicular to u , then it has no effect on that vector. If it acts on any vector

*
parallel to u , it just stretches that vector by a factor of α. We will call this transformation

*
a STRETCHING transformation in the direction of u , with stretching factor α. If α

is negative, the “stretching” just reverses the direction as well as changing the length;

the things we normally call “reflections” are “stretches” of that sort (or products of such

*
stretches in higher dimensions). If α is zero, then everything in the direction of u is

crushed to zero size, and this is also a sort of stretching or extreme compression.

EXAMPLE: A ROTATION is a map of space to itself that preserves all angles and

4
lengths but which isn’t a reflection. Such a map turns straight lines into straight lines so

it is linear.

SOMETHING TO NOTICE ABOUT ALL OF THESE EXAMPLES: I never men-

tioned the word, “matrix”! Linear transformations are MAPPINGS, they are things that

do something to space.

THAT HAS NOTHING TO DO WITH MATRICES!!!!!!!!!!!!!!!!

So why do people tend to think that Linear Algebra is all about matrices?

Because we need a language in which to TALK about linear transformations! And we

humans like to talk, perhaps too much.

1.2. THE MATRIX OF A LINEAR TRANSFORMATION

The usual vectors î and ĵ define a square:

Let’s call this the BASIC BOX in two dimensions. Similarly, î, ĵ, and k̂ define the BASIC

BOX in 3 dimensions.

5
Now let T be any linear transformation. You know that any 2-dimensional vector can be

written as aî + bĵ, for some numbers a and b. So for any vector, we have

T (aî + bĵ) = aT î + bT ĵ.

This formula tells us something very important: IF I KNOW WHAT T DOES TO î and

ĵ, THEN I KNOW EVERYTHING ABOUT T - because now I can tell you what T does

to ANY vector.

EXAMPLE: Suppose I know that T (î) = î + 14 ĵ and T (ĵ) = 1

4 î + ĵ. Then what is

T (2î + 3ĵ)?

   
Answer: T (2î + 3ĵ) = 2T î + 3T ĵ = 2 î + 14 ĵ + 3 14 î + ĵ = 2î + 12 ĵ + 34 î + 3ĵ = 11 7
4 î + 2 ĵ.

Since T î and T ĵ tell me everything I need to know, this means that I can tell you everything

about T by telling you WHAT IT DOES TO THE BASIC BOX.

EXAMPLE: Let T be the same transformation as above, T (î) = î+ 14 ĵ and T (ĵ) = 14 î+ ĵ.

6
The basic box has been squashed a bit! Pictures of WHAT T DOES TO THE BASIC

BOX tell us everything about T !

* *
EXAMPLE: If D is the transformation D u = 2 u , then the Basic Box just gets expanded

by a factor of 2 in all directions:

So every LT can be pictured by

seeing what it does to the Basic Box.

There is another way!

   
a b
Let T î = and T ĵ = . Then we DEFINE THE MATRIX OF T RELATIVE TO
c d
 
a b
î, ĵ as , that is, the first COLUMN tells us what happened to î, and the second
c d

column tells us what happened to ĵ.

   
1 0
EXAMPLE: Let I be the identity transformation. Then I î = î = , I ĵ = ĵ = ,
0 1
 
1 0
so the matrix of the identity transformation relative to îĵ is .
0 1

α
EXAMPLE: Remember the stretching transformation S* . Let’s consider two dimensions
u

and take the special case Sî3 . It maps î to 3î, and it maps ĵ to itself. So the matrix is
   
3 0 5 1 0
. Similarly the matrix of Sĵ is .
0 1 0 5
   
* * 2 0
EXAMPLE: If D u = 2 u , then Dî = and Dĵ = so the matrix of D relative to
0 2
 
2 0
î, ĵ is .
0 2

7
 1
 
1
EXAMPLE: If T î = î + 14 ĵ and T ĵ = 14 î + j, then the matrix is 1
4 .
4 1
 
0 1
EXAMPLE: If T î = ĵ and T ĵ = î, the matrix is . Basic box is REFLECTED
1 0

8
EXAMPLE: Suppose in 3 dimensions T î = î+4ĵ+7k̂, T ĵ = 2î+5ĵ+8k̂, T k̂ = 3î+6ĵ+9k̂,
 
1 2 3
then the matrix is  4 5 6 , relative to îĵ k̂.
7 8 9

EXAMPLE: Suppose T î = î + ĵ + 2k̂ and T ĵ = î − 3k̂. This is an example of a LT

that eats 2-dimensional vectors but PRODUCES 3-dimensional vectors. But it still has a
 
1 1
matrix,  1 0 . It’s just that this matrix is not a SQUARE MATRIX, that is, it is not
2 −3
2 by 2 or 3 by 3. Instead it is 3 by 2.

We shall say that a linear transformation is a 2-dimensional L.T. if it eats 2-dimensional

vectors AND produces 2-dimensional vectors. A 2-dimensional L.T. has a square, 2 by

2 matrix relative to î, ĵ. Similarly a 3-dimensional linear transformation has a 3 by 3

matrix. In this chapter we are mainly interested in these two cases; more general cases

will be treated later.

EXAMPLE: Suppose T is a linear transformation that eats 3-dimensional vectors and

produces

2-dimensional vectors according to the rule T î = 2î, T ĵ = î + ĵ, T k̂ = î − ĵ. What is its

matrix?

 
2 1 1
Answer: , a 2 by 3 matrix.
0 1 −1

EXAMPLE: Suppose you take a flat square of rubber and SHEAR it, as shown.

In other words, you don’t

9
change its volume,

you just push it like a pack

of cards. The base stays fixed but the top moves a distance tan(θ). (The height remains

the same, 1 unit.) Clearly the shearing transformation S satisfies S î = î, S ĵ = î tan θ + ĵ,
 
1 tan θ
so the matrix of S relative to î, ĵ is .
0 1

 
1 1
EXAMPLE: Suppose T i = î + ĵ and T ĵ = î + ĵ. Matrix is and basic box is
1 1

SQUASHED FLAT!

EXAMPLE: Rotations in the plane. Suppose you ROTATE the whole plane through an

angle θ (anti-clockwise). Then simple trigonometry shows you that

Rî = cos θî + sin θĵ

Rĵ = − sin θi + cos θĵ

So the rotation matrix is

 
cos θ − sin θ
R(θ) = .
sin θ cos θ

10
Application: Suppose an object is moving on a circle at constant angular speed ω. What

is its acceleration?

*
Answer: Let its position vector at t = 0 be r0 . Because the object is moving on a circle,

*
its position at a later time t is given by rotating r0 by an angle θ(t). So

 
* cos θ − sin θ *
r (t) = r0
sin θ cos θ

Differentiate

 
*
dr − sin θ − cos θ *
= θ̇ r by the chain rule. Here θ̇ is actually ω, so
dt cos θ − sin θ 0
 
*
dr − sin θ − cos θ *
= ω r0 . Differentiate again,
dt cos θ − sin θ
*
d2 r
 
− cos θ sin θ *
= ω 2 r0
dt2 − sin θ − cos θ

 
2 cos θ − sin θ *
= −ω r0
sin θ cos θ
*
Substitute the equation for r (t),
*
d2 r *
2
= −ω 2 r ,
dt

which is a well-known formula. Notice that something interesting is going on here: we

are really talking about an uncountably infinite SET of rotations, parametrised by t. This

kind of set is extremely important, and it has a name: it is called a Lie group.

11
1.3. COMPOSITE TRANSFORMATIONS AND MATRIX MULTIPLICA-

TION.

You know what it means to take the COMPOSITE of two functions: if f (u) = sin(u), and

u(x) = x2 , then f ◦ u means: “please do u FIRST, THEN f , so

f ◦ u(x) = sin(x2 ). NOTE THE ORDER!!

u ◦ f (x) = sin2 (x), NOT the same!

Similarly if A and B are linear transformations, then AB means “do B FIRST, then A”.

NOTE: BE CAREFUL! According to our definition, A and B both eat vectors and both

produce vectors. But then you have to take care that A can eat what B produces!

EXAMPLE: Suppose A eats and produces 2-dimensional vectors, and B eats and pro-

duces 3-dimensional vectors. Then “AB” would not make sense!

EXAMPLE: Suppose B eats 2-d vectors and produces 3-d vectors (so its matrix relative
 
b11 b12
to îĵ k̂ looks like this:  b21 b22 , a 3 by 2 matrix) and suppose A eats 3-d vectors and
b31 b32
produces 2-d vectors. Then AB DOES make sense, because A can eat what B produces.

(In this case, BA also makes sense.).

12
IMPORTANT FACT: Suppose aij is the matrix of a linear transformation A relative to

îĵ k̂, and suppose bij is the matrix of the Linear Transformation B relative to îĵ k̂. Suppose

that AB makes sense. Then the matrix of AB relative to îĵ or îĵ k̂ is just the matrix

product of aij and bij .

 
1
EXAMPLE: What happens to the vector if we shear 45◦ parallel to the x axis and
2

then rotate 90◦ anticlockwise? What if we do the same in the reverse order?

Answer: Shear

 
1 tan θ
→
0 1
   
1 1 cos θ − sin θ
so in this case it is . A rotation through θ has matrix , so here it
0 1 sin θ cos θ
 
0 −1
is Hence
1 0

SHEAR, THEN ROTATE

    
0 −1 1 1 0 −1
→ =
1 0 0 1 1 1

ROTATE, THEN SHEAR

    
1 1 0 −1 1 −1
→ = .
0 1 1 0 1 0

So shear, then rotate

      
1 0 −1 1 −2
→ = .
2 1 1 2 3

13
Rotate, then shear
      
1 1 −1 1 −1
→ =
2 1 0 2 1

Very different!

EXAMPLE: Suppose B is a LT with matrix

 
1 0  
 0 −1  and A is a LT with matrix 0 1 1 . What is the matrix of AB? Of BA?
−1 1 0
−1 1

Answer:  
 1
 0  
0 1 1  −1 0
0 −1  = = AB
−1 1 0 −1 −1
−1 1

2 by 3 3 by 2 2 by 2
   
1 0   0 1 1
0 1 1
 0 −1  = 1 −1 0 
−1 1 0
−1 1 −1 0 −1

3 by 2 2 by 3 3 by 3

EXAMPLE: Suppose you take a piece of rubber in 2 dimensions and shear it parallel to

the x axis by θ degrees, and then shear it again by φ degrees. What happens?

    
1 tan φ 1 tan θ 1 tan θ + tan φ
=
0 1 0 1 0 1

which is also a shear, but NOT through θ + φ!

The shear angles don’t add up, since tan θ + tan θ 6= tan(θ + φ).

14
EXAMPLE: Rotate 90◦ around z-axis, then rotate 90◦ around x-axis in 3 dimensions.

[Always anti-clockwise unless otherwise stated.] Is it the same if we reverse the order?
 
0 −1 0
Rotate about z axis → i becomes j, j becomes −i, k stays the same, so  1 0 0 .
 0 0 1
1 0 0
Rotate about x axis, i stays the same, j becomes k, k becomes −j, so  0 0 −1 , and
0 1 0
  
1 0 0 0 −1 0
 0 0 −1   1 0 0 
0 1 0 0 0 1
  
0 −1 0 1 0 0
6=  1 0 0   0 0 −1  ,
0 0 1 0 1 0
so the answer is NO!

1.4 DETERMINANTS

You probably know that the AREA of the box defined by two vectors is

* *
| u × v |, magnitude

of the vector product.

If you don’t know it, you can easily check it, since the area of any parallelogram is given

by
AREA = HEIGHT × Base

*  *
= | v | sin θ × | u |

* *
= | u | | v | sin θ

* *
= | u × v |.

15
Similarly, the VOLUME of a “three-dimensional parallelogram” [called a PARALLELOP-

IPED!] is given by

VOLUME = (AREA OF BASE) × HEIGHT.

* * *
If you take any 3 vectors in 3 dimensions, say u , v , w, then they define a 3-dimensional

* *
parallelogram. The area of the base is | u × v |,

* π


height is |w| | sin 2 −θ |

where θ is the angle between

* * *
u × v and w, so VOLUME

* * *
defined by u , v , w is just
* * *
π 
| u × v | |w| | sin −θ |
2
* * *
=| u × v | |w| | cos θ|

* * *
=| u × v · w|.

[Check: Volume of Basic Box defined by îĵ k̂ is

|î × ĵ · k̂| = |k̂ · k̂| = 1, correct!

Now let T be any linear transformation in two dimensions. [This means that it acts on

vectors in the xy plane and turns them into other vectors in the xy plane.]

We let T act on the Basic Box, as usual.

16
Now T î and T ĵ still lie in the same plane as î and ĵ, so (T î) × (T ĵ) must be perpendicular

to that plane. Hence it must be some multiple of k̂. We define the DETERMINANT of

T to be that multiple, that is, by definition, det(T ) is the number given [STRICTLY IN 2

DIMENSIONS] by

(T î) × (T ĵ) = det(T )k̂.

EXAMPLE: If I = identity, then

I î × I ĵ = î × ĵ = k̂ = 1k̂

so det(I) = 1.

* *
EXAMPLE: D u = 2 u

Dî × Dĵ = 4î × ĵ = 4k̂ → det(D) = 4

EXAMPLE: T î = î + 14 ĵ, T ĵ = 14 î + ĵ,

   
1 1 1
T î × T ĵ = î + ĵ × î + ĵ = î × ĵ + ĵ × î
4 4 16
15 15 15
= î × ĵ = k̂ → det T = .
16 16 16

EXAMPLE: T î = ĵ, T ĵ = î,

T î × T ĵ = ĵ × î = −k̂ → det T = −1

17
EXAMPLE: Shear, S î = î, S ĵ = î tan θ + ĵ,

S î × S ĵ = k̂ → det S = 1.

EXAMPLE: T î = î + ĵ = T ĵ,

*
T î × T ĵ = 0 → det T = 0.

EXAMPLE: Rotation

Rî × Rĵ = (cos θî + sin θĵ) × (− sin θî + cos θĵ)

= (cos2 θ − − sin2 θ)k̂ = k̂ → det(R) = 1.

The area of the Basic Box is initially |î × ĵ| = 1. After we let T act on it, the area becomes

|T î × T ĵ| = |det T | |k̂| = | det T |.

So

Final Area of Basic Box | det T |

= = | det T |
Initial Area of Basic Box 1

so | det T | TELLS YOU THE AMOUNT BY WHICH AREAS ARE CHANGED BY T .

So det T = ±1 means that the area is UNCHANGED (Shears, rotations, reflections) while

det T = 0 means that the Basic Box is squashed FLAT, zero area.

18
  
a b
Take a general 2 by 2 matrix M = . We know that this means M î = aî + cĵ,
c d
   
M ĵ = bî + dĵ. Hence M î × M ĵ = aî + cĵ × bî + dĵ = (ad − bc)k̂, so
 
a b
det = ad − bc.
c d

   
2 0 1 tan θ
Check: det = 4, det = 1,
0 2 0 1

   
cos θ − sin θ 1 1
det = 1, det = 0.
sin θ cos θ 1 1

IN THREE dimensions there is a similar gadget. The Basic Box is defined by îĵ k̂, and

we can let any 3-dimensional L.T. act on it, to get a new box defined by T î, T ĵ, T k̂. We

define
     
det T = T î × T ĵ · T k̂

where the dot is the scalar product, as usual. Since |T î × T ĵ · T k̂| is the volume of the

3-dimensional parallelogram defined by T î, T ĵ, T k̂, we see that

Final Volume of Basic Box

| det T | = ,
Initial Volume of Basic Box

that is, | det T | tells you how much T changes volumes.

REMARK: You don’t really need to use the standard ijk basis to make all of this work.

* * *
It will work for any triple of unit vectors u , v , and w as long as these three vectors (in this

19
order) form a right-handed basis. This is obvious geometrically: the linear transformation

doesn’t care about your choice of basis, and of course the amount by which areas and

volumes are stretched or compressed cannot depend on that choice.

IMPORTANT REMARK: If T squashes the Basic Box flat, then

det T = 0.

*
EXAMPLE: You can easily show that, for every unit vector u , the stretching transforma-
 
α
tion satisfies det S* = α. (This is obvious geometrically. To prove it from the definition,
u

* * *
choose a basis consisting of u together with any two unit vectors v and w perpendicular

* * * *
to each other and to u , such that u , v , and w form a right-handed basis.)
 
a b
Just as det = ad − bc, there is a formula for the determinant of a 3 by 3 matrix.
c d

The usual notation is this. We DEFINE

 
a b a b
= det = ad − bc.
c d c d

a11 a12 a13

Similarly a21 a22 a23 is the determinant of
 a31 
a32 a33
a11 a12 a13
 a21 a22 a23  and there is a formula for it, as follows:
a31 a32 a33

a11 a12 a13

a21 a22 a23
a31 a32 a33
a22 a23 a a23 a a22
= a11 − a12 21 + a13 21 .
a32 a33 a31 a33 a31 a32

20
In other words, we can compute a three-dimensional determinant if we know how to work

out 2-dimensional determinants.

COMMENTS:

[a] We worked along the top row. Actually, a THEOREM says that you can use ANY

ROW OR ANY COLUMN!

[b] How did I know that a12 had to multiply the particular 2-dimensional determinant

a21 a23
? Easy: I just struck out EVERYTHING IN THE SAME ROW AND COLUMN
a31 a33
∗ ∗ ∗
as a12 : a21 ∗ a23 and just kept the survivors!
a31 ∗ a33
This is the pattern, for example if you expand along the second row you will get
∗ a12 a13 a11 ∗ a13
− a21 ∗ ∗ ∗ + a22 ∗ ∗ ∗
∗ a32 a33 a31 ∗ a33

a11 a12 ∗
− a23 ∗ ∗ ∗
a31 a32 ∗
[c] What is the pattern of the + and − signs? It is an (Ang Moh ) CHESSBOARD,
+ − +
starting with a + in the top left corner: − + −
+ − +

[d] You can do exactly the same thing in FOUR dimensions, following this pattern, using
+ − + −
− + − +
because now you know how to work out 3-dimensional determinants. And
+ − + −
− + − +
so on!

21
Example:
1 −1 0
1 −1 1 −1 1 1
1 1 −1 = 1 − −1 +0
0 0 2 0 2 0
2 0 0

=0+2+0=2
(expanding along the top row) or, if you use the second row,

1 −1 0
1 1 −1
2 0 0
−1 0 1 0 1 −1
= −1 +1 − −1
0 0 2 0 2 0

=0+0+2=2

or
1 −1 0
1 1 −1
2 0 0
1 −1 −1 0 −1 0
=1 −1 +2
0 0 0 0 1 −1

=0+0+2=2
(expanding down the first column).

Important Properties of Determinants

[a] Let S and T be two linear transformations such that det S and det T are defined. Then

det ST = det T S = (det S) × (det T ).

From the point of view of pure algebra, this is a totally amazing formula! How on earth

do the different bits of the two determinants disentangle themselves like that? (Think of

22
trying to prove it using the above formula for the determinant! No thanks!) But from a

geometric point of view, thinking in terms of volumes, it is completely obvious. Geometry

FTW.

Therefore, det[ST U ] = det[U ST ] = det[T U S] and so on: det doesn’t care about the order.

Remember however that this DOES NOT mean that ST U = U ST etc etc.

[b] If M is a square matrix, then

det M T = det M.

[c] If c is a number and M is an n by n matrix, then

det(cM ) = cn det M.

EXAMPLE: By definition, an ORTHOGONAL matrix satisfies M M T = I. So det(M M T ) =

det I = 1. But det(M M T ) = det(M ) det(M T ) = det(M ) × det(M ) = (det M )2 , thus

det M = ±1

for any orthogonal matrix.

1.5. INVERSES.

23
 *
If I give you a 3-dimensional vector u and a 3-dimensional linear transformation T , then

* *
T sends u to a particular vector, it never sends u to two DIFFERENT VECTORS! So

this picture is impossible:

*
Tu

*
u

*
Tu

But what about this picture:

*
u

* *
Tu =Tv

*
v

Can T send TWO DIFFERENT VECTORS TO ONE? Yes!

       
1 0 0 0 0 1 0 0 0
0 0 01 = 0 = 0 0 00
0 0 0 0 0 0 0 0 1

So it can happen! Notice that this transformation destroys ĵ (and also k̂). In fact if

* * * * * * * * *
u 6= v and T u = T v , then T ( u − v ) = 0, that is, T w = 0 where w IS NOT THE

*
ZERO VECTOR. So if this happens, T destroys everything in the w direction. That is, T

SQUASHES 3-dimensional space down to two or even less dimensions. This means that T

*
LOSES INFORMATION − it throws away all of the information stored in the w direction.

24
Clearly T squashes the basic box down to zero volume, so

det T = 0

and we say T is SINGULAR.

SUMMARY: A SINGULAR LINEAR TRANSFORMATION

[a] Maps two different vectors to one vector

[b] Destroys all of the vectors in at least one direction

[c] Loses all information associated with those directions

[d] Satisfies det T = 0.

Conversely, a NON-SINGULAR transformation never maps 2 vectors to one,

* *

u → Tu

Different
* *

v → Tv 

* *
Therefore if I give you T u , THERE IS EXACTLY ONE u . The transformation that takes

* *
you from T u back to u is called the INVERSE OF T . The idea is that since a NON-

*
SINGULAR linear transformation does NOT destroy information, we can re-construct u

*
if we are given T u . Clearly T HAS AN INVERSE, CALLED T −1 , if and only if det T 6= 0.

All this works in other dimensions too.

25
EXAMPLE:
         
1 1 3 7 1 1 4 7
= , = ,
1 1 4 7 1 1 3 7

two different vectors to one!

     
1 1 1 1 1 2
=
1 1 −1 1 1 −2
    
1 1 13.59 0
= =
1 1 −13.59 0

It destroys everything

in that direction!
 
1 1
Finally det = 0.
1 1

So it is SINGULAR and
 
1
has NO inverse.
−1

     
0 −1 α a
EXAMPLE: Take and suppose it acts on and and sends them to the
1 0 β b

same vector, so
     
0 −1 α 0 −1 a
= .
1 0 β 1 0 b

Then
       
−β −b α=a α a
= → → =
α a β=b β b
   
α a
so and are the same − this transformation never maps different vectors to the
β b

same vector. No vector is destroyed, no information is lost, nothing gets squashed! And

26
  
−10
det = 1, NON-SINGULAR.
01
 
α
EXAMPLE: Since det S* = α, stretching transformations are always non-singular
u

except when α = 0.

How to FIND THE INVERSE.

* *
By definition, T −1 sends T u to u , i.e.

* * *
T −1 (T ( u )) = u = T (T −1 ( u )).

* *
But u = I u (identity) so T −1 satisfies

T −1 T = T T −1 = I.

   
0 −1 a b
So to find the inverse of we just have to find a matrix such that
1 0 c d
      
a b 0 −1 1 0 0 1
= , → b = 1, a = 0, d = 0, c = −1 so answer is .
c d 1 0 0 1 −1 0

In fact it’s easy to show that

 −1  
a b 1 d −b
= .
c d ad − bc −c a

α
EXAMPLE: For the stretching transformation S* with α 6= 0, the inverse is obtained
u

by replacing α with 1/α.

For bigger square matrices there are many tricks for finding inverses. A general [BUT

NOT VERY PRACTICAL] method is as follows:

27
[a] Work out the matrix of COFACTORS. [A cofactor is what you get when you work out

the smaller determinant obtained by striking out a row and a column, for example the
1 2 3
1 2
cofactor of 6 in 4 5 6 is = −6. You can do this for each element in a given
7 8
7 8 9
matrix, to obtain a new matrix of the same size. For example, the matrix of cofactors of
   
1 0 1 1 0 0
0 1 0  is  0 1 0 .
0 0 1 −1 0 1
 
+ − + 1 0 0
[b] Keep or reverse the signs of every element according to − + − (you get  0 1 0 
+ − + −1 0 1
above.)
 
1 0 −1
[c] Take the TRANSPOSE,  0 1 0 .
0 0 1

[d] Divide by the determinant of the original matrix. THE RESULT IS THE DESIRED
 
1 0 −1
INVERSE.  0 1 0  in this example. Check:
0 0 1
    
1 0 1 1 0 −1 1 0 0
0 1 00 1 0  = 0 1 0.
0 0 1 0 0 1 0 0 1

INVERSE OF A PRODUCT:

(AB)−1 = B −1 A−1

Note the order! Easily checked:

(AB)−1 AB = B −1 (A−1 A)B = B −1 IB = I.

28
APPLICATION: SOLVING LINEAR SYSTEMS.

Suppose you want to solve

x + 2y + 3z = 1
4x + 5y + 6z = 2
7x + 8y + 9z = 4.

One way is to write it as

    
1 2 3 x 1
4 5 6y  = 2.
7 8 9 z 4
 −1
1 2 3
Then all you have to do is find  4 5 6  and multiply it on both sides, so you get
7 8 9
   −1  
x 1 2 3 1
y  = 4 5 6   2  = answer.
z 7 8 9 4
 
1 2 3
So this is a systematic way of solving such problems! Now actually det  4 5 6  = 0,
     7 8 9
1 2 3 1 0
and you can see why:  4 5 6   −2  =  0 .
7 8 9 1 0
 
1
So this transformation destroys everything in the direction of  −2 . In fact it squashes
1
3-dimensional space down to a certain 2-dimensional space. [We say that the matrix has

RANK 2. If it had squashed everything down to a 1-dimensional space, we would say that
 
1
it had RANK 1.] Now actually  2 . DOES NOT lie in that two-dimensional space. Since
  4
1 2 3
 4 5 6  squashes EVERYTHING into that two-dimensional space, it is IMPOSSIBLE
7 8 9

29
     
1 2 3 x 1
for  4 5 6   y  to be equal to  2 . Hence the system has NO solutions.
7 8 9 z   4
1 1
If we change  2  to  2 , this vector DOES lie in the special 2-dimensional space, and
 4  3   
1 2 3 x 1
the system  4 5 6   y  =  2  DOES have a solution − in fact it has infinitely
7 8 9 z 3
many!

SUMMARY:

Any system of linear equations can be written as

* *
Mr = a

* *
where M is a matrix, r = the vector of variables, and a is a given vector. Suppose M is

square.

[a] If det M 6= 0, there is exactly one solution,

* *
r = M −1 a .

[b] If det M = 0, there is probably no solution. But if there is one, then there will be many.

1.6 EIGENVECTORS AND EIGENVALUES.

Remember we said that a linear transformation USUALLY changes the direction of a

vector. But there may be some special vectors which DON’T have their direction changed!

30
    
1 2 1
EXAMPLE: clearly DOES change the direction of î and ĵ, since is not
2 −2 2
 
2
parallel to î and is not parallel to ĵ. BUT
−2

      
1 2 2 4 2
= =2
2 −2 1 2 1

 
2
which IS parallel to .
1
*
In general if a transformation T does not change the direction of a vector u , that is

* *
T u = λu

*
for some λ (SCALAR), then u is called an EIGENVECTOR of T . The scalar λ is called

*
the EIGENVALUE of u .

EXAMPLE: The stretching transformation has eigenvalues α, 1, 1, 1, . . ..

1.7 FINDING EIGENVALUES AND EIGENVECTORS.

There is a systematic way of doing this. Take the equation

* *
T u = λu

* *
and write u = I u , I = identity. Then

* *
(T − λI) u = 0

31
 * * *
Let’s suppose u 6= 0 [of course, 0 is always an eigenvector, that is boring]. So the equation

*
says that T − λI SQUASHES everything in the u direction. Hence

det(T − λI) = 0.

This is an equation which can be SOLVED to find λ.


2 1
EXAMPLE: Find the eigenvalues of :
−2 2
   
1 2 1 0
det −λ =0
2 −2 0 1
 
1−λ 2
→ det =0
2 −2 − λ

→ − (1 − λ)(2 + λ) − 4 = 0

→ λ = 2 OR − 3
So there are TWO answers for a 2 by 2 matrix. Similarly, in general there are three answers

for 3 by 3 matrices, etc. Unfortunately, however, sometimes the “two” or “three” answers

can be repeated; that is, you don’t necessarily get three DIFFERENT answers. This is a

great evil.

What are the eigenvectors for λ = 2, λ = −3?

* *
IMPORTANT POINT: Let u be an eigenvector of T . Then 2 u is also an eigenvector

with the same eigenvalue!

* * * *
T (2 u ) = 2T u = 2λ u = λ × (2 u ).

32
 * *
Similarly 3 u , 13.59 u etc are all eigenvectors! SO YOU MUST NOT EXPECT A UNIQUE

ANSWER!
 
α
OK, with that in mind, let’s find an eigenvector for λ = 2. Let’s call an eigenvector .
β

Then
* *
(T − λI) u = 0
  
1−λ 2 α
→ =0
2 −2 − λ β
  
−1 2 α
→ =0
2 −4 β

→ − α + 2β = 0

→ 2α − 4β = 0

But these equations are actually the SAME, so we really only have ONE equation for 2

unknowns. We aren’t surprised, because we did not expect a unique answer anyway! We

can just CHOOSE α = 1 (or 13.59 or whatever) and then solve for β. Clearly β = 12 , so
     
1 2 100
an eigenvector corresponding to λ = 2 is 1 . But if you said or that is also
2 1 50

correct!

What about λ = −3?   

4 2 α
=0
2 1 β

→ 4α + 2β = 0

→ 2α + β = 0
 
1
Again we can set α = 1, then β = −2, so an eigenvector corresponding to λ = −3 is
−2

33
    
2 −10
or or etc.
−4 20
 
0 −1
EXAMPLE: Find the eigenvalues, and corresponding eigenvectors, of .
1 0
√
 
−λ −1
Answer: We have det = 0 → λ2 + 1 = 0 → λ = ±i, i = −1.
1 −λ

Eigenvector for i: we set

  
1 −i −1
=0
β 1 −i
 
1
→ −i − β = 0 → β = −i so an eigenvector for i is . For λ = −i we have
−i
  
i −1 1
=0
1 i β
 
1
→ i − β = 0 → β = i so an eigenvector for −i is . Note that a REAL matrix can have
i
 
0 −1
COMPLEX eigenvalues and eigenvectors! This is happening simply because is
1 0

a ROTATION through 90◦ , and of course such a transformation leaves NO [real] vector’s

direction unchanged (apart from the zero vector).

1.8. DIAGONAL FORM OF A LINEAR TRANSFORMATION.

Remember that we defined the matrix of a linear transformation T WITH RESPECT TO

î, ĵ by letting T act on î and ĵ and then putting the results in the columns. So to say that
 
a b
T has matrix with respect to î, ĵ means that
c d
T î = aî + cĵ

T ĵ = bî + dĵ.

34
What’s so special about the two vectors î and ĵ? Nothing, except that EVERY vector in

two dimensions can be written as αî + β ĵ for some α, β.

Now actually we only really use î and ĵ for CONVENIENCE. In fact, we can do this with

* *
ANY pair of vectors u , v in two dimensions,

PROVIDED that they are not parallel.

*
That is, any vector w can be

expressed as

* * *
w = αu + β v

* * *
for some scalars α, β. You can see this from the diagram − by stretching u to α u and v

* *
to β v , we can make their sum equal to w.

* *
We call u , v a BASIS for 2-dimensional vectors. Let
 
* P11
u = P11 î + P21 ĵ =
P21
 
* P12
v = P12 î + P22 ĵ =
P22

 
* * P11 P12
 
Then the transformation that takes î, ĵ to ( u , v ) has matrix = P . In order
P21 P22
* *
for u , v to be a basis, P must not squash the volume of the Basic Box down to zero, since

* *
otherwise u and v will be parallel. So we must have

det P 6= 0.

35
The same idea works in 3 dimensions: ANY set of 3 vectors forms a basis PROVIDED

that the matrix of components satisfies det P 6= 0.

     
* 1 * 1 1 1
EXAMPLE: The pair of vectors u = ,v = forms a basis, because det =
0 1 0 1

1 6= 0.

Now of course the COMPONENTS of a vector will change if you choose a different basis.

We want to see how that works.

I will [temporarily] use subscripts to indicate which basis I am using. For example,
 
1
= 1î + 2ĵ.
2 (î,ĵ)

BUT if we express this SAME vector in terms of a different basis we will get different

components.

In fact,

* *
1î + 2ĵ = −î + 2(î + ĵ) = − u + 2 v ,

 
* * −1
so the components of this vector relative to u , v are . Is there a systematic
2 * *
(u,v )

way of getting these components?

 
* * 1 1
As usual, set u = P î, v = P ĵ where P = . We want to find α, β such that
0 1

* *
1î + 2ĵ = α u + β v

36
.

The thing to remember here is that these two expressions refer to the SAME vector.

* *
The idea now is that, in order for that to be true, the change from (î, ĵ) to ( u , v ) has to

* *
be “cancelled out”. But surely if going from (î, ĵ) to ( u , v ) is done by multiplying by P ,

then we expect that the cancelling has to be done by multiplying the coefficients by P −1 .

And that does work!

        
α 1 1 −1 1 −1
= P −1 = = ,
β 2 0 1 2 2

* *
that is, the components of this vector relative to u , v are found as

   
−1 1
= P −1
2 * *
(u,v )
2 (î,ĵ)

* *
THE COMPONENTS RELATIVE TO u , v ARE OBTAINED BY MULTIPLYING P −1

INTO THE COMPONENTS RELATIVE TO î, ĵ.

The same sort of thing happens for linear transformations − if a certain linear trans-
 
1 2
formation T has matrix relative to î, ĵ it will have a DIFFERENT matrix
0 −1 îĵ
* *
relative to u , v . What is that matrix?

For example, you can easily verify that

     
1 2 1 5
= .
0 −1 (î,ĵ)
2 (î,ĵ) −2 (î,ĵ)

37
Now multiply both sides by P −1 , and, in the middle, “multiply and divide” by P − that

is, insert P P −1 , which of course we are allowed to do since that is just the identity matrix:

     
−1 1 2 −1 1 −1 5
P PP =P ,
0 −1 (î,ĵ)
2 (î,ĵ) −2 (î,ĵ)

which of course I can write [just by inserting brackets!] as

(   )    
−1 1 2 1 5
P P P −1 = P −1 .
0 −1 (î,ĵ)
2 (î,ĵ) −2 (î,ĵ)
   
−1 1 1
But, by what we said above, P gives exactly the components of relative
2 (î,ĵ) 2 (î,ĵ)
   
* * 5 5
to the ( u , v ) basis, and similarly P −1 gives the components of relative
−2 (î,ĵ) −2 (î,ĵ)
* *
to ( u , v ). So we conclude that the components of the matrix relative to the new basis

must be
 
1 2
P −1 P.
0 −1 (î,ĵ)

We can check this out: you can show that the components of the two vectors change in

the following way:    

−1 1
= P −1 and
2 * *
(u,v )
2 (î,ĵ)
   
7 5
= P −1
−2 * *
(u,v )
−2 (î,ĵ)
and we claim that the matrix changes according to

     
1 2 1 −1 1 2 1 1
P −1 P =
0 −1 0 1 0 −1 0 1
 
1 4
= .
0 −1

38
    
−1 7
And that is correct: if you multiply this into , you will get exactly !
2 * *
(u,v )
−2 * *
(u,v )
* *
We conclude that THE MATRIX OF T RELATIVE TO u , v , IS OBTAINED BY

MULTIPLYING P −1 ON THE LEFT AND P ON THE RIGHT INTO THE MATRIX

OF T RELATIVE TO î, ĵ.

So now we know how to work out the matrix of any linear transformation relative to ANY

basis.

* *
Now let T be a linear transformation in 2 dimensions, with eigenvectors e1 , e2 , eigenvalues

* *
λ1 , λ2 . Now e1 and e2 may or may not give a basis for 2-dimensional space (because we

can’t be sure that they are not parallel to each other). But suppose they do.

* *
QUESTION: What is the matrix of T relative to e1 , e2 ?

* *
ANSWER: As always, we see what T does to e1 and e2 , and put the results into the

columns!

By definition of eigenvectors and eigenvalues,

* * * *
T e1 = λ 1 e1 = λ 1 e1 + 0 e2

* * * *
T e2 = λ 2 e2 = 0 e1 + λ 2 e2
 
λ1 0
So the matrix is .
0 λ2 * *
(e1 ,e2 )  
  α 0 0
a 0
We say that a matrix of the form or  0 β 0  is DIAGONAL. So we see
0 d
0 0 γ

39
that THE MATRIX OF A TRANSFORMATION RELATIVE TO ITS OWN EIGEN-

VECTORS (assuming that these form a basis, which tragically is not always the case) is

DIAGONAL.

EXAMPLE: We know that the eigenvectors of

         
1 2 1 1 1 1 −1 2 −2 −1
are 1 and . So here P = 1 , P = −5 ,
2 −2 2 −2 2 −2 − 12 1
     
1 2 2 −2 −1 1 2 1 1
P −1 P =−
2 −2 5 − 12 1 2 −2 1
2 −2
  
2 −2 −1 2 −3
=−
5 − 12 1 1 6
 
2 −5 0
=− 15
5 0 2
 
2 0
= as expected since the
0 −3

eigenvalues are 2 and -3.

 
1 tan θ
EXAMPLE: The shear matrix .
0 1
 
1 − λ tan θ
Eigenvalues: det = 0 → (1 − λ)2 = 0 → λ = 1. Only one eigenvector,
0 1−λ
 
1
namely , so the eigenvectors DO NOT give us a basis in this case → NOT possible to
0

diagonalize this matrix! This evil is responsible for most of the complications in matrix

theory; it is indeed a great evil.

EXAMPLE: Suppose you want to do a reflection ρ of the entire 2-dimensional plane

around a straight line that passes through the origin and makes an angle of θ with the

40
  
cosθ
x-axis. Then the vector lies along this line, so it is left unchanged by the reflection;
sinθ

in other words, it is an eigenvector of ρ with eigenvalue 1. On the other hand, the vector
 
− sinθ
is perpendicular to the first vector [check that their scalar product is zero] so it
cosθ

is reflected into its own negative by ρ. That is, it is an eigenvector with eigenvalue − 1.

So ρ has a matrix with these eigenvectors and eigenvalues. The P matrix in this case

is
 
cos θ − sin θ
P = ,
sin θ cos θ
 
cos θ sin θ
and clearly P −1 = , and since the eigenvalues are ±1, we have
− sin θ cos θ

   
cos θ − sin θ 1 0 cos θ sin θ
ρ= .
sin θ cos θ 0 −1 − sin θ cos θ

Doing the matrix multiplication and using the trigonometric identities for cos2θ and sin2θ,

you will find that

 
cos 2θ sin 2θ
ρ= .
sin 2θ − cos 2θ

Notice that the determinant is − 1, as is typical for a reflection. Check that this gives

the right answer for reflections around the 45 degree diagonal and around the x-axis.

1.9 APPLICATION − MARKOV CHAINS.

Let’s construct a simple model of weather forecasting. We assume that each day is either

41
RAINY or SUNNY.

Rainy today → probably rainy tomorrow (probability 60%).

Sunny today → probably sunny tomorrow (probability 70%).

Since probabilities have to add up to 100%, you can easily see that Rainy → Sunny has

probability 40% and Sunny → Rainy has probability 30%. We can organise these data into

a matrix by putting the probabilities into the columns. That is, all of the probabilities of

the form Rainy → (Rainy or Sunny) go into the f irst column, and all of the probabilities

of the form Sunny → (Rainy or Sunny) go into the second column, and so on if there are

more possible weather states. So we get

 
Rainy → Rainy Sunny → Rainy
M=
Rainy → Sunny Sunny → Sunny
 
0.6 0.3
= .
0.4 0.7

(So if there were three states, say Rainy, Sunny, Hazy, then we would put the

three numbers Rainy → Rainy, Rainy → Sunny, Rainy → Hazy down the first

column of a 3 × 3 matrix, and similarly for the second and third columns.)

Question: Suppose today is sunny. What is the probability that it will be rainy 4

days from now? To see how to proceed, we make a “tree” like this: [R = rain, S = sun]

for the first two days:

42
We have, similarly, that the probability of Rainy → Sunny over 2 days is

RS2 = 0.6 × 0.4 + 0.4 × 0.7

By constructing a tree starting with S, you will find that the probability of rain 2 days

after a sunny day is

SR2 = 0.6 × 0.3 + 0.3 × 0.7

and similarly

SS2 = 0.4 × 0.3 + 0.7 × 0.7

So now the matrix of probabilities is

 
RR2 SR2
RS2 SS2
 
0.6 × 0.6 + 0.3 × 0.4 0.6 × 0.3 + 0.3 × 0.7
= .
0.6 × 0.4 + 0.4 × 0.7 0.4 × 0.3 + 0.7 × 0.7
  
0.6 0.3 0.6 0.3
But this is exactly = M 2.
0.4 0.7 0.4 0.7

So matrix multiplication actually allows you to compute all of the probabilities in this

“Markov Chain”. To predict the weather 4 days from now, we need

   
RR4 SR4 4 0.43
2 2 0.43
=M =M M = .
RS4 SS4 0.57 0.57

So if it is rainy today, the probability of rain in 4 days is 0.43=43%. If you want 20 days,

just compute M 20 . A very complicated problem without matrix multiplication!

43
 There is an easy way to work this out using eigenvalues.

 
−1 λ1 0
Suppose I can diagonalize M , that is, I can write P MP = D = for some
0 λ2

matrix P . Then

M = P DP −1

M 2 = (P DP −1 )(P DP −1 )

= P DP −1 P DP −1 = P D2 P −1

M 3 = M M 2 = P DP −1 P D2 P −1 = P D3 P −1 etc

M 30 = P D30 P −1 .

 
λ30 0
But D30 is very easy to work out − it is just 1
. Let’s see how this works!
0 λ30
2
     
0.6 0.3 1 1
Eigenvectors and eigenvalues of are (eigenvalue 0.3) and 4 (eigenvalue
0.4 0.7 −1 3

1) so

4
− 37
    
1 1 0.3 0 −1
P = 4 ,D = , P = 73 3
−1 3 0 1 7 7

2 × 10−16
   
30 (0.3)30 0 0
D = ≈
0 1 0 1

44
so
2 × 10−16
 4
− 37
  
30 1 1 0 7
M = 4 3 3
−1 3 0 1 7 7

1 3 + 8 × 10−16
  3 3

3 − 6 × 1016
= ≈ 7 7
7 4 − 8 × 10−16 4 + 6 × 10−16 4
7
4
7

So if it is rainy today, the probability of rain tomorrow is 60%, but the probability of rain

3
30 days from now is only 7 ≈ 43%. As we go forward in time, the fact that it rained today

becomes less and less important. The probability of rain in 31 days is almost the same as

the probability of rain in 30 days!

1.10 SUMS AND PRODUCTS OF EIGENVALUES.

Let M be any square matrix. Then the TRACE of M , denoted T rM , is defined as the
   
  1 2 3 1 5 16
1 0
sum of the diagonal entries: T r = 2, T r  4 5 6  = 15, T r  7 2 15  = 11,
0 1
7 8 9 11 9 8
etc.

In general it is NOT true that T r(M N ) = T rM T rN BUT it is true that T rM N =

T rN M .

Hence T r(P −1 AP ) = T r(AP P −1 ) = T rA so THE TRACE OF A MATRIX IS AL-

WAYS THE SAME NO MATTER WHICH BASIS YOU USE! This is why the trace is

interesting: it doesn’t care which basis you use. In particular, if A is diagonalizable,

45
  
λ1 0
T rA = T r = λ1 + λ2 .
0 λ2

So the trace is equal to the sum of the eigenvalues. This gives a quick check that you

have not made a mistake in working out the eigenvalues: they have to add up to the same

number as the trace of the original matrix. Check this for the examples in this chapter.

By the same sort of argument you can see that the DETERMINANT of such a matrix

is the PRODUCT of the eigenvalues. Now think about what the transformation is doing

to a small box inside some object. (Let’s think about the case where all eigenvalues and

eigenvectors are real.) Along the directions of the eigenvectors, the box is simply being

stretched or compressed, not rotated or twisted. (In other words, a diagonal matrix is

just a product of several stretching matrices.) The amount of stretching or compression

is given by the eigenvalues. So the (absolute value of the) product of the eigenvalues tells

you how much the volume of that small box is being changed. So now we have a better

understanding of why the determinant measures the changes in volumes under a linear

transformation. The point is that this argument works in ALL dimensions, not just 2 and

3. We will come back to this later.

(It is a fact, which we will prove later, that for every square matrix, one can find a

basis such that the original matrix is transformed to a product of stretches with shears,

rotations, and reflections. The last three all have determinants with absolute values equal

46
to one, so the discussion in the previous paragraph actually works whether or not the

matrix is diagonalizable with real eigenvalues. The point I am making is that you will

understand linear algebra a LOT better if you have clear pictures in your mind, and that

is true even of the much more abstract stuff we are about to study!)

47