Linear Algebra 1 - Notes for Lecture 6 1

More on Linear Systems & Inverses

Outcomes In today’s lecture we will learn how to:

• Recognize the special nature of homogeneous systems and understand the associated
terminology.

• Determine the inverse of a matrix by the Gauss Jordan method.

• Identify in which circumstances it is advantageous to solve Ax = b as x = A−1 b.

Contents

• The solution of homogeneous systems.

• The Gauss Jordan method to solve linear systems.

• Elementary matrices and their relation to e.r.o.’s.

• Using the Gauss Jordan method to determine inverses.

• The limited role of inverses in solving linear systems.

Exercises

1. Solve the following homogeneous system.

Z1 + Z2 − 2Z3 − Z5 = 0
2Z1 + 2Z2 − Z3 + Z5 = 0
−Z1 − Z2 + 2Z3 − 3Z4 + Z5 = 0
Z3 + Z4 + Z5 = 0

2. For the following linear system, determine all values of k for which the system has
nontrivial solutions and describe these.
x1 + 3x2 + 6x3 = 0
x2 + 5x3 = 0
2x1 + 3x2 + kx3 = 0

3. For each of the following homogeneous systems, determine by inspection (i.e. you
shouldn’t have to write down any calculations) whether or not they have nontrivial
solutions.
Linear Algebra 1 - Notes for Lecture 6 2

3x1 + 4x2 − 7x3 = 0

5x1 + 2x2 = 0
(a) 17x2 − 5x3 = 0 (b)
15x1 + 6x2 = 0
12x3 = 0
x1 + 3x2 + 2x3 + x4 = 0
a11 x1 + a12 x2 + a13 x3 = 0
(c) 2x1 − x2 − x3 + x4 = 0 (d)
a21 x1 + a22 x2 + a23 x3 = 0
14x1 + 3x2 = 0
4. Solve Exercise 7 from Lecture 5 using the Gauss Jordan method.

5. Which of the following are elementary matrices?

   
" # " # " # 0 0 1 1 0 0
1 0 −2 1 1 0
(a) (b) (c) (d)  0 1 0  (e)  0 0 0 
   
3 1 1 0 0 14
1 0 0 0 1 0

6. Find the inverses of each of the following matrices.

α 0 0 0
 
     
" # 2 6 6 −1 3 −4 a 0 0
1 4  1 α 0 0 
A= ,B =  2 7 6 ,C =  2 4 1 ,D =  0 b 0 ,E = 
       
2 7 0 1 α 0

−4 2 −9
 
2 7 7 0 0 c
0 0 1 α

7. Solve the following systems of linear equations by using the inverse of the coefficient
matrix.
5x1 + 7x2 + 4x3 = 1
3x1 + 4x2 = 10
(a) (b) 3x1 − x2 + 3x3 = 1
7x1 + 9x2 = 20
6x1 + 7x2 + 5x3 = 1

These exercises should take around 70 minutes to complete.

(Answers: 1. Z1 = −s − t, Z2 = s, Z3 = −t, Z4 = 0, Z5 = t; 2. k"= −3, x1 #= 9t,

−7 4
x2 = −5t, x3 = t; 3. (b), (c) and (d); 5. (a), (c) and (d); 6. A−1 = ,
2 −1
 
1
α
0 0 0
 
 
7
 1
− α12 1
    
2
0 −3 0 0  0 0 
 a α
 
−1 1
B = 
 −1 1 0 , C is not invertible, D =  0 b
0 
, E =

 ;

1 1
0 −1 1 − α12 1
 
0 0 c 
 α3 α
0 

 
 
− α14 1
α3
− α12 1
α
7.(a) x1 = −10, x2 = 10 (b) x1 = 8, x2 = −1, x3 = −8)
Linear Algebra 1 - Notes for Lecture 6 3

Homogeneous Systems

Homogeneous systems are simply linear systems where the right hand side of each of the
equations is equal to zero. The term homogeneous is used widely in mathematics, usually
referring to equations in the same way it does here. In general, we have

a11 x1 + a12 x2 + a13 x3 + . . . + a1n xn = 0

a21 x1 + a22 x2 + a23 x3 + . . . + a2n xn = 0
a31 x1 + a32 x2 + a33 x3 + . . . + a3n xn = 0
..
.
am1 x1 + am2 x2 + am3 x3 + . . . + amn = 0

where the variables are x1 , x2 , . . . , xn and the coefficients aij , i = 1, . . . , m, j = 1, . . . , n

are given. In the usual matrix form, we may write

Ax = 0

Note that x = 0 is always a solution to these systems (we call this the trivial solution)
and hence a homogeneous system is always consistent. Another way of looking at this
is to view a homogeneous system as a special case of Ax = b, with b = 0. We have
r([A|b]) = r([A|0]) = r(A), regardless of what the matrix A is. Thus, the system will
always be consistent and we only need to distinguish between two cases:

(i) If r(A) = n, the trivial solution is the only solution.

(ii) If r(A) < n, we get infinitely many solutions. Amongst these is the trivial (i.e. zero)
solution as well as an infinite number of non trivial (i.e. nonzero) solutions.

In the case of infinitely many solutions, we again use parameters to describe these.

Ex: Solve the following homogeneous system:

4x1 +2x2 −x3 = 0

2x1 −3x2 +x3 = 0
6x1 −x2 = 0
   
4 2 −1 | 0 R1 → R2 2 −3 1 | 0
Soln: [A|b] = [A|0] =  2 −3 1 | 0  R2 → R1 ∼  4 2 −1 | 0  R2 → R2 − 2R1
   

6 −1 0 | 0 6 −1 0 | 0 R3 → R3 − 3R1
   
2 −3 1 | 0 2 −3 1 | 0
∼  0 8 −3 | 0  ∼  0 8 −3 | 0 
   

0 8 −3 | 0 R3 → R3 − R2 0 0 0 | 0
Linear Algebra 1 - Notes for Lecture 6 4

Note that it is not actually necessary to carry along the zero column for the augmented
matrix here, but most people seem to do it naturally anyway. As r(A) = 2 is less than the
number of variables, n = 3, we get infinitely many solutions. The non-leading variable in
this case is x3 , so we assign x3 = t. Then 8x2 − 3x3 = 0, i.e. x2 = 38 x3 = 38 t. Finally,
2x1 − 3x2 + x3 = 0, i.e. x1 = 32 x2 − 12 x3 = 32 ( 38 t) − 2t = 16t . In vector form, the solution is
 t   1 
   16  16
x1
 
   
 3   3 
x =  x2  =  t = t .
  
 8   8 
x3 







t 1
This essentially describes a line passing through the origin in 3 space. Infinitely many
solutions to homogeneous systems always generate lines or planes which pass through
the origin and we’ll consider this in more detail later. Finally, note that we end up
solving homogeneous systems frequently when we look at eigenvectors towards the end of
semester.

Reduced Row Echelon Form and the Gauss Jordan Method

A variation of Gaussian elimination is the Gauss Jordan method, which involves manipu-
lating the augmented matrix of a system into the reduced row echelon form. This simply
means that all leading entries in the matrix are 1, any leading entry occurs to the right of
the leading entry in the row above, all zero rows are at the bottom of the matrix and any
column containing a leading entry has only zeros in the remaining entries. In practice,
we first manipulate a given matrix into the row echelon form (working forward along the
columns) and then use further e.r.o.’s (now working backwards along the columns) to
obtain the reduced row echelon form. This is best illustrated by an example.
x1 − x2 + 2x2 = −1
Ex: Solve 2x1 + x2 − 2x3 = −2
−x1 + 2x2 − 4x3 = 1
   
1 −1 2 | −1 1 −1 2 | −1
Soln: [A|b] =  2

1 −2 | −2  R2 → R2 − 2R1 ∼  0 3 −6 | 0 
 
 R2 → R3
−1 2 −4 | 1 R3 → R3 + R 1 0 1 −2 | 0 R3 → R2
   
1 −1 2 | −1 1 −1 2 | −1 R1 → R1 + R 2
∼  0 1 −2 | 0  ∼  0 1 −2 | 0 
   

0 3 −6 | 0 R3 → R3 − 3R2 0 0 0 | 0
 
1 0 0 | −1
∼  0 1 −2 | 0 . We put x3 = t, then x2 = 2t and x1 = −1.
 

0 0 0 | 0
Linear Algebra 1 - Notes for Lecture 6 5

Elementary Matrices and Calculating Inverses

An n×n matrix is called elementary if it can be obtained from the n×n identity matrix by
a single elementary row operation. e.g. E1 , E2 and E3 below are all elementary matrices:
   
1 0 0 R1 → R2 0 1 0
 0 1 0  R2 → R1 ∼  1 0 0  = E1
   

0 0 1 0 0 1
   
1 0 0 1 0 0
 0 1 0  R2 → R2 − 3R1 ∼  −3 1 0  = E2
   

0 0 1 0 0 1
   
1 0 0 1 0 0
 0 1 0  ∼  0 1 0  = E3
   

0 0 1 R3 → 5R3 0 0 5
 
1 3 1 2
To see the use of elementary matrices, consider the effect of pre-multiplying A =  2 1 3 7 
 

4 2 2 1
by each of these elementary matrices:
    
0 1 0 1 3 1 2 2 1 3 7
E1 A =  1 0 0   2 1 3 7  =  1 3 1 2 
    

0 0 1 4 2 2 1 4 2 2 1
    
1 0 0 1 3 1 2 1 3 1 2
 −3 1 0   2 1 3 7  =  −1 −8 0 1 
E2 A =     

0 0 1 4 2 2 1 4 2 2 1
    
1 0 0 1 3 1 2 1 3 1 2
E3 A =  0 1 0   2 1 3 7  =  2 1 3 7 
    

0 0 5 4 2 2 1 20 10 10 5
Now notice what happens if we apply the same e.r.o.’s to A directly
   
1 3 1 2 R1 → R2 2 1 3 7
 2 1 3 7  R2 → R1 ∼  1 3 1 2  = E 1 A
A=   

4 2 2 1 4 2 2 1
   
1 3 1 2 1 3 1 2
A =  2 1 3 7  R2 → R2 − 3R1 ∼  −1 −8 0 1  = E2 A
   

4 2 2 1 4 2 2 1
   
1 3 1 2 1 3 1 2
A= 2 1 3 7  ∼  2 1 3 7  = E3 A
   

4 2 2 1 R3 → 5R3 20 10 10 5
Linear Algebra 1 - Notes for Lecture 6 6

Observation: Let E be obtained from Im by a particular e.r.o. Let A be any m × n

matrix. Then performing the same e.r.o. on A will result in the matrix EA. (i.e. an
e.r.o. is equivalent to a matrix multiplication by the corresponding elementary matrix).

Note that every elementary matrix is invertible and its inverse is also an elementary
matrix (and simply corresponds to the e.r.o. which does the reverse of the original). For
example, consider
   
1 0 0 1 0 0
 0 1 0  R2 → R2 + 3R1 ∼  3 1 0  = E4
   

0 0 1 0 0 1
Then, the operation used to create E4 is just the reverse of the operation used to obtain
E2 and     
1 0 0 1 0 0 1 0 0
E2 E4 =  −3 1 0   3 1 0 = 0 1 0 
,
   

0 0 1 0 0 1 0 0 1
as expected, so E4 = E2−1 .

Note that we could reduce a matrix to row echelon form with a series of matrix
 multi-

1 2 1
plications by elementary matrices if we wanted to. For example, let A =  2 5 3 
 

3 4 5
and
       
1 0 0 1 0 0 1 0 0 1 0 0
E1 =  −2 1 0 

, E2 =  0 1 0 

, E3 =  0 1 0 

, E4 =  0 1 0 

.
1
0 0 1 −3 0 1 0 2 1 0 0 4
Then
       
1 2 1 1 2 1 1 2 1 1 2 1
 0 1 1  = E4 E3  0 1 1  = E4  0 1 1  =  0 1 1  .
E4 E3 E2 E1 A = E4 E3 E2        

3 4 5 0 −2 2 0 0 4 0 0 1
While this idea doesn’t lend itself to hand computation, it is easy to incorporate into
software. Our purpose for introducing elementary matrices is to justify the use of the
Gauss Jordan technique for calculating the inverse of a matrix.

Let A be an invertible n × n matrix (we also use the terminology that A is non-singular).
Then it turns out that we can always reduce A to In by a sequence of e.r.o.’s. Hence,
there is a sequence of elementary matrices E1 , E2 , . . ., Ek such that

Ek Ek−1 . . . E2 E1 A = I

Post multiplying each side by A−1 , we get

Ek Ek−1 . . . E2 E1 AA−1 = IA−1

Linear Algebra 1 - Notes for Lecture 6 7

or
Ek Ek−1 . . . E2 E1 I = A−1
i.e. if we apply the same sequence of e.r.o.’s to I, we will end up with A−1 !

Hence, the technique to obtain the inverse of A is as follows. Form the augmented matrix
[A|I] (i.e. append the identity matrix to the right hand side of A) and then apply e.r.o.’s
to this augmented matrix until the left hand side turns into the identity. The right hand
side will then automatically become A−1 .
 
1 1 2
Ex: Find the inverse of A =  2 4 −3 .
 

3 6 −5
 
1 1 2 | 1 0 0
Soln: [A|I] =  2 4 −3 | 0 1 0 

 R2 → R2 − 2R1
3 6 −5 | 0 0 1 R3 → R3 − 3R1
 
1 1 2 | 1 0 0
1
∼  0 2 −7 | −2 1 0 

 R2 → 2 R2
0 3 −11 | −3 0 1
 
1 1 2 | 1 0 0
∼  0 1 − 72 | −1 21 0 
 

0 3 −11 | −3 0 1 R3 → R3 − 3R2
 
1 1 2 | 1 0 0
∼  0 1 − 2 | −1 12 0 
 7

0 0 − 12 | 0 − 32 1 R3 → −2R3
 
1 1 2 | 1 0 0 R1 → R1 − 2R3
∼  0 1 − 2 | −1 2 0  R2 → R2 + 27 R3
7 1
 

0 0 1 | 0 3 −2
 
1 1 0 | 1 −6 4 R1 → R1 − R2
 0 1 0 | −1 11 −7 
∼ 

0 0 1 | 0 3 −2
 
1 0 0 | 2 −17 11
∼  0 1 0 | −1 11 −7  = [I|A−1 ],
 

0 0 1 | 0 3 −2
 
2 −17 11
i.e. A−1 =  −1 11 −7 . Check for yourself that AA−1 = I.
 

0 3 −2
What happens to this procedure if A is not invertible?
Linear Algebra 1 - Notes for Lecture 6 8
 
1 1 2
Ex: Try to find the inverse of A =  2 4 −3 .
 

1 3 −5
 
1 1 2 | 1 0 0
Soln: [A|I] =  2 4 −3 | 0 1 0 

 R2 → R2 − 2R1
1 3 −5 | 0 0 1 R3 → R3 − R1
 
1 1 2 | 1 0 0
∼  0 2 −7 | −2 1 0 


0 2 −7 | −1 0 1 R3 → R3 − R2
 
1 1 2 | 1 0 0
∼  0 2 −7 | −2 1 0 
 

0 0 0 | 1 −1 1
i.e. this will show up as a row of zeros when trying to perform the Gauss Jordan procedure
and we can’t continue. A matrix which is not invertible is also called singular.

Invertibility and Solutions of Systems

Consider a system of n equations in n unknowns, i.e.

Ax = b
   
x1 b1

 x2   b2 
 
where A is n × n, x =  ..  and b =  .. 
 
.
.   . 


xn bn

Clearly, if A is invertible with inverse A−1 , we have

A−1 Ax = A−1 b
i.e. Ix = A−1 b
i.e. x = A−1 b
   
1 1 2 1
Ex: Solve the system Ax = b, where A =  2 4 −3  and b =  −1 .
   

3 6 −5 0
 
2 −17 11
Soln: From our previous lecture, we have A−1  −1 11 −7 . Thus,
= 

0 3 −2
    
2 −17 11 1 19
−1
x = A b =  −1 11 −7   −1  =  −12  .
    

0 3 −2 0 −3
Linear Algebra 1 - Notes for Lecture 6 9

This example raises the question as to why we should solve a system of n equations in
n unknowns by Gaussian Elimination when we could just use the formula above? The
answer is simply that the amount of computation required for Gaussian elimination is less
than the amount of computation required in finding the inverse of the coefficient matrix.
Particularly for large systems, it usually requires much less effort to employ Gaussian
elimination.

However, in some practical problems, we end up having to solve Ax = b many times over
for the same A but for different vectors b. Here, calculating A−1 once is more efficient
than solving several sets of equations by Gaussian elimination.

Ex: Solve the previous example again with b replaced by

 
1
0
b = 2 
 

1
    
2 −17 11 1 −21
−1 0
Soln: x = A b =  −1 11 −7   2  =  14 .
    

0 3 −2 1 4

Also note that x = A−1 b does not work when A is not invertible. Without any further
work (i.e. Gaussian Elimination), we can only conclude that Ax = b is either inconsistent
(no solution) or it has infinitely many solutions.

Finally, for the homogeneous system Ax = 0, if A is invertible, then x = A−1 0 = 0, i.e.

the zero solution is the only solution. In this case, if A is not invertible, we can conclude
that Ax = 0 must have infinitely many solutions.

Clearly, then, we would like to be able to tell whether a square matrix is invertible or not,
preferably without having to actually attempt to calculate the inverse. This is the goal
of our next lecture.