2 Systems of Linear Equations

2.1 Introduction

This section is all about methods for solving systems of linear equations. A system of linear equations is
a set of equations in the same variables. For example:

−x + y + 2z = 2
3x − y + z = 6
−x + 3y + 4z = 4

This system has three equations and three unknowns, but in general this could be different. For example:

w − x + y + 2z = 2
w + 3x − y + z = 6

In general a system of m linear equations in n unknowns will have the form:

a11 x1 + a12 x2 + a13 x3 + . . . + a1n xn = b1

a21 x1 + a22 x2 + a23 x3 + . . . + a2n xn = b2
.. .
. = ..
am1 x1 + am2 x2 + am3 x3 + . . . + amn xn = bm

Definition 2.1.1 Given a system of m linear equations in n unknowns we can write this in matrix form
as follows:
AX = B
⎛ ⎞ ⎛ ⎞
x1 b1
⎜ x2 ⎟ ⎜ b2 ⎟
⎜ ⎟ ⎜ ⎟
where X = ⎜ . ⎟ and B = ⎜ . ⎟ are column matrices, and
⎝ .. ⎠ ⎝ .. ⎠
xn xm
⎛ ⎞
a11 · · · a1n
a12
⎜ a21 · · · a2n
a22 ⎟
⎜ ⎟
A=⎜ . ⎟ is an m × n matrix.
⎝ .. ⎠
am1 am2 · · · amn
We can also use an Augmented Matrix to represent the system of linear equations:

⎛ ⎞
a11 a12 ··· a1n b1
⎜ a21 a22 ··· a2n b2 ⎟
⎜ ⎟
⎜ .. .. ⎟
⎝ . . ⎠
am1 am2 · · · amn bm

4
 Example 2.1.2.

w − x + y + 2z = 2
w + 3x − y + z = 6

Could be written as ⎛ ⎞
 w 
1 −1 1 2 ⎜ x ⎟
⎜ ⎟= 2
1 3 −1 1 ⎝ y ⎠ 6
z

The Augmented matrix would be:


1 −1 1 2 2
1 3 −1 1 6

Remark 2.1.3 You should have seen some matrix multiplication already (e.g. see Problem sheet 0 on
blackboard). Notice that matrix multiplication is defined precisely so that the above equation works out.

2.2 Matrix Algebra

We will very briefly go over Matrix algebra. You should make sure you go over the exercises on Problem
sheet 0. For the moment we will mostly assume that the matrices take their values in R (at the end of this
section we will see that we could have chosen to take values from any Field F ).
If we want to add two matrices, they must have the same size and shape (the same order). Then we
can simply add corresponding elements. Formally:

Definition 2.2.1. Given m × n matrices, A = [aij ]m×n and if B = [bij ]m×n , then the (matrix) sum
of A and B is the m × n matrix C = [cij ]m×n where cij = aij + bij .

We can also multiply by a scalar product (any element of the field - here R):

Definition 2.2.2. Let A = [aij ] be any matrix, and let λ ∈ R. Then the scalar multiple of A by
λ, denoted by λA, is obtained by multiplying every element of A by λ. Thus if A = [aij ]m×n then
λA = [λaij ]m×n .

See the handout sheet for properties of matrix addition and scalar multiplication.
We can also multiply two matrices together.

5
 Definition 2.2.3. Let A = (aij )p×q and B = (bij )q×r . Then the matrix product of A and B,
denoted by AB, is the matrix C, where

q
C = (cij )p×r , where cij = k=1 aik bkj

Hopefully you will have done lots of examples of this already. Let’s have look at some properties of
matrix multiplication.

Theorem 2.2.4. Matrix multiplication is associative. That is

Let A, B, C be matrices, and α ∈ R, then (AB)C = A(BC).
Proof For A(BC) to be defined, we require the respective sizes of the matrices to be m × n, n ×
p, p × q in which case the product A(BC) is also defined. Calculating the (i, j)th element of this
product, we obtain,
n n p
[A(BC)]ij = aik [BC]kj = aik ( bkt ctj )
k=1 k=1 t=1
n p
= aik bkt ctj
k=1 t=1

If we now calculate the (i, j)th element of (AB)C we obtain the same result:
p p n
[(AB)C]ij = [AB]it ctj = ( aik bkt )ctj
t=1 t=1 k=1
p n
= aik bkt ctj
t=1 k=1

Consequently, we see that A(BC) = (AB)C.

Example 2.2.5. Matrix multiplication is not commutative (i.e. AB = BA)

Proof: To show this we just need one counterexample. Lets try to make it as simple as possible.

• 1 × 1 matrices - multiplying these is just like multiplying elements of R and that is commu-
tative!

• So we have to look at the 2 × 2 matrices.

  
a11 a12 ∗ b12 ∗ a11 b12 + a12 b22
AB = =
 ∗ ∗  ∗ b22  ∗ ∗
b11 b12 ∗ a12 ∗ b11 a12 + b12 a22
BA = =
∗ ∗ ∗ a22 ∗ ∗

Set a11 = b12 = a12 = b22 = b11 = 1...get AB = BA only if a22 = 1

• *****MENTIMETRE*********Is there another way of seeing this in full generality?

6
 Exercise 2.2.6. Let A, B be matrices with entries in R. Show λAB = A(λB).
Proof This splits into two cases:

• AB is not defined therefore λAB is not either. Also B is the same order as λB thus A(λB)
is also not defined.
q
• In the case where AB is defined let cij = k=1 aik bkj be such that cij is the ij th entry of
AB. Then the ij th entry of λAB is λcij .
The ij th entry of A(λB) is
q
dij = k=1 aik λbkj
= λ qk=1 aik bkj
= λcij
As the two matrices have the same entries they are equal.

2.3 Row Operations

Recall the definition of an Augmented Matrix from the first lecture. Here’s an example to help.
Exercise 2.3.1. Find the Augmented matrix for the following system of linear equations

−x + y + 2z = 2
3x − y + z = 6
−x + 3y + 4z = 4
⎛ ⎞⎛ ⎞ ⎛ ⎞
−1 1 2 x 2
⎝ 3 −1 1 ⎠ ⎝ y ⎠ = ⎝ 6 ⎠
−1 3 4 z 4
⎛ ⎞
−1 1 2 2
⎝ 3 −1 1 6 ⎠
−1 3 4 4

From School you know how to solve systems of linear equations. There are 3 operations you can do:

• multiply an equation by a non-zero factor.

• Add a multiple of one equation to another

• Swap equations around.

In the augmented matrix format we can do these operations more efficiently.

7
 Definition 2.3.2. Elementary row operations (e.r.o’s) are performed on an augmented matrix.
There are three allowable operations:

• Multiply a row by any (non-zero) number

• Add to any row a multiple of another row

• Interchange two rows

Note that the elementary row operations amount to the actions we could take on the original equa-
tions.

Remark 2.3.3 1. Performing row operations preserves the solutions of a linear system.

2. Each row operation has an inverse row operation.

Example 2.3.4.

⎛ ⎞
3 −2 1 −6 R3 → 14 R3
3x − 2y + z = −6 (1) ⎝ 4 6 −3 5 ⎠ −−−−− −→
4x + 6y − 3z = 5
⎛ −4 4 0 12 ⎞
(2)
−4x + 4y = 12 (3) 3 −2 1 −6
R →R +4R3
⎝ 4 6 −3 5 ⎠ −−2−−−2−−−→
R1 →R1 +3R3
First multiply (3) by 14 : ⎛ −1 1 0 3 ⎞
0 1 1 3
R →R −10R1
⎝ 0 10 −3 17 ⎠ −−2−−−2−−−−→
−x + y = 3
⎛ −1 1 0 3 ⎞
(4)
0 1 1 3 R2 →− 1 R2
Then add 3 × (4) to (1) and 4 × (4) to (2) ⎝ 0 0 −13 −13 ⎠ −−−−−13 −−→
−1
⎛ 1 0 3⎞
0 1 1 3
y+z = 3 (5) ⎝ 0 0 1 1 ⎠ R1 →R1 −R2
−− −−−−−→
10y − 3z = 17 (6) −1 1 0 3
⎛ ⎞
0 1 0 2
R →−R3 +R1
Then take 10 × (5) from (6) ⎝ 0 0 1 1 ⎠ −−3−−−−− −−→
⎛ −1 1 0 3 ⎞
−13z = −13 (7) 0 1 0 2
R1 →R2 , R2 →R3
⎝ 0 0 1 1 ⎠ −−−−−−−−−−−−→
R3 →R1
So z = 1. Plug this into (5): ⎛ 1 0 0 −1 ⎞
1 0 0 −1
⎝ 0 1 0 2 ⎠
y+1=3 0 0 1 1

We can read this off:

So y = 2. Plug this into (4):
⎛ ⎞⎛ ⎞ ⎛ ⎞
1 0 0 x −1
−x + 2 = 3 ⎝ 0 1 0 ⎠⎝ y ⎠ = ⎝ 2 ⎠
0 0 1 z 1
So x = −1 So we get x = −1, y = 2, z = 1.

8
 Definition 2.3.5. Two systems of linear equations are equivalent if either:

• They are both inconsistent.

• The augmented matrix of the first system can be obtained using row operations from the
augmented matrix of the second system and vice versa.

Remark 2.3.6 Equivalently, by Remark 2.3.3 two systems of linear equations are equivalent if and only
if they have the same set of solutions.

If a row consists of mainly 0s and 1s it becomes easier to read off the solutions to the equations. For
example:

Example 2.3.7. If we are working in unknowns x, y, z:


−2 1 2 2 Whereas 
3 −3 1 5 0 1 0 2
0 0 1 5
Corresponds to
Corresponds to
−2x + y + 2z = 2
3x − 3y + z = 5 y = 2
z = 5

Definition 2.3.8. We say a matrix is in echelon form (ef) if must satisify the following:

• All of the zero rows are at the bottom.

• The first non-zero entry in each row is 1.

• The first non-zero entry in row i is strictly to the left of the first non-zero entry in row i + 1.

We say a matrix is in row reduced echelon form (rref) if it is in echelon form and:

• The first non-zero entry in row i appears in column j, then every other element in column j
is zero.

Example 2.3.9. ⎛ ⎞ ⎛ ⎞
1 1 2 2 1 1 0 0
⎜ 0 1 7 12 ⎟ ⎜ 0 0 1 0 ⎟
⎜ ⎟ ⎜ ⎟
⎝ 0 0 1 −10 ⎠ ⎝ 0 0 0 1 ⎠
0 0 0 0 0 0 0 0
EF RREF

2.4 Elementary matrices

Elementary row operations can be carried out using matrix multiplication.

9
Definition 2.4.1. Any matrix that can be obtained from an identity matrix by means of one elemen-
tary row operation is an elementary matrix.
There are three types of elementary matrix:

• The general form of the elementary matrix which multiplies a row by any (non-zero) number,
α is of the form ⎛ ⎞
1 ... 0 ... 0
⎜ ... ... ... ... ... ⎟
⎜ ⎟
Er (α) = ⎜
⎜ 0 . . . α . . . 0 ⎟
⎟
⎝ ... ... ... ... ... ⎠
0 ... 0 ... 1
where all elements on row r is multiplied by α.

• The general form of the elementary matrix which adds a multiple of a row by any non-zero
number α to another is of the form
⎛ ⎞
1 ... 0 ... 0 ... 0
⎜ ... ... ... ... ... ... ... ⎟
⎜ ⎟
Ers (α) = ⎜⎜ 0 ... 1 ... α ... 0 ⎟
⎟
⎝ ... ... ... ... ... ... ... ⎠
0 ... 0 ... 0 ... 1

where all elements of s are multiplied by α and added to row r.

• The general form of the elementary matrix which interchanges two rows is of the form
⎛ ⎞
1 ... 0 ... 0 ... 0
⎜ ... ... ... ... ... ... ... ⎟
⎜ ⎟
⎜ 0 ... 0 ... 1 ... 0 ⎟
⎜ ⎟
Ers = ⎜
⎜ . . . . . . . . . . . . . . . . . . . . . ⎟
⎟
⎜ 0 ... 1 ... 0 ... 0 ⎟
⎜ ⎟
⎝ ... ... ... ... ... ... ... ⎠
0 ... 0 ... 0 ... 1

where r and s are the rows to interchange.

10
Example 2.4.2. Find the string of elementary matrices that correspond to the following row opera-
tions:

⎛ ⎞
⎛ ⎞ 1 0 0
0 1 1 3 R2 →− 1 R2 ⎝ 0 −1 0 ⎠
⎝ 0 0 −13 −13 ⎠ −−−−−13 −−→ 13
−1 1 0 3 ⎛0 0 1⎞
⎛ ⎞ 1 −1 0
0 1 1 3
R1 →R1 −R2 ⎝ 0 1 0 ⎠
⎝ 0 0 1 1 ⎠ −− −−−−−→
⎛ 0 0 1 ⎞
⎛ −1 1 0 3 ⎞ 1 0 0
0 1 0 2
R →−R3 +R1 ⎝ 0 1 0 ⎠
⎝ 0 0 1 1 ⎠ −−3−−−−− −−→
⎛ 1 0 −1 ⎞ ⎛ ⎞
⎛ −1 1 0 3 ⎞ 1 0 0 0 1 0
0 1 0 2
R1 →R2 ,R2 →R3 ,R3 →R1 ⎝ 0 0 1 ⎠ ⎝ 1 0 0 ⎠
⎝ 0 0 1 1 ⎠ −−−−−−−−−−−−−−−→
0 1 0 0 0 1
⎛ 1 0 0 −1 ⎞
1 0 0 −1
⎝ 0 1 0 2 ⎠
0 0 1 1

Exercise 2.4.3. Find the string of elementary matrices corresponding to the following row opera-
tions.

⎛ ⎞
3 −2 1 −6 R3 → 14 R3
⎝ 4 6 −3 5 ⎠ −−−−− −→
⎛ −4 4 0 12 ⎞
3 −2 1 −6
R →R −4R3
⎝ 4 6 −3 5 ⎠ −−2−−−2−−−→
R1 →R1 +3R3
⎛ −1 1 0 3 ⎞
0 1 1 3
R2 →R2 −10R1
⎝ 0 10 −3 17 ⎠ −−−−−−−−−→
−1 1 0 3

*****Mentimeter***************

Theorem 2.4.4. Let A be a m × n matrix and let E be an elementary m × m matrix. The matrix
multiplication EA applies the same elementary row operation on A that was performed on the iden-
tity matrix to obtain E.

Proof: exercise.

11
2.5 More matrices
Definition 2.5.1. We say a matrix is square if it has the same number of rows as it does columns
(i.e. its a member of Mn×n (F ) for some field F ).

Definition 2.5.2.
A square matrix A = aij ∈ Mn×n (F ) is said to be:

1. upper triangular if aij = 0 wherever i > j . A has zeros for all its elements below the
diagonal.

2. lower triangular if aij = 0 wherever i < j . A has zeros for all its elements above the
diagonal.

3. diagonal if aij = 0 wherever i = j . That is to say A has zeros for all its elements except
those on the main diagonal.

Example 2.5.3. ⎛ ⎞ ⎛ ⎞ ⎛ ⎞
1 1 2 1 0 0 1 0 0
⎝ 0 1 7 ⎠ ⎝ 2 0 0 ⎠ ⎝ 0 −2 0 ⎠
0 0 1 0 0 0 0 0 0
Upper triangular Lower triangular diagonal

Definition 2.5.4. The n × n identity matrix is denoted by In . An identity matrix has all of its
diagonal entreis equal to 1 and all other entries equal to 0. It is called the identity matrix because it
is the multiplicative identity matrix for n × n matrices, i.e.
For A ∈ Mn×n (R), In A = AIn = A

Definition 2.5.5. If, for a square matrix B, if there exists another square matrix B−1 such that
BB−1 = I = B−1 B, then we say that B is invertible, and B−1 is an inverse of B.

It is important to realise that the matrix B might not have an inverse: B−1 might not exist.

Definition 2.5.6. A matrix without an inverse is called a singular matrix.

  1
2 0 2 0
Example 2.5.7. Let A = , verify that it has an inverse: B= 1 .
1 −1 2 −1
  1 
2 0 2 0 1 0
AB = 1 =
 11 −1  2 −1  0 1
2 0 2 0 1 0
BA = 1 =
2 −1 1 −1 0 1

12
 Theorem 2.5.8. The inverse of a given matrix is unique. If there exist square matrices A, B, C
such that AB = I = CA, then B = C.

Proof Suppose that AB = BA = I and AC = CA = I, then

B = BI
= B(AC)
= (BA)C
= IC
= C

This theorem shows that if a matrix A is invertible, we can talk about the inverse of A, denoted by A−1 . In
some circumstances, we can say that a matrix is invertible, and we can find an expression for its inverse,
without knowing exactly what the matrix is.
Exercise 2.5.9. Suppose A, B ∈ Mn×n (R) are both invertible. Show that AB is invertible by
finding its inverse.
********MENTIMETER***************

(a) A−1 B −1

(b) B −1 A−1

(c) BAA−1 B −1 B −1 A−1

Definition 2.5.10. If A = [aij ]m×n , then the Transpose of A is AT = [aji ]n×m .

Example 2.5.11. If
⎛ ⎞
 1 4
1 0 5
A= , then AT = ⎝ 0 2 ⎠
4 2 1
5 1

and we can see that the transpose of a 2 × 3 matrix must be a 3 × 2 matrix.

Exercise 2.5.12. Let A ∈ Mn×m (R), B ∈ Mm×p (R), (AB)T = B T AT .

Proof:
First remark that B T AT is defined and has order p × n, not also AB has order n × p so (AB)T has
order p × n.
Let A = (aij ) and B = (bij )

m
• The ij th entry of AB is k=1 aik bkj . This is the jith entry of (AB)T
m m m
• The jith entry of B T AT is T T
k=1 (b )jk (a )ki = k=1 (b)kj (a)ik = k=1 aik bkj

13
 Theorem 2.5.13. Given an invertible square matrix A, then AT is also invertible, and (AT )−1 =
(A−1 )T .

Proof From the definition of the inverse

AA−1 = I
(AA−1 )T = IT
= I
−1 T T
(A ) A = I

Also

A−1 A = I
(A−1 A)T = IT
= I
T −1 T
A (A ) = I

Equations 8 and 8 prove that (A−1 )T is the (unique) inverse of AT , as required.

2.6 Inverses using row operations

We can use Elementary matrices to find inverses of matrices (it they exist).

Theorem 2.6.1. Every elementary matrix is invertible and the inverse is also an elementary matrix.

Proof
Matrix multiplication can be used to check that

Er (α)Er (α−1 ) = Er (α−1 )Er (α) = I

Ers (α)Ers (α−1 ) = Ers (α−1 )Ers (α) = I
Ers (α)Ers (α) = I

Alternatively, the results can be checked by considering the corresponding ero’s. Hence

Er (α)−1 = Er (α−1 ), Ers (α)−1 = Ers (−α), Ers −1 = Ers

14
Theorem 2.6.2. If the square matrix A can be row-reduced to an identity matrix by a sequence of
elementary row operations, then A is invertible and the inverse of A is found by applying the same
sequence of elementary row operations to I.

Proof
Let A be a square matrix, then A can be row-reduced to I by a sequence of elementary row op-
erations. Let E1 , E2 , E3 . . . , Er be the elementary matrices corresponding to the elementary row
operations, so that
Er . . . E2 E1 A = I (8)

But Theorem 2.6.1 states that Er , . . . , E2 , E1 are invertible. Multiplying Equation 8 by E−1 −1 −1
1 E2 . . . E r
gives A = E−1 −1 −1
1 E2 . . . Er . Since A is a product of elementary matrices, it is invertible (using The-
orem 2.6.1) and
A−1 = (E−1 −1 −1 −1
1 E 2 . . . Er ) = (Er . . . E2 E1 )I

⎛ ⎞
1 0 1
Example 2.6.3. Let A = ⎝ 1 2 0 ⎠ find A−1 .
3 0 4
The method consists of writing the identity matrix I to the right of our given matrix, and then using
the same elementary row operations on both matrices to turn the left-hand matrix into I. When this
has been achieved, the right-hand matrix will have been transformed into the inverse matrix, A−1 .
First, we construct the augmented matrix A|I, by writing the identity matrix to the right of the matrix
A, ⎛ ⎞
1 0 1 1 0 0
⎝ 1 2 0 0 1 0 ⎠
3 0 4 0 0 1
After our row operations, this matrix will be transformed into I|A−1 .

15
The steps might be as follows:
R →R −3R
−−3−−−3−−−→
1
⎛ ⎞
1 0 1 1 0 0
⎝ 1 2 0 0 1 0 ⎠
0 0 1 −3 0 1
R →R −R
2
−−−−−2−−→
1
⎛ ⎞
1 0 1 1 0 0
⎝ 0 2 −1 −1 1 0 ⎠
0 0 1 −3 0 1
R →R +R
2
−−−−−2−−→
3
⎛ ⎞
1 0 1 1 0 0
⎝ 0 2 0 −4 1 1 ⎠
0 0 1 −3 0 1
R →R −R
1
−−−−−1−−→
3
⎛ ⎞
1 0 0 4 0 −1
⎝ 0 2 0 −4 1 1 ⎠
0 0 1 −3 0 1
R2 → 1 R2
2
−−−−− −→ ⎛ ⎞
1 0 0 4 0 −1
⎝ 0 1 0 −2 1 1 ⎠
2 2
0 0 1 −3 0 1

We have found the inverse of our matrix. We could check by doing the matrix multiplication:
⎛ ⎞⎛ ⎞ ⎛ ⎞
4 0 −1 1 0 1 1 0 0
⎝ −2 1 1 ⎠ ⎝ 1 2 0 ⎠ = ⎝ 0 1 0 ⎠
2 2
−3 0 1 3 0 4 0 0 1

as desired.

16