SYSTEM OF LINEAR EQUATIONS

Dr. Isaac Azure

Department of Mathematics
Kwame Nkrumah University of Science and Technology
Google Scholar: https://scholar.google.com/citations?user=t3p4ESAAAAAJhl=enauthuser=1

ResearchGate ID: https://www.researchgate.net/profile/Isaac-Azure/research https://www.researchgate.net/profile/Isaac-Azure/research

Dr. Azure (KNUST-Maths) System of Linear Equations 1 / 41

Lecture Outline

Introduction

1 Gaussian Elimination

2 Gauss-Jordan Elimination Method

3 Inverse Approach
Computing Inverse

Dr. Azure (KNUST-Maths) System of Linear Equations 2 / 41

 Introduction

Introduction to linear systems

This lecture introduces the basics of solving linear equations using some elimination methods.
A system of n linear equations in n unknowns x 1 , x 2 , · · · , x n is a family of equations

a 11 x 1 + a 12 x 2 + · · · + a 1n x n = b 1 (1)
a 21 x 1 + a 22 x 2 + · · · + a 2n x n = b 2 (2)
.. .
. = ..
a n1 x 1 + a n2 x 2 + · · · + a nn x n = b n (3)

We wish to determine if such a system has a solution, that is to find out if there exist numbers
x 1 , x 2 , · · · , x n that satisfy each of the equations simultaneously.
We say that the system is consistent if it has a solution. Otherwise, the system is called
inconsistent.
More generally, the above is called a homogeneous system of linear equations when b1 = b2 =
· · · = bn = 0

Dr. Azure (KNUST-Maths) System of Linear Equations 3 / 41

 Introduction

Geometrically, solving a system of linear equations in two (or three) unknowns is equivalent to
determining whether or not a family of lines (or planes) has a common point of intersection.

(a) Consistent with independent solutions (b) Inconsistent (c) Consistent with dependent solution.

Dr. Azure (KNUST-Maths) System of Linear Equations 4 / 41

 Introduction

Example
Solve the system

2x + 3y = 6
x−y =2

By the elimination or substitution method, the solution to the system is

x = 12/5, y = 2/5 (4)

This approach would be tedious and virtually unworkable for a large number of equations. We will
develop a means of solving systems by using the matrix form of the equation.

Dr. Azure (KNUST-Maths) System of Linear Equations 5 / 41

 Introduction

Solving Square Linear Systems

1 Given the system
a 11 x 1 + a 12 x 2 + · · · + a 1n x n = b 1 (5)
a 21 x 1 + a 22 x 2 + · · · + a 2n x n = b 2 (6)
.. .
. = ..
a n1 x 1 + a n2 x 2 + · · · + a nn x n = b n (7)

2 This can be recast in matrices as

a 11 a 12 ··· a 1n x1 b1
    
 a 21 a 22 ··· a 2n   x2   b2 
 . .. ..   . = .  (8)
    
 .. ..
. . .   ..   .. 
a n1 a n2 ··· a nn xn bn

3 Which is of the general from

Ax = b (9)
Dr. Azure (KNUST-Maths) System of Linear Equations 6 / 41
 Introduction

The matrix A is appended by b to form what we call the Augmented Matrix. This is denoted by

a 11 a 12 a 13 · · · a 1n b1
 
 a 21 a 22 a 23 · · · a 2n b2 
A|b =  . .. .. ..  (10)
 
 .. ..
. . . . 
a n1 a n2 a n3 · · · a nn bn

The matrix is really just a compact way of writing the system of equations.

Dr. Azure (KNUST-Maths) System of Linear Equations 7 / 41

 Introduction

Elementary row operations

Three main types of elementary row operations can be performed on matrices:
1 Interchanging two rows: R i ↔ R j interchanges rows i and j (Partial Pivoting).
2 Multiplying a row by a nonzero scalar: R i → t Ri multiplies row i by the nonzero scalar t .
3 Adding a multiple of one row to another row: R j → R j + t R i add t times row i to row j .

Note
The above operations are implemented on the augmented matrix during elimination approach, and
will not change the solution to the system.

Note that performing these operations on the matrix is equivalent to performing the same operations
directly on the equations.

Definition (Row equivalence)

Matrix A is row-equivalent to matrix B if B is obtained from A by a sequence of elementary row
operations.

Dr. Azure (KNUST-Maths) System of Linear Equations 8 / 41

 Gaussian Elimination

Gaussian Elimination
Gaussian elimination performs row operations on the augmented matrix until the portion
corresponding to the coefficient matrix is reduced to upper-triangular form.

Definition (Upper-Triangular Matrix)

An n × n matrix A whose entries are of the form
(
ai j i≤j
Ui j =
0 i>j

is called an upper triangular matrix. Specifically:

a 11 a 12 ··· a 1n
 
 0 a 22 ··· a 2n 
U = . .. .. 
 
 .. ..
. . . 
0 0 ··· a nn

Dr. Azure (KNUST-Maths) System of Linear Equations 10 / 41

 Gaussian Elimination

The method starts with

a 11 a 12 a 13 · · · a 1n b1
 
 a 21 a 22 a 23 · · · a 2n b2 
A|b =  . .. .. ..  (11)
 
 .. ..
. . . . 
a n1 a n2 a n3 · · · a nn bn
then using row elimination produces a matrix in upper triangular form

c 11 c 12 c 13 · · · c 1n d1
 
 0 c 22 c 23 · · · c 2n d2 
 . .. .. ..  (12)
 
 .. ..
. . . . 
0 0 0 · · · c nn dn
which is easy to solve using back substitution.

Dr. Azure (KNUST-Maths) System of Linear Equations 11 / 41

 Gaussian Elimination

Gaussian Elimination Procedure

1
Begin at element a11 . If a11 = 0, exchange rows so a11 ̸= 0.

2
Now make all the elements below a11 zero using elementary row operations
(ERO)

3
Now perform the same process of elimination using a22 as the pivot.

4
Repeat this process until the matrix is in upper-triangular form.

5
Then execute back substitution to compute the solution.

Dr. Azure (KNUST-Maths) System of Linear Equations 12 / 41

 Gaussian Elimination

Example
Solve the following system of equations using the Gaussian elimination method

x1 + x2 + x3 = 1
4x 1 + 3x 2 − x 3 = 6
3x 1 + 5x 2 + 3x 3 = 4

Dr. Azure (KNUST-Maths) System of Linear Equations 13 / 41

 Gaussian Elimination

Solution
This problem is first recast into matrix form as
    
1 1 1 x1 1
4 3 −1 x 2  = 6 (13)
3 5 3 x3 4

The augmented matrix is deduced from (13) as

 
1 1 1 1
 4 3 −1 6  (14)
3 5 3 4
Now, let reduce (14) to an upper triangular matrix using elementary row operations.

Dr. Azure (KNUST-Maths) System of Linear Equations 14 / 41

 Gaussian Elimination

Iteration 1
The first pivot point is 1 in the first column (first term).
 
1 1 1 1
 4 3 −1 6  (15)
3 5 3 4
We are to reduce the values beneath 1, that is 4 and 3 to zeros using elementary row operations.
The following manipulations are used here.
N R 2 = R 2 − 4R 1 , =⇒ 4 → 0
N R 3 = R 3 − 3R 1 , =⇒ 3 → 0
Note that these computations affect the entire row.
Note
NR is used to denote New Row, such that N R 2 is read ‘new row 2’.
R is used to denote Row, such that R 1 is read ‘row 1’.

Therefore the new matrix is  

1 1 1 1
 0 −1 −5 2  (16)
0 2 0 1
Dr. Azure (KNUST-Maths) System of Linear Equations 15 / 41
 Gaussian Elimination

Iteration 2
The second pivot point is 1 in the second column (diagonal value).

1 1 1 1
 
 0 −1 −5 2  (17)
0 2 0 1
We are to reduce the value beneath 1, that is 2 to zero using elementary row operations. The
following manipulations are used here.
N R 3 = R 3 + 2R 2 , =⇒ 2 → 0
Therefore the new matrix is  
1 1 1 1
 0 −1 −5 2  (18)
0 0 −10 5

Dr. Azure (KNUST-Maths) System of Linear Equations 16 / 41

 Gaussian Elimination

Now we have an upper triangular matrix. So the solution could be finally obtained using back
substitution. That is substitution and solving from the last row. We have

−10x 3 = 5 =⇒ x 3 = −0.5

−x 2 − 5x 3 = 2, but x 3 = −0.5
−x 2 − 5(−0.5) = 2
x 2 = 0.5

x1 + x2 + x3 = 1
x 1 + 0.5 − 0.5 = 1
x1 = 1

These are the solutions to the given problem.

Dr. Azure (KNUST-Maths) System of Linear Equations 17 / 41

 Gaussian Elimination

Dimension
Suppose a system of n linear equations in n unknowns has augmented matrix C and that C is row-
equivalent to a matrix D in upper-triangular form. Then C and D have dimension n × (n + 1).

Inconsistent System
If we perform elementary row operations on the augmented matrix of the system and get a matrix
with one of its rows equal to
[0 0 0 · · · 0 b], where b ̸= 0 (19)
or a row of the form
[0 0 0 · · · 0] (20)
then there may be no solution or infinitely many solutions.

Dr. Azure (KNUST-Maths) System of Linear Equations 18 / 41

 Gaussian Elimination

Homogeneous Systems

An n × n system of homogeneous linear equations

a 11 x 1 + a 12 x 2 + · · · + a 1n x n = 0
a 21 x 1 + a 22 x 2 + · · · + a 2n x n = 0
.. .
. = ..
a n1 x 1 + a n2 x 2 + · · · + a nn x n = 0

is always consistent since x 1 = 0, · · · , x n = 0 is a solution. This solution is called the trivial solution,
and any other solution is called a nontrivial solution.

If the homogeneous system Ax = 0 has only the trivial solution, then A is nonsingular; that is A −1
exists.

Dr. Azure (KNUST-Maths) System of Linear Equations 19 / 41

 Gaussian Elimination

Definition (Elementary Matrix)

An elementary matrix is any matrix that can be obtained from the identity matrix by performing a
single elementary row operation.

The elementary matrix formed by interchanging the first and third rows of the 3 × 3 identity matrix I ,
is  
0 0 1
0 1 0
1 0 0

Dr. Azure (KNUST-Maths) System of Linear Equations 20 / 41

 Gauss-Jordan Elimination Method

Gauss-Jordan Elimination Method

1 This method is based on the idea of reducing the given system of equation

Ax = b

to a diagonal matrix of the form

Dx = z
D is the diagonal matrix, and z is the new column vector on the right hand side.
2 All solution techniques of the Gaussian elimination method do apply here. While a given
problem is reduced to an upper triangular matrix in the case of Gaussian elimination, it is
reduced to a diagonal matrix in the case of Gauss-Jordan elimination.

Dr. Azure (KNUST-Maths) System of Linear Equations 22 / 41

 Gauss-Jordan Elimination Method

Example
Solve the following system of equations using the Gauss-Jordan elimination method

x1 + x2 + x3 = 1
4x 1 + 3x 2 − x 3 = 6
3x 1 + 5x 2 + 3x 3 = 4

This problem is first recast into matrix form as

    
1 1 1 x1 1
4 3 −1 x 2  = 6 (21)
3 5 3 x3 4

Dr. Azure (KNUST-Maths) System of Linear Equations 23 / 41

 Gauss-Jordan Elimination Method

The augmented matrix is deduced from (41) as

 
1 1 1 1
 4 3 −1 6  (22)
3 5 3 4
Note that we are to reduce value above and beneath the leading diagonals to zeros.
Now, let reduce (22) to an upper triangular matrix using elementary row operations.

Dr. Azure (KNUST-Maths) System of Linear Equations 24 / 41

 Gauss-Jordan Elimination Method

Iteration 1

The first pivot point is 1 in the first column (first term).

 
1 1 1 1
 4 3 −1 6  (23)
3 5 3 4

We are to reduce the values beneath 1, that is 4 and 3 to zeros using elementary row operations.
The following manipulations are used here.
N R 2 = R 2 − 4R 1 , =⇒ 4 → 0
N R 3 = R 3 − 3R 1 , =⇒ 3 → 0
Therefore the new matrix is  
1 1 1 1
 0 −1 −5 2  (24)
0 2 0 1

Dr. Azure (KNUST-Maths) System of Linear Equations 25 / 41

 Gauss-Jordan Elimination Method

Iteration 2

The second pivot point is 1 in the second column (diagonal value).

1 1 1 1
 
 0 −1 −5 2  (25)
0 2 0 1
We are to reduce the value beneath 1, that is 2 to zero using elementary row operations. The
following manipulations are used here.
N R 3 = R 3 + 2R 2 , =⇒ 2 → 0
Therefore the new matrix is  
1 1 1 1
 0 −1 −5 2  (26)
0 0 −10 5

Dr. Azure (KNUST-Maths) System of Linear Equations 26 / 41

 Gauss-Jordan Elimination Method

Iteration 3

Since we need a diagonal matrix, we will start manipulating the other non-diagonal matrix to zero
starting from the leading diagonal of the last column.

1 1 1 1
 
 0 −1 −5 2  (27)
0 0 −10 5
We are to reduce the values above -10, that is -5 and 1 to zeros elementary row operations. The
following manipulations are used here.
N R 2 = 2R 2 − R 3 , =⇒ −5 → 0
N R 1 = 10R 1 + R 3 , =⇒ 1 → 0
The new matrix is

10 10 0 15
 
 0 −2 0 −1  (28)
0 0 −10 5

Dr. Azure (KNUST-Maths) System of Linear Equations 27 / 41

 Gauss-Jordan Elimination Method

Iteration 4

The next pivot point is -2 in the second column (diagonal value).

 
10 10 0 15
 0 −2 0 −1  (29)
 
0 0 −10 5
We are to reduce the value above −2, that is 10 to zero using elementary row operations. The
following manipulations are used here.
N R 1 = R 1 − 5R 2 , =⇒ 10 → 0
The new matrix

10 0 0 10
 
 0 −2 0 −1  (30)
0 0 −10 5

Dr. Azure (KNUST-Maths) System of Linear Equations 28 / 41

 Gauss-Jordan Elimination Method

Since we have reduced the system to a diagonal matrix, the values of x can be obtained using
direct substitution. That is

10x 1 = 10 =⇒ x 1 = 1
−2x 2 = −1 =⇒ x 2 = 0.5
−10x 3 = 5 =⇒ x 3 = −0.5

Dr. Azure (KNUST-Maths) System of Linear Equations 29 / 41

 Inverse Approach Computing Inverse

Computing Inverse
Begin by setting an augmented matrix of the form A|I . For 3 × 3 matrix we have
 
a 11 a 12 a 13 1 0 0
A|I =  a 21 a 22 a 23 0 1 0  (31)
a 31 a 32 a 33 0 0 1

Then perform ERO on the coefficient matrix to obtain a diagonal matrix. All the while performing
the row operations on the augmented matrix A|I . When the Gauss-Jordan procedure is completed,
we obtain
[A|I ] −→ [I |A −1 ] (32)

Note
Partial pivoting can also be done using the augmented matrix [A|I ]. However, we cannot first
interchange the rows of A and then find the inverse. Then, we would be finding the inverse of a
different matrix.

Dr. Azure (KNUST-Maths) System of Linear Equations 31 / 41

 Inverse Approach Computing Inverse

Example
 
1 1 1
Find the inverse of the matrix A = 4 3 −1
3 5 3

We perform the following elementary row transformations and do the eliminations

Iteration 1

   
1 1 1 1 0 0 −→ 1 1 1 1 0 0
 4 3 −1 0 1 0  N R 2 = R 2 − 4R 1  0 −1 −5 −4 1 0  (33)
3 5 3 0 0 1 N R 3 = R 3 − 3R 1 0 2 0 −3 0 1

Dr. Azure (KNUST-Maths) System of Linear Equations 32 / 41

 Inverse Approach Computing Inverse

Iteration 2

   
1 1 1 1 0 0 1 1 1 1 0 0
−→
 0 −1 −5 −4 1 0   0 1 5 4 −1 0  (34)
N R 2 = −R 2
0 2 0 −3 0 1 0 2 0 −3 0 1

Iteration 3

   
1 1 1 1 0 0 −→ 1 0 −4 −3 1 0
 0 1 5 4 −1 0  N R1 = R1 − R2  0 1 5 4 −1 0  (35)
0 2 0 −3 0 1 N R 3 = R 3 − 2R 2 0 0 −10 −11 2 1

Dr. Azure (KNUST-Maths) System of Linear Equations 33 / 41

 Inverse Approach Computing Inverse

Iteration 4

 
1 0 −4 −3 1 0
 0 −→
1 5 4 −1 0 
N R 3 = R 3 /(−10)
0 0 −10 −11 2 1
 
1 0 −4 −3 1 0
 0 1 5 4 −1 0 
0 0 1 11/10 −2/10 −1/10

Dr. Azure (KNUST-Maths) System of Linear Equations 34 / 41

 Inverse Approach Computing Inverse

Iteration 5

 
1 0 −4 −3 1 0 −→
 0 1 5 4 −1 0  N R 1 = R 1 + 4R 3
0 0 1 11/10 −2/10 −1/10 N R 2 = R 2 − 5R 3
 
1 0 0 14/10 2/10 −4/10
 0 1 0 −15/10 0 5/10 
0 0 1 11/10 −2/10 −1/10

Therefore, the inverse of the given matrix is given by

 
7/5 1/5 −2/5
 −3/2 0 1/2 
11/10 −1/5 −1/10

Solve the same problem by employing partial pivoting.

Dr. Azure (KNUST-Maths) System of Linear Equations 35 / 41
 Inverse Approach Computing Inverse

Solving A System of Linear Equation Equation using the Inverse

Approach
1 Given the system

a 11 x 1 + a 12 x 2 + · · · + a 1n x n = b 1 (36)
a 21 x 1 + a 22 x 2 + · · · + a 2n x n = b 2 (37)
.. .
. = ..
a n1 x 1 + a n2 x 2 + · · · + a nn x n = b n (38)

2 Which is of the general from

Ax = b (39)
3 Pre-multiply both sides by A −1
x = A −1 b (40)
−1
Note that A A =I

Dr. Azure (KNUST-Maths) System of Linear Equations 36 / 41

 Inverse Approach Computing Inverse

Example
Solve the following system of equations using the inverse approach

x1 + x2 + x3 = 1
4x 1 + 3x 2 − x 3 = 6
3x 1 + 5x 2 + 3x 3 = 4

This problem is first recast into matrix form as

    
1 1 1 x1 1
4 3 −1 x 2  = 6 (41)
3 5 3 x3 4

Then we are to find

x = A −1 b (42)

Dr. Azure (KNUST-Maths) System of Linear Equations 37 / 41

 Inverse Approach Computing Inverse

Using A −1 as obtained from above we have

    
x1 7/5 1/5 −2/5 1
x 2  =  −3/2 0 1/2  6 (43)
x3 11/10 −1/5 −1/10 4

7 6 8
x1 = + − =1 (44)
5 5 5
3 4 1
x2 = − + 0 + = (45)
2 2 2
11 6 4 1
x3 = − − =− (46)
10 5 10 2

Dr. Azure (KNUST-Maths) System of Linear Equations 38 / 41

 Inverse Approach Computing Inverse

Exercise 1

1 Solve the homogeneous system

−3x 1 + x 2 + x 3 + x 4 = 0
x 1 − 3x 2 + x 3 + x 4 = 0
x 1 + x 2 − 3x 3 + x 4 = 0
x 1 + x 2 + x 3 − 3x 4 = 0

2 For which rational numbers λ does the homogeneous system have a nontrivial solution?

x + (λ − 3)y = 0
(λ − 3)x + y = 0

Dr. Azure (KNUST-Maths) System of Linear Equations 39 / 41

 Inverse Approach Computing Inverse

Exercise 2
Solving the following system of equations using
1 Gaussian elimination method without pivoting
2 Gaussian elimination method with partial pivoting
3 Gaussian elimination method with complete pivoting
4 Gauss-Jordan elimination method.
5 Inverse Approach

10x + 4y − 2z = 20 2a + b + c + d = 2
3x + 12y − z = 28 4a + 2c + d = 3
x + 4y + 7z = 2 3a + 2b + 2c = −1
a + 3b + 2c + 6d = 2

Dr. Azure (KNUST-Maths) System of Linear Equations 40 / 41

 END OF LECTURE
THANK YOU

Dr. Azure (KNUST-Maths) System of Linear Equations 41 / 41