SUPPLEMENTARY LECTURE NOTES

Gaussian Elimination

After reading this lecture, you should be able to:

1. solve a set of simultaneous linear equations using Naive Gaussian elimination,
2. learn the pitfalls of the Naive Gaussian elimination method,
3. understand the effect of round-off error when solving a set of linear equations with
the Naive Gaussian elimination method,

How is a set of equations solved numerically?

One of the most popular techniques for solving simultaneous linear equations is the Gaussian
elimination method. The approach is designed to solve a general set of n equations and n
unknowns
a11x1  a12x2  a13x3  ...  a1n xn  b1
a21x1  a22 x2  a23x3  ...  a2n xn  b2
. .
. .
. .
an1x1  an 2 x2  an3 x3  ...  ann xn  bn
Gaussian elimination consists of two steps
1. Forward Elimination of Unknowns: In this step, the unknown is eliminated in each
equation starting with the first equation. This way, the equations are reduced to one
equation and one unknown in each equation.
2. Back Substitution: In this step, starting from the last equation, each of the unknowns
is found.

Forward Elimination of Unknowns:

In the first step of forward elimination, the first unknown, x1 is eliminated from all rows
below the first row. The first equation is selected as the pivot equation to eliminate x1 . So,
to eliminate x1 in the second equation, one divides the first equation by a11 (hence called the
pivot element) and then multiplies it by a 21 . This is the same as multiplying the first
equation by a21 / a11 to give
a21 a a
a21 x1  a12 x2  ...  21 a1n xn  21 b1 Now, this equation can be subtracted from the
a11 a11 a11
second equation to give
 a   a  a
 a 22  21 a12  x2  ...   a 2 n  21 a1n  xn  b2  21 b1
 a11   a11  a11
or
 x2  ...  a2 n xn  b2
a22

1
Numerical Analysis MATH351/352

where
a21
  a22 
a22 a12
a11

a21
a2 n  a2 n  a1n
a11
This procedure of eliminating x1 , is now repeated for the third equation to the n th equation
to reduce the set of equations as
a11x1  a12 x2  a13 x3  ...  a1n xn  b1
a22 x2  a23
 x3  ...  a2 n xn  b2
a32 x2  a33
 x3  ...  a3 n xn  b3
. . .
. . .
. . .
an 2 x2  an 3 x3  ...  ann
 xn  bn
This is the end of the first step of forward elimination. Now for the second step of forward
elimination, we start with the second equation as the pivot equation and a22  as the pivot
element. So, to eliminate x2 in the third equation, one divides the second equation by a22 
(the pivot element) and then multiply it by a32  . This is the same as multiplying the second
equation by a32  / a22  and subtracting it from the third equation. This makes the coefficient of
x2 zero in the third equation. The same procedure is now repeated for the fourth equation
till the n th equation to give
a11x1  a12 x2  a13 x3  ...  a1n xn  b1
a22 x2  a23  x3  ...  a2 n xn  b2
a33 x3  ...  a3n xn  b3
. .
. .
. .
an3 x3  ...  ann
 xn  bn
The next steps of forward elimination are conducted by using the third equation as a pivot
equation and so on. That is, there will be a total of n  1 steps of forward elimination. At the
end of n  1 steps of forward elimination, we get a set of equations that look like
a11x1  a12 x2  a13 x3  ...  a1n xn  b1
a22 x2  a23  x3  ...  a2 n xn  b2
 x3  ...  a3n xn  b3
a33
. .
. .
. .
ann xn  bnn 1
n 1

2
Gaussian Elimination MATH351/352

Back Substitution:
Now the equations are solved starting from the last equation as it has only one unknown.
b ( n 1)
x n  n( n 1)
a nn
Then the second last equation, that is the (n  1) th equation, has two unknowns: x n and xn1 ,
but x n is already known. This reduces the (n  1) th equation also to one unknown. Back
substitution hence can be represented for all equations by the formula
bii 1   aiji 1 x j
n

j i 1
xi  for i  n 1, n  2,,1
aiii 1
and
bn( n 1)
xn  ( n 1)
a nn

Example 1
The upward velocity of a rocket is given at three different times in Table 1.

Table 1 Velocity vs. time data.

Time, t (s) Velocity, v (m/s)
5 106.8
8 177.2
12 279.2

The velocity data is approximated by a polynomial as

vt   a1t 2  a2t  a3 , 5  t  12
The coefficients a1, a2 , and a3 for the above expression are given by
 25 5 1  a1  106.8 
 64 8 1 a   177.2 
   2  
144 12 1  a3  279.2
Find the values of a1, a2 , and a3 using the Naive Gaussian elimination method. Find the
velocity at t  6, 7.5, 9, 11 seconds.

Solution
Forward Elimination of Unknowns
Since there are three equations, there will be two steps of forward elimination of unknowns.
First step
Divide Row 1 by 25 and then multiply it by 64, that is, multiply Row 1 by 64/25  2.56 .
25 5 1 106.8 2.56 gives Row 1 as

3
Numerical Analysis MATH351/352

64 12.8 2.56 273.408

Subtract the result from Row 2
64 8 1 177.2
 64 12.8 2.56 273.408
0  4.8  1.56  96.208
to get the resulting equations as
 25 5 1   a1   106.8 
 0  4.8  1.56 a    96.208
   2  
144 12 1   a3   279.2 
Divide Row 1 by 25 and then multiply it by 144, that is, multiply Row 1 by 144/25  5.76 .
25 5 1 106.8 5.76 gives Row 1 as
144 28.8 5.76 615.168
Subtract the result from Row 3
144 12 1 279.2
 144 28.8 5.76 615.168
0  16.8  4.76  335.968
to get the resulting equations as
25 5 1   a1   106.8 
 0  4.8  1.56  a     96.208 
   2  
 0  16.8  4.76  a3   335.968
Second step
We now divide Row 2 by –4.8 and then multiply by –16.8, that is, multiply Row 2 by
16.8/  4.8  3.5 .
0  4.8 1.56  96.208 3.5 gives Row 2 as
0  16.8  5.46  336.728
Subtract the result from Row 3
0  16.8  4.76  335.968
 0  16.8  5.46  336.728
0 0 0.7 0.76
to get the resulting equations as
25 5 1   a1   106.8 
 0  4.8  1.56 a    96.208
   2  
 0 0 0.7   a3   0.76 

Back substitution
From the third equation
0.7a3  0.76
0.76
a3 
0.7

4
Gaussian Elimination MATH351/352

 1.08571
Substituting the value of a3 in the second equation,
 4.8a2  1.56a3  96.208
 96.208  1.56a3
a2 
 4.8
 96.208  1.56  1.08571

 4.8
 19.6905
Substituting the value of a 2 and a3 in the first equation,
25a1  5a2  a3  106.8
106.8  5a 2  a3
a1 
25
106.8  5  19.6905  1.08571

25
 0.290472
Hence the solution vector is
 a1  0.290472
a    19.6905 
 2  
 a3   1.08571 
The polynomial that passes through the three data points is then
vt   a1t 2  a2 t  a3
 0.290472t 2  19.6905t  1.08571, 5  t  12
Since we want to find the velocity at t  6, 7.5, 9 and 11 seconds, we could simply substitute
each value of t in vt   0.290472t 2  19.6905t  1.08571 and find the corresponding
velocity. For example, at t  6
v6  0.2904726  19.69056  1.08571
2

 129.686 m/s
However we could also find all the needed values of velocity at t = 6, 7.5, 9, 11 seconds
using matrix multiplication.
t 2 
 
vt   0.290472 19.6905 1.08571  t 
1
 
So if we want to find v6, v7.5, v9, v11, it is given by

6 2 7.5 2 9 2 112 
v6 v7.5 v9 v11  0.290472 19.6905 1.08571  6 7.5 9 11 

1 1 1 1 


5
Numerical Analysis MATH351/352

36 56.25 81 121

 0.290472 19.6905 1.08571  6 7.5 9 11 
 1 1 1 1 
 129.686 165.104 201.828 252.828
v(6)  129.686 m/s
v(7.5)  165.104 m/s
v(9)  201.828 m/s
v(11)  252.828 m/s

Example 2
Use Naive Gaussian elimination to solve
20 x1  15x2  10 x3  45
 3x1  2.249 x2  7 x3  1.751
5x1  x2  3x3  9
Use six significant digits with chopping in your calculations.
Solution
Working in the matrix form
 20 15 10  x1   45 
 3  2.249 7   x  = 1.751
   2  
 5 1 3   x3   9 
Forward Elimination of Unknowns
First step
Divide Row 1 by 20 and then multiply it by –3, that is, multiply Row 1 by  3 / 20  0.15 .
20 15 10 45 0.15 gives Row 1 as
 3  2.25 1.5  6.75
Subtract the result from Row 2
 3  2.249 7 1.751
  3  2.25  1.5  6.75
0 0.001 8.5 8.501
to get the resulting equations as
20 15 10   x1   45 
 0 0.001 8.5  x  = 8.501
   2  
 5 1 3   x3   9 
Divide Row 1 by 20 and then multiply it by 5, that is, multiply Row 1 by 5 / 20  0.25
20 15 10 45 0.25 gives Row 1 as
5 3.75 2.5 11.25
Subtract the result from Row 3

6
Gaussian Elimination MATH351/352

5 1 3 9
 5 3.75 2.5 11.25
0  2.75 0.5  2.25
to get the resulting equations as
20 15 10   x1   45 
 0 0.001 8.5  x  =  8.501 
   2  
 0  2.75 0.5  x3   2.25
Second step
Now for the second step of forward elimination, we will use Row 2 as the pivot equation and
eliminate Row 3: Column 2.
Divide Row 2 by 0.001 and then multiply it by –2.75, that is, multiply Row 2 by
 2.75 / 0.001  2750 .
0 0.001 8.5 8.501 2750 gives Row 2 as
0  2.75  23375  23377.75
Rewriting within 6 significant digits with chopping
0  2.75  23375  23377.7
Subtract the result from Row 3
0  2.75 0.5  2.25
 0  2.75  23375  23377.7
0 0 23375.5 23375.45
Rewriting within 6 significant digits with chopping
0 0 23375.5  23375.4
to get the resulting equations as
20 15 10   x1   45 
 0 0.001 8.5   x 2  =  8.501 

 0 0 23375.5  x3  23375.4
This is the end of the forward elimination steps.
Back substitution
We can now solve the above equations by back substitution. From the third equation,
23375.5x3  23375.4
23375.4
x3 
23375.5
 0.999995
Substituting the value of x3 in the second equation
0.001x2  8.5x3  8.501
8.501  8.5 x3
x2 
0.001
8.501  8.5  0.999995

0.001

7
Numerical Analysis MATH351/352

8.501  8.49995

0.001
0.00105

0.001
 1.05
Substituting the value of x3 and x 2 in the first equation,
20 x1  15x2  10 x3  45
45  15 x2  10 x3
x1 
20
45  15  1.05  10  0.999995

20
45  15.75  9.99995

20
29.25  9.99995

20
19.2500

20
 0.9625
Hence the solution is
 x1 
[ X ]   x2 
 x3 
 0.9625 
  1.05 
0.999995

Compare this with the exact solution of

 x1 
X    x2 
 x3 
1
 1
1

Are there any pitfalls of the Naive Gaussian elimination method?

Yes, there are two pitfalls of the Naive Gaussian elimination method.
Division by zero: It is possible for division by zero to occur during the beginning of the
n  1 steps of forward elimination.
For example

8
Gaussian Elimination MATH351/352

5x2  6 x3  11
4 x1  5x2  7 x3  16
9 x1  2 x2  3x3  15
will result in division by zero in the first step of forward elimination as the coefficient of x1
in the first equation is zero as is evident when we write the equations in matrix form.
0 5 6  x1  11
4 5 7  x   16
  2   
9 2 3  x3  15
But what about the equations below: Is division by zero a problem?
5x1  6 x2  7 x3  18
10 x1  12 x2  3x3  25
20 x1  17 x2  19 x3  56
Written in matrix form,
 5 6 7   x1  18 
10 12 3   x   25
  2   
20 17 19  x3  56
there is no issue of division by zero in the first step of forward elimination. The pivot element
is the coefficient of x1 in the first equation, 5, and that is a non-zero number. However, at the
end of the first step of forward elimination, we get the following equations in matrix form
5 6 7   x1   18 
0 0  11  x     11
  2   
0  7  9   x3   16
Now at the beginning of the 2nd step of forward elimination, the coefficient of x2 in Equation
2 would be used as the pivot element. That element is zero and hence would create the
division by zero problem.
So it is important to consider that the possibility of division by zero can occur at the
beginning of any step of forward elimination.

Round-off error: The Naive Gaussian elimination method is prone to round-off errors.
This is true when there are large numbers of equations as errors propagate. Also, if there is
subtraction of numbers from each other, it may create large errors. See the example below.

Example 3
Remember Example 2 where we used Naive Gaussian elimination to solve
20 x1  15x2  10 x3  45
 3x1  2.249 x2  7 x3  1.751
5x1  x2  3x3  9
using six significant digits with chopping in your calculations? Repeat the problem, but now
use five significant digits with chopping in your calculations.

9
Numerical Analysis MATH351/352

Solution
Writing in the matrix form
 20 15 10
 x1   45 
 3  2.249 7   x  = 1.751
 
 2  
 5 1 3 
 x3   9 
Forward Elimination of Unknowns
First step
Divide Row 1 by 20 and then multiply it by –3, that is, multiply Row 1 by  3 / 20  0.15 .
20 15 10 45 0.15 gives Row 1 as
 3  2.25 1.5  6.75
Subtract the result from Row 2
 3  2.249 7 1.751
  3  2.25  1.5  6.75
0 0.001 8.5 8.501
to get the resulting equations as
20 15 10   x1   45 
 0 0.001 8.5  x  = 8.501
   2  
 5 1 3   x3   9 
Divide Row 1 by 20 and then multiply it by 5, that is, multiply Row 1 by 5 / 20  0.25 .
20 15 10 45 0.25 gives Row 1 as
5 3.75 2.5 11.25
Subtract the result from Row 3
5 1 3 9
 5 3.75 2.5 11.25
0  2.75 0.5  2.25
to get the resulting equations as
20 15 10   x1   45 
 0 0.001 8.5  x  =  8.501 
   2  
 0  2.75 0.5  x3   2.25
Second step
Now for the second step of forward elimination, we will use Row 2 as the pivot equation and
eliminate Row 3: Column 2.
Divide Row 2 by 0.001 and then multiply it by –2.75, that is, multiply Row 2 by
 2.75 / 0.001  2750 .
0 0.001 8.5 8.501 2750 gives Row 2 as
0  2.75  23375  23377.75
Rewriting within 5 significant digits with chopping
0  2.75  23375  23377
Subtract the result from Row 3

10
Gaussian Elimination MATH351/352

0  2.75 0.5  2.25

 0  2.75  23375  23377
0 0 23375 23374
Rewriting within 6 significant digits with chopping
0 0 23375  23374
to get the resulting equations as
20 15 10   x1   45 
 0 0.001 8.5   x  =  8.501 
   2  
 0 0 23375  x3  23374
This is the end of the forward elimination steps.
Back substitution
We can now solve the above equations by back substitution. From the third equation,
23375x3  23374
23374
x3 
23375
 0.99995
Substituting the value of x3 in the second equation
0.001x2  8.5x3  8.501
8.501  8.5 x3
x2 
0.001
8.501  8.5  0.99995

0.001
8.501  8.499575

0.001
8.501  8.4995

0.001
0.0015

0.001
 1.5
Substituting the value of x3 and x 2 in the first equation,
20 x1  15x2  10 x3  45
45  15 x2  10 x3
x1 
20
45  15 1.5  10  0.99995

20

11
Numerical Analysis MATH351/352

45  22.5  9.9995

20
22.5  9.9995

20
12.5005

20
12.500

20
 0.625
Hence the solution is
 x1 
X    x2 
 x3 
 0.625 
  1.5 
0.99995
Compare this with the exact solution of
 x1  1
X    x2   1
 x3  1

12