Solution of Systems of Linear Equations

A system of n equations is represented generally as

a 11 x 1 +a 12 x 2+ …..+ a1 n 1 x n=b 1
a 21 x 1+ a22 x 2 +… ..+a2 n x n=b2

……………………………………..
a n 1 x 1+ an 2 x 2 +… ..+a nn x n=b n

In matrix notation ,(1) can be expressed Ax=b

Where A is a n × n matrix, b is an n vector, and x is a vector of n unknown.The
technique and methods for solving systems of linear algebraic equations belongs to
two fundamentally approaches:
1 Elimination approach or direct method
2 Iterative approach
1. Elimination approach or direct method:
(i) Basic Gauss elimination method
(ii) Gauss elimination with pivoting
(iii) LU decomposition methods
(iv) Matrix inverse method
2. Iterative approach:
(i) Jacobi iteration method
(ii) Gauss-Seidel iteration method
(iii) Successive over relaxation method
Gauss Elimination Method: This is the elementary elimination method and it
reduces the systems to an equivalent upper-triangular system, which can be solved
by back substitution.
Let the system of n linear equations in n unknowns be given by
a 11 x 1 +a 12 x 2+ …..+ a1 n 1 x n=b 1
a 21 x 1+ a22 x 2 +… ..+a2 n x n=b2 (2)
……………………………………..
a n 1 x 1+ an 2 x 2 +… ..+a nn x n=b n
There are two steps in the solution of the system given in equation (2), viz., the
elimination of unknowns and back substitution.
Step1:The unknowns are eliminated to obtain an upper-triangular system.
To eliminate x 1 from the second equation, we multiply the first equation by
(−a21 /a11 ) and obtain
a 21 a21 a21
−a 21 x 1−a12 x 2 +…−a1 n x n=−b1
a11 a11 a11

Adding the above equation to the second equation of Eq. (2), we obtain
¿
Which can be written as
á 22 x 2+ á23 x 3 +…+ á2 n x n= b́2,

a21
wherea´22=(a 22−a12 a ). Similarly we can multiply the first equation by (−a31 /a11 ) and
11

add it to the third equation of the system (2). This eliminates the unknown x 1 from
the third equation of Eq.(2) and we obtain
á 32 x 2+ á33 x 3 +…+ á3 n x n= b́3

We can eliminate x 1from the remaining equations and after eliminating x 1 from the
last equation of Eq.(2), we obtain the system
a 11 x 1 +a 12 x 2+ a13 x 3 +…+ a1 n 1 x n=b 1
a´22 x 2+ á23 x 3 +…+ á2 n x n= b́2

a´32 x 2+ á33 x 3 +…+ á3 n x n= b́3 (3)

……………………………………..
a´n 2 x 2+ a´n 3 x3 + …+ a´nn x n=b́n

Now, to eliminate x 2 from the third equation of (3), we multiply the second
equation by −a´32/ a´22 and add it to the third equation. Repeating this process with the
remaining equations, we obtain the system

a 11 x 1 +a 12 x 2+ a13 x 3 +…+ a1 n 1 x n=b 1

 a´22 x 2+ á23 x 3 +…+ á2 n x n= b́2

a¨33 x 3+ …+ a¨3 n x n=b¨3 (4)

……………………………………..
a¨n 3 x 3 +…+ a¨nn x n=b¨n

It is easily seen that this procedure can be continued to eliminate x 3 from the fourth
equation onward, x 4 from the fifth equation onwards etc., till we finally obtain the
upper-triangular form:
a 11 x 1 +a 12 x 2+ a13 x 3 +…+ a1 n 1 x n=b 1
a´22 x 2+ á23 x 3 +…+ á2 n x n= b́2

a¨33 x 3+ …+ a¨3 n x n=b¨3 (5)

……………………………………..
(n−1) (n−1)
a nn xn =bn

Where a(n−1)
nn indicates that the element a nn has changed (n-1) times.
Step2:we now have to obtain the required solution from the system (5) . From the
last equation of this system , we obtain
(n−1)
bn
x n= (6)
a(n−1)
n

We have therefore first computed x n an then x n−1 , x n−2 , … … . , x 2 , x1.

Example1: Use Gauss elimination to solve the system
2 x+ y+ z =10
3 x+ 2 y +3 z=18 (7)
x +4 y+ 9 z =16
Solution: We first eliminate x from the second and third equations. For this we
multiply the first equation by(-3/2) and ad to the second to get
y +3 z=6
Similarly, we multiply the first equation by (-1/2) and ad to the third to get
7 y +17 z=22
We have the following derived system
2 x+ y+ z =10
y +3 z=6 (8)
 7 y +17 z=22

Next we have to eliminated y from second and third equation of the systems(8).
For this multiplying second by (-7) and add to third, this gives
−4 z=−20

The system now has been reduced to an upper triangular form

2 x+ y+ z =10
y +3 z=6 (9)
−4 z=−20

By back substituting we get, z=5,

Then y +3 ×5=6 or y=−9

Finally, 2 x−9+5=10 or x=7

Hence the solution ( x , y , z )=( 7 ,−9 , 5).

Gauss Elimination Method with Pivoting :

Necessity for Pivoting:
We now come to important case of the pivot being zero or very close to zero. If the
pivot is zero , the entire process fails and if it is close to zero, round-off error may
occur. These problems can be avoided by adopting a procedure called pivoting. If
a 11 is either zero or very small compared to the other coefficients of the equation,

then we find the largest available coefficient in the columns below the pivot
equation and then interchanging the two rows. In this way, we obtain a new pivot
equation with a nonzero pivot. Such a process is called partial pivoting, since in
this case we search only the column below for the largest below.
If on the other hand, we search both column and rows for the largest element, the
procedure is called complete pivoting. It is obvious that that complete pivoting
involves more complexity in computations since interchanging of columns means
change of order of unknowns which invariably requires more programming effort.
In comparison, partial pivoting i.e., row interchanges, is easily adopted in
programming. Due to this reason, complete pivoting is rarely used.
Example2: Solve the system using partial pivoting
2 x+ y+ z =10
3 x+ 2 y +3 z=18

x +4 y+ 9 z =16
Solution:
3 x+ 2 y +3 z=18

2 x+ y+ z =10
x +4 y+ 9 z =16 (1)
We first eliminate x from the second and third equations. For this we multiply the
first equation by(-2/3) and ad to the second to get
−1
y−z=−2
3
Similarly, we multiply the first equation by (-1/3) and ad to the third to get
10
y +8 z=10
3
We have the following derived system
3 x+ 2 y +3 z=18
1
y−z =−2
3
10
y +8 z=10 (2)
3
The system written as
3 x+ 2 y +3 z=18
10
y +8 z=10
3
1
y−z =−2 (3)
3
Next we have to eliminate y from second and third equation of the systems (3). For
this multiplying second by (-1/10) and add to third, this gives
−18/10 z=−3

The system now has been reduced to an upper triangular form

 3 x+ 2 y +3 z=18
10
y +8 z=10
3
−18/10 z=−3 (4)
By back substituting we get, z=5 /3 ,
10 5
Then y +8 × =10 or y=−1
3 3
5
Finally, 2 x+ 2(−1)+3 ( )=18 or x=5
3
Hence the solution ( x , y , z )=( 5 ,−1 ,5 /3).

Gauss –Jordan Method:

Example 3:
2 x+ 4 y−6 z=−8
x +3 y+ z=10

2 x−4 y−2 z=−12 (1)

Solution: Step 1: Normalize the first equation by dividing it by 2(pivot element)
x +2 y−3 z=−4

x +3 y+ z=10

2 x−4 y−2 z=−12 (2)

Step 2: Eliminate x from the second equation, subtracting 1 time the first equation
from it. Similarly, eliminate x from the third equation by subtracting 2 times the
first equation from it. The result is:
−¿ x +2 y−3 z=−4

0+ y+ 4 z=14

0−8 y +4 z =−4 (3)

Step 3: Normalize the second equation. ( Note that it is already in normalized
form)
Step 4: Following similar approach, Eliminate y from the first and third equations.
This gives
 x +0−11 z =−32
0+ y+ 4 z=14

0+ 0+36 z=108 (4)

Step 5: Normalize third equation.
x +0−11 z =−32
0+ y+ 4 z=14

0+ 0+ z=3 (5)
Step 6: Eliminate z from the first and second equations. We get

x +0+0=1
0+ y+ 0=2

0+ 0+ z=3 (6)
Hence the solution ( x , y , z )=( 1, 2 , 3)

LU Decomposition Method:
Consider a system of equation
a 11 x 1 +a 12 x 2+ a13 x 3=b 1
a 21 x 1+ a22 x 2 +a23 x3 =b2 (1)
a 31 x 1+ a32 x 2+ a33 x 3=b3

This can be written in the form

AX=B (2)
Let A=LU

( ) ( ) ( )
a11 a12 a13 1 0 0 u 11 u12 u 13
Where, A= a21 a22 a23 , L= l 21 1 0 and U = 0 u22 u 23
a31 a32 a33 l 31 l32 1 0 0 u 33

Then (2) becomes

LUX =B (3)
Setting UX =Y , the equation (3) becomes
LY =B (4)
The eq. (4) is equivalent to the system
 y 1=b 1
l 21 y 1 + y 2=b 2 (5)
l 31 y 1 +l 32 y 2 + y 3=b 3

By the forward substituting we get the values of y 1 , y 2 , y 3 .

When we know the Y, the system U X =Y gives:
u11 x 1+ u12 x 2+ u13 x 3= y 1

u22 x 2 +u23 x3 = y 2 (6)

u33 x 3 = y 3

By the backward substituting we get the values of x 1 , x , x 3 .

From the relation A=LU , we get

( )( )( )
a11 a12 a13 1 0 0 u11 u12 u13
a21 a22 a23 = l 21 1 0 0 u22 u23
a31 a32 a33 l 31 l 32 1 0 0 u33

( )( )
a11 a12 a13 u 11 u12 u13
a21 a22 a23 = l 21 u 11 l 21 u12 +u22 l 21 u 13+u23
a31 a32 a33 l 31 u 11 l 31 u12 +l 32 u22 l 31 u 13+l 32 u 23

We have, u11 =a11 u12=a 12 , u13=a13

l 21 u11 =a 21, l 21 u12 +u22=a 22 l 21 u13 +u23=a23

l 31 u11 =a31 , l 31 u12 +l 32 u22=a 32 l 31 u13 +l 32 u23=a33

From the above equation, we obtain

a21 a31
l 21= , l 31= ,
a11 a11
a21 a21
u22=a22− a , u23=a23− a ,
a11 12 a11 13
a32−a31 ×a 12/a 11
l 32=
u12

from which u33 can be computed.

Example1:
3 x 1+2 x 2 + x3 =10
2 x1 +3 x 2 +2 x 3=14 (1)
x 1+ 2 x 2 +3 x3 =14
Solution: Let AX=B, and A=LU , then LUX =B

( )( ) ( )
a11 a12 a13 3 2 1 1 0 0
where A= a21 a22 a23 = 2 3 2 , L= l 21 1 0
a31 a32 a33 1 2 3 l 31 l 32 1

( )
u 11 u12 u 13
and U = 0 u22 u 23 .
0 0 u 33

We have, l 11=1 and

u11 =a11 =3 u12=a 12=2 , u13=a13=1

a21 2
l 21= = , l 22=1
a11 3
2 5 2 4
u22=a22−l 21 a12=3− ×2= , u23=a23−l 21 a 13=2− ×1=
3 3 2 3

a31 1 a32−l u 2−1/3 ×2 4

l 31= = , l 32 = =31 12
= , l 33=1
a11 3 u22 5/3 5
1 4 4 24
u33=a33−l 31 u13−l 32 u 23=1− ×1− × =
3 5 3 15

( ) ( )
1 0 0 3 2 1
Thus we have, L= 2/3 1 0 and U = 0 5/3 4 /3
1/3 4 /5 1 0 0 24 /15

Solving LY =B by forward substituting, we get

y 1=b 1=10
2 22
y 2=b 2−l 21 y 1=14= × 10=
3 3
1 4 22 72
y 3=b3−l 31 y 1−l 32 y 2=14− ×10− × =
3 5 3 15
Solving UX =Y by backward substituting, we get
y 3 72 /15
x 3= = =3
u33 24 /15
y 2−u23 x 3 22/3−(4 /3)× 3
x 2= = =2
u 22 5/3
 y 1−u12 x 2−u13 x3 10−2× 2−1 ×3
x 1= = =1
u11 3
∴(x 1 , y1 , z 1)=(1 ,2 , 3)

Matrix Inversion Method:

Consider the system of equations AX=B. (1)
Let A be a non-singular matrix i.e., det A ≠ 0. Then A−1 exists. Multiplying both side
of (1) by A−1, we get
AX= A B , (2)
−1 −1
A
Since A−1 A=I , the identity matrix, equation (2) becomes
X =A B .
−1

Computing Matrix Inverse:

Gauss-Jordan Method:
1. Augment the coefficient matrix A with an identity matrix as shown below:

[ ]
a11 a12 a13 ⋮ 1 0 0
a21 a22 a23 ⋮ 0 1 0
a31 a32 a33 ⋮ 0 0 1

2. Applying the Gauss-Jordan method to the augmented matrix to reduce A to an

identity matrix. The result will be shown below:

[ ]
1 0 0 ⋮ á11 á12 á13
0 1 0 ⋮ á21 á22 á23
0 0 1 ⋮ á31 á32 á33

The right-hand side of the augmented matrix is the inverse of A . Now we can
obtain the solution as follow:
x 1=á 11 × b1 + á 12 ×b 2+ á13 × b3
x 2=á 21 ×b 1+ á22 × b2 + á23 ×b 3
x 3=á 31 ×b 1+ á32 × b2 + á33 ×b 3