SYSTEMS OF LINEA R EQUATIONS MATH 19 – NUM ERICA L SOLUTIONS TO CE PROBLEM S

III. SYSTEMS OF LINEAR EQUATIONS

Subtopics:

1.1 Methods of Solution

1.2 Matrix Inversion Method
1.3 Gauss Elimination Method
1.4 Gauss-Jordan Method
1.5 LU Decomposition Methods
1.6 Jacobi’s Iteration Method
1.7 Gauss-Seidel Iteration Method

Notations of Linear Equations

Matrix Notations

or simply Ax = b

Augmented Matrix

Obtained by adjoining the constant vector b to the coefficient matrix A

Uniqueness of Solution

A system of n linear equations in n unknowns has a unique solution, provided that:

 determinant of the coefficient matrix is nonsingular, i.e., if |A| ≠ 0

 rows and columns of a nonsingular matrix are linearly independent
- no row (or column) is a linear combination of other rows (or columns)
3.1 METHODS OF SOLUTIONS

Compiled by: Redin Jhon T. Carriedo, ECE Page 1 of 17

SYSTEMS OF LINEA R EQUATIONS MATH 19 – NUM ERICA L SOLUTIONS TO CE PROBLEM S

A. Direct Methods

 Matrix Inverse Method

 Gauss Elimination Method
 Gauss-Jordan Method
 LU Decomposition Methods

Definitions:

 transforms the original equation into equivalent equations that can be solved more easily
 transformation is carried out by applying certain operations
Advantages and Drawbacks

does not contain any truncation errors

round-off errors is introduced due to floating-point operations

B. Indirect or Iterative Methods

 Jacobi’s Iteration Method

 Gauss-Seidel Iteration Method

Definitions:

 start with a guess of the solution x

 then repeatedly refine the solution until a certain convergence criterion is reached
 less efficient than direct methods due to the large number of operations or iterations
required

Advantages and Drawbacks

more useful to solve a set of ill-conditioned equations

round off errors (or even arithmetic mistakes) in one iteration cycle are corrected in
subsequent cycles
contains truncation error
does not always converge to the solution
- the initial guess affects only the number of iterations that are required for
convergence

3.2 MATRIX INVERSION METHOD

Inverse of a matrix is obtained by dividing its adjoint matrix by its determinant |A|

𝑎𝑑𝑗 𝑨
𝐴−1 =
det 𝑨

Requirements for obtaining a unique inverse of a matrix:

1. The matrix is a square matrix.
2. The determinant of the matrix is not zero (the matrix is non-singular)
if |A| = 0, then the elements of A-1 approach infinity
the inverse of a matrix is also defined by the relationship
A-1 .A = I

Compiled by: Redin Jhon T. Carriedo, ECE Page 2 of 17

SYSTEMS OF LINEA R EQUATIONS MATH 19 – NUM ERICA L SOLUTIONS TO CE PROBLEM S

Solving Linear Equations

Consider a set of three simultaneous linear algebraic equations:

a11 x 1 + a12 x 2 + a13 x 3 = b1

a21 x 1 + a22 x 2 + a23 x 3 = b2

a31 x 1 + a32 x 2 + a33 x 3 = b3

can be expressed in the matrix form:
Ax = b
we obtain the solution of x as:

x = A-1b
Example:
Solve the following simultaneous linear equations:

2x + 3y – z = - 10
-x + 4y + 2z = - 4
2x – 2y + 5z = 35

Solution:

Compiled by: Redin Jhon T. Carriedo, ECE Page 3 of 17

SYSTEMS OF LINEA R EQUATIONS MATH 19 – NUM ERICA L SOLUTIONS TO CE PROBLEM S

Compiled by: Redin Jhon T. Carriedo, ECE Page 4 of 17

SYSTEMS OF LINEA R EQUATIONS MATH 19 – NUM ERICA L SOLUTIONS TO CE PROBLEM S

3.3 GAUSS ELIMINATION METHOD

Definitions:

a popular technique for solving simultaneous algebraic equations

𝑨𝒙 = 𝒃

reduces the coefficient matrix into an upper triangular matrix

𝑼𝒙 = 𝒄

consists of two parts:

- elimination phase
- solution phase

Initial Form: 𝑨𝒙 = 𝒃

Final Form: 𝑼𝒙 = 𝒄

Gauss Elimination Operations:

1. Multiplication of one equation by a non-zero constant.
2. Addition of a multiple of one equation to another equation.
3. Interchange of two equations.
- Ax = b and Ux = c are equivalent if the sequence of operations produce the new
system Ux = c
- A is invertible if U is invertible

Gauss Elimination Process:

1. Eliminate x1 from the second and third equations
assuming a11 ≠ 0.
2. Eliminate x2 from the third row assuming a' 22 ≠ 0.
3. Apply back substitution:
x3 from a''33 x3 = b''3
x2 from a'22 x2 + a'23 x3 = b'2
x1 from a11 x1 + a12 x2 + a13 x3 = b1

Pivoting:
Gauss elimination method fails if any one of the pivots becomes zero.

What if pivot is zero?

Solution: interchange the equation with its lower equations such that the pivots are not zero

3.4 GAUSS-JORDAN METHOD

Ax = b is reduced to a diagonal set Ix = b’
where:
I = a unit matrix or identity matrix
Ix = b’ equivalent to x = b’ where b’ solution vector
implements the same series of row operations as implemented by Gauss except that it
applies below as well as above the main diagonal
- all off-diagonal elements are reduced to zero
- all main diagonal elements become 1
Gauss-Jordan Process:
Compiled by: Redin Jhon T. Carriedo, ECE Page 5 of 17
SYSTEMS OF LINEA R EQUATIONS MATH 19 – NUM ERICA L SOLUTIONS TO CE PROBLEM S

1. Determine if pivot is non-zero. If it is zero, swap row to succeeding rows with non-zero element.
2. Divide the pivot by itself to make the pivot equal to 1.
3. Eliminate all other elements on that column where pivot is located.
4. Go to the next row, and repeat all steps until reaching the last row.

Example:

Solve the following systems using Gaussian elimination and Gauss-Jordan process.
2x + y – 3z = 11
4x – 2y + 3z = 8
-2x + 2y – z = -6
Solution:
GAUSSIAN ELIMINATION

Compiled by: Redin Jhon T. Carriedo, ECE Page 6 of 17

SYSTEMS OF LINEA R EQUATIONS MATH 19 – NUM ERICA L SOLUTIONS TO CE PROBLEM S

GAUSS-JORDAN

Compiled by: Redin Jhon T. Carriedo, ECE Page 7 of 17

SYSTEMS OF LINEA R EQUATIONS MATH 19 – NUM ERICA L SOLUTIONS TO CE PROBLEM S

3.6 LU DECOMPOSITION METHODS

Expressing the matrix as the multiplication of a lower triangular matrix L and an upper triangular
matrix U
A = LU

- Doolittle’s Method
- Crout’s Method

LU Decomposition
aka LU Factorization
process of computing L and U for a given A
expressed as a product of a lower triangular matrix L and an upper triangular matrix U

Constraints

LU decomposition is not unique unless certain constraints are placed on L or U

3.6.1 DOOLITTLE’S DECOMPOSITION METHOD

transforms Ax = b to LUx = b

Compiled by: Redin Jhon T. Carriedo, ECE Page 8 of 17

SYSTEMS OF LINEA R EQUATIONS MATH 19 – NUM ERICA L SOLUTIONS TO CE PROBLEM S

Example:
Use Doolittle’s decomposition method to solve the equations Ax = b, where

Solution:

Decomposition Phase:

3.6.2 CROUT’S DECOMPOSITION METHOD

Example:
Solve the following set of equations by Crout’s method:
2𝑥 + 𝑦 + 4𝑧 = 12
8x – 3y + 2z = 20
4x + 11y – z = 33
Solution:

Compiled by: Redin Jhon T. Carriedo, ECE Page 9 of 17

SYSTEMS OF LINEA R EQUATIONS MATH 19 – NUM ERICA L SOLUTIONS TO CE PROBLEM S

Problem:

Solve the following set of equations by using the Crout’s Method:

2x 1 + x 2 + x 3 = 7
x 1 + 2x 2 + x 3 = 8
x 1 + x 2 + 2x 3 = 9

3.7 JACOBI’S ITERATION METHOD

Consider the equation:

3x + 1 = 0

which can be cast into an iterative scheme as:

𝑥+1
2𝑥 = −𝑥 − 1 𝑜𝑟 𝑥= −
2

which can be expressed as:

1 1
𝑥 𝑘+1 = − 𝑥𝑘 −
2 2
Compiled by: Redin Jhon T. Carriedo, ECE Page 10 of 17
SYSTEMS OF LINEA R EQUATIONS MATH 19 – NUM ERICA L SOLUTIONS TO CE PROBLEM S

Iterations:

𝑥𝑘 1
𝑥 𝑘+1 = − −
2 2

𝑥0 1
𝑥1 = − −
2 2

where 𝑥 0 is the initial guess

𝑥1 1
𝑥2 = − −
2 2
𝑥2 1
𝑥3 = − −
2 2

Will it converge?

Another iterative scheme:

𝑥 = −2𝑥 − 1
𝑥 𝑘+1 = −2𝑥 𝑘 − 1

Will this converge?

Jacobi’s Iteration Method

aka the method of simultaneous displacements
applicable to predominantly diagonal systems

- Consider the system of linear equations:

where 𝑎11 , 𝑎22 , and 𝑎33 are the largest coefficients

- Unknowns are solved using the equations:

Approximations and Iterations:

Let the initial approximations be 𝑥 10 , 𝑥 20 , and 𝑥 30 respectively, it is a general practice to

assume 𝑥 10 = 𝑥 20 = 𝑥 30 = 0

Compiled by: Redin Jhon T. Carriedo, ECE Page 11 of 17

SYSTEMS OF LINEA R EQUATIONS MATH 19 – NUM ERICA L SOLUTIONS TO CE PROBLEM S

Iteration process is continued until the values of 𝑥 1 , 𝑥 2 and 𝑥 3 are found to a pre-assigned
degree of accuracy

Example:

Solve the following equations by Jacobi’s method.

15x + 3y – 2z = 85
2x + 10y + z = 51
x – 2y + 8z = 5

Solution:

Compiled by: Redin Jhon T. Carriedo, ECE Page 12 of 17

SYSTEMS OF LINEA R EQUATIONS MATH 19 – NUM ERICA L SOLUTIONS TO CE PROBLEM S

Problem 1:

Use the Jacobi iterative scheme to obtain the solutions of the system of equations correct to three
decimal places.

x + 2y + z = 0
3x + y – z = 0
x – y + 4z = 3

Problem 2:

Use Jacobi iterative scheme to obtain the solution of the system of equations correct to two decimal
places.

3.8 GAUSS-SEIDEL’S ITERATION METHOD

also called the method of successive approximations

applicable to predominantly diagonal systems
absolute value of the diagonal element in each case is larger than or equal to the sum of the
absolute values of the other elements in that row

Gauss-Seidel vs. Jacobi

Each iteration of Jacobi method updates the whole set of N variables at a time
Compiled by: Redin Jhon T. Carriedo, ECE Page 13 of 17
SYSTEMS OF LINEA R EQUATIONS MATH 19 – NUM ERICA L SOLUTIONS TO CE PROBLEM S

Gauss-Seidel can speed up the convergence by using all the most recent values of variables
for updating each variable even in the same iteration

Gauss-Seidel Generalization Formula

Gauss-Seidel Iterations

Compiled by: Redin Jhon T. Carriedo, ECE Page 14 of 17

SYSTEMS OF LINEA R EQUATIONS MATH 19 – NUM ERICA L SOLUTIONS TO CE PROBLEM S

Example:

Solve the following equations by Gauss-Seidel method.

4x + y – z = 4
x – 8y + 3z = -4
2x + y + 9z = 12

Solutions:

Problem:

Solve the following equations by the Gauss-Seidel method.

1) 4x – y + z = 12
-x + 4y – 2z = -1
x – 2y + 4z = 5

2) 2x – y + 3z = 4
x + 9y – 2z = -8
4x – 8y + 11z = 15

Exercise no. 3.1

Solve the following set of simultaneous linear equations by the matrix- inverse method.

1. 10x + 3y + 10z = 5
8x – 2y + 9z = 2
8x + y – 10z =35
2. x + 3y = 5
4x – y = 12

3. x – y + 3z = 5
4x + 2y – z = 0
x + 3y + z = 5

4. w + x = 7
2w + 3x – y = 9
4x + 2y + 3z = 10
2y – 4z = 12
Compiled by: Redin Jhon T. Carriedo, ECE Page 15 of 17
SYSTEMS OF LINEA R EQUATIONS MATH 19 – NUM ERICA L SOLUTIONS TO CE PROBLEM S

Exercise no. 3.2

1. Solve by Gauss Elimination

1.1 2x + 4y – 6z = -4
x + 5y + 3z = 10
x + 3y + 2z = 5

1.2 6x + 3y + 6z = 30
2x + 3y + 3z = 17
x + 2y + 2z = 11

2. Use the method of Gaussian elimination to solve the following system of linear
equations:
𝑥1 + 𝑥2 + 𝑥3 − 𝑥4 = 2
4𝑥 1 + 4𝑥 2 + 𝑥 3 + 𝑥 4 = 11
𝑥 1 − 𝑥 2 − 𝑥 3 + 2𝑥 4 = 0
2𝑥 1 + 𝑥 2 + 2𝑥 3 − 2𝑥 4 = 2

3. Solve the following equations by Gauss-Jordan Method.

2x – y + 4z = 13
x + 3y + 2z = 17
x + 2y +3z = 16

4. Solve the following system of linear equations using the Gauss-Jordan method.
x – 2y = -4
5y + z = -9
4x – 3z = -10

5. Solve the following system of linear equations using the Gauss-Jordan method.
2x1 + x2 – 3x3 = 11
4x1 – 2x2 + 3x3 = 8
-2x1 + 2x2 – x3 = -6

6. Solve the following system of equations:

2x + 6y + z = 7
x + 2y – z = -1
5x + 7y – 4z = 9

using:

(a) Gaussian elimination and

(b) Gauss-Jordan elimination

Compiled by: Redin Jhon T. Carriedo, ECE Page 16 of 17

SYSTEMS OF LINEA R EQUATIONS MATH 19 – NUM ERICA L SOLUTIONS TO CE PROBLEM S

Exercise no. 3.3

1. Solve Ax = B with Doolittle’s decomposition and compute |A|, where

2. Solve using appropriate LU method.

2.1 2x – y = 3
-x + 2y – z = -3
-y + z = 2

2.2 x + y + z = 7
3x + 3y + 4z = 23
2x + y + z = 10

Exercise no. 3.4

Solve using Jacobi’s and Gauss-Seidel Iteration methods.

1. 4x – 3y + 5z = 34
2x – y – z = 6
x + y + 4z = 15

2. 2x – y + 5z = 15
2x + y + z = 7
x + 3y + z = 10

Compiled by: Redin Jhon T. Carriedo, ECE Page 17 of 17