Chapter 2: SYSTEM OF LINEAR EQUATIONS

We consider the system AX = B where A : n-dimentional squared matrix; X,B: vectors.

Find X with A, B are given.

1) A is an upper-triangular matrix:
𝑎11 𝑎12 𝑎13 𝑥1 𝑥3 = 𝑏3 /𝑎33
𝐴 = [ 0 𝑎22 𝑎23 ] ⇒ 𝑋 = [𝑥2 ] 𝑎𝑛𝑑 { 𝑥2 = (𝑏2 − 𝑎23 𝑥3 )/𝑎22
0 0 𝑎33 𝑥3 𝑥1 = (𝑏1 − 𝑎12 𝑥2 − 𝑎13 𝑥3 )/𝑎11

2) A is an lower-triangular matrix:
𝑎11 0 0 𝑥1 𝑥1 = 𝑏1 /𝑎11
𝐴 = [𝑎21 𝑎22 0 ] ⇒ 𝑋 = [𝑥2 ] 𝑎𝑛𝑑 { 𝑥2 = (𝑏2 − 𝑎21 𝑥1 )/𝑎22
𝑎31 𝑎32 𝑎33 𝑥3 𝑥3 = (𝑏3 − 𝑎31 𝑥1 − 𝑎32 𝑥2 )/𝑎33

I. Matrix Factorization:
A = LU where L - lower-triangular matrix, and U - upper-triangular matrix. Then

𝐿𝑌 = 𝐵
𝐴𝑋 = 𝐵 ⟺ (𝐿𝑈)𝑋 = 𝐵 ⟺ {
𝑈𝑋 = 𝑌

1) Doolittle’s Method:

1 0 0 𝑢11 𝑢12 𝑢13

𝐴 = 𝐿𝑈 where 𝐿 = [𝑙21 1 0] and 𝑈 = [ 0 𝑢22 𝑢23 ]. We have:
𝑙31 𝑙32 1 0 0 𝑢33

𝑎21 𝑎31
𝑢11 = 𝑎11 ; 𝑢12 = 𝑎12 ; 𝑢13 = 𝑎13 ; 𝑙21 = ; 𝑙31 = ;
𝑢11 𝑢11
𝑢22 = 𝑎22 − 𝑙21 𝑢12 ; 𝑢23 = 𝑎23 − 𝑙21 𝑢13 ;
𝑎32 − 𝑙31 𝑢12
𝑙32 = ; 𝑢33 = 𝑎33 − 𝑙31 𝑢13 − 𝑙32 𝑢23
{ 𝑢22

Example 1: Solve the following system using Doolittle’s method:

𝑥1 − 2𝑥2 + 3𝑥3 = 1 1 −2 3
{−2𝑥1 + 5𝑥2 + 𝑥3 = 1 ⟹ 𝐴 = [−2 5 1]
𝑥1 + 2𝑥2 − 2𝑥3 = 1 1 2 −2
+ Factorization:
−2 1
𝑢11 = 1; 𝑢12 = −2; 𝑢13 = 3; 𝑙21 = = −2; 𝑙31 = = 1;
1 1
𝑢22 = 5 − (−2)(−2) = 1; 𝑢23 = 1 − (−2)(3) = 7;
2 − (1)(−2)
𝑙
{ 32 = = 4; 𝑢33 = −2 − (1)(3) − (4)(7) = −33
1
 1 0 0 1 −2 3
So 𝐿 = [−2 1 0] and 𝑈 = [0 1 7 ]
1 4 1 0 0 −33
+ Solving:
1 0 0 1 1
𝐿𝑌 = 𝐵 ⟺ [−2 1 0] 𝑌 = [1] ⟹ 𝑌 = [ 3 ]
1 4 1 1 −12
1 −2 3 1 9/11
𝑈𝑋 = 𝑌 ⟺ [0 1 7 ] 𝑋 = [ 3 ] ⟹ 𝑋 = [5/11]
0 0 −33 −12 4/11

3.5 3.8 4.2

Example 2: Given 𝐴 = [2.1 2.9 3.3]. Using Doolittle’s method find l32 and u33?
1.7 3.6 5.6

SOLUTION.
u11 = 3.5; u12 = 3.8; u13 = 4.2; l21 = 2.1/3.5 = 0.6; l31 = 1.7/3.5 = 17/35;
u22 = 2.9 – (0.6)(3.8) = 0.62; u23 = 3.3 – (0.6)(4.2) = 0.78;
l32 = (3.6 – (17/35)(3.8))/0.62 = 614/217 ~ 2.8295
u33 = 5.6 – (17/35)(4.2) – (614/217)(0.78) = 1468/1085 ~ 1.3530

2.7 1.6 3.8

Example 3: Given 𝐴 = [3.3 5.1 4.9]. Using Doolittle’s method find l32 and u33?
2.4 2.8 4.5
SOLUTION.
u11 = 2.7; u12 = 1.6; u13 = 3.8; l21 = 3.3/2.7 = 11/9; l31 = 2.4/2.7 = 8/9;
u22 = 5.1 – (11/9)(1.6) = 283/90; u23 = 4.9 – (11/9)(3.8) = 23/90;
l32 = (2.8 – (8/9)(1.6))/(283/90) = 124/283 ~ 0.4382
u33 = 4.5 – (8/9)(3.8) – (124/283)(23/90) = 2859/2830 ~ 1.0102

Example 4: Solve the following system using Doolittle’s method:

2𝑥1 + 𝑥2 − 2𝑥3 = 4 2 1 −2 4
{𝑥1 + 𝑥2 + 3𝑥3 = 5 ⟹ 𝐴 = [1 1 3 ] 𝐵 = [5]
3𝑥1 − 𝑥2 + 4𝑥3 = 6 3 −1 4 6
SOLUTION.
1 0 0 2 1 −2 16/9
𝐿 = [1/2 1 0] and 𝑈 = [0 1/2 4 ] => 𝑋 = [14/9]
3/2 −5 1 0 0 27 5/9
 2) Crout’s Method:

𝑙11 0 0 1 𝑢12 𝑢13

𝐴 = 𝐿𝑈 where 𝐿 = [𝑙21 𝑙22 0 ] and 𝑈 = [0 1 𝑢23 ]. We have:
𝑙31 𝑙32 𝑙33 0 0 1
𝑎12 𝑎13
𝑙11 = 𝑎11 ; 𝑙21 = 𝑎21 ; 𝑙31 = 𝑎31 ; 𝑢12 = ; 𝑢13 = ;
𝑙11 𝑙11
𝑙22 = 𝑎22 − 𝑙21 𝑢12 ; 𝑙32= 𝑎32 − 𝑙31 𝑢12 ;
𝑎23 − 𝑙21 𝑢13
𝑢23 = ; 𝑙33 = 𝑎33 − 𝑙31 𝑢13 − 𝑙32 𝑢23
{ 𝑙22

Example 1: Solve the following system using Crout’s method:

𝑥1 − 2𝑥2 + 3𝑥3 = 1 1 −2 3
{−2𝑥1 + 5𝑥2 + 𝑥3 = 1 ⟹ 𝐴 = [−2 5 1]
𝑥1 + 2𝑥2 − 2𝑥3 = 1 1 2 −2
Solution: + Factorization:
−2 3
𝑙11 = 1; 𝑙21 = −2; 𝑙31 = 1; 𝑢12 = = −2; 𝑢13 = = 3 ;
1 1
𝑙22 = 5 − (−2)(−2) = 1; 𝑙32 = 2 − (1)(−2) = 4;
1 − (−2)(3)
{ 𝑢 23 = = 7; 𝑙33 = −2 − (1)(3) − (4)(7) = −33
1
1 0 0 1 −2 3
So 𝐿 = [−2 1 0 ] and 𝑈 = [0 1 7]
1 4 −33 0 0 1
+ Solving:
1 0 0 1 1
𝐿𝑌 = 𝐵 ⟺ [−2 1 0 ] 𝑌 = [1] ⟹ 𝑌 = [ 3 ]
1 4 −33 1 4/11
1 −2 3 1 9/11
𝑈𝑋 = 𝑌 ⟺ [0 1 7] 𝑋 = [ 3 ] ⟹ 𝑋 = [5/11]
0 0 1 4/11 4/11

3) Cholesky’s Method:
a. A matrix A is symmetric if AT = A (aij = aji for all i, j)
b. A matrix A is positive-definite if for all vector 𝑋 ≠ 0 then XTAX > 0.

Theorem: The matrix A is positive-definite if and only if ∆𝑘 > 0, ∀𝑘 = 1,2, … , 𝑛, where

𝑎11 … 𝑎1𝑘
∆𝑘 = | … … … |: main subdeterminant of order k.
𝑎𝑘1 … 𝑎𝑘𝑘
 1 −1 1
Example: Consider the following matrix 𝐴 = [−1 2 0]. We have: A is symmetric, and
1 0 5
1 −1 1
1 −1
∆1 = 1 > 0, ∆2 = | | = 1 > 0, ∆3 = |−1 2 0| = 10 + 0 − 2 − 0 − 5 = 3 > 0
−1 2
1 0 5
So A is positive-definite

c. Theorem: (Cholesky) The matrix A is symmetric and positive-definite when and

only when there exists a nonsingular, lower-triangular matrix C such that A = C.CT
Notations:
𝐶𝑌 = 𝐵
 𝐴𝑋 = 𝐵 ⟺ (𝐶𝐶 𝑇 )𝑋 = 𝐵 ⟺ { 𝑇
𝐶 𝑋=𝑌
𝑎 𝑎
𝑐11 = √𝑎11 ; 𝑐21 = 21 ; 𝑐31 = 31
𝑐11 0 0 𝑐11 𝑐11
𝑎 −𝑐 𝑐
 𝐶 = [𝑐21 𝑐22 0 ] ⟹ 𝑐22 = √𝑎22 − 𝑐21 2
; 𝑐32 = 32 31 21
𝑐22
𝑐31 𝑐32 𝑐33
2 2
{ 𝑐33 = √𝑎33 − 𝑐31 − 𝑐32
Example 1: Solve the following system using Crout’s method:
𝑥1 − 2𝑥2 + 𝑥3 = 1 1 −2 1
{−2𝑥1 + 5𝑥2 + 𝑥3 = 1 ⟹ 𝐴 = [−2 5 1]
𝑥1 + 𝑥2 + 12𝑥3 = 1 1 1 12

Solution: A is symmetric, and

1 −2 1
1 −2
∆1 = 1 > 0, ∆2 = | | = 1 > 0, ∆3 = |−2 5 1 | = 60 − 2 − 2 − 5 − 1 − 48 = 2
−2 5
1 1 12
So A is positive-definite.
−2 1
𝑐11 = √1 = 1; 𝑐21 = = −2; 𝑐31 = = 1
1 1 1 0 0
1 − (1)(−2) ⟹ 𝐶 = [−2 1 0 ]
𝑐22 = √5 − (−2)2 = 1; 𝑐32 = =3
1 1 3 √2
{ 𝑐33 = √12 − (1)2 − (3)2 = √2
1 0 0 1 1
𝐶𝑌 = 𝐵 ⟺ [−2 1 0 ] 𝑌 = [1] ⟹ 𝑌 = [ 3 ]
1 3 √2 1 −9/√2
77
1 2
1 −2 1 38.5
𝑇 3 33
𝐶 𝑋=𝑌⟺[ 0 1 3 ]𝑋 = [ 9 ] ⟹ 𝑋 = = [ 16.5 ]
0 0 √2 − 2 −4.5
√2 9
[− 2 ]
II. Iterative Methods: