MATH3230B Numerical Analysis

Tutorial 6

1 Recall:
1. Symmetric positive definite matrix (SPD matrix):
Some useful properties of a SPD matrix are:

(a) A SPD matrix is nonsingular.

(b) Any diagonal square submatrix of an SPD matrix is also a SPD matrix.
(c) Any eigenvalues of a SPD matrix is positive.
(d) For any rectangular matrix U , if its column vectors are linearly independent, then the matrix U T U is a
SPD matrix.

To cehck whether a symmetric matrix is positive definite or not, we have several ways:
(a) The Sylvester’s criterion states that a real-symmetric matrix A is positive definite if and only if all the
leading principal minors of A are positive.
(b) The eigenvalues of the matrix A are all positive.
(c) Use the Cholesky Factorization to check (Matlab)
2. Cholesky factorization:
Let us write
aT rT
   
α u11
A= , U=
a A11 0 U11
Then the Cholesky factorization runs as follows:

(a) α = u211 .
(b) aT = u11 rT .
(c) A11 = rrT + U11
T
U11 .
Or equivalently, we can write
√
(a) u11 = α. (Take only the positive one)
(b) rT = aT /u11 .
T
(c) U11 U11 = A11 − rrT = Â11 .
One can repeat the above procedure for the submatrix Â11 . So the Cholesky factorization proceeds in n steps.

3. LU factorization:
The Guassian elimination is basically a process of the so-called LU factorization for the matrix A. More
preciously, if a matrix A can be written into A = LU , where the matrix L is a n × n lower triangular matrix
with 1 as its diagonal entries, and the matrix U is an n × n upper triangular matrix. Then we say that A
admits a LU factorization.

1
2 Exercises:
Please do the star problem (*) in tutorial class and finish the rest after class.

1. (a) * Write down the definition of a symmetric positive definite matrix.

(b) * For any real m × n matrix M with its column vectors being linearly independent, prove that M T M is
a symmetric positive definite matrix.
(c) * Write down a criterion to determine whether a matrix A is SPD. Check whether the following matrix
is SPD by this criterion.  
8 6 3
 6 7 2 
3 2 4

(d) Suppose A is SPD, prove that A−1 is also SPD by using eigenvalues of A and A−1 .

Solution. (a) An n × n matrix A is said to be symmetric and positive definite if it satisfies

i. A is symmetric.
ii. xT Ax > 0 for all x 6= 0.
(b) Since (M T M )T = M T M , it is symmetric.
For any non-zero vector x, M x is also a non-zero vector since the column vectors of M are independent.
Therefore
xT M T M x = (M x)T (M x) > 0.
Therefore M T M is a positive definite.
(c) One of the following:
i. The Sylvester’s criterion states that a real-symmetric matrix A is positive definite if and only if all
the leading principal minors of A are positive.
ii. The eigenvalues of the matrix A are all positive.
iii. Use the Cholesky Factorization to check
Now we use (i) to check.
The first order leading principal minor is D1 = 8. The second order leading principal minor is

8 6
D2 = = 20
6 7

The third order leading principal minor is

8 6 3
D3 = 6 7 2 = 57
3 2 4

Therefore the matrix is SPD.

(d) Assume λ is an eigenvalue of A, x is the eigenvector corresponding to λ. Then we have Ax = λx.
Furthermore, we have
A−1 x = λ−1 x
Therefore if λ is an eigenvalue of A, λ−1 is an eigenvalue of A−1 . When λ > 0, we also have λ−1 > 0.
Hence A−1 is also a SPD.

2. Let A be a n × n matrix.

(a) * Write down the definition of the Cholesky factorization.

(b) Write down the general process to compute the Cholesky factorization of A.

2
(c) * Consider a SPD matrix A given by
 
2 −2 0
A =  −2 4 −2  .
0 −2 4
Compute the Cholesky factorization of this matrix A.
(d) * In the algorithm, we generate the matrix Â11 = A11 − rrT in each step. Prove that the new matrix
Â11 is also a SPD matrix.
(e) * Using the result of the Cholesky factorization to show that the inverse of a SPD matrix A is also a
SPD matrix.

Solution. (a) If A is an SPD matrix, then A can be factorized as U T U , where U is a upper triangular matrix.
If, in addition, we require the diagonal entries of U to be positive, then the factorization is unique and
is called the Cholesky factorization of A.
(b) Given a SPD matrix, we can always find its Cholesky factorization. To factorize A = U T U , we have the
following algorithm:
i. α = u211 .
ii. aT = u11 rT .
iii. A11 = rrT + U11
T
U11 .
Then we repeat the step above.
(c) Update the first row and the submatrix at the right bottom corner:
  √ √ 
2 −2 0 2 − 2 0
−2 4 −2 →  ∗ 2 −2
0 −2 4 ∗ −2 4
Update the second row and the submatrix at the right bottom corner:
√ √  √ √ 
2 − 2 0 2 −√ 2 0
√
 ∗ 2 −2 →  ∗ 2 − 2
∗ −2 4 ∗ ∗ 2
Update the third row and the submatrix at the right bottom corner:
√ √  √ √ 
2 −√ 2 0
√ 2 −√ 2 0
√
 ∗ 2 − 2 →  ∗ 2 −√ 2
∗ ∗ 2 ∗ ∗ 2
Hence, if we set √ √ 
2 −√ 2 √0
U = 0 2 −√ 2 ,
0 0 2
then we have
A = U T U.
(d) Using the same notation, we want to prove Â11 := A11 − aaT /α = U11 T
U11 is also symmetric positive
definite if A is symmetric positive definite.
To show that the matrix Â11 is indeed an SPD matrix, for ∀x 6= 0, xT ∈ Rn−1 , we construct [x1 , x]T ∈ Rn .
Then we have
[x1 , x]A[x1 , x]T = x21 α + x1 aT x + x1 xT a + xT A11 x
1
= x21 α + 2x1 (aT x) + (aT x)(aT x) + xT Â11 x
α

Now we find x1 such that x21 α + 2x1 (aT x) + α1 (aT x)(aT x) = 0. Note that the above equation is a simple
second order nonlinear equation. Also note that 4(aT x)2 − 4α · α1 (aT x)(aT x) = 0. Therefore x1 exists.
For such x1 , we have [x1 , x]A[x1 , x]T = xT Â11 x. Since x 6= 0, we have [x1 , x]T 6= 0. As A is SPD, we
have xT Â11 x 6= for all x 6= 0. Therefore Â11 is also SPD

3
 (e) We set
B = U −1 (U −1 )T .
For the result above we know that B is a SPD matrix and

AB = U T U U −1 (U −1 )T = I

BA = U −1 (U −1 )T U T U = I
So B = A−1

3. * Let A be a n × n non-singular matrix.

(a) Write down the definition of an LU factorization of A.

(b) Consider the following system of linear equation Ax = b:

 x + 2y + 3z = 15

2x + 5y + 8z = 37

3x + 4z = 10


Find a LU factorization of A.
(c) Is your result in (b) a unique LU factorization of A? If not, please give an example of another LU
factorization of A.
(d) Write down the corresponding steps of Gaussian elimination and then solve the above system.

Solution. (a) If there exist an n × n lower triangular matrix L with 1 as its diagonal entries and an n × n
upper matrix U such that
A = LU,
then we say that A admits a LU factorization.
(b) Let    
1 0 0 1 2 3
L1 =  −2 1 0  , then L1 A =  0 1 2 
−3 0 1 0 −6 −5
   
1 0 0 1 2 3
L2 =  0 1 0  , then L2 L1 A =  0 1 2 =U
0 6 1 0 0 7
Let  
1 0 0
L = (L2 L1 )−1 = 2 1 0 
3 −6 1
Then   
1 0 0 1 2 3
A = LU =  2 1 0  0 1 2 
3 −6 1 0 0 7

(c) Yes.
(d) The Gaussian elimination steps are the same as the steps that we do LU factorization in (b).
First we solve Ly = b, we have

y1 = 15
y2 = 7
y3 = 7

4
Then we solve U x = y, we have

x = 2
y = 5
z = 1