Numerical Analysis

10th ed

R L Burden, J D Faires, and A M Burden

Beamer Presentation Slides

Prepared by
Dr. Annette M. Burden
Youngstown State University

September 6, 2015
 Chapter 6.1: Direct Methods For Solving Linear
Systems 1

Operations
E1 : a11 x1 + a12 x2 + · · · + a1n xn = b1 ,
E2 : a21 x1 + a22 x2 + · · · + a2n xn = b2 ,
..
.
En : an1 x1 + an2 x2 + · · · + ann xn = bn .
is a linear system with given constants aij , for each i, j = 1, 2, . . . , n, and bi ,
for each i = 1, 2, . . . , n, and we need to determine the unknowns x1 , . . . , xn .

1. Equation Ei can be multiplied by any nonzero constant with the

resulting equation used in place of Ei . This operation is denoted
( Ei ) ! (Ei ).
2. Equation Ej can be multiplied by any constant and added to equation
Ei with the resulting equation used in place of Ei . This operation is
denoted (Ei + Ej ) ! (Ei ).
3. Equations Ei and Ej can be transposed in order. This operation is
denoted (Ei ) $ (Ej ).

| Numerical Analysis 10E

 Chapter 6.1: Direct Methods For Solving Linear
Systems 2

Definition (6.1)
An n ⇥ m (n by m) matrix is a rectangular array of elements
with n rows and m columns in which not only is the value of an
element important, but also its position in the array.

The notation for an n ⇥ m matrix will be a capital letter such as

A for the matrix and lowercase letters with double subscripts,
such as aij , to refer to the entry at the intersection of the ith row
and jth column; that is,
2 3
a11 a12 · · · a1m
6 a21 a22 · · · a2m 7
6 7
A = [aij ] = 6 . .. .. 7 .
4 .. . . 5
an1 an2 · · · anm

| Numerical Analysis 10E

 Chapter 6.1: Direct Methods For Solving Linear
Systems 3

An n ⇥ (n + 1) matrix can be used to represent the linear system

a11 x1 + a12 x2 + · · · + a1n xn = b1 ,

a21 x1 + a22 x2 + · · · + a2n xn = b2 ,
.. ..
. .
an1 x1 + an2 x2 + · · · + ann xn = bn ,

by constructing the augmented matrix

2 3
a11 a12 · · · a1n b1
6a21 a22 · · · a2n b2 7
6 7
[A, b] = 6 . .. .. .. 7 .
4 .. . . .5
an1 an2 ··· ann bn
.

| Numerical Analysis 10E

 Chapter 6.1: Linear Systems of Equations
4

Gaussian elimination with backward substitution

Through a sequential procedure for i = 2, 3, . . . , n 1 we perform the
operation

(Ej (aji /aii )Ei ) ! (Ej ) for each j = i + 1, i + 2, . . ., n,

provided aii 6= 0. This eliminates (changes the coefficient to zero) xi in each

row below the ith for all values of i = 1, 2, . . . , n 1. The resulting matrix has
the form: 2 3
a11 a12 · · · a1n a1,n+1
6 0 a22 · · · a2n a2,n+1 7
˜ 6 7
à = 6 . . . 7,
4 .. .. .. 5
0 ··· 0 ann an,n+1
where, except in the first row, the values of aij are not expected to agree with
˜
those in the original matrix à = [A, b]. The matrix à represents a linear
system with the same solution set as the original system .

| Numerical Analysis 10E

 Chapter 6.1: Linear Systems of Equations
5

Algorithm 6.1: GAUSSIAN ELIMINATION WITH

BACKSUB
To solve the n ⇥ n linear system

E1 : a11 x1 + a12 x2 + ··· + a1n xn = a1,n+1

E2 : a21 x1 + a22 x2 + ··· + a2n xn = a2,n+1
.. .. .. .. ..
. . . . .
En : an1 x1 + an2 x2 + ··· + ann xn = an,n+1

INPUT number of unknowns and equations n; augmented matrix

A = [aij ], where 1  i  n and 1  j  n + 1.
OUTPUT solution x1 , x2 , . . . , xn or message that the linear system has
no unique solution.

| Numerical Analysis 10E

 Chapter 6.1: Linear Systems of Equations
6

Algorithm 6.1: GAUSSIAN ELIMINATION WITH

BACKSUB
Step 1 For i = 1, . . . , n 1 do Steps 2–4. (Elimination process.)
Step 2 Let p be the smallest integer with i  p  n and api 6= 0.
If no integer p can be found
then OUTPUT (‘no unique solution exists’); STOP.
Step 3 If p 6= i then perform (Ep ) $ (Ei ).
Step 4 For j = i + 1, . . . , n do Steps 5 and 6.
Step 5 Set mji = aji /aii .
Step 6 Perform (Ej mji Ei ) ! (Ej );
Step 7 If ann = 0 then OUTPUT (‘no unique solution exists’); STOP.
Step 8 Set xn = an,n+1 /ann . (Start backward substitution.)
h Pn i
Step 9 For i = n 1, . . . , 1 set xi = ai,n+1 j=i+1 aij xj aii .
Step 10 OUTPUT (x1 , . . . , xn ); (Procedure completed successfully.)
STOP.
| Numerical Analysis 10E
 Chapter 6.1: Linear Systems of Equations
7

Operation Counts
Both the amount of time required to complete calculations and the
subsequent round-off error depend on the number of floating-point
arithmetic operations needed to solve a routine problem.

Multiplications/divisions
The total number of multiplications and divisions in Algorithm 6.1

2n3 + 3n2 5n n2 + n n3 n
+ = + n2 .
6 2 3 3

Additions/subtractions
The total number of additions and subtractions in Algorithm 6.1

n3 n n2 n n3 n2 5n
+ = + .
3 2 3 2 6
| Numerical Analysis 10E
 Chapter 6.2: Pivoting Strategies
8

Partial Pivoting
The simplest strategy is to select an element in the same
column that is below the diagonal and has the largest absolute
value; specifically, we determine the smallest p k such that
(k ) (k )
|apk | = max |aik |
k in

and perform (Ek ) $ (Ep ). In this case no interchange of

columns is used.

| Numerical Analysis 10E

 Chapter 6.2: Pivoting Strategies
9

Algorithm 6.2: GAUSSIAN ELIMINATION WITH PARTIAL

PIVOTING
To solve the n ⇥ n linear system

E1 : a11 x1 + a12 x2 + ··· + a1n xn = a1,n+1

E2 : a21 x1 + a22 x2 + ··· + a2n xn = a2,n+1
.. ..
. .
En : an1 x1 + an2 x2 + ··· + ann xn = an,n+1

INPUT number of unknowns and equations n; augmented matrix A = [aij ]

where 1  i  n and 1  j  n + 1.
OUTPUT solution x1 , . . . , xn or message that the linear system has no unique
solution.

Step 1 For i = 1, . . . , n set NROW(i) = i. (Initialize row pointer.)

| Numerical Analysis 10E

 Chapter 6.2: Pivoting Strategies
10

Algorithm 6.2: GAUSSIAN ELIMINATION WITH PARTIAL

PIVOTING
Step 2 For i = 1, . . . , n 1 do Steps 3–6. (Elimination process.)
Step 3 Let p be the smallest integer with i  p  n and
|a(NROW(p), i)| = maxijn |a(NROW(j), i)|.
(Notation: a(NROW(i), j) ⌘ aNROW i ,j .)
Step 4 If a(NROW(p), i) = 0 then OUTPUT (‘no unique solution exists’);
STOP.
Step 5 If NROW(i) 6= NROW(p) then set NCOPY = NROW(i);
NROW(i) = NROW(p);
NROW(p) = NCOPY.
(Simulated row interchange.)
Step 6 For j = i + 1, . . . , n do Steps 7 and 8.
Step 7 Set m(NROW(j), i) = a(NROW(j), i)/a(NROW(i), i).
Step 8 Perform (ENROW (j) m(NROW(j), i) · ENROW (i) ) ! (ENROW (j) ).
Step 9 If a(NROW(n), n) = 0 then OUTPUT (‘no unique solution exists’);
STOP.

| Numerical Analysis 10E

 Chapter 6.2: Pivoting Strategies
11

Algorithm 6.2: GAUSSIAN ELIMINATION WITH PARTIAL

PIVOTING
Step 10 Set xn = a(NROW(n), n + 1)/a(NROW(n), n).
(Start backward substitution.)
Step 11 For i = n 1, . . . , 1 Pn
a(NROW(i), n + 1) j=i+1 a(NROW(i), j) · xj
set xi = .
a(NROW(i), i)
Step 12 OUTPUT (x1 , . . . , xn ); (Procedure completed successfully.)
STOP.

| Numerical Analysis 10E

 Chapter 6.2: Pivoting Strategies
12

Algorithm 6.3: GAUSSIAN ELIMINATION WITH SCALED

PIVOTING
The only steps in this algorithm that differ from those of
Algorithm 6.2 are:

Step 1 For i = 1, . . . , n set si = max1jn |aij |;

if si = 0 then OUTPUT (‘no unique solution exists’);
STOP.
set NROW(i) = i.
Step 2 For i = 1, . . . , n 1 do Steps 3–6. (Elimination process.)
Step 3 Let p be the smallest integer with i  p  n and
|a(NROW(p), i)| |a(NROW(j), i)|
= maxijn .
s(NROW(p)) s(NROW(j))

| Numerical Analysis 10E

 Chapter 6.2: Pivoting Strategies
13

COMPLETE PIVOTING
Pivoting can incorporate interchange of both rows and columns.
Complete (or maximal) pivoting at the k th step searches all the
entries aij , for i = k , k + 1, . . . , n and j = k , k + 1, . . . , n, to find the
entry with the largest magnitude. Both row and column interchanges
are performed to bring this entry to the pivot position. The total
additional time required to incorporate complete pivoting into
Gaussian elimination is
n
X n(n 1)(2n + 5)
(k 2 1) =
6
k =2

comparisons. Complete pivoting is the strategy recommended only

for systems where accuracy is essential and the amount of execution
time needed for this method can be justified.

| Numerical Analysis 10E

 Chapter 6.3: Linear Algebra and Matrix Inversion
14

Definition (6.2)
Two matrices A and B are equal if they have the same number of
rows and columns, say n ⇥ m, and if aij = bij , for each i = 1, 2, . . . , n
and j = 1, 2, . . . , m.

Definition (6.3)
If A and B are both n ⇥ m matrices, then the sum of A and B,
denoted A + B, is the n ⇥ m matrix whose entries are aij + bij , for
each i = 1, 2, . . . , n and j = 1, 2, . . . , m.

Definition (6.4)
If A is an n ⇥ m matrix and is a real number, then the scalar
multiplication of and A, denoted A, is the n ⇥ m matrix whose
entries are aij , for each i = 1, 2, . . . , n and j = 1, 2, . . . , m.

| Numerical Analysis 10E

 Chapter 6.3: Linear Algebra and Matrix Inversion
15

We let O denote a matrix all of whose entries are 0.

Theorem (6.5)
Let A, B, and C be n ⇥ m matrices and and µ be real numbers. The
following properties of addition and scalar multiplication hold:

(i) A + B = B + A, (ii) (A + B) + C = A + (B + C),

(iii) A + O = O + A = A, (iv) A + ( A) = A + A = 0,

(v) (A + B) = A + B, (vi) ( + µ)A = A + µA,

(vii) (µA) = ( µ)A, (viii) 1A = A.

All these properties follow from similar results concerning the real
numbers.

| Numerical Analysis 10E

 Chapter 6.3: Linear Algebra and Matrix Inversion
16

Definition (6.6)
Let A be an n ⇥ m matrix and b an m-dimensional column vector. The
matrix-vector product of A and b, denoted Ab, is an n-dimensional
column vector given by
2 3 2 3 2Pm 3
a11 a12 · · · a1m b1 Pi=1 a1i bi
6a21 a22 · · · a2m 7 6 b2 7 6 m a2i bi 7
6 7 6 7 6 i=1 7
Ab = 6 . .. .. 7 6 .. 7 = 6 .. 7.
4 .. . . 5 4 . 5 4 . 5
Pm
an1 an2 · · · anm bm i=1 ani bi

NOTE: For this product to be defined the number of columns of the

matrix A must match the number of rows of the vector b, and the
result is another column vector with the number of rows matching the
number of rows in the matrix.

| Numerical Analysis 10E

 Chapter 6.3: Linear Algebra and Matrix Inversion
17

Definition (6.7)
Let A be an n ⇥ m matrix and B an m ⇥ p matrix. The matrix product of A
and B, denoted AB, is an n ⇥ p matrix C whose entries cij are
m
X
cij = aik bkj = ai1 b1j + ai2 b2j + · · · + aim bmj ,
k =1

for each i = 1, 2, . . . n, and j = 1, 2, . . . , p.

Theorem (6.8)
Let A be an n ⇥ m matrix, B be an m ⇥ k matrix, C be a k ⇥ p matrix, D be an
m ⇥ k matrix, and be a real number. The following properties hold:

(a) A(BC) = (AB)C; (b) A(B + D) = AB + AD;

(c) (AB) = ( A)B = A( B).

| Numerical Analysis 10E

 Chapter 6.3: Linear Algebra and Matrix Inversion
18

Definition (6.9)
(i) A square matrix has the same number of rows as columns.
(ii) A diagonal matrix D = [dij ] is a square matrix with dij = 0 whenever
i 6= j.
(iii) The identity matrix of order n, In = [ ij ], is a diagonal matrix whose
diagonal entries are all 1s. When the size of In is clear, this matrix is
generally written simply as I.

Definition (6.10)
An upper-triangular n ⇥ n matrix U = [uij ] has, for each j = 1, 2, . . . , n, the
entries
uij = 0, for each i = j + 1, j + 2, . . . , n;
and a lower-triangular matrix L = [lij ] has, for each j = 1, 2, . . . , n, the
entries
lij = 0, for each i = 1, 2, . . . , j 1.

| Numerical Analysis 10E

 Chapter 6.3: Linear Algebra and Matrix Inversion
19

Definition (6.11)
An n ⇥ n matrix A is said to be nonsingular (or invertible) if an n ⇥ n
matrix A 1 exists with AA 1 = A 1 A = I. The matrix A 1 is called the
inverse of A. A matrix without an inverse is called singular (or
noninvertible).

Theorem (6.12)
For any nonsingular n ⇥ n matrix A:
1
(i) A is unique.
1 1 1
(ii) A is nonsingular and (A ) = A.
(iii) If B is also a nonsingular n ⇥ n matrix, then
(AB) 1 = B 1 A 1 .

| Numerical Analysis 10E

 Chapter 6.3: Linear Algebra and Matrix Inversion
20

Definition (6.13)
The transpose of an n ⇥ m matrix A = [aij ] is the m ⇥ n matrix
At = [aji ], where for each i, the ith column of At is the same as the
ith row of A. A square matrix A is called symmetric if A = At .

Theorem (6.14)
The following operations involving the transpose of a matrix hold
whenever the operation is possible:

(i) (At )t = A, (iii) (AB)t = B t At ,

t
(ii) (A + B) = A + B , t t (iv) if A 1 exists, then
(A 1 )t = (At ) 1 .

| Numerical Analysis 10E

 Chapter 6.4: The Determinant of a Matrix
21

Definition (6.15)
Suppose that A is a square matrix.
(i) If A = [a] is a 1 ⇥ 1 matrix, then det A = a.
(ii) If A is an n ⇥ n matrix, with n > 1 the minor Mij is the determinant of the
(n 1) ⇥ (n 1) submatrix of A obtained by deleting the ith row and jth
column of the matrix A.
(iii) The cofactor Aij associated with Mij is defined by Aij = ( 1)i+j Mij .
(iv) The determinant of the n ⇥ n matrix A, when n > 1, is given either by
n
X n
X
det A = aij Aij = ( 1)i+j aij Mij , for any i = 1, 2, . . . , n,
j=1 j=1

or by
n
X n
X
det A = aij Aij = ( 1)i+j aij Mij , for any j = 1, 2, . . . , n.
i=1 i=1

| Numerical Analysis 10E

 Chapter 6.4: The Determinant of a Matrix
22

Theorem (6.16)
Suppose A is an n ⇥ n matrix:
(i) If any row or column of A has only zero entries, then det A = 0.
(ii) If A has two rows or two columns the same, then det A = 0.
(iii) If à is obtained from A by the operation (Ei ) $ (Ej ), with i 6= j, then
det à = det A.
(iv) If à is obtained from A by the operation ( Ei ) ! (Ei ), then
det à = det A.
(v) If à is obtained from A by the operation (Ei + Ej ) ! (Ei ) with i 6= j,
then det à = det A.
(vi) If B is also an n ⇥ n matrix, then det AB = det A det B.
(vii) det At = det A.
1 1 1
(viii) When A exists, det A = (det A) .
(ix) If A is an
Qupper triangular, lower triangular, or diagonal matrix, then
det A = ni=1 aii .
| Numerical Analysis 10E
 Chapter 6.4: The Determinant of a Matrix
23

Theorem (6.17)
The following statements are equivalent for any n ⇥ n matrix A:
(i) The equation Ax = 0 has the unique solution x = 0 .
(ii) The system Ax = b has a unique solution for any n-dimensional
column vector b.
1
(iii) The matrix A is nonsingular; that is, A exists.
(iv) det A 6= 0.
(v) Gaussian elimination with row interchanges can be performed
on the system Ax = b for any n-dimensional column vector b.

Corollary (6.18)
Suppose that A and B are both n ⇥ n matrices with either AB = I or
BA = I. Then B = A 1 (and A = B 1 ).
| Numerical Analysis 10E
 Chapter 6.5: Matrix Factorization
24

Theorem (6.19)
If Gaussian elimination can be performed on the linear system
Ax = b without row interchanges, then the matrix A can be factored
into the product of a lower-triangular matrix L and an upper-triangular
(i) (i)
matrix U, that is, A = LU, where mji = aji /aii ,
2 3
(1) (1) (1)
a11 a12 a1n 2 3
6 7 1 0 0
6 0
(2)
a22 7 6 m21 7
6 7 1
U=6 7 , and L = 6
4
7.
6 (n 1)
an 7 0 5
4 1,n 5 mn1 mn,n 1
(n) 1
0 0 ann

| Numerical Analysis 10E

 Chapter 6.5: Matrix Factorization
25

Algorithm 6.4: LU FACTORIZATION

To factor the n ⇥ n matrix A = [aij ] into the product of the
lower-triangular matrix L = [lij ] and the upper-triangular matrix
U = [uij ]; that is, A = LU, where the main diagonal of either L or U
consists of all ones:
INPUT dimension n; the entries aij , 1  i, j  n of A; the diagonal
l11 = · · · = lnn = 1 of L or the diagonal u11 = · · · = unn = 1 of U.
OUTPUT the entries lij , 1  j  i, 1  i  n of L and the entries, uij ,
i  j  n, 1  i  n of U.

Step 1 Select l11 and u11 satisfying l11 u11 = a11 .

If l11 u11 = 0 then OUTPUT (‘Factorization impossible’);
STOP.
Step 2 For j = 2, . . . , n set u1j = a1j /l11 ; (First row of U.)
lj1 = aj1 /u11 . (First column of L.)

| Numerical Analysis 10E

 Chapter 6.5: TMatrix Factorization
26

Algorithm 6.4: LU FACTORIZATION

Step 3 For i = 2, . . . , n 1 do Steps 4 and 5.
Pi 1
Step 4 Select lii and uii satisfying lii uii = aii k =1 lik uki .
If lii uii = 0 then OUTPUT (‘Factorization impossible’);
STOP.
Step 5 For j = i + 1, . .h. , n i
1
Pi 1
set uij = lii aij k =1 lik ukj ; (ith row of U.)
h Pi 1 i
1
lji = uii aji k =1 ljk uki . (ith column of L.)
Pn 1
Step 6 Select lnn and unn satisfying lnn unn = ann k =1 lnk ukn .
(Note: If lnn unn = 0, then A = LU but A is singular.)
Step 7 OUTPUT (lij for j = 1, . . . , i and i = 1, . . . , n);
OUTPUT (uij for j = i, . . . , n and i = 1, . . . , n);
STOP.

| Numerical Analysis 10E

 Chapter 6.5: Matrix Factorization
27

Permutation matrix
An n ⇥ n permutation matrix P = [pij ] is a matrix obtained by
rearranging the rows of In , the identity matrix. This gives a
matrix with precisely one nonzero entry in each row and in
each column, and each nonzero entry is a 1.

NOTE: Any nonsingular matrix A can be factored into

A = P t LU.

| Numerical Analysis 10E

 Chapter 6.6: Special Types of Matrices
28

Definition (6.20)
The n ⇥ n matrix A is said to be diagonally dominant when
n
X
|aii | |aij | holds for each i = 1, 2, . . . , n.
j=1,
j6=i

A diagonally dominant matrix is said to be strictly diagonally

dominant when the inequality in (6.10) is strict for each n, that is,
when
n
X
|aii | > |aij | holds for each i = 1, 2, . . . , n.
j=1,
j6=i

| Numerical Analysis 10E

 Chapter 6.6: Special Types of Matrices
29

Theorem (6.21)
A strictly diagonally dominant matrix A is nonsingular.
Moreover, in this case, Gaussian elimination can be performed
on any linear system of the form Ax = b to obtain its unique
solution without row or column interchanges, and the
computations will be stable with respect to the growth of
round-off errors.

Definition (6.22)
A matrix A is positive definite if it is symmetric and if xt Ax > 0
for every n-dimensional vector x 6= 0.

| Numerical Analysis 10E

 Chapter 6.6: Special Types of Matrices
30

Theorem (6.23)
If A is an n ⇥ n positive definite matrix, then

(i) A has an inverse; (ii) aii > 0, for each i = 1, 2, . . . , n;

(iii) max1k ,jn |akj |  (iv) (aij )2 < aii ajj , for each i 6= j.
max1in |aii |;

Definition (6.24)
A leading principal submatrix of a matrix A is a matrix of the form
2 3
a11 a12 · · · a1k
6 a21 a22 · · · a2k 7
6 7
Ak = 6 . . . . 7,
4 .. .. .. .. 5
ak 1 ak 2 · · · akk

for some 1  k  n.
| Numerical Analysis 10E
 Chapter 6.6: Special Types of Matrices
31

Theorem (6.25)
A symmetric matrix A is positive definite if and only if each of its
leading principal submatrices has a positive determinant.

Theorem (6.26)
The symmetric matrix A is positive definite if and only if
Gaussian elimination without row interchanges can be
performed on the linear system Ax = b with all pivot elements
positive. Moreover, in this case, the computations are stable
with respect to the growth of round-off errors.

| Numerical Analysis 10E

 Chapter 6.6: Special Types of Matrices
32

Corollary (6.27)
The matrix A is positive definite if and only if A can be factored
in the form LDLt , where L is lower triangular with 1s on its
diagonal and D is a diagonal matrix with positive diagonal
entries.

Corollary (6.28)
The matrix A is positive definite if and only if A can be factored
in the form LLt , where L is lower triangular with nonzero
diagonal entries.

| Numerical Analysis 10E

 Chapter 6.6: Special Types of Matrices
33

Algorithm 6.5: LDLt FACTORIZATION

To factor the positive definite n ⇥ n matrix A into the form LDLt , where
L is a lower triangular matrix with 1s along the diagonal and D is a
diagonal matrix with positive entries on the diagonal:
INPUT the dimension n; entries aij , for 1  i, j  n of A.
OUTPUT the entries lij , for 1  j < i and 1  i  n of L, and di , for
1  i  n of D.

Step 1 For i = 1, . . . , n do Steps 2–4.

Step 2 For j = 1, . . . , i 1, set vj = lij dj .
Pi 1
Step 3 Set di = aii j=1 lij vj .
Pi 1
Step 4 For j = i + 1, . . . , n set lji = (aji k =1 ljk vk )/di .
Step 5 OUTPUT (lij for j = 1, . . . , i 1 and i = 1, . . . , n);
OUTPUT (di for i = 1, . . . , n);
STOP.
| Numerical Analysis 10E
 Chapter 6.6: Special Types of Matrices
34

Corollary (6.29)
Let A be a symmetric n ⇥ n matrix for which Gaussian
elimination can be applied without row interchanges. Then A
can be factored into LDLt , where L is lower triangular with 1s on
(1) (n)
its diagonal and D is the diagonal matrix with a11 , . . . , ann on
its diagonal.

| Numerical Analysis 10E

 Chapter 6.6: Special Types of Matrices
35

Algorithm 6.6: CHOLESKY FACTORIZATION

To factor the positive definite n ⇥ n matrix A into LLt , where L is lower
triangular:
INPUT the dimension n; entries aij , for 1  i, j  n of A.
OUTPUT the entries lij , for 1  j  i and 1  i  n of L. (The entries of
U = Lt are uij = lji , for i  j  n and 1  i  n.)
p
Step 1 Set l11 = a11 .
Step 2 For j = 2, . . . , n, set lj1 = aj1 /l11 .
Step 3 For i = 2, . . . , n 1 do Steps 4 and 5.
⇣ Pi 1 2 ⌘1/2
Step 4 Set lii = aii k =1 lik .
⇣ Pi ⌘
1
Step 5 For j = i + 1, . . . , n set lji = aji k =1 ljk lik /lii .
⇣ Pn 1 2 ⌘1/2
Step 6 Set lnn = ann k =1 lnk .
Step 7 OUTPUT (lij for j = 1, . . . , i and i = 1, . . . , n);
STOP.
| Numerical Analysis 10E
 Chapter 6.6: Special Types of Matrices
36

Definition (6.30)
An n ⇥ n matrix is called a band matrix if integers p and q, with 1 < p,
q < n, exist with the property that aij = 0 whenever p  j i or q  i j. The
band width of a band matrix is defined as w = p + q 1.

Matrices of bandwidth 3 occurring when p = q = 2 are called tridiagonal

because they have the form
2 3
a11 a12 0 ··· ··· 0
6 .. .. 7
6 . 7
6 a21 a22 a23 ··· . 7
6 7
6
6 0 . . .
.
7
7
6 a32 a33 a34 . . 7
A=6 7.
6 .. .. .. .. .. 7
6 .
6 . . . . 0 7
7
6 . 7
6 . .. .. .. 7
4 . . . . an 1,n 5
0 ··· ··· 0 an,n 1 ann

| Numerical Analysis 10E

 Chapter 6.6: Special Types of Matrices
37

Algorithm 6.7: CROUT FACTORIZATION TRI DIAG

To solve the n ⇥ n linear system
E1 : a11 x1 + a12 x2 = a1,n+1 ,
E2 : a21 x1 + a22 x2 + a23 x3 = a2,n+1 ,
.. .. ..
. . .
En 1 : an 1,n 2 xn 2 + an 1,n 1 xn 1 + an 1,n xn = an 1,n+1 ,
En : an,n 1 xn 1 + ann xn = an,n+1 ,

which is assumed to have a unique solution:

INPUT the dimension n; the entries of A.
OUTPUT the solution x1 , . . . , xn .

(Steps 1–3 set up and solve Lz = b.)

| Numerical Analysis 10E

 Chapter 6.6: Special Types of Matrices
38

Algorithm 6.7: CROUT FACTORIZATION TRI DIAG

Step 1 Set l11 = a11 ;
u12 = a12 /l11 ;
z1 = a1,n+1 /l11 .
Step 2 For i = 2, . . . , n 1 set li,i 1 = ai,i 1 ; (ith row of L.)
lii = aii li,i 1 ui 1,i ;
ui,i+1 = ai,i+1 /lii ; ((i + 1)th column of U.)
zi = (ai,n+1 li,i 1 zi 1 )/lii .
Step 3 Set ln,n 1 = an,n 1 ; (nth row of L.)
lnn = ann ln,n 1 un 1,n .
zn = (an,n+1 ln,n 1 zn 1 )/lnn .
(Steps 4 and 5 solve Ux = z.)
Step 4 Set xn = zn .
Step 5 For i = n 1, . . . , 1 set xi = zi ui,i+1 xi+1 .
Step 6 OUTPUT (x1 , . . . , xn );
STOP.

| Numerical Analysis 10E

 Chapter 6.6: Special Types of Matrices
39

Theorem (6.31)
Suppose that A = [aij ] is tridiagonal with ai,i 1 ai,i+1 6= 0, for
each i = 2, 3, . . . , n 1. If |a11 | > |a12 |, |aii | |ai,i 1 | + |ai,i+1 |,
for each i = 2, 3, . . . , n 1, and |ann | > |an,n 1 |, then A is
nonsingular and the values of lii described in the Crout
Factorization Algorithm are nonzero for each i = 1, 2, . . . , n.

| Numerical Analysis 10E