Homework 1

§20.1
1. Solve the following linear systems by Gauss
 elimination.
{  5x1 + 3x2 + x3 = 2
6x1 + x2 = −3
(a) (b) − 4x2 + 8x3 = −3
4x1 − 2x2 = 6 
10x1 − 6x2 + 26x3 = 0
 
 4x1 + 4x2 + 2x3 = 0  x1 + x 2 + x3 = 3
(c) 3x − x2 + 2x3 = 0 (d) 4x + 2x2 − x3 = 5
 1  1
3x1 + 7x2 + x3 = 0 9x + 5x2 − x3 = 13
[ ] [ ] 1
6 1 −3 6 1 −3
Sol. (a) −→ −→ x2 = −3, x1 = 0
4 −2 6 0 − 38 8
     
5 3 1 2 10 −6 26 0 10 −6 26 0
(b)  0 −4 8 −3  −→  0 −4 8 −3  −→  0 −4 8 −3 
10 −6 26 0 5 3 1 2 0 6 −12 2
 
10 −6 26 0
−→  0 −4 8 −3  −→ No solutions.
0 0 0 − 25
     
4 4 2 0 4 4 2 0 4 4 2 0
(c)  3 −1 2 0  −→  0 −4 1
2 0
 −→  0 −4 1 0 
2
3 7 1 0 0 4 − 12 0 0 0 0 0
−→ x3 = 8t, x2 = t, x1 = −3t for arbitrary t.
     
1 1 1 3 1 1 1 3 1 1 1 3
(d)  4 2 −1 5  −→  0 −2 −5 −7  −→  0 −2 −5 −7 
9 5 −1 13 0 −4 −10 −14 0 0 0 0
−→ x3 = 2t, x2 = −5t − 72 , x1 = 3t + 21 for arbitrary t.

2. Find the a and b such that ax1 + x2 = b, x1 + x2 = 3 has

(a) a unique solution, (b) infinitely many solutions, (c) no solutions.
Sol.
[ Using] Gauss elimination,
[ ] [ ]
a 1 b 1 1 3 1 1 3
−→ −→
1 1 3 a 1 b 0 1 − a b − 3a
(a) If a ̸= 1, it has a unique solution x1 = 1−a
3−b
, x2 = b−3a
1−a .
(b) If a = 1 and b = 3, it has infinitely many solutions x1 = 3 − t, x2 = t for
arbitrary t.
(c) If a = 1 and b ̸= 3, it has no solutions.

Computational Science & Engineering (CSE) T. Jeong

 Homework 2

§20.2
1. Show the LU-factorization and solve by Doolittle’s method.
{  2x1 + x2 + 2x3 = 0
2x1 + 9x2 = 82
(a) (b) −2x1 + 2x2 + x3 = 0
3x1 − 5x2 = −62 
x1 + 2x2 − 2x3 = 18
Sol. (a) The[LU-factorization
] with
[ Doolittle’s
][ method ] is
2 9 1 0 2 9
= LU = .
3 −5 1.5 1 0 −18.5
[ ][ ] [ ] [ ] [ ]
1 0 y1 82 y1 82
= =⇒ =
1.5 1 y2 −62 y2 −185
[ ][ ] [ ] [ ] [ ]
2 9 x1 82 x1 −4
= =⇒ =
0 −18.5 x2 −185 x2 10
(b) The LU-factorization
 with Doolittle’s
  method
is 
2 1 2 1 0 0 2 1 2
 −2 2 1  = LU =  −1 1 0   0 3 3  .
1 2 −2 1 1
1 0 0 − 29
    2  2    
1 0 0 y1 0 y1 0
 −1 1 0   y2  =  0  =⇒  y2  =  0 
1 1
 2 2 1   y3  
18
 
y3
 
18

2 1 2 x1 0 x1 2
 0 3 3   x2  =  0  =⇒  x2  =  4 
0 0 − 92 x3 18 x3 −4

2. Show
 the LU-factorization and solve by Cholesky’s method.
 4x1 + 6x2 + 8x3 = 0
(a) 6x + 34x2 + 52x3 = −160
 1
8x + 52x2 + 129x3 = −452
 1
 4x1 + 2x3 = 1.5
(b) 4x2 + x3 = 4.0

2x1 + x2 + 2x3 = 2.5
Sol. (a) TheLU-factorization
 with Cholesky’s
 method
 is 
4 6 8 2 0 0 2 3 4
 6 34 52  = LL =  3 5 0   0 5 8  .
T

8 52 129 4 8 7 0 0 7
        
2 0 0 y1 0 y1 0
 3 5 0   y2  =  −160  =⇒  y2  =  −32 
4 8 7 y3 −452 y3 −28

Computational Science & Engineering (CSE) T. Jeong

 Homework 3
        
2 3 4 x1 0 x1 8
 0 5 8   x2  =  −32  =⇒  x2  =  0 
0 0 7 x3 −28 x3 −4

(b) (a) The LU-factorization withCholesky’smethod is 

 
4 0 2 2 0 0 2 0 1
 0 4 1  = LL =  0 2 0   0 2 21 
  
√ 
T
√
.
2 1 2 1 3 3
1 2 2 0 0 2
        3 
2 0 0 y1 1.5 y1
 0 2 0        24 
 √ 
y 2 = 4.0 =⇒ y 2 =  √ 
1 3 y3 2.5 y3 3
1 2 2
    3     1 
2
2 0 1 x1 x1
 0 2 1    2 
4
  
8
3 
 √2  x 2 =  √  =⇒ x 2 = 4
3 x 3 x 1
0 0 2 3 2 3

§20.3
1. Using the Gauss-Seidel iteration, solve the following system with 5 step and 6S.
Compare
 it with the exact solution. Start from x  = [1 1 1]T .
 4x1 − x2 = 21  3x1 + 2x2 + x3 = 7
(a) −x1 + 4x2 − x3 = −45 (b) x + 3x2 + 2x3 = 4
  1
− x2 + 4x3 = 33 2x1 + x2 + 3x3 = 7
Sol. (a) The exact solution is x = [3, − 9, 6]T .
x(0) = [1.000000, 1.000000, 1.000000]T
x(1) = [5.500000, − 9.625000, 5.843750]T
x(2) = [2.843750, − 9.078125, 5.980469]T
x(3) = [2.980469, − 9.009766, 5.997559]T
x(4) = [2.997559, − 9.001221, 5.999695]T
x(5) = [2.999695, − 9.000153, 5.999962]T

(b) The exact solution is x = [2, 0, 1]T .

x(0) = [1.000000, 1.000000, 1.000000]T
x(1) = [1.333333, 0.222222, 1.370370]T
x(2) = [1.728395, − 0.156379, 1.233196]T
x(3) = [2.026520, − 0.164304, 1.037088]T
x(4) = [2.097174, − 0.057116, 0.954256]T
x(5) = [2.053325, 0.012721, 0.960210]T

Computational Science & Engineering (CSE) T. Jeong

 Homework 4

2. Using the Jacobi iteration, solve the following system with 5 step and 6S. Com-
parewith the Gauss-Seidel iteration. Start from x= [1 1 1]T .
 4x1 − x2 = 21  3x1 + 2x2 + x3 = 7
(a) −x1 + 4x2 − x3 = −45 (b) x + 3x2 + 2x3 = 4
  1
− x2 + 4x3 = 33 2x1 + x2 + 3x3 = 7
Sol. (a) The exact solution is x = [3, − 9, 6]T .
x(0) = [1.000000, 1.000000, 1.000000]T
x(1) = [5.500000, − 10.750000, 8.500000]T
x(2) = [2.562500, − 7.750000, 5.562500]T
x(3) = [3.312500, − 9.218750, 6.312500]T
x(4) = [2.945313, − 8.843750, 5.945313]T
x(5) = [3.039063, − 9.027344, 6.039063]T

(b) The exact solution is x = [2, 0, 1]T .

x(0) = [1.000000, 1.000000, 1.000000]T
x(1) = [1.333333, 0.333333, 1.333333]T
x(2) = [1.666667, 0.000000, 1.333333]T
x(3) = [1.888889, − 0.111111, 1.222222]T
x(4) = [2.000000, − 0.111111, 1.111111]T
x(5) = [2.037037, − 0.074074, 1.037037]T

3. Compute the Frobenius norm, column sum norm, and row sum norm of following

matrices.     
10 1 1 4 0 5 5 1 2
(a)  1 10 1  (b)  1 6 2  (c)  1 4 −2 
1 1 10 8 2 1 2 3 8
√∑ ∑ √
3 3 2
Sol. (a) Frobenius norm : j=1 k=3 cjk = 303 = 17.41
column sum norm : max{11, 11, 11} = 11
row sum norm : max{11,√∑ 11, 11} = 11
3 ∑3 2 =
√
(b) Frobenius norm : j=1 c
k=3 jk 151 = 12.29
column sum norm : max{13, 8, 8} = 13
row sum norm : max{9,√∑9, 11} = 11
3 ∑3 2 =
√
(c) Frobenius norm : j=1 c
k=3 jk 128 = 11.31
column sum norm : max{8, 8, 12} = 12
row sum norm : max{8, 7, 13} = 13

Computational Science & Engineering (CSE) T. Jeong

 Homework 5

§20.4
1. (a) Compute the l1 , l2 , and l∞ norm of [4, 1, 8].
(b) For what x = [a b c] will ∥x∥1 = ∥x∥2 ?
(c) Show that ∥x∥∞ ≤ ∥x∥2 ≤ ∥x∥1 .
Sol. (a) l1 √
norm : 4 + 1 + 8 = 13
l2 norm : 16 + 1 + 64 = 9
l∞ norm : maxj |xj | = 8
(b) ∥x∥21 = (|a| + |b| + |c|) = (a2 + b2 + c2 ) = ∥x∥22 implies |ab| + |bc| + |ca| = 0.
Since |ab|, |bc|, |ca| ≥ 0, we have |ab| = |bc| = |ca| = 0. If a ̸= 0, then b = c = 0.
Hence [a 0 0], [0 b 0], and [0 0 c] are the vectors that satisfy ∥x∥1 = ∥x∥2 .

2. Compute the matrix norm and the condition number corresponding to the l1
norm.  
[ ] [ ] −2 4 −1
2 1 −3 4
(a) A = (b) B = (c) C =  −2 3 0 
0 4 1 2
7 −12 2
Sol. Note that
[ the l1 ]norm of matrix is the column sum norm.
4 −1
(a) A−1 = 81 , ∥A∥1 = 5 and ∥A−1 ∥1 = 12 ,
0 2
the condition number κ(A) = ∥A∥1 ∥A−1 ∥1 = 52
[ ]
2 −4
(b) B −1 = − 10
1
, ∥B∥1 = 6 and ∥B −1 ∥1 = 10
7
,
−1 −3
the condition number κ(B) = ∥B∥1 ∥B −1 ∥1 = 42
  10
6 4 3
(b) C =  4 3 2 , ∥C∥1 = 19 and ∥C −1 ∥1 = 13,
−1

3 4 2
the condition number κ(C) = ∥C∥1 ∥C −1 ∥1 = 247

3. Solve Ax = b1 and Ax = b2 , comparing the solutions. Compute the condition

number. [ ] [ ] [ ]
5 −7 −2 −2
A= b1 = b2 =
−7 10 3 3.1
T T
Sol. From
[ Ax = ] b1 , we have x = [1 1] . From Ax = b2 , we have x = [1.7 1.5] .
10 7
A−1 = , ∥A∥1 = 17 and ∥A−1 ∥1 = 17,
7 5
the condition number κ(A) = ∥A∥1 ∥A−1 ∥1 = 289

Computational Science & Engineering (CSE) T. Jeong

 Homework 6

§20.5
1. (a) Fit a straight line to the given point (x, y) by least squares.
(0, 2), (2, 0), (3, − 2), (5, −3)
(b) Fit a parabola to the given point (x, y) by least squares.
(−1, 5), (1, 3), (2, 4), (3, 8)
Sol. (a) Let the straight line y = a + bx be fitted.
∑ ∑ 2 ∑ ∑
n = 4, xj = 10, xj = 38, yj = −3, xj yj = −21
The normal equations are 4a + 10b = −3 and 10a + 38b = −21.
The solution is y = 1.8462 − 1.0385x.
(b) Let the parabola curve y = a + bx + cx2 be fitted.
∑ ∑ 2 ∑ 3 ∑ 4
n = 4, xj = 5, xj = 15, xj = 35, xj = 99,
∑ ∑ ∑ 2
yj = 20, xj yj = 30, xj yj = 96
The normal equations are
4a + 5b + 15c = 20
5a + 15b + 35c = 30
15a + 35b + 99c = 96
The solution is y = 2.9545 − 1.1591x + 0.9318.

2. The least squares approximation of a function f (x) on an interval a ≤ x ≤ b by

a function
Fm (x) = a0 y0 (x) + · · · + am ym (x)
where y0 (x), · · · , ym (x) are given, requires the determination of the coefficients
a0 , · · · , am such that
∫ b
∥f − Fm ∥ = |f (x) − Fm (x)|2 dx
a

becomes minimum.
Show that this leads to m + 1 normal equations
∑
m
hjk ak = bj for j = 0, · · · , m
k=0

where ∫ ∫
b b
hjk = yj (x)yk (x)dx, bj = f (x)yj (x)dx.
a a
proof. Substituting Fm (x) into the integral, we have
∫ b ∑
m ∫ b ∑
m ∑
m ∫ b
∥f − Fm ∥ = |f (x)| − 2
2
aj f (x)yj (x)dx + aj ak yj (x)yk (x)dx.
a j=0 a j=0 k=0 a

Computational Science & Engineering (CSE) T. Jeong

 Homework 7

Taking partial derivative with respect to al ,

∫ b ∑m ∫ b
∂
∥f − Fm ∥ = 0 − 2 f (x)yl (x)dx + aj yj (x)yl (x)dx = 0
∂ai a j=0 a

or ∫ ∫
∑
m b b
aj yj (x)yl (x)dx = f (x)yl (x)dx.
j=0 a a

§20.8
1. Apply the power method without scaling(3 step), using x0 = [1, 1]T . Give the
Rayleigh
[ quotient
] and error [ bound.]
7 −3 9 4
(a) (b)
−3 −1 4 3
Sol. Let xn+1 = Axn .
(a) Apply
[ ]the power method [ again] to obtain the first eigenvalue
1 4
x0 = , x1 = Ax0 = , q = 0.0000, ϵ ≤ 4.0000
1 −4
[ ]
40
x2 = Ax1 = , q = 6.0000, ϵ ≤ 4.0000
−8
[ ]
304
x3 = Ax2 = , q = 7.8462, ϵ ≤ 1.2308
−112
We choose the first eigenvalue λ1 = 7.8462
Let B = A − kI = A − 8I. Apply the power method again to obtain the second
eigenvalue,
[ ] [ ]
1 −4
x0 = , x1 = Bx0 = , q = −8.0000, ϵ ≤ 4.0000
1 −12
[ ]
40
x2 = Bx1 = , q = −10.0000, ϵ ≤ 0.0000
120
[ ]
−400
x3 = Bx2 = , q = −10.0000, ϵ ≤ 0.0000
−1200
We choose the second eigenvalue λ2 = −10.0000 + k = −2.0000
Note that the exact eigenvalues are 8 and −2.

(b) Apply
[ ] the power method
[ again
] to obtain the first eigenvalue
1 13
x0 = , x1 = Ax0 = , q = 10.0000, ϵ ≤ 3.0000
1 7
[ ]
145
x2 = Ax1 = , q = 10.9908, ϵ ≤ 0.3028
73
[ ]
1597
x3 = Ax2 = , q = 10.9999, ϵ ≤ 0.0275
799

Computational Science & Engineering (CSE) T. Jeong

 Homework 8

We choose the first eigenvalue λ1 = 10.9999

Let B = A − kI = A − 11I. Apply the power method again to obtain the second
eigenvalue,
[ ] [ ]
1 2
x0 = , x1 = Bx0 = , q = −1, ϵ ≤ 3.0000
1 −4
[ ]
−20
x2 = Bx1 = , q = −10, ϵ ≤ 0.0000
40
[ ]
200
x3 = Bx2 = , q = −10, ϵ ≤ 0.0000
−400
We choose the second eigenvalue λ2 = −10 + k = 1
Note that the exact eigenvalues are 11 and 1.

2. Apply the power method with scaling(3 step), using x0 = [1, 1]T . Give the
Rayleigh
[ quotient
] and error [ bound.
]
7 −3 9 4
(a) (b)
−3 −1 4 3
Sol. Let x̂n+1 = Axn and xn+1 be the scaled version of x̂n+1 .
(a) Apply
[ ]the power method [ again
] to obtain the first eigenvalue [ ]
1 4 −1
x0 = , x̂1 = Ax1 = , q = 0.0000, ϵ ≤ 4.0000, x1 =
1 −4 1
[ ] [ ]
40 −5
x̂2 = Ax1 = , q = 6.0000, ϵ ≤ 4.0000, x2 =
−8 1
[ ] [ ]
−2.7143 −2.7143
x̂3 = Ax2 = , q = 7.8462, ϵ ≤ 1.2308, x3 =
1 1
We choose the first eigenvalue λ1 = 7.8462 and eigenvector x = [−2.7143, 1]T .
Let B = A − kI = A − 8I. Apply the power method again to obtain the second
eigenvalue,
[ ] [ ] [ 1 ]
1 −4
x0 = , x̂1 = Bx0 = , q = −8.0000, ϵ ≤ 4.0000, x1 = 3
1 −12 1
[ ] [ 1 ]
40
x̂2 = Bx1 = , q = −10.0000, ϵ ≤ 0.0000, x2 = 3
120 1
[ ] [ 1 ]
−400
x̂3 = Bx2 = , q = −10.0000, ϵ ≤ 0.0000, x3 = 3
−1200 1
We choose the second eigenvalue λ2 = −10 + k = −2 and eigenvector x = [ 31 , 1]T
Note that the exact eigenvalues are 8 and −2, and their eigenvectors are [−3, 1]T
and [ 31 , 1]T , respectively.

(b) Apply the power method again to obtain the first eigenvalue

Computational Science & Engineering (CSE) T. Jeong

 Homework 9
[ ] [ ] [ ]
1 13 1.8571
x0 = , x̂1 = Ax0 = , q = 10.0000, ϵ ≤ 3.0000 x1 = ,
1 7 1
[ ] [ ]
145 1.9863
x̂2 = Ax1 = , q = 10.9908, ϵ ≤ 0.3028 x2 = ,
73 1
[ ] [ ]
1597 1.9987
x̂3 = Ax2 = , q = 10.9999, ϵ ≤ 0.0275 x3 = ,
799 1
We choose the first eigenvalue λ1 = 10.9999 and eigenvector x = [1.9987, 1]T .
Let B = A − kI = A − 11I. Apply the power method again to obtain the second
eigenvalue,
[ ] [ ] [ 1 ]
1 2 −2
x0 = , x̂1 = Bx0 = , q = −1, ϵ ≤ 3.0000 x1 = ,
1 −4 1
[ ] [ 1 ]
−20 −2
x̂2 = Bx1 = , q = −10, ϵ ≤ 0.0000 x2 =
40 1
[ ] [ 1 ]
200 −2
x̂3 = Bx2 = , q = −10, ϵ ≤ 0.0000 x3 =
−400 1
We choose the second eigenvalue λ2 = −10 + k = 1
Note that the exact eigenvalues are 11 and 1,and their eigenvectors are [2, 1]T and
[− 21 , 1]T , respectively.

§20.9
1. Tridiagonalize using Householder’s
 method. Show the details.
  5 4 1 1
0 1 1 4 5 1 1
(a) A =  1 0 1  (b) A = 
1 1 4 2

1 1 0
1 1 2 4
Sol. (a) Step I.
S 2 = 12 + 12 = 2,
v1 = 0,
√ √ √
√ 2+ 2
v2 = 0.5(1 + 1/ 2) = 2 = 0.9239,
v3 = 2v 1√2 = √ √1 √ = 0.3827
 2 2+ 2  
2

0 1 0√ 0√
v =  0.9239 , P = I − vvT =  0 −1/√2 −1/√ 2 
0.3827 0 −1/ 2 1/ 2
 √ 
1
√ − 2 0
A1 = P AP =  − 2 1 0
0 0 1
(b) Step I.

Computational Science & Engineering (CSE) T. Jeong

 Homework 10

S 2 = 42 + 12 + 12 = 18,
v1 = 0,
√ √
v2 = 0.5(1 + 4/ 18) = 0.9856,
v3 = 2v 1√18 = 0.1196
2
v4 = 2v 1√18 = 0.1196
2   
0 1 0 0 0
 0.9856   
v= , P = I − vv T
=  0 −0.9428 −0.2357 −0.2357 
 0.1196   0 −0.2357 0.9714 −0.0286 
0.1196 0 −0.2357 −0.0286 0.9714
 
5 −4.2426 0 0
 −4.2426 6 −1 −1 
A1 = P AP =  

0 −1 3.5 1.5 
0 −1 1.5 3.5
2 2 2
Step II. S = 1 + 1 = 2,
v1 = 0,
√ v2 = 0, √
v3 = 0.5(1 + 6/ 2) = 0.9239
v4 = 2v−1 √ = 0.3827
2 18
 
1 0 0 0
0 1 0√ 0√ 
P = I − vvT =   0 0 −1/ 2 −1/ 2 

√ √
0 0 −1/ 2 −1/ 2
 
5 −4.2426 √0 0
 −4.2426 2 0
A2 = P A 1 P =  √6 
 0 2 5 0
0 0 0 2
2. Do three QR-steps to find approximations of the eigenvalues of the matrix in
answer to problem 1-(a).
Sol. From problem 1-(a), A can be reduced to tridiagonal form B0 using House-
holder’s method.  √ 
1
√ − 2 0
B0 =  − 2 1 0
0 0 1
Note that A has the eigenvalue  1. 
cos θ2 sin θ2 0 √
To find a matrix C2 , let C2 =  − sin θ2 cos θ2 0  such that − sin θ2 − 2 cos θ2 =
0 0 1
√
0, that is cos θ2 = √13 and sin θ2 = − √23 .

Computational Science & Engineering (CSE) T. Jeong

 Homework 11
 √  √  √ √ 
√1 − √23 0 1 − 2 0 3 − 2√ 2
0
 √3  √  √3 
R0 = C2 B0 =  √2
3
√1
3
0  − 2 1 0= 0 − 3 0
0 0 1 0 0 1 0 0 1
Hence taking Q0 = C2−1 ,  √  √ 
√
√1 √2 0 3 0 − 3
2√ 2
 √3 3
  √
B0 = Q0 R0 =  − 23
√ √10   0 − 3 0 .
3
0 0 1 0 0 1

Computational Science & Engineering (CSE) T. Jeong