PART B Please complete in BLOCK CAPITALS

Student Number Name Signature

This information will not be visible to the marker.

PART A Please complete clearly

Exam Number
as shown on your university card
Numerical Linear Algebra
MATH10098
Friday 9 December 2022
13:00–15:00

Number of questions: 7
Total number of marks: 100

IMPORTANT PLEASE READ CAREFULLY

Before the examination

1. Put your university card face up on the desk.

2. Complete PART A and PART B above. By completing PART B you are accepting the
University Regulations on student conduct in an examination (see back cover).
3. Complete the attendance slip and leave it on the desk.
4. In this examination, candidates are allowed to have three sheets (six sides) of A4 paper with
whatever notes they desire written or printed on one or both sides of the paper. Magnifying
devices to enlarge the contents of the sheets for viewing are not permitted. No further notes,
printed matter or books are allowed.
5. A scientific calculator is permitted in this examination. It must not be a graphic calculator. It
must not be able to communicate with any other device.

During the examination

1. Write clearly, in ink, in the space provided after each question. If you need more space then
please use the extra pages at the end of the examination script or ask an invigilator for
additional paper.
2. Attempt all questions.
3. If you have rough work to do, simply include it within your overall answer – put brackets at the
start and end of it if you want to highlight that it is rough work.

At the end of the examination

1. This examination script must not be removed from the examination venue.
2. There are extra pages for working at the end of this examination script. If used, you should
clearly label your working with the question to which it relates.
3. Additional paper and graph paper, if used, should be attached to the back of this examination
script. Write your examination number on the top of each additional sheet.
[Do not write on this page]
MATH10098 Do not write above this line December 2022

1. Throughout this question, suppose

- A ∈ Rn×n is invertible,
- U ∈ Rn×n is upper-triangular and invertible,
- D ∈ Rn×n is diagonal and invertible, and
- y ∈ Rn .
(a) Below is pseudo-code for Algorithm MV, which computes the matrix-vector product Ay.

Algorithm MV: Matrix-vector multiplication

Input: A ∈ Rn×n , y ∈ Rn
Output: z = Ay ∈ Rn
1: for i = 1, . . . , n
2: zi = 0
3: for j = 1, . . . , n
4: zi = zi + aij yj
5: end for
6: end for

Adapt the pseudo-code above to an efficient algorithm for computing the matrix-vector prod-
uct Uy, i.e. an algorithm that avoids unnecessary arithmetic operations.
You don’t need to write out the new pseudo-code in full, but you should clearly state what
changes you are making and why. [3 marks]

[Space for working continues. . . ]

Page 1 of 28 [Please turn over]

MATH10098 Do not write above this line December 2022

If you have used additional space for working then please tick here:

Page 2 of 28
MATH10098 Do not write above this line December 2022

(b) Show that the computational cost of the new algorithm from part (a) to compute Uy is
m(m+1)
C(n) = n2 + n. You may use without proof the formula mi=1 i =
P
2 . [5 marks]

If you have used additional space for working then please tick here:

Page 3 of 28 [Please turn over]

MATH10098 Do not write above this line December 2022

(c) Rank the following tasks in terms of computational cost, from lowest to highest:
(i) Computing the matrix-vector product Ay.
(ii) Solving the linear system Ax = y using a direct method.
(iii) Computing the matrix-vector product Uy.
(iv) Solving the linear system Ux = y.
(v) Computing the matrix-vector product Dy.
(vi) Solving the linear system Dx = y.
Explain how you determined your ranking, including stating explicitly the algorithms or
operations that you are using to compute the quantities. You may assume that n is sufficiently
large. [10 marks]

[Space for working continues. . . ]

Page 4 of 28
MATH10098 Do not write above this line December 2022

If you have used additional space for working then please tick here:

Page 5 of 28 [Please turn over]

MATH10098 Do not write above this line December 2022

2. Let D ∈ Rn×n be an invertible, diagonal matrix with diagonal entries d1 , . . . , dn , and let p ∈ [1, ∞].
Denote by ∥D∥p the induced norm of D.
(a) Show that ∥D∥p ≤ maxi∈{1,...,n} |di |. [4 marks]

If you have used additional space for working then please tick here:

Page 6 of 28
MATH10098 Do not write above this line December 2022

(b) Show that ∥D∥p ≥ maxi∈{1,...,n} |di |. [4 marks]

If you have used additional space for working then please tick here:

Page 7 of 28 [Please turn over]

MATH10098 Do not write above this line December 2022

(c) Parts (a) and (b) show that ∥D∥p = maxi∈{1,...,n} |di |. Using this expression, or otherwise,
show that
maxi∈{1,...,n} |di |
κp (D) = ,
mini∈{1,...,n} |di |
where κp (D) is the condition number of D in the p-norm. [4 marks]

If you have used additional space for working then please tick here:

Page 8 of 28
MATH10098 Do not write above this line December 2022

3. Let A ∈ Rn×n and b ∈ Rn be given, and suppose A is invertible.

(a) How would you efficiently solve the system A2 x = b using a direct method? Hint: write the
system as AAx = b. [7 marks]

[Space for working continues. . . ]

Page 9 of 28 [Please turn over]

MATH10098 Do not write above this line December 2022

If you have used additional space for working then please tick here:

Page 10 of 28
MATH10098 Do not write above this line December 2022

(b) By generalising your approach in part (a), or otherwise, explain how to efficiently solve the
system Am x = b, for m ∈ N. What is the computational cost of your approach?
Hint: You do not want a term involving mn3 in the cost, which is prohibitively large for large
m. [7 marks]

[Space for working continues. . . ]

Page 11 of 28 [Please turn over]

MATH10098 Do not write above this line December 2022

If you have used additional space for working then please tick here:

Page 12 of 28
MATH10098 Do not write above this line December 2022

4. Let A ∈ Rn×n and b ∈ Rn be given, and suppose A is invertible. The Kaczmarz method is an
iterative method for solving the system Ax = b, as described in the pseudo-code below.
Algorithm KI: Kaczmarz Iterative Method
Input: A ∈ Rn×n with rows a1T , . . . , anT , b ∈ Rn , x0 ∈ Rn , εr > 0
Output: xk ∈ Rn , with Axk ≈ b
1: k = 0
2: while ∥Axk − b∥p > εr do
3: k =k+1
4: i = k mod n
b −aT x
5: Compute xk = xk−1 + i ∥ai ∥2k−1 ai
i 2
6: end while
Note that the index i cycles over the rows of A: i = 1, 2, . . . , n, 1, 2, . . . , n, 1, 2, . . . .
(a) What is the computational cost of the Kaczmarz method? Explicitly state all algorithms or
operations you use.
You may assume that it stops after kmax iterations, that kmax ≥ n (so that every row of A
gets used at least once), and that k mod n = k − n ∗ ⌊ nk ⌋ can be computed using 3 arithmetic
operations. You do not need to include the cost of line 2. [10 marks]

[Space for working continues. . . ]

Page 13 of 28 [Please turn over]

MATH10098 Do not write above this line December 2022

If you have used additional space for working then please tick here:

Page 14 of 28
MATH10098 Do not write above this line December 2022
" # " #
2 1 3
(b) Consider the example A = and b = . Compute the approximate solutions x1 and
1 2 3
" #
0
x2 produced by the Kaczmarz method, given the starting guess x0 = . [8 marks]
0

[Space for working continues. . . ]

Page 15 of 28 [Please turn over]

MATH10098 Do not write above this line December 2022

If you have used additional space for working then please tick here:

Page 16 of 28
MATH10098 Do not write above this line December 2022
" #
1
(c) The true solution to the system in part (b) is x = . Compute the errors ∥e1 ∥∞ and
1
∥e2 ∥∞ . [4 marks]

If you have used additional space for working then please tick here:

Page 17 of 28 [Please turn over]

MATH10098 Do not write above this line December 2022

5. Let A ∈ Rm×n , with m ≥ n and rank(A) = n, and let A = UΣVT be a singular value de-
composition. Define the pseudo-inverse B† = (BT B)−1 BT , for any B ∈ Rm×n , with m ≥ n and
rank(B) = n.
(a) Show that A† = VΣ† UT . [6 marks]

If you have used additional space for working then please tick here:

Page 18 of 28
MATH10098 Do not write above this line December 2022

(b) Show that Σ† is a diagonal matrix in Rn×m with diagonal entries σ1−1 , . . . , σn−1 , where
σ1 , . . . , σn are the singular values of A. [4 marks]

If you have used additional space for working then please tick here:

Page 19 of 28 [Please turn over]

MATH10098 Do not write above this line December 2022

(c) Let b ∈ Rm . Suppose we compute the minimiser x of ∥Ax−b∥2 as x = A† b, and we compute

A† either as (AT A)−1 AT (approach 1) or as VΣ† UT (approach 2). Which approach do you
think will produce a larger error in the computed solution x
b , and why? [4 marks]

If you have used additional space for working then please tick here:

Page 20 of 28
MATH10098 Do not write above this line December 2022

6. Consider a symmetric matrix A ∈ R3×3 with eigenvalues λ1 = 2, λ2 = 1 and λ3 = 0.25, and

corresponding eigenvectors x1 , x2 and x3 . Suppose you apply Algorithm PI (Power Iteration) and
Algorithm SII (Shifted Inverse Iteration) with shift s = 0 to A, and you observe the following
errors at iteration k = 1:

k |λ1 − λ(k) | ∥x1 − z(k) ∥2 |λ3 − λ̃(k) | ∥x3 − z̃(k) ∥2

1 0.1 0.1 0.1 0.1
2 ? ? ? ?

Here, λ(k) , z(k) and λ̃(k) , z̃(k) denote the outputs of Algorithm PI and Algorithm SII, respectively,
and you may assume that the approximate eigenvectors do not change sign (i.e. σk = 1 in
Theorems 6.5 and 6.6).
Fill in the second row of the table, i.e. predict what errors you will observe at iteration k = 2.
Explain how you found your answer. [8 marks]

[Space for working continues. . . ]

Page 21 of 28 [Please turn over]

MATH10098 Do not write above this line December 2022

If you have used additional space for working then please tick here:

Page 22 of 28
MATH10098 Do not write above this line December 2022

7. Let A ∈ Rn×n be symmetric. Suppose the eigenvalues of A are labelled in decreasing order of
absolute value,
|λ1 | ≥ |λ2 | ≥ · · · ≥ |λn |,
and choose the corresponding eigenvectors x1 , . . . , xn to be an orthonormal basis of Rn .
Suppose we already know λ1 and x1 , and we would now like to compute λ2 and x2 .
Explain why Algorithm PI (Power Iteration), with starting guess z(0) = (A − λ1 I)u for some
u ∈ Rn \ {0}, converges to λ2 and x2 as k → ∞. What assumptions do you need on the
eigenvalues of A and u to guarantee convergence?
You do not need to provide a full convergence proof, although you may do so if you wish. You
should provide a discussion of the behaviour of Ak z(0) as k → ∞.
Algorithm PI: Power Iteration
Input: A ∈ Rn×n symmetric, z(0) ∈ Rn \ {0}, ε > 0
Output: z(k) ∈ Rn , λ(k) ∈ R, with z(k) ≈ xmax and λ(k) ≈ λmax
1: k = 0
2. λ(0) = rA (z(0) )
3. while ∥Az(k) − λ(k) z(k) ∥2 > ε
4. k =k+1
5. z(k) = Az(k−1)
(k)
6. z(k) = ∥zz(k) ∥
2
7. λ(k) = rA (z(k) )
8. end while
[12 marks]

[Space for working continues. . . ]

Page 23 of 28 [Please turn over]

MATH10098 Do not write above this line December 2022

If you have used additional space for working then please tick here:

Page 24 of 28 [End of questions]

MATH10098 Do not write above this line December 2022

This is an extra page for working. Please indicate clearly the question number to which your working
relates, otherwise your working may not be marked.

Page 25 of 28
MATH10098 Do not write above this line December 2022

This is an extra page for working. Please indicate clearly the question number to which your working
relates, otherwise your working may not be marked.

Page 26 of 28
MATH10098 Do not write above this line December 2022

This is an extra page for working. Please indicate clearly the question number to which your working
relates, otherwise your working may not be marked.

Page 27 of 28
MATH10098 Do not write above this line December 2022

This is an extra page for working. Please indicate clearly the question number to which your working
relates, otherwise your working may not be marked.

Page 28 of 28
[Do not write on this page]
 Exam Hall Regulations
The following is a copy of a Notice which is displayed in Edinburgh University Examination Halls for the
information of students and staff.

The University of Edinburgh Exam Hall Regulations

1. An examination attendance sheet is laid on the desk for each student to complete upon arrival. These
are collected by an invigilator after thirty minutes have elapsed from the start of the examination.
Students are not normally allowed to enter the examination hall more than thirty minutes after the start
of the examination.
2. Students arriving after the start of the examination are required to complete a “Late arrival form” which
requires them to sign a statement that they understand that they are not entitled to any additional time.
Students are not allowed to leave the examination hall less than thirty minutes after the
commencement of the examination or within the last fifteen minutes of the examination.
3. Books, papers, briefcases and cases must be left at the back or sides of the examination room. It is an
offence against University discipline for a student to have in their possession in the examination any
material relevant to the work being examined unless this has been authorised by the examiners.
4. Students must take their seats within the block of desks allocated to them and must not communicate
with other students either by word or sign, nor let their papers be seen by any other student.
5. Students are prohibited from deliberately doing anything that might distract other students. Students
wishing to attract the attention of an invigilator shall do so without causing a disturbance. Any student
who causes a disturbance in an examination room may be required to leave the room, and shall be
reported to the University Secretary.
6. Personal handbags must be placed on the floor at the student’s feet; they should be opened only in full
view of an invigilator.
7. An announcement will be made to students that they may start the examination. Students must stop
writing immediately when the end of the examination is announced.
8. Answers should be written in the script book provided. Rough work, if any, should be completed within
the script book and subsequently crossed out. Script books must be left in the examination hall.
9. During an examination, students will be permitted to use only such dictionaries, other reference books,
computers, calculators and other electronic technology as have been issued or specifically authorised
by the examiners. Such authorisation must be confirmed by the Registry.
10. The use of mobile telephones is not permitted and mobile telephones must be switched off during an
examination.
11. It is an offence against University discipline for any student knowingly
• to make use of unfair means in any University examination
• to assist a student to make use of such unfair means
• to do anything prejudicial to the good conduct of the examination, or
• to impersonate another student or allow another student to impersonate them
12. Students will be required to display their University Card on the desk throughout all written degree
examinations and certain other examinations. If a card is not produced, the student will be required to
make alternative arrangements to allow their identity to be verified before the examination is marked.
13. Smoking and eating are not allowed inside the examination hall.
14. If an invigilator suspects a student of cheating, they shall impound any prohibited material and shall
inform the Examinations Office as soon as possible.
15. Cheating is an extremely serious offence, and any student found by the Discipline Committee to have
cheated or attempted to cheat in an examination may be deemed to have failed that examination or
the entire diet of examinations, or be subject to such penalty as the Discipline Committee considers
appropriate.