COL726: Numerical Algorithms

Sem 2, 2024-25
Assignment 1

Deadline - 9 February, 2025

Please carefully read the instructions given below before attempting the questions:
1. Originality and Acknowledgment: All solutions must be written in your own words. If you
collaborate with others, acknowledge their names at the start of the respective question. Similarly,
if you refer to online resources or other materials, cite them clearly. However, the solutions must
still be written independently.
2. Consequences for Non-Compliance: Failure to follow these guidelines will result in a zero score
for the assignment, and further disciplinary actions may be taken if necessary.
3. Use single precision for the numerical problems if nothing is specified. All the programming ques-
tions are to be done in MATLAB.
4. You can submit your solutions in either handwritten format (ensure clear readability) or LaTeX-
typed format.
5. If you find any problem is incorrect/missing information before a week of the deadline, kindly let us
know. If it is in the last week before the deadline, then make your own assumptions. The deadline
for assignment will not be extended.
6. Submit a single PDF file named <Entry number>.pdf (Example - 2020CS10111.pdf ), containing
solutions to all the problems as well as the code to programming questions.

1. Let A be an m × n matrix with singular values σ1 ≥ σ2 ≥ σ3 ≥ · · ·. Let q1 and q2 be two

orthonormal vectors in Rn . Show that:

(a) For any unit vector x ∈ Rn , ⟨q1 , x⟩2 + ⟨q2 , x⟩2 ≤ 1. [1 Marks]
(b) ∥Aq1 ∥22 + ∥Aq2 ∥22 ≤ σ12 + σ22 . [2 Marks]

2. (a) Using the triangle inequality, derive a lower bound for ∥x + y∥ in terms of ∥x∥ and ∥y∥. [1
Marks]
(b) Suppose A is a square matrix, and you are given ∥A∥ for an arbitrary induced norm ∥ · ∥.
Based only on the value of ∥A∥, find a real number δ such that the matrix I + ϵA is invertible
for all ϵ < δ. (Hint: Consider the action of I + ϵA on an arbitrary vector x.) [3 Marks]
 
1 2.03
3. Given the fixed matrix A = , consider the problem f : R2 → R2 of finding the vector
1.03 2.08
x such that Ax = b for an input vector b.
(a) Compute f (b) for the input vector b = [1.02 1.05]T , using any method you have studied
before. Then compute f (b + ∆b) for the slightly perturbed input b + ∆b = [1.02 1.0502]T .
[1 Marks]

1
 (b) Using only these input-output pairs, what can you say about the condition number of f in the
∞-norm? Give as quantitative an answer as possible. [2 Marks]
4. In class, we defined the notion of a backward stable algorithm as follows: an algorithm for computing
a function f (x) computes f (x̂). Then the algorithm is backward stable if

|x̂ − x|
= O(ϵmach ).
|x|

Sometimes, it is more convenient to work with a slightly more general definition. We say that the
algorithm for computing f (x) is stable if it computes a quantity y which is close to f (x̂) for some
vector x̂ in the following sense:
|y − f (x̂)|
= O(ϵmach ),
|f (x̂)|
where
|x̂ − x|
= O(ϵmach ).
|x|

(i) Show that if an algorithm for computing f is stable, then the relative error in the output is
roughly O(kf (x) · ϵmach ), where kf (x) is the condition number of f at x. Hence, we can use
either the notion of stability or backward stability. [2 Marks]
(ii) Consider the function f (x) = 1+x. Show that if we compute this function using this expression,
then the algorithm may not be backward stable, but is always stable. [2 Marks]
(iii) Suppose we compute f (x) = sin(x) for x ∈ [0, 2π] using an algorithm that outputs a value in
the range sin(x)(1 ± ϵmach ). Show that such an algorithm may not be backward stable, but is
always stable. [2 Marks]

5. Let ∥ · ∥α and ∥ · ∥β be given norms on Cn . Let |||·|||α and |||·|||β denote their respective induced
matrix norms on Mn (set of n × n complex matrices), that is,

∥Ax∥α ∥Ax∥β
|||A|||α = max and |||A|||β = max .
x̸=0 ∥x∥α x̸=0 ∥x∥β

Define
∥x∥α ∥x∥β
Rαβ = max and Rβα = max .
x̸=0 ∥x∥β x̸=0 ∥x∥α

Prove that, the following are equivalent: [6 Marks]

(a) Rαβ Rβα = 1.
(b) There is some c > 0 such that ∥x∥α = c∥x∥β for all x ∈ Cn .
(c) |||·|||α = |||·|||β
6. Let A be an m × n matrix. Suppose you are given a singular value decomposition (SVD) of A.
Write down an SVD of the following matrices:
   
0 A I A
[2 Marks], [5 Marks]
AT 0 AT 0

Here, 0 and I denote the all-zero and the identity matrices of suitable dimensions, respectively.
7. Show that for any rank k matrix B, ∥A − B∥2F ≥ σk+1
2
+ · · · + σn2 , where ∥A∥F is the Frobenius
norm of an m × n matrix A. [5 Marks]

8. Consider the function

f (x) = (ex − 1)/x (1)
It can be shown that
lim f (x) = 1 (2)
x→0

2
 (a) Check this result empirically by writing a program to compute f (x) for x = 10−k , k = 1, . . . , 15.
Do your results agree with theoretical expectations? Explain why.
(b) Perform the experiment in the above part, this time with the equivalent formulation

f (x) = (ex − 1)/log(ex ), (3)

evaluated as indicated, with no simplification. Does this work any better? Explain why or
why not.
You will observe that after a certain value, there is an abrupt change in the value of f (x) in both
of the above problems. Mark that point on the x axis for both plots and explain what is happening
after that point. Use the x axis scale to be −log10 (x). Give an error analysis that explains the
reason for the results obtained. [6 Marks]
9. Write a program to generate the first n terms in a sequence given by the difference equation:

xk+1 = 111 − (1130 − 3000/xk−1 )/xk , (4)

with starting values x1 = 11/2 and x2 = 61/11. Submit a plot for xth k entry in the sequence, where
k ∈ 1, . . . , n, with n = 10. The exact solution is a monotonically increasing sequence converging to
6. Does your program show the correct solution? Explain the results. [5 Marks]
10. (a) Write a program to compute an approximate value for the derivative of a function using the
finite difference formula:
′ f (x + h) − f (x)
f (x) ≈ (5)
h
Test your program for the function tan(x) for x = 1. Determine the error by comparing with
the square of built-in method sec(x). Plot the magnitude of error as a function of h. Take
10k h = 10−k , k = 0, 1, . . . , 16. Use logscale for the value of h and the magnitude of error.

h=
 
Report the minimum value for the error magnitude if it exists. [2 Marks]
(b) Repeat the above exercise using central difference approximation: [2 Marks]

′ f (x + h) − f (x − h)
f (x) ≈ (6)
2h

(c) Explain which of the above two formulations result in more accurate results. Explain the
behavior of error as h decreases. [1 Marks]