Math 2050 Fall 2022

Homework 5

Homework 5 is due on Canvas by Monday XXX at 23:59. The total number of points is 60.

1 SVD of an augmented matrix

For a given n × m matrix A, recall that the largest singular value of A is given by the maximum Rayleigh
quotient
∥Ax∥2
σmax (A) = max .
x̸=0 ∥x∥2
Here, we consider the augmented matrix  
A
à := .
Im

1. (3 points) What is the rank of Ã? How many singular values does it have?
2. (3 points) Show that the largest singular value of Ã, σmax (Ã), is related to that of A by
p
σmax (Ã) = σmax (A)2 + 1.
Hint: consider the corresponding Rayleigh quotient.
3. (3 points) Exhibit a pair of left and right singular vectors, that is, ṽ ∈ Rn+m , ũ ∈ Rm such that
Ãṽ = σmax (Ã)ũ, ÃT ũ = σmax (Ã)ṽ, ∥ũ∥2 = ∥ṽ∥2 = 1.
Hint: start with a pair of singular vectors associated with the largest singular value of A.
4. (3 points) Find singular vectors for the other singular values of à based on those of A. Hint: make
a guess based on the previous result.
Solution:
1. The equation Ãx = (Ax, x) = 0 leads to x = 0; thus, Ã is full rank. Since the number of columns
is less than that of rows, the rank of A is m. There are m singular values, all non-zero.
2. We have
2 ∥Ax∥22 + ∥x∥22 ∥Ax∥22
σmax (Ã) = max = 1 + max = 1 + σ(A)2max .
x̸=0 ∥x∥2 x̸ = 0 ∥x∥2
3. Let u, v be a pair of (unit-norm) and right singular vectors associated with the largest singular value
of A. We have
Av = σmax (A)u, AT u = σmax (A)v, ∥u∥2 = ∥v∥2 = 1.
We observe that v achieves the maximum in the Rayleigh quotient expression above, since
∥Ãv∥2 = ∥Av∥22 + 1 = σmax (A)2 + 1 = σmax (Ã)2 .
Hence v is a right singular vector for à as well. A left singular vector is then
 
Ãv 1 σmax (A)v
u= =p .
∥Ãv∥2 σmax (A)2 + 1 v

1
4. Let ui , vi be right and left singular vectors associated with the singular value σi , i = 1, . . .p
, m, with
the convention that σi = 0 for i > r. Based on the previous result, we guess that σ̃i := σi2 + 1,
i = 1, . . . , m, are the singular values of Ã, with associated singular vectors
 
1 σi vi
ṽi := vi , ũi := p 2 , i = 1, . . . , m.
σi + 1 vi

Indeed, we check that, for i = 1, . . . , m:

Ãṽi = σ̃i ũi , ÃT ũi = σ̃i ṽi , ∥ũi ∥2 = ∥ṽi ∥2 = 1.

2
2 Exponent matrices

A n × m matrix is said to be an exponent matrix if its rows satisfy the “row-sum property”: for any pair
j, k ∈ {1, . . . , n} such that jk ≤ n, the corresponding rows ajk , aj , ak ∈ Rm satisfy ajk = aj + ak .
An example of such a matrix is the n × m matrix An of exponents in the prime number factorizations
of the integers from 1 to n, where m = π(n) is the number of primes less than n, omitting 1. We refer to
this particular matrix as the canonical exponent matrix of order n. For example, when n = 20, we have
m = π(20) = 8; omitting zeroes:

2 3 5 7 11 13 17 19
1
2 1
3 1
4 2
5 1
6 1 1
7 1
8 3
9 2
10 1 1
11 1
12 2 1
13 1
14 1 1
15 1 1
16 4
17 1
18 1 2
19 1
20 2 1

You can check the row-sum property holds: for example,

a18 = a9 + a2 = a6 + a3 = 2a3 + a2 .

1. (4 points) Show that the n-vector of logarithms of integers,

xn := (0, log 2, log 3, . . . , log n)

is in the range of An . Express xn as xn = An pn , where pn ∈ Rπ(n) is a vector that you will

determine.

2. (4 points) Show that the canonical exponent matrix An can be written as

 
Cn
An = P (1)
Iπ(n)

where P is a n × n permutation matrix of order n (that is, a permutation of the columns of the
identity matrix). What is the rank of An ? What is the interpretation of the matrix Cn ? Hint: use the
fact that P is an orthonormal matrix (extra credit if you provide a proof).

3
3. (4 points) Show that the nullspace of Cn contains some unit vectors, which you will specify. Interpret
the result, and provide a bound on the rank of Cn in terms of the number of certain primes in an
interval.

Solution:

1. Every integer i ≤ n can be written as

π(n)
Y A (i,j)
i= pj n , (2)
j=1

where pj is the j-th prime number (omitting 1), and aij the corresponding exponent in i. Taking
logarithms results in xn = An pn , with pn ∈ Rπ(n) the vector of logarithms of the prime numbers
less than n.

2. We can re-order rows so as to put the prime numbers last. If P is the corresponding permutation
matrix, we will obtain the desired expression, where Cn is the matrix of exponents of composite
numbers. Here we used the fact that pre-multiplying a matrix by P reorder its rows according to the
permutation. Since P is orthonormal, the rank of An and (CnT Iπ(n) )T coincide. The rank of the latter
matrix is π(n), hence the rank of An is π(n). The matrix Cn corresponds to the composite numbers.
Let us prove that any n × n permutation matrix P is orthonormal. We can express such a matrix as
n
X
P = eρ(i) eTi
i=1

where ρ denotes the mapping from {1, . . . , n} to the permuted indices. Then:
n X
X n n
X n
X
T
P P = eρ(j) (eTi ej )eTρ(j) = eρ(i) eTρ(i) = ei eTi = In .
i=1 j=1 i=1 i=1

In the above, we exploited the fact that, as i spans {1, . . . , n}, ρ(i) spans the same set, hitting each
value exactly once.

3. The nullspace of Cn contains the unit vectors in Rπ(n) corresponding to prime numbers that do not
appear in the factorization of any composite number less or equal to n. In fact, the columns of Cn
corresponding to those numbers are entirely zero. The rank of Cn is thus less than the number of
such primes. In turn, this number is the number of primes in the interval [n/2, n], since the list of
composite numbers that include a prime p as a factor always should include 2p ≤ n.

4
3 SVD of exponent matrices

Consider the canonical exponent matrix of exercise 2 , specifically the matrix Cn in part 2.
1. (3 points) For a generic value of n, how do the singular values of Cn relate to those of An ?
2. (3 points) For the case n = 100, plot the singular values of Cn in decreasing order. Comment.
3. (3 points) Project the data set in A100 (with each row a data point, representing an integer) on a
2D plot. Show primes in a different color. Comment, the best you can. (This corresponds to an
“encoding”, or “embedding”, of numbers represented via rows of the exponent matrix, into lower-
dimensional vectors.)
4. (3 points) What is the relative squared error in Frobenius norm sense, expressed in %, that corre-
sponds to the plot you have made? Express it in terms of singular values of An , without proof.
5. (4 points) Explain how to obtain reconstructed numbers based on the approximation made, and
plot them against the original numbers. (This step is akin to decoding, when the previous one was
encoding.) Hint: remember the formula linking An , the log-primes and the log-integers.
Solution:
1. As seen in the previous part, the matrix An is related to [CnT , Iπ(n) ]T by eq. (1). Since the permutation
matrix P is orthonormal, and based on part 1, we know that the singular values of An are related to
those of Cn via p
σi (An ) = σi (Cn )2 + 1, i = 1, . . . , m.

2. We observe that the matrix Cn is not full rank and appears to have exactly as many zero singular
values as the number of primes that do not appear in the factorization of any integer less than
n = 100.
3. Start from the SVD of An (n = 100):
π(n)
X
An = σi ui viT ,
i=1

where each integer is a data point, represented as a row, and σ1 ≥ σ2 ≥ . . . ≥ σπ(n) . We then make
a rank-two approximation:
X 2
(2)
An ≈ An := σi ui viT .
i=1
Each row is approximated as
2
X
aTj = eTj An ≈ eTj A(2)
n = σi ui (j)viT , j = 1, . . . , n.
i=1

We can plot the coordinates of each row in the basis v1 , v2 , namely the points (σ1 u1 (j), σ2 u2 (j)),
j = 1, . . . , n. The result is in Fig 1.
We observe a definite pattern via different clusters of integers. Primes seem to group in three specific
large clusters (top right), with the left-most (resp. bottom) cluster containing 2 (resp. 3) as the only
prime. In the clusters corresponding to 2, 3, the other numbers in the cluster are all multiples of the
same prime. All the other primes are located in a single cluster, in the top right.

5
Figure 1: Plot of integers numbers from 1 to 100 via the low-rank approximation of the canonical exponent
matrix. Primes are shown in red.

4. The relative squared error (in Frobenius norm) is

∥A − A(2) ∥2F σ32 + . . . + σπ(n)

2
E= = ≈ 22%.
∥A∥2F σ12 + . . . + σπ(n)
2

5. To reconstruct approximate numbers, we start from the formula

vn := exp xn = exp(An log(pn ))

where vn := (1, 2, . . . , n) and pn contains all the primes ≤ n in increasing order. We then replace
An by its rank-two approximation:

vn ≈ exp(A(2)
n log(pn )).

Fig. 2 shows that the approximation is not very good. One reason is the fact that we apply the
exponential function.

6
Figure 2: Plot reconstructed numbers vs the original ones, based on a rank-two approximation.