Scribd
Upload
136 views

Midterm F15 Sols

This document contains solutions to problems on matrix factorization and operations. For problem 1, the solutions show that the inverse of a circulant Toeplitz matrix is also circulant Toeplitz, and provide fast methods to compute the inverse and product of two such matrices using the discrete Fourier transform. Problem 2 proves that the Kronecker product of two orthogonal matrices is also orthogonal. Problem 3 discusses the first step of the Householder QR factorization algorithm to compute an orthogonal matrix Q1 that zeros out part of the first column of the input matrix. Problem 4 compares the complexity of directly solving (I+AB)x=b by computing and factorizing I+AB, versus using the

Uploaded by

Dylan Ler
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
136 views

Midterm F15 Sols

This document contains solutions to problems on matrix factorization and operations. For problem 1, the solutions show that the inverse of a circulant Toeplitz matrix is also circulant Toeplitz, and provide fast methods to compute the inverse and product of two such matrices using the discrete Fourier transform. Problem 2 proves that the Kronecker product of two orthogonal matrices is also orthogonal. Problem 3 discusses the first step of the Householder QR factorization algorithm to compute an orthogonal matrix Q1 that zeros out part of the first column of the input matrix. Problem 4 compares the complexity of directly solving (I+AB)x=b by computing and factorizing I+AB, versus using the

Uploaded by

Dylan Ler
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 5

L.

Vandenberghe
EE133A

11/2/2015

Midterm solutions
Problem 1. In the homework you derived the following factorization of a circulant Toeplitz matrix
T (a) with the n-vector a as its first column:

a1
a2
a3
..
.

an
a1
a2
..
.

T (a) =

an1 an2
an an1

an1
an
a1
..
..
.
.
an3
an2

a3
a4
a5
..
.
a1
a2

a2
a3
a4
..
.

= W H diag(W a)W.
n

an
a1

Here W is the n n discrete Fourier Transform matrix and diag(W a) is the diagonal matrix with
the vector W a (the discrete Fourier transform of a) on its diagonal.
1. Suppose T (a) is nonsingular. Show that its inverse T (a)1 is a circulant Toeplitz matrix.
Give a fast method for computing the vector b that satisfies T (b) = T (a)1 .
2. Let a and b be two n-vectors. Show that the product T (a)T (b) is a circulant Toeplitz matrix.
Give a fast method for computing the vector c that satisfies T (c) = T (a)T (b).

Solution.
1. The inverse is given by
T (a)1 =

1 H
W diag(W a)1 W.
n

because W 1 = (1/n)W H . This can be written as


T (a)1 =

1 H
W diag(W b)W.
n

if we define b = (1/n)W H diag(W a)1 1.


The vector b can be computed in order n log n flops by calculating the DFT W a of a, inverting
it componentwise to get diag(W a)1 1, and then taking the inverse DFT of the result.
2. Using W H W = W W H = nI to simplify the product gives



1 H
1 H
W diag(W a)W
W diag(W b)W
T (a)T (b) =
n
n
1 H
=
W diag(W a) diag(W b)W
n
1 H
=
W diag ((W a) (W b)) W.
n

This can be written as


T (a)T (b) =

1 H
W diag(W c)W
n

with c = (1/n)W H ((W a) (W b)). The vector c can be computed in order n log n flops by
taking the DFTs of a and b, multiplying them componentwise, and taking the inverse DFT.

Problem 2. The Kronecker product of two n n matrices A and B is the n2 n2 matrix

A11 B A12 B A1n B


A21 B A22 B A2n B

AB =
.
..
..
..
..

.
.
.
.
An1 B An2 B Ann B
For example,

3 4
0
0
5 6
0
0
.
=

6 8 3 4
10 12
5 6

1
0
2 1

3 4
5 6

Suppose A and B are orthogonal. Is A B orthogonal? Explain your answer.


Solution. If A and B are orthogonal, then A B is orthogonal. To show this, we need to prove
that (A B)T (A B) = I.
First, we note that the transpose of the Kronecker product is the Kronecker product of the
transposes: (A B)T = AT B T . Therefore
(A B)T (A B)
= (AT B T )(A B)

A11 B T A21 B T An1 B T


A11 B A12 B A1n B
A12 B T A22 B T An2 B T A21 B A22 B A2n B

..
..
..
..
..
..
.
.
.
.

.
.
.
.
.
.
.
.
T
T
T
A1n B
A2n B
Ann B
An1 B An2 B Ann B

2
2
(A11 + + An1 )I
(A11 A12 + + An1 An2 )I (A11 A1n + + An1 Ann )I
(A12 A11 + + An2 An1 )I
(A212 + + A2n2 )I
(A12 A1n + + An2 Ann )I

=
..
..
..
..

.
.
.
.
(A1n A11 + + Ann An1 )I (A1n A12 + + Ann An2 )I
(A21n + + A2nn )I

I 0 0
0 I 0

= . . .
. . ...
.. ..

0 0

because AT A = I and B T B = I.
2

Problem 3. Let A be an m n matrix with linearly independent columns. The Householder


algorithm for the QR factorization of A computes an orthogonal m m matrix Q such that


R
T
Q A=
0
where R is upper triangular with nonzero diagonal elements. The matrix Q is computed as a
product Q = Q1 Q2 Qn1 of orthogonal matrices. In this problem we discuss the first step, the
calculation of Q1 . This matrix has the property that

R11
0

QT1 A = .
..
.. .
.
.
.
.
0
The symbols denote elements that may or may not be zero.
Let a = (A11 , A21 , . . . , Am1 ) be the first column of A. Define an m-vector


1
1
a + se1
v=p
1 + |A11 |/kak kak
where s = 1 if A11 0 and s = 1 if A11 < 0. The vector e1 is the first unit vector (1, 0, . . . , 0).

1. Show that v has norm 2.


2. Define Q1 = I vv T . Show that Q1 is orthogonal.
3. Show that QT1 a = R11 e1 , where R11 = skak.
4. Give the complexity (dominant term in the flop count for large m, n) of computing the
matrix-matrix product
QT1 A = (I vv T )A.
Solution.
1. The square of kvk is
T

v v =
=
=
=


T 

1
1
1
a + se1
a + se1
1 + |A11 |/kak kak
kak
 T

1
a a
A11
2
+ 2s
+s
1 + |A11 |/kak kak2
kak
2 + 2|A11 |/kak
1 + |A11 |/kak
2.

2. We verify that QT1 Q1 = I. Since v T v = 2,


QT1 Q1 = (I vv T )(I vv T ) = I vv T vvv T + (v T v)vv T = I.
3

3. First note that


p
kak + |A11 |
aT a/kak + seT1 a
=p
= kak 1 + |A11 |/kak.
vT a = p
1 + |A11 |/kak
1 + |A11 |/kak
Therefore
QT1 a = a (v T a)v = a kak(
4. We compute QT1 A as

1
a + se1 ) = skake1
kak

QT1 a = A v(v T A).

This takes 2mn flops for the product y = v T A, another mn for the outer product vy T , and
mn for the addition to A. The total is 4mn.

Problem 4. Let A be an n m matrix and B an m n matrix. We compare the complexity of


two methods for solving
(I + AB)x = b.
We assume the matrix I + AB is nonsingular.
1. In the first method we compute the matrix C = I + AB and then solve Cx = b using the
standard method (LU factorization). Give the complexity of this method.
2. Suppose the matrix I + BA is nonsingular. Show that
(I + AB)1 = I A(I + BA)1 B.
3. This suggests a second method for solving the equation: compute the solution via the formula

x = I A(I + BA)1 B b.
Describe an efficient method for evaluating this formula and give the complexity. Which of
the two methods is faster when m  n? Explain your answer.
Solution.
1. Computing C costs 2n2 m flops. Solving the equation costs (2/3)n3 flops.
2. We check that (I + AB)(I A(I + BA)1 B) = I:
(I + AB)(I A(I + BA)1 B = I + AB A(I + BA)1 B ABA(I + BA)1 B
= I + AB A(I + BA)(I + BA)1 B
= I + AB AB
= I.
3. The complexity is 2nm2 + (2/3)m3 flops plus lower order terms.
4

Compute v = Bb (2mn flops).


Compute C = I + BA (2nm2 flops).
Solve Cu = v using the LU factorization of C ((2/3)m3 flops).
Compute x = b Au (2mn flops).
Method 2 is faster when m  n.

You might also like