ECE 403/503 Solutions to Assignment 1

Problems for this assignment: 1.1, 1.2, and 1.3.

1.1 (2 points) Consider a data matrix

1 1 2 2 
X  
1 2 3 2 
(a) Compute C  X X T  by hand.
(b) Compute the nonzero eigenvalues of C by hand.
(c) Use MATLAB function eig to compute the eigenvalues of C and compare them with the
result from part (b).
Solution
(a)                                            
1 1 
 
1 1 2 2   1 2  10 13
C  XX  
T
   
1 2 3 2   2 3  13 18
 
 2 2
(b) By solving the characteristic equation
  10 13 
det   I  C   det     2  28  11  0
 13   18
we obtain the two nonzero eigenvalues 
28  784  44
1,2 
 27.6015, 0.3985  
2
(c) In MATLAB, using  eig(C) returns with two eigenvalues of C as 0.3985 and 27.6015 which
are in perfect agreement with the result from part (b). This suggests that in case of no access to
software, solving the characteristic equation provides a feasible resolution of the problem,
especially when the matris in question is of small size.
■
1.2 (6 points) Download image building256.mat from the course web site, then in MATLAB
normalize the image to X = building256/255;
(a) Use MATLAB to compute C  X X T .
(b) Use MATLAB function eigs to compute the first five largest eigenvalues of C and the
associated eigenvectors, and denote them as { 1 ,  2 ,  3 ,  4 ,  5 } and  {u1 , u2 , u3 , u4 , u5 } ,
respectively.
(c) Use the results from part (b) to compute
                                                              vi   i1/ 2 X T ui     for i = 1, 2, 3, 4, 5
(d) Use the results from parts (b) and (c) to prepare five matrices Xk defined below

1
 k
X k    i ui viT for k = 1, 2, 3, 4, 5
i 1

|| X k  X ||F
(e) Compute the relative approximation errors ek  for k = 1, 2, 3, 4, 5 and report
|| X ||F
the numerical results {ek, k = 1, …, 5}, where || X ||F  denote the Frobenius norm of X
which can be computed using MATLAB as norm(X,’fro’).
(f) Plot images X1, X3, and X5 together with the original X in a single figure for comparison
using the code below:
figure(1)
subplot(221)
imshow(X)
title(‘original building256’)
subplot(222)
imshow(X1)
title(‘rank-1 approximation’)
subplot(223)
imshow(X3)
title(‘rank-3 approximation’)
subplot(224)
imshow(X5)
title(‘rank-5 approximation’)
(g) Image building256 (hence matrix X) is of size 256  256, hence you need to save
2562  65,536 numbers to keep the image. Alternatively, if you happen to like what image X5
provides, how many numbers do you need to save to keep X5? If you denote that number by n5,
2562
compute and report as the “compression ratio”. Report your numerical result.
n5
Solution
(a) X = building256/255;
C = X*X’;
(b) [U,S] = eigs(C,5);
s = diag(S);
u1 = U(:,1); u2 = U(:,2); u3 = U(:,3); u4 = U(:,4); u5 = U(:,5);
(c) v1 = (X’*u1)/sqrt(s(1));
v2 = (X’*u2)/sqrt(s(2));
v3 = (X’*u3)/sqrt(s(3));
v4 = (X’*u4)/sqrt(s(4));
v5 = (X’*u5)/sqrt(s(5));
(d) X1 = sqrt(s(1))*u1*v1’;
X2 = X1 + sqrt(s(2))*u2*v2’;
X3 = X2 + sqrt(s(3))*u3*v3’;
X4 = X3 + sqrt(s(4))*u4*v4’;

2
 X5 = X4 + sqrt(s(5))*u5*v5’;
(e) nx = norm(X,’fro’);
e1 = norm(X1 – X)/nx;
e2 = norm(X2 – X)/nx;
e3 = norm(X3 – X)/nx;
e4 = norm(X4 – X)/nx;
e5 = norm(X5 – X)/nx;
The approximation errors are summarized in a 5-component vector as
 e1  0.1742 
 e   0.1145 
 2  
 e3    0.0828 
   
 e4   0.0713
 e5   0.0648 
(f)

original building256 rank−1 approximation

rank−3 approximation rank−5 approximation

(g) To construct image X5, we need to keep 5 pairs of weighted (in order to absorb the scalars
 i ) vectors and hence a total of n5 = 5  2  256  2,560 numbers. This achieves a compression
ratio
2562 2562
  25.6
n5 5  2  256
■
1.3 (2 points) The orthogonal projection of a vector v onto another vector u is defined as a vector
who (i) has the same direction as vector u and hence assumes the form  u where u is the unit
vector along u and (ii) has the shortest distance to vector v. Show that

3
  vT u 
                                        the orthogonal projection of v onto u = (v T u ) u   T  u             (S1.1)
u u
where u  u / || u ||2 .
Solution
Recall that inner product has a geometrical interpretation
uT v  || u ||2  || v ||2  cos  (S1.2)
see Appendix A. By definition, the orthogonal projection of a vector v onto another vector u
assumes the form  u where u is the unit vector along with u and  is a scalar given by
                                                                              || v ||2 cos  (S1.3)
Since the unit vector along with vector u can be written as
u
                                                                                   u                                                    (S1.4)
|| u ||2
Using (S1.2) – (S1.4), we obtain
orthogonal projection of vector v onto vector u
=  u
 || v ||2 cos   u
 uT v 
=  || v ||2   u
 || u ||2  || v ||2 
 uT v 
  u
 || u ||2 

=  u T v  u
=  v T u  u
Using (S1.4), the last equality of (S1.1) is derived as follows:
 T u  u  vT u   vT u 
   || u ||  || u ||  || u ||2   uT u  u
v T

u 
u = v  u 
 2  2  2   
■