Scribd
Upload
30 views

EMTH211 - Tutorial 9 - Solutions

The document provides solutions to textbook exercises on least squares regression. It uses MATLAB to find the least squares solution for different data sets by calculating the pseudo-inverse of the matrix. It also shows how to orthogonally diagonalize a matrix by computing its eigenvalues and eigenvectors.

Uploaded by

Nixon Kan
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
30 views

EMTH211 - Tutorial 9 - Solutions

The document provides solutions to textbook exercises on least squares regression. It uses MATLAB to find the least squares solution for different data sets by calculating the pseudo-inverse of the matrix. It also shows how to orthogonally diagonalize a matrix by computing its eigenvalues and eigenvectors.

Uploaded by

Nixon Kan
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

EMTH211 Tutorial 9 Solutions

Textbook exercises
7.3, #7,8,13,14,18,27,44
#7 To find the least square solution to


1 1  
0
1 2 c0 = 1
c1
1 3
5
Using MATLAB ([1, 1; 1, 2; 1, 3]\[0; 1; 5]) we get c0 = 3 and c1 = 2.5. So the line
y = 3 + 2.5x best fits these three points.
The residual error is


0
1 1  
0.5
c
r = 1 1 2 0 = 1
c1
5
1 3
0.5

So krk = 1.5.
#8 To find the least square solution to

1 1  
5
1 2 c0 = 3
c1
1 3
2
Using MATLAB ([1, 1; 1, 2; 1, 3]\[5; 3; 2]) we get c0 = 6 13 and c1 = 1.5. So the line
y = 6 13 1.5x best fits these three points. The residual error is

1
5
1 1  
6
c
r = 3 1 2 0 = 31
c1
1
2
1 3
6
So krk 0.4082.
#13,14 In both cases we are trying to find the least square solution to the system of the form


1 x1  
y1
.. .. c0
..
= .
. .
c1
1 xn
yn
With the help of MATLAB (simply using A\y): we get c0 = 0.2, and c1 = 1.4 for #13,
and c0 = 12.7, and c1 = 2.5 for #14. Notice that we have a positive slope for the first
one and a negative one for the second one, which is exactly, what we would expect,
since the measurements for #13 go up, and the ones for #14 go down.
#18 As in the previous exercise, but now were solving


1 x1 x21
y1
c0
.. ..
.. c = ..
. .
.
. 1
2
c2
1 xn xn
yn
1

Using MATLAB ([ones(5, 1) x x. x]\y), we get c0 = 12.4, c1 = 1.8, and c2 = 4. So


the parabola
y = 4x2 + 1.8x 12.4 = 4(x 1.55)(x 2)
best fits these measurements.
#27 We have seen that in this the solution is given by solving the equation
c = R1 QT b ,
instead of solving the slightly more complicated normal equation. So in this case the
least square solution c is

 
1
 2
 
 
1 1
1 5
3
2 2 1 1 15
3
3 =
c=
=
0 1 3 1 2 2
9 6
3 2
1
#44

1. P is symmetric:
P T = (A(AAT )1 AT )T = (AT )T (AAT )T AT = A(AAT )1 AT ,
since AAT is symmetric and therefore also its inverse is; that is (AAT )T =
(AAT )1
2. P is idempotent:
P P = (A(AAT )1 AT )(A(AAT )1 AT ) = A(AAT )1 (AT A)(AAT )1 AT
= A(AAT )1 AT = P

Supplementary exercise
1. Compute the pseudo-inverse for A, where
(a)

(b)

 
1
A=
2

1 3
A = 1 1
0 2

(c)

1 1
A=
0 1
2. Orthogonally diagonalise

3
0
A=
0
1
2

0
2
0
0

0
0
2
0

1
0

0
3

1.

(a) To compute the pseudo-inverse


AT A = [5]
So
(AT A)1 AT =


1
1 2
5

(b) To compute the pseudo-inverse



1 3
1 1 0
2
2
T
1 1 =
A A=
3 1 2
2 14
0 2


So

 1 3 T




1
1 2 4 1
1 8 16 4
14 2
T
1 T

1 1 =
(A A) A =
=
4
1
28 4 2 2
24 4 4
6 1 1
0 2


(c) To compute the pseudo-inverse




1 0
A A=
1 1
T

So


 

1 1
1 1
=
0 1
1 2


 

2 1 1 0
1 1
(A A) A =
=
1 1
1 1
0 1
T

Notice that this is simply the inverse; which, as weve seen in the lectures is always the case if A is square (and the columns of A are invertible, which was a
requirement to actually compute the pseudo-inverse).
2. First, we compute the eigenvalues:
det(AI) = (3)2 (2)2 (2)2 = (2)2 ((3)2 1) = (2)2 (4)(2)
Hence the eigenvalues are 2, 2, 2, 4. The corresponding eigenvector for 4 is

1
0

E4 = span
0
1
The general solution for (A 2I)x = 0 is

s
r
,
t
s
for arbitrary s, t, r. So


1
0
0
0 1 0

E2 = span
0 , 0 , 1
1
0
0
3

These are already orthogonal! So there is nothing to do, apart from scaling them, to get orthonormal vectors. Let

1
2

0
Q=
0

1
2

then A = QDQT , where

4
0
D=
0
0

1
2

0
0
12
0
2
0
0

0
1
0
0

0
0
,
1
0

0 0
0 0
,
2 0
2

You might also like