Scribd
Upload
52 views8 pages

Math 110: Linear Algebra Homework #8: David Zywina

This document contains a student's homework for a linear algebra course. It includes 9 problems solving various linear algebra concepts. Specifically, it shows work for: 1) Proving a property of permutation matrices and matrix multiplication 2) Computing an LU decomposition of a 3x3 matrix 3) Using the LU decomposition to solve two systems of equations 4) Finding the inverse of a lower triangular matrix

Uploaded by

Cody Sage
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
52 views8 pages

Math 110: Linear Algebra Homework #8: David Zywina

This document contains a student's homework for a linear algebra course. It includes 9 problems solving various linear algebra concepts. Specifically, it shows work for: 1) Proving a property of permutation matrices and matrix multiplication 2) Computing an LU decomposition of a 3x3 matrix 3) Using the LU decomposition to solve two systems of equations 4) Finding the inverse of a lower triangular matrix

Uploaded by

Cody Sage
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 8

MATH 110: LINEAR ALGEBRA

HOMEWORK #8
DAVID ZYWINA
First set of problems
Problem 6. If P is a permutation matrix, and Y = XP, show that Y has the same columns
as X but in a permuted order.
For each j {1, , n}, let (j) be the unique element of {1, , n} such that P
(j),j
= 1
(i.e. the unique 1 in the jth column of X occurs in the (j)th row). Since P is a permutation
matrix, the numbers (1), , (n) are a permutation of the numbers 1, , n.
Fix a j {1, , n}. For each i,
(XP)
i,j
=
n

k=1
X
i,k
P
k,j
= X
i,(j)
P
(j),j
= X
i,(j)
.
This shows that the jth column of XP is the (j)th column of X. Therefore Y has the
same columns as X but in a permuted order.
Problem 7. Show that if P
1
and P
2
are permutation matrices, then so is P
1
P
2
.
The previous problem (with X = P
1
and P = P
2
) shows that P
1
P
2
has the same columns
as P
1
but in a permuted order. Since P
1
has the same columns as the identity matrix I
(possibly permuted), this shows that the columns of P
1
P
2
are just a permutation of those of
I. Thus P
1
P
2
is a permutation matrix.
Problem 8. Show that if P is a permutation matrix, so is P
t
, and P
t
= P
1
.
A permutation matrix, by denition, is an n n matrix with exactly one 1 in each row,
one 1 in each column, and the other entries equal to 0. Since P
t
is the matrix where the
rows are swapped with columns, and columns are swapped with rows it is immediate that
P
t
must also be a permutation matrix.
We can write P =
_
c
1
c
2
c
n

, where {c
1
, , c
n
} = {e
1
, , e
n
} (e
i
is the ith
column of I). The transpose of P can be expressed as
P
t
=
_

_
c
t
1
c
t
2
.
.
.
c
t
n
_

_
.
Date: October 27th.
1
2 DAVID ZYWINA
For 1 i, j n,
(P
t
P)
i,j
= c
t
i
c
j
=
i,j
(the last equality follows from the observation that for 1 i, j n, e
t
i
e
j
=
i,j
). So P
t
P = I
and hence P is invertible with inverse P
t
.
Problem 9. Use LU decomposition to compute an LU decomposition of
A =
_
_
0 2 4
0 1 2
3 2 1
_
_
and use it to describe the complete solution set of Ax =
_
_
8
4
4
_
_
and Ax =
_
_
0
1
0
_
_
.
We need to have a non-zero element in the top left corner of our matrix to start, so we
multiply A by the permutation matrix P =
_
_
0 0 1
0 1 0
1 0 0
_
_
which interchanges the rst and
third row; we get
PA =
_
_
3 2 1
0 1 2
0 2 4
_
_
.
The rst column of PA is already done, so we now work on the second column. Subtracting
2 times the second row from the third row, gives us the matrix:
U

=
_
_
3 2 1
0 1 2
0 0 0
_
_
.
We let
L

=
_
_
1 0 0
0 1 0
0 2 1
_
_
(where the 2 is there because we multiplied the 2nd row by 2 before subtracting it from the
3rd row).
You can now check that we indeed have PA = L

as desired. So multiplying both sides


by P
1
(= P
t
= P), we have
A = PL

=
_
_
0 0 1
0 1 0
1 0 0
_
_
_
_
1 0 0
0 1 0
0 2 1
_
_
_
_
3 2 1
0 1 2
0 0 0
_
_
.
You might argue (rightly) that this is not the LU decomposition dened in class. Since A
has rank 2, L should be a 32 matrix and U should be a 23 matrix. This can be achieved
by simply removing the last column of L

and the last row of U

(This might seem slightly


tricky but it can always be done in this situation. Convince yourself that when multiplying
MATH 110: HOMEWORK #8 3
the matrices the removed column and row contribute nothing.).
A =
_
_
0 0 1
0 1 0
1 0 0
_
_
_
_
1 0
0 1
0 2
_
_
_
3 2 1
0 1 2
_
= PLU.
Now that we have computed the LU decomposition we can solve Ax = PLUx =
_
_
8
4
4
_
_
.
We rst solve the equation (where y = Ux)
Ly = P
1
_
_
8
4
4
_
_
= P
_
_
8
4
4
_
_
=
_
_
4
4
8
_
_
.
This is the same as the equations:
y
1
= 4
y
2
= 4
2y
2
= 8.
These equations are consistent and have solution y =
_
4
4
_
. It remains to solve the
equation Ux = y =
_
4
4
_
. This is the same as the equations:
3x
1
+ 2x
2
+ x
3
= 4
x
2
+ 2x
3
= 4.
The second equation shows that x
2
= 4 2x
3
. Substituting into the rst equation we get
3x
1
= 4 2(4 2x
3
) x
3
= 12 + 3x
3
, thus x
1
= 4 + x
3
. Therefore the solutions to
Ax =
_
_
8
4
4
_
_
are
_
_
_
_
_
4
4
0
_
_
+ t
_
_
1
2
1
_
_
: t R
_
_
_
.
We nally solve Ax = PLUx =
_
_
0
1
0
_
_
. We rst solve the equation (where y = Ux)
Ly = P
1
_
_
0
1
0
_
_
= P
_
_
0
1
0
_
_
=
_
_
0
1
0
_
_
.
4 DAVID ZYWINA
This is the same as the equations:
y
1
= 0
y
2
= 1
2y
2
= 0.
However, these equations are not consistent (they imply that y
2
= 0 and 1). Thus the
original equation Ax =
_
_
0
1
0
_
_
has no solutions.
Second set of problems
Problem 1. Use substitution to solve Lx = b where
L =
_
_
1 0 0
2 1 0
4 3 1
_
_
b =
_
_
3
4
0
_
_
.
Writing x = [x
1
, x
2
, x
3
]
t
the equation Lx = b becomes:
x
1
= 3
x
2
+ 2x
1
= 4
x
3
+ 3x
2
+ 4x
1
= 0.
Solving these equations is easy. We have immediately that x
1
= 3. Substituting this into
the second equation and solving for x
2
gives us, x
2
= 10. Substituting the values of x
1
and
x
2
into the third equation gives us, x
3
= 18. Therefore the solution is
x =
_
_
3
10
18
_
_
.
Problem 2. Use substitution to compute L
1
, where L is given above.
We can write L
1
= [x y z], where x, y and z are column vectors. We then have
[e
1
e
2
e
3
] = I = LL
1
= L[x y z] = [Lx Ly Lz].
Therefore we need only solve the equations Lx = e
1
, Ly = e
2
and Lz = e
3
.
We now solve the rst equation, Lx = e
1
, which can be rewritten as:
x
1
= 1
x
2
+ 2x
1
= 0
x
3
+ 3x
2
+ 4x
1
= 0.
We have immediately that x
1
= 1. Substituting this into the second equation and solving for
x
2
gives us, x
2
= 2. Substituting the values of x
1
and x
2
into the third equation gives us,
MATH 110: HOMEWORK #8 5
x
3
= 2. Therefore x =
_
_
1
2
2
_
_
. Similarily, we also compute y =
_
_
0
1
3
_
_
and z =
_
_
0
0
1
_
_
.
Putting everything together we get:
L
1
=
_
_
1 0 0
2 1 0
2 3 1
_
_
.
Problem 3. In lecture we showed that the number of arithmetic operations needed to com-
pute an LU decomposition of an n n matrix with rank n was
2
3
n
3
+ lower order terms.
(3a) How many more arithmetic operations does it take, given the LU decomposition of
A, to compute A
1
? The method to use is for each column e
j
of the identity matrix, solve
Ax = e
j
for x = column j of A
1
. Use substitution with the L and U factors to solve this
equation. Recall that the P
L
and P
R
factors only involve reordering, no arithmetic opera-
tions. Your answer should be of the form c
1
n
c
2
+ lower order terms, where c
1
and c
2
are
constants you need to determine. Add
2
3
n
3
to your answer to determine the total number of
arithmetic operations to compute the inverse of an invertible n n matrix.
To compute A
1
we need to nd column vectors c
1
, , c
n
such that
A
_
c
1
c
2
c
n

=
_
e
1
e
2
e
n

= I.
So computing A
1
=
_
c
1
c
2
c
n

is equivalent to solving the equations


Ac
1
= e
1
, , Ac
n
= e
n
.
The next Lemma analyzes the cost of solving Ax = b for a general b. We can multiply
this cost by n to get the cost of solving the n linear systems Ac
i
= e
i
for i = 1 to n. Then
we show that by exploiting all the zeros in e
i
, we can actually solve Ac
i
= e
i
more cheaply.
Lemma 1. Suppose we have an LU-decomposition, A = P
L
LUP
R
, where A is an n n
invertible matrix.
Then it takes 2n
2
n arithmetic operations to solve an equation
Ax = b.
Proof. The equation is the same as LU(P
R
x) = P
t
L
b. Since multiplying by a permuation
matrix takes no operations (it involves only swapping entries), we may assume our equation
is of the form
LUx = b.
6 DAVID ZYWINA
We rst solve the equation Ly = b for y = [y
1
, , y
n
]
t
. Since L is a lower triangular
matrix with 1s on the diagonals, the equation can be rewritten as:
y
1
= b
1
y
2
+ L
2,1
y
1
= b
2
y
3
+ L
3,2
y
2
+ L
3,1
y
1
= b
3
y
4
+ L
4,3
y
3
+ L
4,2
y
2
+ L
4,1
y
1
= b
4
.
.
.
y
n
+ L
n,n1
y
n1
+ + L
n,2
y
2
+ L
n,1
y
1
= b
n
The rst equation requires 0 operations to solve for y
1
.
The second equation requires 2 operations to solve for y
2
(1 mult. and 1 subtractions).
The third equation requires 4 operations to solve for y
3
(2 mult. and 2 subtractions).
The fourth equation requires 6 operations to solve for y
4
(3 mult. and 3 subtractions).
.
.
.
The nal equation requires 2(n 1) operations to solve for y
n
(n 1 mult. and n 1 sub-
tractions).
Thus the number of arithmetic operations require to solve the equation Ly = b is
0 + 2 + 4 + 6 + + 2(n 1) = 2(1 + 2 + 3 + + (n 1)) = 2
(n 1)n
2
= n
2
n.
Now that we have solved for y, it remains to solve for x in the equation Ux = y. A similar
process shows that solving Ux = y takes n
2
operations. (The proof is the same except that U
does not necessarily have 1s on the diagonal, so n extra divisions will be needed in general).
Putting everything together, it will take (n
2
n) +n
2
= 2n
2
n arithmetic operations to
solve Ax = b.
Using the approach, the cost of inverting A is the cost of LU decomposition (
2
3
n
3
+ lower
order terms) plus the cost of solving Ac
1
= e
1
through Ac
n
= e
n
(n (2n
2
n) = 2n
3
+ lower
order terms), or altogether
8
3
n
3
+ lower order terms.
However, we can do better by taking advantage of the zeros in e
i
. Since the rst i 1
entries of e
i
are zero, it is easy to see that the rst i 1 entries of y = L
1
e
i
are zero (this is
the same as saying that L
1
is also lower triangular). Therefore, we do not need to compute
them. Thus when solving Ly = e
i
we can start with y
i
= 1, then solve L
i+1,i
y
i
+y
i+1
= 0 for
y
i+1
= L
i+1,i
, then solve L
i+2,i
y
i
+ L
i+2,i+1
y
i+1
+ y
i+2
= 0 for y
i+2
, and so on. This is the
same as solving a triangular system of dimension ni+1, for a cost of (ni+1)
2
+(ni+1)
by the same analysis as above. Summing this from i = 1 to n yields the cost of solving all
the Ly = e
i
:
1
.
n

i=1
[(ni+1)
2
+(ni+1)] =
n

i=1
[i
2
+i] =
n(n + 1)(2n + 1)
6
+
n(n + 1)
2
=
n
3
3
+ lower order terms
1
Recall: 1 + 2 + +n =
n(n+1)
2
and 1
2
+2
2
+ +n
2
=
n(n+1)(2n+1)
6
MATH 110: HOMEWORK #8 7
The cost of the subsequent solves with U does not change (n
3
) for a total inversion cost of
2
3
n
3
+
1
3
n
3
+ n
3
= 2n
3
(plus lower order terms).
(3b) Now suppose A is mn with rank r (the general case). How many arithmetic opera-
tions are needed to compute the LU decomposition?
Let C(m, n, r) be the number of arithmetic operations required to compute the LU de-
composition of an mn matrix with rank r.
The following argument will be done using the inductive algorithm done in lecture. First
of all, multiplying by permutation matrices requires no arithmetic operations (this just swaps
rows and columns); so we can ignore the contributions from the permutation matrices since
they do not aect the value of C(m, n, r).
Let A be an mn matrix of rank r. After a possible permuation, we may assume that
A =
_
A
1,1
A
1,2
A
2,1
A
2,2
_
,
where A
1,1
is a non-zero scalar. We then dene X = A
2,1
/A
1,1
and S = A
2,2
XA
1,2
. (For
future reference: A
1,2
is 1 (n 1), A
2,1
is (m1) 1, and A
2,2
is (m1) (n 1))
It takes m1 divisions to calculate X.
It takes (m 1)(n 1) arithmetic operations (all multiplications) to compute XA
1,2
, then
(m1)(n 1) subtractions to compute S.
Note: So far we have done (m1) + 2(m1)(n 1) arithmetic operations.
We then have
A =
_
1 0
X I
m1
__
A
1,1
A
1,2
0 S
_
.
The matrix S is (m 1) (n 1). Since A has rank r and A
1,1
= 0 we see that S has
rank r 1. Thus it takes C(m 1, n 1, r 1) arithmetic operations to compute an LU
decomposition
S = P
LS
L
S
U
S
P
RS
.
In the lecture notes, it is then derived that
A =
_
1 0
0 P
LS
__
1 0
P
t
LS
X L
S
__
A
1,1
A
1,2
P
t
RS
0 U
S
__
1 0
0 P
RS
_
.
This is the desired LU decomposition of A. Note that all of the multiplications in these
matrices are by permutation matrices and, as explained earlier, take no arithmetic operations
to perform.
Thus we have proven the recursive equation,
C(m, n, r) = (m1) + 2(m1)(n 1) + C(m1, n 1, r 1).
The base case for our recursion is C(m, n, 0) = 0 (the zero matrix is the only mn matrix
with rank 0).
8 DAVID ZYWINA
By repeated using the recursive relation until we get to our base case, we get the explicit
formula:
C(m, n, r) = (m1) + (m2) + ... + (mr)
+ 2(m1)(n 1) + 2(m2)(n 2) + ... + 2(mr)(n r)
We now write this in a more explicit form
C(m, n, r) =
r

i=1
(mi) + 2
r

i=1
(mi)(n i)
= rm
r

i=1
i + 2rmn 2(m + n)
r

i=1
i + 2
r

i=1
i
2
= rm
r(r + 1)
2
+ 2rmn 2(m + n)
r(r + 1)
2
+ 2
r(r + 1)(2r + 1)
6
To get a better idea on the growth of this value we collect all the higher order terms (the
terms of degree 3).
C(m, n, r) = 2rmn (m + n)r
2
+
2
3
r
3
+ lower order terms .
For the special case where m = r = n, we get
C(n) := C(n, n, n) =
2
3
n
3
+ lower order terms
as in lecture.

You might also like