Scribd
Upload
90 views

A Review of Linear Algebra: Mohammad Emtiyaz Khan CS, Ubc

This document provides a summary of key concepts in linear algebra, including: 1) It defines basic terms like vectors, matrices, and matrix multiplication. It also covers inverses and solving systems of linear equations. 2) It discusses LU factorization and how it can be used to solve systems of linear equations. It also introduces Cholesky decomposition. 3) It covers concepts like vector spaces, subspaces, linear independence, basis and dimension, and the four fundamental subspaces. 4) It summarizes orthogonality, inner products, and how orthogonal subspaces and complements relate to row and nullspaces.

Uploaded by

Şamil Okan
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
90 views

A Review of Linear Algebra: Mohammad Emtiyaz Khan CS, Ubc

This document provides a summary of key concepts in linear algebra, including: 1) It defines basic terms like vectors, matrices, and matrix multiplication. It also covers inverses and solving systems of linear equations. 2) It discusses LU factorization and how it can be used to solve systems of linear equations. It also introduces Cholesky decomposition. 3) It covers concepts like vector spaces, subspaces, linear independence, basis and dimension, and the four fundamental subspaces. 4) It summarizes orthogonality, inner products, and how orthogonal subspaces and complements relate to row and nullspaces.

Uploaded by

Şamil Okan
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 13

A Review of Linear Algebra

Mohammad Emtiyaz Khan


CS,UBC

A Review of Linear Algebra p.1/13

Basics
Column vector x Rn , Row vector xT , Matrix A Rmn .
Matrix Multiplication, (m n)(n k) m k , AB 6= BA.
Transpose AT , (AB)T = B T AT , Symmetric A = AT
Inverse A1 , doesnt exist always, (AB)1 = B 1 A1 .
xT x is a scalar, xxT is a matrix.
Ax = b, three ways of expressing:
Pn
j=1 aij xj = bj , j

rTj x = bj , j , where rj is j th row.

x1 a1 + x2 a2 + . . . + xn an = b (Linear Combination,
l.c.)

System of equations : Non-singular (unique solution),


singular (no solution, infinite solution).

A Review of Linear Algebra p.2/13

LU factorization

5
2
1 1
x1

4 6 0 x2 = 2 Ax = b
9
x3
2 7 2


2 1
1
5
x1


=
0
8
2
12
x

2
U x = EF Gb
0 0
1
2
x3
|
{z
}
U

1
1
1

1
1
2 1
, L = G1 F 1 E 1

1
1
1
1
1 1
{z
}|
{z
}|
{z
}
|
E

A Review of Linear Algebra p.3/13

LU factorization
(First non-singular case) If no row exchanges are
required, then A = LU (unique).
Solve Lc = b, then U x = c
Another form A = LDU .
(Second non-singular case) There exist a permutation
matrix P that reorders the rows, so that P A = LU .
(Singular Case) No such P exist.
(Cholesky Decomposition) If A is symmetric, and
A = LU can be found without any row exchanges, then
A = LLT (also called square root of a matrix). (proof).
Positive Definite matrix always have a Cholesky
decompostion.
A Review of Linear Algebra p.4/13

Vector Space, Subspace and Matrix


(Real Vector Space) A set of vectors" with rules for
vector addition and multiplication by real numbers. E.g.
R1 , R2 , . . . , R , Hilbert Space.
(8 conditions) Includes an identity vector and zero
vector, closed under addition and multiplication etc. etc.
(Subspace) Subset of a vector space, closed under
addition and multiplication (should contain zero).
Subspace
(Outline the concept)

a matrix
by
spanned"
b1
0
1



x 1 5 + x 2 4 = b2
b3
4
2

A Review of Linear Algebra p.5/13

Linear Independence, Basis, Dimension


(Linear Independence, l.i.) If x1 a1 + x2 a2 + . . . + xn an
only happens when x1 = x2 = . . . = 0, {ak } are called
linearly independent.
A set of n vectors in Rm are not l.i. if n > m (proof).
(Span) If every vector v in V can be expressed as a l.c.
of {ak }, then {ak } are said to span V .
(Basis) {ak } are called basis of V if they are l.i. and
span V (Too many and unique)
(Dimension) Number of vectors in any basis is called
dimension (and is same for all basis).

A Review of Linear Algebra p.6/13

Four Fundamental Spaces


Fundamental Theorem of Linear Algebra I
1. R(A) = Column Space of A; l.c. of columns; dim r.
2. N (A) = Nullspace of A; All x : Ax = 0; dim n r.
3. R(AT ) = Row space of A; l.c. of rows; dim r.
4. N (AT ) = Left nullspace of A; All y : AT y = 0; dim m r.
(Rank) r is called rank of the matrix. Inverse exist iff rank is
as large as possible. Question: Rank of uvT

A Review of Linear Algebra p.7/13

Orthogonality
(Norm) ||x||2 = xT x = x21 + . . . + x2n
(Inner Product) xT y = x1 y1 + . . . + xn yn
(Orthogonal) xT y = 0
Orthogonal l.i. (proof).
(Orthonormal basis) Orthogonal vectors with norm =1
(Orthogonal Subspaces) V W if v w, v V, w W
(Orthogonal Complement) The space of all vectors
orthogonal to V denoted as V .
The row space is orthogonal to the nullspace (in Rn )
and the column space is orthogonal to the left nullspace
(in Rm ).(proof).
A Review of Linear Algebra p.8/13

Finally...
Fundamental Theorem of Linear Algebra II
1. R(AT ) = N (A)
2. R(A) = N (AT )
Any vector can be expressed as
(1)

x = x1 b1 + . . . + xr br + xr+1 br+1 + . . . + xn bn
|
{z
} |
{z
}
xr

(2)

xn

= xr + xn

Every matrix transforms its row space to its column space


(Comments about pseudo-inverse and invertibility)

A Review of Linear Algebra p.9/13

Gram-Schmidt Orthogonalization
(Projection) of b on a is

aT b
a T a a,

for unit vector (aT b)a

(Schwartz Inequality) |aT b| ||a||||b||


(Orthogonal Matrix)Q = [q1 . . . qn ], QT Q = I . (proof).
(Length preservation) ||Qx|| = ||x|| (proof).
Given vectors {ak }, construct orthogonal vectors{qk }
1. q1 = a1 /||a1 ||

2. for each j , aj = aj (qT1 aj )q1 . . . (qTj1 aj )qj1

3. qj = aj /||aj ||
QR Decomposition (Example)
A Review of Linear Algebra p.10/13

Eigenvalues and Eigenvectors


(Invariance)Ax = x.
(Characteristics Equation) (A I)x = 0 (Nullspace)
1 + . . . + n = a11 + . . . + ann .
1 . . . n = det(A).

(A = SS 1 ) Suppose there exist n linear independent


eigenvectors for A. If S is the matrix whose columns are
those independent vectors, then A = SS 1 where
= diag(1 , . . . , n ).
Diagonalizability is concerned with eigenvectors, and
invertibility is concerned with eigenvalues.
(Real symmetric matrix) Eigenvectors are orthogonal.
So A = QQT . (Spectral Theorem)
A Review of Linear Algebra p.11/13

Singular Value Decomposition


Any matrix can be factorized as A = U V T . Insightful? Finish.

A Review of Linear Algebra p.12/13

Finish
Thanks to Maria (Marisol Flores Gorrido) for helping me
with this tutorial.

A Review of Linear Algebra p.13/13

You might also like