Scribd
Upload
26 views8 pages

Fund Theorem Linear Algebra

Uploaded by

Muhasina Ashraf
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
26 views8 pages

Fund Theorem Linear Algebra

Uploaded by

Muhasina Ashraf
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 8

Fundamental Theorem of Linear Algebra

• Orthogonal Vectors
• Orthogonal and Orthonormal Set
• Orthogonal Complement of a Subspace W
• Column Space, Row Space and Null Space of a Matrix A
• The Fundamental Theorem of Linear Algebra
Orthogonality

Definition 1 (Orthogonal Vectors)


Two vectors ~u, ~
v are said to be orthogonal provided their dot product is zero:
u ·~
~ v = 0.

If both vectors are nonzero (not required in the definition), then the angle θ between the
two vectors is determined by
u ·~
~ v
cos θ = = 0,
k~
ukk~
vk
which implies θ = 90◦ . In short, orthogonal vectors form a right angle.
Orthogonal and Orthonormal Set

Definition 2 (Orthogonal Set of Vectors)


A given set of nonzero vectors ~
u1 , . . . , ~
uk that satisfies the orthogonality condition
ui · ~
~ uj = 0, i 6= j,
is called an orthogonal set.

Definition 3 (Orthonormal Set of Vectors)


A given set of unit vectors ~
u1, . . . , ~
uk that satisfies the orthogonality condition is called
an orthonormal set.
Orthogonal Complement W ⊥ of a Subspace W
Definition. Let W be a subspace of an inner product space V , inner product h~ vi. The
u, ~
⊥
orthogonal complement of W , denoted W , is the set of all vectors ~ v in V such that
h~ vi = 0 for all ~
u, ~ u in W . In set notation:
W ⊥ = {~
v : h~ vi = 0 for all ~
u, ~ u in W }
Example. If V = R3 and W = span{~ u2}, then W ⊥ is the span of the calcu-
u1 , ~
u2. The equation dim(W ) + dim(W ⊥) = 3 holds
u1 × ~
lus/physics cross product ~
(in general dim(W ) + dim(W ⊥ ) = dim(V )).
Theorem. If W is the span of the columns ~ un of m × n matrix A (the column
u1 , . . . , ~
space of A), then

u = ~0}.
W ⊥ = nullspace(AT ) = span{Strang’s Special Solutions for AT ~
Proof. Given W = span{~ un}, then
u1 , . . . , ~
W ⊥ = {~ v·w
v : ~ ~ ∈ W}
~ = 0, all w
= {~ uj · ~
v : ~ v = 0, j = 1, . . . , n}
v = ~0}.
v : AT ~
= {~

u = ~0. Therefore,
Strang’s Special solutions are a basis for the homogeneous problem AT ~
W ⊥ = nullspace(AT ) = span{Strang’s Special Solutions for AT ~ u = ~0}.
Column Space, Row Space and Null Space of a Matrix A
The column space, row space and null space of an m × n matrix A are sets in Rn or Rm ,
defined to be the span of a certain set of vectors. The span theorem implies that each of
these three sets are subspaces.
Definition. The Column Space of a matrix A is the span of the columns of A, a subspace
of Rm . The Pivot Theorem implies that

colspace(A) = span{pivot columns of A}.

Definition. The Row Space of a matrix A is the span of the rows of A, a subspace of
Rn. The definition implies two possible bases for this subspace, just one selected in an
application:

rowspace(A) = span{Nonzero rows of rref (A)} = span{pivot columns of AT }.

Definition. The Null Space of a matrix A is the set of all solutions ~


x to the homogeneous
problem A~ x = ~0, a subspace of Rn. Because solution ~ x of A~ x = ~0 is a linear
combination of Strang’s special solutions, then

x = ~0}.
nullspace(A) = span{Strang’s Special Solutions for A~
The Row space is orthogonal to the Null Space
Theorem. Each row vector ~r in matrix A satisfies ~r · ~
x = 0, where ~
x is a solution of the
homogeneous equation A~x = ~0. Therefore
rowspace(A) ⊥ nullspace(A).

The theorem is remembered from this diagram:


       
· · · 0 row 1 · ~
x 0
 · · · ~
x=0 is equivalent to  row 2 · ~
x=0
· · · 0 row 3 · ~
x 0

which says that the rows of A are orthogonal to solutions ~ x = ~0.


x of A~
Computing the Orthogonal Complement of a subspace W
Theorem. In case W is the subspace of R3 spanned by two independent vectors ~ u1 , ~
u2 ,
then the orthogonal complement of W is the line through the origin generated by the cross
u1 × ~
product vector ~ u2 :
W ⊥ = span{~ u2}⊥ = span{~
u1, ~ u1 × ~
u2 }

Theorem. In case W is a subspace of Rm spanned by all column vectors ~


u1, . . . , ~
un of
an m × n matrix A, then the orthogonal complement of W is the subspace

W⊥ = span{~ un}⊥
u1, . . . , ~
= {~ y ·~
y : ~ ui = 0 for all i = 1, . . . , n}
= nullspace(AT )
= span{Strang’s Special Solutions for AT ~u = ~0}

Method. To compute a basis for W ⊥ , find Strang’s special Solutions for the homogeneous
u = ~0. The basis size is k = number of free variables in AT ~
problem AT ~ u = ~0.
Applications may add an additional step to replace this basis by the Gram-Schmidt orthog-
onal basis ~ yk . Then W ⊥ = span{~
y1, . . . , ~ yk }.
y1, . . . , ~
Fundamental Theorem of Linear Algebra
The four fundamental subspaces are rowspace(A), colspace(A),
Definition.
nullspace(A) and nullspace(AT ).
The Fundamental Theorem of Linear Algebra has two parts:
(1) Dimension of the Four Fundamental Subspaces.
Assume matrix A is m × n with r pivots. Then
dim(rowspace(A)) = r , dim(colspace(A)) = r ,
dim(nullspace(A)) = n − r , dim(nullspace(AT )) = m − r
(2) Orthogonality of the Four Fundamental Subspaces.
rowspace(A) ⊥ nullspace(A)
colspace(A) ⊥ nullspace(AT )

Gilbert Strang’s textbook Linear Algebra has a cover illustration for the fundamental theo-
rem of linear algebra. The original article is The Fundamental Theorem of Linear Algebra,
http://www.jstor.org/stable/2324660. The free 1993 jstor PDF is available
via the Marriott library. Requires UofU 2-factor login.

You might also like