Linear Algebra 2

Lecture #12

Gram-Schmidt orthogonalization. Orthogonal

complements

Irena Penev

February 23, 2023

Definition
For a vector space V over R or C, equipped with a scalar product
⟨·, ·⟩, we say that vectors x and y are orthogonal, and we write
x ⊥ y, provided that ⟨x, y⟩ = 0.
Definition
For a vector space V over R or C, equipped with a scalar product
⟨·, ·⟩, we say that vectors x and y are orthogonal, and we write
x ⊥ y, provided that ⟨x, y⟩ = 0.

As we saw in Lecture 11:

the orthogonality relation is symmetric, i.e. x ⊥ y iff y ⊥ x;
0 is orthogonal to every vector in V .
Definition
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. For a non-zero vector u ∈ V and any vector v ∈ V ,
the orthogonal projection of v onto u is the vector
⟨v,u⟩
proju (v) := ⟨u,u⟩ u.

Note: proju (v) is a scalar multiple of u.

Definition
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. For a non-zero vector u ∈ V and any vector v ∈ V ,
the orthogonal projection of v onto u is the vector
⟨v,u⟩
proju (v) := ⟨u,u⟩ u.

Note: proju (v) is a scalar multiple of u.

Proposition 1.1
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let u be a non-zero vector in V , let v be any vector
in V , and set z := v − proju (v). Then z ⊥ u.
v

z = v−proju(v)

proju(v) u
Proposition 1.1
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let u be a non-zero vector in V , let v be any vector
in V , and set z := v − proju (v). Then z ⊥ u.

Proof. We have that

⟨z, u⟩ = ⟨v − proju (v), u⟩

⟨v,u⟩
= ⟨v − ⟨u,u⟩ u, u⟩

⟨v,u⟩
= ⟨v, u⟩ − ⟨u,u⟩ ⟨u, u⟩

= ⟨v, u⟩ − ⟨v, u⟩

= 0,

and consequently, z ⊥ u. Q.E.D.

Definition
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩ and the norm || · || induced by ⟨·, ·⟩.
An orthogonal set of vectors in V is a collection of pairwise
orthogonal vectors in V .
An orthonormal set of vectors is an orthogonal set of vectors
that satisfies the additional property that all vectors in this set
are of length 1 (i.e. every vector v in the set satisfies ||v|| = 1).
An orthogonal basis (resp. orthonormal basis) for V is an
orthogonal (resp. orthonormal) set in V that is also a basis of
V.
Theorem 2.1
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let {u1 , . . . , un } be an orthogonal basis of V . Then
for all v ∈ V , we have that
n n
P ⟨v,ui ⟩ P
v = ⟨ui ,ui ⟩ ui = projui (v).
i=1 i=1
Theorem 2.1
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let {u1 , . . . , un } be an orthogonal basis of V . Then
for all v ∈ V , we have that
n n
P ⟨v,ui ⟩ P
v = ⟨ui ,ui ⟩ ui = projui (v).
i=1 i=1

Corollary 2.2
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩ and the norm || · || induced by ⟨·, ·⟩. Let {u1 , . . . , un }
be an orthonormal basis of V . Then for all v ∈ V , we have that
n
⟨v, ui ⟩ ui
P
v =
i=1

Coefficients ⟨v, ui ⟩ from Corollary 2.2 are called the Fourier

coefficients.
Proof of Theorem 2.1.
Proof of Theorem 2.1. The second equality follows immediately
from the definition of projection. Let us prove the first equality.
Since v ∈ V and {u1 , . . . , un } is a basis of V , there exist scalars
α1 , . . . , αn such that
n
P
v = αi ui .
i=1
Proof of Theorem 2.1. The second equality follows immediately
from the definition of projection. Let us prove the first equality.
Since v ∈ V and {u1 , . . . , un } is a basis of V , there exist scalars
α1 , . . . , αn such that
n
P
v = αi ui .
i=1

Now, fix any j ∈ {1, . . . , n}. We then have that

n
P
⟨v, uj ⟩ = ⟨ αi ui , uj ⟩
i=1

n
P
= αi ⟨ui , uj ⟩
i=1

(∗)
= αj ⟨uj , uj ⟩,

where (*) follows from the fact that u1 , . . . , un are pairwise

orthogonal.
Proof of Theorem 2.1 (continued). Reminder: ⟨v, uj ⟩ = αj ⟨uj , uj ⟩.
Proof of Theorem 2.1 (continued). Reminder: ⟨v, uj ⟩ = αj ⟨uj , uj ⟩.
Since uj ̸= 0 (because {u1 , . . . , un } is a basis of V ), we see that
⟨uj , uj ⟩ =
̸ 0, and we deduce that
⟨v,uj ⟩
αj = ⟨uj ,uj ⟩ .
Proof of Theorem 2.1 (continued). Reminder: ⟨v, uj ⟩ = αj ⟨uj , uj ⟩.
Since uj ̸= 0 (because {u1 , . . . , un } is a basis of V ), we see that
⟨uj , uj ⟩ =
̸ 0, and we deduce that
⟨v,uj ⟩
αj = ⟨uj ,uj ⟩ .

Since j ∈ {1, . . . , n} was chosen arbitrarily, we now deduce that

n n
P P ⟨v,ui ⟩
v = αi ui = ⟨ui ,ui ⟩ ui .
i=1 i=1

Q.E.D.
Theorem 2.1
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let {u1 , . . . , un } be an orthogonal basis of V . Then
for all v ∈ V , we have that
n n
P ⟨v,ui ⟩ P
v = ⟨ui ,ui ⟩ ui = projui (v).
i=1 i=1
Theorem 2.1
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let {u1 , . . . , un } be an orthogonal basis of V . Then
for all v ∈ V , we have that
n n
P ⟨v,ui ⟩ P
v = ⟨ui ,ui ⟩ ui = projui (v).
i=1 i=1

Corollary 2.2
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩ and the norm || · || induced by ⟨·, ·⟩. Let {u1 , . . . , un }
be an orthonormal basis of V . Then for all v ∈ V , we have that
n
⟨v, ui ⟩ ui
P
v =
i=1

Coefficients ⟨v, ui ⟩ from Corollary 2.2 are called the Fourier

coefficients.
Corollary 2.2
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩ and the norm || · || induced by ⟨·, ·⟩. Let {u1 , . . . , un }
be an orthonormal basis of V . Then for all v ∈ V , we have that
n
⟨v, ui ⟩ ui
P
v =
i=1

Proof.
Corollary 2.2
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩ and the norm || · || induced by ⟨·, ·⟩. Let {u1 , . . . , un }
be an orthonormal basis of V . Then for all v ∈ V , we have that
n
⟨v, ui ⟩ ui
P
v =
i=1

Proof. Since {u1 , . . . , un } be an orthonormal basis of V , we know

that ||u1 || = · · · = ||un || = 1, and consequently (by the
construction of || · ||), we have that ⟨u1 , u1 ⟩ = · · · = ⟨un , un ⟩ = 1.
The result now follows immediately from Theorem 2.1. Q.E.D.
Our next goal is to describe the “Gram-Schmidt
orthogonalization process,” which gives a recipe for
transforming an arbitrary basis of a vector space over R or C,
equipped with a scalar product and the norm induced by it,
into an orthogonal (and even orthonormal) basis.
Our next goal is to describe the “Gram-Schmidt
orthogonalization process,” which gives a recipe for
transforming an arbitrary basis of a vector space over R or C,
equipped with a scalar product and the norm induced by it,
into an orthogonal (and even orthonormal) basis.
But first, we need a couple of technical propositions.
Proposition 2.3
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let {u1 , . . . , uk } be an orthogonal set of non-zero
vectors in V . Then {u1 , . . . , uk } is linearly independent.
Proof.
Proposition 2.3
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let {u1 , . . . , uk } be an orthogonal set of non-zero
vectors in V . Then {u1 , . . . , uk } is linearly independent.
Proof. Fix scalars α1 , . . . , αk such that
α1 u1 + · · · + αk uk = 0.
WTS α1 = · · · = αk = 0.
Proposition 2.3
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let {u1 , . . . , uk } be an orthogonal set of non-zero
vectors in V . Then {u1 , . . . , uk } is linearly independent.
Proof. Fix scalars α1 , . . . , αk such that
α1 u1 + · · · + αk uk = 0.
WTS α1 = · · · = αk = 0. Fix any i ∈ {1, . . . , k}. Then
⟨α1 u1 + · · · + αk uk , ui ⟩ = ⟨0, ui ⟩ = 0.
Proposition 2.3
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let {u1 , . . . , uk } be an orthogonal set of non-zero
vectors in V . Then {u1 , . . . , uk } is linearly independent.
Proof. Fix scalars α1 , . . . , αk such that
α1 u1 + · · · + αk uk = 0.
WTS α1 = · · · = αk = 0. Fix any i ∈ {1, . . . , k}. Then
⟨α1 u1 + · · · + αk uk , ui ⟩ = ⟨0, ui ⟩ = 0.
Now, note that
⟨α1 u1 + · · · + αk uk , ui ⟩ = α1 ⟨u1 , ui ⟩ + · · · + αk ⟨uk , ui ⟩

(∗)
= αi ⟨ui , ui ⟩,
where (*) follows from the fact that {u1 , . . . , uk } is an orthogonal
set.
Proposition 2.3
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let {u1 , . . . , uk } be an orthogonal set of non-zero
vectors in V . Then {u1 , . . . , uk } is linearly independent.
Proof. Fix scalars α1 , . . . , αk such that
α1 u1 + · · · + αk uk = 0.
WTS α1 = · · · = αk = 0. Fix any i ∈ {1, . . . , k}. Then
⟨α1 u1 + · · · + αk uk , ui ⟩ = ⟨0, ui ⟩ = 0.
Now, note that
⟨α1 u1 + · · · + αk uk , ui ⟩ = α1 ⟨u1 , ui ⟩ + · · · + αk ⟨uk , ui ⟩

(∗)
= αi ⟨ui , ui ⟩,
where (*) follows from the fact that {u1 , . . . , uk } is an orthogonal
set. So, αi ⟨ui , ui ⟩ = 0.
Proposition 2.3
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let {u1 , . . . , uk } be an orthogonal set of non-zero
vectors in V . Then {u1 , . . . , uk } is linearly independent.
Proof. Fix scalars α1 , . . . , αk such that
α1 u1 + · · · + αk uk = 0.
WTS α1 = · · · = αk = 0. Fix any i ∈ {1, . . . , k}. Then
⟨α1 u1 + · · · + αk uk , ui ⟩ = ⟨0, ui ⟩ = 0.
Now, note that
⟨α1 u1 + · · · + αk uk , ui ⟩ = α1 ⟨u1 , ui ⟩ + · · · + αk ⟨uk , ui ⟩

(∗)
= αi ⟨ui , ui ⟩,
where (*) follows from the fact that {u1 , . . . , uk } is an orthogonal
set. So, αi ⟨ui , ui ⟩ = 0. Since ui ̸= 0, we see that ⟨ui , ui ⟩ =
̸ 0;
consequently, αi = 0. Q.E.D.
Proposition 2.4
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let {u1 , . . . , uk } be an orthogonal set of non-zero
k k
⟨v,ui ⟩
vectors in V . Let v ∈ V , and set y :=
P P
projui (v) = ⟨ui ,ui ⟩ ui
i=1 i=1
and z := v − y.
(a) {u1 , . . . , uk , z} is an orthogonal set of vectors;
(b) z = 0 if and only if v ∈ Span(u1 , . . . , uk );
(c) Span(u1 , . . . , uk , v) = Span(u1 , . . . , uk , z).

Proof.
Proposition 2.4
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let {u1 , . . . , uk } be an orthogonal set of non-zero
k k
⟨v,ui ⟩
vectors in V . Let v ∈ V , and set y :=
P P
projui (v) = ⟨ui ,ui ⟩ ui
i=1 i=1
and z := v − y.
(a) {u1 , . . . , uk , z} is an orthogonal set of vectors;
(b) z = 0 if and only if v ∈ Span(u1 , . . . , uk );
(c) Span(u1 , . . . , uk , v) = Span(u1 , . . . , uk , z).

Proof. First of all, we remark that Proposition 2.3 guarantees that

{u1 , . . . , uk } is a linearly independent set, and we deduce that
{u1 , . . . , uk } is an orthogonal basis of Span(u1 , . . . , uk ).
Proof of Proposition 2.4 (continued).
(a) WTS {u1 , . . . , uk , z} is an orthogonal set of vectors.
Proof of Proposition 2.4 (continued).
(a) WTS {u1 , . . . , uk , z} is an orthogonal set of vectors.
Since {u1 , . . . , uk , z} is an orthogonal set of vectors, it suffices to
show that ⟨z, uj ⟩ = 0 for all j ∈ {1, . . . , k}.
Proof of Proposition 2.4 (continued).
(a) WTS {u1 , . . . , uk , z} is an orthogonal set of vectors.
Since {u1 , . . . , uk , z} is an orthogonal set of vectors, it suffices to
show that ⟨z, uj ⟩ = 0 for all j ∈ {1, . . . , k}. But indeed, for each
j ∈ {1, . . . , k}, we have that
k
⟨v,ui ⟩
⟨z, uj ⟩ ⟨v − ⟨ui ,ui ⟩ ui , uj ⟩
P
=
i=1
k
⟨v,uj ⟩
⟨v, uj ⟩ − ⟨ui ,ui ⟩ ⟨ui , uj ⟩
P
=
i=1
(∗) ⟨v,uj ⟩
= ⟨v, uj ⟩ − ⟨uj ,uj ⟩ ⟨uj , uj ⟩
= ⟨v, uj ⟩ − ⟨v, uj ⟩
= 0.

where (*) follows from the fact that {u1 , . . . , uk } is an orthogonal

set. This proves (a).
Proof of Proposition 2.4 (continued).
(b) WTS z = 0 iff v ∈ Span(u1 , . . . , uk ).
Proof of Proposition 2.4 (continued).
(b) WTS z = 0 iff v ∈ Span(u1 , . . . , uk ).
k
P ⟨v,ui ⟩
Clearly, z = 0 iff v = ⟨ui ,ui ⟩ ui .
i=1
Proof of Proposition 2.4 (continued).
(b) WTS z = 0 iff v ∈ Span(u1 , . . . , uk ).
k
P ⟨v,ui ⟩
Clearly, z = 0 iff v = ⟨ui ,ui ⟩ ui .
i=1

So, it is enough to show that v ∈ Span(u1 , . . . , uk ) iff

k
P ⟨v,ui ⟩
v= ⟨ui ,ui ⟩ ui .
i=1
Proof of Proposition 2.4 (continued).
(b) WTS z = 0 iff v ∈ Span(u1 , . . . , uk ).
k
P ⟨v,ui ⟩
Clearly, z = 0 iff v = ⟨ui ,ui ⟩ ui .
i=1

So, it is enough to show that v ∈ Span(u1 , . . . , uk ) iff

k
P ⟨v,ui ⟩
v= ⟨ui ,ui ⟩ ui .
i=1
k
P ⟨v,ui ⟩
If v = ⟨ui ,ui ⟩ ui , then v is a linear combination of the vectors
i=1
u1 , . . . , uk , and consequently, v ∈ Span(u1 , . . . , uk ).
Proof of Proposition 2.4 (continued).
(b) WTS z = 0 iff v ∈ Span(u1 , . . . , uk ).
k
P ⟨v,ui ⟩
Clearly, z = 0 iff v = ⟨ui ,ui ⟩ ui .
i=1

So, it is enough to show that v ∈ Span(u1 , . . . , uk ) iff

k
P ⟨v,ui ⟩
v= ⟨ui ,ui ⟩ ui .
i=1
k
P ⟨v,ui ⟩
If v = ⟨ui ,ui ⟩ ui , then v is a linear combination of the vectors
i=1
u1 , . . . , uk , and consequently, v ∈ Span(u1 , . . . , uk ).
On the other hand, if v ∈ Span(u1 , . . . , uk ), then Theorem 2.1
k
P ⟨v,ui ⟩
guarantees v = ⟨ui ,ui ⟩ ui . This proves (b).
i=1
Proof of Proposition 2.4 (continued).
(c) WTS Span(u1 , . . . , uk , v) = Span(u1 , . . . , uk , z).
Proof of Proposition 2.4 (continued).
(c) WTS Span(u1 , . . . , uk , v) = Span(u1 , . . . , uk , z).
Fix any vector x ∈ V . WTS x ∈ Span(u1 , . . . , uk , v) iff
x ∈ Span(u1 , . . . , uk , z).
Proof of Proposition 2.4 (continued).
(c) WTS Span(u1 , . . . , uk , v) = Span(u1 , . . . , uk , z).
Fix any vector x ∈ V . WTS x ∈ Span(u1 , . . . , uk , v) iff
x ∈ Span(u1 , . . . , uk , z).
Suppose first that x ∈ Span(u1 , . . . , uk , v). Then there exist scalars
α1 , . . . , αk , β s.t. x = α1 u1 + · · · + αk uk + βv.
Proof of Proposition 2.4 (continued).
(c) WTS Span(u1 , . . . , uk , v) = Span(u1 , . . . , uk , z).
Fix any vector x ∈ V . WTS x ∈ Span(u1 , . . . , uk , v) iff
x ∈ Span(u1 , . . . , uk , z).
Suppose first that x ∈ Span(u1 , . . . , uk , v). Then there exist scalars
α1 , . . . , αk , β s.t. x = α1 u1 + · · · + αk uk + βv. But now
x = α1 u1 + · · · + αk uk + βv
Pk
= ( αi ui ) + β(y + z)
i=1
k  P k 
P ⟨v,ui ⟩
= ( αi ui ) + β ( ⟨u ,u
i i ⟩ ui ) + z
i=1 i=1
P k 
⟨v,ui ⟩
= (αi + β ⟨ui ,ui ⟩ )ui + βz,
i=1

and we deduce that x ∈ Span(u1 , . . . , uk , z).

Proof of Proposition 2.4 (continued).
(c) WTS Span(u1 , . . . , uk , v) = Span(u1 , . . . , uk , z).
Fix any vector x ∈ V . WTS x ∈ Span(u1 , . . . , uk , v) iff
x ∈ Span(u1 , . . . , uk , z).
Suppose first that x ∈ Span(u1 , . . . , uk , v). Then there exist scalars
α1 , . . . , αk , β s.t. x = α1 u1 + · · · + αk uk + βv. But now
x = α1 u1 + · · · + αk uk + βv
Pk
= ( αi ui ) + β(y + z)
i=1
k  P k 
P ⟨v,ui ⟩
= ( αi ui ) + β ( ⟨u ,u
i i ⟩ ui ) + z
i=1 i=1
P k 
⟨v,ui ⟩
= (αi + β ⟨ui ,ui ⟩ )ui + βz,
i=1

and we deduce that x ∈ Span(u1 , . . . , uk , z).

The other direction is similar. This proves (c).
Proposition 2.3
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let {u1 , . . . , uk } be an orthogonal set of non-zero
vectors in V . Then {u1 , . . . , uk } is linearly independent.

Proposition 2.4
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let {u1 , . . . , uk } be an orthogonal set of non-zero
k k
⟨v,ui ⟩
vectors in V . Let v ∈ V , and set y :=
P P
projui (v) = ⟨ui ,ui ⟩ ui
i=1 i=1
and z := v − y.
(a) {u1 , . . . , uk , z} is an orthogonal set of vectors;
(b) z = 0 if and only if v ∈ Span(u1 , . . . , uk );
(c) Span(u1 , . . . , uk , v) = Span(u1 , . . . , uk , z).
The Gram-Schmidt orthogonalization process (version 1)
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩ and the norm || · || induced by ⟨·, ·⟩. Let v1 , . . . , vk be
a linearly independent set of vectors in V . ∀ℓ ∈ {1, . . . , k}, set
ℓ−1 ℓ−1
⟨vℓ ,ui ⟩
uℓ = vℓ − projui (vℓ ) = vℓ −
P P
⟨ui ,ui ⟩ ui .
i=1 i=1

Then {u1 , . . . , uk } is an orthogonal basis of Span(v1 , . . . , vk ).

Moreover, { ||uu11 || , . . . , ||uukk || } is an orthonormal basis of
Span(v1 , . . . , vk ).
The Gram-Schmidt orthogonalization process (version 1)
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩ and the norm || · || induced by ⟨·, ·⟩. Let v1 , . . . , vk be
a linearly independent set of vectors in V . ∀ℓ ∈ {1, . . . , k}, set
ℓ−1 ℓ−1
⟨vℓ ,ui ⟩
uℓ = vℓ − projui (vℓ ) = vℓ −
P P
⟨ui ,ui ⟩ ui .
i=1 i=1

Then {u1 , . . . , uk } is an orthogonal basis of Span(v1 , . . . , vk ).

Moreover, { ||uu11 || , . . . , ||uukk || } is an orthonormal basis of
Span(v1 , . . . , vk ).

u1 = v1 ;
u2 = v2 − proju1 (v2 );
 
u3 = v3 − proju1 (v3 ) + proju2 (v3 ) ;
..
.  
uk = vk − proju1 (vk ) + proju2 (vk ) + · · · + projuk−1 (vk ) .
First an example, then the proof.
 First an example, then the proof.

Example 2.6
Consider the vectors following vectors
     
3 −5 1
 1   1   1 
v1 =  , v2 =  , v3 =  .
     
 −1   5   −2 
3 −7 8

in R4 , and set U := Span(v1 , v2 , v3 ). Using the Gram-Schmidt

orthogonalization process (version 1):
(a) find an orthogonal basis of U (with respect to the standard
scalar product · in R4 ).
(b) find an orthonormal basis of U (with respect to the standard
scalar product · in R4 and the norm || · || induced by it).
Solution of Example 2.6
Solution of Example 2.6
h iT
u1 := v1 = 3 1 −1 3
Solution of Example 2.6
h iT
u1 := v1 = 3 1 −1 3
h iT
v2 ·u1
u2 := v2 − u1 ·u1 u1 = 1 3 3 −1
Solution of Example 2.6
h iT
u1 := v1 = 3 1 −1 3
h iT
v2 ·u1
u2 := v2 − u1 ·u1 u1 = 1 3 3 −1
  h iT
v3 ·u1 v3 ·u2
u3 := v3 − u1 ·u1 u1 + u2 ·u2 u2 = −3 1 1 3 .
Solution of Example 2.6
h iT
u1 := v1 = 3 1 −1 3
h iT
v2 ·u1
u2 := v2 − u1 ·u1 u1 = 1 3 3 −1
  h iT
v3 ·u1 v3 ·u2
u3 := v3 − u1 ·u1 u1 + u2 ·u2 u2 = −3 1 1 3 .
     
3 1 −3
n 1   3   1 o
So, B := {u1 , u2 , u3 } = , ,  is an
     
−1 3 1

     
3 −1 3
orthogonal basis of U.
To obtain an orthonormal basis of U, we “normalize” the vectors
from the orthogonal basis B of U from part (a), that is, we rescale
them so that their length becomes 1.
Solution of Example 2.6
h iT
u1 := v1 = 3 1 −1 3
h iT
v2 ·u1
u2 := v2 − u1 ·u1 u1 = 1 3 3 −1
  h iT
v3 ·u1 v3 ·u2
u3 := v3 − u1 ·u1 u1 + u2 ·u2 u2 = −3 1 1 3 .
     
3 1 −3
n 1   3   1 o
So, B := {u1 , u2 , u3 } = , ,  is an
     
−1 3 1

     
3 −1 3
orthogonal basis of U.
To obtain an orthonormal basis of U, we “normalize” the vectors
from the orthogonal basis B of U from part (a), that is, we rescale
them so that their length becomes 1.
√ √ √
We compute ||u1 || = 2 5, ||u2 || = 2 5, ||u3 || = 2 5.
Solution of Example 2.6 (continued).
We now see that
C := { ||uu11 || , ||uu22 || , ||uu33 || }
     
3 1 −3
n
1
 1   3   1 o
= √ , 1  , 1 
  √   √  
2 5 −1 2 5 3 2 5 1
 
     
3 −1 3

is an orthonormal basis of U.
The Gram-Schmidt orthogonalization process (version 1)
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩ and the norm || · || induced by ⟨·, ·⟩. Let v1 , . . . , vk be
a linearly independent set of vectors in V . ∀ℓ ∈ {1, . . . , k}, set
ℓ−1 ℓ−1
⟨vℓ ,ui ⟩
uℓ = vℓ − projui (vℓ ) = vℓ −
P P
⟨ui ,ui ⟩ ui .
i=1 i=1

Then {u1 , . . . , uk } is an orthogonal basis of Span(v1 , . . . , vk ).

Moreover, { ||uu11 || , . . . , ||uukk || } is an orthonormal basis of
Span(v1 , . . . , vk ).

u1 = v1 ;
u2 = v2 − proju1 (v2 );
 
u3 = v3 − proju1 (v3 ) + proju2 (v3 ) ;
..
.  
uk = vk − proju1 (vk ) + proju2 (vk ) + · · · + projuk−1 (vk ) .
Proof (outline).
Proof (outline). If {u1 , . . . , uk } is orthogonal basis of
Span(v1 , . . . , vk ), then obviously, { ||uu11 || , . . . , ||uukk || } is an
orthonormal basis of Span(v1 , . . . , vk ) (since we simply rescaled
each vector so that its length is 1).
Proof (outline). If {u1 , . . . , uk } is orthogonal basis of
Span(v1 , . . . , vk ), then obviously, { ||uu11 || , . . . , ||uukk || } is an
orthonormal basis of Span(v1 , . . . , vk ) (since we simply rescaled
each vector so that its length is 1).
Let us prove that {u1 , . . . , uk } is an orthogonal basis of
Span(v1 , . . . , vk ).
Proof (outline). If {u1 , . . . , uk } is orthogonal basis of
Span(v1 , . . . , vk ), then obviously, { ||uu11 || , . . . , ||uukk || } is an
orthonormal basis of Span(v1 , . . . , vk ) (since we simply rescaled
each vector so that its length is 1).
Let us prove that {u1 , . . . , uk } is an orthogonal basis of
Span(v1 , . . . , vk ). For each ℓ ∈ {1, . . . , k}, we let
Uℓ := Span(v1 , . . . , vℓ ), and we prove that {u1 , . . . , uℓ } is an
orthogonal basis of Uℓ .
Proof (outline). If {u1 , . . . , uk } is orthogonal basis of
Span(v1 , . . . , vk ), then obviously, { ||uu11 || , . . . , ||uukk || } is an
orthonormal basis of Span(v1 , . . . , vk ) (since we simply rescaled
each vector so that its length is 1).
Let us prove that {u1 , . . . , uk } is an orthogonal basis of
Span(v1 , . . . , vk ). For each ℓ ∈ {1, . . . , k}, we let
Uℓ := Span(v1 , . . . , vℓ ), and we prove that {u1 , . . . , uℓ } is an
orthogonal basis of Uℓ . Obviously, this is enough, because for
k = ℓ, we get that {u1 , . . . , uk } is an orthogonal basis of
Uk = Span(v1 , . . . , vk ).
Proof (outline). If {u1 , . . . , uk } is orthogonal basis of
Span(v1 , . . . , vk ), then obviously, { ||uu11 || , . . . , ||uukk || } is an
orthonormal basis of Span(v1 , . . . , vk ) (since we simply rescaled
each vector so that its length is 1).
Let us prove that {u1 , . . . , uk } is an orthogonal basis of
Span(v1 , . . . , vk ). For each ℓ ∈ {1, . . . , k}, we let
Uℓ := Span(v1 , . . . , vℓ ), and we prove that {u1 , . . . , uℓ } is an
orthogonal basis of Uℓ . Obviously, this is enough, because for
k = ℓ, we get that {u1 , . . . , uk } is an orthogonal basis of
Uk = Span(v1 , . . . , vk ).
Since {v1 , . . . , vk } is linearly independent, we see that v1 , . . . , vk
are all non-zero, and in particular, {v1 } is linearly independent.
Proof (outline). If {u1 , . . . , uk } is orthogonal basis of
Span(v1 , . . . , vk ), then obviously, { ||uu11 || , . . . , ||uukk || } is an
orthonormal basis of Span(v1 , . . . , vk ) (since we simply rescaled
each vector so that its length is 1).
Let us prove that {u1 , . . . , uk } is an orthogonal basis of
Span(v1 , . . . , vk ). For each ℓ ∈ {1, . . . , k}, we let
Uℓ := Span(v1 , . . . , vℓ ), and we prove that {u1 , . . . , uℓ } is an
orthogonal basis of Uℓ . Obviously, this is enough, because for
k = ℓ, we get that {u1 , . . . , uk } is an orthogonal basis of
Uk = Span(v1 , . . . , vk ).
Since {v1 , . . . , vk } is linearly independent, we see that v1 , . . . , vk
are all non-zero, and in particular, {v1 } is linearly independent.
Since U1 = Span(v1 ) and u1 = v1 , we deduce that {u1 } is a basis
of U1 .
Proof (outline, continued).
Reminder: Uℓ := Span(v1 , . . . , vℓ ) ∀ℓ ∈ {1, . . . , k}.
Proof (outline, continued).
Reminder: Uℓ := Span(v1 , . . . , vℓ ) ∀ℓ ∈ {1, . . . , k}.
Now, fix ℓ ∈ {1, . . . , k − 1}, and assume inductively that
{u1 , . . . , uℓ } is an orthogonal basis of Uℓ . WTS {u1 , . . . , uℓ , uℓ+1 }
is an orthogonal basis of Uℓ+1 .
Proof (outline, continued).
Reminder: Uℓ := Span(v1 , . . . , vℓ ) ∀ℓ ∈ {1, . . . , k}.
Now, fix ℓ ∈ {1, . . . , k − 1}, and assume inductively that
{u1 , . . . , uℓ } is an orthogonal basis of Uℓ . WTS {u1 , . . . , uℓ , uℓ+1 }
is an orthogonal basis of Uℓ+1 . Since {u1 , . . . , uℓ } and {v1 , . . . , vℓ }
are two bases of Uℓ , it is clear that
Span(u1 , . . . , uℓ , vℓ+1 ) = Span(v1 , . . . , vℓ , vℓ+1 ) = Uℓ+1 .
Proof (outline, continued).
Reminder: Uℓ := Span(v1 , . . . , vℓ ) ∀ℓ ∈ {1, . . . , k}.
Now, fix ℓ ∈ {1, . . . , k − 1}, and assume inductively that
{u1 , . . . , uℓ } is an orthogonal basis of Uℓ . WTS {u1 , . . . , uℓ , uℓ+1 }
is an orthogonal basis of Uℓ+1 . Since {u1 , . . . , uℓ } and {v1 , . . . , vℓ }
are two bases of Uℓ , it is clear that
Span(u1 , . . . , uℓ , vℓ+1 ) = Span(v1 , . . . , vℓ , vℓ+1 ) = Uℓ+1 . On the
other hand, by the construction of uℓ+1 and by Proposition 2.4(c),
we have that Span(u1 , . . . , uℓ , vℓ+1 ) = Span(u1 , . . . , uℓ , uℓ+1 ).
Proof (outline, continued).
Reminder: Uℓ := Span(v1 , . . . , vℓ ) ∀ℓ ∈ {1, . . . , k}.
Now, fix ℓ ∈ {1, . . . , k − 1}, and assume inductively that
{u1 , . . . , uℓ } is an orthogonal basis of Uℓ . WTS {u1 , . . . , uℓ , uℓ+1 }
is an orthogonal basis of Uℓ+1 . Since {u1 , . . . , uℓ } and {v1 , . . . , vℓ }
are two bases of Uℓ , it is clear that
Span(u1 , . . . , uℓ , vℓ+1 ) = Span(v1 , . . . , vℓ , vℓ+1 ) = Uℓ+1 . On the
other hand, by the construction of uℓ+1 and by Proposition 2.4(c),
we have that Span(u1 , . . . , uℓ , vℓ+1 ) = Span(u1 , . . . , uℓ , uℓ+1 ). So,
Span(u1 , . . . , uℓ , uℓ+1 ) = Uℓ+1 .
Proof (outline, continued).
Reminder: Uℓ := Span(v1 , . . . , vℓ ) ∀ℓ ∈ {1, . . . , k}.
Now, fix ℓ ∈ {1, . . . , k − 1}, and assume inductively that
{u1 , . . . , uℓ } is an orthogonal basis of Uℓ . WTS {u1 , . . . , uℓ , uℓ+1 }
is an orthogonal basis of Uℓ+1 . Since {u1 , . . . , uℓ } and {v1 , . . . , vℓ }
are two bases of Uℓ , it is clear that
Span(u1 , . . . , uℓ , vℓ+1 ) = Span(v1 , . . . , vℓ , vℓ+1 ) = Uℓ+1 . On the
other hand, by the construction of uℓ+1 and by Proposition 2.4(c),
we have that Span(u1 , . . . , uℓ , vℓ+1 ) = Span(u1 , . . . , uℓ , uℓ+1 ). So,
Span(u1 , . . . , uℓ , uℓ+1 ) = Uℓ+1 . Since dim(Uℓ+1 ) = ℓ + 1, the fact
that {u1 , . . . , uℓ , uℓ+1 } spans Uℓ+1 implies that {u1 , . . . , uℓ , uℓ+1 }
is in fact a basis of Uℓ+1 .
Proof (outline, continued).
Reminder: Uℓ := Span(v1 , . . . , vℓ ) ∀ℓ ∈ {1, . . . , k}.
Now, fix ℓ ∈ {1, . . . , k − 1}, and assume inductively that
{u1 , . . . , uℓ } is an orthogonal basis of Uℓ . WTS {u1 , . . . , uℓ , uℓ+1 }
is an orthogonal basis of Uℓ+1 . Since {u1 , . . . , uℓ } and {v1 , . . . , vℓ }
are two bases of Uℓ , it is clear that
Span(u1 , . . . , uℓ , vℓ+1 ) = Span(v1 , . . . , vℓ , vℓ+1 ) = Uℓ+1 . On the
other hand, by the construction of uℓ+1 and by Proposition 2.4(c),
we have that Span(u1 , . . . , uℓ , vℓ+1 ) = Span(u1 , . . . , uℓ , uℓ+1 ). So,
Span(u1 , . . . , uℓ , uℓ+1 ) = Uℓ+1 . Since dim(Uℓ+1 ) = ℓ + 1, the fact
that {u1 , . . . , uℓ , uℓ+1 } spans Uℓ+1 implies that {u1 , . . . , uℓ , uℓ+1 }
is in fact a basis of Uℓ+1 . By the induction hypothesis, vectors
u1 , . . . , uℓ are pairwise orthogonal non-zero vectors, and so by the
costruction of uℓ+1 and by Proposition 2.4(a), we have that
u1 , . . . , uℓ , uℓ+1 are pairwise orthogonal.
Proof (outline, continued).
Reminder: Uℓ := Span(v1 , . . . , vℓ ) ∀ℓ ∈ {1, . . . , k}.
Now, fix ℓ ∈ {1, . . . , k − 1}, and assume inductively that
{u1 , . . . , uℓ } is an orthogonal basis of Uℓ . WTS {u1 , . . . , uℓ , uℓ+1 }
is an orthogonal basis of Uℓ+1 . Since {u1 , . . . , uℓ } and {v1 , . . . , vℓ }
are two bases of Uℓ , it is clear that
Span(u1 , . . . , uℓ , vℓ+1 ) = Span(v1 , . . . , vℓ , vℓ+1 ) = Uℓ+1 . On the
other hand, by the construction of uℓ+1 and by Proposition 2.4(c),
we have that Span(u1 , . . . , uℓ , vℓ+1 ) = Span(u1 , . . . , uℓ , uℓ+1 ). So,
Span(u1 , . . . , uℓ , uℓ+1 ) = Uℓ+1 . Since dim(Uℓ+1 ) = ℓ + 1, the fact
that {u1 , . . . , uℓ , uℓ+1 } spans Uℓ+1 implies that {u1 , . . . , uℓ , uℓ+1 }
is in fact a basis of Uℓ+1 . By the induction hypothesis, vectors
u1 , . . . , uℓ are pairwise orthogonal non-zero vectors, and so by the
costruction of uℓ+1 and by Proposition 2.4(a), we have that
u1 , . . . , uℓ , uℓ+1 are pairwise orthogonal. So, {u1 , . . . , uℓ , uℓ+1 } is
an orthogonal basis of Uℓ+1 . This completes the induction. Q.E.D.
The Gram-Schmidt orthogonalization process (version 1)
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩ and the norm || · || induced by ⟨·, ·⟩. Let v1 , . . . , vk be
a linearly independent set of vectors in V . ∀ℓ ∈ {1, . . . , k}, set
ℓ−1 ℓ−1
⟨vℓ ,ui ⟩
uℓ = vℓ − projui (vℓ ) = vℓ −
P P
⟨ui ,ui ⟩ ui .
i=1 i=1

Then {u1 , . . . , uk } is an orthogonal basis of Span(v1 , . . . , vk ).

Moreover, { ||uu11 || , . . . , ||uukk || } is an orthonormal basis of
Span(v1 , . . . , vk ).

u1 = v1 ;
u2 = v2 − proju1 (v2 );
 
u3 = v3 − proju1 (v3 ) + proju2 (v3 ) ;
..
.  
uk = vk − proju1 (vk ) + proju2 (vk ) + · · · + projuk−1 (vk ) .
Corollary 2.5
Let V be a finite-dimensional vector space over R or C, equipped
with a scalar product ⟨·, ·⟩ and the norm || · || induced by ⟨·, ·⟩. Let
U be a subspace of V . Then all the following hold:
(a) U has an orthogonal basis;
(b) any orthogonal basis of U can be extended to an orthogonal
basis of V ;
(c) U has an orthonormal basis;
(d) any orthonormal basis of U can be extended to an
orthonormal basis of V .
Proof.
Corollary 2.5
Let V be a finite-dimensional vector space over R or C, equipped
with a scalar product ⟨·, ·⟩ and the norm || · || induced by ⟨·, ·⟩. Let
U be a subspace of V . Then all the following hold:
(a) U has an orthogonal basis;
(b) any orthogonal basis of U can be extended to an orthogonal
basis of V ;
(c) U has an orthonormal basis;
(d) any orthonormal basis of U can be extended to an
orthonormal basis of V .
Proof. Consider any basis {v1 , . . . , vk } of U.
Corollary 2.5
Let V be a finite-dimensional vector space over R or C, equipped
with a scalar product ⟨·, ·⟩ and the norm || · || induced by ⟨·, ·⟩. Let
U be a subspace of V . Then all the following hold:
(a) U has an orthogonal basis;
(b) any orthogonal basis of U can be extended to an orthogonal
basis of V ;
(c) U has an orthonormal basis;
(d) any orthonormal basis of U can be extended to an
orthonormal basis of V .
Proof. Consider any basis {v1 , . . . , vk } of U. Then the
Gram-Schmidt orthogonalization process applied to the vectors
v1 , . . . , vk yields a sequence of vectors u1 , . . . , uk such that
{u1 , . . . , uk } is an orthogonal and { ||uu11 || , . . . , ||uukk || } an orthonormal
basis of U = Span(v1 , . . . , vk ). This proves (a) and (c).
(a) U has an orthogonal basis;
(b) any orthogonal basis of U can be extended to an orthogonal
basis of V ;
(c) U has an orthonormal basis;
(d) any orthonormal basis of U can be extended to an
orthonormal basis of V .
Proof (continued).
(a) U has an orthogonal basis;
(b) any orthogonal basis of U can be extended to an orthogonal
basis of V ;
(c) U has an orthonormal basis;
(d) any orthonormal basis of U can be extended to an
orthonormal basis of V .
Proof (continued). For (b), consider any orthogonal basis
{v1 , . . . , vk } of U, and extend it to a basis
{v1 , . . . , vk , vk+1 , . . . , vn } of V .
(a) U has an orthogonal basis;
(b) any orthogonal basis of U can be extended to an orthogonal
basis of V ;
(c) U has an orthonormal basis;
(d) any orthonormal basis of U can be extended to an
orthonormal basis of V .
Proof (continued). For (b), consider any orthogonal basis
{v1 , . . . , vk } of U, and extend it to a basis
{v1 , . . . , vk , vk+1 , . . . , vn } of V . We then apply the Gram-Schmidt
orthogonaliation process to the sequence v1 , . . . , vk , vk+1 , . . . , vn ,
and we obtain some sequence u1 , . . . , uk , uk+1 , . . . , un such that
{u1 , . . . , uk , uk+1 , . . . , un } is an orthogonal basis of V .
(a) U has an orthogonal basis;
(b) any orthogonal basis of U can be extended to an orthogonal
basis of V ;
(c) U has an orthonormal basis;
(d) any orthonormal basis of U can be extended to an
orthonormal basis of V .
Proof (continued). For (b), consider any orthogonal basis
{v1 , . . . , vk } of U, and extend it to a basis
{v1 , . . . , vk , vk+1 , . . . , vn } of V . We then apply the Gram-Schmidt
orthogonaliation process to the sequence v1 , . . . , vk , vk+1 , . . . , vn ,
and we obtain some sequence u1 , . . . , uk , uk+1 , . . . , un such that
{u1 , . . . , uk , uk+1 , . . . , un } is an orthogonal basis of V . However,
since v1 , . . . , vk were pairwise orthogonal to begin with, we see
from the description of the Gram-Schmidt orthogonalization
process that u1 = v1 , . . . , uk = vk .
(a) U has an orthogonal basis;
(b) any orthogonal basis of U can be extended to an orthogonal
basis of V ;
(c) U has an orthonormal basis;
(d) any orthonormal basis of U can be extended to an
orthonormal basis of V .
Proof (continued). For (b), consider any orthogonal basis
{v1 , . . . , vk } of U, and extend it to a basis
{v1 , . . . , vk , vk+1 , . . . , vn } of V . We then apply the Gram-Schmidt
orthogonaliation process to the sequence v1 , . . . , vk , vk+1 , . . . , vn ,
and we obtain some sequence u1 , . . . , uk , uk+1 , . . . , un such that
{u1 , . . . , uk , uk+1 , . . . , un } is an orthogonal basis of V . However,
since v1 , . . . , vk were pairwise orthogonal to begin with, we see
from the description of the Gram-Schmidt orthogonalization
process that u1 = v1 , . . . , uk = vk . So, the orthogonal basis
{u1 , . . . , uk , uk+1 , . . . , un } of V extends the orthogonal basis
{v1 , . . . , vk } of U. This proves (b).
(a) U has an orthogonal basis;
(b) any orthogonal basis of U can be extended to an orthogonal
basis of V ;
(c) U has an orthonormal basis;
(d) any orthonormal basis of U can be extended to an
orthonormal basis of V .
Proof (continued).
(a) U has an orthogonal basis;
(b) any orthogonal basis of U can be extended to an orthogonal
basis of V ;
(c) U has an orthonormal basis;
(d) any orthonormal basis of U can be extended to an
orthonormal basis of V .
Proof (continued). For (d), consider any orthonormal basis
{u1 , . . . , uk } of U.
 (a) U has an orthogonal basis;
(b) any orthogonal basis of U can be extended to an orthogonal
basis of V ;
(c) U has an orthonormal basis;
(d) any orthonormal basis of U can be extended to an
orthonormal basis of V .
Proof (continued). For (d), consider any orthonormal basis
{u1 , . . . , uk } of U. In particular, the basis {u1 , . . . , uk } of U is
orthogonal, and so by (b), it can be extended to an orthogonal
basis {u1 , . . . , uk , uk+1 , . . . , un } of V .
 (a) U has an orthogonal basis;
(b) any orthogonal basis of U can be extended to an orthogonal
basis of V ;
(c) U has an orthonormal basis;
(d) any orthonormal basis of U can be extended to an
orthonormal basis of V .
Proof (continued). For (d), consider any orthonormal basis
{u1 , . . . , uk } of U. In particular, the basis {u1 , . . . , uk } of U is
orthogonal, and so by (b), it can be extended to an orthogonal
basis {u1 , . . . , uk , uk+1 , . . . , un } of V . But then { ||uu11 || , . . . , ||uunn || }
is an orthonormal basis of V . But since the basis {u1 , . . . , uk } of
U is orthonormal, we know that ||u1 || = · · · = ||uk || = 1, and it
follows that ||uu11 || = u1 , . . . , ||uukk || = uk .
 (a) U has an orthogonal basis;
(b) any orthogonal basis of U can be extended to an orthogonal
basis of V ;
(c) U has an orthonormal basis;
(d) any orthonormal basis of U can be extended to an
orthonormal basis of V .
Proof (continued). For (d), consider any orthonormal basis
{u1 , . . . , uk } of U. In particular, the basis {u1 , . . . , uk } of U is
orthogonal, and so by (b), it can be extended to an orthogonal
basis {u1 , . . . , uk , uk+1 , . . . , un } of V . But then { ||uu11 || , . . . , ||uunn || }
is an orthonormal basis of V . But since the basis {u1 , . . . , uk } of
U is orthonormal, we know that ||u1 || = · · · = ||uk || = 1, and it
follows that ||uu11 || = u1 , . . . , ||uukk || = uk . So, our orthonormal basis
{ ||uu11 || , . . . , ||uunn || } of V in fact extends the orthonormal basis
{u1 , . . . , uk } of U. This proves (d).
The Gram-Schmidt orthogonalization process (version 2)
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩ and the norm || · || induced by ⟨·, ·⟩. Let v1 , . . . , vk be
a linearly independent set of vectors in V . ∀ℓ ∈ {1, . . . , k}, set
ℓ−1 ℓ−1
uℓ = vℓ − projui (vℓ ) = vℓ − ⟨vℓ , zi ⟩ zi ;
P P
i=1 i=1

uℓ
zℓ = ||uℓ || .

Then {z1 , . . . , zk } is an orthonormal basis of Span(v1 , . . . , vk ).

The Gram-Schmidt orthogonalization process (version 2)
recursively creates two sequences of vectors, namely, u1 , . . . , uk
and z1 , . . . , zk , as follows:
u1 = v1 ;
u1
z1 = ||u1 || ;
u2 = v2 − projz1 (v2 );
u2
z2 = ||u2 || ;
 
u3 = v3 − projz1 (v3 ) + projz2 (v3 ) ;
u3
z3 = ||u3 || ;
..
.  
uk = vk − projz1 (vk ) + projz2 (vk ) + · · · + projzk−1 (vk ) ;
uk
zk = ||uk || .
Example 2.7
Consider the vectors following vectors
     
3 −5 1
 1   1   1 
v1 =  , v2 =  , v3 =  .
     
 −1   5   −2 
3 −7 8

in R4 , and set U := Span(v1 , v2 , v3 ). Using the Gram-Schmidt

orthogonalization process (version 2), find an orthonormal basis of
U (with respect to the standard scalar product · in R4 and the
norm || · || induced by it).

Solution. See the Lecture Notes.

Definition
Let V be a vector space V over R or C, equipped with a scalar
product ⟨·, ·⟩.
For a vector v ∈ V and a set of vectors A ⊆ V ,a we say that v
is orthogonal to A, and we write v ⊥ A, provided that v is
orthogonal to all vectors in A.b
For a set A ⊆ V , the orthogonal complement of A, denoted by
A⊥ , is the set of all vectors in V that are orthogonal to A, i.e.

A⊥ = {v ∈ V | v ⊥ A}

= {v ∈ V | v ⊥ a ∀a ∈ A}

= {v ∈ V | ⟨v, a⟩ = 0 ∀a ∈ A}.
a
A may, but need not be, a subspace of V .
b
By definition, this means that for all a ∈ A, we have that ⟨v, a⟩ = 0.
Proposition 3.1
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let A, B ⊆ V . Then
(a) A⊥ is a subspace of V ;
(b) if A ⊆ B, then A⊥ ⊇ B ⊥ .

Proof (outline).

.
Proposition 3.1
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let A, B ⊆ V . Then
(a) A⊥ is a subspace of V ;
(b) if A ⊆ B, then A⊥ ⊇ B ⊥ .

Proof (outline). For (a), check that 0 ∈ A⊥ , and that A is closed

under vector addition and scalar multiplication (details: Lecture
Notes).

.
Proposition 3.1
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let A, B ⊆ V . Then
(a) A⊥ is a subspace of V ;
(b) if A ⊆ B, then A⊥ ⊇ B ⊥ .

Proof (outline). For (a), check that 0 ∈ A⊥ , and that A is closed

under vector addition and scalar multiplication (details: Lecture
Notes).
For (b), suppose that A ⊆ B. WTS A⊥ ⊇ B ⊥ .

.
Proposition 3.1
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let A, B ⊆ V . Then
(a) A⊥ is a subspace of V ;
(b) if A ⊆ B, then A⊥ ⊇ B ⊥ .

Proof (outline). For (a), check that 0 ∈ A⊥ , and that A is closed

under vector addition and scalar multiplication (details: Lecture
Notes).
For (b), suppose that A ⊆ B. WTS A⊥ ⊇ B ⊥ . Fix x ∈ B ⊥ ; WTS
x ∈ A⊥ .
.
Proposition 3.1
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let A, B ⊆ V . Then
(a) A⊥ is a subspace of V ;
(b) if A ⊆ B, then A⊥ ⊇ B ⊥ .

Proof (outline). For (a), check that 0 ∈ A⊥ , and that A is closed

under vector addition and scalar multiplication (details: Lecture
Notes).
For (b), suppose that A ⊆ B. WTS A⊥ ⊇ B ⊥ . Fix x ∈ B ⊥ ; WTS
x ∈ A⊥ . Fix a ∈ A.
.
Proposition 3.1
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let A, B ⊆ V . Then
(a) A⊥ is a subspace of V ;
(b) if A ⊆ B, then A⊥ ⊇ B ⊥ .

Proof (outline). For (a), check that 0 ∈ A⊥ , and that A is closed

under vector addition and scalar multiplication (details: Lecture
Notes).
For (b), suppose that A ⊆ B. WTS A⊥ ⊇ B ⊥ . Fix x ∈ B ⊥ ; WTS
x ∈ A⊥ . Fix a ∈ A. Then a ∈ B (because A ⊆ B), and so x ⊥ a
(because x ∈ B ⊥ ).
Proposition 3.1
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let A, B ⊆ V . Then
(a) A⊥ is a subspace of V ;
(b) if A ⊆ B, then A⊥ ⊇ B ⊥ .

Proof (outline). For (a), check that 0 ∈ A⊥ , and that A is closed

under vector addition and scalar multiplication (details: Lecture
Notes).
For (b), suppose that A ⊆ B. WTS A⊥ ⊇ B ⊥ . Fix x ∈ B ⊥ ; WTS
x ∈ A⊥ . Fix a ∈ A. Then a ∈ B (because A ⊆ B), and so x ⊥ a
(because x ∈ B ⊥ ). So, x ∈ A⊥ . This proves that A⊥ ⊇ B ⊥ .
Proposition 3.2
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let u1 , . . . , uk ∈ V . Then
{u1 , . . . , uk }⊥ = Span(u1 , . . . , uk )⊥ .

Proof.
Proposition 3.2
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let u1 , . . . , uk ∈ V . Then
{u1 , . . . , uk }⊥ = Span(u1 , . . . , uk )⊥ .

Proof. Since {u1 , . . . , uk } ⊆ Span(u1 , . . . , uk ), Proposition 3.1(b)

guarantees that {u1 , . . . , uk }⊥ ⊇ Span(u1 , . . . , uk )⊥ .
Proposition 3.2
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let u1 , . . . , uk ∈ V . Then
{u1 , . . . , uk }⊥ = Span(u1 , . . . , uk )⊥ .

Proof. Since {u1 , . . . , uk } ⊆ Span(u1 , . . . , uk ), Proposition 3.1(b)

guarantees that {u1 , . . . , uk }⊥ ⊇ Span(u1 , . . . , uk )⊥ . Let us prove
the reverse inclusion.
Proposition 3.2
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let u1 , . . . , uk ∈ V . Then
{u1 , . . . , uk }⊥ = Span(u1 , . . . , uk )⊥ .

Proof. Since {u1 , . . . , uk } ⊆ Span(u1 , . . . , uk ), Proposition 3.1(b)

guarantees that {u1 , . . . , uk }⊥ ⊇ Span(u1 , . . . , uk )⊥ . Let us prove
the reverse inclusion. Fix x ∈ {u1 , . . . , uk }⊥ . WTS
x ∈ Span(u1 , . . . , uk )⊥ .
Proposition 3.2
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let u1 , . . . , uk ∈ V . Then
{u1 , . . . , uk }⊥ = Span(u1 , . . . , uk )⊥ .

Proof. Since {u1 , . . . , uk } ⊆ Span(u1 , . . . , uk ), Proposition 3.1(b)

guarantees that {u1 , . . . , uk }⊥ ⊇ Span(u1 , . . . , uk )⊥ . Let us prove
the reverse inclusion. Fix x ∈ {u1 , . . . , uk }⊥ . WTS
x ∈ Span(u1 , . . . , uk )⊥ . Fix u ∈ Span(u1 , . . . , uk ).
Proposition 3.2
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let u1 , . . . , uk ∈ V . Then
{u1 , . . . , uk }⊥ = Span(u1 , . . . , uk )⊥ .

Proof. Since {u1 , . . . , uk } ⊆ Span(u1 , . . . , uk ), Proposition 3.1(b)

guarantees that {u1 , . . . , uk }⊥ ⊇ Span(u1 , . . . , uk )⊥ . Let us prove
the reverse inclusion. Fix x ∈ {u1 , . . . , uk }⊥ . WTS
x ∈ Span(u1 , . . . , uk )⊥ . Fix u ∈ Span(u1 , . . . , uk ). Then there
exist scalars α1 , . . . , αk such that u = α1 u1 + · · · + αk uk .
Proposition 3.2
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let u1 , . . . , uk ∈ V . Then
{u1 , . . . , uk }⊥ = Span(u1 , . . . , uk )⊥ .

Proof. Since {u1 , . . . , uk } ⊆ Span(u1 , . . . , uk ), Proposition 3.1(b)

guarantees that {u1 , . . . , uk }⊥ ⊇ Span(u1 , . . . , uk )⊥ . Let us prove
the reverse inclusion. Fix x ∈ {u1 , . . . , uk }⊥ . WTS
x ∈ Span(u1 , . . . , uk )⊥ . Fix u ∈ Span(u1 , . . . , uk ). Then there
exist scalars α1 , . . . , αk such that u = α1 u1 + · · · + αk uk . But now
⟨u, x⟩ = ⟨α1 u1 + · · · + αk uk , x⟩
= α1 ⟨u1 , x⟩ + · · · + αk ⟨uk , x⟩
(∗)
= α1 0 + · · · + αk 0
= 0,

where (*) follows from the fact that x ∈ {u1 , . . . , uk }⊥ .

Proposition 3.2
Let V be a vector space over R or C, equipped with a scalar
product ⟨·, ·⟩. Let u1 , . . . , uk ∈ V . Then
{u1 , . . . , uk }⊥ = Span(u1 , . . . , uk )⊥ .

Proof. Since {u1 , . . . , uk } ⊆ Span(u1 , . . . , uk ), Proposition 3.1(b)

guarantees that {u1 , . . . , uk }⊥ ⊇ Span(u1 , . . . , uk )⊥ . Let us prove
the reverse inclusion. Fix x ∈ {u1 , . . . , uk }⊥ . WTS
x ∈ Span(u1 , . . . , uk )⊥ . Fix u ∈ Span(u1 , . . . , uk ). Then there
exist scalars α1 , . . . , αk such that u = α1 u1 + · · · + αk uk . But now
⟨u, x⟩ = ⟨α1 u1 + · · · + αk uk , x⟩
= α1 ⟨u1 , x⟩ + · · · + αk ⟨uk , x⟩
(∗)
= α1 0 + · · · + αk 0
= 0,

where (*) follows from the fact that x ∈ {u1 , . . . , uk }⊥ . This

proves that x ⊥ u, and consequently, x ∈ Span(u1 , . . . , uk )⊥ .
Theorem 3.3
Let V be a finite-dimensional vector space over R or C, equipped
with a scalar product ⟨·, ·⟩ and the norm || · || induced by ⟨·, ·⟩. Let
U be a subspace of V . Then all the following hold:
(a) if {u1 , . . . , uk } is an orthogonal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthogonal basis of V , then {uk+1 , . . . , un } is an orthogonal
basis of U ⊥ ;
(b) if {u1 , . . . , uk } is an orthonormal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthonormal basis of V , then {uk+1 , . . . , un } is an
orthonormal basis of U ⊥ ;
(c) dim(U) + dim(U ⊥ ) = dim(V );
(d) V = U + U ⊥;
(e) (U ⊥ )⊥ = U;
(f) U ∩ U ⊥ = {0}.
(a) If {u1 , . . . , uk } is an orthogonal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthogonal basis of V , then {uk+1 , . . . , un } is an orthogonal
basis of U ⊥ .
Proof of (a).
(a) If {u1 , . . . , uk } is an orthogonal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthogonal basis of V , then {uk+1 , . . . , un } is an orthogonal
basis of U ⊥ .
Proof of (a). Assume that {u1 , . . . , uk } is an orthogonal basis of
U, and that {u1 , . . . , uk , uk+1 , . . . , un } is an extension of that
basis to an orthogonal basis of V .
(a) If {u1 , . . . , uk } is an orthogonal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthogonal basis of V , then {uk+1 , . . . , un } is an orthogonal
basis of U ⊥ .
Proof of (a). Assume that {u1 , . . . , uk } is an orthogonal basis of
U, and that {u1 , . . . , uk , uk+1 , . . . , un } is an extension of that
basis to an orthogonal basis of V . (This implies that
{u1 , . . . , uk , uk+1 , . . . , un } is linearly independent, and
consequently, u1 , . . . , uk , uk+1 , . . . , un are all non-zero.)
(a) If {u1 , . . . , uk } is an orthogonal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthogonal basis of V , then {uk+1 , . . . , un } is an orthogonal
basis of U ⊥ .
Proof of (a). Assume that {u1 , . . . , uk } is an orthogonal basis of
U, and that {u1 , . . . , uk , uk+1 , . . . , un } is an extension of that
basis to an orthogonal basis of V . (This implies that
{u1 , . . . , uk , uk+1 , . . . , un } is linearly independent, and
consequently, u1 , . . . , uk , uk+1 , . . . , un are all non-zero.) WTS
{uk+1 , . . . , un } is an orthogonal basis of U ⊥ .
(a) If {u1 , . . . , uk } is an orthogonal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthogonal basis of V , then {uk+1 , . . . , un } is an orthogonal
basis of U ⊥ .
Proof of (a). Assume that {u1 , . . . , uk } is an orthogonal basis of
U, and that {u1 , . . . , uk , uk+1 , . . . , un } is an extension of that
basis to an orthogonal basis of V . (This implies that
{u1 , . . . , uk , uk+1 , . . . , un } is linearly independent, and
consequently, u1 , . . . , uk , uk+1 , . . . , un are all non-zero.) WTS
{uk+1 , . . . , un } is an orthogonal basis of U ⊥ . Clearly,
{uk+1 , . . . , un } is an orthogonal set of vectors, and so it suffices to
show that {uk+1 , . . . , un } is in fact a basis of U ⊥ .
(a) If {u1 , . . . , uk } is an orthogonal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthogonal basis of V , then {uk+1 , . . . , un } is an orthogonal
basis of U ⊥ .
Proof of (a). Assume that {u1 , . . . , uk } is an orthogonal basis of
U, and that {u1 , . . . , uk , uk+1 , . . . , un } is an extension of that
basis to an orthogonal basis of V . (This implies that
{u1 , . . . , uk , uk+1 , . . . , un } is linearly independent, and
consequently, u1 , . . . , uk , uk+1 , . . . , un are all non-zero.) WTS
{uk+1 , . . . , un } is an orthogonal basis of U ⊥ . Clearly,
{uk+1 , . . . , un } is an orthogonal set of vectors, and so it suffices to
show that {uk+1 , . . . , un } is in fact a basis of U ⊥ . By
Proposition 2.3, {uk+1 , . . . , un } is linearly independent, and so we
need only show that Span({uk+1 , . . . , un }) = U ⊥ .
(a) If {u1 , . . . , uk } is an orthogonal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthogonal basis of V , then {uk+1 , . . . , un } is an orthogonal
basis of U ⊥ .
Proof of (a) (continued). WTS Span({uk+1 , . . . , un }) = U ⊥ .
(a) If {u1 , . . . , uk } is an orthogonal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthogonal basis of V , then {uk+1 , . . . , un } is an orthogonal
basis of U ⊥ .
Proof of (a) (continued). WTS Span({uk+1 , . . . , un }) = U ⊥ .
WTS Span({uk+1 , . . . , un }) ⊇ U ⊥ .
(a) If {u1 , . . . , uk } is an orthogonal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthogonal basis of V , then {uk+1 , . . . , un } is an orthogonal
basis of U ⊥ .
Proof of (a) (continued). WTS Span({uk+1 , . . . , un }) = U ⊥ .
WTS Span({uk+1 , . . . , un }) ⊇ U ⊥ . Fix x ∈ U ⊥ .
(a) If {u1 , . . . , uk } is an orthogonal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthogonal basis of V , then {uk+1 , . . . , un } is an orthogonal
basis of U ⊥ .
Proof of (a) (continued). WTS Span({uk+1 , . . . , un }) = U ⊥ .
WTS Span({uk+1 , . . . , un }) ⊇ U ⊥ . Fix x ∈ U ⊥ . Then x ∈ V , and
so since {u1 , . . . , un } is an orthogonal basis of V , Theorem 2.1
guarantees that
n
P ⟨x,ui ⟩
x = ⟨ui ,ui ⟩ ui .
i=1
(a) If {u1 , . . . , uk } is an orthogonal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthogonal basis of V , then {uk+1 , . . . , un } is an orthogonal
basis of U ⊥ .
Proof of (a) (continued). WTS Span({uk+1 , . . . , un }) = U ⊥ .
WTS Span({uk+1 , . . . , un }) ⊇ U ⊥ . Fix x ∈ U ⊥ . Then x ∈ V , and
so since {u1 , . . . , un } is an orthogonal basis of V , Theorem 2.1
guarantees that
n
P ⟨x,ui ⟩
x = ⟨ui ,ui ⟩ ui .
i=1

Since x ∈ U ⊥, and since u1 , . . . , uk ∈ U, we know that ⟨x, ui ⟩ = 0

for all i ∈ {1, . . . , k}.
(a) If {u1 , . . . , uk } is an orthogonal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthogonal basis of V , then {uk+1 , . . . , un } is an orthogonal
basis of U ⊥ .
Proof of (a) (continued). WTS Span({uk+1 , . . . , un }) = U ⊥ .
WTS Span({uk+1 , . . . , un }) ⊇ U ⊥ . Fix x ∈ U ⊥ . Then x ∈ V , and
so since {u1 , . . . , un } is an orthogonal basis of V , Theorem 2.1
guarantees that
n
P ⟨x,ui ⟩
x = ⟨ui ,ui ⟩ ui .
i=1

Since x ∈ U ⊥, and since u1 , . . . , uk ∈ U, we know that ⟨x, ui ⟩ = 0

for all i ∈ {1, . . . , k}. Consequently,
n
P ⟨x,ui ⟩
x = ⟨ui ,ui ⟩ ui .
i=k+1
(a) If {u1 , . . . , uk } is an orthogonal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthogonal basis of V , then {uk+1 , . . . , un } is an orthogonal
basis of U ⊥ .
Proof of (a) (continued). WTS Span({uk+1 , . . . , un }) = U ⊥ .
WTS Span({uk+1 , . . . , un }) ⊇ U ⊥ . Fix x ∈ U ⊥ . Then x ∈ V , and
so since {u1 , . . . , un } is an orthogonal basis of V , Theorem 2.1
guarantees that
n
P ⟨x,ui ⟩
x = ⟨ui ,ui ⟩ ui .
i=1

Since x ∈ U ⊥, and since u1 , . . . , uk ∈ U, we know that ⟨x, ui ⟩ = 0

for all i ∈ {1, . . . , k}. Consequently,
n
P ⟨x,ui ⟩
x = ⟨ui ,ui ⟩ ui .
i=k+1

Thus, x is a linear combination of the vectors uk+1 , . . . , un , and

so x ∈ Span(uk+1 , . . . , un ).
(a) If {u1 , . . . , uk } is an orthogonal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthogonal basis of V , then {uk+1 , . . . , un } is an orthogonal
basis of U ⊥ .
Proof of (a) (continued). WTS Span({uk+1 , . . . , un }) = U ⊥ .
WTS Span({uk+1 , . . . , un }) ⊇ U ⊥ . Fix x ∈ U ⊥ . Then x ∈ V , and
so since {u1 , . . . , un } is an orthogonal basis of V , Theorem 2.1
guarantees that
n
P ⟨x,ui ⟩
x = ⟨ui ,ui ⟩ ui .
i=1

Since x ∈ U ⊥, and since u1 , . . . , uk ∈ U, we know that ⟨x, ui ⟩ = 0

for all i ∈ {1, . . . , k}. Consequently,
n
P ⟨x,ui ⟩
x = ⟨ui ,ui ⟩ ui .
i=k+1

Thus, x is a linear combination of the vectors uk+1 , . . . , un , and

so x ∈ Span(uk+1 , . . . , un ). Thus, Span({uk+1 , . . . , un }) ⊇ U ⊥ .
(a) If {u1 , . . . , uk } is an orthogonal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthogonal basis of V , then {uk+1 , . . . , un } is an orthogonal
basis of U ⊥ .
Proof of (a) (continued). WTS Span({uk+1 , . . . , un }) = U ⊥ .
WTS Span({uk+1 , . . . , un }) ⊆ U ⊥ .
(a) If {u1 , . . . , uk } is an orthogonal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthogonal basis of V , then {uk+1 , . . . , un } is an orthogonal
basis of U ⊥ .
Proof of (a) (continued). WTS Span({uk+1 , . . . , un }) = U ⊥ .
WTS Span({uk+1 , . . . , un }) ⊆ U ⊥ . Fix x ∈ Span(uk+1 , . . . , un );
WTS x ∈ U ⊥ .
(a) If {u1 , . . . , uk } is an orthogonal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthogonal basis of V , then {uk+1 , . . . , un } is an orthogonal
basis of U ⊥ .
Proof of (a) (continued). WTS Span({uk+1 , . . . , un }) = U ⊥ .
WTS Span({uk+1 , . . . , un }) ⊆ U ⊥ . Fix x ∈ Span(uk+1 , . . . , un );
WTS x ∈ U ⊥ . Fix scalars αk+1 , . . . , αn such that
x = αk+1 uk+1 + · · · + αn un .
(a) If {u1 , . . . , uk } is an orthogonal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthogonal basis of V , then {uk+1 , . . . , un } is an orthogonal
basis of U ⊥ .
Proof of (a) (continued). WTS Span({uk+1 , . . . , un }) = U ⊥ .
WTS Span({uk+1 , . . . , un }) ⊆ U ⊥ . Fix x ∈ Span(uk+1 , . . . , un );
WTS x ∈ U ⊥ . Fix scalars αk+1 , . . . , αn such that
x = αk+1 uk+1 + · · · + αn un .

Fix u ∈ U; WTS x ⊥ u.
(a) If {u1 , . . . , uk } is an orthogonal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthogonal basis of V , then {uk+1 , . . . , un } is an orthogonal
basis of U ⊥ .
Proof of (a) (continued). WTS Span({uk+1 , . . . , un }) = U ⊥ .
WTS Span({uk+1 , . . . , un }) ⊆ U ⊥ . Fix x ∈ Span(uk+1 , . . . , un );
WTS x ∈ U ⊥ . Fix scalars αk+1 , . . . , αn such that
x = αk+1 uk+1 + · · · + αn un .

Fix u ∈ U; WTS x ⊥ u. Since {u1 , . . . , uk } is a basis of U, there

exist scalars α1 , . . . , αk such that u = α1 u1 + · · · + αk uk .
(a) If {u1 , . . . , uk } is an orthogonal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthogonal basis of V , then {uk+1 , . . . , un } is an orthogonal
basis of U ⊥ .
Proof of (a) (continued). WTS Span({uk+1 , . . . , un }) = U ⊥ .
WTS Span({uk+1 , . . . , un }) ⊆ U ⊥ . Fix x ∈ Span(uk+1 , . . . , un );
WTS x ∈ U ⊥ . Fix scalars αk+1 , . . . , αn such that
x = αk+1 uk+1 + · · · + αn un .

Fix u ∈ U; WTS x ⊥ u. Since {u1 , . . . , uk } is a basis of U, there

exist scalars α1 , . . . , αk such that u = α1 u1 + · · · + αk uk . If V is a
vector space over C (it’s similar for R), then we have that
⟨x, u⟩ = ⟨αk+1 uk+1 + · · · + αn un , α1 u1 + · · · + αk uk ⟩
n k (∗)
αi αj ⟨ui , uj ⟩ = 0,
P P
=
i=k+1 j=1

where (*) follows from the orthogonality of {u1 , . . . , un }. So,

x ⊥ u, and consequently, x ∈ U ⊥ . This proves (a).
(b) If {u1 , . . . , uk } is an orthonormal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthonormal basis of V , then {uk+1 , . . . , un } is an
orthonormal basis of U ⊥ .
Proof of (b). Analogous to (a).
(f) U ∩ U ⊥ = {0}.
Proof of (f). By hypothesis, U is a subspace of V , and by
Proposition 3.1(a), U ⊥ is also a subspace of V . So, both U and
U ⊥ contain 0, i.e. {0} ⊆ U ∩ U ⊥ . Now, fix any u ∈ U ∩ U ⊥ ; we
must show that u = 0. Since u ∈ U and u ∈ U ⊥ , we have that
u ⊥ u, i.e. ⟨u, u⟩ = 0. But then by the definition of a scalar
product, we have that u = 0. This proves (f).
(c) dim(U) + dim(U ⊥ ) = dim(V );
(d) V = U + U ⊥ ;
(e) (U ⊥ )⊥ = U.
Proof.
(c) dim(U) + dim(U ⊥ ) = dim(V );
(d) V = U + U ⊥ ;
(e) (U ⊥ )⊥ = U.
Proof. First, since V is finite-dimensional, so is U. So, by
Corollary 2.5(a), U has an orthogonal basis {u1 , . . . , uk }. By
Corollary 2.5(b), {u1 , . . . , uk } can be extended to an orthogonal
basis {u1 , . . . , uk , uk+1 , . . . , un } of V . By (a), {uk+1 , . . . , un } is
an orthogonal basis of U ⊥ .
(c) dim(U) + dim(U ⊥ ) = dim(V ).
Proof of (c). Reminder: {u1 , . . . , uk , uk+1 , . . . , un } is an
orthogonal basis of V ; {u1 , . . . , uk } is an orthogonal basis of U;
{uk+1 , . . . , un } is an orthogonal basis of U ⊥ .
(c) dim(U) + dim(U ⊥ ) = dim(V ).
Proof of (c). Reminder: {u1 , . . . , uk , uk+1 , . . . , un } is an
orthogonal basis of V ; {u1 , . . . , uk } is an orthogonal basis of U;
{uk+1 , . . . , un } is an orthogonal basis of U ⊥ .
Note that
dim(V ) = n;
dim(U) = k;
dim(U ⊥ ) = n − k.
(c) dim(U) + dim(U ⊥ ) = dim(V ).
Proof of (c). Reminder: {u1 , . . . , uk , uk+1 , . . . , un } is an
orthogonal basis of V ; {u1 , . . . , uk } is an orthogonal basis of U;
{uk+1 , . . . , un } is an orthogonal basis of U ⊥ .
Note that
dim(V ) = n;
dim(U) = k;
dim(U ⊥ ) = n − k.

So, dim(U) + dim(U ⊥ ) = k + (n − k) = n = dim(V ). This

proves (c).
(d) V = U + U ⊥ .
Proof of (d). Reminder: {u1 , . . . , uk , uk+1 , . . . , un } is an
orthogonal basis of V ; {u1 , . . . , uk } is an orthogonal basis of U;
{uk+1 , . . . , un } is an orthogonal basis of U ⊥ .
(d) V = U + U ⊥ .
Proof of (d). Reminder: {u1 , . . . , uk , uk+1 , . . . , un } is an
orthogonal basis of V ; {u1 , . . . , uk } is an orthogonal basis of U;
{uk+1 , . . . , un } is an orthogonal basis of U ⊥ .
It is clear that U + U ⊥ ⊆ V , and we need only show that
V ⊆ U + U ⊥.
(d) V = U + U ⊥ .
Proof of (d). Reminder: {u1 , . . . , uk , uk+1 , . . . , un } is an
orthogonal basis of V ; {u1 , . . . , uk } is an orthogonal basis of U;
{uk+1 , . . . , un } is an orthogonal basis of U ⊥ .
It is clear that U + U ⊥ ⊆ V , and we need only show that
V ⊆ U + U ⊥ . Fix v ∈ V .
(d) V = U + U ⊥ .
Proof of (d). Reminder: {u1 , . . . , uk , uk+1 , . . . , un } is an
orthogonal basis of V ; {u1 , . . . , uk } is an orthogonal basis of U;
{uk+1 , . . . , un } is an orthogonal basis of U ⊥ .
It is clear that U + U ⊥ ⊆ V , and we need only show that
V ⊆ U + U ⊥ . Fix v ∈ V . Since {u1 , . . . , uk , uk+1 , . . . , un } is a
basis of V , we know that there exist scalars α1 , . . . , αn such that
v = α1 u1 + · · · + αn un .
(d) V = U + U ⊥ .
Proof of (d). Reminder: {u1 , . . . , uk , uk+1 , . . . , un } is an
orthogonal basis of V ; {u1 , . . . , uk } is an orthogonal basis of U;
{uk+1 , . . . , un } is an orthogonal basis of U ⊥ .
It is clear that U + U ⊥ ⊆ V , and we need only show that
V ⊆ U + U ⊥ . Fix v ∈ V . Since {u1 , . . . , uk , uk+1 , . . . , un } is a
basis of V , we know that there exist scalars α1 , . . . , αn such that
v = α1 u1 + · · · + αn un .
Set v1 := α1 u1 + · · · + αk uk and v2 := αk+1 uk+1 + · · · + αn un .
(d) V = U + U ⊥ .
Proof of (d). Reminder: {u1 , . . . , uk , uk+1 , . . . , un } is an
orthogonal basis of V ; {u1 , . . . , uk } is an orthogonal basis of U;
{uk+1 , . . . , un } is an orthogonal basis of U ⊥ .
It is clear that U + U ⊥ ⊆ V , and we need only show that
V ⊆ U + U ⊥ . Fix v ∈ V . Since {u1 , . . . , uk , uk+1 , . . . , un } is a
basis of V , we know that there exist scalars α1 , . . . , αn such that
v = α1 u1 + · · · + αn un .
Set v1 := α1 u1 + · · · + αk uk and v2 := αk+1 uk+1 + · · · + αn un .
Then v = v1 + v2 .
(d) V = U + U ⊥ .
Proof of (d). Reminder: {u1 , . . . , uk , uk+1 , . . . , un } is an
orthogonal basis of V ; {u1 , . . . , uk } is an orthogonal basis of U;
{uk+1 , . . . , un } is an orthogonal basis of U ⊥ .
It is clear that U + U ⊥ ⊆ V , and we need only show that
V ⊆ U + U ⊥ . Fix v ∈ V . Since {u1 , . . . , uk , uk+1 , . . . , un } is a
basis of V , we know that there exist scalars α1 , . . . , αn such that
v = α1 u1 + · · · + αn un .
Set v1 := α1 u1 + · · · + αk uk and v2 := αk+1 uk+1 + · · · + αn un .
Then v = v1 + v2 . Since {u1 , . . . , uk } is a basis of U, we see that
v1 ∈ U, and since {uk+1 , . . . , un } is a basis of U ⊥ , we see that
v2 ∈ U ⊥ .
(d) V = U + U ⊥ .
Proof of (d). Reminder: {u1 , . . . , uk , uk+1 , . . . , un } is an
orthogonal basis of V ; {u1 , . . . , uk } is an orthogonal basis of U;
{uk+1 , . . . , un } is an orthogonal basis of U ⊥ .
It is clear that U + U ⊥ ⊆ V , and we need only show that
V ⊆ U + U ⊥ . Fix v ∈ V . Since {u1 , . . . , uk , uk+1 , . . . , un } is a
basis of V , we know that there exist scalars α1 , . . . , αn such that
v = α1 u1 + · · · + αn un .
Set v1 := α1 u1 + · · · + αk uk and v2 := αk+1 uk+1 + · · · + αn un .
Then v = v1 + v2 . Since {u1 , . . . , uk } is a basis of U, we see that
v1 ∈ U, and since {uk+1 , . . . , un } is a basis of U ⊥ , we see that
v2 ∈ U ⊥ . So, v = v1 + v2 belongs to U + U ⊥ , and it follows that
V ⊆ U + U ⊥ . This proves (d).
(a) If {u1 , . . . , uk } is an orthogonal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthogonal basis of V , then {uk+1 , . . . , un } is an orthogonal
basis of U ⊥ .
(e) (U ⊥ )⊥ = U.
Proof of (e). Reminder: {u1 , . . . , uk , uk+1 , . . . , un } is an
orthogonal basis of V ; {u1 , . . . , uk } is an orthogonal basis of U;
{uk+1 , . . . , un } is an orthogonal basis of U ⊥ .
(a) If {u1 , . . . , uk } is an orthogonal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthogonal basis of V , then {uk+1 , . . . , un } is an orthogonal
basis of U ⊥ .
(e) (U ⊥ )⊥ = U.
Proof of (e). Reminder: {u1 , . . . , uk , uk+1 , . . . , un } is an
orthogonal basis of V ; {u1 , . . . , uk } is an orthogonal basis of U;
{uk+1 , . . . , un } is an orthogonal basis of U ⊥ .
We know that {uk+1 , . . . , un } is an orthogonal basis of U ⊥ , and
that {u1 , . . . , uk , uk+1 , . . . , un } is an orthogonal basis of V
extending {uk+1 , . . . , un }.
(a) If {u1 , . . . , uk } is an orthogonal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthogonal basis of V , then {uk+1 , . . . , un } is an orthogonal
basis of U ⊥ .
(e) (U ⊥ )⊥ = U.
Proof of (e). Reminder: {u1 , . . . , uk , uk+1 , . . . , un } is an
orthogonal basis of V ; {u1 , . . . , uk } is an orthogonal basis of U;
{uk+1 , . . . , un } is an orthogonal basis of U ⊥ .
We know that {uk+1 , . . . , un } is an orthogonal basis of U ⊥ , and
that {u1 , . . . , uk , uk+1 , . . . , un } is an orthogonal basis of V
extending {uk+1 , . . . , un }. So, by (a), {u1 , . . . , uk } is an
orthogonal basis of (U ⊥ )⊥ .
(a) If {u1 , . . . , uk } is an orthogonal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthogonal basis of V , then {uk+1 , . . . , un } is an orthogonal
basis of U ⊥ .
(e) (U ⊥ )⊥ = U.
Proof of (e). Reminder: {u1 , . . . , uk , uk+1 , . . . , un } is an
orthogonal basis of V ; {u1 , . . . , uk } is an orthogonal basis of U;
{uk+1 , . . . , un } is an orthogonal basis of U ⊥ .
We know that {uk+1 , . . . , un } is an orthogonal basis of U ⊥ , and
that {u1 , . . . , uk , uk+1 , . . . , un } is an orthogonal basis of V
extending {uk+1 , . . . , un }. So, by (a), {u1 , . . . , uk } is an
orthogonal basis of (U ⊥ )⊥ . But by construction, {u1 , . . . , uk } is
an orthogonal basis of U.
(a) If {u1 , . . . , uk } is an orthogonal basis of U, and
{u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an
orthogonal basis of V , then {uk+1 , . . . , un } is an orthogonal
basis of U ⊥ .
(e) (U ⊥ )⊥ = U.
Proof of (e). Reminder: {u1 , . . . , uk , uk+1 , . . . , un } is an
orthogonal basis of V ; {u1 , . . . , uk } is an orthogonal basis of U;
{uk+1 , . . . , un } is an orthogonal basis of U ⊥ .
We know that {uk+1 , . . . , un } is an orthogonal basis of U ⊥ , and
that {u1 , . . . , uk , uk+1 , . . . , un } is an orthogonal basis of V
extending {uk+1 , . . . , un }. So, by (a), {u1 , . . . , uk } is an
orthogonal basis of (U ⊥ )⊥ . But by construction, {u1 , . . . , uk } is
an orthogonal basis of U. So, U = Span(u1 , . . . , uk ) = (U ⊥ )⊥ .
This proves (e).