Unit 9: Vector Spaces and Inner Product Spaces

Session 22

Inner product Spaces

Contents:
Introduction, p 84
22.1 inner Product, p 84
22.2 Norm and Distance, p 86
22.3 Orthogonality, p 87
Solutions to Activities, p 91
Summary, p 93
Learning Outcomes, p 93

Introduction

In this session we discuss vector spaces having an additional kind of structure

called an inner product, which generalizes the idea of the dot product of vectors
in Rn.

22.1 Inner product

Definition

Let 𝑉 be a vector space over a field 𝔽 (𝔽 may be ℝ or ℂ or any other field). An

inner product on 𝑉 is a function which assigns to each ordered pair of vectors
𝑢. 𝑣 𝜖 𝑉, a scalar 〈𝑢, 𝑣〉𝜖 𝔽 with the following properties.

〈𝑢 + 𝑤, 𝑣〉 = 〈𝑢, 𝑣〉 + 〈𝑤, 𝑣〉

and 〈𝑢, 𝑣 + 𝑤〉 = 〈𝑢, 𝑣〉 + 〈𝑢, 𝑤〉 for all 𝑢, 𝑣, 𝑤 ∈ 𝑉.

〈𝛼𝑢, 𝑣〉 = 𝛼〈𝑢, 𝑣〉 for all 𝑢, 𝑣 ∈ 𝑉 and 𝛼 ∈ 𝔽.

84
Copyright © 2018, The Open University of Sri Lanka
 Unit 9: Vector Spaces and Inner Product Spaces

〈𝑢, 𝑣〉 = ̅̅̅̅̅̅̅
〈𝑢, 𝑣〉 for all 𝑢, 𝑣 ∈ 𝑉.

〈𝑣, 𝑣〉 ≥ 0 and 〈𝑣, 𝑣〉 = 0 if and only if 𝑣 = 0.

where the bar in (iii) denotes the complex conjugate.

A vector space 𝑉 with inner product is called an inner product space. In

particular a finite dimensional real inner product space is called a Euclidean
space and a complex inner product space is called a unitary space.

Examples

1 .We know that ℝ3 is a vector space over ℝ with the usual addition and
multiplication by a scalar. For 𝑢 = (𝑢1 , 𝑢2 , 𝑢3 ) and 𝑣 = (𝑣1 , 𝑣2 , 𝑣3 ) ∈ ℝ3
define 〈𝑢. 𝑣〉 = 𝑢 ∙ 𝑣 (the dot product). Then it can be easily shown that it is an
inner product on ℝ3 .

2. For 𝑢, 𝑣 ∈ ℂ𝑛 , 〈𝑢, 𝑣〉 = 𝑢 ∙ 𝑣̅ is an inner product.

For e.g. let 𝑢 = (1 + 𝑖, −3, 4 − 3𝑖) and = (2 − 𝑖, −𝑖, 2 + 𝑖) ∈ ℂ3 , ℂ3 is a vector

space over ℂ. Then

〈𝑢, 𝑣〉 = 𝑢 ∙ 𝑣̅ = (1 + 𝑖, −3, 4 − 3𝑖) ∙ (2 + 𝑖, 𝑖, 2 − 𝑖)

= (1 + 𝑖)(2 + 𝑖) + (−3𝑖) + (4 − 3𝑖)(2 − 𝑖).

3. For 𝑎 = 𝑎0 + 𝑎1 𝑥 + 𝑎2 𝑥 2 and 𝑏 = 𝑏0 + 𝑏1 𝑥 + 𝑏2 𝑥 2 in the vector space of all

polynomials of degree 2 or less over ℝ (i.e. in 𝑃2 (𝑥)), 〈𝑎, 𝑏〉 is defined by
〈𝑎, 𝑏〉 = 𝑎0 𝑏0 + 𝑎1 𝑏1 + 𝑎2 𝑏2 . 𝑃2 (𝑥) is an inner product space with the above
definition of 〈𝑎, 𝑏〉.

85
Copyright © 2018, The Open University of Sri Lanka
 Unit 9: Vector Spaces and Inner Product Spaces

4. The vector space of all polynomial functions 𝑃 with real coefficients is an

1
inner product space if 〈𝑝, 𝑞〉 = ∫0 𝑝(𝑥)𝑞(𝑥)𝑑𝑥, for 𝑝 = 𝑝(𝑥) and
𝑞 = 𝑞(𝑥) ∈ 𝑃.

5 .The vector space of all continuous functions in the interval [𝑎, 𝑏] is an inner
𝑏
product space for 〈𝑓, 𝑔〉 is defined by 〈𝑓, 𝑔〉 = ∫𝑎 𝑓(𝑥)𝑔(𝑥)𝑑𝑥

Activity 22.1

For 𝑢 = (𝑢1 , 𝑢2 ), 𝑣 = (𝑣1 , 𝑣2 ) ∈ ℝ2 define 〈𝑢, 𝑣〉 = 𝑢1 𝑣1 − 𝑢2 𝑣2 .

Is this an inner product on ℝ2 ?

22.2. Norm and Distance

The norm or the magnitude of a vector in an inner product space is defined by

‖𝑣‖ = √〈𝑣, 𝑣〉.

The distance from 𝑢 to 𝑣 in an inner product space is defined as 𝑑(𝑢, 𝑣) =

‖𝑢 − 𝑣‖.

Example 22.1

In the inner product space of all polynomial functions with real coefficients with
1
the inner product defined by 〈𝑝, 𝑞〉 = ∫0 𝑝(𝑥)𝑞(𝑥)𝑑𝑥,

Find the norm of the polynomial 𝑝(𝑥) = 2𝑥 − 1.

Compute 𝑑(𝑥 2 + 1, 𝑥 + 1), the distance from 𝑥 2 + 1 to 𝑥 + 1.

‖2𝑥 − 1‖2 = 〈2𝑥 − 1,2𝑥 − 1〉

1 1 1
= ∫0 (2𝑥 − 1)2 𝑑𝑥 = ∫0 (4𝑥 2 − 4𝑥 + 1) 𝑑𝑥 = 3

86
Copyright © 2018, The Open University of Sri Lanka
 Unit 9: Vector Spaces and Inner Product Spaces

1
∴ ‖2𝑥 − 1‖ = .
√3

𝑑(𝑥 2 + 1, 𝑥 + 1) = ‖𝑥 2 + 1 − (𝑥 + 1)‖ = ‖𝑥 2 − 𝑥‖

Now,

‖𝑥 2 − 𝑥‖2 = 〈𝑥 2 − 𝑥, 𝑥 2 − 𝑥〉

1 1 1
= ∫0 (𝑥 2 − 𝑥)2 𝑑𝑥 = ∫0 (𝑥 4 − 2𝑥 3 + 𝑥 2 )𝑑𝑥 =
30

1
∴ ‖𝑥 2 − 𝑥‖ = .
√30

22.3. Orthogonality

A vector 𝑣 such that. ‖𝑣‖ = 1 is called a normalized vector or a unit vector. If

〈𝑢, 𝑣〉 = 0, then 𝑢 and 𝑣 are said to be orthogonal (or perpendicular) to each
other.

A subset 𝑆 of 𝑉 is called an orthogonal set if the elements of 𝑆 are mutually

orthogonal. An orthogonal set 𝑆 is called an orthonormal set if each vector of 𝑆
has unit length.

Now we show that every finite dimensional inner product space has a basis
consisting of orthonormal vectors. The proof of this theorem gives a procedure
for determining an orthonormal basis from any given basis of 𝑉. In general, it
gives a procedure to obtain an orthonormal set from a given set of linearly
independent vectors.

87
Copyright © 2018, The Open University of Sri Lanka
 Unit 9: Vector Spaces and Inner Product Spaces

Theorem 22.1(Gram-Schmidt Orthogonalization procedure)

Every finite dimensional inner product space has a basis

consisting of orthogonal (orthonormal) vectors, obtained by the
following way. If {𝑣1 , 𝑣2 , . . . , 𝑣𝑛 } is a basis of an inner product
space V over F, then the set of vectors 𝑢1 , 𝑢2 , . . . , 𝑢𝑛 defined by

𝑢1 = 𝑣1

〈𝑣2 ,𝑢1 〉
𝑢2 = 𝑣2 − ‖𝑢1 ‖2
𝑢1

〈𝑣3 ,𝑢2 〉 〈𝑣3 ,𝑢1 〉

𝑢3 = 𝑣3 − ‖𝑢2 ‖2
𝑢2 − ‖𝑢1 ‖2
𝑢1

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

〈𝑣𝑛 ,𝑢𝑛−1 〉 〈𝑣𝑛 ,𝑢1 〉

𝑢𝑛 = 𝑣𝑛 − ‖𝑢𝑛−1 ‖2
𝑢𝑛−1 − . . . − ‖𝑢1 ‖2

is an orthogonal set.

Proof: By induction.

𝑛 = 2,

Consider

𝑢1 = 𝑣1

〈𝑣2 , 𝑢1 〉
𝑢2 = 𝑣2 − 𝑢
‖𝑢1 ‖2 1

〈𝑣2 ,𝑢1 〉
then 〈𝑢2 , 𝑢1 〉 = 〈𝑣2 , 𝑢1 〉 − ‖𝑢1 ‖2
〈𝑢1 , 𝑢1 〉 = 0.

Hence {𝑢1 , 𝑢2 } is an orthogonal set.

𝑛 > 2:

88
Copyright © 2018, The Open University of Sri Lanka
 Unit 9: Vector Spaces and Inner Product Spaces

Let 𝑚 < 𝑛

Suppose {𝑢1 , 𝑢2 , . . . , 𝑢𝑚−1 } is an orthogonal set.

To show that {𝑢1 , 𝑢2 , . . . , 𝑢𝑚 } is an orthogonal set, we just have to show that

〈𝑢𝑚 , 𝑢𝑖 〉 = 0 for 𝑖 = 1, 2, … , 𝑚 − 1, as

〈𝑢𝑖 , 𝑢𝑗 〉 = 0 for 𝑖 ≠ 𝑗, 𝑖, 𝑗 = 1, 2, … , 𝑚 − 1 (inductive hypothesis)

Now

〈𝑣𝑚 , 𝑢𝑚−1 〉 〈𝑣𝑚 , 𝑢1 〉

〈𝑢𝑚 , 𝑢𝑖 〉 = 〈𝑣𝑚 , 𝑢𝑖 〉 − 〈𝑢𝑚−1 , 𝑢𝑖 〉 − ⋯ − 〈𝑢1 , 𝑢𝑖 〉
‖𝑢𝑚−1 ‖2 ‖𝑢1 ‖2

〈𝑣𝑚 ,𝑢𝑖 〉
= 〈𝑣𝑚 , 𝑢𝑖 〉 − ‖𝑢𝑖 ‖2
〈𝑢𝑖 , 𝑢𝑖 〉

(By inductive hypothesis 〈𝑢𝑘 , 𝑢𝑖 〉 = 0 for 𝑘 ≠ 𝑖, and 𝑘 = 1, 2, … , 𝑚 − 1).

= 0, as 〈𝑢𝑖 , 𝑢𝑖 〉 = ‖𝑢𝑖 ‖2 .

∴ {𝑢1 , 𝑢2 , . . . , 𝑢𝑚 } is an orthogonal set.

∴ By induction the result is true for all 𝑛.

Furthermore, we can show that this set is linearly independent.

Consider the expression 𝛼1 𝑢1 + 𝛼2 𝑢2 + ⋯ + 𝛼𝑛 𝑢𝑛 = 0 , 𝛼𝑖 ∈ 𝔽.

Then 〈𝛼1 𝑢1 + 𝛼2 𝑢2 + ⋯ + 𝛼𝑛 𝑢𝑛 , 𝑢𝑘 〉 = 〈0, 𝑢𝑘 〉 = 0

But

〈𝛼1 𝑢1 + 𝛼2 𝑢2 + ⋯ + 𝛼𝑛 𝑢𝑛 , 𝑢𝑘 〉 = 𝛼1 〈𝑢1 , 𝑢𝑘 〉 + 𝛼2 〈𝑢2 , 𝑢𝑘 〉 + ⋯ + 𝛼𝑛 〈𝑢𝑛 , 𝑢𝑘 〉

= 𝛼𝑘 〈𝑢𝑘 , 𝑢𝑘 〉, ∵ 〈𝑢𝑖 , 𝑢𝑘 〉 = 0 for 𝑖 ≠ 𝑘.

Therefore, we have 𝛼𝑘 〈𝑢𝑘 , 𝑢𝑘 〉 = 0 ⟹ 𝛼𝑘 = 0, ∵ ‖𝑢𝑘 ‖2 > 0.

89
Copyright © 2018, The Open University of Sri Lanka
 Unit 9: Vector Spaces and Inner Product Spaces

This is true for all 𝑘 = 1, 2, … , 𝑛. Hence the set {𝑢1 , 𝑢2 , . . . , 𝑢𝑛 } is linearly

𝑢
independent. Therefore, it is a basis of 𝑉 (orthogonal basis). Letting 𝑤𝑖 = ‖𝑢𝑖 ‖
𝑖

for 𝑖 = 1, 2, … , 𝑛, we get an orthonormal basis for 𝑉.

Note:

Any linearly independent set of vectors (not necessarily be a basis) can be

transformed into an orthogonal set.

Example 22.2

Using Gram-Schmidt process, we obtain an orthonormal basis from the basis

{(1,1,1), (1,1,0), (1,0,0)} of ℝ3 . (In this case the inner product is the dot
product of the two vectors). Let us denote this set by {𝑣1 , 𝑣2 , 𝑣3 }.

Then 𝑢1 = 𝑣1 = (1, 1, 1) and ‖𝑢1 ‖2 = 3.

〈𝑣2 , 𝑢1 〉 {(1, 1, 0) ∙ (1, 1, 1)}

𝑢2 = 𝑣2 − 𝑢1 = (1, 1, 0) − (1, 1, 1)
‖𝑢1 ‖2 3

2 1 1 2
= (1, 1, 0) − 3 (1, 1, 1) = (3 , 3 , − 3)

As any scalar multiple of this vector is normal to 𝑢1 , for convenience we

take 𝑢2 = (1, 1, −2).

Then ‖𝑢2 ‖2 = 6.

Now,

〈𝑣3 ,𝑢2 〉 〈𝑣3 ,𝑢1 〉

𝑢3 = 𝑣3 − 𝑢2 − 𝑢1
‖𝑢2 ‖2 ‖𝑢1 ‖2

[(1,0,0)∙(1,1,−2)] [(1,0,0)∙(1,1,1)]
= (1, 0, 0) − (1, 1, −2) − (1, 1, 1)
6 3

1 1 1 1
= (1, 0, 0) − 6 (1, 1, −2) − 3 (1, 1, 1) = (2 , − 2 , 0)

90
Copyright © 2018, The Open University of Sri Lanka
 Unit 9: Vector Spaces and Inner Product Spaces

Again for convenience we take 𝑢3 = (1, −1, 0). Hence

{(1, 1, 1), (1, 1, −2), (1, −1, 0)} is an orthogonal set which is also a basis for
1 1 1
ℝ3 and the set { (1, 1, 1), (1, 1, −2), (1, −1, 0)} is an orthonormal
√3 √6 √2

basis of ℝ3 .

Activity 22.2

Compute an orthonormal basis taking

𝑣1 = (1, 0, 0) 𝑣2 = (1, 1, 0) 𝑣3 = (1, 1, 1)

(Use the dot product as the inner product)

Solutions of Activities

Activity 22.1

𝑢 = (𝑢1 , 𝑢2 ) 𝑣 = (𝑣1 , 𝑣2 )

〈𝑢, 𝑣〉 = 𝑢1 𝑣1 − 𝑢2 𝑣2

Let 𝑢, 𝑣, 𝑤 ∈ ℝ2

1. 〈𝑢 + 𝑤, 𝑣〉 = 〈(𝑢1 + 𝑤1 , 𝑢2 + 𝑤2 ), (𝑣1 , 𝑣2 )〉

= (𝑢1 + 𝑤1 )𝑣1 - (𝑢2 + 𝑤2 ) 𝑣2

= 𝑢1 𝑣1 + 𝑤1 𝑣1 − 𝑢2 𝑣2 − 𝑤2 𝑣2

= 𝑢1 𝑣1 − 𝑢2 𝑣2 + 𝑤1 𝑣1 − 𝑤2 𝑣2

= 〈𝑢, 𝑣〉 + 〈𝑤, 𝑣〉

〈𝑢, 𝑣 + 𝑤〉 = 〈(𝑢1 , 𝑢2 ), (𝑣1 + 𝑤1 , 𝑣2 + 𝑤2 )〉

91
Copyright © 2018, The Open University of Sri Lanka
 Unit 9: Vector Spaces and Inner Product Spaces

= 𝑢1 (𝑣1 + 𝑤1 ) − 𝑢2 (𝑣2 + 𝑤2 )

= 𝑢1 𝑣1 + 𝑤1 𝑢1 − 𝑢2 𝑣2 − 𝑢2 𝑤2

= 𝑢1 𝑣1 − 𝑢2 𝑣2 + 𝑢1 𝑤1 − 𝑢2 𝑤2

= 〈𝑢, 𝑣〉 + 〈𝑢, 𝑤〉

2. 〈𝛼𝑢, 𝑣〉 = 〈(𝛼𝑢1 , 𝛼𝑢2 ), (𝑣1 , 𝑣2 ) 〉

= 𝛼𝑢1 𝑣1 − 𝛼𝑢2 𝑣2

= 𝛼 (𝑢1 𝑣1 − 𝑢2 𝑣2 )

= 𝛼 (𝑢, 𝑣)

3. Since 𝑢, 𝑣 ∈ ℝ2

〈𝑢, − 𝑣〉 = 〈𝑣, 𝑢 〉

4. 〈𝑣, 𝑣〉 = 𝑣1 2 − 𝑣2 2

Suppose𝑣 = (2, 3). Then 〈𝑣, 𝑣〉 = 4 − 9 = −5 ≯ 0

Therefore 〈𝑢, 𝑣〉 is not an inner product.

Activity 22.2

2
𝑢1 = 𝑣1 = (1, 0, 0) and ‖𝑢1 ‖ = 1

〈𝑣2 , 𝑢1 〉
𝑢2 = 𝑣2 −
‖𝑢1 ‖2

(1, 1, 0). (1, 0 ,0)

𝑢1 = (1, 1, 0) −
1

1
= (1, 1, 0) − (1, 0, 0)
1

= (0, 1, 0)

Then ‖𝑢2 ‖2 = 1

〈 𝑣3 , 𝑢2 〉 〈𝑣3 , 𝑢1 〉
𝑢3 = 𝑣3 − 𝑢2 − 𝑢1
‖𝑢2 ‖2 ‖𝑢1 ‖2

92
Copyright © 2018, The Open University of Sri Lanka
 Unit 9: Vector Spaces and Inner Product Spaces

(1,1,1).(0,1,0) (1,1,1). (1,0 ,0)

= (1, 1,1) − (0, 1, 0) − (1, 0, 0)
1 1

= (1, 1, 1) − (0, 1, 0) − (1, 0, 0)

= (0 ,1, 1)

Hence {(1, 0, 0), (0, 1, 0), (0, 0, 1)} is an orthogonal set which is also a basis for ℝ3 . Since all
three vectors are normal the above set is an orthonormal basis for ℝ3

Summary

This session concludes with the student gaining knowledge in inner product
spaces and unitary spaces. Also, we have introduced norm and distance of a
vector and orthogonality of vectors.

Learning Outcomes

At the end of the session, students will be able to

 Define an inner product space and identify its properties

 Compute orthogonal bases and orthonormal bases using Gram-Schmidt
process.

93
Copyright © 2018, The Open University of Sri Lanka