Inner Product Spaces

Dr. Juhi Kesarwani Dr. Ashish Kumar Kesarwany

Mathematics Division, School of Advanced Sciences and Languages

VIT Bhopal University, Bhopal-Indore Highway, Kothrikalan, Sehore, Madhya Pradesh, 466114, India

July 31, 2024

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 1 / 61
 Table of Contents

1 Dot Product

2 Inner Product Space

3 Matrix representation of inner product

4 Orthogonal Projection

5 The Gram-Schmidt orthogonalization process

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 2 / 61
 Dot Product

Definition (Dot product)

For x, y ∈ R3 , the dot product of x and y , denoted by x · y , is defined by

x · y = x1 y1 + x2 y2 + x3 y3 ,
   
x1 x2
where y1  and y2 .
z1 z2

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 3 / 61
 Dot Product

Definition (Dot product)

For x, y ∈ R3 , the dot product of x and y , denoted by x · y , is defined by

x · y = x1 y1 + x2 y2 + x3 y3 ,
   
x1 x2
where y1  and y2 .
z1 z2

Example
   
2 −1
Consider x = 1 and y =  1  then
1 1

x · y = −2 + 1 + 1 = 0

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 3 / 61
 Properties of dot product
The dot product on R3 has the following properties.
❃ x · y = y · x.
❃ x · (y + z) = x · y + x · z.
❃ (αx) · y = α(x · y )
 
0
❃ x · x ≥ 0 and x · x = 0 if and only if x = 0.
0
for all x, y ∈ R3 and α ∈ R.

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 4 / 61
 Properties of dot product
The dot product on R3 has the following properties.
❃ x · y = y · x.
❃ x · (y + z) = x · y + x · z.
❃ (αx) · y = α(x · y )
 
0
❃ x · x ≥ 0 and x · x = 0 if and only if x = 0.
0
for all x, y ∈ R3 and α ∈ R.

Length of a vector
For all x ∈ R3 , the length of x is denoted by |x| and is defined as
√
|x| = x · x

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 4 / 61
 Properties of dot product
The dot product on R3 has the following properties.
❃ x · y = y · x.
❃ x · (y + z) = x · y + x · z.
❃ (αx) · y = α(x · y )
 
0
❃ x · x ≥ 0 and x · x = 0 if and only if x = 0.
0
for all x, y ∈ R3 and α ∈ R.

Length of a vector
For all x ∈ R3 , the length of x is denoted by |x| and is defined as
√
|x| = x · x

Remark
If we think of a vector as a point instead of as an arrow, then |x| should be
interpreted to mean the distance from the origin to the point x.
Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 4 / 61
 Angle between two vectors
Let x, y ∈ R3 be two non-zero vectors then angle θ between these two vectors is
given by
x ·y
cos θ =
|x||y |
Thus, from here, we get another definition of dot product, i.e.,

x · y = |x||y | cos θ.

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 5 / 61
 Angle between two vectors
Let x, y ∈ R3 be two non-zero vectors then angle θ between these two vectors is
given by
x ·y
cos θ =
|x||y |
Thus, from here, we get another definition of dot product, i.e.,

x · y = |x||y | cos θ.

Example
   
1 1
Let x = 2 and y = 1, then x · y = 1 · 1 + 2 · 1 + 3 · 1 = 6,
3 1
p √ p √
|x| = 12 + 22 + 32 = 12 + 1 2 + 1 2 = 3
14 and |y | =
!
x ·y 6 −1 6
cos θ = =√ =⇒ θ = cos √
|x||y | 42 42
Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 5 / 61
 Inner Product Space

Definition (Inner Product Space)

Let V be a real vector space then an inner product on V is a function that associates
a real number ⟨x, y ⟩ to each pair of vectors x and y in V such a way that the
following rules are satisfied for all x, y , z ∈ V and α ∈ R:

➀ ⟨x, y ⟩ = ⟨y , x⟩.

➁ ⟨x + y , z⟩ = ⟨x, z⟩ + ⟨y , z⟩.

➂ ⟨αx, y ⟩ = α⟨x, y ⟩.

➃ ⟨x, x⟩ ≥ 0 and ⟨x, x⟩ = 0 iff x = 0.

A pair (V , ⟨ ⟩) of a vector space V and an inner product ⟨ ⟩ is called a inner product

space. In particular, the pair (Rn , ·) is called the Euclidean n-space.

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 6 / 61
 Example
   
x1 y1
Consider the vector space R3 (R). For x = x2  and y = y2  in R3 define
x3 y3

⟨x, y ⟩ = x1 y1 + x2 y2 + x3 y3 .

Then ⟨, ⟩ is an inner product on R3 .

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 7 / 61
 Example
   
x1 y1
Consider the vector space R3 (R). For x = x2  and y = y2  in R3 define
x3 y3

⟨x, y ⟩ = x1 y1 + x2 y2 + x3 y3 .

Then ⟨, ⟩ is an inner product on R3 .

Example
Let V = C [0, 1] be the vector space of all real-valued continuous functions on [0, 1].
For any two functions f (x) and g (x) in V , define
Z 1
⟨f , g ⟩ = f (x)g (x)dx.
0

Then ⟨, ⟩ is an inner product on V .

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 7 / 61
 Definition(Magnitude or Length or Norm)
Let (V , ⟨ ⟩) be an inner product space. By the fourth axiom of an inner product,
⟨u, u⟩ is non-negative for any vector u ∈ V . Thus, its positive square root exists.
We use the notation p
∥u∥ = ⟨u, u⟩.
This non-negative number is called the norm or length or magnitude of u. The
relation ∥u∥2 = ⟨u, u⟩ will be used frequently.

The distance between two vectors u and v , denoted by d(u, v ), is defined as

d(u, v ) = ∥u − v ∥.

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 8 / 61
 Definition(Magnitude or Length or Norm)
Let (V , ⟨ ⟩) be an inner product space. By the fourth axiom of an inner product,
⟨u, u⟩ is non-negative for any vector u ∈ V . Thus, its positive square root exists.
We use the notation p
∥u∥ = ⟨u, u⟩.
This non-negative number is called the norm or length or magnitude of u. The
relation ∥u∥2 = ⟨u, u⟩ will be used frequently.

The distance between two vectors u and v , denoted by d(u, v ), is defined as

d(u, v ) = ∥u − v ∥.

Remark
If ∥u∥ = 1 or, equivalently, if ⟨u, u⟩ = 1, then u is called a unit vector. Every
non-zero vector v in V can be multiplied by the reciprocal of its length to obtain
the unit vector
v
⃗v =
∥v ∥
which is a positive multiple of v . This process is called normalizing v .
Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 8 / 61
 The angle between two vectors
The real number 0 in the interval [0, π] that satisfies

⟨u, v ⟩
⟨u, v ⟩ = ∥u∥ ∥v ∥ cos θ or cos θ =
∥u∥ ∥v ∥

is called the angle between u and v .

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 9 / 61
 Example
    
x1 y
2
In R equipped with an inner product x= ,y = 1 = 2x1 y1 + 3x2 y2 , the
x2 y2
   
1 1
angle between and is computed as
2 0

⟨x, y ⟩ 2 1
cos θ = =√ =√
∥x∥ ∥y ∥ 14 × 2 7
 
1
=⇒ θ = cos−1 √
7

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 10 / 61
 Properties of length in an inner product space V
❀ ∥u∥ ≥ 0.

❀ ∥u∥ = 0 if and only if u = 0.

❀ ∥ku∥ = |k|∥u∥.

❀ ∥u + v ∥ ≤ ∥u∥ + ∥v ∥ (Triangular inequality)

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 11 / 61
 Exercise 1:
Consider the inner product space (R2 [x], ⟨ ⟩) equipped with the inner product
Z 1
⟨f , g ⟩ = f (x)g (x)dx for f , g ∈ R2 [x].
0

Find the angle between f (x) = 3x − 1 and g (x) = x 2 .

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 12 / 61
 Definition (Orthogonal Vectors)
Let (V , ⟨ ⟩) be an inner product space. Then two vectors u and v of V are said to
be orthogonal or perpendicular if ⟨u, v ⟩ = 0.

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 13 / 61
 Definition (Orthogonal Vectors)
Let (V , ⟨ ⟩) be an inner product space. Then two vectors u and v of V are said to
be orthogonal or perpendicular if ⟨u, v ⟩ = 0.

Remark
π
Note that for non-zero vectors u and v , ⟨u, v ⟩ = 0 if and only if θ = 2.

∗ The word orthogonal comes from the Greek word orthogonios, which
means right-angled.

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 13 / 61
 Example
Consider the inner product space (R3 , ⟨⟩) where ⟨ ⟩ is defined as
*   +
x1 x2
y1  , y2  = x1 x2 + y1 y2 + z1 z2
z1 z2
*   +  
1 1 1
Since 1 ,  1  = 1 · 1 + 1 · 1 + 2 · (−1) = 0 and hence vectors 1 and
  2 −1 2
1
 1  are orthogonal vectors.
−1

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 14 / 61
 Exercise 2:    
1 2
Find a non-zero vector w that is orthogonal to u1 = 2 and u2 = 5 in R3 .
1 4

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 15 / 61
 Pythagorean Theorem
Let (V , ⟨ ⟩) be an inner product space, and let x and y be any two vectors in V
with the angle θ. Then

∥x + y ∥2 = ∥x∥2 + ∥y ∥2 + 2∥x∥ ∥y ∥ cos θ.

Moreover, it deduces the Pythagorean theorem: ∥x + y ∥2 = ∥x∥2 + ∥y ∥2 for any

orthogonal vector x and y .

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 16 / 61
 Proof
Since angle between vectors x and y is θ and therefore

⟨x, y ⟩ = ∥x∥ ∥y ∥ cos θ

By definition

∥x + y ∥2 = ⟨x + y , x + y ⟩
= ⟨x, x⟩ + ⟨x, y ⟩ + ⟨y , x⟩ + ⟨y , y ⟩
= ∥x∥2 + 2⟨x, y ⟩ + ∥y ∥2
= ∥x∥2 + ∥y ∥2 + 2∥x∥ ∥y ∥ cos θ
π
If vectors x and y are orthogonal then by putting θ = 2, we get

∥x + y ∥2 = ∥x∥2 + ∥y ∥2

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 17 / 61
 Parallelogram Equality
Let (V , ⟨ ⟩) be an inner product space. Suppose u, v ∈ V . Then

∥u + v ∥2 + ∥u − v ∥2 = 2(∥u∥2 + ∥v ∥2 ).

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 18 / 61
 Parallelogram Equality
Let (V , ⟨ ⟩) be an inner product space. Suppose u, v ∈ V . Then

∥u + v ∥2 + ∥u − v ∥2 = 2(∥u∥2 + ∥v ∥2 ).

Proof
We have
∥u + v ∥2 + ∥u − v ∥2 = ⟨u + v , u + v ⟩ + ⟨u − v , u − v ⟩
= ∥u∥2 + ∥v ∥2 + ⟨u, v ⟩ + ⟨v , u⟩ + ∥u∥2 + ∥v ∥2 − ⟨u, v ⟩ − ⟨v , u⟩
= 2(∥u∥2 + ∥v ∥2 ),

as desired.

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 18 / 61
 Exercise 3:
Let (V , ⟨ ⟩) be an inner product space. Suppose u, v ∈ V are such that

∥u∥ = 3, ∥u + v ∥ = 4, ∥u − v ∥ = 6

What number must ∥v ∥ equal?

Exercise 4:
Prove that if V is a real inner-product space, then

∥u + v ∥2 − ∥u − v ∥2
⟨u, v ⟩ =
4
for all u, v ∈ V .

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 19 / 61
 Example
Let (V , ⟨ ⟩) be an inner product space then for any two vectors x and y in V

∥x∥ − ∥y ∥ ≤ ∥x − y ∥

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 20 / 61
 Example
Let (V , ⟨ ⟩) be an inner product space then for any two vectors x and y in V

∥x∥ − ∥y ∥ ≤ ∥x − y ∥

Solution

∥x∥ = ∥(x − y ) + y ∥ ≤ ∥x − y ∥ + ∥y ∥ [Triangula inequallity]

=⇒ ∥x∥ − ∥y ∥ ≤ ∥x − y ∥ (1)

Again,

∥y ∥ = ∥(y − x) + x∥ ≤ ∥y − x∥ + ∥x∥ [Triangula inequallity]

=⇒ ∥y ∥ − ∥x∥ ≤ ∥y − x∥
=⇒ −(∥x∥ − ∥y ∥) ≤ ∥x − y ∥ (2)

From (1) and (2), we have

∥x∥ − ∥y ∥ ≤ ∥x − y ∥
Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 20 / 61
 Example
Let (V , ⟨ ⟩) be an inner product space and if x1 , x2 , . . . , xk are non-zero vectors in
V that are mutually orthogonal then they are linearly independent.

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 21 / 61
 Example
Let (V , ⟨ ⟩) be an inner product space and if x1 , x2 , . . . , xk are non-zero vectors in
V that are mutually orthogonal then they are linearly independent.

Solution
Suppose α1 x1 + α2 x2 + · · · + αk xk = 0. Then for each i = 1, . . . , k,

⟨α1 x1 + α2 x2 + · · · + αk xk , xi ⟩ = ⟨0, xi ⟩ = 0
=⇒ α1 ⟨x1 , xi ⟩ + α2 ⟨x2 , xi ⟩ + · · · + αi ⟨xi , xi ⟩ + · · · + αk ⟨xk , xi ⟩ = 0
=⇒ αi ∥xi ∥2 = 0
=⇒ αi = 0, ∀ i = 1, . . . , k

This shows that the vectors x1 , x2 , . . . , xk are linearly independent.

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 21 / 61
 Matrix representation of inner product

Let (V , ⟨ ⟩) be an inner product space and let {x1 , x2 , . . . , xm } be an ordered basis

Xm m
X
of V . Then for any x = αi xi and y = βi xi , we have
i=1 i=1

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 22 / 61
 Matrix representation of inner product

Let (V , ⟨ ⟩) be an inner product space and let {x1 , x2 , . . . , xm } be an ordered basis

Xm m
X
of V . Then for any x = αi xi and y = βi xi , we have
i=1 i=1

⟨x, y ⟩ =⟨α1 x1 + α2 x2 + · · · + αm xm , β1 x1 + β2 x2 + · · · + βm xm ⟩
=α1 β1 ⟨x1 , x1 ⟩ + α1 β2 ⟨x1 , x2 ⟩ + · · · + α1 βm ⟨x1 , xm ⟩
+ α2 β1 ⟨x2 , x1 ⟩ + α2 β2 ⟨x2 , x2 ⟩ + · · · + α2 βm ⟨x2 , xm ⟩
.. .. ..
. . .
+ αm β1 ⟨xm , x1 ⟩ + αm β2 ⟨xm , x2 ⟩ + · · · + αm βm ⟨xm , xm ⟩

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 22 / 61
 This implies
  
⟨x1 , x1 ⟩ ⟨x1 , x2 ⟩ ··· ⟨x1 , xm ⟩ β1
 2 , x1 ⟩
⟨x ⟨x2 , x2 ⟩ ··· ⟨x2 , xm ⟩ 
   β2 
⟨x, y ⟩ = α1 α2 ··· αm  .
 
.. .. ..   .. 
 .. . . .  . 
⟨xm , x1 ⟩ ⟨xm , x2 ⟩ ··· ⟨xm , xm ⟩ βm
β 
1
   β2 
= α1 α2 ··· αm A .. 
.
βm

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 23 / 61
 This implies
  
⟨x1 , x1 ⟩ ⟨x1 , x2 ⟩ ··· ⟨x1 , xm ⟩ β1
 2 , x1 ⟩
⟨x ⟨x2 , x2 ⟩ ··· ⟨x2 , xm ⟩ 
   β2 
⟨x, y ⟩ = α1 α2 ··· αm  .
 
.. .. ..   .. 
 .. . . .  . 
⟨xm , x1 ⟩ ⟨xm , x2 ⟩ ··· ⟨xm , xm ⟩ βm
β 
1
   β2 
= α1 α2 ··· αm A .. 
.
βm

 
⟨x1 , x1 ⟩ ⟨x1 , x2 ⟩ ··· ⟨x1 , xm ⟩
 ⟨x2 , x1 ⟩ ⟨x2 , x2 ⟩ ··· ⟨x2 , xm ⟩ 
where A =  .  = [⟨xi , xj ⟩] is called the matrix of
 
.. .. ..
 .. . . . 
⟨xm , x1 ⟩ ⟨xm , x2 ⟩ · · · ⟨xm , xm ⟩
inner product with respect to ordered basis {x1 , x2 , . . . , xm }.

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 23 / 61
 Example
Consider the inner product space (R3 , ⟨ ⟩), where ⟨ ⟩ is given by
*x  x +
1 2
y1  , y2  = x1 x2 + y1 y2 + z1 z2 .
z1 z2

Find
 the  matrix
 of inner product space with respect to ordered basis
 1 0 1 
1 , 1 , 0 .
0 1 1
 

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 24 / 61
 Example
Consider the inner product space (R3 , ⟨ ⟩), where ⟨ ⟩ is given by
*x  x +
1 2
y1  , y2  = x1 x2 + y1 y2 + z1 z2 .
z1 z2

Find
 the  matrix
 of inner product space with respect to ordered basis
 1 0 1 
1 , 1 , 0 .
0 1 1
 

Solution
The matrix of inner product space is given by
   
⟨x1 , x1 ⟩ ⟨x1 , x2 ⟩ ⟨x1 , x3 ⟩ 2 1 1
A = ⟨x2 , x1 ⟩ ⟨x2 , x2 ⟩ ⟨x2 , x3 ⟩ = 1 2 1
⟨x3 , x1 ⟩ ⟨x3 , x2 ⟩ ⟨x3 , x3 ⟩ 1 1 2

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 24 / 61
 Example
Consider the inner product space (R2 [x], ⟨ ⟩), where ⟨ ⟩ is given by
Z 1
⟨p(x), q(x)⟩ = p(x)q(x)dx.
0

Find the matrix of inner product space with respect to the ordered basis {1, x, x 2 }.

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 25 / 61
 Solution
The matrix of given inner product is given by

⟨1, x 2 ⟩
 
⟨1, 1⟩ ⟨1, x⟩
A =  ⟨x, 1⟩ ⟨x, x⟩ ⟨x, x 2 ⟩ 
⟨x 2 , 1⟩ ⟨x 2 , x⟩ ⟨x 2 , x 2 ⟩

Z 1 Z 1 1
Z 1 1
⟨1, 1⟩ = 1 · 1dx = 1 ⟨1, x⟩ = 1 · xdx = ⟨1, x 2 ⟩ = 1 · x 2 dx =
0 0 2 0 3
Z 1 1
Z 1 1
Z 1 1
⟨x, 1⟩ = x · 1dx = ⟨x, x⟩ = x · xdx = ⟨x, x 2 ⟩ = x · x 2 dx =
0 2 0 3 0 4
Z 1 1
Z 1 1
Z 1 1
2 2 2 2 2 2
⟨x , 1⟩ = x · 1dx = ⟨x , x⟩ = x · xdx = ⟨x , x ⟩ = x 2 · x 2 dx =
0 3 0 4 0 5
Thus,
1 1
 
1 2 3
A =  21 1
3
1
4
1 1 1
3 4 5

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 26 / 61
 Exercise 5:      
1 1 1
The vectors u1 = 1, u2 = 2, u3 = 3 form a basis S for Euclidean space
0 3 5
R3 . Find the matrix A that represents the inner product in R3 relative to this
basis S.

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 27 / 61
 Orthonormal Vectors

Orthonormal Vectors
Let (V , ⟨ ⟩) be an inner product space. Then a set of vectors {x1 , x2 , . . . , xk } in V
is said to be orthonormal if ∥xi ∥ = 1 and ⟨xi , xj ⟩ = 0 for i ̸= j.

A subset {x1 , x2 , . . . , xk } of V is said to be an orthonormal basis for V if it is a

basis of V and is orthonormal set.

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 28 / 61
 Example
Consider the inner product space (R2 , ⟨ ⟩), where
   
x1 x
, 2 = x1 x2 + y1 y2 .
y1 y2
(" # " #)
√1 √1
Then √1
2 , 2 is an orthonormal basis of R2 as
2
− √12
" # " #
√1 √1
2 = 2 =1
√1 − √12
2

and *" # " #+

√1 √1
2 , 2 =0
√1 − √12
2

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 29 / 61
 Result
If B = {v1 , v2 , . . . , vm } is an orthonormal basis of an inner product space V and x
is any vector in V then

x = ⟨x, v1 ⟩v1 + ⟨x, v2 ⟩v2 + · · · + ⟨x, vm ⟩vm .

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 30 / 61
 Result
If B = {v1 , v2 , . . . , vm } is an orthonormal basis of an inner product space V and x
is any vector in V then

x = ⟨x, v1 ⟩v1 + ⟨x, v2 ⟩v2 + · · · + ⟨x, vm ⟩vm .

Proof
Since B is a basis of V and x ∈ V and hence let

x = α1 v1 + α2 v2 + · · · + αm vm .

However, for each i = 1, 2, . . . , m, we have

⟨x, vi ⟩ =⟨α1 v1 + α2 v2 + · · · + αi vi + · · · + αm vm , vi ⟩
=α1 ⟨v1 , vi ⟩ + α2 ⟨v2 , vi ⟩ + · · · + αi ⟨vi , vi ⟩ + · · · + αm ⟨vm , vi ⟩
=αi ∥vi ∥2 = αi

Therefore
x = ⟨x, v1 ⟩v1 + ⟨x, v2 ⟩v2 + · · · + ⟨x, vm ⟩vm .
Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 30 / 61
 Orthogonal Projection

Let U and W be subspaces of an inner product space (V , ⟨ ⟩).

❁ Two subspaces U and W are said to be orthogonal, written by U ⊥ W , if

⟨u, w ⟩ = 0 for all u ∈ U and w ∈ W .

❁ The set of all vectors in V that are orthogonal to every vector in U is called
the orthogonal complement of U and is denoted by U ⊥ , i.e.,

U ⊥ = {v ∈ V : ⟨u, v ⟩ = 0 ∀u ∈ U}

Note
➀ The orthogonal complement U ⊥ of U is a subspace of the vector space V .

➁ U ∩ U ⊥ = {0} and V = U + U ⊥ .

➂ dim V = dim U + dim U ⊥ .

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 31 / 61
 Definition (Direct Sum)
Let V be a vector space and let U and W be two subspaces of V such that

V =U +W and U ∩ W = {0}.

Then V is called direct sum of U and W and is written as V = U ⊕ W .

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 32 / 61
 Example
Let V = R2 and let U and W be two subspaces of R2 given by
  
x
U= : x ∈R
0
  
0
W = : y ∈R
y

Then show that R2 = U ⊕ W .

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 33 / 61
 Solution
 
x
Suppose ∈ R2 then
y
     
x x 0
= +
y 0 y
     
x 0 a
Now ∈ U and ∈ W and hence V = U + W . Next, suppose ∈ U ∩W.
0 y b
Then    
a a
=⇒ ∈U and ∈W
b b
=⇒ b = 0 and a=0
   
a 0
=⇒ =
b 0
 
0
Thus u ∩ W = { }. This shows that R2 is direct sum of U and W , i.e.,
0

R2 = U ⊕ W

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 34 / 61
 Example
Describe the orthogonal complement V ⊥ of V where
   
1 0
V = span −3 , 1 ,
2 1

where ⟨ ⟩ is Euclidean inner product.

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 35 / 61
 Solution
   
1 0
The subspace V of R3 is spanned by the two vectors v1 = −3 and v2 = 1.
2 1
Therefore its orthogonal complement V ⊥ is the set of vectors which are orthogonal
to both v1 and v2 , i.e.,
  *   + *x  0+ 
 x x 1 
V ⊥ = y  : y  , −3 = 0 and y  , 1 = 0 .
z z 2 z 1
 

 
x
Let y  ∈ V ⊥ then
z
*x   1 +
y  , −3 = 0 =⇒ x − 3y + 2z = 0 (3)
z 2
*   +
x 0
y  , 1 = 0 =⇒ y + z = 0 (4)
z 1
Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 36 / 61
 The augmented matrix of above homogeneous system of linear equation is
   
1 −3 2 | 0 RREF 1 0 5 | 0
−−−→ .
0 1 1 | 0 0 1 1 | 0

System corresponding to above RREF is

x + 5z = 0 =⇒ x = −5z
y + z = =⇒ y = −z

Thus,
      
 x   −5 
V⊥ = y  : x = −5z&y = −z = z −1 : z ∈ R .
z 1
   

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 37 / 61
 Exercise 6:
Compute the orthogonal complement of the following subspace of R3
  
 x 
W = y  : 3x + 2y = z ,
z
 

where ⟨ ⟩ is Euclidean inner product.

Exercise 7:
Compute W ⊥ , where    
1 1
W = span  1  , 1 ,
−1 1
with respect to Euclidean inner product.

Exercise 8:  
1
Find all vectors perpendicular to  1  with respect to Euclidean inner product.
−1
Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 38 / 61
 Exercise 9:  
1
Find a basis for the subspace u of R3 , where u = span 3.
4

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 39 / 61
 Definition(Orthogonal Projection)
Let (V , ⟨ ⟩) be an inner product space, and let U be a subspace of V so that V = U⊕
U ⊥ . Then the projection of V onto U along U ⊥ is called the orthogonal projection
of V onto U, denoted ProjU . For x ∈ V , the component vector ProjU (x) ∈ U is
called the orthogonal projection of x into U.

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 40 / 61
 Result
Let U be a subspace of an inner product space (V , ⟨ ⟩), and let {u1 , u2 , . . . , um } be
an orthogonal basis for U. Then, for any x ∈ V , the orthogonal projection ProjU (x)
of x into U is

⟨x, u1 ⟩ ⟨x, u2 ⟩ ⟨x, um ⟩

ProjU (x) = 2
u1 + 2
u2 + · · · + um .
∥u1 ∥ ∥u2 ∥ ∥um ∥2

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 41 / 61
 Example
Consider the inner product space (R3 , ⟨ ⟩), where ⟨ ⟩ is Euclidean inner product.
Let W be the subspace of R3 given by
   
0 0
W = span 1 , −1 .
1 1
 
1
Find the projection of 2 onto W .
3

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 42 / 61
 Solution
   
0 0
Since vectors 1 , −1 are orthogonal vectors and hence the orthogonal projec-
  1 1
1
tion of 2 onto W is given by
3
*1 0+ *1  0 +
  2 , 1   2 , −1  
1 3 1 0 3 1 0
ProjU 2 =   2 1 +   2 −1
3 0 1 0 1
1 −1
1 1
   
0 0
2 + 3   −2 + 3  
= 1 + −1
2 2
1 1
 
0
= 2
3
Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 43 / 61
 Example
Let V = R3 [x], the vector space of polynomials of degree less than or equal to 3
equipped with the inner product
Z 1
⟨f , g ⟩ = f (x)g (x)dx for any f , g ∈ R3 [x].
0

Let W be the subspace of V given by W = span(1, x). Let f (x) = x 2 . Find the
orthogonal projection ProjW (f ) of f onto W .

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 44 / 61
 The Gram-Schmidt orthogonalization process

The Gram-Schmidt orthogonalization process

Let (V , ⟨ ⟩) be an inner product space and let {x1 , x2 , . . . xm } be a basis for V . Let

v1 = x1
⟨x2 , v1 ⟩
v2 = x2 − v1
∥v1 ∥2
⟨x3 , v1 ⟩ ⟨x3 , v2 ⟩
v3 = x3 − v1 − v2
∥v1 ∥2 ∥v2 ∥2
.. .. ..
. . .
⟨xm , v1 ⟩ ⟨xm , v2 ⟩ ⟨xm , vm−1 ⟩
vm = xm − 2
v1 − 2
v2 − · · · − vm−1 .
∥v1 ∥ ∥v2 ∥ ∥vm−1 ∥2

Then {v1 , v2 , . . . , vm } is an orthogonal basis of V .

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 45 / 61
 Note
( )
v1 v2 vm
❁ If {v1 , v2 , . . . , vm } is an orthogonal basis of V then , ,...,
∥v1 ∥ ∥v2 ∥ ∥vm ∥
will be an orthonormal basis of V .

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 46 / 61
 Note
( )
v1 v2 vm
❁ If {v1 , v2 , . . . , vm } is an orthogonal basis of V then , ,...,
∥v1 ∥ ∥v2 ∥ ∥vm ∥
will be an orthonormal basis of V .

❁ Any basis of an inner product space (V , ⟨ ⟩) is either an orthonormal basis or

can be converted into orthonormal basis by using the Gram-Schmidt orthogo-
nalization process.

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 46 / 61
 Example
Consider 
the inner product space (R2 , ⟨ ⟩) where ⟨ ⟩ is Euclidean inner product.
 
1 2
Let B = , be a basis of R2 . Use the Gram-Schmidt orthogonalization
1 1
process to find an orthogonal basis.

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 47 / 61
 Solution
   
1 2
Setting x1 = and x2 = . Then using Gram-Schmidt orthogonalization
1 1
process, we get
 
1
v1 = x1 =
1
⟨x2 , v1 ⟩
v2 = x2 − v1
∥v1 ∥2
   
2 3 1
= −
1 2 1
 1 
= 21
−2
   1 
1
Thus , 21 is an orthogonal basis of R2 .
1 −2

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 48 / 61
 Example
Consider the inner product space (R2 [x], ⟨ ⟩), equipped with the inner product
Z 1
⟨f , g ⟩ = f (x)g (x)dx.
0

Let B = {1, x, x 2 } be a basis of R2 [x]. Find an orthogonal basis of R2 [x].

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 49 / 61
 Solution
Setting f1 = 1, f2 = x, f3 = x 2 . Then

v1 = f1 = 1
⟨f2 , v1 ⟩
v2 = f2 − v1
∥v1 ∥2
⟨x, 1⟩ 1
=x− 2
1=x−
∥1∥ 2

⟨f3 , v1 ⟩ ⟨f3 , v2 ⟩
v3 = f3 − v1 − v2
∥v1 ∥2 ∥v2 ∥2
1
= x2 − x +
6
Thus 1, x − 12 , x 2 − x + 1

6 is an orthogonal basis of R2 [x].

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 50 / 61
 Example
Find the closest point to x in the subspace W of R4 spanned by v1 and v2 , where
     
3 3 1
1 1 −1
x =   , v1 =   , v2 =  
    
5 −1 1
1 1 −1

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 51 / 61
 Solution
Since ⟨v1 , v2 ⟩ = 3 · 1 + 1 · (−1) + (−1) · 1 + 1 · (−1) = 0 and {v1 , v2 } is linearly
independent and hence {v1 , v2 } forms an orthogonal basis of W . The closest point
to x in the subspace W is ProjW (x). Thus

⟨x, v1 ⟩ ⟨x, v2 ⟩
ProjW (x) = v1 + v2
∥v1 ∥2 ∥v2 ∥2
   
3 1
9+1−5+1 1  3 − 1 + 5 − 1 −1
=  +  
9 + 1 + 1 + 1 −1 1 + 1 + 1 + 1  1 
1 −1
 
3
−1
=1


−1

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 52 / 61
 Example
Find an orthogonal basis for the column space of the following matrix
 
1 2 5
−1 1 −4
 
−1 4 −3
A=  1 −4 7  .

 
 1 −4 7 
1 2 1

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 53 / 61
 Solution
First of all let us find a basis of col(A)
 
1 2 5    
−1 1 −4 R2 → R2 + R1 1 2 5 1 2 5
−1 4 −3 R3 → R3 + R1 0 3
   −1 0
 R3 → R3 − 2R2  3 1

 1 −4 7  R4 → R4 − R1 0 6 2 R4 → R4 + 2R2 0 0 0
   

 1 −4 7  R5 → R5 − R1 0 −6 2
    0 0 4
0 0 4 0 0 4
1 2 1
   
1 2 5 1 2 5
0 3 1 0 3 1
   
R5 → R5 − R 4 
0 0  R3 ↔ 0
0  0 4

0 0 4 0 0 0
0 0 0 0 0 0
Each column contains a leading entry and hence a basis for Col(A) is
     

 1 2 5  
−1   1  −4


     

−1 ,  4  , −3
     
  −4  7 
 1


 

1 2 7
 
Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 54 / 61
 Now, we convert this basis into
 orthogonal
 basis
 using Gram-Schmidt
 orthogonal-
1 2 5
−1 1 −4
     
ization process. Setting x1 = −1
 
 , x2 =  4  , x3 = −3. Then
   
1 −4 7
1 2 7
   
1 3
−1 0
  ⟨x2 , v1 ⟩  
−1 , v2 = x2 − ∥v1 ∥2 v1 =  3 
v1 = x1 =    
1 −3
1 3
 
2
0
⟨x3 , v1 ⟩ ⟨x3 , v2 ⟩  
v3 = x3 − 2
v1 − 2
v2 = 2

∥v1 ∥ ∥v2 ∥ 2
−2

{v1 , v2 , v3 } is an orthogonal basis.

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 55 / 61
 Example
Use the Gram-Schmidt orthogonalization process on the Euclidean space R4 to
transform the basis        

 0 −1 1 −1 
       
1 1 2
 , , , 0  .

 1  0   0   0 
 
0 0 −1 −1
 

into an orthonormal basis.

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 56 / 61
 Solution
       
0 −1 1 −1
1 1 2 0
Setting x1 = 
1 , x2 =  0  , x3 =  0  , x4 =  0 . Then
      

0 0 −1 −1
   
0 −1
1 ⟨x 2 , v 1 ⟩  1 
1 , v2 = x2 − ∥v1 ∥2 v1 = − 1 
v1 = x1 =    2 
2
0 0
 1 
2
⟨x3 , v1 ⟩ ⟨x3 , v2 ⟩  1 
v3 = x3 − 2
v1 − 2
v2 =  2 
− 1 
∥v1 ∥ ∥v2 ∥ 2
− 12
 1
−3
⟨x4 , v1 ⟩ ⟨x4 , v2 ⟩ ⟨x4 , v3 ⟩ − 1 
v4 = x4 − v1 − v2 − v3 = 
 1 
3
∥v1 ∥2 ∥v2 ∥2 ∥v3 ∥2 3
−1

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 57 / 61
 Thus      1   1 
0 −1 − 
   1   21   31 


1 ,  21  ,  21  , −13 

1 − 2  − 2   3 
 
0 0 − 12 −1
 

is an orthogonal basis of R4 and dividing each vector by its magnitude, we get

   √     √ 
 0 − 6 1 − 63 
 √1   √63   21   √3 
 

−

 12  ,  6√  ,
  2  ,
 6
√ 

 √2  − 6  − 12   63 
 
 6 √ 
− 12

 0 
0 3 
2

which is an orthonormal basis of R4 .

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 58 / 61
 Example
 
1
Find the point on the plane x − y − z = 0 that is closest to p = 2.
0

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 59 / 61
 Solution
Given plane can be expressed as subspace of R3 as following
     
 x   y +z 
W = y  ∈ R3 : x − y − z = 0 =  y  : y , z ∈ R
z z
   
          
 y z   1 1 
= y  + 0 : y , z ∈ R = y 1 + z 0 : y , z ∈ R
0 z 0 1
   
   
1 1
= span 1 , 0
0 1
   
 1 1 
Since 1 , 0 is a linearly independent subset of R3 and hence this forms a
0 1
 
basis of W .

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 60 / 61
 Next, we’ll find
 an
 orthogonal
  basis using Gram-Schmidt orthogonalization process.
1 1
Setting x1 = 1 , x2 = 0, we have
0 1
   1 
1
⟨x2 , v1 2
v1 = x1 = 1 , v2 = x2 − 2
v1 = − 12 
∥v1 ∥
0 1
   1 
 1 2 
Thus, 1 , − 12  is an orthogonal basis of W . Therefore the point on the
0 1
 
 
1
plane x − y − z = 0 that is closest to p = 2 is
0
 4 
⟨p, v1 ⟩ ⟨p, v2 ⟩ 3
ProjW (p) = 2
v1 + 2
v2 =  53 
∥v1 ∥ ∥v2 ∥
− 13

Dr. Juhi Kesarwani & Dr. Ashish Kesarwany (VITB) Inner Product Spaces July 31, 2024 61 / 61