Section 6.

1
Inner Products
Throughout this course, we have used different dimensions of Euclidean space, such as R2 , R3 , etc.,
as a “model” for other spaces. Indeed, much of the motivation for defining a vector space was that
many spaces share the vector space properties with Rn .
Of course, Rn has much more structure than what we have discussed so far in this class. In
particular, there is a notion of length of a vector, and distance and angle between vectors in Rn . In
Calculus III, you learned an operation on Rn called the dot product that allows us to easily calculate
length and distance:
Definition. Given vectors    
u1 v1
 u2   v2 
   
u =  .  and v =  . 
 ..   .. 
un vn
in Rn , the dot product u · v of u and v is the number
u · v = u1 v1 + u2 v2 + . . . + un vn .
Again, the dot product is useful in that it gives us a meaningful way to calculate distance and
length:
Theorem. Given vectors    
u1 v1
 u2   v2 
   
u =  .  and v =  . 
 . 
.  .. 
un vn
in Rn , the length of of v can be calculated as
√ √
||v|| = v · v = v12 + v22 + . . . + vn2 ,

and the distance d(u, v) between u and v is given by

√
d(u, v) = ||u − v|| = (u − v) · (u − v).
The dot product obeys several nice properties: for instance, it is easy to show that
u·v = v·u
(u + v) · w = u · w + v · w
(ku) · v = k(u · v).

As you may have suspected, many vector spaces other than Euclidean space also have a notion
of distance, length, and angle between vectors that is quite similar to the dot product operation
described above. Our goal in this section is to describe such vector spaces and the associated
operations.

1
Section 6.1
Inner Products
The dot product on Rn is our model for the idea of an inner product on a general vector space:

Definition 1. An inner product on a real vector space V is a function that associates a real
number denoted ⟨u, v⟩ with each pair of vectors in V , and that satisfies the following properties
for all vectors u, v, and w in V and all scalars k.

1. ⟨u, v⟩ = ⟨v, u⟩

2. ⟨u + v, w⟩ = ⟨u, w⟩ + ⟨v, w⟩

3. ⟨ku, v⟩ = k⟨u, v⟩

4. ⟨v, v⟩ ≥ 0, and ⟨v, v⟩ = 0 if and only if v = 0.

A vector space that is equipped with an inner product is called an inner product space.

Again, the idea behind the definition of an inner product is that we would like to have a
meaningful way to measure distance and length in a general vector space, say in the vector space
Mm×n of m × n matrices or the vector space Pn of polynomials of degree no more than n.

Examples of Inner Products and Inner Product Spaces

Example 1: The Dot Product and Rn
As our inspiration for defining the inner product was the dot product, it should not surprise you
that the dot product defined above on Rn is an example of an inner product. Indeed, the dot
product
⟨u, v⟩ = u · v = u1 v1 + u2 v2 + . . . + un vn
of two vectors satisfies all of the properties from the definition, so it is an inner product on Rn ,
usually referred to as the standard inner product or Euclidean inner product on Rn .

Example 2: The Standard Inner Product on Mn

We can define an inner product on the vector space Mn of n × n (real) matrices by using the trace
function. If U and V are vectors in Mn (that is, they are n × n (real) matrices), then we define
⟨U, V ⟩ by
⟨U, V ⟩ = tr (U ⊤ V ).
For example, set ( ) ( )
3 1 1 7
U= and V = .
5 2 0 −1

2
Section 6.1
Then the inner product ⟨U, V ⟩ is given by

⟨U, V ⟩ = tr (U ⊤ V )

(( )⊤ ( ))
3 1 1 7
= tr
5 2 0 −1

(( )( ))
3 5 1 7
= tr
1 2 0 −1

( )
3 16
= tr
1 5

= 3+5
= 8.

Notice that the output ⟨U, V ⟩ of the inner product function is indeed a real number, as required by
the definition for an inner product.

We can easily check that ⟨·, ·⟩ is an inner product on the vector space M2 of all 2 × 2 matrices
by checking all four of the properties in Definition 1. However, to save time, we will only check the
first and fourth:

1. ⟨u, v⟩ = ⟨v, u⟩:

Given a pair of matrices
( ) ( )
u11 u12 v11 v12
U= and V = ,
u21 u22 v21 v22

let’s calculate each of the inner products ⟨U, V ⟩ and ⟨V, U ⟩:

(a)

⟨U, V ⟩ = tr (U ⊤ V )

(( )⊤ ( ))
u11 u12 v11 v12
= tr
u21 u22 v21 v22

(( )( ))
u11 u21 v11 v12
= tr
u12 u22 v21 v22

( )
u11 v11 + u21 v21 u11 v12 + u21 v22
= tr
u12 v11 + u22 v21 u12 v12 + u22 v22

= u11 v11 + u21 v21 + u12 v12 + u22 v22 .

3
Section 6.1
(b)

⟨V, U ⟩ = tr (V ⊤ U )

(( )⊤ ( ))
v11 v12 u11 u12
= tr
v21 v22 u21 u22

(( )( ))
v11 v21 u11 u12
= tr
v12 v22 u21 u22

( )
v11 u11 + v21 u21 v11 u12 + v21 u22
= tr
v12 u11 + v22 u21 v12 u12 + v22 u22

= v11 u11 + v21 u21 + v12 u12 + v22 u22 .

Comparing terms, we see that

⟨U, V ⟩ = u11 v11 + u21 v21 + u12 v12 + u22 v22

= v11 u11 + v21 u21 + v12 u12 + v22 u22 since uij and vij are real numbers
= ⟨V, U ⟩,

so that ⟨·, ·⟩ satisfies property 1.

4. ⟨v, v⟩ ≥ 0, and ⟨v, v⟩ = 0 if and only if v = 0:

Let’s calculate ⟨v, v⟩:
(( )⊤ ( ))
v11 v12 v11 v12
⟨v, v⟩ = tr
v21 v22 v21 v22

(( )( ))
v11 v21 v11 v12
= tr
v12 v22 v21 v22

( 2 + v2
)
v11 21 v11 v12 + v21 v22
= tr 2 + v2
v12 v11 + v22 v21 v12 22

2 2 2 2
= v11 + v21 + v12 + v22 .

We have
⟨v, v⟩ = v11
2 2
+ v21 2
+ v12 2
+ v22 ;
since this number is a sum of squared terms, ⟨v, v⟩ ≥ 0. Indeed, the only way to make

⟨v, v⟩ = 0

is to choose
v11 = v12 = v21 = v22 = 0,

4
Section 6.1
i.e. choose ( )
0 0
V = = 0.
0 0

The operation ⟨·, ·⟩ satisfies properties 1 and 4 of an inner product, and can easily be shown to
satisfy properties 2 and 3. Thus the operation ⟨·, ·⟩ defined by

⟨U, V ⟩ = tr (U ⊤ V )

is an inner product on M2 (and indeed on Mn ).

Example 3: The Integral Inner Product and C[a, b]

The set of all continuous functions on an interval from x = a to x = b is a vector space with the
usual notion of function addition and scalar products, and is denoted by C[a, b]. For example,
1
f = f (x) = tan x and g = g(x) =
x − π/2

are both vectors in C[−π/4, π/4].

Functions continuous on an interval [a, b] are also integrable on [a, b]; we can use this fact to
define an inner product on C[a, b] by
∫ b
⟨f , g⟩ = f (x)g(x) dx.
a

As a very simple example, f = f (x) = 1 and g = g(x) = x2 are both vectors in C[−1, 1]. The
inner product ⟨f , g⟩ is given by
∫ 1
⟨f , g⟩ = 1 · x2 dx
−1
∫ 1
= x2 dx
−1
1 3 1
= x
3 −1
1 1
= +
3 3
2
= .
3

We can check the properties for an inner product quite easily in this case, as continuous functions
behave quite nicely; for example, property 2 says that ⟨u + v, w⟩ = ⟨u, w⟩ + ⟨v, w⟩. The integral

5
Section 6.1
inner product clearly obeys this property:
∫ b
⟨u + v, w⟩ = (u(x) + v(x))w(x) dx
a
∫ b
= u(x)w(x) + v(x)w(x) dx
a
∫ b ∫ b
= u(x)w(x) + v(x)w(x) dx
a a
= ⟨u, w⟩ + ⟨v, w⟩.

Example 4: The Standard Inner Product on Pn

Recall that the vector space Pn consists of all polynomials of degree no more than n. Given vectors

f = f (x) = a0 + a1 x + . . . + an xn and g = g(x) = b0 + b1 x + . . . + bn xn

in Pn , the standard inner product on Pn is defined by

⟨f , g⟩ = a0 b0 + a1 b1 + . . . + an bn .

For example, the inner product ⟨f , g⟩ of vectors f = f (x) = 1 − x3 + 2x5 and g = g(x) =
2x − x2 + x3 is given by

⟨f , g⟩ = 1 · 0 + (0 · 2) + (0 · −1) + (−1 · 1) + (2 · 0)
= −1.

You should check for yourself that the operation defined above is indeed in inner product.

Key Point. You should be aware that a particular vector space may have many different ways of
defining an inner product. For example, your book discuss the the weighted inner product on Rn :
given vectors    
u1 v1
 u2   v2 
   
u =  .  and v =  . 
 . 
.  .. 
un vn
in Rn and weights (scalars) w1 , w2 , . . . , wn , the weighted inner product u · v of u and v is the
number
u · v = w1 u1 v1 + w2 u2 v2 + . . . + wn un vn .
The weighted inner product is just as legitimate a way to define an inner product on Rn as is
the usual dot product, and has advantages over the dot product in many applications.

6
Section 6.1
Length and Distance
While notions of length of a vector and distance between vectors are harder to visualize for vector
spaces consisting of functions (such as C[a, b]) or matrices (such as Mmn ), they are useful concepts
to have at our disposal. The definitions below for length and distance are given in terms of the
inner product on the space, just as length and distance in Rn can be defined using the dot product:

Definition 2. If V is a (real) inner product space equipped with the inner product ⟨·, ·⟩, then the
norm (or length) of a vector v in V , denoted ||v||, is given by
√
||v|| = ⟨v, v⟩.

The distance between a pair of vectors u and v, denoted by d(u, v), is given by
√
d(u, v) = ||u − v|| = ⟨u − v, u − v⟩.

A vector v such that ||v|| = 1 is called a unit vector.

These definitions, taken together with the properties of the inner product, lead to the following
observations:

Theorem 6.1.1. If u and v are vectors in a (real) inner product space V equipped with the inner
product ⟨·, ·⟩, and if k is any scalar, then:

(a) ||v|| ≥ 0 and ||v|| = 0 if and only if v = 0.

(b) ||kv|| = |k| ||v||

(c) d(u, v) = d(v, u)

(d) d(u, v) ≥ 0 and d(u, v) = 0 if and only if u = v.

Theorem 6.1.1 is an indicator that our definition of the inner product on a general vector space
makes sense as a generalization of the dot product; for instance, part (a) of the theorem assures us
that a vector’s “length” is never negative, and that the only vector of length 0 is the zero vector.

Example
Let (1 ) ( )
2
1
2
1 −1
U= and V = .
1
2
1
2
4 0
Using the inner product on M2 defined by

⟨U, V ⟩ = tr (U ⊤ V ),

find

1. ||U ||

2. ||V ||

7
Section 6.1
3. d(U, V )

1. ||U ||: To calculate the norm of U , we use the formula

√ √
||U || = ⟨U, U ⟩ = tr (U ⊤ U ) :
√
√ (1 1
)⊤ ( 1 1
)
tr (U ⊤ U ) = tr ( 2
1
2
1
2
1
2
1
)
2 2 2 2

√ (1 ) (1 )
1 1
2 2 2 2
= tr ( 1 1 1 1
)
2 2 2 2

√ (1 )
1
2 2
= tr 1 1
2 2

√
= 1
2
+ 12

= 1.

Thus U is a unit vector under ⟨·, ·⟩ in M2 .

2. ||V ||: As before, we use the formula
√ √
||V || = ⟨V, V ⟩ = tr (V ⊤ V ) :
√
√ ( )⊤ ( )
1 −1 1 −1
tr (V ⊤ V ) = tr ( )
4 0 4 0

√ ( )( )
1 4 1 −1
= tr ( )
−1 0 4 0

√ ( )
17 −1
= tr
−1 1

√
= 17 + 1
√
= 18
√
= 3 2.

So under the inner product we have defined on M2 , U is a “shorter” vector than is V .

8
Section 6.1
3. d(U, V ): To calculate the distance between matrices U and V under the inner product, we
use the formula

d(U, V ) = ||U − V ||
√
= ⟨U − V, U − V ⟩
√ ( )
= tr (U − V )⊤ (U −V) .

Since ( 1 )
−2 3
U −V = 2
,
− 72 1
2

we have
√ ( )
d(U, V ) = ⊤
tr (U − V ) (U − V )

√
(( )⊤ ( 1 ))
− 12 3
2
−2 3
2
= tr
− 72 1
2
− 72 1
2

√ (( )( ))
− 12 − 72 − 12 3
2
= tr 3
2
1
2
− 72 1
2

√ (( ))
25
2
− 52
= tr
− 5
2
5
2

√
= 25
2
+ 52

√
= 15.