LINEAR ALGEBRA NOTE SET-4 (Prepared for the students of CET)

Foreword : This is only a glimpses in to the syllabus of linear algebra to

students of Engineering classes. Students are advised to refer text books for
more problems and examples

Inner Product Spaces

An inner product on a vector space V over R or C is a function that to each pair of vectors u and v in V
associates a scalar (real or complex ) denoted by < 𝐮, 𝐯 > and satisfies the following properties

1. < 𝐮 + 𝐯, 𝐰 > = < 𝐮, 𝐰 > +< 𝐯, 𝐰 >

2. < 𝑐𝐮, 𝐯 > = 𝑐 < 𝐮, 𝐯 >

3. < 𝐮, 𝐯 > = ̅̅̅̅̅̅̅̅̅̅̅̅

< 𝐯, 𝐮 >

4. < 𝐮, 𝐮 > ≥ 0 ,

5. < 𝐮, 𝐮 > = 0 if and only if 𝐮 = 𝟎 (Positive definiteness property)

Note 1. We can combine the properties 1 and 2 to get

< 𝛼𝐮 + 𝛽𝐯, 𝐰 > = 𝛼 < 𝐮, 𝐰 > +𝛽 < 𝐯, 𝐰 >

2. When V is a real vector space axiom 3 becomes < 𝐮, 𝐯 > =< 𝐯, 𝐮 >

Example 1.

Consider 𝑉 = ℝ2 . For any two vectors 𝐮 = (𝑥1 , 𝑥2 ) and 𝐯 = (𝑦1 , 𝑦2 ) inℝ2 , define

< 𝐮, 𝐯 > = 𝑥1 𝑦1 + 𝑥2 𝑦2 .

With this definition, ℝ2 becomes an inner product space.(A vector space equipped with an inner
product is called an inner product space). For 𝒖 = (𝑥1 , 𝑥2 ), 𝒗 = (𝑦1 , 𝑦2 ) and 𝒘 = (𝑧1 , 𝑧2 ) in ℝ2
and real numbers (scalars) α and 𝛽 from the underlying field R, we have

α𝒖 = (α𝑥1 , 𝛼𝑥2 ), 𝛽𝒗 = (𝛽𝑦1 , 𝛽𝑦2 )

α𝒖 + β𝐯 = (α𝑥1 + 𝛽𝑦1 , 𝛼𝑥2 + 𝛽𝑦2 ),.

< 𝛼𝐮 + β𝐯, 𝐰 > = (α𝑥1 + 𝛽𝑦1 )𝑧1 + (α𝑥2 + 𝛽𝑦2 )𝑧2

= 𝛼𝑥1 𝑧1 + 𝛽𝑦1 𝑧1 + 𝛼𝑥2 𝑧2 + 𝛽𝑦2 𝑧2 ……………………………………… (1)

< 𝛼𝐮, 𝐰 > =< (α𝑥1 , 𝛼𝑥2 ), (𝑧1 , 𝑧2 ) >= 𝛼𝑥1 𝑧1 + 𝛼𝑥2 𝑧2 …………………………… (2)

< β𝐯, 𝐰 > =< (β𝑦1 , 𝛽𝑦2 ), (𝑧1 , 𝑧2 ) >= 𝛽𝑦1 𝑧1 + 𝛽𝑦2 𝑧2 ………………………………. (3)

Eq. (2) + eq. (3) gives

< 𝛼𝐮, 𝐰 > + < β𝐯, 𝐰 >= 𝛼𝑥1 𝑧1 + 𝛼𝑥2 𝑧2 + 𝛽𝑦1 𝑧1 + 𝛽𝑦2 𝑧2 ……………………………… (4)
From eq. (2) and eq.(4)

< α𝐮 + β𝐯, 𝐰 > =< 𝛼𝐮, 𝐰 > + < β𝐯, 𝐰 >

< 𝐮, 𝐯 > = 𝑥1 𝑦1 + 𝑥2 𝑦2 and

< 𝐯, 𝐮 > = 𝑦1 𝑥1 + 𝑦2 𝑥2

Since < 𝐯, 𝐮 > = 𝑦1 𝑥1 + 𝑦2 𝑥2 is real, ̅̅̅̅̅̅̅̅̅̅̅̅

< 𝐯, 𝐮 > =< 𝐯, 𝐮 > = 𝑦1 𝑥1 + 𝑦2 𝑥2 =< 𝐮, 𝐯 >

< 𝐮, 𝐮 > = 𝑥1 𝑥1 + 𝑥2 𝑥2 = 𝑥12 + 𝑥22 ≥ 0

< 𝐮, 𝐮 > = 0 if and only if 𝑥12 + 𝑥22 = 0 . This implies that 𝑥1 = 0 and 𝑥2 = 0 so that

𝒖 = (𝑥1 , 𝑥2 ) = 0.Thus all the axioms for an inner products are satisfied by the above definition
of inner product in ℝ2 . Hence ℝ2 is an inner product space. (i. p. s)

Example2. Let V = ℝ𝑛 . For u = (𝑥1 , 𝑥2 , … . . , 𝑥𝑛 ), v = (𝑦1 , 𝑦2 , … … , 𝑦𝑛 ) in V = ℝ𝑛 , define

< 𝐮, 𝐯 > = 𝑥1 𝑦1 + 𝑥2 𝑦2 + ⋯ + 𝑥𝑛 𝑦𝑛 . Then ℝ𝑛 is an i. p. s with this definition. This inner product

is called the standard inner product or usual inner product or usual inner product.

Note: The same vector space can be equipped with a different inner product. Consider the
following example in which we equip ℝ2 with an additional inner product other than standard
inner product
𝑥1 𝑦1
Example: For 𝒖 = [𝑥 ] , and 𝒗 = [𝑦 ] in ℝ2 , define < 𝐮, 𝐯 > = 2𝑥1 𝑦1 − 𝑥1 𝑦2 − 𝑥2 𝑦1 + 5𝑥2 𝑦2 .
2 2

With this definition ℝ2 is an inner product space.

Proof : Left to the students as a Compulsory Exercise.

Example: Let 𝑉 = 𝑀𝑚,𝑛 (𝑅) be the space of all 𝑚 × 𝑛 matrices over R. For A, B 𝐴, 𝐵 ∈ 𝑀𝑚,𝑛 (𝑅),
define

< A, B > = 𝑇𝑟(𝐵𝑇 𝐴).

Hint

For 𝑚 = 2, 𝑛 = 3, consider 𝑉 = 𝑀2,3 (𝑅)

𝑏11 𝑏21
𝑎11 𝑎12 𝑎13 𝑏11 𝑏12 𝑏13
Take any 𝐴 = [𝑎 𝑎22 𝑎23 ] and 𝐵 = [𝑏21 ], 𝐵𝑇 = [𝑏12 𝑏22 ]
21 𝑏22 𝑏23
𝑏13 𝑏23

𝑏11 𝑏21
𝑎 𝑎12 𝑎13
𝑇
𝐵 𝐴 = [𝑏12 𝑏22 ] [𝑎11 𝑎22 𝑎23 ]
21
𝑏13 𝑏23

𝑏11 𝑎11 + 𝑏21 𝑎21 𝑏11 𝑎12 + 𝑏21 𝑎22 𝑏11 𝑎13 + 𝑏21 𝑎23
= [𝑏12 𝑎11 + 𝑏22 𝑎21 𝑏12 𝑎12 + 𝑏22 𝑎22 𝑏12 𝑎13 + 𝑏22 𝑎23 ]
𝑏13 𝑎11 + 𝑏23 𝑎21 𝑏13 𝑎12 + 𝑏23 𝑎22 𝑏13 𝑎13 + 𝑏23 𝑎23

< A, B >= 𝑇𝑟(𝐵𝑇 𝐴) = 𝑏11 𝑎11 + 𝑏21 𝑎21 + 𝑏12 𝑎12 + 𝑏22 𝑎22 + 𝑏13 𝑎13 + 𝑏23 𝑎23
 𝑎11 + 𝑏11 𝑎12 + 𝑏12 𝑎13 + 𝑏13
𝐴+𝐵 =[ ]
𝑎21+ 𝑏21 𝑎22 + 𝑏22 𝑎23 + 𝑏23

𝑇𝑟(𝐶 𝑇 𝐴) = 𝑐11 𝑎11 + 𝑐21 𝑎21 + 𝑐12 𝑎12 + 𝑐22 𝑎22 + 𝑐13 𝑎13 + 𝑐23 𝑎23

𝑇𝑟(𝐶 𝑇 𝐵) = 𝑐11 𝑏11 + 𝑐21 𝑏21 + 𝑐12 𝑏12 + 𝑐22 𝑏22 + 𝑐13 𝑏13 + 𝑐23 𝑏23

𝑇𝑟(𝐶 𝑇 𝐴) + 𝑇𝑟(𝐶 𝑇 𝐵)
= 𝑐11 𝑎11 + 𝑐21 𝑎21 + 𝑐12 𝑎12 + 𝑐22 𝑎22 + 𝑐13 𝑎13 + 𝑐23 𝑎23 + 𝑐11 𝑏11 + 𝑐21 𝑏21
+ 𝑐12 𝑏12 + 𝑐22 𝑏22 + 𝑐13 𝑏13 + 𝑐23 𝑏23
𝑐11 𝑐21
𝑎 + 𝑏11 𝑎12 + 𝑏12 𝑎13 + 𝑏13
𝑇𝑟𝐶 𝑇 (𝐴 + 𝐵) = [𝑐12 𝑐22 ] [ 11 ]
𝑐13 𝑐23 𝑎21 + 𝑏21 𝑎22 + 𝑏22 𝑎23 + 𝑏23

= 𝑐11 (𝑎11 + 𝑏11 ) + 𝑐21 (𝑎21 +𝑏21 ) + 𝑐12 (𝑎12 + 𝑏12 ) +

𝑐22 (𝑎22 + 𝑏22 ) + 𝑐13 (𝑎13 + 𝑏13 ) + 𝑐23 (𝑎23 + 𝑏23 )

= 𝑐11 𝑎11 + 𝑐11 𝑏11 + 𝑐21 𝑎21 + 𝑐21 𝑏21 + 𝑐12 𝑎12 + 𝑐12 𝑏12 + 𝑐22 𝑎22 + 𝑐22 𝑏22 + 𝑐13 𝑎13 + 𝑐13 𝑏13
+ 𝑐23 𝑎23 + 𝑐23 𝑏23 = 𝑇𝑟(𝐶 𝑇 𝐴) + 𝑇𝑟(𝐶 𝑇 𝐵)

< A + B, C >= 𝑇𝑟[𝐶 𝑇 (𝐴 + 𝐵)] = 𝑇𝑟(𝐶 𝑇 𝐴 + 𝐶 𝑇 𝐵)

= 𝑇𝑟𝐶 𝑇 𝐴 + 𝑇𝑟𝐶 𝑇 𝐵 =< A, C > +< B, C >

< 𝛼A, B >= 𝑇𝑟(𝐵𝑇 𝛼𝐴) = 𝑏11 𝛼𝑎11 + 𝑏21 𝛼𝑎21 + 𝑏12 𝛼𝑎12 + 𝑏22 𝛼𝑎22 + 𝑏13 𝛼𝑎13 + 𝑏23 𝛼𝑎23

= 𝛼(𝑏11 𝑎11 + 𝑏21 𝑎21 + 𝑏12 𝑎12 + 𝑏22 𝑎22 + 𝑏13 𝑎13 + 𝑏23 𝑎23 ) = 𝛼 < A, B >

< A, B > = 𝑇𝑟(𝐵𝑇 𝐴)

𝑎11 𝑎21
𝑏 𝑏12 𝑏13
𝑇
< B, A >= 𝑇𝑟(𝐴 𝐵) = [𝑎12 𝑎22 ] [ 11 ]
𝑎13 𝑎23 21 𝑏 𝑏22 𝑏23

= 𝑎11 𝑏11 + 𝑎21 𝑏21 + 𝑎12 𝑏12 + 𝑎22 𝑏22 + 𝑎13 𝑏13 + 𝑎23 𝑏23

= 𝑏11 𝑎11 + 𝑏21 𝑎21 + 𝑏12 𝑎12 + 𝑏22 𝑎22 + 𝑏13 𝑎13 + 𝑏23 𝑎23 =< A, B >

< A, B > =< B, A > as the inner product is real

< A, A >= 𝑇𝑟(𝐴𝑇 𝐴) = 𝑎11 𝑎11 + 𝑎21 𝑎21 + 𝑎12 𝑎12 + 𝑎22 𝑎22 + 𝑎13 𝑎13 + 𝑎23 𝑎23 ≥ 0

< A, A > = 0 Implies 𝑎11 2 + 𝑎21 2 + 𝑎12 2 + 𝑎22 2 + 𝑎13 2 + 𝑎23 2 = 0

This implies 𝑎11 = 𝑎21 = 𝑎12 = 𝑎22 = 𝑎13 = 𝑎23 = 0, i. e A = O

General proof is left to the students as an exercise.

Example: = 𝐶[𝑎, 𝑏] , the space of all continuous real valued functions on[𝑎, 𝑏] is an inner product
space with
 𝑏
< 𝐟, 𝐠 > = ∫𝑐 𝑓(𝑡)𝑔(𝑡) 𝑑𝑡.

Left to the students as a Compulsory Exercise.

𝑏
Hint: < 𝐟 + 𝐠, 𝐡 > = ∫𝑐 (𝑓(𝑡) + 𝑔(𝑡))ℎ(𝑡) 𝑑𝑡.

𝑏
= ∫ (𝑓(𝑡)ℎ(𝑡) + 𝑔(𝑡)ℎ(𝑡))𝑑𝑡
𝑐

𝑏 𝑏
= ∫ 𝑓(𝑡)ℎ(𝑡)𝑑𝑡 + ∫ 𝑔(𝑡)ℎ(𝑡)𝑑𝑡 =< 𝐟, 𝐡 > +< 𝐠, 𝐡 >
𝑐 𝑐

𝑏 𝑏
< 𝛼𝐟, 𝐠 >= ∫ 𝛼𝑓(𝑡)𝑔(𝑡) 𝑑𝑡 = 𝛼 ∫ 𝑓(𝑡)𝑔(𝑡) 𝑑𝑡 = 𝛼 < 𝐟, 𝐠 >
𝑐 𝑐

𝑏
< 𝐟, 𝐠 > = ∫ 𝑓(𝑡)𝑔(𝑡) 𝑑𝑡
𝑐

𝑏 𝑏
< 𝐠, 𝐟 > = ∫ 𝑔(𝑡)𝑓(𝑡) 𝑑𝑡 = ∫ 𝑓(𝑡)𝑔(𝑡) 𝑑𝑡 =< 𝐟, 𝐠 >
𝑐 𝑐

𝑏
< 𝐟, 𝐟 > = ∫ [𝑓(𝑡)]2 𝑑𝑡 ≥ 0
𝑐

𝑏
< 𝐟, 𝐟 > = 0 implies ∫𝑐 [𝑓(𝑡)]2 𝑑𝑡 = 0 . This implies 𝑓(𝑡) = 0

Compulsory exercise

1) In each of the following cases, determine whether the given formula provides an inner
product for the following vector spaces

i) In 𝑉 = ℝ2 , for 𝒖 = (𝑥1 , 𝑥2 ), and 𝒗 = (𝑦1 , 𝑦2 ) in ℝ2 , define < 𝐮, 𝐯 > = 𝑥1 𝑦1 − 𝑥2 𝑦2 .

ii) In 𝑉 = ℝ2, for 𝒖 = (𝑥1 , 𝑥2 ), and 𝒖 = (𝑦1 , 𝑦2 ) in ℝ2 , define

< 𝐮, 𝐯 > = 𝑥1 𝑦1 − 𝑥2 𝑦1 − 𝑥1 𝑦2 + 4𝑥2 𝑦2 .

iii) In 𝑉 = 𝑀2 (𝑅), the space of all second order matrices with real entries , for 𝐴, 𝐵 ∈ 𝑀2 (𝑅)
define < 𝐀, 𝐁 > = 𝑇𝑟(𝑨𝑩).

Requirement for further study from complex algebra

1) For complex number any z  x  iy , it’s conjugate is z  x  iy

2) For any complex number any z  x  iy , z  z

3) For any two complex numbers z1 , z 2 , z1  z 2 = z1  z 2

4) For complex number any z  z z  x  y ,

2 2
Example 3. Consider the vector space V = ℂ2 = {(𝑧1 , 𝑧2 ) 𝑧1 , 𝑧2 ∈ 𝐶 } where C is the field of
complex numbers. For any two vectors 𝐮 = (𝑥1 , 𝑥2 ) and 𝐯 = (𝑦1 , 𝑦2 ) in ℂ2 where 𝑥1 , 𝑥2 , 𝑦1 , 𝑦2 are
complex numbers, define an inner product as

< 𝐮, 𝐯 > = 𝑥1 𝑦̅1 + 𝑥2 𝑦̅2 .

For 𝒖 = (𝑥1 , 𝑥2 ), 𝒗 = (𝑦1 , 𝑦2 ) and 𝒘 = (𝑧1 , 𝑧2 ) in ℂ2

𝒖 + 𝐯 = (𝑥1 + 𝑦1 , 𝑥2 + 𝑦2 )

< 𝐮 + 𝐯, 𝐰 > = (𝑥1 + 𝑦1 ) 𝑧̅1 + (𝑥2 + 𝑦2 ) 𝑧̅2

= 𝑥1 𝑧̅1 + 𝑦1 𝑧̅1 + 𝑥2 𝑧̅2 + 𝑦2 𝑧̅2

=𝑥1 𝑧̅1 + 𝑥2 𝑧̅2 + 𝑦1 𝑧̅1 + 𝑦2 𝑧̅2

=(𝑥1 𝑧̅1 + 𝑥2 𝑧̅2 ) + (𝑦1 𝑧̅1 + 𝑦2 𝑧̅2 )

= < 𝐮, 𝐰 > +< 𝐯, 𝐰 >

< α𝐮, 𝐯 > =< (α𝑥1 , 𝛼𝑥2 ), (𝑦1 , 𝑦2 ) > 𝛼𝑥1 𝑦̅1 + 𝛼𝑥2 𝑦̅2 = 𝛼(𝑥1 𝑦̅1 + 𝑥2 𝑦̅2 ) = 𝛼 < 𝐮, 𝐯 >

< 𝐮, 𝐯 > = 𝑥1 𝑦̅1 + 𝑥2 𝑦̅2

< 𝒗, 𝒖 > = 𝑦1 𝑥̅1 + 𝑦2 𝑥̅2

< 𝒗, 𝒖 > = ̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅

̅̅̅̅̅̅̅̅̅̅̅ 𝑦1 𝑥̅1 + 𝑦2 𝑥̅2

̅̅̅̅̅̅
=𝑦 ̅̅̅̅̅̅
1 𝑥̅1 + 𝑦2 𝑥̅2

= 𝑦̅1 𝑥̅̅1 + 𝑦̅2 𝑥̅̅2

= 𝑦̅1 𝑥1 + 𝑦̅2 𝑥2

= 𝑥1 𝑦̅1 + 𝑥2 𝑦̅2 . (product of complex numbers is

commutative)

=< 𝐮, 𝐯 >

Thus ̅̅̅̅̅̅̅̅̅̅̅
< 𝐮, 𝐯 > =< 𝒗, 𝒖 >

< 𝐮, 𝐮 > = 𝑥1 𝑥̅1 + 𝑥2 𝑥̅2 = |𝑥1 |2 + |𝑥2 |2 ≥ 0

< 𝐮, 𝐮 > = 0 if and only if |𝑥1 |2 + |𝑥2 |2 = 0 ,This implies |𝑥1 |2 = 0 and |𝑥2 |2 = 0

Hence 𝑥1 = 0 and 𝑥2 = 0 so that 𝒖 = (𝑥1 , 𝑥2 ) = 0.

Note : 1.The inner product defined as above in ℝn or ℂn is called the standard inner product.

2. The standard inner product in ℝn is also referred to as the dot product. It is denoted by
𝐮. 𝐯

Therefore for 𝐮 = (𝑥1 , 𝑥2 , … … , 𝑥𝑛 ), 𝐯 = (𝑦1 , 𝑦2 , … … , 𝑦𝑛 ), 𝐮. 𝐯 = 𝑥1 𝑦1 + 𝑥2 𝑦2 + ⋯ … + 𝑥𝑛 𝑦𝑛

Example: Consider the vector space V of all continuous complex valued functions on [a ,b]. Define
𝑏
< 𝒇, 𝒈 > = ∫ 𝑓(𝑡) ̅̅̅̅̅̅
𝑔(𝑡) 𝑑𝑡
𝑎

Then < 𝒇, 𝒈 > is an inner product on V.

Proof is left as an exercise.

Length of a vector: The length of a vector u denoted by ‖𝐮‖ is defined by the positive square root
√< 𝒖, 𝒖 > . If ‖𝐮‖ = 1, the vector is called a unit vector.

Orthogonal vectors: Two vectors u and v are orthogonal (perpendicular) if < 𝐮, 𝐯 > = 0

Example: In ℝ2 , the vectors 𝐞1 = (1,0), e𝟐 = (0,1) are orthogonal. In fact ‖‖𝐞1 ‖ = 1, ‖𝐞2 ‖ = 1.
Since they are orthogonal unit vectors, they are called orthonormal vectors. Thus {𝐞1 , 𝐞2 } form an
orthonormal basis for ℝ2.

Note. If u and v are orthogonal, we write u  v

Example: 0  u for every vector u in V

Proof: < 𝟎, 𝐮 > =< 0𝐮, 𝐮 > = 0 < 𝐮, 𝐮 > = 0‖𝐮‖2 = 0

Theorem. Let V be an inner product space over R or C. Then the following are satisfied for any
vectors u and v and for any scalar α.

(a) ‖𝐮‖ ≥ 0, ‖𝐮‖ = 0 if and only if u=0

(b) ‖α𝐮‖ = |𝛼|‖𝐮‖

(c) For any nonzero vector u , the scalar multiple u/‖𝒖‖ is a unit vector .

(d) Pythagoras identity. If u  v , then ‖𝐮 + 𝐯‖2 = ‖𝐮‖2 + ‖𝐯‖2

Proof: We prove the result for real inner product space

‖𝐮 + 𝐯‖2 = < 𝐮 + 𝐯, 𝐮 + 𝐯 > = < 𝐮, 𝐮 > +< 𝐮, 𝐯 > +< 𝐯, 𝐮 > +< 𝐯, 𝐯 >

= ‖𝐮‖2 + ‖𝐯‖2 where < 𝐮, 𝐯 > = 0, < 𝐯, 𝐮 > = 0 as u and v are orthogonal

(e)Cauchy –Schwarz inequality

|< 𝐮, 𝐯 >| ≤ ‖𝐮‖‖𝐯‖

Proof: When 𝐮 = 𝟎 , the above inequality is satisfied. If 𝐮 ≠ 𝟎 put

<𝐯,𝐮>
𝜸=𝒗− ‖𝐮‖2
𝐮

<𝐯,𝐮> <𝐯,𝐮>
Then 0 ≤ ‖𝛄‖2 =< 𝒗 − ‖𝐮‖2
𝐮, 𝒗 − ‖𝐮‖2
𝐮 >
 <𝐯,𝐮> <𝐯,𝐮> <𝐯,𝐮>
=< 𝒗, 𝒗 − ‖𝐮‖2
𝐮 > +< − ‖𝐮‖2
𝐮, 𝒗 − ‖𝐮‖2 𝐮 >

<𝐯,𝐮> <𝐯,𝐮> <𝐯,𝐮>

=< 𝒗, 𝒗 − ‖𝐮‖2
𝐮 >− ‖𝐮‖2
< 𝐮, 𝒗 − ‖𝐮‖2
𝐮 >

̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅
<𝐯,𝐮> <𝐯,𝐮> ̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅
<𝐯,𝐮>
= =< 𝒗 − ‖𝐮‖2 𝐮, 𝐯 >− ‖𝐮‖2 < 𝒗 − ‖𝐮‖2 𝐮, 𝐮 >

̅̅̅̅̅̅̅̅̅̅̅
< 𝐯, 𝐮 > < 𝐯, 𝐮 > ̅̅̅̅̅̅̅̅̅̅̅
< 𝐯, 𝐮 >< 𝐯, 𝐮 >
̅̅̅̅̅̅̅̅̅̅̅
=< 𝒗, 𝒗 >− 2
̅̅̅̅̅̅̅̅̅̅̅
< 𝐮, 𝐯 >− 2
̅̅̅̅̅̅̅̅̅̅̅
< 𝐯, 𝐮 >+ 4
‖𝐮‖2
‖𝐮‖ ‖𝐮‖ ‖𝐮‖
<𝐮,𝐯>
=< 𝒗, 𝒗 > − ̅̅̅̅̅̅̅̅̅̅̅
< 𝐮, 𝐯 >
‖𝐮‖2

|<𝐮,𝐯>|2
=< 𝒗, 𝒗 > − ‖𝐮‖2

|<𝐮,𝐯>|2
< 𝒗, 𝒗 > − ‖𝐮‖2
≥0

|<𝐮,𝐯>|2
< 𝒗, 𝒗 > ≥ ‖𝐮‖2

|<𝐮,𝐯>|2
‖𝐮‖2
≤ ‖𝐯‖2

|< 𝐮, 𝐯 >|2 ≤ ‖𝐯‖2 ‖𝐮‖2

< 𝒖, 𝒗 > ≤ ‖𝐮‖‖𝐯‖

(f) Triangular inequality

‖𝐮 + 𝐯‖ ≤ ‖𝐮‖ + ‖𝐯‖

Proof :

‖𝐮 + 𝐯‖2 = < 𝐮 + 𝐯, 𝐮 + 𝐯 > = < 𝐮, 𝐮 + 𝐯 > +< 𝐯, 𝐮 + 𝐯 >=<

̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅
𝐮 + 𝐯, 𝐮 >+< ̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅
𝐮 + 𝐯, 𝐯 >

̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅
=< 𝐮, 𝐮 > +< 𝐯, 𝐮 >+< ̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅
𝐮, 𝐯 > +< 𝐯, 𝐯 >

̅̅̅̅̅̅̅̅̅̅̅
=< 𝐮, 𝐮 >+< ̅̅̅̅̅̅̅̅̅̅̅
𝐯, 𝐮 >+< ̅̅̅̅̅̅̅̅̅̅̅
𝐮, 𝐯 >+< ̅̅̅̅̅̅̅̅̅̅̅
𝐯, 𝐯 >

= ‖𝐮‖2 +< 𝐮, 𝐯 > +< 𝐮, 𝐯 > + ‖𝐯‖2

̅̅̅̅̅̅̅̅̅̅̅

= ‖𝐮‖2 + 2 𝑅𝑒 < 𝐮, 𝐯 > +‖𝐯‖2

≤ ‖𝐮‖2 + 2|< 𝐮, 𝐯 >| + ‖𝐯‖2 Since Re (z) ≤ |𝑧| for any znumber z

≤ ‖𝐮‖2 + 2‖𝐮‖ ‖𝐯‖ + ‖𝐯‖2 = (‖𝐮‖ + ‖𝐯‖)2

by Cauchy –Schwarz inequality

‖𝐮 + 𝐯‖ ≤ ‖𝐮‖ + ‖𝐯‖
<𝐮,𝐯>
Definition : The angle between two vectors u and v in is defined by 𝒄𝒐𝒔 𝜽 = ‖𝐮‖‖𝐯‖

When two vectors u an v are perpendicular, cos 90 =0. This implies that < 𝐮, 𝐯 >= 𝟎
Note: The process of finding a unit vector in the direction of a given vector is called normalization.

Problem: Normalize 1,2, 2, 4  OR Find a unit vector in the direction of 1,2, 2, 4 

T T

Take 𝐮 = (1, −2, 2, 4)𝑇 . ‖𝐮‖ = √12 + −22 + 22 + 42 = 5

𝟏 𝟏
The required unit vector 𝒗 = ‖𝒖‖ 𝒖 = 𝟓 (1, − 2 , 2, 4)𝑻

Distance : The distance between two vectors u and v is defined by 𝑑(𝐮, 𝐯) = ‖𝐮 − 𝐯‖

Example: Find the distance between the vectors 1, 2,1, 0  and 2,1, 3, 1
T T

Take u= 1, 2,1, 0  and v= 2,1, 3, 1

T T

u-v=  1, 3,  4,  1
T

𝑑(𝐮, 𝐯) = ‖𝐮 − 𝐯‖ = √−12 + 32 + −42 + −12 = √27

Orthogonal complement
Let W be subspace of a vector space V. The set of all vectors in V that are orthogonal to each vector
in W is called the orthogonal complement of W. It is denoted by W ⟘

Theorem. W ⟘ is a subspace of a vector space V

Proof i, The Zero vector 𝟎 ∈ W ⟘ since it is orthogonal to every vector in W.

ii, Let u ,v ∈ W ⟘ . Then for any 𝐰 ∈ W, < 𝐮, 𝐰 > = 0 and < 𝐯, 𝐰 > = 0

Now < 𝐮 + 𝐯, 𝐰 > = < 𝐮, 𝐰 > +< 𝐯, 𝐰 > = 0. Therefor 𝐮 + 𝐯 ∈ W ⟘

Iii, For any u ∈ W ⟘ and v ∈ W , < 𝑐𝐮, 𝐯 > = 𝑐 < 𝐮, 𝐯 > = 0 .therefor c𝐮 ∈ W

Hence elements in W ⟘ satisfies all the three conditions for a subset to be a vector space. Hence
W ⟘ is a subspace of W.

1 1
Problem : Find the orthogonal complement of the plane spanned by the vectors [1] and [2].
2 3
1 1 1 1
Solution: W=Span {[1] , [2] } consists of the linear combinations of [1] and [2].
2 3 2 3

Hence any vector orthogonal to the given two vectors will be orthogonal to each vector in W.
𝑥 𝑥 1
Thus if [𝑦] ∈ W ⊥ , then 𝑥 + 𝑦 + 2𝑧 = 0 (follows from the orthogonally of [𝑦] and [1] )
𝑧 𝑧 2
 𝑥 1
and 𝑥 + 2𝑦 + 3𝑧 = 0,( follows from the orthogonally of [𝑦] and [2] )
𝑧 3
𝑥
1 1 2 𝑦 0
In matrix form [ ][ ] = [ ]
1 2 3 𝑧 0

𝑥
1 1 2
i.e 𝐴𝑿 = 0 where 𝐴 = [ ] and 𝑿 = [𝑦] ,
1 2 3 𝑧

Hence orthogonal complement of W is the null space of A where the rows of A are the given
𝑎 1
𝑎
vectors. Solving the above equations, a solution is given by [ ] = 𝑎 [ 1 ]
−𝑎 −1
1
where a is any real number. Thus W ⟘ = span {[ 1 ]} .
−1
1 2
Problem: Find a basis for the orthogonal complement of 𝑊 = 𝑠𝑝𝑎𝑛 {[ 2 ] , [4]}
−1 3
4 5

Theorem: Let A be an 𝑚 × 𝑛 matrix. The orthogonal complement of the row space of A is the
null space of A and the orthogonal complement of the column space of A is the null space of 𝐴𝑇 .

1 1 2
Example: Orthogonal complement for the row and column spaces of A where 𝐴 = [ ]
1 2 3

A vector in the orthogonal complement of R(A) is an element of ℝ3 and a vector in the

orthogonal complement of Col(A) is an element of ℝ2 .As seen in the above problem

1 𝑎
span {[ 1 ]} is the orthogonal complement of the row space of A . If [ ] is an element of the
𝑏
−1
orthogonal complement of the column space of A, then 𝑎 + 𝑏 = 0. 𝑎 + 2𝑏 = 0, 2𝑎 + 3𝑏 = 0

1 1 𝑎 0
In matrix form [1 2] [ 𝑏 ] = [ 0]
2 3 0
𝑎
i. e 𝐴𝑇 𝒀 = 𝑶. This shows that [ ] is in the orthogonal complement of AT
𝑏
0
Here 𝑊 = 𝑐𝑜𝑙(𝐴) = ℝ2 . Hence W ⟘ = {[ ]}
0
1 1
Problem: Find a unit vector orthogonal to 𝒖 = [1] and 𝒗 = [ 2 ]
1 −3
𝑥
Answer: Let 𝒘 = [𝑦] be orthogonal to both u and v, then
𝑧

𝑥 + 𝑦 + 𝑧 = 0 , 𝑥 + 2𝑦 − 3𝑧 = 0
 𝑥 −5𝑎 −5
Solving 𝒘 = [𝑦] = [ 4𝑎 ] = 𝑎 [ 4 ]
𝑧 𝑎 1
𝑥 −5 𝑥 −5
Any vector spanned by 𝒘 = [𝑦] = 𝑎 [ 4 ]. In particular when a=1, 𝒘 = [𝑦] = [ 4 ]
𝑧 1 𝑧 1
𝒘
A unit vector orthonormal to both u and v is given by ‖𝒘‖

‖𝒘‖ = √−52 + 42 + 12 = √42

−5
𝒘 1
A unit vector orthonormal to both u and v is ‖𝒘‖ = [4]
√42
1

Result: If 𝒖𝟏 , 𝒖2 , 𝒖3 , … … … , 𝒖𝑝 are mutually orthogonal, the they are linearly independent

Proof: Let 𝑐1 𝒖𝟏 + 𝑐2 𝒖2 + 𝒄𝟑 𝒖3 + … … … + 𝒄𝒑 𝒖𝑝 = 𝟎. Taking inner product with 𝑢𝑗 for any j we

2
have 𝑐𝑗 < 𝑢𝑗 , 𝑢𝑗 > =< 𝑢𝑗 , 𝟎 > = 0, since < 𝑢𝑗 , 𝑢𝑖 > = 0 for 𝑖 ≠ 𝑗. i. e 𝑐𝑗 ‖𝒖𝒋 ‖ = 0, Since 𝒖𝑗 ≠ 0
we get 𝑐𝑗 = 0 for 𝑗 = 1, 2, … . . , 𝑝. Thus the vectors 𝒖𝟏 , 𝒖2 , 𝒖3 , … … … , 𝒖𝑝 are linearly
independent.