SC223 - Linear Algebra

Aditya Tatu

Lecture 14

September 9, 2022
Vector Spaces
Definition: A Vector space is a set V with a field (F, +F , ×),
and two binary operations, vector addition + and scalar
multiplication · that satisfy the following axioms:
 (V , +) is an Abelian group:
 ∀x, y ∈ V , x + y ∈ V
 ∃θ ∈ V , ∀x ∈ V , x + θ = θ + x = x
 ∀x ∈ V , ∃y ∈ V , x + y = y + x = θ. We will denote y by
−x.
 ∀x, y , z ∈ V , (x + y ) + z = x + (y + z).
 ∀x, y ∈ V , x + y = y + x.
 Closure with respect to Scalar multiplication: ·F × V → V .
 Scalar Multiplication identity: ∃1 ∈ F such that
1 · v = v , ∀v ∈ V .
 Distributivity: ∀a ∈ F, ∀u, v ∈ V , a · (u + v ) = a · u + a · v ,
and ∀a, b ∈ F, ∀u ∈ V , (a +F b) · u = a · u + b · u.
 Compatibility of field and scalar multiplication:
∀a, b ∈ F, ∀u ∈ V , (a × b) · u = a · (b · u).
Examples of Vector spaces
(R, +, ·) over R.
(Rn , +, ·) over R.
(Cn , +, ·) over C.
(R∞ , +, ·) over R, where R∞ is the set of all doubly-infinite
sequences.
(P(R), +, ·) over R, where P(R) is the set of all polynomials of
one variable with real coefficients.
(L2 (R), +, ·) over R, where L2 (R) denotes the set of all
square-integrable functions f : R → R.
(Mn (R), +, ·) over R, where Mn (R) denotes the set of all
square matrices of size n with real number entries.
Properties of Vector Spaces
Proposition 1: Every Vector space has a unique additive
identity.
Properties of Vector Spaces
Proposition 1: Every Vector space has a unique additive
identity.
Proposition 2: Every vector in a vector space has a unique
additive inverse.
Properties of Vector Spaces
Proposition 1: Every Vector space has a unique additive
identity.
Proposition 2: Every vector in a vector space has a unique
additive inverse.
Proposition 3: ∀v ∈ V , 0 · v = θ
Properties of Vector Spaces
Proposition 1: Every Vector space has a unique additive
identity.
Proposition 2: Every vector in a vector space has a unique
additive inverse.
Proposition 3: ∀v ∈ V , 0 · v = θ
Proposition 4: ∀a ∈ F, a · θ = θ.
Properties of Vector Spaces
Proposition 1: Every Vector space has a unique additive
identity.
Proposition 2: Every vector in a vector space has a unique
additive inverse.
Proposition 3: ∀v ∈ V , 0 · v = θ
Proposition 4: ∀a ∈ F, a · θ = θ.
Proposition 5: ∀v ∈ V , (−1) · v = −v .
Subspace
Definition: (Subspace) Let (V , +, ·) be a vector space over F.
A subset W ⊆ V is said to be a subspace of V if (W , +, ·) is a
Vector space over F.
Subspace
Definition: (Subspace) Let (V , +, ·) be a vector space over F.
A subset W ⊆ V is said to be a subspace of V if (W , +, ·) is a
Vector space over F.
Note on notation: Capitals(U) for subspaces, small letters(u)
for vectors(elements).
Subspace
Definition: (Subspace) Let (V , +, ·) be a vector space over F.
A subset W ⊆ V is said to be a subspace of V if (W , +, ·) is a
Vector space over F.
Note on notation: Capitals(U) for subspaces, small letters(u)
for vectors(elements).
Examples:
Subspace
Definition: (Subspace) Let (V , +, ·) be a vector space over F.
A subset W ⊆ V is said to be a subspace of V if (W , +, ·) is a
Vector space over F.
Note on notation: Capitals(U) for subspaces, small letters(u)
for vectors(elements).
Examples:
 For any vector space V , V and {θ} are always subspaces.
These are called trivial subspaces.
Subspace
Definition: (Subspace) Let (V , +, ·) be a vector space over F.
A subset W ⊆ V is said to be a subspace of V if (W , +, ·) is a
Vector space over F.
Note on notation: Capitals(U) for subspaces, small letters(u)
for vectors(elements).
Examples:
 For any vector space V , V and {θ} are always subspaces.
These are called trivial subspaces.
 V = R2 , W = {(x, y ) ∈ V | ax + by = c}.
Subspace
Definition: (Subspace) Let (V , +, ·) be a vector space over F.
A subset W ⊆ V is said to be a subspace of V if (W , +, ·) is a
Vector space over F.
Note on notation: Capitals(U) for subspaces, small letters(u)
for vectors(elements).
Examples:
 For any vector space V , V and {θ} are always subspaces.
These are called trivial subspaces.
 V = R2 , W = {(x, y ) ∈ V | ax + by = c}.W is a subspace iff
c = 0.
Subspace
Definition: (Subspace) Let (V , +, ·) be a vector space over F.
A subset W ⊆ V is said to be a subspace of V if (W , +, ·) is a
Vector space over F.
Note on notation: Capitals(U) for subspaces, small letters(u)
for vectors(elements).
Examples:
 For any vector space V , V and {θ} are always subspaces.
These are called trivial subspaces.
 V = R2 , W = {(x, y ) ∈ V | ax + by = c}.W is a subspace iff
c = 0.
 V = P(R), W = Pn (R), where Pn (R) denotes the set of all
polynomials of one variable with real coefficients with degree at
most n.
Subspace
Definition: (Subspace) Let (V , +, ·) be a vector space over F.
A subset W ⊆ V is said to be a subspace of V if (W , +, ·) is a
Vector space over F.
Note on notation: Capitals(U) for subspaces, small letters(u)
for vectors(elements).
Examples:
 For any vector space V , V and {θ} are always subspaces.
These are called trivial subspaces.
 V = R2 , W = {(x, y ) ∈ V | ax + by = c}.W is a subspace iff
c = 0.
 V = P(R), W = Pn (R), where Pn (R) denotes the set of all
polynomials of one variable with real coefficients with degree at
most n. R∞
 V = L2 (R), W = {f ∈ V | −∞ f (t) dt = 0}.
 Proposition 6: A non-empty subset W of a vector space V is a
subspace if and only if
 Proposition 6: A non-empty subset W of a vector space V is a
subspace if and only if
 W is closed with respect to vector addition, and
 Proposition 6: A non-empty subset W of a vector space V is a
subspace if and only if
 W is closed with respect to vector addition, and
 W is closed with respect to scalar multiplication.
 Proposition 6: A non-empty subset W of a vector space V is a
subspace if and only if
 W is closed with respect to vector addition, and
 W is closed with respect to scalar multiplication.
Familiar examples of Subspaces: Let A ∈ Rm×n .
 Proposition 6: A non-empty subset W of a vector space V is a
subspace if and only if
 W is closed with respect to vector addition, and
 W is closed with respect to scalar multiplication.
Familiar examples of Subspaces: Let A ∈ Rm×n . Then,
C (A), N(AT ) and N(A), C (AT ) are subspaces of Rm and Rn
respectively.
Generating New subspaces from Old
Let U, W be subspaces of V .
Is U ∪ W a subspace of V ?
Generating New subspaces from Old
Let U, W be subspaces of V .
Is U ∪ W a subspace of V ? No.
Is U ∩ W a subspace of V ?
Generating New subspaces from Old
Let U, W be subspaces of V .
Is U ∪ W a subspace of V ? No.
Is U ∩ W a subspace of V ? Yes.
Generating New subspaces from Old
Let U, W be subspaces of V .
Is U ∪ W a subspace of V ? No.
Is U ∩ W a subspace of V ? Yes.
Definition: (Sum of subspaces): Let U1 , . . . , Un be subspaces
of V . The sum of subspaces U1 , . . . , Un is defined as:

U1 + . . . + Un =: {u1 + u2 + . . . + un | ui ∈ Ui , i = 1, . . . , n}
Generating New subspaces from Old
Let U, W be subspaces of V .
Is U ∪ W a subspace of V ? No.
Is U ∩ W a subspace of V ? Yes.
Definition: (Sum of subspaces): Let U1 , . . . , Un be subspaces
of V . The sum of subspaces U1 , . . . , Un is defined as:

U1 + . . . + Un =: {u1 + u2 + . . . + un | ui ∈ Ui , i = 1, . . . , n}

Proposition 7: The sum of subspaces U1 , . . . , Un of V is a

subspace.
 If v = u1 + . . . + un , ui ∈ Ui , i = 1, . . . n, we say that
(u1 , . . . , un ) is a decomposition of v .
 If v = u1 + . . . + un , ui ∈ Ui , i = 1, . . . n, we say that
(u1 , . . . , un ) is a decomposition of v .
Is this decomposition unique?
 If v = u1 + . . . + un , ui ∈ Ui , i = 1, . . . n, we say that
(u1 , . . . , un ) is a decomposition of v .
Is this decomposition unique?
Definition: (Direct Sum of Subspaces) In a VS V with
subspaces U1 , . . . , Un , W = U1 + . . . + Un is said to be a Direct
Sum if ∀w ∈ W , w is uniquely expressed as a sum of elements
wi ∈ Ui , i = 1, . . . , n.
 If v = u1 + . . . + un , ui ∈ Ui , i = 1, . . . n, we say that
(u1 , . . . , un ) is a decomposition of v .
Is this decomposition unique?
Definition: (Direct Sum of Subspaces) In a VS V with
subspaces U1 , . . . , Un , W = U1 + . . . + Un is said to be a Direct
Sum if ∀w ∈ W , w is uniquely expressed as a sum of elements
wi ∈ Ui , i = 1, . . . , n.
Direct sum notation: W = U1 ⊕ U2 ⊕ . . . ⊕ Un .
 Proposition 8: Let U1 , . . . , Un be subspaces of V . Then
V = U1 ⊕ . . . ⊕ Un if and only if: (1) V = U1 + . . . + Un , and (2)
The only representation of θ ∈ V is (θ, . . . , θ).
 Proposition 9: Let V be a VS with subspaces U1 , U2 . Then
V = U1 ⊕ U2 iff V = U1 + U2 and U1 ∩ U2 = {θ}.
Span and Linear Independence
Let V be a VS, and U = {v1 , . . . , vn }.
Span and Linear Independence
Let V be a VS, and U = {v1 , . . . , vn }.
Definition: (Span of a set) We define the Span of U as

span(U) := {a1 v1 + . . . + an vn | ∀a1 , . . . , an ∈ F},

i.e., the set of all possible linear combinations of elements from U.

Span and Linear Independence
Let V be a VS, and U = {v1 , . . . , vn }.
Definition: (Span of a set) We define the Span of U as

span(U) := {a1 v1 + . . . + an vn | ∀a1 , . . . , an ∈ F},

i.e., the set of all possible linear combinations of elements from U.

If |U| = ∞?
Span and Linear Independence
Let V be a VS, and U = {v1 , . . . , vn }.
Definition: (Span of a set) We define the Span of U as

span(U) := {a1 v1 + . . . + an vn | ∀a1 , . . . , an ∈ F},

i.e., the set of all possible linear combinations of elements from U.

If |U| = ∞?
Is span(U) a subspace of V ?
Span and Linear Independence
Let V be a VS, and U = {v1 , . . . , vn }.
Definition: (Span of a set) We define the Span of U as

span(U) := {a1 v1 + . . . + an vn | ∀a1 , . . . , an ∈ F},

i.e., the set of all possible linear combinations of elements from U.

If |U| = ∞?
Is span(U) a subspace of V ?
Proposition 10: Let U ⊆ V . Then span(U) is a subspace of V .
 Let V be a VS, and let W ⊂ V . If span(W ) = V , we say that
W is a spanning set of V , or W spans V .
 Let V be a VS, and let W ⊂ V . If span(W ) = V , we say that
W is a spanning set of V , or W spans V .
We say that a vector space V is finite dimensional if there
exists a finite spanning set. Acronym for finite dimensional vector
space: FDVS.
 Let V be a VS, and let W ⊂ V . If span(W ) = V , we say that
W is a spanning set of V , or W spans V .
We say that a vector space V is finite dimensional if there
exists a finite spanning set. Acronym for finite dimensional vector
space: FDVS.
Linearly independent set: Let V be a vector space and let
W = {v1 , . . . , vn } ⊂ V . We say that the set W is a set of linear
independent vectors, if

a1 v1 + . . . + an vn = θ ⇒ ai = 0, i = 1, . . . , n
 Let V be a VS, and let W ⊂ V . If span(W ) = V , we say that
W is a spanning set of V , or W spans V .
We say that a vector space V is finite dimensional if there
exists a finite spanning set. Acronym for finite dimensional vector
space: FDVS.
Linearly independent set: Let V be a vector space and let
W = {v1 , . . . , vn } ⊂ V . We say that the set W is a set of linear
independent vectors, if

a1 v1 + . . . + an vn = θ ⇒ ai = 0, i = 1, . . . , n

What if |W | = ∞.
 Proposition 11: In a FDVS, the number of elements in a
linearly independent set of vectors is always less than equal to the
number of elements in a spanning set.
 Proposition 11: In a FDVS, the number of elements in a
linearly independent set of vectors is always less than equal to the
number of elements in a spanning set.
Proof: Let V be a VS and
U = {u1 , . . . , un }, W = {w1 , . . . , wm } be its subsets such that
span(U) = V and W is LI.
 Proposition 11: In a FDVS, the number of elements in a
linearly independent set of vectors is always less than equal to the
number of elements in a spanning set.
Proof: Let V be a VS and
U = {u1 , . . . , un }, W = {w1 , . . . , wm } be its subsets such that
span(U) = V and W is LI.
{w1 , u1 , . . . , un } is
 Proposition 11: In a FDVS, the number of elements in a
linearly independent set of vectors is always less than equal to the
number of elements in a spanning set.
Proof: Let V be a VS and
U = {u1 , . . . , un }, W = {w1 , . . . , wm } be its subsets such that
span(U) = V and W is LI.
{w1 , u1 , . . . , un } is LD,
 Proposition 11: In a FDVS, the number of elements in a
linearly independent set of vectors is always less than equal to the
number of elements in a spanning set.
Proof: Let V be a VS and
U = {u1 , . . . , un }, W = {w1 , . . . , wm } be its subsets such that
span(U) = V and W is LI.
{w1 , u1 , . . . , un } is LD, i.e.,
∃uj ∈ U, uj ∈ span({w1 , ui , i = 1, . . . , n, i 6= j}).
 Proposition 11: In a FDVS, the number of elements in a
linearly independent set of vectors is always less than equal to the
number of elements in a spanning set.
Proof: Let V be a VS and
U = {u1 , . . . , un }, W = {w1 , . . . , wm } be its subsets such that
span(U) = V and W is LI.
{w1 , u1 , . . . , un } is LD, i.e.,
∃uj ∈ U, uj ∈ span({w1 , ui , i = 1, . . . , n, i 6= j}).
span({w1 , ui , i = 1, . . . , n, i 6= j}) =
 Proposition 11: In a FDVS, the number of elements in a
linearly independent set of vectors is always less than equal to the
number of elements in a spanning set.
Proof: Let V be a VS and
U = {u1 , . . . , un }, W = {w1 , . . . , wm } be its subsets such that
span(U) = V and W is LI.
{w1 , u1 , . . . , un } is LD, i.e.,
∃uj ∈ U, uj ∈ span({w1 , ui , i = 1, . . . , n, i 6= j}).
span({w1 , ui , i = 1, . . . , n, i 6= j}) = V
 Proposition 11: In a FDVS, the number of elements in a
linearly independent set of vectors is always less than equal to the
number of elements in a spanning set.
Proof: Let V be a VS and
U = {u1 , . . . , un }, W = {w1 , . . . , wm } be its subsets such that
span(U) = V and W is LI.
{w1 , u1 , . . . , un } is LD, i.e.,
∃uj ∈ U, uj ∈ span({w1 , ui , i = 1, . . . , n, i 6= j}).
span({w1 , ui , i = 1, . . . , n, i 6= j}) = V
{w1 , w2 , ui , i = 1, . . . , n, i 6= j} is LD
 Proposition 11: In a FDVS, the number of elements in a
linearly independent set of vectors is always less than equal to the
number of elements in a spanning set.
Proof: Let V be a VS and
U = {u1 , . . . , un }, W = {w1 , . . . , wm } be its subsets such that
span(U) = V and W is LI.
{w1 , u1 , . . . , un } is LD, i.e.,
∃uj ∈ U, uj ∈ span({w1 , ui , i = 1, . . . , n, i 6= j}).
span({w1 , ui , i = 1, . . . , n, i 6= j}) = V
{w1 , w2 , ui , i = 1, . . . , n, i 6= j} is LD i.e.,
∃uk ∈ U, k 6= j, uk ∈ span({w1 , w2 , ui , i = 1, . . . , n, i 6= j, k}).
 Proposition 11: In a FDVS, the number of elements in a
linearly independent set of vectors is always less than equal to the
number of elements in a spanning set.
Proof: Let V be a VS and
U = {u1 , . . . , un }, W = {w1 , . . . , wm } be its subsets such that
span(U) = V and W is LI.
{w1 , u1 , . . . , un } is LD, i.e.,
∃uj ∈ U, uj ∈ span({w1 , ui , i = 1, . . . , n, i 6= j}).
span({w1 , ui , i = 1, . . . , n, i 6= j}) = V
{w1 , w2 , ui , i = 1, . . . , n, i 6= j} is LD i.e.,
∃uk ∈ U, k 6= j, uk ∈ span({w1 , w2 , ui , i = 1, . . . , n, i 6= j, k}).
span({w1 , w2 , ui , i = 1, . . . , n, i 6= j, k}) = V
 Proposition 11: In a FDVS, the number of elements in a
linearly independent set of vectors is always less than equal to the
number of elements in a spanning set.
Proof: Let V be a VS and
U = {u1 , . . . , un }, W = {w1 , . . . , wm } be its subsets such that
span(U) = V and W is LI.
{w1 , u1 , . . . , un } is LD, i.e.,
∃uj ∈ U, uj ∈ span({w1 , ui , i = 1, . . . , n, i 6= j}).
span({w1 , ui , i = 1, . . . , n, i 6= j}) = V
{w1 , w2 , ui , i = 1, . . . , n, i 6= j} is LD i.e.,
∃uk ∈ U, k 6= j, uk ∈ span({w1 , w2 , ui , i = 1, . . . , n, i 6= j, k}).
span({w1 , w2 , ui , i = 1, . . . , n, i 6= j, k}) = V
The number of elements in the set
{w1 , w2 , ui , i = 1, . . . , n, i 6= j, k} remains n.
 Proposition 11: In a FDVS, the number of elements in a
linearly independent set of vectors is always less than equal to the
number of elements in a spanning set.
Proof: Let V be a VS and
U = {u1 , . . . , un }, W = {w1 , . . . , wm } be its subsets such that
span(U) = V and W is LI.
{w1 , u1 , . . . , un } is LD, i.e.,
∃uj ∈ U, uj ∈ span({w1 , ui , i = 1, . . . , n, i 6= j}).
span({w1 , ui , i = 1, . . . , n, i 6= j}) = V
{w1 , w2 , ui , i = 1, . . . , n, i 6= j} is LD i.e.,
∃uk ∈ U, k 6= j, uk ∈ span({w1 , w2 , ui , i = 1, . . . , n, i 6= j, k}).
span({w1 , w2 , ui , i = 1, . . . , n, i 6= j, k}) = V
The number of elements in the set
{w1 , w2 , ui , i = 1, . . . , n, i 6= j, k} remains n.
Is it possible that m > n?
 Proposition 11: In a FDVS, the number of elements in a
linearly independent set of vectors is always less than equal to the
number of elements in a spanning set.
Proof: Let V be a VS and
U = {u1 , . . . , un }, W = {w1 , . . . , wm } be its subsets such that
span(U) = V and W is LI.
{w1 , u1 , . . . , un } is LD, i.e.,
∃uj ∈ U, uj ∈ span({w1 , ui , i = 1, . . . , n, i 6= j}).
span({w1 , ui , i = 1, . . . , n, i 6= j}) = V
{w1 , w2 , ui , i = 1, . . . , n, i 6= j} is LD i.e.,
∃uk ∈ U, k 6= j, uk ∈ span({w1 , w2 , ui , i = 1, . . . , n, i 6= j, k}).
span({w1 , w2 , ui , i = 1, . . . , n, i 6= j, k}) = V
The number of elements in the set
{w1 , w2 , ui , i = 1, . . . , n, i 6= j, k} remains n.
Is it possible that m > n?
If so, after n iterations, we will reach a contradiction:
span({w1 , w2 , . . . , wn }) = V
Basis of a Vector space
Definition: (Hamel Basis) Let V be a finite dimensional vector
space. An ordered set β := {v1 , . . . , vn } is said to be a (Hamel)
basis of V if (1) span(β) = V , and (2) β is a set of linearly
independent vectors.
Basis of a Vector space
Definition: (Hamel Basis) Let V be a finite dimensional vector
space. An ordered set β := {v1 , . . . , vn } is said to be a (Hamel)
basis of V if (1) span(β) = V , and (2) β is a set of linearly
independent vectors.
Examples:
 Proposition 12: For a FDVS, every spanning set can be
reduced to a basis.
 Proposition 12: For a FDVS, every spanning set can be
reduced to a basis.
Proposition 13: Every FDVS has a basis.
 Proposition 12: For a FDVS, every spanning set can be
reduced to a basis.
Proposition 13: Every FDVS has a basis.
Proposition 14: Any set of basis vectors of a VS contains the
same number of elements.
 Proposition 12: For a FDVS, every spanning set can be
reduced to a basis.
Proposition 13: Every FDVS has a basis.
Proposition 14: Any set of basis vectors of a VS contains the
same number of elements.
Dimension of a Vector Space: Let V be a FDVS. For any set
of basis vectors β of V , we define the dimension of V as
dim(V ) := |β|.