MATHS 2 | Amethyst Minds

4️⃣
IIT M BS Data Science
Week
L4.1 What is the basis for a vector space?
Subspaces
Subspaces are smaller sets within vector spaces that are themselves
vector spaces.

To qualify as a subspace, the smaller set must satisfy closure

properties.

Checking Closure for Subspaces:

Closure means that when you multiply a vector by a scalar or add

two vectors from the set, the result remains in the set.

MATHS 2 | Amethyst Minds 1

 Span of a Set of Vectors
Linear combination: The sum of elements from a vector space
multiplied by scalars.

Span: The set of all possible linear combinations of elements.

It is denoted as span(S), where S is the set of vectors.

The span of elements is a subspace of the vector space.

It includes the zero vector as a result of the empty linear

combination.

Some Examples of Spanning Sets:

Example 1: Span of {1, 0} in R2 is the x-axis.

Example 2: Span of {1, 1} in R2 is the line y = x.

MATHS 2 | Amethyst Minds 2

 Example 3: Span of {(1, 0, 0),(0,1,0)} in R3 is the x-y plane.

Example 4:

MATHS 2 | Amethyst Minds 3

 Hence span is the smallest subspace containing the given
elements, representing their combinations.

Spanning Set and Subsets:

If a set T is contained in a set S, then span(T) is contained in

span(S).

If T is a spanning set, S (a superset of T) is also a spanning set.

Building a Spanning Set:

Starting with an empty set (S0 ), vectors are appended one by one.
​

Vectors are chosen to be outside the span of the vectors added so

far.

This process continues until a spanning set is formed.

The Concept of a Basis
A basis (B) for a vector space V is a linearly independent subset of V
that also spans V.

A basis is a set that optimally balances size and linear independence

while ensuring full coverage of the vector space.

Standard Basis for Rn:

In Rn, the standard basis consists of vectors with one element

being 1 and others being 0.

Example: In R3, the standard basis includes {1, 0, 0}, {0, 1, 0}, and
{0, 0, 1}.

This basis is both linearly independent and spanning for Rn.

MATHS 2 | Amethyst Minds 4

 Building a spanning set by appending vectors outside the span is a
method to find a basis.
L4.2 Finding bases for vector spaces
Finding Bases
A subset B of V is a basis if and only if it meets these conditions:

B is linearly independent.

The span of B is V.

B is a maximal linearly independent set.

B is a minimal-spanning set.

Maximal Linearly Independent set

Linearly independent, and appending any vector makes it

dependent.

Minimal Spanning Sets

Spanning, and removing any vector makes it no longer spanning.

Methods for Finding a Basis:

Method 1: Start with an empty set, append vectors not in the

current span, until we obtain a spanning set.

Method 2: Begin with a spanning set, and delete vectors that can
be expressed as linear combinations of others until no further
deletions are possible.

Example of Finding Bases

Example 1: Starting with an empty set, append vectors (1, 2) and (2,
3) to form a basis for R2.

MATHS 2 | Amethyst Minds 5

 Example 2: Starting with a spanning set, delete vectors (0, 4, 2) and
(0, 2, 3) to form a basis for R3.

L4.3 Rank/Dimension for a vector space

Definition of Rank/Dimension
The dimension (also called rank) of a vector space is determined by
the size (or cardinality) of its basis.

MATHS 2 | Amethyst Minds 6

 In general, vector spaces may have bases of infinite size. Still, for the
purposes of this course, our focus is primarily on finite sets as they
align with the finite datasets commonly encountered in introductory
data science.

Therefore in this context, rank refers to the number of elements in

the basis set.

Every vector space has a basis.

All bases in a vector space contain the same number of elements.

The dimension or rank of a vector space V is uniquely defined as dim

(V )or rank(V ).
The dimension of Rn 
Each space has 'standard basis vectors'.

A standard basis vector has mostly zeros, with a single '1' in it.

For example, in R3 , the standard basis vectors are (1,0,0), (0,1,0),

and (0,0,1).

In general

The ith standard basis vector in Rn is denoted as ei . ​

ei is a vector with a ‘1’ in the ith position and ‘0’ in all other
​

positions.

The set [e1 , e2 , ..., en ]forms the standard basis of Rn .

​ ​ ​

The total number of standard basis vectors in Rn is n.

Therefore, the dimension of Rn is also n.

Example

MATHS 2 | Amethyst Minds 7

 Using Matrices to Find Dimension
The vectors that span (or generate) subspace W are written as rows
in a matrix:

MATHS 2 | Amethyst Minds 8

 This final matrix is in the reduced row echelon form of the initial
matrix.

The rows of this matrix give a basis for the subspace W .

Notably, there are 2 non-zero rows in the final matrix.

Thus, the dimension of the subspace W is 2, denoted as

dim(W ) = 2.
The Rank of a matrix
We're considering a matrix Awith dimensions m × n.

The column space of the matrix A:

Represents the subspace of Rm .

Is spanned by the column vectors of the matrix A.

The row space of the matrix A:

Represents the subspace of Rn .

Is spanned by the row vectors of the matrix A.

Column rank of the matrix A:

Refers to the dimension of its column space.

MATHS 2 | Amethyst Minds 9

 Row rank of the matrix A:

Refers to the dimension of its row space.

A key fact:

The column rank of the matrix Ais always equal to its row rank.

This common number is known as the rank (dimension) of the

matrix A.
L4.4 Rank/Dimension using Gaussian
elimination
Using Gaussian Elimination
Gaussian elimination is a direct algebraic method to find dimension
and basis.

Steps:

1. Form a matrix with these vectors as rows.

2. Reduce the matrix to row echelon form.

3. The number of non-zero rows in the resulting matrix is the

dimension.

4. The non-zero rows correspond to the basis.

It was previously used to find solutions to systems of linear equations.

Example 1

MATHS 2 | Amethyst Minds 10

 Example 2

Column-Based Method
The row-based method produces a basis but may not contain the
original vectors from the spanning set.

The column-based method aims to find a basis with vectors from the
original set.

A fact is used: Columns of a matrix resulting from row reduction with

pivot elements form a basis for the column space.

Pivot elements are non-zero elements in a matrix after performing

Gaussian elimination.

Here's a step-by-step explanation of the column-based method:

1. Start with a spanning set and write vectors as columns in a matrix.

2. Perform row reduction on the matrix to obtain row echelon form.

3. Identify pivot elements (leading ones) in the matrix.

MATHS 2 | Amethyst Minds 11

 4. Pivot columns correspond to vectors in the basis.

5. The dimension of the subspace is the number of pivot elements.

6. This method yields a basis including vectors from the original set if
desired.

Example 1

Example 2

MATHS 2 | Amethyst Minds 12

 Summary
Two algorithmic methods to obtain a basis from a spanning set were
discussed.

The row-based method converts vectors to rows, while the column-

based method uses vectors as columns.

Both methods yield the dimension and a basis for the vector space,
but the column-based method provides vectors from the original
spanning set as the basis elements

MATHS 2 | Amethyst Minds 13