EE 650 - Basics of Modern Control Systems

Vector Spaces
Definition, Linear independence, Linear combination, Span, Basis, Dimension, Subspace, Linear transforma-
tions, Invariance, Change of basis, Similarity transformation, Examples.

1 Definition
A vector space is a set V of elements called vectors which satisfy the following axioms:

x, y V, x + y V. (CLOSURE)

x, y V, x + y = y + x. (COMMUTATIVE)

x, y, z V, x + (y + z) = (x + y) + z. (ASSOCIATIVE)

in V a unique vector 0 such that x + 0 = x, X V. (ADDITIVE IDENTITY)

to every vector x V there corresponds a unique vector x such that x + (x) = 0. (ADDITIVE
INVERSE)

F and x V, x V. (CLOSURE UNDER SCALAR MULTIPLICATION)

, F and x V, (x) = ()x.

1x = x for every vector x V.

(x + y) = x + y F and x, y V. (DISTRIBUTIVE w.r.t vectors)

( + )x = x + x , F and x V. (DISTRIBUTIVE w.r.t. scalars)

1.1 Examples

Z - set of integers, Q - set of rationals, R - real line, Rn , P - set of polynomials and lots of more.
Is a set of 3 3 matrices a vector spaces? Is a set of 3 3 invertible matrices a vector space?

2 Linear Independence
A finite set {vi } of vectors is linearly dependent if there exists a corresponding set {i } of scalars, not all zero
such that
X
i vi = 0.
i
P
i i vi = 0 i = 0 i = 1, 2, . . . and nonzero vi V
m
The set {vi } of vectors is linearly independent.

1
3 Linear combinations
Every vector in V can be expressed as a linear combination of other vectors in V.
X
v= i w i ,
i

where v, wi V and i F for all i = 1, 2, . . .. A set {xi } of vectors in V is the span of V if every vector in V
can be expressed as a linear combination of the vectors xi .

A basis in a vector space V is a set X of linearly independent vectors such that every vector in V is a linear
combination of elements of X . In other words, basis in V is a set of linearly independent vectors that span the
entire vector space V. If the basis of V has finite number of elements then the vector space V is finite-dimensional.

3.1 Examples
{(1, 0, 0), (1, 2, 1), (1, 3, 4), (0, 1, 0)} span R3 . Note that the vectors in the set are not linearly independent.
{(1, 0, 0), (0, 1, 0), (0, 0, 1)} are linearly independent and span R3 . So the set is the basis of R3 . Similarly,
{(1, 0, 0), (0, 1, 0), (1, 2, 1)} is a basis of R3 . The basis of a vector space is not unique. The dimension of R3 is
three. A set of polynomials P is infinite-dimensional.

A vector space definition is complete with its underlying field (from where the scalars are taken). To illus-
trate this consider the example of complex vector space C2 and real vector space C2 . A vector in C2 is
( + i, + i). The basis for the former vector space is {(1, 0, 0), (0, 1, 0), (0, 0, 1)} and for the latter is
{(1, 0, 0), (i, 0, 0), (0, 1, 0), (0, i, 0), (0, 0, 1), (0, 0, i)}. While the former vector space is of dimension three, the
latter has a dimension of six.

4 Subspace
A non-empty subset M of a vector space V is a subspace if along with every pair, x and y, of vectors contained
in M, every linear combination x + y is also contained in M. Since M is a subset of V it inherits all the
properties of V. The 0 of V is also contained in M. The subspace M is itself a vector space.

Remark: The two trivial subspaces of a vector space V are

(i) the origin (or the 0) only.
(ii) the whole vector space V.
Let the dimension of a vector space V be n and M is a subspace of V. Then the dimension of M is m n.

4.1 Examples
(i) The real line R is a subspace of the vector space R2 . Similarly, R2 is a subspace of the vector space R3 .
(ii) The polynomial set P n (which contains polynomials of the order < n) is a subspace of P.
(iii) A set of 3 3 skew-symmetric matrices S is a subspace of the vector space containing all 3 3 matrices.

5 Linear Transformations
A linear transformation (or operator) A on a vector space V is a correspondence that assigns to every vector x
in V a vector Ax in V, in such a way that

A(x + y) = Ax + Ay

2
identically in the vectors x and y and the scalars and .

5.1 Examples
Pn1 j Pn1
For any x P n , x(t) = j=0 j t , we define (Dx)(t) = j=0 jj tj1 . Here D is a linear transformation
which the derivative action.

6 Transformations as vectors
The set of all linear transformations on a vector space is itself a vector space.

Let A, B and C be three linear transformations on the vector space V. It is easy to see that
(A + B)x = (B + A)x V.
(A + (B + C))x = ((A + B) + C)x.
(A + 0)x = Ax = (0 + A)x.
If we define the transformation (A) such that (A)x = (Ax), then A + (A) = 0.
For any scalar and any transformation A, we define the product A as (A)x = (Ax). (A)x V.
((A))x = (()A)x.
(1A)x = Ax.
((A + B))x = (A + B)x.
(( + )A)x = (A + A)x.
A few more properties of linear transformations:
A0 = 0A = 0
A1 = 1A = A
A(B + C) = AB + AC
(A + B)C = AC + BC
A(BC) = (AB)C

7 Inverses of Transformations
The transformation A is invertible if and only if
If x1 6= x2 , then Ax1 6= Ax2 .
To every vector y there corresponds (at least) one vector x such that Ax = y.
The inverse of A is denoted by A1 . For any invertible A we have

AA1 = A1 A = 1

A linear transformation A on a finite-dimensional vector space V is invertible if and only if Ax = 0 implies that
x = 0, or, alternatively, if and only if every y in V can be written in the form y = Ax.

If A and B invertible, then AB is invertible and (AB)1 = B 1 A1 . If A invertible and 6= 0, then A

1
is invertible and (A)1 = A1 . If A is invertible, then A1 is invertible and (A1 )1 = A.

3
8 Matrix association with a Linear transformation
Let V be an ndimensional vector space, let X = {x1 , , xn } be any basis of V, and let A be a linear
transformation on V. Since every vector is a linear combination of the xi , we have in particular
X
Axj = ij xi
i

for j = 1, , n. The matrix associated with the linear transformation A is denoted by [A] and is

11 12 1n
21 22 2n
[A] = ..

.. ..
. . .
n1 n2 nn

Note that if the basis changes, the matrix [A] associated with the linear transformation A also changes. Let us
see an example. Consider the differential transformation D on the space P n , and the basis {x1 , , xn } defined
by xi = ti1 , i = 1, , n. We have to find the matrix [D] associated with the D. We have

Dx1 = 0x1 + 0x2 + + 0xn1 + 0xn

Dx2 = 1x1 + 0x2 + + 0xn1 + 0xn
Dx3 = 0x1 + 2x2 + + 0xn1 + 0xn
.. ..
.=.
Dxn = 0x1 + 0x2 + + (n 1)xn1 + 0xn

So

0 1 0 0
0
0 2 0

[D] = ... .. .. ..

. . .

0 0 0 n 1
0 0 0 0

A polynomial p(t) = 1 + 2 t + 3 t2 + + n tn1 in P n can be represented as a vector [1 2 3 n ]T

in Rn . Hence, we have

D.x1 = D.1 = [D][1 0 0 0]T = [0 0 0 0 0]T

D.x2 = D.t = [D][0 1 0 0]T = [1 0 0 0 0]T
D.x3 = D.t2 = [D][0 0 1 0]T = [0 2 0 0 0]T
.. ..
.=.
D.xn = D.tn1 = [D][0 0 0 1]T = [0 0 0 n 1 0]T

In a fixed coordinate system X = {x1 , , xn } and with the knowledge of the matrices of A and B we have the
following:

If C = A + B, then [C] = (ij ) where ij = aij + bij .

P
If C = AB, then [C] = (ij ) where ij = k ik kj .

If [0] = (oij ) and [1] = (eij ), then oij = 0 and eij = ij (Kroneckerdelta).

4
 P P
Let x V and x = j j xj . Let y = Ax. Since y V, y = i i xi . We have to find what is the relation
between i and j when the matrix [A] is known.
X X X XX
Ax = j Axj = (j ( ij xi )) = ( ij j )xi
j j i i j
P P
But we have y = Ax = i i xi . So we have i = j ij xij .

9 Invariance
Let V be a vector space of which M is a subspace. M is invariant under the linear transformation A if x M,
Ax is also in M. Let the basis X = {x P 1 , , xn } of V be such that x1 , , xm are in M and xm+1 , , xn
are not.PFor m + 1 j n, Axj = i ij xi . For 1 j m, Axj M as M is invariant under A. So
Axj = m i=1 ij xi . Hence, ij = 0 for j = m + 1, , n. Hence, the matrix [A] of A, in this coordinate system
will have the form
 
[A1 ] [B0 ]
[A] =
[0] [A2 ]

where [A1 ] is the matrix of A considered as a linear transformation on the space M with respect to the coordinate
system {x1 , , xm }, [A2 ] and [B0 ] are some arrays of scalars, and [0] denotes the array consisting of zeros only.
What are the sizes of these matrices?

10 Change of basis
Let V be an ndimensional
P Pvector space and let X = {x1 , , xn } and Y = {y1 , , yn } be two bases of V.
If x is in V, x = i i xi = i i yi , what is the relation between its coordinates (1 , , n ) with respect to X
and its coordinates (1 , , n ) with respect to Y?

Consider a linear transformation A defined by Axi = yi , i = 1, , n. So

X X
A( i xi ) = i yi .
i i
P
Let (i j ) be the matrix of A in the basis X
P , that is, y j = Axj = i ij xi . Note that for every y in V there exists
a vector x in V such that Ax = y. Since i i yi = 0 i = 0, i = 1, , n, so Ax = 0 x = 0. Hence, A is
invertible.

X X X X
j yj = j Axj = j ij xi
j j j i
XX
= ( ij j )xi
i j

ij j , i = 1, , n. Hence, = [A], where = [1 , , n ]T and = [1 , , n ]T .

P
So i = j

11 Similarity
First we will try to establish a few relations.

Consider a linear transformation B on the ndimensional vector space V. Let X and Y be the two bases of
V. Let the matrices associated with B with respect to X and Y be (ij ) and (ij ) respectively. How are these

5
matrices related to each other?

We have
X
Bxj = ij xi
i

and
X
Byj = ij yi
i

Let A be a linear transformation on V defined by Axi = yi , i = 1, , n. Now we have

X
Byj = BAxj = B( kj xk )
k
X X X
= kj Bxk = kj i kxi
k k i
XX
= ( ik kj )xi .
i k

Also
X X
Byj = kj yk = kj Axk
k k
X X XX
= kj ik xi = ( ik kj )xi .
k i i k

Comparing the equations we get

X X
ik kj = ik kj .
k k

Hence, we have

(ij )(ij ) = (ij )(ij )

(ij ) = (ij )1 (ij )(ij )

Two matrices (ij ) and (ij ) are said to be similar if there exists an invertible matrix (ij ) such that

(ij ) = (ij )1 (ij )(ij )

Next we look into another case.

Let B and C be two linear transformations on the vector space V. Let X and Y be two bases of V. Let the
matrix of B with respect to X be (ij ). Let the matrix of C with respect to Y be (ij )? How are the two linear
transformations related?

Note that there are two liner transformations P and the matrix ofPboth transformations with respect to two dif-
ferent bases are same. So we have Bxj = i ij xj and Cyj = i ij yi . Consider an invertible transformation
A on V defined by Axi = yi , i = 1, , n. We have

Cyj = CAxj

6
and
X X X
Cyj = ij yi = ij Axi = A( ij xi ) = ABxj
i i i

So, we have

CA = AB
C = ABA1

Two linear transformations B and C are said to be similar if there exists an invertible transformation A such that
C = ABA1 .

B
x u

A A

y v
C