eNote 7 1

eNote 7

Matrices and Matrix Algebra

This eNote introduces matrices and arithmetic operations for matrices and deduces the relevant
arithmetic rules. Math knowledge comparable to that of a Danish gymnasium (high school)
graduate is the only requirement for benefitting from this note, but it is a good idea to acquaint
oneself with the number space Rn that is described in eNote 5 The Number Spaces.

(Updated: 24.09.2021 David Brander)

A matrix is an array of numbers. Here is an example of a matrix called M:

 
1 4 3
M= (7-1)
−1 2 7

A matrix is characterized by the number of rows and columns, and the matrix M above
is therefore called a 2 × 3 matrix. The matrix M is said to contain 2 · 3 = 6 elements. In
addition to rows and columns a number of further concepts are connected. In order to
describe these we write a general matrix, here called A, as:
 
a11 a12 . . . a1n
a
 21 a22 . . . a2n 

A = . .. ..  (7-2)
 .. . . 
am1 am2 . . . amn

The matrix A has m rows and n columns, and this can indicated by writing Am×n or the
m × n matrix A. The matrix A is also said to be of the type m × n.

Two m × n-matrices A and B are called equal if the elements in each matrix are equal,
eNote 7 7.1 MATRIX SUM AND THE PRODUCT OF A MATRIX BY A SCALAR 2

and we then write A = B .

A matrix with a single column (n = 1), is called a column matrix. Similarly a matrix with
only one row (m = 1), a row matrix.

A matrix with the same number of row and columns (m = n), is called a square matrix.
Square matrices are investigated in depth in eNote 8 Square Matrices.

If all the elements in an m × n-matrix are real numbers, the matrix is called a real matrix.
The set of these matrices is denoted Rm×n .

7.1 Matrix Sum and the Product of a Matrix by a Scalar

It is possible to add two matrices if they are of the same type. You then add the elements
with the same row and column numbers and in this way form a new matrix of the same
type. Similarly you can multiply any matrix by a scalar (a number), this is done by
multiplying all the elements by the scalar. The matrix in which all elements are equal to
0 is called the zero matrix regardless of the type, and is denoted 0 or possibly 0m×n . In
these notes, all other matrices are called proper matrices.
eNote 7 7.1 MATRIX SUM AND THE PRODUCT OF A MATRIX BY A SCALAR 3

Definition 7.1 Matrix Sum and Multiplication by a Scalar

Given a scalar k ∈ R and two real matrices Am×n and Bm×n :
   
a11 a12 . . . a1n b11 b12 . . . b1n
a
 21 a22 . . . a2n   b21 b22 . . . b2n
  
A = . og B =  . (7-3)

. . .. ..
 .. .. ..   ..
 
. . 
am1 am2 . . . amn bm1 bm2 . . . bmn

The sum of the matrices is defined as:

 
a11 + b11 a12 + b12 ... a1n + b1n
 a +b a22 + b22 ... a2n + b2n 
 21 21
A + B = (7-4)

.. .. .. 
 . . . 
am1 + bm1 am2 + bm2 . . . amn + bmn

The sum is only defined when the matrices are of the same type.

The product of the matrix A by the scalar k is written kA or Ak and is defined as:
 
k · a11 k · a12 . . . k · a1n
 k·a
21 k · a22 . . . k · a2n 

kA = Ak =  . (7-5)

.. .. 
 .. . . 
k · am1 k · am2 . . . k · amn

The opposite matrix −A (additive inverse) to a matrix A is defined by the matrix that
results when all the elements in A are multiplied by −1 . It is seen that −A = (−1)A .

Example 7.2 Simple Matrix Operations

Define two matrices A andB by:

   
4 −1 −4 3
A= and B = (7-6)
8 0 9 12

The matrices are both of the type 2 × 2. We wish to determine a third and fourth matrix
eNote 7 7.1 MATRIX SUM AND THE PRODUCT OF A MATRIX BY A SCALAR 4

C = 4A and D = 2A + B. This can be done through the use of the definition 7.1.
     
4 −1 4 · 4 4 · (−1) 16 −4
C = 4A = 4 · = =
8 0 4·8 4·0 32 0
      (7-7)
8 −2 −4 3 4 1
D = 2A + B = + 1 =
16 0 9 2 25 21

In the following theorem we summarize the arithmetic rules that are valid for sums of
matrices and multiplication by a scalar.

Theorem 7.3 Arithmetic Rules for the Matrix Sum and the Product by a
Scalar
For arbitrary matrices A, B and C in Rm×n and likewise arbitrary real numbers k1
and k2 the following arithmetic rules are valid:
1. A+B = B+A Addition is commutative
2. (A + B) + C = A + (B + C) Addition is associative
3. A+0 = A 0 is a neutral matrix for addition in Rm×n
4. A + (−A) = 0 Every matrices in Rm×n has an opposite matrix
5. k 1 ( k 2 A) = ( k 1 k 2 )A Product of a matrix by scalars is associative

6. ( k 1 + k 2 )A = k 1 A + k 2 A
The distributive rules are valid
7. k 1 (A + B) = k 1 A + k 1 B
8. 1A = A The scalar 1 is neutral in the product by a matrix

The arithmetic rules in Theorem 7.3 can be proved by applying the ordinary arithmetic
rules for real numbers. The method is demonstrated for two of the rules in the following
example.

Example 7.4 Demonstration of Arithmetic Rule

Given the two matrices

   
a a b b
A = 11 12 and B = 11 12 (7-8)
a21 a22 b21 b22

plus the constants k1 and k2 . We now try by way of example to show the distributive rules in
eNote 7 7.1 MATRIX SUM AND THE PRODUCT OF A MATRIX BY A SCALAR 5

Theorem 7.3. First we have:

   
a11 a12 (k1 + k2 ) a11 (k1 + k2 ) a12
( k 1 + k 2 )A = ( k 1 + k 2 ) =
a21 a22 (k1 + k2 ) a21 (k1 + k2 ) a22
      (7-9)
k1 a11 k1 a12 k2 a11 k2 a12 k1 a11 + k2 a11 k1 a12 + k2 a12
k1 A + k2 A = + =
k1 a21 k1 a22 k2 a21 k2 a22 k1 a21 + k2 a21 k1 a22 + k2 a22

If you take a11 , a12 , a21 anda22 outside the parentheses in each of the elements in the last ex-
pression, it is seen that (k1 + k2 )A = k1 A + k2 A in this case. The operation of taking the
a-elements outside the parentheses is exactly equivalent to be using the distributive rule for
the real numbers.

The second distributive rule is demonstrated for given matrices and constants:
   
a11 + b11 a12 + b12 k ( a + b11 ) k1 ( a12 + b12 )
k 1 (A + B) = k 1 = 1 11
a21 + b21 a22 + b22 k1 ( a21 + b21 ) k1 ( a22 + b22 )
      (7-10)
k a k a k b k b k a + k1 b11 k1 a12 + k1 b12
k1 A + k1 B = 1 11 1 12 + 1 11 1 12 = 1 11
k1 a21 k1 a22 k1 b21 k1 b22 k1 a21 + k1 b21 k1 a22 + k1 b22

If k1 is taken outside of the parenthesis in each of the elements in the matrix in the last ex-
pression it is seen that the second distributive rule also is valid in this case: k1 (A + B) =
k1 A + k1 B. The distributive rule for real numbers is again used for each element.

Note that the zero matrix in Rm×n is the only matrix Rm×n that is neutral with
respect to addition, and that −A is the only solution to the equation A + X = 0.

Definition 7.5 Difference Between Matrices

The difference A − B between two matrices A and B of the same type is introduced
by:
A − B = A + (−B). (7-11)
In other words B is subtracted from A by subtracting each element in B from the
corresponding element in A.
eNote 7 7.2 MATRIX-VECTOR PRODUCTS AND MATRIX-MATRIX PRODUCTS 6

Example 7.6 Simple Matrix Operation with Difference

With the matrices given in Example 7.2 we get

     
8 −2 4 −3 12 −5
D = 2A − B = 2A + (−1)B = + = (7-12)
16 0 −9 − 12 7 − 12

7.2 Matrix-Vector Products and Matrix-Matrix Products

In this subsection we describe the multiplication of a matrix with a vector and then the
multiplication of matrix by another matrix.

A vector v = (v1 , v2 , . . . , vn ) can be written as a column matrix and is then called a

column vector:  
v1
 v2 
v = ( v1 , v2 , . . . , v n ) =  .  (7-13)
 
 .. 
vn
Using this concept you can divide a matrix Am×n into its column vectors. This is written
like this:
 
A = a1 a2 . . . a n
       
a11 a12 a1n a11 a12 . . . a1n
 a   a22   a2n   a
  21 a22 . . . a2n  (7-14)

 21  
=  ..   ..  · · ·  ..  =  ..
 
.. .. 
 .   .   .   . . . 
am1 am2 amn am1 am2 . . . amn
Accordingly there are n column vectors with m elements each.

Notice that the square brackets around the column vectors can be removed just
like that! This can be done in all dealings with matrices, where double square
brackets occur. It is always the innermost brackets that are removed. In this
way there is no difference between the two expressions. The last expression is
always preferred, because it is the easier to read.

We now define the product of a matrix and a vector, in which the matrix has as many
columns as the vector has elements:
eNote 7 7.2 MATRIX-VECTOR PRODUCTS AND MATRIX-MATRIX PRODUCTS 7

Definition 7.7 Matrix-Vector Product

Let A be an arbitrary matrix in Rm×n , and let v be an arbitrary vector in Rn .

The matrix-vector product of A with v is defined as:

 
v1
  v2 

  
Av = a1 a2 . . . an  .  = v1 a1 + v2 a2 + . . . + vn an (7-15)
 .. 
vn

The result is a column vector with m elements. It is the sum of the products of
the k’th column in the matrix and the k’th element in the column vector for all k =
1, 2, . . . , n.

It is necessary that there are as many columns in the matrix as there are rows in the
column vector, here n.

Notice the order in the matrix-vector product: first matrix, then vector! It is not
a vector-matrix product so to speak. The number of rows and columns will not
match in the other configuration unless the matrix is of the type 1 × 1.

Example 7.8 Matrix-Vector Product

The following matrix and vector (a column vector) are given:

 
  3
a b c
A = A2×3 = and v =  4 . (7-16)
d e f
−1
We now form the matrix-vector product of A with v by use of definition 7.7:
 
  3         
a b c  a b c 3a + 4b − c
Av = 4 = 3 +4 + (−1) = (7-17)
d e f d e f 3d + 4e − f
−1
If A is given like this
 
−1 2 6
A= , (7-18)
2 1 4
you get the product
   
3 · (−1) + 4 · 2 − 6 −1
Av = = (7-19)
3·2+4·1−4 6
eNote 7 7.2 MATRIX-VECTOR PRODUCTS AND MATRIX-MATRIX PRODUCTS 8

It is seen that the result (in both cases) is a column vector with as many rows as there are
rows in A.

Exercise 7.9 Matrix-Vector Product

Form the matrix-vector product A with x in the equation Ax = b, when it is given that
     
a11 a12 a13 x1 b1
A =  a21 a22 a23  , x =  x2  and b =  b2  (7-20)
a31 a32 a33 x3 b3

Is this something you have seen before? From where does it come?

As we have remarked above a matrix can be viewed as a number of column vectors

aligned after one another. This is used in the following definition of a matrix-matrix
product as a series of matrix-vector products.

Definition 7.10 Matrix-Matrix Product

Let A be an arbitrary matrix in Rm×n , and let B be an arbitrary matrix in Rn× p .

The matrix-matrix product or just the matrix product of A and B is defined like this:
   
AB = A b1 b2 . . . b p = Ab1 Ab2 . . . Ab p (7-21)

The result is matrix of type m × p. The k’th column in the resulting matrix is a
matrix-vector product of the first matrix (here A) and the k’th column vector in the
last matrix (here B), cf. definition 7.7.

There must be as many columns in the first matrix as there are rows in the last
matrix.

Example 7.11 Matrix-Matrix Product

Given two matrices A2×2 and B2×3 :

   
4 5 −8 3 3
A= og B= (7-22)
1 2 2 9 −9
eNote 7 7.2 MATRIX-VECTOR PRODUCTS AND MATRIX-MATRIX PRODUCTS 9

We wish to form the matrix-matrix product of A and B. This is done by use of definition 7.10.
        
4 5 −8 4 5 3 4 5 3
AB =
1 2 2 1 2 9 1 2 −9
    (7-23)
4 · (−8) + 5 · 2 4 · 3 + 5 · 9 4 · 3 + 5 · (−9) −22 57 −33
= =
−8 + 2 · 2 3+2·9 3 + 2 · (−9) −4 21 −15
NB: It is not possible to form the matrix-matrix product BA, because there are not as many
columns in B as there are rows in A (3 6= 2).

Example 7.12 Matrix-Matrix Product two Ways

Given the two matrices A2×2 and B2×2 :

   
3 2 4 4
A= and B = (7-24)
−5 1 −1 0

Because the two matrices are square matrices of the same type both matrix-matrix products
AB and BA can be calculated. We use the definition 7.10.
     
3 2 4 3 2 4
AB =
−5 1 −1 −5 1 0
   
3 · 4 + 2 · (−1) 3·4+2·0 10 12
= =
−5 · 4 + 1 · (−1) −5 · 4 + 1 · 0 −21 −20
      (7-25)
4 4 3 4 4 2
BA =
−1 0 −5 −1 0 1
   
4 · 3 + 4 · (−5) 4 · 2 + 4 −8 12
= =
−1 · 3 −1 · 2 −3 −2

We see that AB 6= BA. The factors are not interchangeable!

Here we summarize the arithmetic rules that apply to matrix-matrix products and ma-
trix sums. Because the matrix-vector product is a special case of the matrix-matrix prod-
uct, the rules also apply for matrix-vector products.
eNote 7 7.2 MATRIX-VECTOR PRODUCTS AND MATRIX-MATRIX PRODUCTS 10

Theorem 7.13 Arithmetic Rules for Matrix Sum and Product

For arbitrary matrices A, B and C and likewise an arbitrary real number k the follow-
ing arithmetic rules are valid, in so far as the matrix-matrix products can be formed:
(kA)B = A(kB) = k(AB) Product with a scalar is associative

A(B + C) = AB + AC
the distributive rules apply
(A + B)C = AC + BC
A(BC) = (AB)C Matrix-matrix products are associative

Analogous to the demonstration of the arithmetic rules in Theorem 7.3 we demonstrate

the last arithmetic rule in Theorem 7.13:

Example 7.14 Are Matrix Products Associative?

The last arithmetic rule in 7.13 is tested on the three matrices:

4 −5
 
   
1 2 −3 −2 −1
A= , B= and C =  2 1 (7-26)
3 4 0 0 7
1 −3

First we calculate AB and BC:

          
1 2 −3 1 2 −2 1 2 −1 −3 −2 13
AB = =
3 4 0 3 4 0 3 4 7 −9 −6 25
 −5 (7-27)
    
  4   
−3 −2 −1   −3 −2 −1  −17 16
BC =  2 1   =
0 0 7 0 0 7 7 −21
1 −3

Then we calculate A(BC) and (AB)C:

    
  
1 2 −17
1 2 16 −3 −26
A(BC) = =
3 4 3 4 −21
7 −23 −36
 −5 (7-28)
    
  4   
− 3 − 2 13 − 3 − 2 13 − 3 − 26
(AB)C =  2  1  =
−9 −6 25 −9 −6 25 −23 −36
1 −3

We see that A(BC) = (AB)C, and therefore it doesn’t matter which of the matrix products
AB and BC we calculate first. This is valid for all matrices (although not proved here).

As is done in example 7.14 we can demonstrate the other arithmetic rules. By writing
eNote 7 7.3 TRANSPOSE OF A MATRIX 11

down carefully the formula for each element of a matrix in the final product, in terms of
the elements of the other matrices, one can prove the rules properly.

Exercise 7.15 Demonstration of Arithmetic Rule

Demonstrate the first arithmetic rule in Theorem 7.13 with two real matrices A2×2 and B2×2
and the constant k.

7.3 Transpose of a Matrix

By interchanging rows and columns in a matrix the transpose matrix is formed as in

this example:
 
  a d
a b c
A= has the transpose A> =  b e  (7-29)
d e f
c f

A> is ’A transpose’. In addition you have that (A> )> = A. Here is a useful arithmetic
rule for the transpose of a matrix-matrix product.

Theorem 7.16 Transpose of a Matrix

Let there be given two arbitrary matrices Am×n and Bn× p . You form the transposed
matrices , A> and B> respectively, by interchanging the columns and rows of each
matrix.

The transpose of a matrix-matrix product AB is equal to the matrix-matrix product

of B> with A> (that is, in reverse order):

(AB)> = B> A> (7-30)

In the following example Theorem 7.16 is tested.

eNote 7 7.3 TRANSPOSE OF A MATRIX 12

Example 7.17 Demonstration of Theorem 7.16

 
  9 1
0 1 6
Given the two matrices A = and B =  1 0 . Then
7 −3 2
−6 3
    
  9   1
0 1 6  0 1 6  
AB =  1 0
7 −3 2 7 −3 2
−6 3 (7-31)
   
1·1−6·6 6·3 −35 18
= =
7·9−3·1−2·6 7·1+2·3 48 13

We now try to form the matrix-matrix product B> A> and we find
 
0 7  
> > 9 1 − 6
A =  1 −3  and B = (7-32)
1 0 3
6 2

and then     
  0   7
9 1 − 6 9 1 − 6
B> A> =  1  −3 
1 0 3 1 0 3
6 2 (7-33)
   
1·1−6·6 9·7−1·3−6·2 −35 48
= =
3·6 1·7+3·2 18 13
The two results look identical:
 >  
−35 18 −35 48
= ⇔ (AB)> = B> A> , (7-34)
48 13 18 13

in agreement with Theorem 7.16

Exercise 7.18 Matrix Product and the Transpose

Given the matrices    

1 1 2 0 −1 −1
A= and B = (7-35)
1 2 −1 1 2 1
Calculate if possible the following:

a) 2A − 3B, b) 2A> − 3B> , c) 2A − 3B> , d) AB,

e) AB> , f) BA> , g) B> A, h) A> B.
eNote 7 7.4 SUMMARY 13

7.4 Summary

• Matrices are arrays characterized by the number of columns and rows, determining
the type of the matrix. An entry in the matrix is called an element.

• The type of a matrix is denoted as: Am×n . The matrix A has m rows and n columns.

• Matrices can be multiplied by a scalar by multiplying each element in the matrix

by the scalar.

• Matrices can be added if they are of the same type. This is done by adding corre-
sponding elements.

• The matrix-vector product, of Am×n with the vector v with n elements, is defined
as:
 
v1
  v2 

  
Am×n v = a1 a2 . . . an  .  = a1 v1 + a2 v2 + . . . + an vn , (7-36)
 .. 
vn

where a1 , a2 , . . . , an are the column vectors in A.

• The matrix-matrix product (or just the matrix product) is defined as a series of
matrix-vector products:
   
Am×n Bn× p = A b1 b2 . . . b p = Ab1 Ab2 . . . Ab p (7-37)

• More arithmetic rules for matrix sums, matrix products and matrix-scalar prod-
ucts are found in Theorem 7.3 and Theorem 7.13.

• The transpose A> of a matrix A is determined by interchanging rows and columns

in the matrix.