1 INTRODUCTION

Matrices provide a succinct way of presenting data. Therefore, we find value in

knowing the characteristics of a matrix—its representation and its properties. Further,
it is important to know how to operate on these matrices. Matrix Algebra shall revolve
around these topics and more. This course shall be useful in simplifying description
and promoting development of many analysis methods which rely on matrices.

Section 1.1 defines the matrix including a brief description of several special
matrices. Meanwhile, vectors and scalars are presented in Section 1.2. A review of the
summation notation as well as a discussion of the dot notation is given in Section 1.3.
Finally, Section 1.4 presents some important uses and applications of Matrix Theory in
Statistics.

1.1 Definition of a Matrix

Matrices are valuable aids in organizing data because it provides a convenient

starting point for its description. We illustrate this through the following example.

Illustration

Suppose that data on the food expenditure (in Php) of 3 UP students for 4 school
days is given as follows:

Tuesday Wednesday Thursday Friday

Student 1 313 402 426 450
Student 2 97 171 178 131
Student 3 203 156 190 301

Now, consider the array of numbers in the table extracted and written as:

313 402 426 450

A = [ 97 171 178 131]
203 156 190 301

where the position of the entry in this array determines its meaning; for example,
the entry in the 3rd row and 3rd column, 190, represents the amount spent by
Student 3 on food for Day 3 (Thursday), or the entry in the 2nd row and 1st column,
97, represents the amount spent by Student 2 for Day 1 (Tuesday). In the array of
numbers, A, a row represents a particular student and a column represents a
particular day of the week. This array of numbers is called a matrix.

1|Stat 135: Matrix Theory for Statistics

 A matrix is a rectangular array of numbers set out in rows and columns treated
as a single entity. The individual entries in the array are called elements or
terms of the matrix which can be numbers of any sort (real or complex, rational
or irrational).

Remarks:

• A matrix is named using an underlined capital letter, say A, and its elements are
denoted by lowercase letters with subscripts indicating the position in the
matrix.
• A matrix A is denoted by

𝑎11 𝑎12 … 𝑎1𝑗 … 𝑎1𝑐

𝑎21 𝑎22 … 𝑎2𝑗 … 𝑎2𝑐
⋮ ⋮ … ⋮ … ⋮
A= 𝑎 𝑎𝑖2 … 𝑎𝑖𝑗 … 𝑎𝑖𝑐
𝑖1
⋮ ⋮ … ⋮ … ⋮
[ 𝑎𝑟1 𝑎𝑟2 … 𝑎𝑟𝑗 … 𝑎𝑟𝑐 ]

• Equivalently, A can be denoted A = {𝑎𝑖𝑗 }, where 𝑎𝑖𝑗 is the element in the 𝑖 𝑡ℎ row
and 𝑗 𝑡ℎ column, 𝑖=1,2,…,r and 𝑗=1,2,…,c
• 𝑎11 is called the leading element of the matrix A.
• If A is an 𝑟𝑥𝑐 matrix, then 𝑟 indicates the number of rows (horizontal) and 𝑐
indicates the number of columns (vertical).
• The size of the matrix, 𝑟𝑥𝑐, is called its order or dimension.
• The rows of a matrix are of equal length, as are the columns.

Exercises
1. Determine which of the following are matrices and find the dimension.
1 3
a. A = [5 4]
6 2
45 34 66
b. B = [67 56 84]
98
1
−√2 11
c. C = [ ]
𝜋 3

2. Write down the following matrices.

a. D = {𝑑𝑖𝑗 } where 𝑑𝑖𝑗 =𝑖 for 𝑖 = 𝑗 and 𝑑𝑖𝑗 =0 for 𝑖 ≠ 𝑗, 𝑖, 𝑗 = 1,2,3
b. E = {𝑒𝑖𝑗 } where 𝑒𝑖𝑗 = 𝑖 + 𝑗, 𝑖 and 𝑗 = 1,2
c. F = {𝑓ℎ𝑘 } where 𝑓ℎ𝑘 = 2ℎ + 𝑘, ℎ = 1,2,3,4 and 𝑘=1
𝑗
d. G = {𝑔𝑗𝑘 } where 𝑔𝑗𝑘 =𝑘, 𝑗 = 1, 𝑘 = 1,2,3

2|Stat 135: Matrix Theory for Statistics

Special Matrices
Square matrix
A is a square matrix if r=c, i.e., the number of rows equals the number of
columns. We write it as Ar.

The elements 𝑎11 , 𝑎22 , …, 𝑎𝑟𝑟 of a square matrix are referred to as the diagonal
elements or the diagonal of the matrix.

The elements of a square matrix that lie in a line parallel to and just below the
diagonal are sometimes referred to as the subdiagonal elements.

Elements of a square matrix other than the diagonal elements are called off-
diagonal or nondiagonal elements.

Triangular Matrix

A triangular matrix is a square matrix with all elements above (or below) the
diagonal being zero.

An upper triangular matrix is a triangular matrix whose elements below the

diagonal are all zero.

A lower triangular matrix is a triangular matrix whose elements above the

diagonal are all zero.

Example:

upper triangular matrix lower triangular matrix

8 2𝜋 2 −0.3 0 0
[0 −3 5] [ 7 0.5 0]
0 0 0 4 −1 𝑒

Diagonal Matrix
A diagonal matrix is a square matrix having zero for all its nondiagonal
elements, i.e., a matrix D = {𝑑𝑖𝑗 } is a diagonal matrix if 𝑑𝑖𝑗 = 0 ∀ 𝑖 ≠ 𝑗, 𝑖, 𝑗 = 1,2,
…, r It is denoted by D{ 𝑎11 , 𝑎22 , 𝑎33 , … , 𝑎𝑟𝑟 } where 𝑎𝑖𝑖 are the diagonal elements,
𝑖 = 1,2,…,r.

3|Stat 135: Matrix Theory for Statistics

 Example:
1 0 0
D = [0 3 0] = D {1, 3, 5} = diag{1, 3, 5}
0 0 5

Scalar Matrix
A scalar matrix is a diagonal matrix with all diagonal elements equal.

Example:
3 0 0
S = [0 3 0 ]
0 0 3

Identity Matrix
An identity matrix of order 𝑟, denoted by Ir, is a diagonal matrix having all
diagonal elements equal to one.

Example:
1 0 0
1 0
I2 = [ ] I 3 = [0 1 0 ]
0 1
0 0 1

Null Matrix
A null matrix, denoted by O, is a matrix of zeros, i.e., every element is zero. It is
also called the zero matrix.

Example:
0 0
0 0
O2 = [ ] O3x2 = [0 0]
0 0
0 0

1.2 Vectors and Scalars

A matrix consisting of only a single column is called a column vector.

A matrix consisting of only a single row is called a row vector.

Remarks:
• A column vector is denoted by an underlined small letter.
• A row vector is denoted by an underlined small letter with a prime (‘).

4|Stat 135: Matrix Theory for Statistics

 Example:
2
𝑥 = [7 ] 𝑦’ = [1 5 −3 0 2 6]
4
• A matrix can be expressed as an array of vectors. That is,
𝑎 𝑎 … 𝑎 𝑎1′
11 12 1𝑐
𝑎21 𝑎22 … 𝑎2𝑐 𝑎2 ′
A=[ ⋮ ⋮ ⋮ ] = = [𝑎1 𝑎2 ⋯ 𝑎𝑐 ]
⋱ ⋮
𝑎𝑟1 𝑎𝑟2 … 𝑎𝑟𝑐
[ 𝑎 𝑟 ′]

A single number is called a scalar. A scalar can be thought of as a matrix of

order 1x1, i.e., a matrix with one column and one row. A scalar is denoted by a
small letter.

1.3 Summation and Dot Notation

Summation

𝑛
∑ 𝑋𝑖 = 𝑋1 + 𝑋2 + 𝑋3 + ⋯ + 𝑋𝑛
𝑖=1

Properties of Summation:
• The summation of the sum of two or more terms equals the sum of the
individual summations.
𝑛 𝑛 𝑛 𝑛

∑(𝑋𝑖 + 𝑌𝑖 + 𝑍𝑖 ) = ∑ 𝑋𝑖 + ∑ 𝑌𝑖 + ∑ 𝑍𝑖
𝑖=1 𝑖=1 𝑖=1 𝑖=1

• The summation of the product of a constant, c, with 𝑋𝑖 equals the product the
constant with the summation of 𝑋𝑖 .
𝑛 𝑛

∑ 𝑐𝑋𝑖 = 𝑐 ∑ 𝑋𝑖
𝑖=1 𝑖=1

• The summation of a constant, c, from 𝑖=1 to n, equals the product of n and c.

𝑛

∑ 𝑐 = 𝑛𝑐
𝑖=1

5|Stat 135: Matrix Theory for Statistics

 Dot Notation

𝑟
• 𝑎.𝑗 = ∑ 𝑎𝑖𝑗
𝑖=1

𝑐
• 𝑎𝑖. = ∑ 𝑎𝑖𝑗
𝑗=1

𝑟 𝑐 𝑟 𝑐
• 𝑎.. = ∑ 𝑎𝑖.𝑗 = ∑ 𝑎.𝑗 = ∑ 𝑖𝑗 ∑ 𝑎𝑖𝑗
𝑖=1 𝑗=1 𝑖=1 𝑗=1

Product Notation
𝑛

∏ 𝑏𝑖 = (𝑏1 )(𝑏2 ) … (𝑏𝑛 )

𝑖=1

Remarks:

• 𝑎.𝑗 gives the 𝑗 𝑡ℎ column total

• 𝑎𝑖. gives the 𝑖 𝑡ℎ row total
• 𝑎.. gives the grand total of all elements

Exercises

A. Expand the following.

2
1. ∑𝑛𝑘=1 𝑋𝑘 3. (∑𝑛𝑘=1 𝑋𝑘 )2
2. ∑5𝑖=2 𝑋𝑖 𝑌𝑖 4. ∑5𝑖=2 𝑋𝑖 ∑5𝑖=2 𝑌𝑖
3. ∑4𝑖=1,𝑖≠3 ∑2𝑗=1 𝑋𝑖𝑗

B. Given matrix W and H,

5 3 8 0 2 4
W=[ 2 4 7 ] and H = [ 6 −3 5]
−2 0 −1 9 6 4

show that

1. ∑3𝑖=1 ∑3𝑗=1(𝑤𝑖𝑗 + ℎ𝑖𝑗 ) = 59 4. 𝑤1 . = 16, 𝑤2 .=13, and 𝑤3 . =-3

2. ∑3𝑗=1 ℎ2𝑗 =8 5. 𝑤.1 = 5, 𝑤.2=7, and 𝑤.3 =14
3. ∏3𝑖=1 ℎ𝑖2 = -36 6. ∑3𝑖=1,𝑖≠2 ℎ𝑖𝑗 = 9, 8, and 8 for 𝑗=1,2,3

6|Stat 135: Matrix Theory for Statistics

1.4 Applications of Matrix Algebra in Statistics

Some applications of matrix algebra are: 1) in regression analysis for calculating

estimates for 𝛽; 2) in population dynamics where in studying a biological population,
we want to investigate the distribution of individuals according to their age; 3) in
performing multivariate analysis on data; and 4) estimating the parameters for an
experimental designs study.

For example, suppose an ad agency is interested in investigating the

relationship between advertising cost and the sales of a certain product. A
mathematical equation that allows us to predict values of the dependent variable,
𝑠𝑎𝑙𝑒𝑠, from known values of the independent variable, 𝑎𝑑𝑠, is called a regression
equation. Through the regression equation, 𝑠𝑎𝑙𝑒𝑠̂ = 𝑏𝑜 + 𝑏1 𝑎𝑑𝑠, we can infer about
the linear association that exists between the two variables. However, an integral step
is estimating 𝑏𝑜 and 𝑏1 which relies on matrix algebra.

Another application is representing the random vector. Suppose there is a

random variable 𝑋 with mean 𝜇 and variance 𝜎 2 . This can be extended in the bivariate
case, two random variables 𝑋 and 𝑌 with means 𝜇𝑥 and 𝜇𝑦 , variances 𝜎𝑥2 and 𝜎𝑦2 and
the covariance of 𝑋 which we can summarize as follows:

𝑋 𝑋 𝜇𝑥 𝑋 𝜎𝑥2 𝜎𝑥𝑦
[ ] 𝐸 ([ ])=[𝜇 ] 𝑉𝑎𝑟 ([ ])=[ ]
𝑌 𝑌 𝑦 𝑌 𝜎𝑥𝑦 𝜎𝑦2

This can be further extended to the k-variate case wherein there are 𝑘 random
variables, say 𝑋1 , 𝑋2 , … , 𝑋𝑘 have means 𝜇1 , 𝜇2 , … , 𝜇𝑘 , variances 𝜎12 , 𝜎22 , … , 𝜎𝑘2 and
covariances 𝜎12 , 𝜎13 , … 𝜎𝑘−1,𝑘 which can be represented by the following vectors.

𝑋1 𝜇1 𝜎12 𝜎12 ⋯ 𝜎1𝑘

𝑋2 𝜇2 𝜎22 ⋯ 𝜎2𝑘
𝑋= [ ] 𝐸(𝑋) = 𝜇 = [ ⋮ ] 𝑉𝑎𝑟(𝑋) = Σ = 𝜎12
⋮ ⋮ ⋮ ⋱ ⋮
𝑋𝑘 𝜇𝑘 [𝜎1𝑘 𝜎2𝑘 ⋯ 𝜎𝑘2 ]

7|Stat 135: Matrix Theory for Statistics