Linear Algebra and Matrix Theory

Chapter 1 - Linear Systems, Matrices and Determinants

This is a very brief outline of some basic definitions and theorems of linear

algebra. We will assume that you know elementary facts such as how to add

two matrices, how to multiply a matrix by a number, how to multiply two

matrices, what an identity matrix is, and what a solution of a linear system

of equations is. Hardly any of the theorems will be proved. More complete

treatments may be found in the following references.

1. References

(1) S. Friedberg, A. Insel and L. Spence, Linear Algebra, Prentice-Hall.

(2) M. Golubitsky and M. Dellnitz, Linear Algebra and Differential Equa-

tions Using Matlab, Brooks-Cole.

(3) K. Hoffman and R. Kunze, Linear Algebra, Prentice-Hall.

(4) P. Lancaster and M. Tismenetsky, The Theory of Matrices, Aca-

demic Press.
1
2

2. Linear Systems of Equations and Gaussian Elimination

The solutions, if any, of a linear system of equations

(2.1) a11 x1 + a12 x2 + · · · + a1n xn = b1

a21 x1 + a22 x2 + · · · + a2n xn = b2

..
.

am1 x1 + am2 x2 + · · · + amn xn = bm

may be found by Gaussian elimination. The permitted steps are as follows.

(1) Both sides of any equation may be multiplied by the same nonzero

constant.

(2) Any two equations may be interchanged.

(3) Any multiple of one equation may be added to another equation.

Instead of working with the symbols for the variables (the xi ), it is eas-

ier to place the coefficients (the aij ) and the forcing terms (the bi ) in a

rectangular array called the augmented matrix of the system.

 
 a11 a12 . . . a1n | b1 
 
 
a
 21 a22 . . . a2n | b2 

(2.2) 
 .


 .. 
 
 
 
am1 am2 . . . amn | bm

The steps of Gaussian elimination are carried out by elementary row oper-

ations applied to the augmented matrix. These are:

 3

(1) Any row of the matrix may be multiplied throughout by any nonzero

number.

(2) Any two rows of the matrix may be interchanged.

(3) Any multiple of one row may be added to another row.

When Gaussian elimination steps are applied to a linear system (i.e. when

elementary row operations are applied to the augmented matrix), the result

is an equivalent system, that is, one that has exactly the same set of solu-

tions. The objective of Gaussian elimination is to transform a given linear

system through a sequence of elementary row operations to obtain an equiv-

alent linear system whose solutions are easy to find.

Solved Problems:

Problem 1:

x+y+z =1

x − 2y + 3z = 2

2x − y + 4z = 3

Solution: Let M (i, c) denote the operation of multiplying row i by the con-

stant c. Let A(c, i, k) denote the operation of adding c times row i to row k.

Let S(i, k) denote the operation of swapping rows i and k. Then, starting
4

with the augmented matrix of the system,

   
1 1 1 | 1 1 1 1 | 1
   
1 −2 3 | 2 A(−1,1,2)
   
 −→ 0 −3 2 | 1
 

   
   
2 −1 4 | 3 2 −1 4 | 3
   
1 1 1 | 1 1 1 1 | 1
   
A(−2,1,3)   A(−1,2,3)  
−→ 0 −3 2 | 1 −→ 0 −3 2 | 1
  

   
   
0 −3 2 | 1 0 0 0 | 0
   
1 1 1 | 1 1 0 53 | 2
3
   
M (2,−1/3)   A(−1,2,1)  
−→  2
0 1 − 3 | 3  −→ 
1
0 1 − 3
2
| 1
3
   
   
0 0 0 | 0 0 0 0 | 0

No further simplification is possible. Reintroducing the variables, the equiv-

alent system is

5 2
x+ z =
3 3
2 1
y− z=
3 3

Thus, the original system has infinitely many solutions obtained by assigning

an arbitary value to z and computing x and y from the two equations above.

Geometrically, the set of solutions is a straight line in 3-space.

Problem 2: Solve  
1 1 1 | 1 
 
 
1 −2 3 | 2 
 
 
 
2 −1 4 | −1
 5

Solution:
   
1 1 1 | 1  1 1 1 | 1 
   
A(−1,1,2)   A(−2,1,3)  
−→  0 −3 2 | 1  −→ 0 −3 2 | 1 
  
   
   
2 −1 4 | −1 0 −3 2 | −3

No further steps are necessary because it is obvious that the last two equa-

tions have no solutions. Nevertheless, we shall continue.

   
1 1 1 | 1  1 1 1 | 1
  1
 
A(−1,2,3)   M (3,− 4 )  
−→  0 −3 2 | 1  −→ 0 −3 2 | 1
  
   
   
0 0 0 | −4 0 0 0 | 1
   
5 4
1 1 1 | 1  1 0 3 | 3

   
M (2,− 31 )   A(−1,2,1)  
−→ 0 1 − 2 | − 3  −→ 
1 2
0 1 − 3
1
| −3
 3
   
   
0 0 0 | 1 0 0 0 | 1
   
1 0 35 | 4
3 1 0 53 | 0
   
A( 31 ,3,2)   A(− 43 ,3,1)  
−→  0 1 − 3
2
| 0 −→ 0 1 − 2 | 0
 3 
   
   
0 0 0 | 1 0 0 0 | 1

Again, it is obvious that the system has no solution because the last

equation has no solution. The matrix just above is in a special form called

reduced row echelon form or Hermite normal form. We will discuss it further

in a later section.

Unsolved Problems:

Find all solutions of the following systems. For systems with two or three

variables describe the solution set geometrically if you can.

6

1.

x1 + 2x2 − x3 + 2x4 = 1

2x1 − x2 + 2x3 − x4 = 2

3x1 + x2 + x3 + x4 = 3

−x1 + 3x2 − 3x3 + 3x4 = −1

2.
 
1 1 1| 2
 
 
0
 1 2 | 1 
 
 
2
 3 4 | 5 
 
 
1 0 −1 | 1

3.

x1 + 2x2 − x3 = −1

2x1 + 2x2 + x3 = 1

3x1 + 5x2 − 2x3 = −1

4.
 
 1 −4 −1 1 | 3 
 
 
 2 −8 1 −4 | 9 
 
 
 
−1 4 −2 5 | −6
 7

3. Matrix Notation

The linear system (2.1) is written in matrix form as

(3.1) Ax = b,

where  
 a11 a12 · · · a1n 
 
 . 
 ..
A= 

 
 
am1 am2 · · · amn

is the m × n coefficient matrix,

 
 x1 
 
 
x 
 2
x=
 . 

 .. 
 
 
 
xn

is the n × 1 column vector of unknowns, and

 
 b1 
 
 
b 
 2
b=
 . 

 .. 
 
 
 
bm

is the m × 1 column vector of forcing terms. All of these must have entries

or components from the same number field or scalar field F. Here the scalar

field will almost always be either the real numbers F = R or the complex

numbers F = C.
8

The set of all m × n matrices with entries from the field F is denoted by

Fm×n . The set Fm×n is a vector space over the field F.

We denote the augmented matrix of the linear system (3.1) by (A|b).

This is an example of a partitioned matrix, whose parts or submatrices are

separated by the vertical bar. It should be noted that if B ∈ Fk×m , then

B(A|b) = (BA|Bb).

A similar result holds for matrices partitioned into blocks of columns as

(A1 |A2 | · · · |Ak ). This follows from the rules for matrix multiplication.

4. Row Equivalence and Rank

Definition 4.1. Two matrices A and B ∈ Fm×n are row equivalent if A

can be obtained from B by a sequence of elementary row operations.

This definition is symmetric with respect to A and B because each ele-

mentary row operation has an inverse operation of the same type, that is,

a row operation of the same type which undoes it. The inverse of A(c, i, k)

is A(−c, i, k), the inverse of M (i, c) is M (i, c−1 ), and the inverse of S(i, k)

is S(i, k).

Elementary Matrices:

Let Im denote the m × m identity matrix. An elementary m × m matrix

is one obtained from Im by an elementary row operation. For example,

 9

applying the operation A(−2, 3, 1) to I3 yields the elementary matrix

 
1 0 −2
 
 
0 1 0 
 
 
 
0 0 1

Multiplying a given matrix on the left by an elementary matrix has the

same effect as applying the corresponding elementary operation to the ma-

trix. Therefore, two matrices are row equivalent if one can be obtained from

the other by a sequence of left multiplications by elementary matrices.

Reduced Row Echelon Form:

When we refer to a zero row of a matrix we mean a row whose entries are

all 0. A matrix B is in reduced row echelon form if

(1) All the nonzero rows of B precede all the zero rows of B.

(2) The first nonzero entry in each nonzero row is 1.

(3) The leading 1 in each nonzero row is to the right of the leading 1’s

in all the preceding rows.

(4) The leading 1 in each nonzero row is the only nonzero entry in its

column.

Example 4.1.  
1 −2 0 1 0
 
 
0 0 1 −1 0
 
 
 
0 0 0 0 1
 
 
 
0 0 0 0 0
10

Theorem 4.1. For each matrix A ∈ Fm×n there is a unique matrix B ∈

Fm×n such that B in in reduced row echelon form and is row equivalent to

A.

We shall refer to B in the statement of the theorem as the reduced row

echelon form of A. The reduced row echelon form of a matrix may be found

by Gaussian elimination.

Definition 4.2. The rank of a matrix is the number of nonzero rows in its

reduced row echelon form.

The idea of the rank of a matrix is connected to the existence of solutions

of a linear system through the following theorem.

Theorem 4.2. A linear system Ax = b has a solution if and only if the

rank of the coefficient matrix A is the same as the rank of the augmented

matrix (A|b).

Unsolved Problems: Find the reduced row echelon form and rank of each

of the following matrices.

1.
 
 0 1 2 1
 
 
−2 0 2 0
 
 
 
3 2 1 2
 11

2.
 
2 −2 1
 
 
1 0 0
 
 
 
2 −1 1

5. Invertible Matrices

A square matrix A ∈ Fn×n is invertible if there is a matrix A−1 ∈ Fn×n

such that AA−1 = A−1 A = In . Elementary matrices are invertible since

each elementary operation can be undone by an elementary operation of the

same type. It is clear that products of invertible matrices are invertible.

Indeed, (AB)−1 = B −1 A−1 . From this, it follows that a square matrix is

invertible if and only if its reduced row echelon form is invertible. However,

the only reduced row echelon matrices that are invertible are the identity

matrices. Thus, we have the following theorem.

Theorem 5.1. Let A ∈ Fn×n . The following are equivalent.

(1) A is invertible.

(2) The rank of A is n.

(3) The reduced row echelon form of A is In .

(4) A is a product of elementary matrices.

The inverse of a matrix A ∈ Fn×n may be found by Gaussian elimination.

Perform elementary row operations on the n × 2n partitioned matrix (A|In )

until A is in reduced row echelon form. If the rank of A is less than n that
12

will be apparent. If the rank of A is n, the result of the calculations will be

(In |A−1 ).

Other criteria for invertibility are given in the following theorem.

Theorem 5.2. A ∈ Fn×n is invertible if and only if for each b ∈ Fn×1 the

linear system Ax = b has a unique solution x. A ∈ Fn×n is invertible if and

only if the homogeneous linear system Ax = 0 has only x = 0 as a solution.

Solved Problems:

1. Use Gaussian elimination to find the inverse.

 
2 −2 1
 
 
1 0 0
 
 
 
2 −1 1

Solution: Adjoin a 3 × 3 identity matrix to the right of the given matrix and

then use gaussian elimination steps. Two or more steps may be combined.

   
2 −2 1 | 1 0 0 0 −2 1 | 1 −2 0
   
   
1 0 0 | 0 1 0 −→ 1 0 0 | 0 1 0
   
   
   
2 −1 1 | 0 0 1 0 −1 1 | 0 −2 1

   
1 0 0 | 0 1 0 1 0 0 | 0 1 0
   
   
−→ 0 1 −1 | 0 2 −1 −→ 0 1 −1 | 0
   2 −1

   
   
0 −2 1 | 1 −2 0 0 0 1 | −1 −2 2
 13
 
1 0 0 | 0 1 0
 
 
−→ 0 1 0 | −1 0 1


 
 
0 0 1 | −1 −2 2

Thus the inverse of the given matrix is

 
0 1 0
 
 
−1 0 1
 
 
 
−1 −2 2

This should be verified by multiplying the two matrices together.

2. Show that if A, B ∈ Fn×n and AB = In , then BA = In .

Solution: Let E1 , · · · , Ek be a sequence of elementary matrices such that

E1 · · · Ek A = H is in reduced row echelon form. Since AB = I,

HB = E1 · · · Ek

If the last row of H were zero, the last row of the right hand side above would

be zero also. Below you are asked to show that no product of elementary

matrices has a zero row. Therefore, H = I, B = E1 · · · Ek , and BA = I.

Unsolved Problems: Determine whether the matrices below are invertible

or not. If they are, find their inverses.

1.
 
1 0 3
 
 
0 −2 −2
 
 
 
1 −3 1
14

2.  
0 0 1 
 
 
0 1 0 
 
 
 
1 0 0

3. Show that if A ∈ Fm×n has a zero row its rank is less than m. Then

show that a product of elementary matrices does not have a zero row. This

completes the solution of the second solved problem above.

4. Show that if E is an elementary matrix, then its transpose E t is an

elementary matrix of the same type. If F = C, then the conjugate transpose

E ∗ is elementary. matrix.

6. Determinants

A permutation of the set Nn = {1, 2, · · · , n} is a one-to-one function from

this set onto itself. A permutation σ has a sign, either +1 or -1 depending

on whether the number of pairs (i, j such that i < j and σ(i) > σ(j) is even

or odd. The sign of a permutation σ is denoted by sgn(σ).

Definition 6.1. Let B = (bi,j ) ∈ Fn×n . The determinant of B is

X n
Y
det(B) = sgn(σ) bi,σ(i) .
σ i=1

The sum is taken over all permuations σ of Nn .

Determinants are seldom evaluated from the definition. However, several

useful results are easily proved. In the following theorem, we use the same

notation for an elementary matrix as for the elementary row operation it

 15

corresponds to. Also, a matrix B = (bi,j ) is upper triangular if bi,j = 0 for

i > j, lower triangular if bi,j = 0 for i < j, and triangular if it is either upper

triangular or lower triangular.

Theorem 6.1. Properties of determinants:

Q
(1) If B is triangular, det(B) = i bi,i .

(2) det(B t ) = det(B).

(3) det(A(c, i, k)) = 1 for i 6= k.

(4) det(M (i, c)) = c.

(5) det(S(i, k)) = −1 for i 6= k.

(6) The determinant is multiplicative, i.e., if B, C ∈ Fn×n , det(BC) =

det(B)det(C).

For a square matrix B we may write B = E1 · · · Ek H, where each Ei is

elementary and H is the reduced row echelon form of B. det(B) 6= 0 if and

only if det(H) 6= 0. Since H is upper triangular, det(H) 6= 0 if and only if

for each i, hi,i 6= 0. This occurs if and only if H = I, i.e., if and only if B is

invertible.

Theorem 6.2. Let B ∈ Fn×n . B is invertible if and only if det(B) 6= 0.

Geometric Interpretation of Determinants:

Let B ∈ R2×2 and partition B by columns as B = (b,1 |b,2 ). By comparing

det(B) to the cross product of the plane vectors b,1 and b,2 , it can be seen

that the absolute value of det(B) is the area enclosed by the parallogram
16

with adjacent sides b,1 and b,2 . Similarly, if B is a real 3 × 3 matrix, the

absolute value of det(B) is the volume of the parallepiped in 3-space whose

adjacent sides are the column vectors of B. Generalization to spaces of

higher dimension is straightforward and important in multivariable calculus.

Solved Problems:

1. Find the determinant of

 
1 1 1
 
 
1 2 1
 
 
 
3 1 1

Solution: Row reduce the given matrix to upper triangular form.

     
1 1 1  1 1 1 1 1 1 
     
A(−1,1,2)   A(−3,1,3)   A(2,2,3)  
−→ 0 1 0 −→ 0 1
   0  −→ 0 1 0 
 

     
     
3 1 1 0 −2 −2 0 0 −2

The last matrix is triangular, so its determinant is the product of its di-

agonal entries, −2. This is equal to the product of the determinant of the

given matrix and the determinants of the elementary matrices used in the

reduction. Each of the elementary matrices was of type ”A”, which have

determinant 1. Therefore, −2 is the answer.

2.
 
 i 2+i 0 
 
 
−1 3 2i 
 
 
 
0 −1 1 − i
 17

Solution: Instead of writing each row operation in gaussian elimination

steps, we will simply keep track of their determinants. Operations of type

”A” have determinant 1 and thus do not have to be indicated.

   
 1 1 − 2i 0  1 1 − 2i 0 
   
−i    
−→ −1
 3 2i  −→ 0 4 − 2i
  2i 

   
   
0 −1 1−i 0 −1 1−i
   
1 1 − 2i 0  1 1 − 2i 0 
   
  −1  
−→ 0
 0 2 − 4i −→ 0
  −1 1−i 
   
   
0 −1 1−i 0 0 2 − 4i

Thus, the determinant of the given matrix is (−2 + 4i)/((−1)(−i)) = 4 + 2i.

Unsolved Problems:

1. Find the determinant of

 
0 1 1
 
 
1 2 −5
 
 
 
6 −4 3

2. Find the determinant of

 
 −1 2 + i 3 
 
 
1 − i i 1 
 
 
 
3i 2 −1 + i

3. Let B ∈ Fn×n and let c ∈ F. Show that det(cB) = cn det(B).