Matrices and Linear Algebra

The Algebra

of Matrices

1. MATRICES: DEFINITIONS

This book is entitled Matrices and Linear Algebra, and linear will be the most common mathematical term used here. This word has many related meanings, and now we shall explain what a linear equation is. An example of a linear equation is 3x1 + 2x2 = 5, where x1 and x2 are unknowns. In general an equation is called linear if it is of the form

where x1, …, xn are unknowns, and a1, …, an and b are numbers. Observe that in a linear equation, products such as x1x2 or x3⁴ and more general functions such as sin x1 do not occur.

In elementary books a pair of equations such as

is called a pair of simultaneous equations. We shall call such a pair a system of linear equations. Of course we may have more than three unknowns and more than two equations. Thus the most general system of m equations in n unknowns is

The aij are numbers, and the subscript (i, j) denotes that aij is the coefficient of xj in the ith equation.

So far we have not explained what the coefficients of the unknowns are, but we have taken for granted that they are real numbers such as 2, , or π. The coefficients could just as well be complex numbers. This case would arise if we considered the equations

Note that a real number is also a complex number (with imaginary part zero), but sometimes it is important to consider either all real numbers or all complex numbers. We shall denote the real numbers by and the complex numbers by . The reader who is familiar with abstract algebra will note that and are fields. In fact, most of our results could be stated for arbitrary fields. (A reader unfamiliar with abstract algebra should ignore the previous two sentences.) Although we are not concerned with such generality, to avoid stating most theorems twice we shall use the symbol to stand for either the real numbers or the complex numbers . Of course we must be consistent in any particular theorem. Thus in any one theorem if stands for the real numbers in any place, it must stand for the real numbers in all places. Where convenient we shall call a number system.

In Chapter 2 we shall study systems of linear equations in greater detail. In this chapter we shall use linear equations only to motivate the concept of a matrix. Matrices will turn out to be extremely useful not only in the study of linear equations but also in much else.

If we consider the system of equations (1.1.2), we see that the arrays of coefficients

convey all the necessary information. Conversely, given any arrays like

we can immediately write down a corresponding system of equations

Let stand for the real or complex numbers. With this as motivation we adopt the following

( 1.1.4)DEFINITIONA matrix (over ) is a rectangular array of elements of . The elements of that occur in the matrix A are called the entries of A.

Examples of matrices are

The general form of a matrix over is

where each aij is an element of , that is, either a real or complex number.

The horizontal array

is called the ith row of A and we shall denote it by ai*. Similarly, the vertical array

is called the jth column of A, and we shall often denote it by a*j. Observe that aij is the entry of A occurring in the ith row and the jth column.

If the matrix A has m rows and n columns it is called an m × n matrix. In particular, if m = n the matrix is called a square matrix. At times an n × n square matrix is referred to as a matrix of order n. Two other special cases are the m × 1 matrix, referred to as a column vector, and the 1 × n matrix, which is called a row vector. Examples of each special case are

Usually we denote matrices by capital letters (A, B, and C), but sometimes the symbols [aij], [bkl], and [cpq] are used. The entries of the matrix A will be denoted by aij, those of B by bkl, and so forth.

It is important to realize that

are all distinct matrices. To emphasize this point, we make the following

( 1.1.6)DEFINITIONLet A be an m × n matrix and B a p × q matrix. Then A = B if and only if m = p, n = q, and

Thus for each pair of integers m, n we may consider two sets of matrices. One is the set of all m × n matrices with entries from the set of real numbers , and the other is the set of all m × n matrices with entries from the set of complex numbers . We shall denote the first set by m, n and the second by m, n. To each theorem about m, n there often corresponds an analogous theorem that can be obtained by consistently replacing m, n by m, n. In conformity with our use of the symbol , we shall write m, n to stand for either m, n or m, n.

We shall now point out how the use of matrices simplifies the notation for systems of linear equations. Let

All the information in (1.1.2) is contained in A, x, and b. (It is convenient here to call x a matrix or column vector even though its entries are unknowns.) Thus we could, purely symbolically at the moment, write

Ax = b.

Thus we reduce two linear equations in three unknowns to one matrix equation. If we have m linear equations in n unknowns, as in (1.1.3), we can still use the matrix form

where

For the time being, (1.1.7) is merely a symbolic way of expressing the equations (1.1.3). As another example, let

Then Ax = b is shorthand for

In Section 3 we shall see that the left side of (1.1.7) may be read as the product of two matrices. Note that b and x are column vectors; b is a column vector with m elements and x is a column vector with n elements. This method of writing the linear equations concentrates attention upon the essential item, the coefficient array.

EXERCISES

1. Find the matrices (A, b, x), corresponding to the following

systems of equations.

2. What systems of equations correspond to the following pairs of matrices?

2. ADDITION AND SCALAR

MULTIPLICATION OF MATRICES

We wish to see the effect on their corresponding matrices of adding two systems of equations. Consider the following two systems of equations with their corresponding matrices:

Adding the corresponding equations, we obtain a third system of equations,

and its matrices

Here we see how C may be obtained directly from A and B without reference to the original system of equations as such. We simply add the entries in like position. Thus

c22 = + 4 = a22 + b22

and we shall write C = A + B.

We shall define this sum A + B only when A and B have the same number of rows and columns. Two matrices with this property will be called additively conformable. For such matrices we make the following

( 1.2.1)DEFINITIONIf A and B are two m × n matrices, then the sum C = A + B is given by

With m and n fixed we now look at m, n, the set of all m × n matrices whose entries are elements of , together with the operation + defined above for these additively conformable matrices. We have the following

( 1.2.2)THEOREM(1) Closure: m, n is closed under addition in the sense that the sum of any two m × n matrices is defined and is again an m × n matrix.

(2) Associativity: (A + B) + C = A + (B + C).

(3) Existence of a zero matrix 0: There exists an m × n matrix each of whose entries is zero,

with the property that

A + 0 = A

for any matrix A in the set m, n.

(4) Existence of negatives: Given any matrix A in m, n, there is a unique X in m, n such that

A + X = 0

(As with numbers, we denote X by − A. −A is called the additive inverse of A, and for A = [aij] we have −A = [−aij].)

(5) A + B = B + A.

Note that we are using 0 in two senses: (1) the real number zero, and (2) the matrix all of whose entries are the real number zero.

As an example of (4), let m = 3, n = 2, and

PROOF

We will give a proof of (2); the other assertions are equally easy.

Let aij, bij, and cij be the general entries in A, B, and C, respectively. Then the general term of (A + B) + C is given by (aij + bij) + cij, while the general term of A + (B + C) is given by aij + (bij + cij). But the real and the complex numbers satisfy the associative law for addition, so that

Consequently, (A + B) + C = A + (B + C).

Remark Those readers who know some abstract algebra will recognize that Theorem (1.2.2) simply asserts that m, n is an abelian group under addition.

We now examine the result of multiplying the system of equations

and its matrix form

by some number α (which is called a scalar in this connection). Let α = 2. Upon multiplying we obtain

and the matrix form

Inspection of (1.2.4′),

leads immediately to the

( 1.2.5)DEFINITION OF SCALAR MULTIPLICATIONLet A = [aij] be an m × n matrix with entries in the number system , and let α be an element of . Then B = αA = Aα is defined by

Directly from the definition we have the following

(1.2.6)THEOREM

(1) α(A + B) = αA + αB.

(2) (α + β)A = αA + βA.

(3) (αβ)A = α(βA).

(4) 1A = A.

To illustrate (1) let α = 2,

Then

So in this particular case we have α(A + B) = αA + αB. The proofs are straightforward and are left to the reader.

Although αA = Aα for α a scalar and A a matrix, if A is a row vector, we shall always write the product as αA. However, if B is a column vector, the product of B and α will be written as Bα.

EXERCISES

1. None of the following matrices are equal. Check! Which of them are additively conformable? Calculate the sum of all pairs of additively conformable matrices of the same order.

2. Add the following pairs of systems of equations by finding their matrices A, b, and x. After performing the matrix addition translate the result back into equation form. Check your result by adding the equations directly.

3. Check parts (3) and (4) of Theorem (1.2.2).

4. Prove part (5) of Theorem (1.2.2). [Hint: In this and exercise 3 use the definition of matrix addition as illustrated in the proof of part (2).]

5. Let C be a n × n matrix. The trace of C, tr(C), is defined to be the sum of the diagonal entries, that is, . Deduce the following results:

(a) tr(A + B) = tr(A) + tr(B).

(b) tr(kA) = k tr(A).

6.

Find the scalar product of the given systems of equations by 3. First perform the multiplications. Find the associated matrices and calculate the scalar product using the definition. Translate the result back into equations. Compare the respective answers.

7. Check the assertions following the definition of scalar multiplication.

8. For the system of equations

find the product of the corresponding matrix equation with the scalar (1 − i).

3. MATRIX MULTIPLICATION

Suppose we now consider the system of equations

and its matrix form

where

We wish to define a multiplication between matrices so that (1.3.2) is not merely symbolic but will give us back (1.3.1) when the multiplication is carried out.

Suppose we also have

and the corresponding matrices

so that

We can substitute (1.3.1) into (1.3.3) to obtain a new set of equations in terms of z and x. We want to be able to make the corresponding substitution of (1.3.2) into (1.3.4) and have the following hold for the matrices A and B:

If we compute the equations resulting from the substitution of (1.3.1) into (1.3.3) we obtain

so that if the product BA is to be defined we should have

Observe that (1.3.3) was a system of three equations in the two unknowns y1 and y2, and that (1.3.1) had one equation for each of y1 and y2. Thus in our substitution the number of unknowns in (1.3.3) equals the number of equations in (1.3.1).

In terms of matrices this means that the number of columns of B equals the number of rows of A. Further, after the substitution has been carried out in (1.3.6), it is clear that the number of equations equals the number of equations in (1.3.3), while the number of unknowns is the same as in (1.3.1). Thus our new matrix BA will have the same number of rows as B and the same number of columns as A.

With this in mind we shall call an m × n matrix B and an n′ × p matrix A multiplicatively conformable if and only if n = n′, that is, if the number of columns of B equals the number of rows of A. We shall define multiplication only for multiplicatively conformable matrices. Further, BA will be an m × p matrix; that is, BA will have as many rows as B and as many columns as A.

Keeping (1.3.7) and subsequent remarks in mind we make the following

( 1.3.8)DEFINITIONLet B be an m × n matrix and A be an n × p matrix, so that B has as many columns as A has rows. Let

Then the product

is defined by

that is,

We emphasize that the product of an m × n matrix with an n × p matrix yields an m × p matrix. Observe further that in (1.3.7) the product BA is defined, while the product AB is not, unless m = p.

To illustrate further the definition of matrix multiplication, let us look at our previous example with letters in place of the numerical coefficients. The system (1.3.1) becomes

while (1.3.3) and (1.3.6) are rewritten, respectively, as

The corresponding coefficient matrices are

Note that if A is the matrix of (1.1.5) and b and x are the column vectors of (1.1.8), then Ax is precisely the left side of equations (1.1.3). This justifies our use of Ax = b in Section 1.

We can immediately verify the following special cases:

(1) row vector × matrix = row vector.

(2) matrix × column vector = column vector.

(3) row vector × column vector = 1 × 1 matrix (a scalar).

(4) column vector × row vector = matrix.

( 1.3.12)EXAMPLELet

and C = BA. Then

We remark that C can be obtained from B and A by a row into column multiplication. Thus c11 is the sum of the entrywise products going across the first row of B and down the first column of A, so in (1.3.12)

c11 = 1.0 + 1.1 + 2.2 = 5 = b1.a.1,

where b1* is the first row of B and a*1 is the first column of A. Similarly, c32 can be obtained by going across the third row of B and down the second column of A. Again, in (1.3.12) we have

c32 = 1.1 + 4(–1) + 9.0 = –3 = b3.a.2.

In general

The formula (1.3.13) holds not just in this one example but whenever the product C = BA is defined.

Also, in (1.3.12)

Thus

These formulas again illustrate a general situation. Thus, if C = BA, where B is m × n and A is n × p, then

To prove (1.3.14) we need merely compare the components of ci* with those of bi1a1* + … + binan*.

The right side of equation (1.3.14) is obtained by multiplying each row of A by a scalar and then adding. A sum of the form β1a1* + … + βnan* is called a linear combination of the rows a1* to an* (Linear combinations will be of importance in Chapter 3 and will be studied there in great detail.)

In view of the importance of (1.3.14) we shall sum it up by a theorem.

( 1.3.15)THEOREMIf C = BA, then the ith row of C is a linear combination of the rows of A with coefficients from the ith row of B.

Similarly, we observe that in Example (1.3.12),

Again this is a special case of a general formula. Thus if C = BA, where B is m × n and A is n × p, then

Extending the notion of linear combinations to columns in the obvious way we have the analogue to Theorem (1.3.15) in

( 1.3.17)THEOREMIf C = BA, then the jth column of C is a linear combination of the columns of B with coefficients from the jth column of A.

The reader should also check that in Example (1.3.12),

and in general

Note that for each k, b*kak* is an m × p matrix.

Let us now return to some properties of matrix multiplication. We can now prove

( 1.3.18)THEOREMFor all matrices A, B, and C, and any scalar α:

(1) A(BC) = (AB)C.

(2) A(B + C) = AB + AC.

(3) (A + B)C = AC + BC.

(4) α(AB) = (αA)B.

whenever all the products are defined.

PROOF OF

(1) Let AB = D, BC = G, (AB)C = F, and A(BC) = H. We must show that F = H.

Hence

The proofs of (2), (3), and (4) are similar and are left to the reader as an exercise.

As an example of (1) in the theorem, let

Then

We see in this example that F = H.

( 1.3.19)REMARKIf A, B, and C are n × n matrices, then all sums and products in (1.3.18) are defined, and therefore the equalities of that theorem hold. The reader familiar with abstract algebra will realize that Theorems (1.2.2) and (1.3.18) imply that n,n is an algebra over under matrix addition, matrix multiplication, and scalar multiplication.

The identity contained in part (1) of Theorem (1.3.18) is called the associative law for multiplication. The identities contained in parts (2) and (3) together are called the distributive law. The commutative law for multiplication would mean that AB = BA whenever the products are defined, but in the beginning of Section 4 we shall show by an example that there are matrices A and B such that AB and BA are both defined but AB ≠ BA. (In this connection, see exercise 4 of this section.)

EXERCISES

1. Check the following products.

2.Which of the matrices in exercise 1 of Section 2are multiplicatively conformable? Calculate three such products.

3. Check parts (2) and (3) of Theorem (1.3.18). (Hint: Use the definition.)

4. Compute the following products:

5. Give an example of a pair of matrices A and B such that AB = BA and an example of a pair C and D such that CD ≠ DC.

6. A square n × n matrix whose only nonzero entries occur in positions aii, i = 1, 2, …, n is called a diagonal matrix (that is, akl = 0 if k ≠ l). In general a11, a22, a33, …, an n is called the main diagonal. Each of the matrices in 4(a) and (b) is diagonal. What about their products? Prove, in general, that the product of two n × n diagonal matrices D1, D2 is diagonal and that D1 and D2 commute. (Hint: Let D1 = [dij] and D2 = [d′kl]. Use the definition of matrix multiplication to show that there is at most one nonzero term in ci1 = ∑jdijd′ji.) The matrix

is an example of a diagonal matrix.

7. Square matrices whose only nonzero entries occur on or above the main diagonal are called upper triangular matrices (those for which aij = 0 if i > j). The matrices in 4(c) to (e) are examples of upper triangular matrices. Prove that the product of two upper triangular matrices T1, T2 is an upper tria].

8. A strictly upper triangular matrix is a triangular matrix whose diagonal entries are zero (that is, aij = 0 if i ≥ j). Prove that the product of a strictly upper triangular matrix S and an upper triangular matrix T is strictly upper triangular. Since matrix multiplication is not commutative one must check two products: ST and TS. An example of a strictly upper triangular matrix is

9. Let

Calculate J², J³, J⁴, JA, BJ, J²A, and BJ².

10. Let S be any n × n strictly upper triangular matrix. Prove that Sn = 0.

11. Let Eμν = [eij], where

(a) Show that

(b) Show that if B = [bμν] is any n × n matrix .

(c) Let A be an arbitrary n × n matrix with entry aij ≠ 0. Show that

(Note: Here i and j are fixed.)

12. Use mathematical induction to prove

13. Let . Show that A² = 0.

14. Let

Compute the products AB and BC. Note that A ≠ B.

15. Find two square matrices, C and D, such that (C + D)² ≠ C² + 2CD + D². Why does equality not hold in general?

16. Find all 2 × 2 matrices X such that

17. Find 2 × 2 matrices A ≠ 0 and B ≠ 0 such that A²B² = 0.

18. Let

and let B be any 3 × 3 matrix. Show that the columns of BA are equal. What about the rows of AB?

19. Let B and C be n × n matrices such that C² = 0 and BC = CB. If A = B + C, show that for all integers k > 0, Ak + ¹ = Bk(B + (k + 1)C).

4. SQUARE MATRICES, INVERSES,

AND ZERO DIVISORS

Recall that m, n is the set of all matrices having m rows and n columns. If m = n, then any matrix in n, n is said to be a square matrix. If A and B are in n, n we observe that AB and BA are both well defined and again in n, n. But a simple example shows that in general

AB ≠ BA;

that is, multiplication is not commutative. For let

Then

This example also shows that even if A ≠ 0 and B ≠ 0, we may have AB = 0. Of course, 0B = 0 also. Hence if C = 0 and A and B are the matrices of the above example, then AB = CB. We see that if AB = CB it is not necessarily true that A = C.

In n, n we have a special matrix, called the identity matrix,

defined by

For example,

In has the property that for any A in n n,

InA = A = A In.

We note in passing that if B is in m, n,

ImB = BandBIn = B,

and that for any scalar α,

(αIm)B = αBandB(αIn) = αB.

Recall that for any number x ≠ 0, there is a number y = x−1, called the multiplicative inverse (or reciprocal) of x, such that

xx−1 = 1.

This situation does not hold in n, n, however. For let

Then finding an X in 2,2 such that

is not possible. Regardless of the X chosen,

and so the second row of AX must be [0 0]. Thus we can never have equality in (1.4.1). However, in some cases we can have

AX = I = XA.

For example, let

Then

Keeping this last example in mind, we formulate the following

( 1.4.2)DEFINITIONLet A be a n × n matrix. The n × n matrix X is said to be an inverse of A if and only if

XA = IandAX = I.

( 1.4.3)DEFINITIONThe square matrix A is termed nonsingular if and only if A possesses an inverse. If A has no inverse, A is called singular.

The actual computation of inverses will be taken up later.

( 1.4.4)THEOREMLet A be an n × n matrix. If

XA = IandAY = I,

then X = Y.

In particular, if A is nonsingular, the inverse is unique.

PROOF

X = XI = X(AY) = (XA)Y = IY = Y. In particular, if A is nonsingular and if X and Y are two inverses of A, then XA = I and AY = I, so that X = Y follows.

We shall denote the unique inverse of a nonsingular matrix A by A−1. Thus A−1A = I = AA−1. Later we shall show that if XA = I, then X = A−1.

In the example before Definition (1.4.2) we had

The computation there shows that X = A−1. As another example, let

Then

because

( 1.4.5)LEMMAIf the matrices A and B are nonsingular, then the product AB is nonsingular and

(AB)-1 = B-1A-1.

PROOF

(B−1 A−1)(AB) = B−1(A−1A)B = B−1IB = B−1B = I.

Similarly,

(AB)(B-1A-1) = A(BB)-1A-1 = AIA-1 = AA-1 = I.

By Theorem (1.4.4), (AB)−1, if it exists, is the unique matrix X such that X(AB) = I = (AB)X. Hence (AB)−1 exists and (AB)−1 = B−1A−1.

More generally we can prove by induction that if A1, A2, …, As are nonsingular, then the product

AsAs-1...A1

is nonsingular and

(AsAs-1...A1)-1 = A1-1A2-1...As-1.

If we recall the