MA-110 Linear Algebra and Differential

Equations

Rekha Santhanam

Department of Mathematics
Indian Institute of Technology Bombay
Powai, Mumbai - 76

January 15, 2024

Lecture 5 D3

Rekha Santhanam Lecture 5 D3

 Recap

• If A is invertible then the system Ax = b has a unique solution

for every b.
• Since x = 0 is always a solution of Ax = 0, if Ax = 0 has a
non-zero solution, then A is not invertible by the last remark.
• If A is invertible, then the Gaussian elimination of A produces n
pivots.
• A diagonal matrix A is invertible if and only if A is ____? .



• Since AB = AB∗1 AB∗2 · · · AB∗n and I = e1 e2 · · · en ,
if B = A−1 , then B∗j is a solution of Ax = ej for all j .
• Strategy to find A−1 : Let A be an n × n invertible matrix.
Solve Ax = e1 , Ax = e2 , . . . , Ax = en .

Rekha Santhanam Lecture 5 D3

 Solutions to Multiple Systems
     
0 1 −1 −1 1
Q: Let A = 1 0 2 , b1 =  2  b2 = 0. Solve for
1 2 0 0 2
Ax = b1 and Ax = b2 .
Do we apply Gaussian Elimination on two augmented matrices?


Rephrased question: Let B = b1 b2 . Is there a matrix C such
that AC = B , i.e., such that AC∗1 = b1 , AC∗2 = b2 ?
   
0 1 −1 | − 1 1 1 0 2 | 2 0
R1 ↔R2
[A|B] = 1 0 2 | 2 0 −−−−→ 0 1 −1 | −1 1
  
1 2 0 | 0 2 1 2 0 | 0 2
   
1 0 2 | 2 0 1 0 2 | 2 0
R3 −R1 R3 −2R2
−−−−→ 0 1
 −1 | −1 1 −−−−→ 0 1 −1 | −1 1
 
0 2 − 2 | −2 2 0 0 0 | 0 0
Q: Are Ax = b1 and Ax = b2 both consistent?
Rekha Santhanam Lecture 5 D3
 Solutions to Multiple Systems (Contd.)


Q: Given matrices A, B = b1 b2 , is there a matrix C such
that AC = B ?
   
0 1 −1 | − 1 1 1 0 2 | 2 0
[A|B] = 1 0 2 | 2 0 −→ 0 1 −1 | −1 1
1 2 0 | 0 2 0 0 0 | 0 0
   
0 0
A solution to Ax = b1 is e3 = 0 , and to Ax = b2 is e2 = 1.
  
1 0


(Verify)! So C = e3 e2 works! Is it unique?
Revisit the question about matrix inverses. Can you find inverse
of a matrix this way?

Rekha Santhanam Lecture 5 D3

 Finding inverse of matrix
Strategy : . Let A be an n × n matrix. If v1 , v2 , . . . , vn are
solutions of Ax = e1 , Ax = e2 , . . . ,
Ax = en respectively, then if it
exists, A−1 = v1 v2 · · ·


vn .

If Ax = ej is not solvable for some j , then A is not invertible.

Thus, finding A−1 reduces to solving multiple systems of linear
equations with the same coefficient matrix.
Consider the previous example, A. Is it invertible?
   
0 1 −1 1 0 2
A = 1 0 2  −→ 0 1 −1 
1 2 0 0 0 0

Observe: In the above process, we used a row exchange:

R1 ↔ R2 and elimination using pivots: R3 = R3 − R1 ,
R3 = R3 − 2R2 . Row operations can be achieved by left
multiplication by special matrices.
Rekha Santhanam Lecture 5 D3
 Row Operations: Elementary Matrices
    
1 0 0 u u
Example: E x = −2 1 0  v  = v − 2u  .
0 0 1 w w



If A = A∗1 A∗2 A∗3 , then EA = EA∗1 EA∗2 EA∗3 .
Thus, EA has the same effect on A as the row operation
R2 7→ R2 + (−2)R1 on the matrix A.

Note: E is obtained from the identity matrix I by the row

operation R2 7→ R2 + (−2)R1 .
Such a matrix (diagonal entries 1 and atmost one off-diagonal
entry non-zero) is called an elementary matrix.
 
Notation: E := E21 (−2). Similarly define Eij (λ).
 

Rekha Santhanam Lecture 5 D3

 Row Operations: Permutation Matrices
    
0 1 0 u v
Example: P x= 1 0 0
   v = u
 
0 0 1 w w



If A = A∗1 A∗2 A∗3 , then PA = PA∗1 PA∗2 PA∗3 .
Thus PA has the same effect on A as the row interchange
R1 ↔ R2 .
Note: We get P from the I by interchanging first and second
rows. A matrix is called a permutation matrix if it is obtained
from identity by row exchanges (possibly more than one).
 
Notation: P = P12 . Similarly define Pij .
 
Remark: Row operations correspond to multiplication by
elementary matrices Eij (λ) or permutation matrices Pij on the
left.

Rekha Santhanam Lecture 5 D3

 Elementary Matrices: Inverses
For any n × n matrix A, observe that the row operations
R2 7→ R2 − 2R1 , R2 7→ R2 + 2R1 leave the matrix unchanged.
In matrix terms, E21 (2)E21 (−2)A = IA = A since
    
1 0 0 1 0 0 1 0 0
E21 (−2) E21 (2) = −2 1 0
   2 1 0 = 0 1 0 .
 
0 0 1 0 0 1 0 0 1
 
1 0 0
• If E21 (λ) = λ 1
 0, what is your guess for E21 (λ)−1 ?
0 0 1
Verify.
   
0 1 0 e2T
−1
• Let P12 = 1 0 0 = e1T . What is P12
T T
? P12 P12 ? P12 ?
0 0 1 T
e3

Rekha Santhanam Lecture 5 D3

 Permutation Matrices: Inverses

Notice that the row interchange R1 ↔ R2 followed by R1 ↔ R2

leaves a matrix unchanged.
In matrix terms, P12 P12 A = IA = A, since
    
0 1 0 0 1 0 1 0 0
P12 P12 = 1 0 0
   1 0 0 = 0 1 0 .
 
0 0 1 0 0 1 0 0 1

• Let Pij be obtained by interchanging the i th and j th rows of I .

Show that PijT = Pij = Pij−1 .
   
0 0 1 e3T
• Let P = 1 0 0 = e1T . Show that P = P12 P23 .
0 1 0 e2T
Hence, P −1 = (P12 P23 )−1 = P23 −1 −1 T T
P12 = P23 P12 = P T .

Rekha Santhanam Lecture 5 D3

 Elimination using Elementary Matrices

    
2 1 1 u 5
Consider  4 −6 0  v  = −2 (Ax = b)
−2 7 2 w 9
 
Step 1 Eliminate u by R2 7→ R2 + (−2)R1 , R3 7→ R3 + R1 .
 
This corresponds to multiplying both sides on the left first by
E21 (−2) and then by E31 (1). The equivalent system is:

E31 (1)E21 (−2)Ax = E31 (1)E21 (−2)b , i.e.,

    
2 1 1 u 5
0 −8 −2  v  =  − 12 .
0 8 3 w 14

Rekha Santhanam Lecture 5 D3

 Elimination using Elementary Matrices
 
Step 2 Eliminate v by R3 7→ R3 + R2 ,
 
i.e., multiply both sides by E32 (1) toget Ux = c , 
2 1 1
where U = E32 (1)E31 (1)E21 (−2)A =  0 −8 −2 and
0 0 1
 
5
c = E32 (1)E31 (1)E21 (−2)b =  − 12 .
2
Elimination changed A to an upper triangular matrix
and reduced the problem to solving Ux = c .
Observe: The pivots of the system Ax = b are the diagonal
entries of U .

Rekha Santhanam Lecture 5 D3

 Triangular Factorization
   
Thus Ax = b is equivalent to .
Ux = c 
where
E32 (1) E31 (1) E21 (−2) A = U

Multiply both sides by E32 (−1) on the left:

E31 (1) E21 (−2) A = E32 (−1)U
Multiply first by E31 (−1) and then E21 (2) on the left:

A = E21 (2) E31 (−1) E32 (−1) U = LU

where U is upper triangular, which is obtained by forward

elimination, with diagonal entries as pivots and
L = E21 (2) E31 (−1) E32 (−1).

Rekha Santhanam Lecture 5 D3

 Triangular Factorization

Note that each Eij (a) is a lower triangular. Product of lower

triangular matrices is lower triangular. In particular L is lower
triangular, where
L = E21 (2) E31 (−1) E32 (−1) =
     
1 0 0 1 0 0 1 0 0 1 0 0
2 1 0  0 1 0 0 1 0 =  2 1 0
0 0 1 −1 0 1 0 −1 1 −1 −1 1

Observe: L is lower triangular with diagonal entries 1 and below

the diagonals are the multipliers.
(2, −1, −1 in the earlier example).

Rekha Santhanam Lecture 5 D3