Download the Full Version of textbook for Fast Typing at textbookfull.

com

Linear and complex analysis for applications

D'Angelo

https://textbookfull.com/product/linear-and-complex-
analysis-for-applications-dangelo/

OR CLICK BUTTON

DOWNLOAD NOW

Download More textbook Instantly Today - Get Yours Now at textbookfull.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Big and Complex Data Analysis Methodologies and

Applications Ahmed

https://textbookfull.com/product/big-and-complex-data-analysis-
methodologies-and-applications-ahmed/

textboxfull.com

Complex Analysis with Applications 1st Edition Nakhlé H.

Asmar

https://textbookfull.com/product/complex-analysis-with-
applications-1st-edition-nakhle-h-asmar/

textboxfull.com

Semigroups of Linear Operators With Applications to

Analysis Probability and Physics 1st Edition David
Applebaum
https://textbookfull.com/product/semigroups-of-linear-operators-with-
applications-to-analysis-probability-and-physics-1st-edition-david-
applebaum/
textboxfull.com

Systems Analysis Approach for Complex Global Challenges

Priscilla Mensah

https://textbookfull.com/product/systems-analysis-approach-for-
complex-global-challenges-priscilla-mensah/

textboxfull.com
Linear and Generalized Linear Mixed Models and Their
Applications 2nd Edition Jiming Jiang

https://textbookfull.com/product/linear-and-generalized-linear-mixed-
models-and-their-applications-2nd-edition-jiming-jiang/

textboxfull.com

Linear Functional Analysis for Scientists and Engineers

1st Edition Balmohan V. Limaye (Auth.)

https://textbookfull.com/product/linear-functional-analysis-for-
scientists-and-engineers-1st-edition-balmohan-v-limaye-auth/

textboxfull.com

Handbook of Linear Partial Differential Equations for

Engineers and Scientists Second Edition Andrei D. Polyanin

https://textbookfull.com/product/handbook-of-linear-partial-
differential-equations-for-engineers-and-scientists-second-edition-
andrei-d-polyanin/
textboxfull.com

Spectral Mixture for Remote Sensing Linear Model and

Applications Yosio Edemir Shimabukuro

https://textbookfull.com/product/spectral-mixture-for-remote-sensing-
linear-model-and-applications-yosio-edemir-shimabukuro/

textboxfull.com

Analysis and Control of Output Synchronization for Complex

Dynamical Networks Jin-Liang Wang

https://textbookfull.com/product/analysis-and-control-of-output-
synchronization-for-complex-dynamical-networks-jin-liang-wang/

textboxfull.com
 LINEAR AND
COMPLEX ANALYSIS
FOR APPLICATIONS
Advances in Applied Mathematics

Series Editor: Daniel Zwillinger

Published Titles
Advanced Engineering Mathematics with MATLAB, Fourth Edition
Dean G. Duffy
CRC Standard Curves and Surfaces with Mathematica®, Third Edition
David H. von Seggern
Dynamical Systems for Biological Modeling: An Introduction
Fred Brauer and Christopher Kribs
Fast Solvers for Mesh-Based Computations Maciej Paszyński
Green’s Functions with Applications, Second Edition Dean G. Duffy
Handbook of Peridynamic Modeling Floriin Bobaru, John T. Foster,
Philippe H. Geubelle, and Stewart A. Silling
Introduction to Financial Mathematics Kevin J. Hastings
Linear and Complex Analysis for Applications John P. D’Angelo
Linear and Integer Optimization: Theory and Practice, Third Edition
Gerard Sierksma and Yori Zwols
Markov Processes James R. Kirkwood
Pocket Book of Integrals and Mathematical Formulas, 5th Edition
Ronald J. Tallarida
Stochastic Partial Differential Equations, Second Edition Pao-Liu Chow
Quadratic Programming with Computer Programs Michael J. Best
 Advances in Applied Mathematics

LINEAR AND
COMPLEX ANALYSIS
FOR APPLICATIONS

John P. D’Angelo
University of Illinois
Urbana-Champaign, USA
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742

© 2017 by Taylor & Francis Group, LLC

CRC Press is an imprint of Taylor & Francis Group, an Informa business

No claim to original U.S. Government works

Printed on acid-free paper

Version Date: 20161123

International Standard Book Number-13: 978-1-4987-5610-5 (Hardback)

This book contains information obtained from authentic and highly regarded sources. Reasonable
efforts have been made to publish reliable data and information, but the author and publisher cannot
assume responsibility for the validity of all materials or the consequences of their use. The authors and
publishers have attempted to trace the copyright holders of all material reproduced in this publication
and apologize to copyright holders if permission to publish in this form has not been obtained. If any
copyright material has not been acknowledged please write and let us know so we may rectify in any
future reprint.

Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced,
transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or
hereafter invented, including photocopying, microfilming, and recording, or in any information
storage or retrieval system, without written permission from the publishers.

For permission to photocopy or use material electronically from this work, please access
www.copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc.
(CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization
that provides licenses and registration for a variety of users. For organizations that have been granted
a photocopy license by the CCC, a separate system of payment has been arranged.

Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and
are used only for identification and explanation without intent to infringe.
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
 Contents

Preface vii

Chapter 1. Linear algebra 1

1. Introduction to linear equations 1
2. Vectors and linear equations 4
3. Matrices and row operations 9
4. Equivalence relations 12
5. Vector spaces 16
6. Dimension 21
7. Eigenvalues and eigenvectors 25
8. Bases and the matrix of a linear map 29
9. Determinants 33
10. Diagonalization and generalized eigenspaces 37
11. Characteristic and minimal polynomials 41
12. Similarity 43
13. Additional remarks on ODE 44

Chapter 2. Complex numbers 47

1. Basic definitions 47
2. Limits 50
3. The exponential function and trig 53
4. Subsets of C 57
5. Complex logarithms 59
6. Linear fractional transformations 63
7. Linear fractional transformations and matrices 67
8. The Riemann sphere 69

Chapter 3. Vector analysis 73

1. Euclidean geometry 73
2. Differentiable functions 76
3. Line integrals and work 82
4. Surface integrals and flux 86
5. The divergence theorem 90
6. The classical Stokes’ theorem 94
7. A quick look at Maxwell’s equations 96
8. Differential forms 98
9. Inverse and implicit functions 104
v
vi CONTENTS

Chapter 4. Complex analysis 109

1. Open subsets of C 109
2. Complex differentiation 110
3. Complex integration 113
4. The Cauchy theory 114
5. Polynomials and power series 119
6. Singularities 121
7. Residues 124
8. More residue computations 128
Chapter 5. Transform methods 135
1. Laplace transforms 135
2. Generating functions and the Z-transform 145
3. Fourier series 156
4. Fourier transforms on R 162
Chapter 6. Hilbert spaces 173
1. Inner products and norms 173
2. Orthonormal expansion 176
3. Linear transformations 178
4. Adjoints 179
5. Hermitian and unitary operators 182
6. L2 spaces 188
7. Sturm–Liouville equations 191
8. The Green’s function 195
9. Additional techniques 198
10. Spectral theory 202
11. Orthonormal expansions in eigenvectors 204
Chapter 7. Examples and applications 207
1. Least squares 207
2. The wave equation 212
3. Legendre polynomials 216
4. Probability 219
5. Quantum mechanics 224
6. The Dirichlet problem and conformal mapping 230
7. Root finding 235
8. Linear time-invariant systems 243
Chapter 8. Appendix: The language of mathematics 251
1. Sets and functions 251
2. Variables, constants, and parameters 257

References 259
Index 261
 Preface

This book evolved from several of the author’s teaching experiences, his research in complex analysis
in several variables, and many conversations with friends and colleagues. It has been also influenced by
a meeting he attended (Santa Barbara, 2014) on the revitalization of complex analysis in the curriculum.
That meeting suggested that complex analysis was losing its luster as a gem in the curriculum. This book
therefore aims to unify various parts of mathematical analysis, including complex variables, in an engaging
manner and to provide a diverse and unusual collection of applications, both to other fields of mathematics
and to physics and engineering.
The first draft of the book was based on a course taught in Spring 2014 at the University of Illinois. The
course, Electrical and Computer Engineering (ECE) 493, is cross-listed as Math 487. Twenty-one students,
including several graduate students, completed the class. Areas of specialization were electrical, mechanical,
and civil engineering, engineering physics, and mathematics. Background and interests varied considerably;
communicating with all these students at the same time and in the same notation was challenging.
The course began with three reviews. We discussed elementary linear algebra and differential equations,
basic complex variables, and some multi-variable calculus. The students took two diagnostic exams on these
topics. The first three chapters include these reviews but also introduce additional material. The author feels
that spiral learning works well here; in the classroom he prefers (for example) to introduce differential
forms when reviewing line and surface integrals rather than to start anew. The course used these reviews
to gradually seduce the student into a more abstract point of view. This seduction is particularly prominent
in the material on Hilbert spaces. This book aims to develop understanding of sophisticated tools by using
them rather than by filling in all the details, and hence it is not completely rigorous. The review material
helped students relearn things better while creating a desire to see powerful ideas in action.
After the reviews, the course began anew, with about 4 weeks on complex analysis, 1 week on the
Laplace transform, and 3 weeks on Hilbert spaces and orthonormal expansion. We discussed Sturm–
Liouville theory to give examples of orthonormal expansion, and we briefly discussed generating functions
(and the so-called Z-transform) to provide a discrete analogue of the Laplace transform. The unifying
theme of the class remained linearity and related spectral methods. The course sketched a proof of the
Sturm–Liouville theorem, but filling in the details was not possible because student backgrounds in analysis
were so varied.
After typing notes for the course, the author began adding material. The primary influences in selecting
the added material have been honors linear algebra classes, applied complex variable classes, and the excite-
ment coming from seeing the same mathematical principles in diverse settings. The author is happy with the
many novel features, the wide variety of exercises, and the manner in which the text unifies topics.
As a whole, the book might be too mathematical for engineers, too informal for mathematicians, and
too succinct for all but the top students. Below, however, we offer six specific course outlines that overcome
these issues. The appendix, a late addition to the text, might help some readers overcome discomfort with the
use of mathematical jargon. It also provides a diverse list of operations which can be regarded as functions;
this list glimpses the vast scope of topics covered in the book.
vii
viii PREFACE

Mathematical prerequisites for the book include three semesters of calculus, a beginning course in
differential equations, and some elementary linear algebra. Complex variable theory is developed in the
book, and hence is not a prerequisite. First year courses in Physics are useful but not strictly necessary.
Several possible courses can be based upon this book. We offer six possibilities:
(1) The first five chapters form a coherent course on engineering mathematics. Such a course should
appeal to engineering faculty who want an integrated treatment of linear algebra and complex
analysis, including applications, that also reviews vector analysis. The course could then conclude
with Section 8 from Chapter 7 on linear time-invariant systems.
(2) The first five chapters (omitting parts of Chapter 3) together with the last three sections of Chapter 7
form a standard applied complex variables course, including transform methods.
(3) Each section in Chapter 7 begins by stating approximate prerequisites from the book. Some in-
structors might wish to decide a priori which applications in Chapter 7 they wish to discuss, and
then cover Chapters 1 and 2 together with those topics needed for these applications.
(4) After covering the first four chapters, students in mathematics and physics can continue with the
Hilbert space chapter and conclude with the sections on probability and quantum mechanics.
(5) Students with strong backgrounds can read the first three chapters on their own as review, and take
a one-semester course based on Chapters 4 through 7. Such a course would be ideal for many
graduate students in applied mathematics, physics, and engineering.
(6) Chapters 1, 6 and Sections 1, 5, 7 of Chapter 7 make a nice second course in linear algebra.
The book contains more than 450 exercises of considerably varying difficulty and feel. They are placed
at the ends of sections, rather than at the end of chapters, to encourage the reader to solve them as she reads.
Many of the exercises are routine and a few are difficult. Others introduce new settings where the techniques
can be used. Most of the exercises come from mathematics; instructors may wish to provide additional
exercises coming from engineering and physics. The text weaves abstract mathematics, computations, and
applications into a coherent whole, whose unifying theme is linear systems.
The book itself has three primary goals. One goal is to develop enough linear analysis and complex
variable theory to prepare students in engineering or applied mathematics for advanced work. The second
goal is to unify many distinct and seemingly isolated topics. The third goal is to reveal mathematics as both
interesting and useful, especially via the juxtaposition of examples and theorems. We give some examples
of how we achieve these goals:
• Many aspects of the theory of linear equations are the same in finite and infinite dimensions; we
glimpse both the similarities and the differences in Chapter 1 and develop the ideas in detail in
Chapter 6.
• One recurring theme concerns functions of operators. We regard diagonalization of a matrix and
taking Laplace or Fourier transforms as similar (pun intended) techniques. Early in the 493 class,
and somewhat out of the blue, the author asked what √ it would mean to take “half of a derivative.”
One bright engineering student said “multiply by s”. A year and a half later the author asked his
applied complex variables class√ the same question. A bright physics student answered “first take
Laplace transforms and then s”. He did not actually say multiply but surely he understood.
• Choosing coordinates and notation in which computations are easy is a major theme in this book.
Complex variables illustrate this theme throughout. For example, we use complex exponentials
rather than trig functions nearly always.
• A Fourier series is a particular type of orthonormal expansion; we include many examples of
orthonormal expansion as well as the general theory.
• We state the spectral theorem for compact operators on Hilbert space as part of our discussion of
Sturm–Liouville equations, hoping that many readers will want to study more operator theory.
 PREFACE ix

• We express Maxwell’s equations in terms of both vector fields and differential forms, hoping that
many readers will want to study differential geometry.
• We discuss root finding for polynomials in one variable and express the Jury stabilization algorithm
in a simple manner.
Many of the examples will be accessible yet new to most readers. Here are some examples of unusual
approaches and connections among different topics.
• To illustrate the usefulness of changing basis, we determine which polynomials with rational co-
efficients map the integers to the integers. Later on, the same idea helps us to find the generating
function of an arbitrary polynomial sequence.
• We mention the scalar field consisting of two elements, and include several exercises about lights-
out puzzles to illuminate linear algebra involving this field. At the very end of the book, in the
section on root finding, we give the simple exercise: show that the fundamental theorem of algebra
fails in a finite field. (In Chapter 4 we prove this theorem for complex polynomials.)
• We compute line and surface integrals using both differential forms and the classical notation of
vector analysis. We express Maxwell’s equations using differential forms in a later section.
• To illustrate sine series, we prove the Wirtinger inequality relating the L2 norms of a function and
its derivative.
• We develop connections between finite- and infinite-dimensional linear algebra, and many of these
ideas will be new to most readers. For example, we compute ζ(2) by finding the trace of the
Green’s operator for the second derivative.
• We introduce enough probability to say a bit about quantum mechanics; for example, we prove the
Heisenberg uncertainty principle. We also include exercises on dice with strange properties.
• Our discussion of generating functions (Z-transforms) includes a derivation of a general formula
PN
for the sums n=1 np (using Bernoulli numbers) as well as (the much easier) Binet’s formula for
the Fibonacci numbers. The section on generating functions contains several unusual exercises,
including a test for rationality and the notion of Abel summability. There we also discuss the
precise meaning of the notorious but dubious formula 1 + 2 + 3 + · · · = −1 12 , and we mention the
concept of zeta function regularization. This section helps unify many of the ideas in the book,
while at the same time glimpsing more sophisticated ideas.
• The section on Fourier series includes a careful discussion of summation by parts.
Several features distinguish this book from other applied math books. Perhaps most important is how
the book unifies concepts regarded as distinct in many texts. Another is the extensive mathematical treatment
of Hilbert spaces. The scope and diversity of our examples distinguish it from other pure math books.
Formal theoretical items such as theorems, definitions, lemmas, and propositions are numbered as a unit
within each section. Examples are numbered separately, also within each section. Thus Lemma 3.2 must
follow Theorem 3.1 or Definition 3.1 or whatever is the first theoretical item in the section, whereas Example
3.2 could precede Theorem 3.1. Remarks are not numbered.
This book is not an encyclopedia and it does not cover all significant topics in either analysis or en-
gineering mathematics. Omissions include numerical methods and mathematical modeling. For example,
although we write an inversion formula for the Laplace transform involving an integral, we never consider
numerical evaluation of such integrals. We discuss eigenvalues throughout the book, well aware that tech-
niques exist for finding them that are far better than using the characteristic polynomial; nonetheless we
do not develop such techniques. We do show that the eigenvalues of a self-adjoint matrix can be found
via optimization. Our discussion of the condition number of a matrix or linear map is quite brief; we do
provide some exercises about the Hilbert matrix. Although we discuss how to solve a few linear differential
x PREFACE

equations, we say little about specific circumstances whose modeling leads to these equations. One omis-
sion concerns partial differential equations (PDE). We discuss the one-dimensional wave equation, the heat
equation, and the Schrödinger equation, primarily to illustrate separation of variables, but we include little
general theory. Section 6 of Chapter 7 discusses the Dirichlet problem and conformal mapping, but four
pages cannot do justice to this material. Although we mention approximate identities at several places, we
do not discuss distribution theory. The author recommends [T] for an excellent treatment of PDE that nicely
weaves together the theory and important examples. Perhaps the most significant omission is wavelets. Al-
though this topic nicely fits with much of what appears here, there is simply too much information. To quote
rhythm and blues singer Smokey Robinson, “a taste of honey is worse than none at all.” We therefore refer
to [Dau], [M] and [N] for the subject of wavelets.
The author acknowledges the contributions of many people. People involved with the original ECE
class include Jont Allen and Steve Levinson (both in ECE), Sarah Robinson (graduate student in ECE), Tej
Chajed (then a senior in computer science), Pingyu Wang (a student in the class who took class notes on
a tablet), and Jimmy Shan (a mathematics graduate student who typed these notes into a LaTeX file). A
few of the topics resulted directly from questions asked by Jont Allen. Sam Homiller (then a student in
engineering physics) made some of the pictures used in this book. Several of these students were partially
supported by a small grant from IMSE, an initiative at the University of Illinois which is attempting to bring
mathematicians and engineers closer together. Bruce Hajek and Erhan Kudeki (both of ECE) invited the
author to several meetings of their curriculum committee where stimulating discussions ensued. Bruce Hajek
gave the following wonderful advice: take the reader on a guided tour of some infinite forest, pointing out
various highlights and landmarks, but it is not possible to give a complete view in one excursion. Physicist
Simon Kos and the author had several productive discussions about Maxwell’s equations and related topics.
Physicist Mike Stone has provided numerous useful ideas to the author over the years. Ki Yeun (Eunice) Kim
worked with the author in spring 2015 as a research assistant supported by the National Science Foundation.
She provided many of the pictures as well as significantly many useful criticisms and comments. Upon
reading a preliminary draft, Sergei Gelfand asked for a clarification of the goals of the book, thereby leading
to significant improvements and revisions, especially in the last two chapters. Dan Zwillinger provided
a useful balance of praise and criticism of an early draft and also helped me polish a late draft. Robert
Ross offered both encouragement and many useful comments. Charlie Epstein suggested several significant
improvements, while emphatically stating that the book is too mathematical for engineers. Students in
other classes, especially Applied Complex Variables, have indirectly helped the author organize some of the
material. Martin Ostoja-Starzewski (Mechanical Science and Engineering) provided valuable comments and
encouragement while discussing the different perspectives of mathematics and engineering. Mathematics
graduate student Zhenghui Huo influenced the treatment of several topics. Anil Hirani, Rick Laugesen,
and Bill Helton each provided good ideas for bridging the gap between mathematics and engineering. Phil
Baldwin patiently explained to the author what real world scientists actually do; his comments led to several
improvements. The author’s then thirteen-year-old son Henry suggested Exercise 5.1 of Chapter 3.
The author acknowledges support provided by NSF Grant DMS 13-61001.
 CHAPTER 1

Linear algebra

Linear algebra arises throughout mathematics and the sciences. The most basic problem in linear algebra
is to solve a system of linear equations. The unknown can be a vector in a finite-dimensional space, or an
unknown function, regarded as a vector in an infinite-dimensional space. One of our goals is to present
unified techniques which apply in both settings. The reader should be to some extent already familiar with
most of the material in this chapter. Hence we will occasionally use a term before defining it precisely.

1. Introduction to linear equations

We begin by reviewing methods for solving both matrix equations and linear differential equations.
Consider the linear equation Lx = b, where b is a given vector, and x is an unknown vector. To be precise,
x lives in a vector space V and b lives in a vector space W ; in some but not all situations, V = W . The
mapping L : V → W is linear. We give precise definitions of vector spaces and linear maps a bit later.
There are many books on linear algebra; we recommend [HK] for its abstract treatment and [St] for its
applications and more concrete treatment. This chapter includes several topics not generally found in any
linear algebra book. To get started we consider three situations, two just fun, where linear algebra makes an
appearance. After that we will be a bit more formal.
When there are finitely many equations in finitely many unknowns, we solve using row operations, or
Gaussian elimination. We illustrate by discussing a simple Kakuro puzzle. Kakuro is a puzzle where one
is given a grid, blank squares, the row sums, the column sums, and blacked-out squares. One is supposed
to fill in the blanks in the grid, using the digits 1 through 9. There are two added constraints; no repeats
(within a fixed group) are allowed and the solution is known to be unique. In difficult puzzles, knowing that
the solution is unique can sometimes help. We ignore these constraints in the following simple example.
Consider a three-by-three grid, as shown in Figure 1. From left to right, the column sums are given to
be 10, 13, and 13. From top to bottom, the row sums are given to be 17, 7, and 12. Since the solution is
known to use only the whole numbers from 1 through 9, one can easily solve it. In this example, there are
two possible solutions. Starting from the top and going across, the possible solutions are (9, 8, 1, 2, 4, 3, 9)
and (8, 9, 2, 1, 4, 3, 9).

10 13 13
17

12

F IGURE 1. An easy Kakuro puzzle

1
2 1. LINEAR ALGEBRA

Label the unknown quantities x1 , x2 , . . . , x7 . The given row and column sums lead to the following
system of equations of six equations in seven unknowns:

x1 + x2 = 17
x3 + x4 + x5 = 7
x6 + x7 = 12
x1 + x3 = 10
x2 + x4 + x6 = 13
x5 + x7 = 13

To solve this system of equations, we first write it in matrix form

 
1 1 0 0 0 0 0 17
0 0 1 1 1 0 0 7
 
0 0 0 0 0 1 1 12
 . (1)
1 0 1 0 0 0 0 10
 
0 1 0 1 0 1 0 13
0 0 0 0 1 0 1 13

The last column is the right-hand side of the system of equations, the other columns are the coefficients of
the equations, and each row corresponds to an equation. It is convenient to reorder the equations, obtaining:

 
1 1 0 0 0 0 0 17
1 0 1 0 0 0 0 10
 
0 1 0 1 0 1 0 13
 .
0 0 1 1 1 0 0 7
 
0 0 0 0 1 0 1 13
0 0 0 0 0 1 1 12

Reordering the equations does not change the solutions. It is easy to see that we can achieve any order by
several steps in which we simply interchange two rows. Such an interchange is an example of an elementary
row operation. These operations enable us to simplify a system of linear equations expressed in matrix form.
There are three types of elementary row operations. We can add a multiple of one row to another row, we
can interchange two rows, and we can multiply a row by a non-zero constant.
This simple example does not require much effort. By row operations, we get the equivalent system
 
1 1 0 0 0 0 0 17
0
 −1 1 0 0 0 0 −7
0 0 1 1 0 1 0 6
 .
0 0 1 1 1 0 0 7
 
0 0 0 0 1 0 1 13 
0 0 0 0 0 1 1 12
 1. INTRODUCTION TO LINEAR EQUATIONS 3

After a few more simple steps we obtain

 
1 0 0 −1 0 0 1 16
0 1 0
 1 0 0 −1 1
0 0 1 1 0 0 −1 −6.

0 0 0 (2)
 0 0 0 0 0
0 0 0 0 1 0 1 13 
0 0 0 0 0 1 1 12
The matrix equation in (2) has the same solutions as the matrix equation in (1), but it is expressed in a
manner where one can see the solutions. We start with the last variable and work backwards. No row in (2)
determines the last variable x7 , and hence this variable is arbitrary. The bottom row of (2) gives x6 +x7 = 12
and hence determines x6 in terms of x7 . The second row from the bottom determines x5 in terms of x7 . We
proceed, working backwards, and note that x4 is also arbitrary. The third row then determines x3 in terms of
both x4 and x7 . Using the information from all the rows, we write the general solution to (1) in vector form
as follows:        
x1 16 1 −1
 x2   1  −1 1
       
x3  −6 −1 1
       
x =  x4  =  0  + x4  1  + x7 
     
0
 (3)
x5   13  0 −1
       
x6   12  0 −1
x7 0 0 1
Notice that there are two arbitrary parameters in the solution (3): the variables x4 and x7 can be any
real numbers. Setting them both equal to 0 yields one particular solution. Setting x7 = 9 and x4 = 1, for
example, gives the solution x = (8, 9, 2, 1, 4, 3, 9), which solves the Kakuro puzzle. Setting x7 = 9 and
x4 = 2 gives the other Kakuro solution.
When solving a system of linear equations, three things can happen. There can be no solutions, one
solution, or infinitely many solutions. See Theorem 2.5 below. Before proving it, we will need to introduce
some useful concepts. We conclude this section with an example.
E XAMPLE 1.1. Consider the system of three equations in three unknowns given by
x1 + 5x2 − 3x3 = 2
2x1 + 11x2 − 4x3 = 12
x2 + cx3 = b.
Here c and b are constants, and the form of the answer depends on them. In matrix form the system is
 
1 5 −3 2
2 11 −4 12 .
0 1 c b
Two row operations yield the equivalent system
 
1 5 −3 2
0 1 2 8 .
0 0 c−2 b−8
If c = 2, the last row says 0 = 0x1 + 0x2 + 0x3 = b − 8. If b 6= 8, then we get a contradiction and the
system has no solution. If c = 2 and b = 8, then we can choose x3 arbitrarily and satisfy all three equations.
If c 6= 2, then we can solve the system whether or not b = 8, and the solution is unique.
4 1. LINEAR ALGEBRA

E XERCISE 1.1. Put c = 1 in Example 1.1 and solve the resulting system.
E XERCISE 1.2. Solve the Kakuro puzzle in Figure 2. (Do not use linear equations!)

23 18 8 16 13
30

18

8 13

9
7

F IGURE 2. A kakuro puzzle

E XERCISE 1.3. Consider the linear system

x + y + z = 11

2x − y + z = 13
y + 3z = 19.
Convert it to matrix form, solve by row operations, and check your answer.
E XERCISE 1.4. Consider the (augmented) linear system, expressed in matrix form, of four equations in
six unknowns.  
1 2 3 0 0 1 30
0 0 1 0 1 2 8 
 
0 0 0 1 0 0 0 
0 0 0 0 1 5 0
Find the most general solution, and write it in the form
X
x = x0 + cj vj .

2. Vectors and linear equations

We begin an informal discussion of linear equations in general circumstances. This discussion needs to
be somewhat abstract in order to allow us to regard linear differential equations and linear systems (matrix
equations) on the same footing.
Definition 2.1 (informal). A real vector space is a collection of objects called vectors. These objects
can be added together, or multiplied by real scalars. These operations satisfy the axioms from Definition 5.2.
 2. VECTORS AND LINEAR EQUATIONS 5

Definition 2.2. Let V and W be real vector spaces, and suppose L : V → W is a function. Then L is
called a linear transformation (or a linear map or linear) if for all u, v ∈ V , and for all c ∈ R, we have
L(u + v) = Lu + Lv (L.1)
L(cv) = c L(v). (L.2)
Note that the addition on the left-hand side of (L.1) takes place in V , but the addition on the right-hand side
in (L.1) takes place in W . The analogous comment applies to the scalar multiplications in (L.2).
The reader is surely familiar with the vector space Rn , consisting of n-tuples of real numbers. Both
addition and scalar multiplication are formed component-wise. See (7.1) and (7.2) below. Each linear map
from Rn to Rm is given by matrix multiplication. The particular matrix depends on the basis chosen, as we
discuss in Section 8. Let A be a matrix of real numbers with n columns and m rows. If x is an element of
Rn , regarded as a column vector, then the matrix product Ax is an element of Rm . The mapping x 7→ Ax is
linear. The notation I can mean either the identity matrix or the identity linear transformation; in both cases
I(x) = x for each x ∈ Rn .
While Rn is fundamental, it will be insufficiently general for the mathematics in this book. We will also
need both Cn and various infinite-dimensional spaces of functions.
In Example 1.1, V is the vector space R3 . The linear map L : R3 → R3 is defined by
L(x, y, z) = (x + 5y − 3z, 2x + 11y − 4z, y + cz).
We are solving the linear equation L(x, y, z) = (2, 12, b).
When V is a space of functions, and L : V → V is linear, we often say L is a linear operator on V . The
simplest linear operator is the identity mapping, sending each v to itself; it is written I. Other well-known
examples include differentiation and integration. For example, when V is the space of all polynomials in
one variableR x, the map D : V → V given by D(p) = p0 is linear. The map J : V → V defined by
x
(Jp)(x) = 0 p(t)dt is also linear.
E XAMPLE 2.1. We consider the vector space of polynomials of degree at most three, and we define L
by L(p) = p − p00 . We show how to regard L as a matrix, although doing so is a bit clumsy. Put
p(x) = a0 + a1 x + a2 x2 + a3 x3 .
Then we have
(Lp)(x) = p(x) − p00 (x) = (a0 − 2a2 ) + (a1 − 6a3 )x + a2 x2 + a3 x3 .
There is a linear operator on R4 , which we still write as L, defined by
L(a0 , a1 , a2 , a3 ) = (a0 − 2a2 , a1 − 6a3 , a2 , a3 ).
In matrix form, we have  
1 0 −2 0
0 1 0 −6
L=
0
. (∗)
0 1 0
0 0 0 1
Example 2.1 illustrates several basic points. The formula L(p) = p−p00 is much simpler than the matrix
representation in formula (*). In this book we will often use simple but abstract notation. One benefit is that
different applied situations become mathematically identical. In this example we can think of the space V3
of polynomials of degree at most three as essentially the same as R4 . Thus Example 2.1 suggests the notions
of dimension of a vector space and isomorphism.
6 1. LINEAR ALGEBRA

Definition 2.3. Let L : V → V be a linear transformation. Then L is called invertible if there are
linear transformations A and B such that I = LB and I = AL. A square matrix M is invertible if there
are square matrices A and B such that I = M B and I = AM .

R EMARK . A linear map L or a matrix M is invertible if such A and B exist. If they both exist, then
A = B:
A = AI = A(LB) = (AL)B = IB = B.
Note that we have used associativity of composition in this reasoning. When both A and B exist, we
therefore write (without ambiguity) the inverse mapping as L−1 .
Examples where only one of A or B exists arise only in infinite-dimensional settings. See Example 5.3,
which is well known to calculus students when expressed in different words. It clarifies why we write +C
when we do indefinite integrals.
Definition 2.4. Let L : V → W be a linear transformation. The null space of L, written N (L), is the
set of v in V such that Lv = 0. The range of L, written R(L), is the set of w ∈ W such that there is some
v ∈ V with Lv = w.

R EMARK . Let L : Rn → Rn be linear. Then N (L) = {0} if and only if L is invertible. The conclusion
does not hold if the domain and target spaces have different dimensions.
Theorem 2.5. Let V, W be real vector spaces, and suppose L : V → W is linear. For the linear system
Lx = b, exactly one of three possibilities holds:
(1) For each b ∈ V , the system has a unique solution. When W = V , this possibility occurs if and
only if L is invertible, and the unique solution is x = L−1 b. In general, this possibility occurs if
and only if N (L) = {0} (uniqueness) and R(L) = W (existence).
(2) b 6∈ R(L). Then there is no solution.
(3) b ∈ R(L), and N (L) 6= {0}. Then there are infinitely many solutions. Each solution satisfies
x = xpart + v, (∗)
where xpart is a particular solution and v is an arbitrary element of N (L).
P ROOF. Consider b ∈ W . If b is not in the range of L, then (2) holds. Thus we assume b ∈ W is an
element of R(L). Then there is an xpart with L(xpart ) = b. Let x be an arbitrary solution to L(x) = b.
Then b = L(x) = L(xpart ). By linearity, we obtain
0 = L(x) − L(xpart ) = L(x − xpart ).
Hence x − xpart ∈ N (L). If N (L) = {0}, then x = xpart and (1) holds. If N (L) 6= {0}, then we
claim that (3) holds. We have verified (*); it remains to show that there are infinitely many solutions. Since
N (L) is not 0 alone, and it is closed under scalar multiplication, it is an infinite set, and hence the number
of solutions to L(x) = b is infinite as well. 
The main point of Theorem 2.5 is that a system of linear equations with real coefficients (in finitely or
infinitely many variables) can have no solutions, one solution, or infinitely many solutions. There are no
other possibilities. Analogous results hold for systems with complex coefficients and even more generally.
See Exercises 2.5 through 2.8 for easy examples illustrating Theorem 2.5.
This idea of particular solution plus an element of the null space should be familiar from differential
equations. We illustrate by solving an inhomogeneous, constant-coefficient, linear, ordinary differential
equation (ODE).
 2. VECTORS AND LINEAR EQUATIONS 7

E XAMPLE 2.2. We solve the differential equation y 00 + 5y 0 + 6y = 6x − 6. Here we regard y as a vector

in an infinite-dimensional vector space of functions. Let A(y) = y 00 + 5y 0 + 6y and b = 6x − 6. Then A is
linear and we wish to solve Ay = b.
We first find the null space of A by considering the homogeneous equation A(y) = 0. The standard
approach to solve it is to assume a solution of the form eλx . We obtain the equation
(λ2 + 5λ + 6)eλx = 0.
Since eλx 6= 0, we can divide by it and obtain λ2 + 5λ + 6 = 0. This equation is known as the
characteristic equation for the differential equation. Here its solutions are λ = −2, −3. Hence the null
space of A is spanned by the two functions e−2x and e−3x . Thus there are constants c1 and c2 such that
Ay = 0 if and only if y(x) = c1 e−2x + c2 e−3x .
Next we need to find a particular solution. To do so, we seek a particular solution of the form y = cx+d.
Then y 0 = c and y 00 = 0. Plugging in the equation and doing simple algebra yields c = 1 and d = − 11 6 .
The general solution to the differential equation is therefore
11
y(x) = c1 e−2x + c2 e−3x + x − .
6
In a real world problem, we would have additional information, such as the values of y and y 0 at some initial
point. We would then use these values to find the constants c1 and c2 .
R EMARK . In engineering, the current is often written i(t) or I(t). For us i will denote the imaginary
unit (i2 = −1) and I will denote the identity operator or identity matrix. Hence we write I for the current,
both in the next example and in later chapters.
E XAMPLE 2.3. The current in an RLC circuit (Figure 3) is governed by a second-order, constant-
coefficient, linear ordinary differential equation.

R
• •

• •
L

C
• •

• •

V
F IGURE 3. An RLC circuit

Three circuit elements, or impedances, arise in the coefficients of this ODE. Let E(t) be the voltage
drop at time t, and let I(t) be the current at time t. For a resistor, Ohm’s law yields
Eresistor (t) = RI(t).
0
The constant R is called the resistance. Hence Eresistor = RI 0 .
8 1. LINEAR ALGEBRA

For an inductor, the voltage drop is proportional to the time derivative of the current:

Einductor (t) = LI 0 (t).

0
The constant of proportionality L is called the inductance. Hence Einductor = LI 00 .
For a capacitor, the voltage drop is proportional to the charge Q(t) on the capacitor, and the current is
the time derivative of the charge:
1
Ecapacitor (t) = Q(t).
C
0 1
Ecapacitor (t) = I(t).
C
The constant C is called the capacitance.
Using Kirchhoff’s laws, which follow from conservation of charge and conservation of energy, and
assuming linearity, we add the three contributions to obtain the time derivative of the voltage drop:
1
LI 00 (t) + RI 0 (t) + I(t) = E 0 (t). (4)
C

The differential equation (4) expresses the unknown current in terms of the voltage drop, the inductance,
the resistance, and the capacitance. The equation is linear and of the form A(I) = b. Here A is the operator
given by
d2 d 1
A = L 2 + R + I.
dt dt C
The domain of A consists of a space of functions. The current t 7→ I(t) lives in some infinite-dimensional
vector space of functions. We find the current by solving the linear system A(I) = b.
R EMARK . The characteristic equation of the differential equation for an RLC circuit is
1
Lλ2 + Rλ + = 0.
C
Since L, R, C ≥ 0, the roots have non-positive real part. Consider the three cases
L
R2 − 4 > 0
C
L
R2 − 4 = 0
C
2 L
R − 4 < 0.
C
These three cases correspond to whetherqthe roots of the characteristic equation are real and distinct, re-
peated, or complex. The expression ζ = C R
L 2 is called the damping factor of this circuit. Note that ζ > 1
if and only if R2 − 4L 2 4L
C > 0 and ζ < 1 if and only if R − C < 0. The case ζ > 1 is called over-damped.
The case ζ = 1 is called critically damped. The case ζ < 1 is called under-damped and we get oscillation.
L
See Figure 4. The discriminant R2 − 4 C has units ohms squared; the damping factor is a dimensionless
quantity.

E XERCISE 2.1. What is the null space of the operator D3 − 3D2 + 3D − I? Here D denotes differen-
tiation, defined on the space of infinitely differentiable functions.
E XERCISE 2.2. Find the general solution to the ODE y 00 − y = x.
E XERCISE 2.3. Find the general solution to the ODE y 0 − ay = ex . What happens when a = 1?
 3. MATRICES AND ROW OPERATIONS 9

I(t)
1 under-damped
critically damped
over-damped

t
2 4 6 8 10 12 14

−1

F IGURE 4. The damping factor for a circuit

E XERCISE 2.4. For what value of C is the circuit in eqn. (4) critically damped if L = 1 and R = 4?
E XERCISE 2.5. Consider the system of two equations in two unknowns given by x + 2y = 7 and
2x + 4y = 14. Find all solutions and express them in the form (3) from Section 1. For what values of a, b
does the system x + 2y = a and 2x + 4y = b have a solution?
E XERCISE 2.6. Consider the system x + 2y = 7 and 2x + 6y = 16. Find all solutions. For what values
of a, b does the system x + 2y = a and 2x + 6y = b have a solution?
√
E XERCISE 2.7. In this exercise i denotes −1. Consider the system z +(1+i)w = −1 and iz −w = 0,
for z, w unknown complex numbers. Find all solutions.
√
E XERCISE 2.8. Again i denotes −1. Consider the system z + (1 + i)w = −1 and −iz + (1 − i)w = i.
Find all solutions and express in the form (3) from Section 1.

3. Matrices and row operations

We wish to extend our discussion of row operations. We begin by recalling the definitions of matrix
addition and matrix multiplication. We add two matrices of the same size by adding the corresponding
elements. The i, j entry in A + B is therefore given by
(A + B)ij = Aij + Bij .
We can multiply matrices A and B to form BA only when the number of columns of B equals the number
of rows of A. In that case, the i, j entry in BA is defined by
Xn
(BA)ij = Bik Akj .
k=1
Here A has n columns and B has n rows. It follows easily from this definition that matrix multiplication is
associative: A(BC) = (AB)C whenever all the products are defined. Matrix multiplication is not generally
commutative; even if both AB and BA are both defined, they need not be equal. We return to these issues a
bit later.
10 1. LINEAR ALGEBRA

A matrix with m columns and n rows (with real entries) defines a linear map A : Rm → Rn by matrix
multiplication: x 7→ Ax. If x ∈ Rm , then we can regard x as a matrix with 1 column and m rows. The
matrix product Ax is defined and has 1 column and n rows. We identify such a column vector with an
element of Rn .
When we are solving the system Ax = b, we augment the matrix by adding an extra column corre-
sponding to b:
 
a11 a12 · · · a1m b1
 a21 a22 · · · a2m b2 
..  .
 
 .. .. ..
 . . ··· . .
an1 an2 · · · anm bn
We perform row operations on a matrix, whether augmented or not. Let rj denote the j-th row of a
given matrix. The row operations are
• Multiply a row by a non-zero constant: rj → λrj where λ 6= 0.
• Interchange two rows. For k 6= j, rj → rk and rk → rj .
• Add a multiple of a row to another row. For λ 6= 0 and k 6= j, rj → rj + λrk .
Doing a row operation on a matrix A corresponds to multiplying on the left by a particular matrix, called
an elementary row matrix. We illustrate each of the operations in the 2-dimensional case:
 
1 0
(ER.1)
0 λ
 
0 1
(ER.2)
1 0
 
1 λ
(ER.3).
0 1
Multiplying the second row of a (two-by-two) matrix corresponds to multiplying on the left by the
matrix in (ER.1). Switching two rows corresponds to multiplying on the left by the matrix in (ER.2). Adding
λ times the second row to the first row corresponds to multiplying on the left by the matrix in (ER.3).
The language used to describe linear equations varies considerably. When mathematicians say that
Lu = b is a linear equation, L can be an arbitrary linear map between arbitrary vector spaces. We regard b
as known, and we seek all solutions u. Here u is the variable, or the unknown. It could be a column vector
of n unknown numbers, or an unknown function, regarded as a vector in an infinite-dimensional space of
functions. When u is a column vector of n unknown numbers and b consists of m known numbers, we
can regard the linear equation Lu = b as m linear equations in n unknowns. The term system of linear
equations is often used in this case. The author prefers thinking of a system as a single equation for an
unknown vector.
Row operations enable us to replace a linear equation with another linear equation whose solutions are
the same, but which is easier to understand.
Definition 3.1. Let L : V → W1 and M : V → W2 be linear transformations. The linear equations
Lu = b and M v = c are equivalent if they have the same solution sets.
In Definition 3.1, the linear maps L and M must have the same domain, but not necessarily the same
target space. The reason is a bit subtle; the definition focuses on the solution set. Exercise 3.3 shows for
example that a system of two linear equations in three unknowns can be equivalent to a system of three linear
 3. MATRICES AND ROW OPERATIONS 11

equations in three unknowns. In such a situation one of the three equations is redundant. These ideas will
be clarified when we formalize topics such as linear dependence and dimension.
Row operations on a matrix enable us to replace a given linear equation with an equivalent equation,
which we hope is easier to understand. When L is invertible, the equation Lu = b is equivalent to the equa-
tion u = L−1 b, which exhibits the unique solution. We repeat that a row operation amounts to multiplying
on the left by an invertible matrix, and doing so does not change the set of solutions to the equation.
When solving a system, one performs enough row operations to be able to read off the solutions. One
need not reach either of the forms in the following definition, but the terminology is nonetheless useful.
Definition 3.2. A matrix is row-reduced if the first entry (from the left) in each non-zero row equals 1,
and all the other entries in the column corresponding to such an entry equal 0. For each non-zero row, the 1
in the first entry is called a leading 1. A matrix is in row echelon form if it is row-reduced and, in addition,
the following hold:
• Each zero row is below all of the non-zero rows.
• The non-zero rows are ordered as follows: if the leading 1 in row j occurs in column cj , then
c1 < c2 < · · · < ck .
E XAMPLE 3.1. The following matrices are in row echelon form:
 
0 1 2 0 5
0 0 0 1 4
0 0 0 0 0
 
1 1 2 0 0
0 0 0 1 0
0 0 0 0 1
The following matrix is not in row-echelon form:
 
0 1 2 0 5
.
0 1 0 1 4
The matrix (1) from the Kakuro problem is not row-reduced; the matrix (2) is row-reduced but not in row
echelon form.
It is important to understand that AB need not equal BA for matrices A and B. When AB = BA we
say that A and B commute. The difference AB − BA is called the commutator of A and B; it plays a
significant role in quantum mechanics. See Chapter 7. An amusing way to regard the failure of commutivity
is to think of B as putting on your socks and A as putting on your shoes. Then the order of operations
matters. Furthermore, we also see (when A and B are invertible) that (AB)−1 = B −1 A−1 . Put on your
socks, then your shoes. To undo, first take off your shoes, then your socks.
We have observed that matrix multiplication is associative. The identity
(AB)C = A(BC)
holds whenever the indicated matrix products are defined; it follows easily from the definition of matrix mul-
tiplication. If we regard matrices as linear maps, then associativity holds because composition of functions
is always associative.
If some sequence of elementary row operations takes A to B, then we say that A and B are row-
equivalent. Since each elementary row operation has an inverse, it follows that row-equivalence is an
equivalence relation. We discuss equivalence relations in detail in the next section.
E XERCISE 3.1. Prove that matrix multiplication is associative.
12 1. LINEAR ALGEBRA

E XERCISE 3.2. Write the following matrix as a product of elementary row matrices and find its inverse:
 
1 2 3
2 4 7
0 1 0
E XERCISE 3.3. Determine whether the linear equations are equivalent:
     
1 2 3 x 6
1 2 4 y  = 7
0 0 1 z 1
 
  x  
1 2 3   6
y = .
0 0 1 1
z
E XERCISE 3.4. In each case, factor A into a product of elementary matrices. Check your answers.
 
6 1
A=
−3 −1
 
0 1 0 0
0 0 1 0
A= 0 0 0 1


1 0 0 0
E XERCISE 3.5. Find two-by-two matrices A, B such that AB = 0 but BA 6= 0.
E XERCISE 3.6. When two impedances Z1 and Z2 are connected in series, the resulting impedance is
Z = Z1 + Z2 . When they are connected in parallel, the resulting impedance is
1 Z1 Z2
Z= 1 1 = Z +Z .
Z1 + Z2 1 2

A traveler averages v1 miles per hour going one direction, and returns (exactly the same distance) averaging
v2 miles per hour. Find the average rate for the complete trip. Why is the answer analogous to the answer
for parallel impedances?

4. Equivalence relations
In order to fully understand linear systems, we need a way to say that two linear systems, perhaps
expressed very differently, provide the same information. Mathematicians formalize this idea using equiv-
alence relations. This concept appears in both pure and applied mathematics; we discuss it in detail and
provide a diverse collection of examples.
Sometimes we are given a set, and we wish to regard different members of the set as the same. Perhaps
the most familiar example is parity. Often we care only whether a whole number is even or odd. More
generally, consider the integers Z and a modulus p larger than 1. Given integers m, n we regard them as the
same if m − n is divisible by p. For example, p could be 12 and we are thinking of clock arithmetic. To
make this sort of situation precise, mathematicians introduce equivalence relations and equivalence classes.
First we need to define relation. Let A and B be arbitrary sets. Their Cartesian product A × B is
the set of ordered pairs (a, b) where a ∈ A and b ∈ B. A relation R is an arbitrary subset of A × B. A
function f : A → B can be regarded as a special kind of relation; it is the subset consisting of pairs of the
form (a, f (a)). Another example of a relation is inequality. Suppose A = B = R; we say that (a, b) ∈ R
 4. EQUIVALENCE RELATIONS 13

if, for example, a < b. This relation is not a function, because for each a there are many b related to a. A
function is a relation where, for each a, there is a unique b related to a.
We use the term relation on S for a subset of S × S. We often write x ∼ y instead of (x, y) ∈ R.
Definition 4.1. An equivalence relation on a set S is a relation ∼ such that
• x ∼ x for all x ∈ S. (reflexive property)
• x ∼ y implies y ∼ x. (symmetric property)
• x ∼ y and y ∼ z implies x ∼ z. (transitive property)
An equivalence relation partitions a set into equivalence classes. The equivalence class containing x is
the collection of all objects equivalent to x. Thus an equivalence class is a set.
Equality of numbers (or other objects) provides the simplest example of an equivalence relation, but it
is too simple to indicate why the concept is useful. We give several additional examples next, emphasizing
those from linear algebra. The exercises provide additional insight.
E XAMPLE 4.1. Congruence provides a good example of an equivalence relation. Let Z be the set of
integers, and let p be a modulus. We write m ∼ n if m − n is divisible by p. For example, if p = 2, then
there are two equivalence classes, the odd numbers and the even numbers. For example, if p = 3, then there
are three equivalence classes: the set of whole numbers divisible by three, the set of whole numbers one
more than a multiple of three, and the set of whole numbers one less than a multiple of three.
E XAMPLE 4.2. Let A and B be matrices of the same size. We say A and B are row-equivalent if there
is a sequence of row operations taking A to B.
E XAMPLE 4.3. Square matrices A and B are called similar if there is an invertible matrix P such that
A = P BP −1 . Similarity is an equivalence relation.
E XAMPLE 4.4. In Chapter 6 we say that two functions are equivalent if they agree except on a small
set (a set of measure zero). An element of a Hilbert space will be an equivalence class of functions, rather
than a function itself.
In this book we presume the real numbers are known. One can however regard the natural numbers N
as the starting point. Then one constructs the integers Z from N, the rational numbers Q from Z, and finally
the real number system R from Q. Each of these constructions involves equivalence classes. We illustrate
for the rational number system.
E XAMPLE 4.5 (Fractions). To construct Q from Z, we consider ordered pairs of integers (a, b) with
b 6= 0. The equivalence relation here is that (a, b) ∼ (c, d) if ad = bc. Then (1, 2) ∼ (−1, −2) ∼ (2, 4)
and so on. Let ab denote the equivalence class of all pairs equivalent to (a, b). Thus a rational number is an
equivalence class of pairs of integers! We define addition and multiplication of these equivalence classes.
For example, ab + mn is the class containing (an + bm, bn). See Exercise 4.5.

An equivalence class is a set, but thinking that way is sometimes too abstract. We therefore often wish
to choose a particular element, or representative, of a given equivalence class. For example, if we think of a
rational number as an equivalence class (and hence infinitely many pairs of integers), then we might select
as a representative the fraction that is in lowest terms.
In linear algebra, row equivalence provides a compelling example of an equivalence relation. Exercise
4.1 asks for the easy proof. Since an invertible matrix is row equivalent to the identity, we could represent
the equivalence class of all invertible matrices by the identity matrix. Later in this chapter we show that a
matrix is diagonalizable if and only if it is similar to a diagonal matrix. In this case, the diagonal matrix is
often the most useful representative of the equivalence class.
14 1. LINEAR ALGEBRA

Most linear algebra books state the following result, although the language used may differ. The row-
equivalence class containing the identity matrix consists of the invertible matrices.
Theorem 4.2. Let A be a square matrix of real (or complex numbers). Then the following statements
all hold or all fail simultaneously.
• A is invertible.
• A is row equivalent to the identity matrix.
• A is a product of elementary matrices.
We illustrate this theorem by factoring and finding the inverse of the following matrix:
 
2 1
A= .
1 1
We perform row operations on A until we reach the identity:
         
2 1 1 1 1 1 1 0 1 0
→ → → →
1 1 2 1 0 −1 0 −1 0 1
At each step we multiply on the left by the corresponding elementary matrix and express the process by a
product of matrices:
       
1 0 1 1 1 0 0 1 2 1 1 0
= . (5)
0 −1 0 1 −2 1 1 0 1 1 0 1
We think of (5) as E4 E3 E2 E1 A = I, where the Ej are elementary row matrices. Thus A−1 = E4 E3 E2 E1 .
Hence  
1 −1
A−1 = .
−1 2
Taking the inverse matrices and writing the product in the reverse order yields A = E1−1 E2−1 E3−1 E4−1 as
the factorization of A into elementary matrices:
       
2 1 0 1 1 0 1 −1 1 0 1 0
= .
1 1 1 0 2 1 0 1 0 −1 0 1
When finding inverses alone, one can write the same steps in a more efficient fashion. Augment A by
including the identity matrix to the right, and apply the row operations to this augmented matrix. When the
left half becomes the identity matrix, the right half is A−1 . In this notation we get
     
2 1 1 0 1 1 0 1 1 1 0 1
→ → →
1 1 0 1 2 1 1 0 0 −1 1 −2
   
1 0 1 −1 1 0 1 −1
→ . (6)
0 −1 1 −2 0 1 −1 2
Thus row operations provide a method both for finding A−1 and for factoring A into elementary matri-
ces. This reasoning provides the proof of Theorem 4.2, which we leave to the reader.
We conclude this section with an amusing game. The solution presented here illustrates the power of
geometric thinking; orthogonality and linear independence provide the key idea.
E XAMPLE 4.6. There are two players A and B. Player A names the date January 1. Player B must
then name a date, later in the year, changing only one of the month or the date. Then player A continues in
the same way. The winner is the player who names December 31. If player B says January 20, then player
B can force a win. If player B says any other date, then player A can force a win.
 4. EQUIVALENCE RELATIONS 15

We work in two-dimensional space, writing (x, y) for the month and the day. The winning date is
(12, 31). Consider the line given by y = x + 19. Suppose a player names a point (x, y) on this line.
According to the rules, the other player can name either (x + a, y) or (x, y + b), for a and b positive. Neither
of these points lies on the line. The player who named (x, y) can then always return to the line. Thus a
player who names a point on this line can always win. Hence the only way for player B to guarantee a win
on the first turn is to name January 20, the point (1, 20).

(12, 31)

(1, 20)

(1,1)

F IGURE 5. Solution of the game

E XERCISE 4.1. Prove that row equivalence is an equivalence relation.

E XERCISE 4.2. Prove Theorem 4.2.
E XERCISE 4.3. Describe all row equivalence classes of two-by-two matrices.
E XERCISE 4.4. Verify that the three properties of an equivalence relation hold in Example 4.5. Why is
this definition used? What is the definition of multiplication of rational numbers?
E XERCISE 4.5. Let S be the set of students at a college. For a, b ∈ S write a ∼ b if a and b have taken
a class together. Is this relation an equivalence relation?
E XERCISE 4.6. (Dollars and sense). For real numbers, define (x, y) ∼ (a, b) if x + 100y = a + 100b.
Is this relation an equivalence relation? Give a simple interpretation!
 
1 0
E XERCISE 4.7. Determine all two-by-two matrices that are row equivalent to .
0 0
E XERCISE 4.8. There is a pile of 1000 pennies for a two-person game. The two players alternate turns.
On your turn, you may remove 1, 2, or 3 pennies. The person who removes the last penny wins the game.
Which player wins? What if the original pile had 1001 pennies?
E XERCISE 4.9. Which of the following subsets of R2 , thought of as relations on R, define functions?
• The set of (x, y) with x + y = 1.
• The set of (x, y) with x2 + y 2 = 1.
• The set of (x, y) with x3 + y 3 = 1.
16 1. LINEAR ALGEBRA

5. Vector spaces
n
We recall that R consists of n-tuples of real numbers. Thus we put

Rn = {(x1 , x2 , · · · , xn ) : xi ∈ R}
Cn = {(z1 , z2 , · · · , zn ) : zi ∈ C}
In both cases, we define addition and scalar multiplication componentwise:
(u1 , u2 , · · · , un ) + (v1 , v2 , · · · , vn ) = (u1 + v1 , u2 + v2 , · · · , un + vn ) (7.1)

c(u1 , u2 , · · · , un ) = (cu1 , cu2 , · · · , cun ) (7.2)

n n
In both cases, the axioms for a vector space hold. Thus R and C will be two of our most important
examples of vector spaces. It is standard notation within mathematics to regard the n-tuple (x1 , . . . , xn ) as
the same object as the column vector
 
x1
 x2 
 ..  .
 
 . 
xn
The (column) vector (x1 , x2 , . . . , xn ) and the row vector (x1 x2 . . . xn ) are not considered the same object.
We return to this point in Chapter 3 when we introduce dual spaces.
For Rn , the scalars are real numbers. For Cn , the scalars will be complex numbers. The collection of
scalars forms what is known in mathematics as the ground field or scalar field.
We give the formal definition of vector space. First we need to define field. The word field has two
distinct uses in mathematics. Its use in terms such as vector field is completely different from its use in the
following definition. In nearly all our examples, the ground field will be R or C, although other fields also
arise in various applications.
Definition 5.1. A field F is a mathematical system consisting of a set of objects (often called scalars)
together with two operations, addition and multiplication, satisfying the following axioms. As usual, we
write xy for multiplication, instead of a notation such as x ∗ y.
• There are distinct elements of F written 0 and 1.
• For all x, y we have x + y = y + x.
• For all x, y, z we have (x + y) + z = x + (y + z)
• For all x we have x + 0 = x
• For all x there is a −x such that x + (−x) = 0.
• For all x, y we have xy = yx.
• For all x, y, z we have (xy)z = x(yz).
• For all x we have x1 = x.
• For all x 6= 0 there is an x−1 such that xx−1 = 1.
• For all x, y, z we have x(y + z) = xy + xz.
Although these axioms might seem tedious, they are easy to remember. A field is a mathematical system
in which one can add, subtract, multiply, and divide (except that one cannot divide by zero), and the usual
laws of arithmetic hold. The basic examples for us are the real numbers R and the complex numbers C. The
rational numbers Q also form a field, as does the set {0, 1, . . . , p − 1} under modular arithmetic, when p is
a prime number. When p is not prime, factors of p (other than 1) do not have a multiplicative inverse.
 5. VECTOR SPACES 17

R EMARK . Two elementary facts about fields often arise. The first fact states that x0 = 0 for all x. This
result follows by writing
x0 = x(0 + 0) = x0 + x0.
Adding the additive inverse −(x0) to both sides yields 0 = x0. The second fact is that xy = 0 implies that
at least one of x and y is itself 0. The proof is simple. If x = 0, the conclusion holds. If not, multiplying
both sides of 0 = xy by x−1 and then using the first fact yield 0 = y.
R EMARK . Both the rational numbers and the real numbers form ordered fields. In an ordered field, it
makes sense to say x > y and one can work with inequalities as usual. The complex numbers are a field but
not an ordered field. It does not make sense to write z > w if z and w are non-real complex numbers.
Definition 5.2. Let F be a field. A vector space V over F is a set of vectors together with two
operations, addition and scalar multiplication, such that
(1) There is an object 0 such that, for all v ∈ V , we have v + 0 = v.
(2) For all u, v ∈ V we have u + v = v + u.
(3) For all u, v, w ∈ V we have u + (v + w) = (u + v) + w.
(4) For all v ∈ V there is a −v ∈ V such that v + (−v) = 0.
(5) For all v ∈ V , we have 1v = v.
(6) For all c ∈ F and all v, w ∈ V we have c(v + w) = cv + cw.
(7) For all c1 , c2 ∈ F and all v ∈ V , we have (c1 + c2 )v = c1 v + c2 v.
(8) For all c1 , c2 ∈ F and for all v ∈ V , we have c1 (c2 v) = (c1 c2 )v.
The list of axioms is again boring but easy to remember. If we regard vectors as arrows in the plane, or
as forces, then these axioms state obvious compatibility relationships. The reader should check that, in the
plane, the definitions
(x, y) + (u, v) = (x + u, y + v)
c(x, y) = (cx, cy)
correspond to the usual geometric meanings of addition and scalar multiplication.
R EMARK . The axioms in Definition 5.2 imply additional basic facts, most of which we use automati-
cally. For example, for all v ∈ V , we have 0v = 0. To illustrate proof techniques, we pause to verify this
conclusion. Some readers might find the proof pedantic and dull; others might enjoy the tight logic used.
Since 0 + 0 = 0 in a field, we have 0v = (0 + 0)v. By axiom (7), we get 0v = (0 + 0)v = 0v + 0v. By
axiom (4), there is a vector −(0v) with 0 = −(0v) + 0v. We add this vector to both sides and use axioms
(3) and (1):
0 = −(0v) + 0v = −(0v) + (0v + 0v) = (−(0v) + 0v)) + 0v = 0 + 0v = 0v.
We have replaced the standard but vague phrase “a vector is an object with both magnitude and direc-
tion” with “ a vector is an element of a vector space.” Later on we will consider situations in which we can
measure the length (or norm, or magnitude) of a vector.
The diversity of examples below illustrates the power of abstraction. The ideas of linearity apply in all
these cases.
E XAMPLE 5.1 (vector spaces). The following objects are vector spaces:
(1) Rn with addition and scalar multiplication defined as in (7.1) and (7.2).
(2) Cn with addition and scalar multiplication defined as in (7.1) and (7.2).
18 1. LINEAR ALGEBRA

(3) Let S be any set and let VS = {functions f : S → R}. Then VS is a vector space. The zero vector
is the function that is identically 0 on S. Here we define addition and scalar multiplication by
(f + g)(s) = f (s) + g(s) and (cf )(s) = cf (s). (8)
n
(4) The set of real- or complex-valued continuous functions on a subset S of R . Addition and scalar
multiplication are defined as in (8).
(5) The set of differentiable functions on an open subset S of Rn .
(6) The solutions to a linear homogeneous differential equation.
(7) The collection of states in quantum mechanics.
(8) For any vector space V , the collection of linear maps from V to the scalar field is a vector space,
called the dual space of V .
We mention another example with a rather different feel. Let F be the field consisting only of the two
elements 0 and 1 with 1 + 1 = 0. Exercises 5.6-5.10 give interesting uses of this field. Consider n-tuples
consisting of zeroes and ones. We add componentwise and put 1 + 1 = 0. This vector space Fn and its
analogues for prime numbers larger than 2 arise in computer science and cryptography.
E XAMPLE 5.2. Consider the lights-out puzzle; it consists of a five-by-five array of lights. Each light
can be on or off. Pressing a light toggles the light itself and its neighbors, where its neighbors are those
locations immediately above, to the right, below, and to the left. For example, pressing the entry P11 toggles
the entries P11 , P12 , P21 . (There is no neighbor above and no neighbor to the left.) Pressing the entry
P33 toggles P33 , P23 , P34 , P43 , P32 . We can regard a configuration of lights as a point in a 25-dimensional
vector space over the field of two elements. Toggling amounts to adding 1, where addition is taken modulo
2. Associated with each point Pij , there is an operation Aij of adding a certain vector of ones and zeroes
to the array of lights. For example, A33 adds one to the entries labeled P23 , P33 , P43 , P32 , and P34 while
leaving the other entries alone.
The following example from calculus shows one way in which linear algebra in infinite dimensions
differs from linear algebra in finite dimensions.
E XAMPLE 5.3. Let V denote the space of polynomials in one real variable. Thus p ∈ V means
R x we can
PN
write p(x) = j=0 aj xj . Define D : V → V by D(p) = p0 and J : V → V by J(p)(x) = 0 p(t)dt.
Both D and J are linear. Moreover, DJ = I since
Z x
d
(DJ)(p)(x) = p(t)dt = p(x).
dx 0
Therefore DJ(p) = p for all p and hence DJ = I. By contrast,
Z x
(JD)(p)(x) = p0 (t)dt = p(x) − p(0).
0
Hence, if p(0) 6= 0, then (JD)p 6= p. Therefore JD 6= I.
In general, having a one-sided inverse does not imply having an inverse. In finite dimensions, however,
a one-sided inverse is also a two-sided inverse.
In the previous section we showed how to use row operations to find the inverse of (an invertible) square
matrix. There is a formula, in terms of the matrix entries, for the inverse. Except for two-by-two matrices,
the formula is too complicated to be of much use in computations.
E XAMPLE 5.4. Suppose that L : R2 → R2 is defined by
      
x 2 1 x 2x + y
L = = .
y 1 1 y x+y
Exploring the Variety of Random
Documents with Different Content
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

textbookfull.com