Free Textbook Template Access

An unmodified version of the free template used to create this textbook can be found at
https://www.typesetters.se/latex-textbook-template/.

Creative Commons Licensing

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike
4.0 International License.

Cover design by J. Matt Randall.

Contents

1 Polynomials ....................................................... 8
1.1 Univariate Polynomials 8
1.2 Univariate Taylor polynomials and Taylor series 10
1.3 Univariate Lagrange polynomials and interpolation 16
1.4 Multivariate Taylor polynomials 21
1.5 Problems 24

2 Multivariable Calculus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.1 Vector algebra 27
2.2 Derivatives and the gradient operator 36
2.3 Applications of the gradient operator 38
2.4 Divergence and Laplacian 41
2.5 Gauss’ theorem and flux 45
2.6 The curl, circulation, and Stokes’ theorem 49
2.7 Problems 51

3 Introduction to Di↵erential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

3.1 Ordinary vs. partial di↵erential equations 55
3.2 Order of a di↵erential equation 56
3.3 Linear vs. nonlinear 57

3
3.4 Homogeneous vs. non-homogeneous 61
3.5 Constant vs. non-constant coefficient 62
3.6 Initial value vs. boundary value problems 62

4 First Order Linear ODEs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

4.1 Case 1: dy/dx = f (x) 67
4.2 Indefinite and definite integrals for ODEs 68
4.3 Integration by substitution 71
4.4 Interval of interest 72
4.5 Case 2: dy/dx + p(x)y = 0 73
4.6 Case 3: dy/dx + p(x)y = f (x) 75
4.7 Linear vs. exponential growth 77
4.8 Applications 78

5 First Order Nonlinear ODEs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

5.1 Equilibrium solutions 90
5.2 Separable ODEs 91
5.3 Exact ODEs (optional) 96
5.4 Behavior near equilibrium points 98
5.5 Application to chemical kinetics 98

6 Constant Coefficient Linear ODEs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

6.1 Boundary conditions 102
6.2 Complex numbers and Euler’s formula 106
6.3 Hyperbolic functions 108
6.4 Linearly independent functions 108
6.5 Homogeneous ODEs 111
6.6 Nonhomogeneous ODEs 117
6.7 Resonance 122
6.8 Application to circuits 122

7 Systems of Ordinary Di↵erential Equations . . . . . . . . . . . . . . . . . . . . . . . . 127

7.1 Solution by elimination 130
7.2 A brief review of matrices and linear algebra 132
7.3 Systems of ODEs as eigenvalue problems 146
8 Non-constant Coefficient ODEs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
8.1 Power series solutions 155
8.2 The Cauchy-Euler equation 162
8.3 The method of Frobenius 166

9 The Legendre and Bessel Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172

9.1 Legendre equation 172
9.2 Bessel equation 176

10 Partial Di↵erential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177

10.1 Preliminary concepts 177
10.2 The heat, Laplace, and Poisson equations 179
10.3 The wave equation 181
10.4 Solution by integration 182
10.5 Separation of variables 184
10.6 Sturm-Liouville Theory 187
10.7 Application to the 1D wave equation 200
10.8 Application to the 2D wave equation 204
10.9 Non-homogeneous PDEs 208
10.10 Non-homogeneous boundary conditions 212

11 Fourier Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215

11.1 Even and odd functions 215
11.2 Periodic and sinusoidal functions 217
11.3 Fourier series 219
11.4 Application to ODEs 223
11.5 Complex form 226
11.6 The wavenumber and frequency domain 229
11.7 Fourier integrals 231

12 Numerical Solution of IVPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237

12.1 Numerical Integration 237
12.2 Forward Euler method 243
12.3 Backward Euler, leap frog, and Crank-Nicolson methods 248
12.4 Runge-Kutta methods 249
12.5 Stability 252
12.6 Systems of IVPs 255

13 Numerical Solution Boundary Value Problems . . . . . . . . . . . . . . . . . . . . . . 258

13.1 Numerical di↵erentiation 258
13.2 Numerical solution of boundary value problems 264

14 Numerical Solution of Partial Di↵erential Equations . . . . . . . . . . . . . . . . 268

14.1 The 1D Heat equation 268
14.2 Poisson equation 281

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
Preface: Motivation and Objectives

At the graduate level, engineering students face several challenges when it comes to math-
ematical proficiency. First, math and physics in graduate courses and research become in-
extricably tied, and require more intuition and creativity than at the undergraduate level.
The problems are more open-ended, and not easily solved using the “follow-the-recipe” ap-
proach that rote-learning tends to promote. Some universities address this issue by o↵ering
courses dedicated to mathematical modeling. A google search for “mathematical modeling”
returns a host of lecture notes and textbooks that we encourage students to consult. The
second challenge is that graduate courses and research require advanced math that draws
liberally from di↵erential, integral, and vector calculus, as well as linear algebra, complex
numbers, and integral transforms such as Fourier and Laplace transforms. The third chal-
lenge is that real-world systems studied at the graduate level often produce mathematical
models that cannot be solved analytically. This requires students to become proficient in
numerical methods and computer programming.
This text has three primary objectives. The first is to provide students an introduction
to the advanced mathematical concepts they are likely to encounter in their courses and
research. We focus on the topics of ordinary di↵erential equations, partial di↵erential equa-
tions, and Fourier transforms because these are used broadly in solid mechanics, material
science, thermal-fluid sciences, dynamics, automation, and control. As we present these
topics, we also make an e↵ort to review basic concepts from calculus and linear algebra.
Our second objective is to simultaneously give students a rudimentary experience in basic
numerical methods and computer coding with MATLAB. We focus on numerical integration
and di↵erentiation and solving ordinary and partial di↵erential equations using finite di↵er-
ence methods. Finally, we make an e↵ort to develop homework assignments, projects, and
exams with open-ended problems requiring mathematical modeling. While an undergrad-
uate math homework or exam typically asks students to solve a given equation, this courses
presents students with physical problems for which they must first build the mathematical
model before they can solve it.

7
1. Polynomials

1.1 Univariate Polynomials 8

1.2 Univariate Taylor polynomials and Taylor series 10

1.3 Univariate Lagrange polynomials and interpola-

tion 16

1.4 Multivariate Taylor polynomials 21

1.5 Problems 24

Polynomials hold a special place in mathematics and physics because they are easy to eval-
uate and produce smooth, well-behaved, functions that are infinitely di↵erentiable. In this
chapter, we introduce two approaches to approximating functions as polynomials: Taylor se-
ries and Lagrange polynomials. Taylor series play such a key role in math and physics, that it
is difficult to exaggerate their importance. Applications include the derivation of conserva-
tion equations (as demonstrated earlier in Section 2.4), the solution of di↵erential equations
(both analytically andp numerically), and extending our concept of numbers to include the
imaginary unit i = 1. Meanwhile, Lagrange polynomials have important applications to
interpolating data, numerical integration, and solving di↵erential equations using finite ele-
ment methods. Before we explore these approaches, we review some fundamental concepts
in Section 1.1.

1.1 Univariate Polynomials

Univariate polynomials are functions that can be written in the form

p(x) = a0 + a1 x + a2 x2 + ... + an xn , (1.1)

where x is a variable and a0 to an are coefficients. Univariate polynomials can be written

more succinctly as
n
X
p(x) = aj x j . (1.2)
j=0

These are called “univariate” polynomials because they depend on only one variable, x. More
generally, “mutlivariate” polynomials are polynomials that depend on multiple variables,
such as
p(x, y) = 1 + x + xy + x2 + y 2 . (1.3)

8
 Chapter 1. Polynomials 9

For now, we focus on the simpler case of univariate polynomials, and we drop the qualifier
“univariate” for brevity. Note that polynomials can only include integer, non-negative, pow- p
ers of x. For example, f (x) = 2 is a polynomial, but f (x) = 1/x is not, and neither is f (x) = x.
The largest exponent in a polynomial is called the degree of the polynomial. For example,
f (x) = 2 is a zero-degree polynomial, and f (x) = 2 + x2 is a second-order polynomial. Note
that a monomial is a polynomial with only one term, such as f (x) = 2x3 , or more generally
f (x) = axn .

The roots of a polynomial are values of x for which

p(x) = 0. (1.4)

A zero-order polynomial has no roots, except for the trivial case where p(x) = 0 for all x. A
first-order polynomial (a line) has only one root. A second-order polynomial can have up to
two distinct roots, which can be real or complex. If the polynomial can be written as

p(x) = (x a)(x b), (1.5)

then it has the two distinct roots x = a and x = b. If it can be written as

p(x) = (x a)2 , (1.6)

then it has only one distinct root. In that case, we say x = a is a repeated root. A third-order
polynomial has up to three roots. The roots are distinct if p(x) can be written as

p(x) = (x a)(x b)(x c). (1.7)

There are only two roots if

p(x) = (x a)(x b)2 , (1.8)

and only one root if

p(x) = (x a)3 . (1.9)

Theorem 1.1 (Roots of polynomials)

An n’th-order polynomial has n roots, some of which may be repeated.

This theorem is nice, because if you know the n roots of an n’th-order polynomial, you
can immediately write the polynomial as

p(x) = (x a)(x b)(x c).... (1.10)

 10 1.2. Univariate Taylor polynomials and Taylor series

1.2 Univariate Taylor polynomials and Taylor series

Throughout this course, we will find that Taylor expansions play a key role in solving dif-
ferential equations, whether you solve such equations analytically with paper and pencil, or
simulate them numerically with computers. In fact, most computer software developed to
simulate fluid mechanics, solid mechanics, and heat transport ultimately depend on Taylor
expansions. They are a beautiful mathematical concept, whose importance is often under-
appreciated.

1.2.1 Motivition and derivation

Suppose you are asked to evaluate the function f (x) = ex . Nowadays, it’s easy to forget that
without the use of a calculator, evaluating relatively simple functions such ex and sin(x)
is actually quite challenging. For the case f (x) = ex , we can start by recalling that e0 = 1.
Figure 1.1(a) compares the actual variation of ex (black line) with the rough approximation
ex ⇡ 1 (blue line). In the region 0.1  x  0.1, this approximation provides errors within
10%. To get a better approximation, we can use the fact that at x = 0, we also know the slope
df /dx,
df
= ex = 1. (1.11)
dx x=0 x=0

From this, we can get the improved approximation

df
f (x) ⇡ 1 + x = 1 + x. (1.12)
dx x=0

This approximation, shown as a red line in figure 1.1(a), produces relative errors smaller
than 10% when 0.39  x  0.5.
Note that our approximations above are zero and first-order polynomials of the form
p(x) = a and p(x) = a + bx, respectively. Perhaps we can improve the approximation using a
second-order polynomial,
p(x) = a + bx + cx2 . (1.13)

We are also motivated by the fact that ex varies smoothly in figure 1.1(a), and it seems intu-
itive that it can approximated, at least locally, as a polynomial. To determine the coefficients
a, b, and c, we leverage the fact that we know f (x), f 0 (x), and f 00 (x) at x = 0,

p(0) = f (0) = a,
p 0 (0) = f 0 (0) = b, (1.14)
p 00 (0) = f 00 (0) = 2c.

From this, we find

x2 00 x2
f (x) ⇡ f (0) + xf 0 (0) + f (0) = 1 + x + . (1.15)
2 2
 Chapter 1. Polynomials 11

(a)

df
x dx
x=0

(b)
7
ex
5 1 + x + x2 /2
1 + x + x2 /2 + x3 /6
3

-1

-3 -2 -1 0 1 2 3
x

Figure 1.1: Approximations of ex near x = 0.

This approximation, shown as a blue line in in figure 1.1(b), produces errors below 10% for
0.7  x  1. Repeating this procedure for a polynomial of order n,

pn (x) = a + bx + cx2 + ex3 + ... + gxn , (1.16)

it is straightforward to show that

x2 00 x3 000 x4 xn
pn (x) = f (0) + xf 0 (0) + f (0) + f (0) + f 000 (0) + ... + f (n) (0), (1.17)
2 3·2 4·3·2 n!

which can be written more succinctly as

n
X xk
pn (x) = f (k) (0), (1.18)
k!
k=0

where f (k) denotes d k f /dxk , and we must define 0! = 1 to avoid division by zero when k = 0.
Figure 1.1(b) shows the third-order approximation p3 (x) = 1 + x + x2 /2 + x3 /6 as a red line. It
 12 1.2. Univariate Taylor polynomials and Taylor series

Figure 1.2: Approximations of ln(x) about x = 1.

produces errors below 10% when 1  x  1.7. In the region 0.6  x  0.8, the error is less
than 1%.
Now suppose we want to evaluate the function f (x) = ln(x), again without a calculator. In
this case, we must modify the procedure above because f (0) is undefined. At x = 1, however,
we know that ln(x) = 0. We also know all the derivatives of ln(x) at x = 1. In that case, let’s
define a new variable ⇠ = x 1, so we can write f (x) = ln(x) as

fb(⇠) = ln(⇠ + 1). (1.19)

For convenience, we brought in the hat symbol fb to help distinguish fb(⇠) from f (x). Now
that fb(0) = f (1), we can use equation (1.18) above to approximate fb(⇠) as

n
X ⇠ k b(k)
pn (⇠) = f (0), (1.20)
k!
k=0

Equation (1.20) can be transformed back to x by substituting ⇠ = x 1,

n
X (x 1)k (k)
pn (x) = f (1). (1.21)
k!
k=0

This produces the approximation

n
X
(x 1)2 (x 1)3 ( 1)(k 1)
pn (x) = (x 1) + ... = (x 1)k . (1.22)
2 3 k
k=0

Figure 1.2 compares ln(x) (solid black line) with p1 (x) = x 1 (dotted blue line) and p500 (x)
(dotted red line). It is interesting to note that p500 (x) gives excellent results for 0 < x < 2,
such that it is indistinguishable from ln(x) in the figure for x < 2. The approximation p500
deviates significantly, however, for x > 2. This behavior only worsens as we increase the
order n of the polynomial approximation. To understand why, consider that each term in
our approximation pn has a numerator that varies like (x 1)k . For 0 < x < 2, this term gets
increasingly small with increasing k. Meanwhile the denominator k becomes increasingly
 Chapter 1. Polynomials 13

large, such that

(x 1)k
lim = 0, 0 < x < 2. (1.23)
k!1 k

We conclude that as we consider polynomial approximations of higher and higher order, i.e.
n ! 1, the corrections get smaller and smaller, such that pn (x) converges to a finite value
for 0 < x < 2. For x > 2, however, the numerator (x 1)k begins to grow with increasing k.
Though the denominator k also grows, the numerator grows faster, such that pn (x) does not
converge to a finite value with increasing n for x > 2. This can be shown more formally by
taking the limit
(x 1)k
lim , x > 2. (1.24)
k!1 k

Using L’Hopital’s rule, this can be written as

k(x 1)(k 1)
lim = 1, x > 2. (1.25)
k!1 1

More generally, we can now approximate a function f (x) about some point x = x0 as
n
X (x x0 )n j
pn (x) = f (x0 ). (1.26)
j!
j=0

Equation (1.26) is called the n’th Taylor polynomial. In the limit n ! 1,

1
X (x x0 )n j
p(x) = f (x0 ). (1.27)
j!
j=0

Equation (1.27) is called the Taylor series. The special case of x0 = 0, is also called the
Maclaurin series,
1 n
X x
p(x) = f j (0), (1.28)
j!
j=0

1.2.2 Convergence

To calculate a Taylor series, the function f (x) must be infinitely di↵erentiable, and be defined
in a region near x = x0 . Our expansion (1.22) of ln(x) about x0 = 1 in the previous sections
then raises two important issues:

1. For what values of x does a Taylor series converge to a finite value?

2. In cases where p(x) indeed converges, does it converge to the original function f (x)?
 14 1.2. Univariate Taylor polynomials and Taylor series

To answer question 1, we first note that Taylor series are also an example of a power series. A
power series is any series of the form

1
X
ak (x x0 )k = a0 + a1 (x x0 ) + a2 (x x0 )2 + .... (1.29)
k=0

We immediately note that a power series always converges to a0 when x = x0 . The question
is then, for what values about x0 does the series converge? This is answered by the following
theorem, presented without proof.

Theorem 1.2 (Power series convergence)

The power series
1
X
ak (x x0 )k (1.30)
k=0

converges at x = x0 . If it converges at other points as well, then those points form

an interval of convergence |x x0 | < R where the radius of convergence (R) can be
determined from either of the two formulas below

1 1
R= or R= p . (1.31)
lim | aak+1 | lim k |ak |
k!1 k
k!1

To answer the second question above, let us define the remainder Rn (x) as the di↵erence
between the function f (x) and its n’th Taylor polynomial,

Rn (x) = f (x) pn (x). (1.32)

Then, the Taylor series converges to f (x) if

lim Rn (x) = 0. (1.33)

n!1

Theorem 1.3 (Taylor’s inequality)

If |f n+1 (x)|  C for all x in some interval |x x0 | < D, then the remainder Rn (x) of the
Taylor polynomial pn (x) for f (x) about the point x0 satisfies the inequality

C
|Rn (x)|  |x x0 |n+1 , (1.34)
(n + 1)!

for all points |x x0 | < D.

 Chapter 1. Polynomials 15

Let’s now use Taylor’s inequality to analyze the Taylor series of f (x) = ex about x0 = 0, i.e.

X xk 1
x2
p(x) = 1 + x + + ... = . (1.35)
2 k!
k=0

We first note that the (n + 1)’th derivative of ex is always ex . Consequently, for any interval
D  x  D,
|f n+1 (x)|  eD . (1.36)

We then apply Taylor’s inequality

eD
|Rn (x)|  |x|n+1 . (1.37)
(n + 1)!

Taking the limit of the above as n ! 1,

|x|n+1
lim |Rn (x)| = eD lim . (1.38)
n!1 n!1 (n + 1)!

In the interval D  x  D, the numerator |x|n+1 is less than or equal to D n+1 . Consequently,

D n+1
lim |Rn x|  eD lim . (1.39)
n!1 n!1 (n + 1)!

It can be shown that (n + 1)! tends to infinity as n ! 1 faster than D n+1 , such that

lim |Rn x| = 0. (1.40)

n!1

Because D was arbitrary in the above analysis, we conclude that

1 k
X x
ex = , for all x. (1.41)
k!
k=0

1.2.3 Analytic functions

Definition 1.1 (Analytic functions)

A real function f (x) is analytic if it agrees with its local Taylor series in a neighborhood
 16 1.3. Univariate Lagrange polynomials and interpolation

of every point x0 . This requires f (x) to be infinitely di↵erentiable and equal to

1
X (x xo )n j
f (x) = f (x0 ), (1.42)
j!
j=0

in some region about every point x = x0 .

The classic definition of “analytic” above requires f (x) to agree with its local Taylor series
for all points. In practice, however, if a function only agrees with its local Taylor series in
some interval a  x  b, we often say f (x) is analytic in the interval a  x  b. Note that we
don’t require f (x) to be expressed in a single Taylor series that converges for x. It is sufficient
to say that for every point x0 , we can find a local Taylor series that converges in at least some
small interval about x0 . For example, in Section 1.2.1, we saw that the Taylor series of ln(x)
about x0 = 1 does not converge for x > 2. Nevertheless, we can show that ln(x) is analytic
for all x > 0. That’s because we can show that Taylor series about all points x0 > 0 converge
for some interval about x0 . We cannot, however, include points x  0 because ln(x) is not
defined there.
Analytic functions are special because they can be expressed as a polynomial in a neigh-
borhood of every point. This means they are smooth and very well behaved. Three famous
examples of analytic functions include ex , sin(x), and cos(x). Moreover, the Maclaurin series
of these functions have an infinite radius of convergence:

1 k
X
x2 x
ex = 1 + x + + ... = , for all x. (1.43)
2! k!
k=0

1
X
x3 x5 x2j+1
sin(x) = x + ... = ( 1)j , for all x. (1.44)
3! 5! (2j + 1)!
k=0

1
X
x2 x4 x2j
cos(x) = 1 + ... = ( 1)j , for all x. (1.45)
2! 4! (2j)!
k=0

1.3 Univariate Lagrange polynomials and interpolation

Suppose that the values of a function f (x) are only known at

n discrete x locations, as sketched in figure 1.3. Let’s denote f(x)
those x-locations as xj , where j = 1, 2, ..., n. Let’s also use the f3 f4
notation f j to denote f j = f (xj ). The data pairs (xj , f j ) could f2 f5
represent, for example, discrete data measured with a tem- f1
perature or pressure sensor. Interpolation is the process of
x
estimating f (x) for values of x that lie between the discrete x1 x2 x3 x4 x5
locations xj . One obvious way to interpolate data is to fit a
Figure 1.3
polynomial. For that purpose, the following theorem is very
 Chapter 1. Polynomials 17

(a) (b) (c)

f(x)

x x x
x1 x2 x1 x2 x3 x1 x2 x3

Figure 1.4: Fitting polynomials to data points.

useful.

Theorem 1.4 (Polynomial uniqueness and existence)

Given n data points (xj , f j ), for which the xj values are unique, there exists a unique
polynomial of order n 1 or less that passes through the n data points.

Consider the simple case sketched in figure 1.4(a), where f (x) is only known at two
points, x1 and x2 . There are an infinite number of polynomials passing through these two
data points, including the line and two parabolas shown as dashed lines. The lowest-order
polynomial passing through the two points, however, is the line (a first-order polynomial).
Furthermore, this line is unique, i.e. it is the only line passing through both data points.
In contrast, there are an infinite number of parabolas passing through the two data points.
Next, consider the case of three data points sketched in figure 1.4(b). In this case, the lowest-
order polynomial passing through the three points is a parabola (a second-order polyno-
mial). It is now also unique. Usually, when given n data points, the minimum-order polyno-
mial passing through those points is order n 1. Occasionally, however, this is not the case,
as demonstrated in figure 1.4(c), where three data points happen to lie on a line. In that case,
the lowest-order polynomial is a line.

Suppose we want to fit the line, p = a + bx, to the two data points in figure 1.4(a). One
way of finding the coefficients a and b is to solve the system of equations

f 1 = a + bx1 , f 2 = a + bx2 . (1.46)

These equations can be recast as the matrix problem

" #" # " #
1 x1 a f
= 1 . (1.47)
1 x2 b f2

Similarly, if we want to find the second-order polynomial p(x) = a + bx + cx2 passing through
 18 1.3. Univariate Lagrange polynomials and interpolation

the three points in figure 1.4(b), we can build the matrix

2 32 3 2 3
661 x1 x12 77 66a77 66f 1 77
66 76 7 6 7
661 x2 x22 777 666b777 = 666f 2 777 . (1.48)
64 75 64 75 64 75
1 x3 x32 c f3

These are examples of what are called Vandermonde matrices.

Let’s return to the example of two data points in

figure 1.4(a). Instead of building the Vandermonde f(x)
matrix, we can also write the polynomial approxima- L1(x) L2(x)
tion as
1
p(x) = L1 (x)f 1 + L2 (x)f 2 , (1.49)

where L1 (x) is a first-order polynomial that is unity at 0 x

x1 x2
x1 and zero at x2 , and L2 (x) is a first-order polynomial
Figure 1.5
that is unity at x2 and zero at x1 . These polynomials
are sketched in figure 1.5. The advantage to this approach is that L1 (x) and L2 (x) are easily
built by inspection. If L1 (x) must be zero at x = x2 , then x2 is a root. Consequently, L1 must
vary like L1 ⇠ (x x2 ). We can then set L1 (x) to unity at x = x1 by dividing x x2 by x1 x2 ,
i.e.
x x2
L1 (x) = . (1.50)
x1 x2

Similarly, we can find by inspection that

x x1
L2 (x) = . (1.51)
x2 x1

Finally, we can write the polynomial approximation as

x x2 x x1
p(x) = f1 + f . (1.52)
x1 x2 x2 x1 2

If we take this result and rewrite it in the form p = a +bx, it must give us the same result pro-
duced by the matrix approach in equation (1.47), because the first-order polynomial passing
through the two data points is unique.

In the above example, the polynomials L1 and L2 are examples of Lagrange polynomials.
Let’s take a moment to summarize and generalize everything we have explored with respect
to Lagrange interpolation, and then examine another example.
 Chapter 1. Polynomials 19

Lagrange Interpolation: Suppose you are given n data points (xj , f j ), for which the x
values are unique. The polynomial of order n 1 passing through these points can be
written as
Xn
p(x) = Lj (x)f j , (1.53)
j=1

where Lj is the Lagrange polynomial

n
Y x xi (x x1 )(x x2 )...(x xj 1 )(x xj+1 )...(x xn )
Lj (x) = = . (1.54)
xj xi (xj x1 )(xj x2 )...(xj xj 1 )(xj xj+1 )...(xj xn )
i=0, i,j

The Lagrange polynomial Lj (x) is equal to unity at x = xj , and is zero at all other xi .

Let’s demonstrate Lagrange interpolation for

the case of three data points, as sketched in fig- 1.5
L
1
ure 1.4(b). First, we write the second-order poly- L2
L3
nomial as 1

L(x)
0.5
p(x) = L1 (x)f 1 + L2 (x)f 2 + L3 (x)f 3 .
0

The polynomial L1 (x) is equal to unity at x = x1 and

zero at x = x2 and x3 . This means L1 must vary like -0.5
0 1 2 3 4
(x x2 )(x x3 ). To make L1 equal unity at x1 , we x

define Figure 1.6

(x x2 )(x x3 )
L1 (x) = .
(x1 x2 )(x1 x3 )

Similarly, we can find by inspection that

(x x1 )(x x3 ) (x x1 )(x x2 )
L2 (x) = , L3 (x) = .
(x2 x1 )(x2 x3 ) (x3 x1 )(x3 x2 )

Figure 1.6 shows L1 , L2 , and L3 for the case of (x1 , x2 , x3 ) = (1, 2, 3).

Now let’s take a moment to compare Lagrange interpolation to Taylor series:

1. Both methods approximate a function f (x) as a polynomial.

2. Taylor series require we know f (x) and all it’s derivatives at one point. In contrast,
Lagrange interpolation only needs data for f (x) at discrete points.

3. In the limit n ! 1, Taylor series converge to analytic functions exactly. In contrast,

Lagrange polynomials often become ill-behaved as the number of points n tends to
large values.
 20 1.3. Univariate Lagrange polynomials and interpolation

(a) (b) (c)

Figure 1.7: The Runge phenomenon. The black lines show f (x) = 1/(1 + x2 ). The blue lines show the
Lagrange interpolation using (a) n = 10, (b) n = 14, and (c) n = 18 equispaced points.

(a) (b) (c)

Figure 1.8: The Runge phenomenon. The black lines show f (x) = cos(x). The blue lines show the
Lagrange interpolation using (a) n = 8, (b) n = 10, and (c) n = 40 equispaced points.

To demonstrate the poor behavior of Lagrange interpolation as n increases, suppose we

approximate f (x) = 1/(1 + x2 ) using Lagrange interpolation with n equispaced points be-
tween 2⇡  x  2⇡. Figure 1.7 shows the results using 10 points (panel a), 14 points (panel
b), and 18 points (panel c). We see that increasing the number of points causes the growth
of increasingly sharp oscillations near x = ±2⇡. Increasing n further only worsens the oscil-
lations. This behavior is called the Runge phenomenon, and it tends to occur when the data
points are equispaced in the x-axis. By carefully choosing non-equispaced locations for xj ,
the Runge phenomenon can be avoided; however, this is a more advanced topic not covered
here. Unfortunately, because data is often equispaced, the Runge phenomenon places a limit
on the usefulness of Lagrange interpolation. In practice, it is often limited to third or fourth
order. To interpolate f (x) = 1/(1+x2 ) in figure 1.7, for example, we can fit a parabola to three
adjacent data points, instead of trying to fit one high-order polynomial to all the data.
Depending on the data, Lagrange interpolation can work well for much larger values of
n. Figure 1.8, for example, shows that a Lagrange interpolation of cos(x) using 40 equispaced
points produces excellent agreement with the exact function. Though not shown, we only
begin to observe the Runge phenomenon around n = 60 for this particular case. Finally, we
note that if the function f (x) is itself a polynomial of order n 1, then the Lagrange interpo-
 Chapter 1. Polynomials 21

(a) (b) (c)

500 500 500

450 450 450

400 400 400

350 350 350

f (x) f (x)
f (x)
300 300 300

250 250 250

200 200 200

150 150 150

100
100 100
-2 0 2 4 6 8 10 12 -2 0 2 4 6 8 10 12 -2 0 2 4 6 8 10 12
x x x

Figure 1.9: Lagrange interpolation (blue lines) of the function f (x) = 100 + x + 20x2 + 3x3 + 0.01x4 +
0.01x5 (black lines) using (a) n = 4, (b) n = 5, and (c) n = 6 points. The interpolation using n = 6 is
exact, because f (x) is a fifth-order polynomial.

lation using n points will be exact. This is demonstrated in figure 1.9 showing the Lagrange
interpolation of a fifth-order polynomial using n = 4, 5, and 6 points. The interpolation
using 6 points recovers the exact polynomial f (x).

1.4 Multivariate Taylor polynomials

To conclude our discussion of polynomials, it’s worth briefly discussing multivariate Taylor
polynomials. A multivariate polynomial is simply a polynomial that depends on two or more
variables. For example, a polynomial that depends on two variables x and y is a function that
can be written in the form

p(x, y) = a0 + a1 x + a2 y + a3 xy + a4 x2 + a5 y 2 +
|{z} | {z } | {z }
order 0 order 1 order 2

a6 x2 y + a7 xy 2 + a8 x3 + a9 y 3 + ... (1.55)
| {z }
order 3

Note that when working with multivariate polyno-

mials, we define the “order” of a term as the sum of
the exponents in the term. Consequently, the term
a3 xy is second order, and the term a6 x2 y is third or-
der. To be more specific, you can also say the term
a6 x2 y is “second order in x and first order in y.”
For brevity, in this section, we focus on polyno-
mials that depend on only two variables, x and y.*
In that case, it’s worth noting that the first-order
polynomial Figure 1.10

p(x, y) = a0 + a1 x + a2 y (1.56)
* These are called “bivariate” polynomials.
 22 1.4. Multivariate Taylor polynomials

(a) (b)

Figure 1.11: Comparison of the function f = ex+y with the Taylor polynomial p1 = 1 + x + y (panel a)
and p2 = 1 + x + y + xy + x2 /2 + y 2 /2 (panel b).

produces a planar surface when p(x, y) is plotted as in figure 1.10.

To motivate the development of multivariable Taylor polynomials, suppose we need to

approximate some function f (x, y) about the point (x, y) = (0, 0). If we know the value of
f (x, y) at (0, 0), then we can approximate f (x, y) as the zero-order Taylor polynomial

f (x, y) ⇡ f (0, 0). (1.57)

Now suppose we seek a better approximation as a first-order Taylor polynomial,

p1 (x, y) = a0 + a1 x + a2 y. (1.58)

To determine the coefficients, we require

@p1 @f @p1 @f
p1 (0, 0) = f (0, 0), = , and = . (1.59)
@x 0,0 @x 0,0 @y 0,0 @y 0,0

This produces
@f @f
p1 (x, y) = f (0, 0) + x +y . (1.60)
@x 0,0 @y 0,0

For demonstration, figure 1.11(a) compares the function f (x, y) = ex+y with its first-order
Taylor polynomial p1 = 1 + x + y. Note that p1 apprximates f (x, y) as a planar surface. Now
suppose we seek a better approximation as a second-order Taylor polynomial,

p2 (x, y) = a0 + a1 x + a2 y + a3 xy + a4 x2 + a5 y 2 . (1.61)

The coefficients a0 , a1 , and a2 are found as before, for the first-order polynomial. The coeffi-
 Chapter 1. Polynomials 23

cients a3 , a4 , and a5 are found by requiring

@2 p 1 @2 f @2 p1 @2 f @2 p1 @2 f
= , = , and = . (1.62)
@x@y 0,0 @x@y 0,0 @x2 0,0 @x2 0,0 @y 2 0,0 @y 2 0,0

This produces

@f @f @2 f x2 @2 f y 2 @2 f
p2 (x, y) = f (0, 0) + x +y + xy + + . (1.63)
@x 0,0 @y 0,0 @x@y 0,0 2 @x2 0,0 2 @y 2 0,0

For demonstration, figure 1.11(b) compares the function f (x, y) = ex+y with its second-order
Taylor polynomial p1 = 1 + x + y + xy + x2 /2 + y 2 /2.
In the above, we derived the second-order bivariate Taylor polynomial about (x, y) =
(0, 0). More generally, it is straightforward to show that about some point (x, y) = (a, b), the
second-order polynomial takes the form

@f @f
p2 (x, y) = f (a, b) + (x a) + (y b) +
@x a,b @y a,b

@2 f (x a)2 @2 f (y b)2 @2 f
(x a)(y b) + + . (1.64)
@x@y a,b 2 @x2 a,b 2 @y 2 a,b

The extension on the procedure to third-order polynomials is left to homework.

 24 1.5. Problems

1.5 Problems
P1.1) Consider the function
x2
f (x) = e . (1.65)

(a) Compute the 8th -order Taylor polynomial of f (x) about x = 0. Make sure to show all your
work.
(b) From your result in part (a), you should be able to see
a pattern, such that you can express the Taylor polynomial
1
f(x)
pn (x)

for any order N as 0.8

0.6
n
X
pn (x) = ... (1.66) 0.4

j=0 0.2

0
-2 -1 0 1 2

Write a matlab function called Pn.m that takes the input N ,

and outputs a single figure that compares f (x) and PN (x) Figure 1.12
over the interval x 2 [ 2, 2]. Figure 1.12 shows an example
for N = 2. In addition, the function should output the relative error at x = ⇡/2, defined as

|f (⇡/2) pn (⇡/2)|
Err(n) = . (1.67)
f (⇡/2)

Show your plots and errors for N = 6 and N = 10.

(c) Write a second matlab function called PnErr.m that calls the function Pn.m from part (b)
above and generates a log-log plot of Err vs. n for n = 0 to n = 100.

P1.2) Consider the following, incomplete, matlab function:

1 function[] = LagPoly(N,PLOTFLAG)
2
3 xj = linspace(≠2,2,N); % N equispaced points in [≠2,2]
4 x = linspace(≠3,3,500); % 500 equispaced points in [≠3,3]
5
6 PM = ones(N,500);
7 for j=1:N
8 ... add code here ...
9 end
10
11 if (PLOTFLAG==1)
12 figure
13 hold on
14 set(gca,'FontName','Times','FontSize',20)
15 for j=1:N
16 plot(x,PM(j,:),'LineWidth',2)
17 end
 Chapter 1. Polynomials 25

18 plot([≠3 3],[1 1],'≠≠k','LineWidth',2)

19 plot([≠3 3],[0 0],'≠≠k','LineWidth',2)
20 xlabel('$x$','interpreter','latex')
21 ylabel('$P$','interpreter','latex','rot',0)
22 title(strcat('N=',num2str(N)))
23 grid on
24 box on
25 axis([≠3 3 ≠2 2])
26 end

Line 3 of LapPoly.m defines a vector xj of N equispaced points in the interval x 2 [ 2, 2].

Line 4 defines a vector x of 500 equispaced points in the interval x 2 [ 3, 3]. The objective
of the function is to compute the N Lagrange polynomials associated with the N points xj ,
and then make a plot of each polynomial over the interval x 2 [ 3, 3] using the vector x.
For example, suppose we set N = 3. Then we could compute and plot the three Lagrange
polynomials as follows
1 P1 = ( x ≠ xj(2) )*( x ≠ xj(3) )/( xj(1) ≠ xj(2) )/( xj(1) ≠ xj(3) );
2 P2 = ( x ≠ xj(1) )*( x ≠ xj(3) )/( xj(2) ≠ xj(1) )/( xj(2) ≠ xj(3) );
3 P3 = ( x ≠ xj(1) )*( x ≠ xj(2) )/( xj(3) ≠ xj(1) )/( xj(3) ≠ xj(2) );
4
5 figure
6 hold on
7 plot(x,P1)
8 plot(x,P2)
9 plot(x,P3)

However, we want the matlab function to do this for an arbitrary number of N points xj . For
that, the function LagPoly.m builds a matrix PM, of size PM(N,500), whose rows are the N
Lagrange polynomials. For example, if N = 3, then
1 PM(1,:) = ( x ≠ xj(2) )*( x ≠ xj(3) )/( xj(1) ≠ xj(2) )/( xj(1) ≠ xj(3) );
2 PM(2,:) = ( x ≠ xj(1) )*( x ≠ xj(3) )/( xj(2) ≠ xj(1) )/( xj(2) ≠ xj(3) );
3 PM(3,:) = ( x ≠ xj(1) )*( x ≠ xj(2) )/( xj(3) ≠ xj(1) )/( xj(3) ≠ xj(2) );

Add the required code in the for-loop of LagPoly to build the N=3
matrix PM for any input N . The provided code then makes
2

1.5

a plot showing the N polynomials vs. x, as in Figure 1.13 1

for N = 3. Note that the code plots two horizontal dashed 0.5

0
lines at P = 1 and P = 0. Show your results for N = 4, 6, and -0.5

8. Comment on the behavior of the Lagrange polynomials -1

-1.5
in the region |x| > 2 versus that in the region 2  x  2. -2
-3 -2 -1 0 1 2 3

Figure 1.13
 26 1.5. Problems

P1.3) Take your completed matlab function LagPoly.m from problem P1.2, and modify it so
that it uses the N Lagrange polynomials in the matrix PM to approximate the function

x2
f (x) = e . (1.68)

The function should generate a plot that compares the N=8

exact f (x), shown as a solid black line, with the La-
1
f(x)
p (x)
grange polynomial approximation p(x), shown as a red N

0.5
dashed line. Limit the axes so the horizontal x axis
(called the abscissa) varies from x 2 [ 3, 3], and the 0
vertical axis (called the ordinate) varies from 1 to 1.
Figure 1.14 demonstrates the result for N = 8. Show -0.5
your results for N = 4, 8, 16, 32, and 128. Comment
on the behavior of the polynomial approximation as N -1
becomes large.
-3 -2 -1 0 1 2 3

Figure 1.14
P1.4) Suppose we want to approximate a function
f (x, y) about the point (x, y) = (0, 0) using a third-order
Taylor polynomial of the form

p(x, y) = a0 + a1 x + a2 y + a3 xy + a4 x2 + a5 y 2 + a6 x2 y + a7 xy 2 + a8 x3 + a9 y 3 .

Solve for the required coefficients a0 to a9 .

2. Multivariable Calculus

2.1 Vector algebra 27

2.2 Derivatives and the gradient operator 36

2.3 Applications of the gradient operator 38

2.4 Divergence and Laplacian 41

2.5 Gauss’ theorem and flux 45

2.6 The curl, circulation, and Stokes’ theorem 49

2.7 Problems 51

Multivariable calculus is the branch of mathematics concerned with the di↵erentiation and
integration of functions of several variables. That includes di↵erentiation and integration
of vector fields, which play a central role in all fields of mechanical engineering, including
solid mechanics, fluid mechanics, heat transfer, robotics, biomechanics, optimization, and
control.

2.1 Vector algebra

Scalars are quantities such as time, mass, and temperature

z
that have magnitude but no direction. Vectors are quanti- g
ties such as force, velocity, and acceleration that have both m v
magnitude and direction. Consider the small mass m with
x
Cartesian coordinates (x, y, z) sketched in figure 2.1. We z
define the position vector x as y
x
x = x i + y j + z k, (2.1) y
x

where i, j, and k are the unit vectors in the x, y, and z Figure 2.1
directions, respectively. These unit vectors are called basis
vectors because any vector in 3D space can be uniquely expressed as a linear combination of
them.* If the mass travels in the positive y-direction, it has the velocity vector

v = v j, (2.2)

* People also often use the symbol r for the position vector. Some sources also use e , e , and e as the basis
x y z
vector.

27
 28 2.1. Vector algebra

where v is the speed. If it is subjected to gravity in the negative z-direction, it feels the force

F = mg k. (2.3)

Rather than write a vector explicitly in terms of i, j, and k, as in equation (2.1), they are often
written as column vectors
2 3
66x 77
6 7
x = 6666y 7777 . (2.4)
4 5
z

When appearing within text, this is sometimes written as x = [x, y, z]T , where the T denotes
the transpose.
It’s worth noting that, perhaps counter-intuitively, pressure is
usually considered a scalar. For example, the small pressure force
df exerted on a di↵erential area element dA in figure 2.2 must be
written as
n
df = p dA n, (2.5) dA

where n is the normal to dA. Pressure is considered a scalar for

several reasons. First, thermodynamic relations such as the ideal Figure 2.2
gas law treat the pressure as a scalar,
p = ⇢RT , (2.6)

where R is the specific gas constant. Defining p as a vector would require one of the terms
on the right-hand-side of equation (2.6) to be a vector as well. Second, in undergraduate
thermodynamics and fluid mechanics courses, we learn than pressure depends on position,
but not direction. The sense of direction only enters when one asks “what is the pressure
force exerted on a surface?” In that case, the pressure force is always normal to the surface,
and compressive. We consequently say that the sense of direction comes from the surface,
not the pressure. People sometimes make this more explicit by introducing the concept of
“vector area,” so they can rewrite the relation for df as

df = p dA, where dA = dA n. (2.7)

2.1.1 Scalar multiplication and vector summation

Consider the two-dimensional vector a sketched in

figure 2.3. Suppose we express a as a column vector, y y

ka
" #
a a
a= 1 , (2.8)
a2
x x

where a1 and a2 are the vector’s components in the x

Figure 2.3
and y-directions, respectively. Sometimes, it’s clearer
 Chapter 2. Multivariable Calculus 29

to express a vector as
" #
a
a= x , (2.9)
ay

because this helps distinguish that ax and ay are the components in the x and y-directions,
respectively. However, when working with advanced vector operations, this notation turns
out to be less convenient. Scalar multiplication of the vector a by the scalar k is defined as
" #
k a1
ka = . (2.10)
k a2

Graphically, this can be interpreted as scal- y y

ing the vector’s magnitude by k, while main- a
b
taining the same direction, as sketched in a
figure 2.3. Of course, if k is negative, then a +b
the vector ka would point in the opposite di- x x
rection from the original vector a. b
Now consider a second vector b, which Figure 2.4
we express as the column vector
" #
b
b= 1 . (2.11)
b2

We define vector addition a + b as

" #
a1 + b1
a+b = . (2.12)
a2 + b2

This can be interpreted graphically as sketched in figure 2.4. The vector subtraction a b can
now be defined in terms of scalar multiplication and vector summation as
" #
a1 b1
a b = a + ( 1)b = . (2.13)
a2 b2

2.1.2 Dot products

Consider two three-dimensional vectors a and b:

2 3 2 3
66a1 77 66b1 77
6 7 6 7 a
a = 6666a2 7777 , b = 6666b2 7777 . (2.14) z
4 5 4 5
a3 b3 θ
b
y
Any pair of non-parallel vectors defines a plane, as sketched
x
by the region shaded grey in figure 2.5. Within the plane, the Figure 2.5
two vectors are separated by the angle ✓. We define the dot
 30 2.1. Vector algebra

product a · b as the operation

a · b = |a| |b| cos ✓, (2.15)

where |a| and |b| are the magnitudes of vectors a and b, respectively. From the definition
(2.15), we see that two vectors a and b are orthogonal when

a · b = 0. (2.16)

We also see that

p
|a| = a · a (2.17)

In terms of the components of a and b, one can show that†

a · b = a1 b1 + a2 b2 + a3 b3 . (2.18)

This can be expressed as the matrix multiplication aT b,

h i2 3
a · b = a1 a2 a3 66b1 77
66 77
66b2 77 ,
64 75 (2.19)
b3

P
or using the summation symbol ,

3
X
a·b = ai bi . (2.20)
i=1

Dot products are particularly useful for per-

forming projections. Suppose you wish to know
how much of a vector a lies in the direction n, a a
where n is a unit vector. From figure 2.6, we see a⟂
the answer is θ
n a ||
|a| cos(✓) = a · n. (2.21) |a|cos(θ)

Figure 2.6

We call this the scalar projection of a onto n. If desired, we can break up the vector a into two
orthogonal components
a = ak + a? , (2.22)

† The proof is relatively straightforward, and can be found in most vector calculus textbooks.
 Chapter 2. Multivariable Calculus 31

where ak and a? are parallel and perpendicular to n, respectively,

ak = (a · n) n, a? = a ak . (2.23)

Lastly, we note that dot products satisfy commutatitivity, associativity, and distributivity,

a·b = a·b commutativity

(↵a) · b = ↵(a · b) associativity (2.24)
(a + b) · c = a · c + b · c distributivity

Associativity and distributivity are sometimes combined into an equiavelent property called
linearity
(↵a + b) · c = ↵a · c + b · c linearity. (2.25)

Example 2.1 (Forces)

The cable AB in figure 2.7 has a tension of 5 kN. Find the

vector F representing the force applied by the cable to the z

structure. Find the component of F in the direction AC.

Note that this problem has been modified from the text- A

book Engineering Mechanics: Statics by Meriam and Kraige,

Wiley Publishers.
10 m
Solution: The position vectors of points A, B and C are
2 3 2 3 2 3 O
66 0 77 66477 66477
66 77 66 77 6 7 x C
xA = 666 4 777 , xB = 6666777 , xC = 666607777 .
2m
6m
4 5 4 5 4 5
10 0 0 B
4m y

From these, we can compute the vectors AB and AC Figure 2.7

2 3 2 3
66 4 77 66 4 77
66 7 66 7
AB = xB xA = 666 2 7777 , AC = xC xA = 666 4 7777 ,
4 5 4 5
10 10
and the corresponding unit vectors
2 3 2 3
66 4 77 66 4 77
AB 1 66 7 AC 1 66 7
=p 66 2 777 , =p 66 4 777 .
|AB| 120 4 1075
6 |AC| 132 4 1075
6

The force F can now be written as

2 3
66 4 77
5 66 7
F= p 66 2 777 kN, (2.26)
120 4 1075
6
 32 2.1. Vector algebra

and its component in direction AC as

2 3 2 3
66 4 77 66 4 77
5 66 7 1 66 7
p 66 2 777 · p 66 4 777 = 4.29kN (2.27)
120 106
4 7
5 132 1075
6
4

2.1.3 Cross products

Consider the three-dimensional vectors a and b
axb
sketched in figure 2.8. We define the cross product
as the operation b b
θ θ
c = a ⇥ b, (2.28) a a

that produces a vector c whose magnitude is equal bxa

to the area within the parallelogram shaded grey in Figure 2.8
figure 2.8
|c| = |a| |b| sin(✓). (2.29)

The vector c is perpendicular to both a and b, and its orientation follows the right-hand-rule.
As a result, the vector b ⇥ a points in the opposite direction to a ⇥ b, i.e.,

b ⇥ a = a ⇥ b. (2.30)

Due to the above, cross products are not commutative. Rather, they are anti-commutative.
They are, however, associative and distributive,

a⇥b = a⇥b anti-commutativity

(↵a) ⇥ b = ↵(a ⇥ b) associativity (2.31)
(a + b) ⇥ c = a ⇥ c + b ⇥ c distributivity

Cross products arise in statics and dynamics, where they

are used to represent moments and rigid-body rotations. The Ω
motion of a rigid body, such as that shaded grey in figure 2.9,
v
can be decomposed into the translation of the body’s center of
mass with velocity v plus a rotation of the body about its center r
of mass with angular velocity ⌦. The total velocity vt of any
point on the body can then be expressed as
Figure 2.9

vt = v + ⌦ ⇥ r, (2.32)

where r is the position of the point relative to the center of mass. Note that the term ⌦ ⇥ r
accounts for the velocity of the point relative to the body’s center of mass. Because the body
is rigid, this motion can only take the form of a rigid body rotation. In fluid mechanics, the
 Chapter 2. Multivariable Calculus 33

cross product plays an important role in the definition of the vorticity field ! = r ⇥ v. In
this case, the cross product occurs between the gradient operator r and the velocity vector
v. This operation, called the curl, is discussed later in Section 2.6.

The formula for cross-products is not as straightforward as that for dot products. One
form expresses the resultant vector as
2 3
66a2 b3 a3 b2 77
6 7
|a| |b| sin(✓) = 6666a3 b1 a1 b3 7777 . (2.33)
4 5
a1 b2 a2 b1

Rather than memorizing that formula, it’s much easier to express the cross product as the
determinant
i j k
|a| |b| sin(✓) = det a1 a2 a3 . (2.34)
b1 b2 b3

Example 2.2 (Moments)

The cable AB in figure 2.7 has a tension of 5 kN. Calculate z

the moment M that the cable exerts on the structure about
the base point O.
A

Solution: In example 2.1, we found the cable exerts the

force
2 3 10 m
66 4 77
5 66 7
F= p 66 2 777 kN (2.35)
120 1075
6 O
4
x C 2m
6m
4m y
on the structure. The moment exerted about O is given by B

the cross product Figure 2.10

M = r ⇥ F, (2.36)
where r is a vector from point O to any point on the line of action of F.a We can
consequently compute M using the product
2 3
i j k 6 6077
5 5 666 7
M = xB ⇥ F = p 4 6 0 =p 66 40 777 . (2.37)
120 4 2 10 120 4 1675
6

p
Note that we used associativity to bring 5/ 120 out of the determinant. Because r is
 34 2.1. Vector algebra

any vector joining O to the line of action of F, we could also compute M as

2 3
i j k 66 6077
5 5 66 7
M = xA ⇥ F = p 0 4 10 = p 66 40 777 . (2.38)
120 4 2 10 120 4 1675
6

a This formula can be found in any undergraduate statics textbook.

2.1.4 Vector fields

Our discussion of vector algebra in section 2.1 focused on distinct vectors. In contrast, a
vector field assigns a vector to every point in space, as in the velocity field v(x, y, z) of a fluid
flow
v(x, y, z, t) = u(x, y, z, t) i + v(x, y, z, t) j + w(x, y, z, t) k. (2.39)

In equation (2.39), the functions u, v, and w are the velocity components in the x, y, and z
directions, respectively. Figure 2.11 shows the following four velocity fields in the x-y plane

(a) x = i + j, (b) x = xi, (c) x = yi, (d) x = xi yi.

(a) (b)
1 1

0.8 0.8

0.6 0.6

0.4 0.4

0.2 0.2

0 0

0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

(c) (d)
1 1

0.8 0.8

0.6 0.6

0.4 0.4

0.2 0.2

0 0

0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

Figure 2.11
 Chapter 2. Multivariable Calculus 35

Vector fields play key roles in solid mechanics, fluid

mechanics, heat transport, and electromagnetism. As an
z Q
example in electrostatics, suppose we have two electric
charges, q and Q. Suppose the position of q is fixed, while r
that of Q can be varied. Coulomb’s law states that the
q
charge Q feels the force
y
1 qQ
F= r,
b (2.40)
4⇡"0 r 2
x
where "0 is called the permitivity of free space, r is Figure 2.12
the magnitude of the vector r pointing from q to Q, as
r is the unit vector r/r. Note that
sketched in 2.12, and b
when the charges have the same sign, the force is repulsive. Equation (2.40) can be rewritten
as
1 q
F = QE, E= br, (2.41)
4⇡"0 r 2

where Q is called the test charge and E is the electric field generated by q.
Figure 2.13(a) shows the electric field generated by a unit positive charge at the origin.
Note how the arrows radiate outwards from the origin. When Q is acted upon by multi-
ple charges q1 , q2 , ..., qN , we can still express the net force as F = QE, where we sum the
elementary fields,
N
1 X qj
E= brj , (2.42)
4⇡"0 rj2
j=1

where rj is the vector from qj to Q. Figure 2.13(b) shows the net field generated by a positive
charge at (x, y, z) = ( 1, 1, 1) and an opposite charge at (x, y, z) = (1, 1, 1). Note how the
arrows emanate from the positive charge towards the negative charge.

(a) (b)

Figure 2.13
 36 2.2. Derivatives and the gradient operator

2.2 Derivatives and the gradient operator

Consider the real function f (x) sketched in figure 2.14(a). We define the derivative df /dx as

df f (x + h) f (x)
= lim , (2.43)
dx h!0 h

where for this discussion, we assume h 0, so that x + h approaches x from the right. The
derivative df /dx provides the slope of the tangent line to f (x). Besides their many obvious
applications, derivatives play an important role in the concept of how “continuous” or “well-
behaved” a function is. In our definition (2.43) above, the derivative is defined as f (x + h)
approaches f (x) from the right. We could just as well have defined the derivative as

df f (x) f (x h)
= lim , (2.44)
dx h!0 h

where we now approach f (x) from the left. For a sufficiently well-behaved function, both
limits should return the same answer.

Consider again the function f (x) sketched in figure 2.14(a). We say that f (x) is continu-
ous at the location x because

lim f (x + h) = lim f (x h), (2.45)

h!0 h!0

i.e. we recover the same value of f (x) if we approach from the right or left. Figure 2.14(b)
shows a discontinuous function for which this is not true. It is possible for a continuous
function f (x) to have a discontinuous derivative. That is the case for the function sketched in
figure 2.14(c), which has a cusp at point x. More generally, we say a function is C n continuous
if the function and all its derivative up to df n /dxn are continuous. For example, the function
in figure 2.14(c) is only C 0 continuous, while that in figure 2.14(a) is at least C 1 continuous.
Functions that are infinitely di↵erentiable are special functions, because they can always be
written in terms of a polynomial using a Taylor series.

(a) (b) (c)

f(x) f(x) f(x)

x-h x x+h x x-h x x+h x x-h x x+h x

Figure 2.14
 Chapter 2. Multivariable Calculus 37

Consider now the function f (x, y) sketched as the grey

f(x,y)
surface in figure 2.15. For this function, we define the
partial derivative @f /@x as

@f f (x + h, y) f (x, y) y
= lim . (2.46)
@x h!0 h
x
Figure 2.15
This is simply the rate of change of f (x, y) as one walks
along a curve of constant y, such as the red curve in fig- y
ure 2.15. We similarly define the partial derivative @f /@y
as s
n sy
@f f (x, y + h) f (x, y)
= lim , (2.47)
@y h!0 h θ
y
sx
which is the rate of change of f (x, y) along a curve of con-
x
x
stant x, such as the blue curve in figure 2.15. Figure 2.16

Suppose, however, that you require the rate of change of f (x, y) as you walk in the direc-
tion n, where n is a unit vector, as sketched in figure 2.16. In that case, we can define the
derivative
df f (x + sx , y + sy ) f (x, y)
= lim . (2.48)
ds s!0 s

Using a Taylor series, we can approximate f (x + sx , y + sy ) as

@f @f
f (x + sx , y + sy ) ⇡ f (x, y) + s + s , (2.49)
@x x @y y

so that
!
df 1 @f @f
⇡ lim s + s . (2.50)
ds s!0 s @x x @y y

In the limit s ! 0, this approximation becomes exact, and we find that

df @f dsx @f dsy @f @f
= + = cos(✓) + sin(✓). (2.51)
ds @x ds @y ds @x @y

Note, however, that the unit vector n can be written as

n = cos(✓)i + sin(✓)j. (2.52)

We can consequently express the directional derivative as

df
= rf · n, (2.53)
ds
 38 2.3. Applications of the gradient operator

where
@f @f
rf = i+ j. (2.54)
@x @y

Though demonstrated here in two dimensions, we can easily repeat the analysis to show we
find the same result in three dimensions. For generality, we define the gradient operator as

@ @ @
r=i +j +k . (2.55)
@x @y @z

2.3 Applications of the gradient operator

Given some function f (x), the gradient vector rf points in the direction for which f in-
creases most rapidly. The magnitude of the gradient vector |rf | is the rate of increase in that
direction. For demonstration, let’s consider the function

f (x, y) = 10 x2 y 2 , (2.56)

for which the gradient is

rf = 2x i 2y j. (2.57)

The function f (x, y) is shown as a surface in figure 2.17(a), while the vector field rf is shown
as blue arrows in figure 2.17(b). Note how the vectors are perpendicular to the contour
lines (shown in black) and point in the direction of increasing f (x, y). The magnitude of the
vectors tend to zero at the origin, where f (x, y) reaches a maximum and rf = 0.

(a) (b)

0.5

-0.5

-1
-1 -0.5 0 0.5 1

Figure 2.17
 Chapter 2. Multivariable Calculus 39

2.3.1 Modelling di↵usive processes

The gradient operator often appears when modeling di↵usive phenomena, such as heat con-
duction. Suppose the temperature of a solid varies with position as

T (x, y) = 10 x2 y 2 , (2.58)

where x and y are given in meters and T is given in Kelvin (K). Spatial variations in temper-
ature cause energy (heat) to spontaneously flow in the direction of maximum temperature
decrease. This is typically modelled using Fourier’s law of heat conduction,

q = krT (2.59)

where q is the heat flux vector (units Watts per m2 ) and k is the thermal conductivity (W per
m·K). The negative sign appears because rT points in the direction of maximum tempera-
ture increase. If the thermal conductivity of the material is unity, k = 1, then the heat flux
vectors would be equal and opposite to the blue vectors in figure 2.17(b), flowing radially
outward from the origin.
As a second example of di↵usive transport, suppose you place a sugar cube in a glass of
water. Fick’s law of mass di↵usion states that dissolved sugar will spontaneously di↵usive
away from the cube as
J = Drc, (2.60)

where J is the di↵usive flux, D is the mass di↵usivity, and c is the sugar concentration field.
A final example of di↵usion-type behavior is Darcy’s law, which models fluid flow through
porous media as
K
u= rp. (2.61)
µ

This law states that fluid in a porous material will flow in the direction of decreasing pressure
p, where u is the fluid velocity, K is the permeability of the porous material, and µ is the
fluid’s viscosity.

2.3.2 Conservative vector fields and scalar potentials

The gradient operator also often appears when modeling force fields. Specifically, conser-
vative force fields for which the work exerted by the field on a particle only depends on the
initial and final positions of the particle, independent of the path taken between the po-
sitions. Examples include gravitational, electrostatic, and magnetic force fields. Consider
the mass sketched in figure 2.18(a). If the mass moves rightward a distance d under the
influence of the force F, the work W exerted by the force is

W = F · d = F cos(✓)d. (2.62)

Now suppose we raise the mass a distance h from point a to point b along a straight line, as
 40 2.3. Applications of the gradient operator

(a) (b) (c)

b b

F
d dθ
θ dz
m m mg
z
θ dl

a a mg

Figure 2.18

sketched in figure 2.18(b), the work done by the gravitational field is

Wab = mgh, (2.63)

where the work is negative because the gravitational force is opposite the direction of dis-
placement. Suppose instead the mass followed a semicircular path of radius h/2, as sketched
in figure 2.18(c). In this case, the work can be expressed as the integral

Zb
Wab = F · dl, (2.64)
a

where dl is a di↵erential vector along the path, as sketched in figure 2.18(c). Using a bit of
geometry, this can expressed as the integral

Z⇡ ⇡
h mgh
Wab = mg sin ✓ d✓ = cos ✓ = mgh. (2.65)
2 2 0
0

Repeating this procedure for any path between a and b, it can be shown that gravitational
force fields are conservative. Moreover, it can be shown that conservative force fields can
always be expressed as the gradient of a scalar field (x),

F=r , (2.66)

where is called the scalar potential. For the previous example, the scalar potential is

= mgz, (2.67)

because
r = mgk, (2.68)
 Chapter 2. Multivariable Calculus 41

where k is the unit vector in the z-direction. The work done by a conservative field acting
on a particle moving between any two positions a and b can be written in terms of the scalar
potential as
Zb Zb
Wab = F · dl = r · dl = (b) (a). (2.69)
a a

2.3.3 Optimization and the gradient descent method

Due to the fact that rf (x) points in the di-

1
rection of increasing f (x), the gradient op-
erator often appears in optimization algo-
rithms. Suppose you wish to find the min- 0.5
imum of a function f (x, y). Given an initial
guess x0 = (x0 , y0 ), you can seek a minimum
0
using the iterative procedure

xn+1 = xn srf (xn ), (2.70) -0.5

where s is some step-size. This method, is -1

-1 -0.5 0 0.5 1
called the gradient descent method. It works
by taking a step in the direction for which Figure 2.19
f (x) decreases most rapidly. Figure 2.19
demonstrates the method when

f (x, y) = x + x2 x3 + y + y 2 . (2.71)

The initial guess (shown as a blue circle) is set to x0 = (0.75, 0.75) and the step size is set to
s = 0.05. The dashed lines show contours of f (x, y), and the red circles show the iterations
converging to the minimum. For complex multidimensional problems, this method may
converge to a local minima rather than the global minima. Therefore, occasionally this pro-
cess is combined with a genetic algorithm to intelligently generate multiple initial guesses.
Overall, nonlinear optimization is a challenging field in which a global minima/maxima
cannot always be proven algorithmically.

2.4 Divergence and Laplacian

The gradient operator can act on vector fields as well as scalar fields. Consider the vector
field f = f x i + f y j + f z k. The divergence of f is defined as

@f x @f y @f z
r·f = + + . (2.72)
@x @y @z

The divergence arises when we express conservation of mass, momentum, and energy math-
ematically. Suppose, for example, that energy moves through a solid due to heat conduction,
 42 2.4. Divergence and Laplacian

(a) (b)

qy (x,y+δy,t)
y
qx (x,y,t) qx (x+δx,y,t)
δy
δx
(x,y)
x
qy (x,y,t)
Figure 2.20

as sketched in figure 2.20(a). At any point in the solid, the local heat transfer can be ex-
pressed as the heat flux vector q(x, y, t),

q = qx (x, y, t) i + qy (x, y, t) j, (2.73)

where q has units of Joules per area per time, i.e. W/m2 . We ignore heat transfer due to
any other phenomena, such as advection and radiation. Now consider the small element of
volume V– = x y z, with the point x in the lower left corner, as sketched in figure 2.20(b).
Conservation of energy requires that the time rate of change of energy in the volume, EV –,
satisfy
dEV
– = Ė
in Ėout , (2.74)
dt

where Ėin and Ėout represent the rate of energy entering and leaving the faces of the element.
For simplicity, we assume no heat is conducted in the z-direction, so that we only need to
consider heat passing through the four faces sketched in figure 2.20. Using the heat flux
vector, we can approximate the rate of energy entering the west and south faces as

Ėin ⇡ qx (x, y, t) y z + qy (x, y, t) x z. (2.75)

This approximation neglects the variations of qx and qy along the west and south faces. In
the limit V
– ! 0, the approximation becomes exact. We can similarly approximate the rate
of energy leaving the east and north faces as

Ėout ⇡ qx (x + x, y, t) y z + qy (x, y + y, t) x z. (2.76)

 Chapter 2. Multivariable Calculus 43

Using a Taylor series about x, we can further approximate the heat flux vectors on the east
and north faces as

@qx
qx (x + x, y, t) ⇡ qx (x, y, t) + x (2.77)
@x
@qy
qy (x, y + y, t) ⇡ qx (x, y, t) + y . (2.78)
@y

Summing our approximations for Ėin and Ėout , we find that

dEV
– = (r · q) V
–. (2.79)
dt

From equation (2.79), we see that a positive divergence r · q produces a decrease in energy
EV– with time. This occurs because the vector field q tends to radiate away from x0 , such
that heat leaves the volume. In contrast, a negative divergence causes EV– to increase in time,
because the vector field q tends to converge towards x0 .
In the special case of steady heat conduction, the heat flux vector must have a zero diver-
gence,
dEV
– = 0, r · q = 0. (2.80)
dt

Vector fields with zero divergence are called solenoidal or divergence free. One famous exam-
ple of a solenoidal fields occurs in fluid mechanics. Using a procedure similar to that above,
one can show that the density (⇢) of a fluid flow satisfies

1 @⇢
= r · u, (2.81)
⇢ @t

where u is the velocity field. Returning to figure 2.20(b), we see that a velocity field radiating
away from a point tends to carry mass away from the point, thereby decreasing the local
density. Conversely, a velocity field radiating towards a point tends to accumulate mass and

(a) (b) (c)

Figure 2.21
 44 2.4. Divergence and Laplacian

increase density at that point. Incompressible flows must consequently satisfy

r · u = 0. (2.82)

We see that the operation r·q is called the divergence because it measures whether the vec-
tor field q tends to locally converge to or diverge away from the point where it is evaluated.
This interpretation, however, can cause confusion. Consider the 2D heat flux q = qx i + qy j,
where
x y
qx = p , qy = p . (2.83)
x2 + y 2 x2 + y 2

This produces the vector field sketched in figure 2.21(a). After some math, the corresponding
divergence is
@qx @qy 1
r·v = + =p . (2.84)
@x @y x + y2
2

This is simply 1/r, where r is the distance from the origin, and is consequently always posi-
tive. You might think this result was expected, because the vectors clearly diverge from the
origin. Now consider the field

x y
qx = , qy = , (2.85)
x2 + y 2 x2 + y 2

which produces the field in figure 2.21(b). Though the vectors radiate outwards from the
origin, a little algebra shows that this field has a zero divergence! Finally, consider the field

qx = 2 x, qy = 0, (2.86)

which produces the field in figure 2.21(c). In this case, the vectors all point rightward, yet
the divergence is everywhere negative

@qx @qy
r·v = + = 1. (2.87)
@x @y

These counter-intuitive results will be explained physically in section 2.5 using Gauss’ theo-
rem.
From the divergence and grad operator, comes another common operator called the
Laplacian, which is a second-order di↵erential operator given by the divergence of the gra-
dient of a field. For a scalar field f , the Laplacian is defined as

@2 f @2 f @2 f
r2 f = r · (rf ) = + + . (2.88)
@x2 @y 2 @z 2

The Laplacian tends to arise when modeling di↵usion processes. To demonstrate how, let’s
 Chapter 2. Multivariable Calculus 45

return to equation (2.79) above,

1 dEV
– = r · q. (2.89)
– dt
V

Now consider that Fourier’s law approximates the conductive heat flux as

q = krT , (2.90)

where k is the thermal conductivity (units W/m/K), and T is the temperature. This leads to
the following energy equation
1 dEV
– = kr2 T . (2.91)
– dt
V

If we further approximate EV
– as
– = ⇢V
EV – cv T , (2.92)

where cv is the specific heat, we arrive at

@T k 2
= r T. (2.93)
@t ⇢cv

This famous di↵erential equation is often called the di↵usion equation.‡

2.5 Gauss’ theorem and flux

The term “flux” is often used in math and physics, but
can have di↵erent meanings. In fluid mechanics and (a) (b)
n q
heat transfer, flux often refers to the rate with which n
some quantity flows per unit area per unit time. We
saw an example of this earlier in section 2.4, where the
heat flux vector q represented joules of thermal energy
flowing peer unit area per unit time, i.e. W/m2 , where dS
W=J/s. However, a second meaning of the term “flux”
Figure 2.22
refers to surface integrals of the form
Z
= f · n dS, (2.94)
S

where f(x, y, z) is some vector field and S is some surface with unit normal n(x, y, z), as
sketched in figure 2.22(a). These integrals represent the net rate with which some quan-
tity leaves a surface. Suppose, for example, that we want to compute the net rate with which
‡ It is also often called the heat equation, but this creates some confusion in the fluid mechanics literature,
where the “heat equation” often includes additional terms due to heat transported by the motion of the fluid.
 46 2.5. Gauss’ theorem and flux

heat leaves the surface S in the figure 2.22(a). For that purpose, we consider the small ele-
ment dS, sketched in figure 2.22(b). The rate with which thermal energy leaves this element
equals
d = q · n dS. (2.95)

The quantity q · n represents the component of the heat flux vector q that is leaving the sur-
face. Multiplying this component by the area dS returns the rate of Joules passing through
the element. To obtain the net rate of Joules leaving the surface, we must integrate over the
full surface S,
Z
= q · ndS. (2.96)
S

When the surface S is a closed surface, i.e. the surface encloses a volume V– , Gauss’ law
states that the surface integral can be converted to a volume integral as follows
Z Z
= q · ndS = (r · q) d V
–. (2.97)
S –
V

Theorem 2.1 (Gauss’ Theorem (The Divergence Theorem))

The surface integral of a vector field f over a closed surface is mathematically equiva-
lent to the volume integral of the divergence of the field over the volume enclosed by
the surface. Mathematically, this is written as
Z Z
= f · ndS = (r · f) d V
–. (2.98)
S –
V

Note: Gauss’ theorem assumes that f exists everywhere in V

– . That requires that f have
no singularities in V
–.

We experienced Gauss’ theorem earlier in sec-

tion 2.4, where we derived the conservation equation
qy (x,y+δy,t)
(2.79)
y
dEV
– = (r · q) V
–. (2.99) qx (x,y,t) qx (x+δx,y,t)
dt δy
δx
If you carefully review this derivation, you will see (x,y)
x
that the right-hand-side was derived by approximat-
ing the surface integral qy (x,y,t)

Z Figure 2.23
q · n dS, (2.100)
S
 Chapter 2. Multivariable Calculus 47

(a) (b) (c)

Figure 2.24

where S surrounds the volume V

– = x y z in figure 2.23. The result
Z
q · n dS ⇡ (r · q) V
–, (2.101)
S

is actually an approximation of the volume integral

Z
(r · q) d V
– ⇡ (r · q) V
–. (2.102)
–
V

Finally, note that if we include Fourier’s law of heat conduction,

q = krT , (2.103)

we find that
Z Z
q · n dS = r2 T d V
–. (2.104)
S –
V

Once again, we see how the Laplacian appears when modelling di↵usive phenomena.
Using Gauss’ theorem, we can now revisit the three vector fields plotted earlier in figure
2.21. The first field,
x y
qx = p , qy = p , (2.105)
x2 + y 2 x2 + y 2

produces the vector field sketched in figure 2.24(a), and has a positive divergence. Suppose
we perform the integral
Z
q · n dS (2.106)
S
 48 2.5. Gauss’ theorem and flux

over a cylinder of radius R and length L, sketched in figure 2.24(a). This integral is zero on
the two ends of the cylinder, because qz = 0. Meanwhile, on the sides of the cylinder, you
can show that in polar coordinates, the vector field 2.105 is equivalent to

q = er , (2.107)

where er is the unit vector pointing in the radial direction. The surface integral consequently
simplifies to the area
Z
q · n dS = 2⇡RL. (2.108)
S

Using Gauss’ theorem, we can also compute the flux exiting the cylinder as
Z Z
1
(r · q) d V
– = –,
dV (2.109)
r
–
V –
V

where we used the fact that r·q = 1/r. Using the volume element d V
– = 2⇡Ldr, this produces
the same answer as above,
ZR
2⇡L dr = 2⇡RL. (2.110)
0

Next, let’s reconsider the field

x y
qx = , qy = , (2.111)
x2 + y 2 x2 + y 2

sketched in figure 2.24(b). This field has zero divergence, yet if we compute the flux sur-
face integral over the same cylinder as in panel (a), we clearly get a nonzero answer. The
disagreement occurs because the vector field q in this case is equivalent to

1
q = er , (2.112)
r

i.e. it points in the radial direction, and it’s magnitude decreases as 1/r. This creates a
singularity at the origin that invalidates Gauss’ theorem. If instead, we integrate over the
volume contained in the annulus sketched in figure 2.24(b), you will indeed find that
Z Z
q · n dS = (r · q) d V
– = 0. (2.113)
S –
V

The surface integral is zero because the rate of heat entering the volume through the inner
surface balances that leaving through the outer surface. More generally, these integrals will
 Chapter 2. Multivariable Calculus 49

be zero for any volume/surface pair that does not include the origin. Here, we picked the
annulus because it made the integral easier.
Finally, consider the field
qx = 2 x, qy = 0, (2.114)

sketched in figure 2.21(c). In this case, it might be counterintuitive that vectors pointing
rightward have negative divergence. However, if we perform the integral flux integral over
the cube skecthed in the figure, we see this occurs becuase more heat enters from the left
than what exits from the right. This causes heat to accumulate in the volume.

2.6 The curl, circulation, and Stokes’ theorem

The gradient operator can also act on a vector field q through the cross product. The result-
ing operation, r ⇥ q, is called the curl,

i j k ! ! !
@ @ @ @qz @qy @qz @qx @qy @qx
r⇥q = @x @y @z = i j +k . (2.115)
@y @z @x @z @x @y
qx qy qz

In comparison to the divergence and

Laplacian, the curl is more challenging to y x y y
visualize or interpret physically. It can be
shown, however, that the curl r ⇥ q is a
x x x
measure of much the vector field q “curls”
(a) (b) (c)
about the point at which which it is evalu-
ated.§ For example, in figure 2.25, the vec- Figure 2.25
tor fields in panels (a) and (b) have zero curl at the point x, while the field in panel (c) has a
nonzero curl.
The curl has many important applications in electromagnetism and fluid mechanics. To
demonstrate its application in fluid mechanics, suppose a fluid flow has a velocity field

v(x, t) = vx (x, t) i + vy (x, t) j + vz (x, t) k. (2.116)

It can be shown that the curl of the velocity field is twice the angular velocity with which
small fluid elements rotate about their centroid, as sketched in figure 2.26,
ω = 2Ω
z
r ⇥ v = 2⌦. (2.117)

y
This angular rotation plays an important role in understanding the in-
fluence of viscous e↵ects in fluid flows, and is particularly important x
Figure 2.26
in the physics of turbulence. The curl of the velocity field is suffi-

§ This interpretation is taken nearly verbatim from the textbook of “Introduction to Electrodynamics” by D.
J. Griffiths.
 50 2.6. The curl, circulation, and Stokes’ theorem

ciently important to be given a dedicated name and symbol, called the vorticity !,

! = r ⇥ v. (2.118)

The curl of a vector field is closely related to a second quan-

tity called the circulation. Staying with our focus on fluid me- C
n
chanics, the circulation of a fluid velocity field v is defined as
z
the following line integral around a closed curve C,
dl
I
= v · dl. (2.119) x u
y
C
Figure 2.27
In the above, dl is a small element of the curve C, and u · dl
represents the velocity tangential to the curve, as sketched in figure 2.27. While the vor-
ticity is a measure of local rotation, the circulation is a macroscopic measure of rotation or
“swirl’ about a closed curve. The vorticity is related to the circulation through the following
theorem.

Theorem 2.2 (Stokes’ Theorem)

The circulation of a velocity field v about a curve C is equal to the flux of r⇥v through
any open surface S for which C is the boundary,
I Z
= v · dl = (r ⇥ v) · n dS. (2.120)
C S

This is usually written more compactly in terms of the vorticity as

I Z
= v · dl = ! · n dS. (2.121)
C S
 Chapter 2. Multivariable Calculus 51

2.7 Problems

P2.1) Suppose the temperature in the x y plane varies like

T = x2 + y 2 + (x 1)(y 2). (2.122)

Three organisms are placed at location (x, y) = (1, 1). One organism prefers hot temperatures,
while one prefers cooler temperatures. The third likes the current temperature, but needs
to move in order to seek food. What directions should each organism travel? Express your
answers as unit direction vectors.

P2.2) The temperature of a metal plate varies in the

x-y plane as (a)

T = cos(x) cos(y) (2.123)

where x 2 [0, 2⇡] and y 2 [0, 2⇡]. The temperature vari-

ation generates heat flow due to conduction.

a) Use matlab’s “surf” function to plot T (x, y). For

demonstration, panel (a) of the image to the right
shows an example in the range x 2 [0, ⇡] and y 2 [0, ⇡].
Make sure to label the axes.
(b)
b) Use matlab’s “contour” function to make a contour
plot of T . Then use the “quiver” function to overlay
arrows showing the heat flux vector field q = KrT ,
where K is the thermal conductivity. For convenience,
set K = 1. Panel (b) to the right shows an example in
the range x 2 [0, ⇡] and y 2 [0, ⇡].

c) Make a surface plot and contour plot of the diver-

gence r · q.

d) If heat is only transported due to conduction, where

in the x-y plane is @T /@t = 0, where t is time. Where is
@T /@t > 0, and where is @T /@t < 0?
 52 2.7. Problems

P2.3) Consider the function

f (x, y) = x2 y + sin(⇡x) sin(⇡y). (2.124)

Suppose we want to find minima of f (x, y) in the region 1  x  1 and 1  y  1. For that,
write a matlab function GradDesc.m

1 function[] = GradDesc(x0,y0,s,N)
2
3 %≠≠≠inputs ≠≠≠%
4 % x0,y0: initial guess
5 % s: step size
6 % N: number of steps

Given an initial guess (x0 , y0 ), the function GradDesc.m should perform N steps of the gradi-
ent descent method discussed in section 2.3.3,

xn+1 = xn srf (xn ), (2.125)

where s is the input step size. The function should generate a plot like that in figure 2.19,
where isocontours of f (x, y) are shown as dashed lines, the initial guess is shown as a blue
circle, and the N steps of the gradient descent algorithm are shown as red dots. Show the
resulting plots for the inputs

(x0 , y0 ) = ( 0.6, 0.6), s = 0.05, N = 15, (2.126)

and the inputs

(x0 , y0 ) = ( 0.6, 0.5), s = 0.05, N = 15. (2.127)

(a) y (b) y
P2.4) Consider the vector field
(1,1)
1 1
u = x i + y j.

x x
Compute the flux integral 1 1

Z
= (u · n) dS,
S

over the volumes (a) and (b) sketched to the right. Both volumes have a unit depth in the
z-direction.
 Chapter 2. Multivariable Calculus 53

P2.5) Compute the heat flux integral

y
Z
= q · n dS,
S 1

where S is the surface of a cylinder of length L = 1 and radius

x
R = 1, centered at (x, y) = (1, 1). The heat flux satisfies Fourier’s law 1

q = krT ,

where k = 10 is the thermal conductivity and

x2 y2
T= + xy.
2 2
3. Introduction to Di↵erential Equations

3.1 Ordinary vs. partial di↵erential equations 55

3.2 Order of a di↵erential equation 56

3.3 Linear vs. nonlinear 57

3.4 Homogeneous vs. non-homogeneous 61

3.5 Constant vs. non-constant coefficient 62

3.6 Initial value vs. boundary value problems 62

Di↵erential equations are equations that contain the deriva-

tive of the dependent variable (the unknown) with respect to
the independent variable. For example, the damped spring
and mass system sketched in figure 3.1 can be modeled by the
di↵erential equation

d 2y dy
m 2
+b + k(y y0 ) = mg, (3.1)
dt dt

where the dependent variable y is the vertical position of the

mass m, and the independent variable is time, t. The constant
Figure 3.1
parameters m, b, k, y0 and g are the mass, damping constant,
spring constant, equilibrium position, and gravitational acceleration. Depending on the
characteristics of a di↵erential equation, there can be one or more methods to seek a solution
analytically or numerically. These characteristics include

1. is the equation an ordinary or partial di↵erential equation,

2. is the equation a first-order or higher-order di↵erential equation,

3. is the equation linear or nonlinear,

4. is the equation a homogeneous or non-homogeneous di↵erential equation

5. is the equation a constant or non-constant coefficient di↵erential equation,

6. is the equation an initial value problem, boundary value problem, or both,

7. what is the nature of the initial conditions and/or boundary conditions that must be
satisfied by the di↵erential equation.

The remainder of this introductory chapter focuses on defining these various characteristics.

54
 Chapter 3. Introduction to Di↵erential Equations 55

3.1 Ordinary vs. partial di↵erential equations

Definition 3.1 (Ordinary vs. partial di↵erential equations)

An ordinary di↵erential equation (ODE) has only one independent variable. The
derivatives in ODE’s are consequently normal derivatives, such as dx/dt. In contrast,
a partial di↵erential equation (PDE) has two or more independent variables, and con-
sequently contains partial derivatives, such as @x/@t.

An ordinary di↵erential equation (ODE) is

a di↵erential equation with only one inde- (a) (b)
pendent variable. Two classic examples are
k
sketched in figure 3.2. Suppose a coin is m
dropped under the influence of gravity, as
in figure 3.2(a). Neglecting e↵ects due to air
resistance, a force balance in the direction x x
of gravity produces the ODE k

dv
= g, (3.2)
dt Figure 3.2

where v is the coin velocity, measured positive downwards in the direction of gravity, and
g = 9.81 m/s2 is the gravitational acceleration. This can also be written in terms of the coin
position x,
d 2x
= g, (3.3)
dt 2

where x is the position measured downwards.

The second example is the mass-spring system sketched in figure 3.2(b). Suppose the
spring and mass are at equilibrium when x = 0. If we displace the mass rightward a distance
x, the spring exerts a leftward force equal to kx, where k is the spring constant. Neglecting
any other forces due to friction, Newton’s laws of motion require

ma = kx. (3.4)

Setting a = d 2 x/dt 2 produces the ODE

d 2x
m + kx = 0. (3.5)
dt 2

Note that when writing ODE’s, we typically isolate all terms involving the dependent
variable on the left-hand-side, as demonstrated in equations (3.3) and (3.5) above. We also
typically order the terms on the left-hand-side starting with the highest-order derivative.
For example, the following ODE

d 2x d 3x
+ + cos(t) 3 sin(2t) = x (3.6)
dt 2 dt 3
 56 3.2. Order of a di↵erential equation

would be reorganized as

d 3x d 2x
+ x = cos(t) + 3 sin(2t). (3.7)
dt 3 dt 2

Partial di↵erential equations (PDE) are di↵erential

h(x,t)
equations with two or more independent variables. Sup-
pose, for example, you pluck the taut string sketched in
figure 3.3. The ensuing vibration of the string satisfies the
x
PDE
@2 h 2
2@ h
= c , (3.8)
@t 2 @x2 Figure 3.3

where h(x, t) is the vertical position of the string and c2 = ⌧/ is the ratio between the string’s
tension ⌧ and mass per length . In contrast to ODEs, we often do not isolate the indepen-
dent variables on the left-hand-side of PDE’s. However, If time is an independent variable,
we often put the derivative of the dependent variable with time as the first term on the
left-hand-side, as demonstrated above in equation (3.8).

3.2 Order of a di↵erential equation

Definition 3.2 (Order of a Di↵erential Equations)

The order of a di↵erential equation (whether an ODE or PDE) is the order of the high-
est derivative in the equation.

Identifying the order of an ODE is relatively straightforward, because there is only one
independent variable. For example, those below are both first-order,

dm dv
= ṁin ṁout , = g, (3.9)
dt dt

while those below are both second-order

d 2x d 2x
= g, m + kx = 0. (3.10)
dt 2 dt 2

Identifying the order of a partial di↵erential equation can be

less obvious than for an ODE. Consider the transport of heat in the z
air surrounding a light bulb. If the air is at rest and the temperature
field has reached a steady state (i.e. does not vary in time), the
temperature field T (x, y, z) in the air surrounding the bulb could be y
modeled using the PDE
x
@2 T @2 T @2 T Figure 3.4
+ + = 0. (3.11)
@x2 @y 2 @z2
 Chapter 3. Introduction to Di↵erential Equations 57

This PDE is clearly second-order. Now suppose you want to model the change in tempera-
ture field with time after the light bulb has just been turned o↵ or on. The PDE must be then
be changed to
!
@T @2 T @2 T @2 T
=↵ + + , (3.12)
@t @x2 @y 2 @z2

where ↵ is called the thermal di↵usivity. Technically, this PDE is still second-order, because
the highest-order derivative is second-order. In practice, however, it is more helpful to iden-
tify an order for each independent variable. With this approach, equation (3.12) is “first
order in time” and “second order in x, y, and z.” We could also simply say “first order in
time and second order in space.”

3.3 Linear vs. nonlinear

A di↵erential equation is linear if the dependant variable and its derivatives only appear in
linear combinations. For example, a linear second-order ODE can always be written as

d 2x dx
a(t) 2
+ b(t) + c(t)x = f (t), (3.13)
dt dt

where x is the dependent variable and t is the independent variable. A di↵erential equation
is nonlinear if the dependant variable or it derivatives appear in nonlinear combinations.
Examples include
2
x ddt x2 = f (t),
d2x (3.14)
dt 2
+ x2 = f (t),
d2x
dt 2
+ cos(x) = f (t),

where the nonlinear terms are shown in red font.

A more formal definition of linear vs. nonlinear di↵erential equations uses the concept
of di↵erential operators. Consider equation (3.5) governing the mass-spring system discussed
earlier in figure 3.2(b), repeated below as

d 2x
m + kx = 0. (3.15)
dt 2

To generalize our discussion a little further, suppose we also apply some force F(t) to the
mass. The governing equation then becomes.

d 2x
m + kx = F(t). (3.16)
dt 2

We can rewrite this equation as

" #
d2
m 2 + k x(t) = F(t), (3.17)
dt
 58 3.3. Linear vs. nonlinear

where we have factored out the operations acting on x(t). We can now write this equation as

L x = F(t), (3.18)

where L is the di↵erential operator,

" #
d2
L = m 2 +k . (3.19)
dt

Similarly, let’s reconsider the wave equation (3.8) discussed earlier in figure 3.3,

@2 h 2
2@ h
= c . (3.20)
@t 2 @x2

We can write this as

" #
@2 @22
c h(x, t) = 0, (3.21)
@t 2 @x2

or simply
L h(x, t) = 0. (3.22)

where
" #
@2 2@2
L= c . (3.23)
@t 2 @x2

The di↵erential operators from the examples above can help determine whether or not the
equations are linear. This depends on the properties of the operator. If the operator is linear,
then it satisfies the conditions defined below.

Definition 3.3 (Linear di↵erential operator)

A di↵erential operator is linear if it satisfies the following two conditions

L (x1 + x2 ) = L x1 + L x2 , (3.24)

and
L (cx) = cL x, (3.25)

where c is some scalar. Alternatively, you can combine these two constraints into the
single constraint
L (ax1 + bx2 ) = aL x1 + bL x2 , (3.26)

where a and b are scalars.

 Chapter 3. Introduction to Di↵erential Equations 59

Consider the operator

" #
d2
L = m 2 +k . (3.27)
dt

This operator is linear because

" # " # " #
d2 d2 d2
m 2 + k (ax1 + bx2 ) = a m 2 + k x1 + b m 2 + k x2 . (3.28)
dt dt dt

Similarly,
" #
@2 2@2
L= c , (3.29)
@t 2 @x2

is also linear because

" #" #
@2 2@2
c ah1 (x, t) + bh2 (x, t)
@t 2 @x2
" 2 2
# " 2 # (3.30)
@ 2 @ @ 2@2
=a 2 c 2
h1 (x, t) + b c h2 (x, t) .
@t @x @t 2 @x2

Definition 3.4 (Linear di↵erential equation) Linear di↵erential equations involve only
linear di↵erential operators.

To demonstrate a nonlinear di↵erential equation, let’s recon-

sider the example of a falling coin, as sketched in figure 3.5. In FD
addition to gravity, suppose the coin experiences a drag force FD h
due to friction e↵ects with the surrounding air. Such drag forces
D
can often be approximated as

⇢CD A 2
FD ⇡ v , (3.31) mg
2

where ⇢ is the air density, v is the coin velocity, CD is a constant

coefficient, and A = Dh is an area. For ease of notation, lets de-
Figure 3.5
fine b = ⇢CD A/2. Summing forces in the vertical direction then
requires
ma = mg bv 2 . (3.32)

Substituting a = d 2 x/dt 2 and v = dx/dt produces the nonlinear ODE

!2
d 2x dx
m 2 +b = mg. (3.33)
dt dt
 60 3.3. Linear vs. nonlinear

To see why this ODE is nonlinear, let’s focus on the operation

!2
dx
. (3.34)
dt

Setting x = x1 + x2 , we see that this operation is nonlinear, because

!2 !2 !2 !2
dx dx1 dx1 dx1 dx dx dx2
= + = +2 1 2 + , (3.35)
dt dt dt dt dt dt dt

which does not equal

!2 !2
dx1 dx2
+ . (3.36)
dt dt

Example 3.1 (Linear vs Nonlinear) Suppose that y is a function of x, i.e. y(x). Deter-
mine whether the following ODE’s are linear or nonlinear. If the ODE is linear, write
out the linear operator.
1.
d 2 y dy
+ + y = 0.
dx2 dx

2.
d 2 y dy 1
+ + = 0.
dx2 dx y

3.
d 2y dy
ex + sin(x) + y = 0.
dx2 dx

4.
d 2y
= sin(y).
dx2

5.
dy
y = f (x).
dx

6.
1 dy
= f (x).
y dx
 Chapter 3. Introduction to Di↵erential Equations 61

3.4 Homogeneous vs. non-homogeneous

Definition 3.5 (Non-homogeneous di↵erential equation)

A non-homogeneous di↵erential equation has one or more terms that do not depend on
the dependent variable. Otherwise, the equation is called homogeneous.

Let’s reconsider the earlier example of the mass-spring system sketched in figure 3.6. If
a force F(t) is applied to the mass, a force balance in the x-direction produces the ODE

d 2x
m + kx = F(t). (3.37)
dt 2

This equation is non-homogeneous due to the term on the

right-hand-side, which does not contain the dependent
x
variable x. When writing ODE’s we usually put all terms k
m
involving the dependent variable on the left-hand-side,
and we put all other terms on the right-hand-side. Us-
ing this convention, an ODE is homogeneous if the right-
hand-side is zero, and is non-homogeneous otherwise.
k F(t)
For an example of a non-homogeneous PDE, let’s con- m
sider unsteady heat transport in the bar of metal sketched
in figure 3.7. If the bar is thermally insulated on its lateral
Figure 3.6
edges, heat is constrained to travel in the x-direction. We
can then generate variations of temperature T (x, t) within
the bar by changing the temperatures TL and TR on the
left and right edges of the bar. These variations satisfy the
TL x TR
PDE
@T @2 T L
⇢cp =K 2, (3.38)
@t @x Figure 3.7

where ⇢, cp , and K are the density, heat capacity, and thermal conductivity of the metal, re-
spectively. This is a homogeneous PDE, because all terms involve the independent variable
T . Now suppose a chemical reaction generates heat within the bar, and that the heat gener-
ated by this reaction is independent of the temperature. The temperature field will then be
governed by the non-homogeneous PDE

@T @2 T
⇢cp = K 2 + q(x, t), (3.39)
@t @x

where the added term accounts for the heat generation within the bar.

Definition 3.6 (Forcing terms)

Non-homogeneous terms in di↵erential equations are often called forcing terms. For
example, q(x, t) is the forcing term in equation (3.39).
 62 3.5. Constant vs. non-constant coefficient

3.5 Constant vs. non-constant coefficient

Di↵erential equations are further characterized by whether or not the coefficients multiply-
ing the function and its derivative are constant. The following ODE,

d 2x
m + kx = cos(t), (3.40)
dt 2

is a constant coefficient ODE because the coefficients multiplying d 2 x/dt 2 and x on the left-
hand-side are constants. In contrast, the ODE

d 2x
m + cos(t)x = 0, (3.41)
dt 2

is a non-constant ODE, because the coefficient multiplying x is a function of t. Note that the
classification of whether a di↵erential equation is constant coefficient or not has nothing to
do with the forcing term. As one more example, suppose we reconsider the PDE governing
heat transport in the bar of metal sketched in figure 3.7,

@T @2 T
⇢cp =K 2. (3.42)
@t @x

This PDE assumes that ⇢, cp , and K are all constants. If instead the thermal conductivity K
is a function of position x, heat transport satisfies the modified PDE

@T @2 T dK @T
⇢cp = K(x) 2 + , (3.43)
@t @x dx @x

which is a non-constant coefficient PDE.

3.6 Initial value vs. boundary value problems

Definition 3.7 (Initial value vs. boundary value problems)

An ordinary di↵erential equation is an initial value problem (IVP) if the independent
variable is time (t). It is a boundary value problem (BVP) if the independent variable is
a spatial coordinate (x, y, or z). A partial di↵erential equation can be both an IVP and
BVP if the independent variables include both time and space.

To demonstrate why the distinction between IVP and BVP is important, lets reconsider
the example of a dropped coin, as sketched in figure 3.8. If gravity is the only force acting
on the coin, its position satisfies the IVP

d 2x
= g. (3.44)
dt 2
 Chapter 3. Introduction to Di↵erential Equations 63

To solve this equation, we integrate both sides with respect to t, pro-

ducing
dx
= gt + A. (3.45)
dt

where A is any constant. Integrating once again, we find that x

gt 2
x(t) = + At + B, (3.46)
2
Figure 3.8
where B is another constant.
Solving an ODE usually produces one or more unknown constants of integration, such
as A and B above. For IVPs, these are determined by applying initial conditions. Suppose
the coin is dropped at time t = 0 with an initial position x = 0. This produces the initial
condition
x(0) = 0. (3.47)

Substituting this condition in solution (3.46) requires

x(0) = B = 0, (3.48)

from which we conclude that

gx2
x(t) = + Ax. (3.49)
2

To determine A, we require a second initial condition. Suppose the coin is released with an
initial downward velocity of zero. This produces the initial condition

dx
= 0. (3.50)
dt t=0

This requires
dx
= A = 0. (3.51)
dt t=0

We finally conclude that

gx2
x(t) = . (3.52)
2

Let’s contrast the above solution of an IVP with a BVP. Consider steady heat transport in
the bar of metal sketched in figure 3.9. The temperature distribution satisfies the BVP

d 2T
= 0. (3.53)
dx2
 64 3.6. Initial value vs. boundary value problems

Integrating twice with respect to x produces the solution

T = Ax + B, (3.54) TL x TR

where A and B are determined from boundary conditions. L

If the temperatures on the left and right edges of the bar
Figure 3.9
are fixed at TL and TR , respectively, the boundary condi-
tions are
T (0) = TL , T (L) = TR . (3.55)

Substituting these conditions into solution (3.54), we find that

✓ ◆
TR TL
T= x + TL . (3.56)
L

In summary, we see that initial value problems are solved by applying one or more initial
conditions at one moment in time. In contrast boundary value problems can be solved by
applying boundary conditions at two or more di↵erent locations in space. Sometimes, how-
ever, this distinction between IVPs and BVPs is muddied. For example, we could have solved
for the temperature distribution in the metal bar by applying a temperature TL at x = 0 as
well as a heat flux qL at x = 0. This produces the boundary conditions

dT
T (0) = TL , K = qL , (3.57)
dx x=0

where K is the thermal conductivity of the metal. We still refer to these as boundary condi-
tions, however, because they are applied on a boundary in space. Moreover, these conditions
are applied for all times, as opposed to only one moment in time.
An additional distinction between IVPs and BVPs concerns the directions in which infor-
mation “flows” in space and time. Information in IVPs usually only flows forwards in time
(i.e. not backwards in time). That is to say, the solution of the system at some time t depends
on the behavior of the system at previous times, but not later times. In contrast, information
in BVPs often travels in multiple spatial directions. In the case of a metal bar, for example,
heat will travel from left to right if the temperature on the left (TL ) is greater than that on
the right (TR ). The rate of heat flow nevertheless depends on both boundary temperatures
TL and TR . Information travels both downstream and upstream.
Sometimes, the spatial flow of information in a boundary value problem can be one-way,
like time in an IVP. Suppose a pollutant is released into a water canal in which the water
flows rightward with constant velocity U, as sketched in figure 3.10(a). If the pollutant is
insoluble in water, the concentration c(x, t) of pollutant satisfies the PDE

@c @c
+U = 0, (3.58)
@t @x
 Chapter 3. Introduction to Di↵erential Equations 65

called the “advection equation.” This equa-

tion is both an IVP and a BVP. You can show,
however, that information only travels in the
(a)
downstream x-direction. Because the pollu- x
tant cannot dissolve in the water, it is trans-
ported downstream, and the concentration
c(x, t) translates rightward as sketched in fig-
ure 3.10(b). You will only encounter the pol- C
lutant if it was upstream of your position at (b) C(x,t0) C(x,t1)
some previous time. In this case, x behaves
x
much like time. In contrast, if the pollutant is
soluble in water, it can then dissolve and dif-
fuse both upstream and downstream. It then C
satisfies the modified PDE (c) C(x,t0) C(x,t1)

@c @c @2 c x
+U = D 2, (3.59)
@t @x @x Figure 3.10

called the “advection-di↵usion equation.” The concentration c(x, t) evolves as sketched in fig-
ure 3.10(c).

Summary:
The characteristics of ODEs and PDEs discussed in this chapter have more signifi-
cance than the brief definitions given here. Each classification provides insight as to
which analytical and numerical solutions are relevant to solving the problem. Thus,
the first step in any solution of a di↵erential equation is to identify the distinguishing
elements, and then the solution pathway will become more clear. For example,
consider how we classify the equations below.

Classification: Classify each di↵erential equation.

00 0
a) cos(2x2 )y + x2 y + xy = 0 is a second order, non-constant coefficient, homogeneous,
linear ODE.
000
b) yy = sin(x) is a third order, non-constant coefficient, non-homogeneous, nonlinear
ODE.
c) @u/@y + @u/@x = 0 is a first order, constant coefficient, homogeneous, linear PDE.
4. First Order Linear ODEs

4.1 Case 1: dy/dx = f (x) 67

4.2 Indefinite and definite integrals for ODEs 68

4.3 Integration by substitution 71

4.4 Interval of interest 72

4.5 Case 2: dy/dx + p(x)y = 0 73

4.6 Case 3: dy/dx + p(x)y = f (x) 75

4.7 Linear vs. exponential growth 77

4.8 Applications 78

A linear n’th-order ODE can be written in the form

d ny d n 1y dy
an (x) n
+ a n 1 (x) n 1
+ ... + a1 (x) + a0 (x)y = g(x), (4.1)
dx dx dx

where y is the dependent variable and x is the independent variable. In this chapter, we
consider the case of linear first-order ODEs,

dy
a1 (x) + a0 (x)y = g(x). (4.2)
dx

Assuming a1 (x) is not zero, we will divide this equation by a1 , and write it in the form

dy
+ p(x)y(x) = f (x). (4.3)
dx

In the following sections, we divide this equation into three cases of increasing complex-
ity. Case 1 considers the ODE when p(x) = 0,

dy
= f (x). (4.4)
dx

Case 2 considers the ODE when f (x) = 0,

dy
+ p(x)y(x) = 0. (4.5)
dx

Finally, case 3 considers the general form (4.3) above.

66
 Chapter 4. First Order Linear ODEs 67

4.1 Case 1: dy/dx = f (x)

Perhaps the simplest first-order linear ODE occurs when p(x) = 0, such that

dy
= f (x). (4.6)
dx

These ODE’s can be solved by “integrating both sides of the equation.” Consider the example

dy
= cos(x). (4.7)
dx

Integrating both sides of the equation with respect to x produces

y = sin(x) + C, (4.8)

where C is an arbitrary constant. The idea here is that if you take the derivative of the left
and right-hand-sides of equation (4.8), you indeed recover the original ODE (4.7).

Solution (4.8) is called a general solution, because it includes all possible solutions of
the ODE. To determine the constant C, we need to apply a boundary condition (or initial
condition if x refers to time). Suppose

y(x0 ) = y0 . (4.9)

This requires

y(x0 ) = sin(x0 ) + C = y0 , (4.10)

from which we conclude that C = y0 sin(x0 ) and

y = sin(x) + y0 sin(x0 ). (4.11)

Technically, the solution above is still a general solution, because y0 and x0 can be set to
recover any possible solution of the ODE. Now suppose we specify that y(0) = 0. In that
case,
y = sin(x). (4.12)

Solution (4.12) above is called a particular solution, because all arbitrary constants have been
set.
 68 4.2. Indefinite and definite integrals for ODEs

4.2 Indefinite and definite integrals for ODEs

What does it really mean when we integrate both sides of an equation? Students sometimes
conceptualize the procedure by breaking it into the following steps:

1. Multiply both sides of dy/dx = f (x) by dx,

dy
dx = f (x)dx. (4.13)
dx

2. Cancel the dx in the numerator and denominator of the left-hand-side,

dy = f (x)dx. (4.14)

3. Left-multiply both sides with the integral symbol,

Z Z
dy = f (x)dx. (4.15)

4. Now solve the integrals on both sides of the equation to get

Z
y= f (x)dx. (4.16)

These steps produce the correct answer, but are conceptually incorrect. First, steps 1 and
2 treat dy/dx as a fraction, which it is not. A quick internet search of “is dy/dx a fraction”
demonstrates this is a common misconception. Steps 3 and 4 are equally troublesome. Are
you really allowed to integrate the left-hand-side of an equation with respect to y and the
right-hand-side with respect to x? As a counterargument, consider the equation

y = 2x. (4.17)

Di↵erentiating the left-hand-side with respect to y and the right-hand-side with x produces
the nonsense result
1 = 2. (4.18)

If you cannot di↵erentiate the left and right-hand-sides of an equation with respect to two
di↵erent variables, why would you be allowed to do integrate the two sides with respect to
di↵erent variables?
To address these issues we need to review the concepts of antiderivatives, indefinite inte-
grals, definite integrals, and integration by substitution. We will review the first three here, and
discuss integration by substitution in Section 4.3. Let’s begin by considering the function

f (x) = x2 . (4.19)
 Chapter 4. First Order Linear ODEs 69

The corresponding function

x3
F(x) = , (4.20)
3

is called an antiderivative of f (x), because

dF
= f (x). (4.21)
dx

The function f (x) has an infinite number of antiderivatives that can all be written as

x3
F(x) = + C, (4.22)
3

where C is a constant. Some people define the indefinite integral as the operation that gener-
ates this infinite family of antiderivatives,
Z
x3
f (x) dx = + C. (4.23)
3

In practice, however, others drop the constant, and simply write

Z
x3
f (x) dx = . (4.24)
3

A definite integral is defined in terms of the antiderivative F(x) as the operation,

Zb
f (x) dx = F(b) F(a). (4.25)
a

When performing a definite integral, you can use any of the infinite antiderivatives F(x),
because the constant term will cancel. For example

Zb " #b
2 x3 b3 a3
x dx = +C = . (4.26)
3 a 3 3
a

In the discussion above, we have been using the uppercase symbol F(x) to denote the
antiderivative, but this is not established notation. More commonly, people denote an an-
tiderivative in terms of a definite integral as

Zx
F(x) = f (x) dx, (4.27)
a
 70 4.2. Indefinite and definite integrals for ODEs

where a is some constant. It is considered poor practice, however, to have the same inde-
pendent variable in both the integrand and limits. In more formal settings, this operation is
often written as
Zx
F(x) = f (⇠) d⇠, (4.28)
a

where f (x)dx is replaced with f (⇠) d⇠. For the current textbook, however, we will be lenient
with this notation.

Now, let’s reconsider the ODE

dy
= cos(x). (4.29)
dx

Conceptually, we can solve this equation by taking the antiderivative of both sides, which
we can write using indefinite integrals as
Z Z
dy
dx = cos(x) dx. (4.30)
dx

This has the solution

y + C1 = sin(x) + C2 , (4.31)

where C1 and C2 are constants. These can be merged into a single constant C = C2 C1 ,

y = sin(x) + C. (4.32)

Instead of using indefinite integrals, you can also solve the ODE using definite integrals.
Suppose the ODE satisfies the boundary condition y(x0 ) = y0 , then we can integrate both
sides of the equation as
Zx Zx
dy
dx = cos(x) dx. (4.33)
dx
x0 x0

This produces
y(x) y0 = sin(x) sin(x0 ), (4.34)

which is identical to our solution (4.11) earlier.

 Chapter 4. First Order Linear ODEs 71

4.3 Integration by substitution

Now let’s reconsider the steps labelled 1-4 earlier in Section 4.2. We can show that the left-
hand-side of equation (4.30) can indeed be written as
Z Z
dy
dx = dy. (4.35)
dx

Consequently, equation (4.13) in step 3 above is actually correct, though the steps 1 and 2
were faulty. To do it properly, we need the concept of integration by substitution.

Definition 4.1 (Integration by substitution)

Suppose you can write an indefinite integral in the form
Z ⇣ ⌘
f g(x) g 0 (x) dx. (4.36)

It can then be shown that

Z ⇣ ⌘ Z
f g(x) g 0 (x) dx = f (u) du, (4.37)

where
u = g(x). (4.38)

This rule is easier to follow if we define y = g(x). Integration by substitution can then
be written as
Z Z
dy
f (y) dx = f (y) dy. (4.39)
dx

Now suppose you have the definite integral

Zb
dy
f (y) dx. (4.40)
dx
a

This can be written as

Zb y(b)
Z
dy
f (y) dx = f (y) dy. (4.41)
dx
a y(a)

Suppose we set f (y) = 1 in equation (4.39), we then find that

Z Z
dy
dx = dy. (4.42)
dx
 72 4.4. Interval of interest

In summary, though dy/dx is not a fraction, the relation above is nevertheless true due to
the concept of integration by substitution.
Now let’s reconsider the earlier solution (4.33) in terms of definite integrals

Zx Zx
dy
dx = cos(x) dx. (4.43)
dx
x0 x0

Using integration by substitution for definite integrals, we can write this as

Zx y(x)
Z
dy
dx = dy. (4.44)
dx
x0 y0

Summary: You can write the solution of

dy
= f (x) (4.45)
dx

in terms of the indefinite integrals

Z Z Z Z
dy
dx = f (x) dx or dy = f (x) dx (4.46)
dx

or the definite integrals

Zx Zx Zy Zx
dy
dx = cos(x) dx or dy = cos(x) dx. (4.47)
dx
x0 x0 y0 x0

4.4 Interval of interest

When presenting an ODE, it is sometimes important

to specify the interval of interest. Consider, for ex- 100

ample, the ODE

50

dy 1
= , (4.48)
dx (1 x)2 0

-50
subject to the initial condition
-100
y(0) = 1. (4.49) 0 0.5 1 1.5 2

Figure 4.1
 Chapter 4. First Order Linear ODEs 73

This has the solution

1
y= + C, (4.50)
(1 x)

where the initial condition requires C = 0. As demonstrated in figure 4.1, this solution
becomes discontinuous at x = 1 where the derivative tends to infinite values. If we are only
interested in the solution to the ODE over a finite interval not including x = 1, say 0  x  0.5,
then we can likely ignore the behavior outside the interval. On the other hand, if our interval
includes x = 1, then we may need to reconsider whether the ODE is appropriate for whatever
system we are modeling.

4.5 Case 2: dy/dx + p(x)y = 0

When q(x) = 0, equation (4.3) becomes

dy
+ p(x)y = 0. (4.51)
dx

Such ODEs can be solved by first rearranging them as

1 dy
= p(x). (4.52)
y dx

This requires us to assume y , 0, which is an assumption we need to revisit after solving the
ODE. We then integrate both sides of the equation with respect to x,
Z Z
1 dy
dx = p(x) dx. (4.53)
y dx

Using the integration by substitution formula (4.39), we can express the left-hand-side of
equation (4.53) as
Z Z
1
dy = p(x) dx. (4.54)
y

We conclude that
Z
ln|y| = p(x) dx + C, (4.55)

where C is an arbitrary constant, and the absolute value |y| arises because the logarithm is
not defined when y < 0.
The presence of the absolute value |y| requires some careful attention. We first note that
from the definition of the logarithm, equation (4.55) requires
R
p(x) dx+C
|y| = e . (4.56)
 74 4.5. Case 2: dy/dx + p(x)y = 0

Because y could be positive or negative, this requires

R
p(x) dx+C
y = ±e , (4.57)

where the sign of the right-hand-side is set when we apply the boundary or initial condition.
In that case, we can simply write the general solution as
R
p(x) dx
y = Ae , (4.58)

where A = ±eC . This is the same solution we would have obtained if we had omitted the
absolute value symbols in equation (4.55), i.e. writing it as
Z
ln(y) = p(x) dx + C. (4.59)

This is very often done in practice, with the understanding that it leads to the correct answer.
For brevity, we often take that approach in the current textbook, as demonstrated in the
example below.

Example 4.1 First-order chemical reactions have the unique feature that the rate of
change of reactant concentration (dy/dt) varies linearly with the reactant concentra-
tion (y) following the relation
dy
= ky, (4.60)
dt

where k is called the rate coefficient. Using the procedure described above, this can be
rearranged and integrated with respect to t,
Z Z
1 dy
dt = k dt. (4.61)
y dt

Using integration by substitution, this in turn can be expressed as

Z Z
1
dy = k dt. (4.62)
y

This has the solution

ln(y) = kt + C. (4.63)

Using the definition of logarithms, this can be expressed as

y = eC e kt
. (4.64)
 Chapter 4. First Order Linear ODEs 75

Finally, we can define A = eC , and express the general solution as

kt
y = Ae . (4.65)

Now suppose the initial reactant concentration is y0 at time t = 0. Applying this initial
condition, we find that
kt
y = y0 e . (4.66)

Finally, we need to revisit equation (4.52),

1 dy
= p(x), (4.67)
y dx

which assumes that y , 0. Our resulting solution,

R
p(x) dx
y = Ae , (4.68)

is indeed never zero, except in certain limits. The constant A = ±eC is never zero, unless you
try to include the limit C ! 1. Similarly, the exponential term is never zero, unless you
try to include the limit e 1 . In the previous example, this occurred as t ! 1.
To consider the case y = 0 properly, we need to return to the original ODE

dy
+ p(x)y = 0. (4.69)
dx

which reduces to
dy
= 0. (4.70)
dx

We see that y = 0 is a special solution for which y is constant. Such solutions, called equilib-
rium solutions, play an in important role in nonlinear ODEs, and will be discussed in more
detail in Chapter 5. For now, we note that we can include the equilibrium solution into the
general solution by simply allowing A = 0 in
R
p(x) dx
y = Ae . (4.71)

4.6 Case 3: dy/dx + p(x)y = f (x)

Let’s now consider the general form

dy
+ p(x)y(x) = f (x). (4.72)
dx
 76 4.6. Case 3: dy/dx + p(x)y = f (x)

If we rearrange the ODE as we did for Case 2,

1 dy f (x)
= p(x) + , (4.73)
y(x) dx y(x)

integrate with respect to x,

Z Z Z
1 dy f (x)
dx = p(x) dx + dx, (4.74)
y(x) dx y(x)

and apply integration by substitution,

Z Z Z
1 f (x)
dy = p(x) dx + dx, (4.75)
y(x) y(x)
| {z }
impasse

we reach an impasse due to the new term on the right-hand-side generated by the non-
homogeneous term f (x). This term cannot be integrated, because y(x) is unknown.
To solve ODE (4.72), suppose we first multiply the equation by some function (x),

dy
(x) + (x)p(x)y(x) = (x)f (x). (4.76)
dx

We can now choose (x) so that

" #
dy d
(x) + (x)p(x)y(x) = (x)y(x) . (4.77)
dx dx

This allows to write the ODE as

" #
d
(x)y(x) = (x)f (x), (4.78)
dx

which can be integrated with respect to x, producing

Z
(x)y(x) = (x)f (x) dx + C, (4.79)

where C is an arbitrary constant. Finally, we solve for y(x),

2Z 3
1 666 77
y(x) = 6 (x)f (x) dx + C 775. (4.80)
(x) 4

Note that this last step requires (x) , 0.

 Chapter 4. First Order Linear ODEs 77

The function (x) is called an integrating factor. It must satisfy

dy d d dy
+ py = ( y) = y+ , (4.81)
dx dx dx dx

where we have used the product rule to expand d( y)/dx. This simplifies to

d
p(x) = 0. (4.82)
dx

From our work in Section 4.5, we can solve this ODE to show that
R
p(x)dx
(x) = Ae , (4.83)

where A is an arbitrary constant. This constant plays no actual role in applying boundary
conditions, and we consequently set to A = 1 for convenience,
R
p(x)dx
(x) = e . (4.84)

4.7 Linear vs. exponential growth

Two famous first-order linear ODEs that deserve further discussion are the initial value prob-
lems
dy
=k (4.85)
dt

and
dy
= ky. (4.86)
dt

These produce the solutions

y = kt + C and y = Cekt , (4.87)

respectively, where C is an arbitrary constant. ODEs of the form (4.85) produce linear growth
(when k > 0) or decay (when k < 0), while equations of the form (4.86) produce exponential
growth (k > 0) or decay (k < 0).

A classic example of exponential growth is that of bacteria undergoing binary fission.

Suppose a petri dish has a single bacterium at time t = 0. In the first hour, the bacterium
divides into two. In the second hour, each bacteria divides again, such that there are four
bacteria. Growing in this manner, there are 8, 16, and 32 bacteria in the third, fourth, and
fifth hours. We see that the rate of change of the bacteria population (dy/dt) is proportional
 78 4.8. Applications

to the population size (y), i.e.

dy
= ky, (4.88)
dt

such that y = Cekt . To determine k, we consider that at times t1 and t2 ,

y1 = Cekt1 , y2 = Cekt2 . (4.89)

Taking the logarithm of y2 /y1 , we find that

!
y2
ln = k(t2 t1 ), (4.90)
y1

which we then solve for k,

!
1 y2
k= ln . (4.91)
t2 t1 y1

For the example of bacteria, we see that

✓ ◆
1 4
k= ln = ln(2). (4.92)
2 1 2

We conclude that
y = Celn(2)t . (4.93)

Using logarithm identities, we can write this as

t
y = Celn(2 ) = C2t . (4.94)

The initial condition then requires C = 1, such that

y = 2t . (4.95)

4.8 Applications
In this section, we demonstrate how first-order ODEs arise in modeling electric circuits,
chemical kinetics, and stirred tank reactors. Our objectives are threefold. First, we want
to demonstrate how ODE’s arise when modeling real-word systems. Second, we want to
demonstrate how to solve those ODEs using the techniques discussed earlier in the chapter.
Third, we want to demonstrate how we can use these solutions to explore the physics of the
systems being studied.
 Chapter 4. First Order Linear ODEs 79

4.8.1 Circuits

Here we consider applications of first-

order linear ODEs to modeling simple (a) (b) (c)
circuits composed of a power supply
providing a constant voltage v, a wire
v v v
carrying a current i, a switch, a resis- R L R

tor, and an inductor. Let’s begin by con- i i L

sidering the circuit shown in Fig. 4.2(a)
with a single resistor. The basic func- Figure 4.2
tion of a resistor is to reduce the current
in a circuit. Suppose the switch is suddenly closed at time t = 0. The current through the
resistor is then given by Ohm’s law,
v
i= , (4.96)
R

where R is the resistance.

Now let’s consider the circuit in figure 4.2(b) with a single inductor. The function of an
inductor is to resist changes in current. If the switch is suddenly closed at t = 0, the current
in the circuit will evolve as
di v
= , (4.97)
dt L

where L is the inductance. This ODE (4.97) has the solution

v
i= t + A, (4.98)
L

where A is an arbitrary constant. Applying the initial condition i(0) = 0, we find that the
current grows linearly without bound,

v
i= t. (4.99)
L

Figure 4.3 shows the resulting variation of cur-

rent with time produced by a single resistor (blue i
line) and inductor (red line). Neither result is en-
tirely physical. The circuit with a single resistor im- v
R
mediately produces the current v/R upon closing the
switch. In actuality, it should take some finite time v
R
for the current to increase from an initial value of
zero at t = 0. Meanwhile, the current in the circuit t
with a single inductor grows without bound. We can Figure 4.3
address these issues by realizing that the wire and re-
sistor also have some inductance. Meanwhile, the inductor has some resistance. In that
case, let’s consider the more realistic “RL circuit” sketched in figure 4.2(c). Kirchho↵’s loop
rule states that the sum of the voltage drops across resistor and inductor equal the voltage
 80 4.8. Applications

provided by the power supply, i.e.

di
L + Ri = v. (4.100)
dt

Dividing by L,
di R v
+ i= , (4.101)
dt L L

we see this equation can be solved using the integrating factor

R
(R/L)dt
(x) = e = eRt/L . (4.102)

Multiplying equation (4.101) by (x), we can rewrite the equation as

" #
Rt/L di Rt/L R d Rt/L v
e +e i= ie = eRt/L . (4.103)
dt L dt L

This has the solution

v Rt/L
ieRt/L = e + A, (4.104)
R

where A is an arbitrary constant. Applying the initial condition i(0) = 0, we find that

v
+ A = 0, (4.105)
R

and
" #
v Rt/L
i= 1 e . (4.106)
R

Figure 4.4 shows that the response of the RL cir-

cuit can be decomposed into a steady term and tran- 1

sient term, 0.8

v v t/⌧ 0.6
i= e , (4.107)
R R
|{z} | {z } 0.4

steady-state transient 0.2

0
where ⌧ = L/R is a time-constant. 0 2 4 6 8 10

Let’s now repeat the above analysis for a power Figure 4.4
supply providing the alternating voltage

v(t) = V cos(!t), (4.108)

 Chapter 4. First Order Linear ODEs 81

where V is the amplitude and ! is the angular frequency. The RL circuit now satisfies the
ODE
di R V
+ i = cos(!t). (4.109)
dt L L

Using the same integrating factor as before, we can write this ODE as
" #
d Rt/L V
ie = eRt/L cos(!t). (4.110)
dt L

After some integration and algebra, this produces the general solution
" #
V cos(!t) + !⌧ sin(!t) t/⌧
i= + Ae , (4.111)
R 1 + !2 ⌧ 2

where ⌧ = L/R, and A is an arbitrary constant. If we assume the switch is suddenly closed at
t = 0, and apply the initial condition i(0) = 0, we can write the final solution as

" #
V 1 t/⌧
i= cos(!t) + !⌧ sin(!t) e . (4.112)
R 1 + !2 ⌧ 2

When analyzing and interpreting general solutions such as that above, it can be helpful
to first identify any special cases. Consider, for example, the three cases below.

Case 1. When ! = 0, the power supply provides a constant voltage V , and we recover our
earlier solution (4.106)
" #
V t/⌧
i= 1 e . (4.113)
R

Case 2. If we remove the inductor by setting L = 0, the solution simplifies to

V
i= cos(!t), (4.114)
R

which is simply Ohm’s law for an oscillating voltage.

Case 3. If we remove the resistor by setting R = 0, the general solution (4.106) is invalid,
because R appears in the denominator. In this case, we return to our original ODE (4.109),
which now simplifies to
di V
= cos(!t). (4.115)
dt L
 82 4.8. Applications

This has the solution

V
i= sin(!t) + A, (4.116)
!L

where A is an arbitrary constant. Setting i(0) = 0, we find that A = 0, and

V
i= sin(!t). (4.117)
!L

Comparing this result for that of a single resistor in equation (4.114), we see that the inductor
imposes an e↵ective resistance R = !L and also causes the current to be 90 out-of-phase with
the voltage signal.

To interpret the general case, we first use the theorem of harmonic addition (shown later
below) to express
p
cos(!t) + !⌧ sin(!t) = 1 + !2 ⌧ 2 cos(x + ), (4.118)

where = arctan( !⌧). This allows us to express the general solution as

V V e t/⌧
i= p cos(!t + ) (4.119)
R 1 + !2 ⌧ 2 R(1 + ! 2 ⌧ 2 )

Next, we see that the second term on the right-hand-side is a transient term that decays
to zero as
p t ! 1. If we focus on the late-time behavior, and define an e↵ective resistance
Ref f = R 1 + ! 2 ⌧ 2 , we can write the solution as

V
i= cos(!t + ). (4.120)
Ref f

We conclude that when an RL circuit is subjected to an input voltage v = V cos(!t), the cir-
cuit has an e↵ective resistance Ref f > R and the current has the phase shift = arctan( !⌧).

Theorem 4.1 (Harmonic addition)

The linear combination of a sine and cosine wave is equivalent to a single cosine wave
with a phase shift and amplitude G,

a cos(x) + b sin(x) = G cos(x + ), (4.121)

where
p !
b
G = sign(a) a2 + b2 , = arctan . (4.122)
a
 Chapter 4. First Order Linear ODEs 83

Proof: Using the trig identity for cos(x + ), we see that

G cos(x + ) = G cos(x) cos( ) G sin(x) sin( ) = a cos(x) + b sin(x). (4.123)

Equating coefficients, we find that

G cos( ) = a G sin( ) = b. (4.124)

Taking the ratio of these relations, we find that

G sin( ) b
= tan( ) = . (4.125)
G cos( ) a

Taking the sum of the squares of the relations, we find

G 2 cos2 ( ) + G 2 sin2 ( ) = a2 + b2 , (4.126)

p
which requires G = ± a2 + b2 . To choose the proper sign for G, we note that when
b = 0, we recover
a cos(x) = G cos(x). (4.127)

Consequently, G must have the same sign as a.

4.8.2 Chemical kinetics

Chemical kinetics is the branch of chemistry concerned with understanding and modeling
the rates of chemical reactions. Consider the irreversible* reaction in which the reactant A
produces the product B,
A ! B. (4.128)

Hereinafter, let’s assume all reactions occur “in solution,” such that the reactants and prod-
ucts are dissolved in water, and we can express the amounts of reactants and products as
concentrations measured in units of moles per liter. The reaction rate (r) for the example
above is defined as
dCA dCB
r= = , (4.129)
dt dt

where CA (t) and CB (t) are the concentrations of A and B, respectively. Note r is defined such
that it is positive. For an irreversible reaction, the reaction rate is primarily a function of the

* An irreversible reaction is one for which the products cannot convert back to the reactants.
 84 4.8. Applications

reactant concentration. For the simple A ! B reaction, this is expressed as the rate equation

r = kCAn , (4.130)

where k 0 has units 1/sec, and n is called the “order of the reaction.”
The concepts above can be extended to more more complicated reactions as follows. Con-
sider,
A + 2B ! E. (4.131)

For this case, we define the reaction rate as

dCA 1 dCB dCE

r= = = , (4.132)
dt 2 dt dt

where the fraction (1/2) is placed in front of dCB /dt to account for the fact that 2 moles of
B are consumed for every mole of A. Extending this concept further, consider the general
reaction
aA + bB ! eE + gG, (4.133)

in which A and B react to form E and G. We define the reaction rate as

1 dCA 1 dCB 1 dCE 1 dCG

r= = = = , (4.134)
a dt b dt e dt g dt

and we express the rate equation as

r = kCAn CBm , (4.135)

where n + m is called the overall reaction order.

Di↵erential equations play a key role in chemical kinetics. For example, let’s return to
the simple A ! B reaction. If the reaction is first-order, the concentrations CA and CB satisfy

dCA dCB
= kCA , = kCA . (4.136)
dt dt

The ODE for CA is easily solved to show that

kt
CA = A0 e , (4.137)

where A0 is the initial concentration of A at t = 0. Product B consequently satisfies the ODE

dCB kt
= kA0 e , (4.138)
dt
 Chapter 4. First Order Linear ODEs 85

which has the solution

1
kt
CB = A0 e + P, (4.139) CA
0.8
CB

0.6
where P is an arbitrary constant. If there is ini-
tially no product at t = 0, then P = A0 , and 0.4

⇣ ⌘ 0.2
kt
CB = A0 1 e . (4.140)
0
0 1 2 3 4 5 6
For simplicity, we can define the non-dimensional
b = C/A0 and time b Figure 4.5
concentration C t = kt, and
write the solution as
bA = e bt ,
C bB = 1 e bt ,
C (4.141)

which is plotted in figure 4.5.

As a more complicated example, let’s consider the two-step reaction

k1 k2
A ! B ! C. (4.142)

The k1 and k2 placed above the arrows tell us that the single step reaction A ! B satisfies the
rate equations
dCA dCB
= k1 CA , = k1 CA , (4.143)
dt dt

while the single step reaction B ! C satisfies

dCB dCC
= k2 CB , = k2 CB . (4.144)
dt dt

In the two-step reaction, both single-step reactions occur simultaneously, such that

dCA dCB dCC

= k1 CA , = k1 CA k2 CB , = k2 CB . (4.145)
dt dt dt

Suppose we begin the reaction with the initial concentrations CA (0) = A0 , and CB (0) =
CC (0) = 0. The ODE for CA produces the solution

k1 t
CA = A0 e , (4.146)

such that CB satisfies

dCB k1 t
+ k2 CB = k1 A0 e . (4.147)
dt
 86 4.8. Applications

Using the integrating factor ek2 t , we can rewrite this ODE as

d ⇣ ⌘
k1 )
CB ek2 t = k1 A0 et(k2 . (4.148)
dt

At this point, we need to consider two cases. First, let’s assume that k2 , k1 . In that case, it is
straightforward to solve for CB and CC ,

k1 A0 ⇣ k1 t k2 t
⌘
CB = e e , (4.149)
k2 k1

A0 ⇣ k1 t k2 t
⌘
CC = k e k1 e k2 + k1 . (4.150)
(k2 k1 ) 2

On the other hand, if k1 = k2 = k, then equation (4.148) above simplifies to

d ⇣ ⌘
CB ekt = kA0 . (4.151)
dt

Integrating and applying the initial condition, we find that

kt
CB = kA0 te . (4.152)

The concentration CC then satisfies the ODE

dCC
= k 2 A0 te kt
. (4.153)
dt

Integrating and applying the initial condition produces

kt
CC = A0 (1 + kt) e + A0 . (4.154)

To explore our two solutions, let define the non-dimensional concentration C b = C/A0 and
non-dimensional time bt = k1 t. For the case k1 , k2 we can write the solution as

1 ⇣ ⌘
bA = e bt ,
C bB =
C e
b
t
e ↵b
t
, (4.155)
↵ 1
1 ⇣ ⌘
↵b
t b
bC =
C e ↵e t + ↵ 1 , (4.156)
(↵ 1)

where ↵ = k2 /k1 .
 Chapter 4. First Order Linear ODEs 87

(a) (b)
1 1
C C
A A
C C
B B
0.8 0.8
CC CC

0.6 0.6

0.4 0.4

0.2 0.2

0 0
0 1 2 3 4 5 0 10 20 30 40

Figure 4.6

Figure 4.6 shows the resulting variation of concentrations when ↵ = 10 (panel a) and
↵ = 0.1 (panel b). When ↵ = 10, k2 is ten times the rate of k1 . As a result, the product B
generated by reaction A ! B is quickly consumed by the subsequent reaction B ! C. In fact,
if we take the limit ↵ ! 1 our solution simplifies to

bA = e bt ,
C bB = 0
C bC = 1 e bt ,
C (4.157)

where we needed to use L’Hospital’s rule for the limit of CC . This limit recovers the one-step
reaction A ! C. In contrast, when ↵ = 0.1, the reaction A ! B is ten times faster than the
subsequent B ! C. This causes A to be quickly consumed, thereby generating a surplus of
B, which requires considerable time to consume. Note that the case ↵ = 10 quickly reaches
it’s steady state in roughly 5 seconds, while ↵ = 0.1 requires more then 40 seconds.

4.8.3 Stirred tank reactors

Figure 4.7 illustrates a continuously stirred tank reactor

(CSTR).† Reactants are continuously fed into the tank where
they undergo a chemical reaction. The reaction products are
simultaneously extracted at the same flow rate as the inlet re-
actants to maintain a constant volume in the tank. CSTR’s have
broad applications in chemical and environmental engineer-
ing, pharmaceuticals, wastewater treatment, biofuel produc-
tion, and oil and gas operations. For our purposes, suppose a
liquid solution enters the tank with constant volumetric flow
rate Q (m3 /s). The entering solution has a constant concentra-
tion Cin (moles/m3 ) of a single reactant A. In the tank, reactant

† Daniele Pugliesi, CC BY-SA 3.0

<https://creativecommons.org/licenses/by-sa/3.0>, Wikimedia Commons Figure 4.7
 88 4.8. Applications

A is converted to product B through the chemical reaction,

A ! B. (4.158)

Suppose we want to estimate the time for the reactor to reach steady-state operation. A
mass-balance on the number of moles NA of reactant within the tank produces the di↵eren-
tial equation
dNA
= Ṅin Ṅout Ṅcons , (4.159)
dt

where Ṅin and Ṅout represent the rates with which moles of A enter and leave the tank,
respectively, and Ṅcons represents the rate with which moles of A are consumed in the chem-
ical reaction. The simplest model assumes the tank’s contents are uniformly mixed, such
that the output composition is identical to that in the tank. The concentration CA of reactant
A within the tank then satisfies the di↵erential equation

dCA
V = QCin QCA rV , (4.160)
dt

where V is the volume of the tank and r is the rate with which A is consumed per unit
volume, units mol/(m3 s). Now suppose the reaction is a first-order reaction for which r is
linearly proportional to CA ,
r = kCA , (4.161)

where k is some constant (units 1/s). We can then express the di↵erential equation as
✓ ◆
dCA Q Q
+ + k CA = Cin , (4.162)
dt V V

where we have isolated all terms with the unknown CA on the left-hand-side. The ratio V /Q
has units of time, and is often interpreted as the residence time ( = V /Q) that the flowing
solution spends in the tank. Using this definition, we can write the equation as
✓ ◆
dCA 1 C
+ + k CA = in . (4.163)
dt

This equation has the solution

t Cin
CA (t) = De + , (4.164)
1+ k

where = (1/ ) + k, and D is any constant. To determine D, suppose that CA at the initial
startup (t = 0) equals zero. This requires

Cin
CA (0) = D + = 0. (4.165)
1+ k
 Chapter 4. First Order Linear ODEs 89

We conclude that D = Cin /(1 + k), and

Cin Cin
CA = e t + (4.166)
1+ k 1+ k
| {z } | {z }
initial transient steady-state

The initial transient term tends to zero as 10-3

6
time t tends to large values. The second term is
the remaining steady-state concentration of re- 5
actant A. Suppose we consider a 1 m3 tank with
4
an inlet concentration of 1 mol/m3 and a reac-
tion constant k = 1 s 1 . The black line in fig- 3

ure 4.8 shows the temporal evolution of CA when 2

Qin = 0.001 m3 /s (one liter per second). The blue
line shows the result when the inlet flow rate is 1

increased to 0.005 m3 /s (5 liters per second). The 0

red dashed lines show the corresponding steady- 0 2 4 6 8 10

state values. The steady-state value increases as

Figure 4.8
Q increases because this decreases the residence
time during which the reaction can occur within
the tank. 10-3

To explore the time required to reach steady- 2

state, we need to investigate the exponential

term in solution (4.166). This can be written as 1.5

t t/⌧
e =e , (4.167) 1

0.5
where ⌧ is called the exponential time constant,
0
1 0 2 4 6 8 10
⌧= = . (4.168)
1+ k
Figure 4.9

⌧ is the time required for the exponential term to reduce to 1/e ⇡ 0.37 times its initial value.
A more intuitive measure of the rate of decay is the half-life t1/2 , which is the time required
for the exponential term to decay to one half of its initial value,

t1/2 = ⌧ln2. (4.169)

The half-lives for the two cases plotted in figure 4.8 are nearly identical, t1/2 = 1.998 for
Q = 0.001 and t1/2 = 1.99 for Q = 0.005.
Suppose we reconsider the same system, but fix the inlet flow rate to Qin = 0.001 m3 /s,
and vary the reaction constant k. Figure 4.9 shows the results when k = 2 (black line) and
k = 0.5 (blue line). The steady-state concentration of A is greater when k = 0.5, because the
slower reaction consumes fewer moles of A during the residence time within the tank. The
slower reaction also has a longer half life, t1/2 = 3.99, compared to t1/2 = 1 for k = 2.
5. First Order Nonlinear ODEs

5.1 Equilibrium solutions 90

5.2 Separable ODEs 91

5.3 Exact ODEs (optional) 96

5.4 Behavior near equilibrium points 98

5.5 Application to chemical kinetics 98

Chapter 4 showed that we can typically solve first-order linear ODEs analytically (i.e. with
paper and pencil). At the worst, we may resort to a numerical method for approximating a
difficult integral. In contrast, first-order nonlinear ODEs can only be solved analytically in
special cases. These usually require that we can first write the ODE as

dy
= f (x, y), (5.1)
dx

where f (x, y) is some nonlinear function of x and y. Unfortunately, not all first-order ODEs
can be written in this form, such as that below
!
dy dy
+ y cos = x. (5.2)
dx dx

5.1 Equilibrium solutions

Suppose a nonlinear ODE can be written as

dy
= f (x, y). (5.3)
dx

An equilibrium solution ye of this ODE is any solution for which dy/dx is always zero. Con-
sider the ODE
dy
= 1 y2. (5.4)
dx

Setting 1 y 2 = 0, we see the ODE has the two equilibrium solutions

ye = ±1. (5.5)

90
 Chapter 5. First Order Nonlinear ODEs 91

It is important to stress that equilibrium solutions produce dy/dx = 0 for all x. For example,
the ODE
dy
= sin(x) (5.6)
dx

has no equilibrium solutions, because dy/dx is only zero at the discrete x values for which
sin(x) = 0. In contrast, the ODE
dy
= sin(y) (5.7)
dx

has the equilibrium solutions

ye = 0, ±⇡, ±2⇡, ±3⇡, ... (5.8)

Equilibrium solutions of first-order ODEs are constants, and consequently also often
called constant solutions. They are important for several reasons. For nonlinear ODEs, they
are sometimes the only solutions we can find without resorting to numerical methods. Equi-
librium solutions also often correspond to important real-world solutions. Finally, we can
often approximate the solution of a nonlinear ODE in the vicinity of an equilibrium solu-
tion. This plays a key role in all sorts of optimal control strategies used in everything from
robotics to fluid mechanics. We discuss that later in Section 5.4.

5.2 Separable ODEs

Separable ODEs can be written in the form

dy
= X(x)Y (y), (5.9)
dx

where X(x) is a function of only x, and Y (y) is a function of only y. For example, the following
functions are all separable,

y(y 2)
cos(x) sin(y), , y2, 1 + y + y2, (5.10)
x(y 1)

while the following are not,

x + y, ln(xy), x sin(y) + cos(y). (5.11)

When solving a separable ODE, it’s good practice to first identify any equilibrium solu-
tions. These occur when
Y (y) = 0. (5.12)
 92 5.2. Separable ODEs

For example, the separable ODE

dy y(y 2)
= , (5.13)
dx x(y 1)

has the equilibrium solutions

ye = 0, ye = 2. (5.14)

After identifying the equilibrium solutions, we rearrange the ODE as

1 dy
= X(x). (5.15)
Y (y) dx

This assumes that Y (y) , 0, which is fine because we are seeking solutions other than the
equilibrium solutions already identified. We then integrate the equation with respect to x
Z Z
1 dy
dx = X(x) dx, (5.16)
Y (y) dx

and use integration by substitution to express the left-hand-side as an equivalent integral

with respect to y,
Z Z
1
dy = X(x) dx. (5.17)
Y (y)

To demonstrate the procedure, let’s consider the following two examples.

Example 5.1 (Separable ODEs)

Solve the following di↵erential equation

dy x2
= , (5.18)
dx y

subject to the initial condition

y(0) = 1. (5.19)

Solution:

In this case Y (y) = 1/y, and there are no equilibrium solutions. We consequently pro-
ceed to rewrite the ODE as
dy
y = x2 , (5.20)
dx
 Chapter 5. First Order Nonlinear ODEs 93

and integrate both sides with respect to x,

Z Z
dy
y dx = x2 dx. (5.21)
dx

Using integration by substitution, we express the left-hand-side as

Z Z
y dy = x2 dx. (5.22)

Integrating, we find that

y 2 x3
= + C. (5.23)
2 3

Solving for y, we find that

r
2x3
y=± + 2C. (5.24)
3

The initial condition y(0) = 1 requires that we take the positive solution. We can also
replace “2C” above with simply “C,” because both are arbitrary constants,
r
2x3
y= + C. (5.25)
3

Finally, the initial condition requires C = 1

r
2x3
y= + 1. (5.26)
3

Note that when solving separable ODEs, people often treat dy/dx as a fraction and
rewrite the ODE as
1
dy = X(x)dx, (5.27)
Y (y)

which they then integrate as

Z Z
1
dy = X(x)dx. (5.28)
Y (y)

Though the logic is incorrect, this nevertheless produces the correct result. The next exam-
ple demonstrates the importance of considering equilibrium solutions, as these may be the
solutions selected by the initial/boundary conditions.
94 5.2. Separable ODEs

Example 5.2 (Separable ODEs)

Solve the following di↵erential equation

dy
= x2 y 2 . (5.29)
dx

Consider two possible initial conditions

y(0) = 1 and y(0) = 0. (5.30)

Solution:

We first note that the ODE has the equilibrium solution

ye = 0. (5.31)

We then rewrite the ODE as

1 dy
= x2 , (5.32)
y 2 dx

and integrate both sides with respect to x,

Z Z
1 dy
dx = x2 dx. (5.33)
y 2 dx

Using integration by substitution, we express the left-hand-side as

Z Z
1
dy = x2 dx. (5.34)
y2

Integrating, we find that

x3
1/y = + C, (5.35)
3

which can be solved for y as

3
y= , (5.36)
x3 + C

In summary, the ODE has the solutions

3
y=0 and y= . (5.37)
x3 + C
 Chapter 5. First Order Nonlinear ODEs 95

The initial condition y(0) = 1 can only be satisfied by the second solution using C = 3,

3
y= . (5.38)
x3 3

The initial condition y(0) = 0 can only be satisfied by the equilibrium solution y = 0.

In both examples above, our procedure produced explicit formulas for y in terms of
x. In other cases however, we can find only implicit solutions, i.e. a formula in which we
cannot isolate y. The next example demonstrates one of these cases. Methods of dealing with
implicit solutions for practical applications is beyond the time constraints of this course, but
can be found in many textbooks and online resources.

Example 5.3 (Separable ODEs)

Solve the following di↵erential equation

dy x
= . (5.39)
dx y + cos(y)

subject to the initial condition

y(0) = 1. (5.40)

Solution:
We first note that there are no equilibrium solutions. Using the usual procedure, we
rearrange and integrate the ODE,
Z Z
(y + cos y) dy = x dx, (5.41)

producing the solution

y2 x2
+ sin y = + C, (5.42)
2 2

where C is an arbitrary constant. We can express this solution as

y 2 + 2 sin y = x2 + C. (5.43)

Applying the initial condition y(0) = 1,

1 + 2 sin 1 = C, (5.44)

we find that
y 2 + 2(sin y sin 1) = x2 + 1. (5.45)
 96 5.3. Exact ODEs (optional)

5.3 Exact ODEs (optional)

A second class of first-order nonlinear ODEs (called exact ODEs) can be solved using the
chain rule for di↵erentiation. Consider the function (x, y) where y is itself a function of x,

( x, y(x) ). (5.46)

The chain rule states that

d @ @ dy
= + . (5.47)
dx @x @y dx

For example, consider

(x, y) = x2 y 2 , where y = sin(x). (5.48)

Then
d
= 2xy 2 + 2x2 y cos(x). (5.49)
dx

To see how we can use the chain rule to solve nonlinear ODEs, consider the example

dy sin y
= . (5.50)
dx 2y x cos y

We can rewrite this ODE as

dy
sin y + (2y x cos y) = 0. (5.51)
dx

The idea is to now find a function (x, y) so that this is equivalent to

d @ @ dy
= + = 0. (5.52)
dx @x @y dx

This requires
@ @
= sin(y), and = 2y x cos y. (5.53)
@x @y

The first equation above requires

(x, y) = x sin y + g(y), (5.54)

where g(y) is an arbitrary function of y. Substituting this result into the second constraint
 Chapter 5. First Order Nonlinear ODEs 97

produces
@ dg
= x cos(y) + = 2y x cos y. (5.55)
@y dy

That in turn requires

dg
= 2y, g = y 2 + C, (5.56)
dy

where C is an arbitrary constant. We conclude that the original ODE is equivalent to

d
= 0, (5.57)
dx

where
(x, y) = x sin y + y 2 + C. (5.58)

Integrating both sides of equation (5.57) with respect to x produces

x sin y + y 2 + C = C2 , (5.59)

where C2 is another arbitrary constant. Finally, we can combine the two constants to express
the solution as
y 2 x sin y = C, (5.60)

which is an implicit solution to the ODE.

To generalize the procedure, we first assume the ODE can be written as

dy M(x, y) dy
= or equivalently M(x, y) + N (x, y) = 0. (5.61)
dx N (x, y) dx

We then introduce a function (x, y) such that

@ @
= M(x, y), and = N (x, y). (5.62)
@x @y

If you can solve for , then the final solution takes the form

(x, y) = C. (5.63)

Unfortunately, a solution (x, y) does not always exist. To test whether it is worth seeking a
solution, we note that if exists and is well-behaved, then

@ @ @ @
= . (5.64)
@y @x @x @y
 98 5.4. Behavior near equilibrium points

In other words, the mixed derivative of with respect to x and y does not depend on the
order of di↵erentiation. This is equivalent to

@M @N
= . (5.65)
@y @x

In summary we say that an ODE of the form

dy M(x, y)
= (5.66)
dx N (x, y)

is an exact ODE if
@M @N
= . (5.67)
@y @x

5.4 Behavior near equilibrium points

5.5 Application to chemical kinetics

In Section 4.8.2 we modeled first-order reactions of the form A ! B and A ! B ! C us-
ing first-order linear ODEs. Here we consider second-order chemical reactions. One such
reaction is
A + A ! B, (5.68)

for which we define the reaction rate

1 dCA dCB
r= = . (5.69)
2 dt dt

The rate equation is given by

r = kCA2 , (5.70)

where k has units of L·mol 1 s 1 . The system is consequently governed by

dCA dCB
= 2kCA2 , = kCA2 . (5.71)
dt dt

The ODE for CA is a separable ODE that can be solved by rearranging and integrating
both sides with respect to t,
Z Z
1 dCA
dt = 2 k dt. (5.72)
CA2 dt
 Chapter 5. First Order Nonlinear ODEs 99

(a) (b)
1
C
A 1
C
B
0.8
0.5 CA
0.8
CB
0.6
0.6

0.4
0.4

0.2 0.2

0 0
0 5 10 15 20 0 1 2 3 4 5 6

Figure 5.1

This neglects the equilibrium solution CA = 0, which only occurs if the initial concentration
of CA is zero. Using integration by substitution, we find that
Z Z
1
dCA = 2 k dt. (5.73)
CA2

Which can be solved to show that

1
CA = , (5.74)
2kt + P

where P is an arbitrary constant. Applying the initial condition CA (0) = A0 , we find that

A0
CA = . (5.75)
2kA0 t + 1

Finally, we solve the ODE for CB

dCB kA20
= kCA2 = , (5.76)
dt (2kA0 t + 1)2

subject to the initial condition CB (0) = 0, producing

A0 A
CB (t) = + 0. (5.77)
4kA0 t + 2 2

b = C/A0 and
To investigate our solution, we define the non-dimensional concentration C
time t = kA0 t. We can then write the solution as
b

bA = 1 bB = 1 1
C , C + . (5.78)
2t + 1
b 4t + 2 2
b
 100 5.5. Application to chemical kinetics

Figure 5.1(a) shows the resulting variation of CbA and C bB . There are a few important di↵er-
ences from the results we obtained for the simpler first-order reaction A ! B considered in
Section 4.8.2. For comparison, that result is shown in figure 5.1(b). CB in panel (a) tends to
0.5 as t ! 1 while it tends to 1.0 in panel (b). This is because reaction A + A ! B produces
only one mole of B for every 2 moles of A. We also note that the curves in panel (a) asymptote
more slowly to their steady-state values than in panel (b). This is because reaction A + A ! B
produces transient terms that decay like 1/t, while A ! B produces transient terms that
decay exponentially.
6. Constant Coefficient Linear ODEs

6.1 Boundary conditions 102

6.2 Complex numbers and Euler’s formula 106

6.3 Hyperbolic functions 108

6.4 Linearly independent functions 108

6.5 Homogeneous ODEs 111

6.6 Nonhomogeneous ODEs 117

6.7 Resonance 122

6.8 Application to circuits 122

In earlier chapters, we have focused on first-order linear and non-linear ODEs. In this chap-
ter, we move on to solving higher-order ODEs. Specifically, we consider constant-coefficient,
linear, ODEs. These can be written as

d ny d n 1y dy
an + an 1 + ... + a1 + a0 y = f (x), (6.1)
dxn dxn 1 dx

where n is the order of the ODE, the constant coefficients are denoted by ai , and the depen-
dent variable y is a function of one independent variable x. Equation (6.1) is homogeneous
when f (x) = 0.

We begin in Section 6.1 by considering the simplest case, in which

d ny
= f (x). (6.2)
dxn

Using this case, we also explore the di↵erent types of boundary/initial conditions encoun-
tered when solving higher-order ODEs. In Section 6.5, we move on to second-order homo-
geneous ODEs of the form
dy 2 dy
a2 2
+ a1 + a0 y = 0. (6.3)
dt dt

The solution of these ODEs requires, however, some background theory on the concept of
linear dependence and complex numbers, which we discuss in Sections 6.2 and 6.4. Finally,
we discuss the solution of non-homogeneous ODEs in Section 6.6.

101
 102 6.1. Boundary conditions

6.1 Boundary conditions

The simplest constant-coefficient ODEs are of the

form (a)
d ny L
= f (x). (6.4)
dxn
x
These can be solved by repeatedly integrating the left
and right-hand-sides with respect to x, and then ap-
plying boundary/initial conditions to determine any (b)
arbitrary constants. For demonstration, let’s recon-
sider the problem of 1-D, steady, heat transport in a T
metal bar, as sketched in figure 6.1(a). The tempera-
ture field T (x) in the bar satisfies the ODE
x
d 2T
= 0. (6.5)
dx2 Figure 6.1

Integrating twice with respect to x produces the general solution

T = Ax + B, (6.6)

where A and B are arbitrary constants. We see that all solutions vary linearly with x, as
sketched in figure 6.1(b).
To determine the constants A and B in solution (6.6), we must apply two independent
boundary conditions. The four most common types of boundary conditions are called Dirich-
let, Neumann, Robin, and periodic boundary conditions. A Dirichlet condition fixes the value
of the dependent variable on one of the boundaries, i.e.

T (0) = a. (6.7)

For example, fixing the temperature at both ends of the bar produces the two Dirichlet con-
ditions
T (0) = TL , T (L) = TR . (6.8)

Applying these conditions requires

T (0) = B = TL ,
(6.9)
T (L) = AL + B = TR .

Solving, we find that B = TL and A = (TR TL )/L,

TR TL
T (x) = x + TL . (6.10)
L
 Chapter 6. Constant Coefficient Linear ODEs 103

A Neumann condition fixes the derivative of the dependent variable on a boundary, i.e.

dT
= a. (6.11)
dx x=0

Suppose, for example, that the heat flux exiting the bar on the right equals q. From Fourier’s
law, this requires
dT
K = q, (6.12)
dx x=L

where K is the thermal conductivity of the metal. This condition requires

KA = q. (6.13)

If we fix the temperature on the left of the bar as above, the final solution takes the form

q
T (x) = x + TL . (6.14)
K

For the current example, the following five combinations of Dirichlet and Neumann con-
ditions at x = 0 or L are sufficient to solve for A and B:

1. Dirichlet at x = 0 and x = L

T (0) = a, T (L) = b.

2. Dirichlet at x = 0 and Neumann x = L

dT
T (0) = a, =b
dx x=L

3. Neumann at x = 0 and Dirichlet x = L

dT
= a, T (L) = b.
dx x=0

4. Neumann and Dirichlet both at x = 0

dT
= a, T (0) = b.
dx x=0

5. Neumann and Dirichlet both at x = L

dT
= a, T (L) = b.
dx x=L
 104 6.1. Boundary conditions

It is important to stress, however, that not all random combinations of Neumann and Dirich-
let conditions produce valid solutions. Consider the following combination of two Neumann
conditions,
dT dT
= a, = b. (6.15)
dx x=0 dx x=L

This combination attempts to force a straight line to have two di↵erent slopes. The only
valid combination occurs when a = b, but in this case, we can determine only the coefficient
A and the constant B remains unknown.

Now suppose that heat generation within the bar causes the temperature to satisfy the
ODE
d 2T
= 1. (6.16)
dx2

Integrating twice with respect to x, we find that

dT x2
= x + A, T (x) = + Ax + B. (6.17)
dx 2

The temperature now varies non-linearly, and can accommodate di↵erent values of dT /dx at
the extremities. Yet, if we attempt to apply the Neumann conditions (6.15), we find that

dT dT
= A = a, = L + A = b. (6.18)
dx dx
x=0 x=L

We see that two the Neumann conditions require A = a and A = b L. This is only compatible
if
a = b L. (6.19)

Physically, this compatibility condition arises because the two Neumann conditions fix the
heat conduction entering the left and right-hand-sides of the bar. The di↵erence in these
heat transfers must balance the heat generated within the bar. Otherwise, the boundary
conditions violate the conservation of energy principles that we used to derive the ODE.

In addition to the Dirichlet and Neumann condition, a third common condition is the
Robin condition, which is a linear combination of the Dirichlet and Neumann conditions,

dT
↵ + T = a, (6.20)
dx x=x0
x=x0

where ↵ and are constant coefficients. For the case of heat transport in a metal bar, this
condition arises when the rate of heat loss from the metal bar at x = 0 or x = L is proportional
 Chapter 6. Constant Coefficient Linear ODEs 105

to the di↵erence in the temperatures between the bar and its surroundings (Ts ),

✓ ◆
dT
K = T Ts . (6.21)
dx x=x0
x=x0

The above is known as “Newton’s law of cooling.”

Finally, some physical systems produce periodic boundary conditions. As an example,
consider the ODE
d 3y
= cos(x), (6.22)
dx3

where y is periodic over the interval 0  x  2⇡. Integrating the ODE three times, we find
that
y = sin(x) + Ax2 + Bx + C. (6.23)

Periodicity requires

y(0) = y(2⇡), y 0 (0) = y 0 (2⇡), y 00 (0) = y 00 (2⇡). (6.24)

After a little algebra, this produce A = B = 0, but C remains unknown,

y = sin(x) + C, (6.25)

because any constant value C is indeed periodic over the interval. To determine C, additional
information would need to be provided.

Summary of important concepts:

1. An ODE of order n typically requires n initial/boundary conditions to determine

the resulting coefficients of integration.

2. The four most common conditions are Dirichlet, Neumann, Robin, and periodic
conditions.

3. The initial/boundary conditions should also be consistent with the underlying

physics that led to the ODE. Randomly applying n conditions can otherwise lead
to incompatibilities.

4. In the case of periodic conditions, it may not be possible to solve for all constants
without additional information being provided.
 106 6.2. Complex numbers and Euler’s formula

6.2 Complex numbers and Euler’s formula

Complex numbers are numbers of the form

z = a + ib, (6.26)

p
where a and b are real numbers, and i = 1 is called the imaginary unit. For interested
readers, there is a book dedicated to the history of complex numbers [1], as well as a nice lec-
ture by the physicist Richard Feynman.* Complex numbers arose as mathematicians grap-
pled with how to solve quadratic and cubic equations. An early mention dates to Heron of
Alexandria in the first century AD, but it wasn’t until the 16’th and 17’th centuries that they
were more seriously considered. The term “imaginary” was coined by René Descartes in the
1600’s, who used it as a derogatory term, because it was unclear how to interpret complex
numbers geometrically.
Let’s take the approach of Feynman, and present imag-
inary numbers as a gradual abstraction of our concept of negative
integers
positive
integers
numbers. Suppose we already know of the existence of

{
{
zero and the positive integers. This let’s us solve equations 3 2 1 0 1 2 3
such as x = 3 + 5. However, if we try to solve the equation
x = 3 5, we discover we cannot express the answer with Figure 6.2
our current number system. We consequently invent the
concept of “negative integers.” To interpret these new integers geometrically, we could plot
the axis in figure 6.2 on which we place zero in the middle, positive integers on the right,
and negative integers on the left. We denote the integers with tick marks, all separated by
an equal distance, which we set to the unit integer 1. Eventually, however, we try to solve
the equation x = 3/5, and again discover that our system of numbers is not complete. We
consequently invent the concept of rational numbers, which are numbers that can be writ-
ten as x = n/d, where n and d are integers. Things go well, until we try to compute the ratio
between a circle’s circumference and diameter, at which point we discover the answer (⇡)
cannot be expressed as a rational number. We consequently once again expand our concept
of numbers to irrational numbers.
Finally, we one day try to model a real-world system that produces the equation x2 =
1. In some cases, such equations correspond to unphysical solutions that can simply be
ignored. Counter-intuitively, however, such equations are also often physical, and cannot be
ignored. We consequently need to once again p expand our concept of numbers. To deal with
these types of equations, we define i = 1. Any pure multiple of i, such as 7i, is called
a pure imaginary number. To interpret these numbers geometrically, we invent the complex
plane sketched in figure 6.3(a). The plane consists of two axes called the real and imaginary
axes. We put all real numbers on the horizontal axis, which is simply an abstraction of
the axis we drew earlier in figure 6.2. On this axis, we imagine that real non-integers lie
between the integer tick marks. We find no way, however, to imagine how a pure imaginary
number could lie on the real axis. We consequently put such numbers on an entirely new
axis perpendicular to the real axis. By convention, we often label the real and imaginary
axes as x and iy respectively. We then interpret a complex number z = a + ib as a vector in
the complex plane.

* available at www.feynmanlectures.caltech.edu
 Chapter 6. Constant Coefficient Linear ODEs 107

With the extension to complex

(a) (b)
numbers, mathematicians needed to iy iy
also extend the various mathemati-
cal operations, such as addition, sub- b 4
traction, and multiplication. The

imaginary
z2
complex plane allows us to interpret

axis
2
addition and subtraction as vector z
z1
operations. Figure 6.3(b) shows the z3

example of z3 = z1 + z2 , where z1 = a
x
3 5
x
real axis
3 + 4i and z2 = 2 2i. One particu-
Figure 6.3
larly important operation is the ex-
ponential of a complex quantity,
ex+iy . (6.27)

To interpret this operation, we use a Taylor series to show that

!
x+iy x iy x y2 y3 y4
e = e e = e 1 + iy i + ... . (6.28)
2 3! 4!

This can be re-ordered as

" ! !# " #
x+iy x y2 y4 y3 y5 x
e =e 1+ + ... + i y + ... = e cos(y) + i sin(y) . (6.29)
2 4! 3! 5!

Consequently, we conclude that

eiy = cos(y) + i sin(y), (6.30)

which is called Euler’s formula.

Rather than expressing a complex number as z = a + ib, Euler’s formula shows we can
also express complex numbers as

iy
z = rei✓ , (6.31)
b
where r and ✓ are the polar coordinates sketched in Fig. 6.4.
Using Euler’s formula, we see that
r
z = rei✓ = r cos(✓) + ir sin(✓). (6.32)
θ
x
We conclude that a

p Figure 6.4
a = r cos(✓), b = r sin(✓), r = a2 + b 2 . (6.33)
 108 6.3. Hyperbolic functions

With the acceptance of complex numbers, we also create

a new operation called complex conjugation. The operation iy
simply changes the sign of the imaginary part of a complex
number. For example, if z = a + ib, then the complex complex b
conjugate, denoted z, is defined as z

z = a ib. (6.34) x
a
z
Figure 6.5 shows the complex conjugate graphically. One prac-
tical use of the complex conjugate is the relation b

zz = (a + ib)(a ib) = a2 + b2 = r 2 . (6.35) Figure 6.5

We see that complex conjugation can be used to obtain r. Another practical use when ma-
nipulating complex numbers is the following. Suppose you want to compute the real and
imaginary parts of the ratio
1
z= . (6.36)
2 + 3i

To extract the real and imaginary parts of z, we multiply the numerator and denominator by
the complex conjugate of the denominator, i.e.

1 2 3i 2 3i 2 3
z= · = 2 2
= i . (6.37)
2 + 3i 2 3i 2 + 3 13 13

We conclude that the real and imaginary parts are 2/13 and 3i/13, respectively.

6.3 Hyperbolic functions

6.4 Linearly independent functions

Linear dependence and independence describe how a set of functions do or do not relate
to one another. Being able to correctly identify whether a set of functions is dependent or
independent plays a key role in solving di↵erential equations of second-order or higher.

Definition 6.1 (Linear Dependence)

A set of functions { f 1 (x), f 2 (x), ..., f n (x) } is linearly dependent if at least one function
can be written as a linear combination of the others. Otherwise, the set is linearly
independent.

Consider, for example, the set of functions

{2, y, y2, y3, y + 2y 2 }. (6.38)

This set is linearly dependent, because the function y + 2y 2 is a linear combination of two
 Chapter 6. Constant Coefficient Linear ODEs 109

other functions in the set. In contrast, the set

{y 2 , y4, y 6 }, (6.39)

is linearly independent, because no function is a linear combination of the others. Finally,

consider the set
{ex , e x, sinh(x)}. (6.40)

This set is linearly dependent because sinh(x) is a linear combination of ex and e x .

For the applications considered in this textbook, it is usually straightforward to identify

whether a set of functions is linearly dependent or independent by inspection. For com-
pleteness, however, we note that there are more formal methods of testing whether a set is
dependent or not. For that purpose, we first note the following theorem.

Theorem 6.1 (Linear dependence)

A set of functions { f 1 (x), f 2 (x), ..., f n (x) } is linearly dependent on some interval a  x 
b if there exists a set of scalars ↵j , not all zero, for which

↵1 f 1 + ↵2 f 2 + ... + ↵n f n = 0, (6.41)

for all x in the interval. If the above equation is only true when all ↵j = 0, then the set
is linearly independent.

As an example, let’s reconsider set of functions

{2, y, y2, y3, y + 2y 2 }. (6.42)

This set is linearly dependent because

↵1 2 + ↵2 y + ↵3 y 2 + ↵4 y 3 + ↵5 (y + 2y 2 ) = 0, (6.43)

for all x when

↵1 = 0, ↵2 = 1, ↵3 = 2, ↵4 = 0, ↵5 = 0. (6.44)

In contrast, the set

{y 2 , y4, y 6 }, (6.45)

is linearly independent, because

↵1 y 2 + ↵2 y 4 + ↵3 y 6 (6.46)
 110 6.4. Linearly independent functions

is only zero for all x when

↵1 = ↵2 = ↵3 = 0. (6.47)

Now consider that if a linear combination of functions is zero for all x,

↵1 f 1 + ↵2 f 2 + ... + ↵n f n = 0, (6.48)

then the derivative with respect to x must also equal zero,

↵1 f 10 + ↵2 f 20 + ... + ↵n f n0 = 0, (6.49)

as well as the second derivative

↵1 f 100 + ↵2 f 200 + ... + ↵n f n00 = 0, (6.50)

and all subsequent higher derivatives. This provides us n equations for the n unknown
coefficients as follows. Suppose the set has three functions,

{ f 1 (x), f 2 (x), f (3) }. (6.51)

The set is linearly dependent if the system of equations

2 32 3
66 f 1 (x) f 2 (x) f 3 (x) 77 66↵1 77
66 0 76 7
66 f 1 (x) f 20 (x) f 30 (x) 777 666↵2 777 = 0 (6.52)
64 00 75 64 75
f 1 (x) f 200 (x) f 300 (x) ↵3

has a non-trivial solution† . In linear algebra, it can be shown that the system only has a
non-trivial solution if the determinant of the leading matrix equals zero,

f 1 (x) f 2 (x) f 3 (x)

f 10 (x) f 20 (x) f 30 (x) = 0. (6.53)
f 100 (x) f 200 (x) f 300 (x)

On the other hand, if the determinant is non-zero, the system only has the trivial solution,
and the set is consequently linearly independent. We call the determinant above the Wron-
skian. More generally, if the set has n functions, the Wronskian is defined as

f 1 (x) f 2 (x) ... f n (x)

f 10 (x) f 20 (x) ... f n0 (x)
.. .. .. .. . (6.54)
. . . .
f 1n 1 (x) f 2n 1 (x) ... f nn 1 (x)

† The trivial solution is the solution for which all ↵ equal zero. A non-trivial solution is a solution for which
j
not all ↵j are zero.
 Chapter 6. Constant Coefficient Linear ODEs 111

Example 6.1
Apply the Wronskian to determine if y 2 , y 4 , y 6 is linearly dependent.

Solution: Building the Wronskian, we find that

y2 y4 y6
4y 3 6y 5 2y 6y 5 6 2y 4y 3
2y 4y 3 6y 5 = y 2 y4 + y (6.55)
12y 2 30y 4 2 30y 4 2 12y 2
2 12y 2 30y 4

= 76y 9 60y 8 . (6.56)

Because the Wronskian is not zero, the set is linearly independent.

Example 6.2
Apply the Wronskian to determine if y, y 2 , y y 2 is linearly dependent.

Solution: Building the Wronskian, we find that

y y2 y y2
2y 1 2y 1 1 2y 1 2y
1 2y 1 2y = y y2 + (y y2) (6.57)
2 2 0 2 0 2
0 2 2

= 0. (6.58)

Because the Wronskian is zero, the set is linearly dependent.

6.5 Homogeneous ODEs

The concept of linear independence is key to solving ODEs due to the following theorem.

Theorem 6.2 (Higher order linear homogeneous ODEs)

Linear n’th-order homogeneous ODEs can be written in the form

d ny d n 1y dy
an (x) n
+ a n 1 (x) n 1
+ ... + a1 (x) + a0 (x)y(x) = 0. (6.59)
dx dx dx

If the coefficients an (x), an 1 (x), ... , a0 (x) are all continuous on an interval x0  x  x1 ,
then equation (6.59) admits n linearly independent solutions y1 (x), y2 (x), ... , yn (x),
and the general solutions can be written as

y(x) = C1 y1 (x) + C2 y2 (x) + ... + Cn yn (x). (6.60)

 112 6.5. Homogeneous ODEs

In the current section, we apply this theorem to solve constant coefficient, homogeneous
ODEs of the form
d 2y dy
a +b + cy = 0, (6.61)
dx2 dx

where a, b, and c are constants. The approach is equally valid, however, for constant-
coefficient ODEs of any order. To motivate the approach let’s first consider the special cases

dy d 2y d 2y dy
= y, = y, = , (6.62)
dx dx2 dx 2 dx

where is some constant. Functions whose first or second derivatives are proportional to
the function itself include exponential (ex ), sinusoidal (sin x, cos x), and hyperbolic functions
(cosh x, sinh x). These functions can all be written in terms of exponential functions as

eix + e ix eix e ix
cos(x) = , sin(x) = , (6.63)
2 2i

eax + e ax eax e ax
cosh(ax) = , sinh(ax) = , (6.64)
2 2

eiax = cos(ax) + i sin(ax), e ax = cosh(ax) + sinh(ax). (6.65)

This suggests that we seek solutions to the general ODE (6.61) in the form

y = e x, (6.66)

where is an unknown constant. Substituting this form into the general ODE, we find that

2
a + b + c = 0. (6.67)

If this quadratic equation produces two distinct roots, 1 and 2, Theorem 6.2 above guar-
antees that the general solution to the ODE can be written as

y = C1 e 1x + C2 e 2x . (6.68)

To demonstrate the procedure, let’s consider the

damped oscillator composed of a mass m attached to x
a spring and dampener as sketched in figure 6.6. If
m
the mass is displaced a distance x from its equilib-
rium position, the spring will exert a force equal to
kx, where k is the spring constant. Any subsequent Figure 6.6
motion of the mass is also resisted by the dampener,
which applies a force c(dx/dt), where c is called the damping constant. Newton’s second law
 Chapter 6. Constant Coefficient Linear ODEs 113

can be applied to show that the motion of the mass satisfies the ODE,

d 2x dx
m +c + kx = 0. (6.69)
dt 2 dt

The system also has two initial conditions,

dx
x t=0 = x0 , = U0 , (6.70)
dt t=0

where the first prescribes the initial position, and the second prescribes the initial velocity.
In the following discussion, we assume the constants m, k, and c are all positive numbers, be-
cause that is the case for physical damped oscillators. For other real-world systems, however,
one or more of the constants could be negative.

Seeking a solution to equation (6.69) in the form x(t) = e t produces an equation for ,

2
m + c + k = 0. (6.71)

This is called the characteristic equation. Applying the quadratic formula, we find the two
roots
p p
c + c2 4mk c c 2 4mk
1= , 2= . (6.72)
2m 2m

The ODE consequently has the general solution

x(t) = c1 e 1t + c2 e 2t , (6.73)

where c1 and c2 are constants determined by the initial conditions. There are several cases
that arise from equation (6.73):

• Case 1: c = 0, two purely imaginary roots.

• Case 2: c2 4mk > 0, two real roots.

• Case 3: c2 4mk < 0 and c , 0, two complex roots.

• Case 4: c2 4mk = 0, repeated roots.

Each of these cases create a unique final form for our solution. In the following subsections
we provide more details.
 114 6.5. Homogeneous ODEs

6.5.1 Case 1: Purely imaginary roots

Purely imaginary roots occur when c = 0,

���
r
k ���
= ±i , (6.74)
m � ���

p - ���
where i = 1. This produces the general solution
- ���
pk pk � �
�
�� ��
i mt i mt
x(t) = c1 e + c2 e . (6.75)
Figure 6.7

Instead of writing the solution in this form, it is usually more convenient to write the solution
in terms of sines and cosines as follows. First, we use the identity e iax = cos(ax) + i sin(ax)
(called the Euler formula) to write equation (6.75) as
" # " #
x(t) = c1 cos (!t) + i sin (!t) + c2 cos ( !t) + i sin ( !t) , (6.76)

p
where ! = k/m. Next, we recall that cos( t) = cos(t) and sin( t) = sin(t) to write this as

x(t) = (c1 + c2 ) cos (!t) + (ic1 ic2 ) sin (!t) . (6.77)

Finally, we define new constants A1 = c1 + c2 and A2 = i(c1 c2 ), so that

x(t) = A1 cos (!t) + A2 sin (!t) . (6.78)

A unique solution can then be obtained by solving for A1 and A2 using the initial conditions.
An example plot is shown in figure 6.7.

Conclusion: if you have two imaginary roots of the form = ±i!, the general solution
can be written as
x(t) = A1 cos (!t) + A2 sin (!t) . (6.79)

6.5.2 Case 2: Two real roots

When c2 4mk > 0, both roots are real, and the solution can be written as

x(t) = c1 e 1t + c2 e 2t . (6.80)

For a general second order ODE, these could lead to exponential growth or decay. For a
damped oscillator, however, both roots will be negative, such that the motion is an exponen-
tial decay back to the equilibrium position (x = 0) due to the dampener, as demonstrated in
figure 6.8.
 Chapter 6. Constant Coefficient Linear ODEs 115

Occasionally, when solving a second order ODE,

the characteristic equation can produce roots of the ���

form ���

���
= ±↵, (6.81) �
���

���
where ↵ is a real number. In that case, we can use
a procedure similar to that used in Section 6.5.1 to ���
� � �� ��
write the solution in terms of cosh(↵x) and sinh(↵x). �

Specifically, we can use the identity e ax = cosh(ax) + Figure 6.8

sinh(ax) to write the solution as

x(t) = A1 cosh(↵t) + A2 sinh(↵t), (6.82)

where A1 and A2 are new constants, generally not equal to c1 and c2 .

Conclusion: if you have two real roots of the form = ±↵, the general solution can be
written as
x(t) = A1 cosh (↵t) + A2 sinh (↵t) . (6.83)

6.5.3 Case 3: Complex roots

The characteristic equation will produce two com-

plex roots when c2 4mk < 0 and c , 0. Suppose we ���

write the roots as ���

���

± = ↵ ±i . (6.84) � ���

���

This produces the solution ���

- ���
� � �� ��
x(t) = c1 e↵+i t
+ c 2 e↵ i t
�
(6.85)
= e ↵t (c1 ei t
+ c2 e i t
). Figure 6.9

Using the identity e iat = cos(at) + i sin(at), we can write this as

" #
↵t
x(t) = e A1 cos( t) + A2 sin( t) . (6.86)

For an oscillator, this produces an oscillatory motion with an amplitude that decays in time,
as demonstrated in figure 6.9.
 116 6.5. Homogeneous ODEs

6.5.4 Case 4: Repeated roots

p
When the quantity c2 4mk = 0, one obtains the re-
���
peated root
���
c
1= 2= , (6.87)
2m �
���

���

which produces the solution ���

���
� � �� ��
x(t) = C1 e 1t . (6.88) �
Figure 6.10

This solution has only one coefficient of integration (c1 ), and is not sufficiently general to
satisfy two initial conditions.
Repeated roots occur when there is a solution to the second-order ODE (6.69) of the form

x(t) = f (t)e t . (6.89)

This solution cannot be found by seeking solutions of the form x = e t . To find this second
solution, we substitute equation (6.89) into the ODE (6.69),
" # " #
d2 t d t t
m 2 f (t)e + c f (t)e + kf (t)e = 0, (6.90)
dt dt

which simplifies to
d 2f df ⇣ 2
⌘
m 2
+ (2m + c) + m + c + k f = 0. (6.91)
dt dt

Now using the fact that = c/2m, this further simplifies to

d 2f 1 ⇣ 2 ⌘
m c 4mk f = 0. (6.92)
dt 2 4m

Recall, however, that the repeated root occurs when c2 4mk = 0. Consequently, this further
simplifies to
d 2f
= 0, (6.93)
dt 2

which requires
f = C2 t + C3 , (6.94)

where C2 and C3 are constants of integration. Adding our two solutions, we now have

x(t) = C1 e t + (C2 t + C3 ) e t
= (C1 + C3 + C2 t) e t , (6.95)
 Chapter 6. Constant Coefficient Linear ODEs 117

which can be written as

x(t) = (A1 + A2 t) e t , (6.96)

where the two constants A1 and A2 allow us to satisfy the two initial conditions. An example
solution is shown in Figure 6.10.
For second-order ODEs, the above discussion is sufficient. For third-order and higher,
the characteristic equation can produce a root of multiplicity k, i.e. an equation of the form

( )k = 0. (6.97)

In that case, it can be shown that the repeated root produces the k independent solutions

e t, te t , t2e t , ...t k 1 e t . (6.98)

6.6 Nonhomogeneous ODEs

To solve non-homogeneous linear ODEs, we leverage the following theorem.

Theorem 6.3 (Non-homogeneous ODEs)

Linear n’th-order non-homogeneous ODEs can be written in the form

d ny d n 1y dy
an (x) n
+ a n 1 (x) n 1
+ ... + a1 (x) + a0 (x)y(x) = f (x). (6.99)
dx dx dx

The general solution to this ODE can be written as

y(x) = yh (x) + yp (x), (6.100)

where yh (x) is the solution to the corresponding homogeneous ODE, and yp is any
particular solution to the non-homogeneous ODE.

This theorem allows us to solve linear non-homogeneous ODEs by first solving the cor-
responding homogeneous ODE using the techniques demonstrated in Section 6.5. If we can
then find any particular solution, even if by trial-and-error, we have e↵ectively solved the
full ODE using Theorem 6.3. Two common approaches to seeking particular solutions are
the method of variation of parameters and method of undetermined coefficients. For brevity, we
only demonstrate the latter, and refer readers to any good textbook on ODEs for the former.
The method of undetermined coefficients only works for linear constant-coefficient ODEs
for which repeated di↵erentiation of the forcing function f (x) with respect to x produces a
finite family of linearly independent functions. Consider the forcing function

f (x) = x2 . (6.101)
 118 6.6. Nonhomogeneous ODEs

Repeated di↵erentiation with respect to x produces the finite family of linearly independent
functions
x2 , 2x, 2. (6.102)

Now consider the forcing function

xekx . (6.103)

We see that repeated di↵erentiation of f (x),

df d 2f
= ekx + xkekx , = kekx + kekx + xk 2 ekx , (6.104)
dx dx2

produces the finite family of linearly independent functions

xex , ex . (6.105)

Finally, consider the forcing function

f (x) = ln x. (6.106)

Repeated di↵erentiation produces the infinite family of functions

1 1 1 1
, , , , ... (6.107)
x x2 x3 x4

The method of undetermined coefficients is consequently not applicable to this forcing func-
tion.

To demonstrate the method of undetermined coefficients, consider the ODE

d 4y d 2y
+ 4 = ex , (6.108)
dx4 dx2

subject to the four boundary conditions

dy dy
y(0) = 0, = 0, y(1) = 0, = 0. (6.109)
dx x=0 dx x=1

We begin by solving the corresponding homogeneous ODE

d 4 yh d 2 yh
+ 4 = 0, (6.110)
dx4 dx2

by seeking solutions in the form yh ⇠ e x , producing the characteristic equation

2 2
( + 4) = 0. (6.111)
 Chapter 6. Constant Coefficient Linear ODEs 119

This has the roots

1,2 = 0, 3,4 = ±2i. (6.112)

The repeated roots 1,2 produce the solution

yh = C1 + C2 x, (6.113)

while roots 3,4 produce the solution

yh = C3 sin(2x) + C3 cos(2x). (6.114)

The net homogeneous solution is consequently

yh = C1 + C2 x + C3 sin(2x) + C3 cos(2x). (6.115)

To find a particular solution, we first note that repeated di↵erentiation of the forcing
function produces only one function ex . We consequently seek a particular solution in the
form
y = Aex , (6.116)

where A is an unknown constant. Substituting this form in the full ODE, we find that

5Aex = ex . (6.117)

This requires A = 1/5. The general solution is consequently

1
y = C1 + C2 x + C3 sin(2x) + C4 cos(2x) + ex . (6.118)
5

Applying the boundary conditions, we find that

1
y(0) = C1 + C4 + 5 = 0,
1
y 0 (0) = C2 + 2C3 + 5 = 0,
⇡ 1 ⇡/2
(6.119)
y(⇡/2) = C1 + 2 C2 C4 + 5e = 0,
1 ⇡/2
y 0 (⇡/2) = C2 2C3 + 5e = 0.

After some algebra, we find that

(⇡ 4) 1 ⇣ ⌘
C1 = C2 , C2 = 1 + e⇡/2 , (6.120)
2 10
1 ⇣ ⌘ 1
C3 = 3 + e⇡/2 , C4 = C1 . (6.121)
20 5
 120 6.6. Nonhomogeneous ODEs

Now suppose we replace the right-hand-side of equation (6.108) with

d 4y d 2y
4
+ 4 2
= ex + cos(x). (6.122)
dx dx

In this case, we can express the general solution as

y = yh + yp1 + yp2 , (6.123)

where yh and yp1 are the homogeneous solution and particular solution already determined
above, and yp2 is any particular solution to

d 4y d 2y
+ 4 = cos(x). (6.124)
dx4 dx2

To find yp2 we repeatedly di↵erentiate cos x, and find this produces the family of linearly
independent functions
{ cos(x), sin(x) }. (6.125)

We consequently seek a particular solution in the form

yp2 = A cos(x) + B sin(x), (6.126)

where A and B are undetermined coefficients. Substituting this form into ODE (6.124), we
find that
(A cos x + B sin x) + 4 ( A cos x B sin x) = cos(x). (6.127)

Collecting like terms, we find that

( 3A 1) cos x + ( 3B) sin x = 0. (6.128)

Because sin(x) and cos(x) are linearly independent, the above equation can only be true for
all x if
3A 1 = 0, 3B = 0. (6.129)

We conclude that A = 1/3 and B = 0. The general solution is consequently

1 1
y = C1 + C2 x + C3 sin(2x) + C4 cos(2x) + ex cos x. (6.130)
5 3
 Chapter 6. Constant Coefficient Linear ODEs 121

Now suppose we consider the ODE

d 4y d 2y
+ 4 = cos(2x). (6.131)
dx4 dx2

Repeatedly di↵erentiating the forcing function suggests we seek a particular solution in the
form
yp = A cos(2x) + B sin(2x). (6.132)

Substituting this form into the ODE (6.131), we find that

" # " #
16A cos(2x) + 16B sin(2x) + 4 4A cos(2x) 4B sin(2x) = cos(2x). (6.133)

The left-hand-side of the ODE then vanishes, producing

0 = cos(2x), (6.134)

which is not true for all x. This issue arose because sin(2x) and cos(2x) both appear in the
homogeneous solution of the ODE. We deal with this issue by seeking a solution in the form

yp = Ax cos(2x) + Bx sin(2x). (6.135)

As a final example, consider the ODE

d 4y d 2y
+ 4 = x2 . (6.136)
dx4 dx2

Repeated di↵erentiation of the forcing function suggests we seek a solution in the form

yp = A + Bx + Cx2 . (6.137)

Unfortunately, we see that A and Bx are already present in the homogeneous solution. If we
multiply the solution by x,
yp = Ax + Bx2 + Cx3 , (6.138)

we see that Ax is still present in the homogeneous solution. We consequently multiply the
solution form by x again, producing

yp = Ax2 + Bx3 + Cx4 . (6.139)

Substituting this into ODE (6.136), we find that

 122 6.7. Resonance

We can now summarize the method of undetermined coefficients as follows. Suppose

you want to solve the constant-coefficient non-homogeneous ODE:

d ny d n 1y
an + an 1 + ... + a0 y = f 1 (x) + f 2 (x) + ... + f k (x). (6.140)
dxn dxn 1

First solve the homogeneous problem for yh (x). Then for each forcing function f j (x),
perform the following steps.

• Verify that repeated di↵erentiation of the forcing function produces a finite lin-
early independent family of functions

{ g1 (x), g2 (x), ... , gm (x) }. (6.141)

• Seek a particular solution in the form

yp (x) = A1 g1 (x) + A2 g2 (x) + ... + Am gm (x). (6.142)

• Check for duplication between the functions gj (x) and the homogeneous solu-
tion yh (x). If there is any duplication, repeatedly multiply yp (x) by x until the
duplication is removed.

• Substitute yp into the ODE and solve for the undetermined coefficients.

After repeating this procedure for each forcing function, the general solution can be
written as
y(x) = yh (x) + yp1 (x) + yp2 (x) + ... + ypk (x). (6.143)

As a final step, apply the boundary/initial conditions.

6.7 Resonance

6.8 Application to circuits

In this section we discuss applications of constant coeffi-

cient second-order ODEs with respect to circuits. These
types of systems tend to arise when there are resistors
along with two or more storage elements present in the
circuit, such as inductors or capacitors. Previously, we
briefly introduced both resistors and inductors and their
relationships between voltage (v) and current (i). As a re-
minder the current through a resistor, with resistance R, Figure 6.11
 Chapter 6. Constant Coefficient Linear ODEs 123

is given by Ohm’s law,

v
i= . (6.144)
R

Since the inductor resists changes in the rate of current, the derivative

di v
= (6.145)
dt L

describes the voltage and current relationship for an inductor with inductance L. Capacitors
on the other hand have not yet been discussed. The purpose of a capacitor is to passively
store energy in an electric field. The voltage across a capacitor is

Q
v= , (6.146)
C

where Q is the charge across the capacitor and C is the capacitance. The charge can be found
by integrating the current over time,
Z t
Q= i(t) dt. (6.147)
0

Now that we have introduced the capacitor as a building block, let’s consider the RLC
circuit in figure 6.11. According to Kirchho↵’s law, the sum of voltage drops across the com-
ponents in our closed circuit must equal the applied voltage. Rearranging equations (6.144),
(6.145), and (6.146) for their respective voltage drops and adding them together we get,
Z t
di 1
L + iR + i(t) dt = v(t), (6.148)
dt C 0

where we substituted our definition for Q so that we can write each term on the left hand
side in terms of our dependent variable i. Due to the integral in equation (6.148), we do not
yet have an ODE that we can solve using the methods discussed in this chapter. To fix this,
we di↵erentiate both sides to obtain

d 2i di 1
L 2
+ R + i = v 0 (t). (6.149)
dt dt C

To solve equation (6.149), let’s first consider that v(t) = 0 and that our current is driven
by some initial charge on the capacitor. In this case v 0 (t) = 0 and we have a homogeneous
ODE. To determine the appropriate solution to this ODE, we will solve for the roots of the
characteristic polynomial,
1
Lr 2 + Rr + = 0. (6.150)
C

With some simplification, the result is

s
!2
R R 1
r= ± . (6.151)
2L 2L LC
 124 6.8. Application to circuits

Figure 6.12: Current responses for an RLC circuit. (a) Overdamped, (b) underdamped, and (c) criti-
cally damped cases depend upon the relationship between the damping factor and resonant frequency.

p
For simplification purposes we will use the substitutions ↵ = R/2L and !o = 1/ LC. This
allows us to rewrite the roots of our ODE as
q
r = ↵± ↵ 2 !o2 . (6.152)

The terms that ↵ and !o represent frequently show up in circuits problems and therefore
have been given special names. More specifically, ↵ is referred to as the damping factor and
!o is called the resonant frequency. The damping factor provides insight into how quickly
the overall signal would fade to zero without an applied voltage. The resonant frequency
describes how the current would naturally oscillate without any additional applied oscilla-
tions due to a voltage source. The relationship between these two terms will determine the
form of our solution, as we will show below.
If we consider physical cases in which the resistance is nonzero then our roots may end
up being complex, real and distinct, or real and repeated based on the relative values of ↵
and !o . Let’s consider the following three relationships

↵ > !o , ↵ = !o , ↵ < !o . (6.153)

According to the cases discussed in Section 6.5, these three relationships produce di↵erent
general solutions. When ↵ > !o the characteristic polynomial results in two real and distinct
roots. Therefore, the general solution takes the form

i(t) = Ae r1 t + Ber2 t , (6.154)

where A and B are constants and require two initial conditions to solve. Figure 6.12(a) shows
a sketch of what this general solution looks like. We can see that the current decays to zero
without any oscillations. This is referred to as an overdamped circuit. Since the damping
factor is higher than the resonant frequency, the natural oscillations are not observed. In
other words, the period between oscillations that would have been observed is shorter than
the time it takes for the current to decay to zero.
 Chapter 6. Constant Coefficient Linear ODEs 125

When ↵ < !o , the roots of the characteristic polynomial are complex. If we write the
roots as
r1 = ↵ + i , r2 = ↵ i , (6.155)

p
where = ↵ 2 !o2 then the general solution can be written as
⇣ ⌘
↵
i(t) = e A cos( t) + B sin( t) (6.156)

where A and B are again constants. In comparison to our overdamped solution, the family
of solutions that this expression generates is significantly di↵erent. An example is sketched
in figure 6.12(b). Here, we can clearly see the natural oscillations in the current along with
its slow decay to zero. Perhaps unsurprisingly, this is referred to as an underdamped circuit.
Compared to the overdamped circuit, an underdamped circuit has a natural frequency that
oscillates with a period that is shorter than the rate of decay. The amplitude of the oscilla-
tions is bounded by
p
i(t) = ± A2 + B2 e ↵t , (6.157)

as shown by the dashed lines in the figure.

Lastly, we consider when ↵ = !o . This relationship generates repeated roots for the char-
acteristic polynomial of ↵. The general solution for this case is

↵t ↵t
i(t) = Ae + Bte . (6.158)

Figure 6.12(c) shows an example this type of solution. Although we see no oscillations as we
did with the underdamped case, this solution is just on the verge of providing an oscillation.
This situation is referred to as a critically damped response. It provides the fastest possible
decay time that can be had without providing oscillations. Although overdamped responses
also show no oscillating behavior, the time that it takes to reach a steady-state value is always
greater than that of a critically damped circuit. To verify this visually, the overdamped curve
has been overlaid with the critically damped curve in figure 6.12(c).
The circuit from figure 6.11 can also be made to be non-homogeneous by applying a
voltage that varies in time. Consider an alternating voltage given by

v(t) = Eo cos( t), (6.159)

where Eo and are constants. By equation (6.149), we require v 0 (t) to determine our gov-
erning ODE for the circuit. Taking this derivative we arrive at the governing ODE,

d 2 i R di 1
2
+ + i = Fo sin( t), (6.160)
dt L dt LC

where we divided each term by L from the original expression to have the leading coefficient
of our second-order term be equal to one. The constant Fo incorporates this division as
well as the constants that came from di↵erentiating the right hand side. The solution to
 126 6.8. Application to circuits

equation (6.160) takes the form

i(t) = ih + ip , (6.161)

where ih is the one of the appropriate homogeneous solutions discussed earlier, based on
the values of the characteristic polynomial. We can determine the particular solution, ip , by
examining the forcing term. Since the forcing term is a sine function, we might guess that a
function in terms of cosine and/or sine may also satisfy this ODE. If we choose

ip (t) = A cos( t) + B sin( t), (6.162)

where A and B are constants, we can check to see if it satisfies our ODE. Substituting in the
appropriate derivatives of equation (6.162) into equation (6.160) we are left with

2 2 R⇣ ⌘
A cos( t) B sin( t) + A sin( t) + B cos( t)
L (6.163)
1 ⇣ ⌘
+ A cos( t) + B sin( t) = Fo sin( t).
LC

Comparing the left hand and right hand sides of equation (6.163) we see that with the correct
values of A and B, this expression can be satisfied. To determine these constants, we compare
the coefficients for each of the cos( t) terms to get

2 RB A
A + + = 0. (6.164)
L LC

Similarly, we can compare the coefficients for each of the sin( t) terms to get

2 RA B
B + = Fo . (6.165)
L LC

Theses two independent expressions can be used to solve for our two unknowns. Once
they have been found, we can write the final form of our solution by adding together the
homogeneous and particular components.
7. Systems of Ordinary Di↵erential Equations

7.1 Solution by elimination 130

7.2 A brief review of matrices and linear algebra 132

7.3 Systems of ODEs as eigenvalue problems 146

Systems of ODEs often arise when modeling dynamical systems with multiple degrees of
freedom. Consider the mass-spring system sketched in figure 7.1. If we assume the only
forces impacting the masses are the springs, a force balance on mass m1 produces

X d 2 x1
F = k1 x1 k12 (x1 x2 ) = m1 , (7.1)
dt 2

while a balance on m2 produces

X d 2 x2
F = k12 (x2 x1 ) k2 x2 = m2 . (7.2)
dt 2

These can then be rearranged into the following system of coupled, second-order, constant-
coefficient ODE’s,
2
m1 ddtx21 + (k1 + k12 )x1 k12 x2 = 0,
2
(7.3)
m2 ddtx22 + (k2 + k12 )x2 k12 x1 = 0.

x1 x2
k1 k12 k2
m1 m2

Figure 7.1

127
 128

This system requires four initial conditions, such as

dx1
x1 (0) = a, = b, (7.4)
dt t=0

dx2
x2 (0) = e, = g,
dt t=0

Systems of ODEs can also occur when solving a single higher-order initial-value problem,
such as
d 3x dx
= cos(t) + 2tx. (7.5)
dt 2 dt

This can be expressed as an equivalent system of first-order ODEs by defining the new vari-
ables
dx d 2x
u= , v= . (7.6)
dt dt 2

These allow us to define the equivalent system

2 3 2 3
66 x 77 66 u 77
d 6666 7777 66
66
77
77
6u 7 = 66 v 77 . (7.7)
dt 6664 7775 66
4
77
5
v cos(t)u + 2tx

This equivalent first-order system is then straightforward to solve using the numerical meth-
ods discussed in Chapter 12. We can similarly transform the system (7.3) of second-order
ODEs into an equivalent first-order system by defining the new variables

dx1 dx2
u1 = , u2 = . (7.8)
dt dt

This produces
2 3
2 3 66 u1 77
66 x1 77 66 77
66 77 66 77
6 7 66 u2 77
d 6666 x2 7777 66 ⇣
66 k1 +k12 ⌘ ⇣ k ⌘ 7777 .
6 7 = (7.9)
dt 6666u1 7777 66
66 m1 x1 + m12 x2 777
1 77
66 77 66 77
4 5 66 ⇣ ⌘ ⇣ ⌘
u2 4 k2 +k12 x + k12 x 75
m2 2 m2 1

Before we discuss solution methods, it is important to emphasize the reduction in com-

plexity that we get from rewriting these problems as a system of first-order ODEs. To reit-
erate how this was done above, the first step in this process is to substitute the first-order
derivatives in the equations with new variables. The number of variables that are introduced
depend on the order of the ODE (or the order of each ODE for systems). Specifically, n’th or-
 Chapter 7. Systems of Ordinary Di↵erential Equations 129

der ODEs will require n 1 new variables to arrange them into a system of first-order ODEs.
The examples below demonstrate this process.

Example 7.1 (Systems of first-order ODEs)

Rewrite the following ODE as a system of first-order ODEs.

d 4 y dy
+ = f (t) (7.10)
dt 4 dt

. As a 4th order ODE, this problem will require three new variables to arrange it into
the following system of first order ODEs. The system of ODEs will be as follows:

dy
u= (7.11)
dt

du d 2 y
v= = 2 (7.12)
dt dt

dv d 3 y
w= = 3 (7.13)
dt dt

dw
+ u = f (t) (7.14)
dt

The same process can be applied to systems of higher-order ODEs to form systems of
first-order ODEs. To accomplish this, we set each first derivative of each dependent variable
equal to their own new variable. From there additional variables are included to equate
higher-order derivatives to a first-order derivative, just as we showed for single equation
ODEs.

Example 7.2 (Systems of first-order ODEs)

Consider the system of higher-order ODEs:

d 4 y1 dy1 dy2
+ + = f 1 (t) (7.15)
dt 4 dt dt

d 4 y2 dy2 dy1
+ + = f 2 (t) (7.16)
dt 4 dt dt

Next, set the following first-order derivatives equal to new variables.

dy1
=a (7.17)
dt
 130 7.1. Solution by elimination

dy2
=u (7.18)
dt

da d 2 y1
= =b (7.19)
dt dt 2

du d 2 y2
= =u (7.20)
dt dt 2

db d 3 y1
= =c (7.21)
dt dt 3

dv d 3 y2
= =w (7.22)
dt dt 3

Now the original system of higher order ODEs can be rewritten as a system of first
order ODEs.
dc
+ a + u = f 1 (t) (7.23)
dt

dw
+ u + a = f 2 (t) (7.24)
dt

This technique is extremely important for simplifying solution methods as systems of

first-order ODEs are much easier to solve.

7.1 Solution by elimination

To demonstrate the method of elimination, consider the mass-spring system sketched in

figure 7.1. To simplify the example, suppose we set m1 = 2, m2 = 4, k1 = 1, k12 = 2 and k2 = 3.
The system of equations then simplifies to

2
2 ddtx21 + 3x1 2x2 = 0,
2
(7.25)
4 ddtx22 + 5x2 2x1 = 0.

The method of elimination proceeds by first rewriting the system as

" #
d2
2 dt 2 + 3 x1 2x2 = 0,
" # (7.26)
d2
4 dt 2 + 5 x2 2x1 = 0,
 Chapter 7. Systems of Ordinary Di↵erential Equations 131

or more simply as
L 1 x1 2x2 = 0,
(7.27)
2x1 + L2 x2 = 0,

where L1 and L2 are the linear operators

" 2 # " 2 #
d d
L1 = 2 2 + 3 , L2 = 4 2 + 5 . (7.28)
dt dt

We now operate on the two equations so that when they are added, one of the unknown
variables is eliminated. In this case, we can operate on the second equation with L1 ,

2L1 x1 4x2 = 0,
(7.29)
2L1 x1 + L1 L2 x2 = 0,

and then add the two equations,

L1 L2 x2 4x2 = 0. (7.30)

The operation L1 L2 is equivalent to

" 2 #" 2 #
d d x2 d 4x d 2x d 2x
2 2 + 3 4 2 + 5x2 = 8 42 + 10 22 + 12 22 + 15x2 . (7.31)
dt dt dt dt dt

We can also work with the operators L1 and L2 directly,

" #" 2 #
d2 d d4 d2
2 2 + 3 4 2 + 5 = 8 4 + 22 2 + 15. (7.32)
dt dt dt dt

Either way, the method of elimination produces the fourth-order ODE

d 4 x2 d 2 x2
8 + 22 + 11x2 = 0. (7.33)
dt 4 dt 2

This can be solved by seeking a solution in the form x2 ⇠ e t , producing the characteristic
equation
4 2
8 + 22 + 11 = 0. (7.34)

Defining ↵ = 2, we can write this as

8↵ 2 + 22↵ + 11 = 0, (7.35)
 132 7.2. A brief review of matrices and linear algebra

which produces the roots

p p
11 33 11 + 33
↵1 = , ↵2 = (7.36)
8 8

These produce four values of

s s
p p
11 33 11 + 33
1,2 = ±i , 2,3 = ±i . (7.37)
8 8

The solution for x2 can consequently be written as

x2 = C1 sin(!1 t) + C2 cos(!1 t)
(7.38)
+ C3 sin(!2 t) + C4 cos(!2 t),

where
s s
p p
11 33 11 + 33
!1 = , !2 = . (7.39)
8 8

At this point, it’s important to note that while a single mass-spring system has only one
natural frequency, a system of two masses can simultaneously oscillate with two frequencies.
To solve for x1 , we return to the original system of ODEs and see that

d 2 x2 5
x1 = 2 + x2 , (7.40)
dt 2 2

which requires

" # " #
5 5
x1 = C1 !12 sin(!1 t) + C2 !12 cos(!1 t)
2 2
" # " # (7.41)
5 5
+ C3 !22 sin(!2 t) + C4 !22 cos(!2 t).
2 2

To determine the four constants of integration, we would need to apply the four initial con-
ditions (7.4).

7.2 A brief review of matrices and linear algebra

The mathematics of matrices is a rich field of study typically covered in a course on linear
algebra. Here we only review the minimum concepts required for our purposes. Matrices
 Chapter 7. Systems of Ordinary Di↵erential Equations 133

arise when considering systems of linear equations, such as

2x1 2x2 + 6x3 = 8,

x1 + 3x3 = 3, (7.42)
2x1 x2 = 9.

This example system can be written as the equivalent matrix equation

2 32 3 2 3
662 2 677 66x1 77 66877
66 76 7 6 7
661
64 0 37777 6666x2 7777 = 666637777 , (7.43)
54 5 4 5
2 1 0 x3 9

which can also be abbreviated as

Ax = b, (7.44)

where A, x, and b are all matrices,

2 3 2 3 2 3
662 2 677 66x1 77 66877
6 7 6 7 6 7
A = 66661 0 37777 , x = 6666x2 7777 , b = 666637777 . (7.45)
4 5 4 5 4 5
2 1 0 x3 9

Let’s now define a matrix more abstractly as a two-dimensional array of numbers, such
as
2 3
66a11 a12 a13 77
6 7
A = 6666a21 a22 a23 7777 . (7.46)
4 5
a31 a32 a33

The numbers within a matrix are called elements, and are often denoted using indices aij .
We define the dimension of a matrix as N ⇥ M, where N and M are the number of rows and
columns. The matrix below has the dimension 2 ⇥ 4
" #
1 2 3 7
A= . (7.47)
1 3 7 13

A square matrix has the same number of rows and columns, N ⇥ N . These matrices arise
when solving a system of N equations for N unknowns. When a matrix has more rows
than columns, that often means the system is overdetermined, i.e. it has more equations than
unknowns. When a matrix has fewer rows than columns, that typically means the system is
underdetermined, i.e. it has fewer equations than unknowns. A vector with only one column
is called a column vector
2 3
66177
6662777
A = 6666 7777 , (7.48)
66377
4 5
4
 134 7.2. A brief review of matrices and linear algebra

and a vector with only one row is called a row vector

h i
A= 1 2 3 4 . (7.49)

7.2.1 Gaussian elimination and inverse matrices

Matrices permit three elementary row operations:

1. interchange two rows,

2. multiply a row by a non-zero constant,

3. add or subtract a multiple of one row to another.

Gaussian elimination uses these operations to solve linear systems as follows. Consider the
linear system (7.42). We first express the system using the augmented matrix
2 3
r1 6662 2 6 877
77
6
r2 6661 0 3 3777 , (7.50)
6 7
r3 42 1 0 95

where the fourth column represents the vector b, and we have labelled the rows r1 , r2 , r3 for
convenience. Gaussian elimination begins by dividing the first row by entry a11 ,
2 3
r1 /2 6661 1 3 477
77
6
r2 6661 0 3 3777 . (7.51)
6 7
r3 42 1 0 95

We then replace row 2 with r2 r1 , and replace row 3 with r3 2r1 . This gives
2 3
r1 661 1 3 4 77
66 77
r2 r1 6660 1 0 1777 . (7.52)
6 7
r3 2r1 40 1 6 15

Replacing row 3 with (r2 r3 )/6 produces

2 3
r1 661 1 3 4 77
66 77
r2 660 1 0 1 777 . (7.53)
66 7
(r2 r3 )/6 40 0 1 1/35

We now replace row 1 with r1 3r3

2 3
r1 3r1 6661 1 0 5 77
77
6
r2 6660 1 0 1 777 . (7.54)
6 7
r3 40 0 1 1/35
 Chapter 7. Systems of Ordinary Di↵erential Equations 135

And finally, we replace row 1 with r1 + r2

2 3
r1 + r2 6661 0 0 4 77
77
6
r2 6660 1 0 1 777 . (7.55)
6 7
r3 40 0 1 1/35

This final result is equivalent to the system I x = x, where I is the identity matrix,
2 32 3 2 3
661 0 077 66x1 77 66 4 77
66 76 7 6 7
660 1 0777 666x2 777 = 666 1 777 . (7.56)
64 75 64 75 64 75
0 0 1 x3 1/3

The steps described above can also be interpreted in terms of the inverse matrix. Given a
matrix A, its corresponding inverse matrix A 1 is defined such that

A 1 A = I. (7.57)

Solving the system Ax = b can be interpreted as left-multiplying the system by A 1 ,

A 1 Ax = A 1 b, (7.58)

because this equals

x = A 1 b. (7.59)

To compute the inverse of a matrix, consider again the matrix A in equation (7.45). The
inverse of A can be computed by first defining the augmented matrix
2 3
661 1 3 1 0 077
66 77
661 0 3 0 1 0777 . (7.60)
66 7
42 1 0 0 0 15

We now perform the same elementary operations demonstrated above to convert the matrix
A into the identity matrix. For example, we replace row 2 with r2 r1 , and replace row 3
with r3 2r1 ,
2 3
r1 6661 1 3 1 0 077
77
6
r2 r1 6660 1 0 1 1 0777 (7.61)
6 7
r3 2r1 40 1 6 2 0 15

Continuing in this manner, we find that

2 3
661 1 3 1/2 1/2 1/2 77
66 77
660 1 0 1 1 0 777 (7.62)
66 7
40 1 6 1/6 1/6 1/6.5
 136 7.2. A brief review of matrices and linear algebra

The inverse matrix is consequently

2 3
66 1/2 1/2 1/2 77
6 7
A 1 = 6666 1 1 0 7777 (7.63)
4 5
1/6 1/6 1/6.

7.2.2 Linear dependence of vectors and matrix rank

Considerable theory is dedicated to understanding whether a matrix A has an inverse A 1 .
Inverse matrices play a central role in engineering and physics because mathematical mod-
els of real-world systems often boil down to solving systems Ax = b, where the matrix A
can have millions of rows and columns. The matrices are often so large that they cannot be
stored in the memory (RAM) used to perform mathematical operations by computers. Nu-
merical simulations of real-world systems (fluid flows, solid deformations, robotic motions)
also often require you to repeatedly solve a large system Ax = b for many di↵erent right-
hand-sides b. If you can find the inverse matrix A 1 , then you can easily solve the system
Ax = b for any right-hand-side b by performing a single matrix multiplication x = A 1 b.
That operation can also be parallelized on large computer clusters to permit the simulation
of such complicated systems as weather patterns and climate change.

The possible existence of an inverse A 1 depends on the linear dependence of the rows
and columns of A. We first note that the rows and columns of an N ⇥N matrix can interpreted
as a set of N row and column vectors, as sketched below.

1 2 3 1 2 3 1 2 3
12 5 9 12 5 9 12 5 9
7 18 9 7 18 9 7 18 9

Figure 7.2

A set of vectors {u1 , u2 , ..., un } is linearly dependent if at least one of the vectors is a linear
combination of the others. The following set is linearly independent,
2 3 2 3 2 3
66177 66077 66077
66 77 66 77 66 77
66077 , 66177 , 66077 , (7.64)
64 75 64 75 64 75
0 0 1

while the following is linearly dependent

2 3 2 3 2 3
66177 66077 66277
66 77 66 77 66 77
66077 , 66177 , 66377 . (7.65)
64 75 64 75 64 75
1 0 2
 Chapter 7. Systems of Ordinary Di↵erential Equations 137

When solving a system of N equations for N unknowns, the N equations must be linearly
independent. Consider, for example, the system

2x1 2x2 + 6x3 = 8,

x1 + 3x3 = 3, (7.66)
3x1 2x2 + 9x3 = 11.

Suppose we try to solve this system using Gaussian elimination. We first build the aug-
mented matrix,
2 3
662 2 6 8 77
66 77
661 0 3 3 777 . (7.67)
66 7
43 2 9 115

Performing the steps of Gaussian elimination, we eventually simplify the system to

2 3
661 0 3 3 77
66 77
660 1 0 1777 , (7.68)
66 7
40 0 0 05

in which the third row is all zeros. The inverse matrix A 1 for this system consequently does
not exist. The zero row occurred because the third equation in the system (7.66) is a linear
combination of the other two. The remaining two rows of the augmented matrix produce
the two equations
x1 + 3x3 = 3,
(7.69)
x2 = 1.

These produce an infinite number of solutions which we can express by setting x3 to a ran-
dom constant x3 = C, so that
2 3 2 3
x1 = 3 3C, 66 3 77 66 377
66 77 6 7
x2 = 1 = 66 177
64 75 + C 6666 0 7777 . (7.70)
4 5
x3 = C 0 1

The discussion above leads to the definition of something called the rank of a matrix. We
define this term below.

Definition 7.1 (Rank of a matrix)

The number of linearly independent rows in a matrix equals the number of linearly
independent columns and is called the rank of the matrix. The maximum rank of a
square N ⇥ N matrix is N , i.e. the number of rows and columns. If the rank is less than
N , the matrix is called rank deficient. Rank deficient matrices do not have an inverse.
 138 7.2. A brief review of matrices and linear algebra

7.2.3 Determinants

Though the existence of an inverse is related to the concept of rank, computing the rank
requires us to check whether Gaussian elimination produces one or more rows of zeros. For
large systems, that isn’t practical. Instead, we compute something called the determinant.

To motivate the definition of the determinant, consider a system of two equations,

ax1 + bx2 = e,
(7.71)
cx1 + dx2 = f .

The matrix A for this system is

" #
a b
A= . (7.72)
c d

We define the determinant of a 2 ⇥ 2 matrix as

a b
det(A) = = ad cb, (7.73)
c d

where the determinant is denoted using either det(A) or by replacing the square brackets
of the matrix with vertical lines. Now suppose that the two equations in system (7.71) are
linearly dependent. This requires the second row of matrix A to be a multiple of the first, i.e.

c = ↵a, d = ↵b, (7.74)

where ↵ is some constant. Substituting these relations into the determinant, we find that

det(A) = ad cb = ↵ab ↵ab = 0. (7.75)

For this 2 ⇥ 2 example, the inverse A 1 only exists if det(A) , 0.

We can similarly define determinants for any square N ⇥ N matrix, though the definition
quickly becomes very laborious. For a 3 ⇥ 3 matrix, we define the determinant as

a b c
e f d f d e
d e f =a b +c . (7.76)
h i g i g h
g h i

For a fourth-order matrix, it is defined as

a b c d
f g h e g h e f h e f g
e f g h
=a j k l b i k l +c i j l d i j k , (7.77)
i j k l
n o p m o p m n p m n o
m n o p
 Chapter 7. Systems of Ordinary Di↵erential Equations 139

and so on for higher-order matrices.

Determinants have the following properties:

1. If a row or column of a matrix A is all zeros, then the determinant is zero, det(A) = 0.

2. If any row or column of a matrix is a linear combination of the other rows or columns,
then the determinant is zero, det(A) = 0.

3. The determinant of a product of matrices is the product of the determinants, i.e.

det(AB) = det(A)det(B).

4. The determinant of a sum of matrices is not equal to the sum of determinants, i.e.
det(A + B) , det(A) + det(B).

5. The determinant of the identity matrix is unity, i.e. det(I) = 1.

Finally, we conclude with the following theorem which allows us to determine the exis-
tence of an an inverse matrix A 1 .

Theorem 7.1 (Existence of inverse matrices)

• If A is an N ⇥ N matrix and det(A) , 0, then the inverse matrix A 1 exists and is

unique

• If det(A) = 0, then the inverse matrix A 1 does not exist, and A is called a singular
matrix.

7.2.4 Homogeneous matrix problems, Ax = 0

When modeling real-world systems, we sometimes encounter matrix equations of the form

Ax = 0, (7.78)

where the right-hand-side is a column vector of all zeros. Such systems are called homo-
geneous matrix problems. They pose a unique challenge because one obvious solution is the
trivial solution,
x = 0. (7.79)

This trivial solution is usually not the solution we are interested in. The question then arises,
“when can a homogeneous problem Ax = 0 produce nontrivial solutions?” To answer this
question, we use theorem 7.1 from section 7.2.3. This states that if det(A) , 0, then A 1 exists
and is unique. In this case, the homogeneous problem Ax = 0 has only the trivial solution,
because
x = A 1 0 = 0. (7.80)

That leads to the following theorem.

 140 7.2. A brief review of matrices and linear algebra

Theorem 7.2 (Homogeneous matrix problems)

The homogeneous matrix problem Ax = 0 admits nontrivial solutions only if det(A) =
0. In that case, the problem admits an infinite number of solutions.

Consider for example the homogeneous problem

2 32 3 2 3
661 2 377 66x1 77 66077
66 76 7 6 7
664 5 6777 666x2 777 = 6660777 . (7.81)
64 75 46 75 64 75
7 8 9 x3 0

The determinant of the A matrix is

5 6 4 6 4 5
det(A) = 2 +3 = 0. (7.82)
8 9 7 9 7 8

This system consequently yields non-trivial solutions. Performing Gaussian elimination, we

find that the system reduces to
2 32 3 2 3
661 2 377 66x1 77 66077
66 76 7 6 7
660 1 2777 666x2 777 = 6660777 . (7.83)
64 75 64 75 64 75
0 0 0 x3 0

As expected, Gaussian elimination produces a row of zeros, because the matrix is singular.
The remaining two rows imply
x1 = 2x2 3x3 ,
(7.84)
x2 = 2x3 .

If we set x3 = C, we can express the infinite solutions as

2 3
66 1 77
6 7
x = C 6666 27777 . (7.85)
4 5
1

Note that the trivial solution is recovered when C = 0.

Let’s now consider the slightly di↵erent matrix problem
2 32 3 2 3
661 2 3 77 66x1 77 66077
66 76 7 6 7
664 5 6 777 666x2 777 = 6660777 , (7.86)
64 75 64 75 64 75
7 8 10 x3 0

where the element a33 has been set to 10. The determinant of the A matrix is now

5 6 4 6 4 5
det(A) = 2 +3 = 3. (7.87)
8 10 7 10 7 8
 Chapter 7. Systems of Ordinary Di↵erential Equations 141

Because the determinant is now nonzero, the inverse matrix exists and the system only pro-
duces the trivial solution. Performing Gaussian elimination, we indeed find that the system
reduces to
2 32 3 2 3
661 2 377 66x1 77 66077
6660 1 2777 666x 777 = 6660777 , (7.88)
66 77 66 2 77 66 77
4 54 5 4 5
0 0 1 x3 0

which produces only the trivial solution.

7.2.5 Eigenvalue problems

Another common class of matrix problems are those of the form

Ax = x, (7.89)

where A is a given square matrix, is an unknown scalar, and x is an unknown vector. Such
problems are called eigenvalue problems. When solving eigenvalue problems, the objective is
to determine what combinations of and x satisfy equation (7.89). The scalars are called
eigenvalues, while the vectors x are called eigenvectors.
When seeking solutions to eigenvalue problems, one obvious solution is the trivial so-
lution x = 0. Unfortunately, this is typically not the solution we are interested in. To find
nontrivial solutions, we rewrite the eigenvalue problem as an equivalent homogeneous prob-
lem,
Ax x = (A I ) x = 0. (7.90)

From our work in section 7.2.4, we know that nontrivial solutions only exist when

det (A I ) = 0. (7.91)

This provides a relation from which we can determine permissible eigenvalues .

To demonstrate the solution of an eigenvalue problem, consider the example
" #" # " #
2 1 x1 x1
= . (7.92)
1 2 x2 x2

We begin by writing the equivalent homogeneous equation

" #" # " #
2 1 x1 0
= . (7.93)
1 2 x2 0

This produces nontrivial solutions when

2 1
= (2 )2 1 = 2
4 + 3 = 0. (7.94)
1 2
 142 7.2. A brief review of matrices and linear algebra

The polynomial equation 2 4 +3 = 0 is called the characteristic equation. In this case, the
characteristic equation produces two eigenvalues,

1 = 1, 2 = 3. (7.95)

To find the corresponding eigenvectors, we begin by substituting 1 = 1 into the homoge-

neous equation (7.93), producing
" #" # " #
1 1 x1 0
= , (7.96)
1 1 x2 0

which simplifies to
" #" # " #
1 1 x1 0
= . (7.97)
0 0 x2 0

As expected, the resulting matrix problem is singular, and provides only one equation for x1
and x2 ,
x1 + x2 = 0. (7.98)

If we arbitrarily set x2 = C, the above equation requires x1 = C. We can consequently write

the eigenvector as
" #
1
x1 = C . (7.99)
1

The subscript 1 denotes that x1 is the eigenvector corresponding to 1. The pair ( 1 , x1 ) is

often called an eigenpair or eigenmode. Repeating the procedure for 2 = 3, we find that the
homogeneous problem (7.93) reduces to
" #" # " #
1 1 x1 0
= . (7.100)
0 0 x2 0

Setting x2 = C, the eigenvector for 2 can be written as

" #
1
x2 = C . (7.101)
1

As demonstrated above, we can only determine eigenvectors up to a constant factor C. Eigen-

vectors are often reported with having a magnitude equal to unity. Using this approach, the
example above has the eigenvectors
" # " #
1 1 1 1
x1 = p , x2 = p . (7.102)
2 1 2 1
 Chapter 7. Systems of Ordinary Di↵erential Equations 143

An N ⇥ N matrix produces an N ’th-order characteristic polynomial with N roots. How-

ever, some of these roots may be repeated, such that the matrix has less than N distinct
eigenvalues. Consider, for example, the problem
2 32 3 2 3
662 2 177 66x1 77 66x1 77
66 76 7 66 77
661 3 1777 666x2 777 = 66x2 77 . (7.103)
64 75 64 75 64 75
1 2 2 x3 x3

The eigenvalues are given by the determinant

2 2 1
1 3 1 = 0, (7.104)
1 2 2

which after some algebra produces the characteristic polynomial

( 5)( 1)2 = 0. (7.105)

Due to the repeated root = 1, the matrix has only two distinct eigenvalues

1 = 5, 2 = 1. (7.106)

To determine the eigenvectors, we substitute 1 and 2 into the homogeneous equation

2 32 3 2 3
662 2 1 77 66x1 77 66077
66 77 66 77 66 77
66 1 3 1 77 66x2 77 = 66077 . (7.107)
64 75 64 75 64 75
1 2 2 x3 0

For 1 = 5, this eventually produces

2 32 3 2 3
66 3 2 1 77 66x1 77 66077
66 76 7 6 7
66 0 1
64 17777 6666x2 7777 = 666607777 . (7.108)
54 5 4 5
0 0 0 x3 0

This leads to the eigenvector

2 3
66177
6 7
x1 = C 666617777 . (7.109)
4 5
1

Repeating for 2 = 1 produces the system

2 32 3 2 3
661 2 177 66x1 77 66077
66 76 7 6 7
660 0 0777 666x2 777 = 6660777 . (7.110)
64 75 64 75 64 75
0 0 0 x3 0
 144 7.2. A brief review of matrices and linear algebra

We see that for the repeated eigenvalue 2, two rows of the matrix reduce to zeros, such that
the only remaining equation is
x1 + 2x2 + x3 = 0. (7.111)

If we arbitrarily set x2 = C2 and x3 = C3 , the remaining equation requires x1 = 2C2 C3 . We

can consequently express the eigenvector as
2 3 2 3 2 3
66 2C2 C3 77 66 277 66 177
66 77 66 77 6 7
x = 666 C2 77 = C2 66 1 77 + C3 666 0 777 . (7.112)
4 7
5 6
4 5 7 64 75
C3 0 1

We consequently see that the repeated eigenvalue produces two linearly independent eigen-
vectors
2 3 2 3
66 277 66 177
66 77 6 7
x2 = C2 666 1 777 , and x3 = C3 6666 0 7777 . (7.113)
4 5 4 5
0 1

In other cases, a repeated eigenvalue might produce only one eigenvector. Consider, for
example,
2 32 3 2 3
661 0 177 66x1 77 66x1 77
66 76 7 6 7
661 1 0777 666x2 777 = 666x2 777 . (7.114)
64 75 64 75 64 75
0 0 1 x3 x3

The eigenvalues are given by the determinant

1 0 1
1 1 0 = 0, (7.115)
0 0 1

which after some algebra produces the characteristic polynomial

(1 )3 = 0, (7.116)

such that we obtain only one eigenvalue

= 1. (7.117)

Solving for the corresponding eigenvector, we find that

2 32 3 2 3
660 0 177 66x1 77 66077
66 76 7 6 7
661 0 0777 666x2 777 = 6660777 . (7.118)
64 75 46 75 64 75
0 0 0 x3 0
 Chapter 7. Systems of Ordinary Di↵erential Equations 145

The first two rows require x1 = x3 = 0. Setting x2 = C, we express the eigenvector as

2 3
66077
6 7
x = C2 666617777 . (7.119)
4 5
0

Matrices can also produce complex eigenvalues, even when all the elements of the matrix
are real. Consider, for example, the matrix
" #
1 1
A= . (7.120)
1 1

Computing det(A I ) = 0,
1 1
= 0, (7.121)
1 1

produces the characteristic polynomial

2
2 + 2 = 0. (7.122)

This has the roots

1 = 1 + i, 2 = 1 i. (7.123)

Just as for real eigenvalues, we compute the corresponding eigenvectors by substituting 1

and 2 into the homogeneous problem
" #" #
1 1 x1
= 0. (7.124)
1 1 x2

For 1 = 1 + i, this produces

" #" #
i 1 x1
= 0. (7.125)
1 i x2

Multiplying the first row by i, produces

" #" # " #" #
1 i x1 1 i x1
= = 0. (7.126)
1 i x2 0 0 x2

This produces the eigenvector

" #
i
x1 = C . (7.127)
1
 146 7.3. Systems of ODEs as eigenvalue problems

Repeating this procedure, we find that the eigenvector for 2 = 1 i equals

" #
i
x2 = C . (7.128)
1

Note that the eigenvalues and eigenvectors of the above matrix appear as complex con-
jugates. This always occurs for complex eigenvalues of real matrices.

Theorem 7.3 (Complex eigenvalues)

Complex eigenvalues and eigenvectors of real matrices always occur as complex con-
jugates, i.e.
= ↵ ±i , x = a ± ib. (7.129)

7.3 Systems of ODEs as eigenvalue problems

To demonstrate the application of eigenvalue problems to solving systems of ODEs, consider

the simple system
dx1
dt = ax1 + bx2 + f 1 (t),
(7.130)
dx2
dt = cx1 + gx2 + f 2 (t).

We can write this as the equivalent matrix problem

" # " #" # " #
d x1 a b x1 f 1 (t)
= + , (7.131)
dt x2 c g x2 f 2 (t)

or more compactly as
dx
= Ax + f(t). (7.132)
dt

For now, let’s consider the homogeneous case f = 0,

dx
= Ax. (7.133)
dt

We’ll explore the non-homogeneous case later, in section 7.3.4.

To motivate our approach, let’s first consider the system (7.133) when the o↵-diagonal
elements b and c are both zero,
" # " #" #
d x1 a 0 x1
= . (7.134)
dt x2 0 g x2
 Chapter 7. Systems of Ordinary Di↵erential Equations 147

The o↵-diagonal elements b and c are responsible for the coupling between x1 and x2 . When
they are both zero, the two ODEs become uncoupled,
dx1
dt = ax1
(7.135)
dx2
dt = gx2 ,

and can be solved by seeking solutions in the form x1 ⇠ e 1t and x2 ⇠ e 2t . This produces

x1 = k1 eat , x2 = k2 egt , (7.136)

where k1 and k2 are arbitrary constants. In vector form, this is equivalent to

" # " # " #
x1 k1 at 0 gt
= e + e . (7.137)
x2 0 k2

Now suppose that b = 0, but c , 0. In this case, x1 is uncoupled from x2 , but x2 is coupled
to x1 , i.e. there is a “one-way coupling,”
" # " #" #
d x1 a 0 x1
= . (7.138)
dt x2 c g x2

We can proceed by first solving the uncoupled ODE for x1 , which has the solution

x1 = k1 eat . (7.139)

We can now write the remaining ODE as

dx2
gx2 = k1 eat , (7.140)
dt |{z}
forcing

where the coupling to x1 is a forcing term. Solving the homogeneous problem first,

dxh
= gxh , (7.141)
dt

we find that
xh = k2 egt . (7.142)

Assuming that g , a, we can seek a particular solution as*

xp = Ceat , (7.143)

* Note that if g equaled a, we would need to modify our approach because the forcing term is also a solution
to the homogeneous problem. That case isn’t important for our discussion here.
 148 7.3. Systems of ODEs as eigenvalue problems

where the constant C is determined by substituting xp in equation (7.140). In this manner,

we eventually find that
" # " # " #
x1 k1 at 0 gt
= e + e . (7.144)
x2 C k2

The above examples suggest that when neither b nor c are zero, we should seek solutions
in the form
" # " #
x1 k
= 1 e t, (7.145)
x2 k2

which we can abbreviate as

x = ke t . (7.146)

Substituting this form in dx/dt = Ax produces,

k = kA. (7.147)

This is an eigenvalue problem for eigenvalue and eigenvector k. To determine the eigen-
values, we solve the characteristic polynomial

det(A I) = 0. (7.148)

In the following subsections, we address three possible cases in which the characteristic
polynomial produces

1. distinct real eigenvalues,

2. distinct complex eigenvalues,

3. repeated eigenvalues.

7.3.1 Distinct roots

Consider the system

dx1
dt = 2x1 + 3x2 ,
(7.149)
dx2
dt = 4x1 + 3x2 ,

which is equivalent to
" # " #" #
d x1 2 3 x1
= . (7.150)
dt x2 4 3 x2
 Chapter 7. Systems of Ordinary Di↵erential Equations 149

Seeking solutions in the form x = ke t produces the characteristic polynomial

2 3
= (2 )(3 ) 12 = ( 6)( + 1) = 0, (7.151)
4 3

from which we find

1 = 6, 2 = 1. (7.152)

Solving for the corresponding eigenvectors, we find

" # "
#
3 1
k1 = , k2 = . (7.153)
4 1

This produces the solution

" # " #
3 6t 1
x = C1 e + C2 e t. (7.154)
4 1

Suppose we apply the initial conditions

x1 (0) = A, x2 (0) = B. (7.155)

" # " # " #

3 1 A
C1 + C2 = . (7.156)
4 1 B

Solving, we find that

A+B 4A 3B
C1 = , C2 = . (7.157)
7 7

7.3.2 Complex roots

Consider the system

" # " #" #
d x1 2 5 x1
= . (7.158)
dt x2 1 2 x2

Solving for the eigenvalues, we find that

2 5 2
= (2 )( 2 )+5 = + 1 = 0. (7.159)
1 2

This produces

1 = i, 2 = i. (7.160)
 150 7.3. Systems of ODEs as eigenvalue problems

Solving for the corresponding eigenvectors, we find that

" # " #
2+i 2 i
k1 = , k2 = , (7.161)
1 1

which produces the solution

" # " #
2 + i it 2 i it
x = C1 e + C2 e . (7.162)
1 1

Using Euler’s formula, eit = cos(t) + i sin(t), we can express this solution in terms of sine
and cosine functions. For generality, we first note that complex eigenvalues and eigenvectors
of a real matrix A must appear as conjugates,

1,2 = ↵ ±i , k1,2 = a ± ib. (7.163)

The resulting solution consequently takes the form

x = C1 (a + ib) e(↵+i )t
+ C2 (a ib) e(↵ i )t
. (7.164)

Applying Euler’s formula and some algebra, this produces the following result.

Complex Eigenvalues:

When a system of ODEs

dx
= Ax (7.165)
dt

produces complex conjugate eigenvalues and eigenvectors of the form

1,2 =↵ ± i , k1,2 = a ± ib, (7.166)

the corresponding solution for x can be written as

" # " #
↵t ↵t
x = A1 e a sin t + b cos t + A2 e a cos t b sin t (7.167)

For the current example, we see that

" # " #
2 1
↵ = 0, = 1, a= , b= . (7.168)
1 0
 Chapter 7. Systems of Ordinary Di↵erential Equations 151

The solution can consequently be written as

8" # " # 9 8" # " # 9
>
> 2 1 >
> >
> >
>
< = < 2 1 =
x = A1 >
> sin(t) + cos(t) >
> + A 2>
> cos(t) sin(t)>
> . (7.169)
: 1 0 ; : 1 0 ;

The solution can be written much more compactly by collecting terms involving sines and
cosines,
" # " #
A1 + 2A2 2A1 A2
x = cos(t) + sin(t). (7.170)
A2 A1

7.3.3 Repeated roots

When it comes to repeated roots, we will consider two cases. To start, take the system
" # " #" #
d x1 2 0 x1
= . (7.171)
dt x2 0 2 x2

Upon inspection, we can already see that x1 and x2 are uncoupled in this example. Regard-
less, if we use our eigenvalue approach then we find

2 0
= (2 )2 = 0. (7.172)
0 2

This produces

1 = 2, 2 = 2. (7.173)

In this case, the repeated eigenvalues produce two eigenvectors. These can be represented
as any two vectors that are linearly independent. For example
" # " #
1 0
k1 = , k2 = . (7.174)
0 1

Perhaps unsurprisingly we could have reached this same solution by solving for x1 and x2
separately due to the uncoupled nature of this problem. The final result, either way, pro-
duces
" # " #
1 2t 0 2t
x = C1 e + C2 e . (7.175)
0 1

A more interesting case involves repeated roots when x1 and x2 maintain their coupled
nature. As an example, consider
" # " #" #
d x1 1 1 x1
= . (7.176)
dt x2 1 3 x2
 152 7.3. Systems of ODEs as eigenvalue problems

This system results in the characteristic polynomial

1 1
= (1 )(3 ) + 1 = 0. (7.177)
1 3

Simplifying this polynomial, we are left with

( 2)2 = 0, (7.178)

which solves to give

1 = 2 = 2. (7.179)

Although up to this point each of these steps is identical to previously covered problems, we
see an issue arise when it comes to solving for the eigenvectors. The system
" #" #
1 2 1 k1
= 0, (7.180)
1 3 2 k2

reduces to
" #" #
1 1 k1
= 0. (7.181)
0 0 k2

This reduction shows that we are left with an underdetermined system that consists of two
unknowns and only one equation,
k1 + k2 = 0. (7.182)

If we say k2 = C, then k1 = C, and we are left with only one eigenvector,

" #
1
. (7.183)
1

To determine a second linearly independent solution, let’s attempt

x = kte2t (7.184)

as a possible solution. Substitution of this expression into x0 = Ax provides

ke2t + k2te2t = Akte2t . (7.185)

Dividing by e 2t and rearranging, we are left with

k + t(2k Ak) = 0. (7.186)

 Chapter 7. Systems of Ordinary Di↵erential Equations 153

This requires
k = 0, and 2k = Ak. (7.187)

The result of these requirements is the trivial solution k = 0.

Next, let’s seek a solution as

x = kte2t + ⌘ e 2t . (7.188)

Substituting this into x0 = Ax gives

ke2t + k2te2t + ⌘ 2e2t = Akte2t + A⌘

⌘ e 2t . (7.189)

Diving each term by e2t and rearranging yields

(k + 2⌘
⌘ A⌘
⌘ ) + t(2k Ak) = 0. (7.190)

This requires
A⌘
⌘ = k + 2⌘
⌘, and Ak = 2k, (7.191)

which can be rewritten as

⌘ = k,
(A 2I)⌘ and (A 2I)k = 0. (7.192)

These two expressions can be used together to determine the values for k and ⌘ . We start by
acknowledging that (A 2I)k = 0 is satisfied if k is an eigenvector corresponding to = 2.
This was shown above with the result
" #
1
k= . (7.193)
1

⌘ = k. This can be written as

Next, we will turn to solving for ⌘ by solving (A 2I)⌘
" #" # " #
1 2 1 ⌘1 1
= , (7.194)
1 3 2 ⌘2 1

which reduces to
" #" # " #
1 1 ⌘1 1
= . (7.195)
0 0 ⌘2 0

This provides a relationship which can be used for ⌘1 and ⌘2 . Say ⌘2 = C, which gives

⌘1 + C = 1 =) ⌘1 = 1 C. (7.196)
 154 7.3. Systems of ODEs as eigenvalue problems

Therefore, we conclude that

" # " # " #
1 C 1 1
⌘= = +C . (7.197)
C 0 1

Recall that the first solution was

" #
1 2t
x1 = C1 e . (7.198)
1

Now, after seeking a second solution, we also have

h i
x2 = C2 kte2t + ⌘ e 2t
" # (" # " # )
1 2t 1 1
= C2 te + C2 + C3 e 2t . (7.199)
1 0 1
| {z }
similar to x1

Note that the term labeled with the underbrace is similar to x1 and therefore can be absorbed
by the first solution. For the final solution, we take the sum x1 + x2 . Consequently, we can
finally write
" # " # " #
1 2t 1 2t 1 2t
x = C1 e + C2 te + C2 e . (7.200)
1 1 0

Repeated Eigenvalues:
In general, if you have a repeated root of order two, then the two solutions take the
form
⇣ ⌘
x = C1 ke t + C2 kte t + ⌘ e t , (7.201)

where ⌘ is any solution of

(A I)⌘
⌘ = k. (7.202)

7.3.4 Nonhomogeneous systems

8. Non-constant Coefficient ODEs

8.1 Power series solutions 155

8.2 The Cauchy-Euler equation 162

8.3 The method of Frobenius 166

8.1 Power series solutions

Though we have relatively straightforward methods of solving ODEs with constant coeffi-
cients, non-constant coefficient ODEs generally require us to resort to more difficult approx-
imations in the form of series solutions. Consider a second-order ODE of the form

y 00 + p(x)y 0 + q(x)y = 0. (8.1)

The power-series approach discussed in this section supposes that p(x) and q(x) can be ex-
pressed as Taylor series, such as

1
X
(x xo )2 (x xo )n
p(x) = p(xo ) + p 0 (xo )(x xo ) + p 00 (xo ) + ... = p n (xo ) , (8.2)
2 n!
n=0

where p n denotes the n’th derivative, and we define 0! = 1. To express p(x) and q(x) in Taylor
series, we require that

(1) p(x) and q(x) are infinitely di↵erentiable

(2) the resulting series converge in some interval |x xo | < R

(3) the series converge to p(x) and q(x)

Condition (3) might appear strange; however, there are rare cases where the series converges,
but not to p(x) and q(x).

155
 156 8.1. Power series solutions

Definition 8.1 (Analytic and singular functions)

If some function f (x) can be expressed as a Taylor series in some interval |x xo | < R,
then f (x) is analytic at x = xo . Otherwise, f (x) is said to be singular at xo .

As an example, the function sin(x) is analytic for all x; however, not all functions are
analytic for all values of x. Consider the function 1/(1 x), which is singular at x = 1.

Now that we have introduced the concept of a function begin analytic at a particular
value of x, we can introduce a fundamental theorem that will be used to solve some instances
of non-constant coefficient ODEs.

Theorem 8.1 (Power series solution)

If p(x) and q(x) are analytic at x = xo , then the solution y(x) to the ODE

y 00 + p(x)y 0 + q(x)y = 0 (8.3)

is also analytic at xo , and can be expressed as

1
X
y(x) = an (x xo )n . (8.4)
n=0

To demonstrate how theorem (8.1) can be leveraged to solve ODE’s, consider the follow-
ing equation,
y 00 + y = 0, (8.5)

for which p(x) = 0 and q(x) = 1. This constant-coefficient ODE can be easily solved by seeking
solutions in the form y ⇠ e x . That approach produces the general solution

y = A cos(x) + B sin(x), (8.6)

where A and B are constants. Nevertheless, let’s see how we can also solve this equation
using a power-series approach. Because p(x) = 0 and q(x) = 1 are analytic for all values of
x, theorem (8.1) guarantees that y(x) is also analytic for all x. In that case, we can seek a
solution in the form
X1
y(x) = an (x xo )n . (8.7)
n=0

For simplicity, let’s set xo = 0, so that

1
X
y= an xn = a0 + a1 x + a2 x2 + ... (8.8)
n=0
 Chapter 8. Non-constant Coefficient ODEs 157

Di↵erentiating with respect to x, we find that

1
X
2
0
y = a1 + 2a2 x + 3a3 x + ... = nan xn 1 , (8.9)
n=1

and
1
X
y 00 = n(n 1)an xn 2 . (8.10)
n=2

Substituting these expressions into the ODE (8.5), we find

1
X 1
X
2
n(n 1)an xn + an xn = 0. (8.11)
n=2 n=0

We now perform a series of steps with the objective of rewriting the above equation as a
single summation of the form
1 h
X i
... xn = 0, (8.12)
n=0

where the dependence on x appears as xn . Using the substitution m = n 2, we can rewrite

the first summation as
1
X 1
X
n 2
n(n 1)an x = (m + 2)(m + 1)am+2 xm . (8.13)
n=2 m=0

Since m is simply a dummy variable in the summation, we can now replace it again with n,
so that we have
X1 1
X
(n + 2)(n + 1)an+2 xn + an xn = 0. (8.14)
n=0 n=0

The two summations above now have equal limits (from n = 0 to 1), and can consequently
be combined as
X1 h i
(n + 2)(n + 1)an+2 + an xn = 0. (8.15)
n=0

For equation (8.15) to be satisfied for all x, the expression within the square brackets must
equal zero,
(n + 2)(n + 1)an+2 + an = 0. (8.16)
 158 8.1. Power series solutions

This requires
an
an+2 = . (8.17)
(n + 2)(n + 1)

Equation (8.17) provides a recursion relation that we can use to write out the final solu-
tion as follows. Substituting n = 0 and n = 1, we find that

a0
n = 0, a2 = (8.18)
(2)(1)
a1
n = 1, a3 = . (8.19)
(3)(2)

We see that a2 and a3 depend on a0 and a1 , respectively. Moving forward to n = 2 and n = 3,

we find that
a2 a0
n = 2, a4 = = (8.20)
(4)(3) (4)(3)(2)(1)
a3 a1
n = 3, a5 = = . (8.21)
(5)(4) (5)(4)(3)(2)(1)

From the emerging pattern, we can write our solution for y(x) as
! !
x2 x4 x3 x5
y(x) = a0 1 + ... + a1 x + ... , (8.22)
2! 4! 3! 5!

or as the two summations

1
X X 1
1n 2n 1n
y(x) = a0 x + a1 x2n+1 (8.23)
(2n)! (2n + 1)!
n=0 n=0

We now require two independent boundary conditions to solve for a0 and a1 . Suppose
we are given the conditions
y(0) = 4 and y 0 (0) = 1. (8.24)

Applying the first condition, we find that

y(0) = a0 = 4. (8.25)

Applying the second, we find that

y 0 (0) = a1 = 1. (8.26)
 Chapter 8. Non-constant Coefficient ODEs 159

Now suppose that the boundary conditions been given at the point x = 3,

y(3) = 4, y 0 (3) = 1. (8.27)

In that case, applying the conditions is less straightforward. The first condition, for example,
produces
1
X X 1
1n 2n 1n
y(3) = a0 3 + a1 32n+1 = 4. (8.28)
(2n)! (2n + 1)!
n=0 n=0

In this case, we see that it would have been advantageous to seek our power-series solution
about xo = 1,
1
X
y= an (x 1)n . (8.29)
n=0

Finally, note that cos(x) and sin(x) can be expressed in the Taylor series

x2 x4
cos(x) = 1 + ... (8.30)
2! 4!

x3 x5
sin(x) = x + ... (8.31)
3! 5!

These relationships allow us to write our final result as

y = 4 cos(x) sin(x), (8.32)

which is the same solution we would have obtained by solving the ODE using our methods
for constant-coefficient ODE’s.

Example 8.1 (Apply a power series solution to an ODE)

Solve the following ODE over the domain 4  x  1,

(x 1)y 00 + y 0 + 2(x 1)y = 0, (8.33)

subject to the boundary conditions y(4) = 5 and y 0 (4) = 0.

Solution:
We begin by dividing both sides of the ODE by (x 1),

1
y 00 + y 0 + 2y = 0. (8.34)
(x 1)
160 8.1. Power series solutions

so we can identify
1
p(x) = and q(x) = 2. (8.35)
(x 1)

Though p(x) is singular at x = 1, both functions are analytic for 4  x  1. Because the
boundary conditions are given at x = 4, we seek a power series solution about xo = 4,

1
X
y= an (x 4)n . (8.36)
n=0

The first and second derivative are consequently given by

1
X 1
X
n 1
0
y = an n(x 4) , y = 00
an n(n 1)(x 4)n 2 . (8.37)
n=1 n=2

Before substituting these expressions into the ODE, we first rewrite the ODE as
h i h i
3 + (x 4) y 00 + y 0 + 2 3 + (x 4) y = 0, (8.38)

where we have used the equality

x 1 = 3 + (x 4). (8.39)

Our series solution then produces

h 1
iX 1
X
2 1
3 + (x 4) n(n 1)an (x 4)n + nan (x 4)n
n=2 n=1
1 (8.40)
h iX
+ 2 3 + (x 4) an (x 4)n = 0,
n=0

Distributing and collecting like terms, we find that

1
X 1
X 1
X
2 1 1
3n(n 1)an (x 4)n + n(n 1)an (x 4)n + nan (x 4)n
n=2 n=2 n=1
1 1 (8.41)
X X
n n+1
+ 6an (x 4) + 2an (x 4) = 0.
n=0 n=0

We now manipulate the exponents and limits of each summations so we can combine
them into a single summation. If we define m = n 2, the first summation can be
Chapter 8. Non-constant Coefficient ODEs 161

written as
1
X
3(m + 2)(m + 1)am+2 (x 4)m . (8.42)
m=0

If we define m = n 1, the second summation can be written as

1
X
(m + 1)(m)am+1 (x 4)m . (8.43)
m=1

We can also change the lower limit to m = 0,

1
X 1
X
(m + 1)(m)am+1 (x 4)m = (m + 1)(m)am+1 (x 4)m , (8.44)
m=1 m=0

because (m + 1)m = 0 when m = 0. Applying m = n 1 to the third summation produces

1
X
am+1 (m + 1)(x 4)m . (8.45)
m=0

Finally, we apply m = n + 1 to the fifth summation, producing

1
X
2am 1 (x 4)m (8.46)
m=1

We then lower the limit to m = 0,

1
X 1
X
2am 1 (x 4)m = 2am 1 (x 4)m , (8.47)
m=1 m=0

by imposing a 1 = 0. We can now write our solution in the form

1 "
X #
2
3(n + 2)(n + 1)an+2 + (n + 1) an+1 + 6an + 2an 1 (x 4)n = 0. (8.48)
n=0

This requires
3(n + 2)(n + 1)an+2 + (n + 1)2 an+1 + 6an + 2an 1 = 0, (8.49)
 162 8.2. The Cauchy-Euler equation

which provides the recursive relationship

(n + 1)an+1 2an 2an 1

an+2 = . (8.50)
3(n + 2) (n + 2)(n + 1) 3(n + 2)(n + 1)

For n = 0, the recursive relation produces

a1 2a0 2a 1 a
a2 = = 1 a0 , (8.51)
3(2) (2)(1) 3(2)(1) 6

where we used our constraint that a 1 = 0. For n = 1, the recursive relation produces

2a2 a1 a0
a3 = . (8.52)
9 3 9

Using our result for a2 above, this can be written as

✓ ◆
2 a1 a1 a0 8 1
a3 = a0 = a1 + a0 . (8.53)
9 6 3 9 27 9

We can now express the solution as

h 1 i
y(x) = a0 1 (x 4)2 + (x 4)3 + ...
9 (8.54)
h 1 8 i
+ a1 (x 4) (x 4)2 (x 4)3 + ... .
6 27

Finally, applying our boundary conditions, we find a0 = 5 and a1 = 0.

8.2 The Cauchy-Euler equation

Cauchy-Euler equations are non-constant coefficient ODE’s of the form

d ny n 1y
1d
xn + C1 x n 1
+ ... + Cn y = 0. (8.55)
dxn dxn

Such equations appear often in engineering and physics when modeling systems using cylin-
drical and spherical coordinates. Cauchy-Euler equations are also important for the method
of Frobenius discussed later in section 8.3. To demonstrate the solution of Cauchy-Euler
equations, we will focus here on the second-order Cauchy-Euler equation of the form

x2 y 00 + C1 xy 0 + C2 y = 0. (8.56)
 Chapter 8. Non-constant Coefficient ODEs 163

The methods demonstrated are applicable, however, to higher-order Cauchy-Euler equations

as well.
Suppose we limit ourselves to the range 0  x  1. In that case, the Cauchy-Euler equa-
tion admits solutions of the form
y=x . (8.57)

Substituting this form in the ODE (8.56) produces

2
(1 C1 ) + C2 = 0. (8.58)

Applying the quadratic formula, we find is

p
1 C1 ± (1 C1 )2 4C2
= . (8.59)
2

Depending on the values of C1 and C2 , the quadratic formula can produce distinct real roots,
distinct complex roots, or repeated roots. These three cases are discussed separately in the
following sections.

8.2.1 Case 1: real and distinct roots

Consider the problem

x2 y 00 2xy 0 10y = 0. (8.60)

Setting y = x produces the polynomial

2
3 10 = 0. (8.61)

The solutions to this polynomial are 1 = 2 and 2 = 5. This means that both y = Ax 2 and
y = Bx5 are each solutions to the original ODE, where A and B are unknown constants that
require boundary conditions to determine. Since the original ODE is linear, the sum of any
number of solutions is also a solution. Therefore, we can write the final form for y as

A
y= + Bx5 . (8.62)
x2

8.2.2 Case 2: complex roots

Applying the solution y = x to the governing ODE

x2 y 00 2xy 0 + 4y = 0, (8.63)
 164 8.2. The Cauchy-Euler equation

produces the roots

p
3 7
= ±i . (8.64)
2 2

For simplicity, suppose we express these roots as

= ↵ ±i . (8.65)

This produces the solution

⇣ ⌘
y = Ax↵+i + Bx↵ i
= x↵ Axi + Bx i
(8.66)

where A and B are constants. To find a more manageable form of this solution, we will use
the identity
x = eln(x) . (8.67)

This allows us to express xi as

i
xi = e ln(x )
= ei ln(x)
(8.68)

This allows us to rewrite our solution as

⇣ ⌘
ln(x) i ln(x)
y = x↵ Ae i + Be (8.69)

Using the Euler formula, this can then be expressed as

h i
y = x↵ A cos( ln(x)) + B sin( ln(x)) , (8.70)

where A and B are new constants. Returning to our original example, the solution to the
given ODE is
" p ! p !#
3/2 7 7
y=x A cos ln(x) + B sin ln(x) . (8.71)
2 2

8.2.3 Case 3: repeated roots

In cases of a repeated root, we obtain only one solution of the form

y = Ax . (8.72)
 Chapter 8. Non-constant Coefficient ODEs 165

This occurs because the second solution is not of the form x . To find a second solution, we
seek the solution in the more general form

y = B(x) x , (8.73)

where is the repeated root obtained earlier, and B(x) is some arbitrary function of x.
Though we do not show the algebra here, substituting this form in the the Cauchy-Euler
equation produces
d 2 B dB
x + = 0. (8.74)
dx2 dx

To solve for B(x), we define a new function P(x)

dB
P(x) = , (8.75)
dx

and rewrite equation (8.74) as

dP
x + P = 0. (8.76)
dx

From Chapter 4, we can solve this first-order linear ODE to find

C
P= , (8.77)
x

where C is a constant. Using our definition for P above, we can integrate to find B as

B = C ln(x) + D, (8.78)

where C and D are constants. The full solution to the ODE can now be written as
h i
y = A + C ln(x) + D x , (8.79)

or more simply
h i
y = C ln(x) + D x . (8.80)

8.2.4 The negative domain

For negative values of x, we must consider that ln(x) becomes undefined. To handle this, we
perform the change of variable x = ⇠. With this substitution, we will define

y(x) = y( ⇠) = z(⇠). (8.81)

 166 8.3. The method of Frobenius

This allows us to write dy/dx using the chain rule as

dy dz d⇠ dz
= = . (8.82)
dx d⇠ dx d⇠

Similarly, we find that

" # " #
d 2y d dz d dz d⇠ d 2z
2
= = = . (8.83)
dx dx d⇠ d⇠ d⇠ dx d⇠ 2

We can now write the Cauchy-Euler equation as

d 2z dz
⇠2 2
+ C1 ⇠ + C2 z = 0. (8.84)
d⇠ d⇠

If we now limit our domain to ⇠ > 0, this ODE produces the same results as before. Since ⇠ >
0 represents the same domain as x < 0, this means that our solutions should be independent
of negative values. Therefore, we can generalize the way we write our solutions for the entire
domain of 1  x  1 as

y = A|x| 1 + B|x| 2 , (8.85)

h i
y = |x|↵ A cos( ln( |x| )) + B sin( ln( |x| )) , (8.86)
y = [A + B ln( |x| )]|x| . (8.87)

As before, each of these solutions is specific to the value(s) of that are derived from the
governing ODE. In order they should be used when is found to represent two real and
distinct roots, a complex conjugate pair, or a repeated root.

8.3 The method of Frobenius

Before we introduce the idea and applications for the method of Frobenius, it is necessary
that we remind the reader of some necessary vocabulary and introduce some new definitions.
Recall that for the ODE
y 00 + p(x)y 0 + q(x)y = 0, (8.88)

we say that p(x) and q(x) are “analytic” at x = xo if they can be represented as Taylor series
in some interval |x xo | < R. With this understanding of an analytic function, we introduce
the definitions for ordinary, singular, regular, and irregular points below.

Definition 8.2 (Ordinary vs singular points)

If p(x) and q(x) are analytic at x = xo , then the ODE in equation (8.88) can be solved
using a regular power series. The point xo is then called an ordinary point. Otherwise,
xo is called a singular point.
 Chapter 8. Non-constant Coefficient ODEs 167

Definition 8.3 (Regular vs irregular singular points)

Suppose xo = 0 is a singular point. We can still solve the ODE in equation (8.88) at xo
if we can multiply the ODE by x2 ,

x2 y 00 + x[xp(x)]y 0 + [x2 q(x)]y = 0, (8.89)

and find that xp(x) and x2 q(x) are analytic.

Using this idea, we say that xo is a regular singular point if (x xo )p(x) and (x xo )2 q(x)
are analytic at xo . Otherwise, xo is called an irregular singular point.

With these definitions in mind, we introduce the idea behind the method of Frobenius.
Suppose xo = 0 is a regular singular point and we have multiplied our ODE by x2 to write

x2 y 00 + x[xp(x)]y 0 + [x2 q(x)]y = 0. (8.90)

If we expand xp(x) and x2 q(x) in Taylor series as

1
X
xp(x) = pn x n , (8.91)
n=0

1
X
x2 q(x) = qn x n , (8.92)
n=0

where pn and qn are expansion coefficients, then we write

x2 y 00 + x[p0 + xp1 + ...]y 0 + [q0 + q1 x + ...]y = 0. (8.93)

As x ! 0, equation (8.93) can be approximated as

x2 y 00 + p0 xy 0 + q0 y = 0. (8.94)
 168 8.3. The method of Frobenius

Equation (8.94) is a Cauchy-Euler equation that admits at least one solution that behaves
like xr , where r is a constant. To recover this solution as x ! 0, we will seek a solution as
1
X
y = xr (a0 + a1 x + a2 x2 + ...) = an xn+r . (8.95)
n=0

To demonstrate this method, we will now work through an example. Starting with the
ODE

6x2 y 00 + 7xy 0 (1 + x2 )y = 0 for 0 < x < 1, (8.96)

we must first identify p(x) and q(x) by dividing each term by the leading coefficient of y 00 ,
i.e. 6x2 . This provides

7x 7 (1 + x2 )
p(x) = 2
= , q(x) = , (8.97)
6x 6x 6x2

showing that p(x) and q(x) are both singular at x = 0. But, if we multiply through by x2 , we
can show that
7 (1 + x2 )
xp(x) = , x2 q(x) = , (8.98)
6 6

are both analytic at x = 0. This means that x = 0 is a regular singular point and that the
method of Frobenius is appropriate to apply to this ODE.

Starting with our assumed solution

1
X
y= an xn+r = a0 xr + a1 xr+1 + a2 xr+2 + ..., (8.99)
n=0

we can di↵erentiate to find y 0 and y 00 . The results are

1
X
1
y 0 = a0 rxr + a1 (r + 1)xr + ... = (n + r)an xn+r 1 , and (8.100)
n=0
1
X
y 00 = (n + r)(n + r 1)a + nxn+r 2 . (8.101)
n=0

Note that the lower limit on all of these summations remains n = 0. Taking these three
expressions, we can rewrite the ODE as

1
X 1
X 1
X
6x2 (n + r)(n + r 1)an xn+r 2
+ 7x (n + r)an xn+r 1
(1 + x2 ) an xn+r = 0. (8.102)
n=0 n=0 n=0
 Chapter 8. Non-constant Coefficient ODEs 169

This expression can be expanded to

1
X 1
X
6(n + r)(n + r 1)an xn+r + 7(n + r)an xn+r
n=0 n=0
1 1 (8.103)
X X
n+r n+r+2
an x an x = 0.
n=0 n=0

Consider the last term from the previous expression. This is the only term which has
an exponent other than n + r. If we can manipulate the summation, then we can collect
the coefficients and write our expression in a more concise manner. To accomplish this set
m + r = n + r + 2. This is the equivalent to saying m = n + 2. Using this equality when n = 0,
m = 2, so
X1 1
X
an xn+r+2 = am 2 xm+r . (8.104)
n=0 m=2

If we set a 2 =a 1 = 0, then we can write this summation again with a lower limit of zero,
1
X
am 2 xm+r . (8.105)
m=0

Finally, with this manipulation the ODE becomes

1 (h
X i
)
n+r
6(n + r)(n + r 1) + 7(n + r) 1 an an 2 x = 0. (8.106)
n=0

For equation (8.106) to be true for all x in the domain, each of the coefficients must be equal
to zero,
h i
6(n + r)(n + r 1) + 7(n + r) 1 an an 2 = 0. (8.107)

If we consider the coefficient for n = 0 we are left with

⇣ ⌘
6r 2 + r 1 a0 = 0. (8.108)

For this expression to be true, either 6r 2 + r 1 = 0 or a0 = 0. If we suppose a0 = 0 then

y = xr (a0 + a1 x + a2 x2 + ...)
= xr (a1 x + a2 x2 + ...) (8.109)
r+1
=x (a1 + a2 x + ...).

Defining b
r = r + 1, we can show that this is the same as our original expansion

y = xbr (a1 + a2 x + ...). (8.110)

 170 8.3. The method of Frobenius

Consequently, we will force a0 , 0 and require 6r 2 + r 1 = 0. This produces the roots

1 1
r1 = , r2 = , (8.111)
2 3

which leads to two solutions. Let’s first consider r1 = 1/2. By rearranging equation (8.107)
and substituting in r1 , we can determine an expression for the coefficients

an 2
an = . (8.112)
6(n 1/2)2 + n 3/2

We have already covered the case for n = 0 above, so let’s examine this expression for the
next few cases

a 2
n = 1, a1 = =) a1 = 0, (8.113)
6(1 1/2)2 + 1 3/2

1
n = 2, a2 = a , (8.114)
14 0
1
n = 3, a3 = a =) a3 = 0. (8.115)
39 1

Continuing to evaluate these coefficients, we find that

" #
1/2 1 2 1 4
y1 (x) = a0 x 1+ x + x + ... . (8.116)
14 76(14)

Performing a similar analysis with r2 = 1/3, you’ll find that

" #
1/3 1 2 1 4
y2 (x) = a0 x 1+ x + x + ... . (8.117)
34 116(34)

Finally, our ultimate solution is obtained from a combination of these two independent so-
lutions. It can be written as

" # " #
1/2 1 2 1/3 1 2
y(x) = C1 x 1 + x + ... + C2 x 1 + x + ... , (8.118)
14 34

where C1 and C2 are two unique constants that cannot be determined without two indepen-
dent boundary conditions.
 Chapter 8. Non-constant Coefficient ODEs 171

8.3.1 Special complications for the method of Frobenius

Depending on the roots (i.e., r1 and r2 ) that are obtained during the application of the
method of Frobenius, certain complications can arise. These complications include

(1) If you have repeated roots. In this case, you can show that

1
X
y2 (x) = y1 (x) ln(x) + Cn xn+r . (8.119)
n=1

(2) When r1 and r2 di↵er by an integer, e.g. r1 r2 = 3. Here, a subtle issue arises depending
on the relative values of r1 and r2 . For additional information on this complication,
please refer to Chapter 4 of Advanced Engineering Mathematics by Greenberg [2].
9. The Legendre and Bessel Equations

9.1 Legendre equation 172

9.2 Bessel equation 176

In this chapter we introduce two famous non-constant-coefficient ODEs. The first, discussed
in Section 9.1, is called the Legendre equation. The second, discussed in Section 9.2, is called
the Bessel equation. These equations are important for several reasons. First, they both ap-
pear in important physical phenomena. The Legendre equation appears in steady heat con-
duction in a sphere, while the Bessel equation appears in transient heat conduction in a
disk. The Legendre and Bessel equations are also important because they both yield useful
families of orthogonal polynomials (see Chapter 1). Finally, both equations are pedagogi-
cal examples of solving non-constant-coefficient ODEs. we will see that the solution of the
Legendre equation is an example of a power series solution, while the solution of the Bessel
equation is an example of a Frobenius solution.

9.1 Legendre equation

The Legendre equation is the ODE

⇣ ⌘ d 2y dy
1 x2 2x + y = 0, 1 < x < 1, (9.1)
dx2 dx

where is a constant. Some textbooks write

the constant as = n(n+1), where n is another (a) z
R=c
(b)
constant. We use for now, but the choice z
= n(n + 1) becomes evident later. r
φ
The Legendre equation appears in di↵u- y
sion problems in spherical coordinates and is y
x
used to determine wave functions of electrons θ
x
orbiting an atom, electromagnetic potentials Figure 9.1
in spherical geometries, and heat transfer and

172
 Chapter 9. The Legendre and Bessel Equations 173

nuclear reactor physics. Consider steady heat conduction in the sphere of radius c sketched
in figure 9.1(a). Steady heat conduction satisfies

r2 T = 0, (9.2)

where T is the temperature. Suppose we consider the spherical coordinates (✓, , r) sketched
in figure 9.1(b). Let’s also consider the simpler case of “axisymmetric” heat conduction,
for which @T /@✓ = 0. In that case, equation (9.2) can be written as the partial di↵erential
equation
@2 T 2 @T 1 @2 T cot( ) @T
+ + + = 0, (9.3)
@r 2 r @r r 2 @ 2 r2 @

where
0  r  c, 0  ⇡. (9.4)

Suppose we seek a solution to this equation in the form T (r, ) = R(r)y( ), where R(r) is
a function of only r, and y( ) is a function of only . This is called “seeking a separable
solution.” We will learn more about that method of solving PDEs later in Chapter 10. Sub-
stituting this form in equation (9.3), we find that the equation can be rewritten as

r 2 R00 (r) + 2rR0 (r) y 00 ( ) + cot( )y 0 ( )

= , (9.5)
R(r) y( )

where the prime symbols denote di↵erentiation (R0 = dR/dr and y 0 = dy/d ). The left-hand-
side of equation (9.5) is only a function of r, while the right-hand-side is only a function of
. The only way this can be true is if both sides of the equation equal the same constant.
Suppose we call this constant ,

r 2 R00 (r) + 2rR0 (r) y 00 ( ) + cot( )y 0 ( )

= = , (9.6)
R(r) y( )

We can now replace our original PDE with two ODEs

r 2 R00 (r) + 2rR0 (r) R(r) = 0, y 00 ( ) + cot( )y 0 ( ) + y( ) = 0. (9.7)

The second ODE in (9.7) can then be transformed into a Legendre equation using the substi-
tution x = cos( ). This produces

⇣ ⌘ d 2 y(x) dy(x)
1 x2 2x + y = 0, 1 < x < 1. (9.8)
dx2 dx

Note that as varies from = 0 to = ⇡, the variable x varies between x = 1 and x = 1.

Equation (9.8) is identical to the Legendre equation (9.1).
 174 9.1. Legendre equation

Important conceptual point:

At this point, it’s very important to stress that the constant is unknown. In fact,
plays a similar role to the symbol used in solutions e x in solutions to constant
coefficient ODEs. It also plays a similar role to in our solutions of systems of ODEs.
In fact, the Legendre equation is often written as

⇣ ⌘ d 2y dy
x2 1 2
+ 2x = y. (9.9)
dx dx

or more succinctly
" #
⇣ ⌘ d2 d
2
x 1 + 2x y = y. (9.10)
dx2 dx
| {z }
L

or even more succinctly

Ly = y. (9.11)

We see that the Legendre equation is in fact an eigenvalue problem for the eigenvalue
and the eigenfunction y(x).

9.1.1 Power series solution

The Legendre equation was studied by such mathematicians as Adrien-Marie Legendre (1752-
1833) and Pierre-Simon Laplace (1749-1827), who both studied it for applications to gravi-
tational potentials and celestial mechanics. If we rewrite the Legendre equation (9.1) as

d 2y 2x dy
+ y = 0, (9.12)
dx2 (1 x2 ) dx (1 x2 )

we see that p(x) and q(x),

2x
p(x) = q(x) = , (9.13)
(1 x2 ) (1 x2 )

are analytic for 1 < x < 1. Note that this interval does not include the points x = ±1, where
p(x) and q(x) are singular. Surprisingly, we can nevertheless find solutions to the Legendre
equation that are valid in the entire interval 1  x  1 by seeking a solution in the form of
a regular power series about x = 0,

1
X
y(x) = ak x k . (9.14)
k=0
 Chapter 9. The Legendre and Bessel Equations 175

Substituting this power series into the Legendre equation produces the recurrence relation

k(k + 1)
ak+2 = a , (9.15)
(k + 1)(k + 2) k

where we have used the index k in our summation, instead of the usual n, because we will
use n for something else later. This relation produces the solution

y(x) = a0 y1 (x) + a1 y2 (x), (9.16)

where
(6 ) (20 )(6 )
y1 (x) = 1 x2 x4 x6 ..., (9.17)
2 24 720

and
(2 ) (12 )(2 )
y2 (x) = x + x3 + x5 + ... (9.18)
6 120

Now let’s go back to our physical application of heat conduction in a sphere. We saw in
that example that the constant appeared when we sought a solution to the PDE (9.3) in
the form of a separable solution. We are only interested in values of that can satisfy the
boundary conditions of our problem. These boundaries occur at x = ±1. It turns out that
random values of usually cause both y1 (x) and y2 (x) to tend to ±1 as x tends to ±1. In that
case, the only way to get a finite value for y(x) is to set both a0 and a1 to zero, which just gets
us the trivial solution.
There are specific values of , however, for which either y1 (x) or y2 (x) are bounded. Con-
sider y1 (x). We see that when = 0, all terms in y1 (x) are zero, except for the first term,

y1 (x) = 1. (9.19)

On the other hand, if = 6, then only the first two terms are non-zero

y1 (x) = 1 3x2 . (9.20)

Similarly, if = 20, then three terms survive. If we consider y2 (x), we see that a similar
situation occurs for = 2 and = 12. More generally, we can show that if

= n(n + 1), n = 0, 1, 2, ... (9.21)

the series solutions for either y1 (x) or y2 (x) will terminate, such that they are bounded at
x = ±1. These are the only values of that we care about, because they are the only values
that can satisfy our boundary conditions.
 176 9.2. Bessel equation

9.1.2 Legendre polynomials

The values = n(n +1) produce a family of orthogonal polynomials Pn (x), called the Legendre
Polynomials. For convenience, the polynomials are usually normalized so they equal Pn (x) = 1
at x = 1. The first four polynomials are

1⇣ 2 ⌘ 1⇣ 3 ⌘
P0 = 1, P1 = x, P2 = 3x 1 , P3 = 5x 3x . (9.22)
2 2

Knowing P0 (x) and P1 (x), the remaining polynomials can be generated using the recurrence
relation
(n + 1)Pn+1 (x) = (2n + 1)xPn (x) nPn 1 (x). (9.23)

Bounded solutions of the Legendre equation can be expressed as a sum of Legendre poly-
nomials
1
X
y(x) = bn Pn (x). (9.24)
n=0

The Legendre polynomials can be generated by the formula

" #
1 dn ⇣ 2 ⌘n
Pn (x) = n x 1 . (9.25)
2 n! dxn

The Legendre polynomials are orthogonal with respect to the inner product

Z1 8
< 0,
D E >
> i ,j
Pi (x), Pj (x) = Pi (x)Pj (x) dx = >
> (9.26)
: 2 i =j
2j+1
1

9.2 Bessel equation

10. Partial Di↵erential Equations

10.1 Preliminary concepts 177

10.2 The heat, Laplace, and Poisson equations 179

10.3 The wave equation 181

10.4 Solution by integration 182

10.5 Separation of variables 184

10.6 Sturm-Liouville Theory 187

10.7 Application to the 1D wave equation 200

10.8 Application to the 2D wave equation 204

10.9 Non-homogeneous PDEs 208

10.10 Non-homogeneous boundary conditions 212

In this course, we focus on four famous second-order PDEs called the Laplace, Poisson, heat,
and wave equations. We begin by discussing some preliminary concepts in Section 10.1, and
then derive the Laplace, Poisson, and heat equations in Section 10.2, and the wave equation
in Section 10.3.

10.1 Preliminary concepts

A di↵erential equation is called a partial di↵erential equation (PDE) if its dependent variable
u depends on multiple variables. For example, the equation below is an ordinary di↵erential
equation,
d 2u
= f (x), (10.1)
dx2

for which u(x) is a function of only x. The equation below, called the Poisson equation, is a
PDE,
@2 u @2 u
+ = f (x, y), (10.2)
@x2 @y 2

for which u(x, y) is a function of x and y. Common independent variables in PDEs include
space and time. The Poisson equation above involves derivatives only in space (x and y).
The equation below, called the advection equation, involves derivatives in both space (x)
and time (t),
@u @u
+c = 0. (10.3)
@t @x

We define the order of an ODE as the highest derivative therein. For example, the Poisson
equation (10.2) is second-order, while the advection equation (10.3) is first-order. Consider

177
 178 10.1. Preliminary concepts

the equation below, called the heat equation,

@u @2 u
=↵ 2. (10.4)
@t @x

Using our definition above, this is correctly labeled a second-order PDE. Very often, however,
you will here people say that the heat equation is “first-order in time, and second order in
space,” to distinguish between the highest derivative in time and space. That information is
often very useful. For example, it tells us that the PDE will require one initial condition for
u(x, t) in time, say at time t = t0 ,
u(x, t0 ) = g(x), (10.5)

and two boundary conditions in space, say at x = a and x = b

u(a, t) = h(t), u(b, t) = s(t). (10.6)

Note that the advection equation (10.3) is consequently both an initial value problem (IVP)
and a boundary value problem (BVP). Sometimes, you will hear people go so far as to call it
an “initial boundary value problem” (IBVP).

Just as for ODEs, we often write PDEs in terms of operators. For example, the Poisson
equation (10.2) can be written as
!
@2 @2
+ u(x, y) = f (x, y), (10.7)
@x2 @y 2
| {z }
L
or simply
Lu = f (x, y). (10.8)

PDEs that can be written as

Lu = 0, (10.9)

are called homogeneous, while those of the form

Lu = f (x, y, t), (10.10)

are called non-homogeneous. Linear ODEs involve linear operators, such as

" #
@2 @2
a(x, y) 2 + b(x, y) 2 u(x, y) = f (x, y). (10.11)
@x @y

All of the equations seen above are linear PDEs. Nonlinear equations, however, involve
operators such as
" #
@2 @2
a(x, y, u) 2 + b(x, y, u) 2 u(x, y) = f (x, y). (10.12)
@x @y
 Chapter 10. Partial Di↵erential Equations 179

A particularly famous example is the Burgers equation

@u @u @2 u
+u =↵ 2, (10.13)
@t @x @x

which is studied extensively for its similarity to the nonlinear Navier-Stokes equations used
in fluid mechanics.

10.2 The heat, Laplace, and Poisson equations

The heat, Laplace, and Poisson equations have

qn
numerous applications in solid mechanics,
fluid mechanics, heat and mass transfer, and
electromagnetism. Here we derive them using δy
the simple example of conductive heat trans- qw qe
fer in a solid body, such as that sketched in fig- y δx
ure 10.1. Consider the small solid element of
dimensions x by y by z, centered about the qs
point p, sketched as a dot in figure 10.1. The
z-direction points into the page. For simplic-
ity, we will assume there is no heat conduction
x
in the z-direction. Because the element is very
Figure 10.1
small, we can estimate its energy as E = mcv Tp ,
where m and cv are the element’s mass and specific heat, respectively, and Tp is the temper-
ature at point p, which is located at the centroid of the element. Conservation of energy
requires the time-rate-of-change E equal

dE d
= (mcv Tp ) = Ėin Ėout + Ėgen , (10.14)
dt dt

where Ėin and Ėout represent the rate of energy entering and leaving the element from its
faces, and Ėgen represents heat generation within the element.
Heat generation Ėgen can be approximated as

f p x y z, (10.15)

where f p is the rate of heat generation per unit volume. If heat only crosses the element’s
surface due to conduction, we can write Ėin Ėout as

(qw qe ) y z + (qs qn ) x z, (10.16)

where qe , qn , qw , and qs denote the conductive heat fluxes on the east, north, west, and south
faces of the element, as sketched in figure 10.1. These fluxes can be expressed in terms of
temperature using Fourier’s law of heat conduction, which states that the conductive heat
 180 10.2. The heat, Laplace, and Poisson equations

flux vector q is linearly proportional to the temperature gradient,

q = KrT , (10.17)

where K is the thermal conductivity, and the negative sign occurs because heat always flows
in the direction of decreasing temperature (from hot to cold). On the east face, the heat flux
vector is q = KrTe . However, only the x-component of this vector crosses the east face, such
that qe = K(@Te /@x). Applying Fourier’s law to the four faces, we find that

@Te @Tn @Tw @Ts

qe = K , qn = K , qw = K , qs = K , (10.18)
@x @y @x @y

where the subscripts denote the gradients evaluated on the four faces.
If we suppose that m, cv , and K are all constant, we can write the conservation of energy
equation as
! !
@Tp @Te @Tw @Tn @Ts
mcv =K y z +K x z + f p x y z. (10.19)
@t @x @x @y @y

If we express m as ⇢ x y z, where ⇢ is the element’s density, we can rewrite the above as

! !
@Tp K @Te @Tw K @Tn @Ts
⇢cv = + + fp . (10.20)
@t x @x @x y @y @y

If we then take the limit as x and y tend to zero,

! !
K @Te @Tw @2 T K @Tn @Ts @2 T
lim =K 2, lim =K 2, (10.21)
x!0 x @x @x @x y!0 y @y @y @y

the equation becomes,

@T
⇢cv = Kr2 T + f , (10.22)
@t

where
@2 @2
r2 = + , (10.23)
@x2 @y 2

is called the Laplacian.

In the case of steady heat conduction without heat sources, the above equation becomes
the Laplace equation,
r2 T = 0. (10.24)

The Laplace equation has second-order derivatives in both x and y. That means it requires
 Chapter 10. Partial Di↵erential Equations 181

a total of four boundary conditions. Suppose we solve the Laplace equation in the box
sketched in figure 10.2 with the temperature set to zero on all boundaries

T x=a
= 0, T x=b
= 0, T y=c
= 0, T y=d
= 0. (10.25)

The Laplace equation will only return the trivial solution T (x, y) = 0. If we apply homoge-
neous Neumann conditions on all boundaries,

@T @T @T @T
= 0, = 0, = 0, = 0, (10.26)
@x x=a @x x=b @y y=c @y y=d

then the Laplace equation produces T (x, y) = A, where A is any constant. This can be recast
into the trivial solution by defining the new temperature Tb = T A.
In order to get a non-trivial solution, the Laplace equation
y
must be solved with at least one non-homogeneous boundary
condition. If we include steady heat generation, the Laplace
equation is called the Poisson equation d

r2 T = f (x, y) . (10.27) c
x
a b
Because the Poisson equation is forced by the term f (x, y), it Figure 10.2
produces nontrivial solutions, even when T is set to zero on all
boundaries.
Finally, let’s consider the case of unsteady heat conduction without heat generation,

@T
⇢cv = Kr2 T . (10.28)
@t

This is called the heat equation, which is usually written as

@T
= ↵r2 T , (10.29)
@t

where ↵ = K/(⇢cv ) is called the thermal di↵usivity. In contrast to the Laplace and Poisson
equations, the heat equation requires an initial condition,

T t=0
= T0 (x, y). (10.30)

Note that if the solution of the heat equation tends to a steady-state Ts (x, y), then the steady-
state solution satisfies the Laplace equation r2 Ts = 0.

10.3 The wave equation

Another equation arising in physics and engineering is the wave equation. To demonstrate
its derivation, consider a string under constant tension ⌧ as sketched in figure 10.3(a). At
time t = 0, the string is disturbed, and then released, as sketched in figure 10.3(b). If we
 182 10.4. Solution by integration

define y(x, t) as the normal distance of the string from it’s initially undisturbed position,
what equation governs the subsequent evolution of y(x, t)?

We begin by applying Newton’s law, neglecting

gravity, to a small segment of the string in the
y-direction. Using the parameters defined in fig-
ure 10.4, this can be written as

@2 y @2 y
⌧ sin ✓1 + ⌧ sin ✓2 = m = s , (10.31)
@t 2 @t 2

where s is the length of the segment, and is the

string’s mass per unit length. For small angles of de-
flection, it can be shown that

@y
sin ✓ ⇡ , s ⇡ x. (10.32)
@x Figure 10.3

With these approximations, we can rewrite equa-

tion (10.31) as
" !#
1 @y @y @2 y
⌧ = . (10.33)
x @x x+ x @x x @t 2

Taking the limit as x ! 0,

@2 y 2
2@ y
= c , (10.34) Figure 10.4
@t 2 @x2

where
r
⌧
c= . (10.35)

Equation (10.34) is called the wave equation. The generalized form of is written as

@2 u
= c2 r2 u. (10.36)
@t 2

10.4 Solution by integration

In some cases, PDE’s can be solved using straightforward integration. As an example, con-
sider the PDE
@2 u
= 1, (10.37)
@x2
 Chapter 10. Partial Di↵erential Equations 183

where u(x, y) is a function of x and y. We can integrate both sides of this equation with
respect to x,
@u
= x + f (y). (10.38)
@x

However, because u is a function of x and y, we must add an unknown function f (y). Inte-
grating again, we find that
x2
u= + xf (y) + g(y), (10.39)
2

where g(y) is another function of y. To determine the functions f (y) and g(y), we require
more information. For example, perhaps we are given the boundary conditions

@u
= cos(y), u(1, y) = cos(y), (10.40)
@x x=0

The Neumann condition at x = 0 requires f (y) = cos(y)

@u
= f (y) = cos(y). (10.41)
@x x=0

The Dirichlet condition at x = 1 requires g(y) = 1/2

1
u(1, y) = + cos(y) + g(y) = cos(y). (10.42)
2

It’s worth noting that when we introduce f (y) and g(y) during the integration of the PDE,
what we really mean is that f and g are “at most” functions of y. They can also be constants,
as in the case of g(y) above.

PDE’s that can be solved by simple integration are

common in fluid mechanics. For example, consider
steady incompressible flow between two converging y
plates, as sketched in figure 10.5. One can argue that u(x,y)
h(x)
the fluid velocity u(x, y) in the x-direction satisfies the x
PDE
@2 u 1 dp
= , (10.43)
@y 2 µ dx
Figure 10.5
where µ is a constant, and dp/dx is the pressure gra-
dient. In this case, the pressure gradient is at most a function of x. Integrating twice with
respect to y, we find that
y 2 dp
u= + yf x) + g(x), (10.44)
2µ dx
 184 10.5. Separation of variables

where f and g are functions of x. These can be determined by the the “no-slip” boundary
condition, which requires u to be zero at y = ±h(x), where h(x) is the distance from the
centerline to the plates. After some algebra, we find that

1 dp ⇣ 2 ⌘
u= y h(x)2 . (10.45)
2µ dx

10.5 Separation of variables

In cases for which direct integration is not possible,

L
separation of variables is a more general technique
known to yield solutions to many linear PDE’s, in-
cluding the Laplace, Poisson, heat, and wave equa- x
tions. To demonstrate the method, let’s consider tran-
sient heat conduction in the solid bar sketched in fig-
ure 10.6. The bar is of length L, and insulated on Figure 10.6
its lateral surfaces such that its temperature can only
vary as a function T (x, t). Heat transport within the bar is governed by the heat equation

@T @2 T
=↵ 2, (10.46)
@t @x

where ↵ is the thermal di↵usivity of the solid. Let’s consider the case where the tempera-
tures of the bar at x = 0 and x = L are maintained at T = 0, and the bar is given an initial
temperature profile f (x) at time t = 0,

T (0, t) = 0, T (L, t) = 0, T (x, 0) = f (x). (10.47)

Equation (10.46) cannot be solved by simple integration. Instead, separation of variables

seeks a solution in the form
T (x, t) = X(x)F(t), (10.48)

where X(x) is a function of only x, and F(t) is a function of only t. Substituting this form
into the PDE (10.46) requires
X(x)F 0 (t) = ↵X 00 (x)F(t), (10.49)

which can be rearranged as

X 00 1 F 0
= . (10.50)
X ↵ F(t)

Note that the left-hand-side of equation (10.50) is only a function of x, while the right-hand-
side is only a function of t. The only way this can be true is if both sides equal the same
 Chapter 10. Partial Di↵erential Equations 185

constant,
X 00 1 F 0
= = , (10.51)
X ↵ F(t)

where is some unknown constant. The negative sign is placed in front of for convenience,
because it allows us to write the resulting equations for X(x) and F(t) as

X 00 + X = 0, F 0 + ↵F = 0. (10.52)

The boundary conditions require

T (0, t) = X(0)F(t) = 0, T (L, t) = X(L)F(t) = 0. (10.53)

If we try to satisfy these conditions by setting F(t) = 0, we get the trivial solution. Conse-
quently, we require
X(0) = 0, X(L) = 0. (10.54)

At this stage, separation of variables has produced a BVP for X(x) with well-defined
boundary conditions,

X 00 + X = 0, X(0) = 0, X(L) = 0. (10.55)

The same is not true, however, for the function F(t), because substituting the separable form
T (x, t) = X(x)F(t) into the initial condition requires

f (x)
F(0) = = , (10.56)
X(x)

where is some constant. This constrains X(x) to be a multple of f (x)

X(x) = f (x), (10.57)

which may not necessarily satisfy the BVP (10.55).

For now, we accept there is an issue with our initial condition for F(t), and focus instead
on the BVP (10.55) for X(x). Though this BVP is a simple constant coefficient ODE, its
solution is complicated by the fact that is unknown. For now we assume that is real. In
that case, we know from our experience solving constant coefficient ODEs that this BVP can
produce very di↵erent behavior if < 0, = 0, or > 0. We consider those three cases below.

Case 1: = 0. In this case, the ODE for X(x) reduces to

X 00 = 0, (10.58)
 186 10.5. Separation of variables

which has the solution

X = Ax + B. (10.59)

The conditions X(0) = X(L) = 0, however, produce the trivial solution A = B = 0.

Case 2: < 0. In this case, we set = µ2 , where µ > 0, and solve the ODE

X 00 µ2 X = 0, (10.60)

producing the solution

X = Aeµx + Be µx
. (10.61)

The conditions X(0) = X(L) = 0 again produce the trivial solution A = B = 0.

Case 3: > 0. In this case, we set = µ2 , where µ > 0, and solve the ODE

X 00 + µ2 X = 0, (10.62)

producing the solution

X = A cos(µx) + B sin(µx). (10.63)

The condition X(0) = 0 requires A = 0, while the condition X(L) = 0 requires

B sin(µL) = 0. (10.64)

If we satisfy this condition by setting B = 0, we recover the trivial solution. We must conse-
quently set sin(µL) = 0, which requires

µL = 0, ⇡, 2⇡, 3⇡... (10.65)

Note, however, that the case µL = 0 again produces the trivial solution. Consequently, we
require
⇡ 2⇡ 3⇡
µ= , , ..., (10.66)
L L L

We conclude that BVP (10.55) produces an infinite number of non-trivial solutions, for which
✓ ◆2 ✓ ◆
n⇡ n⇡x
n= , Xn (x) = sin , n = 1, 2, 3... (10.67)
L L

Now that we have determined , let’s turn our attention to the IVP for F(t), which we can
 Chapter 10. Partial Di↵erential Equations 187

write as
n2 ⇡ 2 ↵
F 0 (t) + F(t) = 0. (10.68)
L2

This has the solution

n2 ⇡ 2 ↵
t
F(t) = Ae L2 . (10.69)

Note that F(t) tends to zero as t ! 1.

We conclude that seeking a seperable solution T = X(x)F(t) produces an infinite number

of nontrivial solutions of the form
✓ ◆
n⇡x n⇡ 2
Tn (x, t) = sin e ↵( L ) t , n = 1, 2, 3, ... (10.70)
L

Unfortunately, none of these solutions satisfies the initial condition T (x, 0) = f (x), except in
the special cases where f (x) happens to equal Xn (x). However, because the governing PDE
is linear, we can seek a more general solution in the form of a superposition

1
X 1
X ✓ ◆
n⇡x n⇡ 2
T (x, t) = an Tn (x, t) = an sin e ↵( L ) t . (10.71)
L
n=1 n=1

The initial condition then requires,

1
X ✓ ◆
n⇡x
T (x, 0) = an sin = f (x). (10.72)
L
n=1

We now have a chance to satisfy our initial condition if we can find a combination of expan-
sion coefficients an that satisfies condition (10.72). It turns out that this is possible, because
the solutions Xn (x) are orthogonal functions generated by a Sturm-Liouville problem, as
discussed in the next section.

10.6 Sturm-Liouville Theory

The ODE (10.55) encountered earlier in section 10.5,

X 00 (x) + X(x) = 0, X(0) = 0, X(L) = 0, (10.73)

is an example of what is called a Sturm-Liouville problem. Sturm-Liouville theory originated

in a series of papers published by Charles-François Sturm (1803-1855) and Joseph Liouville
(1809-1882) in the 1830’s [3]. Sturm and Liouville observed that seeking separable solutions
to partial di↵erential equations led to BVP’s such as equation (10.92). When solving such
BVP’s, we are seeking values of that produce non-trivial solutions X(x). This search can be
 188 10.6. Sturm-Liouville Theory

interpretted as an eigenvalue problem by first rewriting equation (10.92) as

d 2X
= X, (10.74)
dx2

and then introducing the di↵erential operator

d2
L= . (10.75)
dx2

Equation (10.76) can then be written as

L (x) = (x), (10.76)

where we have replaced X(x) with the symbol (x) to be consistent with the notation used
later in this section. The values n that produce non-trivial solutions are called eigenval-
ues. The corresponding functions n (x) are called an eigenfunctions. Given such eigenvalue
problems, Sturm and Liouville studied the resulting solutions, and under what conditions
one could expand an arbitrary function f (x) in an infinite series of the eigenfunctions

1
X
f (x) = an n (x). (10.77)
n=1

Here we only briefly summarize the relevant results of Sturm and Liouville. For deeper
discussion, we refer to the review by Lützen [3].
“Regular” Sturm-Liouville (SL) problems are second-order ODEs of the form
" #
d d
p(x) + q(x) + w(x) = 0, a  x  b, (10.78)
dx dx

subject to the homogeneous Robin boundary conditions

↵ (a) + 0 (a) = 0,
0 (b) = 0, (10.79)
(b) +

where p(x), p 0 (x), q(x), and w(x) are all continuous on x 2 [a, b]. Furthermore, a, b, p(x),
q(x), w(x), ↵, , , and are all real. Finally, we require p(x) > 0 and w(x) > 0 on x 2
[a, b]. In addition to the regular SL problems above, there are additional SL problems called
“singular” and “periodic” SL problems. These are not discussed here.
The Sturm-Liouville problem can be interpretted as an eigenvalue problem by rewriting
it as
" #
p(x) d 2 p 0 (x) d q(x)
= , (10.80)
w(x) dx2 w(x) dx w(x)
| {z }
L
 Chapter 10. Partial Di↵erential Equations 189

or simply
L = . (10.81)

Equation (10.92) is a regular Sturm-Liouville problem for which

p(x) = w(x) = 1, q(x) = 0, (10.82)

and
↵= = 1, = = 0. (10.83)

Theorem 10.1 (Sturm-Liouville Theorem)

Let’s denote the eigenvalues of the SL problem as j , where j = 0, 1, 2, .... For each
eigenvalue, let’s denotes the eigenfunction as j (x). The following four statements are
true:

(i) There are an infinite number of eigenvalues j , and they are all real. The eigen-
values can be ordered in increasing magnitude such that

1 < 2 < 3 < ... < j (10.84)

where j ! 1 as j ! 1. By the above, we also mean there is a unique 1 of

smallest value.

Statement (i) simply means that there are an infinite number of eigenvalues, and they
can be ordered such that they increase with j, and tend to infinity as j ! 1.

(ii) For each eigenvalue j , there is one corresponding eigenfunction j (x). Further-
more, j (x) is linearly independent from all the other eigenfunctions.

(iii) The eigenfunctions are orthogonal from each other with respect to the inner
product
(
D E 0, when j , k
j (x), k (x) = (10.85)
k j (x)k2 , when j = k

Where

D E Zb
j (x), k (x) = j (x) k (x) w(x) dx. (10.86)
a

Just like for vectors, the magnitude of the eigenfunction is the square root of the
 190 10.6. Sturm-Liouville Theory

inner product between the eigenfunction and itself:

Zb
2
k j (x)k = j (x) j (x) w(x) dx. (10.87)
a

Though the eigenvalues j are always real, the eigenfunctions j (x) can be com-
plex. In that case, the inner product is written as

D E Zb
j (x), k (x) = j (x) k (x) w(x) dx, (10.88)
a

where the overline denotes the complex conjugate.

Statements (ii) and (iii) basically state that the eigenfunctions form a family of orthogonal
functions. The functions are orthogonal with respect to an inner product, and the weight
function w(x) in the inner product is the same w(x) appearing in the SL di↵erential equation.
It’s important to note that though the eigenvalues are all real, the eigenfunctions can be
complex. For example, sin(x) and cos(x) often appear as eigenfunctions. For convenience it
is often simpler to write these in terms of eix . We will learn more about that when we explore
Fourier series.

(iv) Suppose a function f (x) and its deriva- f(x)

tive f 0 (x) are “piecewise continuous”
on the domain a  x  b. A func- f(x-)
tion is called piecewise continuous on
an interval if the interval can be bro- f(x+)
ken into a finite number of subintervals
on which the function is continuous, as
sketched in figure 10.7. The Sturm-
Liouville theorem guarantees that in a x b
the interval a < x < b, the series Figure 10.7

1
X
aj j (x), (10.89)
j=0

where
D E
j (x), f (x)
aj = D E, (10.90)
j (x), j (x)

converges to f (x) if the function is continuous at x. At points of discontinuity,

 Chapter 10. Partial Di↵erential Equations 191

the expansion converges to

f (x ) + f (x+)
. (10.91)
2

Example 1: We already solved a Sturm-Liouville problem in section 10.5,

00
(x) + = 0, (0) = 0, (L) = 0, (10.92)

and found that

✓ ◆2 ✓ ◆
j⇡ j⇡x
j = , j (x) = sin , j = 1, 2, 3... (10.93)
L L

Let’s now confirm that the eigenfunctions j (x) are indeed orthogonal. Because w(x) = 1, the
inner product of two eigenfunctions is given by

ZL ZL ✓ ◆ !
D E j⇡x k⇡x
j (x), k (x) = j (x) k (x) dx = sin sin dx. (10.94)
L L
0 0

There are several methods of evaluating integrals like this. One is to express the sinusoidal
functions in terms of complex exponential functions,

✓ ◆ ! 0 i j⇡x j⇡x
10 1
j⇡x k⇡x BB e L e i L CC BB ei k⇡x
L e i k⇡x
L C
CC
sin sin = BB@B C
CC BB@ CA (10.95)
L L 2i A 2i

This can be rearranged as

(j k)⇡ (j k)⇡ (j+k)⇡ (j+k)⇡

ei L x +e i L x ei L x +e i L x
. (10.96)
4 4

The integral can then be expressed in cosines:

ZL 8
> " # " #9
D 1 E >
< (j k)⇡ (j + k)⇡ >>
=
j (x), k (x) = >
> cos x cos x >
> dx. (10.97)
2 : L L ;
0

If j = k, then

D E ZL " ✓ ◆#
1 2j⇡ L
j (x), j (x) = 1 cos x dx = . (10.98)
2 L 2
0
 192 10.6. Sturm-Liouville Theory

However, if j , k, then you find that

D E
j (x), k (x) = 0. (10.99)

We conclude that
(
D E 0, j ,k
j (x), j (x) = L (10.100)
2, j = k.

Example 2: Expand the function f (x) = x3 x4 be- (a)

tween 0  x  1, shown in figure 10.8(a), in terms of 0.12

the eigenfunctions j (x) = sin(j⇡x), i.e. 0.1

0.08
N
X f (x)
f (x) = aj j (x). (10.101) 0.06

j=0 0.04

0.02
Let’s first use this as an opportunity to prove that
0
0 0.2 0.4 0.6 0.8 1
D E
j (x), f (x)
(b) x

aj = D E, (10.102) 0.12
j (x), j (x) 0.1
f(x)
N=1
N=2
0.08 N=4

which is given in statement (iv) of the Sturm- 0.06

f (x)
Liouville theorem. We start by multiplying equa- 0.04
tion (10.101) with i (x),
0.02

0
N
X
i (x)f (x) = aj i (x) j (x), (10.103) -0.02
0 0.2 0.4 0.6 0.8 1
j=0 x

Figure 10.8
and then integrate over the domain 0  x  1

Z1 N
X Z1
i (x)f (x)dx = aj i (x) j (x)dx. (10.104)
0 j=0 0

Due to orthogonality, all terms in the sum are zero, except for the case where j = i

Z1 Z1
i (x)f (x)dx = ai i (x) i (x)dx. (10.105)
0 0
 Chapter 10. Partial Di↵erential Equations 193

This is equivalent to
⌦ ↵ ⌦ ↵
i (x), f (x) = ai i (x), i (x) , (10.106)

which is the same result

D given
E in the Sturm-Liouville theorem. We already know from an
earlier example that j , j = 1/2, so we must evaluate the integral

Z1 ⇣ ⌘
aj = 2 sin(j⇡x) x3 x4 dx. (10.107)
0

This integral can be evaluated analytically, with some patience. As an exercise, let’s compute
it numerically using the trapezoidal rule. The following matlab function Coe↵.m computes
the j’th coefficient aj using the trapezoidal rule over N subintervals. I found that N = 200
provided excellent accuracy. The subsequent function Expansion.m calls Coe↵.m to approx-
imate f (x) using the finite expansion (10.101). Figure 10.8(b) compares f (x) (black line) with
the expansion using N = 1, 2, and 4 terms. Though not shown, the expansion is virtually
indistinguishable from f (x) when N = 6.

1 function[a] = Coeff(j,N)
2
3 dx = 1/N;
4 x = linspace(0,1,N+1);
5
6 I = 2*sin(j*pi*x).*(x.^3≠x.^4);
7
8 a=0;
9 for i=1:N
10 a = a + 0.5*dx*(I(i)+I(i+1));
11 end

1 function[] = Expansion(N)
2
3 x = linspace(0,1,200);
4 f = x.^3≠x.^4;
5
6 Exp=0*x;
7 for j=1:N
8 a = Coeff(j,200);
9 Exp = Exp + a*sin(j*pi*x);
10 end
11
12 figure
13 set(gca,'FontName','Times','FontSize',20)
14 hold on
15 plot(x,f,'≠k','LineWidth',2.0)
16 plot(x,Exp,'≠≠b','LineWidth',2.0)
 194 10.6. Sturm-Liouville Theory

17 xlabel('$x$','Interpreter','latex')
18 ylabel('$f(x)$','Interpreter','latex','rot',0)
19 grid on
20 box on

Example 3: Let’s now leverage Sturm-Liouville theory to complete our solution of the PDE
considered in section 10.5,
@T @2 T
=↵ 2, (10.108)
@t @x

T (0, t) = 0, T (L, t) = 0, T (x, 0) = f (x). (10.109)

Seeking a separable solution T (x, t) = X(x)F(t), produces an infinite number of nontrivial

solutions of the form
✓ ◆
n⇡x n⇡ 2
Tn (x, t) = sin e ↵( L ) t , n = 1, 2, 3, ... (10.110)
L

To satisfy the initial condition, we seek a more general solution in the form of a superposition

1
X 1
X ✓ ◆
n⇡x n⇡ 2
T (x, t) = an Tn (x, t) = an sin e ↵( L ) t . (10.111)
L
n=1 n=1

The initial condition then requires,

1
X ✓◆
n⇡x
an sin = f (x). (10.112)
L
n=1

To determine the coefficients an , we leverage the Sturm-Liouville theorem, which states

that the eigenfunctions n (x) are orthogonal with respect to the inner product

ZL 8
D E >
> 0, j ,n
<
j (x), n (x) = j (x) n (x) dx = >
> , (10.113)
:k 2 (x)k, j = n
0 j

where
ZL ✓ ◆
2 j⇡x L
k j (x)k = sin2 dx = . (10.114)
L 2
0

We can now determine the coefficients an by multiplying equation (10.72) with the eigen-
 Chapter 10. Partial Di↵erential Equations 195

(a) (b)
1
1 t=0

0.8
0.8 0.001

0.6 T (x, t)
f (x) 0.6
0.005
0.4
0.4
0.02
0.2 0.2
0.1

0 0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
x x

Figure 10.9

function j (x) and integrating over the domain

1
X ZL ZL
an j (x) n (x) dx = j (x)f (x) dx. (10.115)
n=1 0 0

Orthogonality requires
ZL
L
ai = i (x)f (x) dx, (10.116)
2
0

from which we conclude that

ZL
2
aj = j (x)f (x) dx. (10.117)
L
0

We conclude that the PDE and associated boundary and initial conditions are satisfied by
the solution

1
X ✓◆ ZL ✓ ◆
n⇡x n⇡ 2 2 n⇡x
T (x, t) = an sin e ↵( L ) t , an = sin f (x) dx. (10.118)
L L L
n=1 0

As a more concrete example, let’s consider the heat equation when L = 1, and the initial
 196 10.6. Sturm-Liouville Theory

temperature profile is set to

" ✓ ◆2 #
1
f (x) = exp 200 x . (10.119)
2

This produces the profile in figure 10.9a. Using orthogonality of the eigenfunctions, we
can easily compute the coefficients an using numerical integration, as demonstrated earlier.
We can then plot T (x, t) at any given time. Figure 10.9b shows the resulting temperature
profiles at times t = 0, 0.001, 0.005, 0.02, and 0.1 when we set ↵ = 1. At early times, t 
0.005, the temperature gradient @T /@x is large. This cause the maximum temperature to
decrease rapidly, from T = 1 at t = 0 to around T = 0.45 at t = 0.005. For t > 0.005, the
temperature decays more slowly. At the final time, the solution is essentially equal to the
first eigenfunction,

1 = sin(⇡x), (10.120)

because the first eigenfunction has the slowest decay rate,

↵⇡ 2 t
e . (10.121)

Example 4: Heat generation in a metal bar of length L = 1 satisfies the PDE

@T @2 T
= (10.122)
@t @x2

subject to the boundary conditions

@T
T (0, t) + (0, t) = 0, T (1, t) = 0, (10.123)
@x

and initial condition

T (x, 0) = f (x). (10.124)

a) Seek a separable solution and show that the solution to the PDE simplifes to two ODEs.
Show the boundary/initial conditions for the ODEs, and identify the resulting Sturm-Liouville
problem.

b) Solve for the eigenvalues n of the Sturm-Liouville problem. The final relationship you
find for n cannot be solved explicitly, but you can show the resulting n graphically.

Solution: As already demonstrated in section 10.5, seeking a separable solution of the form
 Chapter 10. Partial Di↵erential Equations 197

T (x, t) = X(x)F(t) produces the ODE’s

X 00 + X = 0, F 0 + ↵F = 0. (10.125)

The boundary condition at x = 0 requires

X(0)F(t) + X 0 (0)F(t) = 0. (10.126)

To avoid a trivial solution, we must require

X(0) + X 0 (0) = 0. (10.127)

Similarly, the boundary condition at x = 1 requires

X(1) = 0. (10.128)

We conclude that X(x) is given by the BVP problem

X 00 + X = 0, X(0) + X 0 (0) = 0, X(1) = 0. (10.129)

Comparing with our definition of a regular SL problem earlier in this section, we see this is
a regular SL problem for which

p(x) = w(x) = 1, q(x) = 0, ↵= = = 1, = 0. (10.130)

The IVP for F(t), however, is not as clearly define because the initial condition T (x, 0) = f (x)
requires
T (x, 0) = X(x)F(0) = f (x), (10.131)

which implies
f (x)
F(0) = . (10.132)
X(x)

In actuality, we will enforce the initial condition T (x, 0) = f (x) by seeking a more general
solution to T (x, t) in the form of a superposition.

To solve the SL problem, we must consider the three cases < 0, = 0, and > 0.

Case 1: = 0. In this case, the ODE for X(x) reduces to

X 00 = 0, (10.133)
 198 10.6. Sturm-Liouville Theory

which has the solution

X 0 = A, X = Ax + B, (10.134)

The condition X(0) + X 0 (0) requires

X = A + B = 0, (10.135)

such that B = A, and

X = A(x 1). (10.136)

This automatically satisfies the condition X(1) = 0, such that = 0 is an eigenvalue, for which
the corresponding eigenfunction is
X = x 1. (10.137)

Case 2: < 0. In this case, we set = µ2 . The resulting solution can be written as

X = A cosh(µx) + B sinh(µx), X 0 = µA sinh(µx) + µB cosh(µx) (10.138)

The Robin condition requires

X 0 (0) + X(0) = µB + A = 0, (10.139)

from which we conclude that B = A/µ. Applying X(1) = 0 requires

A
X(1) = A cosh(µ) sinh(µ) = 0. (10.140)
µ

Assuming A , 0, this requires 1.5

1
cosh(µ) sinh(µ) = 0, (10.141) 1
µ

0.5
or equivalently

µ tanh(µ) = 0. (10.142) 0
0 0.5 1 1.5 2

This, however is only true when µ = 0, which can be Figure 10.10

seen in figure , showing a plot of µ tanh(µ) vs. µ.

Case 3: > 0. In this case, we set = µ2 , where µ > 0, producing

X(x) = A cos(µx) + B sin(µx), X 0 (x) = µA sin(µx) + µB cos(µx) (10.143)

 Chapter 10. Partial Di↵erential Equations 199

The Robin condition at x = 0 requires

X(0) + X 0 (0) = A + µB = 0. (10.144)

Setting A = µB, the Dirichlet condition at x = 1 requires

X(1) = µB cos(µ) + B sin(µ) = 0. (10.145)

To avoid a trivial solution, we must assume B , 0. We

14
consequently require 12
10
µ cos(µ) + sin(µ) = 0, (10.146) 8
6
4
which can be written as
2
0
tan(µ) = µ, (10.147) -2
0 5 10 15

Figure 10.11
Though we cannot solve this relation explicitly for µ,
we can show the result graphically as in figure 10.11, showing the roots of the equation as
the points where plots of tan(µ) and µ intersect.
 200 10.7. Application to the 1D wave equation

10.7 Application to the 1D wave equation

As a second demonstration of separation of variables,

we consider the motion of waves along the string
sketched in figure 10.12. The vertical position of the (a)
string u(x, y) is governed by the wave equation,
f(x)
@2 u 2
2@ u L
= c , (10.148)
@t 2 @x2
x

where c2 = ⌧/ is the ratio between the string tension

⌧ and mass per length . Because the wave equation
(b)
is second-order in x, it requires two boundary con-
ditions. When the string is fixed at both ends, as in
figure 10.12(a), the string satisfies the boundary con-
ditions,

u(0, t) = 0, u(L, t) = 0. (10.149) Figure 10.12

If the right-end of the string is free to translate along the right boundary without friction, as
in figure 10.12(b), the string satisfies,

@u
u(0, t) = 0, = 0. (10.150)
@x x=L

Because the wave equation is second-order in time, it requires two initial conditions. Let’s
consider the case where the string is initially displaced such that

u(x, 0) = f (x). (10.151)

Furthermore, if the string is initially at rest, and suddenly released at t = 0, it must also
satisfy
@u
= 0. (10.152)
@t t=0

Seeking a solution in the form u(x, y) = X(x)F(t), we find that

X 00 1 F 00
= 2 = , (10.153)
X c F(t)

which requires
X 00 + X = 0, F 00 + c2 F = 0. (10.154)

Let’s consider the case where the string is fixed at x = 0 but free to translate along x = L. This
 Chapter 10. Partial Di↵erential Equations 201

requires
@u
u(0, t) = X(0)F(t) = 0, = X 0 (L)F(t) = 0.
@x x=l

These produce the following conditions for X(x),

X(0) = 0, X 0 (L) = 0. (10.155)

These boundary conditions produce a Sturm-Liouville problem for X(t). Meanwhile, the
initial conditions require

@u
u(x, 0) = X(x)T (0) = f (x), = X(x)T 0 (0) = 0. (10.156)
@t t=0

These conditions do not produce a Sturm-Liouville problem for F(t). The reason is that (i)
the conditions on F(t) are not boundary conditions (because they are both applied at t = 0),
and (ii), the condition X(x)T (0) = f (x) is a non-homogeneous condition. Sturm-Liouville
problems must have homogeneous boundary conditions.
We consequently start by solving the Sturm-
Liouville problem 1

X 00 + X = 0.
0.5
(10.157)
φn (x)
0

Just as in our previous application to the heat equa- φ1(x)

-0.5 φ2(x)
tion, this produces trivial solutions when  0. For φ3(x)
> 0, however, it produces the solution -1
0 0.2 0.4 0.6 0.8 1
x/L
X = A cos(µx) + B sin(µx). (10.158)
Figure 10.13

The condition X(0) = 0 requires A = 0, while the condition X 0 (L) = 0 requires

Bµ cos(µL) = 0. (10.159)

For non-trivial solutions, this requires

⇡ 3⇡ 5⇡
µL = , , ... (10.160)
2 2 2

This produces the eigenvalues

✓ ◆2 ✓ ◆2 ✓ ◆2
⇡ 3⇡ 5⇡
= , , ... (10.161)
2L 2L 2L

and eigenfunctions
✓ ◆ ✓ ◆ ✓ ◆
⇡x 3⇡x 5⇡x
(x) = sin , sin , sin , ... (10.162)
2L 2L 2L
 202 10.7. Application to the 1D wave equation

These can be written more succinctly as

!
(2n 1)2 ⇡2 (2n 1)⇡x
n= , n (x) = sin , n = 1, 2, 3... (10.163)
4L 2L

The first three eigenfunctions are plotted in figure 10.13.

We now turn our attention to the ODE for F(t)

2
F 00 + nc F = 0. (10.164)

Using the usual procedure for constant-coefficient ODEs, we find this has the solution
⇣ p ⌘ ⇣ p ⌘
F(t) = A cos c nt + B sin c nt , (10.165)

for which the initial condition F 0 (0) requires B = 0. We conclude that

⇣ p ⌘
Fn (t) = A cos c nt . (10.166)

We now seek a solution to u(x, t) in the form of a superposition

1
X
u(x, t) = an n (x)Fn (t) (10.167)
n=1

for which the initial condition requires

1
X
u(x, 0) = an n (x) = f (x). (10.168)
n=1

The Sturm-Liouville theorem guarantees that n (x) is a family of orthogonal functions,

8
D >
> E 0, j ,n
<
j (x), n (x) = >
> 2, j = n (10.169)
:k j (x)k

A little math shows that k j (x)k = L/2. Consequently, to find the coefficients an , we take the
inner product of the initial condition (10.168) with j

1
X D E D E
an j (x), n (x) = j,f (x) . (10.170)
n=1

from which we find

2D E
aj = j , f (x) . (10.171)
L
 Chapter 10. Partial Di↵erential Equations 203

Figure 10.14

Figure 10.14 shows the result when we set c = 1, L = 1 and

" ✓ ◆2 #
1
f (x) = exp 200 x . (10.172)
2

This generates a pair of rightward and leftward traveling waves that interact with the left
and right boundaries.
 204 10.8. Application to the 2D wave equation

10.8 Application to the 2D wave equation

Consider a drum made by stretching a membrane

over a rectangular frame with dimensions Lx by Ly , y
as sketched in figure 10.15. The displacement of the
membrane in the z-direction (out of the page) is gov-
erned by the PDE Ly
!
@2 u 2
2 @ u @2 u
=c + , (10.173) x
@t 2 @x2 @y 2
Lx
Figure 10.15
where c is a positive constant. We suppose the drum
satisfies the homogeneous Dirichlet conditions

u x=0,Lx
= 0, u y=0,Ly
= 0. (10.174)

We assume the drum is released from rest with an initial displacement f (x, y), such that

@u
u(x, y, 0) = f (x, y), = 0. (10.175)
@t t=0

We begin by seeking the separable solution u(x, y, t) = X(x)Y (y)F(t). Subsituting this form
in the PDE (10.173), produces
h i
X(x)Y (y)F 00 (t) = c2 X 00 (x)Y (y)F(t) + X(x)Y 00 (y)F(t) (10.176)

which can be reordered as

1 F 00 (t) X 00 (x)Y (y) + X(x)Y (y)00

= (10.177)
c 2 F(t) X(x)Y (y)

Because the left-hand-side is only a function of t, while the right-hand-side is only a function
of x and y, both sides must be equal a constant, which we set to so that

F 00 (t) + c2 F(t) = 0, (10.178)

and
X 00 (x)Y (y) + X(x)Y (y)00
= . (10.179)
X(x)Y (y)

Equation (10.189) can in turn be rearranged as

X 00 (x) Y 00 (y)
= . (10.180)
X(x) Y (y)
 Chapter 10. Partial Di↵erential Equations 205

Once again, we can set both sides of this equation to a constant. Choosing ⌘ produces

X 00 + ⌘X = 0, Y 00 + ( ⌘)Y = 0. (10.181)

The boundary conditions then require

X(0) = 0, X(Lx ) = 0, Y (0) = 0, Y (Ly ) = 0. (10.182)

We begin by solving the Sturm-Liouville problem

X 00 + ⌘X = 0, X(0) = 0, X(Lx ) = 0. (10.183)

From section 10.5, we already know this problem produces the non-trivial solutions
!2 !
n⇡ n⇡x
⌘n = , Xn (x) = sin , n = 1, 2, 3... (10.184)
Lx Ly

Next we write the ODE for Y (y) in the form of a second Sturm-Liouville problem by intro-
ducing the eigenvalue = ⌘,

Y 00 + Y = 0, where = ⌘. (10.185)

Again, from section 10.5, this produces the non-trivial solutions

!2 !
m⇡ m⇡y
m= , Ym (y) = sin , m = 1, 2, 3... (10.186)
Ly Ly

Note that we use the index m to distinguish the eigenvalues m from the eigenvalues ⌘n .
Solving for , we see that
!2 !2
n⇡ m⇡
nm = ⌘n + m = + . (10.187)
Lx Ly

Combining our results for X(x) and Y (y), we see that the spatial variation of the drum is
given by
! !
n⇡x m⇡y
Xn (x)Ym (y) = sin sin . (10.188)
Lx Ly

These eigenfunction combinations are referred to as modes of vibration. Figure 10.16 shows
the first four modes for which (n, m) = (1, 1) (panel a), (n, m) = (2, 1) (panel b), (n, m) = (1, 2)
(panel c), (n, m) = (2, 2) (panel d).
 206 10.8. Application to the 2D wave equation

(a) (b)

x x
y y

(c) (d)

x x
y y

Figure 10.16: The first four drum modes for which (n, m) = (1, 1) (panel a), (n, m) = (2, 1) (panel b),
(n, m) = (1, 2) (panel c), (n, m) = (2, 2) (panel d).

We can now express the ODE for F(t) as

2
F 00 (t) + !nm F(t) = 0, (10.189)

p
where !nm = c nm . Recall that c and nm are both positive. This has the solution

Fnm = Anm cos(!nm t) + Bnm sin(!nm t). (10.190)

We can now build a more general solution to our PDE in the form of a superposition

1 "
1 X
X # ! !
n⇡x m⇡y
u(x, y, t) = Anm cos(!nm t) + Bnm sin(!nm t) sin sin . (10.191)
Lx Ly
n=1 m=1

The initial condition for @u/@t requires

1 X
X 1 ! !
@u n⇡x m⇡y
= !nm Bnm sin sin = 0. (10.192)
@t t=0 Lx Ly
n=1 m=1

Because the eigenfunctions are linearly independent, this requires Bnm = 0. The remaining
 Chapter 10. Partial Di↵erential Equations 207

initial condition requires

1 X
X 1 ! !
n⇡x m⇡y
u(x, y, 0) = Anm sin sin = f (x, y). (10.193)
Lx Ly
n=1 m=1

To determine the expansion coefficients Anm , we multiply both sides of the initial condi-
tion by sin(j⇡x/Lx ) and integrate in the x-direction from 0 to Lx ,

ZLx ! 1 X
X 1 ! ZLx ! !
j⇡x m⇡y n⇡x j⇡x
f (x, y) sin dx = Anm sin sin sin dx (10.194)
Lx Ly Lx Lx
0 n=1 m=1 0

Using orthogonality, this produces

ZLx ! 1
X !
j⇡x m⇡y Lx
f (x, y) sin dx = Ajm sin . (10.195)
Lx Ly 2
0 m=1

Next, we multiply both sides of the above by sin(k⇡y/Ly ) and integrate in the y-direction
from 0 to Ly ,

ZLy ZLx ! ! 1
X ZLy ! !
j⇡x k⇡y Lx m⇡y k⇡y
f (x, y) sin sin dxdy = A sin sin dy. (10.196)
Lx Ly 2 jm Ly Ly
0 0 m=1 0

From orthogonality, this produces

ZLy ZLx ! !
j⇡x k⇡y L Ly
f (x, y) sin sin dxdy = Ajk x . (10.197)
Lx Ly 2 2
0 0

We conclude that

1 X
X 1 ! !
n⇡x m⇡y
u(x, y, t) = Anm cos(!nm t) sin sin , (10.198)
Lx Ly
n=1 m=1

where

ZLy ZLx ! !
4 n⇡x m⇡y
Anm = f (x, y) sin sin dxdy (10.199)
Lx Ly Lx Ly
0 0
 208 10.9. Non-homogeneous PDEs

10.9 Non-homogeneous PDEs

Suppose we reconsider the heat equation solved in
Section 10.5, but with the addition of a volumetric L
heating term q(x, t),
x
@T @2 T
= ↵ 2 + q(x, t), (10.200)
@t @x
Figure 10.17
subject to the same boundary conditions as in Section
10.5.
T (0, t) = 0, T (L, t) = 0, T (x, 0) = f (x). (10.201)

The addition of the heating term makes the PDE non-homogeneous, such that we can no
longer perform separation of variables as before. Indeed, if we try to set T = X(x)F(t), we
find that
X(x)F 0 (t) = ↵X 00 (x)F(t) + q(x, t). (10.202)

If we try to rearrange this equation as before

1 F 0 (t) X 00 (x) q(x, t)

= + , (10.203)
↵ F(t) X(x) ↵F(t)X(x)

we no longer get a situation where both sides of the equation must equal a constant.
 Chapter 10. Partial Di↵erential Equations 209

To proceed, we begin by solving the homogeneous problem, by setting q(x, t) = 0. From

Section 10.5, we already know the homogeneous problem has the solution

1
X 2
↵ ( n⇡
L ) t
T (x, t) = an n (x)e , (10.204)
n=1

where
✓ ◆
n⇡x
n (x) = sin . (10.205)
L

Now recall that the Sturm-Liouville theorem guarantees that any piecewise continuous func-
tion g(x) can be expanded in n (x),

1
X
g(x) = an n (x). (10.206)
n=1

That means that at any fixed time t = t1 , the solution to the non-homogeneous PDE can also
be expressed as
1
X
T (x, t1 ) = an n (x). (10.207)
n=1

If we perform this expansion at every time t, we would find that the solution can be ex-
pressed as
1
X
T (x, t) = an (t) n (x), (10.208)
n=1

where the expansion coefficients an (t) are now functions of time. We can similarly express
the heating term q(x, t) as
1
X
q(x, t) = bn (t) n (x). (10.209)
n=1

Substituting these equations into the non-homogeneous PDE (10.200), we find that

1
X 1
X 1
dan (t) d 2 n (x) X
n (x) = ↵ an (t) + bn (t) n (x). (10.210)
dt dx2
n=1 n=1 n=1

From the expression n = sin(n⇡x/L), we know that

✓ ◆
d2 n n2 2 n⇡x n2 2
= sin = n. (10.211)
dx2 L2 L L2
 210 10.9. Non-homogeneous PDEs

We consequently find that the three sums in equation (10.210) can be combined into one
sum of the form
X1 " #
dan (t) ↵n2 ⇡2
+ an (t) bn (t) n (x) = 0. (10.212)
dt L2
n=1

Because the eigenfunctions are linearly independent, this requires

dan (t) ↵n2 ⇡2

+ an (t) = bn (t). (10.213)
dt L2

We conclude that the solution of the non-homogeneous PDE (10.200) can be written as

1
X
T (x, t) = an (t) n (x), (10.214)
n=1

where each expansion coefficient an (t) must satisfy the ODE (10.213).

As a simple example, let’s consider transient heat conduction in a bar of length L = 1, for
which
@T @2 T
= ↵ 2 + sin(⇡x), (10.215)
@t @x

subject to the boundary conditions and initial conditions

T (0, t) = 0, T (1, t) = 0, T (x, 0) = 0. (10.216)

For this simple case, it happens that q(x, t) = 1 (x), such that b1 = 1 and bn = 0 for n 2.
Consequently, the expansion coefficient a1 must satisfy

da1 (t)
↵⇡2 a1 (t) = 1. (10.217)
dt

The homogeneous equation for a1 (t) can be written as

1 da1 d
= (ln a1 ) = ↵⇡2 , (10.218)
a1 dt dt

which has the solution

2
a1 = C1 e↵⇡ t . (10.219)

We can seek a particular solution in the form of a constant a1 = B, which requires

↵⇡2 B = 1. (10.220)
 Chapter 10. Partial Di↵erential Equations 211

We conclude that
2 1
a1 = C1 e↵⇡ t
. (10.221)
↵⇡2

The remaining expansion coefficients satisfy the homogeneous ODE,

dan ↵n2 ⇡2
an = 0, n = 2, 3, 4..., (10.222)
dt L2

which has the solution

2
⇡2 t
an = Cn e↵n . (10.223)

We conclude that
" #
2 1 ↵22 ⇡2 t
T (x, t) = a1 (t) 1 (t) + a2 (2) 2 (t) + ... = C1 e↵⇡ t 1 (x) + C2 e 2 + ... (10.224)
↵⇡2

1 ↵⇡ 2 t ↵22 ⇡2 t
= 1 (x) + C1 e 1 (x) + C2 e 2 + ... (10.225)
↵⇡2

or simply
1
X
1 2
⇡2 t
T (x, t) = 1 (x) + Cn e↵n n (x). (10.226)
↵⇡2
n=1

The initial condition then requires

1
X
1
T (x, 0) = 1 (x) + Cn n (x) = 0. (10.227)
↵⇡2
n=1

If we take the inner product of the initial condition with 1,

1
1 ⌦ ↵ X ⌦ ↵
1 (x), 1 + Cn 1, n (x) = 0. (10.228)
↵⇡2
n=1

Orthogonality causes this to simplify to

1 1 1
+ C1 = 0, (10.229)
↵⇡2 2 2

from which we can solve for C1 ,

1
C1 = . (10.230)
↵⇡2
 212 10.10. Non-homogeneous boundary conditions

If take the inner product of the initial condition with any other eigenfunction j (x), j , 1,
we find that Cj = 0. In conclusion, we find the solution to the non-homogeneous PDE can is
" #
sin(⇡x) ↵⇡2 t
T (x, t) = e 1 . (10.231)
↵⇡2

10.10 Non-homogeneous boundary conditions

Suppose we reconsider the heat equation solved in

Section ??, governed by the heat equation L

@T @2 T x
=↵ 2, (10.232)
@t @x

but subject to non-homogeneous boundary condi- Figure 10.18

tions that fix the temperatures at x = 0 and x = L,

T (0, t) = T0 , T (L, t) = TL , T (x, 0) = f (x). (10.233)

Repeating the analysis of Section ??, we can seek a solution in the form T = X(x)F(t), and
find that X(x) and F(t) must satisfy

X 00 + X = 0, F 0 + ↵F = 0. (10.234)

The boundary conditions, however, now require

X(0)F(t) = T0 , X(L)F(t) = TL . (10.235)

The resulting ODE for X(x) is no longer a Sturm-Liouville problem, because Sturm-Liouville
problems must have homogeneous boundary conditions.
For this relatively simple example, we can address the issue by realizing that for late
times t ! 1, the temperature in the bar should tend to a steady-state solution Ts (x) satisfying

d 2 Ts
= 0, Ts (0) = T0 , Ts (L) = TL . (10.236)
dx2

This has the solution

x
Ts (x) = T0 + (T T0 ) . (10.237)
L L

We can consequently seek a solution to the full problem in the form

b(x, t).
T (x, t) = Ts (x) + T (10.238)
 Chapter 10. Partial Di↵erential Equations 213

b satisfies the usual heat equation,

Substituting this form into the original PDE, we find that T

b
@T @2 T
b
=↵ 2. (10.239)
@t @x

Substituting the form (10.238) into the boundary and initial conditions, we find that

b(0, t) = T0 ,
T (0, t) = Ts (0) + T b(L, t) = TL ,
T (L, t) = Ts (L) + T (10.240)

b(x, 0) = f (x).
T (x, 0) = Ts (x) + T (10.241)

This requires
b(0, t) = 0,
T b(L, t) = 0,
T b(x, 0) = g(x),
T (10.242)

b satisfies homogeneous boundary conditions, such that

where g(x = f (x) Ts (x). We see that T
it can now be solved using the usual approach.
The above approach also works well when the temperature is fixed on one boundary, and
the gradient is fixed on the other, for example

@T
T (0, t) = T0 , = a, (10.243)
@x x=L

where T0 and b are both constants. In that case, we would solve

d 2 Ts dTs (L)
= 0, Ts (0) = T0 , = b. (10.244)
dx2 dx

In some cases, however, this approach does not work. Suppose, for example, that we apply
Neumann boundary conditions to both ends of the rod,

@T @T
= a, = b, (10.245)
@x x=0 @x x=L

where a and b are both constants. In this case, there is no steady-state solution, except for
the special case where a = b. You can show this by attempting to solve

d 2 Ts dTs (0) dTs (L)

= 0, =a = b, (10.246)
dx2 dx dx

for which you will find that no solution exists. The reason no steady state solution exists,
is that the Neumann conditions fix the rate with which heat enters the bar from the left
and right boundaries. If these rates of heat entering and leaving the bar do not balance,
then energy will either accumulate or diminish within the bar, without every achieving a
steady-state.
 214 10.10. Non-homogeneous boundary conditions

For cases such as the above, for which we cannot find an appropriate steady-state solu-
tion, we must seek a solution to T in the form

b(x, t),
T (x, t) = g(x) + T (10.247)

where g(x) is any function satisfying the boundary conditions. In the last example, we need
to find a function g(x) for which g 0 (0) = a and g 0 (L) = b. Suppose we seek g(x) as a quadratic
function,
g = c1 + c2 x + c3 x 2 . (10.248)

Applying the boundary condition at x = 0, we find that

g 0 (0) = c2 = a. (10.249)

Applying the condition at x = L, we find that

g 0 (0) = a + 2c3 L = b, (10.250)

which requires c3 = (b a)/(2L). We see, however, that any value of c1 satisfies the boundary
conditions. We consequently set c1 = 0, so that

b a 2
g = ax + x . (10.251)
2L

After finding a suitable function g(x), we substitute the solution form (10.247) into the
original PDE, and find that
b
@T @2 T
b d 2g
= ↵ 2 +↵ 2. (10.252)
@t @x dx

We see that the solution form (10.247) adds a non-homogeneous term into the heat equation.
The boundary conditions, however, are now homogeneous. Luckily, we know how to solve
these problems, as outlined in Section 10.9.
11. Fourier Series

11.1 Even and odd functions 215

11.2 Periodic and sinusoidal functions 217

11.3 Fourier series 219

11.4 Application to ODEs 223

11.5 Complex form 226

11.6 The wavenumber and frequency domain 229

11.7 Fourier integrals 231

Fourier series, Fourier transforms, and the broader field of Fourier analysis originated in
work by Daniel Bernoulli (1700-1782), Leonhard Euler (1707-1783), and Joseph Fourier
(1768-1830) to understand physical phenomena such as vibrating strings and heat conduc-
tion. Fourier analysis is now used extensively in all branches of engineering and physics,
including optics, acoustics, signal processing, optimal control theory, electromagnetism, cir-
cuit design, crystallography, computer imaging, fluid mechanics, weather forecasting, and
modeling any wavelike or periodic phenomenon, including fluctuations in stock markets.

11.1 Even and odd functions

Even functions f (x) satisfy the condition f ( x) = f (x), as sketched in figure 11.1(a). Note that
any function of the form f (x) = ax2n , where n = 0, 1, 2, 3, ... is an even function. Consequently

(a) (b)
4 1.5
1 x
2
x 1 x3
3 cos(x) sin(x)

0.5
2
f (x) f (x)
0
1
-0.5

0
-1

-1 -1.5
-2 -1 0 1 2 -2 -1 0 1 2
x x
Figure 11.1: Examples of even (panel a) and odd (panel b) functions.

215
 216 11.1. Even and odd functions

any polynomial of the form

N
X
f (x) = a0 + a1 x2 + a2 x4 + ... = an x2n (11.1)
n=0

is also an even function. That includes cos(x), which can be written as the Taylor series

1
X
x2 x4 x2j
cos(x) = 1 + ... = ( 1)j . (11.2)
2! 4! (2j)!
k=0

Odd functions satisfy the condition f ( x) = f (x), as sketched in figure 11.1(b). Any func-
tion of the form f (x) = ax(2n 1) , where n = 1, 2, 3, ..., is an odd function. Consequently any
polynomial of the form

N
X
f (x) = a1 x + a2 x3 + a3 x5 + ... = an x2n 1
(11.3)
n=1

is an odd function, including sin(x), which can be written as the Taylor series

1
X
x3 x5 x2j+1
sin(x) = x + ... = ( 1)j . (11.4)
3! 5! (2j + 1)!
k=0

Due to their symmetry about x = 0, any even function f e (x) satisfies

Za Za
f e (x) dx = 2 f e (x) dx, (11.5)
a 0

while any odd function f o (x) satisfies

Za
f o (x) dx = 0. (11.6)
a

Furthermore, products of even and odd functions satisfy

f e (x)ge (x) = even,

f o (x)go (x) = even, (11.7)

f e (x)go (x) = odd,

where ge and go denote an even and odd function, respectively. Furthermore, we can show
 Chapter 11. Fourier Series 217

that any function f (x) can be decomposed into the sum of an even and odd function,

f (x) = f e (x) + f o (x). (11.8)

This can be proven by expressing f (x) as

f (x) + f (x) f ( x) f ( x)
f (x) = + , (11.9)
2 2

which can be rearranged as

f (x) + f ( x) f (x) f ( x)
f (x) = + . (11.10)
2 2
| {z } | {z }
f e (x) f o (x)

Note that
f ( x) + f (x)
f e ( x) = = f e (x). (11.11)
2

f ( x) f (x)
f o ( x) = = f o (x). (11.12)
2

As a more concrete example, it is worth noting that cosh x and sinh(x) can be interpreted as
the even and odd components of f (x) = ex ,

ex + e x ex e x
ex = + . (11.13)
2 2
| {z } | {z }
cosh(x) sinh(x)

11.2 Periodic and sinusoidal functions

Periodic functions are functions that exhibit a regularly repeating pattern. Let’s consider
some function f (x), where x denotes a position in space. In that case, we say a function is
periodic over the wavelength if f (x) = f (x + ) for all x. Two classic examples of periodic
functions are sin(x) and cos(x), which are both periodic over the wavelength = 2⇡, as
demonstrated in figure 11.2(a). More generally, let’s consider the functions

A sin(↵x) and A cos(↵x), (11.14)

where A is called the amplitude, and ↵ is called the wavenumber. These functions perform a
full oscillation in space as ↵x varies between

0  ↵x  2⇡. (11.15)
 218 11.2. Periodic and sinusoidal functions

(a) (b)
1 sin(x)
cos(x) 0.75

0.5 0.5

f (x) 0.25
0
f (x)
0

-0.5
-0.25 0.25*sin(0.5x)
0.5*sin(2x)
0.75*cos(0x)
-1 -0.5
-6 -4 -2 0 2 4 6 -6 -4 -2 0 2 4 6
x x
Figure 11.2: Examples of periodic functions.

(a) (b)

Figure 11.3: E↵ects of phase and phase shift.

Consequently, the wavelength is given by

2⇡
= . (11.16)
↵

We see that large wavenumbers produce small wavelengths, and small wavenumbers pro-
duce large wavelengths. People sometimes prefer to write sinusoidal functions directly in
terms of their wavelength,
✓ ◆ ✓ ◆
2⇡x 2⇡x
A sin and A cos . (11.17)

The roles of A and ↵ are demonstrated in figure 11.2(b). We see, for example, that f (x) =
0.25 sin(0.5x) has an amplitude of 0.25 and a wavelength of = 2⇡/0.5 = 4⇡. One special
case occurs when ↵ = 0. In this case, sin(↵x) = 0 and cos(↵x) = 1. From the relation = 2⇡/↵,
one can think of these functions as having infinite wavelength. However, it is obvious from
figure 11.2(b) that they are periodic over any wavelength . The function sin(0x) = 0 is of
little practical use, but the case of cos(0x) = 1, plays an important role in Fourier analysis.
 Chapter 11. Fourier Series 219

For even more generality, let’s consider sinusoidal functions, which are any functions of
the form
A sin(↵x ) and A cos(↵x ), (11.18)

where is often called the phase.* Figure 11.3(a) compares the function sin(x) (solid blue
line) with sin(x 2) (red dashed line). We see that the phase = 2 shifts the sine function
rightward a distance of 2 units, because sin(x 2) = 0 when x = 2. Figure 11.3(b) compares
the functions sin(x 2) (solid blue line) with sin(2x 2) (red dashed line). In the case, the
sinusoidal function is only shifted rightward a distance of x = 1, because sin(2x 2) = 0 at
x = 1. Consequently, people often prefer to write sinusoidal functions in the form
h i h i
A sin ↵(x xs ) and A cos ↵(x xs ) , (11.19)

where xs , called the phase shift, is the actual distance the sinusoidal function is shifted right-
ward.
The terms wavelength and wavenumber are usually reserved for sinusoidal functions of
space. When the independent variable is time t, we usually write sinusoidal functions as

A sin(!t ) and A cos(!t ), (11.20)

where ! is called the angular frequency, which has units of radians per second. The time T it
takes for a function to undergo one full oscillation is called the Period, which is related to !
by
2⇡
T= . (11.21)
!

Occasionally, people use the definition of regular frequency f = 1/T , often measured in
Hertz (Hz) or kiloHertz (KHz). In that case, sinusoidal functions can be written as

A sin(2⇡f t ) and A cos(2⇡f x ), (11.22)

11.3 Fourier series

Suppose that some function f (x) is piecewise continuous and periodic over the wavelength
= 2l. Then it can be shown that f (x) can be expressed as an infinite sum of sinusoidal
functions,
1
X ✓ ◆ X1 ✓ ◆
n⇡x n⇡x
f (x) = an cos + bn sin , (11.23)
l l
n=0 n=0

where an and bn are expansion coefficients. This is called a Fourier series. Note that the
functions sin(n⇡x/l) and cos(n⇡x/l) are all periodic, with the wavelength = 2l. Also note
* In actuality, one finds several competing terms for , and some researchers even use “phase” to refer to
↵x , which is the full argument within the sin and cos functions.
 220 11.3. Fourier series

that the limit in the second sum is often set to n = 1, because sin(n⇡x/l) = 0 when n = 0.
Furthermore, due to that fact that cos(n⇡x/l) = 1 when n = 0, people often write the Fourier
series as
1
X ✓ ◆ X1 ✓ ◆
n⇡x n⇡x
f (x) = a0 + an cos + bn sin . (11.24)
l l
n=1 n=1

Conversely, we often combine both sums in the form

1 "
X ✓ ◆ ✓ ◆#
n⇡x n⇡x
f (x) = an cos + bn sin . (11.25)
l l
n=0

To determine the expansion coefficients an and bn , we use the fact that the functions
sin(n⇡x/l) and cos(n⇡x/l) are orthogonal with respect to the inner product

Zl
h h(x), g(x) i = h(x)g(x) dx. (11.26)
l

Indeed, a bit of calculus shows that

Zl 8
✓ ◆ ✓ ◆ >
> 0 n,m
m⇡x n⇡x <
cos cos dx = >
> l n=m,0 (11.27)
l l :
l 2l n = m = 0

Zl 8
✓ ◆ ✓ ◆ >
> 0 n,m
m⇡x n⇡x <
sin sin dx = >
> l n = m,0 (11.28)
l l :
l 0 n=m=0

Zl ✓ ◆ ✓ ◆
m⇡x n⇡x
sin cos dx = 0. (11.29)
l l
l

Suppose then that we expand a function f (x) in a Fourier series,

1 "
X ✓ ◆ ✓ ◆#
n⇡x n⇡x
f (x) = an cos + bn sin . (11.30)
l l
n=0

We can multiply both sides of the above equation by cos(m⇡x/l) and integrate over x 2 [ l, l]

Zl ✓ ◆ 1 Zl
X ✓ ◆" ✓ ◆ ✓ ◆#
n⇡x n⇡x n⇡x n⇡x
cos f (x) dx = cos an cos + bn sin dx. (11.31)
l l l l
n=0
l l
 Chapter 11. Fourier Series 221

Due to orthogonality, only one term survives the summation of the right-hand-side

Zl ✓ ◆ Zl ✓ ◆ ✓ ◆
n⇡x n⇡x m⇡x
cos f (x) dx = am cos cos dx. (11.32)
l l l
l l

We conclude that

Zl Zl ✓ ◆
1 1 n⇡x
a0 = f (x) dx and an = cos f (x) dx. (11.33)
2l l l
l l

Similarly, you can show that

Zl ✓ ◆
1 n⇡x
bn = sin f (x) dx. (11.34)
l l
l

Note that for even functions f (x), the expansion coefficients bn = 0. Conversely, for odd
functions, the expansion coefficients an = 0.

Theorem 11.1 (Fourier Convergence Theorem)

Let f (x) be a piecewise continuous function that is periodic over = 2l. Then the
Fourier series converges to f (x) at every point for which f (x) is continuous, and to
[f (x+ ) + f (x )]/2 at every point for which f (x) is discontinuous.

Example 11.1
Consider the following function f (x) that is periodic over = 2⇡
8
>
>
<1, 0  x < ⇡
f (x) = >
> (11.35)
:0, ⇡  x < 2⇡.

We start by computing the coefficient a0 ,

Z⇡ Z⇡
1 1 1
a0 = f (x) dx = 1 dx = . (11.36)
2⇡ 2⇡ 2
⇡ 0

Note that a0 is the average value of f (x). We next compute the coefficients an for n > 0,

Z⇡ Z⇡ " #⇡
1 1 1 1
an = f (x) cos(nx) dx = cos(nx) dx = sin(nx) = 0. (11.37)
⇡ ⇡ ⇡ n 0
⇡ 0
222 11.3. Fourier series

We could have actually foreseen that an = 0, because f (x) a0 is an odd function.

Finally, we compute the coefficients bn ,

Z⇡ Z⇡ " #
1 1 1
bn = f (x) sin(nx) dx = sin(nx) dx = cos(n⇡) 1 . (11.38)
⇡ ⇡ ⇡n
⇡ 0

We find that
8
>
< ⇡2
> n = 1, 3, 5, ...
bn = >
> (11.39)
:0 n = 2, 4, 6, ...

We conclude that
1
1 2X 1 h i
f (x) = + sin (2n 1)x . (11.40)
2 ⇡ 2n 1
n=1

(a) (b)

(c) (d)

Figure 11.4: Fourier approximations of the function (11.35) using (a) N = 0, (b) N = 1, (c)
N = 5, and (d) N = 40.

Figure 11.4 shows approximations to f (x) when the summation in equation (11.40) is
taken to N = 0, 1, 5, and 40 terms. We see that with increasing N , the approximation
improves in regions where f (x) is continuous, but the approximation becomes increas-
ingly oscillatory at the discontinuous points. This behavior is called Gibbs phenomenon.
It only occurs for discontinuous functions.
 Chapter 11. Fourier Series 223

11.4 Application to ODEs

Fourier series are useful for modeling dynamical sys-

tems subject to discontinuous forcing. Consider, (a)
for example, the simple mass-spring system in fig-
ure 11.5(b), which satisfies the ODE k x

d 2x
m f(t)
m 2 + kx = f (t). (11.41)
dt

Let us also consider the initial conditions (b)

x(0) = x0 (0) = 0. (11.42)

Now suppose that the forcing term f (t) is the function

shown in figure 11.5(b), which is periodic over a period
of T = 4
8
>
>
> 0 2  t  1,
>
>
<1 + t 1  t  0,
f (t) = >
> (11.43)
>
> 1 t 0  t  1,
>
: 0 Figure 11.5
1t2

Finding the homogeneous solution xh to this

p problem is straightforward. We seek a solution
t
in the form xh ⇠ e , and find that = ±i k/m. For simplicity, lets assume k = m = 1. The
homogeneous solution can then be written as

xh = A sin(t) + B cos(t). (11.44)

Unfortunately, if the forcing function f (t) is left in the piecewise form (11.43), it is not at all
clear how to obtain a particular solution. For example, the method of undetermined coeffi-
cients requires us to repeatedly di↵erentiate f (t) with respect to time, and yet the derivative
of f (t) is discontinuous for this case. Fourier series o↵er a solution to this issue by expressing
f (t) as an infinite sum of sinusoidal functions that are di↵erentiable.

Because the function f (t) is an even function, it can be expressed as a cosine series,

1
X ✓ ◆
n⇡t
f (t) = an cos , (11.45)
l
n=0

where l = 2. We begin by computing a0 ,

Z2 Z2 Z1 " #1
1 1 1 1 t2 1
a0 = f (t) dt = f (t) dt = (1 t) dt = t = . (11.46)
4 2 2 2 2 0 4
2 0 0
 224 11.4. Application to ODEs

Note that because f (t) is even, we were able to replace the integral from l to l by an integral
from 0 to l. Furthermore, note that a0 is equal to the average area under the curve f (t). Next,
we compute the remaining an

Z2 ✓ ◆ Z2 ✓ ◆ Z1 ✓ ◆
1 n⇡t n⇡t n⇡t
an = cos f (x) dx = cos f (x) dx = cos (1 t) dx (11.47)
2 2 2 2
2 0 0

2 ✓ ◆ ✓ ◆ 31 2 ✓ ◆ 37
2 666 n⇡t 2 n⇡t 777 4 666 n⇡ 7
= 64 (t 1) sin + cos 75 = 2 2
64 cos 1775. (11.48)
n⇡ 2 n⇡ 2 n ⇡ 2
0

Figure 11.6 shows that the resulting cosine series

(red dashed line) is virtually indistinguishable from
f (t) using only only N = 10 terms. We can now write
the ODE governing the particular solution xp as
1
X ✓ ◆
d 2x n⇡t
+x = an cos . (11.49)
dt 2 2
n=0

Let’s write out the first three terms of the forcing

Figure 11.6
function,
✓ ◆
d 2x ⇡t
2
+ x = a0 + a1 cos + a2 cos (⇡t) + ... (11.50)
dt 2

The method of undetermined coefficients suggests we seek a corresponding particular solu-

tion in the form
✓ ◆ ✓ ◆
⇡t ⇡t
xp = C0 + C1 cos + C2 sin + C3 cos (⇡t) + C4 sin (⇡t) + ... (11.51)
|{z} 2 2 | {z }
| {z }
a0 a2 cos(⇡t)
a1 cos(⇡t/2)

where the brackets show how the components of the particular solution are generated by the
three terms of the forcing function f (t). Including all terms of the forcing function, we can
seek a solution in the form
X1 " ✓ ◆ ✓ ◆#
n⇡t n⇡t
xp = sn cos + pn sin , (11.52)
2 2
n=0

where sn and pn are to be determined. Taking the second derivative with respect to time, we
find that
X1 " ✓ ◆ ✓ ◆#
00 n2 ⇡ 2 n⇡t n⇡t
xp = sn cos + pn sin . (11.53)
4 2 2
n=0

It’s worth noting that xp = s0 and xp00 = 0 when n = 0. Substituting xp and xp00 into the ODE
 Chapter 11. Fourier Series 225

(11.49), we find that

1 "
X ! ✓ ◆ ! ✓ ◆# X1 ✓ ◆
n2 ⇡ 2 n⇡t n2 ⇡ 2 n⇡t n⇡t
1 sn cos + 1 pn sin = an cos .
4 2 4 2 2
n=0 n=0

This requires
an 4an
sn = ⇣ ⌘= , pn = 0. (11.54)
1 n2 ⇡ 2 4 n2 ⇡ 2
4

The full solution x = xh + xp can now be written as

1
X ✓ ◆
4an n⇡t
x(t) = A sin(t) + B cos(t) + cos . (11.55)
4 n2 ⇡ 2 2
n=0

Finally, we must consider the initial conditions. The

condition x(0) = 0 requires

1
X 4an
x(0) = B + = 0, (11.56)
4 n2 ⇡ 2
n=0

from which we conclude that

1
X 4an
B= . (11.57)
4 n2 ⇡ 2
n=0 Figure 11.7

The condition x0 (0) = 0 requires

x0 (0) = A = 0. (11.58)

We conclude that
1
X 1
X ✓ ◆
4an 4an n⇡t
x(t) = cos(t) + cos , (11.59)
4 n2 ⇡ 2 4 n2 ⇡ 2 2
n=0 n=0

or more compactly,
1 2 ✓ ◆ 3
X 4an 666 n⇡t 77
x(t) = 2 2
64 cos cos(t)775. (11.60)
4 n ⇡ 2
n=0

Figure 11.7 shows the solution x(t) using N = 10 terms, which was found the be sufficiently
accurate. Note that though the particular solution xp is periodic over the period T = 4, the
full solution x(t) (red dashed line) is not periodic over the same period as f (t), due to the
 226 11.5. Complex form

e↵ect of the homogeneous solution.

11.5 Complex form

In practice, it often more convenient to express Fourier series in terms of complex exponen-
tials. Using Euler’s formula, we can express sin(n⇡x/l) and cos(n⇡x/l) as

ein⇡x/l e in⇡x/l ein⇡x/l + e in⇡x/l

sin(n⇡x/l) = , cos(n⇡x/l) = , (11.61)
2i 2
p
where i = 1. Substituting this form into

1 "
X ✓ ◆ ✓ ◆#
n⇡x n⇡x
f (x) = an cos + bn sin , (11.62)
l l
n=0

we find that
1 "
X #
ein⇡x/l + e in⇡x/l ein⇡x/l e in⇡x/l
f (x) = an + bn , (11.63)
2 2i
n=0

which can be re-arranged as

1 "
X ! ! #
an ibn in⇡x/l an + ibn in⇡x/l
f (x) = e + e . (11.64)
2 2
n=0

Note that when n = 0, this simplifies to

! !
a0 ib0 0 a0 + ib0 0
e + e = a0 . (11.65)
2 2

We can consequently break the Fourier series into the three sums

1
X ! 1 !
an ibn in⇡x/l X an + ibn in⇡x/l
f (x) = a0 + e + e . (11.66)
2 2
n=1 n=1

By substituting n = m, the second summation in the above can be written as

1
X ! 1
X !
an + ibn in⇡x/l a m + ib m
e = eim⇡x/l . (11.67)
2 2
n=1 m= 1
 Chapter 11. Fourier Series 227

Now setting m = n, we can express the Fourier series as

1
X ! 1 !
an ibn in⇡x/l X a n + ib n
f (x) = a0 + e + ein⇡x/l , (11.68)
2 2
n=1 n= 1

or simply
1
X
f (x) = cn ein⇡x/l , (11.69)
n= 1

where
8
>
>
> a0 , n=0
>
>
<
cn = >
> (an ibn )/2 n>0 (11.70)
>
>
>
: (a n + ib n )/2 n < 0.

Substituting our integral relationships for an and bn , you will find that cn for all cases can be
computed using the simple integral,

Zl
1 in⇡x/l
cn = f (x)e . (11.71)
2l
l

We can also prove the above relation for cn by noting that the functions

in⇡x/l
n (x) = e , n = 0, 1, 2, 3, .... (11.72)

are orthogonal with respect to the inner norm

Zl
⌦ ↵
n (x), m (x) = n (x) m (x) dx, (11.73)
l

where the overline denotes the complex conjugate. More specifically,

Zl Zl
⌦ ↵ in⇡x/l im⇡x/l
n (x), m (x) = e e dx = ei(n m)⇡x/l
dx. (11.74)
l l

When n , m, the integrand oscillates about zero with a periodicity of 2l, such that the inte-
gral is zero. When n = m, the integrand simplifies to e0 = 1, such that
8
< 0, n , m
⌦ >
> ↵
n (x), m (x) = >
> (11.75)
:2l, n = m
 228 11.5. Complex form

Now suppose we seek an expansion to f (x) in the form

1
X
f (x) = cn ein⇡x/l . (11.76)
n= 1

We can multiply both sides by exp( im⇡x/l) and integrate over l  x  l,

Zl 1
X Zl
im⇡x/l
f (x)e dx = cn ein⇡x/l e im⇡x/l
dx. (11.77)
n= 1
l l

Due to orthogonality, this simplifies to

Zl
im⇡x/l
f (x)e dx = 2lcm . (11.78)
l

In summary, we can express a Fourier series in the compact form

1
X Zl
in⇡x/l 1 in⇡x/l
f (x) = cn e , cn = f (x)e . (11.79)
n= 1
2l
l

Example 11.2
Let’s reconsider the function f (x) considered in figure 11.4 of Section 11.3,
8
>
>
<1, 0  x < ⇡
f (x) = >
> (11.80)
:0, ⇡  x < 2⇡.

Substituting the function into our formula for cn , we find that

Z⇡  
1 inx 1 1 inx
⇡ i in⇡
cn = e dx = e = e 1, (11.81)
2⇡ 2⇡ in 0 2⇡n
0

where we have used the fact that 1/i = i. Note that the above integral assumes n , 0,
to avoid division by zero. Using the Euler formula, our expression for cn simplifies to

i
cn = cos(n⇡) 1 (11.82)
2⇡n
 Chapter 11. Fourier Series 229

We find that
8
>
> 0 n = ±2, ±4, ...
<
cn = >
> i (11.83)
: ⇡n n = ±1, ±3, ...

For the case n = 0, we must reconsider the integral

Z⇡
1 1
c0 = e0 dx = , (11.84)
2⇡ 2
0

which the identical to our result for a0 in Section 11.3.

Note that in the above example, the expansion coefficients for n < 0 are simply the com-
plex conjugate of the coefficients for n > 0. It turns out that this is true for any real function,
i.e.

When f (x) is a real function,

c n = cn . (11.85)

We consequently only need to compute the expansion for coefficients for n 0.

11.6 The wavenumber and frequency domain

Rather than expressing the Fourier series as

1 "
X ✓ ◆ ✓ ◆#
n⇡x n⇡x
f (x) = an cos + bn sin , (11.86)
l l
n=0

we can express it in terms of the wavenumber ↵n

1 "
X #
f (x) = an cos (↵n x) + bn sin (↵n x) , (11.87)
n=0

where ↵n = n⇡x/l. The wavenumbers ↵n represent all the permissible wavelengths n =

2⇡/↵n that are periodic over the fundamental wavelength = 2l.
Similarly, if the independent variable is time, f (t), we can write the Fourier series as

1 "
X #
f (t) = an cos (!n t) + bn sin (!n t) , (11.88)
n=0

where !n is the frequency. We can furthermore express the Fourier series in complex form
 230 11.6. The wavenumber and frequency domain

as
1
X 1
X
f (x) = cn ei↵n x , f (t) = cn ei!n t , (11.89)
n= 1 n= 1

where !n = n⇡/l represents all the possible frequencies that are periodic over the fundamen-
tal period T = 2l.
We refer to ↵n and !n as the spectrum of the functions f (x) and f (t). Sometimes to be
more specific, we refer to the latter as the “wavenumber spectrum” and the “frequency spec-
trum” This also motivates the concept of a wavenumber domain and frequency domain. Let’s
reconsider the function (11.43) considered previously in Section 11.4. We can interpret the

time domain frequency domain

Figure 11.8

space domain wavenumber domain

Figure 11.9
 Chapter 11. Fourier Series 231

Fourier series as taking a function f (t) in the time domain and expressing it as a linear
combination of sinusoids in the frequency domain, as sketched in figure 11.8. Smooth func-
tions produce spectra that decay rapidly to zero as n ! 1. In figure 11.8, an is on the order of
0.001 when n = 20. Discontinuous functions tend to generate spectra that decay more slowly,
because the discontinuity excite high frequencies/wavenumbers. Consider for example the
step function 11.35 which we expressed in Section 11.5 as

1
X
f (x) = cn einx , (11.90)
n= 1

where
8
>
>
> 1/2 n=0
>
>
>
< 0
cn = >
> n = ±2, ±4, ... (11.91)
>
>
>
> i
n = ±1, ±3, ...
: ⇡n

Figure 11.9 shows we can interpret the Fourier series as taking a function f (x) in the space
domain and expressing it as a linear combination of sinusoids in the wavenumber domain,
where the horizontal axis shows ↵n and the vertical axis shows the |cn |. Note that because
cn is complex, we plot its magnitude. We also only show results for positive wavenumbers
↵n . Note that the spectrum for the step function decays very slowly to zero because the
discontinuous nature of the step function excites high wavenumbers. These are required to
resolve rapid variations in the vicinity of the discontinuities.

11.7 Fourier integrals

Suppose we have a function f (t) that is defined over 1 < t < 1 but is not periodic, such as
2
f (t) = e t . We can extend the Fourier concept to f (t) by letting the period T ! 1. Consider

1 "
X ! !#
n⇡t n⇡t
f (t) = an cos + bn sin (11.92)
l l
n=0
X1 " #
= an cos(!n t) + bn sin(!n t) ,
n=0

where !n = n⇡/l is the “spectrum,” i.e. !n represents all possible frequencies. Consider the
following spectra for increasing l,

l = ⇡, !n = n = 0, 1, 2, 3, ... (11.93)
n 1 3
l = 2⇡, !n = = 0, , 1, , ... (11.94)
2 2 2
n
l = 10⇡, !n = = 0, 0.1, 0.2, 0.3, ... (11.95)
10
 232 11.7. Fourier integrals

From this pattern, we observe that as l ! 1, the di↵erence between adjacent frequencies,

⇡
! = !n !n 1 = , (11.96)
l

tends to zero, such that the discrete spectrum tends to a continuous spectrum

0  ! < 1. (11.97)

In this limit, the Fourier series must be replaced with the Fourier integral
Z 1" #
f (t) = a(!) cos(!t) + b(!) sin(!t) d!, (11.98)
0

where
Z 1
1
a(!) = f (t) cos(!t)dt, (11.99)
⇡ 1
Z 1
1
b(!) = f (t) sin(!t)dt. (11.100)
⇡ 1

This can also be written more succinctly in complex form as

Z 1
1
f (t) = c(!)ei!t d! (11.101)
2⇡ 1

where
Z 1
i!t
c(!) = f (x)e dt. (11.102)
1

Usually, however, we prefer the symbol fb(!),

fb(!) = c(!) (11.103)

and define
Z 1
fb(!) = F{f (t)} = f (t)e i!t
dt , (11.104)
1
| {z }
F{ } = Fourier transform
(11.105)
Z 1
1
f (t) = F 1 {fb(!)} = fb(!)ei!t d! . (11.106)
2⇡ 1
| {z }
F 1{ } = inverse Fourier transform
 Chapter 11. Fourier Series 233

11.7.1 Application to PDEs

The propagation of a wave travelling on an infinite

string is governed by the PDE u
c
c
f(x)
@2 u 2
2@ u x
= c , 1 < x < 1, t > 0, (11.107)
@t 2 @t 2

where u(x, t) is the height of the wave measured from Figure 11.10
the strings undisturbed position. At t = 0, the wave
has an initial shape f (x) and temporal derivative g(x),

@u
u t=0
= f (x), = g(x), (11.108)
@t t=0

as sketched in figure 11.10, We assume that the initial disturbance is finite in spatial extent,
such that f (x) tends to zero as x tends to ±1. Experimentally, we find that the initial dis-
turbance of a string generates waves that travel leftward and rightward with a speed c, as
sketched in figure 11.10. Because the initial condition has a finite spatial extent, and the
string has an infinite length, we can apply the boundary conditions

u x!±1 = 0. (11.109)

We can solve this problem by seeking a solution in the form of a Fourier integral,
Z 1
1
u(x, t) = b(!, t)ei!x d!.
u (11.110)
2⇡ 1

From this assumption, we can define the partial derivatives

Z 1
@2 u 1
2
= b(!, t)( ! 2 )ei!x d!,
u (11.111)
@x 2⇡ 1
Z 1
@2 u 1 @2 u
b i!x
2
= 2
e d!. (11.112)
@t 2⇡ 1 @t

Plugging these partial derivatives into the governing PDE and simplifying, we find
Z 1 " #
@2 u
b 2 2
+ c ! u ei!x d! = 0,
b (11.113)
1 @t 2

which requires

@2 u
b 2 2
+c ! u
b = 0. (11.114)
@t 2
 234 11.7. Fourier integrals

Although equation (11.114) has partial derivatives, it is basically an ODE for a fixed !. This
allows us to seek a solution in the form

b t
b = A(!)e
u (11.115)

as a solution. Taking the derivative with respect to t and plugging the result into equa-
tion (11.114), we are left with

2b
A(!)e t
+ c2 !2 A(!)e
b t
=0 (11.116)

which simplifies to
2
+ c 2 ! 2 = 0. (11.117)

Solving for , we find

= ±ic!. (11.118)

This means that the solution to u

b is of the form

b ic!t b ic!t
b = A(!)e
u + B(!)e . (11.119)

In addition to the above solution, we need to consider the special case where ! = 0, for which

@2 u
b b b
= 0, and b = D(!)
u + E(!)t. (11.120)
@t 2

b
The D(!) is recovered by solution (11.119) above when ! = 0. We can consequently write
the net solution as
b ic!t b ic!t b
b = A(!)e
u + B(!)e + E(!)t. (11.121)

We can now recover u(x, t) using the inverse transform

Z 1 h i
1 ic!t ic!t
u(x, t) = b
A(!)e b
+ B(!)e b
+ E(!)t ei!x d!, (11.122)
2⇡ 1

which we can write as

Z 1 Z 1 Z 1
1 b i!(x ct) 1 b i!(x+ct) t b i!x
u(x, t) = A(!)e d! + B(!)e d! + E(!)e d!. (11.123)
2⇡ 1 2⇡ 1 2⇡ 1

In the above, the combination x ± ct in the first two integrals plays the role of x in our usual
 Chapter 11. Fourier Series 235

Fourier transform. This consequently produces

u(x, t) = A(x ct) + B(x + ct) + E(x)t. (11.124)

where A is function of x ct and B is function of x + ct. The term E(x)t grows unbounded in
time, and is not observed physically. We consequently set E(x) = 0, producing

u(x, t) = A(x ct) + B(x + ct)+ . (11.125)

Solution (11.125) can be interpreted as the sum

of two waves that keep their shapes, but travel left
or right with speed c. To demonstrate why, consider
A(ξ)
that the function A(x ct) is stationary when plotted Ao
as a function of ⇠ = x ct, as sketched in figure 11.11.
Now suppose that at time to and position xo , we ob-
serve the local wave height to be Ao = A(⇠0 ), where ξ
ξo
⇠ o = xo cto . (11.126)

Figure 11.11
If we now travel rightward with velocity c,

x = ⇠o + cto , (11.127)

the local value of A appears fixed.

To solve for the functions A and B, we apply the initial conditions. The initial condition
for u(x, 0) produces

u(x, 0) = A(x) + B(x) = f (x). (11.128)

To apply the second condition, note that we can write the solution as

u(x, t) = A( ⇠(x, t) ) + B( ⇠(x, t) ), (11.129)

where ⇠ = x ± ct. Using the chain rule for integration, we find that

@u dA @⇠ dB @⇠ dA dB
= + = c +c . (11.130)
@t d⇠ @t d⇠ @t d⇠ d⇠

We can now apply the initial condition on @u/@t as

dA dB
c +c = g(x), (11.131)
dx t=0 dx t=0

where we have used the fact that ⇠ = x at time t = 0.

 236 11.7. Fourier integrals

For demonstration, suppose we set f (x) =

exp( x2 ) and g(x) = 0. These require
1
u t=0
x2 dA dB 0.5
A(x) + B(x) = e , = . (11.132)
dx dx 0
1
u t = 0.5
The second condition produces 0.5

0
B(x) = A(x) + K, (11.133) 1
u t = 1.0
0.5
where K is any constant. The first condition 0
then requires u 1
t = 1.5
0.5
1 x2 K 1 K x2
A(x) = e , B(x) = e . + 0
2 2 2 2
(11.134) 1
t = 2.0
u
We see that we can drop the constant K, 0.5
because it will cancel when A and B are 0
-8 -6 -4 -2 0 2 4 6 8
summed. The final solution is then given by x
Figure 11.12
1 (x ct)2 1 (x+ct)2
u(x, t) = e + e . (11.135)
2 2

Figure 11.12 shows the resulting solution at 5 equispaced time intervals when c = 1.
12. Numerical Solution of IVPs

12.1 Numerical Integration 237

12.2 Forward Euler method 243

12.3 Backward Euler, leap frog, and Crank-Nicolson

methods 248

12.4 Runge-Kutta methods 249

12.5 Stability 252

12.6 Systems of IVPs 255

12.1 Numerical Integration

The objective of Numerical integration is to compute an ap-

proximate solution to definite integrals of the form

Zb
I= f (x) dx. (12.1) V1 V2
a Figure 12.1

In some cases, we resort to numerical integration because

the integrand f (x) has no simple antiderivative. Just as often, however, we use numerical
integration because f (x) is an unknown function for which we only have discrete data. Sup-
pose, for example, that we wish to compute the work exerted by a gas as it expands within
a piston and cylinder from volume V1 to V2 , as sketched in figure 12.1. In thermodynamics,
we earn that the resulting work is give by the integral

ZV2
W= p dV , (12.2)
V1

where p is the pressure. If the pressure is measured experimentally using a probe, we have
no analytical expression for p(x). Instead we have a collection of discrete data points (p, V ).

237
 238 12.1. Numerical Integration

(a) (b) (c)

f(x)

L
x x x
x0 = a x1 x2 x3 = b x0 x1 x2 x3 x0 x1 x2 x3

Figure 12.2

12.1.1 Newton-Cotes methods

Suppose we want to approximate the integral

Zb
I= f (x) dx. (12.3)
a

Newton-Cotes methods divide the interval x 2 [a, b] into N subintervals of equal width h =
(b a)/N , as sketched in figure 12.2(a) for the case N = 3. The function f (x) is then approxi-
mated in each subinterval as a polynomial that can be integrated analytically. Newton-Cotes
methods were initially proposed by Isaac Newton in 1676 and then further developed by
Roger Cotes (1682-1716) [4]. To demonstrate, let’s start by noting that the N intervals are
bounded by the N +1 points xj , defined as

xj = a + hj, j = 0, 1, ... , N . (12.4)

The simplest method then approximates the integral as the sum

X1
N
I ⇡ hf (x0 ) + hf (x1 ) + ... + hf (xN 1) = h f (xj ), (12.5)
j=0

as sketched in figure 12.2(b). This is called the “left hand rule” because it estimates the
area beneath the curve in each subinterval using f (x) evaluated at the left boundary of each
subinterval, as sketched in figure 12.2(b).
Similarly, the right hand rule estimates the integral using f (x) evaluated at the right edge
of each interval, as sketched in figure 12.2(c). This can be written as the sum

N
X
I ⇡h f (xj ). (12.6)
j=1
 Chapter 12. Numerical Solution of IVPs 239

(a) (b)
1.4
9 1.2
7.5 1
0.8
6
0.6
4.5

3 0.4

1.5

0 0.2
0 0.5 1 1.5 2 2.5 3 10 20 30 40 50

Figure 12.3

The only di↵erence between equations (12.5) and (12.6) are the limits of the summation.
The left and right-hand-rules are both called “first-order accurate” because their errors are
linearly proportional to h, i.e.
Err ⇠ O(h). (12.7)

This can be written in terms of N as

✓ ◆
1
Err ⇠ O . (12.8)
N

Figure 12.3(a) shows the left hand rule applied to f (x) = x2 when a = 0, b = ⇡, and N = 10.
Figure 12.3(b) shows a corresponding log-log plot of the error (asterisk symbols) vs. N . The
dashed line shows a corresponding plot of 1/N , confirming the first-order accuracy of the
method.
Two Newton Cotes methods that achieve second-order accuracy are the midpoint and
trapezoidal rules. The midpoint rule estimates the area in each subinterval using f (x) eval-
uated at the midpoint of the interval, as sketched in figure 12.4(a),
X1
N
I ⇡h f (xj+1/2 ), (12.9)
j=0

(a) (b)
f(x) f(x)

x x
x0 x1 x2 x3 x0 x1 x2 x3
Figure 12.4
 240 12.1. Numerical Integration

(a) (b) (c)

-1
10
9 9

7.5 7.5

6 6 10
-2

4.5 4.5

3 3 midpoint
trapezoid
-3 1/N
1.5 1.5 10
1/N2
0 0
0 0.5 1 1.5 2 2.5 3 0 0.5 1 1.5 2 2.5 3 10 20 30 40 50

Figure 12.5

while the trapezoidal rule approximates the area in each interval as a trapezoid, as sketched
in figure 12.4(b),
X1 1 h
N i
I ⇡h f (xj ) + f (xj+1 ) . (12.10)
2
j=0

Figure 12.5 shows the midpoint rule (panel a) and trapezoidal rule (panel b) applied to
f (x) = x2 when L = ⇡ and N = 10. Figure 12.5(c) shows a log-log plot of the errors generated
by both approaches as a function of N . The dash-dotted line shows a corresponding plot of
1/N 2 , confirming second-order accuracy of both methods.
The trapezoidal rule is equivalent to approximating the
function in each interval as a first-order Lagrange poly- interpolating
polynomial
nomial. To derive a higher-order Newton-Cotes formula,
we can approximate the function using a second-order La- f(x)
grange polynomial that passes through the left, middle, and
right points of each subinterval. For example, the area in
the first interval can be approximated as

Zh Zh h i x
f (x)dx ⇡ a + bx + cx2 dx, (12.11) x0 x1/2 x1
0 0 Figure 12.6

where the coefficients a, b, and c are chosen so the polynomial passes through f (x0 ), f (x1/2 ),
and f (x1 ), as sketched in figure 12.6. After some algebra, it can be shown that this leads to
2 3
/2 1
NX /2 1
NX
h 6666 77
7
I ⇡ 66f (x0 ) + 2 f (x2j ) + 4 f (x2j+1 ) + f (xN )777 , (12.12)
34 5
j=1 j=0

where this formula assumes N is even. This method, known as Simpson’s rule, is fourth-order
accurate.

12.1.2 Gaussian quadrature

Deriving higher-order Newton-Cotes methods, beyond Simpson’s rule, is limited by the fact
that as we increase the order of the interpolating polynomial, the polynomials become ill-
 Chapter 12. Numerical Solution of IVPs 241

behaved due to the Runge phenomena described in Section 1.3. See, for example, our discus-
sion of figure 1.7. In this section, we provide a brief introduction to a high-order method of
numerical integration, called Gaussian quadrature, which plays an important role in finite-
elements and spectral methods used to numerically approximate di↵erential equations.

Gaussian quadrature begins by noting that an integral over the interval x 2 [a, b] can
always be transformed to an integral over the interval x 2 [ 1, 1]

Z1
I= f (x) dx, (12.13)
1

by performing a change of variable. Now recall that the Newton-Cotes formulas in Sec-
P
tion 12.1.1 could all be written in the form of summations of the form f (xj ), where the
points xj are equispaced. Gaussian quadrature generalizes this procedure by approximating
the integral as
N
X
I⇡ wj f (xj ), (12.14)
j=1

where wj are called “weights,” and the points xj (called nodes) are no longer necessarily
equispaced.

To demonstrate, suppose we want numerically approximate I using f (x) evaluated at

only two points,
I ⇡ w1 f (x1 ) + w2 f (x2 ). (12.15)

The two weights (w1 , w2 ) and node locations (x1 and x2 ) give us four degrees of freedom
to maximize the accuracy of the approximation. We do this by maximizing the polynomial
order for which the method is exact, i.e.

R1
1dx = w1 + w2 = 2,
1
R1
xdx = w1 x1 + w2 x2 = 0,
1 (12.16)
R1
2
x2 dx = w1 x12 + w2 x22 = 3,
1
R1
x3 dx = w1 x13 + w2 x23 = 0.
1

This provides four equations for the four unknowns. The system of equations, however, is
nonlinear, and not straightforward to solve. More advanced theory deals with this hurdle
using an alternate approach based on the theory of orthogonal polynomials. As that is be-
yond our scope, we will assume here that we can solve the systems of equations using brute
 242 12.1. Numerical Integration

Gauss-Legendre Gauss-Lobatto

N wj xj Precision N wj xj Precision
p
2 1 1/p 3 3 2 1 -1 1
1 1/ 3 1 1
p
3 5/9 3/5 5 3 1/3 -1 3
8/9 0 4/3 0
p
5/9 3/5 1/3 1

Table 12.1: Weights and nodes for Gauss-Legendre and Gauss-Lobatto quadratures.

p p
force. For a two point approximation, this yields w1 = w2 = 1, x1 = 1/ 3, and x2 = 1/ 3, i.e.

Z1 ! !
1 1
I= f (x) dx ⇡ f p + f p . (12.17)
3 3
1

This formula is exact for any polynomial of order 3 or lower. As an example, consider for
example the integral
Z1 h i
I= 1 + 2x + 3x2 + 4x3 dx (12.18)
1

with the exact solution I = 4. The two-point numerical approximation produces the exact
answer,

" ! ✓ ◆ !# " ! ✓ ◆ !#
1 1 1 1 1 1
1 2 p +3 4 p + 1+2 p +3 +4 p = 4. (12.19)
3 3 3 3 3 3 3 3
| {z } | {z }
⇣ ⌘ ⇣ ⌘
1 1
f p f p
3 3

Following a similar approach, one can show that the three-point Gaussian quadrature
requires,
Z1 0 r 1 0r 1
5 BB
BB 3 CCC 8 5 BB 3 CC
BB C
I= f (x) dx ⇡ f @ CA + f (0) + f @ 5 CA . (12.20)
9 5 9 9
1

This formula is exact for any polynomial f (x) of order 5 or less. The weights wj and node
locations xj for the 2 and 3 point formulas are summarized in table 12.1. These are called the
“Gauss-Legendre quadratures.” An alternate quadrature, called “Gauss-Lobatto,” constrains
the node locations so that x1 = a and xN = b, i.e. the limits of the integral. The weights
and nodes for this approach are also summarized in table 12.1. The degree of precision,
i.e. the maximum order of polynomial for which the formulas are exact, are also listed. For
 Chapter 12. Numerical Solution of IVPs 243

Gauss-Legendre quadrature, the degree equals 2N 1, while for Gauss-Lobatto it is 2N 3.

12.2 Forward Euler method

The forward Euler method (often just called “Euler’s

method”) is the simplest numerical method for approxi- f(t,y(t))
mating IVPs. Suppose you want to solve the initial value
problem
dy
= f (t, y). (12.21)
dt

To solve this problem numerically, suppose y is known at

some time tn . We begin by integrating equation (12.21)
from time tn to tn+1 = tn + t,
tn tn+1
tn+1
Z tn+1
Z Figure 12.7
dy
dt = f (t, y(t))dt. (12.22)
dt
tn tn

The left-hand-side can be integrated exactly, producing

tn+1
Z
n+1 n
y =y + f (t, y(t))dt, (12.23)
tn

where y n denotes y evaluated at time tn . The remaining integral can be estimated using the
left-hand-rule,
y n+1 = y n + tf (tn , y(tn )), (12.24)

as sketched in figure 12.7. This produces a famous numerical method called the forward
Euler method, which can be abbreviated as

y n+1 = y n + tf n , forward Euler (12.25)

where f n denotes f (tn , y(tn )).

The forward Euler method can be interpreted physically as sketched in figure 12.8. The
initial condition y(0) = y 0 provides us a starting point as well as the initial slope dy/dx = f 0 ,
as sketched in panel (a). That allows us to estimate the solution at time t = t as y 0 + tf 0 ,
as sketched in panel (b). We then repeat as sketched in panel (c). To demonstrate, consider
the simple IVP
dy
= y cos(t), y(0) = 1, (12.26)
dt
 244 12.2. Forward Euler method

(a) (b) (c)

y1 + Δt f 1
y y y

1
f
0 y1
y + Δt f
0

0 0
y 0
f y 0
f y0
t t t

Figure 12.8

which has the exact solution

y = esin(t) . (12.27)

Given y(0) = 1, we compute y 1 as

y 1 = 1 + t cos(0) = 1 + t. (12.28)

We then compute y 2 as
y2 = 1 + t + t (1 + t) cos t (12.29)
|{z} | {z }
y1 f1

and so on.
The code below demonstrates how this can be implimented in matlab. Figure 12.9 shows
results when the ODE is integrated numerically from t = 0 to t = 2⇡ using 10, 20, and 50
time steps.
1 function[Err]=ForwardEuler(N,Tf,PLOTFLAG)
2
3 % ≠≠≠ Inputs ≠≠≠ %
4 % N = number of time steps
5 % Tf = final time
6 % PLOTFLAG = will make a plot when equal to 1
7
8 % ≠≠≠ Outputs ≠≠≠ %
9 % Err = Error computed at final time
10
11 dt = Tf/N; % compute time step
12 y(1)=1; % set initial conditions
13 t(1)=0; % set initial time
14
15 % march in time
16 for i=1:N
 Chapter 12. Numerical Solution of IVPs 245

(a) (b) (c)

3 3 3
ye ye ye
2.5 yn 2.5 yn 2.5 yn

2 2 2

1.5 1.5 1.5

1 1 1

0.5 0.5 0.5

0 0 0
0 1 2 3 4 5 6 0 1 2 3 4 5 6 0 1 2 3 4 5 6

Figure 12.9: Results for N = 10 (panel a), 20 (panel b), and 50 (panel c).

17
18 y(i+1) = y(i) + dt*y(i)*cos(t(i));
19 t(i+1)=i*dt;
20
21 end
22
23 % compute error at final time step
24 Err = abs(y(N+1)≠exp(sin(Tf)))
25
26 % make a plot comparing exact and numerical answer
27 if (PLOTFLAG==1)
28
29 t2 = linspace(0,Tf,200);
30 ye = exp(sin(t2));
31
32 figure
33 set(gca,'FontName','Times','FontSize',24)
34 hold on
35 plot(t2,ye,'≠k','LineWidth',2)
36 plot(t,y,'≠≠r','LineWidth',2)
37 plot(t,y,'.r','MarkerSize',20)
38 box on
39 grid on
40 xlabel('$t$','interpreter','latex')
41 ylabel('$y$','interpreter','latex','rot',0)
42 legend('y_e','y_n')
43
44 end

To determine the accuracy of the Euler method, we note that from a Taylor series

t 2 00
y(t + t) = y(t) + ty 0 (t) + y (t) + ... (12.30)
2
 246 12.2. Forward Euler method

(a) (b)

Err
1/N

10-1 10-1
Err
dt

102 10-1

Figure 12.10

Substituting y 0 (t) = f (t), we conclude that

y n+1 = y n + tf n + O( t 2 ). (12.31)

From this perspective, you might conclude the method is second-order accurate. Suppose,
however, that you want to integrate from t = 0 to some final time t = T using N time steps.
Because you make a second-order error every time step, the net error Err is proportional to

Err ⇠ N t 2 . (12.32)

Substituting N = T / t, we find that

✓ ◆
T 1
Err ⇠ t 2 ⇠ O ( t) ⇠ O . (12.33)
t N

We conclude that the forward Euler method is first-order accurate.

The matlab code below generates the error plots shown in figure 12.11. These confirm
that the error decreases as 1/N or t.
1 function[]=ErrPlot()
2
3 j=1;
4 for i=25:25:500
5
6 Err(j)=ForwardEuler(i,2*pi,0);
7 N(j)=i;
8 dt(j) = 2*pi/i;
9 j=j+1;
10
11 end
12
 Chapter 12. Numerical Solution of IVPs 247

(a) (b) (c)

f(t,y(t)) f(t,y(t)) f(t,y(t))

tn tn+1 tn tn+1/2 tn+1 tn tn+1

Figure 12.11

13 figure
14 set(gca,'FontName','Times','FontSize',24)
15 hold on
16 plot(N,Err,'*r','MarkerSize',8,'LineWidth',2)
17 plot(N,8./N,'≠≠k','LineWidth',2.0)
18 set(gca,'Yscale','log')
19 set(gca,'Xscale','log')
20 xlabel('$N$','interpreter','latex')
21 ylabel('$Err$','interpreter','latex')
22 box on
23 grid on
24 axis tight
25 legend('Err','1/N')
26
27
28 figure
29 set(gca,'FontName','Times','FontSize',24)
30 hold on
31 plot(dt,Err,'*r','MarkerSize',8,'LineWidth',2)
32 plot(dt,dt,'≠≠k','LineWidth',2.0)
33 set(gca,'Yscale','log')
34 set(gca,'Xscale','log')
35 xlabel('$dt$','interpreter','latex')
36 ylabel('$Err$','interpreter','latex')
37 box on
38 grid on
39 axis tight
40 legend('Err','dt')
 248 12.3. Backward Euler, leap frog, and Crank-Nicolson methods

12.3 Backward Euler, leap frog, and Crank-Nicolson methods

Three additional methods of approximating IVPs are derived by approximating the inte-
gral in equation (12.23) using the right-hand-rule, mid-point rule, and trapezoidal rule, as
sketched in figure 12.11. This produces the backward Euler, mid-point, and Crank-Nicolson
methods below,

y n+1 = y n + tf n+1 backward Euler (12.34)

y n+1 = y n + tf n+1/2 mid-point method (12.35)

t⇣ n ⌘
y n+1 = y n + f + f n+1 Crank-Nicolson (12.36)
2

In practice, the mid-point method is often implemented over two time steps by writing it as

y n+2 = y n + 2 tf n+1 leapfrog method, (12.37)

in which case it is often called the leap frog method.

Though not proven here, it can be shown that the backward Euler method is first-order
accurate, while the mid-point, leap-frog, and Crank-Nicolson methods are all second-order
accurate. In practice, the forward and backward Euler methods are rarely used because of
their low order-of-accuracy. One exception is when integrating an ODE to a steady-state
solution. If you are uninterested in accurately approximating the initial transient, then the
forward and backward Euler methods can be used.
Of the four methods discussed so far (forward/backward Euler, leap frog, and Crank-
Nicolson), the forward Euler method is called an explicit method because f (t, y) is only eval-
uated at the previous time-step. In contrast, the remaining three methods are all implicit
methods. Implicit methods are more difficult to implement, but have greater numerical sta-
bility than explicit methods. The concept of stability is defined and discussed in section
12.5. For now, we simply state that explicit methods tend to behave poorly when the time
step t is too large, whereas implicit methods permit larger time steps.
To demonstrate, the application of an implicit method,
let’s consider the application of the backward Euler 3
ye
method to the IVP considered in section 12.2, 2.5 yn

2
dy
= y cos(t), y(0) = 1. (12.38) 1.5
dt
1

0.5
The Backward Euler method can be written as
0
0 1 2 3 4 5 6
n+1 n n+1
y = y + ty cos(tn+1 ), (12.39)
Figure 12.12
 Chapter 12. Numerical Solution of IVPs 249

and solved for y n+1

yn
y n+1 = , (12.40)
1 t cos(tn+1 )

Figure 12.12 shows the results when the IVP is integrated to t = 2⇡ using 50 time steps.
For linear IVPs such as equation (12.38) above, the application of backwards Euler ap-
pears straightforward. Now consider the nonlinear IVP

dy
= y 2 cos y. (12.41)
dt

In this case, the backward Euler method produces a nonlinear relationship for y n+1 ,

y n+1 = y n + t(y n+1 )2 cos y n+1 , (12.42)

which cannot be explicitly solved for y n+1 ,

h i
y n+1 1 ty n+1 cos y n+1 = y n . (12.43)

In this case, we would need to solve the nonlinear relationship for y n+1 using some sort of
iterative procedure at at each time step.

12.4 Runge-Kutta methods

Of the numerical methods discussed so far, the second-order Crank-Nicolson and leap-frog
methods are both implicit, which makes them impractical for nonlinear IVPs. Two fami-
lies of higher-order explicit methods include the Adams-Bashforth methods and Runge-Kutta
methods. For brevity, we only discuss Runge-Kutta methods in this text.
To motivate Runge-Kutta methods, let’s reconsider the mid-point method below,

y n+1 = y n + tf (tn+1/2 , y n+1/2 ). (12.44)

We can make this method explicit by first approximating y n+1/2 using the the forward Euler
method over a half time-step t/2.

t n
y n+1/2 ⇡ y n + f . (12.45)
2

This produces the following explicit implementation of the mid-point method

y⇤ = yn + t n
2 f ,
explicit midpoint method. (12.46)
y n+1 = y n + tf (tn+1/2 , y ⇤ ),
 250 12.4. Runge-Kutta methods

Similarly, we can approximate the Crank-Nicolson method explicitly as

y⇤ = yn + tf n ,
⇣ ⌘ improved Euler method. (12.47)
t
y n+1 = y n + 2 f (tn , y n ) + f (tn+1 , y n+1 ,

This is often called the improved Euler method or Heun’s method.

The explicit mid-point method and improved Euler method are both examples of two-
stage Runge-Kutta methods, which can be written in the general form

y n+1 = y n + a1 k1 + a2 k2 , (12.48)

where
k1 = t f (tn , y n )
k2 = t f (tn + ↵ t, y n + k1 )

The explicit mid-point method is recovered when a1 = 0, a2 = 1, and ↵ = = 1/2

y n+1 = y n + k2 ,

k1 = t f (tn , y n )
1 1
k2 = t f (tn + t, y n + k1 ),
2 2

while the improved Euler method is recovered when a1 = a2 = 1/2 and ↵ = = 1,

y n+1 = y n + k1 + k2 ,

k1 = t f (tn , y n ),
k2 = t f (tn + t, y n + k1 ).

The accuracy of two-stage Runge-Kutta methods is determined by performing a Taylor

series of y n+1 about tn ,

dy n t2 d 2yn
y n+1 = y n + t + + O( t 3 ). (12.49)
dt 2 dt 2

From the given di↵erential equation we can replace dy/dt in the Taylor series with f (t, y).
We can also replace d 2 y/dt 2 with

d 2y d @f @f dy
2
= f (t, y) = + , (12.50)
dt dt @t @y dt

where we have used the chain rule to account for the fact y(t) is itself a function of t. This in
 Chapter 12. Numerical Solution of IVPs 251

turn can written as

d 2 y @f @f
2
= +f . (12.51)
dt @t @y

We can now rewrite the Taylor series as

" #
t 2 @f n n @f
n
y n+1 n
= y + tf n
+ +f + O( t 3 ), (12.52)
2 @t @y

where the subscript n implies that everything on the right-hand-side is evaluated using tn
and y n .

To derive a second-order accurate Runge-Kutta method, we must choose the coefficients

a1 , a2 , ↵, such that equation (12.48) agrees with the Taylor series equation (12.52). For that
purpose, we expand f (tn + ↵ t, y n + k1 ) in equation (12.48) using a two-dimensional Taylor
series,

@f n @f n
f (tn + ↵ t, y n + k1 ) = f (tn , yn ) + ↵ t + k1 + O( t 2 ). (12.53)
@t @y

Note that ↵ t and k1 = tf (tn , y n ) are both of order t. The remainder above is consequently
order t 2 . Substituting this expansion in the Runge-Kutta formula (12.48), we find that
!
@f n @f n
y n+1 n n
= y + a1 t f + a2 n
t f +↵ t + k1 + O( t 3 ). (12.54)
@t @y

If we further substitute k1 = t f n and collect similar terms, we can rearrange this as

@f n @f n
y n+1 = y n + (a1 + a2 ) tf n + a2 ↵ t 2 + a2 t2f n + O( t 3 ). (12.55)
@t @y

The above equation (12.55) agrees with the Taylor expansion (12.52) when

1 1
a1 + a2 = 1, a2 ↵ = , a2 = . (12.56)
2 2

These provide three equations for four unknown coefficients. We can consequently solve for
a1 , a2 , and in terms of ↵

1 1
a1 = 1 , a2 = , = ↵, (12.57)
2↵ 2↵
 252 12.5. Stability

(a) (b)
1
4

0.5 2

0 0

-0.5 -2

-1
0 1 2 3 4 5 0 1 2 3 4 5

Figure 12.13

and rewrite all second-order Runge-Kutta methods as

k1 = t f (tn , y n ),

k2 = t f (tn + ↵ t, y n + ↵k1 ), (12.58)

⇣ ⌘
1 1
y n+1 = y n + 1 2↵ k1 + 2↵ k2 ,

Though all such methods are second-order accurate, the choice of ↵ also impacts the numer-
ical stability of the method, as well as other properties that are beyond our scope.
Though there are many additional third-order and fourth-order Runge-Kutta methods
(including both explicit and implicit methods), we do not present them here. We only briefly
present a famous fourth-order method, which can be written as

k1 = tf (tn , y n )

k2 = tf (tn+1/2 , y n + k21 )

k3 = tf (tn+1/2 , y n + k22 ) fourth-order Runge-Kutta. (12.59)

k4 = tf (tn+1 , y n + k3 )

y n+1 = y n + 16 (k1 + 2k2 + 2k3 + k4 ) ,

12.5 Stability

The stability of numerical methods for approximating IVPs is an important topic. For the
limited scope of this introductory course, however, we only touch on some key concepts, and
encourage students to consult more advanced textbooks for further discussion. Let’s begin
 Chapter 12. Numerical Solution of IVPs 253

by considering the simple IVP

dy
= cy, y(0) = y0 , (12.60)
dt

where c is a real constant coefficient. This IVP has the exact solution

y = y0 ect , (12.61)

and the equilibrium solution,

ye = 0. (12.62)

The equilibrium solution is stable when c < 0, because the solution y(t) converges to y = 0
with time, for any initial value y0 , as sketched in figure 12.13(a). The equilibrium solution
is unstable if c > 0, because y(t) diverges from y = 0 for any finite initial condition (y0 , 0),
as sketched in figure 12.13(b).

Ideally, numerical methods of approximating IVPs should match the stability of the IVP
being approximated. A great way of studying the stability of numerical methods is to apply
them to the IVP (12.60) above, because it has only one equilibrium solution, and the stability
of that solution is well understood. Consider the forward Euler method

y n+1 = y n + tf n . (12.63)

Setting f = cy, we find that

y n+1 = y n + tcy n . (12.64)

This can be rearranged as

y n+1
G= = 1 + c t, (12.65)
yn

where G is called the amplification. We define a method as stable if the absolute value of the
amplification is less than 1, i.e.
1 < G < 1. (12.66)

For stability, the forward Euler method requires

1 < 1 + c t < 1, (12.67)

which in turn requires

2 < c t < 0. (12.68)
 254 12.5. Stability

We make the following conclusions about the forward Euler method:

1. When the IVP is stable (c < 0), the forward Euler method is only stable when
t < 2/|c|.

2. When the IVP is unstable (c > 0), the forward Euler method is also unstable.

Next, let’s consider the backwards Euler method

y n+1 = y n + tf n+1 . (12.69)

Setting f = cy, we find that

y n+1 = y n + tcy n+1 . (12.70)

This can be rearranged as

y n+1 1
G= = (12.71)
y n (1 c t)

Stability occurs when the absolute value of the denominator is greater than unity, i.e.

|1 c t| > 1. (12.72)

When c is negative, this is always true. When c is positive, stability occurs when

1 c t < 1, (12.73)

or
2
t> . (12.74)
c

We make the following conclusions about the backward Euler method:

1. When the IVP is stable (c < 0), the backward Euler method is also stable.

2. When the IVP is unstable (c < 0), the backward Euler method is unstable when
t < 2/c.
 Chapter 12. Numerical Solution of IVPs 255

12.6 Systems of IVPs

To demonstrate the numerical approximation of systems
of IVPs, let’s consider an application to ballistics. A pro- y
jectile is launched from the position (x, y) = (0, 0) with an
initial speed V0 and angle ✓, as sketched in figure 12.14.
Assuming the only force is gravity, Newton’s laws require
V 0
g
ax = 0, ay = g, (12.75) θ
x
Figure 12.14
where g = 9.81 m/s2 is the gravitational acceleration, and
ax and ay are the components of the projectile acceleration in the x and y directions, re-
spectively. The above equations can be expressed as the following system of four first-order
IVP’s,
2 3 2 3
66 x 77 66 u 77
6 7 66 77
d 666 y 777 6v7
66 77 = 666 777 . (12.76)
6
dt 66u 77 7 66 0 77
4 5 6
4 5 7
v g

This can expressed be more compactly as

2 3 2 3
66 x 77 66 u 77
66 77 66 77
dy 6y 7 6v7
= f, where y = 6666 7777 , f = 6666 7777 , (12.77)
dt 66u 77 66 0 77
4 5 4 5
v g

subject to the initial condition

2 3
66 0 77
66 7
6 0 777
y(0) = 6666 77 . (12.78)
66V0 cos ✓ 777
4 5
V0 sin ✓

We can apply the forward Euler method to the system (12.77) as

yn+1 = yn + tfn . (12.79)

For example,
2 13 2 03 2 3
66 x 77 66 x 77 66V0 cos ✓ 77
66 1 77 66 0 77 66 7
66 y 77 6y 7 6 V sin ✓ 777
66 1 77 = 666 0 777 + t 666 0 7 (12.80)
66u 77 66u 77 66 0 777 .
64 1 75 64 0 75 64 75
v v g

The following matlab code demonstrates the procedure.

 256 12.6. Systems of IVPs

1 function[x,y,t]=Ballistics(Vo,theta,N,dt)
2
3 % ≠≠≠ Inputs ≠≠≠ %
4 % Vo = initial speed
5 % theta = initial angle
6 % N = number of time steps
7 % dt = time step
8
9 % ≠≠≠ Outputs ≠≠≠ %
10 % x,y,t = vectors of length N+1 of containing
11 % projectile locations and time
12
13 g = 9.81; % gravitational acceleration
14
15 % initial condition
16 Y0 = [0; 0; Vo*cos(theta); Vo*sin(theta)];
17
18 % initialize outputs
19 x(1) = 0;
20 y(1) = 0;
21 t(1) = 0;
22
23 % open a figure
24 figure
25 set(gca,'FontName','Times','FontSize',20)
26
27 for i = 1:N
28
29 F = [Y0(3); Y0(4); 0; ≠g];
30 Y1 = Y0 + dt*F;
31
32 t(i+1) = i*dt;
33 x(i+1) = Y1(1);
34 y(i+1) = Y1(2);
35
36 % update plot
37 hold off
38 plot(x,y,'≠≠k','LineWidth',2)
39 hold on
40 plot(x(i+1),y(i+1),'.k','MarkerSize',20)
41 grid on
42 box on
43 xlabel('$x$','interpreter','latex','FontSize',20)
44 ylabel('$y$','interpreter','latex','FontSize',20)
45 axis([0 Vo*cos(theta)*N*dt 0 Vo*sin(theta)*N*dt])
46 pause(.02)
47
48 Y0 = Y1;
 Chapter 12. Numerical Solution of IVPs 257

49
50 end
13. Numerical Solution Boundary Value Problems

13.1 Numerical di↵erentiation 258

13.2 Numerical solution of boundary value problems 264

13.1 Numerical di↵erentiation

13.1.1 Approximating the first derivative

Numerical di↵erentiation refers to methods of approx-

f(x)
imating the derivative of a function without perform-
ing the derivative analytically. Usually, we resort to
numerical di↵erentiation because the function is only fj-1 fj+1
known at discrete points, in which case there is no an-
alytical relation for f (x). Suppose a function f (x) is fj
known at the discrete points xj sketched in figure 13.1.
Δx Δx
Though not required generally, we assume hereinafter
that the points xj are separated by the constant x. x j-1 xj x j+1
This is called an “equispaced” grid. For convenience, Figure 13.1
we also use the notation f j to denote f (xj ).

Using the three points sketched, there are three obvious approximations to the derivative
at xj . The first, called a forward di↵erence, approximates the derivative using information at
xj and xj+1 ,
df j f j+1 f j
⇡ . (13.1)
dx x

This produces the slope shown as a blue line in figure 13.1. The second estimate, called a

258
 Chapter 13. Numerical Solution Boundary Value Problems 259

backward di↵erence, approximates the derivative using information at xj and xj 1 ,

df j fj fj 1
⇡ . (13.2)
dx x

This produces the slope shown as a red line in figure 13.1. The third estimate, called a
centered di↵erence, approximates the derivative using information at xj+1 and xj 1 ,

df j f j+1 f j 1
⇡ . (13.3)
dx 2 x

This produces the slope shown as a green line in figure 13.1.

As we take the limit x ! 0, all three estimates above converge to the exact derivative
at xj . We can show, however, that the centered di↵erence converges much faster than the
forward and backwards di↵erence. To show why, let’s express f j+1 in a Taylor series about
xj ,

⇣ ⌘2 ⇣ ⌘3
⇣ ⌘ df j xj+1 xj d 2 f j xj+1 xj d 3 f j
f j+1 = f j + xj+1 xj + + + ... (13.4)
dx 2 dx2 6 dx3

Using the fact that xj+1 xj = x, we can rewrite this as

2 3
df j x2 d f j x3 d f j
f j+1 = f j + x + + + ... (13.5)
dx 2 dx2 6 dx3

We can then solve this expression for df j /dx,

2 3
df j f j+1 f j x d fj x2 d f j
= ... (13.6)
dx x 2 dx2 6 dx3
| {z }
leading error term

As x tends to small values, the error term proportional to x2 becomes negligible compared
to the leading error term, which is proportional to x. This result is often written as

df j f j+1 f j
= + O( x). (13.7)
dx x

The “Big O” notation O( x) means the error is of order x.

Now suppose we expand f j 1 about xj ,

2 3
df j x2 d f j x3 d f j
fj 1 = fj x + ... (13.8)
dx 2 dx2 6 dx3
 260 13.1. Numerical di↵erentiation

Solving for df j /dx as before, we find that

df j fj fj 1
= + O( x). (13.9)
dx x

We see that the forward and backward di↵erences have the same order of accuracy O( x).
Finally, to find the error for the centered di↵erence, we expand both f j+1 and f j 1 about
the point xj ,

2 3
df x2 d f j x3 d f j
f j+1 = f j + x dxj + 2 dx2 + 6 dx3 + ...
(13.10)
2 3
df x2 d f j x3 d f j
fj 1 = fj x dxj + 2 dx2 6 dx3 + ...

Subtracting these equations, we find that,

3
df j x3 d f j
f j+1 f j 1 = 2 x + + ... , (13.11)
dx 3 dx3

which can be solved for df j /dx to show that

df j f j+1 f j 1
⇡ + O( x2 ). (13.12)
dx 2 x

We see that the error for the centered di↵erence formula varies like x2 . The forward and
backward di↵erences above are said to be first-order accurate, while the centered di↵erence
approximation is called second-order accurate. If you make x ten times smaller, we expect
the error of the forward and backward di↵erence to decrease tenfold, while the centered
di↵erence approximation will decrease 100-fold. For this reason, numericists avoid first-
order approximation at all costs. In practice they are found to be too inaccurate, particularly
when used to solve partial di↵erential equations. At the minimum, numericists always strive
to achieve second-order, or higher, approximations.

As an example, suppose we want to approximate

f(x)
the derivative of esin(x) numerically between x = 1 and
x = 10. We start by making a grid of N + 1 equispaced
points between x = 1 and x = 10. These are sketched Δx
x0 x1 x2 x3 x4 x5
in figure 13.2 for the case N + 1 = 4. The grid spacing
1 10
is given as x = 9/N . We also add the points x0 and x5
Figure 13.2
outside to allow us to apply the finite-di↵erence for-
mulas at x = 1 and x = 10. The following MATLAB code computes the numerical derivatives
using the centered di↵erence formula and makes a plot of the exact and numerical answers:
1 function[Err]=FD(N)
2
3 dx = 9/N; % grid spacing
4
5 % compute grid points, including the two extra points
 Chapter 13. Numerical Solution Boundary Value Problems 261

6 x=linspace(1≠dx,10+dx,N+3);
7
8 f = exp(sin(x)); % f(x) at grid points
9
10 dfe = cos(x).*exp(sin(x)); % exact derivative
11
12 % compute derivative using centered≠differences
13 df = 0.0*x;
14 for i=2:N+2
15 df(i) = ( f(i+1) ≠ f(i≠1) )/(2*dx);
16 end
17
18 % Compute Error
19 Err = max(abs(df(2:N+2)≠dfe(2:N+2)))/max(abs(dfe));
20
21 figure
22 set(gca,'FontName','Times','FontSize',20)
23 hold on
24 plot(x(2:N+2),dfe(2:N+2),'≠k','LineWidth',2.0)
25 plot(x(2:N+2),df(2:N+2),'≠≠r','LineWidth',2.0)
26 xlabel('$x$','interpreter','latex')
27 ylabel('$\frac{df}{dx}$','interpreter','latex','rot',0)
28 legend('Exact','Numerical')
29 grid on
30 box on
31 axis tight

Note that the code measures the error as

kf N0 f e0 k1
Err = , (13.13)
kf e0 k1

(a) (b) (c)

Exact Exact
1 Numerical Numerical
Err
1 1/N2
10-1
0.5
0.5
df df
Err

dx 0 dx 0
10-2
-0.5 -0.5

-1
-1
10-3 1
2 4 6 8 10 2 4 6 8 10 10 102
x x N

Figure 13.3: Results produced by the MATLAB code. Panels (a) and (b) show the exact and numerical
results for N = 10 and N = 50, respectively. Panel (c) shows a log-log plot of the error. The dashed line
shows that Err varies as 1/N 2 , which is expected for centered di↵erences.
 262 13.1. Numerical di↵erentiation

where f N0 and f e0 are the numerical and exact derivatives at the grid points. The symbol k k1
is called the infinity norm. It corresponds to the maximum absolute value of a vector. In
MATLAB, I implemented this as max(abs( )). Also note that the error is normalized with
respect to the infinity norm of the exact answer.
Figures 13.3(a) and 13.3(b) compare plots of the exact and numerical derivatives using
N = 10 and N = 50. The accuracy of a finite di↵erence method is often evaluated by plotting
the error vs N , using a logarithmic scale for both axes. Figure 13.3(c) shows a log-log plot of
the error vs. N for the current example. Suppose the error of a method varies with x as

Err ⇠ xb , (13.14)

where b = 2 for the centered di↵erence formula. If the domain width is L, then x = L/N .
This allows us to write
b
Err ⇠ N . (13.15)

Taking the logarithm of both sides, we find that

log(Err) ⇠ b log(N ). (13.16)

Consequently, we expect a log-log plot of Err vs. N to decrease with slope b. In fig-
ure 13.3(c), we see the error (shown as symbols) decreases with a slope of 1/N 2 (shown
as dashed line). This confirms the second-order accuracy of the method.

13.1.2 One-sided di↵erences

Suppose we wish to approximate the derivative f 0 (x) at the

four locations marked in figure 13.4. For the interior points
f(x)
x2 and x3 , we can achieve second-order accuracy using the
centered-di↵erence formula (13.3). At the two end points,
however, the centered di↵erence formula isn’t applicable.
To maintain second-order accuracy, we can derive a second-
order accurate one-sided di↵erence approximation as fol-
lows. We expand f 2 and f 3 in a Taylor series as,
x
x2 00 x1 x2 x3 x4
f2 = f1 + xf 10 + 2 f1 + O( x3 )
Figure 13.4
f 3 = f 1 + 2 xf 10 + 2 x2 f 100 + O( x3 )

We then see that we can cancel the f 100 term by taking the linear combination 4f 2 f 3 ,

4f 2 f 3 = 3f 1 + 2 xf 10 + O( x3 ), (13.17)

which can be solved for f 10

3f 1 + 4f 2 f 3
f 10 = + O( x2 ). (13.18)
2 x
 Chapter 13. Numerical Solution Boundary Value Problems 263

Note that the error term fell to order x2 because solving for f 10 required division by x.
Following a similar approach, you can show that f 40 can be estimated to second order as

3f 4 4f 3 + f 2
f 40 = + O( x2 ). (13.19)
2 x

In the interest of generality, these relations can be written as

3f j + 4f j+1 f j+2 3f j 4f j 1 + fj 2
f j0 ⇡ f j0 ⇡ . (13.20)
2 x 2 x

13.1.3 Approximating the second derivative

An approximation to the second derivative can be derived in a similar manner to the cen-
tered di↵erence approximation for f j0 . We first expand f j 1 and f j+1 in a Taylor series about
the value f j

x2 00 x3 000 x4 d 4 fj
f j+1 = f j + xf j0 + 2 fj + 6 fj + 24 dx4
+ ...
d 4 fj
(13.21)
x2 00 x3 000 x4
fj 1 = fj xf j0 + 2 fj 6 fj + 24 dx4
...

Summing these equations, we find that the terms involving odd-powered derivatives cancel,
such that
d 2fj 4
x4 d f j
f j+1 + f j 1 = 2f j + x2 + + ... (13.22)
dx2 12 dx4

This can then be solved for d 2 f j /dx2 to show that

d 2fj fj 1 2f j + f j+1
= + O( x2 ). (13.23)
dx2 x2

We see that this produces a second-order accurate approximation to f j00 .

 264 13.2. Numerical solution of boundary value problems

13.2 Numerical solution of boundary value problems

Suppose we want to solve the following ODE using

u(x)
finite-di↵erence methods

d 2u Δx
= f (x), a  x  b, (13.24) x0 x1 x2 x3 x4 x5
dx2
a b

subject to the boundary conditions Figure 13.5

u(a) = ↵, u(b) = . (13.25)

We start by building a grid of N + 1 points

xj = jh, j = 0, 1, ..., N , (13.26)

separated by the constant distance h = (b 1)/N , as sketched in figure 13.6 for the case N = 5.
At the interior points x1 to xN 1 , we can approximate ODE (13.24) using the finite-di↵erence
approximation
uj 1 2uj + uj+1
= f (xj ). (13.27)
h2

For the example, in figure 13.6, this relation can be applied at the interior points x1 to x4 .
At the first and last grid point, we apply the boundary condition u0 = ↵ and uN = . This
produces N + 1 equations for the N + 1 unknown values of uj , which we collect into the
following matrix problem
2 32 3 2 3
66 1 0 0 0 0 0 77 66u0 77 66 ↵ 77
66 2 76 7 6 7
661/h 2/h2 1/h2 0 0 0 777 666u1 777 666f 1 777
66 76 7 6 7
66 0
66 1/h2 2/h2 1/h2 0 0 7777 6666u2 7777 6666f 2 7777
76 7 = 6 7 (13.28)
66 0
66 0 1/h2 2/h2 1/h2 0 777 666u3 777 666f 3 777
66 0 76 7 6 7
66 0 0 1/h2 2/h2 1/h2 7777 6666u4 7777 6666f 4 7777
4 54 5 4 5
0 0 0 0 0 1 u5

The following matlab code solves the ODE

d 2u
= sin(x), u(0) = 0, u(L) = sin(L), (13.29)
dx2

which has the exact solution u = sin(x). Figure 13.6(a) shows the resulting plot of the exact
and numerical answers when N = 32 and L = 20.

1 function[Err]=BVP(N,L,PLOTFLAG)
2
3
4 % N+1 equsipaced points
 Chapter 13. Numerical Solution Boundary Value Problems 265

5 % note that the apostrophe after linspace makes x

6 % a column vector
7 x=linspace(0,L,N+1)';
8
9
10 % grid spacing
11 h=L/N;
12
13 % build right≠hand side
14 f=≠sin(x);
15 f(1) = sin(0);
16 f(N+1) = sin(L);
17
18 % build matrix
19 A=zeros(N+1,N+1);
20
21 % interior rows
22 for j=2:N
23 A(j,j) = ≠2/h^2;
24 A(j,j≠1) = 1/h^2;
25 A(j,j+1) = 1/h^2;
26 end
27
28 %boundary conditions
29 A(1,1) = 1;
30 A(N+1,N+1) = 1;
31
32 % solve matrix
33 u=A\f;
34

(a) (b)

Figure 13.6: (a) Results produced by BVP.m when N = 32 and L = 20. (b) Error plot generated by
BVPErr.m when L = 20.
 266 13.2. Numerical solution of boundary value problems

35 % compute error
36 ue = sin(x);
37 Err = max(abs(u≠ue));
38
39 if (PLOTFLAG == 1)
40
41 figure
42 hold on
43 set(gca,'FontName','Times','FontSize',20)
44 plot(x,ue,'.k','MarkerSize',18)
45 plot(x,u,'≠≠r','LineWidth',2.0)
46 legend('Exact','Numerical')
47 xlabel('$x$','interpreter','latex','FontSize',20)
48 ylabel('$u$','interpreter','latex','FontSize',20,'rot',0)
49 grid on
50 box on
51 axis tight
52
53 end

The next matlab function uses BVP.m above to build an error plot demonstrating the
second-order accuracy of the method. Figure 13.6(b) shows the resulting plot when L = 20.
1 function[]=BVPErr(L)
2
3 l=1;
4 for j=20:20:200
5
6 Err(l) = BVP(j,L,0);
7 N(l) = j;
8 l=l+1;
9
10 end
11
12
13 figure
14 hold on
15 set(gca,'FontName','Times','FontSize',20)
16 plot(N,Err,'.k','MarkerSize',20)
17 plot(N,10*N.^(≠2),'≠≠r','LineWidth',2.0)
18 legend('Err','1/N^2')
19 xlabel('$N$','interpreter','latex','FontSize',20)
20 ylabel('$Err$','interpreter','latex','FontSize',20,'rot',0)
21 set(gca,'xscale','log')
22 set(gca,'yscale','log')
23 grid on
24 box on
25 axis tight
 Chapter 13. Numerical Solution Boundary Value Problems 267

As a final example, suppose we consider the slightly more complicated ODE,

d 2u du
+x = sin(x) + x cos(x), (13.30)
dx2 dx

subject to the conditions

u(0) = 0, u(L) = sin(L). (13.31)

This ODE also has the exact solution u = sin(x). On the internal nodes of our grid we must
modify the code BVP.m to approximate the ODE as

uj 1 2uj + uj+1 uj+1 uj 1

+ xj = f (xj ), (13.32)
h2 2h

where we have used a centered di↵erence approximation for du/dx to maintain the second-
order accuracy of our method. To solve the ODE numerically, we only need to make several
small changes to the our earlier code BVP.m. First, we replace line 14 with
1 f=≠sin(x)+x.*cos(x);

Second, we now build the internal rows of the matrix as

1 for j=2:N
2 A(j,j) = ≠2/h^2;
3 A(j,j≠1) = 1/h^2≠x(j)/(2*h);
4 A(j,j+1) = 1/h^2+x(j)/(2*h);
5 end

One final topic we have not discussed here is the application of Neumann and Robin
boundary conditions. Those are left to homeworks.
14. Numerical Solution of Partial Di↵erential Equations

14.1 The 1D Heat equation 268

14.2 Poisson equation 281

14.1 The 1D Heat equation

In this section, we consider the 1D heat equation,

@u @2 u
= ↵ 2 + q(x, t), (14.1)
@t @x

subject to general Robin boundary conditions of the form

@u
a u(0, t) + b = g0 (t), (14.2)
@x x=0

@u
c u(L, t) + d = gL (t), (14.3)
@x x=L

and the initial condition

u(x, 0) = f (x). (14.4)

Because the heat equation (14.1) is an IBVP (Initial-Boundary-Value-Problem) that de-

pends on both time and space, we need to consider it’s discretization in both time and space.
For the scope of this textbook, we’ll assume that we discretize the heat equation spatially
using a finite-di↵erence grid of N + 1 equispaced points between x = 0 to x = L,

L
xj = (j 1) x, j = 1, 2, ... , N + 1, x= . (14.5)
N

268
 Chapter 14. Numerical Solution of Partial Di↵erential Equations 269

That allows us to approximate the second derivative in the heat equation as

@2 u j uj 1 2uj + uj+1
⇡ , (14.6)
@x2 x2

where uj = u(xj , t). Next, we assume that time is discretized using a constant time step t,
and introduce the notation

ujn = u(xj , tn ), where t n = n t. (14.7)

In the next sections, we demonstrate the temporal integration of the heat equation us-
ing the forward and backward Euler methods. We also demonstrate matlab codes for both
approaches using the forcing term q(x, t)

q(x, t) = ! sin(x) sin(!t) + ↵ sin(x) cos(!t). (14.8)

This forcing term satisfies the exact solution

u(x, t) = sin(x) cos(!t), (14.9)

which is steady when ! = 0 and unsteady otherwise. In section 14.1.3, we use this exact
solution to verify the spatial and temporal accuracy of our methods.

14.1.1 Forward Euler

Recall from section 12.2, that given an IVP

du
= f (u, t), (14.10)
dt

the forward Euler method approximates the solution as

u n+1 = u n + tf (u n , t n ), (14.11)

where u n is the solution at time t n = n t. To discretize the heat equation (14.1) using the
forward Euler method, we set

@2 u n
f (u n , t n ) = ↵ + q(x, t n ). (14.12)
@x2
 270 14.1. The 1D Heat equation

Figure 14.1 shows the finite-di↵erence

grid when N = 7. On the internal points u(x,t) u51
u41 u61
x2 to xN , we can approximate f (u n , t n ) to u1
3
u71
second-order accuracy using the finite dif- u21
ference approximation,
u40 u50 u60 u70
ujn 1 2ujn + uj+1
n
u30 u80
n n n
f (u , t ) = ↵ + q(xj , t ). u0

x2 u10
2

internal nodes
L
Substituting this approximation into the x
x1 x2 x3 x4 x5 x6 x7 x8
forward Euler formula, we find that on the
Figure 14.1
internal grid points, we can solve for ujn+1
using only information from the previous time step
" n #
uj 1 2ujn + uj+1
n
ujn+1 = ujn + t ↵ n
+ q(xj , t ) . (14.13)
x2

Figure 14.1 demonstrates that if we are given the 8 initial values u 0 , we can solve for the 6
internal values u 1 . The red dashed lines show that u21 depends on u10 , u20 , and u30 . Similarly,
u61 depends on u50 , u60 , and u70 .

After solving for uj1 on the internal nodes, we can solve for the boundary nodes u11 and
1
uN +1 using the boundary conditions. For example, using a simple one-sided di↵erence, we
can approximate the Robin condition (14.2) at x = 0 as

u21 u11
au11 + b = g0 (t 1 ). (14.14)
x

This can be rearranged to solve for u11 as

x g0 (t 1 ) bu21
u11 = . (14.15)
a x b

Note that the boundary condition is applied at the time level t 1 . To ensure second-order
spatial accuracy, we should use a second-order accurate one-sided di↵erent to approximate
@u/@x in the boundary conditions. This is left as a homework problem.

The following matlab code implements the forward Euler method discussed above using
the forcing term
q(x, t) = ! sin(x) sin(!t) + ↵ sin(x) cos(!t) (14.16)

and boundary conditions

u(0, t) = 0, u(L, t) = sin(L) cos(!t). (14.17)

 Chapter 14. Numerical Solution of Partial Di↵erential Equations 271

(a) (b)
1 1

0.5 0.5

0 0

-0.5 -0.5

-1 -1
0 2 4 6 0 2 4 6

Figure 14.2: Results produced by the forward Euler discretization of heat equation when ! = 0, u 0 = 0,
and N = 32 (panel a) and N = 64 (panel b). The black lines show the exact steady state solution
u = sin(x). The red lines show snapshots of the numerical approximation.

When the input IC is set to 0, the code sets ! = 0 and uses the initial condition

u(x, 0) = 0. (14.18)

The objective is to then integrate in time to steady-state, at which point we can compare
with the exact solution u = sin(x). When I C = 1, the code sets ! = 2⇡ and uses the initial
condition
u(x, 0) = sin(x). (14.19)

The objective is to then integrate to some final time tf , and compare with the exact solu-
tion u = sin(x) cos(2⇡tf ). These two choices of initial conditions are useful for verifying the
temporal and spatial accuracy of the code, as demonstrated later in section 14.1.3.
Figure 14.2(a) shows results produced by the matlab code when I C = 0, L = 2⇡, dt = 0.01,
and N = 32. The black line shows the exact steady-solution u = sin(x), while the dashed
red lines show snapshots of the numerical approximation at several early times. For this
combination of N and dt, the simulation eventually reaches a steady state, and produces
a maximum relative error below 0.1 %. Figure 14.2(b) shows that when the simulation is
repeated for the finer grid N = 64, a numerical instability appears, taking the form of rapid
oscillations. These grow in amplitude with time steps, such that the code diverges, and
never reaches a steady-state. The instability disappears, however, if the time step is reduced
to t = 0.0048. Further tests show that for each N (or x), there is a maximum time step,
tmax , above which the method becomes unstable. This maximum time step varies with N
as
1
tmax ⇠ 2 ⇠ x2 , (14.20)
N

such that finer grids require increasingly small time steps.

1 function[U1] = Main(alpha,N,L,Nt,dt,INCR,IC)
2
 272 14.1. The 1D Heat equation

3 %≠≠≠≠≠ INPUTS ≠≠≠≠≠≠≠≠≠≠≠≠%

4 % alpha = thermal diffusivity
5 % N = resolution of finite difference grid
6 % L = domain length
7 % Nt = number of time steps
8 % dt = time step
9 % INCR: will plot result every INCR time steps
10 % IC = initial condition
11
12 % build finite difference grid.
13 % Note that i use the apostrophe to make the
14 % array a column
15
16 x=linspace(0,L,N+1)';
17 dx = L/N;
18
19 % Set initial condition
20 if (IC == 0)
21
22 omega = 0;
23 U0 = 0*x;
24 U1 = 0*x;
25
26 else
27
28 omega = 2*pi;
29 U0 = sin(x);
30 U1 = 0*x;
31
32 end
33
34 t = 0; % set initial time
35 figure % open figure for plotting purposes
36
37 % start time loop
38 for k = 1:Nt
39
40 % update internal nodes
41 for i=2:N
42
43 q = ≠omega*sin(x(i))*sin(omega*t) + ...
44 alpha*sin(x(i))*cos(omega*t);
45
46 F = alpha*(U0(i≠1) ≠ 2*U0(i) + U0(i+1))/dx^2 + q;
47
48 U1(i) = U0(i) + dt*F;
49
50 end
 Chapter 14. Numerical Solution of Partial Di↵erential Equations 273

51
52 % advance time before applying boundary conditions
53 t = t + dt;
54
55 % apply boundary conditions
56 U1(1) = sin(0)*cos(omega*t);
57 U1(N+1) = sin(L)*cos(omega*t);
58
59 % every INCR time steps, output the time step,
60 % residue and update the plot
61 if (mod(k,INCR)==0)
62
63 k
64 RES = max(abs(U1≠U0))/dt
65
66 hold off
67 plot(x,sin(x)*cos(omega*t),'≠k')
68 hold on
69 plot(x,U1,'≠≠r')
70 xlabel('x')
71 ylabel('u')
72 grid on
73 box on
74 axis tight
75 axis([0 L ≠1 1])
76 pause(.1)
77
78 end
79
80 % update solution
81 U0 = U1;
82
83 end
84
85 % compute error
86 Ue = sin(x)*cos(omega*t);
87 Err = max(abs(U1≠Ue))

14.1.2 Backwards Euler

Recall from section 12.2, that given an IVP

du
= f (u, t), (14.21)
dt
 274 14.1. The 1D Heat equation

the backward Euler method approximates the solution as

u n+1 = u n + tf (u n+1 , t n+1 ). (14.22)

Setting f (u n+1 , t n+1 ) to

@2 u n+1
f (u n+1 , t n+1 ) = ↵ + q(x, t n+1 ), (14.23)
@x2

and substituting into the backward Euler formula produces

ujn+1 ujn ujn+1

1 2ujn+1 + uj+1
n+1
=↵ + q(xj , t n+1 ). (14.24)
t x2

Isolating the unknowns on the left side produces

" #
1 2↵ n+1 ↵ n+1 ↵ n+1 ujn
+ u u uj+1 = + q(xj , t n+1 ). (14.25)
t x2 j x2 j 1 x 2 t

For brevity, suppose we apply the Dirichlet boundary conditions

u(0, t) = g0 (t), u(L, t) = gL (t). (14.26)

These can be assembled with relation 14.25 into the matrix system
2 32 3 2 n+1 3
661 0 0 0 0 0 0 077 66u1n+1 77 66 g0 77
666a 7 6 7 66 n+1 77
66 b c 0 0 0 0 0777 6666u2n+1 7777 66f 2 77
66 n+1 77
660 76 7
66 a b c 0 0 0 07777 666u3n+1 777 66f
66 3 777
7
66 77 666 n+1 777 6 n+1 77
660 0 a b c 0 0 077 66u4 77 6 f
66 77 66 n+1 77 = 6666 4n+1 7777 , (14.27)
660 0 0 a b c 0 0777 66u5 77 66f 5 77
66 77 666 n+1 777 66 n+1 77
660 0 0 0 a b c 077 66u6 77 66f 7
66
660 77 66 n+1 77 66 6 777
66 0 0 0 0 a b c 777 66u7 77 6
66f 7 777
n+1
4 5 64 n+1 75 64 n+1 75
0 0 0 0 0 0 0 1 u8 gL

where
1 2↵ ↵ ujn
b= + , a=c= , f jn+1 = + q(xj , t n+1 ).
t x2 x2 t

The following matlab code demonstrates the implementation of the backward Euler method
for the same test problem discussed in section 14.1.1. Note that the commands “A=sparse(A)”
and “A=decomposition(A)” take advantage of the sparse nature of the matrix to precompute
a sparse LU decomposition of the matrix. This speeds up the code considerably.
Figure 14.3 shows results produced by the matlab code when I C = 0, L = 2⇡, dt = 0.01,
and N = 8 (panel a) and N = 1024 (panel b). The black line shows the exact steady-solution
u = sin(x), while the dashed red lines show snapshots of the numerical approximations at
 Chapter 14. Numerical Solution of Partial Di↵erential Equations 275

(a) (b)
1 1

0.5 0.5

0 0

-0.5 -0.5

-1 -1
0 2 4 6 0 2 4 6

Figure 14.3: Results produced by the backwards Euler discretization of heat equation when ! = 0,
u 0 = 0, and N = 8 (panel a) and N = 1024 (panel b). The black lines show the exact steady state
solution u = sin(x). The red lines show snapshots of the numerical approximation.

several early times. Both simulations reach a steady state, without the appearance of numer-
ical instability. Though beyond the scope of this textbook, we can show that our implicit
discretization is “absolutely stable,” such that it will always converge to a steady-state no
matter how large the time step.
1 function[U1] = Main(alpha,N,L,Nt,dt,INCR,IC)
2
3 %≠≠≠≠≠ INPUTS ≠≠≠≠≠≠≠≠≠≠≠≠%
4 % alpha = thermal diffusivity
5 % N = resolution of finite difference grid
6 % L = domain length
7 % Nt = number of time steps
8 % dt = time step
9 % INCR: will plot result every INCR time steps
10 % IC = initial condition
11
12 % build finite difference grid.
13 % Note that i use the apostrophe to make the
14 % array a column
15
16 x=linspace(0,L,N+1)';
17 dx = L/N;
18
19 % Set initial condition
20 if (IC == 0)
21
22 omega = 0;
23 U0 = 0*x;
24 U1 = 0*x;
25
26 else
27
 276 14.1. The 1D Heat equation

28 omega = 2*pi;
29 U0 = sin(x);
30 U1 = 0*x;
31
32 end
33
34 % build matrix
35 A=zeros(N+1,N+1);
36 A(1,1)=1;
37 A(N+1,N+1)=1;
38
39 b = 1/dt + 2*alpha/dx^2;
40 a = ≠alpha/dx^2;
41 c = ≠alpha/dx^2;
42
43 for j=2:N
44 A(j,j) = b;
45 A(j,j≠1) = a;
46 A(j,j+1) = c;
47 end
48
49 % leverage the fact that A is sparse
50 A=sparse(A);
51
52 % precompute the LU decomposition of A
53 A=decomposition(A);
54
55 t = 0; % set intiial time
56 figure % open figure for plotting purposes
57
58 % start time loop
59 for k = 1:Nt
60
61 % compute time n+1
62 t = t + dt;
63
64 % buld right≠hand≠side vector
65 q =≠omega*sin(x)*sin(omega*t) + ...
66 alpha*sin(x)*cos(omega*t);
67
68 F = U0/dt + b;
69
70 F(1) = sin(0)*cos(omega*t);
71 F(N+1) = sin(L)*cos(omega*t);
72
73 % solve matrix problem for u^{n+1}
74 U1 = A\F;
75
 Chapter 14. Numerical Solution of Partial Di↵erential Equations 277

76
77 if (mod(k,INCR)==0)
78
79 k
80 RES = max(abs(U1≠U0))/dt
81
82 hold off
83 plot(x,sin(x)*cos(omega*t),'≠k')
84 hold on
85 plot(x,U1,'≠≠r')
86 xlabel('x')
87 ylabel('u')
88 grid on
89 box on
90 axis tight
91 axis([0 L ≠1 1])
92 pause(.1)
93
94 end
95
96 U0 = U1;
97
98
99 end
100
101 t
102 Ue = sin(x)*cos(omega*t);
103 Err = max(abs(U1≠Ue))/max(abs(Ue))

14.1.3 Verification with manufactured solutions

In the field of computational physics, the terms “verification” and “validation” typically re-
fer to two di↵erent types of analyses. Verification refers to analyses performed to show
that a numerical method correctly approximates the underlying equations with the expected
spatial-temporal accuracy. Validation refers to comparisons with other independent exper-
imental, theoretical, or numerical results to show that the numerical method reproduces
what is observed in nature or previous literature. For the scope of this introductory text-
book, we briefly discuss here the process of verification using the method of “manufactured
solutions.”
The method of manufactured solutions adds forcing terms to the governing equations
and boundary conditions so they satisfy a desired exact solution. Suppose for example, we
want the heat equation to satisfy

ue (x, t) = sin(x) cos(!t). (14.28)

 278 14.1. The 1D Heat equation

For that purpose, we substitute the desired solution in the heat equation,

@u @2 u
= ↵ 2 + q(x, t), (14.29)
@t @x

and solve for the required forcing term q(x, t)

@u @2 u
q(x, t) = ↵ = ! sin(x) sin(!t) + 2↵ sin(x) cos(!t). (14.30)
@t @x2

We must also substitute the desired solution in the boundary conditions

@u
a u(0, t) + b = g0 (t), (14.31)
@x x=0

@u
c u(L, t) + d = gL (t), (14.32)
@x x=L

to find the required forcing terms g0 (t) and gL (t),

g0 (t) = b cos(!t). (14.33)

h i
gL (t) = c sin(L) + d cos(L) cos(!t). (14.34)

The coefficients a, b, c, and d can then be set to test the code for Dirichlet (a = c = 1, b = d = 0),
Neumann (a = c = 0, b = d = 1), or Robin (a = c = b = d = 1) boundary conditions.
To test the spatial accuracy of the forward and backward Euler methods discussed in
sections 14.1.1 and 14.1.2, we set ! = 0 and the initial condition to

u(x, 0) = 0. (14.35)

We then integrate the numerical method in time until steady-state. In the matlab codes
shown earlier, we know when steady-state has been reached by monitoring the “residue,”
defined as
kun+1 un k1
Res = , (14.36)
t

where un+1 and un are vectors containing the numerical approximation at the N + 1 grid
points at two subsequent time steps. The symbol k k1 denotes the infinity-norm, which is
simply the maximum absolute value of a vector. The residue can be interpreted as a measure
of the maximum value of @u/@t over all grid points. Note that the steady-state numerical
solution is not sensitive to t. Any time step for which the methods are stable can be used.
After reaching steady-state, we can measure the spatial error by comparing the numerical
 Chapter 14. Numerical Solution of Partial Di↵erential Equations 279

(a) (b)
10-2
EN Et
2 t
1/N
-2
10
10-3

10-3 10-4

10 20 30 40 50 60 10-4 10-3 10-2

Figure 14.4: Spatial (panel a) and temporal (panel b) error plots for the forward Euler method when
L = 2⇡.

result with the exact steady solution ue = sin(x). For that purpose, we define the error ErrN
as
kuN ue k1
EN = , (14.37)
kue k1

where uN is a vector of the numerical approximation using N + 1 grid points, and ue is a

vector of the exact solution evaluated at the same grid points. The denominator is included
because it is good practice to normalize the error with respect to some measure of the mag-
nitude of exact solution. For the current example, this denominator can be omitted, because
k sin(x)k1 = 1. Figure 14.4(a) shows the resulting variation of EN with N for the forward
Euler method. Comparison with the line 1/N 2 confirms second-order spatial accuracy. The
corresponding error plot for the backward Euler method (not shown) is indistinguishable.
Verifying the temporal accuracy of our schemes is more involved. We begin by setting
! = 2⇡ and use the exact initial condition

u(x, 0) = sin(x). (14.38)

The idea is to then integrate from t = 0 to t = 1, after which we can compare with the exact
solution
ue (x, 1) = sin(x). (14.39)

Unfortunately, if we measure the error at time t = 1 as before,

kuN ue k1
EN = , (14.40)
kue k1

our measure of error will include contributions from both spatial and temporal sources.
With the backward Euler method, we can isolate the temporal error by setting N to a large
value, say N = 1024, so that the spatial error is small compared to the temporal error. That
approach, however, does not work the forward Euler method, because we cannot increase N
 280 14.1. The 1D Heat equation

independently of t. Instead, we suggest the following approach, which works equally well
for both the forward and backward Euler methods.
We begin by setting N to a fixed value. The matlab code below uses N = 32. We then
integrate from t = 0 to t = 1 using a very small time step. The matlab code below uses
t = 10 6 . We save the result at the final time as a reference solution uref . We then repeat
the simulation for a range of time steps larger than 10 6 , always using the same value of N as
the reference solution. The attached code uses the time steps t = 0.01, 0.005, 0.002, 0.001,
0.0005, 0.0002, and 0.0001. For each of these time steps, we define the temporal error as

kuN uref k1
E t = . (14.41)
kuref k1

The idea here is that because uref is computed with a very small time step, it is approxi-
mately equal to the exact answer plus the spatial error. When we subtract uref from uN ,
we are left with only the temporal error contained in uN . Figure 14.4(b) shows the resulting
variation of E t with t for the forward Euler method. Comparison with the line t confirms
first-order temporal accuracy. The corresponding error plot for the backward Euler method
(not shown) is indistinguishable.
1 function[]=TempErr()
2
3 % compute reference solution
4 N=32;
5 L=2*pi;
6 dt=1e≠6;
7 Nt = 1000000;
8 IC=1;
9 Uref = Main(N,L,Nt,dt,Nt/10,IC);
10
11 % repeat simulation for larger time steps
12 DT = [0.01 0.005 0.002 0.001 0.0005 0.0002 0.0001];
13 NT = [100 200 500 1000 2000 5000 10000];
14
15 for i=1:length(DT)
16
17 U=Main(N,L,NT(i),DT(i),NT(i)/10,IC);
18 ERR(i) = max(abs(U≠Uref));
19
20 end
21
22 figure
23 set(gca,'FontName','Times','FontSize',24)
24 hold on
25 plot(DT,ERR,'.r','MarkerSize',24)
26 plot(DT,DT,'≠≠k','LineWidth',2)
27 set(gca,'xscale','log')
28 set(gca,'yscale','log')
29 xlabel('$\Delta t$','interpreter','latex','FontSize',24)
 Chapter 14. Numerical Solution of Partial Di↵erential Equations 281

30 ylabel('$E_t$','interpreter','latex','FontSize',24,'rot',0)
31 legend('E_t','\Delta t')
32 grid on
33 box on
34 axis tight

14.1.4 Comparison of forward and backward Euler

In section 14.1.3, we saw that the forward and backward Euler methods give comparable
accuracy. The primary reasons for which one might choose one method over the other have
to do with their numerical stability and computational complexity. Each time step of the
explicit method requires a simple for-loop over each grid point. In contrast, each time step of
the implicit method requires building the right-hand-side vector and then solving a matrix
problem. As a result, each time step of the explicit method is typically much faster than
the implicit method. Though beyond the scope of this course, explicit methods also tend
to parallelize efficiently on computer clusters, while parallelizing implicit methods is often
more difficult.
The price to pay for the simplicity of explicit methods comes from their reduced stabil-
ity. Our implicit discretization of the heat equation is absolutely stable, which permits us
to use large time steps, and converge to steady solutions quickly. In contrast, the explicit
discretization becomes unstable beyond a maximum time step tmax ,

1
tmax ⇠ . (14.42)
N2

Consequently, if N is increased 10-fold, the maximum permissible time step decreases 100-
fold. As a result, one usually chooses an implicit method for cases where one wants to
use large steps, perhaps because u(x, t) varies slowly in time. On the other hand, if u(x, t)
varies rapidly in time, and requires small time steps anyways, then explicit methods become
attractive.

14.2 Poisson equation

Consider the two-dimensional Poisson equation

@2 u @2 u
+ = f (x, y) (14.43)
@x2 @y 2

in the domain 0  x  Lx , 0  y  Ly , subject to the Dirichlet conditions

u(0, y) = gw (y), u(0, Lx ) = ge (y), u(x, 0) = gs (x), u(0, Ly ) = gn (x). (14.44)

 282 14.2. Poisson equation

To solve this equation numerically, we begin by introducing a finite-di↵erence grid defined

by the points (xj , yi ), where

Lx
xj = (j 1) x, x= j = 1, 2, ... , Nx (14.45)
(Nx 1)

Ly
yi = (i 1) y, y= i = 1, 2, ... , Ny . (14.46)
(Ny 1)

Figure 14.5(a) shows the grid when Nx = Ny = 5. On the boundary grid points labelled red,
we apply the Dirichlet boundary conditions (14.44). In some cases, there is some ambiguity
as to what to apply on the four corners of the domain. For example, on the corner (x, y) =
(0, 0), should we apply u(0, 0) = gw (0) or u(0, 0) = gs (0) ? In the current textbook, we assume
the problem is well posed, with consistent boundary conditions, such that gw (0) and gs (0) are
equal. In practice, however, that is not always true. Such situations need to be considered
case-by-case to decide which choice is more physically or numerically appropriate.

On the internal grid points labelled with blue dots in figure 14.5(a) we approximate the
Laplacian operator using second-order accurate finite di↵erence formulas. Consider the grid
point labelled P in 14.5(a). At that point, we can approximate the PDE as

uW 2uP + uE uS 2uP + uN
+ = f (xP , yP ). (14.47)
x2 y2

Together with the boundary conditions applied on the red grid points, these provide Nx ⇥ Ny
equations for Nx ⇥ Ny unknowns (if we treat the boundary grid points as unknowns). To
solve the system of equations, we assemble them in a matrix problem of the form

Au = f, (14.48)

where u is a vector of the Nx ⇥ Ny unknown u values.

To assemble the matrix problem (14.48), we need to choose how to order the unknowns
within the vector u. For that purpose, suppose we denote the unknown u values as uij ,
where the index i denotes the y-position on the grid, and the index j denotes the x-position.
Figure 14.5(b) shows the resulting labeling of unknowns uij for Nx = Ny = 5. We then place
these unknowns in the column vector u starting with u11 , and working rightward along the
rows of the grid, as sketched in figure 14.5(c). Using this approach, the first Nx rows of
the matrix A correspond to grid points on the lower boundary. We consequently apply the
boundary conditions by inserting 1 on the main diagonal of the matrix, and inserting the
desired boundary value on the corresponding row of f, as sketched in figure 14.5(c).

On rows corresponding to internal grid points, we apply the finite di↵erence relation
(14.47), which we rewrite in the form
!
1 1 1 1 2 2
2
uW + 2
uE + 2
uS + uN + uP = f P . (14.49)
x x y y2 y2 y2
 Chapter 14. Numerical Solution of Partial Di↵erential Equations 283

(a) (b)
u51 u52 u53 u54 u55

u41 u42 u43 u44 u45

Δx

N
u31 u32 u33 u34 u35
Δy
W P E
S
u21 u22 u23 u24 u25
S
u11 u12 u13 u14 u15

(c) A u f

{
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 u11 ug11
first row
0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 u12 gu12
of grid 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 u13 gu13
0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 u14 gu14
0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 u15 gu15

{
0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 u21 g21
nd
2 row 0 S 0 0 0 WP E 0 0 0 N 0 0 0 0 0 0 0 0 u22 f 22
of grid 0 0 S 0 0 0 W P E 0 0 0 N 0 0 0 0 0 0 0 u23 = f 23
0 0 0 S 0 0 0 WP E 0 0 0 N 0 0 0 0 0 0 u24 f 24
0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 u25 g25

{
0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 u31 gu31
rd
3 row 0 0 0 0 0 0 S 0 0 0 WP E 0 0 0 N 0 0 0 u32 uf 32
of grid 0 0 0 0 0 0 0 S 0 0 0 W P E 0 0 0 N 0 0 u33 uf 33
0 0 0 0 0 0 0 0 S 0 0 0 W P E 0 0 0 N 0 u34 uf 34
0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 u35 gu35

Figure 14.5

This produces the rows shown in blue of figure 14.5(c), where W = E = 1/ x2 , S = N = 1/ y 2 ,

and P = ( 2/ x2 2/ y 2 ). The following matlab code solves the 2D Poisson equation when

f = 2 sin(x) sin(y), (14.50)

and compares the result with the exact solution

u = sin(x) sin(y). (14.51)

1 function[] = Main(Lx,Ly,Nx,Ny)
2
 284 14.2. Poisson equation

3 % compute grid points in x and y directions

4 x=linspace(0,Lx,Nx+1)';
5 y=linspace(0,Ly,Ny+1)';
6
7 % compute exact solution
8 [X,Y] = meshgrid(x,y);
9 Ue = sin(X).*sin(Y);
10
11 % initialize a matrix of all zeros
12 NT=(Nx+1)*(Ny+1); % total number of unknowns
13 A=zeros(NT,NT);
14
15 % apply lower/upper BC
16 for j=1:Nx+1
17
18 A(j,j) = 1; % lower BC
19
20 A(Ny*(Nx+1) + j,Ny*(Nx+1) + j) = 1; % upper BC
21 end
22
23 % apply left/right BC
24 for i=2:Ny
25
26 row = (i≠1)*(Nx+1) + 1
27 A(row,row) = 1; % left BC
28
29 row = (i)*(Nx+1)
30 A(row,row) = 1; % right BC
31
32 end
33
34 % build rows corresponding to internal grid points
35 dx = Lx/Nx;
36 dy = Ly/Ny;
37
38 P = ≠2/dx^2≠2/dy^2;
39 E = 1/dx^2;
40 W = E;
41 S = 1/dy^2;
42 N = S;
43
44 for i = 2:Ny
45 for j = 2:Nx
46
47 row = (i≠1)*(Nx+1)+j;
48 A(row,row) = P;
49 A(row,row≠1) = W;
50 A(row,row+1) = E;
 Chapter 14. Numerical Solution of Partial Di↵erential Equations 285

51 A(row,row≠(Nx+1)) = S;
52 A(row,row+(Nx+1)) = N;
53
54 end
55 end
56
57 % build right≠hand≠side vector
58 F=zeros(NT,1);
59
60 % apply lower/upper BC
61 for j=1:Nx+1
62
63 F(j) = sin(x(j))*sin(0); % lower BC
64
65 F(Ny*(Nx+1) + j) = sin(x(j))*sin(Ly); % upper BC
66
67 end
68
69 % apply left/right BC
70 for i=2:Ny
71
72 row = (i≠1)*(Nx+1) + 1;
73 F(row) = sin(0)*sin(y(i)); % left BC
74
75 row = (i)*(Nx+1);
76 F(row) = sin(Lx)*sin(y(i)); % right BC
77
78 end
79
80 % interior grid points
81 for i = 2:Ny
82 for j = 2:Nx
83
84 row = (i≠1)*(Nx+1)+j;
85 F(row) = ≠2*sin(x(j))*sin(y(i));
86
87 end
88 end
89
90 % solve matrix problem and unpack
91 % solution in a matrix of size Ny+1 by Nx+1
92 TMP = A\F;
93
94 Un=zeros(Ny+1,Nx+1);
95
96 for i = 1:Ny+1
97 for j = 1:Nx+1
98
 286 14.2. Poisson equation

99 row = (i≠1)*(Nx+1)+j;
100 Un(i,j) = TMP(row);
101
102 end
103 end
104
105 % plot exact solution, numerical solution, and error
106
107 figure
108
109 subplot(1,3,1)
110 surf(X,Y,Ue)
111 shading interp
112 xlabel('x')
113 ylabel('y')
114 zlabel('Ue')
115
116 subplot(1,3,2)
117 surf(X,Y,Un)
118 shading interp
119 xlabel('x')
120 ylabel('y')
121 zlabel('Un')
122
123 subplot(1,3,3)
124 surf(X,Y,abs(Ue≠Un))
125 shading interp
126 xlabel('x')
127 ylabel('y')
128 zlabel('Err')
129
130 Err = max(max(abs(Ue≠Un)))
Bibliography

p
[1] P. J. Nahin. An Imaginary Tale: The Story of 1. Princeton University Press, 1998.
[2] M. Greenberg. Advanced Engineering Mathematics, 2’nd Edition. Pearson, 1998.
[3] Jesper Lützen. “Sturm and Liouville’s Work on Ordinary Linear Di↵erential Equa-
tions. The Emergence of Sturm-Liouville Theory”. In: Archive for History of Exact Sci-
ences 29.4 (1984), pp. 309–376. (Visited on 04/11/2022).
[4] Walter Gautschi. “A Survey of Gauss-Christo↵el Quadrature Formulae”. In: E. B.
Christo↵el: The Influence of His Work on Mathematics and the Physical Sciences. Ed. by
P. L. Butzer and F. Fehér. Basel: Birkhäuser Basel, 1981, pp. 72–147.

287
Index

antiderivative, 68 homogeneous matrix problems, 139

hyperbolic functions, 108
backward Euler, 248
boundary value problem, 62 identity matrix, 135
igaminary units, 106
complex conjugation, 108 indefinite integral, 68
complex number, 106 initial value problem, 62
complex plane, 106 integrating factor, 76
conservative force fields, 39 integration by substitution, 68
constant solution, 90
Crank-Nicolson, 248 leap frog, 248
left hand rule, 238
definite integral, 68 Linear Dependence, 108
di↵erential operator, 57 linear dependence, 108

eigenvalue, 141 midpoint rule, 240

eigenvalue problems, 141
eigenvector, 141 non-homogeneous di↵erential equation, 61
equilibrium solution, 90 ordinary di↵erential equation, 55
Euler’s formula, 107
exact ODEs, 96 partial di↵erential equation, 55
exponential time constant, 89 particular solution, 67

forcing terms, 61 rate equation, 83

forward Euler method, 243 rate of reaction, 83
reaction rate, 83
Gaussian elimination, 134 right hand rule, 238
general solution, 67 row operations, 134
Runge-Kutta methods, 249
half-life, 89
homogeneous di↵erential equation, 61 scalar, 27

288
 Index 289

scalar potential, 40
Separable ODEs, 91
Simpson’s rule, 240
singular matrix, 139

trapezoidal rule, 240

undetermined coefficients, 117

variation of parameters, 117