i

Queens University
Mathematics and Engineering and Mathematics and Statistics

MTHE / MATH 237

Differential Equations for Engineering Science

Supplemental Course Notes

Serdar Yuksel

November 26, 2015

ii

This document is a collection of supplemental lecture notes used for MTHE / MATH 237: Differential Equations for
Engineering Science.
Serdar Yuksel

Contents

Introduction to Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1.2 Classification of Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1.2.1

Ordinary Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1.2.2

Partial Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1.2.3

Homogeneous Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1.2.4

N-th order Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1.2.5

Linear Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1.3 Solutions of Differential equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1.4 Direction Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1.5 Fundamental Questions on First-Order Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

First-Order Ordinary Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2.2 Exact Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2.3 Method of Integrating Factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2.4 Separable Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2.5 Differential Equations with Homogenous Coefficients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2.6 First-Order Linear Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2.7 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Higher-Order Ordinary Linear Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3.2 Higher-Order Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3.2.1

Linear Independence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

3.2.2

Wronskian of a set of Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

iv

Contents

3.2.3
4

Non-Homogeneous Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Higher-Order Ordinary Linear Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
4.2 Homogeneous Linear Equations with Constant Coefficients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
4.2.1

L(m) with distinct Real roots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

4.2.2

L(m) with repeated Real roots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

4.2.3

L(m) with Complex roots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

4.3 Non-Homogeneous Equations and the Principle of Superposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

4.3.1

Method of Undetermined Coefficients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

4.3.2

Variation of Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

4.3.3

Reduction of Order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4.4 Applications of Second-Order Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

4.4.1
5

Resonance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Systems of First-Order Linear Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
5.1.1

Higher-Order Linear Differential Equations can be reduced to First-Order Systems . . . . . . . . . . . . . . 22

5.2 Theory of First-Order Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

6

Systems of First-Order Constant Coefficient Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
6.2 Homogeneous Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
6.3 How to Compute the Matrix Exponential . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
6.4 Non-Homogeneous Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
6.5 Remarks on the Time-Varying Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

Stability and Lyapunovs Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
7.2 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
7.2.1

Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

7.2.2

Non-Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

7.3 Lyapunovs Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

8

Laplace Transform Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Contents

8.2 Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

8.2.1

Laplace Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

8.2.2

Properties of the Laplace Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

8.2.3

Laplace Transform Method for solving Initial Value Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

8.2.4

Inverse Laplace Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

8.2.5

Convolution: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

8.2.6

Step Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

8.2.7

Impulse response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

On Partial Differential Equations and Separation of Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

10 Series Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
10.2 Taylor Series Expansions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
11 Numerical Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
11.2 Numerical Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
11.2.1 Eulers Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
11.2.2 Improved Eulers Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
11.2.3 Including the Second Order Term in Taylors Expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
11.2.4 Other Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
A

Brief Review of Complex Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

A.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
A.2 Eulers Formula and Exponential Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

Similarity Transformation and the Jordan Canonical Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

B.1 Similarity Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
B.2 Diagonalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
B.3 Jordan Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

1
Introduction to Differential Equations

1.1 Introduction
Many phenomena in engineering, physics and broad areas of applied mathematics involve entities which change as a
function of one or more variables. The movement of a car along a road, the propagation of sound and waves, the path an
airplane takes, the amount of charge in a capacitor in an electrical circuit, the way a cake taken out from a hot oven and left
in room temperature cools down and many such processes can all be modeled and described as differential equations.
In class we discussed a number of examples such as the swinging pendulum, a falling object with drag and an electrical
circuit involving resistors and capacitors.
Let us make a definition.
Definition 1.1.1 A differential equation is an equation that relates an unknown function and one or more of its derivatives
of with respect to one or more independent variables.
For instance, the equation
dy
= 5x
dx
relates the first derivative of y with respect to x, with x. Here x is the independent variable and y is the unknown function
(dependent variable).
In this course, we will learn how to solve and analyze properties of the solutions to such differential equations. We will
make a precise definition of what solving a differential equation means shortly.
n

d y
Before proceeding further, we make a remark on notation. Recall that dx
n is the n-th derivative of y with respect to x. One
n
d y
(n)
can also use the notation y
to denote dxn . It is further convenient to write y = y (1) and y = y (2) . In physics, the
notation involving dots is also common, such that y denotes the first-order derivative.

Different classifications of differential equations are possible and such classifications make the analysis of the equations
more systematic.

1.2 Classification of Differential Equations

There are various classifications for differential equations. Such classifications provide remarkably simple ways of finding
the solutions (if they exist) for a differential equation.
Differential equations can be of the following classes.

1 Introduction to Differential Equations

1.2.1 Ordinary Differential Equations

If the unknown function depends only on a single independent variable, such a differential equation is ordinary.
The following is an ordinary differential equation:
L

d
1
d2
Q(t) + R 2 Q(t) + Q(t) = E(t),
2
dt
dt
C

which is an equation which arises in electrical circuits. Here, the independent variable is t.

1.2.2 Partial Differential Equations

If the unknown function depends on more than one independent variables, such a differential equation is said to be partial.
Heat equation is an example for partial differential equations:
2

2
f (x, t) = f (x, t)
x2
t

Here x, t are independent variables.

1.2.3 Homogeneous Differential Equations

If a differential equation involves terms all of which contain the unknown function itself, or the derivatives of the unknown
function, such an equation is homogeneous. Otherwise, it is non-homogeneous.

1.2.4 N-th order Differential Equations

The order of an ordinary differential equation is the order of the highest derivative that appears in the equation.
For example
2y (2) + 3y (1) = 5,
is a second-order equation.

1.2.5 Linear Differential Equations

A very important class of differential equations are linear differential equations. A differential equation
F (y, y (1) , y (2) , . . . , y (n) )(x) = g(x)
is said to be linear if F is a linear function of the variables y, y (1) , y (2) , . . . , y (n) .
Hence, by the properties of linearity, any linear differential equation can be written as follows:
n
X

ani (x)y (i) (x) = g(x)

i=0

for some a0 (x), a1 (x), . . . , an (x) which dont depend on y.

For example,

1.4 Direction Fields

ay + by (1) + cy = 0
is a linear differential equation, whereas
yy (1) + 5y = 0,
is not linear.

If a differential equation is not linear, it is said to be non-linear.

1.3 Solutions of Differential equations

Definition 1.3.1 To say that y = g(x) is an explicit solution of a differential equation
F (x, y,
on an interval I R, means that

F (x, g(x),

dy dn y
)=0
,
dx dxn

dg(x)
dn g(x)
) = 0,
,...,
dx
dxn

for every choice of x in the interval I.

Definition 1.3.2 We say that the relation G(x, y) = 0 is an implicit solution of a differential equation
F (x, y,

dy dn y
)=0
,
dx dxn

if for all y = g(x) such that G(x, g(x)) = 0, g(x) is an explicit solution to the differential equation on I.
Definition 1.3.3 An n-th parameter family of functions defined on some interval I by the relation
h(x, y, c1 , . . . , cn ) = 0,
is called a general solution of the differential equation if any explicit solution is a member of the family. Each element of
the general solution is a particular solution.
Definition 1.3.4 A particular solution is imposed by supplementary conditions that accompany the differential equations.
If all supplementary conditions relate to a single point, then the condition is called an initial condition. If the conditions
are to be satisfied by two or more points, they are called boundary conditions.
Recall that in class we used the falling object example to see that without a characterization of the initial condition (initial
velocity of the falling object), there exist infinitely many solutions. Hence, the initial condition leads to a particular solution,
whereas the absence of an initial condition leads to a general solution.
Definition 1.3.5 A differential equation together with an initial condition (boundary conditions) is called an initial value
problem (boundary value problem).

1.4 Direction Fields

The solutions to first-order differential equations can be represented graphically, without explicitly solving the equation.
This also provides further understanding to the solutions to differential equations.

1 Introduction to Differential Equations

In particular, consider a point P = (x0 , y0 ) R2 and the equation

dy
= f (x, y).
dx
dt
= f (x0 , y0 ). What this means is that the rate of change in y with respect to x at this point is given
It follows that at P , dx
by f (x0 , y0 ). Hence, a line segment showing the change is a line with slope f (x0 , y0 ).

The differential equation above tells us what the slope in the graph, that is the ratio of a small change in y with respect to a
small change in x, is. As such, one can draw a graph to obtain the direction field.
Upon a review of preliminary topics on differential equations, we proceed to study first-order ordinary differential equations.

1.5 Fundamental Questions on First-Order Differential Equations

Given an equation, one may ask the following questions: Does there exist a solution? How many solutions do exist? And
finally, if the answer of the existence question is affirmative, how do we find the solution(s)?
Here is an important theorem for first-order differential equations. We refer to this as the existence and uniqueness theorem.

Theorem 1.5.1 Suppose that the real-valued function f (x, y) is defined on a rectangle U = [a, b] [c, d], and suppose

f (x, y) and y
f (x, y) are continuous on U . Suppose further that (x0 , y0 ) is an interior point of U . Then there is an open
subinterval (a1 , b1 ) [a, b], x0 (a1 , b1 ) such that there is exactly one solution to the differential equation
that is defined on (a1 , b1 ) and passes trough the point (x0 , y0 ).

dy
dx

= f (x, y)

The above theorem is important as it tells us that under certain conditions, there can only exist one solution. For example
dy
dy
1/3
might not
dx = sin(y) has a unique solution that passes through a point in the (x, y) plane. On the other hand dx = y
1/3
2/3
have a unique solution in the neighborhood around y = 0, since y y
= (1/3)y
is not continuous when y = 0.
In class, we will have some brief discussion on the existence and uniqueness theorem via the method of successive approximations (which is also known as Piccards method). A curious student is encouraged to think a bit more about the
theorem, on why the conditions are needed.

2
First-Order Ordinary Differential Equations

2.1 Introduction
In this chapter, we consider first-order differential equations. Such a differential equation has the form:
y (1) = f (x, y).
We start the analysis with an important sub-class of first order differential equations. These are exact differential equations.

2.2 Exact Differential Equations

Definition 2.2.1 Let F (x, y) be any function of two-variables such that F has continuous first partial derivatives in a
rectangular region U R2 . The total differential of F is defined by
dF (x, y) =

F (x, y)dx +
F (x, y)dy
x
y

Definition 2.2.2 A differential equation of the form

M (x, y)dx + N (x, y)dy = 0
is called exact if there exists a function F (x, y) such that
M (x, y) =

F (x, y)
x

N (x, y) =

F (x, y)
y

and

In this case,
M (x, y)dx + N (x, y)dy = 0,
is called an exact differential equation.
We have the following theorem, to be proven in class:
Theorem 2.2.1 If the functions M (x, y), N (x, y) and their partial derivatives

2
y N (x, y) are continuous on a domain U R , then the differential equation

y M (x, y), x M (x, y)

and

x N (x, y),

2 First-Order Ordinary Differential Equations

M (x, y)dx + N (x, y)dy = 0,

is exact if and only if

M (x, y) =
N (x, y),
y
x

x, y U )

If the equation is exact, the general implicit solutions are given by

F (x, y) = c,
where F has partial derivatives as M (x, y) and N (x, y) on U .
Exercise 2.2.1 Show that
ydx + xdy = 0,
is an exact differential equation and find the solution passing through the point (1, 3) for x, y > 0.

2.3 Method of Integrating Factors

Sometimes an equation to be solved is not exact, but can be made exact. That is, if
M (x, y)dx + N (x, y)dy = 0
is not exact, it might be possible to find (x, y) such that
(x, y)(M (x, y)dx + N (x, y)dy) = (x, y)M (x, y)dx + (x, y)N (x, y)dy = 0,
is exact. The function (x, y) is called an integrating factor. In this case, if (x, y) is non-zero, then, the solution to the
new equation becomes identical to the solution of the original, non-exact equation.
Now, let us see what the integrating factor should satisfy for exactness. It must be that,





(x, y)M (x, y) =

(x, y)N (x, y)
y
x
and hence,
(

(x, y))M (x, y) + (x, y)( M (x, y)) = ( (x, y))N (x, y) + (x, y)( N (x, y))
y
y
x
x

It is not easy to find such a (x, y) in general. But, if in the above, we further observe that (x, y) is only a function of x,
that is can be written as (x), then we have:
(x)
This can be written as
(

M (x, y) =
(x))N (x, y) + (x) N (x, y)
y
dx
x

d
(x))N (x, y) = (x)( N (x, y)
M (x, y))
dx
x
y

and

( x N (x, y) y M (x, y))

d
(x)) = (x)
dx
N (x, y)

If
P (x) =

( y
M (x, y)

x N (x, y))

N (x, y)

(2.1)

2.5 Differential Equations with Homogenous Coefficients

is only a function of x, then we can solve the equation (2.1) as a first-order, ordinary differential equation as
(

d
(x)) = (x)P (x),
dx

for which the solution is:

(x) = e

Rx

P (u)du

(2.2)

K,

for some constant K.

A brief word of caution follows here. The integrating factor should not be equal to zero, that is (x, y) 6= 0 along the path
of the solution. In such a case, the solution obtained might be different, and one should check the original equation given
the solution obtained by the method of integrating factors.
Exercise 2.3.1 Consider the following differential equation
dy
+ T (x)y = R(x),
dx
Find an integrating factor for the differential equation.
See Section 2.6 below for an answer.

2.4 Separable Differential Equations

A differential equation of the form
F (x)G(y)dx + f (x)g(y)dy = 0,
is called a separable differential equation.
For such DEs, there is a natural integrating factor: (x, y) =

1
G(y)f (x) ,

the substitution of which leads to

F (x)
g(y)
dx +
dy = 0,
f (x)
G(y)
which is an exact equation. An implicit solution can be obtained by:
Z
Z
g(y)
F (x)
dx +
dy = c,
f (x)
G(y)
One again needs to be cautious with the integrating factor. In particular, if G(y0 ) = 0, then it is possible to lose the solution
y(x) = y0 for all x values. Likewise, if there is an x0 such that f (x0 ) = 0, then, x(y) = x0 for all y is also a solution.
Remark 2.1. If the initial condition is y0 such that G(y0 ) = 0, then y(x) = y0 is a solution.

2.5 Differential Equations with Homogenous Coefficients

-This definition should not be confused with the definition of homogeneous equations discussed earlier.A differential equation of the form
dy
= f (x, y),
dx
is said to have homogenous coefficients if one can write

2 First-Order Ordinary Differential Equations

y
f (x, y) = g( ),
x
for some function g(.).
For differential equations with homogenous coefficients, the transformation y = vx reduces the problem to a separable
equation in x and v. Note that, the essential step is that
dy
d(vx)
dv
=
=v+x .
dx
dx
dx
Alternatively,
M (x, y)dx + N (x, y)dy = 0,
has homogenous coefficients if
M (x, y)
M (x, y)
=
,
N (x, y)
N (x, y)

R, 6= 0

2.6 First-Order Linear Differential Equations

Consider a linear differential equation:
dy
+ P (x)y = Q(x),
dx
We can express this as
(Q(x) P (x)y)dx dy = 0
This equation is not exact. Hence, following the discussion in Section 2 above, we wish to obtain an integrating factor. It
should be observed that

y ((Q(x) P (x)y)) x (1)

= P (x),
1
is only a function of x and hence we can use the integrating factor
(x) = e
to obtain an exact equation:
e

P (x)dx

P (x)dx

(Q(x) P (x)y)dx e

P (x)dx

dy = 0

Exercise 2.6.1 Consider the following differential equation

dy
+ y = et ,
dt
Find an integrating factor for the differential equation. Obtain the general solution.

2.7 Applications
In class, applications in numerous of areas of engineering and applied mathematics will be discussed.

3
Higher-Order Ordinary Linear Differential Equations

3.1 Introduction
In this chapter, we consider differential equations with arbitrary finite orders.

3.2 Higher-Order Differential Equations

In class we discussed the Existence and Uniqueness Theorem for higher-order differential equations. In this chapter, we
will restrict the analysis to such linear equations.
Definition 3.2.1 An nth order linear differential equation in a dependent variable y and independent variable x defined
on an interval I R has the form
an (x)y (n) + an1 (x)y (n1) + + a1 (x)y (1) + a0 (x)y = g(x).

(3.1)

If g(x) = 0 for all x in I, then the differential equation is homogeneous, otherwise, it is non-homogeneous.
For notational convenience, we write the left hand side of (3.1) via:
P(D)(y)(x) = an (x)D(n) (y)(x) + an1 (x)D(n1) (y)(x) + + a1 (x)D(1) (y)(x) + a0 (x)D(0) (y)(x),
which we call as the polynomial operator of order n.
The following is the fundamental theorem of existence and uniqueness applied to such linear differential equations:
Theorem 3.2.1 If a0 (x), a1 (x), . . . , an (x) and g(x) are continuous and real-valued functions of x on an interval I R
and an (x) 6= 0, x I, then the differential equation has a unique solution y(x) which satisfies the initial condition
y(x0 ) = y0 , y (1) (x0 ) = y1 , . . . , y (n1) (x0 ) = yn1 ,
where x0 I, y0 , y1 , . . . , yn1 R.
The following is an immediate consequence of the fundamental theorem:
Theorem 3.2.2 Consider a linear differential equation defined on I:
P(D)(y(x)) = 0,

10

3 Higher-Order Ordinary Linear Differential Equations

such that an (x) 6= 0,

x I and a0 (x), a1 (x), . . . , an (x) and g(x) are continuous and real-valued functions of x with
y(x0 ) = 0, y (1) (x0 ) = 0, . . . , y (n1) (x0 ) = 0,

I.

Then,
x I

y(x) = 0,
Exercise 3.2.1 Let f1 , f2 , . . . , fn be solutions to

P(D)(y(x)) = 0.
Then,
c1 f1 (x) + c2 f2 (x) + + cn fn (x) = 0,
is also a solution, for all c1 , c2 . . . , cn R.
3.2.1 Linear Independence
Linear independence/dependence is an important concept which arises in differential equations, as well as in linear algebra.
Definition 3.2.2 The set of functions {f1 (x), f2 (x), f3 (x), . . . , fn (x)} are linearly independent on an interval I if there
exist constants, c1 , c2 , . . . , cn ; not all zero, such that
c1 f1 (x) + c2 f2 (x) + + cn fn (x) = 0,

x I.

If this condition implies that

c1 = c2 = = cn = 0,
the functions f1 (x), f2 (x), f3 (x), . . . , fn (x) are said to be linearly independent.
Theorem 3.2.3 Consider the differential equation
P(D)(y(x)) = 0,
defined on I. There exist n linearly independent solutions to this differential equation. Furthermore, every solution to the
differential equation (for a given initial condition) can be expressed as a linear combination of these solutions.
Exercise 3.2.2 Consider y (2) + y = 0 with y(0) = 5, y (0) = 1. Verify that y1 (x) = sin(x) and y2 (x) = cos(x) both
solve y (2) + y = 0. Hence, the general solution is c1 sin(x) + c2 cos(x). Find c1 and c2 .

3.2.2 Wronskian of a set of Solutions

An important issue is with regard to the Wronskian, which will help us understand the issues on linear dependence.
Definition 3.2.3 Let f1 , f2 , . . . , fn be n linearly functions with (n 1) derivatives on an interval I. The determinant

W (f1 , f2 , . . . , fn ) = det

f1
(1)
f1
...
...
(n1)

f1

is called the Wronskian of these functions.

f2
. . . fn
(1)
(1)
. . . , fn
f2
... ... ...

... ... ...

(n1)
(n1)
. . . , fn
f2

3.2 Higher-Order Differential Equations

11

Theorem 3.2.4 Let f1 , f2 , . . . , fn be n 1st order differentiable functions on an interval I. If these functions are linearly
dependent, then the Wronskian is identically zero on I.
The proof of this is left as an exercise. The student is encouraged to use the definition of linear independence to show that
the Wronskian is zero.
The main use of Wronskian is given by the following, which allows us to present an even stronger result. It says that if the
Wronskian is zero at any given point, then the set of solutions are linearly dependent.
Theorem 3.2.5 Let y 1 , y 2 , . . . , y n be n solutions to the linear equation.
P(D)(y(x)) = 0.
They are linearly dependent if and only if the Wronskian is zero on I. The Wronskian of n solutions to a linear equation is
either identically zero, or is never 0.
The proof of this for a second-order system will be presented as an exercise.

3.2.3 Non-Homogeneous Problem

Consider the differential equation
P(D)(y(x)) = g(x).
Suppose yp1 and yp2 are two solutions to this equation. Then, it must be that, by linearity,
P(D)(yp1 (x) yp2 (x)) = g(x) g(x) = 0
Hence, yp1 (x) yp2 (x) solves the homogenous equation:
P(D)(y(x)) = 0.
Hence, any solution to a particular equation can be obtained by obtaining one particular solution to the non-homogenous
equation, and then adding the solutions to the homogenous equation, and finally obtaining the coefficients for the
homogenous-part of the solution.
We will discuss non-homogeneous equations further while studying methods for solving higher-order linear differential
equations.

4
Higher-Order Ordinary Linear Differential Equations

4.1 Introduction
In this chapter, we will develop methods for solving higher-order linear differential equations.

4.2 Homogeneous Linear Equations with Constant Coefficients

Consider the following differential equations:
an y (n) + an1 y (n1) + + a1 y (1) + a0 y = 0.
Here {an , an1 , . . . , a1 , a0 } are R-valued constants
For such equations, we look for solutions of the form
y(x) = emx .
The substitution of this solution into the equation leads to
(an mn + an1 mn1 + + a1 m + a0 )emx = 0
We call L(m) = an mn + an1 mn1 + + a1 m + a0 , the characteristic polynomial of the equation.
Recall that an nth order linear differential equation has n linearly independent solutions. Furthermore, every particular
solution to a given initial value problem can be expressed as a linear combination of these n solutions.
As such, our goal is to obtain n linearly independent solutions which satisfy a given differential equation.
With this reminder, we now proceed to the analysis for L(m) = 0. There are three possibilities.

4.2.1 L(m) with distinct Real roots

In case L(m) = 0 has n distinct roots, m1 , m2 , . . . , mn are different and each of
y(x) = emi x ,

i = 1, 2, . . . , n

is a solution. In this case,

y(x) = c1 em1 x + c2 em2 x + + cn emn x
is the general solution.

14

4 Higher-Order Ordinary Linear Differential Equations

4.2.2 L(m) with repeated Real roots

Suppose m1 is a root with multiplicity k. In this case, each of
xi em1 x
is a solution for i = 1, 2 . . . , k 1. Note that these are linearly independent. You are encouraged to prove linear independence of these by using the definition of linear independence.

4.2.3 L(m) with Complex roots

With complex roots, the analysis is identical to Real roots.
In particular, emx is a solution. If m = a + ib, then e(a+ib)x is a solution.
As such,
e(a+ib)x = eax (cos(bx) + i sin(bx))
is a solution.
Since L(m) = 0 only has Real coefficients in the polynomial values, it follows that if m is a complex valued root, then the
complex conjugate of m is also a solution. Hence,
e(aib)x = eax (cos(bx) + i sin(bx))
is also a solution.
As such a linear combination of e(a+ib)x and e(aib)x is also a solution. Instead of working with c1 eax (cos(bx) +
i sin(bx)) + c2 eax (cos(bx) i sin(bx)), we could have
c1 eax cos(bx) + c2 eax sin(bx)).
Here, in general c1 and c2 can be complex valued, but for the initial value problems with Real valued initial values, it
suffices to search for c1 , c2 to be Real valued.
In case m1 = a + ib and m2 = a ib are repeated complex valued roots k times, then, each of the following are solutions:
eax cos(bx), eax sin(bx), xeax cos(bx), xeax sin(bx), . . . , xk1 eax cos(bx), xk1 eax sin(bx)

4.3 Non-Homogeneous Equations and the Principle of Superposition

Consider the differential equation
P(D)(y(x)) = g(x).
Suppose yp1 and yp2 are two solutions to this equation. Then, it must be that, by linearity,
P(D)(yp1 (x) yp2 (x)) = g(x) g(x) = 0
Hence, yp1 (x) yp2 (x) solves the homogenous equation:
P(D)(y(x)) = 0.

4.3 Non-Homogeneous Equations and the Principle of Superposition

15

Hence, any solution to a particular equation can be obtained by obtaining one particular solution to the non-homogenous
equation, and then adding the solutions to the homogenous equation, and finally obtaining the coefficients for the
homogenous-part of the solution.
Theorem 4.3.1 Let yp be a particular solution to a differential equation
P(D)(y(x)) = g(x),

(4.1)

and let yc = c1 y1 (x) + c2 y2 (x) + + cn yn (x), be the general solution corresponding to the homogeneous equation
P(D)(y(x)) = 0
Then, the general solution of (4.2) is given by
y(x) = yp (x) + yc (x)
Here yc (x) is called the complementary solution.
We now present a more general result. This is called the principle of superposition.
Theorem 4.3.2 Let ypi (x) be respectively particular solutions of
P(D)(y(x)) = gi (x),

(4.2)

for i = 1, 2 . . . , m. Then,
a1 yp1 (x) + a2 yp2 (x) + + am ypm (x)
is a particular solution to the DE
P(D)(y(x)) = a1 g1 (x) + a2 g2 (x) + + am gm (x)
In the following, we discuss how to obtain particular solutions. We first use the method of undetermined coefficients and
then the method of variation of parameters.
The method of undetermined coefficients has limited use, but when it works, is very effective. The method of variation of
parameters is more general, but requires more steps in obtaining solutions.

4.3.1 Method of Undetermined Coefficients

The derivative of a polynomial is a polynomial. As such, the polynomial operator applied to a polynomial function leads
to a polynomial term. Furthermore, the order of the polynomial which results from being subjected to the polynomial
operator, cannot be larger than the polynomial itself.
A similar observation applies to the exponentials. The derivative of an exponential is another exponential.
These observation leads us to the following result.
Theorem 4.3.3 Consider a linear, non-homogeneous, constant-coefficient ordinary differential equation
P(D)(y(x)) = f (x)
Let f (x) be of the form
f (x) = eax cos(bx)(po + p1 x + p2 x2 + + pm xm ) + eax sin(bx)(qo + q1 x + q2 x2 + + qm xm ),

16

4 Higher-Order Ordinary Linear Differential Equations

for some constants a, b, p0 , p1 , . . . , pm , q0 , q1 , . . . , qm . If a + ib is not a solution to L(m) = 0, then there exists a particular
solution of the form
yp (x) = eax cos(bx)(Ao + A1 x + A2 x2 + + Am xm ) + eax sin(bx)(Bo + B1 x + B2 x2 + + Bm xm ),
where A0 , A1 , . . . , Am , B0 , B1 , . . . , Bm can be obtained by substitution.
If a + ib is a root of the polynomial with multiplicity k, then the assumed particular solution should be modified by
multiplying with xk .
4.3.2 Variation of Parameters
A particularly effective method is the method of variation of parameters.
Consider,
an (x)y (n) + an1 y (n1) + + a1 (x)y (1) + a0 y = g(x),
where all of the functions are continuous on some interval, where an (x) 6= 0.
Suppose
y(x) = c1 y1 (x) + c2 y2 (x) + + cn yn (x)
is the general solution to the corresponding homogeneous equation. This method, replaces ci with some function ui (x).
That is, we look for u1 (x), u2 (x), . . . , un (x) such that
yp (x) = u1 (x)y1 (x) + u2 (x)y2 (x) + + un (x)yn (x)
This method, due to Lagrange, allows a very large degree of freedom on how to pick the functions u1 , , un . One
restriction is the fact that the assumed solution must satisfy the differential equation. It turns out that, through an intelligent
construction of n 1 other constraints, we can always find such functions: As there are n unknowns, we need n equations,
for which can have freedom on how to choose. We will find these equations as follows:



T
Let U (x) = u1 (x) u2 (x) . . . un (x) and Y (x) = y1 (x) y2 (x) . . . yn (x) . Let us write in vector inner
product form:

yp (x) = Y (x)U (x)

The first derivative writes as
yp (x) = Y (x)U (x) + Y (x)U (x)
Lets take Y.U = 0. In this case,
yp (x) = Y (x)U (x) + Y (x)U (x)
and let us take Y .U = 0. By proceeding inductively, we obtain
yp(n) (x) = Y (n) U + Y (n1) .U
For the last equation, we substitute the expression into the differential equation. The differential equation is that:
an (x)yp(n) + an1 yp(n1) + + a1 (x)yp(1) + a0 yp = g(x),
Hence,
an (Y (n) U + Y (n1) U ) + an1 Y (n1) U + + Y U = g(x)
but
an (Y (n) U ) + an1 Y (n1) U + + Y U = 0
and hence

4.3 Non-Homogeneous Equations and the Principle of Superposition

17

an Y (n1) U = g(x)
Hence, we obtain:

y1
y1
..
.
(n1)

y1

y2
y2
..
.
(n1)

y2

...
...

yn
yn
..
.

...
(n1)
. . . yn

u1
u2

.. =
.

un

0
0
..
.

g/an

You may recognize that the term above is the matrix M , whose determinant is the Wronskian. We already know that this
matrix is invertible, since the functions y1 , y2 , . . . , yn are linearly independent solutions of the corresponding homogeneous
differential equation.

u1
u2

It follows that, we can solve for . by
..
un

0
0
..
.

(M (y1 , y2 , . . . , yn )(x))1

g(x)
an (x)

We will have to integrate out this term and obtain:

u1 (x)
Z
u2 (x)

.. = (M (y1 , y2 , . . . , yn )(x))1
.

un (x)

0
0
..
.
g(x)
an (x)

dx + C,

for some constant C (this constant will also serve us in the homogeneous solution).
Hence, we can find the functions u1 (x), u2 (x), . . . , un (x), and using these, we can find
yp (x) = u1 (x)y1 (x) + u2 (x)y2 (x) + + un (x)yn (x)
As we observed, if n linearly independent solutions to the corresponding homogeneous equation are known, then a particular solution can be obtained for the non-homogenous equation. One question remains however, on how to obtain the
solutions to the homogeneous equation. We now discuss a useful method for obtaining such solutions.

4.3.3 Reduction of Order

If one solution y1 (x) to an 2nd order linear differential equation is known, then by substitution of y = v(x)y1 (x) into the
equation
y + p(x)y + q(x)y = 0,
with y (x) = v (x)y1 (x) + v(x)y1 (x), y (x) = v (x)y1 (x) + 2v (x)y1 (x) + v(x)y1 (x)
v(y + p(x)y + q(x)y) + v (2y1 + py1 ) + v y1 = 0
Thus, it follows that
v +

(2y1 + py1 )
v =0
y1

18

4 Higher-Order Ordinary Linear Differential Equations

This is a first-order equation in v . Thus, v can be solved, leading to a solution for v, and ultimately solving for y(x) =
v(x)y1 (x).
As such, if one solution to the homogeneous equation is known, another independent solution can be obtained.
The same discussion applies for an nth order linear differential equation. If one solution is known, then by writing y(x) =
v(x)y1 (x) and substituting this into the equation (as we did above), a differential equation of order n 1 for v (x) can be
obtained. The n 1 linearly independent solutions for v can all be used to recover n 1 linearly independent solutions to
the original differential equation. Hence, by knowing only one solution, n linearly independent solutions can be obtained.

4.4 Applications of Second-Order Equations

Consider an electrical circuit consisting of a resistor, capacitor, inductor and a power supply.
The equation describing the relation between the charge around the capacitor, the current and the power supply voltage is
given as follows:
di(t)
q(t)
+ iR +
= E(t),
dt
C
where q(t) is the charge in the capacitor, i(t) is the current, L, R, C are inductance, resistance and capacitance values,
respectively, and E(t) is the power supply voltage.
L

Recognizing that i(t) =

dqt(t)
dt ,

and dividing both sides by L,we have that

d2 q(t) R dq(t) q(t)
E(t)
+
+
=
dt2
L dt
LC
L

This is a second order differential equation with constant coefficients. The equation is non-homogeneous. As such, we first
need to obtain the solution to the corresponding homogeneous equation.
This writes as:

d2 q(t) R dq(t) q(t)

+
+
=0
dt2
L dt
LC

The characteristic polynomial is:

L(m) = m2 +

m
R
m+
= 0,
L
LC

R
m1 =
+
2L

with solutions:

or

m1 = +
with =

R
2L

and w0 =

1
LC

1
R 2
)
2L
LC

q
(2 w02

The second root is:

m2 =

q
(2 w02

There are three cases:

2 > w02 . In this case, we have two distinct real roots: The general solution is given as qc (t) = c1 em1 t + c2 em2 t

2 = w02 . In this case, we have two equal real roots: The general solution is given as qc (t) = c1 et + c2 tet

4.4 Applications of Second-Order Equations

19

p
2 = w02 . p
In this case, we have two complex valued roots: The general solution is given as qc (t) = c1 et cos( (2 w02 t)+
c2 et sin( (2 w02 t)

We now can solve the non-homogeneous problem by obtaining a particular solution and adding the complimentary solution
above to the particular solution.
Let E(t) = E cos(wt). In this case, we can use the method of undetermined coefficients to obtain a particular solution.
Suppose w 6= w0 .
Using the method of undetermined coefficients, we find that a particular solution is given by
qp (t) = A cos(wt) + B sin(wt),
where
A=

E
1
2
L w02 w + (2w)
2
2
w0 w

and
B=

2Aw
w02 w2

Hence, the general solution is

q(t) = qp (t) + qc (t)
When R = 0, that is there is no resistance in the circuit, then = 0 and the solution simplifies as the B term above
becomes zero.

4.4.1 Resonance
Now, let us consider the case when = 0 and w = w0 , that is the frequency of the input matches the frequency of the
homogeneous equation solution.
In this case, when we use the method of undetermined coefficients, we need to multiply our candidate solution by t to
obtain:
qp (t) = At cos(w0 t) + Bt sin(w0 t),
Substitution yields,
A = 0, B =

E
2w0 L

In this case, the solution is:

qp (t) = c1 (w0 t) + c2 sin(w0 t) +

E
t sin(w0 t)
2w0 L

As can be observed, the magnitude of qp (t) grows over time. This leads to breakdown in many physical systems. However,
if this phenomenon can be controlled (say by having a small non-zero value), resonance can be used for important
applications.

5
Systems of First-Order Linear Differential Equations

5.1 Introduction
In this chapter, we investigate solutions to systems of differential equations.
Definition 5.1.1 A set of differential equations which involve more than one unknown functions and their derivatives with
respect to a single independent variable is called a system of differential equations.
For example
(1)

y1 = f1 (x, y1 , y2 , . . . , yn )
(1)

y2 = f2 (x, y1 , y2 , . . . , yn )
and up to:
yn(1) = fn (x, y1 , y2 , . . . , yn )
is a system of differential equations.
Clearly, the first order equation that we discussed earlier in the semester of the form
y (1) = f (x, y),
is a special case. If each of the functions {f1 , f2 , . . . , fn } is a linear function of {x1 , x2 , . . . , xn }, then the system of
equations is said to be linear. In this case, we have:
(1)

y1 = 11 (t)y1 + 12 (t)y2 + + 1n (t)yn + g1 (t)

(1)

y2 = 21 (t)y1 + 22 (t)y2 + + 2n (t)yn + g2 (t)

and up to:
yn(1) = n1 (t)y1 + n2 (t)y2 + + nn (t)yn + gn (t)
If g1 , g2 , . . . , gn are zero, then the equation is said to be homogeneous.
We now state an existence and uniqueness theorem. Consider the following system of equations:
(1)

x1 = f1 (t, x1 , x2 , . . . , xn )
(1)

x2 = f2 (t, x1 , x2 , . . . , xn )
and up to:

22

5 Systems of First-Order Linear Differential Equations

x(1)
n = fn (t, x1 , x2 , . . . , xn )
fi
Theorem 5.1.1 Let { x
}, for all i {1, 2, . . . , n}, j {1, 2, . . . , n}, exist and be continuous in an n + 1-dimensional
j
domain containing the point (t0 , x01 , x02 , . . . , x0n ). Then, there exists an interval [t0 h, t0 + h] with h > 0, such that for
all t [t0 h, t0 + h], there exists a unique solution

x1 (t) = 1 (t),

x2 (t) = 2 (t), . . . , xn (t) = n (t)

of the system of differential equations which also satisfies

x1 (t0 ) = x01 ,

x2 (t0 ) = x02 , . . . , xn (t0 ) = x0n

For a vector x(t) Rn , x (t) denotes the derivative of the vector x(t), and it exists when all the components of x(t) are
differentiable. The derivative of x(t) with respect to t is a vector consisting of the individual derivatives of the components
of x:

x1 (t)
x2 (t)

x (t) = .
..
xn (t)

We note that, the integral of a vector is also defined in a similar pattern, that is, the following holds:

R
R x1 (t)dt
Z
x2 (t)dt

x(t)dt =

..

R .
xn (t)dt
5.1.1 Higher-Order Linear Differential Equations can be reduced to First-Order Systems
One important observation is that any higher-order linear differential equation
y (n) + an1 (x)y (n1) + an2 (x)y (n2) + + a1 (x)y (1) + a0 (x)y = 0,
can be reduced to a first-order system of differential equation by defining:
x1 = y,

x2 = y ,

x3 = y , . . . , xn = y (n1)

It follows that,
x1 = y
x2 = y = x1
x3 = y = x2
until
xn = xn1
As such, we obtain:

0
1
0
x1
x2 0
0
1

x3 0
0
0
=
.. ..
..
..
. .
.
.
a0 (x) a1 (x) a2 (x)
xn

x1
x2

x3

..
.
...
xn
. . . an1 (x)

...
...
...

0
0
0
..
.

5.2 Theory of First-Order Systems

23

Hence, if we write:

x1
x2


x = x3
..
.
xn

0
0
0
..
.

1
0
0
..
.

0
1
0
..
.

...
...
...

0
0
0
..
.

A(x) =

...
a0 (x) a1 (x) a2 (x) . . . an1 (x)
We obtain a first-order differential equation:
x (t) = A(x)x(t)
As such, first-order systems of equations are very general.
Exercise 5.1.1 Express 5y (3) + y (2) + y (1) + y = 0 as a system of differential equations, by defining x1 = y, x2 = y and
x3 = y .

5.2 Theory of First-Order Systems

We first consider homogeneous linear differential equations with g(t) = 0:
x (t) = A(x)x(t),
where A(t) is continuous.
Theorem 5.2.1 Let x1 (t), x2 (t), . . . , xn (t) be solutions of the linear homogeneous equation. If these solutions are linearly
independent, then the determinant of the matrix


M (t) = x1 (t) x2 (t) . . . xn (t)

is non-zero for all t values where the equation is defined.

Definition 5.2.1 A linearly independent set of solutions to the homogeneous equation is called a fundamental set of solutions.
Theorem 5.2.2 Let {x1 , x2 , . . . , xn } be a fundamental set of solutions to
x (t) = A(t)x(t)
in an interval < t < . Then, the general solution x(t) = (t) can be expressed as
c1 x1 (t) + c2 x2 (t) + . . . cn xn (t),
and for every given initial set of conditions, there is a unique set of coefficients {c1 , c2 , . . . , cn }.
Hence, the main issue is to obtain a fundamental set of solutions. Once we can obtain this, we could obtain the complete
solution to a given, homogeneous differential equation.

24

5 Systems of First-Order Linear Differential Equations

We will observe that, we will be able to use the method of variation of parameters for systems of equations as well. The
fundamental set of solutions will be useful for this discussion as well.
The next topic will focus on system of differential equations which are linear, and constant-coefficient.

6
Systems of First-Order Constant Coefficient Differential Equations

6.1 Introduction
In this chapter, we consider systems of differential equations with constant coefficients.

6.2 Homogeneous Case

If a system of linear equations consist of constant-coefficient equations, then, a linear equation becomes of the form:
x (t) = Ax(t)
where A is a constant (independent of t) matrix.
We observed that, we need to find a fundamental set of solutions to be able to solve any such differential equation with
an arbitrary initial condition. Recall further that, a fundamental set of solutions is any set of linearly independent solutions
satisfying the equation.
Let us first assume that A is a Real-Symmetric matrix (and as such, has n full eigenvectors, which are linearly independent).
In this case, we have that if Av i = i v i , then
x1 = ei t v i
is a solution to the equation
x (t) = Ax(t)
Let us verify this:
d
d i
x = (ei t v i ) = i v i ei t = i v i ei t = Axi
dt
dt
Hence, we can find n such solutions, and these provide a fundamental set of solutions.
However, it is not always the case that we can find n eigenvectors. Sometimes, one has use generalized eigenvectors. We
now provide the general solution where A can also have generalized eigenvectors.
Theorem 6.2.1 Consider an initial value problem, with initial conditions x(t) = x0 . In this case, the following is a
solution:
x(t) = eAt x0
where
eAt = I + At + A2

t2
tn
+ . . . An + . . .
2
n!

26

6 Systems of First-Order Constant Coefficient Differential Equations

is the matrix exponential.

The above can be verified by direct substitution. Furthermore, by the uniqueness of the solution, this is the solution to the
equation. As such, eAt can be used directly to provide solutions.

6.3 How to Compute the Matrix Exponential

First, let us consider a 2 2 matrix

 
10
A=I=
01

In this case,
eAt = I + It + I 2

t2
tn
+ + In + . . .
2!
n!

Since I n = I for any n, it follows that

e

At

et 0
=
0 et


With similar arguments, if A is diagonal

we obtain

1 0 0
A = 0 2 0 ,
0 0 3
eAt

e 1 0 0
= 0 e2 t 0 ,
0 0 e3 t

Hence, it is very easy to compute the exponential when the matrix has a nice form.
What if A has a Jordan form? We now discuss this case.
First, we use a result that if AB = BA, that is if A and B commute, then
e(A+B) = eA eB
You will prove this in your assignment. In this case, we can write a matrix

1 1 0
A = 0 1 1
0 0 1
as B + C, where

1 0 0
B = 0 1 0
0 0 1

010
C = 0 0 1
000

We note that BC = CB, for B is the identity matrix multiplied by a scalar number. Hence,
eAt = eBt eCt .
All we need to compute is eCt , as we have already discussed how to compute eBt .

6.4 Non-Homogeneous Case

27

It should be observed that C 3 = 0.

For a Jordan matrix where the number of 1s off the diagonal is k 1, the kth power is equal to 0.
Now,
eCt = I + Ct + C 2
becomes
eCt
Hence,
eAt

t2
t3
+ C3 + . . . ,
2!
3!

1 t t2 /2
t
= I + Ct + C 2 = 0 1 t
2!
00 1
2

2
e 1 0 0
1 t t2 /2
e1 t te1 t t2 e1 t
= 0 e1 t 0 0 1 t = 0 e1 t te1 t
00 1
0 0 e1 t
0
0
e1 t

Now that we know how to compute the exponential of a Jordan form, we can proceed to study a general matrix. Let
A = P BP 1 ,
where B is in a Jordan form. Then,
A2 = (P BP 1 )2 = P BP 1 P BP 1 = P B 2 P
A3 = (P BP 1 )3 = P BP 1 P BP 1 P BP 1 = P B 3 P
and hence,
Ak = (P BP 1 )3 = P BP 1 P BP 1 (P BP 1 )k1 = P B k P
Finally,
eA = P (eB )P 1
and
eAt = P (eBt )P 1
Hence, once we obtain a diagonal matrix or a Jordan form matrix B, we can compute the exponential eAt very efficiently.

6.4 Non-Homogeneous Case

Consider
x (t) = Ax(t) + g(t),
with initial condition x(0) = x0 . In this case, via variation of parameters, we look for a solution of the form
x(t) = X(t)v(t),
where X(t) is a fundamental matrix for the homogeneous equation. We obtain the solution (with the details presented in
class) as:
Z
t

eA(t ) g( )d

x(t) = eAt x0 +

You could verify this result by substitution. The uniqueness theorem reveals that this has to be the solution.
This equation above is a fundamentally important one for mechanical and control systems. Suppose a spacecraft needs to
move from the Earth to the moon. The path equation is a more complicated version of the equation above, and g(t) is the
control term.

28

6 Systems of First-Order Constant Coefficient Differential Equations

6.5 Remarks on the Time-Varying Case

We note that the discussions above also apply to the case when A is a function of t. In this case,
x (t) = A(t)x(t),
and x(t) = X(t)c, where
d
X(t) = A(t)X(t)
dt
and
det(X(t)) 6= 0, t
Definition 6.5.1 If X(t) is a fundamental matrix, then (t, t0 ) = X(t)X(t0 )1 is called a state-transition matrix such
that
x(t) = (t, t0 )x(t0 )
Hence, the state-transition matrix (t1 , t0 ), transfers the solution at time t0 to another time t1 .

7
Stability and Lyapunovs Method

7.1 Introduction
In many engineering applications, one wants to make sure things behave nicely in the long-run; without worrying too much
about the particular path the system takes (so long as these paths are acceptable).
Stability is the characterization ensuring that things behave well in the long-run. We will make this statement precise in the
following.

7.2 Stability
Before proceeding further, let us recall the l2 norm: For a vector x Rn ,
v
u n
uX
x2i ,
||x||2 = t
i=1

where xi denotes the ith component of the vector x.

We have three definitions of stability:

1. Local Stability: For every > 0, > 0 such that ||x(0)||2 < implies that ||x(t)||2 < , t 0 (This is also known
as stability in the sense of Lyapunov).
2. Local Asymptotic Stability: > 0 such that ||x(0)||2 < implies that limt ||x(t)||2 = 0.
3. Global Asymptotic Stability: For every x(0) Rn , limt ||x(t)||2 = 0. Hence, here, for any initial condition, the
system converges to 0.
One could consider an inverted pendulum as an example of a system which is not locally stable.
It should be added that, stability does not necessarily need to be with regard to the origin; that is stability can hold for any
x0 . In this case, the above norms should be replaced with ||x(t) x0 ||2 (such that, for example for the asymptotic stability
case, x(t) will converge to x0 ).

7.2.1 Linear Systems

Theorem 7.2.1 For a linear differential equation

30

7 Stability and Lyapunovs Method

x = Ax,
the solution is locally and globally asymptotically stable if and only if
max{Re{i }} < 0,
i

where Re{.} denotes the real part of a complex number, and i denotes the eigenvalues of A.
We could also further strengthen the theorem.
Theorem 7.2.2 For a linear differential equation
x = Ax,
the system is locally stable if and only if

max{Re{i }} 0,
i

If Re{i } = 0, for some i , the algebraic multiplicity of this eigenvalue should be the same as the geometric multiplicity.

Here, Re{.} denotes the real part of a complex number, and i denotes the eigenvalues of A.
Exercise 7.2.1 Prove the theorem.

7.2.2 Non-Linear Systems

In practice, many systems are not linear, and hence the above theorem is not applicable. For such systems, there are two
approaches:

7.3 Lyapunovs Method

A common approach is via the so-called Lyapunovs second method. In class we defined Lyapunov (Energy) functions:
V (x) : Rn R is a Lyapunov function if
1. V (x) > 0, x 6= 0
2. V (x) = 0, if x = 0.
3. V (x) is continuous, and has continuous partial derivatives.
First we present results on local asymptotic stability. Let Rn be a closed, bounded set containing 0.
Theorem 7.3.1 a) For a given differential equation x (t) = f (x(t)) with f (0) = 0, and continuous f , if we can find a
Lyapunov function V (x) such that
d
V (x(t)) 0,
dt
for all x 6= 0 and x , then, the system is locally stable (stable in the sense of Lyapunov).

7.3 Lyapunovs Method

31

b) For a given differential equation x (t) = f (x(t)) with f (0) = 0, and continuous f , if we can find a Lyapunov function
V (x) such that
d
V (x(t)) < 0,
dt
for x(t) = x \ {0} where is a closed and bounded set containing the origin, the system is locally asymptotically
stable.
Proof. a) Let > 0 be given. We will show the existence of > 0 such that for all ||x(0)||2 , ||x(t)||2 for all
t 0. Define m = minx:||x||2= V (x) (such an m exists by the continuity of V ). Let 0 < < so that for all ||x||2 it
follows that V (x) < m. Such a exists by continuity of V . Now, for any ||x(0)||2 , V (x(0)) < m and furthermore by
d
V (x(t)) 0, V (x(t)) < m. This then implies that ||x(t)||2 < since otherwise, there would have to be
the condition dt
some time t1 so that ||x(t1 )||2 = which would result in V (x(t1 )) m, leading to a contradiction.
b) Let be so that with = r, satisfy the condition in part a so that with ||x(0)||2 , ||x(t)||2 r for all t so that
d
V (x(t)) < 0, V (x(t)) is a monotonically decreasing family of non-negative numbers and
x(t) . Since we have that dt
it therefore has a limit. Call this limit c. We will show that c = 0. Suppose c 6= 0. Let be such that for
2 ,
 all ||x||
V (x) c. It then follows that for all t, x(t) {x : ||x||2 r}. Define a = max{x:||x||2r} x V (x) f (x).

The number a exists since the functions considered are continuous, and it is either zero or a negative number, but it
cannot be zero by the assumptions stated in the theorem. Therefore, a < 0. This implies then that V (x(t)) = V (x(0)) +
Rt d
s=0 ds V (x(s))ds V (x(0))+at, which implies that V (x(t)) will be negative after a finite t. This cant be true, therefore
c cannot be non-zero.

The theorem above can be strengthened to global asymptotic stability if part b of the Theorem above is also satisfied by
some Lyapunov function with the following property:
1. The Lyapunov function V (x) is radially unbounded, that is lim||x||2 V (x) = and

d
dx V

(x) < 0 for all x 6= 0.

Exercise 7.3.1 Show that x = x3 is locally asymptotically stable, by picking V (x) = x2 as a Lyapunov function. Is this
solution globally asymptotically stable?
Exercise 7.3.2 Consider x + x + x = 0. Is this system asymptotically stable?
Hint: Convert this equation into a system of first-order differential equations, via x1 = x and x2 = x1 , x2 = x1 x2 .
Then apply V (x1 , x2 ) = x21 + x1 x2 + x22 as a candidate Lyapunov function.

Linearization
Consider x = f (x), where f is a function which has continuous partial derivatives so that it locally admits a first-order
Taylors expansion representation in the following form:
f (x) = f (x0 ) + f (x0 )(x x0 ) + h.o.d.,
where h.o.d. stands for higher order terms. Now consider x0 = 0 and further with the condition that f (0) = 0 and f is
continuously differentiable so that we can write
f (x) = f (0) + f (0)(x) + b(x)x,
where limx0 b(x) = 0. One then studies the properties of the linearized system by only considering the properties of
f (x) at x = 0. Then, if f (0) is a matrix with all of its eigenvalues having negative real parts, then the system is locally
stable.

32

7 Stability and Lyapunovs Method

However, it should be noted that, this method only works locally, and hence, the real part of the eigenvalues being less than
zero only implies local stability due to the Taylor series approximation.
You will encounter many applications where stability is a necessary prerequisite for acceptable performance. In particular,
in control systems one essential goal is to adjust the system dynamics such that the system is either locally or globally
asymptpotically stable.
For the scope of this course, all you need to know are the notions of stability and application of Lyapunov theory to
relatively simple systems. The above is provided to provide a more complete picture for the curious.
If time permits, we will discuss these issues further at the end of the semester, but this is highly unlikely. Mathematics and
Engineering students will revisit some of these discussions in different contexts, but with essentially same principles, in
MATH 332, MATH 335 and MATH 430.

8
Laplace Transform Method

8.1 Introduction
In this chapter, we provide an alternative approach to solve constant coefficient differential equations. This method is
known as the Laplace Transform Method.

8.2 Transformations
A transformation T is a map from a signal set to another one, such that T is onto and one-to-one (hence, bijective).
Let us define a set of signals on the time-domain. Consider the following mapping:
Z
K(s, t)x(t)dt, s R.
g(s) =
=

Here K(s, t) is called the Kernel of the transformation.

The idea is to transform a problem involving x(t) to a simpler problem involving g(s), obtaining a solution, and then
transforming back to x(t).

8.2.1 Laplace Transform

Let x(t) be a Rvalued function defined on t 0.
Let s R be a real variable. The set of improper integrals for s R give rise to the Laplace transform.
Z
x(t)est dt, s R
X(s) =
=0

We will also denote X(s) by

L(x),
evaluated at s. Hence, you might imagine L(x) as a function, and X(s) is the value of this function at s.
The Laplace transform exists for the set of functions such that
|f (t)| M et , t 0,

34

8 Laplace Transform Method

for some finite M < and > 0 and f (t) is a piece-wise continuous function in every interval [0, ], 0, R. We
say a function is piece-wise continuous if every finite interval can be divided into a finite number of subintervals in which
the function is continuous, and the function has a finite left-limit and a right-limit in the break-points.

8.2.2 Properties of the Laplace Transform

Laplace transform is linear.
Theorem 8.2.1 Let f be a real, continuous function and |f (t)| M et . Furthermore, suppose f (t) is piece-wise continuous on every finite interval. Then,
L(f ) = sL(f ) f (0),
for all s > .
The same discussion applies to the derivatives of higher order:
Theorem 8.2.2 If f, f (1) , f (2) , . . . , f (n1) are continuous and f (n) is piecewise continuous on every bounded interval,
and
|f (i) (t)| M et ,
for all i = 1, 2, . . . , n 1 and all t 0, then for all s
L(f n ) = sn L(f ) sn1 f (0) sn2 f (1) (0) sn3 f (2) (0) f (n1) (0),
Another important property of Laplace Transforms is that
L(eat f t ) = F (s a)
And
L(tn f t ) = (1)n

dn
F (s)
dsn

8.2.3 Laplace Transform Method for solving Initial Value Problems

Consider a linear DE with constant coefficients:
an y (n) + an1 y (n1) + + a0 y = b(t),
with y(0) = c0 , y (1) (0) = c1 , . . . , y (n1) (0) = cn1 . Then, we have
L(y (n) ) = sn L(y) sn1 c0 sn2 c1 cn1
L(y (n1) ) = sn1 L(y) sn2 c0 sn1 c1 cn2
and
L(y (1) ) = sL(y) c0
Combining all, we obtain:
(an sn + an1 sn1 + + a0 )L(y) c0 (an sn1 + a1 sn2 + + a1 )
c1 (an sn2 + a1 sn3 + + a2 ) cn1 an

= B(s).

(8.1)

As such, we could obtain the Laplace transform of y(t). We now will have to find the function which has its Laplace
transform as Y (s). Thus, we need to find an inverse Laplace transform.

8.2 Transformations

35

8.2.4 Inverse Laplace Transform

The inverse of the Laplace Transform is not necessarily unique. But, when we assume the functions to be piecewise
continuous, the inverse is essentially unique (up to a number of discontinuities). We will find the inverse, by a table lookup.
In general, there is an inverse formula, but this involves complex integration. We will not consider this in this course.

8.2.5 Convolution:
Let f and g be two functions that are piecewise continuous on every finite closed interval, and be less that M et , t 0.
and f (t) = g(t) = 0 for t 0. The function
Z
f ( )g(t )d
h(t) = (f g)(t) =
0

is called the convolution of f and g.Then,


 

L(f g)(s) = L(f ) (s) L(g) (s),
for all s values.

8.2.6 Step Function

We define a step function for a 0 as follows:

ua (t) = 1(a0) ,

where 1(.) is the indicator function. We can show that for s > 0
L(ua (t)) =

1 as
e
s

8.2.7 Impulse response

Consider the following function:
F (t) =

if 0 t ,
else.

Now, consider lim0 F (t). This limit is such that, for every t 6= 0, the limit is zero valued. When t = 0, however, the
limit is . of We denote this limit by (t).
This limit is known as the impulse. The impulse is not a regular function. It is not integrable. The proper way to study such
a function is via distribution theory. This will be further covered in Math 334 and Math 335.
For the purpose of this course however, we will always use the impulse function under the integral sign.
In class, we showed that
lim

for all s values.

est F (t)dt = 1,

Consider a linear constant coefficient system

P(D)(y) = an y (n) + an1 y (n1) + . . . a1 y (1) + a0 y = g(t),

36

8 Laplace Transform Method

with y(0) = y (1) (0) = = y (n1) (0) = 0. We had observed that the Laplace transform of the solution Y (s) =
G(s)
an sn +an1 sn1 ++a0

If g(t) = (t), then G(s) = 1 and hence,

Y (s) =

1
.
an sn + an1 sn1 + + a0

We denote y(t) when g(t) = (t), by h(t) and call it the impulse response of a system governed by a linear differential
equation.
Exercise 8.2.1 Consider the linear equation above. If g(t) = p(t) for some arbitrary function p(t) defined on t 0, then
the resulting solution is equal to
y(t) = p(t) h(t)

9
On Partial Differential Equations and Separation of Variables

Recall that a differential equation that entails more than one independent variable is called a partial differential equation.
An important example is the heat equation
u
2u
= k 2,
t
x

(9.1)

where u is the dependent function and x, t are the independent variables. Here, k is a constant.
This equation models the variation of temperature u with position x and time t in a heated rod where it is assumed that at
a given cross-section at length x from one end of the rod the temperature is a constant.
Suppose that the rod has a finite length L, extending from x = 0 to x = L. Its temperature function u(x, t) solves the heat
equation. Suppose that the initial temperature is given by
u(x, 0) = f (x)

(9.2)

One can also restrict fixed temperatures at the two ends of the rod so that
u(0, t) = u(L, t) = 0.

(9.3)

A physical setting would be the case where the rod is connected to large objects with constant temperatures.
These equations define a boundary value problem, where (9.1) is defined for 0 < x < L, t > 0.
This example will be studied in this chapter; the ideas used here are applicable to a large class of such differential equations.
First, as in the ordinary differential equations, due to linearity, the principle of superposition holds (see Chapter 4): That
is, if u1 and u2 solve (9.1), so does c1 u1 + c2 u2 for any scalars c1 , c2 . Furthermore, the homogenous conditions (9.3)
are also satisfied by the linear combinations. On the other hand, the condition (9.2) is a non-homogenous condition. If
u(x) = c1 u1 (x, 0) + c2 u2 (x, 0) = f (x) and u1 and u2 satisfy (9.1) and(9.3), then u is a solution to the heat equation given
the boundary conditions.
The solution technique is then to consider first the homogenous problem, and then arrange the terms so as to arrive at the
non-homogenous problem. A common first attempt is to look for a solution that has the structure that:
u(x, t) = X(x)T (t),
where X only depends on X and T only depends on T . Such a method is called the method of separation of variables.
Assuming such a structure holds, (9.1) reduces to:
T (1)
X (2)
=
,
X
kT

38

9 On Partial Differential Equations and Separation of Variables

Now, suppose that both of these terms are equal to a constant . Of course, there may be multiple such values, and we
will need to account for all such values. First, consider the term:
X (2)
+=0
X
We know from our earlier analysis that the general solution for such a problem will be given by either an exponential or a
harmonic family, depending on whether 0 or < 0. The conditions u(0, t) = u(L, t) = 0 actually rule out the case
with 0 for a non-zero solution, and it turns out that the general solution will write as:

Xn (x) = sin(| |x),

2

where { nL2 , n Z+ }. Now with this set of valus, we can solve the second equation:
n2 2
T (1)
,
=
kT
L2
which leads to the solution:
Tn (t) = Kn e

n2 2 k
t
L2

where Kn is a constant depending on n. Thus, a candidate solution will be

Kn e

n2 2 k
t
L2

sin(

n
x)
L

Using the principle of superposition, we can take a linear combination so that any such linear combination
u(x, t) =

X
n

Kn e

n2 2 k
t
L2

sin(

n
x)
L

is a solution to the homogenous problem. Now, if we also satisfy the initial condition:
X
n
u(x, 0) = f (x, 0) =
Kn sin( x),
L
n
given by the non-homogenous boundary condition, we can find the {Kn , n Z+ } coefficients and obtain a solution to the
heat equation.
Next year, in MTHE 334 and MTHE 335; we will see that any continuous function defined one a bounded interval can
be expressed in terms of sines and cosines, through an expansion known as the Fourier Transform. This will allow for the
combination of the Kn terms above, leading to a solution to the heat equation.
Partial differential equations are significantly more challenging in general than ordinary differential equations. Our discussion in this chapter is introductory. However, the method of separation of variables is an effective solution technique
applicable to a large class of problems.

10
Series Method

10.1 Introduction
In this chapter, we discuss the series method to solve differential equations. This is a powerful technique essentially building
on the method of undetermined coefficients.

10.2 Taylor Series Expansions

11
Numerical Methods

11.1 Introduction
In this chapter, we discuss numerical methods to compute solutions to differential equations. Such methods are useful when
a differential equation does not belong to the classifications we have investigated so far in the course.

11.2 Numerical Methods

11.2.1 Eulers Method
Consider
dx
= f (x, t)
dx
Let x(t) be the solution, such that x(t) admits a Taylor expansion around t0 . We may express x(t) around x(t0 ) as
x(t) = x(t0 ) +

d
(t t0 )2
d2
x(t)|t=t0 (t t0 ) + 2 x(t)|t=t0
+ h.o.t.,
dt
dt
2!

where h.o.t. (higher order terms) converge to zero as t t0 .

One method to approximate the solution x(t) with a function (t) by eliminating the terms in Taylor expansion after the
first order term. That is, by obtaining:
(t1 ) = x(t0 ) + f (x(t0 ), t0 )(t1 t0 )
in the interval t1 [t0 , t0 + h], for some small h. Likewise,
(t2 ) = (t1 ) + f ((t1 ), t1 )(t2 t1 )
in the interval t2 [t1 , t1 + h]

(t3 ) = (t2 ) + f ((t2 ), t2 )(t3 t2 )

and
(tk ) = (tk1 ) + f ((tk1 ), tk1 )(tk tk1 )
This method is known as the Eulers method.

42

11 Numerical Methods

11.2.2 Improved Eulers Method

One way to improve the above method is via the averaging of the end-points in the interval [tk , tk + h]:


1
f ((tk1 ), tk1 ) + f ((tk ), tk ) (tk tk1 )
(tk ) = (tk1 ) +
2
But, this requires the use of f ((tk ), tk ) to compute (tk ). This is be computationally difficult.
The improved Eulers method replaces this term with (tk1 ) + f ((tk1 ), tk1 )(tk tk1 )
Hence, for all k 0, the improved Eulers method works as:



1
(tk ) = (tk1 ) +
f ((tk1 ), tk1 ) + f (tk1 ) + f ((tk1 ), tk1 )(tk tk1 ) (tk tk1 ),
2
for tk [tk1 , tk1 + h].
11.2.3 Including the Second Order Term in Taylors Expansion
Let for simplicity h = tk+1 tk for all k values.
Let for all k 0

1
(tk + h) = (tk ) + x (tk )h + x (tk )h2
2

That is, in the above, we truncate the Taylor expansion at the second term, instead of the first one.
Here one has to compute the second derivative. This can be done as follows:
x =

d dx
d
x
x dx
x
x
d2 x
=
=
x
=
+
=
+
x
dt2
dt dt
dt
t
y dt
t
x

11.2.4 Other Methods

In the above methods, a smaller h leads to a better approximation, with the additional complication of higher number of
computations to reach from any point to another one.
If the differential equation involves continuous functions, then there always exists a sufficiently small h, with which you
can carry over the numerical approximation.
It should be noted that, there are many other possibilities to further improve the approximation errors, such as the higherorder Taylor approximations.
One other popular method which we will not discuss is the Runge-Kutta method, which assigns numerical weights on the
average of the derivatives at various locations on the (x, t) values. However, the Taylor expansion based methods are more
general.
The approximations can be generalized for higher dimensional settings; in particular for systems of differential equations.
These follow the same basic principles. One way is to linearize a system with the first order Taylors approximation for
each of the terms in the system.

A
Brief Review of Complex Numbers

A.1 Introduction
The square root of -1 does not exist in the space of real numbers. We extend the space of numbers to the so-called complex
ones, so that all polynomials have roots.
We could define the space of complex numbers C as the space of pairs (a, b), where a, b R, on which the following
operations are defined:
(a, b) + (c, d) = (a + b, c + d)
(a, b).(c, d) = (ac bd, ad + bc)
We could also use the notation:
(a, b) = a + bi.
In the above representation, the notation i stands for imaginary. As such b is the imaginary component of the complex
number. With this definition, the square root of -1 is (0, 1) and this is denoted by the symbol i (some texts use the term j,
in particular the literature in physics and electrical engineering heavily use j, instead of i). As such, complex number has
the form
a + bi,
where a denotes the real term, and b is the imaginary term in the complex number. Two complex numbers are equal if and
only if both their real parts and imaginary parts are equal. We denote the space of complex numbers as C.
One could think of a complex number as a vector in a two dimensional space; with the x-axis denoting the real component,
and the imaginary term living on the y-axis. Observe that with b = 0, we recover the real numbers R.
We define the conjugate of a complex number a + bi, is a bi. The absolute value of a complex number is the Euclidean
norm of the vector which represents the complex number, that is
p
|a + bi| = a2 + b2 ,

where |.| denotes the Euclidean norm of a vector.

Complex numbers adopt the algebraic operations of addition, subtraction, multiplication, and division operations that
generalize those of real numbers. These operations are conveniently carried out when the complex number is represented
in terms of exponentials.

A.2 Eulers Formula and Exponential Representation

The power series representation for ex in powers of x around 0 is given by

44

A Brief Review of Complex Numbers

ex = 1 + x +

xk
x2
+ +
+ ...
2!
k!

For any complex number z, we define ez by the power series:

ez = 1 + z +

z2
zk
+ +
+ ...
2!
k!

In particular, if we assume the above holds for complex numbers as well, it follows that:
(i)2
(i)k
+ +
+ ...
2!
k!
2
3
4
(i)k
= 1 + i
i +
+
+ ...
2!
3!
4!
k!

ei = 1 + i +

(A.1)

Let us recall the Taylor series for cos() and sin() around 0:
cos() = 1
sin() =

2
4
6
2n
+

+ + (1)n
+ ...
2
4!
6!
(2n)!

5
7
2n+1
3
+

+ + (1)n
+ ...
3
5!
7!
(2n + 1)!

The above hold for all R. This can be verified by the fact that series converges and the error in the approximation
converges to 0.
Thus, ei satisfies:
ei = cos() + i sin()
The above is known as Eulers formula and is one of the fundamental relationships in applied mathematics. Also, note that
via the equations above, it follows that
ei + ei
,
cos() =
2
and
ei ei
sin() =
2i
Now, we could represent complex numbers in terms of exponentials. For example
ei/2 = i
ei = 1

ei3/2 = i
ei2 = 1

Via an extension of this discussion, we can represent an arbitrary complex number z = a + ib as

rei
with
r = |z| =
and is such that

p
a2 + b 2

tan() = (b/a),

since a = a2 + b2 cos() and b = a2 + b2 sin() Such an exponential representation of a complex number also lets us
take roots of complex numbers: For example let us compute:

A.2 Eulers Formula and Exponential Representation

45

(1)1/4
This might arise for example, when one tries to solve the homogenous differential equation
y (4) + y = 0

(A.2)

In this case, one needs to solve the polynomial

L(m) = m4 + 1 = 0.
Since this is a fourth order equation, there needs to be four roots. We could compute the roots by expressing 1 as an
exponential. Observe that
ei = ei(+2) .
Hence, we could have four different representations for 1 as follows:
1 = ei = ei3 = ei5 = ei7
Hence,
m1 = ei/4
m2 = ei3/4
m3 = ei5/4
m4 = ei7/4
All of which satisfy: m4i = 1, for i = 1, 2, 3, 4. Hence, after a few steps involving sines and cosines, the general solution
to the homogenous equation
y (4) + y = 0,
can be found to be:

y(x) = k1 ei 4 x + k2 ei 4 x + k3 ei3 4 x + k3 ei3 4 x ,

or

1
1
1
1
y(x) = c1 cos( x) + c2 sin( x) + c3 cos(3 x) + c4 sin(3 x).
4
4
4
4

B
Similarity Transformation and the Jordan Canonical Form

B.1 Similarity Transformation

If A and B are two real square matrices related by
B = P 1 AP,
where P is a nonsingular matrix, then A and B are said to be similar. Since
det(I B) = det(P 1 (I A)P ) = det(P 1 )det(I A)det(P ) = det(I A),
it follows that A and B have the same eigenvalues.

B.2 Diagonalization
If A is a real symmetric matric, and if we choose P to be a matrix whose columns are the orthonormal eigenvectors of A,
then the matrix B = P 1 AP is diagonal, which is also real, by an argument we proved in class. Hence, real symmetric
matrices can be diagonalized by a similarity transformation.
However, if the matrix A is not symmetric, then diagonalization might not be possible.
For such systems, a structure, which is close to a diagonal form, known as the Jordan canonical form, can be obtained:

B.3 Jordan Form

If A is a real matrix, then there exists a non-singular matrix P such that, B

B1 0 . . . 0
0 ...
0 B2 . . . 0
0 ...

.. .. .. ..
..
..
. . . .
.
.

B=
0 0 . . . Bk 0 . . .
0 0 . . . 0 Bk+1 . . .

.. .. .. ..
..
..
. . . .
.
.
0

where Bi , 1 i m have the form:

0 0

= P 1 AP , and B has the following form:

0
0

..
.

..
.

. . . Bm

48

B Similarity Transformation and the Jordan Canonical Form

i
0

Bi = .
..

1
i
..
.

0 ...
1 ...
.. ..
. .

0
0

.. .
.

0 0 0 . . . i

One could also have a Jordan canonical matrix with only real-valued entries (but we will not consider this).
In class we defined the notions of algebraic multiplicity and geometric multiplicity of an eigenvalue. Recall that the algebraic multiplicity of an eigenvalue i is equal to the number of times i appears as a root to det(A I) = 0.
The geometric multiplicity is the size of N (A I) (the null space of A I). This is equal to the number of linearly
independent eigenvectors that can be obtained as a result of the equation: Ax = i x.
If the matrix A has at least one eigenvalue with its algebraic multiplicity greater than its geometric multiplicity, then
diagonalization is not possible (Hence, it follows that, having a repeated eigenvalue does not automatically mean that
diagonalization is not possible).
A convenient way to generate a Jordan canonical form is through the use of eigenvectors and generalized eigenvectors:
That is, the columns of the matrix P may consist of the eigenvectors and the generalized eigenvectors.
Clearly, any n n square matrix with distinct eigenvalues (with n different eigenvalues) can be transformed into a diagonal
matrix, as there will be n Jordan blocks which form the diagonal matrix.