Mathematical Methods

Martin Stmpfle

January 30, 2013

 3

Contents

1 Numerical Analysis 5

2 Ordinary Differential Equations 7

2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.1.1 Theoretical Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.1.2 Numerical Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.2 Eulers Method and Variants . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.2.1 Eulers Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.2.2 Implicit Eulers Method . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.2.3 Improved Eulers Method . . . . . . . . . . . . . . . . . . . . . . . . 21
2.2.4 Heuns Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.3 Runge-Kutta Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.3.1 Runge-Kutta Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.3.2 Step Size Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.3.3 The One-Step Approach . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.4 Multistep Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.4.1 Linear Multistep Methods . . . . . . . . . . . . . . . . . . . . . . . . 28
2.4.2 Adams-Bashforth Methods . . . . . . . . . . . . . . . . . . . . . . . 29
2.4.3 Adams-Moulton Methods . . . . . . . . . . . . . . . . . . . . . . . . 30
2.4.4 Predictor-Corrector Methods . . . . . . . . . . . . . . . . . . . . . . 31
2.4.5 The General Multistep Approach . . . . . . . . . . . . . . . . . . . . 32
2.5 Stability and Stiffness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.5.1 Stability of One-Step Methods . . . . . . . . . . . . . . . . . . . . . 33
2.5.2 Stability of Multistep Methods . . . . . . . . . . . . . . . . . . . . . 39
2.5.3 Stiffness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2.6 Initial Value Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.6.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.6.2 Matlab Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.7 Boundary Value Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
2.7.1 Shooting Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
2.7.2 Finite Difference Methods . . . . . . . . . . . . . . . . . . . . . . . . 45
2.8 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.8.1 Biology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.8.2 Mechanics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
2.8.3 Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
2.8.4 Vehicle Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
2.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4

3 Partial Differential Equations 57

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.1.1 Theoretical Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.1.2 Numerical Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
3.2 Finite Difference Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.2.1 Finite Differences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.2.2 Difference Methods for 2D Elliptic Equations . . . . . . . . . . . . . 69
3.2.3 Difference Methods for 1D Parabolic Equations . . . . . . . . . . . 74
3.2.4 Difference Methods for 2D Parabolic Equations . . . . . . . . . . . 81
3.2.5 Difference Methods for 1D Hyperbolic Equations . . . . . . . . . . . 84
3.3 Finite Element Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
3.3.1 Meshes, Partitions, and Triangulations . . . . . . . . . . . . . . . . . 87
3.3.2 Variational Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
3.3.3 Function Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
3.3.4 Piecewise Linear Finite Elements . . . . . . . . . . . . . . . . . . . . 97
3.3.5 Galerkins Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
3.3.6 Rayleigh-Ritzs Method . . . . . . . . . . . . . . . . . . . . . . . . . 99
3.4 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
3.4.1 Heat Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
3.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

A Appendix 121
A.1 Greek Letters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

Bibliography 123

Index 125
 5

1 Numerical Analysis
 7

2 Ordinary Differential Equations

Ordinary differential equations are equations where the unknown is a funtion in one vari-
able. Many models from engineering sciences include ordinary differential equations. The
most popular class are multi body systems, where several components are linked together
by rods, springs, dampers, or elastomers. Depending on the constraints we distinguish
between initial and boundary value problems.
8

2.1 Introduction
We briefly discuss the basics of ordinary differential equations. Besides theoretical aspects
we introduce numerical definitions as well.

2.1.1 Theoretical Basics

Ordinary differential equations can be classified according to their properties. Among
these characteristics are form, order, linearity, or the type of given boundary values.

Definition 2.1 (Ordinary differential equation)

An equation involving a function y(t) of one independent variable t and its n derivatives
y (t), . . . , y (n) (t) is called an ordinary differential equation (ODE) of order n.

Ordinary differential equations are equations involving a function and some of its deriva-
tives. Some approaches to find solutions require the equation in such a special form that
the highest derivative is isolated on one side of the equation.

Definition 2.2 (Implicit and explicit form)

An ordinary differential equation
in the general form

F (t, y, y , y , . . . , y (n) ) = 0

is called an ordinary differential equation in implicit form.

in the special form

y (n) = f (t, y, y , y , . . . , y (n1) )

is called an ordinary differential equation in explicit form.

Linearity is also a fundamental property of an ordinary differential equation. The set of

all solutions then can be calulated always with a fixed scheme.

Definition 2.3 (Linear ordinary differential equation)

An ordinary differential equation is called linear if y and all its derivaties appear only
as linear terms.
 9

Example 2.1 (Ordinary differential equations)

a) The equation y = y is an explicit, first-order, linear equation. The general solution reads
y(t) = c et with c R.
b) The equation y = y is an explicit, second-order, linear equation with constant coefficients.
The set of all solutions is given by y(t) = c1 sin t + c2 cos t with c1 , c2 R.
c) The second-order, linear equation

2y + 4t2 y 8y = 0 y = 2t2 y + 4y

can be rewritten in explicit form.

d) The equation y 7y sin t = 0 is a third-order, linear equation in implicit form.
e) The second-order, non-linear equation y = y 2 + 3 sin t is in explicit form.
f) The first-order, non-linear equation y + 3 cos y = 8 y 2 cannot be transformed into an explicit
form.

The general solution of an ordinary differential equation typically is a set of infinite many
functions. Depending on appropriate additional initial conditions the solution then is
unique.

Definition 2.4 (Initial value problem)

An ordinary differential equation of order n for y
n=2
a function y(t) together with n initial condi-
tions (IC) at one single point t0 is called an
initial value problem (IVP):
y0
y(t0 ) = y0
y (t0 ) = y1
t0 t
y (n1) (t0 ) = yn1

Example 2.2 (Initial value problems)

a) Using the general solution from Example 2.1 the initial value problem has the following
unique solution:

y = y, y(0) = 2 y(t) = 2 et .

b) Using the set of all solutions from Example 2.1 the initial value problem has the following
unique solution:

y = y, y(0) = 1, y (0) = 0 y(t) = cos t .

For boundary conditions the situation is different. The existence and uniqueness of solu-
10

tions is not closely related to the number of boundary values.

Definition 2.5 (Boundary value problem)

An ordinary differential equation of order n to- y
n=4
gether with n boundary conditions (BC) at at
least two different points t0 and t1 is called a
y1
boundary value problem (BVP):

y(t0 ) = y0 , y(t1 ) = y1
y0
y (t0 )

= y2 , y (t1 )

= y3
t0 t1 t
y (m) (tk ) = yn1

Example 2.3 (Boundary value problems)

a) With the set of all solutions y(t) = c1 sin t+c2 cos t from Example 2.1 the following boundary
value problem has a unique solution:

y = y,

y(0) = 1, y ( ) = 1 y(t) = sin t + cos t .
2

b) Using the general solution y(t) = c1 sin t + c2 cos t from Example 2.1 the following boundary
value problem has no solution:

y = y, y(0) = 1, y() = 0 c2 = 1, c2 = 0 .

c) Again using the general solution y(t) = c1 sin t + c2 cos t from Example 2.1 the following
boundary value problem has infinite many solutions:

y = y, y(0) = 1, y() = 1 y(t) = c1 sin t + cos t .

Boundary value problems

Existence and uniqueness are more complicated for boundary value problems than for
initial value problems.

For first-order ordinary differential equation in explicit form the right-hand side of the
equation states the slopes of the solution curves. Using this information a vector field can
be drawn. Connecting the the small line elements gives the set of all solutions.
 11

Definition 2.6 (Vector field, flow lines, line elements)

Let y = f (t, y) be a first-order ordinary differ- y
ential equation. Then a set of small flow lines
or line elements at points (t, y) with slopes
f (t, y) is called a vector field of the differen-
tial equation.

Example 2.4 (Vector fields)

a) The vector field of the differential equation y

y =
y
t
1
consists of line elements that are in the direction
of lines through the origin. No line elements are 1 t
defined on the y-axis. So, the solutions are half
lines that end at the origin.

b) The line elements of the differential equation y

y =
t
y
1
are orthogonal to the line elements of y = yt . No
line elements are defined on the t-axis. Hence, 1 t
the solutions are half circles centered at the ori-
gin.

Theorem 2.1 (Vector field)

Each regular point in the vector field corresponds uniquely to one single solution.
Solutions cannot intersect.

Ordinary differential equations can be generalized from single equations to systems. Each
model state typically contributes one equation to the system. In general, the equations
are coupled. This means that more than one state is involved in each equation.
12

Definition 2.7 (System of ordinary differential equations)

A set of first-order ordinary differential equations

y1 = f1 (t, y1 , . . . , yn )
y
= f (t, y)
yn = fn (t, y1 , . . . , yn )

is called a first-order system of ordinary differential equations.

If the right-hand side of a system of equations does not depend on the independent
variable t, the system is called autonomous. Then, the change in the states depends only
the states themselves and not on time. Autonomous systems are special systems.

Definition 2.8 (Autonomous system of ordinary differential equations)

A set of first-order ordinary differential equations is called autonomous if all functions
fk on the right-hand side do not depend on t:

y1 = f1 (y1 , . . . , yn )
y
= f (y)
yn = fn (y1 , . . . , yn )

By introducing new states ordinary differential equation of higher order can be transformed
into first-order systems. This is important, because many numerical methods only deal
with first-order equations.

Transformation into a system

An ordinary differential equation of order n can be transformed into a system of n
first-order differential equations:

y1 = x y1 = y2
y2 = x
y2 = y3
x(n) = f (x, x , . . . , x(n1) ) ,
yn1 = x(n2) yn1

= yn
yn = x (n1)
yn = f (y1 , . . . , yn )
 13

Example 2.5 (Systems of differential equations)

a) The third-order differential equation

x 3tx + 5x = sin t

can be transformed into a system:

y1 = x y1 = y2
y2 = x y2 = y3
y3 = x y3 = 5y1 + 3ty2 + sin t

Since the differential equation is linear, the system also can be written using matrices:

y1 0 1 0 y1 0
y2 = 0 0 1 y2 + 0 .
y3 5 3t 0 y3 sin t

b) The nonlinear, second-order differential equation with initial values

x = cos(tx) + 4t2 + 6x, x(0) = 2, x (0) = 1

can be written as a nonlinear system:

y1 = x y1 = y2 , y1 (0) = 2
y2 = x y2 = cos(ty1 ) + 4t2 + 6y1 , y2 (0) = 1

c) The nonlinear second-order system with initial values

x = 6xx z + tet , x(1) = 0 , x (1) = 4

z = t + x + z + x + z , z(1) = 1 , z (1) = 2

can be transformed into a first-order system

y1 = x y1 = y3 , y1 (1) = 0
y2 = z y2 = y4 , y2 (1) = 1
y3 = x y3 = 6y1 y3 y4 + tet , y3 (1) = 4
y4 = z y4 = t + y1 + y2 + y3 + y4 , y4 (1) = 2

14

Example 2.6 (Vibration equation)

The vibration equation is defined as

x + 2 x + 02 = 0 ,

where 0 is the damping constant and 0 > 0 is the angular frequency. Using the new states

y1 = x , y2 = x

the second-order differential equation can be transformed into a linear first-order system:

y1 0 1 y1
=
y2 0 2 y2
2

Transformation of a system into an autonomous system

A non-autonomous first-order system of ordinary differential equations can be trans-
formed into an autonomous system using the time variable as an additional state
yn+1 = t:

y1 = f1 (t, y1 , . . . , yn ) y1 = f1 (yn+1 , y1 , . . . , yn )

yn = fn (t, y1 , . . . , yn ) yn = fn (yn+1 , y1 , . . . , yn )
yn+1

= 1

Example 2.7 (Autonomous system)

The third-order differential equation from Example 2.5

x 3tx + 5x = sin t

can be transformed into an autonomous system:

y1 = x y1 = y2
y2 = x y2 = y3
y3 = x y3 = 5y1 + 3y2 y4 + sin y4
y4 = t y4 = 1

Note, that the original differential equation is linear, whereas the autonomous system has lost
the property of linearity.

There are different types of solutions. A solution that does not change its state at all is
called an equilibrium solution. Other special solutions are periodic solutions. For periodic
solutions, the system states are repeated again and again.
 15

Definition 2.9 (Equilibrium solution, periodic solution)

A solution of a system of ordinary differential equations in the form
y(t) = y 0 with y 0 Rn is called an equilibrium solution.

y(t) = y(t + T ) with T > 0 is called a periodic solution. The parameter T denotes
the period.

Example 2.8 (Types of solutions)

a) Consider again the differential equation y = y from Example 2.1. The function y(t) = 0 is
an equilibrium solution. All solutions y(t) = c1 sin t + c2 cos t are periodic.
b) Consider the differential equation y = y. The trivial function y(t) = 0 is an equilibrium
solution. No solution y(t) = c1 et + c2 et is periodic.

Definition 2.10 (Phase-plane, phase-space, phase portrait)

The plane or space can be used to visualize the dynamics of differential equations.
The plane in which autonomous, two-dimensional first-order systems can be ana-
lyzed is called phase-plane.
For higher dimensional systems the dynamics can be analyzed in the phase-space.

The phase-plane or phase-space together with some solution curves is called phase
portrait.
16

Example 2.9 (Phase plane)

a) The initial value problem y2
2
x + x (0) = 0 .
4
x = 0, x(0) = 3,
9 1
can be transformed into a system:
3 2 1 1 2 3 y1
y1 = y2 , y1 (0) = 3 1
4
y2 = y1 , y2 (0) = 0 2
9

The solution curve is an ellipse that is oriented counterclockwise:

2 2
y1 (t) = 3 cos ( t) , y2 (t) = 2 sin ( t) .
3 3

b) Rewriting the initial value problem y2

2
x +
52 4
x + x = 0, x(0) = 3, x (0) = 0 .
255 9 1
yields the linear system
3 2 1 1 2 3 y1
y1 = y2 , y1 (0) = 3 1
4 52
y2 = y1 y2 , y2 (0) = 0 2
9 255

The solution curve is a spiral that tends to the origin:

y1 (t) = 3 e 255 t cos ( y2 (t) = 3 e 255 t (

26 168 26 26 168 168 168
t) , cos ( t) sin ( t)) .
255 255 255 255 255

Finally, we introduce the definition of stability. Stability of a solution in general means that
nearby solutions always converge towards the stable solution. This principle of stability is
especially important for equilibrium solutions.

Definition 2.11 (Stability of an equilibrium solution)

An equilibrium solution y(t) = y 0 of an autonomous system is called stable if any
solution in the neighborhood of y 0 will always approach this equilibrium solution as
t . In this case y(t) y 0 for t . Otherwise the equilibrium solution is called
unstable.
 17

2.1.2 Numerical Basics

This section is about the very basics of numerical solutions for ordinary differential equa-
tions. Many numerical methods generate sequences of points as solutions. The numerical
solution is calculated only at discrete points.

Definition 2.12 (Discretization)

The transformation of a continuous solution y
y(t) for t [0, ) into a discrete sequence of wk
points (yk ) for k N0 at points in time tk with y(t)

yk = y(tk ) wk

is called discretization. If the points in time yk = y(tk )

tk are equidistant with h > 0 then tk = hk.
Numerical approximations of y are denoted by t
w.

The error of numerical solutions has two main sources. Firstly, any numerical algorithm
is an approximation scheme. For instance, derivatives can be substituted by difference
formulas. Such discretizations cause truncation errors. Secondly, on computers, numbers
are stored with a fixed number of digits. Any floating-point operation thus causes roundoff
d
errors. As a rule of thumb, there optimal step size is h0 = 10 2 , where d is the number
of digits. On general purpose personal computers, d = 16 and thus h0 = 108 . For many
real-world applications, step sizes between 105 and h = 103 yield solutions that are
sufficiently accurate.

Definition 2.13 (Truncation error, roundoff error)

The error of a numerical method caused by 



 

discretization is called truncation error, 
    

the limited machine precision is called    

roundoff error.
Both errors depend on the step size h and con-
tribute to the total error.  

The approximation quality of numerical solutions is a central issue in the context of initial
value problems. The most important error is the global error that describes the difference
between the exact and the numerical solution. In most cases, this global error is very
hard to calculate or estimate. As a substitute, the local error is defined as the difference
between numerical and exact solution in one single step. This local error is much easier
18

to compute.

Definition 2.14 (Local and global truncation error)

Given a numerical integration method.
The local truncation error E(t, h) at time t with step size h is defined as the
difference between the numerical method applied to the exact solution and the
exact solution itself.
The global truncation error EG (t, h) at time t with step size h is defined as the
difference between the numerical solution w(t, h) and the exact solution y(t):

EG (t, h) = w(t, h) y(t) .

A numerical integration method should be at least consistent. This minimum requirement

ensures that the numerical approximation solves the given problem and not any other
one. The order of a numerical integration method describes the speed of convergence. In
general, the higher the order, the more accurate is the solution.

Definition 2.15 (Consistence, convergence, order)

A numerical integration method with step size h, global truncation error EG (t, h) and
local truncation error E(t, h) is called
consistent if the local truncation error devided by the step size tends to zero:

1
E(t, h) 0 (h 0) t ,
h

of order p > 0 if the local truncation error is bounded by h(p+1) :

E(t, h) = O(hp+1 ) t ,

convergent if the global truncation error tends to zero:

EG (t, h) 0 (h 0) t .
 19

2.2 Eulers Method and Variants

In this section, we discuss simple numerical integration schemes. They are easy to under-
stand, quick to program, and helpful to get a first impression of the solution. Of course,
quite small step sizes are necessary to achieve a reasonable precision.

2.2.1 Eulers Method

The simplest approach to solve a first-order ordinary differential equation numerically is
to substitute the derivative by the forward difference formula:
y(t + h) y(t)
y (t) .
h
Let wk denote the numerical solution at tk = kh with step size h. Then, inserting the
forward difference formula into the differential equation yields
wk+1 wk
= f (tk , wk ) .
h
Resolving this equation with respect to the next iteration wk+1 is called Eulers method.

Definition 2.16 (Eulers method)

Let y = f (t, y) be a first-order differential
equation. Then, Eulers method generates a
solution (wk ) at points in time tk = t0 +hk with
step size h beginning with w0 = y0 and using


wk+1 = wk + h f (tk , wk )



 

for k N0 . The increment function is the
slope at the left point of each interval.

To investigate the order of Eulers approach, we discuss the local truncation error. We
insert the exact solution y into Eulers formula and get
E(t, h) = y(t) + hf (t, y(t)) y(t + h) .

Euler exact
Now we use Taylor series expansion and obtain

E(t, h) = (y(t) + hf (t, y(t))) (y(t) + hy (t) + O(h2 )) = O(h2 ) .

Since y (t) = f (t, y) the leading expressions disappear.

20

Theorem 2.2 (Eulers method)

Eulers method is consistent and of order 1. Each iteration step requires 1 function
evaluation.

2.2.2 Implicit Eulers Method

The idea of the implicit variant of Eulers method is to use the backward difference
formula:
y(t + h) y(t)
y (t + h) .
h
Let wk denote the numerical solution at tk = kh with step size h. Then, inserting the
backward difference formula into the differential equation yields
wk+1 wk
= f (tk+1 , wk+1 ) .
h
This formula is called Eulers method.

Definition 2.17 (Implicit Eulers method)

Let y = f (t, y) be a first-order differential equation. Then, the implicit Eulers
method generates a solution (wk ) at points in time tk = t0 + hk with step size h
beginning with w0 = y0 and using

wk+1 = wk + h f (tk+1 , wk+1 )

for k N0 . The increment function is the slope at the right point of each interval.

We discuss the order of the implicit Euler: Inserting the exact solution y into the formula
for the local truncation error yields
E(t, h) = y(t) + hf (t + h, y(t + h)) y(t + h) .

implicit Euler exact
Now, we use Taylor series expansion and obtain

E(t, h) = (y(t) + h (f (t, y(t)) + O(h)) ) (y(t) + hy (t) + O(h2 )) = O(h2 ) .

Since y (t) = f (t, y) the leading expressions disappear.

Theorem 2.3 (Implicit Eulers method)

The implicit Eulers method is consistent and of order 1. Each iteration step requires
the solution of a nonlinear equation. In contrast to explicit methods this method has
a large region of stabiliy.
 21

2.2.3 Improved Eulers Method

Eulers method uses the slope at the end points of the interval [t, t + h]. To improve
accuracy, the slope at the midpoint of this interval can be used as well. This gives an
improved version of Euler.

Definition 2.18 (Improved Eulers method)

Let y = f (t, y) be a first-order differen-

tial equation. Then, the improved Eulers  

method generates a solution (wk ) at points
in time tk = t0 + hk with step size h beginning
with w0 = y0 and using  
 

K1 = f (tk , wk )

K2 = f (tk + 21 h, wk + 12 h K1 )        
 
wk+1 = wk + h K2


for k N0 . The increment function is the slope at the midpoint of each interval.

To discuss the order of the improved Euler, we insert the exact solution y into the formula
of the local truncation error:
1 1
E(t, h) = y(t) + hf (t + h, y(t + h) + hf (t, y(t))) y(t + h) .
2 2
exact
improved Euler

For simplicity, we now omit the argument (t) for function y and the arguments (t, y(t))
for function f and its derivatives. Again, using Taylor series expansion yields
1
E(t, h) = h (ft + fy (f + O(h))) + O(h2 )))
(y + h (f +
2
1
(y + hy + h2 y + O(h3 )) = O(h3 ) .
2
The derivative of y (t) = f (t, y(t)) with respect to t is obtained by the general chain rule.
Since y = f and y = ft + fy f , the leading expressions disappear.

Theorem 2.4 (Improved Eulers method)

The improved Eulers method is consistent and of order 2. Each iteration step requires
2 function evaluations.
22

2.2.4 Heuns Method

The idea of Heuns approach is to average both slopes at the end points of the inter-
val [t, t + h]. This also gives an improved version of Euler. The precision achieved is
comparable to that of the improved Eulers method.

Definition 2.19 (Heuns method)

Let y = f (t, y) be a first-order differential

equation. Then, Heuns method generates a  

solution (wk ) at points in time tk = t0 +hk with
step size h beginning with w0 = y0 and using
 
K1 = f (tk , wk )  

K2 = f (tk + h, wk + h K1 )
   
 
wk+1 = wk + h ( 21 K1 + K2 ) 1
2


for k N0 . The increment function is the average of the slopes at the left and right
point of each interval.

The formula of the local truncation error has to used to proof the order of Heuns method:
1 1
E(t, h) = y(t) + h ( f (t, y(t)) + f (t + h, y(t) + hf (t, y(t)))) y(t + h) .
2 2
exact
Heun
Again, for simplicity, we now omit the arguments (t) and (t, y(t)). Using Taylor series
expansion yields
1 1
E(t, h) = hf + h (f + h (ft + fy (f + O(h))) + O(h2 )))
(y +
2 2
1 2
(y + hy + h y + O(h3 )) = O(h3 ) .

2
The derivative of y (t) = f (t, y(t)) with respect to t is obtained by the general chain rule.
Since y = f and y = ft + fy f , the leading expressions disappear.

Theorem 2.5 (Heuns method)

Heuns method is consistent and of order 2. Each iteration step requires 2 function
evaluations.
 23

2.3 Runge-Kutta Methods

Runge-Kutta methods represent a very useful and popular class of one-step methods.

2.3.1 Runge-Kutta Methods

Definition 2.20 (Classical Runge-Kutta method)

Let y = f (t, y) be a first-order differential equation. Then, the classical Runge-
Kutta method generates a solution (wk ) at points in time tk = t0 + hk with step size
h beginning with w0 = y0 and using  
K1 = f (tk , wk )
  
K2 = f (tk + 12 h, wk + 21 h K1 ) 
 
K3 = f (tk + 21 h, wk + 21 h K2 )
 


  
K4 = f (tk + h, wk + h K3 )


 
wk+1 = wk +h ( 16 K1 + 62 K2 + 26 K3 + 61 K4 )


for k N0 . The increment function is an average of two slopes at the left and right
point and two slopes at the midpoint of each interval.

Theorem 2.6 (Classical Runge-Kutta method)

The classical Runge-Kutta method is consistent and of order 4. Each iteration step
requires 4 function evaluations.
24

Definition 2.21 (General Runge-Kutta method)

Let y = f (t, y) be a first-order differential equation. Then, the r-stage general Runge-
Kutta method generates a solution (wk ) at points in time tk = t0 + hk with step size
h beginning with w0 = y0 and using
i1
Ki = f (t + ai h, y + h bi,j Kj )
j=1
r
wk+1 = wk + h ci Ki
i=1


for k N0 . The coefficients can be listed in a table:

a1
a2 b2,1
a3 b3,1 b3,2

ar br,1 br,2 ... br,r1
c1 c2 ... cr1 cr

Theorem 2.7 (General Runge-Kutta method)

The maximum order p of a general Runge-Kutta method depends on the stages r:

stage r 1 2 3 4 5 6 7 8 9
order p 1 2 3 4 4 5 6 6 7

Special Runge-Kutta methods of order 2

The coefficient tables of special Runge-Kutta methods of order 2 are:

0 0
1 1
2 2
1 1
1 1
0 1 2 2

Improved Eulers method Heuns method

 25

Special Runge-Kutta methods of order 3

The coefficient tables of special Runge-Kutta methods of order 3 are:

0
1 1
2 2
1 1 2
1 4 1
6 6 6

Kuttas method

Special Runge-Kutta methods of order 4

The coefficient tables of special Runge-Kutta methods of order 4 are:

0 0
1 1 1 1
2 2 3 3
1
2
0 1
2
2
3
13 1
1 0 0 1 1 1 1 1
1 2 2 1 1 3 3 1
6 6 6 6 8 8 8 8

Classical RK method 3/8 method

26

2.3.2 Step Size Control

Definition 2.22 (Step size control)

The principle of step size control can be applied using two numerical methods of
different order or of different discretization. The global error can be estimated by the
difference of two numerical solutions (wk ) and (wk ):

E = wk+1 wk+1 .

The step size h is adapted in dependence of a given error bound :

If E > h then redo the computation of wk+1 with 1
2
h.

If E < 1
4
h then accept wk+1 and set the step size to 2h for the next step.
Otherwise accept wk+1 and continue.

The numerical solution (wk+1 ) is used only to control the step size.

Special embedded Runge-Kutta methods of order 2/3

The coefficient tables of special embedded Runge-Kutta methods of order 2/3 are:

0 0
1 1
2 2
1 1
1 1 2 1
2
1
4
1
4
1 1
0 1 2 2
1 4 1 1 1 4
6 6 6 6 6 6

Embedded improved Eulers method Embedded Heuns method

 27

Definition 2.23 (Runge-Kutta-Fehlberg method)

Let y = f (t, y) be a first-order differential equation. Then, the embedded 6-stage
Runge-Kutta-Fehlberg method of order 4/5 with step size control generates a solu-
tion (wk ) using the coefficients

0
1 1
4 4
3 3 9
8 32 32
12
13
1932
2197
7200
2197
7296
2197
1 439
216
8 3680
513
4104
845

1
2
27
8
2 2565
3544 1859
4104
40
11

25
216
0 1408
2565
2197
4104
51
16
135
0 6656
12825
28561
56430
50
9 2
55

2.3.3 The One-Step Approach

Definition 2.24 (One-step method, increment function)

A numerical integration method that generates a solution (wk ) at points in time (tk )
with step size h of the form

a0 wk + a1 wk+1 = h (tk , tk+1 , wk , wk+1 )

with coefficients ai is called one-step method. The function is called increment

function.
28

2.4 Multistep Methods

Runge-Kutta methods all have in common that they use only the last iterate to calculate
the next one. As an alternative, more than one old value can integrated into a numerical
algorithm. This idea leads to multistep methods. They use several old iterates to compute
the next one.

2.4.1 Linear Multistep Methods

The special approach of linear multistep methods uses a linear combination of the function
values f (tk+i , wk+i ) to build up the incremental function. It is important to mention the
fact, that linear multistep methods are not restricted to linear differential equations. The
linearity refers to the function values in the formula.

Definition 2.25 (Linear multistep methods)

Let y = f (t, y) be a first-order differential equation. Then, a linear multistep method
or linear m-step method generates a solution (wk ) at points in time tk = t0 + hk with
step size h beginning with w0 = y0 and using
m m
ai wk+i = h bi f (tk+i , wk+i )
i=0 i=0

for k N0 . The coefficients with a20 + b20 0 and a2m + b2m 0 can be listed in a table:

a0 a1 ... am
b0 b1 ... bm

A method is explicit if bm = 0 and implicit otherwise. The first iterates w1 , . . . , wm1

are computed with a one-step method.

Runge-Kutta methods can be compared with multistep methods. Characteristical features

are the number of function calls, the achieved order, or the possibility to adapt the step
size. Since multistep methods need several historic values, they always are accomplished
with one-step methods with which the first iterates are computed.
 29

Linear multistep methods

Linear multistep methods have some characteristics in common:
The general advantages are high order and high accuracy with few computational
effort.
The general disadvantage are the limitation to constant step size and the necessity
for seed computation for the first iterates.

2.4.2 Adams-Bashforth Methods

A very popular class of linear multistep variants are the so-called Adams-Bashforth meth-
ods. They are named after the English mathematicians John Couch Adams and Francis
Bashforth. They all have in common that only the last two coefficients am1 = am = 1
are nonzero. Moreover, this class contains only explicit methods with bm = 0.

Definition 2.26 (Adams-Bashforth methods)

Explicit linear multistep methods with

a0 = a1 = . . . = am2 = 0 , am1 = 1 , am = 1 , bm = 0

are called Adams-Bashforth methods (AB).

Investigating the local truncation error a set of possible method variants can be derived.
The achievable order depends on the number of included old values. Eulers method can
be regarded as a special Adams-Bashforth method. The general approach for m is

wk+2 = wk+1 + h(b0 f (tk , wk ) + b1 f (tk+1 , wk+1 )) .

Inserting this approach into the local truncation error formula yields

E = y(t + h) + h(b0 f (t, y(t)) + b1 f (t + h, y(t + h))) y(t + 2h) .

A Taylor expansion with center point t + h leads to the error representation

1 2
E = y + h(b0 (f h(ft + fy f ) + O(h2 )) + b1 f ) (y + hy + h y + O(h3 )) .
2
Here, the arguments (t) and (t, y(t)) are omitted. Comparing all expressions with y = f
and y = ft + fy f , respectively, the following equations result:

1
hb0 + hb1 h = 0 , h2 b0 h2 = 0 .
2
30

Dividing by h and h2 and rearranging the expressions yields

1
b0 + b1 = 1 , b0 = .
2
This system of linear equations has the solution b0 = 21 und b1 = 32 .

Adams-Bashforth methods
The coefficients of m-step Adams-Bashforth methods together with the order p are:

m am1 am b0 b1 b2 b3 b4 b5 p
1 1 1 1 0 1
2 1 1 12 3
2
0 2
3 1 1 5
12
16
12
23
12
0 3
4 1 1 24
9 37
24
24
59 55
24
0 4
5 1 1 251
720
1274
720
2616
720
2774
720
1901
720
0 5

The special case p = 1 is Eulers method.

Example 2.10 (Adams-Bashforth Methods)

a) For m = 1, we obtain Eulers method as a special case of an Adams-Bashforth variant:

wk+1 = wk + hf (tk , wk ) .

b) For m = 2, a possible Adams-Bashforth variant with p = 2 is

1 3
wk+2 = wk+1 + h ( f (tk , wk ) + f (tk+1 , wk+1 )) .
2 2

2.4.3 Adams-Moulton Methods

The implicit analogon to Adams-bashforth methods are Adams-Moulton methods. This
class of multistep variants are named after the English and American mathematicians
John Couch Adams and Forest Ray Moulton. Again, only the last two coefficients
am1 = am = 1 are nonzero. In contrast, now bm 0, which implies implicit formulas.

Definition 2.27 (Adams-Moulton methods)

Implicit linear multistep methods with

a0 = a1 = . . . = am2 = 0 , am1 = 1 , am = 1 , bm 0

are called Adams-Moulton methods (AM).

 31

To get appropriate coefficients bk , the local truncation error is stated with the objective to
cancel out as many leading expressions as possible. As in the explicit case, the achievable
order depends on the number of included old values. The implicit Eulers method is a
special Adams-Moulton method.

Adams-Moulton methods
The coefficients of m-step Adams-Moulton methods together with the order p are:

m am1 am b0 b1 b2 b3 b4 p
1 1 1 0 1 1
1 1 1 1
2
1
2
2
2 1 1 12
1 8
12
5
12
3
3 1 1 1
24
24
5 19
24
9
24
4
4 1 1 720
19 106
720
264
720
646
720
251
720
5

The special case p = 1 is the implicit Eulers method. The special case p = 2 is the
trapezoidal method.

Example 2.11 (Adams-Moulton Methods)

a) For m = 1, we obtain the implicit Eulers method of order 1:

wk+1 = wk + hf (tk , wk+1 ) .

b) Again for m = 1, there is a another Adams-Moulton method. This method is called trape-
zoidal method and has order 2:
1 1
wk+1 = wk + h ( f (tk , wk ) + f (tk+1 , wk+1 )) .
2 2

2.4.4 Predictor-Corrector Methods

Definition 2.28 (Predictor corrector methods)

Multistep predictor corrector methods (PC) or Adams-Bashforth-Moulton meth-
ods (ABM) are defined as follows:
(1) Compute predictor wk+m

with explicit Adams-Bashforth m-step method.
(2) Compute corrector wk+m with implicit Adams-Moulton m 1-step method using
wk+m

in the right-hand side of the Adams-Moulton scheme.
32

Predictor corrector methods

The advantage of predictor corrector methods is that there is no need for rootfinding
subroutines. Using the predictor in the right-hand side of the scheme transforms a
nonlinear problem into a linear one.

2.4.5 The General Multistep Approach

The linear multistep approach can be generalized into a nonlinear variant. Then, the
right-hand side of the multistep method formula has nonlinear incremental function of
any possible form.

Definition 2.29 (Multistep method, increment function)

A numerical integration method that generates a solution (wk ) at points in time (tk )
with step size h of the form
m
ai wk+i = h (tk , . . . , tk+m , wk , . . . , wk+m )
i=0

with coefficients ai is called multistep method or m-step method. The function

is called increment function.
 33

2.5 Stability and Stiffness

There are several aspects that contribute to a good numerical solution. Of course, pre-
cision and accuracy are among these aspects. Moreover, a weaker requirement is that
a numerical solution shows the same tendency as the exact solution. Stiff differential
equations are a special challenge for numerical integration methods. They require small
iteration steps although the solution is quite smooth.

2.5.1 Stability of One-Step Methods

If the exact solution of a differential equation tends to zero, the same behavior is expected
from the numerical solution. This requirement ensures that the numerical method is
consistent with the differential equation and thus behaves stable. In general, stability is
achieved only for step sizes below critical bounds. To investigate the issue of stability, we
consider the simple linear test differential equation

y = y , C.

Since most real-world models embody friction or damping, the interesting case is Re() < 0.
Applying a one-step method with step size h to this equation generates a recursive formula
of the form

wk+1 = g(h)wk .

for the numerical solution wk . Here, g(z) denotes some complex function with a complex
argument z = h C. If Re() < 0, the exact solution y(t) tends to zero. The numerical
solution wk has the same behavior, if g(z) < 1.

Definition 2.30 (Region of stability for one-step methods)

Consider the test differential equation y = y. For a one-step method with a numerical
solution (wk ) and a step size h of the form

wk+1 = g(h)wk

the region of stability is defined as that part of the complex plane where g is con-
tracting:

G = {z C g(z) < 1} .

Within G and the left-half plane the numerical solution has the same quantitative
behavior as the exact solution.
34

Boundary of region of stability for one-step methods

The boundary of the region of stability consists of all solutions z of the equations

g(z) = ei , 0 < 2 ,

where g results from the one-step method applied to the test differential equation.

Example 2.12 (Stability of Eulers method)

Applying Eulers method

wk+1 = wk + h f (tk , wk )

to the test differential equation y = y yields

wk+1 = wk + hwk = (1 + h)wk .

Using z = h, the stability function reads

g(z) = 1 + z .

The boundary of the region of stability is defined as g(z) = 1. Since all complex points on the
unit circle are of the form ei , we get

1 + z = ei z = 1 + ei .

With [0, 2] the boundary of the region of stability describes a circle with center 1 and
radius 1. The region of stability lies within the circle.
 35

Example 2.13 (Stability of improved Eulers and Heuns method)

a) Applying the improved Eulers method
1 1
wk+1 = wk + h f (tk + h, wk + h f (tk , wk ))
2 2
to the test differential equation y = y yields
1 1 1
wk+1 = wk + h (wk + h f (tk , wk )) = wk + h (wk + hwk ) = (1 + h + h2 2 ) wk .
2 2 2
Using z = h, the stability function reads
1 2
g(z) = 1 + z + z .
2
The boundary of the region of stability is defined as g(z) = 1. Since all complex points on
the unit circle are of the form ei , we get
1 2
1+z+ z = ei z1,2 = 1 2 ei 1 .
2
With [0, 2] the boundary of the region of stability describes a closed curve which looks
like an oval. For special values we obtain

= 0 z = 2, 0 , = z = 1 3i .

b) Applying Eulers method

1 1
wk+1 = wk + h ( f (tk , wk ) + f (tk + h, wk + hf (tk , wk)))
2 2
to the test differential equation y = y yields
1 1 1
wk+1 = wk + h ( wk + (wk + hwk )) = (1 + h + h2 2 ) wk .
2 2 2
Hence, the stability function of Heuns method
1 2
g(z) = 1 + z + z .
2
is the same as for the improved Eulers method.
36

Example 2.14 (Stability of Kuttas method)

Applying Kuttas method to the test differential equation y = y yields

K1 = f (tk , wk ) = wk

K2 = f (tk + 1
2
h, wk + h 12 K1 ) = (1 + 1
2
h) wk

K3 = f (tk + h, wk + h(K1 + 2K2 )) = (1 + h + h2 2 ) wk .

All together we obtain

wk+1 = wk + h( 61 wk + 46 (1 + 1
2
h) wk + 16 (1 + h + h2 2 ) wk )

= (1 + h + 1
2
h2 2 + 1
6
h3 3 ) wk .

Using z = h, the stability function reads

1 2 1 3
g(z) = 1 + z + z + z .
2 6
The boundary of the region of stability is defined as g(z) = 1. Since all complex points on the
unit circle are of the form ei , we get
1 2 1 3
1+z+ z + z = ei .
2 6
This equation can be solved either numerically or with a formula for the roots of cubic polynomials.
With [0, 2] the boundary of the region of stability describes a closed curve in the complex
plane.

Example 2.15 (Stability of classical Runge-Kuttas method)

Applying the classical Runge-Kuttas method to the test differential equation y = y yields the
stability function
1 2 1 3 1 4
g(z) = 1 + z + z + z + z .
2 6 24
The boundary of the region of stability is defined as g(z) = 1. Since all complex points on the
unit circle are of the form ei , we get
1 2 1 3 1 4
1+z+ z + z + z = ei .
2 6 24
This equation can be solved either numerically of with a formula for the roots of cubic polynomials.
With [0, 2] the boundary of the region of stability describes a closed curve in the complex
plane.
 37

Stability of Runge-Kutta methods

Runge-Kutta methods are explicit methods. Im RK4
The regions of stability are bounded and cover RK3
only small parts of the left-half plane. RK2
i
RK1: Euler RK1
6 5 4 3 2 1 1 Re
RK2: Improved Euler and Heun i

RK3: Kutta
RK4: Classical Runge-Kutta

In terms of stability, good numerical methods cover a wide range of the left-half plane.
There are a couple of methods that cover the complete left-half plane. Such methods are
called A-stable. They are optimal with respect to stability.

Definition 2.31 (A-stability)

An integration method is called A-stable if the region of stability G covers the complete
left-half plane:

G {z C Re(z) < 0} .

Example 2.16 (Stability of implicit Eulers method)

Applying Eulers method

wk+1 = wk + h f (tk+1 , wk+1 )

to the test differential equation y = y yields

1
wk+1 = wk + hwk+1 wk+1 = wk .
1 h
Using z = h, the stability function reads
1
g(z) = .
1z
The boundary of the region of stability is defined as g(z) = 1. Since all complex points on the
unit circle are of the form ei , we get

z = 1 ei .
1
= ei
1z
With [0, 2] the boundary of the region of stability describes a circle with center 1 and radius
1. The region of stability lies outside the circle and thus covers the whole left half plane. The
implicit Eulers method is A-stable. This is a big advantage of this approach.
38

Example 2.17 (Stability of trapezoidal method with one-step definition)

Applying the trapezoidal method
1 1
wk+1 = wk + h ( f (tk , wk ) + f (tk+1 , wk+1 ))
2 2
to the test differential equation y = y yields
1 1 2 + h
wk+1 = wk + h ( wk + wk+1 ) wk+1 = wk .
2 2 2 h
Using z = h, the stability function reads
2+z
g(z) = .
2z
The boundary of the region of stability is defined as g(z) = 1. Since all complex points on the
unit circle are of the form ei , we get

2+z ei 1 2 sin
= ei z=2 = i.
2z ei + 1 1 + cos
The last equation results from expanding the fraction with the complex conjugate of the denom-
inator ei + 1 and using the relations

(e ei ) (e + ei ) .
1 i 1 i
sin = , cos =
2i 2
With [0, 2] the boundary of the region of stability is the imaginary axis. The region of
stability lies left to the imaginary axis and thus covers the whole left half plane. The trapezoidal
method is A-stable.

Definition 2.32 (Characteristic polynomials)

The coefficients of a multistep method can be used to build up two polynomials:
m m
() = ai i , () = bi i .
i=0 i=0

are called the first characteristic polynomial and the second characteristic poly-
nomial of the given multistep method.
 39

Example 2.18 (Characteristic polynomials of Adams methods)

a) The characteristic polynomials of the Adams-Bashforth method with m = 1 are

() = 1 + , () = 1 .

b) The characteristic polynomials of the Adams-Bashforth method with m = 2 are

1 3
() = + 2 , () = + .
2 2

c) The characteristic polynomials of the Adams-Bashforth method with m = 3 are

5 16 23
() = 2 + 3 , () = + 2 .
12 12 12

d) The characteristic polynomials of the Adams-Moulton method with m = 1 and p = 1 are

() = 1 + , () = .

e) The characteristic polynomials of the Adams-Moulton method with m = 1 and p = 2 are

1 1
() = 1 + , () = + .
2 2

f) The characteristic polynomials of the Adams-Moulton method with m = 2 are

1 8 5
() = + 2 , () = + 2 .
12 12 12

2.5.2 Stability of Multistep Methods

Once again, we consider the test differential equation

y = y , C

with Re() < 0. With this equation, a linear multistep method with step size h reads
m m m
ai wk+i = h bi wk+i (ai zbi )wk+i = 0 ,
i=0 i=0 i=0

where z = h. This is a linear difference equation for the sequence wk . The general
solution of that difference equation is based on the powers k , where are the zeros of
the polynomial
m
p() = (ai zbi )i = 0 .
i=0

Hence, the general solution tends to zero, if all zeros lie within the unit circle.
40

Definition 2.33 (Region of stability for linear multistep methods)

Consider the test differential equation y = y. For a multistep methods with a numer-
ical solution (wk ) and a step size h of the form
m m
ai wk+i = h bi wk+i
i=0 i=0

the region of stability is defined as that part of the complex plane where the stability
polynomial p has only zeros within the unit circle:
m
G = {z C p() = (ai zbi )i has only zeros with < 1 } .
i=0

Within G and the left-half plane the numerical solution has the same quantitative
behavior as the exact solution.

The boundary of the region of stability is reached if one zero crosses the unit circle:

= 1 = ei , [0, 2) .

Inserting this value into the characteristic polynomial yields the boundary points
m (ei )
(ai zbi ) = 0 z() =
i
.
i=0 (ei )

Hence, all boundary points z() are obtained by varying in the interval [0, 2).

Boundary of region of stability for linear multistep methods

Using the characteristic polynomials and of a multistep method, the boundary of
the region of stability consists of all points

(ei )
z= , 0 < 2 .
(ei )

Example 2.19 (Stability of Eulers method with multistep definition)

Using the characteristic polynomials, we obtain the boundary points of the region of stability:
1 +
z= = 1 + ei .
1
This expression is the same as in Example 2.12. Again, all points z lie on a circle with center 1
and radius 1.
 41

Stability of Adams-Bashforth methods

Adams-Bashforth methods are explicit meth- Im
ods. AB1 is Eulers method. The regions of
stability are bounded and cover very small parts AB2
of the left-half plane. With increasing order the AB1 i AB4
regions are getting even smaller. A point that
6 5 4 3 2 1 1 Re
belongs to any stability region is plotted in light i
red. AB3

Example 2.20 (Stability of trapezoidal method with multistep definition)

Using the characteristic polynomials, we obtain the boundary points of the region of stability:

1 + ei 1
z= = 2 i .
1
2
+2 e +1

This expression is the same as in Example 2.17. Again, all points z lie on the imaginary axis.
Thus, the trapezoidal method is A-stable.

Stability of Adams-Moulton methods

Adams-Moulton methods are implicit methods. Im
AM1 is the implicit Eulers method. The re- AM2
gion of stability of AM1 is the area outside the AM1
unit circle with center z = 1. The region of AM3 AM4 i
stability of AM2 is the left half of the plane.
6 5 4 3 2 1 1 Re
The regions of stability of AM3 and AM4 are i
bounded. AM1 and AM2 are A-stable. A point
that belongs to any stability region is plotted in
light red.

2.5.3 Stiffness
Stiffness describes the relation between necessary step size and solution smoothness. If
a solution behaves continuous and smooth without sudden changes of the slopes on one
hand and requests quite a small step size on the other hand, such a paradox situation is
called stiff. Stiffness is a matter of performance, not of precision. Stiffness also is a local
matter. A differential equation can behave normal in some region and stiff in another.
42

Definition 2.34 (Stiffness)

An ordinary differential equation y = f (t, y) is called stiff at (t, y)
if the solution being sought varies slowly, but there are nearby solutions that vary
rapidly, so the numerical method must take small steps to obtain satisfactory pre-
cision or
if there is an Eigenvalue of f y (t, y) with Re() << 0 or
if the Eigenvalues of f y (t, y) have very different negative parts.

Example 2.21 (Vibration equation with stiff behavior)

Using the special values 0 = 1 and = 5.05 in the y
vibration system of Example 2.6 the system reads 1 z1 (t)
0 1
y = Ay with A=( ) .
1 10.1
0.5
The characteristic polynomial of A
z2 (t)
det(A I) = 2 + 10.1 + 1
1 2 3 t
has two zeros which generate the basis functions:

1 = 0.1 , 2 = 10 z1 (t) = e0.1 t , z2 (t) = e10 t .

The corresponding eigenvector equations are

0.1 1 10 1
(A 1 I)v1 = ( ) v1 = 0, (A 2 I)v2 = ( ) v2 = 0
1 10 1 0.1

with eigenvectors

10 1
v1 = ( ) , v2 = ( ) .
1 10

Using constants C1 and C2 , we obtain the general solution

y(t) = C1 z1 (t) v 1 + C2 z2 (t) v 2 .

Since 1 and 2 are very different, the system behaves stiff. In comparison with z1 , the function
z2 contributes almost nothing to the general solution.
 43

Example 2.22 (Vibration equation solved with Euler)

We consider again the vibration equation of Exam- y
ple 2.6 with the parameters of Example 2.21. For the 1
initial value
y1 (t)
1
y(0) = ( ) , 0.5
0 h = 0.17
h = 0.23
the constants in the general solution are
10 10 1 2 3 t
C1 = , C2 = .
99 99

Using Eulers method, the limit stability number is z = h = 2. This implies

2 2 2 2
h1 = = = 20 , h2 = = = 0.2 , h = min{h1 , h2 } = h2 = 0.2 .
1 0.1 2 10
The absolute value of eigenvalue 1 is much smaller than that of eigenvalue 2 . So, the limit
step size h1 derived from 1 is much larger than the limit step size h2 derived from 2 . Although
the solution part z1 is dominant, the solution part z2 limits the step size. With h = 0.17 < h the
solution behavior is generated correctly. In contrast, with h = 0.23 > h, strong oscillations occur
in the numerical solution. This effect is called stiffness.
44

2.6 Initial Value Problems

Initial value problems can be solved numerically in many different ways. Most popular
methods integrate from the initial point in time to the end point step by step. The
resulting solution is a sequence of approximation points.

2.6.1 Summary
Initial value problems can be solved numerically in many different ways. Most popular
methods integrate from the initial point in time to the end point step by step. The
resulting solution is a sequence of approximation points.

2.6.2 Matlab Functions

Matlab provides a variety of built-in functions to solve ordinary differential euqations with
given initial values. These functions differ in precision, efficiency, and applicability to stiff
problems. One-step, multistep, explicit, and implicit methods all have their advantages
and disadvantages.

MATLAB solvers
MATLAB has a variety of built-in solvers for initial value problems:
solver problem type type of algorithm
ode23 nonstiff explicit Runge-Kutta pair, orders 2 and 3
ode45 nonstiff explicit Runge-Kutta pair, orders 4 and 5 (workhorse!)
ode113 nonstiff explicit linear multistep, orders 1 to 13
ode15s stiff implicit linear multistep, orders 1 to 5
ode15i fully implicit implicit linear multistep, orders 1 to 5
ode23s stiff modified Rosenbrock pair (one-step), orders 2 and 3
ode23t mildly stiff trapeziodal rule (implicit), orders 2 and 3
ode23tb stiff implicit Runge-Kutta-type algorithm, orders 2 and 3
 45

2.7 Boundary Value Problems

Ordinary differential equations appear together with initial or boundary values. If infor-
mation about the solution is known at at least two points, the problem is called boundary
value problems. Calculating the bend of a fixed rod with load is a classical boundary value
problem. Another famous application is the re-entry problem of a space shuttle from the
orbit into the atmosphere.

2.7.1 Shooting Methods

Numerical methods for initial value problems can be used for boundary value problems as
well. Typically, the slope at the initial point is varied. This slope is adjusted such that
the difference between the numerical solution at the end point and the given boundary
value becomes as small as possible.

Definition 2.35 (Shooting method)

Boundary value problems can be solved with shooting methods:
(1) Transform the boundary value problem

y = f (t, y, y ) , y(a) = , y(b) =

into an initial value problem:

y = f (t, y, y ) , y(a) = , y (a) = s .

(2) Define the shooting function

g(s) = y(b, s)

(3) Find a root of the function g. Use Newtons method or the secant formula.

2.7.2 Finite Difference Methods

The idea of the finite difference approach to boundary value problems is to discretize
the integration interval and substitute the derivatives by difference formulas. Typically,
this yields systems of linear equations. The solution at all discrete points is calculated
simultaneously. this is in contrast to the shooting approach, where the solution points are
calculated from the starting point to the end point in an iterative process.
46

Definition 2.36 (Finite difference method)

Boundary value problems can be solved with finite difference methods:
(1) State the boundary value problem

y = f (t, y, y ) , y(a) = , y(b) =

(2) Set up finite differences for y and y with step size h:

y(tk+1 ) y(tk1 )
y (tk ) = + O(h2 )
2h
y(tk+1 ) 2y(tk ) + y(tk1 )
y (tk ) = + O(h2 )
h2

(3) Discretize the problem into n = bah

intervals and substitute y(tk ) in the fi-
nite difference formulas by wk . Solve the resulting system of linear equations
in w0 , . . . , wn .
 47

2.8 Applications
2.8.1 Biology
Example 2.23 (Predator-Prey model)
The Lotka-Volterra predator-prey model describes the relation between two populations. One
population behaves as predator the other as prey. The model consists of a two-dimensional
system of first-order differential equations. The 4 parameters are a, b, c, d > 0.

x = ax + bxy
y = cy + dxy

48

Example 2.24 (Competing population model)

The competing population model describes the two populations that compete in their living
environment. The model consists of a two-dimensional system of first-order differential equations.
The parameter r > 0 describes the effect of the growth of one species on the growth of the other
species.

x = x x2 rxy
y = y y 2 rxy

First, we discuss the case r = 0.5:

x-nullclines: x = 0 and y = 2x + 2
y-nullclines: y = 0 and y = 12 x + 1
Equilibrium solution: (0, 0), (1, 0), (0, 1) and ( 32 , 23 ) (stable)

1.5

0.5

0 0.5 1 1.5 2
Second, we discuss the case r = 2:
x-nullclines: x = 0 and y = 21 x + 1
2
y-nullclines: y = 0 and y = 2x + 1
Equilibrium solution: (0, 0), (1, 0), (0, 1) and ( 31 , 13 ) (unstable)

0.8

0.6

0.4

0.2

0.2
0.2 0 0.2 0.4 0.6 0.8 1 1.2

 49

2.8.2 Mechanics
Example 2.25 (Spring damper oscillator)
Let x denote the deflection of a ball with mass m F(t)
and c and k denote the spring and damper coefficient,
respectively. Let F (t) be an external force. Newtons c
law on the balance of forces states:

mx + kx + cx = F (t) . m
x(t)
Using the new states k
y1 = x, y2 = x

the second-order differential equation can be transformed into a linear first-order system:

y1 0 1 0
1 + 1
y
=

y2
c k F (t)
y2 m
m m

Example 2.26 (Coupled oscillator)

Let x1 and x2 denote the deflection of two balls with
masses m1 and m2 and c1 , c2 , and c3 denote the
spring coefficients. According to Newtons law, we
can model the system with two differential equations: c1 m1 c2 m2 c3

m1 x1 + x1 c1 + (x1 x2 )c2 = 0
m2 x2 + x2 c3 + (x2 x1 )c2 = 0

Using new states x1 x2

y1 = x1 , y2 = x2 , y3 = x1 , y4 = x2

this second-order system can be written as a linear first-order system with constant coefficients:

0 0 1 0
y 1
y1
y 1
y2
0 0 0
2
= c +c
c2
y3 1 2
0 0 y3

y4
m1 m1
c2 c2 + c3 y4
0 0
m2 m2

50

2.8.3 Engineering
Example 2.27 (Liquid tank)
The model describes the height of a liquid in a spherical tank. The model consists of a first-order
differential equation. The parameter r > 0 describes the tank radius. The parameter cd denotes
a liquid dependent parameter (water: 0.6). The parameter A describes the hole area.

h =
cd A 2gh
2rh h2
For example we can set r = 1.5 m, cd = 0.6, and A = 0.0152 m2 .
3

2.5

2
h [m]

1.5

0.5

0
0 500 1000 1500 2000 2500
time [s]

 51

2.8.4 Vehicle Dynamics

Example 2.28 (Quarter car model with one mass)
The model describes a simple quarter car with one mass:

Source: Mitschke

The model consists of a second-order linear differential equation. The state variables are

z
y=( ) .
z

The model oriented differential equation reads

mz + kz + cz = kh + ch .

The numerical solver oriented differential equation reads

y = (
0 1 0
k )y + ( )
mc
m c
m
h+ m
k
h

52

Example 2.29 (Quarter car model with three masses)

The model describes a simple quarter car with three coupled masses:

Source: Mitschke

The model consists of a system of three second-order linear differential equations. The differential
equations are

m1 z1 k2 (z2 z1 ) c2 (z2 z1 ) + c1 z1 = c1 h
m2 z2 k3 (z3 z2 ) c3 (z3 z2 ) + k2 (z2 z1 ) + c2 (z2 z1 ) = 0
m3 z3 + k3 (z3 z2 ) + c3 (z3 z2 ) = 0

The state variables are

z1
z2
z1
z z3
z = z2 , y=( )=



z3 z z1

z2
z3

The mass matrix, the right-hand side matrix and the amplification by the road read

m1 0 0 c1 0 0 h1
M = 0 m2 0 , R= 0 0 0 , h= 0 .
0 0 m3 0 0 0 0

The spring and damper matrices read

k2 k2 0 c1 + c2 c2 0
K = k2 k2 + k3 k3 , C = c2 c2 + c3 c3 .
0 k3 k3 0 c3 c3

Thus, the model oriented differential equation reads

M z + K z + Cz = Rh .

The solver-oriented differential equation reads

y = (
0 E 0
)y + ( ) .
M 1 C M 1 K M 1 Rh

 53

2.9 Exercises
Theoretical Exercises
Exercise 2.1
For each ODE that ist given, determine its order, check if it is linear, and check if the given
functions are solutions.
a) y = 2t with y(t) = t2 + c.
t
x2
b) y = 2ty + 1 with y(t) = et
2 2
e dx + et .

0

c) y + 2y 3y = 0 with y1 (t) = e3t and y2 (t) = et .

d) y + 4y + 3y = t with y1 (t) = and y2 (t) = et + .

t t
3 3

Exercise 2.2
Consider the ODE

y 3y + 2y = t .

a) Transform the ODE into a first-order system.

b) Transform the ODE into an autonomous first-order system.

Exercise 2.3
Reformulate each of the following IVPs below into a new IVP which involves a first-order system.
a) y + y et y = sin(3t) with y(0) = 1, y (0) = 2, y (0) = 3.

b) x = xy + t2 + 1 and y = x cos y with x(10) = 4, x (10) = 1 and y(10) = 1.

Exercise 2.4
Suppose we had three different methods M1, M2, and M3 for solving IPVs which had orders 1,
2, and 4, respectively. Suppose also that it is known that for a certain IVP, the constant c of the
global error bound

y(tk ) wk chp , k

could be taken to be 2 for all three methods.

a) Find the resulting error bounds for each of the three methods using a step size h = 0.1.
b) Find the resulting error bounds for each of the three methods using a step size h = 0.05.
c) Compare the results.
54

Exercise 2.5
Compute the region of stability for the trapezoid method
1 1
wk+1 = wk + h ( f (tk , wk ) + f (tk+1 , wk+1 )) .
2 2

Computational Exercises
Exercise 2.6
Consider the IVP

y = y, y(t0 ) = y0 , y (t0 ) = y0

and the BVP

y = y, y(t0 ) = y0 , y(t1 ) = y1 .

a) Find the general solution of the differential equation.

b) MATLAB: Set up the IVP solution and draw a graph for t [0, 10].
c) MATLAB: Check if a BVP solution exists. In that case draw a graph for t [0, 10].

Exercise 2.7
MATLAB: Plot the vector fields of the following ODEs for the range [5, 5] [5, 5]. Describe
the solution curves.
a) y =
y
b) y =
t
c) y = 10
sin t
t y 1 10y

Exercise 2.8
MATLAB: Consider the IVP

y = sin(t2 ), y(0) = 1 .

a) Plot the solution of the IVP for t [0, 5] using the function quad with stepsize 0.01.
b) Which element of the discrete solution vector y(k) gives y(3)?
c) What is the value of y(3)?
 55

Exercise 2.9
Consider the simple IVP

y = y, y(0) = 1 .

a) Determine the exact solution.

b) MATLAB: Implement Eulers method and integrate the system with h = 0.2 for t [0, 2].
Plot both, the numerical solution and the exact solution. Compare the curves.
c) MATLAB: Decrease the step size h. Does the numerical solution tend to the exact solution?

Exercise 2.10
Consider the test initial value problem

y = 2y, y(0) = 5, t [0, 50] .

a) Calculate the exact solution.

b) MATLAB: Integrate the IVP with Eulers method using the step sizes h = 1.015 and 0.9.
What can be observed?
c) MATLAB: Integrate the IVP with the classical Runge-Kutta method of order 4 using the step
sizes h = 1.41 and 1.2. What can be observed?

Application-based Exercises
Exercise 2.11
Population model with two competing species: Consider the system

x = x x2 y = y y2
1 1
xy, xy .
2 2
Draw a phase-plane diagram for x 0, y 0 which includes all equilibrium solutions, x- and
y-nullclines along with the exact flow directions on the nullclines and approximate flow directions
in the regions between nullclines.
Exercise 2.12
After a skydiver jumps from an airplane and until the parachute opens, the air resistance is
proportional to v1.5 , and the maximum speed that the skydiver can reach is 130km/h.
a) Set up the differential equation for the vertical velocity. What is the initial condition?
b) MATLAB: Implement the Runge-Kutta method and integrate the system with h = 0.01 for
t [0, 10]. Plot the solution curve and include the curve if there were no air resistance.
c) MATLAB: How many seconds would it take for the skydiver to break a falling speed of
100km/h?
56

Exercise 2.13
The vertical dynamical behavior of a vehicle can be studied using a quarter car with three masses.
this simplified model consists of a wheel with mass m1 , the body with mass m2 , and the driver
with mass m3 . The masses are linked with springs and dampers.
a) Set up three scalar differential equations of order 2 describing the vertical dynamics using the
positions z1 , z2 , and z3 as state variables. Let h denote the road height.
b) Transform the differential equation into a second-order system with mass matrix M , damper
matrix K, and spring matrix C. Use z = (z1 , z2 , z3 )T as state variable.
c) Transform the system into an explicit first-order system. Use y = (z, z)T as state variable.
d) MATLAB: Implement the right-hand side function of the ODE system as set up in c).
e) MATLAB: Implement the road height function. Let x denote the road position. Then,

0 if x 4


h(x) = 0.2 if 4 < x 12



0 if 12 < x

f) MATLAB: Assume a constant velocity of 10 m

s
. Furthermore, assume the following masses

m1 = 10 , m2 = 100 , m3 = 30

and spring and damping constants

k2 = 1000 , k3 = 100 , c1 = 100000 , c2 = 1000 , c3 = 1000 .

Simulate the quarter car using the built-in solver ode45. Plot the curves z1 (t), z2 (t), z3 (t),
and h(t).
g) MATLAB: Change the parameters mi , ki , and ci and discuss the behavior.
h) MATLAB: Implement the classical RK4 method. Compare the simulation results with this
method and the built-in ode45 method.
i) MATLAB: Animate the model and study the oscillations.

Exercise 2.14
When lighting a match, the ball of flame grows rapidly until it reaches a critical size. Then it
remains at that size because the amount of oxygen being consumed by the combustion in the
interior of the ball balances the amount available through the surface. The simple model with y
representing the radius of the ball is

y = y2 y3 ,
2
y(0) = , t [0, ] .

a) Show that y(t) = 1 is a steady state solution.
b) MATLAB: Integrate the IVP with the MATLAB routine ode45 using = 0.01. Set the relative
error to 104 .
c) MATLAB: Integrate the IVP with the MATLAB routines ode45 and ode23s using = 0.0001.
Set the relative error to 104 . Compare the number of steps needed. Zoom into the second
half of the graph.
 57

3 Partial Differential Equations

Partial differential equations are used as mathematical models in a vast range of technical
applications. Heat distributions, diffusion processes and wave dynamics can be described
with partial differential equations. Any kind of fluid dynamics is mathematically expressed
by the famous Navier-Stokes equations.
58

3.1 Introduction
First of all, we briefly discuss the most important basics on partial differential equations.

3.1.1 Theoretical Basics

The main difference between ordinary and partial differential equations is the number of
independent variables. Solutions of partial differential equations are functions of at least
two variables. Thus, any derivative involved in a partial differential equation is a partial
derivative. Most common partial differential equations are of second-order.

Definition 3.1 (Partial differential equation)

An equation involving a function u(x, y, t) of the independent variables of space x and
y, the independent variable of time t and some partial derivatives of u of highest order
n is called a partial differential equation (PDE) of order n. Special cases involve
functions u(x), u(x, y), and u(x, t).

Example 3.1 (Partial differential equations)

a) The equation ux + uy = 1 is a first-order partial differential equation because two partial
derivatives are involved. The solution is a function u(x, y).
b) The equation ux = t u is also a first-order partial differential equation. Having the indepen-
dent variables x and t the solution is a function u(x, t).
c) The second-order equation uxx + uyy + uzz = t u2 comprises the space variables x, y, and z,
and time t. Thus, a solution function is of the form u(x, y, z, t).

The independent variables typically describe time and space. It is common to denote
problems in one spacial variable as 1D problems. Likewise, problems in two or three
spacial variables are called 2D and 3D problems.

Dimension
Partial differential equations are called
1D problems if only one spacial variable is involved,
2D problems if two spacial variable are involved,
3D problems if three spacial variable are involved.

This classification holds no matter if time is a further variable or not.

 59

Example 3.2 (Problem dimension)

a) The equation ux + uy = 1 is a 2D problem.
b) The equation ux = t u is a 1D partial differential equation although x and time t are both
independent variables.
c) The equation uxx + uyy + uzz = t u2 is a 3D problem.

Example 3.3 (Laplace and Poisson equation)

a) The 1D and 2D Laplace equation reads

u = 0, u = uxx + uyy = 0 .

b) The 1D and 2D Poisson equation reads

u = f (x), u = uxx + uyy = f (x, y) .

Example 3.4 (Heat equation)

With > 0 the 1D and 2D heat equations are defined as

ut = uxx , ut = (uxx + uyy ) = u .

Example 3.5 (Wave equation)

The 1D and 2D wave equations are defined as

utt = 2 uxx , utt = 2 (uxx + uyy ) .

Example 3.6 (Tricomi equation)

The second-order, linear Tricomi equation is defined as y uxx + uyy = 0.

Solutions of the Laplace equation are called harmonic. Conversely, any function with
u = 0 solves the Laplace equation.

Definition 3.2 (Solution)

A function u is called solution of a partial differential equation if the equation is
satisfied at all points of the domain when inserting u and its partial derivatives.
60

Example 3.7 (Solutions of the heat equation)

a) The function u(x, t) = ax + b solves the heat equation since ut = uxx = 0.

b) The function u(x, t) = e t (c1 cos ( x) + c2 sin ( x) ) also solves the heat equation
2

for any coefficients c1 and c2 since

2 e t (
2
ut (x, t) = c1 cos ( x) + c2 sin ( x) )

e t ( c1 cos ( x) c2 sin ( x) )
2
uxx (x, t) =

c) The function u(x, t) = e t (c1 e + c2 e
2
x x
) also is a solution of the heat equation for
any coefficients c1 and c2 .

Example 3.8 (Solutions of the Tricomi equation)

The function u(x, y) = y (x2 1
6
y 3 ) solves the Tricomi equation y uxx + uyy = 0 since

2 3
ux = 2xy, uxx = 2y, uy = x2 y , uyy = 2y 2 .
3

Definition 3.3 (Harmonic function)

A function u is called harmonic if it solves the Laplace equation u = 0.

Example 3.9 (Harmonic functions)

a) The function u(x, y) = x2 + y 2 is not harmonic, since u = 2 + 2 = 4 0.
b) The function u(x, y) = x2 y 2 is harmonic, because u = 2 2 = 0.
c) The function u(x, y) = log(x2 + y 2 ) is harmonic, since

2x 2y 2x2 + 2y 2 2x2 2y 2
u = + = + 2 =0.
x x + y
2 2 y x + y
2 2 (x + y )
2 2 2 (x + y 2 )2

d) Any linear function u(x, y, z) = ax + by + cz + d is harmonic, because all second-order partial

derivatives vanish.

Definition 3.4 (Linear partial differential equation)

An partial differential equation is called linear if u and all its derivaties appear only as
linear terms.

Example 3.10 (Linear and non-linear equations)

a) The equation ux + uy = 1 is linear.
b) The equation ux = t u is also linear.
c) The equation uxx + uyy + uzz = t u2 is non-linear, because u appears as a square.
d) The equation 5ux 7t2 uyy = sin t + et utt is linear. This is not in conflict with the non-linear
coefficients t2 , sin t, or et .
 61

Many important partial differential equations are linear and of second order:

a uxx + 2b uxy + c uyy + 2 d1 ux + 2 d2 uy + d3 u = f (x, y).

In general, the coefficients a, b, c and d1 , d2 , d3 depend on (x, y). The factors 2 simplify
some equations related to this quadratic form. Using differentials we can rewrite the
equation as


a b x

( d1 u = f .
)
x y
, + 2 ( , ) ( ) + d
x y d 2
3

b c
y

A
Finding the zeros of the characteristical polynomial of matrix A

det (A I) = 2 (a + c) + (ac b2 ) = 0

yields the eigenvalues of matrix A:

(a + c) (a + c)2 + 4(b2 ac) (a + c) (a c)2 + 4b2
1,2 = = .
2 2
The second expression shows that both eigenvalues are always real numbers. The type
of A follows from the first expression: If b2 ac < 0, then 1,2 > 0 or 1,2 < 0 and A is
positive or negative definite. If b2 ac = 0, then 1 > 0 or 1 < 0 and 2 = 0. Hence, A
is positive or negative semi-definite. Finally, if b2 ac > 0, then 1 > 0 and 2 < 0. This
implies that A is indefinite. So, linear second-order partial differential equations can be
divided into three different types. Note, that in general the type of equation depends of
x and y.

Definition 3.5 (Classification of linear second-order equations)

The linear second-order partial differential equation

a uxx + 2 b uxy + c uyy + 2 d1 ux + 2 d2 uy + d3 u = f (x, y)

is called
elliptic if b2 ac < 0,
parabolic if b2 ac = 0,
hyperbolic if b2 ac > 0.
In general, the coefficients a, b, c and d1 , d2 , d3 depend on (x, y). Hence, the type of
equation depends on (x, y).
62

Example 3.11 (Laplace, heat, and wave equation)

a) For the 2D Laplace and Poisson equation we have

uxx + uyy = 0 a = 1, b = 0, c=1

and with that b2 ac = 1 < 0. Hence, the 2D Laplace equation is of second-order, linear,
and elliptic.
b) In terms of classification, the 1D heat equation gives

ut = uxx a = , b = 0, c=0

which yields b2 ac = 0. Hence, the 1D heat equation is of second-order, linear, and parabolic.
c) For the 1D wave equation we find

utt = 2 uxx a = 2 , b = 0, c = 1

and with that b2 ac = 2 > 0. Hence, the 1D wave equation is of second-order, linear, and
hyperbolic.

Example 3.12 (Tricomi equation)

Regarding classification the Tricomi equation has the coefficients

y uxx + uyy = 0 a(x, y) = y, b(x, y) = 0, c(x, y) = 1 .

This gives b2 ac = y. Hence, the Tricomi equation is elliptic, if y > 0, parabolic if y = 0, and
hyperbolic if y < 0.

Solutions of linear second-order equations

Typical solutions of
elliptic equations are steady-state and do not depend on time.
parabolic equations evolve towards steady-state solutions.

hyperbolic equations do not tend to settle into steady-state.

As with ordinary differential equations partial differential equations typically have infinite
many solutions. To define a well-posed problem with a unique solution, additional initial
and boundary conditions are necessary.
 63

Definition 3.6 (Boundary conditions for elliptic equations)

Typical boundary conditions of elliptic partial differential equations with solutions
u(x, y) on a domain D R2 are
y
Dirichlet boundary conditions

u(x, y) = g(x, y), (x, y) D

P D
Neumann boundary conditions
D
un (x, y) = g(x, y), (x, y) D
x
where un is the partial derivative of u in
normal direction. The special case un = 0 is
called natural boundary conditions.

The solution at P (x0 y0 ) D depends on the whole boundary D.

Example 3.13 (Boundary conditions for the Laplace equation)

a) Consider the 2D Laplace equation on the unit square D = [0, 1] [0, 1]. With

u(0, y) = u(1, y) = 0, u(x, 0) = 1, u(x, 1) = 2

Dirichlet boundary conditions are defined.

b) The 2D Laplace equation on the unit circle D = {(x, y) x2 + y 2 1} together with the
Dirichlet boundary condition

u(x, y) = sin(arg(x, y)) for x2 + y 2 = 1

define a unique solution. Here, arg(x, y) denotes the angle between the vector (x, y)T and
the positive x-axis.
c) Consider the 2D Laplace equation on the triangle D = {(x, y) x, y 0, x + y 1}. With

u(x, 1 x) = 1, ux (0, y) = uy (x, 0) = 0

mixed Dirichlet and Neumann boundary conditions are defined.

64

Definition 3.7 (Boundary conditions for parabolic equations)

Typical boundary conditions of parabolic partial differential equations with solutions
u(x, t) on a domain D = [a, b] R+0 are
t
initial boundary conditions D
u(x, 0) = g(x), x [a, b] P
constant boundary conditions
u(a, t) = c1 , u(b, t) = c2 , t > 0
D
natural boundary conditions
ux (a, t) = ux (b, t) = 0, t > 0 a b x
periodic boundary conditions
u(a, t) = u(b, t), ux (a, t) = ux (b, t), t> 0

The solution at P (x0 t0 ) D depends on that part of the boundary D where t t0 .

Example 3.14 (Boundary conditions for the heat equation)

a) Consider the 1D heat equation on the domain D = [0, 1] R+0 . With

u(x, 0) = sin( x), u(0, t) = u(1, t), ux (0, t) = ux (1, t)

initial and periodic boundary conditions are defined.

b) The 2D heat equation on the domain D = {(x, y) x2 + y 2 1} R+0 together with the initial
and constant boundary conditions

u(x, y, 0) = e(x +y 2 )
2
, u(1, y) = u(1, y) = u(x, 1) = 0, u(x, 1) = 1

define a unique solution.

Definition 3.8 (Boundary conditions for hyperbolic equations)

Typical boundary conditions of hyperbolic partial differential equations with solutions
u(x, t) on a domain D = [a, b] R+0 are
t
initial boundary conditions D
P
u(x, 0) =g(x), ut (x, 0) = g(x), x[a, b]

constant boundary conditions

D
u(a, t) = c1 , u(b, t) = c2 , t>0
a b x
The solution at P (x0 t0 ) D depends on that part of the boundary D that lies
behind the propagating line of P .
 65

Example 3.15 (Boundary conditions for the wave equation)

Consider the 1D wave equation on the domain D = [0, ] R+0 . With

u(x, 0) = cos x, u(0, t) = 1, u(, t) = 1

initial and constant boundary conditions are defined.

3.1.2 Numerical Basics

Some numerical definitions and principles are fundamental and used in the context of any
numerical method for partial differential equations. At least for finite difference approaches
a numerical solution is represented by a finite sequence of grid nodes.

Definition 3.9 (Discretization)

The transformation of a continuous solution into a discrete sequence of points is called
1D-discretization or 2D-discretization in space

u(x) ui = u(xi ) wi
u(x, y) ui,j = u(xi , yj ) wi,j

1D-discretization or 2D-discretization in space and time

u(x, t) ui,k = u(xi , tk ) wi,k

u(x, y, t) ui,j,k = u(xi , yj , tk ) wi,j,k

For the continuous solution x [a, b], y [c, d], and t 0. The discretized solution is
defined for i = 0, 1, . . . , n, j = 0, 1, . . . , m, and k N0 . If the points are equidistant with
h > 0 and t > 0 then xi = a + hi, yj = c + hj, and tk = t k. Numerical approximations
of u are denoted by w.

If the problem is 2D or higher, all grid nodes can be ordered linearly in a vector. One of
the possible variants to do this is to use the reading or alphabetical order. All grid nodes
are ordered with ascending index values. The first index has highest and the last index
lowest priority.
66

Definition 3.10 (Reading or alphabetical order)

A two-dimensional solution matrix in space with y
elements wi,j can be transformed into a one- y2 w0,2 w1,2 w2,2 w3,2
dimensional solution vector w using the read-
ing order or alphabetical order. The elements y1 w0,1 w1,1 w2,1 w3,1
wi,j are sorted with ascending index order. The
comparison starts with the first index and in y0 w0,0 w1,0 w2,0 w3,0
case of equality continues with the second in-
dex. x0 x1 x2 x3 x

w = (w0,0 , w0,1 , w0,2 , . . . , w1,0 , w1,1 , w1,2 , . . . , wn,m ) .

Example 3.16 (Reading order)

Consider the domain D = [0, 2] [3, 4]. Using an equally spaced grid with h = 0.1 we obtain the
positions

x0 = 0, x1 = 0.1, x2 = 0.2, . . . , x21 = 2, y0 = 3, y1 = 3.1, y2 = 3.2, . . . , y11 = 4 .

The grid consists of 21 11 = 231 nodes with 19 9 = 171 inner nodes and 20 + 10 + 20 + 10 = 60
boundary nodes. If the solution function is, for example, w(x, y) = xy 2 , the vector

w = ( 0, 0, 0, . . . , 0.9, 0.961, 1.024, . . . , 1.8, 1.922, 2.048, . . . , . . . , 32)

x=0 x = 0.1 x = 0.2
consists of all grid values in reading order.
 67

3.2 Finite Difference Methods

One of the two main streams to solve partial differential equations are finite difference
methods. The idea is to discretize the domain and to approximate all partial derivatives
by finite difference formulas. This is especially simple if an equidistant and orthogonal
grid is used. Then, the resulting linear systems are obtained in a quick and simple way.
This is the main advantage of finite differences. However, if the domain has a curved
boundary or if adaptive grid sizes are used, the difference formulas become much more
complicated.

3.2.1 Finite Differences

Derivatives of functions can easily be approximated by finite differences. Most formulas
result from Taylor expansions. The simple forward difference formula for the first derivative
of a function f is obtained in the following way:

f (t + h) f (t)
f (t + h) = f (t) + h f (t) + O(h2 ) f (t) = + O(h) .
h
In the same way, the backward difference formula reads

f (t) f (t h)
f (t h) = f (t) h f (t) + O(h2 ) f (t) = + O(h) .
h
Both approximations are of first order O(h). Subtracting the second from the first ex-
panded polynomial

h2 h3
f (t + h) = f (t) + h f (t) + f (t) + f (t) + O(h4 )
2 6
h2 h3
f (t h) = f (t) h f (t) + f (t) f (t) + O(h4 )
2 6
yields the central formula for the first derivative

f (t + h) f (t h)
f (t) = + O(h2 )
2h
which is of second order O(h2 ).
68

Definition 3.11 (Finite differences for the first derivative)

The first derivative of a function f R R can be approximated using h > 0 by
the first-order forward and backward difference formula
f (t + h) f (t) f (t) f (t h)
f (t) , f (t) ,
h h

the second-order central difference formula

f (t + h) f (t h)
f (t) .
2h

Likewise, adding the expanded polynomials for f (t + h) and f (t h) up to third order

results in the central formula for the second derivative. This central formuls is also of
second order O(h2 ).

Definition 3.12 (Finite differences for the second derivative)

The second derivative of a function f R R can be approximated using h > 0 by
the second-order central difference formula
f (t + h) 2f (t) + f (t h)
f (t) .
h2

These finite differences in one dimension now can be applied to a function u depending on
x and y. Using these differences, approximations w of u are involved only at grid points.
In this context, we assume a regular grid with equidistant points.

Finite differences
The second-order partial derivatives of a function u(x, y) at point xi , yj can be ap-
proximated using h > 0 by the numerical values
wi+1,j 2wi,j + wi1,j
uxx (xi , yj ) (wxx )i,j =
h2
wi,j+1 2wi,j + wi,j1
uyy (xi , yj ) (wyy )i,j =
h2
 69

3.2.2 Difference Methods for 2D Elliptic Equations

We derive a numerical approximation scheme for the 2D Laplace equation. Therefore,
we substitute the second-order partial derivatives of the Laplace equation at each node
(xi , yj )

(uxx + uyy )(xi , yj ) = 0

by finite differences for the second derivative:

wi+1,j 2wi,j + wi1,j wi,j+1 2wi,j + wi,j1
+ =0.
h2 h2
At each node (xi , yj ) the corresponding equation comprises the central value wi,j and
the 4 neighbors wi1,j and wi,j1 . This molecule is called the 5-point-star.

Definition 3.13 (5-point-star)

A numerical solution wi,j of the 2D Laplace y
equation can be obtained using the grid size h
and the approximation scheme 1

wi1,j +wi+1,j +wi,j1 +wi,j+1 4wi,j 1 4 1

=0
h2
which is called the 5-point-star. The coeffi- 1
cients of the 5-point-star molecule are obtained
from the equation multiplied by 1. x

The equations for all numerical values wi,j can be collected in a linear system Aw = b.
If the domain is rectangular, the system matrix A has a special structure, it consists of
5 diagonals. Using a grid with n + 1 nodes in both directions, the vector w consists of
(n + 1)2 elements in reading order and the matrix A has (n + 1)4 elements!
70

Linear system for 5-point-star

Using the 5-point-star to solve the 2D Laplace equation results in the linear system
Aw = b with

4 1 1 0
1

1
A=
1
,
b=

.


1

1 1 4 0

This linear system is valid for all inner nodes wi,j . Boundary conditions for boundary
nodes have to be added separately. Using the reading order in a retangular grid with
n + 1 nodes in x-direction and m + 1 nodes in y-direction, the values 1 are separated
by m 1 zero elements.

Since all finite differences used in the numerical solution scheme are of order 2, the 5-
point-star is of second-order. This is sufficient and gives reasonable precision for normal,
well-conditioned problems.

Theorem 3.1 (Order of the 5-point-star)

The numerical method using the 5-point-star with grid size h to solve the 2D Laplace
equation is of order O(h2 ).

Until now, the linear system Aw = b is valid only for inner nodes. Nodes on the boundary
typically have only 2 or 3 neighbors within the domain. Thus, the 5-point-star is not
applicable.

Ghost nodes
Applying the 5-point-star to boundary nodes y
ghost nodes
generates ghost nodes outside the domain. ym
These ghost nodes are eliminated again with
modified difference formulas. Thus, for bound-
ary nodes the 5-point-star changes into formu-
las involving less than 5 nodes. y0

x0 xn x

To include Dirichlet boundary conditions there are two alternatives. The first alternative is
to keep the boundary nodes in the linear system and replace the corresponding 5-point-star
equations by trivial equations.
 71

Boundary nodes with Dirichlet condition

To add Dirichlet boundary conditions, the linear system Aw = b is modified. Any row
k of A that refers to a boundary node wk with Dirichlet conditions is replaced by zeros
and the value 1 on the diagonal. The corresponding element bk of the right-hand side
vector is replaced by the given Dirichlet value gk :

ak,` = 0, ` k, ak,k = 1, bk = gk .

Example 3.17 (Laplace equation on a square)

Consider the 2D Laplace problem y
4
u =0 on D = {(x, y) 0 x, y 2}
10 if y = 2 3
u(x, y) = { w0,2 w2,2
0 otherwise on D 2
The domain is discretized with a grid using h = 1. 1
First of all, 9 nodes w0,0 w2,0
1 2 3 4 5 6 x
wi,j = w(ih, jh) with i, j = 0, 1, 2

are introduced.

According to the reading order, we define the vector

w = (w0,0 , w0,1 , w0,2 , w1,0 , w1,1 , w1,2 , w2,0 , w2,1 , w2,2 )T .

Using the 5-point-star we obtain the linear system Aw = b with

1 0 0 0 0 0 0 0 0 0
0 1 0 0 0 0 0 0 0 0

0 0 1 0 0 0 0 0 0 10

0
0 0 0 1 0 0 0 0 0

A= 0 1 0 1 4 1 0 1 0 , b= 0 .

0 0 0 0 0 1 0 0 0 10

0
0 0 0 0 0 0 1 0 0

0 0 0 0 0 0 0 1 0 0
0 0 0 0 0 0 0 0 1 10

The value of the center node is w1,1 = 2.5.

The second alternative is to delete all boundary node equations from the linear system. As
an advantage, this implies a smaller matrix A. However, matrix A is then less structured.
This may be regarded as a disadvantage.
72

Example 3.18 (Laplace equation on an L-shaped domain)

Consider the 2D Laplace problem y
4
u =0 on D
w3,3
5 if y = 0 3
u(x, y) = {
0 otherwise on D 2
w1,1 w3,1
on the given domain D. Using a grid with h = 1 the 1
domain has 5 inner nodes:
1 2 3 4 5 6 x
w = (w1,1 , w2,1 , w3,1 , w3,2 , w3,3 )T .

Since the values of the boundary nodes are known through the Dirichlet conditions, we set up a
linear system resulting from the 5-point-star including only the inner nodes:

4w11 w21 = 5 4 1 0 0 0 5
4w21 w11 w31 = 5 1 4 1 0 0 5

4w31 w21 w32 = 5 0 1 4 1 0 w = 5 .

4w32 w31 w33 = 0 0 0 1 4 1 0
4w33 w32 = 0 0 0 0 1 4 0

Using the Gau algorithm we get the scheme

4 1 0 0 0 0
0 15 4 0 0 25
0 0 56 15 0 100
0 0 0 209 56 100
0 0 0 0 780 100

and thus the solution values for the inner nodes

w1,1 1.7949, w2,1 2.1795, w3,1 1.9231, w3,2 0.5128, w3,3 0.1282 .

To include Neumann boundary conditions there are at least two approaches. In a first
approach we could use forward or backward difference formulas for the boundary nodes.
The advantage is that these formulas do not include nodes outside the domain. The big
disadvantage however is that their order is only O(h). This order does not match with
the second-order differences O(h2 ) of the 5-point-star for the inner nodes. A second
approach makes use of temporary ghost nodes outside the domain. If wn,j is a boundary
node on the right part of a boundary, the ghost node wn+1,j lies outside the domain. Let
u
(xn , yj ) = gn,j
n
be the value of the normal derivate at the boundary node. Then, using the central
difference formula in x-direction for the normal derivative gives the approximation
wn+1,j wn1,j
= gn,j wn+1,j = 2h gn,j + wn1,j .
2h
 73

Inserting the ghost node wn+1,j into

4wn,j wn1,j wn+1,j wn,j1 wn,j+1 = 0

finally yields the modified boundary equation

4wn,j 2wn1,j wn,j1 wn,j+1 = 2h gn,j

in which the ghost node is eliminated again.

Boundary nodes with Neumann condition

To add Neumann boundary conditions the linear system Aw = b is modified. In any
row k of A that refers to a boundary node wk with Neumann conditions the element
ak,` that corresponds with the node w` opposite the ghost node outside the domain is
replaced by the value 2. The corresponding element bk of the right-hand side vector
is replaced by a given Neumann value gk and the grid size h:

ak,` = 2, bk = 2h gk .

As an alternative 9 points can be included in a finite difference scheme. In addition to the

neighbors to the left and right and upwards and downwards, the four diagonal neighbors
can be involved in the approximation formula. A symmetric approach uses the coefficients

The coefficients , , and are chosen such that the resulting local discretisation error
is as small as possible. One possible choice defines the 9-point-star.

Definition 3.14 (9-point-star)

A numerical solution wi,j of the 2D Laplace y
equation can be obtained using the grid size h
and the approximation scheme 1 4 1

wi1,j1 + 4wi1,j + 4wi,j1 20wi,j 4 20 4

=0
6h2
which is called the 9-point-star. The coeffi- 1 4 1
cients of the 9-point-star molecule are obtained
from the equation multiplied by 1. x

It can be shown that the 9-point-star is of higher order than the 5-point-star. To be more
precise, the order is O(h4 ) in comparison with O(h2 ). However, 9 neighboring points are
74

involved in each equation. The corresponding matrix has more non-zero elements than
the 5-point-star matrix.

Theorem 3.2 (Order of the 9-point-star)

The numerical method using the 9-point-star with grid size h to solve the 2D Laplace
equation is of order O(h4 ).

3.2.3 Difference Methods for 1D Parabolic Equations

The finite difference approach for parabolic equations is quite similar to that for elliptic
ones. For 1D problems the partial derivates with respect to x and t have to substituted
by finite difference formulas.

Definition 3.15 (Explicit 4-point-star)

A numerical solution wi,k of the 1D heat equa- t
tion ut = uxx can be obtained using the grid
sizes h and t and the approximation scheme
1
wi,k+1 wi,k wi1,k 2 wi,k
=
t h2 12
which is called the explicit 4-point-star. The
t
constant is = 2 . x
h

In contrast to solutions of elliptic problems those of parabolic equations depend on time.

This implies that the numerical solution is computed from one time level to the next one.
Instead of solving one single linear system, an interative process has to be run.

Linear system for explicit 4-point-star

Using the explicit 4-point-star to solve the 1D heat equation ut = uxx results in the
linear system wk+1 = A wk with

1 2
t
A= , = .
h2
1 2

This linear system is valid for all inner nodes wi,k+1 with 1 i n 1 and k 0.
Boundary conditions for boundary nodes have to be added separately.
 75

Example 3.19 (Heat equation on a band domain)

Consider the 1D heat problem t
4
ut = uxx on D = {(x, t) 0 x 4, 0 t}
u(0, t) = 0, u(4, t) = 10 for 0 t 3

u(x, 0) = 0 for 0 < x < 4 2

The domain is discretized with a grid using h = 1 and 1

t = . Using the explicit method, in each interation
1
2
step 1 2 3 4 5 6 x

1 0 0 0 0
1 1
0 0 0
2 2

wk+1 = A wk with A= 0 1
0 1
0 .
2 2
1
0 0 1
0
0 0
2
0 0 1
2

has to be computed. The first and last row in A are due to the Dirichlet boundary conditions.
Starting with the initial condition, the first iterates and the limit interate read

0 0 0 0 0
0 0 0 1.25 2.5

w0 = 0 , w1 = 0 , w2 = 2.5 , w3 = 2.5 , . . . , w = 5 .

0 5 5 6.25 7.5
10 10 10 10 10

The simple and straightforward explicit method is of order O(h2 ) and only of order O(t).
Moreover, a stability conditon can be derived. This condition is obtained by investigating
the eigenvalues of matrix A.

Theorem 3.3 (Order and stability of the explicit 4-point-star)

The numerical method using the explicit 4-point-star with grid sizes h and t to solve
the 1D heat equation is of order O(h2 ) and O(t). The method is stable if
t 1
= .
h2 2
76

Example 3.20 (Heat equation on a band domain with unstable solution)

Consider again the 1D heat problem t
4
ut = uxx on D = {(x, t) 0 x 4, 0 t}
u(0, t) = 0, u(4, t) = 10 for 0 t 3

u(x, 0) = 0 for 0 < x < 4 2

from example 3.19 now with a grid using h = 1 and 1

t = 1. According to the stability criterion
1 2 3 4 5 6 x
t 1
= =1>
h2 2

stability is not guaranteed. Using the explicit method, in each interation step

1 0 0 0 0
1 1 1 0 0

wk+1 = A wk with A= 0 1 1 1 0 .

0 0 1 1 1
0 0 0 0 1

has to be computed. The first and last row in A are due to the Dirichlet boundary conditions.
Starting with the initial condition, the first iterates read

0 0 0 0 0
0 0 0 10 20

w0 = 0 , w1 = 0 , w2 = 10 , w3 = 10 , w4 = 40 , . . .

0 10 0 20 20
10 10 10 10 10

This numerical solution shows increasing oscillations and thus do not behave similiar to the real
solution.

Instead of using the forward difference formula for the time discretisation the backward
one can be used as well. The 4-point-star then is established in such a way that just 1
node lies on the old time level and 3 nodes lie on the new one.

Definition 3.16 (Implicit 4-point-star)

A numerical solution wi,k of the 1D heat equa- t
tion ut = uxx can be obtained using the grid
sizes h and t and the approximation scheme
1+2
wi,k+1 wi,k wi1,k+1 2wi,k+1
=
t h2 1
which is called the implicit 4-point-star. The
t
constant is = 2 . x
h
 77

In this implicit approach the matrix times vector multiplication from the explicit method
changes into the solution of a linear system. This implies that the computational effort
increases for the implicit variant.

Linear system for implicit 4-point-star

Using the implicit 4-point-star to solve the 1D heat equation ut = uxx results in the
linear system B wk+1 = wk with

1 + 2
t
B= , = .
h2
1 + 2

This linear system is valid for all inner nodes wi,k+1 with 1 i n 1 and k 0.
Boundary conditions for boundary nodes have to be added separately.

The implicit method is still quite simple and also of order O(h2 ) and only of order O(t).
But in contrast to the explicit approach, the implicit variant is unconditionally stable. This
is a major advantage.

Theorem 3.4 (Order and stability of the implicit 4-point-star)

The numerical method using the implicit 4-point-star with grid sizes h and t to solve
the 1D heat equation is of order O(h2 ) and O(t). The method is unconditionally
stable.
78

Example 3.21 (Heat equation on a band domain with implicit solution)

Consider again the 1D heat problem t
4
ut = uxx on D = {(x, t) 0 x 4, 0 t}
u(0, t) = 0, u(4, t) = 10 for 0 t 3

u(x, 0) = 0 for 0 < x < 4 2

from example 3.19 with a grid using h = 1. Using 1

the implicit method, with t = 1 the corresponding
linear system reads 1 2 3 4 5 6 x

1 0 0 0 0
1 3 1 0 0

B wk+1 = wk with B= 0 1 3 1 0 .

0 0 1 3 1
0 0 0 0 1

The first and last row in B are due to the Dirichlet boundary conditions. Starting with the initial
condition, the iterates read

0 0 0 0
0 0.476 1.043 2.5

w0 = 0 , w1 1.428 , w2 2.653 , . . . , w = 5 .

0 3.809 5.487 7.5
10 10 10 10

Likewise, with t = 1
2
the corresponding linear system reads

11 0 0 0 0
2 2 12 0 0

B wk+1 = wk with B= 0 12 2 21 .
0
1
0 0 12 2 2
0 0 0 0 1

Starting with the initial condition, the iterates read

0 0 0 0
0 0.178 0.471 2.5

w0 = 0 , w1 0.714 , w2 1.530 , . . . , w = 5 .

0 2.678 4.221 7.5
10 10 10 10

When making a comparison between the results obtained with the two values of t we have to
be careful. Only every second iterate obtained with t = 21 corresponds to an iterate obtained
with t = 1.

Now, we combine explicit and implicit approach. The two 4-point-stars merge into one
6-point-star called Crank-Nicolson method. This method has been published in a paper
in 1947.
 79

Definition 3.17 (6-point-star Crank-Nicolson method)

A numerical solution wi,k of the 1D heat equa- t
tion ut = uxx can be obtained using the grid
sizes h and t and an approximation scheme
which is the average of the explicit and implicit 2+2
4-point-star method and is called the 6-point-
star Crank-Nicolson method. The constant 22
t
is = 2 and the approximation scheme
h
reads
x

wi,k+1 wi,k wi1,k + wi1,k+1 2 wi,k 2 wi,k+1

= .
t 2 h2

Like in the implicit case the Crank-Nicolson method requires the solution of a linear system
in each iteration step. Note, that for 1D parabolic equations the size of the resulting linear
system is comparable to that of 1D elliptic equations and thus in general much smaller
than the size of 2D elliptic equations.

Linear system for 6-point-star Crank-Nicolson method

Using the 6-point-star Crank-Nicolson method to solve the 1D heat equation ut = uxx
t
results in the linear system B wk+1 = A wk with = 2
h

2 2 2 + 2

A= , B= .

2 2 2 + 2

This linear system is valid for all inner nodes wi,k+1 with 1 i n 1 and k 0.
Boundary conditions for boundary nodes have to be added separately.

The Crank-Nicolson method is in several ways satisfying. Firstly, the method is of second-
order for h as well as for t. Secondly, the method is unconditionally stable. There is no
restriction regarding the time step size t.

Theorem 3.5 (Order and stability of the 6-point-star Crank-Nicolson method)

The numerical method using the 6-point-star Crank-Nicolson method with grid sizes h
and t to solve the 1D heat equation is of order O(h2 ) and O(t2 ). The method is
unconditionally stable.
80

Example 3.22 (Heat equation on a band domain with Crank-Nicolson)

Consider again the 1D heat problem from exam- u
ple 3.19 with a grid using h = 1. Applying Crank- 10 t = 1
2
Nicolsons method, the corresponding linear system
reads 7.5 w

B wk+1 = A wk . 5 w0
2.5
With t = 1 the corresponding matrices are

1 2 3 4 5 6 x

1 0 0 0 0 1 0 0 0 0
1 0 1 0 0 1 4 1 0 0

A= 0 1 0 1 0 , B= 0 1 4 1 0 .

0 0 1 0 1 0 0 1 4 1
0 0 0 0 1 0 0 0 0 1

The first and last row in B are due to the Dirichlet boundary conditions. Starting with the initial
condition, the iterates read

0 0 0 0
0 0.357 1.173 2.5

w0 = 0 , w1 1.428 , w2 3.265 , . . . , w = 5 .

0 5.357 6.173 7.5
10 10 10 10

Likewise, with t = 1
2
the corresponding matrices are

1 0 0 0 0 11 0 0 0 0
2 12
1 1
1 0 0 3 0 0
2 2

A= 0 1
1 1
0 , B= 0 12 3 21 0 .
2 2
1 1
0 0 1
1 0 0 12 3 2
0 0
2
0 0 1
2
0 0 0 0 1

Starting with the initial condition, the iterates read

0 0 0 0
0 0.098 0.407 2.5

w0 = 0 , w1 0.588 , w2 1.660 , . . . , w = 5 .

0 3.431 4.852 7.5
10 10 10 10

When making a comparison between the results obtained with the two values of t we have to
be careful. Only every second iterate obtained with t = 21 corresponds to an iterate obtained
with t = 1.
 81

3.2.4 Difference Methods for 2D Parabolic Equations

The extension of difference methods from 1D to 2D in space is straight forward. The
Laplace operator is applied to x and y. This implies the well-known 5-point-star on
the x-y-plane. Like in 1D, there are explicit, implicit, and combined Crank-Nicolson
approaches in 2D as well.

Definition 3.18 (Explicit 6-point-star)

A numerical solution wi,k of the 2D heat equa- t
tion ut = u can be obtained using the grid
sizes h and t and an approximation scheme 1
which is called explicit 6-point-star. The con-
stant is
y
t
= x 14
h2

and the approximation scheme reads

wi,j,k+1 wi,j,k wi1,j1,k 4 wi,j,k
= .
t h2

The explicit variant result in a molecule with 6 points. The time update is calculated by
the multiplication of a matrix with a vector. Using n + 1 nodes in x-direction and m + 1
nodes in y-direction, the update matrix A consists of (n + 1)2 (m + 1)2 elements.

Linear system for explicit 6-point-star

Using the explicit 6-point-star to solve the 2D heat equation ut = u results in the
linear system wk+1 = A wk with

1 4


t

A= , =
.
h2


1 4

This linear system is valid for all inner nodes wi,j,k+1 with 1 i n1 and 1 j m1
and k 0. Boundary conditions for boundary nodes have to be added separately.

The order of the explicit approach is the same in 1D and 2D. To ensure stability of the
numerical solution, the coefficient has to be below the value 14 . This value in 2D is
different to the value 21 in 1D.
82

Theorem 3.6 (Order and stability of the explicit 6-point-star)

The numerical method using the explicit 6-point-star with grid sizes h and t to solve
the 2D heat equation is of order O(h2 ) and O(t). The method is stable if
t 1
= .
h2 4

Swapping the difference formulas of time layers k and k + 1 yields the implicit variant.
Here, the backward difference formula with respect to time is applied. This results again
in a 6-point-star.

Definition 3.19 (Implicit 6-point-star)

A numerical solution wi,k of the 2D heat equa- t
tion ut = u can be obtained using the grid

sizes h and t and an approximation scheme 1+4

which is called implicit 6-point-star. The con-
stant is
t
y
= x 1
h2

and the approximation scheme reads

wi,j,k+1 wi,j,k wi1,j1,k+1 4 wi,j,k+1
= .
t h2

A single node of time k + 1 is computed from several nodes of time k. This again implies
the solution of a linear system. In contrast to elliptic problems, auch a linear system has
to be solved in each time step.
 83

Linear system for implicit 6-point-star

Using the implicit 6-point-star to solve the 2D heat equation ut = u results in the
linear system B wk+1 = wk with

1 + 4


t
B=

,
= .
h2


1 + 4

This linear system is valid for all inner nodes wi,j,k+1 with 1 i n1 and 1 j m1
and k 0. Boundary conditions for boundary nodes have to be added separately.

The order of simple explicit and implicit methods is O(h2 ) and O(t). Thus, the implicit
variant has no advantage in terms of precision. But, concerning the overall behavior of
the numerical solution, there is no limit of t. Even with large t, the numerical solution
behaves in the same way as the exact solution.

Theorem 3.7 (Order and stability of the implicit 6-point-star)

The numerical method using the implicit 6-point-star with grid sizes h and t to solve
the 2D heat equation is of order O(h2 ) and O(t). The method is unconditionally
stable.

A further improvement is obtained, using the average of the explicit and the implicit
method. In this approach, 10 points in total are included in the methods molecule. 5
points refer to time k and another 5 points refer to time k + 1.

Definition 3.20 (10-point-star Crank-Nicolson method)

A numerical solution wi,k of the 2D heat equa- t
tion ut = u can be obtained using the grid

sizes h and t and an approximation scheme 2+4

which is the average of the explicit and implicit
6-point-star method and is called the 10-point-
star Crank-Nicolson method. The constant y
t
is = 2 and the approximation scheme x 24
h
reads

wi,j,k+1 wi,j,k wi1,j1,k + wi1,j1,k+1 2 wi,j,k 2 wi,j,k+1

= .
t 2 h2
84

The time update involves two matrices. One matrix is multiplied with wk+1 , the other
one with wk . Hence, a matrix multiplication and the solution of a linear system have to
be calculated.

Linear system for 10-point-star Crank-Nicolson method

Using the 10-point-star Crank-Nicolson method to solve the 2D heat equation ut = u
t
results in the linear system B wk+1 = A wk with = 2
h

2 4 2 + 4



A=

, B =

.




2 4 2 + 4

This linear system is valid for all inner nodes wi,j,k+1 with 1 i n1 and 1 j m1
and k 0. Boundary conditions for boundary nodes have to be added separately. Using
the reading order in a retangular grid with n + 1 nodes in x-direction and m + 1 nodes
in y-direction the values and are separated by m 1 zero elements.

It can be shown, that the order of the Crank-Nicolson approach is 2 for h as well as for
t. Moreover, the advantage of unconditional stbility of the implicit method is inherited
to the Crank-Nicolson method. These advantages make the Crank-Nicolson approach to
a commonly used solver for parabolic problems.

Theorem 3.8 (Order and stability of the 10-point-star Crank-Nicolson method)

The numerical method using the 10-point-star Crank-Nicolson method with grid sizes
h and t to solve the 2D heat equation is of order O(h2 ) and O(t2 ). The method
is unconditionally stable.

3.2.5 Difference Methods for 1D Hyperbolic Equations

The third class of linear, second-order partial differential equations deals is called hyper-
bolic equations. Typical applications include second partial derivatives with respect to
space and time. In 1D, central difference formulas yield the hyperbolic 5-point-star. This
molecule looks similar to the 5-point-star of elliptic problems but has different coefficients.
 85

Definition 3.21 (Hyperbolic 5-point-star)

A numerical solution wi,k of the 1D wave equa- t
tion utt = 2 uxx can be obtained using the grid
sizes h and t and the approximation scheme 1

wi,k1 2wi,k wi1,k 2wi,k 22

2
= 2
t h2
which is called the hyperbolic 5-point-star. 1
2
t
The constant is = 2 2 . x
h

In general, the discretization of space and time is different. This implies a coefficient ,
that also depends on the factor of the differential equation. Moreover, three time layers
are included in the hyperbolic 5-point-star.

Linear system for hyperbolic 5-point-star

Using the hyperbolic 5-point-star to solve the 1D wave equation utt = 2 uxx results in
the second-order linear system wk+1 = A wk wk1 with

2 2
t
2
A= , = 2 .
h2
2 2

This linear system is valid for all inner nodes wi,k+1 with 1 i n 1 and k 1.
Boundary conditions for boundary nodes have to be added separately. In addition, a
special initial step is necessary for k = 0.

Since the central difference formula is used for time discretization in the hyperbolic 5-
point-star, this approach is of second order with respect to space and time. The Courant-
Friedrichs-Levy condition describes the limit of stability.

Theorem 3.9 (Order and stability of the hyperbolic 5-point-star)

The numerical method using the hyperbolic 5-point-star with grid sizes h and t to
solve the 1D wave equation utt = 2 uxx is of order O(h2 ) and O(t2 ). The method
is stable if the Courant-Friedrichs-Levy condition holds:
2
t
= 2 1.
h2

Given the initial conditions

u(x, 0) = g(x), ut (x, 0) = g(x) ,

86

ghost nodes for the seed computation in the first step are used:
wi,1 wi,1
= gi wi,1 = wi,1 2 t gi .
2 t
Inserting this formula into the hyperbolic 5-point-star yields

wi,1 = 2(1 )wi,0 + (wi+1,0 + wi1,0 ) wi,1

= 2(1 )gi + (gi+1 + gi1 ) (wi,1 2 t gi ) .

Rearranging the expressions gives the special seed formula

wi,1 = (1 )gi + (gi1 + gi+1 ) + t gi .
2

Initial nodes with hyperbolic 5-point-star

The initial step using the hyperbolic 5-point-star to solve the 1D wave equation
utt = 2 uxx for the initial conditions

u(x, 0) = g(x), ut (x, 0) = g(x)

can be computed using the central difference formula in t-direction with ghost nodes
wi,1 outside the domain:

wi,1 = (1 )gi + (gi1 + gi+1 ) + t gi , 1in1.
2
This method is of order O(t2 ) .

Finally, one can compare the parts of boundary that influence the exact and the numerical
solution. To ensure stability, the numerically used boundary region has to be superset of
the analytical relevant region.

Stability of hyperbolic difference methods

A numerical method using a difference scheme t
is stable if the numerical region of influence cov-
ers completely the theoretical region of influ- u
w
ence.

x
 87

3.3 Finite Element Methods

The first finite element approaches date back to 1943. In the 1950s and 1960s a prac-
tical approach has been established by engineers. After then, in the 1960s and 1970s
mathematicians founded a theoretical basis for finite elements.
The overall idea is to split the domain into small pieces, so-called elements. The original
partial differential equation then is converted into a variational problem. The solution is
a linear combination of basis functions in a finite dimensional function subspace. Finally,
this approach yields a linear system.

3.3.1 Meshes, Partitions, and Triangulations

The basis of any finite element approach is the decomposition of the domain. Independent
of the problem dimension meshes and nodes are the terms used.

Definition 3.22 (Mesh and node)

A mesh decomposes a 1D or 2D domain into a set of elements. In 1D the elements
are subintervals and in 2D the elements can be triangles or quadrangles. The mesh
points are also called nodes.

Typically, in one space dimension a domain is an interval. For 1D problems meshes

specializes to partitions.

Definition 3.23 (Partition)

A partition decomposes a 1D domain into a set of subintervals. The result is a special
mesh on the domain.

Example 3.23 (Partition of an interval)

a) The interval [a, b] = [2, 6] can be decomposed into subintervals in many different ways. One
possible choice is

x0 = a = 2, x1 = 2.5, x2 = 3.5, x3 = 5, x4 = b = 6 .

This partition is not equidistant.

b) The interval [a, b] = [2, 6] can also be decomposed into n equidistant subintervals using

ba
xk = a + h k, h= , k = 0, 1, 2, . . . , n
n
which yields for n = 40 the nodes

x0 = 2, x1 = 2.1, x2 = 2.2, ... , x3 = 2.3, ... , x4 = 2.4, ..., x40 = 6 .

88

For 2D problems meshes can be triangulations. The common way of representing a 2D

triangulation is to use two vectors x and y for the mesh nodes a matrix T for the
triangles. In T only the indices of the nodes are stored. Thus, the elements of T are
natural numbers. The order of the triangle vertices can be either ascending or according
to the orientation. The second variant starts with the vertex with the smallest index and
then continues counter-clockwise.

Definition 3.24 (Triangulation)

A triangulation decomposes a 2D domain into y
a set of triangles T1 , . . . , Tm . The result is a
mesh on the domain. The triangulation can be
described with two vectors x Rn and y Rn
containing the coordinates of the n nodes and
a matrix T Rn3 containing in each row the
indices of the three nodes of the corresponding
triangle.
x

Example 3.24 (Triangulation of a square)

We consider a simple square domain with 1 inner and y
4 boundary nodes:
n3
1 1 2 n2
2 2 1 2 3 T1
1 3 4
x= 0 , y = 2 , T = .
1 4 5 T2 n1 T4
0 0 1
1
2 0 5 2
T3 n5
In each row of T the node indices are arranged start- n4 1 2 3 4 x
ing with the smallest index and then continuing coun-
terclockwise.

Example 3.25 (Triangulation of a domain)

We consider a domain including 1 inner and 5 bound- y
ary nodes. The triangulation consists of 5 triangles:
n3 n2
1 1 2
2.5 2 1 2 3 T1
1 3 4
0 2
x=
0 , y = 0 , T = 1 4 5 . T2 n1 T4
1
1 5 2 T5
2.5 0 2 T3 n6
4 0 5 6
n4 1 2 n5 3 4 x
In each row of T the node indices are arranged start-
ing with the smallest index and then continuing coun-
terclockwise.

 89

Example 3.26 (Triangulation of a house shaped domain)

We consider a domain with 1 inner and 6 boundary y
nodes. The triangulation consists of 6 triangles: n1

2 2.5 2
1 1.5 1 2 3 T1 T2
1 3 4 n2 n3 n4
2 1.5 T3
T5
x= , y = 1.5 , T = 2 3 5 . 1
3 3 T4 T6
0.5 5 6 n5 n7
1
3 4 6 n6
2 0.5 4
0.5 6 7 1 2 3 4 x
3

In each row of T the node indices are arranged in

ascending order.

The process of decomposing a 2D domain is quite different from that for a 1D interval.
Finding good triangulations is an art in some sense. The Delaunay triangulation is such
that the smallest angle in any triangle is as large as possible. This ensures numerical
stability during the calculation of the finite element solution.

Definition 3.25 (Delaunay triangulation)

The Delaunay triangulation of the convex
hull of given nodes is a triangulation where the
smallest angle in any triangle Tk is as large
as possible:

min max
Tk
Delaunay
The Delaunay triangulation has the property
that no other node lies in the circumcircle of
any triangle.
90

Example 3.27 (Delaunay triangulation of a quarter circle)

We consider a quarter circle shaped domain with 1 y
inner and 7 boundary nodes. Hence, 8 x-coordinates 2
and 8 y-coordinates have to be stored.

1 1
0 0 1

1 0

2 0

x = , y= ,
3 1
1 2 3 x
1 3

0 2

0 1

The initial triangulation is changed into a Delaunay y

n7
triangulation with large angles. 2 n6 n1
6 7 8
T1
1 6 8
n8 T2 T3
n5
1 5 6 1

T =
2 3 8 .

T5 T6

1 8 3 T4 T7 n4

1 3 5
n2 1 n3 2 3 x
3 4 5

In each row of T the node indices are arranged starting with the smallest index and then continuing
counterclockwise. Finite elements are well suited for curved shaped domains. Hence, for a quarter
circle domain, the finite element method is advantageous in comparison with the finite difference
method.

3.3.2 Variational Problems

This section is a litte bit theoretical, but includes the main principle of the finite element
approach. The idea is to replace the original partial differential equation either by the
principle of virtual work or the minimum potential energy functional. In the latter case,
an optimization problem is solved.

Definition 3.26 (Differential operator)

A partial differential equation can be written with a differential operator in the form

Lu = f .
 91

Example 3.28 (Poisson equation with differential operator)

a) The 1D Poisson equation u = f (x) can be reformulated with a differential operator:

d2
Lu = f, L= .
d x2

b) In 2D, the Poisson equation uxx + uyy = f (x, y) can be written with an operator:

2 2
Lu = f, L = = 2 = + .
x2 y 2
Sometimes, the equation is multiplied by 1, yielding

Lu = f, L = , f = f .

Definition 3.27 (Inner product)

A mapping u, v of two functions u and v into a real number is called an inner product
if the mapping is
symmetric: u, v = v, u
positive definite: u, u > 0 for u 0
linear: 1 u1 + 2 u2 , v = 1 u1 , v + 2 u2 , v
The inner product is a symmetric, positive definite bilinear form.
92

Example 3.29 (Integral expressions as inner products)

a) The integral of the product of two functions u and v on a 1D interval [a, b]
b
u, v = u(x) v(x) d x
a

defines an inner product. Symmetry, linearity, and positive definiteness result directly form
the corresponding properties of the integral.
b) Likewise, the integral of the product of two functions u and v on a 2D domain D

u, v = u(x, y) v(x, y) d x d y
D

defines an inner product.

c) Using the first derivatives of two functions u and v on a 1D interval [a, b] the expression
b
u , v = u (x) v (x) d x
a

is an inner product.
d) In the same way, using the gradients of two functions u and v on a 2D domain D

u, v = u(x, y) v(x, y) d x d y
D

defines an inner product. Note that the function within the integral is real-valued.

Laplace operator and inner product

For the negative Laplace operator Lu = u the inner product is

Lu, v = u, v .

For the negative Laplace operator Lu = u the inner product with a function v is defined
as

Lu, v = u, v = u v dx dy .
D

Using Greens formula the inner product can be written as

u
Lu, v = v ds + u v dx dy = u, v .
D n D

0
u
Here, n denotes the normal derivative of u. The last expression shows that it is useful
to start with the negative Laplace operator instead of the Laplace operator itself.
 93

Definition 3.28 (Properties of a differential operator)

A differential operator L is called
symmetric if Lu, v = u, Lv
positive definite if Lu, u > 0 for u 0
94

Example 3.30 (Properties of the negative Laplace operator)

a) We consider the negative 1D Laplace operator Lu = u on [a, b] together with the integral
inner product for functions that are zero on the interval bounds a and b. First, we show the
symmetry of the Laplace operator: Using integration by parts we get
b b
u (x) v(x) d x = u (x) v(x) + u (x) v (x) d x .
b
Lu, v =
a a a

Since v(a) = v(b) = 0, the first expression vanishes. Likewise, we obtain

b b
u(x) v (x) d x = u(x) v (x) + u (x) v (x) d x .
b
u, Lv =
a a a

Here again, the first expression vanishes. Second, we proof positive definiteness:
b b
u (x) u(x) d x = u (x) u(x) + (u (x))2 d x > 0
b
Lu, u =
a a a

for any function u 0 with u(a) = u(b) = 0. From this, we can understand that for positive
definiteness it is necessary to use the negative of the original Laplace operator.
b) In 2D, the negative Laplace operator is also symmetric and positive definite for functions
that are zero on the boundary D. Starting with

Lu, v = u(x, y) v(x, y) d x d y

D

and using Greens formula we obtain

u
Lu, v = (s) v(s) d s + u(x, y) v(x, y) d x d y .
D n D

u
Here, denotes the derivative with respect to the normal direction n on the boundary of
n
the domain. With v = 0 on D again the first expression vanishes. Likewise, starting with

u, Lv = u(x, y) v(x, y) d x d y
D

and again using Greens formula yields

v
u, Lv = u(s) (s) d s + u(x, y) v(x, y) d x d y .
D n D

With u = 0 on D we finally get symmetry. Positive definiteness can be seen starting with

Lu, u = u(x, y) u(x, y) d x d y

D

and once again using Greens formula:

u
Lu, u = (s) u(s) d s + (u(x, y))2 d x d y > 0 .
D n D

This holds for any function u 0 with u = 0 on D.

One of the key ideas of the finite element approach is to rewrite the original partial
differential equation as a variational or an optimization problem. The strong formulation
 95

then changes into a weak formulation.

Theorem 3.10 (Differential equation and variational problems)

For a symmetric, positive definite, and linear differential operator L the following prob-
lems are equivalent:
(A) The function u solves the partial differential equation

Lu = f .

(B) The function u satisfies the principle of virtual work

Lu, v = f, v for all functions v .

(C) The function u minimizes the potential energy functional

1
F (v) = Lv, v f, v
2

The theorem is fundamental. Hence, it is worth to proof the equality between the three
formulations.
a) (A) (B)
Take the inner product on both sides of the equation Lu = f with an arbitrary
function v.
b) (B) (A)
The opposite direction follows immediately with the fundamental lemma of the cal-
culus of variations: If Lu f, v = 0 for any function v then Lu f has to be the
zero function.
c) (B) (C)
Let v be an arbitrary function. Then, with w = v u we get
1
F (v) = F (u + w) = L(u + w), u + w f, u + w .
2
Applying the linearity of L and the inner product, the functional F (v) reads
1
F (v) = (Lu, u + Lu, w + Lw, u + Lw, w) (f, u + f, w) .
2
Since L is assumed to be symmetric, the second and third inner product are the
same:
1 1
F (v) = Lu, u f, u + Lu, w f, w + Lw, w .
2 2

F (u) 0 0
96

The last expression is positive, since L is assumed to be positive definite. The

inequality F (v) F (u) shows the minimum of F for the function u.
d) (C) (B)
Again, let v be an arbitrary function. Then, we consider the one-dimensional function
() = F (u + v) which can be expanded to
1
() = (Lu, u + Lu, v + Lv, u + 2 Lv, v) (f, u + f, v) .
2
Using the symmetry of L we can again combine the second and third inner product.
Hence, the derivative with respect to is

() = Lu, v + Lv, v f, v .

F has a minimum at = 0. This yields

0 = (0) = Lu, v f, v .

Theorem 3.11 (Variational problems for Poisson equation)

For the Poisson equation the following problems are equivalent:
(A) The function u solves the partial differential equation

u =f.

(B) The function u satisfies the principle of virtual work

u, v + f, v = 0 for all functions v S .

(C) The function u minimizes the potential energy functional

1
F (v) = v, v + f, v
2

Using again the negative Laplace operator Lu = u in combination with the function
f = f the main theorem specializes to the following:
(A) The original partial differential equation yields the Poisson equation:

Lu = f u = f u =f .

(B) The original principle of virtual work can be written as the following equation:

Lu, v = f, v u, v = f, v u, v = f, v .
 97

(C) The original potential energy functional specializes to

1 1 1
F (v) = Lv, v f, v = v, v f, v = v, v + f, v .
2 2 2

Hence, for the Laplace or Poisson equation the principle of virtual work and the potential
energy functional can be formulated with gradients.

3.3.3 Function Spaces

According to the strong or weak formulation of the partial differential equation the solution
has to fulfill different conditions of differentiability. It is common to search numerical
approximations of the solution in finite-dimensional subspaces. This results in a well-
posed problem.

Function spaces
A solution u with u = 0 on D of a partial differential equation on a domain D can be
searched for in the following function spaces S S S :
The function space with strong differentiability conditions on u

S = {u D R u C 2 (D)} .

The function space with weak differentiability conditions on u

S = {u D R u C(D), u C 1 piecewise on D} .

A finite-dimensional subspace of S that consists of all piecewise linear functions on

a given mesh of the domain D

S = {u D R u C(D), u piecewise linear on the mesh} .

3.3.4 Piecewise Linear Finite Elements

A numerical solution can be represented by a piecewise linear function. On one hand, such
elements are simple to understand and easy to use. On the other, the precision achieved
is limited. Piecewise linear functions are composed of hat functions. These hat functions
form a basis of the finite-dimensional subspace of all piecewise linear functions. Moreover,
the support of hat functions, that is that part of the domain, where the functions are non-
zero is quite small.
98

Definition 3.29 (1D hat functions)

The continuous 1D hat functions on the mesh y
a = x1 < . . . < xn = b of the interval [a, b]

1 if x = xj

j (x)
j (x) = 0 if x = xi , i j 1

linear

otherwise

with 1 j n define a basis of the space of all

piecewise linear functions S on the mesh. a xj b x

Similar to 1D, in 2D hat functions also have a very small support. They look like a
carneval hat or a pyramid.

Definition 3.30 (2D hat functions)

The continuous 2D hat functions on a mesh z
with nodes (x1 , y1 ), . . . , (xn , yn ) of the domain 1
D j (x, y)

1 if (x, y) = (xj , yj )


j (x, y) = 0 if (x, y) = (xi , yi ), i j yj



xj y
linear otherwise x
with 1 j n define a basis of the space of all
piecewise linear functions S on the mesh.

3.3.5 Galerkins Method

Addressing the principle of virtual work which is variant (B) in the main theorem, this
approach is called Galerkins method. A linear combination of the hat functions
n
w = cj j
j=1

is used for the unknown function u. Likewise, the arbitrary function v is one after the
other replaced by the hat functions 1 , . . . , n . In this way, n equations are obtained.
 99

Definition 3.31 (Galerkins method)

A numerical solution w of the differential equation Lu = f using Galerkins method
can be found by satisfying the principle of virtual work

Lu, v = f, v

on a finite-dimensional function space S . With basis functions 1 , . . . , n satisfying

homogeneous boundary conditions the solution has the form
n n
w(x) = cj j (x) or w(x, y) = cj j (x, y) .
j=1 j=1

Using u = w and v = j for 1 j n the principle of virtual work then consists of n

equations for the coefficients c1 , . . . , cn .

3.3.6 Rayleigh-Ritzs Method

Addressing the potential energy functional which is variant (C) in the main theorem,
this approach is called Rayleigh-Ritzs method. Again, a linear combination of the hat
functions
n
w = cj j
j=1

is used for the function v. As a necessary condition the gradient of the functional is set
to zero. Like in the Galerkin approach, here also n equations are obtained.

Definition 3.32 (Rayleigh-Ritzs method)

A numerical solution w of the differential equation Lu = f using Rayleigh-Ritzs
method can be found by minimizing the potential energy functional
1
F (v) = Lv, v f, v
2
on a finite-dimensional function space S . With basis functions 1 , . . . , n satisfying
homogeneous boundary conditions the solution has the form
n n
w(x) = cj j (x) or w(x, y) = cj j (x, y) .
j=1 j=1

Using v = w the potential energy functional F then is a function F (c1 , . . . , cn ) and

optimal values for the coefficients c1 , . . . , cn are determined by solving F = 0.

Now we focus on Rayleigh-Ritzs approach for the Poisson equation. In the following, we
100

omit the functions arguments. Inserting

n n
w = cj j w = cj j
j=1 j=1

in the functional we obtain

1 n n n
F (w) = cj j , cj j + f , cj j .
2 j=1 j=1 j=1

This functional can be interpreted as a function depending on c1 , . . . , cn :

1 n n n
F (c1 , . . . , cn ) = cj ck j , k + cj f, j .
2 j=1 k=1 j=1

The partial derivative with respect to ci then reads

1 n
F (c1 , . . . , cn ) = 2 cj i , j + 2ci i , i + f, i .
ci 2 ji

The first two expressions can be collected in one sum, and then the partial derivative is
set to zero:
n

F (c1 , . . . , cn ) = cj i , j + f, i = 0 .
ci j=1

The inner products can be interpreted as matrix and vector elements:

ai,j = i , j , bi = f, i , i, j = 1, . . . , n .

Note, that the matrix defined as above is symmetric.

Definition 3.33 (Stiffness matrix for Poisson equation)

The symmetric stiffness matrix A Rnn for the 1D Poisson equation consists of the
values

ai,j = i (x) j (x) dx, i, j = 1, . . . , n

D

and for the 2D Poisson equation of the values

ai,j = i (x, y) j (x, y) dx dy, i, j = 1, . . . , n .

D

The functions 1 , . . . , n are the basis functions on the domain D.

 101

Definition 3.34 (Load vector for Poisson equation)

The load vector b Rn for the 1D Poisson equation consists of the values

bi = f (x) i (x) dx, i = 1, . . . , n

D

and for the 2D Poisson equation of the values

bi = f (x, y) i (x, y) dx dy, i = 1, . . . , n .

D

The functions 1 , . . . , n are the basis functions and f is the right-hand side of the
Poisson equation on the domain D.

Using the stiffness matrix and the load vector the Rayleigh-Ritz method for the Poisson
equation finally can be written as a linear system: Ac = b.

Theorem 3.12 (Linear system for Poisson equation)

The values of the Rayleight-Ritz coefficients c1 , . . . , cn can be obtained by solving the
linear system Ac = b, where A is the stiffness matrix and b is the load vector.

The next step addresses the question how the stiffness matrix can be computed. Calcu-
lating the elements ai,j one after the other in two nested loops is straightforward but not
efficient, because same integrals are computed several times.

Assembly
The stiffness matrix A and the load vector b can be assembled
nodewise: Each value ai,j and bi is computed separately. This approach is time-
consuming since same parts of the integrals are computed several times.
elementwise: The integrals are computed locally on each mesh element Tk . This
approach generates element stiffness matrices Ak and element load vectors bk .
102

The element-wise assembly calculates the inte-

grals separately on each triangle element. Let
nk1
Tk be one triangle with nodes

nk 1 , nk2 , nk3 . Tk nk3

Since the hat function j is non-zero only on

those triangles that include node nj , we obtain nk2
a non-zero integral value on Tk only if

i, j {k1 , k2 , k3 } .

This means, that for each triangle, 9 integrals contributing to the stiffness matrix have to
be computed. These 9 values define the element stiffness matrix.

Definition 3.35 (Element stiffness matrix for 2D Poisson equation)

A symmetric element stiffness matrix Ak R33 for the 2D Poisson equation using
a triangulation with triangles Tk consists of the values

ai,j = ki (x, y) kj (x, y) dxdy, i, j = 1, . . . , 3 .

(k)
Tk

The functions k1 , k2 , and k3 are the basis functions corresponding to the nodes
k1 , k2 , and k3 of triangle Tk .

In the same way, only if i {k1 , k2 , k3 } we obtain a non-zero entry in the load vector.
Thus, 3 integrals contribute to the element load vector.

Definition 3.36 (Element load vector for 2D Poisson equation)

An element load vector bk R3 for the 2D Poisson equation using a triangulation
with triangles Tk consists of the values

= f (x, y) ki (x, y) dxdy, i = 1, . . . , 3 .

(k)
bi
Tk

The functions k1 , k2 , and k3 are the basis functions corresponding to the nodes
k1 , k2 , and k3 of triangle Tk . The function f is the right-hand side of the Poisson
equation.

Now we address to the computation of the element stiffness matrix. We calculate the 9
 103

(k)
values ai,j simultaneously. Let

xk1 xk2 xk 3
nk 1 = ( ), nk2 = ( ), nk3 = ( )
yk1 yk2 yk3
be the coordiantes of the three nodes of triangle Tk . On one hand, the numerical solution
w on Tk is a linear combination of the three relevant hat functions:
w(x, y) = ck1 k1 (x, y) + ck2 k2 (x, y) + ck3 k3 (x, y) .
The transposed gradient of w can be written as a matrix vector product:
T w(x, y) = ck1 T k1 (x, y) + ck2 T k2 (x, y) + ck3 T k3 (x, y)
ck1
= (T k1 (x, y), T k2 (x, y), T k3 (x, y)) ck2 .
ck3

On the other hand, the numerical solution w on Tk is a linear function:

w(x, y) = 1 + 2 x + 3 y .
At the triangle nodes the solution takes the values ck1 , ck2 , and ck3 :

1 xk1 yk1 1 ck1

1 xk2 yk2 2 = ck2 .
1 xk3 yk3 3 ck3

Fk
Since Tk is a triangle, matrix F k is regular and dk = F k 0. The determinant dk
is positive, if the nodes k1 , k2 , k3 are ordered counterclockwise and negative otherwise.
Hence, we can use Cramers rule to state the components 2 and 3 explicitly:

y yk3 yk3 yk1 yk1 yk2 k1

c
2 1
T w(x, y) = ( )= ( k2 ) ck2 .
3 dk xk3 xk2 xk1 xk3 xk2 xk1 c
k3
Bk
Now we can compare the two representations of T w. Since ck1 , ck2 , and ck3 are arbitrary
numbers we have equality of the two matrices:
(T k1 (x, y), T k2 (x, y), T k3 (x, y)) = B k .
The gradients of the hat functions are row vectors with constant numbers. This implies

k1 1
Ak = k2 (T k1 , T k2 , T k3 ) 1 dx dy = B Tk B k dk
k3 T k 2

using the area 1

2
dk of triangle Tk .
104

Element stiffness matrix for 2D Poisson equation with hat functions

The element stiffness matrix Ak R33 on a triangle Tk with vertices

xk1 xk 2 xk3
( ), ( ), ( )
yk1 yk2 yk3

can be computed with

1
Ak = dk B Tk B k
2
using the area 1
2
dk of the triangle Tk and coordinates matrix B k :

RRR 1 xk1 yk1 RRR

R RRR y yk3 yk3 yk1 yk1 yk2
dk = RRRRR 1
1
xk2 yk2 RRR , Bk = ( k2 ).
RRR 1 xk3 yk3 RRR dk xk 3 xk 2 xk1 xk3 xk2 xk1
R R
 105

Example 3.31 (Element stiffness matrices for a square domain)

We again consider example 3.24. The determinants for T1 and T2 are
RRR 1 1 1 RRR RRR 1 1 1 RRR
RR RRR RR RRR
d1 = RRRR 1 2 2 RRR = 2 , d2 = RRRR 1 0 2 RRR = 2 ,
RRR RRR RRR RRR
RR 1 0 2 RR RR 1 0 0 RR
and in the same way we obtain d3 = 2 and d4 = 2. Since all triangle nodes are in counterclockwise
order, all values dk are positive. The matrix consisting of the gradients of triangle T1 reads

1 22 21 12 1 0 1 1
B1 = ( )= ( ) .
2 02 10 21 2 2 1 1

Likewise, the matrix consisting of the gradients of triangle T2 is

1 20 01 12 1 2 1 1
B2 = ( )= ( ) .
2 00 10 01 2 0 1 1

In the same way, for the two other triangles we get

1 0 1 1 1 2 1 1
B3 = ( ) , B4 = ( ) .
2 2 1 1 2 0 1 1

When calculating B 4 we have to be careful about the not ascending order of the node indices:
1, 5, 2. This means k1 = 1, k2 = 5, and k3 = 2. Using the formula Ak = 12 dk B Tk B k , in this
example we obtain 4 equal stiffness matrices:

1 1
1
2
A1 = A2 = A3 = A4 = 1 1 0 .
2
1 0 1

Note the special structure of the element stiffness matrices. The diagonal elements are positive.
The sum in each row and column is zero.
106

Example 3.32 (Element stiffness matrices)

We again consider example 3.25. The determinants for T1 and T2 are
RRR 1 1 1 RRR RRR 1 1 1 RRR
RR RRR RR RRR
d1 = RRRR 1 2.5 2 RRR = 2.5 , d2 = RRRR 1 0 2 RRR = 2
RRR RRR RRR RRR
RR 1 0 2 RR RR 1 0 0 RR
and in the same way we obtain d3 = 2.5, d4 = 3, and d5 = 3. Since all triangle nodes are ordered
counterclockwise, all values dk are positive. The matrix consisting of the gradients of triangle
T1 reads

1 22 21 12 0 0.4 0.4
B1 = ( )=( ) .
2.5 0 2.5 10 2.5 1 1 0.4 0.6

Likewise, the matrix consisting of the gradients of triangle T2 is

1 20 01 12 1 0.5 0.5
B2 = ( )=( ) .
2 00 10 01 0 0.5 0.5

We also show how the matrix with the gradients of triangle T3 is computed:

1 00 01 10 0 0.4 0.4
B3 = ( )=( ) .
2.5 2.5 0 1 2.5 01 1 0.6 0.4

For the remaining triangle T4 we obtain

1 02 21 10 0.667 0.333 0.333

B4 = ( )=( ) ,
3 2.5 2.5 1 2.5 2.5 1 0 0.5 0.5

and finally for the last triangle T5

1 00 02 20 0 0.667 0.667
B5 = ( )=( ) .
3 4 2.5 2.5 4 2.5 2.5 0.5 0.5 0

Using the formula Ak = 1

2
dk B Tk B k we obtain the first element stiffness matrix

1.25 0.5 0.75

1
A1 = d1 B T1 B 1 = 0.5 0.4 0.1 .
2 0.75 0.1 0.65

The next two element stiffness matrices read

1 0.5 0.5 1.25 0.75 0.5

A2 = 0.5 0.5 0 A3 = 0.75 0.65 0.1
0.5 0 0.5 0.5 0.1 0.4

and the last two stiffness matrices are

0.667 0.333 0.333 0.375 0.375 0

A4 = 0.333 0.542 0.208 A5 = 0.375 1.042 0.667 .
0.333 0.208 0.542 0 0.667 0.667

In matrices B k and Ak , some numbers are truncated to 3 digits after the decimal point.
 107

Example 3.33 (Element stiffness matrices for a house shaped domain)

We again consider example 3.26. The determinants for the 6 triangles are

d1 = 1, d2 = 1, d3 = 1, d4 = 1, d5 = 1, d6 = 1.

Since the orientation of the triangle nodes is not the same we obtain positive and negative values.
The absolute value is constant because the triangulation is regular and all triangles have the same
area. The matrices B k read

1 0 1 1 1 0 1 0 1
B1 = B4 = B6 = ( ) , B2 = ( ) , B3 = B5 = ( ).
0 1 1 0 1 1 1 1 0

The equality of some of the matrices B k is reflected in the element stiffness matrices. The first
two different matrices are
1 1
1 1
1 0 1 0
A1 = A4 = A6 = 0 1 1 , A2 = 1 2 1 ,
2 2
1 1 2 0 1 1

and the third appearing matrix reads

1 1
1
2
A3 = A5 = 1 1 0 .
2
1 0 1

108

Example 3.34 (Element stiffness matrices for a quarter circle domain)

We again consider example 3.27. The determinants for T1 and T2 are
R
RRRR 1 1 3 RRR RRRR 1 1 RR
RR
R R R 1 RRRR
d1 = RR 1 0 2 RRRR = 1 , d2 = RRRR 1 1 3 RRR = 3 1 ,
RRR R RRR R
RR 1 0 1 RRRR RR 1 0 1 RRRR

and in the same way we obtain

d3 = 2(2 3) , d4 = 1 , d5 = 1 , d6 = 31, d7 = 1 .

Since all triangle nodes are in counterclockwise order, all values dk are positive. The matrix
consisting of the gradients of triangle T1 reads

1 21 1 3 32 1 1 3 32
B1 = ( )=( ) ,
1 00 10 01 0 1 1

Likewise, the matrix consisting of the gradients of triangle T2 is

1 31 11 1 3 1 31 0 1 3
B2 = ( )= ( ) .
31 01 10 11 31 1 1 0

In the same way matrix B 3 can be computed:

1 1 3 31 0
B3 = ( ) .
2(2 3) 1 3 0 31

The next two matrices read

1 1 0 1 1 0
B4 = ( ) , B5 = ( ) ,
1 0 1 1 0 1

and the last two matrices are similar to the first two matrices:
1 1 0 1 1 1 0
B6 = ( ) , B7 = ( ) .
31 31 1 3 0 32 1 3 1

For the element stiffness matrices we switch to floating point numbers:

0.5 0.366 0.134 1.049 0.683 0.366

A1 = 0.366 0.768 0.402 , A2 = 0.683 0.683 0 .
0.134 0.402 0.536 0.366 0 0.366

The next element stiffness matrices are

1 0.5 0.5
A3 = A4 = A5 = 0.5 0.5 0
0.5 0 0.5

and the last two element stiffness matrices read

1.049 0.366 0.683 0.536 0.402 0.134

A6 = 0.366 0.366 0 , A7 = 0.402 0.768 0.366 .
0.683 0 0.683 0.134 0.366 0.5

 109

In the same way the element load vector can be calculated. Since the function f of the
Poisson equation in general is nonlinear, the resulting integrals have to be computed using
numerical integration methods. We do not discuss this issue here.

Elementwise assembly for triangulations

If the mesh is a triangulation with m triangles the elementwise assembly can be per-
formed as follows:
(1) Initialize the stiffness matrix A = 0 and the load vector b = 0.

(2) For k = 1, . . . , m: Add element matrix Ak R33 and element vector bk R3 of

triangle Tk with nodes k1 , k2 , and k3 to A and b.

k1

k2

,

k3



k1 k2 k3

Example 3.35 (Assembly for a square domain)

We once again consider example 3.24 with the element stiffness matrices from example 3.31.
The assembly starts with A = 0. In the first two steps, matrices A1 and A2 are added to A:

2 1 1 0 0 4 1 2 1 0
1 1
1 1
1 0 0 0 1 0 0 0

1 0 1 0 0 2 0 2 0 0 .
2
0

2
1


0 0 0 0 0 0 1 0
0 0 0 0 0 0 0 0 0 0

Next, matrix A3 is added. Due to the non-ascending node order we have to add A4 carefully:

6 1 2 2 1 8 2 2 2 2
1 2
1 1
1 0 0 0 2 0 0 0

2 0 2 0 0 2 0 2 0 0 .
2


2



2 0 0 2 0 2 0 0 2 0
1 0 0 0 1 2 0 0 0 2

The first row of A remembers of the 5-point-star finite difference method:

4 1 1 1 1
1 1 0 0 0

A= 1 0 1 0 0 .

1 0 0 1 0
1 0 0 0 1

110

Example 3.36 (Assembly)

We once again consider example 3.25 with the element stiffness matrices from example 3.32.
The assembly starts with A = 0. After adding 5 matrices we obtain the stiffness matrix

4.167 0.833 1.25 1.25 0.833 0

0.833 1.317 0.1 0 0.583 0

1.25 0.1 1.15 0 0 0
A=
1.25
.

0 0 1.15 0.1 0

0.833 0.583 0 0.1 1.983 0.667
0 0 0 0 0.667 0.667

 111

Example 3.37 (Assembly for a house shaped domain)

We once again consider example 3.26 with the element stiffness matrices from example 3.33.
The assembly starts with A = 0. In the first two steps, matrices A1 and A2 are added to A:

1 0 1 0 0 0 0 2 0 2 0 0 0 0
0 1 1 0 0 0 0 0 1 1 0 0 0 0

1 1 2 0 0 0 0 2 1 4 1 0 0 0
1
0
1
0

.
1
2 2
0 0 0 0 0 0 0 1 0 0 0
0 0
0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0

In the next two steps, matrices A3 and A4 are added to A:

2 0 2 0 0 0 0 2 0 2 0 0 0 0
0 3 2 0 1 0 0 0 3 2 0 1 0 0

2 2 5 1 0 0 0 2 2 6 1 0 1 0
1
0
1
0

.
1 1
2 2
0 1 0 0 0 0 1 0 0 0
0 1 0 1 1
0 0 1 0 0 0 0 2 0

0 0 0 0 0 0 0 0 0 1 0 1 2 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0

In the last two steps, matrices A5 and A6 are added to A:

2 0 2 0 0 0 0 2 0 2 0 0 0 0
0 3 2 0 1 0 0 0 3 2 0 1 0 0

2 2 8 2 0 2 0 2 2 8 2 0 2 0
1
0
1
0

.
2 2 1
2 2
0 2 0 0 0 0 3 0 0
0 1 1 0 1 1
0 0 2 0 0 0 2 0

0 0 2 0 1 3 0 0 0 2 0 1 4 1
0 0 0 0 0 0 0 0 0 0 1 0 1 2

After adding 6 matrices we obtain the stiffness matrix

1 0 1 0 0 0 0
0 1.5 1 0 0.5 0 0

1 1 4 1 0 1 0

A=
0 0 1 1.5 0 0 0.5 .

0 0.5 0.5
0 0 1 0

0 0 1 0 0.5 2 0.5
0 0 0 0.5 0 0.5 1

112

Example 3.38 (Assembly for a quarter circle domain)

We once again consider example 3.27 with the element stiffness matrices from example 3.34.
The assembly starts with A = 0. After adding 7 matrices we obtain the stiffness matrix

4.098 0 0.866 0 1.183 1.183 0 0.866

0 1 0.5 0 0 0 0 0.5

0.866 0.5 1.902 0.402 0.134 0 0 0

0 0 0.402 0.768 0.366 0 0 0
A=

.

1.183 0 0.134 0.366 1.683 0 0 0

1.183 0 0 0 0 1.683 0.366 0.134

0 0 0 0 0 0.366 0.768 0.402
0.866 0.5 0 0 0 0.134 0.402 1.902

For the Laplace equation the load vector b up to now is just a zero vector. A nonzero
numerical solution is obtained if boundary conditions are added. As in the finite difference
approach, here we also impress the given boundary conditions on the linear system Ac = b.

Boundary nodes with Dirichlet condition using hat functions

To add Dirichlet boundary conditions the linear system Ac = b is modified. Any row
k of A that refers to a boundary node with Dirichlet conditions is replaced by zeros
and the value 1 on the diagonal. The corresponding element bk of the right-hand side
vector is replaced by a given Dirichlet value gk :

ak,` = 0, ` k, ak,k = 1, bk = gk .

Example 3.39 (Boundary conditions for a square domain)

We consider the result of example 3.35. Impressing Dirichlet boundary conditions g2 , . . . , g5 on
the boundary nodes n2 , . . . , n5 yields the simple linear system

4 1 1 1 1 c1 0
0 1 0 0 0 c2 g2

0 0 1 0 0 c3 = g3 .

0 0 0 1 0 c4 g4
0 0 0 0 1 c5 g5

Hence, the value of the unknown inner node w1 is just the average of the boundary nodes:
1
w1 = (g2 + g3 + g4 + g5 ) .
4

 113

Example 3.40 (Boundary conditions for a domain)

We consider the result of example 3.36. Impressing Dirichlet boundary conditions g2 , . . . , g6 on
the boundary nodes n2 , . . . , n6 yields the linear system

4.167 0.833 1.25 1.25 0.833 0 c1 0

0 1 0 0 0 0 c2 g2

0 0 1 0 0 0 c3 g3
= .

0 0 0 1 0 0 c4 g4

0 0 0 0 1 0 c5 g5
0 0 0 0 0 1 c6 g6

Hence, the value of the unknown inner node w1 is a weighted average of the 4 adjacent boundary
nodes. Boundary value g6 is not relevant for w1 .

Now we have discussed the complete procedure of deriving the overall linear system that
results from the finite element approach using Rayleigh-Ritzs method with hat functions
for the Laplace equation.

Ritzs method with hat functions for Laplace equation with Dirichlet boundary
The finite element solution using Rayleigh-Ritzs method with hat functions k on a
mesh with n nodes and m elements for the 2D Laplace equation with Dirichlet boundary
can be computed as follows:

(1) Initialize the stiffness matrix A Rnn and the load vector b Rn to 0.
(2) Compute the element stiffness matrices Ak and add them to A for k = 1, . . . , m.
(3) Impress Dirichlet boundary conditions on A and b.
(4) Solve the linear system Ac = b. The numerical solution is
n
w(x, y) = cj j (x, y) .
j=1
114

3.4 Applications
3.4.1 Heat Distribution
Example 3.41 (Steady-state heat distribution on a square metal plate)
Consider the 2D Laplace equation uxx + uyy = 0 on y
the given domain D with Dirichlet boundary condi- 10
tions on the outer boundary 8

u(x, 10) = 100, u(x, 0) = u(0, y) = u(10, y) = 0 6

4
and Dirichlet boundary conditions on the inner
boundary (hole) 2

u(x, 7) = 100, u(x, 3) = u(3, y) = u(7, y) = 0. 2 4 6 8 10 12 x

The function u(x, y) describes the steady-state tem-
perature distribution on the metal plate. The solu-
tion of the elliptic problem can be computed with
finite differences using the 5-point-star.

Example 3.42 (Time-dependent heat distribution on a square metal plate)

Consider the 2D heat equation uxx + uyy = 0 on the y
given domain D with Dirichlet boundary conditions 10
on the outer boundary 8

u(x, 10, t) = 100 6

u(x, 0, t) = u(0, y, t) = u(10, y, t) = 0 4

and Dirichlet boundary conditions on the inner 2

boundary (hole)
2 4 6 8 10 12 x
u(x, 7, t) = 100
u(x, 3, t) = u(3, y, t) = u(7, y, t) = 0.

The initial conditions are y

10
u(x, y, t) = 0, (x, y) D D . 8
The function u(x, y, t) describes the time-dependent 6
temperature distribution on the metal plate. The so-
4
lution of the parabolic problem can be computed with
finite differences using the Crank-Nicolson method. 2

2 4 6 8 10 12 x

 115

Example 3.43 (steady-state heat distribution on a round metal plate)

Consider the 2D Laplace equation uxx + uyy = 0 on y
the given domain D with Dirichlet boundary condi- 10
tions 8
u = 2, (x 5)2 + (y 5)2 = 22 6
u = cos(2), (x 5)2 + (y 5)2 = 52 . 4
The function u(x, y) describes the steady-state tem- 2
perature distribution on the metal plate. The solu-
tion of the elliptic problem can be computed with 2 4 6 8 10 12 x
finite elements using Ritzs method with hat func-
tions.

116

3.5 Exercises
Theoretical Exercises
Exercise 3.1
Show that the function

u(x, y) = ex+y arccos(x y)

solves the partial differential equation

ux + uy 2u = 0 .

Exercise 3.2
Consider Laplaces differential equation

uxx + uyy = 0 .

a) For which constant c is the function u = x3 + cxy 2 a solution of the differential equation?

b) Show that u = ln (x a)2 + (y b)2 solves the differential equation.

Exercise 3.3
State the type of the following linear second-order PDEs. If the type changes for different
values of the independent variables, indicate precisely how the type varies with the independent
variables.
a) uxy = ux b) (x2 1)uxx + 2xyuxy + (y 2 1)uyy = xux + yuy

Exercise 3.4
Show that the local truncation error for the Crank-Nicolson method for the heat equation ut =
uxx is O(t2 ) + O(x2 ) provided that the solution is sufficiently differentiable.
 117

Computational Exercises
Exercise 3.5
Consider the 2D-Laplace equation

uxx + uyy = 0, 0 x, y 1

with mixed boundary conditions

u(x, 0) = 0, u(0, y) = 10, uy (x, 1) = 3, ux (1, y) = 0 .

a) Set up a grid over the domain using the step size h = 0.25 . Introduce the nodes wi,j
u(xi , yj ) for 0 i, j 4 (including boundary nodes).
b) Discretize the problem using the 5-point stencil for the PDE and the central difference formula
for the Neumann boundaries.
c) Transform the 2D nodes w0,0 to w4,4 into a 1D node vector w using the reading order. Set
up the matrix A R2525 and the vector b R25 for the linear system Aw = b.

Exercise 3.6
Consider the 2D Laplace equation

uxx + uyy = 0, 0 x, y 3

with Dirichlet boundary conditions

u(x, 3) = 100, u(x, 0) = u(0, y) = u(3, y) = 0 .

a) Discretize the problem using the 5-point-star with the grid size h = 1.
b) Set up a linear system Aw = b with A R44 and b R4 for the 4 inner nodes.
c) Solve the linear system Aw = b.
d) Triangulate the squares in the grid by lines from left down to right up.
e) Solve the problem with finite element approach. Use Ritzs method with 2D hat functions.
f) Compare the finite difference solution with the finite element solution.
118

Exercise 3.7
Consider the 2D Laplace equation y
uxx + uyy = 0, 0 x, y 2
2
with Dirichlet boundary conditions

u(x, 2) = 100, u(x, 0) = u(0, y) = u(2, y) = 0 1

on the given domain.

1 2 3 x

a) Set up the stiffness matrix and the load vector of the linear system Ac = b using Ritzs method
with 2D hat functions.
b) Solve the linear system Ac = b.

Exercise 3.8
Consider the 2D Laplace equation y
uxx + uyy = 0
2
with Dirichlet boundary conditions

u(x, 2) = 100, u(x, y) = 0, (x, y) D, y 2 1

on the given domain D.

1 2 3 x

a) Set up the stiffness matrix and the load vector of the linear system Ac = b using Ritzs method
with 2D hat functions.
b) Solve the linear system Ac = b.
 119

Application-based Exercises
Exercise 3.9
Consider the 2D-Laplace equation u = uxx + uyy = 0
y
on the following domain with the Dirichlet boundary 10
conditions on the outer boundary
8
u(x, 10) = 100, u(x, 0) = u(0, y) = u(10, y) = 0 6
and with the Dirichlet boundary conditions on the inner 4
boundary (hole) 2
u(x, 7) = 100, u(x, 3) = u(3, y) = u(7, y) = 0 .
2 4 6 8 10 12 x

a) Set up the finite difference method for the steady-state heat problem for the Laplace equation
with Dirichlet boundary conditions using a common step size h = 1.
b) MATLAB: Solve the resulting linear system and give a surface plot of the solution.
c) MATLAB: Give a corresponding isotherm plot.
d) Repeat parts a) through c) using the step size h = 0.5.

Exercise 3.10
Consider the 2D-Laplace equation u = uxx + uyy = 0
y
on the following domain D with the Dirichlet boundary 10
conditions on the upper boundary
8
u(x, 8) = 100 6
and with zero Dirichlet boundary conditions on the rest 4
of the boundary 2
u(x, y) = 0, (x, y) D, y8
2 4 6 8 10 12 x

a) Set up the finite difference method for the steady-state heat problem for the Laplace equation
with Dirichlet boundary conditions using the common step sizes h = 1 and h = 0.5.
b) MATLAB: Solve the resulting linear system and give a surface plot of the solution.
120

Exercise 3.11
Consider the 2D-Laplace equation u = uxx + uyy = 0
y
on the following domain D with the Dirichlet boundary 10
conditions on the outer boundary
8
u(x, 10) = 100, u(x, 0) = u(0, y) = u(10, y) = 0 6
and with the Dirichlet boundary conditions on the inner 4
boundary (hole) 2
u(x, 7) = 100, u(x, 3) = u(3, y) = u(7, y) = 0 .
2 4 6 8 10 12 x
The initial conditions are

u(x, y) = 0, (x, y) D D .

a) Set up the 2D Crank-Nicolson method for the time-dependent heat problem. Use h = 0.5
and t = 0.1 for t [0, 5].
b) MATLAB: Solve the resulting linear system and give a surface plot animation of the solution.
c) Set up the explicit difference method for the above problem and compare the resulting linear
system of equations with the Crank-Nicolson scheme.
d) MATLAB: Implement the explicit difference method and compute the solution for t = 0.05
and t = 1.5. Compare the solutions.

Exercise 3.12
Consider the 2D-Laplace problem
y
10
u = uxx + uyy = 0
8
with the Dirichlet conditions
6
u=2 , (x 5)2 + (y 5)2 = 22
4
u = cos(2) , (x 5)2 + (y 5)2 = 52
2
on the given domain.
2 4 6 8 10 12 x

a) MATLAB: Generate approximately n approximately equally spaced nodes within the domain.
Choose a method that respects the symmetry of the domain, and therefore use nodes on
circles with radii between the inner and the outer boundary circle.
b) MATLAB: Generate a triangulation of the domain. Introduce a temporary ghost node to
delete triangles outside the domain.
c) MATLAB: Set up the stiffness matrix and the load vector for the given equation. Impress the
boundary conditions in a final step.
d) MATLAB: Solve the problem and plot a graph of the solution.
e) MATLAB: Vary the number of nodes n.
 121

A Appendix
A.1 Greek Letters

Capital Letter Small Letter Name

A Alpha
B Beta
Gamma
Delta
E , Epsilon
Z Zeta
H Eta
, Theta
I Iota
K Kappa
Lambda
M My
N Ny
Xi
O o Omikron
Pi
P , % Rho
Sigma
T Tau
Ypsilon
, Phi
X Chi
Psi
Omega
 123

Bibliography

[Burden] Burden, Faires: Numerical Analysis, Brooks/Cole

[Hairer] Hairer, Nrsett, Wanner: Solving Ordinary Differential Equations I, II, Springer

[Holmes] Holmes: Introduction to Numerical Methods in Differential Equations, Springer

[Huebner] Huebner, Dewhirst, Smith, Byrom: The Finite Element Method for Engineers,
Wiley

[Smith] Smith: Numerical Solution of Partial Differential Equations: Finite Difference

Methods, Oxford University Press

[Stanoyevitch] Stanoyevitch: Introduction to Numerical Ordinary and Partial Differential

Equations Using MATLAB, Wiley

Vehicle Dynamics

[Genta] Genta: Motor Vehicle Dynamics: Modeling and Simulation, World Scientific

[Gillespie] Gillespie: Fundamentals of Vehicle Dynamics, Society of Automotive Engineers,

Warrendale

[Rajamani] Rajamani: Vehicle Dynamics and Control, Springer

Mathematics

[Koch/Stmpfle] Koch, J., Stmpfle, M.: Mathematik fr das Ingenieurstudium, 2nd edition,
Carl Hanser Verlag, Mnchen, 2012

Formularies

[Bartsch] Bartsch, H.-J.: Taschenbuch mathematischer Formeln, Fachbuchverlag Leipzig

im Carl Hanser Verlag, 21. Auflage, 2007

[Bronstein] Bronstein, Semendjajew, Musiol, Mhlig: Taschenbuch der Mathematik, Harri

Deutsch

[Mohr] Mohr, R.: Mathematische Formeln fr das Studium an Fachhochschulen, Carl

Hanser Verlag, Mnchen, 2011
 125

Index

Symbols Crank-Nicolson method 48, 51

A-stable 23
m-step method 20 D
10-point-star 51 Delaunay triangulation 54
4-point-star, differential operator 55
explicit 46 positive definite 55
implicit 47 symmetric 55
5-point-star 45 Dirichlet boundary condition 41
hyperbolic 52 discretization 11, 43
6-point-star 48
explicit 49
implicit 50 E
9-point-star 46 element load vector 60
element stiffness matrix 60
elliptic 40
A equilibrium solution 10
Adams-Bashforth methods 21 Eulers method 13
Adams-Bashforth-Moulton methods 22 explicit 4-point-star 46
Adams-Moulton methods 21 explicit 6-point-star 49
autonomous 9 explicit form 8

B F
backward difference formula 44 finite difference method 27
boundary condition 9 flow line 9
constant 42 forward difference formula 44
costant 42
Dirichlet 41
initial 42 G
natural 41 f. Galerkins method 57
Neumann 41 general Runge-Kutta method 16
periodic 42 global truncation error 12
boundary conditions 41 f.
boundary value problem 9 H
hat function 57
C heat equation 41
central difference formula 44 Heuns method 15
classical Runge-Kutta method 16 hyperbolic 40
consistent 12 hyperbolic 5-point-star 52
constant boundary condition 42
convergent 12
126

I periodic solution 10
implicit 4-point-star 47 phase portrait 10
implicit 6-point-star 50 phase-plane 10
implicit Eulers method 13 phase-space 10
implicit form 8 Poisson equation 41
improved Eulers method 14 positive definite 55
increment function 14, 20 positive definite differential operator 55
initial boundary condition 42 predictor corrector methods 22
initial condition 8
initial value problem 8 R
inner product 55 Rayleigh-Ritzs method 58
linear 55 reading order 43
positive definite 55 region of stability 23
symmetric 55 roundoff error 11
Runge-Kutta-Fehlberg method 19
L
Laplace equation 41 S
line element 9 shooting method 26
linear 8, 40, 55 stable 11
linear m-step method 20 step size control 18
linear multistep method 20 stiff 25
load vector 59 stiffness matrix 59
local truncation error 12 symmetric 55
symmetric differential operator 55
M system of ordinary differential equations 9
matrix,
element stiffness 60 T
stiffness 59 triangulation 54
mesh 54 Delaunay 54
multistep method 20 truncation error 11

N U
natural boundary condition 41 f. unstable 11
Neumann boundary condition 41
node 54
nullcline 11 V
vector,
element load 60
O load 59
one-step method 14 vector field 9
order 12
ordinary differential equation 8
W
wave equation 42
P
parabolic 40
partial differential equation 40
partition 54
periodic boundary condition 42