Simulation and modeling of natural processes

Week 3: Dynamical systems and numerical integration

Orestis Malaspinas

1/78
General introduction to dynamical
systems

2/78
What is a dynamical system?

• A system that varies through time.

• The system is described by its state s(t).
• The state s can be multivariate (a vector)
s(t) = (s1 (t), ..., sn (t))T .
• Examples: pendulum, population evolution, ...

3/78
Different classes of descriptions

• Discrete dynamical systems are described by recurrence

relations
s(t + ∆t) = f (s(t)),
where f is a function and ∆t the time increment.
• Continuous dynamical systems are described by differential
equations
ds
ṡ ≡ = f (s(t)).
dt

4/78
Generality of the description: continuous case
What happens if we have second derivatives?
ṡ + s̈ = f (s)
Let us then define
ṡ = y,
ẏ = f (s) − y.
By defining u(t) ≡ (s(t), y(t)) we have a system of the form
u̇ = g(u).
5/78
Generality of the description: discrete case
“Second derivatives” equivalent to presence of a t − ∆t term
s(t + ∆t) = f (s(t)) + s(t − ∆t).
Similarly to continuous case
y(t + ∆t) = s(t),
s(t + ∆t) = f (s(t)) + y(t),
and by defining u(t) ≡ (s(t), y(t)) we have a system of the form
u(t + ∆t) = g(u(t)).
6/78
Generality of the description: continuous case
What happens if f explicitly depends on t?
ṡ = f (s, t)
Let us then define
ṡ = f (s, y)
ẏ = 1,
with y(0) = 0. By defining u(t) ≡ (s(t), y(t)) we have a system of
the form
u̇ = g(u).
7/78
 End of module
General introduction to dynamical
systems

Coming next
The random walk
8/78
The random walk

9/78
The random walk
Time evolution of particles, drunk people, ...
x(t + ∆t) = x(t) + ∆t · vx (t),
y(t + ∆t) = y(t) + ∆t · vy (t),
where (vx , vy ) is chosen randomly in the ensemble
{(∆x/∆t, 0), (−∆x/∆t, 0), (0, 0), (0, ∆y/∆t), (0, −∆y/∆t)},
where ∆x and ∆y are the length of the displacements in direction x
and y.
10/78
The random walk: illustration

11/78
The random walk: illustration

12/78
The random walk: illustration

13/78
The random walk: mathematical analysis
What is the “average” behavior of many random walkers?
• Initial position of the walker is (x(0), y(0)) = (x0 , y0 ).
• After a time t = N · ∆t the position of a random walker is
x(t) = x(t − ∆t) + ∆t · vx (t − ∆t)
= x(t − 2∆t) + ∆t · (vx (t − 2∆t) + vx (t − ∆t)) ,
XN
= x(0) + ∆t vx (t − n · ∆t).
n=1

14/78
The random walk: mathematical analysis
What is the average position?
N
X
hx(t)i = x0 + ∆t hvx (t − n · ∆t)i,
n=1

where for any time, τ , one has

 
1 ∆x 3 1 ∆x
hvx (τ )i = − + ·0+ = 0,
5 ∆t 5 5 ∆t
and therefore
hx(t)i = x0 .
15/78
The random walk: mathematical analysis
What is the average displacement?
p
d(t) = h(x(t) − x0 )2 i + h(y(t) − y0 )2 i
Therefore with dx2 = h(x(t) − x0 )2 i
* N N
+
X X
dx2 = ∆t2 vx (t − m∆t) vx (t − n∆t) ,
m=1 n=1
N
X
= ∆t2 hvx (t − m∆t)vx (t − n∆t)i.
m,n=1

16/78
The random walk: mathematical analysis

Statistical independence
⇒ hvx (t − m∆t)vx (t − n∆t)i = 0, m 6= n.
And for m = n
∆x2 ∆x2 2∆x2
   
1 3 1
hv2x i = + ×0+ = .
5 ∆t2 5 5 ∆t2 5∆t2

17/78
The random walk: mathematical analysis

The sum can be separated in two terms

 
N N
X X 
dx2 = ∆t 2
hvx (t − m∆t)vx (t − n∆t)i + hvx (t − m∆t)2 i
,
m,n=1 m=1
m6=n
N
X 2∆x2 2∆x2 2∆x2
= =N =t .
5 5 5∆t
m=1

18/78
The random walk: mathematical analysis
Doing similarly for dy2 on gets
r
q
2 2
4∆x2
d = dx + dy = t .
5∆t
∆x2
Finally by defining the diffusion coefficient D ≡ 5∆t
√
d = 4Dt.
Equivalent to
∂t ρ(x, y, t) = D(∂x2 + ∂y2 )ρ(x, y, t).
19/78
The random walk: 2D diffusion

Parameters of the simulation

t = 100, ∆x = ∆t = 1, (x0 , y0 ) = (0, 0).
Transport parameters
1 √
D = , d = 80 ∼ = 8.9443.
5
Let us measure the average position and the average distance at time
t for a different number of walkers.

20/78
The random walk: N = 100

h(x, y)i = (−0.63, −0.01), d = 9.262

3.0 −15
−15 2.7 0.36
2.4 −10 0.32
−10 0.28
2.1
−5 1.8 −5 0.24
1.5 0.20
0 0 0.16
1.2
0.9 5 0.12
5
0.6 0.08
10 0.3 10 0.04
0.0 0.00
−15 −10 −5 0 5 10 −15 −10 −5 0 5 10

21/78
The random walk: N = 1000

h(x, y)i = (0.382, −0.094), d = 9.037

−20 10.5 −20 3.6
−15 9.0 −15 3.2
−10 7.5 −10 2.8
−5 −5 2.4
6.0 2.0
0 0
4.5 1.6
5 5
3.0 1.2
10 10 0.8
15 1.5 15 0.4
20 0.0 20 0.0
−15−10−5 0 5 10 15 20 −15−10−5 0 5 10 15 20

22/78
The random walk: N = 10000

h(x, y)i = (0.0034, 0.0335), d = 8.972

−20 45 −20 36
40
32
−10 35 −10 28
30 24
0 25 0 20
20 16
10 15 10 12
10 8
20 5 20 4
0 0
−20 −10 0 10 20 −20 −10 0 10 20

23/78
The random walk: N = 100000

h(x, y)i = (−0.00214, −0.0013), d = 8.938

400
−20 −20 360
350 320
300 −10 280
−10
250 240
0 200 0 200
150 160
10 10 120
100 80
20 50 20 40
0 0
−20 −10 0 10 20 −20 −10 0 10 20

24/78
 End of module
The random walk

Coming next
Growth of a population

25/78
Growth of a population

26/78
Growth of a population
• Let us note the number of individuals of a population P(t), and
P(0) = P0 .
• Only two processes are affecting the number of individuals: the
number of births B(t) and the number of deaths D(t).
• The variation of P(t) can be expressed as
Ṗ(t) = B(t) − D(t).
• A possible model for births and deaths is given by
B(t) = rb P(t), D(t) = rd P(t).
27/78
Growth of a population

Total variation of the population

Ṗ(t) = rP(t), r = rb − rd .
The solution of this equation
P(t) = P0 ert .

28/78
Growth of a population
140
120
• r > 0 exponential growth. 100
80

P (t)
• r < 0 exponential decay. 60

• r = 0 constant. 40
20
0
0 1 2 3 4 5 6 7 8 9
t

29/78
Growth of a population

• Of course an infinite growth of population is not realistic.

• One can add a target “capacity” C to the population by r → 0 as
P→C  
P
r =v 1− .
C
• The model becomes
 
P
Ṗ = v 1 − P
C

30/78
Growth of a population
90
80
The solution is a logistic function 70
60

P (t)
C 50
P(t) = C−P0 −vt 40
1+ P0 e 30
20
10
0 1 2 3 4 5 6 7 8 9
t

C = 50, v = 1, P0 = 10 (blue), 90 (red).

31/78
Growth of a population

• P0 < C growth of population and P → C.

• P0 > C decay of population and P → C.
• Except if P0 = 0 then P(t) = 0.

This system has two fixed points: C (attractive) and 0 (repulsive).

32/78
 End of module
Growth of a population

Coming next
Balance equations

33/78
Balance equations I

34/78
Balance equations

As shown with the growth of a population example the basic idea is

to write a balance equation

| {z } | {z rate} − destruction
variation of a quantity = creation | {z rate} .
ex: Ṗ ex: B(t) ex: D(t)

Or to reuse the generic formalism of continuous dynamical systems

ṡ = f (s) = creation rate − destruction rate.

35/78
Balance equations

For discrete dynamical systems this equation becomes

s(t + ∆t) − s(t) = ∆t (creation rate − destruction rate) .
Both formulations are equivalent in the limit ∆t → 0 since
s(t + ∆t) − s(t)
lim = ṡ.
∆t→0 ∆t
Discrete balance equations can be directly implemented on a
computer.

36/78
Lotka–Volterra model (“Prey-Predator”)

Let a be the antelope population and c the

• The antelopes reproduce exponentially fast
cheetah population.
(infinite amount of food).
da • The antelopes are being eaten by the cheetah
= ka a(t) − kc,a c(t)a(t),
dt | {z } | {z } proportionally to their chance to meet.
(1) (2) • The cheetahs die exponentially fast of
dc starvation.
= − kc c(t) + ka,c a(t)c(t)
dt | {z } | {z } • The cheetahs are reproducing proportionally
(3) (4)
to their chance to eat antelopes.

37/78
Lotka–Volterra model (“Prey-Predator”)

Time evolution 6
5

Periodicity 4

a(t), c(t)
3
2
1
0
0 5 10 15 20 25 30
t

38/78
Lotka–Volterra model (“Prey-Predator”)

Parametric plot 10

8
Initial condition 6
sensitivity

c
4

0
0 2 4 6 8 10
a

39/78
 End of module
Balance equations I

Coming next
Balance equations II

40/78
Balance equations II

41/78
Space–time dependent equations

• For certain situations time dependence is not sufficient (e. g.

weather forecast).
• Necessity to add spatial dependence.
• The balance is now done in a volume V ∈ Rd and must not only
take into account the destruction/creation processes but also the
fluxes through the surface of V, noted ∂V.
The flux through of a quantity is the amount of s crossing ∂V per
unit time.
42/78
Control volume

43/78
The continuity equation

• Let us define ρ(t, x) the volume density of the property one wants
to study (mass, electric charge, ...)
Z
s(t) = ρ(t, x)dV.
V

• Balance equation for s(t) in V reads

ṡ = − |{z}
flux through surface + volumic
| destruction/creation
{z rate} .
=j =Σ

44/78
The continuity equation
R
By defining Σ = V σdV
I
ṡ + j · ndS = Σ
∂V

By using the Gauss–Ostrogradski theorem

Z Z Z
∂ρ
dV + ∇ · jdV = σdV,
V ∂t V V
∂ρ
+∇·j=σ
∂t
45/78
Example: The diffusion equation, ρ → C

Fick’s law j = −D∇C, and σ = 0

∂C
= D∇2 C,
∂t
where D is the diffusion coefficient.

46/78
Example: The Navier–Stokes equations, ρ → ρu

Momentum flux j = ρuu + pI − τ , and σ = 0

∂ρu
+ ∇ · (ρuu) = −∇p + ∇ · τ .
∂t
For a Newtonian incompressible fluid
∂u 1
+ (u · ∇)u = − ∇p + ν∇2 u,
∂t ρ
with ν the kinematic viscosity.

47/78
 End of module
Balance equations II

Coming next
Integration of ordinary differential
equations
48/78
Integration of ordinary differential
equations

49/78
Integration of ordinary differential equations

Let us define a one-dimensional differential equation

ṡ = f (s, t), t0 ≤ t ≤ tf , and s(t = t0 ) = s0 .
where f is a function of s and t, and s0 is a prescribed initial
condition. Depending on the form of f , s(t) may be difficult to obtain
analytically.

50/78
Example: Growth of a population
s
ṡ = f (s, t), f (s, t) = v 1 − s.
C
90
80
C 70
s(t) =   60
C−s0

P (t)
1+ s0 e−vt 50
40
30
20
10
0 1 2 3 4 5 6 7 8 9
t

51/78
Explicit Euler scheme, ṡ = f (s, t)

Find approximate solution at t1 = t0 + ∆t (Taylor exp.)

s(t0 + ∆t) = s(t0 ) + ∆t · ṡ(t0 ) + O(∆t2 )
| {z }
error
With the definition of the differential equation we find
s(t1 ) = s(t0 ) + ∆t · f (s, t0 ).

52/78
Explicit Euler scheme (continued)
Iteratively one finds the explicit or forward Euler scheme
s(tn+1 ) = s(tn ) + ∆t · f (s, tn ), tn+1 = tn + ∆t.

Remarks
• It looks a lot like a discrete dynamical system!
• s(tn+1 ) depends explicitly on s(tn ).
• Straightforward to implement on a computer.

53/78
Example: growth of a population

54/78
 End of module
Integration of ordinary differential
equations

Coming next
Error of the approximation
55/78
Error of the approximation

56/78
Error of the approximation
Evaluation of the error

v
u N
u 1 X
E(∆t) = t (s(ti ) − s∆t (ti ))2 ,
N+1
i=0

with N = (tf − t0 )/∆t

Remark
• If no analytical solution exists one can replace s(ti ) → smin(∆t) (ti ).

57/78
Evaluation of the error

Evaluation of the error, si = s(ti )

s1 = s0 + ∆t · f (s0 , t0 ) + O(∆t2 ),
s2 = s1 + ∆t · f (s1 , t1 ) + O(∆t2 ),
⇒s2 ∼
= s0 + ∆t · {f (s0 , t0 ) + f [s0 + ∆t · f (s0 , t0 ), t0 + ∆t]} + 2O(∆t2 ).
The error accumulates: after 2∆t
s(t0 + 2∆t) − s∆t (t0 + 2∆t) ∼ 2 × O(∆t2 ).

58/78
Evaluation of the error
Defining Ei = s(ti ) − s∆t (ti ), we have
E0 = s(t0 ) − s∆t (t0 ) = 0, · · · ,
EN = s(tN ) − s∆t (tN ) = NO(∆t2 ).
Therefore
v v
u N u N
u O(∆t4 ) X √
u 1 X
2
E=t Ei ∼ t i ∼ ∆t4 N 2 ∼ O(∆t),
2
N+1 N+1
i=0 i=0

where we used that N = (tf − ti )/∆t ∼ O(∆t−1 ).

59/78
Order of the scheme
• The forward Euler scheme is of order 1, since E ∼ O(∆t1 ).
• ∆t is divided by a factor M ⇒, E is decreased by a factor M.
• A scheme is of order k if E ∼ O(∆tk ).
Growth of population

60/78
 End of module
Error of the approximation

Coming next
Implicit Euler scheme

61/78
Implicit Euler scheme

62/78
Stability of explicit schemes

Example: ṡ = −10s(t), t0 = 0, tf = 1.5, s0 = 1.

Solution: s = s0 exp (−10t). Euler: s(t + ∆t) = s(t) − 10∆t · s(t).
Explicit Euler scheme
• Difficult to solve stiff problems with
large ∆t.
• Since only information about tn−1 is
know when going to tn “too big”
jumps are possible.

63/78
The implicit Euler scheme

Instead of sj+1 we approximate sj−1

sj−1 = sj − ∆t · f (s, tj ) + O(∆t2 )
Our differential equation is now approximated by the backward
Euler scheme
sj = sj−1 + ∆t · f (sj , tj ) ⇔ sj+1 = sj + ∆t · f (sj+1 , tj+1 ) .
This class of approximation is called implicit.

64/78
 Implicit Euler scheme (continued)

Example: ṡ = −10s(t), t0 = 0, tf = 1.5, s0 = 1.

Solution: s = s0 exp (−10t).
Implicit Euler scheme
3
2 • Implicit scheme • Explicit scheme
1
s(t)

0
sj+1 = sj − 10∆tsj+1 , sj+1 = sj (1−10∆t).
−1
−2 sj+1 = sj /(1 + 10∆t).
−3
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6
t • Instability solved.

65/78
Explicit vs Implicit (continued)

Graphical interpetation
Explicit Implicit
2.6 2.6
2.4 2.4
2.2 2.2
2.0 2.0
1.8 1.8
s(t)

s(t)
1.6 1.6
1.4 1.4
1.2 1.2
1.0 1.0
0.8 0.8
−0.5 0.0 0.5 1.0 1.5 2.0 −0.5 0.0 0.5 1.0 1.5 2.0
t t

66/78
Explicit vs Implicit
Explicit Implicit
• Explicit equation to solve • Implicit equation to solve
• Conditionally stable (on stiff problems) • Unconditionnaly stable
• Simple implementation • Difficult implementation

Explicit/Implicit solution (left) and error (right) on the population growth problem.

67/78
 End of module
Implicit Euler scheme

Coming next
Numerical integration of partial
differential equations
68/78
Numerical integration of partial
differential equations

69/78
The diffusion equation

In 1D the diffusion equation reads

∂C(x, t) ∂ 2 C(x, t)
=D , t ∈ [t0 , tf ], x ∈ [x0 , xf ],
∂t ∂x2
with the following spatial boundary conditions C(x = x0 , t) = C0 et
C(x = xf , t) = C1 and initial condition C(x, t0 ) = Cx .

Not only time must be discretized but also space.

70/78
The diffusion equation: space discretization

• The physical space is divided in M + 1 equidistant points with

M = (xf − x0 )/∆x.
• The set of points is defined such as xi+1 = xi + ∆x and xf = xM .
• The concentration is the defined on each point with Ci ≡ C(xi ).
• The time dependence is understood here.

71/78
The diffusion equation: space discretization
The Taylor expansion of Ci+1 and Ci−1 to second order reads
∂Ci ∆x2 ∂ 2 Ci
Ci+1 = Ci + ∆x + ,
∂x 2 ∂x2
∂Ci ∆x2 ∂ 2 Ci
Ci−1 = Ci − ∆x + .
∂x 2 ∂x2
By summing these equations one gets (xi ∈ [x1 , ..., xM−1 ])
∂ 2 Ci Ci+1 − 2Ci + Ci−1
=
∂x2 ∆x2
72/78
The diffusion equation: space discretization
The discretization of the Laplacian leads to
∂Ci (t) Ci+1 (t) − 2Ci (t) + Ci−1 (t)
=D .
∂t ∆x2
By defining C(t) = (C(x0 , t), C(x1 , t), ..., C(xM , t))T and A as
1 0 0 0 · · · 0 Remark
1 −2 1 0 ··· 0
0 1 −2 1 ··· 0 • The first and last lines of A
A =  ... .. .. .. .. .. 
correspond to boundary

. . . . .
. .. ..
conditions.

.. . . 1 −2 1
0 ··· ··· ··· 0 1

73/78
The diffusion equation: time discretization

We can now rewrite the diffusion equation under a matrix form

∂C(t) D
= AC(t).
∂t ∆x2

Explicit time discretization Implicit time discretization

C(ti+1 ) − C(ti ) D C(ti+1 ) − C(ti ) D
= AC(ti ). = AC(ti+1 ).
∆t ∆x2 ∆t ∆x2

74/78
The diffusion equation: time discretization

Explicit time discretization Implicit time discretization

  −1
∆tD

C(ti+1 ) = A + I C(ti ). C(ti+1 ) = − ∆tD
∆x2 A+I C(ti ).
∆x2
Remarks
• Implicit scheme needs matrix inversion (it is an art!) but stable.
• Explicit scheme easy to implement but unstable (CFL).

75/78
Numerical example

Initial condition Boundary conditions

1

0.8
C(x0 , ti ) = 0, C(xf , ti ) = 1, ∀ti .
0.6
For t → ∞ admits a stationary
0.4
solution.
0.2

0
0 0.2 0.4 0.6 0.8 1

76/78
Numerical example
1
∆t = ∆x = 0.1
0.9 explicit
0.8 implicit
0.7
sationary
0.6

0.5
C

0.4

0.3

0.2

0.1

0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
x
77/78
 End of module
Numerical integration of partial
differential equations

End of Week 3
Thank you for your attention!
78/78