Happy Thanksgiving !

The attendance of Lab 11 is to based on completion of Problem 0

in the Lab session, and submit it to the GSIs.
How will we test linear algebra in final exam ?
E7 – Lecture Series

Lecture 24
Numerical Methods for Ordinary Differential Equations (ODE)

Fall 2021
Instructor: Shaofan Li
The Initial Conditions ---- The Initial-Value Problem (IVP)
A differential equation of order n has a solution containing n arbitrary constant,
which is called the general solution of the ODE.

By letting these constants vary, we can obtain infinitely many solutions.

In practice, we are interested in finding a solution that satisfy some constraints, which is called
the particular solution of the ODE. These constraints can be initial conditions or boundary
conditions.

Recall the example problem: 𝑚𝑢ሷ + 𝑐𝑢ሶ + 𝑘𝑢 = 𝑝(𝑡)

𝑢ሶ 0 = 0 Two initial conditions for

the second order ODE
𝑢(0) = 0
(Initial velocity and displacement of the mass is zero)
Solutions to ODEs: Analytical Solution

Recall the pendulum example:

If we assume that rotations are small, we can use the small-angle approximation

Therefore the ODE becomes

𝑔
𝜃ሷ + 𝜃=0
𝐿
Given the boundary conditions
ሶ = 0) = 0
𝜃(𝑡 𝜃(𝑡 = 0) = 𝜃0

Particular solution can be obtained as:

𝑔
𝜃 𝑡 = 𝜃0 cos( 𝑡)
𝐿
 Euler’s Method

Consider the first order ODE defined on an interval with a given boundary (or initial) condition:
𝑑𝑦
= 𝑓(𝑡, 𝑦) 𝑡 ∈ [𝑡0 , 𝑡𝑛 ] 𝑦(𝑡 = 𝑡0 ) = 𝑦0
𝑑𝑡

We can divide the interval into equal lengths of h (step-size) to obtain the numerical grid:

[𝑡0 , 𝑡1 , 𝑡2 , … , 𝑡𝑛−1 , 𝑡𝑛 ] where 𝑡0 < 𝑡1 < 𝑡2 , … , 𝑡𝑓−1 < 𝑡𝑛 and 𝑡𝑖+1 = 𝑡𝑖 + ℎ 𝑓𝑜𝑟 𝑖 = 1,2, … , 𝑛

Starting with the initial value 𝑦0 , we can predict the next function value by making use of
the slope at 𝑡0 to extrapolate linearly over the step size h
𝑦1 = 𝑦0 + ℎ ∙ 𝑓 𝑡0 , 𝑦0
Euler’s Method

𝑦1 = 𝑦0 + ℎ ∙ 𝑓 𝑡0 , 𝑦0

𝑦1 = 𝑦0 + ℎ ∙ 𝑓 𝑡0 , 𝑦0

𝑦2 = 𝑦1 + ℎ ∙ 𝑓 𝑡1 , 𝑦1

𝑦3 = 𝑦2 + ℎ ∙ 𝑓 𝑡2 , 𝑦2

𝑦𝑛 = 𝑦𝑛−1 + ℎ ∙ 𝑓 𝑡𝑛−1 , 𝑦𝑛−1

We can generalize this idea to obtain the Explicit (or forward) Euler Formula

𝑦𝑖+1 = 𝑦𝑖 + ℎ ∙ 𝑓 𝑡𝑖 , 𝑦𝑖

Explicit Euler Formula

𝑦

𝑦2 𝑦3 𝑦4
The algorithm is ‘explicit’ since it only requires information at
𝑦1
the previous time step 𝑡𝑖 to compute the state at 𝑡𝑖+1

𝑦0 𝑇𝑟𝑢𝑒 𝑠𝑜𝑙𝑢𝑡𝑖𝑜𝑛, 𝑦(𝑡)

𝑡
Implicit Euler Method
𝑡 ∈ [𝑡0 , 𝑡𝑛 ] 𝑦(𝑡 = 𝑡0 ) = 𝑦0
𝑑𝑦
= 𝑓(𝑡, 𝑦)
𝑑𝑡

Another method can be derived using the 1st order Taylor expansion around 𝑡𝑖+1 and evaluating it at 𝑡𝑖

𝑦𝑖+1 = 𝑦𝑖 + ℎ ∙ 𝑓 𝑡𝑖+1 , 𝑦𝑖+1

Implicit Euler Formula

 DEM

Consider the initial value

problem:
𝑦 ′ = −2𝑥 3 + 12𝑥 2 − 20𝑥 + 8.5

𝑦 0 =1 𝑥 ∈ [0,4]

Apply Explicit Euler’s Method

to solve this ODE numerically
in the given domain
Explicit Euler’s Method
Consider the following ODE with a given initial condition: 𝑦 ′ = −2𝑥 3 + 12𝑥 2 − 20𝑥 + 8.5 𝑦 0 =1
Apply Euler’s Method with different step-sizes:

𝐸𝑥𝑎𝑐𝑡 𝑆𝑜𝑙𝑢𝑡𝑖𝑜𝑛 ∶ 𝑦 𝑥 = −0.5𝑥 4 + 4𝑥 3 − 10𝑥 2 + 8.5𝑥 + 1

Euler’s method converges to the analytical solution as step size gets smaller.
(II): Runge-Kutta Method
Second-Order Runge-Kutta (RK2) Method Midpoint rule
We use midpoint rule to integrate this equation: 𝑑𝑦
= 𝑓(𝑡, 𝑦)
𝑑𝑡
Central Difference
Fourth-Order Runge-Kutta (RK4) Method
It is possible to get higher accuracy methods by evaluating the derivative several times at
different locations (in contrast to Euler’s method, where we evaluate the derivative at the
current step only).

Consider the first order ODE: 𝑑𝑦

= 𝑓(𝑡, 𝑦)
𝑑𝑡

RK4 Method: ℎ
𝑦𝑖+1 = 𝑦𝑖 + (𝑘1 + 2𝑘2 + 2𝑘3 + 𝑘4 )
6

where
𝑘1 = 𝑓(𝑡𝑖 , 𝑦𝑖 )
𝑘2 = 𝑓(𝑡𝑖 + ℎ/2 , 𝑦𝑖 + ℎ/2 𝑘1 )
𝑘3 = 𝑓(𝑡𝑖 + ℎ/2 , 𝑦𝑖 + ℎ/2 𝑘2 )
𝑘4 = 𝑓(𝑡𝑖 + ℎ , 𝑦𝑖 + ℎ 𝑘3 )
The Explanation of Fourth-Order Runge-Kutta (RK4) Method

ℎ
𝑦𝑖+1 = 𝑦𝑖 + (𝑘1 + 2𝑘2 + 2𝑘3 + 𝑘4 )
6

𝑘1 = 𝑓(𝑡𝑖 , 𝑦𝑖 )
𝑘2 = 𝑓(𝑡𝑖 + ℎ/2 , 𝑦𝑖 + ℎ/2 𝑘1 )
𝑘3 = 𝑓(𝑡𝑖 + ℎ/2 , 𝑦𝑖 + ℎ/2 𝑘2 )
𝑘4 = 𝑓(𝑡𝑖 + ℎ , 𝑦𝑖 + ℎ 𝑘3 )

k’s are slopes at different locations

Image from Wikipedia

Derivation of the Fourth-Order Runge-Kutta (RK4) Method
We use Simpson’s 1/3 rule to integrate this equation:

𝑘1 = 𝑓(𝑡𝑖 , 𝑦𝑖 )
ℎ 𝑘2 = 𝑓(𝑡𝑖 + ℎ/2 , 𝑦𝑖 + ℎ/2 𝑘1 )
𝑦𝑖+1 = 𝑦𝑖 + (𝑘1 + 2𝑘2 + 2𝑘3 + 𝑘4 )
6 𝑘3 = 𝑓(𝑡𝑖 + ℎ/2 , 𝑦𝑖 + ℎ/2 𝑘2 )
𝑘4 = 𝑓(𝑡𝑖 + ℎ , 𝑦𝑖 + ℎ 𝑘3 )
Fourth-Order Runge-Kutta (RK4) Method

Consider the following ODE with the initial condition

𝑦′ = 𝑦 − 𝑡2 + 1 𝑦 0 = 0.5

Solve the IVP with RK4

0≤𝑡≤2 ∆𝑡 = 0.5

Exact solution: DEM

1 O
𝑦 = 𝑡 2 + 2𝑡 + 1 − 𝑒 𝑡
2
DEMO
DEMO
 MATLAB built-in ODE solvers
Built-in functions of MATALB, to solve first-order ODEs

Look at their documentation if you’re interested!

ODE-Solving with MATLAB
MATLAB ode45 uses adaptive Runge-Kutta algorithm – an
efficient solver for most ODEs

Example:

function time initial

handle interval condition

Analytical solution: y = sin(t) + C

Value of C determined from initial condition: C = y(0)
 MATLAB built-in ODE solvers
Most common solver is ode45
ode45 for vector variables
Numerical solvers that we discussed are specifically designed for first-order ODEs
Luckily, we can always write a higher-order ODE in terms of a system of first-order ODEs !

Consider the pendulum equation

𝑔 ሶ
𝜃(0) = 𝜃ሶ 0
𝜃ሷ + 𝜃=0 𝜃(0) = 𝜃0
𝐿
Introduce the variables 𝑦1 and 𝑦2 :
𝑦1 ∶= 𝜃
𝑦2 ∶= 𝜃ሶ = 𝑦1ሶ 𝑦2ሶ = 𝜃ሷ

Therefore the second order ODE becomes two first-order ODEs

𝑦1ሶ = 𝑦2 𝑦1 0 = 𝜃0
𝑔
𝑦2ሶ = − 𝑦1 𝑦2 0 = 𝜃ሶ0
𝐿
Reduction of order

We can write this in matrix form as

𝑑𝑌 0 1
= 𝑌
𝑑𝑡 −𝑔/𝐿 0
where
𝑦1 𝑑𝑌 𝑦ሶ
𝑌= 𝑦 = 1
2 𝑑𝑡 𝑦2ሶ

with the initial conditions

𝜃0
𝑌(0) =
𝜃ሶ 0
 Reduction or order

We can use ode45 to solve the system of ODEs

𝑑𝑌 0 1 𝜃0
= 𝑌 𝑌(0) = ሶ
𝑑𝑡 −𝑔/𝐿 0 𝜃0

1
Consider the problem in the interval t = [0,10] with given initial conditions 𝑌(0) =
1
Single Pendulum Problem
Order of Accuracy and Stability of Numerical Integration

Accuracy: How fast a scheme gets close to the exact solution, as a function of step size.

Euler’s Method = 𝑂(ℎ)

Central Difference = O(h^2);

RK4 Method = 𝑂(ℎ4 )

Stability: Ability of a scheme to keep the error from growing, as it integrates forward.

Conditionally Stable = Stability of the scheme depends on the choice of the step size
Unconditionally Stable = Scheme is stable no matter the step size
Why we are interested in Implicit Euler Method ?
𝑡 ∈ [𝑡0 , 𝑡𝑛 ] 𝑦(𝑡 = 𝑡0 ) = 𝑦0
𝑑𝑦
= 𝑓(𝑡, 𝑦)
𝑑𝑡

Another method can be derived using the 1st order Taylor expansion around 𝑡𝑖+1 and evaluating it at 𝑡𝑖

𝑦𝑖+1 = 𝑦𝑖 + ℎ ∙ 𝑓 𝑡𝑖+1 , 𝑦𝑖+1

Implicit Euler Formula

 Order of Accuracy and Stability
Solution to the pendulum problem with initial conditions 𝜃ሶ 0 = 0 and 𝜃 0 = 1
by using different numerical schemes, with a fixed step size of h = 0.1

Explicit and implicit schemes are unstable for this choice of the step size.
Which line shows the behavior of the error for
Euler’s method?

A
1 C E=Ch^1
B Log E= logC + 1 * logh
1

2
1 3

1
D

E) None of these
Euler’s method

The slope f(xi, yi) for Euler’s method

yi+1= yi + f(xi,yi) h
Is valid at which point ?

A) At the initial point

B) At the end point
C)At the mid point
D)Over the whole interval
Consider this 2nd order Runge-Kuttaformula
k1=f(xi,yi)
k2=f(xi+0.5h, yi+0.5k1h)
yi+1= yi + k2h

What method that you learned is this equivalent to?

A) Forward Euler;
B) Backward Euler;
C) Midpoint method
D) Simpson’s method
E) I’m not sure
Consider this 2nd order Runge-Kuttaformula
Calculate k1=f(xi,yi)

Predict yi+1/2 with Euler

y*i+1/2=yi + 0.5 f(xi,yi) h = yi + 0.5 k1 h
Calculate
k2=f(xi+1/2, y*i+1/2) -> slope at midpoint based on Euler estimate
k2=f(xi+0.5h, yi+0.5k1h)

Then
yi+1= yi + k2h -> predict yi+1 using midpoint slope over h
yi+1= yi + f(xi+0.5h, yi+0.5k1h) h
yi+1= yi + f(xi+1/2, y*i+1/2) h -> Midpoint rule
Happy Thanksgiving