KNF2053 Numerical

Methods & Statistics

Chapter5:
Numerical Solution of
Ordinary Differential Equations

Dyg nur salmi dharmiza

Faculty of engineering, unimas
 §5 Ordinary Differential Equations

• Introduction
• Euler’s method
• Runge-Kutta methos
• Stiff and multistep methods
 5.1 Introduction
• Equations composed of an unknown function and
its derivatives are called differential equations.

• Play a fundamental role in engineering because

many physical phenomena are best formulated
mathematically in terms of their rate of change.

dv c v- dependent variable
g v t- independent variable
dt m
 Introduction
• This chapter is devoted to solving ordinary differential
equations of the form
dy
 f ( x, y )
dx

• General form of solving the equation using numerical method.

yi 1  yi   h
(New value = old value + slope x step size)
 5.2 Euler’s Method
yi 1  yi yi+1

h  xi 1  xi

Hence, yi
yi 1  yi
f ( xi , yi ) 
h

yi 1  yi  f ( xi , yi )h
 Euler’s Method
• 1st derivative: provides a direct estimate of the slope
at xi
  f ( xi , yi )

where f(xi,yi) is the differential equation evaluated at

xi and yi. This estimate can be substituted into the
equation:
yi 1  yi  f ( xi , yi )h
• A new value of y is predicted using the slope to
extrapolate linearly over the step size h.
 Example 1
Use Euler’s method to numerically integrate
dy
 2 x  12 x  20 x  8.5
3 2

dx
from x=0 to x=4 with a step size of 0.5. The
initial condition at x=0 is y=1.
Solution
 Results after x=4
x y true f(x,y) y euler
0.0 1.00000 8.50 1.000
0.5 3.21875 1.25 5.250
1.0 3.00000 -1.50 5.875
1.5 2.21875 -1.25 5.125
2.0 2.00000 0.50 4.500
2.5 2.71875 2.25 4.750
3.0 4.00000 2.50 5.875
3.5 4.71875 -0.25 7.125
4.0 3.00000 -7.50 7.000
Comparison of true solution with numerical solution
What will happen if we change step size, h ?
 Improvements of Euler’s method
• A fundamental source of error in Euler’s
method is that the derivative at the beginning
of the interval is assumed to apply across the
entire interval.
• Two simple modifications are available to
circumvent this shortcoming:
– Heun’s Method
– The Midpoint (or Improved Polygon) Method

12
 Heun’s Method
• improve the estimate of the slope
– involves the determination of two derivatives for the
interval:
• At the initial point
• At the end point
• The two derivatives are then averaged to obtain an
improved estimate of the slope for the entire interval.

Predictor : yi01  yi  f ( xi , yi )h
f ( xi , yi )  f ( xi 1 , yi01 )
Corrector : yi 1  yi  h
2
Heun’s method
a) Predictor

b) Corrector
The Midpoint (Improved Polygon) Method

• Uses Euler’s method to predict a value of y at

the midpoint of the interval:

yi 1  yi  f ( xi 1/ 2 , yi 1/ 2 )h
The Midpoint (Improved Polygon) Method

yi 1/ 2  yi  f ( xi , yi )h / 2

yi 1  yi  f ( xi 1/ 2 , yi 1/ 2 )h
 5.3 Runge-Kutta Methods (RK)
• Runge-Kutta methods achieve the accuracy of a
Taylor series approach without requiring the
calculation of higher derivatives.
yi 1  yi   ( xi , yi , h)h
  a1k1  a2 k 2    an k n Increment function
a' s  constants
k1  f ( xi , yi )
k 2  f ( xi  p1h, yi  q11k1h)
k3  f ( xi  p3h, yi  q21k1h  q22k 2 h)
p’s and q’s are constants

k n  f ( xi  pn 1h, yi  qn 1k1h  qn 1, 2 k 2 h    qn 1,n 1k n 1h)
 Runge-Kutta Methods
• k’s : recurrence functions.
– Because each k is a functional evaluation, this recurrence
makes RK methods efficient for computer calculations.
• Various types of RK methods can be devised by
employing different number of terms in the
increment function as specified by n.
• First order RK method with n=1 is in fact Euler’s
method.
 2nd Order (n=2) RK Methods
yi 1  yi  (a1k1  a2 k 2 )h
k1  f ( x i , yi )
k 2  f ( xi  p1h, yi  q11k1h)
Values of a1, a2, p1, and q11 are evaluated by setting the second order equation
to Taylor series expansion to the second order term.
Three equations to evaluate four unknowns constants are derived.
a1  a2  1
1 A value is assumed for one of the
a 2 p1 
2 unknowns to solve for the other
1 three.
a2 q11 
2
 2nd Order RK Methods
• Since we can choose an infinite number of values for
a2, there are an infinite number of second-order RK
methods.
• Every version would yield exactly same results if the
solution to the ODE were quadratic, linear, or a
constant.
• Three of the most commonly used methods :
– Heun Method with a Single Corrector (a2=1/2)
– The Midpoint Method (a2=1)
– Raltson’s Method (a2=2/3)
 Heun Method with a Single Corrector (𝑎2=1/2)

1
Assume 𝑎2 =
2
1
Since 𝑎1 + 𝑎2 = 1, 𝑎1 =
2
So, 𝑝1 = 𝑞11 = 1 which yield
1 1
𝑦𝑖+1 = 𝑦𝑖 + 𝑘1 + 𝑘2 ℎ
2 2
Where
𝑘1 = 𝑓(𝑥𝑖 , 𝑦𝑖 )
𝑘2 = 𝑓(𝑥𝑖 + ℎ, 𝑦𝑖 + 𝑘1 ℎ)
 Midpoint method(𝑎2=1)
If 𝑎2 =1, then 𝑎1 = 0, 𝑝1 = 𝑞11 = 1/2,
this results in
𝑦𝑖+1 = 𝑦𝑖 + 𝑘2 ℎ
Where
𝑘1 = 𝑓(𝑥𝑖 , 𝑦𝑖 )
1 1
𝑘2 = 𝑓(𝑥𝑖 + ℎ, 𝑦𝑖 + 𝑘1 ℎ)
2 2
 Ralston’s Method (𝑎2=2/3)
If 𝑎2 =2/3, then 𝑎1 = 1/3, 𝑝1 = 𝑞11 = 3/4,
this results in
1 2
𝑦𝑖+1 = 𝑦𝑖 + 𝑘1 + 𝑘2 ℎ
3 3

Where
𝑘1 = 𝑓(𝑥𝑖 , 𝑦𝑖 )
3 3
𝑘2 = 𝑓(𝑥𝑖 + ℎ, 𝑦𝑖 + 𝑘1 ℎ)
4 4
Comparison of the true solution with the numerical solutions using three
second-order RK methods and Euler’s method
 3rd Order (n=3) RK Method

For dy
 f ( x, y), y(0)  y0
dx
Runge Kutta 3th order method is given by
1
yi 1  yi   k1  4k2  k3  h
6
where
k1  f xi , yi 

 1 1 
k2  f  xi  h, yi  k1h 
 2 2 
k3  f  xi  h, yi  k1h  2k2 h 
 Runge-Kutta 4th Order Method (n=4)

For dy
 f ( x, y), y(0)  y0
dx
Runge Kutta 4th order method is given by

yi 1  yi 
1
k1  2k2  2k3  k4 h
6
where
k1  f xi , yi 
 1 1 
k2  f  xi  h, yi  k1h 
 2 2 
 1 1 
k3  f  xi  h, yi  k2 h 
 2 2 
k4  f xi  h, yi  k3h
 Example 2
Use the classical fourth-order RK method to
integrate
f ( x, y)  2 x  12 x  20 x  8.5
3 2

using a step size of h = 0.5 and initial

condition of y = 1 at x = 0.
Solution
 Example 3
Integrate the following using fourth-order RK

f ( x, y)  4e 0.8 x
 0.5 y

with h = 0.5, y(0) = 2 from x = 0 to 0.5

 Solution
k1  f (0, 2)  4e0.8(0)  0.5(2)  3
k2  f (0.25, 2.75)  4e0.8(0.25)  0.5(2.75)  3.510611
k3  f (0.25, 2.877653)  4e0.8(0.25)  0.5(2.877653)  3.446785
k4  f (0.5,3.723392)  4e0.8(0.5)  0.5(3.723392)  4.105603

Substituted into Runge-Kutta Equation

1 
y (0.5)  2   3  2(3.510611)  2(3.446785)  4.105603 0.5
6 
 3.751699
 5.4 Stiff and multistep methods
• A stiff system is the one involving rapidly changing
components together with slowly changing ones.
• Both individual and systems of ODEs can be stiff:

dy
 1000 y  3000  2000e t
dt

• If y(0)=0, the analytical solution is developed as:

y  3  0.998e1000t  2.002et
 Multistep methods
• Step-Size Control/
– Constant Step Size.
• A value for h must be chosen prior to computation.
• It must be small enough to yield a sufficiently small truncation error.
• It should also be as large as possible to minimize run time cost and
round-off error.
– Variable Step Size.
• If the corrector error is greater than some specified error, the step size
is decreased.
• A step size is chosen so that the convergence criterion of the corrector
is satisfied in two iterations.
• A more efficient strategy is to increase and decrease by doubling and
halving the step size.
 Multistep methods
Integration Formulas:
1. Newton-Cotes Formulas.
Open Formulas.
xi 1

yi 1  yi n  
xi n
f n ( x)dx fn(x) is an nth order interpolating
polynomial.

Closed Formulas.
xi 1

yi 1  yi n 1  f
xi n1
n ( x)dx
2. Adams Formulas (Adams-Bashforth).
Open Formulas.
• The Adams formulas can be derived in a variety of
ways. One way is to write a forward Taylor series
expansion around xi. A second order open Adams
formula:
3 1  5 3
yi 1  yi  h f i  f i 1   h f i O(h 4 )
2 2  12
Closed Formulas.
• A backward Taylor series around xi+1 can be written:
n 1
yi 1  yi  h  k f i 1k  O(h n 1 ) Listed in coefficients and
k 0 truncation error for Adams-
36
Moulton correctors table
 Higher-Order multistep Methods
1. Milne’s Method.
Uses the three point Newton-Cotes open formula as a
predictor and three point Newton-Cotes closed
formula as a corrector.

2. Fourth-Order Adams Method.

Based on the Adams integration formulas. Uses the
fourth-order Adams-Bashforth formula as the
predictor and fourth-order Adams-Moulton formula as
the corrector
 Exercise 5
1. Given
dy
 yx 2  2 y, y(0)  1
dx
a) Determine y (1.0) using analytical method (calculus)
b) Estimate y (1.0) using Euler’s method. Let h = 0.25
c) Calculate the true error percentage 𝜀𝑇

2. Given
dy
 (1  x) y , y(0)  1
dx
The initial condition at x = 0 is y = 1
Estimate y (2.0) using the 4th order Runge Kutta method. Let h=0.5