Numerical Solution of Differential Equations
Numerical Solution of Differential Equations
For many of the differential equations we need to solve in the real world, there is no "nice"
algebraic solution. That is, we can't solve it using the techniques we have met before in calcus
(separation of variables, integrable combinations, or using an integrating factor), or other similar
means.
As a result, we need to resort to using numerical methods for solving such DEs. The concept is
similar to the numerical approaches we saw in an earlier integration chapter (Trapezoidal Rule
and Simpson's Rule ).
In this section first order single ordinary differential equations will be considered.
We are trying to solve problems that are presented in the following way:
𝑑𝑦
= 𝑓(𝑥, 𝑦)
𝑑𝑥
where 𝑓(𝑥, 𝑦) is some function of the variables x, and y that are involved in the problem.
KAU, Rabigh Chemical and Materials Eng. Dep. Dr. Mohamed Helmy
CHEN 342 [email protected]
2
Note that the right hand side is a function of x and y. Let's now see how to solve such problems
using a numerical approach.
Euler's Method
Euler's Method assumes our solution is written in the form of a Taylor's Series.
This gives us a reasonably good approximation if we take plenty of terms, and if the value of h is
reasonably small.
𝑑𝑦
The last term is just h times 𝑑𝑥 expression, so we can write Euler's Method as follows:
The result of using this formula is the value for y, one h step to the right of the current value.
Let's call it 𝑦1 . So we have:
𝑦1 ≈ 𝑦(0) + ℎ 𝑓(𝑥0 , 𝑦0 )
Where
Next value: To get the next value 𝑦2 , we would use the value we just found for 𝑦1 as follows:
𝑦2 ≈ 𝑦(1) + ℎ 𝑓(𝑥1 , 𝑦1 )
Where
𝑥1 = 𝑥𝑜 + ℎ
The right hand side of the formula above means, "start at the known y value, then move one step
h units to the right in the direction of the slope at that point, which is
𝑑𝑦
= 𝑓(𝑥, 𝑦)
𝑑𝑥
We will arrive at a good approximation to the curve's y-value at that new point."
We'll do this for each of the sub-points, h apart, from some starting value x = a to some finishing
value, x=b, as shown in the graph below.
KAU, Rabigh Chemical and Materials Eng. Dep. Dr. Mohamed Helmy
CHEN 342 [email protected]
4
We'll start at the point (xo , yo ) = (2, e) and use step size of h=0.1 and proceed for 10 steps. That
is, we'll approximate the solution from t=2 to t=3 for our differential equation. We'll finish with a
set of points that represent the solution, numerically.
We already know the first value, when xo = 2, which is yo = e (the initial value).
We now calculate the value of the derivative at this initial point. (This tells us the direction to
move.)
𝑑𝑦 𝑒 𝑙𝑛𝑒
= 𝑓(2, 𝑒) = = 1.3591409
𝑑𝑥 2
This means the slope of the line from t =2 to t =2.1 is approximately 1.3591409.
Step 2
Now, for the second step, h = 0.1, the next point is x+h = 2+0.1 =2.1), we substitute what we
know into Euler's Method formula, and we have:
𝑦 (𝑥 + ℎ) ≈ 𝑦(𝑥) + ℎ 𝑓(𝑥, 𝑦)
𝑒
𝑦 (1) ≈ 𝑦(2.1) ≈ 𝑒 + 0.1 ( ) = 2.8541959
2
This means the approximate value of the solution when x=2.1 is 2.8540959.
KAU, Rabigh Chemical and Materials Eng. Dep. Dr. Mohamed Helmy
CHEN 342 [email protected]
5
We'll need the new slope at this point, so we'll know where to head next.
𝑑𝑦 2.8541959 ln 2.8541959
= 𝑓(2.1,2.8541959) = = 1.4254536
𝑑𝑥 2.1
This means the slope of the approximation line from x=2.1 to x =2.2 is 1.4254536. So it's a little
bit steeper than the first slope we found.
Step 3
Now we are trying to find the solution value when x=2.2. We substitute our known values:
𝑦 (𝑥 + ℎ) ≈ 𝑦(𝑥) + ℎ 𝑓(𝑥, 𝑦)
KAU, Rabigh Chemical and Materials Eng. Dep. Dr. Mohamed Helmy
CHEN 342 [email protected]
6
We'll need the new slope at this point, so we'll know where to head next.
𝑑𝑦 2.99664126 ln 2.99664126
= 𝑓(2.2,2.99664126) = = 1.49490457
𝑑𝑥 2.2
This means the slope of the approximation line from x = 2.3 is 1.49490456. So it's a little steeper
than the first 2 slopes we found.
Step 4
Now we are trying to find the solution value when x=2.3. We substitute our known values:
𝑦 (𝑥 + ℎ) ≈ 𝑦(𝑥) + ℎ 𝑓(𝑥, 𝑦)
KAU, Rabigh Chemical and Materials Eng. Dep. Dr. Mohamed Helmy
CHEN 342 [email protected]
7
Subsequent Steps
Of course, most of the time we'll use computers to find these approximations. I used a spreadsheet
to obtain the following values. Don't use your calculator for these problems - it's very tedious and
prone to error.
x y dx /dy
2.0 e = 2.7182818285 (e ln e)/2 = 1.3591409142
2.1 e+0.1(e/2) = 2.8541959199 (2.8541959199 ln 2.8541959199)/2 = 1.4254536226
2.2 2.9967412821 1.4949999323
2.3 3.1462412754 1.5679341197
2.4 3.3030346873 1.6444180873
2.5 3.4674764961 1.7246216904
2.6 3.6399386651 1.8087230858
2.7 3.8208109737 1.8969091045
2.8 4.0105018841 1.9893756448
2.9 4.2094394486 2.08632809
3.0 4.4180722576
KAU, Rabigh Chemical and Materials Eng. Dep. Dr. Mohamed Helmy
CHEN 342 [email protected]
8
(There's no final dy/dx value because we don't need it. We've found all the required y values.)
Here is the graph of our estimated solution values from x=2 to x=3.
Note:
The problem with Euler's Method is that you have to use a small interval size to get a reasonably
accurate result. That is, it's not very efficient.
Runge-Kutta Methods
The Runge-Kutta methods are a series of numerical methods for solving differential equations and
systems of differential equations.
KAU, Rabigh Chemical and Materials Eng. Dep. Dr. Mohamed Helmy
CHEN 342 [email protected]
9
ℎ2 𝑦 ′′ (𝑥)
If the term is included in the solution the method is second order RK method, if the term
2!
ℎ3 𝑦 ′′′ (𝑥) ℎ4 𝑦 ′′′ (𝑥)
the method is the third order RK method and if the term the method is fourth
3! 4!
order RK method.
The Runge-Kutta 2nd order method is a numerical technique used to solve an ordinary differential
equation of the form
𝑑𝑦
= 𝑓(𝑥, 𝑦); 𝑦(0) = 𝑦𝑜
𝑑𝑥
Only first order ordinary differential equations can be solved by using the Runge-Kutta 2nd order
method. In other sections, we will discuss how the Euler and Runge-Kutta methods are used to
solve higher order ordinary differential equations or coupled (simultaneous) differential equations.
Note: Euler’s method can be considered to be the Runge-Kutta 1st order method.
Runge and Kutta did was write the 2nd order method as:
1
𝑦(𝑥 + ℎ) = 𝑦(𝑥) + (𝑘1 + 𝑘2 )
2
𝑘1 = ℎ 𝑓(𝑥, 𝑦)
𝑘2 = ℎ 𝑓(𝑥 + ℎ, 𝑦 + 𝑘1 )
As usual in this work, the more terms we take, the better the solution. In practice, the Order 2
solution is rarely used because it is not very accurate.
1
𝑦(𝑥 + ℎ) = 𝑦(𝑥) + (2𝑘1 + 3𝑘2 + 4𝑘3 )
9
KAU, Rabigh Chemical and Materials Eng. Dep. Dr. Mohamed Helmy
CHEN 342 [email protected]
10
𝑘1 = ℎ 𝑓(𝑥, 𝑦)
ℎ 𝑘1
𝑘2 = ℎ 𝑓(𝑥 + , 𝑦 + )
2 2
3ℎ 3𝑘2
𝑘3 = ℎ 𝑓(𝑥 + ,𝑦 + )
4 4
The most commonly used Runge-Kutta formula in use is the Order 4 formula (RK4), as it gives
the best trade-off between computational requirements and accuracy.
1
𝑦(𝑥 + ℎ) = 𝑦(𝑥) + (𝑘1 + 2𝑘2 + 2𝑘3 + 𝑘4 )
6
𝑘1 = ℎ 𝑓(𝑥, 𝑦)
ℎ 𝑘1
𝑘2 = ℎ 𝑓(𝑥 + , 𝑦 + )
2 2
ℎ 𝑘2
𝑘3 = ℎ 𝑓(𝑥 + , 𝑦 + )
2 2
𝑘4 = ℎ 𝑓(𝑥 + ℎ, 𝑦 + 𝑘3 )
Example
Use Runge-Kutta Method of Order 4 to solve the following, using a step size of h = 0.1 for 0 ≤
𝑥≤1
𝑑𝑦 5𝑥 2 − 𝑦
= ; 𝑦(0) = 1
𝑑𝑥 𝑒 𝑥+𝑦
KAU, Rabigh Chemical and Materials Eng. Dep. Dr. Mohamed Helmy
CHEN 342 [email protected]
11
Solving the problem using spread sheet will give the following data:
KAU, Rabigh Chemical and Materials Eng. Dep. Dr. Mohamed Helmy
CHEN 342 [email protected]
12
𝑑𝑦1
= 𝑓1 (𝑦1 , 𝑦2 , … . . , 𝑦𝑛 , 𝑡); 𝑦1 (0) = 𝑦10
𝑑𝑡
𝑑𝑦2
= 𝑓2 (𝑦1 , 𝑦2 , … . . , 𝑦𝑛 , 𝑡); 𝑦2 (0) = 𝑦20
𝑑𝑡
𝑑𝑦𝑛
= 𝑓𝑛 (𝑦1 , 𝑦2 , … . . , 𝑦𝑛 , 𝑡); 𝑦𝑛 (0) = 𝑦𝑛0
𝑑𝑡
The only difference between solving a single ODE-IVP and solving a system of them is that all
variables and functions become vectors. This is illustrated in the following example.
Suppose the following chemical reactions take place in a continuous stirred tank reactor (CSTR):
𝑘1 𝑘3
𝐴 𝐵 𝐶
𝑘2 𝑘4
The initial charge to the reactor is all A, so the initial conditions are (in mol/L)
An unsteady state mole balance on component leads to the following set of ODEs:
𝑑𝐶𝐴
= −𝑘1 𝐶𝐴 + 𝑘2 𝐶𝐵
𝑑𝑡
𝑑𝐶𝐵
= 𝑘1 𝐶𝐴 − 𝑘2 𝐶𝐵 − 𝑘3 𝐶𝐵 + 𝑘4 𝐶𝐶
𝑑𝑡
𝑑𝐶𝐶
= 𝑘3 𝐶𝐵 − 𝑘4 𝐶𝐶
𝑑𝑡
Solve the system of ODEs using the Euler’s method for time only up to 0.13 min using h = 0.01
min.
KAU, Rabigh Chemical and Materials Eng. Dep. Dr. Mohamed Helmy
CHEN 342 [email protected]
13
Solution:
The following spreadsheet displays a solution to this system using the Euler’s method for time
only up to 0.13 min:
Note:
Homework: Solve the above example using the second order RK method.
KAU, Rabigh Chemical and Materials Eng. Dep. Dr. Mohamed Helmy
CHEN 342 [email protected]
14
A model for a batch reactor in which penicillin is produced by fermentation has been derived as
follows for cell production and penicillin synthesis, respectively:
𝑑𝑦
= 13.1 𝑦 − 13.94 𝑦 2 ; 𝑦(0) = 0.03
𝑑𝑡
𝑑𝑥
= 1.71 𝑦; 𝑥(0) = 0.0
𝑑𝑡
where:
t = dimensionless time, 0 ≤ 𝑡 ≤ 1
Use h = 0.1.
Solution:
a) Euler’s method
t y x dy/dt dx/dt
0 0.03 0 0.380454 0.0513
0.1 0.068045 0.00513 0.82685 0.116358
0.2 0.15073 0.016766 1.657856 0.257749
0.3 0.316516 0.042541 2.749817 0.541242
0.4 0.591498 0.096665 2.871438 1.011461
0.5 0.878642 0.197811 0.748371 1.502477
0.6 0.953479 0.348059 -0.18258 1.630449
0.7 0.93522 0.511104 0.058947 1.599227
0.8 0.941115 0.671026 -0.01801 1.609307
0.9 0.939313 0.831957 0.005608 1.606226
1 0.939874 0.99258 -0.00174 1.607185
KAU, Rabigh Chemical and Materials Eng. Dep. Dr. Mohamed Helmy
CHEN 342 [email protected]
15
y x
t y x dy/dt dx/dt
k1 k2 k1 k2
0 0.03 0 0.380454 0.0513 0.038045 0.084046 0.00513 0.006007
0.1 0.091046 0.005569 1.077145 0.155688 0.107715 0.20928 0.015569 0.018231
0.2 0.249543 0.022469 2.400946 0.426719 0.240095 0.317014 0.042672 0.049969
0.3 0.528097 0.068789 3.030394 0.903046 0.303039 0.142453 0.090305 0.105747
0.4 0.750843 0.166814 1.977153 1.283942 0.197715 0.007313 0.128394 0.15035
0.5 0.853357 0.306186 1.027612 1.459241 0.102761 -0.00271 0.145924 0.170877
0.6 0.903385 0.464587 0.457842 1.544789 0.045784 0.006509 0.154479 0.180895
0.7 0.929532 0.632274 0.132295 1.5895 0.013229 0.014887 0.15895 0.18613
0.8 0.94359 0.804814 -0.05062 1.613539 -0.00506 0.020358 0.161354 0.188945
0.9 0.951238 0.979964 -0.15245 1.626617 -0.01524 0.02361 0.162662 0.190477
1 0.955421 1.156533 -0.20883 1.63377 -0.02088 0.025469 0.163377 0.191314
HIGHER-ORDER ODES
𝑑2𝑦
= 𝑓(𝑦, 𝑦 ′ , 𝑡); 𝑦(0) = 𝑦0 , 𝑦 ′ (0) = 𝑦0′
𝑑𝑡 2
Since all of the methods discussed apply only to first-order equations, one way to solve those of
the form of the above equation is to convert them into two simultaneous first-order equations. This
is easily accomplished by defining a new variable z as follows:
𝑑𝑦
= 𝑧; 𝑦(0) = 𝑦0
𝑑𝑡
The above equation is a first-order ODE-IVP involving both y and z. If this equation is
𝑑2 𝑦
differentiated with respect to t and substituted into the equation [ 𝑑𝑡 2 = 𝑓(𝑦, 𝑦 ′ , 𝑡)] , there results:
𝑑𝑧
= 𝑓(𝑦, 𝑧, 𝑡); 𝑧(0) = 𝑦 ′ (0)
𝑑𝑡
𝑑𝑦
= 𝑧; 𝑦(0) = 𝑦0
𝑑𝑡
𝑑𝑧
= 𝑓(𝑦, 𝑧, 𝑡); 𝑧(0) = 𝑦 ′ (0)
𝑑𝑡
KAU, Rabigh Chemical and Materials Eng. Dep. Dr. Mohamed Helmy
CHEN 342 [email protected]
16
constitute a system of coupled ODE-IVPs. This system can be solved numerically by any of the
methods previously discussed.
Example:
𝑑2 𝑦 𝑑𝑦
+ + 𝑦 = 1; 𝑦(0) = 𝑦 ′ (0) = 0
𝑑𝑡 2 𝑑𝑡
Solution:
In order to solve this model equation numerically with, for example, the Euler method, it must be
converted into two first-order equations as follows:
𝑑𝑦
= 𝑧; 𝑦(0) = 𝑦0
𝑑𝑡
𝑑 2 𝑦 𝑑𝑧
=
𝑑𝑡 2 𝑑𝑡
𝑑𝑧
+𝑧+𝑦 =1
𝑑𝑡
𝑑𝑧
= 1 − 𝑧 − 𝑦 = 1; 𝑧(0) = 𝑦 ′ (0)
𝑑𝑡
𝑑𝑦
= 𝑧; 𝑦(0) = 𝑦0 = 0
𝑑𝑡
𝑑𝑧
= 1 − 𝑧 − 𝑦 = 1; 𝑧(0) = 𝑦 ′ (0)
𝑑𝑡
KAU, Rabigh Chemical and Materials Eng. Dep. Dr. Mohamed Helmy
CHEN 342 [email protected]
17
are two first-order ODE-IVPs that can be solved by any of the methods previously described.
The solution by Euler’s method will be as shown in the following table (using Excel spread sheet).
t y z=dy/dt dz/dt
0 0 0 1
0.1 0 0.1 0.9
0.2 0.01 0.19 0.8
0.3 0.029 0.27 0.701
0.4 0.056 0.3401 0.6039
0.5 0.09001 0.40049 0.5095
0.6 0.130059 0.45144 0.418501
0.7 0.175203 0.49329 0.331507
0.8 0.224532 0.526441 0.249027
0.9 0.277176 0.551344 0.17148
1 0.33231 0.568492 0.099198
1.1 0.38916 0.578411 0.032429
1.2 0.447001 0.581654 -0.02865
1.3 0.505166 0.578789 -0.08395
1.4 0.563045 0.570393 -0.13344
1.5 0.620084 0.557049 -0.17713
1.6 0.675789 0.539336 -0.21513
1.7 0.729723 0.517824 -0.24755
1.8 0.781505 0.493069 -0.27457
1.9 0.830812 0.465611 -0.29642
2 0.877373 0.435969 -0.31334
KAU, Rabigh Chemical and Materials Eng. Dep. Dr. Mohamed Helmy
CHEN 342 [email protected]