An Introduction to Numerical Methods

Math 2C03
Sec 2.6 (Zill), Sec 3.1 (Trench)
January 29, 2020

• We are not always able to analytically find a solution to differential equations, and/or we
must computationally solve a differential equation. To overcome this, we must numerically
solve the differential equation and resort to numerical methods
• Numerical approximations are very common in practice, and several online solvers rely on
numerically computed differential equations

1 Simple Numerical Method

Suppose we have a first-order initial value problem
(
dy
dx = f (x, y), (1)
y(x0 ) = y0 .

We can use tangent lines to approximate the solution to Equation (1)! We shall do this, by
remembering that we can approximate a derivative by

dy y(x) − y(x0 )
≈ , (2)
dx x − x0
and we can use equation of the tangent line
dy
y(x) = y0 + (x − x0 ).
dx
dy
Noting that dx = f (x, y) in Equation (1), we have

y(x) = y0 + f (x0 , y0 ) (x − x0 ) (3)

It is required that x − x0 be reasonably small, in order to obtain a good approximation of our

derivative in Equation (2). Denoting h = x − x0 , we can consider this as a “step-size” from our
initial value x0 . Thus, using our initial condition y(x0 ) = y0 and Equation (3), we can approximate
the solution for y1 = y(x0 + h) by solving

y1 = y0 + f (x0 , y0 ) h.

1
Denoting x1 = x0 + h, we repeat this process, to find y2 = y(x1 + h) = y(x0 + 2h) by solving

y2 = y1 + f (x1 , y1 ) h.

This can be defined as a general recursive formula

yn+1 = yn + h f (xn , yn ), (4)

where xn = x0 + nh, n = 0, 1, 2, . . . Using this algorithm of utilizing successive tangent line

approximation to find y1 , y2 , . . . is known as Euler’s Method.

2 Example of Euler’s Method

Use Euler’s Method to obtain an approximation of y(0.8) for
(
y 0 = y 2 + x2 ,
(5)
y(0) = 1,

with a step-size of 0.1.

Solution Given in the problem (5), we are told that h = 0.1, and we wish to compute the
solution up to x = 0.8. Thus, we have x0 = 0, x1 = 0.1, x2 = 0.2, x3 = 0.3, . . . x7 = 0.7, x8 = 0.8.
Using Euler’s Method: yn+1 = yn + h f (xn , yn )

y(0) = y0 = 1
y1 = y0 + h f (x0 , y0 )
= y0 + h (y02 + x20 )
= 1 + (0.1) · (12 + 02 )
y(0.1) = y1 = 1.1
y2 = y1 + h f (x1 , y1 )
= y1 + h (y12 + x21 )
= 1.1 + (0.1) · (1.12 + 0.12 )
y(0.2) = y2 = 1.222
y(0.3) = y3 = 1.3753284
y(0.4) = y4 = 1.573481220784657
y(0.5) = y5 = 1.837065536000854
y(0.6) = y6 = 2.199546514357064
y(0.7) = y7 = 2.719347001239096
y(0.8) = y8 = 3.507831812553902

2
3 Errors in Approximations
When using an approximation (such as a numerical method), we must remember it is just that,
an approximation. The approximation can be improved upon by refining the step-size, however it
should be noted that when using computers we must be careful not to use too small of a step-size
(this is elaborated upon if you take a numerical analysis course).
A natural question that arises when obtaining numerical solutions is: What is my error to the true
solution? It is important to know the accuracy of the numerical method you are using.

• Euler’s Method’s global truncation error is first-order accurate, so the error are O(h). This
is good for an introduction to numerical methods, but not practical for use (converges too
slowly to true solution)
• There are higher order methods, which can efficiently solve differential equations

• Common methods are “Runge-Kutta” methods, such as the globally accurate fourth-order
RK4 method, which is the “workhorse” for many differential equation solvers
• Here we used equispaced points in our independent variable, however this isn’t always the best
method when approximating solutions (not discussed in this course, but would be covered in
a numerical methods course for differential equations . . . )

To illustrate order of accuracy, we numerically solve the logistic equation

(
u0 = u (1 − u),
(6)
u(0) = 2,

which has solution u(t) = 2−e2 −t . The solution to Equation (6) is shown in Figure 1, with h = 0.1.
We also show the convergence to the true solution in Figure 2 at T = 2.

3
 2

1.9

1.8

1.7

1.6

1.5

1.4

1.3

1.2

1.1

1
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

Figure 1: Solution to the logistic initial value problem Equation (6), using h = 0.1. Shown is the
solution found: analytically (black, solid line), using Euler’s method (blue, dashed-dot line), and
using the RK4 method (red, dashed line).

4
 105

100

10-5

10-10

10-15

10-20
10-4 10-3 10-2 10-1 100

Figure 2: Convergence of the approximated solutions to the analytic solution of Equation (6) at
T = 2, as step-size h is refined. Shown is the solution given by Euler’s method (blue, dashed-dot
line) and RK4 method (red, dashed line).

5
 In addition, there are several numerical methods that can be used to solve several problems,
and some of them work better than others, depending on the problem! To show this, we can solve
this IVP
(
y 0 = |x|3 ,
(7)
y(−1) = 0,

and we show the convergence of the solution at x = 1 in Figure 3. In Figure 4 we show what
the grid points are of an example for an equispaced and non-equispaced numerical method.

100

10-2

10-4

10-6

10-8

10-10

10-12

10-14
10-4 10-3 10-2 10-1 100

Figure 3: Convergence of the numerical solutions to the true solution of the IVP given in Equa-
tion (7) at T = 1, as step-size h is refined. Shown is the solution given by Euler’s method (blue,
dashed-dot line) and RK4 method (red, dashed line) with both use Equispaced grids, and a method
using a Non-Equispaced grid (yellow, line-dotted).

6
 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

Figure 4: Example of grid using 16 equispaced points (blue, circles) vs using 16 non-equispaced
points (red, stars).