Chapter13

ORDINARY DIFFERENTIAL EQUATIONS

13.1 Introduction
Let f ( x , y) be a real val ued funct i on of t wo vari abl es def i ned for a ~< x ~< b
and all real y. Suppose now that y is a real val ued funct i on defi ned on
[a, b ]. The equat i on
y' =f ( x , y ) (13.1)
is called an ordi nary differential equat i on of the first order. Any real val ued
funct i on y, whi ch is differentiable and sat i sfi es (13.1) for a ~< x <~ b, is said
to be a solution of this differential equat i on. In general such a solution, if it
exists, will not be uni que because an arbi t rary const ant is i nt roduced on
i nt egrat i ng (13.1). To make the solution uni que we need to i mpose an ext ra
condi t i on on the solution. This condi t i on usual l y takes the form of requi ri ng
y(x) to have a speci fi ed val ue for a part i cul ar val ue of x, that is,
y( xo) =S (13.2)
where x0 ~ [a, b] is the part i cul ar val ue of x and s is the speci fi ed val ue of
y(xo). Equat i ons (13.1) and (13.2) are col l ect i vel y cal l ed an initial value
problem. Equat i on (13.2) is called the initial condi t i on and s the initial val ue
of y.
Exa mpl e 13.1 The general solution of
y' = ky, (13.3)
where k is a gi ven const ant , is
y (x) = ce ~" (13.4)
where c is an arbi t rary constant. If, for exampl e, we are gi ven the extra
condi t i on
y( 0) = s (13.5)
336 Numerical analysis
then, on putting x = 0 in (13.4), we see that we require c = s. The unique
solution satisfying (13.3) and (13.5) is
y ( x ) = se t'.
The di fferent i al equat i on (13. 3) occurs in the descr i pt i on of many
physi cal phenomena, for exampl e, uni nhi bi t ed bact er i ol ogi cal gr owt h, a
condenser di schar gi ng, a gas escapi ng under pressure f r om a cont ai ner
and r adi oact i ve decay. In all t hese exampl es, t i me is r epr esent ed by x.
The condi t i on (13. 5) gi ves the i ni t i al val ue of the physi cal quant i t y
bei ng measur ed, for exampl e, the vol ume of bact eri a at t i me x = 0.
Many mor e compl ex phenomena may be descri bed by equat i ons of the
form (13. 1). Vl
Duri ng most of this chapter we shall discuss numerical met hods of
solving (13.1) subject to (13.2). By this, we mean that we seek approxi ma-
tions to the numerical values of y ( x ) for particular values of x or we seek a
funct i on, for exampl e a pol ynomi al , whi ch approxi mat es to y over some
range of values of x. Onl y for rare special cases can we obtain an explicit
form of the solution, as in Exampl e 13.1. In most probl ems, f ( x , y ) will be
far too compl i cat ed for us to obtain an explicit solution and we must
therefore resort to numeri cal met hods. We shall also discuss s y s t e ms of
ordinary differential equat i ons and hi gher order differential equations, in
which derivatives of hi gher order than the first occur. The met hods for
both of these are natural ext ensi ons of those for a single first order
equation.
The first question we ask of (13.1) and (13.2) is whether a solution y
necessarily exists and, if not, what additional restrictions must be imposed
on f ( x , y ) to ensure this. The following example illustrates that a solution
does not necessarily exist.
Exampl e 13.2 For 0 ~< x ~< 2, solve
y , = y 2 with y ( 0 ) = l .
The general solution of the differential equation is
1
y ( x ) = - - - - - - - ,
(x + c)
where c is arbitrary. The initial condition implies c = - 1, when
1
y ( x ) = ~ .
(1 - x )
This solution is not defi ned for x = 1 and therefore the given initial value
problem has no s ol ut i on on [0, 2 ]. D
Ordinary differential equations 337
You will notice that in Example 13.2 the function f ( x , y) is the well-
behaved function y2. Obviously continuity of f ( x , y) is not sufficient to
ensure the existence of a unique solution of (13.1) and (13.2). We find that
a Lipschitz condition on f ( x , y) with respect to y is sufficient and we have
the following theorem.
Theorem 1 3.1 The first order differential equation
y' = f ( x , y )
with initial condition
y ( x o ) =S ,
where x0 G [a, b], has a unique solution y on [a, b] if f ( x , y) is continuous
in x and if there exists L, independent of x, such that
I f ( x , y ) - f ( x , z ) ] <-LI y - z l (13.6)
for all x ~ [a, b] and all real y and z.
P r o o f We shall consider an outline of the proof, which is based on the
contraction mapping theorem. We may write the problem in the integral
form
I
X
y( x) = s + f ( x , y( x) ) dr.
Xo
(13.7)
We consider the sequence of functions (Ym(X))m=O defined recursively
by
y o( x ) =s , (13.8)
SX
ym +~ (X) = S + f(x, ym(X)) dx,
X 0"
m = 0, 1 . . . . . (13.9)
We find that, for m I> 1,
Now
[Ym+ ,(X) - ym(X) [ <<-I ~ I f ( x, ym(X)) - f ( x , Ym- ~(X)) [ dx
Xo
<" I x L lye(x) - ym- , (x) l dx.
X0
I f ( x , s) dx ~ M i x Xo l,
l y, ( x) - yo(x) I = l y, ( x) - s [ = ~
338 Numerical analysis
where M I> I f ( x, s) I for x [a, b ]. M, an upper bound of I f ( x, s) l, exists
as f is continuous in x. Thus
l y2(x) - y, ( x) I ~< I ' L l y , ( x ) - y0(x)I dx
X o
I x L ~ t l x - x ol ~ = L A 4 I x - X ol ~
xo 2!
and
x x L2M
[y3(x)- y2(x) I ~< Ix Lly2(x) - y,(x)[ dx ~< I I x - Xo[ z dx
o xo 2!
I x - x 0 [ 3.
3!
We may prove by induction that
[ y ~ ~(x) - ym( X) [
for m ~> 0. The series
ML m
( m+ 1)!
I X -- X o[ m+ 1
yo( x) + ( y l ( x ) - yo( x) ) + (y2(x) - y l ( x ) ) + ""
is thus uniformly convergent on [a, b ], as the absolute value of each term is
less than the corresponding term of the uniformly convergent series
M s L ml x - x o [ m
I S l + L m. , m! "
The partial sums of the first series are y0(x), y~(x), y2(x), ... and this
sequence has a limit, say y ( x ) . This limit y ( x ) can be shown to be a
differentiable function which is the solution of the initial value problem. By
assuming the existence of two distinct solutions and obtaining a contradic-
tion, the uniqueness of y can be shown. V1
The sequence (Ym(X))m_ o defi ned in the proof of the last theorem may
be used to find approximations to the solution. The resulting process is
called P i c ar d iteration, but is only practicable for simple functions f ( x , y)
for which the integration in (13.9) can be performed explicitly.
E x a mp l e 1 3. 3
Use Picard iteration to find an approximate solution of
y' = y, with y(0) - 1
for 0~<x~<2.
Ordinary differential equations 339
In this case, f ( x , y ) = y and t hus
If(x, y ) - f ( x , z) [ = l y - z l,
so we choose L = 1. We start wi t h y o ( X) - 1 when, f r om (13. 9),
I
X
y l ( x ) = 1 + yo( x) d x = 1 + x,
0
2
I x x
y 2 ( x ) - 1 + ( l + x ) d x - l + x + ~ ,
o 2!
x ( x )
y 3 ( x ) - l + I l + x + d x = l + x + - - - + ~
o 2! 3!
and, in general ,
y r ( X) "- 1 + x +
2
X
2~
r
X
+ . . . + ~ .
r!
The exact sol ut i on in t hi s case is
2
X
y ( x ) = e x= 1 + x + ~ + - . .
2!
so that the appr oxi mat i ons obt ai ned by t he Pi car d pr ocess are the part i al
s ums of this i nf i ni t e series. I-1
Ex a mp l e 13. 4 We show t hat
1
Y' - 1 + y 2 ' wi t h y ( x o) = s
has a uni que sol ut i on for x ~ [a, b ]. We have
1
f ( x , y ) =
1 +y2
and f r om t he mean val ue t heor em
f ( x , y) - f ( x , z ) = ( y - z )
Of(x, ~)
ay
wher e ~" lies bet ween y and z. Thus to sat i sf y (13. 6) we choose
L~>
Of(x, y)
, ,
ay
340 Numerical analysis
for all x E [a, b] and y E ( -*, , , **), pr ovi ded such an upper bound L exists.
In this case,
Of - 2 y
-~y - ( 1+y2) 2
and, as may be seen by sket chi ng a graph, this is a ma xi mum for y = + 1/~/3.
We t herefore choose
Of(x, 1/x/3) 343
L ~ . - - . .
igy 8 H
Most numer i cal met hods of sol vi ng the initial val ue pr obl em (13.1) and
(13.2) use a st ep- by- st ep process. Appr oxi mat i ons Y0, Y~, Y2 . . . . are sought t
for the val ues y( xo) , y( x~) , y(x2), . . . of the sol ut i on y at di st i nct poi nt s
x = x0, x~, x2 . . . . . We cal cul at e the y, by means of a recurrence relation
deri ved f r om (13. 1), whi ch expresses y, in t erms of y,_~, Y,-2 . . . . . Y0. We
t ake Yo = y(x0) = s, since this val ue is gi ven. We usual l y t ake the points
x 0, x~ . . . . to be equal l y spaced so that Xr--XO + rh, where h > 0 is called the
st ep si z e or mesh l engt h. We also t ake a = x0 as we are onl y concerned wi t h
poi nt s x, ~ a. ( We coul d equal l y wel l t ake h < 0 but this may be avoi ded by
the si mpl e t ransformat i on x' --- - x. )
The si mpl est suitable recurrence rel at i on is deri ved by repl aci ng the
deri vat i ve in (13.1) by a f or war d di fference. We obt ai n
Yn+l -- Yn
= f ( x . , y. ) ,
whi ch on r ear r angement yi el ds Eul e r ' s met hod
y0 = s,
y.+ i = y . + h f ( x . , y . ) , n = 0 , 1 , 2 . . . . .
The l at t er is a part i cul ar t ype of d i f f erenc e equat i on.
(13.10)
(13.11)
13.2 Difference equations and Inequalities
In order to di scuss the accur acy of appr oxi mat i ons obt ai ned by met hods
such as (13. 11) in sol vi ng initial val ue pr obl ems, we must use some
propert i es of di fference equat i ons. I f ( F, ( wo, w~)) is a sequence of
t We use the notation yr to denote an approximation to Y(Xr), the exact solution at x = x,,
throughout the rest of this chapter.
Ordinary differential equations 341
funct i ons defi ned for n E N, some set of consecut i ve integers, and for all w0
and w~ in some interval I of the real number s, t hen the syst em of equat i ons
F , ( y , , y , + ~ ) = O, (13. 12)
for all n E N, is called a d i f f er enc e e quat i on o f or d e r 1. A sequence ( y, ) ,
wi t h all y, E I and sat i sfyi ng (13.12) for all n E N, is called a sol ut i on of
this difference equation.
Ex ampl e 1 3.5 If
F, ( w0, wl ) = w0 - wl + 2n
for all integers n, we obtain the difference equat i on
y , - y, +t + 2n = O.
The general solution of this equat i on is
y , = n ( n - 1 ) + c ,
where c is an arbitrary constant. Such an arbitrary const ant may be
det ermi ned if the value of one el ement of the sol ut i on sequence is given.
For exampl e, the ' i ni t i al ' value Y0 may be gi ven, when clearly we must
choose c = Y0. The arbitrary const ant is anal ogous to that occurri ng in the
integration of first order differential equat i ons. El
The difference equation of order m is obt ai ned f r om a sequence
( F, ( wo , wl . . . . . Wm) )
of funct i ons of m + 1 variables def i ned for all n ~ N, a set of consecut i ve
integers, and all Wr E I , r = 0 . . . . . m, for some interval I. We seek a
sequence ( y, ) with all y, E I and such that
F , ( y , , y,+~ . . . . . Yn+m) = O,
for all n E N. The simplest difference equat i on of order m is of the form
aoy, + a~y,+~ + . . . + amy, +m=O, (13. 13)
where ao, a~ . . . . . am are i ndependent of n. Equat i on (13. 13) is called an mth
or d e r h o mo g e n e o u s l i near d i f f er enc e equat i on wi t h const ant coeffi ci ent s. It
is homogeneous because if ( y, ) is a sol ut i on, t hen so is (~ty,), where a is
any scalar. Equat i on (13.13) is called linear because it consists of a linear
combi nat i on of el ement s of (y, ). We illustrate the met hod of sol ut i on by
first consi deri ng the second order equat i on,
aoy, + a~y,+~ + a2y, + 2 = 0. (13. 14)
We seek a solution of the form
y . = z " ,
342 Numerical analysis
where z is independent of n. On substituting z n in (1 3.14), we have
aoz n + a l z n + l .1. a2 z n +2 _. 0 ,
that is,
z ~(ao + a l z + a2z 2) = O.
We conclude that for such a solution we require either
Z n = 0
o r
ao + a l z + a2z 2 =0. (13.15)
The former only yields z = 0 and hence the trivial solution y ~ - 0. The latter,
a quadratic equation, is called the c har ac t e r i s t i c equation of the difference
equation (13.14) and, in general, yields two possible values of z. We
suppose for the moment that (13.15) has two distinct roots, Z l and z2. We
then have t wo independent solutions
i'/
y , = z ~ and y n =z 2.
It is easily verified that
/1
yn = c i z ~ + c2 + z2, (13.16)
where c~ and c2 are ar bi t r ar y c ons t ant s , is also a solution of (13.14). This is
the general solution of (13.14) for the case when the characteristic equation
has distinct roots. You will notice that this general solution contains two
arbitrary constants and those familiar with differential equations will know
that two such constants occur in the solution of second order differential
equations.
Now suppose that in (13.15)
a21= 4aoa 2 and a2 * O,
so that the characteristic equation has two equal roots. We only have one
solution
y ~=z ~.
We try, as a possible second solution,
y , = nz ~ (13.17)
in (13.14). The left side becomes
a o n z ~+ a l ( n + 1)z~ ' + a2( n + 2)z~ 2
= z ~[ n( ao + al z i + a2z ~) + z l ( a l + 2a2zl)]
= 0
Ordinary differential equations 343
as z~ = - 8 9 2 is the repeated root of (13.15). Cl earl y (13.17) is a sol ut i on
and the general sol ut i on of (13.14) is
y , = C l Z ~ ' a t " c2rlT,' ]. (13. 18)
The onl y r emai ni ng case is a 2 = 0 , when the di fference equat i on is of onl y
first order and has general sol ut i on
c (
a l l
where c is arbitrary.
I f a2 * 0 we may write (13.14) as the recurrence rel at i on
Y, + 2 - - - ( a o / a 2 ) Y, , - ( a , / a 2) y , , + l .
I f we are to use this to calculate member s of a sequence ( y, ) we need t wo
starting (or initial) values, for exampl e, Yo and y~. Such starting val ues fix
the arbitrary const ant s in the general solution. It can also be seen f r om this
that (13.16) or (13.18) is the general sol ut i on in that any particular sol ut i on
may be expressed in this form. Gi ven any part i cul ar sol ut i on we can al ways
use Y0 and y~ to det ermi ne c~ and c2. The r emai ni ng el ement s of the sol ut i on
are then uni quel y det ermi ned by the recurrence rel at i on above.
Exampl e 13.6 The difference equat i on
y . - 2 cos O.y.+~ + Y,, =0 ,
where/ 9 is a const ant , has characteristic equat i on
1 - 2 cos O. z + z 2 = 0
which has root s z = cos 0 i sin 0 = e 176 The general sol ut i on is:
y . = c~ e "i~ + c2 e - ' ~~ = k~ cos nO + k2 sin nO, 0:# m~ ,
y,, = c~ + c z n, 0 = 2 m~ ,
y . = (cl + c zn ) ( - 1 ) ~, /9= (2m + 1)~r,
where m is any integer and c~, c2 and, hence, kl, k2 are arbitrary. D
To det ermi ne the general solution of the mt h order equat i on (13.13) we
again try y. = z". We require
ao Z n + a l Z n+l + . . . + am Z n+m = 0
and thus either z = 0 or
pr o( Z ) =-- ao + a l z + a2 Z2 + - ' - + amZ
m= o .
(13. 19)
344 Numerical analysis
Equat i on (13.19) is called the characteristic equat i on and P m t he character-
istic pol ynomi al of (13.13). In general, (13.19) has m roots z ~, z 2, . . . , Z m
and, i f these are distinct, the general sol ut i on of (13.13) is
/ I
y , = c l z ~ + c 2z ~ + "'" + c ~ z ~ ,
where the m const ant s c~, c 2, . . . , C m a r e arbitrary. Agai n we may expect m
arbi t rary constants in the general solution of (13. 13), because we need m
starting values when it is expressed as a recurrence relation. For exampl e,
Yo, Y l . . . . . Ym- ~ a r e needed to calculate Ym, Y, . ~ . . . . .
I f P m has a repeated zero, for exampl e if z2 = z~ , we find that y . = nz' ~ i s a
solution. I f the ot her zeros of p,. are distinct, we obtain the general solution
y . = c~z ' ~ + c 2 n z ' ~ + c 3z 3 + . . . + c, , , z m"
I f a zero of p,. is repeated r times, for exampl e if
Z 1 = Z 2 = . . . = Z r ,
the general solution is
t l t l
y . = c l z ~ + c 2 n z ~ + c 3 n 2 z ~ + . . . + c ~ n ~ - ' z ~ + C r + l Z r + 1 "at" " ' " "Jr C mZ m,
agai n assumi ng that Z r + ~ . . . . , Z , . ar e distinct.
E x a mp l e 1 3 . 7 Fi nd the sol ut i on of the third order equat i on
Y. - Y. +~ - Yn+2 + Y,,+3 = 0,
wi t h y_~ = 0, Yo = 1 and y~ = 2.
The characteristic equat i on is
1 - z - z 2 + z 3 = O
wi t h root s z = - 1 , 1, 1. The general solution of (13.20) is
Y. = c i ( - 1 ) " + C 2 + n c 3
and thus we require
We obtain
Y_ ~ = -Cl + C 2 - - C 3 = 0
y0 = C I + C 2 - - - 1
y~ = -c~ + c2+ c3=2.
(13.20)
c~ - 0, c2 = c3 = 1,
and the solution requi red is
y . = l + n .
O
Ordinary differential equations 345
The non- homogeneous difference equat i on
a o y . + a ~y . + " " + a my . + m = b . (13. 21)
is solved by first fi ndi ng the general sol ut i on of the homogeneous equat i on
a o y . + a~y.+~ + . . . + amYn+m = O.
We denot e this general solution by g, . Secondl y we find any part i cul ar
solution ~, of the compl et e equat i on (13.21) by some means (usually by
inspection). The general solution of (13.21) is then
y. = ~. + g. .
There are m arbitrary constants in g. and hence m in y. . (This met hod is
entirely anal ogous to the ' part i cul ar integral plus compl ement ar y f unct i on'
solution of a linear differential equat i on. )
Exa mpl e 13.8 Fi nd the general solution of
2 y . + y.+~ - Yn+2 = 1.
The homogeneous equat i on is
2 y . + y.+~ - Yn+2 = O,
which has general sol ut i on
A particular sol ut i on is
y . = c l ( - 1)" + c2.2 n.
3~. = 89 for all n,
so that the general sol ut i on of the compl et e equat i on is
y. = cj ( - 1)" + c2.2" + 89 i"l
In the error anal ysi s of this chapt er we shall meet d i f f e r e n c e (or
r e c u r r e n t ) i n e qu a l i t i e s . We need the fol l owi ng l emma.
Le mma 13.1 Let the sequence (u,)~_-0 of posi t i ve reals sat i sfy the
difference i nequal i t y
U n + 1 <~ l g n "+" a o u n 4- a l U n _ 1 .-[ - . . . "-]- a mU n _ m "J- b
(13. 22)
for n = m, m + 1, m + 2 . . . . . where a r ~ 0, r = 0, 1 . . . . . m, and b ~> 0. Then, if
A = n0+ a~ + -.- + am,
b
u, ~< A (13. 23)
[ ~ + nb, A= 0
346 Numerical analysis
for n = 0, 1, 2 . . . . . where 5 is any real number sat i sfyi ng
( ~ U r , r = 0 , 1 . . . . , m.
P r o o f T h e pr oof in bot h cases is by induction. Suppose (13. 23) is true
when A > 0 for all n ~< N. On appl yi ng (13.22) with n = N we see that
U N + 1 <<~ I~ + e N A b m e (N- r)A b
- - - - ' ~ " t2L r 5 + - - - + b
A = A
(0 A )
<- + eNA dr" [ ~r + eN A b
ffi A
= + eNA(1 +A) A < + e
(N+ I)A
b
A
since 1 + A < e A. Fr om the defi ni t i on of t~ we can see that (13. 23) holds for
n ~< m. The pr oof for A = 0 is si mpl er but similar. I-!
13.3 One-step methods
We return to the numeri cal solution of the initial value pr obl em
y' = f ( x , y )
with
y ( x o ) = s,
where x E [x0, b]. We shall assume that f satisfies the condi t i ons of
Theor em 13.1, so that the exi st ence of a unique solution is ensured. We
shall also assume that f is ' suf f i ci ent l y' differentiable, that is, all the
deri vat i ves used in the analysis exist. We l ook for a sequence Y0, Y~,
Y2 . . . . . whose el ement s are approxi mat i ons to y ( x o ) , y ( x ~) , Y( X 2 ) . . . . . where
x . = Xo + nh. We seek a first order difference equation to det ermi ne (y. ). A
first order di fference equat i on yi el ds a one - s t e p process, in that we evaluate
y . . ~ f r om onl y y. and do not use y . _ ], Yn-2, . . . explicitly.
One approach is to consi der the Tayl or series for y. We have
h 2 h p
y ( x . l) = y ( x . ) + h y ' ( x . ) + ~ y"(x~) + . . " + ~ y(P)(x. ) + h p+ IRp+ l ( x. ) ,
P!
where
(13.24)
Rp+ ](x .) = ~ 1 y(p and x . < ~5 < x . l -
( p + 1)!
We fi nd we can det ermi ne the deri vat i ves of y f r om f and its partial
Ordinary differential equations 347
derivatives. For example,
y' ( x) = f ( x , y ( x ) )
and, on using the chain rule for partial differentiation,
y" ( x) = Of + Of d y
8x 8 yax
= L +f r y'
= f ~ + f y f . (13.25)
We have written fx and fy to denote the partial derivatives of f with respect
to x and y.
If we define the functions f ( r ) ( x , y ) recursively by
f ( r ) ( x, y ) = fx(r-I)(X, y ) + f ~r- l ) ( x, y ) f ( x , y ) (13.26)
for r~> 1, with f(~ y ) = f ( x , y) , then
y(r+l )(X) = f ( r ) ( x , y ( x ) ) (13.27)
for r I> 0. The proof of this is easily obtained by induction. The result is true
for r = 0 and on differentiating (13.27) we have
y(r+2)(X) = fa!r)(x, y ( x ) ) + f ; r ) ( x, y ( x ) ) y' ( x)
= f~ r) ( X , y ( X ) ) + f( r) ( X , y( x ) ) f ( x , y( x ) )
a y
= f(r y ( x ) ) ,
showing that we can extend the result to the ( r + 2)th derivative of y. Note
t hat t he f (r )(x , y) ar e not deri vat i ves of f in t he usual sense.
If we substitute ( 1 3. 27) in ( 1 3. 24 ) , we obtain
hP f ( p - l +
y(x,,+ 1) = y(x, , ) + hf ( x, , , y(x, , )) + . . . + ~ )(x,,, y( x, , ) ) + h p IRp+ i(xn).
p!
The approximation obtained by neglecting the remainder term is
hP f ( p 1
Y, , 1 = Y, + h f ( x , , y , ) + . . . + ~ - )(x, , y , )
p!
( 1 3. 28 )
(13.29)
(13.30)
(13.31)
f ( ' ) ( x , y ) = f ~ + f y f ,
f a ) ( x, y ) = f x ~') + f ~' ) f
= f , ~ + 2f x y f + fry f 2 + ( f x + f y f ) f y .
and the Taylor series method consists of using this recurrence relation for
n = 0, 1, 2 . . . . together with Y0 = s. The disadvantage of this method is that
the f ( r) ( x, y ) are difficult to determine. For example,
348 Numerical analysis
The met hod mi ght , however , be useful for p ~< 2. For p = 1 we obt ai n Eul er ' s
met hod. For most pr obl ems the accuracy of the met hod i ncreases wi t h p
since the r emai nder t erm decreases as p increases if the deri vat i ves of y( x)
are wel l behaved.
Ex a mp l e 13.9 For the pr obl em
y ' = y wi t h y ( 0 ) = l ,
f ( x , y ) = y, f(~)(x, y ) = f x+ f y f =O+ 1. y = y,
f(2)(x ' y) = f,,!l) + f ~l ) f = y and f (r)(x, y) = y.
Fr om (13. 29), the Tayl or series met hod is
h 2 h p
Y, =y , , + h y , , + - ~ . . y , , + . . . + ~ y , . ,
p!
- 1 + h+ - ~ - . w + . . . + y .
wi t h Yo- 1. Thus we have
y . - l + h + . . . + p ! ] .
El
E x a mp l e 1 3 . 1 0
we obt ai n
For the pr obl em
1
!
y - with y(0) = 1,
1 +y2
1
f(x, y) =
1 +y 2 '
- 2 y
ftl)(X' Y) = f x + f Yf = (1 + y2)3'
ftE)(x ' Y) = 2(5y 2 - 1).
(1 + y2)5
Thus, for p = 3, the Tayl or series met hod (13.29) is
h h2 ( - 2y, ) h3 2(572 _ 1)
Y, + 1 = Y, + + --- + ---
l+y 2! 3!
with Yo- 1.
D
Ordinary differential equations 349
A much more useful class of met hods is derived by considering
approximations to the first p + 1 terms on the fight of (13.28). We seek an
expression of the form
Y . + 1 = Y. + h ~ a i ~ . , ( 1 3 . 3 2 )
i=1
where kl = f ( x . , y.) and
ki = f ( x n + hal , y,, + hl~iki_l),
i >1 .
We try to det ermi ne the coefficients a~, ;t~ and/ , ~ so that the first p + 1
terms in the Tayl or series expansion of (13.32) agree with the first p + 1
terms in (13.28). Tayl or' s theorem for two variables gives
f ( x, , + h2, y,, + hl~k)
= f (x ,,, y,,) + h -~x +~uk f (x ,,, y,,) + ~ ~ +luk f (x ,,, y,,)
+ . . . +
h p - !
( p - 1)!
-~x + t~k f (x. , y. ) + h PSp(x., y. , 2, gk),
(13.33)
where hPSp is the remai nder term. It is clear that equating terms of (13.32),
using the expansions (13.33), with the terms of the expansion of (13.28),
using (13.26), is very tedious. We shall illustrate only the case p = 2, when
(13.28) becomes
I h2(f~, + f y f ) + h3R3(x,,),
y(x,, + l) = y(x,,) + h f + -i
(13.34)
where f , fx and f y have argument (x,, y ( x , ) ) . Clearly there are an infinity of
ways of choosing r, ai , ;ti and/, ~.
If we choose r = 2, (13.32) when expanded becomes
Y.+l = Y. + h(~ + ~ h2~2('~'2fx + ~ 2 f f y )
+ h3~ES2( xn, Yn, ~2,/*2 f ) , (13.35)
where f , fx and f y have arguments (x,, y, ). On compari ng (13.34) and
(13.35) we see that we require
at + a2 1 0 [ , 2~ , 2 ] l ( 1 3 . 3 6 )
One solution is
1
Of..1 --" O , 0 [ 2 = 1 , ;t 2 = ,U2 = ~ ,
350
Numerical analysis
when ( 1 3.32) becomes
y, + 1= y , + h f ( x , + 89 h, y , + 89 h f ( x , , y , ) ) .
(13.37)
Thi s is known as the mod i f i e d Eul e r met hod.
I f we take ~t~ = a2 = 89 t hen 2 2 - " ~U 2 = 1 and we obtain
y,,+, = y , , + 8 9 f ( x , , + , , y . + hf ( x , , . y , , ) ) ] ,
(13.38)
whi ch is known as the s i mpl e R u n g e - K u t t a met hod.
For p = 4 and r = 4, we obt ai n the c l as s i c al R u n g e - K u t t a met hod
Y,,+] = Y,, + ~ h[ k~ + 2k 2 + 2k 3 + k4],
(13.39)
where
k, = f ( x , , , y, , ),
k z = f ( x . + 8 9 Y. + 8 9
k3 = f ( x , , + 89 h, y,, + 89 hk2),
k 4 = f ( x . + i, Y. + hk3).
Thi s is not the onl y choi ce of met hod for p = r = 4. Ral st on and Rabi nowi t z
(1978) gi ve a compl et e account of the derivation of this and ot her similar
met hods.
13.4 Truncati on errors of one-step methods
The general one-st ep met hod of sol vi ng (13.1) may be expressed as
y.+~ = y . + hq~( x. , y. ; h) (13.40)
where q~(x, y; h) is called the i nc r e me nt f unc t i on. For h , 0 , we rewrite
(13. 40) as
Yn+l - Yn
= ~ ( x , , y , ; h ) . (13.41)
We are interested in the accuracy of (13.41) when consi dered as an
approxi mat i on to (13.1). I f y ( x ) is the solution of (13.1) and (13.2), we
defi ne the l oc al t runc at i on error, t ( x ; h) , of (13.40) to be
y ( x + h ) - y ( x )
t (x; h) = - qb(x, y(x); h), (13.42)
h
Ordinary differential equations
351
where x E [ Xo, b] and h>0. We say that (13.41) is a consistent approxi ma-
tion to (13.1) if
t(x; h)---)O, as h----) 0,
uniformly for x ~ [Xo, b]. From (13.42) we see that, for a consistent method,
y' ( x ) - ~( x , y( x) ; O) = f ( x , y( x) ) . (13.43)
More precisely, we say that the method (13.40) is consistent of order p, if
there exists N >. O, ho > 0 and a positive integer p such that
sup It(x; h)[ ~< Nh p, for all h E (0, h0]. (13.44)
Xo~X~b
The Taylor series method (13.29) is consistent of order p as, on
comparing (13.28) and (13.29), we see from (13.42) that
h p
t(x; h) = hPRp+ l(x) = ~ y(P+ 1)(~), (13.45)
( p + 1)!
where x < ~ < x+ h. We choose
1 ( p l
N = max [y + )(x)I. (13.46)
( p + 1)! X o,X ,b
The Runge-Kut t a method (13.32) with p + 1 terms of its expansion
agreeing with the Taylor series will also be consistent of order p. It is
difficult to obtain an explicit form of N. We just observe from (13.33) and
(13.28) that the truncation error takes the form
t(x; h) = hP(Rp+~ ( x ) + ap( x, h) ) , (13.47)
where Qp depends on p, r and the coefficients ai , 2i , / ~. Provided sufficient
derivatives of y and f are bounded, we can choose a suitable N. Full details
are given by Henrici (1962).
There are methods of the Runge- Kut t a type for which it is possible to
estimate the local truncation error t from the values calculated. The most
commonly used method is due to R. H. Merson, and details of this and other
methods may be found in Lambert (1991). Such estimates of the truncation
error may be used to produce methods which automatically adjust the step
size h according to the accuracy desired in the results.
13.5 Convergence of one-step methods
So far we have considered local truncation errors. These give a measure of the
accuracy of the difference equation used to approximate to a differential
352 Numerical analysis
equation. We have not considered how accurately the solution sequence (y,),
of the difference equation, approximates to the solution y ( x ) of the differen-
tial equation. We shall also be interested in knowing whether the solution of
the difference equation converges to the solution of the differential equation
as the step size h is decreased. We call y ( x , ) - y , t he g l obal t r unc at i on error.
We must defi ne convergence carefully. I f we look at the behaviour of y,
as h ~ 0 and keep n fixed, we will not obtain a useful concept. Clearly
x , = Xo + nh ~ Xo, as h ~ 0, with n fixed,
yet we are interested in obtaining solutions for values of x other than x = x0. We
must therefore consider the behaviour of y, as h---~0 with x, =x0 + nh kept
fixed. In order to obtain a solution at a fixed value of x , x0, we must increase
the number of steps required to reach x from x0 if the step size h is decreased. If
l i m y , = y ( x ) (13.48)
h ~ 0
x n = x fixed
for all x E [x0, b ], we say that the met hod is convergent. This definition is
due to G. Dahlquist.
Exampl e 13.11 The Taylor series met hod (13.29) applied to y ' = y with
y(0) = 1, x ~ [0, b], is convergent. In Exampl e 13.9 we obtained
(
y . - l + h + . - . +
Now by Tayl or' s theorem
h p h p + l
e h = l + h + . . . + ~ + , e h', wh e r e O< h ' < h ,
p! (p + 1)!
and thus
nh( tn
= e 1 - e h ' -
( p + 1)!
As x. = nh and y ( x , , ) = e"h we have for the global truncation error
[ y ( x J - y . l - e ~h 1 - e - 1
( p + 1)!
x. n h P + l h ' - h
< ~ e e
(p + 1)!
Ordinary differential equations 353
for h sufficiently small (see Example 2.13). Thus, as e
x n
l y(x. )- y. I <~ e
Xn h p
( p + 1)!
Hence [ y( x. ) - y. [ ---> 0 as h ---> 0 with x. fixed.
We also remark that there exists M such that
We choose any
[ y( x. ) -- y. [ <. Mh p.
x
x e
M ~> max
0..,-.b (p + 1)!
h ' - h < 1 '
(13.49)
(13.50)
!-1
The following theorem provides an error bound for a consistent one-step
method, provided the increment function ~ satisfies a Lipschitz condition
with respect to y. From this error bound it follows that such a method is
convergent.
Theorem 13.2 The initial value problem
y' = f ( x , y) , y ( x o) = S,
is replaced by a one-step method:
x ~ [X o, b],
y o = S
x ,, = Xo + n h [
I
y~ ~ = yn + hqb(x., y.; h)
n =O, 1, 2 . . . .
where q~ satisfies the Lipschitz condition
I q~(x,y; h)- q~(x,z; h)l < - L , l y- z l
(13.51)
for all x ~ [x0, b], - oo< y, z < o,,, h E (0, h0], for some Lr and h0>O.
If the met hod is consistent of order p, so that the local truncation error
defi ned by (13.42) satisfies (13.44), then the global truncation error
y ( x , ) - y, satisfies
[ ( e(X"-x~ ! )NhP '
[ y( x. ) - Y.[ ~< 11~--" ~ -
1 ( xn -- x~
forL~ . 0 ,
f or Lr = 0,
(13.52)
for all x, E [x0, b] and all h E (0, h0], and thus the met hod is convergent, as
(13.48) is satisfied.
354
Numerical analysis
P r o o f We let
Fr om (13. 42),
and
e, , = I y( x, , ) - y,,l.
y ( x , +l ) = y ( x , ) + h ~ ( x , , y( x, ) ; h) + ht ( x, ; h)
(13.53)
y( x, +~) - y, +, = y ( x , ) - y , + h ~ ( x , , y( x. ) ; h ) - h ~ ( x . , y, ; h) + ht ( x, ; h).
Thus
] y(x.+~) - Yn+~ I ~< I y(x. ) - y, ] + hl ~(X n, y(x. ); h) - ~( x. , y.; h) l
+ h ] t ( x , ; h)}
l y ( x . ) - y . I + h L , ] y ( x . ) - y . I + Nh p+~, (13.54)
where we have used (13. 44) and the Li pschi t z condi t i on (13.51). Hence
e,+i ~< (1 + hL ~,)e, + Nh p+~.
Thi s is a difference i nequal i t y of the t ype consi dered in Lemma 13.1. In this
case m = 0 and
= eo = l y ( x 0 ) - Y0 1 = 0
and f r om (13.23). for L , . 0.
nhLr
e - 1
e, <~ NhP+ !
h L ,
(x,, - xo)Lr )
e - 1 Nh p
= z : 9
For L ~ = 0, Lemma 13.1 yi el ds
en <<.n . Nh p+ ~ = (Xn -- xo) Nh p. V!
You will not i ce f r om (13. 52) that, for all h ~ (0, ho] and all x, E [x0, b],
there is a const ant M ~> 0 such that
I y(x. ) - y, I ~< MhP. (13.55)
A met hod for whi ch the gl obal error satisfies (13. 55) is said to be con-
ver gent o f or d er p. We shall nor mal l y expect that a met hod whi ch is
consi st ent of order p is also conver gent of order p and we say si mpl y that
such a met hod is o f or d er p.
It fol l ows f r om (13.43) that, for a consi st ent met hod, (13. 51) reduces to
I f ( x , y ) - f ( x , z ) ] ~L , I y - z ]
Ordinary differential equations 355
when h = 0 . Thus (13.51) implies the existence of a Lipschitz condition on
f ( x , y ) , of the type required to ensure the existence of a unique solution of
the differential equation.
In order to obtain expressions for L~ for the Taylor series and
Runge - Kut t a met hods, we assume that bounds
Of(k)(x, y)
Lk >~ sup
Xo , X , b ~gy
- o o < y < o o
exist for the values of k required in the fol l owi ng analysis.
A s in Ex ampl e 1 3.4 we observe, usi ng t he mean val ue t heorem, t hat we
may choose L = L0. For the Tayl or series met hod ( 1 3. 29) ,
h h p-1
q~(x, y; h) = f ( x , y) + m f ~l )(x, y) + "." + ~ f<P- l)(x, y)
2~
P!
and, on using the mean value t heorem for each of the f<k), we obtain
L h h 2 h p- I
[r y; h) - q~(x, z; h) l ~< 0 + ~ L~ + ~ t 2 + --. + P!
Lp_ ,)[y - z l-
We therefore choose
h 0 h~ -1
L , = Lo + -~. L l + "'" + P! L p _ i, for h E (0, ho].
(13.56)
For the simple Runge- Kut t a met hod (13.38),
~ ( x , y ; h ) = 8 9 f ( x + h , y + h f ( x , y ) ) ] .
Now with L = L0 we have the Lipschitz condition on f ,
I f ( x , Y) - f ( x , z ) [ <. Lo [ y - z I
and, for h E (0, ho],
I f ( x + h, y + h f ( x , y ) ) - f ( x + h, z + h f ( x , z))[
<~ Lo l y + h f ( x , y ) - z - h f ( x , z) l
<~Lo[[ y - z[ + h [ f ( x , y ) - f ( x , z ) [ ]
~< Lo[I y - z 1+ hLo l y - z l]
= L0(1 + hoLo) l y - z [ .
Thus
[ r q~( x , z ; h ) [ ~< 89 Lo(1 + h o L o ) ] l y - z ]
- Lo(1 + 89 hoLo) l Y- z l ,
356
Numerical analysis
so we may choose
L ~ = Lo (1 + 89 hoLo).
For the classical Runge - Kut t a met hod (13.39), we may choose
hoLo h~L~ hoLo
L ~ = L o 1 + + + .
2 6 24
Full details are gi ven by Henrici (1962).
(13.57)
(13.58)
E x a mp l e 1 3. 1 2
obtain a global error bound, using Theorem 13.2.
We know that y ( x ) = e x and thus from (13.46)
N~
We reconsi der Exampl e 13.11, only this time we will
b
1 e
max ly (p l)(x) I = ~ .
(p + 1)! 0, x, b (p + 1)!
From Exampl e 13.9, fCk)= y for all k and thus Lk = 1 for all k. We therefore
choose, from (13.56),
h0 h~- '
L~= 1 + ~ +. - . + ~
2! p!
and ( 13.52) becomes
e x"L' - 1 ) e b
l Y( X , ) - Y,I ~ h p.
(p 1)!
For x E [0, b] this bound is larger and hence not as good as that of (13.49).
r-I
Exa mpl e 13.13 Find a bound of the global error for Eul er ' s method
applied to the probl em
1
t
Y = l + y 2 ' wi t h y ( 0 ) = l and 0 ~ x ~ < 1.
From (13.45) with p = 1 the truncation error is
t(x; h) h h
= ~ Y " (~s) = - - f ( ~) ( ~, y( ~) ) .
2! 2
By sketching a graph, we find that
=fx+ff,=
-2y
(1 + y2) 3
Ordinary differential equations
357
has maxi mum modul us for y = 1/ 45 and thus
2545
[ f(~(x. y(x)) [
108
We therefore obtain
25~/5
It(x; h) l ~< h ~ .
216
Since ~ (x, y; h) = f ( x , y) for Eul er' s met hod, we choose L
(see Exampl e 13.4). The bound (13.52) becomes
l Y, - y ( x , ) l ~ (e x"L' - 1) 2545 h.
8143
For x , = l ,
= L = 3#3/ 8
l y . - y( 1. 0) I < 0.365 h.
This is a very pessimistic bound for the error. Wi t h h = 0.1 the absolute
value of the actual error is 0.00687. I'-1
Fr om the last example it can be seen that Theorem 13.2 may not give a
useful bound for the error. In practice this is often true and in any case it is
usually impossible to determine (13.52) because of the difficulty of
obtaining suitable values of N and L~. The importance of Theorem 13.2 is
that it describes the general behaviour of the global errors as h ~ 0.
13.6 Ef f ect of rounding errors on one-step methods
One difficulty when comput i ng with a recurrence relation such as
y,+~ = y , + h~b(x, , y, ; h)
is that an error will be introduced at each step due to rounding. If we reduce
h so as to reduce the truncation error, then for a given x, we increase the
number of steps and the number of rounding errors. For a given arithmeti-
cal accuracy, there is clearly a mi ni mum step size h below which roundi ng
errors will produce inaccuracies larger than those due to truncation errors.
Suppose that instead of ( y, ) we actually comput e a sequence (z, ) with
z,+~ = z, + h ~ ( x , , z , ; h ) + e, , (13.59)
where e, is due to rounding. From (13.53) and (13.59) we obtain, anal ogous
to (13.54),
I y( x. +, ) - z. I ~ [ y( x. ) - z. [ + hl r y( x, ) ; h) - r z,; h) l
+ h l t ( x . ; h ) l + l e. I.
358 Numerical analysis
Thus, if l en I <<"e for all n, a quantity e / h must be added to N h p i n (13.52)
to bound I y( x~) - z, I. The choice of method and arithmetical accuracy to
which we work should therefore depend on whether e / h is appreciable in
magnitude when compared with N h p.
13.7 Methods based on numeri cal integration;
expl icit methods
On integrating the differential equation
y' = f ( x , y)
between limits xn and xn+ l, we obtain
f Xn+l
y(xn+ , ) = y(x, , ) + f ( x , y ( x ) ) d x. (13.60)
JXn
The basis of many numerical methods of solving the differential equation is
to replace the integral in (13.60) by a suitable approximation. For example,
we may replace f ( x , y ( x ) ) by its Taylor polynomial constructed at x = x~ and
integrate this polynomial. This leads to the Taylor series method described
earlier (see Problem 13.19). A more successful approach is to replace f by
an interpolating polynomial. Suppose that we have already calculated
approximations Yo, Y~ . . . . . y , to y ( x ) at the equally spaced points
Xr = XO + rh, r = O, 1 . . . . . n. We write
f r = f ( Xr, Yr), r = O, 1, . . . , n;
these are approximations to f ( Xr , y(Xr)).
We construct the interpolating polynomial Pm through the m + 1 points
( x n, f ~) , (X ~-l, f~-l) . . . . . ( Xn- m, f , - m) , where m<~n. We use pm( X) as an
approximation to f ( x , y ( x ) ) between x~ and X ~+l and replace (13.60) by
fx
~n+ I
y~+ ~ = y~ + pro(x) dx. (13.61)
n
It should be remembered that Pm(X), being based on the values
f ~ , f ~ - l , . . . . f ~-m, is not the exact interpolating polynomial for f ( x , y ( x ) )
constructed at the points x~, x~_ l . . . . , X~-m.
By the backward difference formula (4.36),
pro(X) = ~[] ( - 1)J V(f,, where s = (x - x , ) / h .
j
Thus, as in w
m
Xn+ |
J ~ pro(x) d x = h ~ b; Wf ~, (13.62)
j = 0
Ordinary differential equations
359
where
( s) s 0 (13. 63)
Not e that the bj are i ndependent of m and n.
On substituting (13.62) in (13. 61), we obt ai n the A d a ms - B a s h f o r t h
al gori t hm for an initial val ue probl em. To comput e appr oxi mat i ons to the
solution y ( x ) at equal l y spaced poi nt s Xr = XO + r h , r = 0 , 1 . . . . . gi ven the
starting approxi mat i ons Yo, Y~ . . . . . Ym, we use the recurrence rel at i on
y . + , = y . + h [ b o f ~ + b , V f . + . . . + bmVmfn] (13. 64)
for n = m, m + 1 . . . . . where f r = f ( Xr , Y r)"
The starting val ues Yo, Y~ . . . . . Ym are usual l y found by a one-st ep met hod.
Tabl e 13.1 exhi bi t s some of the bj. We shall see that (13. 64) is of order
m + 1, that is, bot h the local and gl obal t runcat i on errors are of or der
m+ l .
Tabl e 13.1 The coefficients bj
0 1 2 3 4
1 5 3 251
1 ~ ~
2 12 8 720
In practice we usual l y expand the di fferences in (13.64) so that it takes the
form
y , . ~ = y , + h [ f l o f ~ + fl~ f ~_~ + " " + ~mfn-m] " (13. 65)
The flj now depend on m. For exampl e, when m = 1 we obtain the second
order met hod
y , . ~ = y , + 89 h ( 3 f , - f ~_~) (13. 66)
and, for m = 3, the fourth order met hod
y, . ~ = y, + ~ h (55f~ - 59f , _ ~ + 3 7f , _2 - 9f, _3). (13. 67)
The fol l owi ng al gori t hm is for the fourt h order met hod (13. 67) on the
i nt erval [a, b]. Four starting val ues Y0, Y~, Y2, Y3 are requi red and these are
found by a one-st ep met hod such as the fourth order Ru n g e - Ku t t a met hod
(13.39).
A l g o r i t h m 13.1 ( Ada ms - Ba s hf or t h of order 4). Let F0, F~, F2, F3
represent the val ues of f~, f~_~, f~-2, f~-3 respect i vel y. Choose a step h such
that ( b - a ) / h is an integer.
360 Numerical analysis
Start with Yo, Y~, Y2, Y3
X := a + 3 h; Y := Y3
Fo : =f ( X, y3)
Fl := f ( X - h, Y2)
F 2 : = f ( X - 2 h , yl )
F 3 := f ( X - 3 h, Yo)
r e p e a t
Y' = Y + 2~h (55Fo - 59F1 + 37F2 - 9F3)
F 3 := F2; F 2 := F~; F~ := Fo
X : = X + h
Fo : - f ( X , Y)
un t i l X = b E]
You will not i ce that, in the above al gori t hm, f is eval uat ed onl y once in each
step.
To det ermi ne the t runcat i on error of the Ada ms - Ba s hf or t h formul a, we
rewri t e (13. 64) as
m
Y" = Z bj VJf~ (13. 68)
h i -0
and consi der this as an appr oxi mat i on to (13.1). Now from (7. 22)
I j a n + I
x'*' f ( x , y(x)) dx = y' (x) dx
Xn JXn
m
= h by VYy'(x.) + h m 2bin+ I y
j , , 0
where x, _ m <~ ~ n <~ Xn + 1" T h u s f r om (13. 60)
m
y ( x . + ,) - y( x. ) = Z bj VT( x. . y( x. ) ) + h m + 'bin+, y (m + 2)(~n), (13.69)
h ~o
where VJ f ( x . , y ( x . ) ) = VJ - l f ( x . , y ( x . ) ) - VJ - l f ( x. _l , y ( x . _l ) ).
The last t erm in (13. 69) is the l ocal truncation error. You will notice that
this is of order m + 1. We expand the di fferences in (13.69) to obt ai n
y( x. +l ) = y ( x . ) + h [ f l of ( x . , y ( x . ) + f l ~f ( x. _l , y( x. _~) ) + ...
+ flmf(Xn Y(Xn_m))] + hm+2bm (m+2)
-m, +IY ( ~, ) - (13. 70)
We assume that y(m+2)(X) is bounded for x E [x0, b] , so t here exists gin+ 2
such that
ma x [ y(m+2)(x)[ <~ Mm+ 2. ( 1 3 . 71 )
x E [x o, b]
Ordinary differential equations
361
We agai n let
e. = [ y(x.) - y. [
and, by the Li pschi t z condi t i on on f ,
I f ( x. , y( x, , ) ) - f ( x, , , Y.)I ~< L l y ( x . ) - y,, I = Le,,.
On subt ract i ng (13. 65) f r om (13. 70), we obt ai n
e,,+, <~ e,, + hL[ l flo l e,, + I fl, l e , _ , + ' - ' + I flm l e,,_ml + hm+21bm..., l Mm+2.
We suppose that the starting val ues Y0 . . . . , Ym are such that
er<~ 6 , for r = 0 , 1 . . . . . m.
We now appl y Le mma 13.1 with
e t r - hL l flrl and b = h' bm.,., l Mm
so that
where
A = eto + et~ + . . . + cLm = hL B, , ,
(13. 72)
Bin= I f l o l + l f l , I + - - . + I f l ml . (13. 73)
Fr om (13.23) we obt ai n, r emember i ng that nh = x , , - Xo,
( hm+" bm exp( ( x, _xo) L B, , , )
l y(x. ) - Y.[ = e,, ~< 6 +
LBm
hm
- . (13. 74)
LBm
The i nequal i t y (13. 74) provi des a gl obal t runcat i on error bound for the
Ada ms - Ba s hf or t h al gori t hm. This bound is agai n mor e of qual i t at i ve than
quant i t at i ve interest. It is usual l y i mpossi bl e to obtain usef ul bounds on the
deri vat i ves of y( x) ; however , (13. 74) does provi de much i nformat i on about
the behavi our of the error as h is decreased. Fr om the first t erm on the right-
hand side, it can be seen that if the met hod is to be of order m + 1, we
requi re 6, a bound for the error in the starting val ues, to decrease like h m
as h tends to zero. Thus we should choose a one-st ep met hod of order m + 1
to det ermi ne starting values.
Theorem 1 3.3
order m+ l i f y
The Ada ms - Ba s hf or t h al gori t hm (13. 64) is conver gent of
~m+2)(X ) is bounded for x E [X o, b] and the starting val ues
362 Numerical analysis
Y0 . . . . . Ym are such that
[Y(Xr)-Yr[ <~D h m+l r = O 1 m 9 9 9 . . o ~ 9
for some D ~>0 and all h ~ (0, h0] where h0> 0. Furt hermore, the global
truncation errors satisfy (13.74), where Mm+2 and B m a r e defi ned by (13.71)
and (13.73) and
8 = max lY(Xr)--Yrl" H
O~ r ~ m
Exa mpl e 13.14 As we have seen before, for simple equations it is possible
to det ermi ne bounds on the derivatives of y and hence comput e global error
bounds. We will seek a global error bound for the second order
Adams - Bas hf or t h algorithm applied to the probl em of Exampl e 13.13.
From Exampl e 13.4, we have L = 3~/3/8 and in Exampl e 13.10 we found
f(2)(x ' Y) = 10y 2 - 2.
(1 + y2)5
Thus
lY(3)(x) I = I ft2)(x, Y(x))I <- ma x
- o o < y < oo
1 0 y 2 - 2
(1 + y2)5
=2
and we choose M 3 = 2.
For the Adams - Bas hf or t h al gori t hm of order 2,
m= 1 , b= - ' - ,-~, B , - 1 / 30 1 + I / 3 , 1 - 2
and thus (13.74) yields
lY(X") - Y"I <" ( d + h2 5"~ " 3x/3 22 ) e xp( 3x/ 3( x" - 0) / 4)
5 8 2
_ h 2 ....... .
12 343 2
( 6 1 0 h 2 ) 343x,/410h2
= + 9~/3 e 943" (13.75)
This error bound is consi derabl y larger than the actual error. For exampl e,
wi t h h - 0 . 1 , n - 1 0 and y~ obtained by the simple Runge - Kut t a met hod
(which yields t~-- 10-5), (13.75) becomes
I y(1. 0)- y,ol <~0.0172.
In fact, to five deci mal places,
y( 1. 0) = 1.40629 and Yz0- 1.40584
Ordinary differential equations 363
so that
I y(1. 0) - Y~01 = 0. 00045. l--1
There is no reason why we should not consider the integral form of the
differential equation over intervals other than x, to x, For exampl e,
instead of (13.60), we may consider
X n + l
y ( x , 1) = y ( x , _ ,) + f ( x , y ( x ) ) d x. (13.76)
d Xn- 1
On repl aci ng f ( x , y ( x ) ) by a pol ynomi al as before, we deri ve the
formul a
* b * V f , + + y,,+, = y,,_, + h[ bof , , + . . . b*Vmf , , ] (13.77)
where
b : = ( - 1 ) / I / l ( - ; ) ds. (13.78)
This met hod is attributed to E. J. Nyst r6m.
The Adams - Bas hf or t h and Nyst r6m algorithms are known as expl i c i t or
open methods. The recurrence relations (13.64) and (13.77) do not involve
f ( x , y,+~); they express y, explicitly in terms of y , , y,_~ . . . . . Y,-m" T h e
explicit nature of these algorithms arises from the fact that we use
extrapolation when obtaining an approxi mat i on to f ( x , y ( x ) ) bet ween x, and
x,+~. As was seen in Chapter 4, extrapolation, that is interpolation outside
the range of the interpolating points, is usually less accurate than interpola-
tion inside this range. The class of formulas considered in the next section
avoid extrapolation by introducing the extra point ( x, +~, f , +~) in the
interpolatory process.
13.8 Methods based on numerical integration;
impl icit methods
We again approxi mat e to the integral in (13.60) and thus obtain a
recurrence relation. Suppose that qm. ~( X) is the i nt erpol at i ng pol ynomi al
through the poi nt s ( X , + l , f , + l ) , ( x , , f , ) . . . . . ( X , - , , , f , - m) . We regard qm+l(X)
as an approxi mat i on to f ( x , y ( x ) ) bet ween x, and x, and repl ace (13. 60)
by
I Xn+ 1
Y,+ ] = Y,, + qm+ 1(X ) dx.
X n
Notice that this approxi mat i on does not involve extrapolation.
364
Numerical analysis
By the backwar d di fference f or mul a (4. 36),
j=O
wher e s - (x - x. +~ ) / h, and
wher e
m+ l
I xn+l
q m + 1(x) d x = h c / V J f n + 1,
X n
j =0
( s) o,
(13.79)
The latter are i ndependent of m and n and Tabl e 13.2 gi ves some of the
values. We thus repl ace (13. 60) by the appr oxi mat i on
Y. = Y. + h [ c o f ~ + c i V f . + "'" + C m+lVm+lfn+l ]. (13.80)
Gi ven y, , y, _~, . . . , Yn-m, we seek y, sat i sfyi ng (13.80). In general ,
(13. 80) does not yi el d an expl i ci t expr essi on for y , . ~ as each of the t erms
VJ f , , i nvol ves f ( x , , Y, Usual l y (13. 80) must be sol ved as an al gebrai c
t~) i s t h e ith equat i on in y,+ ~ by an i t erat i ve met hod. Let us suppose that y , l
iterate in this process. Then (13. 80) is al ready in a conveni ent form for
i t erat i on of the ki nd
Wi + l = F ( w i )
where F is some gi ven funct i on and wi = .,v"). + l- The y. , y._~ , . . . . Y.-m remai n
unchanged t hroughout this i t erat i ve process to cal cul at e y.+~. We will
exami ne the conver gence of this process in the next section. To cal cul at e a
t0)
good first appr oxi mat i on y . we empl oy one of the expl i ci t formul as of
the last section, for exampl e, the Ada ms - Ba s hf or t h formul a.
Thi s leads to the A d a ms - M o u l t o n al gori t hm for an initial val ue probl em.
To comput e appr oxi mat i ons to the sol ut i on y ( x ) at equal l y spaced points
xk = Xo + k h , k = 0 , 1 . . . . . gi ven the st art i ng approxi mat i ons Yo, Y~ . . . . . Ym, we
Tabl e 13.2 The coeffi ci ent s c i
0 1 2 3 4
1 1 1 1 9
. . . .
2 12 24 720
Ordinary differential equations
3 6 5
use t he f ol l owi ng for n = m, m + 1 . . . . .
(o) h [ b o f . + V f . + "'" + b mV mf n ] ( 13. 81a)
Predi ct or: y . ~ = y. + b~
(i+1) f ( i ) + C V f (i) "
Correct or: y . + t = Y. + h [c0 ~ ~ j . + ~ ~ ,
, I n+ l " ~ ' ' ' ' " ~ ' C m Vm+ l/(n')+ ]
i - 0 , 1 . . . . . ( I - 1), ( 13. 81b)
(1)
Yn+! = Y .
wher e
and
f ( i )
. 2 4 7 y(i ) ),
n+l f . = f ( x . , y . )
= VJ-~f (i) - W-
VJf(.')+, "i. +, ' f . .
The coef f i ci ent s bj and cj are def i ned by (13. 63) and (13. 79) respect i vel y.
Equat i on (13.81 a) is cal l ed t he p r e d i c t o r and (13.81 b) the c or r ec t or . For each
step, I i nner i t erat i ons of t he correct or (or I correct i ons) are per f or med and
(t) is consi der ed a suffi ci ent l y accurat e appr oxi mat i on to the y.+~ of (13. 80).
Yn + !
We shall pr ove later that (13. 81b) yi el ds a met hod of order m+ 2. The
fol l owi ng al gor i t hm descri bes the process whi ch i nt egrat es a di fferent i al
equat i on over the interval [a, b ].
A l g o r i t h m 13. 2 ( Ad a ms - Mo u l t o n ) . Choose a st ep
N = ( b - a ) / h is an i nt eger.
Obt ai n st art i ng val ues Yo . . . . . Ym by a one st ep met hod.
I / : = m
r e p e a t
cal cul at e y~)+ ~ f r om t he pr edi ct or
i ' =O
r e p e a t
(i ~) f r om t he cor r ect or cal cul at e y .
i ' = i + 1
un t i l i = I
(I)
Y,+l := Y, +l
n : = n + 1
un t i l n = N
h such t hat
ffl
We nor mal l y expand t he di f f er ences so t hat the f or mul as be c ome
Predi ct or: y~)+~ = y , + h[ f l of , , + fl~f,,_~ + "'" + tim f,,-,,,] ( 13. 82a)
Cor r ect or : .,,247 = Y, + h[y_~,+~f(~ + Yo f , + "'" + 7 mf n - m] " ( 13. 82b)
The Br and Yr are dependent on m. For m = 0, we obtain the second order met hod
y~)+ ~ = y , + h f , , ( 13. 83a)
y(i +l ) (i) + f . ) ( 13. 83b)
.+l = y . + 8 9 . . l
366 Numerical ana/ysis
and for m = 2, the fourth order met hod
(0)
Y, = Y, + ~2 h ( 23f ~- 16f~_~ + 5f~_2), (13.84a)
( i + 1 ) =
y, . ~ y, + ~ h (9f(~')+~ + 1 9 f , - 5f~_! + f, -2)- (13.84b)
We may simplify the iteration procedure for the corrector by noting from
(13.82b) that
(i (i)+ + h~, _l ( f ( i ) e ( i - I
Y, =Y ~ j , . ~ - j , . ~ ) ) , i = 1, 2 . . . . . ( I - 1). (13.85)
Therefore we only need to empl oy the full formula (13. 82b) for the first
inner iteration. For subsequent iterations we may use (13.85). Although we
prove in w 13.9 that the inner iteration process is convergent , we shall also
show that we normal l y expect to make only one or t wo iterations.
In the rest of this section, we will discuss the truncation and global errors
of the corrector and will assume that members of the sequence Ym+l,
Ym + 2 . . . . satisfy (13.80) exactly. Of course in practice this will rarely be
true, as only a finite number of inner iterations may be performed for each
step. We leave the discussion of the additional errors due to this and
rounding until the next section.
To derive the local truncation error, we rewrite (13.80) as
Y,
= c o f ~ . ! + "'" + C m+, vm+lfn+l
h
and consider this as an approxi mat i on to (13.1). From (7.24)
L ~~*'
x"+' f ( x , y ( x ) ) d x = y ' ( x ) d x
dXn n
m+ !
= h Z cj VJy'(xn+ 1) + hm+ 3Cm+ 2Y (m+ 3) ( ~n) ,
y=0
where x, _ m < ~ n < Xn * 1" Thus from (13.60)
y ( x , l) - y ( x , ) '~
2C m 2 Y
= cj VYf ( x , 1, y ( x , !)) + hm+ (m+ 3)(~,). (13.86)
h i=o
The last term is the local truncation error and is of order m + 2.
We expand the differences in (13.86) to obtain
y ( x , + ~) = y ( x , ) + h [ y _ ~ f ( x , y ( x , + . . .
+ Ymf(Xn-m, Y(X ,_m))] + hm+3cm+2Y ( m+3) ( ~n) .
As in the analysis of the Adams - Bas hf or t h algorithm, we let
(13.87)
e , = l y ( x , ) - y , [
Ordinary differential equations
367
and, on expandi ng and subtracting (13.80) from (13.87), we obtain
e , , + l < ~ e . + h L [ l y - , l e , , + I r o l e . + " ' + I )' ml e, , -m]
+ h m+3 I cm+2lMm+3, (13.88)
where l Y ~"+3)(x) I ~< Mm+3 for x ~ [X o, b]. If h is sufficiently small, so that
1 - h L I ) , _, I >0, (13.89)
then
1
en+ 1
1 - h L I r - ~l
hL
= e. +
1 - h t l r - , I
hm+3lC m+2lMm.3
+
1-hZl Y_, l
On applying Lemma 13.1, we obtain
6 hm+2lC m+2lMm+3
l y( x. ) - y. l =e~ +
LC m
hm+2lC m+2lMm+3
LC m
{e,, + hL [ [ 70l e, , + "" + lyre I e, -m] + h"+3lC r,,+2lMm+3}
{([y01 + It-, I)e. + ly~ le._, +" " +l y~l e. -~}
) x o,L c )
exp 1-hLl~,_~l
(13.90)
where
fro--17-1l + I>'ol + ' " + 1 9 , . I
and 6 is the maxi mum starting error, so that
[ Y( Xr ) - - Yr l ~<6, r = 0 , 1 . . . . . m.
(13.91)
The inequality (13.90) provides a global truncation error bound for the
Adams - Moul t on algorithm, assuming that the corrector is always satisfied
exactly. Again, this result is qualitative rather than quantitative but it may be
seen that the method is of order m + 2 if the maxi mum starting error 6
decreases like h m+2 as h is decreased. We should therefore choose a one-step
method of order m + 2 to determine starting values.
Theorem 13.4 The Adams - Moul t on algorithm (13.81), with the corrector
satisfied exactly for each step, is convergent of order m + 2 if y (m+3)(X ) is
bounded for x E [x0, b ] and the starting values Yo . . . . . Ym are such that
[ y(Xr)-- Yr [ ~ D hm+2, r =O, 1 . . . . . m,
368 Numerical analysis
for some D ~>0 and all h E (0, h 0] where h0>0. Furthermore, the global
truncation errors satisfy (13.90) provided h < (L ] 7-~ [ ) -~- Vl
You will observe that the order of the corrector in (13.81) is one higher
than the order of the predictor. This is because we have chosen a pair
which require an equal number of starting values. There is no reason why
we should not use other explicit formulas as predictors, as these clearly do
not affect the final accuracy of the corrector, provided I is suitably
adjusted.
It should be noted that, for small h and a given value of t~, the error
bound for the Adams - Moul t on method is smaller than that of the
Adams-Bashf or t h method of the same order, since for m > 0, we can show
that [ Cm [<Ibm [ (see Problem 13.32) and C m < B,,, For the second order
Adams - Moul t on method the relevant constants are c2 = - ~ and Co = 1,
whereas for the second order Adams-Bashf or t h method we have b2 = ~ and
B~ = 2. For nearly all problems the Adams - Moul t on algorithm is far
superior to the Adams-Bashf or t h algorithm of the same order, in that
global truncation errors are much smaller. This might be expected since the
Adams - Moul t on method does not use extrapolation.
Again, we may obtain other corrector formulas by integrating the
differential equation over two or more intervals. By integrating over two
intervals, we obtain a class of formulas which includes Si mpson' s rule (see
Problem 13.33).
Exampl e 13.15 Obtain a global truncation error bound for the second
order Adams - Moul t on algorithm applied to the problem of Example 13.13,
assuming that the corrector is satisfied exactly after each step. From
Example 13.4, L = 3~/3/8 and, from Example 13.14, M 3 = 2. Since c2 = - ~ ,
Co = 1 and ),_~ = 89 (13. 90)becomes
1 - h3~/ 3/ 16 12 343
( 4 h2) ( 6 3x n) 4h2
= 6+ 943 exp 16- 3~/ 3h 943
provided h< 16/(3~/3). For example, with h =0 . 1 , 6= 10 -5 and x , = 1, this
bound is
[ y (1.0) - y 10 [ ~< 0.00248.
Notice that this is much smaller than that in Example 13.14 for the second
order Adams-Bashf or t h algorithm. D
Ordinary differential equations
36 9
13.9 Iterating with the corrector
We first veri fy that the i nner i t erat i ve pr ocess ( 13. 81b) for sol vi ng the
cor r ect or is conver gent for smal l h. We r ear r ange the expanded f or m
( 13. 82b) as
( i +l )
Y.+~ = Y. + h [ y o f . + "-" + Ymf~-m] + hy ~f(x.+~ y~) ), (13. 92)
- ~ n + l
~) may be expr essed as
whi ch, on wri t i ng w/ = y.+~,
Wi + l = F ( w i ) .
In Chapt er 8, Theor em 8.3, it was shown that the sequence (w;) will
conver ge to a sol ut i on if F sat i sfi es cl osure and a sui t abl e Li pschi t z
condi t i on on some interval. In this case, we t ake the i nt erval to be all the real
number s, when cl earl y cl osure is sat i sfi ed, and we seek Lr < 1, such that
I F( w) - F( z) [ ~< L F I w - z I (13. 93)
for all reals w and z. Fr om (13. 92), we have
IF(w) - F( z ) I = I hy_, f (x, , +t , w ) - hy_~f(x,,+~, z) l
<~hLly_~] I w- z l (13. 94)
wher e L is the usual Li pschi t z const ant for f ( x , y) wi t h respect to y. We
obt ai n a condi t i on of the f or m (13. 93) wi t h LF < 1 for
1
h < ~ .
I ,-,It
Thi s does not i mpose a ver y severe rest ri ct i on on h for most pr obl ems. You
will not i ce t hat this is the same condi t i on as (13. 89) whi ch was necessar y for
the val i di t y of the error bound (13. 90).
We also obser ve, for later use, that
~'+~) ~') I = I F (y~.)+ ) - F ( Y ~' - I) l
Yn + l - - Y. + t 1 n + l
- ~!-1) I (13. 95)
~< hLl y_, l ly~+, Y,+t 9
We now i nvest i gat e the effect of per f or mi ng onl y a fi ni t e number of
~t) is the last iterate i t erat i ons wi t h the correct or and we will assume that y ,+
whi ch is act ual l y comput ed. Fr om ( 13. 82b)
(/) ( / +l )
Y.+l = Y. + h[ y ,f~t) + ) ' of . + "'" + ~/mL m] "~" Y{~+, - n + l - -- Yn + l 9
We let
P n = ~,(! + 1) . (!)
. ) ' n + l - - . Yn + l
and assume that
[ P, I -< P, for all n.
370 Numerical analysis
We wi l l also allow for r oundi ng errors in the process and thus, instead of
comput i ng a sequence ( y, ) whi ch sat i sfi es the correct or exact l y for each n,
we act ual l y comput e a sequence (z, ) wi t h
z,,+, = z,, + h [ y _l f ( x , , + i , z , , +, ) + . . . + ymf(Xn_m, Z n_m) ] 4" Pn 4" ~'n
where e, is due to roundi ng. On subt ract i ng (13. 87) and put t i ng
e . = I y ( x . ) - z. [
we obt ai n
e.+,<.e.+hL[ ly_,le.+,+ I Y01 e . + "'" + I ~'~ I e. -m]
+ h m+ 3 l C m+ 2 l M m+ 3 + p + E , (13.96)
where l e, I ~ < e for all n. By compar i ng (13.96) with (13. 88), it can be seen
that an amount ( p + e ) / h must be added to hm+2l c , , +2l Mm+ 3 in the error
bound (13.90). I f we have an est i mat e for Mm+3, we can then deci de how
smal l p and e shoul d be. Since hm+~lCm+21Mm+ 3 is a bound for the local
t runcat i on error of the correct or (see (13. 87)), we are in fact compari ng
( p + e ) / h wi t h the t runcat i on error and in the next section we descri be how
this may be estimated.
Havi ng chosen p, we may use (13. 95) to decide when to stop iterating
wi t h the corrector. We require
I - (t) ( t - /9
Y.+, - Y.+ ~)1 <- ~ . (13.97)
hl~'-,IL
I f L is not known, as is usual , we can use the mean val ue t heorem in
(13. 94) and replace L by I Of/Oylat some poi nt near x = x,+~, y = y, , ~. We
then use the est i mat e
t _~
~f
7y
f(]) _f(o) I
. ( 1 ) . ( 0 ) '
Yn+ 1 --Yn+ !
(13.98)
in pl ace of L in (13.97).
13.10 Milne's method of estimating truncation errors
W. E. Mi l ne first proposed a t echni que for est i mat i ng local t runcat i on errors.
For the t echni que to wor k the predi ct or must be of the same order as the
corrector. However , it is onl y necessary to make one or t wo appl i cat i ons of
the correct or for each step, pr ovi ded h is small.
Ordinary differential equations 371
We consider the following formulas.
Predictor: y ~)+ ~ = y,, + h [ bo f ,, + b ~ V f ,, + . . . + bm + l V m + l f n ] (13. 99a)
(i+ 1) f. ,e(i) ,,. ~7r Vm+ If(i )
Corrector: y, + ~ = y, + h [ + + "" + C m + ]
" OJ n + l ~"l " J n +l 1 . J n + l
i = 0 , 1 . . . . . ( I - 1), (13. 99b)
9 ( ! )
Yn+l = Yn+l"
Apart from the necessity of an extra starting value, these formulas are
i mpl ement ed in an identical manner to (13.81).
From (13.69) and (13.86),
m+l
y(x,,+ , ) = y( x, , ) + h Z by vJf(x. , y( x, , ) ) + h m+ 3bin+ 2y (m+ 3)(/,in),
j=O
m+l
y(x,,+ ,) = y( x, , ) + h Z c1 Vgf(x" +" y(x, , + 1)) + hm+ 3C m+ 2Y
j=O
(13.100a)
(13.100b)
where r/, and ~, are intermediate points. We assume that
y Cm + 3) (/7 n) = Y (m + 3) ( ~ n)
and write
T ,, = hm+2c m+2y( m+3) ( ~n) (13.101)
for the local truncation error of (13.100b). (An additional factor h is
accounted for by writing (13. 100b) in the form (13.86).) On subtracting
(13.100a) from (13. 100b), we obtain
1 T,, = Z c~ V x . +~, y( x . +~)) - b~ V x . , y( x . ) ) . ( 1 3 . 1 0 2 )
C m+ 2 j = O j = O
We now use approximations
V i f ( x . , y ( x . ) ) - - - VJ f . , VJ f ( x . + , y ( x . + , ) ) = Vif(0)
, - - J n + l ,
to obtain
,_(
cj VJ f c~ . . . . . . + ~ - a j V
a m+ 2 - C m+21~j -O j =O
1 ( C m+ 2 )(_ ( l ) _ ( 0 )
= h . . . . Y " + I - Y " + ~)' (13.103)
bin+2 - - C m+ 2
on using (13. 99a) and (13.99b). We may use this last result to estimate the
local truncation error at each step.
372 Numerical analysis
If only one application of the correcter is made, I = 1 and we seek an
estimate of
(l) I <~ hL l 7_l ll " (1) I I P, [ I Y(2+)~ Y, +' = - - Yn + l --Y(On)+l
Now, from (13. 103)and (13.101),
h lT-1 I L l ('' (0,
y . - y . , + l l = h 2 t l F _ l l
b i n + 2 - - C m+ 2
C m+ 2
I r . I
~ hm+4L[ 7_, [ lbm+2-c, , +z lM, , +3,
where Mm. 3 ~ [ Y (m * 3 ) ( X ) ] for x E [X o, b]. Thus an approxi mat e upper
bound for the p, is
fa= hm+4L ] )'_t l [bm+2-Cm+21Mm+3
and ~/ h is small compared with
h m+ 2 l c m+ 2 l M m+ 3 > ] T . I ,
if
1 [Cm+2l
h << . (13.104)
L]),_ ~ ] [bm+2-C m+2]
If this condition is satisfied, there should be no advantage in applying the
correcter more than once. The effects of correcting only once are investigated
more precisely by Henrici (1962). l amber t (1991) also describes the
consequences of applying the correcter only a small number of times. Ralston
and Rabinowitz (1978) show how the estimate (13.103) of the local
truncation error may be used to improve the predicted value .,,+~v (~ at each step.
Exampl e 13.16 We consider the following fourth order pair:
Predictor: -(o)
y.+~ = y. + ~ h( 55f . - 59f._~ + 37f . _ 2 9f . _ 3), (13.105a)
Correcter: ~,(i+~) -(0 + 1 9 f . - 5 f . +f n-2) (13 105b)
." ,,+l = Yn + ~4 h ( 9 J , , + l -1 9 9
Here m= 2 , bm b4=251 = =
and Cm+2 C4 - - ~ . T he local truncation error
of the predictor is
h m+ 2b m 24 7 = 251 h4y( 5)
(,I .)
and that of the correcter is
T , hm+Ecm y ( m+ 3 ) _ h4y(5)( =
Thus, from (13.103),
l ( b c ) 1 o o
T. --- h ' c tY.+l - Y,+ 1) ----" h - (Y(,l+) l - Y,+ I).
(13.106)
Tabl e 13.3 Results for Example 13.17
x~ yt0, jao, y~,, f~,, Tn- t x 1 0 8 y(x,,) Er r o r x l 0 s
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
1.18236027
1.22322920
1.26252409
1.30037111
1.33688243
1.37215806
1.40628744
0.41701839
0.40059453
0.38550990
0.37161390
0.35877598
0.34688284
0.33583594
1
1.04879039
1.09531337
1.13977617
1.18236150
1.22323034
1.26252510
1.30037199
1.33688319
1.37215872
1.40628800
0.5
0.47619926
0.45460510
0.43495475
0.41701788
0.40059409
0.38550953
0.37161358
0.35877571
0.34688262
0.33583576
- 87
- 80
-71
- 62
- 54
- 46
- 39
1.0
1.04879039
1.09531337
1.13977617
1.18236140
1.22323016
1.26252485
1.30037168
1.33688284
1.37215832
1.40628758
0
0
0
0
- 10
- 18
- 25
-31
- 35
- 40
- 42
(,o
-,,4
cA)
374 Numerical analysis
I f onl y one correct i on is made for each step, then f r om (13. 104) with
I ) ' - i [ = ~ we requi re
8 19 0.2
h << 9 _ . (13. 107) r-]
3L 270 L
Ex a mp l e 13. 17 We appl y the fourth order met hod of Exampl e 13.16 to
the pr obl em in Exampl e 13.13. Fr om (13. 107) with L = 3~/3/8, we see that
we requi re h << 0. 289 i f onl y one correct i on is made for each step. Table
13.3 gi ves results wi t h h = 0.1; the T~ are obt ai ned f r om (13. 106). The
starting val ues are cal cul at ed usi ng the fourth order Ru n g e - Ku t t a met hod.
The last col umn gi ves the errors y ( x ~) - y~).
At first sight it mi ght appear that for x~ ~> 0.9 the correct i on process makes
the appr oxi mat i ons worse, as [y~')-y(x~)[> [ y~0) _y( x, ) [ for n~>9.
However , omi ssi on of the correct or affects all approxi mat i ons so that the
fourt h order Ada ms - Ba s hf or t h process does not give the col umn y~0~. 1-!
13.11 Numerical stabil ity
We have seen that the i ncl usi on of roundi ng errors does not radically
change the gl obal t runcat i on error bounds for the met hods i nt roduced earlier
in this chapter. However , the error bounds do not al ways provi de a good
descri pt i on of the effect s of roundi ng errors and starting errors.
We consi der the cal cul at i on of member s of a sequence (y~)~= 0 by means
of a recurrence relation (or difference equat i on) of the form
Y~ = F( y ~, Y~- l , . . . , Yn-m), tl = m, m + 1 . . . . . (13.108)
gi ven starting val ues Yo, Y~ . . . . . y~. We suppose that a si ngl e error is
i nt roduced so that, for some arbitrary r,
Yr = Yr + E
and we comput e the sequence
Yo , Yl . . . . . Yr - l , Yr , Yr . . . . .
We assume that this sequence satisfies (13. 108) for all n > . m e x c e pt
n = r - 1. I f r ~< m the error will occur in the starting val ues and i f r > m the
error will occur when (13. 108) is comput ed for n = r - 1. We say that the
recurrence rel at i on (13. 108) is n u me r i c a l l y uns t abl e if the errors l y~- y*[
are unbounded as n ---> oo.
Cl earl y an unst abl e recurrence relation will not provi de a sat i sfact ory
numeri cal process, as we are certain to introduce roundi ng errors. Of course
Ordinary differential equations
375
in practice we will introduce more than one roundi ng error, but in the
anal ysi s it is more instructive to consi der the effect of an isolated error.
Exa mpl e 13. 18 Suppose we wi sh to calculate ( y. ) f r om
Y,+l = 1 0 0 . 0 1 y , - Yn-I
with Yo = 1, y ~ - 0.01. The general sol ut i on of (13. 109) is
(13. 109)
where c~ and c2 are arbitrary. The solution whi ch satisfies the initial
condi t i ons is
y
for whi ch c~ = 0 and c: = 1. The i nt roduct i on of errors may be consi dered
equi val ent to changi ng the particular solution being comput ed. Thi s will
have the effect of maki ng c~ non-zero, so that a term c~.100" must be added
to y, . Clearly c~. 100 ~ will qui ckl y ' s wamp' c2/100 ~. For exampl e, if
:~ 6
Y0 = Y0 + 10- = 1. 000001,
* 0.01
Yl =Yl =
then
Y2 = 0. 000099 and Y5 --- - 1.0001. Ul
For an approximation to an initial value differential probl em, the function F
in the recurrence relation (13.108) will depend on h and f ( x, y). In defining
stability we do not decrease h as n ---),,,, as was necessary when considering the
convergence of y, to y(x, ). We are now interested in onl y the stability of
recurrence relations resulting from differential equations. Because the precise
form of the recurrence relation depends on f ( x , y), it is difficult to derive
general results regarding the stability of such approximations. We find it
instructive, however, to consider in detail the linear differential equation
y' = - A y , with y( 0) = 1, (13. 110)
where A is a positive constant, whi ch has sol ut i on y ( x ) = e -Ax. Not e that this
sol ut i on converges to zero as x---),,*. Lat er in this section, we show that to a
large extent the lessons to be learned f r om this pr obl em will appl y to other
initial val ue probl ems. We shall certainly expect that any approxi mat e
solution of the test equat i on (13. 110) shoul d sat i sfy y, ---)0 as n---),,o so that
the behavi our of the exact solution is mi mi cked. I f such a propert y is
sat i sfi ed the met hod is said to be absolutely stable.
376
Numerical analysis
We start by considering the mi d poi nt rule
Y.+~ = Yn - i + 2h f .
which is obtained by using
(13.111)
Yn -- Yn - I
2h
=f.
as an approximation to (13.1) or, alternatively, from Nyst r 6m' s formula
(13.77) with m= 0 (see Problem 13.25). For (13.110), f ( x , y ) = - A y and
the method becomes
Y.+I = -2hAy, , + y._~. (13.112)
This is a second order difference equation with characteristic equation
z2 + 2hAz - 1 =0,
whose roots are
z2ZI] - A h + (1 + A2h 2) 1/2.
The general solution of (13.112) is
y . = c l z ~ + C zZ ~. (13.113)
We find that z ~' behaves like y ( x ) as h is decreased with x. fixed, but that
z~ has a completely different behaviour. On expanding the square root, we
obtain
Z l = - A h + 1 +89 2) + O(h4).
By O( h 4) we mean terms involving h 4 and higher powers. On comparison
with
- Ah
= 1 - A h + 89 2) + O( h3) ,
we see that
Z l = e - Ah + O( h 3)
= e-Ah(1 + O( h3) ) ,
since
Thus
e -ah= 1 + O( h ) .
Z ~ = e - A n h ( 1 + O( h3) ) n = e-Anh(1 + n. O( h3) )
= e-aX . (1 + O( h 2 ) ) ,
Ordinary differential equations 377
as x, = nh. Now as h ---)0 with x,, = nh fi xed,
z ~---~e-a~,= y( xn)
and, for smal l val ues of h, z~ behaves like an appr oxi mat i on to y ( x ) . It can
also be shown that z ~'---) 0 as n ---) ** for any fi xed h > 0 and A > 0.
On the ot her hand
z2 = - A h - (1 +AZh2) ~/2 < - 1
since Ah > 0 and thus I z2l > 1. In fact
z2 = - A h - (1 +A2h2) i/z
= - A h - 1 - I A 2 h 2 + O(h 4)
= _e ah + O( h 3)
= --eah(1 + O( h3) )
and thus
z~= (-1)"eA"h(1 + O( h3) ) n
= (--1)"eAx,,(1 + O( h2) ) .
As h is decreased with x, fi xed, z ~' does not appr oxi mat e to y ( x , ) .
The general solution of (13. 112) is, t herefore,
y , =c l e - a ' " ( 1 +O( h2) ) +C 2( - - 1 ) " e ax " ( 1 + O( h2) ) (13. 114)
and for this to be an appr oxi mat i on to y ( x , ) , we requi re
c ~ = l and c 2=0.
We woul d t herefore like to comput e the part i cul ar solution
y , = z ~
of (13. 112). In pract i ce we cannot avoi d errors and we obt ai n, say,
* n
y~ = (1 + 61)z~'+ 62z2.
In this case the error
* n
[ y , - y , [ = [ 6,z~' + 6 2z21 "-')0", as n---).o
since I z21> 1. Therefore the met hod (13. 111) is unst abl e for this differential
equation.
The sol ut i on z~ is called the par as i t i c sol ut i on of the di fference equat i on
(13.112). It appears because we have repl aced a f i r s t order differential
equation by a s e c ond order di fference equation. In this case the parasi t i c
solution ' s wa mps ' the solution we woul d like to det ermi ne.
Even if we coul d use exact ari t hmet i c with the mi dpoi nt rule, we woul d
not avoid numeri cal instability. We need starting val ues Y0 and y~, and y~
378 Numerical analysis
must be obtained by some other method. Let us suppose that this yields the
exact solution y(xl ). Clearly, with
Yl = y ( x l ) = e - Ah * g l ,
we do not obtain c~ = 1 and c2 = 0 in (13.113).
Exampl e 13.19 The initial value problem
y' ( x) = - 3 y ( x ) with y( 0) = 1
has solution y( x) = e -3x. We use the midpoint rule (13.111) with h=0. 1 and
starting values
Y0 = y ( 0 ) = I and y~=0. 7408.
The last value is y ( h ) = e -3h correct to four decimal places. Results are
tabulated in Table 13.4. The oscillatory behaviour for x, > 1 is typical of
unstable processes and is due to the ( - 1 ) " factor in the second term of
(13.114). i-1
The midpoint rule is thus at a major disadvantage when used for problems
with exponentially decreasing solutions and will not produce accurate results
although it has a smaller local truncation error (see Problem 13.38) than the
second order Adams-Bashf or t h algorithm, which we consider next.
From
h
Y, , + , =Y, , + 2( 3 f , , - f , , - , )
with f ( x , y) = - A y (A > 0, as before), we obtain the recurrence relation
y, , +t =( 1- ~Ah) y, , + 8 9
which has characteristic polynomial
z 2 - ( 1 - 3 A h ) z - 8 9
The roots are
!
z~ i ~ 3 Ah) (l _ Ah +
z2 = i [ ( 1 - i ~A9 2h2)1/2 ]
On expanding the square root, we obtain
z~ =89 - ~ a h ) + 1 + 8 9 + 9 a 2 h 2 ) - ~ ( - a h + 9A2h2)2 + O(h3)]
= 1 - A h + 8 9 O( h 3)
= e- a h( 1 + O ( h3) ) .
(13.115)
Ordinary differential equations 379
Th u s
z ~ = e-A"h(1 + O( h 3 ) ) n
= e - a x . ( 1 + O ( h 2 ) ) ,
so that, for small h, z~' is an approximation to y ( x , , ) . We also find that for all
values of h >0 and A>0 we have I z, l< 1 so that z~'---) 0 as n---) =,.
We seek the solution of (13.115) of the form
y , , =z ' ~
but, in practice, obtain
* ) z ~ + ~ 2z~ .
y, = ( 1 + 6 1
No w
z2=[ [(1 - 3 A h ) - (1 - a h + 9 A2h2) l / 2]
and detailed analysis shows that Z z < - I if A h > l , and - 1 <z 2 <0 if
0 < A h < 1. Hence the method is absolutely stable only if h < 1/A. We will
certainly also need to satisfy this inequality if the truncation error is not to
be excessive and, therefore, this is not a severe restriction for the differential
equation y ' = - Ay . However, this restriction may be severe for other
problems.
Similar results may also be obtained for the equation
y ' = - A y + B , (13.116)
where A>0 and B are constants. The constant B will not affect the
homogeneous part of the difference approximations to (13.116) and it is
Tabl e 13.4 An unstable process
xn y .
0 1
0.1 0.7408
0.2 0.5555
0.3 0.4075
0.4 0.3110
0.5 0.2209
0.6 0.1785
0.7 0.1138
0.8 0.1102
0.9 0.0477
1.0 +0.0816
1.1 -0. 0013
1.2 +0.0824
1.3 -0. 0507
1.4 +0.1128
380 Numerical analysis
only necessary to add the particular solution y . = B / A , to the general
solution of the homogeneous equations (13.112) and (13.115). As h--->0,
with nh = x. fi xed, this particular solution converges to y ( x ) = B / A , a
particular solution of (13.116). For both of the met hods investigated in this
section, we woul d like the solution
but actually obtain
y . = z'~+ B / A ,
~[c n
y . = (1 + 61)z~'+ t~2z2 + B/ A .
Thus the absolute stability conditions are identical to those for the problem
(13.110).
I f a first order difference equation is used to approxi mat e a differential
equation then there are no parasitic solutions to give stability difficulties.
However, the only solution of the difference equation may not satisfy our
absolute stability condition: y, - - ) 0 as n--~ oo. If Eul er' s met hod is applied to
(13.110), we obtain
y,,+ l = y , , - A h y .
= (1 - Ah)y, , ,
so that
y,, = ( 1 - Ah ) "Yo.
We can see that y. ---> 0 only if [ 1 - Ah [ < 1, that is, h < 2/ A . In practice, for
(13. 110), this restriction on h should normal l y be satisfied for accuracy
reasons even when A is large. For a large value of A, the exact solution
decreases very rapidly and a small step size would be essential. However,
consi der the equation
y ' = - A ( y - 1)
which has solution y ( x ) = ce -Ax + 1 where c is arbitrary. Eul er ' s met hod is
y,,+~ = y , , - A h ( y . - 1)
so that
y . - c~(1 - Ah ) " + 1
where c~ is arbitrary. Now y(x)---> 1 as x--->**, but y.---> 1 as n--->** with h
fixed, onl y if [ 1 - A h [ < 1, that is, h < 2/ A . This condition may be severe if
A is very large. I f the initial value y( 0) is near + 1, the solution will not vary
rapidly for x > 0 and there should be no need to use a small step size.
There are similar restrictions for all the other met hods introduced in this
book except the Ada ms - Moul t on met hod of order 2. When this method is
Ordinary differential equations 381
applied to ( 1 3.1 1 0 ) we obtain
y.+~ = y. + 89 h ( - A y . - Ay. +~)
so that
Thus
(1 + 8 9 = (1 - 8 9
( )
1 - i A h "
1 Yo
Y" = l + -~ A h
and y.--->0, as n--->**, for all A>0 and fixed h >0 . Results for the
Runge- Kut t a met hods are included in Probl em 13.40 and a full discussion
of stability is given by Lambert (1991).
The stability behaviour of approximations to the general equation
y ' = f ( x , y ) (13.117)
will to a certain extent be similar to those for the special probl em (13.110).
On expanding f ( x , y) using Taylor series at the point x = X , y = Y, we
obtain
f ( x , y) = f ( X , r ) + ( y - Y) f y ( X, r ) + O( x - x ) + O( y - y)2
= y f y ( x , Y) + [ f ( x , Y) - Yf y( X, Y)]
near the point (X , Y). Thus locally (that is, over a small range of values of
x) the problem (13.117) will behave like one of the form (13.116).
Alternatively, we may argue that one woul d need to be rather optimistic to
expect success in computing a decreasing solution of (13.117) by a met hod
which is unstable for the particular case (13.110).
13.12 Systems and higher order equations
Suppose that
f~ = fj (x, z,, z2 . . . . . zk), j = 1 . . . . . k,
is a set of k real valued functions of k + 1 variables which are defi ned for
a < x~< b and all real z~ . . . . . zk. The simultaneous equations
y' ,(x) = f ,(x, y, ( x) . . . . . yk(x))
y 2(x) = f 2( x, y ~(x) . . . . . yk(x)) (13.118)
y'k(x) = f k( x, yl ( x) . . . . . yk(x))
382 Numerical analysis
wher e x E [a, b ], are called a s ys t em of ordi nary differential equations. Any
set of k di fferent i abl e functions'["
yl ( x) , . . . , y k ( x )
sat i sfyi ng (13. 118) is called a solution.
In general , the solution of (13. 118), i f it exists, will not be uni que unless
we are gi ven k ext ra conditions. These usual l y take the form
yj (Xo) = s/, j - 1 . . . . . k, (13. 119)
wher e the sj are known and x0 ~ [a, b ]. We are thus gi ven the val ues of the
y j ( x ) at the poi nt X=Xo. T he pr obl em of sol vi ng ( i 3. 118) subject to
(13. 119) is agai n known as an initial value probl em.
We write (13. 118) and (13. 119) in the form
y' = f ( x, y), (13. 120)
y(x0) = s,
where y( x) , y' ( x) , f ( x , y ) and s are k-di mensi onal vect ors, whose ith
component s are y i ( x ) , y i ( x ) , f i ( x , y~(x) . . . . . yk( x) ) and si respect i vel y. The
pr obl em has a uni que solution if f sat i sfi es a Li pschi t z condi t i on of the
form: there exists L ~> 0 such that, for all real vect ors y and z of di mensi on k
and all x E [a, b ],
I} f ( x, y) - f ( x, z) I I - t II Y - z I1-.
(13. 121)
The norm is def i ned in w 10.2.
Numer i cal met hods anal ogous to those for one equat i on may be deri ved in
an identical manner. For the Tayl or series met hod, we det er mi ne f ( r) ( x, y)
such that
h 2
y(x,,+ i) = y( x, ) + h f ( x , , y(x, )) + ~ f(l)(x,,, y(x, )) + -.-
2~
h p
+ ~ f ( P - l ) ( x . , y ( x . ) ) + h " I R p . l ( x . )
P!
(cf. (13. 28)), where Ro.~ is a r emai nder term. For exampl e,
d f ( x, y( x) ) ~gf ~-~dyj ( x) ~gf
f(~)(x, y( x) ) = dx Ox j...~ dx Oy/
Ox
j__l f j Oy)'
(13. 122)
t" Note that the subscripts denote different functions and not values of approximations to
values of a single function, as in earlier sections.
Ordinary differential equations
383
where the ith component of 3f / 3y j is 3f i / 3yj . The formulas for f(' ) are very
lengthy for r ~> 2. The Tayl or series met hod of order 2 is thus
Y0 =S,
h a (13.123)
y. ~ = y. + hf ( x. , y. ) + ~ f(~)(x~, y.),
2~
where y. is a vector whose ith el ement yi,. is to be considered as an
approxi mat i on to y~(x.) and x. = x o + nh.
We may derive Runge - Kut t a met hods by using Tayl or series in several
variables. Anal ogous to (13.39), we have the classical met hod
Y.+l = Y. + -~ h[kl + 2k2 + 2k3 + k4],
where
k~ = f ( x . , y. ) , k2 = r( x . + 89 h, y. + 89 hkt ) ,
k 3 = f ( x. + ~ h, y. + 89 hk2), k4 = f ( x. +l , y. + hk3).
Pr edi ct or -cor r ect or methods are also identical to those for single equations.
We merel y replace the y, and f , in (13.82) by vectors y, and f, .
Global truncation error bounds for all methods are very similar to
those for a single equation. We replace the quantity l y , , - y ( x , , ) l by
II y . - y(x. ) II..
Ex a mp l e 1 3. 20
with
We have
and
Consider the initial value probl em
y'~(x) = xy~ (x) - y2(x)
y'2(x) = - y, (x) + y2(x)
Yi ( 0) = sl, Y2(O) = s2.
[ y2)]
f(x, y ) = f 2(x, Yl, Y2) = (-Yl + Y2)
Of 3f 3f
3y2
=[Yl]+(Xyl-Y2)[O - l X ] + ( - Yl + Y2) [ - 1 ] 1
( - (1 + x)y ~ + 2y 2)
384
Numerical analysis
The Ta yl or ser i es me t hod o f or der 2 is
YI. O = S I , Y2. 0 = $ 2,
h 2
YI , . + 1 = Yl , n + h ( x , , y l , , , - Y2,.) + ~ {( x2 + 2)y1. .
2
- (1 + x. )Y2, . },
h 2
Y2. n+ 1 = YZ , , , + h ( - y l , . + Y2, n) + ~ 1- ( 1 + x , , ) y l , . + 2Y2,.},
2
f or n = 0 , 1 , 2 . . . . .
The Ad a ms - Mo u l t o n p r e d i c t o r - c o r r e c t o r me t hod of or der 2 (cf.
( 13. 83) ) is as f ol l ows .
Ini t i al val ues: Y~.o = s~, Y2.0 = s2;
[ (0 )
Pr edi ct or : | Y1 ' , + 1 = Y1 . , + h f l . , ,
/ (0) + h f 2 , .
[ Y2 , n+ ~ = Y2, n n,
[ , , ( i + 1) I h(f(i)
Is' l,,,+ i = Yi,,, + 7 1,,,+ l + f l , . ) ,
Co r r e c t o r : / - (~ + ~) ~ h ( f (~)
[ Y2 . n+ 1 = Y2,,, + 7 2,.+ l +f 2, . ) ,
(t)
Yl , n + l = Yl , n + l ,
i = 0 , 1 . . . . . ( I - 1 ) ;
(t) .
Y2, n+ 1 = Y2, n+ 1,
wher e fc~) j, . = f ~( x. 1, Y ci) ci) ).
1. n+l ~ Y 2. n+l
( The super scr i pt i in t he a bove is t o di st i ngui sh di f f er ent i t er at es and does
not denot e di f f er ent i at i on. ) r l
I f t he real val ued f unct i on
f = f ( x , z l , z 2, . . . , z m)
of m var i abl es is de f i ne d f or x E [ a, b ] and all real z~ . . . . . Z m, we say t hat
y ( m) ( x ) = f ( x , y ( x ) , y ' ( x ) . . . . . y ( m - l ) ( x) ) ( 13. 124)
is a di f f er ent i al equat i on o f or de r m. I f t he sol ut i on y ( x ) is to be uni que we
need m ext r a condi t i ons . I f t hese t ake t he f or m
Y (r)(x 0) = tr, r = 0, 1 . . . . . m - 1, ( 13. 125)
wher e x0 ~ [ a, b] and t he t r a r e gi ven, we agai n say t hat we have an i ni t i al
val ue pr obl em. You wi l l not i ce t hat we are gi ven t he val ues of y and its fi rst
m- 1 der i vat i ves at one poi nt x = x0. In Chapt er 14 we wi l l di scuss pr obl e ms
wi t h condi t i ons i nvol vi ng mor e t han one val ue of x.
We ma ke t he s ubs t i t ut i ons
z j ( x ) = y (j - l )(x), j = 1 . . . . , m,
Ordinary differential equations 385
so that
z~ (x) = y ~j) (x) = zj (x), j = 1, . . . , m - 1,
and thus rewri t e the si ngl e equat i on (13. 124) as the s ys t em of fi rst or der
equat i ons
z ' , ( x ) = z~(x)
z'2(x) = z 3 ( x )
t
Z m_l ( X ) ' - Z m( X )
z ' ( x ) = f ( x , z, ( x) , z 2( x) . . . . . Z m( X ) ) .
The condi t i ons (13. 125) become
z, (Xo)= to, z 2(xo) = tl . . . . . Z m( XO) = t ~_t ,
and we have an initial val ue pr obl em of the t ype consi der ed at the begi nni ng
of this section.
Example 13. 21 Consi der
wi t h
Let
when
Eul er ' s met hod is
y " = g ( x , y, y ' )
y( xo) = to, y' (Xo)= tl.
z , ( x ) = y ( x ) , z 2( x) = y' ( x) ,
z ' , ( x) = z ~( x ) ,
z ' z ( x) = g ( x , z l ( x ) , z2(x)).
g 0 = $ ,
z. = z. + hf ( x. , z. ) .
For this exampl e, we obt ai n
zl. o = t 0, Z2.o = t i ,
g l , n + i ~--" gl,n+hg2, n,
z2., = z2., + h g ( x , , z l . , , z 2. , ).
There are al so some special met hods for hi gher or der di fferent i al
equat i ons, part i cul arl y second order equat i ons. See Henr i ci (1962) and
Lamber t (1991).
We have al ready seen in w 13.11 that for Eul er ' s met hod appl i ed to y' = - A y
with A >0, we obt ai n y,--->0 as n--->*,, with h fi xed, onl y if h < 2/ A . I f we
386 Numerical analysis
have a linear system of the form
y ' = - Ay (13.126)
where A is an m x m matrix, then one component of the general sol ut i on
vect or has the form
y(x) = cle -a~x + c2 e-a2~ + . - - + Cm e-a"'x (13. 127)
where ,,1.~, 22 . . . . . A m are the ei genval ues of A. I f ,~,r > 0, r = 1, 2 . . . . . m, the
sol ut i on shown in (13. 127) will sat i sfy y(x)--->O as x--->,,o. It can be shown
that with Eul er ' s met hod we need to choose
h < 2/A,r, for r = 1, 2 . . . . . m (13. 128)
if the solution of the di fference equat i on is to decrease t owards zero as the
number of steps increases. I f 0 < 2 ~ < 2 2 ~ < " - ~ 2 m in (13. 127) then, for
c ~ , 0, the first t erm will be the domi nant term for large x. The inequalities
(13. 128) may i mpose a very severe restriction on h if/~m is very much larger
than ;t~. We illustrate this by the fol l owi ng example.
Ex ampl e 1 3.22 Gi ven
y' (x) = - 8 y ( x ) +7z ( x )
z' (x) = 42y( x) - 43z( x)
with y( 0) = 1 and z(0) = 8, the sol ut i on is
(13. 129)
- 50x
y( x) = 2 e - ~ - e
z (x) = 2e -x + 6e -5~
The equat i ons (13. 129) are of the form (13.126) where A has ei genval ues
2, = 1 and 22 = 50. For large x > 0 the domi nant term in this solution is 2e -x
but we must choose h < 0. 04 i f Eul er ' s met hod is to produce a decreasi ng
solution, rq
Equat i ons such as those of Exampl e 13.22 are called stiff equations.
There are similar di ffi cul t i es with all the met hods we have di scussed except
the Ad a ms - Mo u l t o n met hod of order 2 which, as we saw in w is
al ways absol ut el y stable. The restrictions on h gi ven in Probl em 13.40
for the Ru n g e - Ku t t a met hod and in w 13.11 for the second order Ad a ms -
Bashfort h met hod must be sat i sfi ed for each 2 r in (13.127). Lambert (1991)
gi ves a full di scussi on of the sol ut i on of st i ff equations.
13.13 Comparison of step-by-step methods
The t ypes consi dered are
(T) Tayl or series met hods.
( RK) Runge - Kut t a met hods.
Ordinary differential equations 387
(E) Explicit methods, for example, the Adams-Bashforth method
(13.64) and Algorithm 13.1.
(PC) Predictor-corrector methods, for example, the Adams-Moulton
algorithm 13.2 and the method (13.99).
Eas e of us e
The main consideration is how simple they are to program for a digital
computer.
(T) Difficult to establish suitable formulas, except for low order methods.
No special starting procedure.
(RK) Very simple. No special starting procedure.
(E) Quite straightforward, but special starting procedure required.
(PC) Slightly difficult to implement, due to inner iterations, unless only
one correction is made per step. Special starting procedure
required.
A moun t of c omput a t i on
We usually measure this by the number of evaluations o f f that are needed
for each step. If f is at all complicated, most of the calculation time will be
spent on evaluations of f.
(T) Many evaluations of f and derivatives required per step, for example,
six evaluations for the third order method.
(RK) Comparatively many evaluations of f, for example, four for the
fourth order method. For the commonly used methods, the number of
evaluations equals the order of the method.
(E) One evaluation of f, regardless of the order.
(PC) Two or three evaluations of f, regardless of the order.
Trun c a t i on error
The truncation error gives a measure of the accuracy of a method for a
range of problems but does not necessarily provide a correct comparison of
the accuracy of methods for one particular problem. We consider the
magnitude of the truncation error for a given order.
(T) Not very good.
(RK) May be quite good.
(E) Much poorer than (PC).
(PC) Good.
388
Numerical analysis
Error est i mat i on
(T) Difficult.
(RK) Difficult, although there are some methods which
indication of truncation errors.
(E) Difficult.
(PC) Fairly easy i f Mi l ne' s method (w is used.
give some
St abi l i t y and st i f f syst ems
For all the methods introduced in this chapter, except the Adams-Moul t on
algorithm of order 2, the step size must be restricted for absolute stability
and hence there are difficulties for stiff equations.
St ep size adjust ment
It is sometimes desirable to adjust the step size as the solution proceeds.
This is particularly necessary if there are regions in which the solution
has large derivatives, in which a small step size is required, and other
regions in which derivatives are small, so that a large step size may be
employed.
(T) Very easy to change the step size, as these are one-step methods.
( R K ) A s (T).
( E ) Q u i t e difficult, a s specia l f o r m u l a s a re required.
(PC) As (E).
Choi c e of met hod
(T)
(RK)
(E)
(PC)
Not recommended unless the problem has some special features, for
example if f and its derivatives are known from other information.
Recommended for a quick calculation for which there is little concern
about accuracy. Also recommended to obtain starting values for (E)
and (PC).
Only recommended for problems for which f is particularly difficult
to evaluate.
Recommended (particularly (13.99)) for most problems as
truncation error estimation is possible. The fourth order
predi ct or-correct or (13.105) is probably one of the best methods,
except for stiff equations. It involves comparatively few function
evaluations (for one or two inner iterations) and provides truncation
error estimates.
Ordinary differential equations
389
Probl ems
Sec t i on 1 3.1
13.1 Show that for 0 ~< x ~< 1
y' = xy, wi t h y ( 0 ) = l ,
has a unique solution and find an appr oxi mat e sol ut i on by Pi card iteration.
13.2 Show that
y ' = sin y, wi t h y ( x o) = s,
has a unique solution on any interval cont ai ni ng x0.
13.3 Gi ven that g ( x ) and h( x ) are cont i nuous funct i ons on some interval
[a, b ] cont ai ni ng x0, show that the linear differential equat i on
y' = g ( x ) . y + h ( x ) , wi t h y ( x o) = s,
has a unique sol ut i on on [a, b ].
(Hint: use the fact that g ( x ) is bounded on [a, b] . )
Sec t i on 1 3.2
13.4 Find the general sol ut i on of the homogeneous difference equations:
(i) y . . ~ - ay . = 0, where a is i ndependent of n;
(ii) y . + 2- 4 y . . t + 4y. = 0;
(iii) Y. 2yn Y,,+~ + 2y,, = O;
(iv) Y. +3- 5Y,,+2 + 8y, , +~- 4y. = O;
(V) Y.+3 + 3Y.+2 + 3y,,+~ + y,, = O.
13.5 Find the general sol ut i ons of the di fference equations:
(i) y . + E- 4y . . ~ + 4 y . = 1;
(ii) Yn + Y. +~- 2y. = l;
(iii) Yn 2y. . ~ + y . = n + 1.
(Hint: find particular sol ut i ons by t ryi ng sol ut i ons whi ch are pol ynomi al s in
n, for exampl e, y. = k0, y. = k0 + k~ n, y. = k0 + k~ n + k2 n2, where the k~ are
i ndependent of n.)
13.6 The sequence ( u . ) ~o of positive reals satisfies the recurrent inequality
un+l <-au. + b
for n - 0 , 1 . . . . . where a, b ~> 0 and a * 1. Prove by i nduct i on that
Un <~ anuo + b.
390 Numerical analysis
This is a stronger result than (13.23) when (13.22) holds with m =0 and
a = l + o t o.
S e c t i o n 1 3 . 3
13.7 Derive an explicit formula for the Taylor series method (13.29) with
p - 3 for the differential equation of Problem 13.2.
13.8 The initial value problem,
y' - a x + b, with y ( 0 ) - 0,
has solution
y ( x ) = 89 a x 2 + bx.
If Eul er' s method is applied to this problem, show that the resulting
difference equation has solution
y,, = 89 ax2, + bx, , - 89 ahx, , ,
where x, = nh, and thus that
y ( x , ) - y , = 89 a h x , .
13.9 Show that, for any choice of v, the ' Runge- Kut t a' method
y,,+~ = y , + 89 h[ k2 + k3], (13.130)
kl = f ( x , , y , ) , k2 = f ( x , + v h , y , + v h k l ) ,
k 3= f ( x . + [1 - v ] h , y , + [1 - v ] h k ~) ,
is consistent of order 2. That is, show that the first three terms in the Taylor
series expansion of (13.130) agree with the first three terms ( p = 2) in
(13.28).
13.10 Show that the ' Runge- Kut t a' method
y,,+~ = y , + h[ Z k~ +~k2 +~k3],
kl = f ( x , , y, , ), k 2 = f ( x , , + 89 h, y , + 89 k h ) ,
k3 = f ( x , + ] h, y , + ] k2h ),
is consistent of order 3. (Consider four terms in the Taylor series
expansion.)
13.11 Write a formal algorithm for the classical Runge-Kut t a method
(13.39) when applied to the initial value problem (13.1) and (13.2).
13.12 Solve the equation
- x + x - y, with y( 0) = 0
y, 2
Ordinary differential equations 391
by the simple and classical Runge- Kut t a methods to find an approximation
to y(0.6) using a step size h= 0.2. Compare your solution with the exact
solution
y ( x ) = - e -~ + x 2 - x + 1.
13.13 Use the classical Runge-Kut t a method (13.39) to solve the initial
value problem
y ' = sin y, with y( 0) = 1.
Take h = 0.2 and obtain estimates of the solution for 0 ~< x ~< 1.
Sec t i on 1 3.5
13.14 Show that a suitable value of the Lipschitz constant L~ of (13.51) for
the method of Problem 13.9 is
L r 1 8 9 + [ 1 - v l ] L h 0 ) ,
where L is the usual Lipschitz constant for f. Deduce that the method is
convergent of order 2.
13.15 Show that a suitable value of the Lipschitz constant Lr of (13.51)
for the method of Problem 13.10 is
L 2 L 3
~ .
L r + h 2 3!
13.16 Use Theorem 13.2 to find a global truncation error bound for Eul er' s
method applied to the equation given in Problem 13.12 with 0 ~< x ~< 1.
13.17 Find a global truncation error bound for the Taylor series method of
order 2 applied to the initial value problem of Examples 13.10 and 13.13
with 0~< x ~ 1. ( Hi n t : show first that If(E)(x, y)[ ~< 2.)
13.18 Find a global truncation error bound for Eul er' s method applied to the
differential equation of Problem 13.2 with x0=0, h = 0.1 and 0 ~ x ~< 1.
Sec t i on 1 3.7
13.19 The Taylor polynomial of degree p - 1 for f ( x , y ( x ) ) constructed at
x = Xn is
q ( x) = f ( x ~ , y ( x , ) ) +
( X ~ X n)
1!
~ f ( 1 ) ( x , , y ( x , , ) ) + . . .
( X -- Xn ) p - 1
+ f ( P - ' ) ( x , , y ( x , ) ) ,
( p - 1)!
392 Numerical analysis
where the f ( r ) are as defined in w 13.3. Thus
h p
i x . . , f ( p _ l)(x. , y ( x . ) ) . q( x ) d x - h f ( x , , y ( x , ) ) + . . . +
x. p!
Show that, on approximating to the integral in (13.60) by the above integral
of q and replacing y ( x , ) by y, , we obtain the Taylor series algorithm for an
initial value problem.
13.20 Determine the third order Adams-Bashf or t h algorithm in the form
(13.65), that is, expand the differences.
13.21 Determine an approximation to y( 1. 0) , where y ( x ) is the solution of
y' = 1 - y, with y(0) - 0,
using the Adams-Bashf or t h algorithm of order 2 with h =0. 2 and starting
values y0=0, y~=0. 181 ( =y ( 0 . 2 ) correct to three decimal places).
Compare your calculated solution with
y ( x ) = 1 - e-X.
13.22 Use the fourth order Adams-Bashf or t h algorithm 13.1 to solve the
equation in Problem 13.13. Take h = 0.2 and consider 0 <~ x ~ 1.
13.23 Evaluate the error bound (13.74) for the initial value problem and
method of Problem 13.21. Compare the global error bound with the actual
error for x, = 1.0. (Initial error bound, 6 = 0.0005.)
13.24 Obtain a global truncation error bound for the Adams-Bashf or t h
algorithm of order 2 applied to the equation in Problem 13.2. ( Hi nt : show
that f(2)(x, y) = sin y cos 2y has maxi mum modulus + 1.)
13.25 Derive explicit forms of the Nystr6m method (13.77) for m = 0, 1, 2.
(Expand the differences.)
Sec t i on 1 3.8
13.26 Determine the third order Adams - Moul t on algorithm (m = 1) in the
form (13.82).
13.27 Show that if a predi ct or-correct or method is used to solve the linear
differential equation of Problem 13.3, then Y, may be found expl i c i t l y
from the corrector formula.
13.28 Use the Adams - Moul t on algorithm of order 2 with h = 0 . 2 to
determine an approximation to y(1. 0), where y ( x ) is the solution of the
differential equation of Problem 13.21. (Note the result of Problem
13.27.)
Ordinary differential equations 393
13.29 Use the fourth order Ad a ms - Mo u l t o n pr edi ct or - cor r ect or met hod
(13.84) to solve the equation in Probl em 13.13. Take h - 0. 2, consi der
0~<x<~ 1 and perform just one or t wo correct i ons per step ( I = 1 or 2).
Compar e your results with those of Probl ems 13.13 and 13.22.
13. 30 Eval uat e the global truncation error bound (13. 90) at x, = 1.0 for the
calculation of Probl em 13.28. Compar e the actual error with this error
bound.
13.31 Obt ai n a global truncation error bound for the Ad a ms - Mo u l t o n
met hod of order 2 applied to the equat i on in Probl em 13.2. ( Compar e with
the result of Probl em 13.24 which also cont ai ns a hint. )
1 3 . 3 2 Show that
S'o (- s) ds > j S:(S)ds I j , fo rj~ > 2,
and thus that I bjl >l ml for j~>2, where bj and m are def i ned by (13.63)
and (13.79) respectively.
1 3 . 3 3 Correct or formul as may be obt ai ned by approxi mat i ng to (13.76),
using q,,+ ~ as in w 13.8. Show that, with both m = 1 and m = 2, there results
the formul a
y , + , =y , _~ + ~ h[f, _~ + 4f , +f , +t ].
This is called the Mi l ne - Si mps on formul a and it reduces to Si mps on' s rule
if f is i ndependent of y.
S e c t i o n 1 3 . 1 0
1 3 . 3 4 The pr edi ct or -cor r ect or formul as
ym)+ - _
~= y , , + 8 9 f , ~).
y ( i + i)
,,+, = y. + 89 h( f . + ~.+~t'~i)),
obtained f r om (13.99) with m = 0, are bot h of order 2. Show that the local
truncation error o f the corrector as gi ven by (13. 101) is
T, = - ~ h2y(3)(~n)
and that
1 . (1 ) _ (0)
T ,, = - - - - t y, + I - y,,+ l).
6 h
Di scuss the choi ce of h if onl y one correct i on is to be made for each step.
394 Numerical analysis
13.35 Write a formal algorithm for the met hod of Exampl e 13.16. You may
assume only one correction is made per step.
Sec t i on 13. 11
13.36 Investigate the numerical stability of the following met hods for the
differential equation y' - - A y with A > 0.
(i) y,,+t=y,,+~h[ 5f. + 8L-f~_, ]
( Adams - Moul t on corrector of order 3.)
(ii) y,,+,= y,,_, +~h[ f . +l + 4f . +f . _, ]
( Mi l ne- Si mps on formula. See Probl em 13.33.)
(iii) y.+, = y. _, + 89 h[ f . +, + 2f. + f . _, ]
(Double trapezoidal rule.)
13.37 Investigate the numerical stability of the following method for the
test equation y' = - A y with A > 0.
y. = y._, + 89 h( f . + 3f. _, ).
Show that absolute errors [ y . - y*[ (notation as in w 13.11) will decrease
for sufficiently small h but that relative errors [ ( y . - y , , ) / y . ] are un-
bounded as n ---),,,,.
13.38 Show that the local truncation error of the mi dpoi nt rule (13.111) is
hEy(3)(~,).
13.39 The equation y' =- A y + B has general solution y( x) =c e- ' U+ B/ A
where c is arbitrary and thus y ( x ) - - ) B/ A as x---~**. If Eul er ' s method is
applied to this equation, show that y, ~ B/ A as n ~ ** with the step size h
fixed, only if h < 2/A.
13.40 The equation y' = - A y with A > 0 is to be solved. Show that y, --) 0 as
n--)** with the step size h fixed (and thus there is absolute stability), if
(i)
(ii)
h < 2/,4, for the simple Runge - Kut t a method;
1 2 1 A3h3 1 A4h4
1 - A h + ~ A 2h - m +
2! 3! 4!
< 1 ,
for the classical Runge-Kut t a method.
Sec t i on 1 3.1 2
13.41 Show that, for the function f of the equations in Exampl e 13.20 with
x ~ [0, b] where b > 1, a suitable choice of L, the Lipschitz constant in
(13.121), is L = 1 + b.
Ordinary differential equations 395
13.42 Determine, as in Example 13.20, the difference equations for the
Taylor series method of order 2 applied to the two simultaneous equations
y' l ( x) = x 2y l ( x ) - y2(x)
y' 2(x) = - Yl ( x ) + xy2 ( x) .
with y~ (0) = s~ and y2(0) = s2.
13.43 Solve the equation
y" + 4 x y ' y + 2y 2 =0.
with y(0) = 1, y' (0) = 0, using Euler' s method with h = 0.1, to obtain
approximations to y(0.5) and y' (0.5).
13.44 Determine the difference equations for the classical Runge-Kut t a
method applied to the differential equation of Problem 13.43.
13.45 Derive the difference equations for the Taylor series method of order
2 applied to
y(a) = 2 y" + x 2y + 1 + x,
assuming that y ( x ) , y' ( x ) and y" ( x ) are known for x = x0.
13.46 Determine the difference equations for the Adams-Moul t on
predictor-corrector method of order 2 applied to the differential equation in
Problem 13.45.