1.1 First Order Approximation of Derivatives
1.1 First Order Approximation of Derivatives
The basic concept of the finite difference method relies in the approximation of the
derivative in a point by the quotient of differences in a small interval, transforming the
differential problem in an algebraic problem. For example, the derivative u' of the function
u(x) is approximated using the finite difference method as the secant of the tangent (see
Figure 1).
u' x
u x 2 u x 1
h
, x1 x x2
(0)
Expression (1) is an approximation for almost all the points in the interval (x1,x2), however
Rolle theorem shows that for at least one point there is an strict equality.
If we establish that:
u' x 1
u x 2 u x1
h
(0)
equation (2) represents a forward difference. On the other hand, if we establish that:
u' x 2
u x 2 u x1
h
(0)
u' x 0
u x 2 u x1
h
(0)
Considering some geometric properties, equation (4) represents a better approximation than
equations (2) or (3) because the first equation presents a better alignment with the tangent
line.
In order to demonstrate the previous statement it is necessary to study the precision of each
approximation. This study consists in the analysis of the error associated to the replacement
of the differential equation by a finite difference approximation and what parameters
influence this error. We will start with the centered difference approximation of u' (x). For
convenience Figure 3 shows the centered scheme, where x0 is the middle point between x1
and x2. Using the Taylors series expansion of u(x1) and u(x2) at u(x0)
u x 1 u x 0
h
1 h
u' x 0
2
2! 2
u x 2 u x 0
h
1 h
u' x 0
2
2! 2
u' ' x 0
u' ' x 0
1 h
u ' ' ' 1
3! 2
x1 1 x0
(0)
1 h
u' ' ' 2
3! 2
x0 2 x2
(0)
1
u x 2 u x 1 u ' x 0 1 1 h 2 u ' ' ' 1 u ' ' ' 2
h
3! 2
(0)
Taking the absolute value and replacing u'''(1)+u'''(2) by 2 Max|u'''(x)| it is finally obtain
1
u x 2 u x 1 1 h 2 max u ' ' ' x
h
24
(0)
If u'''(x) is bounded between x1 and x2, the error associated to u'(x 0) using a centered
difference method is limited for a constant multiplied by h 2 or is order h2 (O(h2)). If the
value of h is reduced to a half, the error associated to this method is reduced by a factor of
4.
As we already mentioned, the forward and backward difference approximation of u'(x) are
less precise that the centered difference method. Using Figure 1 and equation (2), the
Taylors series expansion at x1 of u(x2) is
u x 2 u x1 h u' x1
1 2
h u' '
2!
x1 x2
(0)
Therefore:
1
u x 2 u x 1 u ' x 1 1 h max u ' ' x
h
2
(0)
The forward difference method (2) is O(h) only if u''(x) is limited between x1 and x2. If we
reduce the value of h to a half the error associated is reduced by a factor 2. The same result
is obtained for the precision analysis of the backward method.
Until now we have derived approximations of u'(x) using O(h) and O(h2) methods using
geometric considerations. We will present a method to derive systematically
approximations of u'(x) using the following expression:
u' x1 u x1 u x 2
( 0)
where and are coefficient to be determined. Expanding u(x2) in Taylors series at x1 and
replacing in (11) we ended up with:
u ' x 1 u x 1 h u' x 1
h 2
u' '
2
(0)
Making equal the coefficients in (11) and (12) we ended up with +=0 and h=1;
therefore =1/h and =-1/h. Replacing these values in (11) we get expression (2). The
precision of the forward difference method is O(h) for these values of and . It is required
that u'(x1) in (12) must be exact for the constant term and linear in u.
Therefore any difference scheme can be built in this way. In the undetermined coefficient
method the approximation of u'(x) is written as a weighted sum of the u(x i) . The
coefficients have to be determined considering that the method must be exact for the
biggest grade polynomial in u(xi).
A three points approximation for u'(x1) using the undetermined coefficient scheme it is
presented. Assuming u(x1), u(x2) and u(x3) are known at x1, x2=x1+h and x3=x2+h, the
approximation of u'(x1) is written as:
u' x1 u x1 u x 2 u x 3
(0)
h 2
h 3
4h 3
2 h 2 u ' ' x 1
u ' ' ' 1
u' ' ' 2
3!
3
2
u ' x 1 u x 1 h 2 h u ' x 1
(0)
where x1 1 , 2 x2. Making equal the coefficients of equation (14) with equation (2) it
is obtain that ++=0, +2=1/h and +4=0, which implies that u'(x 1) is exact at the
constant, linear and quadratic term. The solution to the previous equations is =-3/2h,
=2/h and =-1/2h. Replacing these values in (13)
u' x1
1
3 u x1 4 u x 2 u x 3
2h
(0)
(0)
Expanding u(x1), u(x2), u(x3) and u(x4) in Taylors series at x0, replacing at (16), making the
coefficients equal to equation (2) and solving the 4 by 4 system of equations for the four
unknowns, the solution is =(12 h) -1, =-(2/3 h)-1 ,=(2/3 h)-1 and =-(12 h)-1. Replacing in
(16)
u' x 0
1
u x 1 8u x 2 8u x 3 u x 4
12h
(0)
u ' x m1
then
u x 3 u x 2
u x 2 u x1
, u' x m 2
h
h
(0)
u x 3 u x 2 u x 2 u x1
h
h
u' ' x
u ' x m 2 u x m1
h
u' ' x
u x 1 2u x 2 u x 3
h2
(0)
(0)
Depending on the point where the equation (20) is evaluated, the approximation will be
forward, backward or centered.
u x1 u x 2 h u' x 2
h2
h3
h 4 'v
u' ' x 2
u' ' ' x 2
u 1 x1 1 x2
2!
3!
4!
(0)
u x 3 u x 2 h u' x 2
h2
h3
h 4 'v
u' ' x 2
u' ' ' x 2
u 2
2!
3!
4!
(0)
x2 2 x3
1
h 2 'v
u
x
2
u
x
u
x
u
'
'
x
u 1 u ' v 2
1
2
3
2
2
4!
h
(0)
or
1
u x 1 2u x 2 u x 3 u ' ' x 2 M
h2
where M
(0)
1
max u ' v x
2
h
Equation (24) shows that the centered difference approximation is O(h2) if u'v(x) is limited.
Using the forward or backward difference method the order of the approximation will be
reducer to O(h).
Abramovitz and Stegun (1977) present a variety of formulas to made approximations of
high order for the derivatives. Godunov and Ryabenki (1964) and Isaacson and Keller
(1966) show a rigorous treatment to the problem of numerical approximations.
1.3 FINITE DIFFERENCE EQUATIONS
Linear differential equations are combination of derivatives of first and second order, order
zero and independent terms. The approximation of these equations using the finite
difference technique is called finite difference equations. As an introduction to the subject
we will present the approximation of a linear homogenous differential equation of first and
second order.
A first order equation example is given by the following initial value problem:
u' x a u x 0
x0
(0)
with u(x0)=u0 and a equal to a constant. A forward finite difference approximation for u'(x)
is given by:
u x 2 u x1
a u x1 0
h
(0)
u x 2 1 ah u x 1 0
(0)
Dividing the x axis in (n+1) equal magnitude segments, numerating these segments as
0,1,2n and calling u(x1)=u(xn)=un and u(x2)=u(x2+h)=un+1, equation (27) is written as:
u n 1 (1 ah )u n 0
(0)
Equation (28) is a finite difference expression of first order because it requires the value of
u at a single point to solve it. In particular, it requires satisfying the initial condition. A
general solution of equation (26) is for example given by u n=c n, where is the solution of
the characteristic equation:
1 ah 0
(0)
1 ah
(0)
Using expression (30) and the initial condition, the solution of equation (28) is given by:
u n u 0 1 ah
(0)
It is observed that the exact solution for equation (28) has different behaviour depending on
the value of (1-ah):
(1-ah) > 1
(1-ah) =1
0<(1-ah)<1
(1-ah)=0
-1<(1-ah)<0
(1-ah)=-1
(1-ah)<-1
It is clear that the solution of the finite difference equation disagree with the behavior of the
differential equation, depending on the value of (1-ah). This behavior is related to the
numerically stability concept of the differential equation.
An example of a second order differential equation is given by the following contour
problem:
u' ' x u' x 0
(0)
And the following boundary conditions u(0)=1 and u(1)=0, where >0. The exact solution
to this equation is:
e e x
u x
e 1
(0)
Expression (32) represents a boundary layer problem (we suggest to draw the exact solution
for =1,0.1 and 0.01). Using centered difference approximation for u'(x) and u''(x), the
following expression is obtained:
u x 1 2u x 2 u x 3
u x u x1
3
0
2
2h
h
(0)
Using the same convention as the previous example, equation (34) can be written as:
1 p u n 1 2u n 1 p u n 1 0
(0)
where p= h/2. Expression (35) represents a second order finite difference equation and it
requires two different points to be solved. In particular equation (35) must satisfy the
boundary conditions stated at (32). In order to find the general solution for equation (35)
using a solution of the type u n=c n. Replacing this solution in (35) a characteristic equation
is obtained.
1 p 2 2 1 p 0
(0)
Expression (36) has two roots 1 and 2, and if they are different they are two particular
solutions of (35). The general solution is a linear combination of both roots:
u n c1 1n c 2 n2
(0)
where c1 and c2 depend on the boundary conditions. In this example both roots are real:
1=1 and 2=1-p2. Independently of the boundary conditions the solution of the finite
difference equation is:
1 p
1 p
u n c1 c 2
(0)
We expect that the solution given by expression (38) has a similar behavior than the
solution to the differential equation. However this is only true when p is less than 1. If p is
greater than 1 the exact solution will have an oscillatory behavior because it changes when
n is even or odd.
A question pops up: how can we avoid the oscillations? The answer is simple: using the
backward or forward finite difference approximation of u'(x) (depending of the sign of ).
References: