Numerical Differentiation

Types of Differentiation:
Forward expansion of Taylor series:
''
' f ( xi ) 2 h=step size
f ( xi 1 ) f ( xi ) f ( xi ) h h ...
2!
' f ( xi 1 ) f ( xi ) first forward
f ( xi ) O (h)
h difference
Backward expansion of Taylor series
''
' f ( xi ) 2
f ( xi 1 ) f ( xi ) f ( xi ) h h ...
2!
f ( xi ) f ( xi 1 ) first backward
'
f ( xi ) O (h) difference
h

Subtract forward expansion from the backward expansion:

' f ( xi 1 ) f ( xi 1 ) 2 centered
f ( xi ) O (h )
2h difference
Higher-accuracy Differentiation Formulas:
High-accuracy diffentiation formulas can be obtained by adding
more terms in Taylor expansion.
Forward Taylor series expansion:
''
' f ( xi ) 2
f ( xi 1 ) f ( xi ) f ( xi ) h h ...
2
or
''
' f ( xi 1 ) f ( xi ) f ( xi ) 2
f ( xi ) h O (h )
h 2

Approximate the second derivative using finite difference formula

'' f ( xi 2
) 2 f ( xi 1 ) f ( xi )
f ( xi ) 2
h
Then,
' f ( xi 1 ) f ( xi ) f ( xi 2
) 2 f ( xi 1 ) f ( xi ) 2
f ( xi ) 2
h O (h )
h 2h
or,
' f ( xi ) 4 f ( xi 1 ) 3 f ( xi ) 2
forward
2
f ( xi ) O (h ) scheme
2h

Backward and centered finite difference formulas can be derived in

a similar way:

' 3 f ( xi ) 4 f ( xi 1 ) f ( xi 2 ) 2 backward
f ( xi ) O (h ) scheme
2h

' f ( xi 2
) 8 f ( xi 1 ) 8 f ( xi 1 ) f ( xi 2 ) 4 centered
f ( xi ) O (h )
12 h scheme
EX: Calculate the approximate derivative of
4 3 2
f ( x) 0 .1 x 0 . 15 x 0 .5 x 0 . 25 x 1 .2

at x=0.5 using a step size of h=0.25 (true value=-0.9125).

First, evaluate the following data points:

xi-2=0 ; f(xi-2)=1.2 xi+1=0.75 ; f(xi+1)=0.6363281
xi-1=0.25 ; f(xi-2)=1.103516 xi+2=1 ; f(xi+2)=0.2
xi=0.5 ; f(xi)=0.925

Forward difference scheme:

' 0 .2 4 ( 0 . 6363281 ) 3 ( 0 . 925 )
f ( xi ) 0 . 859375 t
5 . 82 %
2 ( 0 . 25 )
Backward difference scheme:
' 3 ( 0 . 925 ) 4 (1 . 035156 ) 1 . 2
f ( 0 .5 ) 0 . 878125 t
3 . 77 %
2 ( 0 . 25 )
Centered difference scheme:
' 0 .2 8 ( 0 . 6363281 ) 8 (1 . 035156 ) 1 . 2
f ( xi ) 0 . 9125 t
0%
12 ( 0 . 25 )
Richardson Extrapolation:
As done for the integration, Richardson extrapolation uses two
derivatives of different step sizes to obtain a more accurate
derivative.
In a similar fashion applied for the integration, use two step sizes
such that h2=h1/2. Richardson extrapolation recursive formula:

4 1
D D ( h2 ) D ( h1 )
3 3

O(h4) O(h2) [centered difference scheme]

The approach can be iteratively used by Romberg algorithm to get

higher accuracies.
EX: Use Richardson extrapolation of step sizes h=0.5 and h=0.25 to calculate the
derivative of the function
4 3 2
f ( x) 0 .1 x 0 . 15 x 0 .5 x 0 . 25 x 1 .2

at x=0.5.

Centered scheme finite difference approximation for h=0.5:

0 .2 1 .2
D ( 0 .5 ) 1 .0 t
9 .6 %
1

Centered scheme finite difference approximation for h=0.25:

0 . 6363281 1 . 103516
D ( 0 .5 ) 0 . 934375 t
2 .4 %
0 .5
Apply Richardson extrapolation for improved accuracy:

4 1
D ( 0 .5 ) ( 0 . 934375 ) ( 1) 0 . 9125 t
0%
3 3
Derivatives of Unequally Spaced Data
In the previous discussion, both finite difference approximations
and Richardson extrapolation reqires evenly distributed data. So,
these methods are more suitable to evaulate functions.
Emprically derived data, experimental or from field surveys, are
usually not even.
One way is to fit a second-order Lagrange interpolating polynomial
to each set of three data points. Derivative of the polynomial:

' 2x xi xi 1
2x xi 1
xi 1
2x xi 1
xi
f ( x) f ( xi 1 ) f ( xi ) f ( xi 1 )
( xi 1
x i )( x i 1
xi 1 ) ( xi x i 1 )( x i xi 1 ) ( xi 1
x i 1 )( x i 1
xi )

Using above formula, any point in the range of three data points
can be evaluated.
This equation has the same accuracy of high-accuracy centered
difference approximation even though data is not need to be
equally spaced.
Derivatives and Integrals for Data with Error:
Differentiation process amplifies the error:

y dy/dt

x x
As a remedy, fit a smoother function (low-order polynomial) to the
uncertain data.
On the other hand, integration process reduces the error.
(Succusive negative and positive errors cancel out during
integration). No further action is required.