MAT 202 E

NUMERICAL METHODS

Numerical Differentiation

“These notes are only to be used in class presentations”

Textbook: Numerical Methods for Engineers, S.C. Chapra, R.P. Canale,

7th edition, 2015
 Numerical Differentiation and Integration
• Calculus is the mathematics of change. Since engineers continuously deal with
systems and processes that change, calculus is an essential tool of engineering.
• Standing at the heart of calculus are the concepts of:
• Differentiation represents the rate of change of a dependent variable with respect
to a independent variable y=f(x).
• Integral of the function f(x) with respect to the independent variable x, evaluated
between the limits x=a to x=b.
Differentiation Integration
y f ( xi  x)  f ( xi )
 b
x x I   f ( x)dx
dy f ( xi  x)  f ( xi ) a
 lim
dx x0 x
Where do we use numerical techniques?
– A complicated continuous function that is difficult or
impossible to differentiate or integrate directly
– When we have a sampling of points representing a
function as with the experiments results
 Numerical Differentiation

• To evaluate numerical derivatives we’ll use Finite

Divided Difference Methods
– Forward
– Backward
– Centered
First derivative with forward divided difference
Consider a function f(xi+1) which can be expanded in a Taylor
series in the neighborhood of a point xi
h  xi 1  xi : step size

f ' ( xi ) f " ( xi ) 2 f ( n) ( xi ) n
f ( xi 1 )  f ( xi )  h h  h  Rn
1! 2! n!
f ( n 1) () ( n 1)
Rn  h  O(hn+1),
(n  1)!
If we truncate the series after the 1st derivative term

f ' ( xi )
f ( xi 1 )  f ( xi )  h  R1
1!
Solve for the 1st derivative

' f ( xi 1)  f ( xi )
f ( xi )   O ( h)
h
f ( xi 1)  f ( xi )  f ( xi ) 1st forward divided difference

' f ( xi )
f ( xi )   O
 ( h)
 h 1st order accurate
order of the
1st derivative truncation error
estimate
Forward Divided Difference method uses the value of points in
front of or at the point where the derivative is calculated.
First derivative with backward divided difference

Backward Divided Difference method uses the value of points

behind of or at the point where the derivative is calculated.
Consider a function f(xi-1) which can be expanded in a Taylor
series in the neighborhood of a point xi and

h  xi  xi 1 : step size

f ' ( xi ) f " ( xi ) 2 f (n) ( xi ) n

f ( xi 1)  f ( xi )  h h  h  Rn
1! 2! n!
Truncate the series after the 1st derivative term and solve for the 1st
derivative

' f ( xi )  f ( xi 1)
f ( xi )   O ( h)
h

f ( xi )  f ( xi 1)  f ( xi ) 1st backward divided difference

' f ( xi )
f ( xi )   O
 ( h)
 h 1st order accurate
order of the
1st derivative truncation error
estimate
 First derivative with center divided difference

xi 1  xi 1  2h

Centered Divided Difference uses the value of points both in

front and behind of the point where the derivative is
calculated.
Subtract backward difference Taylor series expansion from
forward difference Taylor series expansion

f ' ( xi ) f (3) ( xi ) 3
f ( xi 1 )  f ( xi 1 )  2 h2 h 
1! 3!
f ( n) ( xi ) n
2 h  Rn
n!

n is odd.
Truncate the series after the 1st derivative term and solve for the
1st derivative

'
f ( xi 1)  f ( xi 1)  2 f ( xi )h  R1 f (3) () 3
R1  h
(3)!

' f ( xi 1 )  f ( xi 1 )
f ( xi )   O(h 2 )
2h 
   order of the 2nd order accurate
1st derivative truncation error
estimate

Centered method is usually more accurate than forward

& backward methods
 Example: Determine the 1st derivative of
f ( x)  0.1x 4  0.15x 3  0.5x 2  0.25x  1.2

at x=0.5 using finite divided differences and a step size of h=0.25.

Forward divided difference

f (0.5)  1.1547
Backward divided difference
f (0.5)  0.7141
Centered divided difference
f (0.5)  0.9344
 Higher order derivatives
Second derivative with forward divided difference

Consider functions f(xi+1) and f(xi+2) which can be expanded in

Taylor series in the neighborhood of a point xi

f ' ( xi ) f " ( xi ) 2 f ( n) ( xi ) n
f ( xi 1 )  f ( xi )  h h  h  Rn
1! 2! n!
f ' ( xi ) f " ( xi ) 2 f ( n)
( xi )
f ( xi  2 )  f ( xi )  2h  ( 2h )    (2h n )  Rn
1! 2! n!
xi 1  xi  h
xi  2  xi  2h
Truncate the series after the 2nd derivative term

f ' ( xi ) f " ( xi ) 2
f ( xi 1 )  f ( xi )  h h  R2
1! 2!
f ' ( xi ) f " ( xi )
f ( xi  2 )  f ( xi )  2h  (2h) 2  R2
1! 2!
Multiply f(xi+1) by 2 and subtract that from f(xi+2)

f ( xi  2 )  2 f ( xi 1 )   f ( xi )  f ( xi )h 2  O (h3 )
 Solve for the 2nd derivative
'' f ( xi  2 )  2 f ( xi 1)  f ( xi )
f ( xi )   O
 ( h)
2
h
  order of the
2nd derivative truncation error
estimate

f ( xi  2 )  2 f ( xi 1)  f ( xi )  2 f ( xi ) 2nd forward divided

2 difference
 f ( xi )
f ( xi )   O ( h)
2
h 1st order accurate

( n) n f ( xi ) nth forward divided

f ( xi )   O ( h) difference
hn
 Second derivative with backward divided difference

Consider functions f(xi-1) and f(xi-2) which can be expanded in

Taylor series in the neighborhood of a point xi

f ' ( xi ) f " ( xi ) 2 f (n) ( xi ) n

f ( xi 1)  f ( xi )  h h  h  Rn
1! 2! n!
f ' ( xi ) f " ( xi ) 2 f ( n)
( xi )
f ( xi  2 )  f ( xi )  2h  ( 2h)    (2h n )  Rn
1! 2! n!
xi  xi 1  h
xi  xi  2  2h
Truncate the series after the 2nd derivative term

f ' ( xi ) f " ( xi ) 2
f ( xi 1 )  f ( xi )  h h  R2
1! 2!
f ' ( xi ) f " ( xi )
f ( xi  2 )  f ( xi )  2h  (2h) 2  R2
1! 2!
Multiply f(xi-1) by 2 and subtract that from f(xi-2)

f ( xi  2 )  2 f ( xi 1 )   f ( xi )  f ( xi )h 2  O(h3 )
 Solve for the 2nd derivative
'' f ( xi  2 )  2 f ( xi 1)  f ( xi )
f ( xi )   O
 ( h)
2
h
  order of the
2nd derivative truncation error
estimate

f ( xi  2 )  2 f ( xi 1)  f ( xi )   2 f ( xi ) 2nd backward divided

2 difference
 f ( xi )
f ( xi )   O ( h)
2 1st order accurate
h

( n)  n f ( xi ) nth backward divided

f ( xi )   O ( h) difference
hn
 Second derivative with center divided difference

f ' ( xi ) f " ( xi ) 2 f ( n) ( xi ) n
f ( xi 1 )  f ( xi )  h h  h  Rn
1! 2! n!
f ' ( xi ) f " ( xi ) 2 f (n) ( xi ) n
f ( xi 1)  f ( xi )  h h  h  Rn
1! 2! n!

Add these terms and truncate the series after the 2nd derivative term
f ( xi 1)  f ( xi 1)  2 f ( xi )  f " ( xi )h 2  O(h 4 )

" f ( xi 1 )  2 f ( xi )  f ( xi 1 )
f ( xi )   O(h 2 ) 2nd order accurate
h2
High Accuracy Differentiation Equations
• High-accuracy divided-difference formulas can be generated by
including additional terms from the Taylor series expansion.
1st derivative with forward divided difference

f ( xi ) 2
f ( xi 1 )  f ( xi )  f ( xi )h  h  R2
2
f ( xi 1 )  f ( xi ) f ( xi )

f ( xi )   h  O(h 2 )
h 2
f ( xi  2 )  2 f ( xi 1 )  f ( xi )
f ( xi ) 
h2
 f ( xi 1 )  f ( xi ) f ( xi  2 )  2 f ( xi 1 )  f ( xi )
f ( xi )   2
h  O ( h 2
)
h 2h

 f ( xi  2 )  4 f ( xi 1 )  3 f ( xi )
f ( xi )   O(h 2 )
2h

• Inclusion of the 2nd derivative term has improved the

accuracy to O(h2).
 1st derivative with backward divided difference

f ( xi ) 2
f ( xi 1 )  f ( xi )  f ( xi )h  h  R2
2
f ( xi )  f ( xi 1 ) f ( xi )

f ( xi )   h  O(h 2 )
h 2
f ( xi 2 )  2 f ( xi 1 )  f ( xi )
f ( xi ) 
h2

f ( xi 2 )  4 f ( xi 1 )  3 f ( xi )
f ( xi )   O( h 2 )
2h
 1st derivative with centered divided difference

 f ( xi  2 )  8 f ( xi 1 )  8 f ( xi 1 )  f ( xi 2 )
f ( xi )   O( h 4 )
12h

Derive the formula for the centered divided difference for the
1st derivative and show that it’s 4th order accurate.
Forward Divided Difference
Backward Divided Difference
Centered Divided Difference
 Example: Determine the 1st derivative of
f ( x)  0.1x 4  0.15x 3  0.5x 2  0.25x  1.2

at x=0.5 using finite divided differences and a step size of

h=0.25.
Use second order accurate forward and backward finite divided
differences.

Forward divided difference

f (0.5)  0.8594
Backward divided difference
f (0.5)  0.8781
 Example: Use the following data to find velocity and acceleration
at t=5s.

Time (s) 0 1 2 3 4 5 6 7 8
Position (m) 0 0.7 1.8 3.4 5.1 6.5 7.3 8.0 8.4

Use second order accurate forward finite divided difference

methods

v(5)  0.85m / s
2
a(5)  0.1m / s
Solution
 Richardson Extrapolation

• There are two ways to improve derivative estimates when

employing finite divided differences:
• Decrease the step size, or
• Use a higher-order formula that employs more points.

• A third approach, based on Richardson extrapolation, uses two

derivative estimates to arrive at a better approximation

34
• D is the true value but unknown and D(h1) is an
approximation based on the step size h1. Reducing the step size
to half, h2 =h1/2, we obtained another approximation D(h2).
• By properly combining the two approximations, D(h1) &
D(h2), the error is reduced to O(h4).

4 1
h2  h1 / 2  D  D(h2 )  D(h1 )
3 3
Example: Determine the 1st derivative of
f ( x)  0.1x 4  0.15x 3  0.5x 2  0.25x  1.2

at x=0.5 using Richardson extrapolation with h1=0.5 and

h2=0.25. Use 2nd order acccurate centered divided difference
method.
'
f (0.5)  0.9125
 Derivatives of Unequally Spaced Data

• For finite divided difference approximation the data have to

be evenly spaced.
• For the Richardson extrapolation the data have to be evenly
spaced and generated for succesively halved intervals.
• But data from experiments or field studies are often
collected at unequal intervals.

• So one way to handle unequally spaced data fit a 2nd order

Lagrange interpolating polynomial to each set of 3 adjacent
points. (xi-1,xi,xi+1)

38
 f ( x)  f ( xi 1 )
x  xi x  xi 1   f ( x ) x  xi 1 x  xi 1   f ( x ) x  xi 1 x  xi 
xi 1  xi xi 1  xi 1  i
xi  xi 1 xi  xi 1  i 1
xi 1  xi 1 xi 1  xi 

Take the derivative

2 x  xi  xi 1 2 x  xi 1  xi 1 2 x  xi 1  xi
f ( x)  f ( xi 1 )  f ( xi )  f ( xi 1 )
xi 1  xi xi 1  xi 1  xi  xi 1 xi  xi 1  xi 1  xi 1 xi 1  xi 

x: the value where you want to estimate the derivative

• These equations can be used to estimate the derivative anywhere

within the range prescribed by 3 points.
•Points don’t have to be equally spaced.
• Derivative estimate is of the same accuracy as the centered divided
difference
Example: Use the following data to find f ’(0.8)

x 0.6 0.8 0.9

f(x) 0.825 0.697 0.622

f (0.8)  0.715
Solution