ICS 226: Numerical Analysis

Oliver Mhlanga

HARARE INSTITUTE OF TECHNOLOGY

[email protected] or [email protected]; 0712531415

April 9, 2018

Numerical Differentiation
Numerical Differentiation
If f (x) can be computed with only n digits of precision, is it also
possible to calculate f 0 (x) numerically with n digits of precision?
This challenge can be traced to the subtraction between quantities
that are nearly equal. In this section, several alternatives are
offered for the numerical computation of f 0 (x) and f 00 (x).
Recall that
f (x + h) − f (x)
f 0 (x) = lim
h→0 (1)
h
Taylor theorem:
∞ n
X 1 (k) X 1 (k)
f (x + h) = f (x)hk = f (x)hk + En+1 ,
k! k!
k=0 k=0
(2)
∞
X 1 (k) 1
En+1 = f (x)hk = f (n+1) ()hn+1 + O(hn+1 )
k! (n + 1)!
k=n+1
Numerical Differentiation

E = O(hn ) ⇒ ∃ real, finite, C : |E | ≤ C |h|n . E is of order hn

implying that E is approaching zero at a rate similar to hn .

f (x0 + h) − f (x0 ) h 00
f 0 (x0 ) = − f ().
h 2
For small values of h,
f (x0 + h) − f (x0 )
f 0 (x0 ) ≈
h
M|h|
with an error bounded by 2 , where M is a bound on |f 00 (x)| for
x ∈ (x0 , x0 + h).
Numerical Differentiation

Forward Difference (FD):

f (x + h) = f (x) + hf 0 (x) + O(h2 )

⇒ hf 0 (x) = f (x + h) − f (x) + O(h2 )
f (x + h) − f (x)
⇒ f 0 (x) = + O(h)
h
Backward Difference (BD):

f (x − h) = f (x) − hf 0 (x) + O(h2 )

⇒ hf 0 (x) = f (x) − f (x − h) + O(h2 )
f (x) − f (x − h)
⇒ f 0 (x) = + O(h)
h
Numerical Differentiation

Central Difference (CD):

h2 00 h3 000 h4 iv
f (x + h) = f (x) + hf 0 (x) + f (x) + f (x) + f (x) + ...
2! 3! 4!
h2 h3 000 h4 iv
f (x − h) = f (x) − hf 0 (x) + f 00 (x) − f (x) + f (x) + ...
2! 3! 4!
h3
⇒ f (x + h) − f (x − h) = 2hf 0 (x) + 2 f 000 (x) + ...
3!
0 f (x + h) − f (x − h)
⇒ f (x) = + O(h2 )
2h
FD, BD and CD formulas each involves 2 function calls, 1
subtraction , and 1 division: same computation time.
FD and BD formulas are comparable in accuracy.
CD formula is the most accurate and most recommended method,
however, sometimes CD can not be applied.
Numerical Differentiation

Higher Order Difference Formulas

h2 00 h3 000 h4
f (x + h) = f (x) + hf 0 (x) + f (x) + f (x) + f iv (x) + ...
2! 3! 4!
h2 h3 000 h4 iv
f (x − h) = f (x) − hf 0 (x) + f 00 (x) − f (x) + f (x) + ...
2! 3! 4!
h2 00 h4 iv
⇒ f (x + h) + f (x − h) = 2f (x) + 2 f (x) + 2 f (x) + ...
2! 4!
f (x + h) − 2f (x) + f (x − h)
⇒ f 00 (x) = + O(h2 )
h2 | {z }
f iv ()h2
Error =− 12
Numerical Differentiation

Other Higher Order Formulas

f (x + h) − 2f (x) + f (x − h)
f 00 (x) =
h2
f (x + 2h) − 2f (x + h) + 2f (x − h) − f (x − 2h)
f 000 (x) =
2h3
f (x + 2h) − 4f (x + h) + 6f (x) − 4f (x − h) + f (x − 2h)
f iv (x) =
h4
00 000
Other formulas for f (x), f (x), ... are also possible.
Numerical Differentiation
Polynomial Interpolation.
A general approach for Numerical Differentiation is to use
polynomial interpolation. Given a function f (x), find a polynomial
p(x) : p 0 ≈ f 0 .
Suppose that n + 1 points, x0 , x1 , ..., xn and values
f (x0 ), f (x1 ), ...f (xn ) have been evaluated, we apply the Lagrange
polynomial interpolation formula to derive

n
X
p(x) = f (xi )Li (x) (3)
i=0

where
n
Y x − xj
Li (x) =
xi − xj
j=0,j6=i
Numerical Differentiation
Since f (x) can be written as
n
X 1
f (x) = f (xi )Li (x) + f (n+1) (x )w (x) (4)
(n + 1)!
i=0
Qn
where w (x) = j=0 (x − xj ).
Taking derivatives , we obtain
f 0 (x) =
P n 0 1 (n+1) ( )w 0 (x) + 1 d (n+1)
i=0 f (xi )Li (x) + (n+1)! f x (n+1)! w (x) dx f (x )
0
P n Q n
w (x) = j=0 i=0,i6=j (x − xi ).
Hence a reasonable approximation for the first derivative of f is

n
X
f 0 (x) ≈ f (xi )L0i (x) = p 0 (x) (5)
i=0
Numerical Differentiation
Richardson Extrapolation Method.
Use less accurate formulas to generate a high accuracy formula.
From Taylor’s Theorem: (CD approach)

h2 00 h3 000 h4 iv
f (x + h) = f (x) + hf 0 (x) + f (x) + f (x) + f (x) + ...
2! 3! 4!
h2 h3 000 h4 iv
f (x − h) = f (x) − hf 0 (x) + f 00 (x) − f (x) + f (x) + ...
2! 3! 4!
h3
⇒ f (x + h) − f (x − h) = 2hf 0 (x) + 2 f 000 (x) + ...
3!
f (x + h) − f (x − h) 2 2
⇒ f 0 (x) = + − f 000 (x) h2 + − f v (x) h4 − ...
2h | 3!{z } | 5!{z }
a2 a4

f (x + h) − f (x − h)
⇒ f 0 (x) = + a2 h2 + a4 h4 + ... (6)
2h
Numerical Differentiation

Let A = f 0 (x), A(h) = f (x+h)−f

2h
(x−h)
, where h is the step size,
Thus, A is the derivative of a function, A(h) is the approximation
of the derivative when we use a step size of h, and set k = 2.
Equation (6) may be written

A = A(h) + ϕhk + O(hk+2 ) (7)

O(hk+1 ) is conventionally used to stand for ” sum of terms of

order hk+1 and higher”.
We can therefore get two different equations to identify both A
and ϕ by just using two different step sizes. Then doing this ,
using step sizes h and h/2, for any h
Numerical Differentiation

A = A(h/2) + ϕ(h/2)k + O(hk+2 ) (8)

taking 2k times (8) and subtracting equation (7) yields

2k A(h/2) − A(h)
A= + O(hk+2 ) (9)
2k − 1
The generation of a new improved approximation for A from two
A(h)’s with different values of h is called Richardson
Extrapolation. The process can be repeated again and again, to
get a new better approximation.
Numerical Differentiation
Numerical Differentiation
Numerical Differentiation

Example 4.1: f (x) = x 3 , f 0 (x) = 3x 2 , compute f 0 (1) using FD,

BD, CD, and CD with Richardson extrapolation.
Solution:
FD: h = 0.1, f 0 (1) = f (1.1)−f
0.1
(1.0)
= 3.31
f (1.0)−f (0.9)
BD: h = 0.1, f 0 (1) = 0.1 = 2.71
0 f (1.1)−f (0.9)
CD: h = 0.1, f (1) = 0.2 = 3.01
CD with Richardson extrapolation
CD : h2 = 0.05, f 0 (1) = f (1.05)−f 0.1
(0.95)
= 3.0025 = A(h/2), using
0
h = 0.1, f (1) = 3.01 = A(h),
2
A = 2 (3.0025−3.01)
22 −1
= 3.0000 = exact value!
Numerical Differentiation

Example 4.2: The distance x of a runner from a fixed point is

measured (in metres) at intervals of half a second. The data
obtained is
t 0.0 0.5 1.0 1.5 2.0
x 0.00 3.65 6.80 9.90 12.15
1. Use central differences to approximate the runners velocity at
times t = 0.5s and t = 1.25s
2. Use a central difference to approximate the runners
acceleration at time t = 1.5s H/W.
Numerical Differentiation

Solution (4.2.1):
Our aim here is to approximate x 0 (t). The choice of h is dictated
by the available data. Using data with t = 0.5s at its centre we
obtain

x(1.0) − x(0.0)
x 0 (0.5) ≈ = 6.80m/s (10)
2 × 0.5
Data centred at t = 1.25s gives us the approximation

x(1.5) − x(1.0)
x 0 (1.25) ≈ = 6.20m/s (11)
2 × 0.25
Note the value of h used.