NUMERICAL DIFFERENTIATION

We will seek a numerical approximations of derivatives.

1 First Derivative Formula

We know that the derivative of the function f at x0 is
f (x0 + h) − f (x0 )
f 0 (x0 ) = lim
h→0 h
This formula gives an obvious way to generate an approximation to f 0 (x0 ). A
simple approximation of the first derivative is
f (x0 + h) − f (x0 )
f 0 (x0 ) ≈
h
for small values of h.

1.1 Two-Point Formulas

Let’s compute the f 0 (x0 ) with an approximation error. Suppose that x0 ∈ (a, b),
where f ∈ C 2 [a, b], and that x1 = x0 + h for some h 6= 0 that is sufficiently small
to ensure that x1 ∈ [a, b]. We construct the first Lagrange polynomial P1 (x) for
f determined by x0 and x1 , with its error term:
(x − x0 )(x − x1 ) 00
f (x) = P1 (x) + f (ξ (x))
2!
(x − x0 )(x − x1 ) 00
= L1,0 (x)f (x0 ) + L1,1 (x)f (x1 ) + f (ξ (x))
2!
x − x1 x − x0 (x − x0 )(x − x1 ) 00
= f (x0 ) + f (x1 ) + f (ξ (x))
x0 − x1 x1 − x0 2!
x − (x0 + h) x − x0 (x − x0 )(x − x0 − h) 00
= f (x0 ) + f (x0 + h) + f (ξ (x))
x0 − (x0 + h) (x0 + h) − x0 2!
where ξ (x) between x0 and x1 . Differentiating f with respect to x gives
f (x0 + h) − f (x0 ) 2(x − x0 ) − h 00 (x − x0 )(x − x0 − h) d 00
f 0 (x) = + f (ξ (x))+ f (ξ (x))
h 2 2! dx
When x is x0 , the formula simplifies to
f (x0 + h) − f (x0 ) h 00
f 0 (x0 ) = − f (ξ (x))
h 2
Hence, for small values of h,
f (x0 + h) − f (x0 )
f 0 (x0 ) ≈
h

1
with an error bounded by M |h|/2, where M is a bound on |f 00 (x)| for x between
x0 and x0 + h. This formula is known as the forward-difference formula if
h > 0 and the backward-difference formula if h < 0.

NOTE (First-Derivative Formulas via Taylor Series)

We can use Taylor’s formula of f (x) at x0 to compute the approximation

of the first derivative. But this method can not be used if the number of the
evaluation points is more than two.
f 00 (ξ)
f (x) = f (x0 ) + f 0 (x0 )(x − x0 ) + (x − x0 )2 , ξ ∈ (x0 , x)
2!
When x = x0 + h
f 00 (ξ)
f (x0 + h) = f (x0 ) + f 0 (x0 )(x0 + h − x0 ) + (x0 + h − x0 )2 , ξ ∈ (x0 , x)
2!
f 00 (ξ)
= f (x0 ) + hf 0 (x0 ) + h2 , ξ ∈ (x0 , x)
2
Hence
f (x0 + h) − f (x0 ) h 00
f 0 (x0 ) = − f (ξ)
h 2

Example 1 Values for f (x) are given in table. Use forward-difference and
backward-difference formulas to approximate f 0 (0.5), f 0 (0.6) and f 0 (0.7).

x 0.5 0.6 0.7

f (x) 0.48 0.56 0.66

Solution We know that

f (x0 + h) − f (x0 )
f 0 (x0 ) ≈ , h = x1 − x0
h
For f 0 (0.5):
x0 = 0.5, x1 = 0.6 ⇒ h = 0.1 > 0
So we will compute f 0 (0.5) by applying the forward-difference formula

f (0.6) − f (0.5) 0.56 − 0.48

f 0 (0.5) ≈ = = 0.8
0.1 0.1
For f 0 (0.6):
x0 = 0.6, x1 = 0.5 ⇒ h = −0.1 < 0
So we will compute f 0 (0.6) by applying the backward-difference formula

f (0.5) − f (0.6) 0.48 − 0.56

f 0 (0.6) ≈ = = 0.8
−0.1 −0.1

2
or
x0 = 0.6, x1 = 0.7 ⇒ h = 0.1 > 0
So we will compute f 0 (0.6) by applying the forward-difference formula

f (0.7) − f (0.6) 0.66 − 0.56

f 0 (0.6) ≈ = =1
0.1 0.1

For f 0 (0.7):
x0 = 0.7, x1 = 0.6 ⇒ h = −0.1 < 0
So we will compute f 0 (0.7) by applying the backward-difference formula

f (0.6) − f (0.7) 0.66 − 0.56

f 0 (0.7) ≈ = =1
−0.1 0.1

Example 2 Use the forward-difference formula to approximate the deriva-

tive of f (x) = lnx at x0 = 1.8 using h = 0.01, and determine bounds for the
approximation error.
Solution The forward-difference formula gives

f (1.8 + h) − f (1.8)
f 0 (1.8) ≈
h
ln (1.81) − ln (1.8)
≈ = 0.5540
0.01
Because f 00 (x) = −1/x2 and 1.8 < ξ < 1.81, a bound for this approximation
error is
h
|R(1.8)| = − f 00 (ξ (x))
2
0.01 1
=
2 ξ2
1 1
≤ = 1.5.10−3
200 (1.8)2

NOTE
f (1.81) − f (1.8)
|R(1.8)| = f 0 (1.8) − ≤ 1.5.10−3
0.01
0.5540 − 1.5.10−3 ≤ f 0 (1.8) ≤ 0.5540 + 1.5.10−3
0.5525 ≤ f 0 (1.8) ≤ 0.5555

and
1
f 0 (1.8) = = 0.5555
1.8

3
1.2 Three-Point Formulas
Suppose that x0 ∈ (a, b), where f ∈ C 3 [a, b], and that x1 = x0 + h, x2 = x0 + 2h
for some h 6= 0 that is sufficiently small to ensure that x1 , x2 ∈ [a, b]. We
construct the second Lagrange polynomial P2 (x) for f determined by x0 , x1
and x2 , with its error term:

(x − x0 )(x − x1 )(x − x2 ) 000

f (x) = P2 (x) + f (ξ (x))
3!
(x − x0 )(x − x1 )(x − x2 ) 000
= L2,0 (x)f (x0 ) + L2,1 (x)f (x1 ) + L2,2 (x)f (x2 ) + f (ξ (x))
3!
(x − x1 ) (x − x2 ) (x − x0 ) (x − x2 )
= f (x0 ) + f (x1 )
(x0 − x1 ) (x0 − x2 ) (x1 − x0 ) (x1 − x2 )
(x − x0 ) (x − x1 ) (x − x0 )(x − x1 )(x − x2 ) 000
+ f (x2 ) + f (ξ (x))
(x2 − x0 ) (x2 − x1 ) 3!
(x − x0 − h) (x − x0 − 2h) (x − x0 ) (x − x0 − 2h)
= 2
f (x0 ) − f (x0 + h)
2h h2
(x − x0 ) (x − x0 − h) (x − x0 )(x − x0 − h)(x − x0 − 2h) 000
+ 2
f (x0 + 2h) + f (ξ (x))(1)
2h 3!
where ξ (x) between x0 and x0 + 2h.
Differentiating f with respect to x and then writing x = x0 give

h2
 
1 3 1
f 0 (x0 ) = − f (x0 ) + 2f (x0 + h) − f (x0 + 2h) + f 000 (ξ (x))
h 2 2 3

Hence, for small values of h,

 
0 1 3 1
f (x0 ) ≈ − f (x0 ) + 2f (x0 + h) − f (x0 + 2h)
h 2 2

with an error
h2 000
f (ξ (x))
3
This formula is known as the three-point endpoint formula.
Differentiating f with respect to x and then writing x = x0 + h give

h2
f 0 (x0 + h) = P20 (x0 + h) − f 000 (ξ (x))
6
h2
 
1 1 1
= − f (x0 ) + f (x0 + 2h) − f 000 (ξ (x))
h 2 2 6

where ξ (x) between x0 and x0 + 2h. Or

h2
 
1 1 1
f 0 (x0 ) = − f (x0 − h) + f (x0 + h) − f 000 (ξ (x))
h 2 2 6

4
where ξ (x) between x0 − h and x0 + h. Hence, for small values of h,
 
0 1 1 1
f (x0 ) ≈ − f (x0 − h) + f (x0 + h)
h 2 2
with an error
h2 000
−
f (ξ (x))
6
This formula is known as the three-point midpoint formula.

Example Values for f (x) are given in table. Use three-point formulas to
approximate f 0 (2).

x 1.9 2 2.1 2.2

f (x) 13.5 12.7 14.8 19.7

Solution According to the table

x0 = 2, x0 − h = 1.9, x0 + h = 2.1, x0 + 2h = 2.2

Using the endpoint formula

 
1 3 1
f 0 (2) ≈ − f (2) + 2f (2.1) − f (2.2)
0.1 2 2
≈ 7

Using the midpoint formula

 
0 1 1 1
f (2) ≈ − f (1.9) + f (2.1)
0.1 2 2
≈ 6.5

2 Second Derivative Midpoint Formula

Suppose that x0 ∈ (a, b), where f ∈ C 4 [a, b]. Taylor’s formula of f (x) at x0
gives

f 00 (x0 ) f 000 (x0 ) f (4) (ξ)

f (x) = f (x0 )+f 0 (x0 )(x−x0 )+ (x−x0 )2 + (x−x0 )3 + (x−x0 )4
2! 3! 4!
(2)
First, write x = x0 + h in (2)

f 00 (x0 ) 2 f 000 (x0 ) 3 f (4) (ξ1 ) 4

f (x0 + h) = f (x0 ) + f 0 (x0 )h + h + h + h , (3)
2! 3! 4!
Then, write x = x0 − h in (2)

f 00 (x0 ) 2 f 000 (x0 ) 3 f (4) (ξ2 ) 4

f (x0 − h) = f (x0 ) − f 0 (x0 )h + h − h + h (4)
2! 3! 4!

5
From equations (3) and (4), one gets

f (4) (ξ1 ) 4 f (4) (ξ2 ) 4

f (x0 + h) + f (x0 − h) = 2f (x0 ) + f 00 (x0 )h2 + h + h
4! 4!
Hence,

f (x0 + h) − 2f (x0 ) + f (x0 − h) h2 (4)

f 00 (x0 ) = − f (ξ) , ξ ∈ (x0 − h, x0 + h)
h2 12
where
1 h (4) i
f (4) (ξ) = f (ξ1 ) + f (4) (ξ2 )
2
Hence, for small values of h,

f (x0 + h) − 2f (x0 ) + f (x0 − h)

f 00 (x0 ) ≈
h2
This formula is known as the second derivative midpoint formula.

Example Values for f (x) = xex are given in table. Use the second
derivative midpoint formula to approximate f 00 (2) and determine bounds for
the approximation error.

x 1.8 1.9 2.0 2.1 2.2

f (x) 10.889365 12.703199 14.778112 17.148957 19.855030

Solution According to table

x0 = 2, x0 − h = 1.9, x0 + h = 2.1 ⇒ h = 0.1

so
f (2.1) − 2f (2) + f (1.9)
f 00 (2) ≈ 2 = 29.593200
(0.1)
or

x0 = 2, x0 − h = 1.8, x0 + h = 2.2 ⇒ h = 0.2

and

f (2.2) − 2f (2) + f (1.8)

f 00 (2) ≈ 2 = 29.704275
(0.2)
For the second case (h = 0.2), we will determine the bound for the approxima-
tion error:
2
(0.2) (4)
|R(2)| = − f (ξ)
12

6
where 1.8 < ξ < 2.2. Since f (4) (x) = (4 + x)ex
2
(0.2)
|R(2)| = − (4 + ξ)eξ
12
1
≤ (4 + 2.2)e2.2 ≈ 0.1865
12.25
NOTE: The exact value of f 00 (2) is

f 00 (2) = (2 + 2)e2 = 4e2 ≈ 29.556224