NUMERICAL DIFFERENTIATION

or
DIFFERENCE APPROXIMATION
LEARNING OUTCOMES
AT THE END OF THIS CHAPTER, STUDENTS SHOULD BE ABLE TO:

1. Approximate a derivative using Finite Difference Methods

2. Increase accuracy of the approximation suing high accuracy Finite

Difference Methods.

3. Analyze the accuracy of the approximation.

2
1 NUMERICAL DIIFERENTIATION
Used to evaluate derivatives of a function
using the functional values at grid points.
They are important in the numerical solution
of both ordinary and partial differential
equations.
 Methods of Approximation
 Forward Difference
 Backward Difference
 Central Difference
Example:
Graph the first derivative from equation
𝑓 𝑥 = 𝑒 𝑥 − 𝑥3 + 1
for 𝑥0 = 1

4
5
6
7
Mathematical formulas for those three graphs are as follows:
𝑓 𝑥0 +ℎ −𝑓 𝑥0
✗ Forward Difference 𝑓′(𝑥) ≈
ℎ
𝑓 𝑥0 −𝑓 𝑥0 −ℎ
✗ Backward Difference 𝑓′(𝑥) ≈ ℎ
𝑓 𝑥0 +ℎ −𝑓 𝑥0 −ℎ
✗ Central Difference 𝑓′(𝑥) ≈ 2ℎ
Question:
How accurately of these formulas are approximating the
derivative ?

8
Big concept
Numerical precision is the
very soul of science.
~D'Arcy Wentworth Thompson

9
TAYLOR EXPANSION METHOD
 Start with notation
𝑓𝑖 = 𝑓 𝑥𝑖 𝑓𝑖+1 = 𝑓 𝑥𝑖+1 𝑓𝑖′ = 𝑓 ′ 𝑥𝑖
where
𝑥𝑖 = 𝑥0 𝑥𝑖+1 = 𝑥0 + ℎ

 Thus, Taylor expansion of 𝑓𝑖+1 about 𝑥𝑖 is

ℎ 2 ℎ 3 ℎ 4
𝑓𝑖+1 = 𝑓𝑖 + ℎ𝑓𝑖′ + 𝑓𝑖′′ + 𝑓𝑖′′′ + 𝑓𝑖′′′′ + ⋯
2! 3! 4!

10
 TAYLOR EXPANSION METHOD
 Solving equation above for 𝑓𝑖 ′ yields
′ 𝑓𝑖+1 − 𝑓𝑖 ℎ ′′ ℎ2 ′′′ ℎ3 ′′′′
𝑓𝑖 = − 𝑓𝑖 − 𝑓𝑖 − 𝑓𝑖 − ⋯
ℎ 2! 3! 4!
Truncated after first term yield forward difference approximation
(FDA)
′ 𝑓𝑖+1 − 𝑓𝑖
𝑓𝑖 =
ℎ
The remainder terms constitute the truncation error. Thus, the FDA
is expressed, including the truncation error effect, as
𝑓𝑖+1 −𝑓𝑖 ℎ
𝑓𝑖′ = + 𝑂 ℎ where 𝑂 ℎ ≅ − 𝑓𝑖 ′′
ℎ 2
11
 TAYLOR EXPANSION METHOD
 The first derivative with backward difference approximation (BDA)
is approximated by using Taylor expansion yield
ℎ 2 ℎ 3 ℎ 4
𝑓𝑖−1 = 𝑓𝑖 − ℎ𝑓𝑖′ + 𝑓𝑖′′ − 𝑓𝑖′′′ + 𝑓𝑖′′′′ + ⋯
2! 3! 4!
 Hence, the BDA is expressed, including the truncation error effect,
as
𝑓𝑖 −𝑓𝑖−1 ℎ
𝑓𝑖′ = + 𝑂(ℎ) where 𝑂 ℎ ≅ 𝑓𝑖 ′′
ℎ 2

12
 TAYLOR EXPANSION METHOD
 The Central difference approximation derived by subtracting the
Taylor expansion of 𝑓𝑖+1 and 𝑓𝑖−1 .
 Hence, we have
𝑓𝑖+1 −𝑓𝑖−1 ℎ2
𝑓𝑖′ = + 𝑂(ℎ2 ) where 𝑂 ℎ2 ≅ − 𝑓𝑖 ′′′
2ℎ 6

Conclusion:
The truncation error of FDA and BDA is proportional to ℎ and the
truncation error of CDA is proportional to ℎ2 . Hence, when ℎ is
decreased, the error of CDA decreases more rapidly than in the other.
13
 EXAMPLE Solutions:

A function table is given as ℎ = 𝑥𝑖+1 − 𝑥𝑖 = 0.2 − 0 = 0.2

follows:
FDA
𝑥 𝒇(𝒙) 𝑓 0.2 − 𝑓 0 4.441 − 4.020
𝑓′ 0 = = = 2.105
−0.2 3.665 0.2 0.2

0 4.020 BDA
𝑓 0 − 𝑓 −0.2 4.020 − 3.665
𝑓′ 0 = = = 1.775
0.2 4.441 0.2 0.2

Derive the forward, backward CDA

𝑓 0.2 − 𝑓 −0.2 4.441 − 3.665
and centered approximations 𝑓′ 0 = =
2(0.2) 0.2
to calculate 𝑓′(0) with the data = 1.940
given !
14
 example
Estimate the first derivative of 𝑔(𝑥), at 𝑥 = 1.5 and step size ℎ = 0.5 by using forward backward and
centered difference approximation with accuracy 𝑂(ℎ), given the data in Table below:

x 1.0 1.5 2.0

𝑔(𝑥) 2.7 10.1 29.5

15
example

18
✗ HIGHER ACCURACY FINITE DIFFERENCE
METHODS

19
As obtained above, a difference approximation for needs at least p+1 data points. If
more data points are used, a more accurate difference approximation may be derived.

Three-point forward difference approximation

′ −𝑓𝑖+2 + 4𝑓𝑖+1 − 3𝑓𝑖
𝑓𝑖 = + 𝑂(ℎ2 )
2ℎ
Three-point backward difference approximation
3𝑓𝑖 − 4𝑓𝑖−1 + 𝑓𝑖−2
𝑓𝑖′ = + 𝑂(ℎ2 )
2ℎ
Five-point centered difference approximation
−𝑓𝑖+2 + 8𝑓𝑖+1 − 8𝑓𝑖−1 + 𝑓𝑖−2
𝑓𝑖′ = + 𝑂(ℎ4 )
12ℎ

20
 example
Use high-accuracy
differentiation
formulas to
estimate the
derivative of
𝜋
cos(𝑥) at 𝑥 = 3 for
ℎ = 0.1.

True value:
𝑓 ′ 𝑥 = − sin 𝑥
𝜋
𝑓′ = −0.866025
3

21
✗ SECOND DERIVATIVE

22
 Forward difference approximation
′′ 𝑓 𝑥 − 2𝑓 𝑥 + ℎ + 𝑓(𝑥 + 2ℎ)
𝑓𝑖 = + 𝑂(ℎ)
SECOND DERIVATIVE ℎ2
Backward difference approximation
To derive the difference 𝑓 𝑥 − 2ℎ − 2𝑓 𝑥 − ℎ + 𝑓(𝑥)
𝑓𝑖′′ = + 𝑂(ℎ)
approximation for the 𝑛−𝑡ℎ ℎ2
derivative, we must to Centered difference approximation
eliminate the first until 𝑓 𝑥 + ℎ − 2𝑓 𝑥 + 𝑓(𝑥 − ℎ)
(𝑛−1)−𝑡ℎ derivative from the 𝑓𝑖′′ = 2 + 𝑂(ℎ2 )
ℎ
Taylor expansions.

23
Higher accuracy formulas for second derivatives
Forward difference approximation
−𝑓 𝑥 + 3ℎ + 4𝑓 𝑥 + 2ℎ − 5𝑓 𝑥 + ℎ + 2𝑓(𝑥)
𝑓𝑖′′ = 2
+ 𝑂(ℎ2 )
ℎ

Backward difference approximation

2𝑓 𝑥 − 5𝑓 𝑥 − ℎ + 4𝑓 𝑥 − 2ℎ − 𝑓(𝑥 − 3ℎ)
𝑓𝑖′′ = 2
+ 𝑂(ℎ2 )
ℎ

Centered difference approximation

−𝑓 𝑥 + 2ℎ + 16𝑓 𝑥 + ℎ − 30𝑓 𝑥 + 16𝑓 𝑥 − ℎ − 𝑓(𝑥 − 2ℎ)
𝑓𝑖′′ = 2
+ 𝑂(ℎ4 )
12ℎ

24
 example
Estimate the second derivative of 𝑔(𝑥), at 𝑥 = 1.5 and step size ℎ = 0.5 by using forward backward and
centered difference approximation with accuracy 𝑂(ℎ), given the data in Table below:

x 1.0 1.5 2.0

𝑔(𝑥) 2.7 10.1 29.5

25
example

26
 summary Numerical Differentiation

Standard Higher Accuracy

Second Derivative Second Derivative

FDA −𝑓𝑖+3 + 4𝑓𝑖+2 − 5𝑓𝑖+1 + 2𝑓𝑖
FDA 𝑓𝑖 − 2𝑓𝑖+1 + 𝑓𝑖+2 𝑂(ℎ2 ) 𝑓𝑖′′ =
𝑂(ℎ) 𝑓𝑖′′ = ℎ2
ℎ2
BDA 2𝑓𝑖 − 5𝑓𝑖−1 + 4𝑓𝑖−2 − 𝑓𝑖−3
BDA 𝑓𝑖−2 − 2𝑓𝑖−1 + 𝑓𝑖 𝑂(ℎ2 ) 𝑓𝑖′′ =
𝑂(ℎ) 𝑓𝑖′′ = ℎ2
ℎ2
CDA 𝑓𝑖+1 − 2𝑓𝑖 + 𝑓𝑖−1 First Derivative CDA
𝑂(ℎ4 )
𝑓𝑖′′
𝑂(ℎ2 ) 𝑓𝑖′′ = −𝑓𝑖+2 + 16𝑓𝑖+1 − 30𝑓𝑖 + 16𝑓𝑖−1 − 𝑓𝑖−2
ℎ2 FDA −𝑓𝑖+2 + 4𝑓𝑖+1 − 3𝑓𝑖 =
First Derivative 𝑂(ℎ2 ) 𝑓𝑖′ = 12ℎ2
2ℎ
FDA 𝑓𝑖+1 − 𝑓𝑖
𝑂(ℎ) 𝑓𝑖′ = BDA
ℎ 3𝑓𝑖 − 4𝑓𝑖−1 + 𝑓𝑖−2
𝑂(ℎ2 ) 𝑓𝑖′ =
BDA 𝑓𝑖 − 𝑓𝑖−1 2ℎ
𝑂(ℎ) 𝑓𝑖′ =
ℎ CDA 𝑓𝑖′
CDA 𝑓𝑖+1 − 𝑓𝑖−1 𝑂(ℎ4 ) −𝑓𝑖+2 + 8𝑓𝑖+1 − 8𝑓𝑖−1 + 𝑓𝑖−2
𝑂(ℎ2 ) 𝑓𝑖′ = =
2ℎ 12ℎ

27
 Tutorial chapter 6
Answer tutorial question 7, 8, 9 and 10 and submit in the
Ulearn no later than 30 December 2020 5pm.

28
Thanks!
Any questions?

29