Lesson 3.

2:
Numeric Differentiation
• Explain the principle behind numeric differentiation
• Perform definite differentiation using numeric methods
Meaning of derivative
Calculus definition:
Line defining 𝐵
𝑓 𝑥 + ∆𝑥 − 𝑓(𝑥) 𝑓(𝑥 + ∆𝑥)
′
𝑓 𝑥 = lim Slope at A
∆𝑥→0 ∆𝑥
or 𝐴
𝑓(𝑥)
“𝑓 ′ 𝑥 is the slope of the tangent line at 𝑥.”

Numeric approach: ∆𝑥
𝑥 𝑥 + ∆𝑥
′
𝑓 𝑥 + ∆𝑥 − 𝑓(𝑥)
𝑓 𝑥 ≅ Line defining
∆𝑥
Slope A to B
where ∆𝑥 ≅ 0
This is called finite-divided-difference method.
 Example
Determine
1
𝑑 1 𝑓 𝑥 =
1 − 𝑥2
𝑑𝑥 1− 𝑥2 𝑑𝑦
𝑑𝑥
at 𝑥 = 0.5

Analytic Solution:

𝑥
𝑓′ 𝑥 = 3
1− 2
𝑥 2
𝒇′ 𝟎. 𝟓 = 𝟎. 𝟕𝟔𝟗𝟖𝟎𝟎𝟑𝟓𝟖𝟗
Example
1 𝑑𝑦
𝑓 𝑥 =
Numeric Solution: 1 − 𝑥2 𝑑𝑥

𝑓 𝑥 + ∆𝑥 − 𝑓(𝑥) 𝐵 𝑓(0.500001)
′
𝑓 𝑥 ≅
∆𝑥
𝐴
Say we use ∆𝑥 = 1 × 10−6 : 𝑓(0.5)

𝑓 0.5 + 0.000001 − 𝑓(0.5)

𝑓′ 0.5 ≅
0.000001 0.000001
0.5 0.500001
1.154700538 … − 1.154701308 …
𝑓′ 0.5 ≅
0.000001
This is called forward finite-
𝒇′ 𝟎. 𝟓 ≅ 𝟎. 𝟕𝟔𝟗𝟖𝟎𝟏𝟗𝟏 divided-difference method.
Analytic Solution:
𝒇′ 𝟎. 𝟓 = 𝟎. 𝟕𝟔𝟗𝟖𝟎𝟎𝟑𝟓𝟖𝟗
Backward finite-divided-difference:
1
𝑓 𝑥 =
′
𝑓 0.5 − 𝑓(0.5 − 0.000001) 1 − 𝑥2
𝑓 0.5 ≅
0.000001 𝑑𝑦
𝑑𝑥
1.154700538 … − 1.154699769 …
′
𝑓 0.5 ≅ 𝐴
0.000001 𝑓(0.5)
𝐵
𝒇′ 𝟎. 𝟓 ≅ 𝟎. 𝟕𝟔𝟗𝟕𝟗𝟖𝟖𝟏 𝑓(0.499999)

Analytic Solution: 0.000001

𝒇′ 𝟎. 𝟓 = 𝟎. 𝟕𝟔𝟗𝟖𝟎𝟎𝟑𝟓𝟖𝟗 0.499999 0.5
Centered finite-divided difference
Forward: 𝑓 𝑥
𝑑𝑦
𝑑𝑥
′
𝑓 𝑥 + ∆𝑥 − 𝑓(𝑥) 𝐵
𝑓 𝑥 ≅ 𝑓(𝑥 + ∆𝑥)
∆𝑥

Backward: 𝑓(𝑥)
𝐴
𝑓(𝑥 − ∆𝑥)
𝑓 𝑥 − 𝑓(𝑥 − ∆𝑥)
𝑓′ 𝑥 ≅ ∆𝑥 ∆𝑥
∆𝑥
𝑥 − ∆𝑥 𝑥 + ∆𝑥
Centered: 𝑥

𝑓 𝑥 + ∆𝑥 − 𝑓(𝑥 − ∆𝑥)
𝑓′ 𝑥 ≅ 2∆𝑥
2∆𝑥
Example
1
Centered: 𝑓 𝑥 =
1 − 𝑥2 𝑑𝑦
𝑓 𝑥 + ∆𝑥 − 𝑓(𝑥 − ∆𝑥) 𝑑𝑥
′
𝑓 𝑥 ≅ 𝐵
2∆𝑥 𝑓(0.5 + 0.000001)

𝑓 0.5 + 0.000001 − 𝑓(0.5 − 0.000001) 𝑓(0.5)

𝑓′ 0.5 ≅ 𝐴
2(0.000001) 𝑓(0.5 − 0.000001)

′
1.154701308 … − 1.154699769 …
𝑓 0.5 ≅
0.000002 0.5 − 0.000001 0.5 + 0.000001
𝒇′ 𝟎. 𝟓 ≅ 𝟎. 𝟕𝟔𝟗𝟖𝟎𝟎𝟑𝟔 0.5

Analytic Solution: 0.000002

𝒇′ 𝟎. 𝟓 = 𝟎. 𝟕𝟔𝟗𝟖𝟎𝟎𝟑𝟓𝟖𝟗
Comparison

1
𝑓 𝑥 =
1 − 𝑥2

Analytic: 𝒇′ 𝟎. 𝟓 = 𝟎. 𝟕𝟔𝟗𝟖𝟎𝟎𝟑𝟓𝟖𝟗

Numeric: (Using ∆𝑥 = 1 × 10−6 )

Forward: 𝑓 ′ 0.5 ≅ 0.76980191 𝑅𝐸 = 0.0002015%

Backward: 𝑓 ′ 0.5 ≅ 0.76979881 𝑅𝐸 = 0.0002012%

More accurate
Centered: 𝑓′ 0.5 ≅ 0.76980036 𝑅𝐸 = 0.00000014036% in most cases.
Seatwork
Determine the analytic and numeric
𝑑𝑦
solution (forward, backward, and
centered FDM) of: 𝑑𝑥
𝑑 𝑥
(𝑒 cos(0.25𝑥))𝑥=0.2
𝑑𝑥

Forward:
𝑓 𝑥 + ∆𝑥 − 𝑓(𝑥)
𝑓′ 𝑥 ≅
∆𝑥
Backward:
′
𝑓 𝑥 − 𝑓(𝑥 − ∆𝑥)
𝑓 𝑥 ≅
∆𝑥
Centered:
𝑓 𝑥 + ∆𝑥 − 𝑓(𝑥 − ∆𝑥)
′
𝑓 𝑥 ≅ Analytic solution:
2∆𝑥
𝒇′ 𝟎. 𝟐 = 𝟏. 𝟐𝟎𝟒𝟔𝟏𝟓𝟏𝟒𝟗 …
Second derivative 𝐵𝐶
𝑓 𝑥
“the rate of change of the first derivative” 𝐴𝐵

Forward FDM: 𝐵
𝑓(𝑥 + 2∆𝑥)
𝐴
𝑓 𝑥 + ∆𝑥 − 𝑓(𝑥) 𝑓(𝑥 + ∆𝑥)
𝑓′ 𝑥 ≅ 𝑋
∆𝑥 𝑓(𝑥)

′
𝑓′ 𝐴 − 𝑓′(𝑋) ∆𝑥 ∆𝑥
𝑓′ 𝑥 ≅
∆𝑥
𝑥
𝑥 + ∆𝑥
𝑓 𝐴 − 𝑓(𝐵) 𝑓 𝐵 − 𝑓(𝑋)
−
𝑓 ′′ 𝑥 ≅ ∆𝑥 ∆𝑥 𝑥 + 2∆𝑥
∆𝑥

𝑓 𝑥 − 2𝑓 𝑥 + ∆𝑥 + 𝑓(𝑥 + 2∆𝑥)
𝑓 ′′ 𝑥 ≅
∆𝑥 2
Example 𝐵𝐶
Approximate 𝑓 𝑥
𝐴𝐵

𝑑2 1 𝐵
𝑓(𝑥 + 2∆𝑥)
𝑑𝑥 2 1− 𝑥2 𝑥=0.5
𝐴
𝑓(𝑥 + ∆𝑥)
𝑋
using Forward FDM: 𝑓(𝑥)
𝑓(𝑥 − 2∆𝑥)

′
𝑓 𝑥 − 2𝑓 𝑥 + ∆𝑥 + 𝑓(𝑥 + 2∆𝑥) ∆𝑥 ∆𝑥
𝑓′ 𝑥 ≅
∆𝑥 2
𝑥
𝑥 + ∆𝑥
Analytic Solution: 𝑥 + 2∆𝑥
2
1 + 2𝑥
𝑓 ′′ 𝑥 = 5
1 − 𝑥2 2
𝒇′′ 𝟎. 𝟓 = 𝟑. 𝟎𝟕𝟗𝟐𝟎𝟏𝟒𝟑𝟔 …
 1
𝐵𝐶
Example 𝑓 𝑥 =
1−𝑥 2
𝐴𝐵

Forward FDM: (say use ∆𝑥 = 1 × 10−6 ) 𝐵

𝑓(𝑥 + 2∆𝑥)
𝑓 𝑥 − 2𝑓 𝑥 + ∆𝑥 + 𝑓(𝑥 + 2∆𝑥) 𝐴
𝑓 ′′ 𝑥 ≅ 𝑓(𝑥 + ∆𝑥)
∆𝑥 2 𝑋
𝑓(𝑥)
𝑓(𝑥 − 2∆𝑥)
𝑓 0.5 = 1.154700538 …
∆𝑥 ∆𝑥
𝑓 0.5 + ∆𝑥 = 1.154701308 …
𝑥 = 0.5
𝑥 + ∆𝑥
𝑓 0.5 + 2∆𝑥 = 1.154702078 …
𝑥 + 2∆𝑥
1.154700538 … − 2(1.154701308 … ) + 1.154702078 …
𝑓 ′′ 0.5 ≅
∆𝑥 2

𝒇′ ′ 𝟎. 𝟓 ≅ 𝟑. 𝟎𝟔 Analytic Solution:
𝒇′′ 𝟎. 𝟓 = 𝟑. 𝟎𝟕𝟗𝟐𝟎𝟏𝟒𝟑𝟔 …
Second derivative 𝑓 𝑥
Centered FDM: 𝐷
𝑓(𝑥 + 2∆𝑥)
′
′
𝑓 (𝐶) − 𝑓′(𝐵) 𝐶
𝑓′ 𝑥 ≅
2∆𝑥 𝑋
𝐵 𝑓(𝑥)
where: 𝐴
𝑓(𝑥 − 2∆𝑥)
∆𝑥 ∆𝑥 ∆𝑥 ∆𝑥
𝑓 𝐷 − 𝑓(𝑋) 𝑓 𝑥 + 2∆𝑥 − 𝑓(𝑥)
𝑓′ 𝐶 = = 𝑥 − 2∆𝑥
2∆𝑥 2∆𝑥 𝐴𝑋 𝑥 + 2∆𝑥

𝑓 𝑋 − 𝑓(𝐴) 𝑓 𝑥 − 𝑓(𝑥 − 2∆𝑥) 𝑥 − ∆𝑥 𝑥 + ∆𝑥

𝑓′ 𝐵 = = 𝑋𝐷
2∆𝑥 2∆𝑥 𝑥

′
𝑓 𝑥 + 2∆𝑥 − 2𝑓 𝑥 + 𝑓(𝑥 − 2∆𝑥)
𝑓′ 𝑥 ≅
4∆𝑥 2
Example
Determine 𝑓 𝑥
𝑑2 1 𝐷
𝑓(𝑥 + 2∆𝑥)
𝑑𝑥 2 1 − 𝑥2 𝑥=0.5 𝐶
Using Centered FDM: 𝑋
𝐵 𝑓(𝑥)
𝐴
𝑓 𝑥 + 2∆𝑥 − 2𝑓 𝑥 + 𝑓(𝑥 − 2∆𝑥) 𝑓(𝑥 − 2∆𝑥)
𝑓 ′′ 𝑥 ≅ ∆𝑥 ∆𝑥 ∆𝑥 ∆𝑥
4∆𝑥 2
𝐴𝑋 𝑥 − 2∆𝑥 𝑥 + 2∆𝑥

𝑥 − ∆𝑥 𝑥 + ∆𝑥
Analytic Solution: 𝑋𝐷
2 𝑥
1 + 2𝑥
𝑓 ′′ 𝑥 = 5
1 − 𝑥2 2
𝒇′′ 𝟎. 𝟓 = 𝟑. 𝟎𝟕𝟗𝟐𝟎𝟏𝟒𝟑𝟔 …
Example
Centered FDM: (say use ∆𝑥 = 1 × 10−6 )

′
𝑓 𝑥 + 2∆𝑥 − 2𝑓 𝑥 + 𝑓(𝑥 − 2∆𝑥)
𝑓′ 𝑥 ≅
4∆𝑥 2

𝑓 0.5 − 2∆𝑥 = 1.154698999 …

𝑓 0.5 = 1.154700538 …

𝑓 0.5 + 2∆𝑥 = 1.154702078 …

1.154702078 … − 2 1.154700538 … + 1.154698999 …

𝑓 ′′ 0.5 ≅
4∆𝑥 2

𝒇′ ′ 𝟎. 𝟓 ≅ 𝟑. 𝟎𝟖𝟐𝟓 Analytic Solution:

𝒇′′(𝟎. 𝟓) = 𝟑. 𝟎𝟕𝟗𝟐𝟎𝟏𝟒𝟑𝟔
Comparison
1
𝑓 𝑥 =
1−𝑥 2

Analytic: 𝒇′′ 𝟎. 𝟓 = 𝟑. 𝟎𝟕𝟗𝟐𝟎𝟏𝟒𝟑𝟔 …

Numeric: (Using ∆𝑥 = 1 × 10−6 )

Forward: 𝑓 ′′ 0.5 ≅ 3.06 𝑅𝐸 = 0.623585%

Centered: 𝑓 ′′ 0.5 ≅ 3.0825 𝑅𝐸 = 0.107124% More accurate in most cases.

 Seatwork (2nd derivative)
Determine the analytic and numeric 𝑑𝑦
solution (forward and centered FDM)
of: 𝑑𝑥

𝑑2 𝑥
(𝑒 cos(0.25𝑥))𝑥=0.2
𝑑𝑥 2

Forward:

𝑓 𝑥 − 2𝑓 𝑥 + ∆𝑥 + 𝑓(𝑥 + 2∆𝑥)
𝑓 ′′ 𝑥 ≅
∆𝑥 2
Centered:

𝑓 𝑥 + 2∆𝑥 − 2𝑓 𝑥 + 𝑓(𝑥 − 2∆𝑥)

𝑓 ′′ 𝑥 ≅
4∆𝑥 2
Higher Accuracy Numeric Differentiation Formulas
First Derivative:
2-point
𝑓 𝑥 + ∆𝑥 − 𝑓(𝑥 − ∆𝑥) Centered FDM
𝑓′ 𝑥 ≅
2∆𝑥
4-point
−𝑓 𝑥 + 2∆𝑥 + 8𝑓 𝑥 + ∆𝑥 − 8𝑓 𝑥 − ∆𝑥 + 𝑓(𝑥 − 2∆𝑥) Centered FDM
𝑓′ 𝑥 ≅
12∆𝑥

Second Derivative:
3-point
𝑓 𝑥 + 2∆𝑥 − 2𝑓 𝑥 + 𝑓(𝑥 − 2∆𝑥)
′
𝑓′ 𝑥 ≅ Centered FDM
4∆𝑥 2

−𝑓 𝑥 + 2∆𝑥 + 16𝑓 𝑥 + ∆𝑥 − 30𝑓 𝑥 + 16𝑓 𝑥 − ∆𝑥 − 𝑓(𝑥 − 2∆𝑥) 5-point

𝑓 ′′ 𝑥 ≅ Centered FDM
12∆𝑥 2

For derivations, see p. 653-655 of Chapra (Numerical Methods for Engineers – 6th Edition)
Example

1
𝑓 𝑥 =
1 − 𝑥2
1
𝑓 𝑥 =
Approximate: 1 − 𝑥2
𝑑𝑦
𝑑𝑥
a) 𝑓′(0.5) using 4-point centered FDM

b) 𝑓′′(0.5) using 5-point centered FDM

Analytic Solution: Analytic Solution:

𝑥
𝑓′ 𝑥 = 1 + 2𝑥 2
2
3 𝑓 ′′ 𝑥 = 5
1− 𝑥 2
1− 𝑥2 2
𝒇′ 𝟎. 𝟓 = 𝟎. 𝟕𝟔𝟗𝟖𝟎𝟎𝟑𝟓𝟖𝟗
𝒇′′(𝟎. 𝟓) = 𝟑. 𝟎𝟕𝟗𝟐𝟎𝟏𝟒𝟑𝟔