MTH2212 – Computational Methods and

Statistics

Numerical Differentiation and Integration

Lecture 8:
Numerical Differentiation
 Objectives

 Introduction
 Taylor Series
 Forward difference method
 Backward difference method
 Central difference method

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 2

 Introduction

 Calculus is the mathematics of change. Because engineers

must continuously deal with systems and processes that
change, calculus is an essential tool of engineering.
 Standing in the heart of calculus are the mathematical
concepts of differentiation and integration:

y f ( xi  x)  f ( xi )

x x
dy f ( xi  x)  f ( xi )
 x lim 0
dx x
b
I   f ( x)dx
a

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 3

 Introduction

 Integration and differentiation are closely linked processes.

They are, in fact, inversely related.

 Types of functions to be differentiated or integrated:

1. Simple polynomial, exponential, trigonometric  analytically
2. Complex function  numerically
3. Tabulated function of experimental data  numerically

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 4

 Applications

 Differentiation has so many engineering applications (heat

transfer, fluid dynamics, chemical reaction kinetics, etc…)

 Integration is equally used in engineering (compute work in

ME, nonuniform force in SE, cross-sectional area of a river,
etc…)

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 5

 Differentiation

y f ( x i  x )  f ( x i ) dy f ( x i  x )  f ( x i )
  lim
x x dx x  0 x

The finite difference becomes a derivative as Δx approaches zero.

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 6
 Integration

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 7

 Differentiation vs. Integration

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 8

 Taylor Series Expansion

 Non-elementary functions such as trigonometric,

exponential, and others are expressed in an approximate
fashion using Taylor series when their values, derivatives,
and integrals are computed.

 Any smooth function can be approximated as a polynomial.

Taylor series provides a means to predict the value of a
function at one point in terms of the function value and its
derivatives at another point.

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 9

 Numerical Application of Taylor Series

 If f (x) and its first n+1 derivatives are continuous on an

interval containing xi+1 and xi , then:
f ''
 xi  
f  xi 1   f  xi   f '  xi  xi 1  xi   xi 1  xi 
2

2!
f  3   xi  f  n   xi 
  xi1  xi   ... 
3
 xi 1  xi  n  Rn
3! n!

Where the remainder Rn is defined as:

f  n1   
Rn   xi1  xi  n1
 n  1!

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 10

 Numerical Application of Taylor Series

 The series is built term by term

f  xi 1   f  xi  zero order approximation

 Continuing the addition of more terms to get better

approximation we have:
f  xi 1   f  xi   f '  xi  xi 1  xi  1st order approximation

f ''  xi 
f  xi 1   f  xi   f  xi  xi 1  xi  
'
 xi1  xi  2 2nd order
2! approximation

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 11

 Numerical Application of Taylor Series

 Finally
f ''  xi 
f  xi 1   f  xi   f  xi  xi 1  xi  
'
 xi1  xi  2
2!
f  3   xi  f  n   xi 
  xi 1  xi   ... 
3
 xi 1  xi  n  Rn
3! n!
where

f  n1   
Rn   xi1  xi  n1
 n  1!
ξ is a value of x that lies somewhere between xi and xi+1.

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 12

 Taylor Series ξ in the Remainder Term

 Limitations
 ξ is not exactly known but lies somewhere between xi and xi+1
To evaluate Rn, the (n+1) derivative of f (x) has to be determined. To
do this f (x) must be known
 if f (x) was known there would be no need to perform the Taylor
series expansion!!!
 Modification
Rn = O(hn+1) the truncation error is of the order of hn+1 . (h = xi+1 – xi)
 If the error is O(h), halving the step size will halve the error.
 If the error is O(h2), halving the step size will quarter the error.
 In general, the truncation error is decreased by addition of more
terms in the Taylor series.

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 13

 Forward Difference Formulas- 1st derivative

 2nd order Taylor series expansion of f(x) can be written as:

f ' ' ( xi ) 2
f ( xi  1 )  f ( x i )  f ' ( x i ) h  h  O(h 3 ) (1)
2
 Then, the first derivative can be expressed as:

f ( xi  1 )  f ( xi ) f ' ' ( x i )
f ' ( xi )   h  O(h 2 ) (2)
h 2
 Given that f“(x) can be approximated by:

f ( x i  2 )  2 f ( xi  1 )  f ( x i )
f ' ' ( xi )   O ( h) (3)
h2
Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 14
 Forward Difference Formulas- 1st derivative

 Substituting equation (3) into equation (2):

f ( x i 1 )  f ( x i ) f ( x i  2 )  2 f ( x i  1 )  f ( x i )
f ' ( xi )   h  O(h 2 )
h 2h 2

 Collecting terms and simplifying, we have:

 f ( x i  2 )  4 f ( xi  1 )  3 f ( xi )
f ' ( xi )   O(h 2 )
2h

 Note that the inclusion of the second-derivative term has

improved the accuracy to O(h2).

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 15

 Forward Difference Formulas- 2nd derivative

 Start with Lagrange interpolation polynomial for f(x) based

on the four points xi, xi+1, xi+2 and xi+3 .
 Differentiate the products in the numerators twice
 Substitute x = xi and consider the fact that xj - xi =(j- i)h
 The expression of the second derivative is then:

 f ( x i  3 )  4 f ( x i  2 )  5 f ( xi  1 )  2 f ( xi )
f ' ' ( xi )   O(h 2 )
h2

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 16

 Backward Difference Formulas- 1st derivative

 Using backward difference in the Taylor series expansion,

f ' ' ( xi ) 2
f ( xi 1 )  f ( xi )  f ' ( xi )h  h  O(h3 )
2
 And given that f“(x) can be approximated by:

f ( xi )  2 f ( xi 1 )  f ( xi  2 )
f ' ' ( xi )  2
 O ( h)
h

 The second-order estimate of f’(x) can be obtained:

3 f ( x i )  4 f ( x i 1 )  f ( x i  2 )
f ' ( xi )   O (h2 )
2h

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 17

 Backward Difference Formulas- 2nd derivative

 Start with Lagrange interpolation polynomial for f(x) based

on the four points xi, xi-1, xi-2 and xi-3 .
 Differentiate the products in the numerators twice
 Substitute x = xi and consider the fact that xj - xi =(j- i)h
 The expression of the second derivative is then:

2 f ( x i )  5 f ( x i 1 )  4 f ( xi  2 )  f ( xi  3 )
f ' ' ( xi )  2
 O ( h 2
)
h

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 18

 Centered Difference Formulas- 1st derivative [O(h2)]

 Start with the 2nd degree Taylor expansions about x for

f(x+h) and f(x-h):
f ' ' ( xi ) 2
f ( xi 1 )  f ( xi )  f ' ( xi ) h  h  O(h 3 ) (4)
2
f ' ' ( xi ) 2
f ( xi 1 )  f ( xi )  f ' ( xi )h  h  O(h3 ) (5)
2
 Subtract (5) from (4)
f ( xi 1 )  f ( xi 1 )  2 f ' ( xi )h  O(h 3 )

 Hence f ( xi 1 )  f ( xi 1 )
f ' ( xi )   O(h 2 )
2h
Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 19
 Centered Difference Formulas- 1st derivative [O(h4)]

 Start with the difference between the 4th degree Taylor

expansions about x for f(x+h) and f(x-h):
2 f ' ' ' ( xi ) 3
f ( xi 1 )  f ( xi 1 )  2 f ' ( xi )h  h  O(h5 ) (6)
3!
 Use the step size 2h, instead of h, in (6)
16 f ' ' ' ( xi ) 3
f ( xi  2 )  f ( xi  2 )  4 f ' ( xi )h  h  O(h5 ) (7)
3!
 Multiply equation (6) by 8, subtract (7) from it, and solve for
f’(x)
 f ( xi  2 )  8 f ( xi 1 )  8 f ( xi 1 )  f ( xi  2 )
f ' ( xi )   O(h 4 )
12h
Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 20
 Centered Difference Formulas- 2nd derivative [O(h2)]

 Start with the 3rd degree Taylor expansions about x for

f(x+h) and f(x-h):
f ' ' ( xi ) 2 f ' ' ' ( xi ) 3
f ( xi 1 )  f ( xi )  f ' ( xi )h  h  h  O(h 4 ) (8)
2 3!
f ' ' ( xi ) 2 f ' ' ' ( xi ) 3
f ( xi 1 )  f ( xi )  f ' ( xi )h  h  h  O(h 4 ) (9)
2 3!
 Add equations (8) and (9), and solve for f’’(x)

f ( xi 1 )  2 f ( xi )  f ( xi 1 )
f ' ' ( xi )  2
 O ( h 2
)
h

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 21

 Centered Difference Formulas- 2nd derivative [O(h4)]

 Start with the addition between the 5th degree Taylor

expansions about x for f(x+h) and f(x-h):
2 f ' ' ( xi ) 2 2 f ( 4 ) ( xi ) 4
f ( xi 1 )  f ( xi 1 )  2 f ( xi )  h  h  O(h 6 ) (10)
2! 4!
 Use the step size 2h, instead of h, in (10)
8 f ' ' ( xi ) 2 32 f ( 4 ) ( xi ) 4
f ( xi  2 )  f ( xi  2 )  2 f ( xi )  h  h  O(h 6 ) (11)
2! 4!
 Multiply equation (10) by 16, subtract (11) from it, and solve

for f’’(x)
 f ( xi  2 )  16 f ( xi 1 )  30 f ( xi )  16 f ( xi 1 )  f ( xi 2 ) 4
f ' ' ( xi )  2
 O ( h )
12h
Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 22
 Error Analysis

 Generally, if numerical differentiation is performed, only

about half the accuracy of which the computer is capable is
obtained unless we are fortunate to find an optimal step
size.
 The total error has part due to round-off error plus a part
due to truncation error.

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 23

 Total Numerical Error

Total numerical error = truncation error + round-off error.

Trade-off between truncation and round-off errors

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 24
 Example 1

Estimate the first and second derivatives of:

f ( x)  1.2  0.25 x  0.5 x 2  0.15 x 3  0.1x 4

at x = 0.5 and h = 0.25 using

a) forward finite-divided difference

b) Centered finite-divided difference

c) backward finite-divided difference?

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 25

 Example 1 - Solution

a)Forward difference
 1st derivative computation

The data needed is:

xi  0.5 f ( xi )  0.925
xi 1  0.75 f ( xi 1 )  0.636328
xi  2  1 f ( xi  2 )  0.2

First derivative:
 f ( xi  2 )  4 f ( xi 1 )  3 f ( xi )
f ' ( xi ) 
2h
 f (1)  4 f (0.75)  3 f (0.5)
f ' (0.5)   0.859375 εt = 5.82%
2(0.25)
True value=-0.9125
Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 26
 Example 1 - Solution

 Second derivative computation

The data needed is:
xi  0.5 f ( xi )  0.925
xi 1  0.75 f ( xi 1 )  0.636328
xi  2  1 f ( xi  2 )  0.2
xi 3  1.25 f ( xi 3 )  1.94336

Second derivative:
 f ( xi 3 )  4 f ( xi  2 )  5 f ( xi 1 )  2 f ( xi )
f ' ' ( xi ) 
h2
 (1.94336)  4(0.2)  5(0.636328)  2(0.925)
f ' ' (0.5)  2
 39.6
(0.25)
Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 27
 Example 1 - Solution

b) Centered finite-divided difference

 The data needed is:
xi 2  0 f ( xi  2 )  1.2
xi 1  0.25 f ( xi 1 )  1.103516
xi 1  0.75 f ( xi 1 )  0.636328
xi  2  1 f ( xi  2 )  0.2
 First derivative:
 f ( xi  2 )  8 f ( xi 1 )  8 f ( xi 1 )  f ( xi 2 )
f ' ( xi ) 
12h
 (0.2)  8(0.636328)  8(1.103516)  (1.2)
f ' ( xi ) 
12(0.25) εt = 0%
f ' ( xi )  0.9125 True value=-0.9125
Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 28
 Example 1 - Solution

c) Backward finite-divided difference

 The data needed is:
xi  2  0 f ( xi 2 )  1.2
xi 1  0.25 f ( xi 1 )  1.103516
xi  0.5 f ( xi )  0.925
 First derivative :
3 f ( xi )  4 f ( xi 1 )  f ( xi  2 )
f ' ( xi ) 
2h
3(0.925)  4(1.103516)  (1.2)
f ' ( xi )   0.878125
2(0.25)
εt = 3.77%
True value=-0.9125
Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 29