1

Chapter 5: Derivative and Integral

Scientific Computing

SoICT 2023

2
 Content

1. Approximation of Derivative
2. Approximation of Integral

3
APPROXIMATION OF DERIVATIVE

4
 Derivative Problem
• The first order Derivative :
f ( x + h) − f ( x)
f ( x ) = lim
'
h →0 h
• Geometric meaning:
• f’(x) is the slope
of tangent at point x f(x+h)
(solid – line) f(x)
• Approximation:
• h≠0
• f’(x) is the slope
x x+h
of secant (dashed – line)

5
 Forward Difference Method (FD)

• Formulate the method: Consider the Taylor expansion of

the function f at the neighborhood of x:
2
h
f ( x + h ) = f ( x ) + f ( x )h + f (ζ )
' ''
(1)
where ξ belong to [x,x+h]. 2!
From (1) =>:
f ( x + h) − f ( x) h
f ( x) =
'
+ f (ζ )
''
( 2)
h 2!
Given that f’’(ξ) h/2 is the truncation error, from (2) =>:
f ( x + h) − f ( x)
f ( x) ≈
'
(3)
h
(3) is the Forward Difference (FD) formula to approximate
the derivative
6
 FD: Error analysis

• Truncation error : f’’(ξ) h/2= O(h)

⇒This method has accuracy of first order
• Rounding error: When calculating f(x) and f(x+h), if there is
a rounding error, the formula for f':
f ( x + h )(1 + δ1 ) − f ( x )(1 + δ 2 ) f ( x + h ) − f ( x ) δ1 f ( x + h ) − δ 2 f ( x )
= +
h h h
Because |δi| is smaller than the accuracy of the computer ε, so
the rounding error when calculating f' is:
ε ( f ( x + h) + f ( x) )
• The total error is minimal when: h

h≈ ε
7
 FD: Example

• Consider the function: f(x) = sin(x). Use the

forward difference method to approximate f'(π/3).
• Error analysis
• Calculate with h=10-k, k = 1,…,16
• Find h for minimum error

8
 Result

h Derivative Error
10-1 0.455901885410761 -0.044098114589239
10-2 0.495661575773687 -0.004338424226313
10-3 0.499566904000770 -0.000433095999230
10-4 0.499956697895820 -0.000043302104180
10-5 0.499995669867026 -0.000004330132974
10-6 0.499999566971887 -0.000000433028113
10-7 0.499999956993236 -0.000000043006764
10-8 0.499999996961265 -0.000000003038736
10-9 0.500000041370186 0.000000041370185

9
 Backward Difference Method (BD)

• Formulate the method: Similar to the forward difference

method, in Taylor expansion we use x-h instead of x+h,
we have :
f ( x) − f ( x − h)
f ( x) ≈
'
(1)
h
• Error: As same as in forward difference, backward
difference has the first order accuracy
• Minimum error when:
h≈ ε
• Exercise: Use the BD to approximate f'(π/3), knowing
that f(x) = sin(x)
10
 Central Difference Method (CD)

• Formulate the method: Consider the Taylor expansion

of the function f at the neighborhood x:
2 3
h h
f ( x − h ) = f ( x ) − f ' ( x )h + f '' ( x ) − f ''' (ζ − ) (1)
2! 3!
2 3
h + h
f ( x + h ) = f ( x ) + f ( x )h + f ( x ) + f (ζ )
' '' '''
( 2)
2! 3!
where ξ+ belongs to [x,x+h], ξ- belongs to [x-h,x]. From
(1) and (2), we have:
f ( x + h) − f ( x − h)
f ( x) =
'
(3)
2h
(3) is the Central Difference (CD) method

11
 CD: Error Analysis

• Truncation error:
1 '''
− f (ζ )h 2 , ζ ∈ [x − h, x + h ]
6
• CD method has accuracy order of 2;
• Error minimal when h = ε1/3

• Exercise: Use the CD method to approximate f'(π/3),

knowing that f(x) = sin(x). Compare with FD and BD
method

12
 Compare error of: FD, BD, CD
h FD BD CC
10-1 ~10-2 ~10-2 ~10-4
10-2 ~10-3 ~10-3 ~10-6
10-3 ~10-4 ~10-4 ~10-8
10-4 ~10-5 ~10-5 ~10-10
10-5 ~10-6 ~10-6 ~10-12
10-6 ~10-7 ~10-7 ~10-11
10-7 ~10-8 ~10-8 ~10-10
10-8 ~10-9 ~10-9 ~10-9
10-9 ~10-8 ~10-8 ~10-8

13
 Approximation of second order derivative

• Consider the Taylor expansion of the function f at the

neighborhood x:
2 3 4 5
h h h h
f ( x − h ) = f ( x ) − f ' ( x )h + f '' ( h ) − f ''' ( x ) + f '''' ( x ) − f ''''' ( x ) + ...(1)
2! 3! 4! 5!
h2 h3 h4 h5
f ( x + h ) = f ( x ) + f ( x ) h + f ( h ) + f ( x ) + f ( x ) + f ( x ) + ...( 2 )
' '' '' ' ''' ' '''''

2! 3! 4! 5!
• From (1) and (2), we have the approximate formula
for 2nd order derivative
f ( x + h) − 2 f ( x) + f ( x − h)
f ( x) =
''
(3)
h2
• Truncation error:
• Minimum when h = ε1/4
1 ''''
− f (ζ )h 2 , ζ ∈ [x − h, x + h ]
12
14
 Approximation of partial derivative

• Similarly, we can formulate the approximate formula

for partial derivative, for example, central difference
for partial derivatives of function f(x,y) as follows:

∂f ( x, y ) f ( x + h, y ) − f ( x − h, y )
=
∂x 2h
∂f ( x, y ) f ( x, y + h ) − f ( x, y − h )
=
∂y 2h

15
APPROXIMATION OF INTEGRAL

16
 Integral Problem

• Integral formula: b
I = ∫ f ( x )dx,
a

where f(x) is an integrable function on the interval [a,b]

• Geometrical meaning:

f(x)

a b
17
 Riemann Sum

• Suppose the function f defined on [a,b] and ∆ is the

division of the interval [a,b] into n closed sub-intervals
Ik=[xk-1,xk], k=1,…,n, where a = x0 < x1<…< xn-1< xn = b.
Choose n points {ck: k=1,…,n}, each of which belongs to a
sub-interval, that is: ck belongs to Ik for all k. The sum:
n

∑
k =1
f ( c k ) ∆ x k = f ( c1 ) ∆ x1 + f ( c 2 ) ∆ x 2 + ... + f ( c n ) ∆ x n

is called the Riemann sum of the function f(x)

corresponding to the division ∆ and the selection points
{ck: k=1,…,n}.

18
 Approximating Integral

• The definite integral of the function f(x) wrt x from a to

b is the limit of the Riemann sum
b n

∫ f ( x ) dx = lim
n→ ∞
∑
k =1
f ( ck ) ∆ xk ,
a

• It is assumed that this limit exists.

• The function f(x) is called the function to be integrated
• a, b are the integral limitations
• [a,b] is the integral interval

19
 Properties of definite integrals
a

∫ a
f ( x ) dx = 0

b a

∫ a
f ( x ) dx = − ∫
b
f ( x ) dx

b b

∫C.f
a
( x ) dx = C . ∫ f ( x ) dx
a

b b b

∫(
a
f ( x ) + g ( x )) dx = ∫
a
f ( x ) dx + ∫
a
g ( x ) dx

b c b

∫ f ( x ) dx = ∫ f ( x ) dx + ∫ f ( x ) dx , c ∈ [a , b ]
a a c

20
 Theorems

• Theorem 1: If f is continuous on [a,b] and F is a

primitive of the function f (F' = f), then:
b

∫a
f ( x ) dx = F ( b ) − F ( a )
• Theorem 2 (Theorem about mean value): If f is
continuous on [a,b], there exists a number c in the
interval [a,b] such that :
b
1
f (c ) =
b−a ∫
a
f ( x ) dx

21
 Newton-Cotes formula

• The first approach to formulate an integral approximation

is to approximate the function f(x) over the integral
interval [a,b] by a polynomial. In each subinterval we
approximate the function f(x) by a polynomial:
pm(x) = a0 + a1x + a2x2 + … + amxm (1)
• We can replace the function f(x) by an interpolated
polynomial (1).
• And then easily calculate the exact integral of interpolated
polynomial

22
 Newton-Cotes formula

• Replace f(x) with Lagrange interpolated polynomial,

we have:
b a  m m x−x 

f ( x ) dx = ∫ ∑ ∏ 
∫
j
f ( x i ) dx
 i=0 j=0 xi − x j 
a b
 j≠i 
m b m
x − xj
= ∑ f ( xi ) ∫ ∏ dx (1)
i=0 j=0 x − x
a j≠i i j

23
 Newton-Cotes formula

• Error of the method is evaluated by:

b b a  m 
( ζ x )  ∏ ( x − x i )  dx
1
∫ f ( x ) dx − ∫ p m ( x ) dx = ∫
( m +1 )
f
( m + 1)! b  i=0 
a a
 
ζ x ∈ [a , b ] (2)

24
 Newton-Cotes formula

• The integral approximation formulas are obtained with this

approach which uses an equal grid in the integral interval,
i.e.:
xi = a+i*h; i=0,1,…,m; h =(b-a)/m,
is called the Newton-Cotes formula.
• For different m, we have different Newton-Cotes formulas
m Order Formula Error
1 1 Trapezoidal rule O(h2 )
2 2 Simpson 1/3 O(h4 )
3 3 Simpson 3/8 O(h4 )

25
 Trapezoidal rule

• For n=1, the interpolated polynomial has the form:

f (b) − f ( a )
p1 ( x ) = f (a ) + ( x − a)
b−a
b b b
 f (b) − f ( a ) 
⇒ I = ∫ f ( x )dx ≈ ∫ p1 ( x )dx = ∫  f ( a ) + ( x − a ) dx
a a a
b−a 

⇒I=
( f ( a ) + f (b) )
(b − a ) (1)
2
• (1) is called the Trapezoidal formula for approximating the
integral

26
 Trapezoidal rule

• Error of trapezoidal rule:

b − a ''
− f (ζ )h 2 , h = b − a, ζ ∈ [a, b]
12
• Geometrical meaning:

fb
fa `

f(x)

a b
27
 Trapezoidal rule: Extension

fb
fb
fa fa
f(x) f(x)

a b a b

• The idea of the extended trapezoidal formula: Divide the

interval [a,b] into subintervals in order to reduce the error

28
 Trapezoidal rule: Extension

• Divide [a,b] into n equal intervals using n+1 points:

x0 = a, x1 = a + h, xn-1 = a + (n-1)*h, xn = a + n*h
where h = (b-a)/n, we have:
b a +h a +2 h a + nh
I = ∫ f ( x)dx = ∫ f ( x)dx + ∫ f ( x)dx + ... + ∫ f ( x)dx (1)
a a a +h a +( n −1) h

• Applying the trapezoidal formula for each subinterval we

have: h n −1

I=  f ( a ) + 2∑ f ( a + ih ) + f (b) ( 2)
2 i =1 

• (2) is called the expanded trapezoid formula

29
 Simpson 1/3 formula

• Substituting n=2 into the Newton-Cotes formula and

integrating, we get:
b
h
I =∫ f ( x )dx = [ f ( x0 ) + 4 f ( x1 ) + f ( x2 )]
a
3
x0 = a, x1 = a + h, x2 = a + 2h = b, (1)

• (1) is called Simpson's formula 1/3

30
 Simpson 1/3 formula: Extension

• Like the extended trapezoidal rule, we divide the interval

[a,b] into several subintervals and apply the Simpson 1/3
to each subinterval, we get the extended Simpson 1/3:
n −1 n −2

b f ( x0 ) + 4 ∑ f (x ) + 2 ∑ f (x ) + f (x )
i j n

I = ∫ f ( x)dx =(b − a ) i =1, 3,5,.. j =2 , 4 ,6,..

a
3n
x0 = a, xi = a + ih, i = 1,..., n, (1)

• Note: We need an even number of subintervals, or an odd

number of points.

31
 Simpson 3/8 formula

• Substituting n=3 into the Newton-Cotes formula and

integrating, we get:
b
3h
I =∫ f ( x )dx = [ f ( x0 ) + 3 f ( x1 ) + 3 f ( x2 ) + f ( x3 )]
a
8
x0 = a, x1 = a + h, x2 = a + 2 h , x3 = a + 3h (1)

• (1) is called Simpson formula 3/8

32
33