Numerical Methods

(CENG-3081)

Instructor: Mastewal Shumet (Ph.D. Candidate)

Lecturer, School of Civil & Environmental Engineering.
Members of Road and Transport Engineering Chair
Addis Ababa Institute of Technology
Addis Ababa University
[email protected]
[email protected]
[email protected]
Numerical Methods
(CENG-3081)
Chapter 2
Roots of Equations
 Quadratic Formula

− b  b 2 − 4 ac
x=
2a
f ( x) = ax 2 + bx + c = 0

This equation gives us the roots of the algebraic function f(x)

i.e. the value of x that makes f(x) = 0

How can we solve for f(x) = e-x - x?

 Roots of Equations
• Plot the function and determine where it
crosses the x-axis
f(x)=e-x-x
• Lacks precision
10
• Trial and error 8
6
4
2
0
-2 -1 -2 0 1 2

-4
 Objectives of Study
• Understand the graphical interpretation of a root
• Know the graphical interpretation of the false-
position method and why it is usually superior to
the bisection method
• Understand the difference between bracketing and
open methods for root location
• Understand the concepts of convergence and
divergence.
• Know why bracketing methods always converge,
whereas open methods may sometimes diverge
 Objectives of Study Cont’d
• Realize that convergence of open methods is more
likely if the initial guess is close to the true root
• Know the fundamental difference between the false
position and secant methods and how it relates to
convergence
• Understand the problems posed by multiple roots
and the modification available to mitigate them
• Use the techniques presented to find the root of an
equation
 Overview of Methods
• Bracketing methods
– Graphing method
– Bisection method
– False position
• Open methods
– Fixed point iteration
– Newton-Raphson
– Secant method
 Graphical
(Limited Practical Value)
f(x) f(x)
consider lower and
upper bound same
sign, no roots or
x even # of roots x

f(x) f(x)
opposite sign,
odd # of roots

x x
 Bisection Method
• Takes advantage of sign changing.
• 𝑓 𝑥𝑙 𝑓 𝑥𝑢 < 0 where the subscripts refer
to lower and upper bounds respectively.
• There is at least one real root

f(x) f(x) f(x)

x x x
 Bisection Method Algorithm
✓ Choose 𝑥𝑢 𝑎𝑛𝑑 𝑥𝑙 . Verify sign change
▪ 𝑓 𝑥𝑙 𝑓 𝑥𝑢 < 0
✓ Estimate root
𝑥𝑙 +𝑥𝑢
▪ 𝑥𝑟 =
2
✓ Determine if the estimate is in the lower or upper
subinterval
▪ 𝑓 𝑥𝑙 𝑓 𝑥𝑟 < 0 then 𝑥𝑢 = 𝑥𝑟 RETURN
▪ 𝑓 𝑥𝑙 𝑓 𝑥𝑟 > 0 then 𝑥𝑙 = 𝑥𝑟 RETURN
▪ 𝑓 𝑥𝑙 𝑓 𝑥𝑟 = 0 then root equals 𝑥𝑟 →COMPLETE
 Termination Criteria and Error Estimates

𝑥𝑟 𝑛𝑒𝑤 − 𝑥𝑟 𝑜𝑙𝑑
𝜀𝑎 = 𝑛𝑒𝑤
100%
𝑥𝑟
Where 𝜀𝑎 = error in percent
𝑥𝑟 𝑛𝑒𝑤 = The root for the present iteration
𝑥𝑟 𝑜𝑙𝑑 = The root from the previous iteration
 Example 1

Use the bisection method to determine the root

𝑓 𝑥 = 𝑒 −𝑥 − 𝑥, 𝑥𝑙 = −1 & 𝑥𝑢 = 1

10

6
f(x)

4 3.7 1 8282

0 -0.6321 2

-2
-2 -1 0 1 2
x
 SOLUTION

10

6
f(x)

4 3.7 1 8282
2 1

0 -0.6321 2

-2
-2 -1 0 1 2
x
 Solution Cont’d

1
f(x)

0
0.1 06531 -0.6321 2

-2
-1 0 1 2
x
 −𝑥
𝑓 𝑥 =𝑒 −𝑥
′
Bisection Method (Solution Cont d)
𝑻𝒓𝒊𝒂𝒍 𝒙𝒍 𝒙𝒓 𝒙𝒖 𝒇(𝒙𝒍 ) 𝒇(𝒙𝒓 ) 𝒇(𝒙𝒍 )𝒇(𝒙𝒓 )
0 -1 0 1 3.718282 1 3.718282
1 0 0.5 1 1 0.106531 0.106531
2 0.5 0.75 1 0.106531 -0.27763 -0.02958
3 0.5 0.625 0.75 0.106531 -0.08974 -0.00956
4 0.5 0.5625 0.625 0.106531 0.007283 0.000776
5 0.5625 0.59375 0.625 0.007283 -0.0415 -0.0003
6 0.5625 0.578125 0.59375 0.007283 -0.01718 -0.00013
7 0.5625 0.570313 0.578125 0.007283 -0.00496 -3.6E-05
8 0.5625 0.566406 0.570313 0.007283 0.001155 8.41E-06
9 0.566406 0.568359 0.570313 0.001155 -0.00191 -2.2E-06
10 0.566406 0.567383 0.568359 0.001155 -0.00038 -4.3E-07
 Solution Cont’d

After 10 trials,
𝒙𝒓 = 0.567383

𝒇 𝒙 = 𝒆−𝒙 − 𝒙

𝒇 𝟎. 𝟓𝟔𝟕𝟑𝟖𝟑 = 𝒆−𝟎.𝟓𝟔𝟕𝟑𝟖𝟑 − 𝟎. 𝟓𝟔𝟕𝟑𝟖𝟑

= −0.00038 ≅ 𝟎

𝜺𝒂 = 𝟎. 𝟏𝟕%
 Example 2
Use the bisection method to determine the root of
the function:
𝒇 𝒙 = 𝒙𝟑 + 𝟒𝒙𝟐 − 𝟏𝟎,

𝒙𝒍 = 𝟏 & 𝒙𝒖 = 𝟐
 𝒇 𝒙 = 𝒙𝟑 + 𝟒𝒙𝟐 − 𝟏𝟎,
Bisection Method, 𝒙𝒍 = 𝟏 & 𝒙𝒖 = 𝟐
𝑻𝒓𝒊𝒂𝒍 𝒙𝒍 𝒙𝒓 𝒙𝒖 𝒇(𝒙𝒍 ) 𝒇(𝒙𝒓 ) 𝒇(𝒙𝒍 )𝒇(𝒙𝒓 )
0 1 1.5 2 -5 2.375 -11.875
1 1 1.25 1.5 -5 -1.79688 8.984375
2 1.25 1.375 1.5 -1.79688 0.162109 -0.29129
3 1.25 1.3125 1.375 -1.79688 -0.84839 1.524448
4 1.3125 1.34375 1.375 -0.84839 -0.35098 0.29777
5 1.34375 1.359375 1.375 -0.35098 -0.09641 0.033838
6 1.359375 1.3671875 1.375 -0.09641 0.032356 -0.00312
7 1.359375 1.3632813 1.3671875 -0.09641 -0.03215 0.0031
8 1.3632813 1.3652344 1.3671875 -0.03215 7.2E-05 -2.3E-06
9 1.3632813 1.3642578 1.3652344 -0.03215 -0.01605 0.000516
10 1.3642578 1.3647461 1.3652344 -0.01605 -0.00799 0.000128
 Solution Cont’d
After 10 trials,
𝒙𝒓 = 1.3647461

𝒇 𝒙 = 𝒙𝟑 + 𝟒𝒙𝟐 − 𝟏𝟎

𝒇 1.3647461 = (𝟏. 𝟑𝟔𝟒𝟕𝟒𝟔𝟏)𝟑 +𝟒(𝟏. 𝟑𝟔𝟒𝟕𝟒𝟔𝟏)𝟐 −𝟏𝟎

= −0.00799 ≅ 𝟎
𝜺𝒂 = 𝟎. 𝟎𝟒%
 False Position Method
• Even though bisection method is a perfectly
valid technique for determining roots
• “Brute-Force” approach of bisection method is
relatively inefficient
• Join points by a straight line
• Improves the estimate
• Replacing the curve by a straight line gives the
“false position”
 Based on similar
triangles
Next estimate, 𝒙𝒓 𝑓(𝑥𝑢 )

𝑥𝑙 f ( xl ) f ( xu )
=
𝑥𝑢 xr − xl xr − xu
𝑓(𝑥𝑙 )

f ( xu )( xl − xu )
xr = xu −
f ( xl ) − f ( xu )
 Example on False-Position Method
Find the root of 𝑓 𝑥 = 𝑥 − 4 2 𝑥 − 2 = 0
Using the Initial Guesses of 𝑥𝑙 = 1 & 𝑥𝑢 = 2.5, and a
pre-specified tolerance 𝜀𝑠 = 0.1%
 Solution
Trial 𝒙𝒍 𝒙𝒖 𝒇(𝒙𝒍 ) 𝒇(𝒙𝒖 ) 𝒙𝒓 𝒇(𝒙𝒓 )
0 1 2.5 -9 1.125 2.333333 0.925926
1 2.333333 1 0.925926 -9 2.208955 0.670295
2 2.208955 1 0.670295 -9 2.125157 0.43993
3 2.125157 1 0.43993 -9 2.072721 0.270115
4 2.072721 1 0.270115 -9 2.041464 0.159049
5 2.041464 1 0.159049 -9 2.023378 0.09134
6 2.023378 1 0.09134 -9 2.013097 0.051702
7 2.013097 1 0.051702 -9 2.00731 0.029026
8 2.00731 1 0.029026 -9 2.004072 0.01622
9 2.004072 1 0.01622 -9 2.002265 0.009041
10 2.002265 1 0.009041 -9 2.00126 0.005032

𝒙𝒓 = 𝟐. 𝟎𝟎𝟏𝟐𝟔
𝜺𝒂 = 𝟎. 𝟎𝟓𝟎𝟑 < 𝟎. 𝟏% (𝜺𝒔 ) 𝑶𝒌!
Exercise on False-Position Method
Determine the root of the following equation
using the false position method starting with an
initial estimate of 𝑥𝑙 = 4.55 𝑎𝑛𝑑 𝑥𝑢 = 4.65
𝑓 𝑥 = 𝑥 3 − 98
30
20
10
f(x) 0
-1 0
-20
-30
-40
4 4.5 5
x
Pitfalls of False Position Method

f(x)=x1 0-1

30
25
20
15
f(x)

10
5
0
-5
0 0.5 1 1 .5
x
 Group Assignment (5 Students )

• On page 142, Chapra 7th edition

• Q# 5.1 to 5.8
• Submission date October 28, 2024
 Open Methods
• Fixed-point iteration
• Newton-Raphson method
• Secant method
• In the previous bracketing methods, the root is
located within an interval prescribed by an
upper and lower boundary.
• In this case a single starting point is used
except the Secant Method
 Open Methods Cont’d
• Such methods are said to be convergent
– solution moves closer to the root as the
computation progresses
• Open method
– single starting value
– two starting values that do not necessarily
bracket the root (secant Method)
• These solutions may diverge
– solution moves farther from the root as the
computation progresses
f(x)

f(xi+1 ) The tangent gives next estimate.

xi

xi+1 x

f(xi)
 Solution can “overshoot” the root and potentially diverge
f(x)

x2 x1

x0 x
 Fixed-Point Iteration
• Open methods employ a formula to predict the root
• In fixed-point iteration, rearrange the function f(x)
so that x is on the left hand side of the equation
✓ For example, 𝑓 𝑥 = 𝑥 2 − 2𝑥 + 3 = 0
𝒙𝟐 +𝟑
▪𝒙 = 𝑜𝑟
𝟐
2 𝟑
▪ 𝑥 − 2𝑥 = −3 → 𝑥 𝑥 − 2 = −3 → 𝒙 =
𝟐−𝒙
– For the given rearranged function 𝑥 = 𝑔 𝑥 ,
– 𝐼𝑓 𝑔′(𝑥) ≤ 1, 𝑐𝑜𝑛𝑣𝑒𝑟𝑔𝑒𝑛𝑡, in the given set of interval
– 𝐼𝑓 𝑔′(𝑥) > 1, 𝑛𝑜𝑡 𝑐𝑜𝑛𝑣𝑒𝑟𝑔𝑒𝑛𝑡
 Fixed-Point Iteration Cont’d
• In fixed point iteration, rearrange the function
f(x) so that x is on the left hand side of the
equation
▪ i.e. for f(x) = sin x = 0
▪ x = sin x + x
• Let x = g(x)
• New estimate based on
➢ x i+1 = g(xi)
 EXAMPLE
• Consider f(x) = e-x -3x
• x=g(x) = e-x / 3 15

• Initial guess x = 0
10
Iter. x g(x) f(x) ea
0 0 0.333 -0.28347
1 0.333 0.239 0.071007 39.561% 5
2 0.239 0.263 -0.01842 9.016%
3 0.263 0.256 0.004737 2.395%
4 0.256 0.258 -0.00122 0.612% 0
-2 -1.5 -1 -0.5 0 0.5 1 1.5 2
5 0.258 0.258 0.000315 0.158%
6 0.258 0.258 -8.1E-05 0.041% -5
7 0.258 0.258 2.09E-05 0.010%
8 0.258 0.258 -5.4E-06 0.003%
9 0.258 0.258 1.39E-06 0.001% -10

After 9th iteration r= 0.2576….

 Newton Raphson

Slope

f(xi)
dy
Slope = = f ' ( x)
dx
f ( xi ) − 0
f ' ( xi ) =
xi − xi +1
xi Re arrange
xi+1
f ( xi )
xi +1 = xi −
f ' ( xi )
 EXAMPLE
Use the Newton Raphson method to determine the root of
f(x) = x2 - 11 using an initial guess of xi = 3
 Solution
𝑓 𝑥 = 𝑥 2 − 11
𝑓 ′ 𝑥 = 2𝑥
𝑥𝑖 = 3
𝑓(𝑥𝑛 )
𝑥𝑛+1 = 𝑥𝑛 −
𝑓′(𝑥𝑛 )
100

80
Iter. 𝒙𝒊 𝒙𝒊+𝟏 𝒇(𝒙) 𝜺𝒂

f(x) = x2 - 11
60
0 3 3.333333 0.111111
40

1 3.333333 3.316667 0.000278 0.503% 20

2 3.316667 3.316625 1.75E-09 0.001% 0

0 2 4 6 8 10 12
3 3.316625 3.316625 0 0.000% -20

After fourth iteration r=3.316625

 In your program code, check for
problems of divergence

• Include an upper limit on the number of

iterations
• Establish a tolerance, es
• Check to see if ea is increasing
What if derivative is difficult to evaluate?
SECANT METHOD
 Secant method

Approximate derivative using a finite divided difference

𝑓(𝑥𝑛 ) − 𝑓(𝑥𝑛−1 )
𝑓′(𝑥𝑛 ) ≅
𝑥𝑛 − 𝑥𝑛−1
What is this? HINT: dy / dx = Δy / Δ x

Substitute this into the formula for Newton Raphson

 𝒇(𝒙𝒏 )
𝒙𝒏+𝟏 = 𝒙𝒏 −
𝒇′(𝒙𝒏 ) Newton Raphson
Substitute finite difference Backward Divided Difference
approximation for the first
derivative into Newton
Raphson
𝑓(𝑥𝑛 ) − 𝑓(𝑥𝑛−1 )
𝑓′(𝑥𝑛 ) ≅
𝑥𝑛 − 𝑥𝑛−1

𝑓(𝑥𝑛 )(𝑥𝑛 −𝑥𝑛−1 ) Secant method

𝑥𝑛+1 = 𝑥𝑛 −
𝑓 𝑥𝑛 − 𝑓 𝑥𝑛−1
 Secant method
𝑓(𝑥𝑛 )(𝑥𝑛 −𝑥𝑛−1 )
𝑥𝑛+1 = 𝑥𝑛 −
𝑓 𝑥𝑛 − 𝑓 𝑥𝑛−1

• Requires two initial estimates, 𝑥0 & 𝑥1

• 𝑓(𝑥) is not required to change signs, therefore this is not a
bracketing method
• Iteration. For n = 1; 2; 3; · · · , until certain stopping
criterion is satisfied (required solution accuracy or
maximal number of iterations is reached).
 Example

• Solve the equation 𝑓 𝑥 = 𝑥 6 − 𝑥 − 1 = 0

using Secant Method
• The two initial estimates, 𝑥0 = 1 & 𝑥1 = 2
 Solution
trial 𝑥𝑛−1 𝑥𝑛 𝑓(𝑥𝑛−1 ) 𝑓(𝑥𝑛 ) 𝑥𝑛+1 𝑓(𝑥𝑛+1 )
0 1 2 -1 61 1.016129 -0.91537
1 2 1.016129 61 -0.915368 1.030675 0.657466
2 1.0161290 1.0306748 -0.9153677 -0.8319214 1.1756889 -0.16849
3 1.0306748 1.1756889 -0.8319214 0.4652272 1.1236791 -0.02244
4 1.1756889 1.1236791 0.4652272 -0.1106329 1.1336711 0.000954
5 1.1236791 1.1336711 -0.1106329 -0.0108059 1.1347527 -5.1E-06
6 1.1336711 1.1347527 -0.0108059 0.0002937 1.1347241 -1.1E-09
7 1.1347527 1.1347241 0.0002937 -0.0000007 1.1347241 0

𝑓(𝑥𝑛 )(𝑥𝑛 −𝑥𝑛−1 )

𝑥𝑛+1 = 𝑥𝑛 −
𝑓 𝑥𝑛 − 𝑓 𝑥𝑛−1
After Eighth iterations, our approximation to the root is 1.1347241.
 Example
As an example of the secant method, suppose we wish to find a root of
the function f(x) = cos (x) + 2 sin (x) + x2.
A closed form solution for x does not exist so we must use a numerical
technique. We will use x0 = 0 and x1 = -0.1 as our initial approximations.
We will let the two values εstep = 0.001 and εabs = 0.001

n xn − 1 xn xn + 1 |f(xn + 1)| |xn + 1 - xn|

0 0 -0.1 -0.514 0.1520 0.4137

1 -0.1 -0.514 -0.610 0.0461 0.0963
2 -0.51371 -0.610 -0.652 0.0066 0.0418
3 -0.60996 -0.652 -0.659 0.0004 0.0070
4 -0.6518 -0.659 -0.659 0.0000 0.0005
5 -0.6588 -0.659 -0.659 0.0000 0.0000
Thus, with the last step, both halting conditions are met, and therefore, after six
iterations, our approximation to the root is -0.6595 .
 FALSE POSITION
f(x)
2
SECANT METHOD
2

f(x)
1

1 x
new est.

The new estimate is selected from

the intersection with the x-axis x
new est.
 Group Assignment 2

• On pages 173-174, Chapra 7th edition

• Q# 6.2, 6.4, and 6.17
• Submission date October 28, 2024 (With Assignment 1)
Thank You!!!