Numerical Methods [CEng2112] – Chapter 2: Solving non-linear equations

Numerical Methods
CEng2112
Chapter 2
Solving non-linear equations of one variable
Contents:-
2.1 Locating roots
2.2 Numerical methods of solving a non-linear equation
I. Bracketing Methods
 Bisection Method
 Method of false position
II. Open Methods
 Iteration Method (Fixed point Iteration)
 Secant Method
 Newton - Raphson Method

Objectives:- At the end of this chapter students should be able to:

 Understand the difference between bracketing and open methods
 Know the graphical interpretation of each method
 Identify the advantage and disadvantage of each method.
 Know the features of each method:
 The Procedure/Implementation of the method
 The condition of Convergence

Page 1 of 16
 Numerical Methods [CEng2112] – Chapter 2: Solving non-linear equations

Introduction:

Activity: Identify the difference between

 Algebraic and transcendental equations
 Linear equations and non-linear equations
 Systems of linear equations and systems non-linear equations

A problem of great importance in science and engineering is that of determining the roots/zeros of an
equation of the form ( ) .

Definition (Root or solution of an equation):

 A number , for which ( ) , is called the root of the equation ( )
 Geometrically, the roots of ( ) are points at which the graph of ( ) crosses or touches the x-
axis. i.e x-intercepts

A root of an equation may be simple or multiple;

 A number is a simple root of ( ) , if ( ) and f ′(r) ≠ 0. Then, we can write f(x) as
f (x) = (x – r) g(x), g(r) ≠ 0. Example: x = 1 is a simple root of ( ) , because
we can write ( ) ( )( ).
 A number is a complex root of ( ) with multiplicity . Then, we can write f (x) as
( ) ( ) ( ) ( ) . Example: x = 2 is a multiple root of ( )
with multiplicity 2 (double root), because we can write ( ) ( ) ( ).
Remark: In this chapter, we focus on different numerical methods of approximating real simple roots
of a non-linear equation of a single variable ( ) .

Types of non-linear equations:

Algebraic equations:
Page 2 of 16
 Numerical Methods [CEng2112] – Chapter 2: Solving non-linear equations

 Polynomial equations
Transcendental equations:
 Exponential equations
 Logarithmic equations
 Trigonometric equations
 Hyperbolic equations

A fundamental technique in computer science to solve equations is Iteration. Iteration is a process

which is repeated until an answer or root is achieved, i.e. producing a sequence of numbers that
hopefully converge towards a limit which is a root. The first values of this sequence are called initial
guesses (values).

Common Iterative numerical methods can be classified as:

Bracketing methods: These methods start with guesses that bracket (contain) the root and then
systematically reduce the width of the bracket.
 Bisection
 False position
Open methods: These methods also involve systematic trial-and-error iterations but do not
require that the initial guesses contain the root.
 Fixed point iteration
 Newton - Raphson method
 Secant method
2.1 Locating roots
Root locating theorem: From the intermediate value theorem of calculus, If ( ) is continuous on
[a, b] & ( ) ( ) , then there is one real number , - such that ( ) .
crosses or touches the at
has a root at
Locating the roots has limited practical value because it is not precise. However, locating the roots
can be utilized to obtain rough estimates of roots. These estimates can be employed as starting guesses
for the numerical methods.
Example: Give an interval that locates a solution of:
a. ( ) c. ( )

b. ( ) d. ( )

Page 3 of 16
 Numerical Methods [CEng2112] – Chapter 2: Solving non-linear equations

2.2 Numerical methods of solving a non-linear equation ( )

These methods are based on the idea of successive approximations. We start with one or two initial
approximations to the root and obtain a sequence of approximations * + which will converge to the
exact root.
Since, we cannot perform infinite number of iterations; we need a criterion to stop the iterations. We
use one or both of the following criterion:
(i) The equation ( ) is satisfied to a given accuracy or ( ) is bounded by an error
tolerance ε.
| ( )|
(ii) The magnitude of the difference between two successive iterates is smaller than a given
accuracy or an error bound ε.
| |
For example, if we require two decimal place accuracy, then we iterate until
| | .
If we require three decimal places accuracy, then we iterate until| | .

Here, we will see each iterative methods of solving a given non-linear equation ( )

I. Bracketing Methods:- these methods contains or brackets the root by an interval.

a. Bisection Method:- The bisection method is a root-finding algorithm which repeatedly
bisects an interval then selects a subinterval in which a root must lie for further processing.

Page 4 of 16
 Numerical Methods [CEng2112] – Chapter 2: Solving non-linear equations

 Procedure: Suppose ( ) is continuous on an interval [a, b]

 Check that ( ) ( ) (i.e the function must have opposite signs at the end
points)

 Let be the first estimate of the root, given by , which is the midpoint of

the interval
 Make the following evaluations to determine the subinterval in which the root lies:
i. If ( ) then is the root of .
ii. If ( ) ( ) then the root lies in the subinterval , -
iii. If ( ) ( ) then the root lies in the subinterval , -
 Continue the process until either we get the exact root or we may have an approximate
root with the required degree of accuracy.

The Bisection method generates a sequence * + of mid points of the reduced interval. i.e

( ) where

 Condition of convergence Theorem: Assume that is continuous on [a, b] and ( ) ( )

, then
 There exists a number , - such that ( ) .
 The sequence * + converges to the zero . That is, .
 Termination criteria for a given tolerance (acceptance) or error :
The number of iteration ( ) process will be terminated when the length of the interval ( )
becomes very small. i.e

Absolute error ( ) | |

Now, to have an absolute error with tolerance of error and number of iteration ( ) should
satisfy:
( )
( )

Example1: Approximate the root of ( ) in [2, 3].

( ) ( ) & also is cont on [2, 3]. Thus has at least one root b/n 2 & 3.

Page 5 of 16
 Numerical Methods [CEng2112] – Chapter 2: Solving non-linear equations

( )
1 2 3 2.5 5.6250 > 0
2 2 2.5 2.25 1.8906 > 0
3 2 2.25 2.125 0.3457 > 0
4 2 2.125 2.0625 -0.3513 < 0
5 2.0625 2.125 2.09375 -0.0089 < 0

 The desired approximation is if

| | | |
Example2: Determine approximately how much iteration is necessary to solve
( ) , with an accuracy of for and
Solution: The maximum number of iteration is given by
( )

( )

It requires iterations to obtain an approximation to an accurate of

 Advantages and Disadvantages:

 It is very easy to implement
 Very reliable and has good error bounds
 Although it is reliable, it converges slowly.
 The bisection method gives only a range where the root exists, rather than a single estimate
for the root's location.
 If has several simple roots in the interval , - then the bisection method will find one of
them.
 We can determine the number of iterations for a given tolerance (acceptable) of error .

b. Method of False position/ Regula Falsi Method /linear interpolation method

One of the shortcoming of bisection method is that, in dividing the interval , - no account is
taken of the magnitude of ( ) ( ) For example if ( ) is much closer to zero than
( ) then it is likely that the root is closer to than .

Page 6 of 16
 Numerical Methods [CEng2112] – Chapter 2: Solving non-linear equations

An alternative method, false position that exploits this graphical insight is to join
( ) ( ) by a straight line (chord).
The intersection of this line with x-axis represents an improved estimate of the root.

 Procedure: In this method we choose two points ( ) ( ) are of opposite

signs. Hence, a root of ( ) must lie in between these points, given that is continuous
on , -
 Now the equation of the chord joining the two points ( ( )) & ( ( )) is
given by
( ) ( ) ( )
=

 The intersection of this line with the x-axis is the -intercept let which is the first
estimate of the root;
( ) ( )
( ) ( )
( )
( )
( ) ( )
 Check the functional value of ( )
 If ( ) then x3 is the exact root of ( )
 If ( ) the root is in [x3, x2] (considering ( ) ), In this case join the
points ( ( )) & ( ( )) using st. line to get a better approximate root .
 If ( ) then the root is in [x1, x3]. If so, join the points ( ( ))
( ( )) using straight line to get a better approximate root .
 Continue the process by bracketing the root until the root is estimated adequately by;

Page 7 of 16
 Numerical Methods [CEng2112] – Chapter 2: Solving non-linear equations

( )
( ) , for the interval , - that contains the root
( ) ( )

 Condition of convergence Theorem:

Assume that is continuous on [a, b] and there exists a number , - such that ( )
. If ( ) ( ) have opposite signs, and

( )
( ) ( ) ( )

represents the sequence of points generated by the false position process, then the sequence
converges to the zero .

That is, .

 Termination criteria for a given tolerance of error : Here, we use the size of | ( )| as the
stopping criterion for it. | ( )|

Example: Find a root of the equation

( ) in [2, 3], using regula falsi method up to 4th iteration.
Solution:
The formula for the method is
( )
- ( ) ( ) ( )

Since ( ) ( ) the root lies b/n 2 & 3

( )
 - = =
( )

Now find ( ) ( ) ( )

The root is in , -
Now use
( )
( ) ( ) ( )
( )( )

Page 8 of 16
 Numerical Methods [CEng2112] – Chapter 2: Solving non-linear equations

etc.

 Advantages and Disadvantages:

 Advantages: faster than bisection method
 Limitations:
o When there is a discontinuity over an interval.
o When there are distinct roots.
o Over a "large" interval.

II. Open Methods: these methods do not contain the root by an interval.

a. Fixed-point Iteration: This method involves writing the equation ( ) in a form of

( ). There are many ways of rewriting ( ) in this form. The function ( ) is called
iteration function.
This method needs one initial value (initial approximate root) and an iteration function.

Definition (Fixed Point): A fixed point of a function ( ) is a number such that ( )

A fixed point is not a root of the equation ( ) , it is a solution of the equation
( ). Geometrically, the fixed points of a function ( ) are the point(s) of intersection of the
curve ( ) and the line .

Now, finding a root of ( ) is same as finding a number such that ( ).

Page 9 of 16
 Numerical Methods [CEng2112] – Chapter 2: Solving non-linear equations

 Procedure:
A function ( ) for computing successive terms is needed, together with a starting value ,
Such that ( ). Then a sequence of values * + is obtained using the iterative rule
( ).

Definition (Fixed Point Iteration): The iteration ( ) for is called fixed

point iteration.

The sequence has the pattern

( )
( )
( )

( )

Convergence of an iteration method depends on the choice of the iteration function ( ), and a
suitable initial approximation , to the root.

 Condition of convergence Theorem:

Page 10 of 16
 Numerical Methods [CEng2112] – Chapter 2: Solving non-linear equations

Theorem: Assume that ( ) is a continuous function and that * + is a sequence

generated by fixed point iteration. If , then is a fixed point of ( ).

The following two theorems establish conditions for the existence of a fixed point and the
convergence of the fixed-point iteration process to a fixed point.

Theorem (Existence and uniqueness): Assume that ( ) is continuous on [a, b], Then we
have the following conclusions.

(i). If the range of the mapping ( ) satisfies , - for all , -, then has
a fixed point in , -.
(ii). Furthermore, suppose that ( ) is defined over ( ) and that a positive constant
exists with | ( )| for all ( ) , then has a unique fixed point in
, -.

Theorem (conditions of convergence): Assume that the following hypothesis hold true.

(a) is a fixed point of a function ,

(b) and are continuous on, -
(c) is a positive constant,
(d) ( ), and
(e) ( ) , - for all , -. Then we have the following conclusions.

(i). If | ( )| for all , - , then the iteration ( ) will converge

to the unique fixed point ( ). In this case, is said to be an attractive fixed point.

(ii). If | ( )| for all , - , then the iteration ( ) will not converge

to . In this case, is said to be a repelling fixed point and the iteration exhibits local
divergence.
 Termination criteria within a given tolerance error :- The stopping criterion of the method
of fixed point iteration is | |<

Example: Use the method of simple iteration to find the root of the function ( ) ,
use
Solution

Take ( )

( )
Page 11 of 16
 Numerical Methods [CEng2112] – Chapter 2: Solving non-linear equations

( )
( )
( )
( )
( )

 Using only five iterations the approximate root of ( ) is 0.257

Example 2: Use the method of simple iteration to find the root of ( ) .Take .
Solution: Let us consider the following two different ways:

I II

( ) ( )

( ) ( )
( ) ( )
( ) ( )
( ) ( )

But, the exact root is √

Remark: The simple iteration may fail to converge. This usually happens when the slope of
( ) (in absolute value) near the root is too large.
Exercise: Observe the convergence of the solution of ( ) such that

( )

b. The Secant method: The secant method is an open method of root-finding algorithm that uses
a succession of roots of secant lines to better approximate a root of a function f. It resembles
totally the method of false position except that no attempt is made to ensure the root is enclosed.
This method needs two initial values (initial approximate roots).
 Procedure: Let be the root of ( ) i.e ( ) , and assume be two
initial approximate roots of .

Page 12 of 16
 Numerical Methods [CEng2112] – Chapter 2: Solving non-linear equations

 Draw the secant line passing through ( ( )) ( ( ))

 Find the point at which the line crosses the -axis, let and from the slope of the
( ) ( ) ( )
line, we have
( )
- ( ) ( ) ( )

 Further approximations of are computed using the iteration;

( )
– ( ) ( ) ( )

 Condition of convergence Theorem:

The iterates xn of the secant method converge to a root of f, if the initial values x0 and x1 are
sufficiently close to the root.
If the initial values are not close to the root, then there is no guarantee that the secant
method converges.
 Termination criteria within a given tolerance error : The stopping criterion for this
method is the size of | ( )|

Example: Use the secant method to approximate the root of ( ) with initial
approximations & & | ( )| < Tol =
Solution
( )
( )

Page 13 of 16
 Numerical Methods [CEng2112] – Chapter 2: Solving non-linear equations

( ), - ( )
( ) ( )

And ( )
( ), - , -
( ) ( )

And ( )
( ), -
–
( ) ( )
And ( )

And ( )
....
 Advantages and Disadvantages:
We are not sure about the convergence of the approximated roots to the exact root. But, the
process is simpler; because the sign of ( ) is not tested, and often converges faster

Exercise: Use the method of regula falsi method to find the root ( ) until

| ( )| in the interval , - (Hint: Use radian measure).

c. Newton - Raphson Method: This open method needs one initial value (approximate root)

 Procedure:
Let denote the known approximate value of the root of ( ) then using Taylor series
expansion about we expand ( ) as:
( ) ( ) ( )( ) ( ) ( ) ( ) ( )

Neglecting the second & higher order derivatives, we get:

( ) ( )( ) =0
( ) ( )
( ) ( )

Replacing x by x1 we get:
( )
( )
Hence, successive approximate roots are given by Newton – Raphson iteration;
( )
( )

Page 14 of 16
 Numerical Methods [CEng2112] – Chapter 2: Solving non-linear equations

OR, Geometrically, Newton – Raphson method consists in replacing the part of the curve between
the point ( ( )) & the -axis by means of the tangent line to the curve at point ( ( )) & is
shown as:

 From the figure, is the root of ( ) . i.e ( )

 And is the initial approximate root of
 Let be the root of the tangent line
( ) ( )
 Since the slope of ( ) ( ) , hence we have ( )

 Similarly, successive approximate roots are given by Newton – Raphson iteration;

( )
( )

 Condition of convergence Theorem:

Assume that is continuous on [a, b] and there exists a number , - , where ( ) .
If ( ) , then there exists a such that the sequence * + defined by the iteration
( )
( ) for
( )

will converge to for any initial approximation , -.

 Termination criteria for a given tolerance of error : Use the value | ( )| as stopping
criterion for Newton’s method.

Example 1: Use the Newton - Raphson method to find a root of the equation with
initial value
Solution: ( )
( ) ,
( )
, n = 0, 1, 2 . . .
( )

Page 15 of 16
 Numerical Methods [CEng2112] – Chapter 2: Solving non-linear equations

( )

is the approximate root in the 2nd iteration.

Example 2: Find the positive root of ( ) using Newton – Raphson method taking
= 5 up to 3rd iteration.

 nth root algorithm:- The principal nth root √ , , of a positive real number A, is the positive
real solution of the equation

( )

There is a very fast-converging nth root algorithm for finding √ i.e Newton's method;

The general iteration scheme of Newton's method to solve ( ) is:

1. Make an initial guess x0

( )
2. Set ( )
3. Repeat step 2 until the desired precision is reached.

The nth root problem can be viewed as searching for a zero of the function ( )

So the derivative is ( ) and the iteration rule is

( )
( )

,( ) - leading to the general nth root algorithm.

Example: Find √ using Newton – Raphson method taking .

Page 16 of 16