Chapter 3

MATH 6370

Annalisa Quaini
[email protected]
Office : PGH 662
Lecture : TuTh 2:30 PM - 4:00 PM in SEC 201
Office hours : Th 1:00 PM - 2:30 PM or BY APPOINTMENT
* No class next week!

* The final is Dec. 9th at 2PM in SEC 201.

* The final is on EVERYTHING we've covered in class.

A. Quaini, UH MATH 6370 1 / 12

 Chapter 6

Convergence Results for Some Fixed-point Methods

The theorems in the previous lecture provide a theoretical tool for

analyzing iterative methods for rootfinding. In this lecture, we are
going to interpret some methods that we have already seen as
fixed point methods.

c 2020, A. Quaini, UH MATH 6370 2/5

 Chapter 6

Chord method

The chord method:

b−a
x (k+1) = x (k) − f (x (k) ) ∀k ≥ 0
f (b) − f (a)

can be seen as a fixed point method with

b−a
φchord (x) = x − f (x).
f (b) − f (a)

We have:
b−a
φ0chord (x) = 1 − f 0 (x).
f (b) − f (a)
Thus, if we have f 0 (α) = 0 we get φ0chord (α) = 1, which means
that the method is not guaranteed to converge.

c 2020, A. Quaini, UH MATH 6370 3/5

 Chapter 6

Chord method

The condition
|φ0chord (α)| < 1
leads to
b−a
1− f 0 (α) < 1,
f (b) − f (a)
that is

c 2020, A. Quaini, UH MATH 6370 4/5

 Chapter 6

Chord method

We get
b−a
0< f 0 (α) < 2.
f (b) − f (a)
This condition is equivalent to requiring that
b−a
f (b) − f (a)

has the same sign as f 0 (α) and that interval [a, b] is s.t.:

f (b) − f (a)
b−a<2 .
f 0 (α)

c 2020, A. Quaini, UH MATH 6370 5/5

More about the Newton method.
The Newton method is a fixed point method: x(k+1) = φ(x(k) ) for the function

f (x)
φ(x) = x − .
f ′ (x)

Let α be a zero of f , i.e. such that f (α) = 0. Note that φ′ (α) = 0, when
f ′ (α) 6= 0. Indeed,

′ [f ′ (x)]2 − f (x)f ′′ (x)

φ (x) = 1 − .
[f ′ (x)]2

Nonlinear equations – p. 32/53

 Chapter 6

Newton method

c 2020, A. Quaini, UH MATH 6370 6 / 12

Theorem 1. If f is twice differentiable, f (α) = 0 and f ′ (α) 6= 0, then there
exists δ > 0 such that, if | x(0) − α |≤ δ, the sequence defined by the Newton
method converges to α.
Moreover, the convergence is quadratic; more precisely

x(k+1) − α f ′′ (α)
lim = .
k→∞ (x(k) − α)2 2f ′ (α)

Proof. The property of convergence comes from the Proposition 2, while the
quadratic convergence is a consequence of the Proposition 3, because
′ φ′′ (α) f ′′ (α)
φ (α) = 0 and 2
= 2f ′ (α)
.

Definition 2. Let α be a zero of f . α is said to have multiplicity m, m ∈ N,

if f (α) = . . . = f (m−1) (α) = 0 and f (m) (α) 6= 0.
A zero that has multiplicity m = 1 is called simple zero.

Nonlinear equations – p. 33/53

Remark 2. If f ′ (α) = 0, the convergence of the Newton method is linear,
not quadratic. We can use the modified Newton method:

(k+1) (k) f (x(k) )

x =x − m ′ (k) , k = 0, 1, 2 . . . . (7)
f (x )

where m is the multiplicity of α.

If the multiplicity m of α is unknown, there are other methods, the adaptive

methods, which can recover the quadratic order of convergence.

Nonlinear equations – p. 34/53

A stopping criterion for Newton
When to stop the Newton method? A good stopping criterion is the control of
the increment : the iterations is completed when

|x(k+1) − x(k) | < ǫ (8)

where ǫ is a fixed tolerance.

Nonlinear equations – p. 35/53

Indeed, if we denote e(k) = α − x(k) is the error of the iteration k, we have

e(k+1) = α − x(k+1) = φ(α) − φ(x(k) ) = φ′ (ξ (k) )e(k) ,

where ξ (k) is between x(k) and α, and

 
(k+1) (k) (k) (k+1) (k) (k+1) (k)
x −x = α−x −α+x =e −e ′
= 1 − φ (ξ ) e(k) . (9)

Assuming that if k is large enough, we have φ′ (ξ (k) ) ≈ φ′ (α) and knowing that
the Newton method for φ′ (α) = 0, if α is a simple zero, we find the estimation

|e(k) | ≈ |x(k+1) − x(k) |.

The error that we commit when we adopt the criterion (8) is smaller than the
fixed tolerance.

Nonlinear equations – p. 36/53

Stopping criteria: the general case
In general, for all discussed methods, we can use two different stopping
criteria: the iterations is completed when

|x(k+1) − x(k) | < ǫ (control of the increment),

or
|f (x(k) )| < ǫ (control of the residual),
where ǫ is a fixed tolerance.

Nonlinear equations – p. 37/53

Using fixed point iterations we obtain the following estimation:

1
e(k) ≈ (x (k+1)
− x (k)
).
(1 − φ′ (α))

We can plot the graph of (1−φ1′ (α)) and comment on the relevance of the
stopping criterium based on the increment:

• if φ′(α) is near to 1 the test is not

satisfactory
• for methods of order 2 (φ′(α) = 0
), the criterium is optimal,
• if −1 < φ′(α) < 0 the criterium is
still all right.

Nonlinear equations – p. 38/53

Stopping Criteria
The stopping criterium based on the control on the residual |f (x(k) )| < ǫ is
satisfactory only if |f ′ | ≃ 1 near the root α. Otherwise it is too strong (if
|f ′ | ≫ 1) or too weak (if |f ′ | ≪ 1):

y f y

f (x(k) )
f

α x(k) α x(k) f (x(k) )

x x
e(k) e(k)
Two cases where the residual is a bad estimator of the error: |f ′ (x)| ≫ 1
(left), |f ′ (x)| ≪ 1 (right)) with x near to α

Nonlinear equations – p. 39/53

Applications
Example. 1 (suite) We draw the graph of f (I) = M − v 1+I I
[(1 + I) n
− 1]
on the interval [0.010.3] with M = 6000, v = 1000 and n = 5:
>> f=@(x) 6000-1000*(1+x).*((1+x).^5 - 1)./x;
>> I = [0.01:0.001:0.3];
>> grid on;plot(I,feval(f,x));
1000
α
0

−1000

−2000

−3000

−4000

−5000

−6000
0 0.05 0.1 0.15 0.2 0.25 0.3

Nonlinear equations – p. 40/53

 The root of f is between 0.05 and 0.1.
We can apply the bisection method on the interval [0.05, 0.1] with a tolerance
10−5
>> [zero,res,niter]=bisection(f,0.05,0.1,1e-5,1000);
The approximate solution after 12 iterations is x̄ = 0.061407470703125.

We can apply the Newton method with initial guess x(0) = 0.05
>> df=@(x) 1000*((1+x).^5.*(1-5*x) - 1)./(x.^2);
>> [zero,res,niter]=newton(f,df,.05,1e-5,1000);
The result is approximately the same, but we need only 3 iterations
The interest rate is 6.14%.

Nonlinear equations – p. 41/53

Example. 2 (cont) We would like to plot the angle θ as function of ω for
0 ≤ ω ≤ π with a1 = 10 cm, a2 = 13 cm, a3 = 8 cm, a4 = 10 cm.
For each ω, we have to solve the nonlinear poblem

a1 a1 a21 + a22 − a23 + a24

f (θ) = cos(ω) − cos(θ) − cos(ω − θ) + = 0. (10)
a2 a4 2a2 a4

To start with, we plot the graph of f (θ) for ω = π/3:

>> F = @(x, a1, a2, a3, a4, omega) ...
(a1/a2)*cos(omega) - (a1/a4)*cos(x) - cos(omega-x) ...
+ ( (a1.^2 + a2.^2 - a3.^2 + a4.^2) / (2*a2*a4) );
>> a1=10; a2=13; a3=8; a4=10; omega=pi/3;
>> f = @(x) F(x,a1,a2,a3,a4,omega);
>> x = [-pi:0.01:pi];
>> plot( x, f(x) ); grid on;

Nonlinear equations – p. 42/53

 Function f with ω = π/3.
3.5

2.5
f (θ)

1.5

0.5

−0.5
−4 −3 −2 −1 0 1 2 3 4
θ
We have two roots, which means that we have two possible configurations.

Nonlinear equations – p. 43/53

Example. 3 (suite) We consider the carbon diox-
ide (CO2 ), for which a = 0.401 P a m6 and b =
42.7 · 10−6 m3 .
We search the volume occupied by N = 1000
molecules of CO2 in temperature T = 300 K and
pressure p = 3.5 · 107 P a. We know that the Boltz-
mann constant is k = 1.3806503 · 10−23 Joule K −1 .

Nonlinear equations – p. 46/53

 We draw the graph of the function
"  2 #
N
f (V ) = p + a (V − N b) − kN T
V

for V > 0 . We do not consider V < 0 (it does not have physical meaning),
because V is the volume of gas.
We use the commands in Matlab/Octave:

>> a=0.401; b=42.7e-6; p=3.5e7; T=300; N=1000; k=1.3806503e-23;

>> f = @(x,p,T,a,b,N,k)(p+a*((N./x).^2)).*(x-N*b)-k*N*T;
>> x=[0.03:0.001:0.1];
>> plot(x,f(x,p,T,a,b,N,k))
>> grid on

Nonlinear equations – p. 47/53

We obtain the graph of the function f (V ):
6
x 10
6

0
f(V)

−2

−4

racine de la fonction f(V)

−6

−8
0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1
V

Nonlinear equations – p. 48/53

 We see that there is a zero for 0.03 < V < 0.1. If we apply the bisection
method on the interval [0.03, 0.1] with a tolerance 10−12 :

[zero,res,niter]=bisection(f,0.03,0.1,1e-12,1000,p,T,a,b,N,k);

then we find, after 36 iterations, the value V = 0.0427.

If we use the Newton method with the same tolerance, starting form the
initial point x(0) = 0.03,

>> df = @(x,p,T,a,b,N,k) -2*a*N^2/(x^3)*(x-N*b)+(p+a*((N./x).^2));

>> [zero,res,niter]=newton(f,df,0.03,1e-12,1000,p,T,a,b,N,k);

then we find the same solution after 6 iterations.

The conclusion is that the volume V occupied by the gas is 0.0427 m3 .

Nonlinear equations – p. 49/53

 Chapter 6

Post-processing Techniques for Iterative Methods

We will see an algorithm that aims at accelerating the convergence

of iterative methods for finding the roots of a function.

Case under consideration: linearly convergence fixed point

methods (p = 1).

Denote by λ an approximation of φ0 (α). We have seen that

x (k+1) − α
lim = φ0 (α).
k→∞ x (k) − α

Thus:

c 2020, A. Quaini, UH MATH 6370 6 / 10

 Chapter 6

Post-processing Techniques for Iterative Methods

Goal: finding a simple way of computing λ s.t. the convergence

of the sequence {x (k) } to α is accelerated.

c 2020, A. Quaini, UH MATH 6370 7 / 10

 Chapter 6

Aitken’s Method

Set
x (k) − x (k−1)
λ(k) = .
x (k−1) − x (k−2)
Let’s check that
lim λ(k) = φ0 (α).
k→∞

c 2020, A. Quaini, UH MATH 6370 8 / 10

 Chapter 6

Aitken’s Method

c 2020, A. Quaini, UH MATH 6370 9 / 10

 Chapter 6

Aitken’s Method

By plugging λ = λ(k) into:

λ
α ' x (k) + (x (k) − x (k−1) )
1−λ

we obtain a new way of approximating α, denoted by x̂ (k) :

c 2020, A. Quaini, UH MATH 6370 10 / 10

 Chapter 6

Aitken’s Method

c 2020, A. Quaini, UH MATH 6370 11 / 10