Errors Analysis and Basic Definitions

in Numerical Analysis

Lecture Notes

Mohammad Sabawi

Department of Mathematics
College of Education for Women
Tikrit University

Email: [email protected]

1
Contents

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1 Introduction 4
1.1 Numerical Analysis: An Introduction . . . . . . . . . . . . . . . . . 4
1.2 Numbers Representation in Computer . . . . . . . . . . . . . . . . 5
1.2.1 Floating-Point Numbers . . . . . . . . . . . . . . . . . . . . 6
1.3 Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3.1 Error Analysis . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3.2 Sources of Error in Numerical Computations . . . . . . . . 7
1.3.3 Absolute and Relative Errors . . . . . . . . . . . . . . . . . 8
1.3.4 Roundoff and Truncation Errors . . . . . . . . . . . . . . . 9
1.4 Stable and Unstable Computations: Conditioning . . . . . . . . . . 10
1.5 Convergence and Order of Approximation . . . . . . . . . . . . . . 12
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2
 Preface
The aim of these class notes is to cover the necessary materials in a stan-
dard numerical analysis course and it s not intended to add to the plethora
of Numerical Analysis texts. We tried our best to write these notes in con-
cise, clear and accessible way, to make them more attractive to the readers.
These lecture notes cover the basic and fundamental concepts and principles
in numerical analysis and it is not a comprehensive introduction to numer-
ical analysis. We emphasise in these notes on the mathematical principles
via explaining them by the aid of numerical software MATLAB. The pre-
requisite material for this course are a course in Calculus, Linear Algebra
and Differential Equations. A basic knowledge in MATLAB is helpful but
it is not necessary. There is a glut of numerical software nowadays, among
these we chose to use MATLAB because of its wide capabilities in scientific
computing.These notes consist of ten chapters and each chapter ends with a
set of exercises address the topics covered in each chapter.

3
Chapter 1

Introduction

1.1 Numerical Analysis: An Introduction

Numerical analysis is a branch of mathematics studies the methods and al-
gorithms which used for solving a variety of problems in different areas of
todays life such as mathematics, physics, engineering, medicine and social
and life sciences. The main objective of numerical analysis is investiga-
tion finding new mathematical approaches for approximating the underlying
problems, and also development of the current algorithms and numerical
schemes to make them more efficient and reliable. The advent of computers
revolutionise numerical analysis and nowadays with parallel and super com-
puters the numerical computations became more easier compared with the
past where solving simple problems take a long time, much effort and require
hard work. In principle, numerical analysis mainly focuses on the ideas of
stability, convergence, accuracy, consistency and error analysis. In the lit-
erature numerical analysis also known as scientific computing, scientific
computation, numerics, computational mathematics and numerical
mathematics. Numerical analysis can be divided into the following fields:

1. Numerical Solutions of Linear Algebraic Equations.

2. Numerical Solutions of Nonlinear Algebraic Equations.

3. Interpolation and Extrapolation.

4. Approximation Theory and Curve Fitting.

5. Numerical Differentiation.

6. Numerical Integration.

4
 7. Numerical Optimisation.

8. Numerical Solutions of Eigenvalue Problems.

9. Numerical Solutions of Ordinary Differential Equations.

10. Numerical Solutions of Partial Differential Equations.

11. Numerical Solutions of Integral Equations.

12. Numerical Modelling.

13. Numerical Simulation.

Numerical analysis is dated back to the Babylonians works in approxi-

mating the square root of 2. During this long journey of evolution many
many scientists contributed to its development and progress among these we
just name a few such as Lagrange, Gauss, Newton, Euler, Legendre
and Simpson.

1.2 Numbers Representation in Computer

Human beings do arithmetic in their daily life using the decimal (base 10)
number system. Nowadays, most computers use binary (base 2) number sys-
tem. We enter the information to computers using the decimal system but
computers transform them to the binary system by using the machine lan-
guage.

Definition 1 (Scientific Notation). Let k be a real number, then k can be

written in the following form

k = m × 10n ,

where m is any real number and the exponent n is an integer. This notation is
called the scientific notation or scientific form and sometimes referred
to as standard form.

Example 1. Write the following numbers in scientific notation:

1. 0.00000834.

2. 25.45879.

3. 3400000.

5
 4. 33.
5. 2, 300, 000, 000.
Solution:
1. 0.00000834 = 8.34 × 10−6 .
2. 25.45879 = 2.545879 × 101 .
3. 3400000 = 3.4 × 106 .
4. 33 = 3.3 × 101 .
5. 2.3 × 109 .

1.2.1 Floating-Point Numbers

In the decimal system any real number a 6= 0 can be written in the decimal
normalised floating-point form in the following way

a = ±0.d1 d2 d3 · · · dk dk+1 dk+2 · · · × 10n , 1 ≤ d1 ≤ 9, 0 ≤ di ≤ 9,

for each i = 2, · · · , and n is an integer called the exponent (n can be
positive, negative or zero). In computers we use a finite number of digits in
representing the numbers and we obtain the following form

b = ±0.d1 d2 d3 · · · dk × 10n , 1 ≤ d1 ≤ 9, 0 ≤ di ≤ 9,
for each i = 2, · · · , k. These numbers are called k-digit decimal ma-
chine numbers.

Also, the normalised floating-point decimal representation of the number

a 6= 0 can be written in other way as
1
a = ±r × 10n , (
≤ r < 1),
10
the number r is called the normalised mantissa.

The floating-point representation in binary number system can be defined

by the same way as in the decimal number system. If a 6= 0, it can be
represented as
1
a = ±p × 2m , ( ≤ p < 1),
2
where p = (0.b1 b2 b3 · · · )2 , b1 = 1.

6
1.3 Errors
Occurrence of error is unavoidable in the field of scientific computing. In-
stead, numerical analysts try to investigate the possible and best ways to
minimise the error. The study of the error and how to estimate and min-
imise it are the fundamental issues in error analysis.

1.3.1 Error Analysis

In numerical analysis we approximate the exact solution of the problem by
using numerical method and consequently an error is committed. The numer-
ical error is the difference between the exact solution and the approximate
solution.

Definition 2 (Numerical Error). Let x be the exact solution of the underlying

problem and x∗ its approximate solution, then the error (denoted by e) in
solving this problem is

e = x − x∗ .

1.3.2 Sources of Error in Numerical Computations

• Blunders (Gross Errors) These errors also called humans errors,
and are caused by humans mistakes and oversight and can be minimised
by taking care during scientific investigations. These errors will add to
the total error of the underlying problem and can significantly affect
the accuracy of solution.

• Modelling Errors These errors arise during the modelling process

when scientists ignore effecting factors in the model to simplify the
problem. Also, these errors known as formulation errors.

• Data Uncertainty These errors are due to the uncertainty of the

physical problem data and also known as data errors.

• Discretisation Errors Computers represent a function of continuous

variable by a number of discrete values. Also, scientists approximate
and replace complex continuous problems by discrete ones and this
results in discretisation errors.

7
1.3.3 Absolute and Relative Errors
Definition 3 (Absolute Error). The absolute error ê of the error e is defined
as the absolute value of the error e

ê = |x − x∗ |.

Definition 4 (Relative Error). The relative error ẽ of the error e is defined

as the ratio between the absolute error ê and the absolute value of the exact
solution x

ê |x − x∗ |
ẽ = = , x 6= 0.
|x| |x|
Example 2. Let x = 3.141592653589793 is the value of the constant ratio π
correct to 15 decimal places and x∗ = 3.14159265 be an approximation of x.
Compute the following quantities:

a. The error.

b. The absolute error.

c. The relative error.

Solution:

a. The error

e = x − x∗ = 3.141592653589793 − 3.14159265 = 3.589792907376932e − 09

= 3.589792907376932 × 10−9 = 0.000000003589792907376932.

b. The absolute error

ê = |x−x∗ | = |3.141592653589793−3.14159265| = 3.589792907376932e−09.

c. The relative error

ê |x − x∗ | 3.141592653589793 − 3.14159265
ẽ = = =
|x| |x| 3.141592653589793
3.589792907376932e − 09
= = 1.142666571770530e − 09.
3.141592653589793

8
1.3.4 Roundoff and Truncation Errors
Computers represent numbers in finite number of digits and hence some
quantities cannot be represented exactly. The error caused by replacing a
number a by its closest machine number is called the roundoff error and
the process is called correct rounding.

Truncation errors also sometimes called chopping errors are occurred

when chopping an infinite number and replaced it by a finite number or by
truncated a series after finite number of terms.

Example 3. Approximate the following decimal numbers to three digits by

using rounding and chopping (truncation) rules:

1. x1 = 1.34579.

2. x2 = 1.34679.

3. x3 = 1.34479.

4. x4 = 3.34379.

5. x5 = 2.34579.

Solution:

(i) Rounding:

(a) x1 = 1.35.
(b) x2 = 1.35.
(c) x3 = 1.34.
(d) x4 = 3.34.
(e) x5 = 2.35.

(ii) Chopping:

(a) x1 = 1.34.
(b) x2 = 1.34.
(c) x3 = 1.34.
(d) x4 = 3.34.
(e) x5 = 2.34.

9
1.4 Stable and Unstable Computations: Con-
ditioning
Stability is one of the most important characteristics in any efficient and
robust numerical scheme.

Definition 5 (Numerical Stability). The numerical algorithm is called sta-

ble if the final result is relatively not affected by the perturbations during
computation process. Otherwise it is called unstable.

The stability notion is analogous and closely related to the notion of

conditioning.

Definition 6 (Conditioning). Conditioning is a measure of how sensitive

the output to small changes in the input data. In literature conditioning is
also called sensitivity.

• The problem is called well-conditioned or insensitive if small changes

in the input data lead to small changes in the output data.

• The problem is called ill-conditioned or sensitive if small changes

in the input data lead to big changes in the output data.

Definition 7 (Condition Number of a Function). If f is a differentiable

function at x in its domain then the condition number of f at x is

|xf 0 (x)|
Cond(f (x)) = , f (x) 6= 0.
|f (x)|
Note: Condition number of a function f at x in its domain sometimes
denoted by Cf (x).

Definition 8 (Condition Number of a Matrix). If A is a non-singular n × m

matrix, the condition number of A is defined by

cond(A) = kAkkA−1 k,
where
kAxk
kAk = max ,
x6=0 kxk
and x is a m × 1 column vector.

Remark 1. Note that:

1. The problem is ill-posed or sensitive if cond  1.

10
 2. The problem is well-posed or insensitive if cond < 1.
√
Example 4. Find the condition number of the function f (x) = x.

Solution:
√ 1
f (x) = x =⇒ f 0 (x) = √ , x 6= 0,
2 x
implies that
|xf 0 (x)| | 2√x x | 1
cond(f (x)) = = √ = .
|f (x)| | x| 2
This indicates that the small changes in the input date lead to changes in the
output data of half size the changes in the input data.
Example 5. Let  
1 −1 1
A =  1 0.5 3  ,
0.1 1 0.3
the inverse of A can be computed by using MATLAB command inv(A) to
obtain  
4.7500 −2.1667 5.8333
A−1 =  0.5000 −0.3333 1.6667  .
−3.2500 1.8333 −4.1667
Also, the condition number of A and its inverse can be computed using MAT-
LAB commands cond(A) and cond(inv(A)) to have cond(A) = 37.8704
and cond(A−1 ) = 37.8704. We notice that the matrix A and its inverse have
the same condition number.
Definition 9 (Well-Posed Problem). The problem is well-posed if satisfies
the following three conditions:
a. The solution exists.
b. The solution is unique.
c. The solution depends continuously on problem data.
Otherwise, the problem is called ill-posed.
Definition 10 (Accuracy). It is a measure of closeness of the approximate
solution to the exact solution.
Definition 11 (Precision). It is a measure of closeness of the two or more
measurements to each other.
Remark 2. Note that the accuracy and precision are different and they are
not related. The problem maybe very accurate but imprecise and vice versa.

11
1.5 Convergence and Order of Approxima-
tion
Convergence of the numerical solution to the analytical solution is one of the
important characteristic in any good and reliable numerical scheme.

Definition 12 (Convergence of a Sequence). Let {an } be an infinite sequence

of real numbers. This sequence is said to be convergent to a real number a
(has a limit at a) if, for any  > 0 there exists a positive integer N () such
that

|an − a| < , whenever n > N (),

Otherwise it is called a divergent sequence, a is called the limit of the
sequence an . Other commonly used notations for convergence are:

lim an = a or an → a as n, or lim (an − a) = 0,

n→∞ n→∞

this means that the sequence {an } converges to a otherwise it diverges.

Definition 13 (Order of Convergence). Let the sequence {an } converges to

a and set en = an − a for any n > 0. If two positive constants M 6= 0 and
q > 0 exist, such that

|an+1 − a| |en+1 |
lim q
= lim = M,
n→∞ |an − a| n→∞ |en |q

then the sequence {an } is to be convergent to a with the order of conver-

gence q.

If q = 1, the convergence is called linear.

If q = 2, the convergence is called quadratic.
If q = 3, the convergence is called cubic.

Note that the convergence gets more rapid as q gets larger and larger.

Example 6. Consider the sequence { n1 }, where n is a positive integer. Ob-

serve that n1 → 0 as n → ∞, it follows that

1
lim = 0.
n→∞ n

12
Definition 14 (Order of Approximation O(hn )). The function f (h) is said
to be big Oh of the function g(h), if two real constants c, and C exist such
that
|f (h)| ≤ C|g(h)| wheneverh < c,
and denoted by f (h) = O(g(h)). The order of approximation is used to
determine the rate at which a function converges.

Example 7. Consider the functions f (x) = x + 1 and g(x) = x2 , where

x ≥ 1. Observe that x ≤ x2 and 1 ≤ x2 for x ≥ 1, hence f (x) = x + 1 ≤
2x2 = 2g(x) for x ≥ 1. Consequently, f (x) = O(g(x)).

13
 Exercises

Exercise 1. Write the following numbers in scientific form:

1. 23.123.

2. 30, 000, 000.

3. 0.000001573.

4. 39776444.

5. −345.386443.

6. −23000000.

Exercise 2. Evaluate error, absolute error and relative error of the following
values and their approximations:

1. x = 1, 000, 000, x∗ = 999, 999

2. y = 0.00012887765, y ∗ = 0.00012897766

3. z = 9776.96544, z ∗ = 9775.66544

Exercise 3. Approximate the following numbers to four digits using rounding

and chopping:

1. 1.98876.

2. 33.87654.

3. 8.98879.

4. 2.88778.

Exercise 4. Compute the condition number of the following functions:

1. f (x) = cos(x).

2. f (x) = cos−1 (x).

14