Azərbaycan Dövlət Neft

və Sənaye Universiteti
Numerical
Methods II
Curve Fitting:

Introduction
We pose three problems concerning the
representation of functions to give an indication
of the current subject matter.

Suppose that we have a table of numerical

values of a
function:
i. Is it possible to find a simple and convenient
formula that reproduces the given points
exactly?
ii. The second problem is similar, but it is
assumed that the given table of numerical
values is contaminated by errors, as might
occur if the values came from a physical
experiment.

Now we ask for a formula that represents the

data (approximately) and, if possible, filters
out the errors.
iii. As a third problem, a function is given,
perhaps in the form of a computer
procedure, but it is an expensive function to
evaluate.
In this case, we ask for another function that
is simpler to evaluate and produces a
reasonable approximation to .

Sometimes in this problem, we want to

approximate with full machine precision.
In all of these problems, a simple function can
be obtained that represents or approximates the
given table or function .
All of these applications are known as .

The representation can always be taken to be a

polynomial, although many other types of
simple functions can also be used.
Once a simple function has been obtained, it
can be used in place of in many situations.

For example, the integral of could be estimated

by the integral of , and the latter should
generally be easier to evaluate.

In many situations, a polynomial solution to the

problems outlined above will be unsatisfactory
from a practical point of view, and other classes
of functions must be considered.
The obvious way in which a polynomial can fail
as a practical solution to one of the preceding
problems is that its degree may be
unreasonably high.

For instance, if the table considered contains

entries, a polynomial of degree may be
required to represent it.
Polynomials also may have the surprising
defect of being highly oscillatory.
If the table is precisely represented by a
polynomial , then for .

For points other than the given , however, may

be a very poor representation of the function
from which the table arose.

The example involving the illustrates this

phenomenon.
In this course, one other class of versatile
functions is discussed: the .
There are two general approaches for curve
fitting that are distinguished from each other on
the basis of the amount of error associated with
the data.

i. First where the data exhibits a significant

degree of error or , the strategy is to derive a
single curve that represents the general trend
of the data.
Because any individual data point may be
incorrect, we make no effort to intersect every
point.

Rather the curve is designed to follow the

pattern of the points taken as a group.

One approach of this nature is called .

ii. Second where the data is known to be very
precise the basic approach is to fit a curve or
a series of curves that pass directly through
each of the points.

Such data usually originates from tables.

The estimation of values between well-
known discrete points is called interpolation.
Two types of applications are generally
encountered when fitting experimental data:
• ;
• .

 represents the process of using the pattern of

the data to make predictions.

Trend analysis may be used to predict or

forecast values of the dependent variable.
This can involve extrapolation beyond the
limits of the observed data or interpolation
within the range of the data.

For cases where the experimental data are

measured with high precision, we might
utilize interpolating polynomials.

Imprecise data are often analyzed with least-

squares regression.
 A second application of experimental curve
fitting is .

Here, an existing mathematical model is

compared with measured data.

If the model coefficients are unknown it may

be necessary to determine values that best fit
the observed data.
On the other hand if estimates of the model
coefficients are already available it may be
appropriate to compare predicted values of
the model with observed values to test the
adequacy of the model.

Often alternative models are compared and

the one is selected on the basis of empirical
observations.
In addition to the above applications, is
important in other numerical methods such as
definite integration and approximate solution of
differential equations.

That is, techniques can be used to derive

simple functions to approximate complicated
functions.
 
The prerequisite mathematical background for
is found in the material on Taylor series
expansions and finite divided differences
introduced in Numerical Methods I.

requires additional information from the field

of statistics – mean, standard deviation, residual
sum of the squares, normal distribution, and
confidence intervals.
 Curve Fitting:
Noncomputer Methods
for Curve Fitting
The simplest method for fitting a curve to data
is to plot the points and then sketch a line that
visually conforms to the data.

Although this is a valid option when quick

estimates are required, the results are
dependent on the subjective viewpoint of the
person sketching the curve.
For example the next
shows sketches
developed from the same
set of data by three
persons.

Three attempts to fit a “best”

curve through five data points.
Least-squares regression;
Linear interpolation;
Curvilinear interpolation.
• The first person did not attempt to connect
the points, but rather, characterized the
general upward trend of the data with a
straight line. (see )

• The second person used straight-line

segments or linear interpolation to connect
the points. (see )

This is a very common practice in science and

engineering.
If the values are truly close to being linear or
are spaced closely, such an approximation
provides estimates that are adequate for
many scientific and engineering calculations.

However, where the underlying relationship

is highly curvilinear or these data are widely
spaced, significant errors can be introduced
by such linear interpolation.
• The third person used curves to try to capture
the meanderings suggested by the data. (see )

A fourth or fifth person would likely develop

alternative fits.

Obviously, our goal here is to develop

systematic and objective methods for the
purpose of deriving such curves.
Curve Fitting:

Interpolation
Scientists and engineers frequently have
occasion to estimate intermediate values
between precise data points.

The most common method used for this

purpose is .

Recall that the general formula for an order

th

polynomial is
For data points there is one and only one
polynomial of order that passes through all the
points.

• For example there is only one straight line

(i.e. a first-order polynomial) that connects
points.

• Similarly only one parabola connects a set of

points.
 lies in determining the unique order
th

polynomial that fits data points.

This polynomial then provides a formula to

compute intermediate values.
In this presentation we will describe two
alternatives that are well-suited for computer
implementation: the Newton and the Lagrange
polynomials.
 Curve Fitting:

Newtons Interpolation
As stated above, there are a variety of
alternative forms for expressing an
interpolating polynomial.

is among the most popular and useful forms

for expressing an interpolating polynomial.

Before presenting the general equation we will

introduce the first and secondorder versions
because of their simple visual interpretation.
 
The simplest form of interpolation is to connect
two data points with a straight line.

This technique, called

is depicted graphically
in the next .
Let’s denote this linear
function by .
Using similar triangles we can write:

It can be rearranged to yield

which is a .
Notice that besides representing the slope of
the line connecting the points the term

is a finite-divided-difference approximation of
the first derivative.

In general the smaller the interval between the

data points the better the approximation.
This is due to the fact that as the interval
decreases a continuous function will be better
approximated by a straight line.

. Estimate the natural logarithm of using linear

interpolation.
First perform the computation by interpolating
between and .
Then repeat the procedure but use a smaller
interval from to .
.
We use Eq. and a linear interpolation for from
to to give

ln 𝟔 −ln 𝟏
ℓ ( 𝟐 )=ln 𝟏+ (𝟐 − 𝟏 )
𝟓
ℓ ( 𝟐 )= 𝟎 .𝟑𝟓𝟖𝟑𝟓𝟏𝟗
.
Note that the true value of is .

The last computation represents an error of

Using the smaller interval from to yields

.
Thus, using the shorter interval reduces the
percent relative error to .

Both
interpolations are
shown in along
with the true
function.
 
A strategy for improving the estimate is to
introduce some curvature into the line
connecting the points.

If three data points are available this can be

accomplished with a second-order polynomial
(also called a or a ).
A particularly convenient form for this purpose
is

Note that although Eq. might seem to differ

from the general polynomial [Eq. ] the two
equations are equivalent.

This can be shown by multiplying the terms in

Eq. .
This produces

or, collecting terms,

where
Thus, equations and are alternative, equivalent
formulations of the unique second-order
polynomial joining the three points.

A simple procedure can be used to determine

the values of the coefficients: , , and .

For Eq. with can be used to compute

Equation can be substituted into Eq. which
then can be evaluated at for

Finally Eqs. and can be substituted into Eq.

which can be evaluated at and solved (after
some algebraic manipulations) for
Notice that, as was the case with linear
interpolation, still represents the slope of the
line connecting points and .

Thus, the first two terms of Eq. are equivalent to

linear interpolation from to as specified
previously in Eq. .

The last term introduces the second-order

curvature into the formula.
If we examine the form of the coefficient we
will find it similar to the finite-divided-
difference approximation of the second
derivative.

Thus, Eq. is beginning to manifest a structure

that is very similar to the .
. Fit a second-order polynomial to the three
points used in :

Use the polynomial to evaluate .

. Applying Eq. yields

. Equation yields

and Eq. gives

. Substituting these values into Eq. yields the
quadratic formula

which can be evaluated at for

This calculation represents a relative error of

. Thus, the curvature introduced by the
quadratic formula (see ) improves the
interpolation compared with the result obtained
using straight
lines in
.
 
The preceding analysis can be generalized to fit
an thorder polynomial to data points.

The thorder polynomial is

As was done previously with the linear and

quadratic interpolations data points can be used
to evaluate the coefficients .
For an th
order polynomial data points are
required:

We use these data points and the following

equations to evaluate the coefficients:
Here the bracketed function evaluations are
finite divided differences:

The second finite divided difference, which

represents the difference of two first divided
differences is expressed generally as
Similarly the th finite divided difference is

These differences can be used to evaluate the

coefficients in Eqs. through which can then be
substituted into Eq. , i.e. polynomial , to yield
the interpolating polynomial.
This is called .
Remark
It should be noted that it is not necessary that
the data points used in Eq. be equally spaced or
that the abscissa values necessarily be in
ascending order.

Notice also how Eqs. through are recursive —

i.e. higher-order differences are computed by
taking differences of lower-order differences.
. In data points at , and were used to estimate
with a parabola.

Now adding a fourth point

estimate with a third-order Newton’s

interpolating polynomial.
The third-order polynomial, Eq. with , is

The first divided differences for the problem are

[Eq. ] as follows:
 𝟏.𝟔𝟎𝟗𝟒𝟑𝟖−𝟏.𝟕𝟗𝟏𝟕𝟓𝟗
𝐟 [ 𝒙𝟑 ; 𝒙𝟐 ]= =𝟎.𝟏𝟖𝟐𝟑𝟐𝟏
𝟓−𝟔
The second divided differences are [Eq. ]
The third divided difference is [Eq. with ]
The results for , , , and represent the coefficients
, , , and , respectively, of Eq. .

Therefore, Eq. yields

This can be used to evaluate , which represents
a relative error of .

The complete
cubic
polynomial
is shown in .
 

Notice that the structure of Eq. is similar to the

Taylor series expansion: terms are added
sequentially to capture the higher-order
behavior of the underlying function.

These terms are finite-divided-differences and

thus represent approximations of the higher-
order derivatives.
Consequently as with the Taylor series if the
true underlying function is an th
order
polynomial, order interpolating polynomial
th

based on data points will yield exact results.

Also as was the case with the Taylor series a

formulation for the truncation error can be
obtained.
Recall that the truncation error for the Taylor
series could be expressed generally as

where is somewhere in the interval to .

For an order interpolating polynomial an

th

analogous relationship for the error is

Here is somewhere in the interval containing
the unknown and the data.

For this formula to be of use the function in

question must be known and differentiable
which is not usually the case.

Fortunately, an alternative formulation is

available that does not require prior knowledge
of the function.
Rather, it uses a finite divided difference to
approximate the th derivative

where is the th finite divided difference.

Because Eq. contains the unknown it cannot be

solved for the error.
However, if an additional data point

is available Eq. can be used to estimate the

error as in
. Use Eq. to estimate the error for the second-
order polynomial interpolation of .

Use the additional data point to obtain your

results.

. Recall that in , the second-order interpolating

polynomial provided an estimate of , which
represents an error of .
. If we had not known the true value, as is most
usually the case, Eq. , along with the additional
value at , could have been used to estimate the
error, as in

where the value for the third-order finite

divided difference is as computed previously
in .
. This relationship can be evaluated at for

which is half the true error (), but of the same

order of magnitude as the true error.
From Eq. it should be clear that the error
estimate for the thorder polynomial is equivalent
to the difference between the th
order and the
order prediction:
th

In other words the increment that is added to

the th
order case to create the order case [i.e.
th

Eq. ] is interpreted as an estimate of the thorder

error.
This can be clearly seen by rearranging Eq. to
give

Note that whereas all other error estimates for

iterative approaches introduced up to this point
have been determined as a present prediction
minus a previous one, Eq. represents a future
prediction minus a present one.
This means that for a series that is converging,
the error estimate of Eq. could be less than the
true error (compare the true error of the last , i.e.
, with the approximate error, i.e. ).

This would represent a highly unattractive

quality if the error estimate were being
employed as a stopping criterion.
However, as will be described in the following
section, higher-order interpolating polynomials
are highly sensitive to data errors — i.e. they are
notoriously ill-conditioned.

When employed for interpolation, they often

yield predictions that diverge significantly from
the true value.

By “looking ahead” to sense errors, Eq. is more

sensitive to such divergence.
As such, it is more valuable for the sort of
exploratory data analysis for which Newton’s
polynomial is best-suited.
 Curve Fitting:
Computer Algorithm for
Newton’s Interpolating
Polynomial
Three properties make Newton’s interpolating
polynomials extremely attractive for computer
applications.

i. As in Eq.

higher-order versions can be developed

sequentially by adding a single term to the
next lower-order equation.
This facilitates the evaluation of several
different-order versions in the same program.

Such a capability is especially valuable when

the order of the polynomial is not known a
priori.

By adding new terms sequentially, we can

determine when a point of is reached.
That is, when addition of higher-order terms
no longer significantly improves the estimate
or in certain situations actually detracts from
it.

The error equations discussed below are

useful in devising an objective criterion for
identifying this point of diminishing terms.
ii. The finite divided differences that constitute
the coefficients of the polynomial [Eqs.
through ]

can be computed efficiently.

That is, as in Eq. and below, lower-order

differences are used to compute higher-order
differences.
 . Graphical depiction of the recursive nature of finite
divided differences.

By utilizing this previously determined

information, the coefficients can be computed
efficiently.
iii. The error estimate [Eq. ]

can be very simply incorporated into a

computer algorithm because of the sequential
way in which the prediction is built.

All the above characteristics can be exploited

and incorporated into a general algorithm for
implementing Newton’s polynomial.
. Error Estimates to Determine the
Appropriate Order of Interpolation
After incorporating the error [Eq. ], utilize the
computer algorithm and the following
information to evaluate
at :
.
The results of employing the algorithm to
obtain a solution are shown in .
.
The error estimates, along with the true error
(based on the fact that ), are depicted in .

Note that the estimated error

and the true error are similar
and that their agreement
improves as the order
increases.
.
From these results, it can be concluded that the
th
order version yields a good estimate and that
higher-order terms do not significantly enhance
the prediction.

This exercise also illustrates the importance of

the positioning and ordering of the points.
.
For example, up through the third-order
estimate, the rate of improvement is slow
because the points that are added (namely, and
) are distant and on one side of the point in
question at .

The fourth-order estimate shows a somewhat

greater improvement because the new point at
is closer to the unknown.
.
However, the most dramatic decrease in the
error is associated with the inclusion of the
th
order term using the data point at .

Not only is this point close to the unknown but

it is also positioned on the opposite side from
most of the other points.
As a consequence, the error is reduced by
almost an order of magnitude.
.
The significance of the position and sequence of
these data can also be illustrated by using the
same data to obtain an estimate for but
considering the points in a different sequence.
. shows results for the case of reversing the
order of the original data, that is, and so forth.
. Because the initial points for this case are
closer to and spaced on either side of the error
decreases much more rapidly than for the
original situation.

By the second-order term, the error has been

reduced to less than .

Other combinations could be employed to

obtain different rates of convergence.
The foregoing example illustrates the
importance of the choice of base points.

As should be intuitively obvious, the points

should be centered around and as close as
possible to the unknown.

This observation is also supported by direct

examination of the error equation [Eq. ]:
If we assume that the finite divided difference
does not vary markedly along the range of these
data, the error is proportional to the product:

Obviously, the closer the base points are to , the

smaller the magnitude of this product.
Thank you for attention!
text

text

text
. text

text

text