CGN 3421 - Computer Methods Gurley

other examples of data sets that we can fit a function to.

height of Oxygen in
dropped soil
object

time temperature
pore Profit
pressure

soil depth paid labor hours

Is a straight line suitable for each of these cases ?

No. But we’re not stuck with just straight line fits. We’ll start with straight lines, then expand the concept.

Linear curve fitting (linear regression)

Given the general form of a straight line

! ( " ) " #" # $

How can we pick the coefficients that best fits the line to the data?

First question: What makes a particular straight line a ‘good’ fit?

Why does the blue line appear to us to fit the trend better?

• Consider the distance between the data and points on the line

• Add up the length of all the red and blue verticle lines

• This is an expression of the ‘error’ between data and fitted line

• The one line that provides a minimum error is then the ‘best’
straight line

Numerical Methods Lecture 5 - Curve Fitting Techniques page 90 of 102

CGN 3421 - Computer Methods Gurley

Quantifying error in a curve fit

assumptions:

1) positive or negative error have the same value

(data point is above or below the line)
(x4,y4)
2) Weight greater errors more heavily
(x2,y2) (x4,f(x4))
we can do both of these things by squaring the distance

denote data values as (x, y) ==============>> (x2,f(x2))

denote points on the fitted line as (x, f(x))
sum the error at the four data points

$ $ $
%&& " ∑ ( '( ) " ( )% & ! ( "% ) ) # ( )$ & ! ( "$ ) )
$ $
''''''''''''''''''''''''''' # ( ) ( & ! ( " ( ) ) # ( ) ) & ! ( " ) ) )

Our fit is a straight line, so now substitute ! ( " ) " #" # $

*'+,-,'./01-2 *'+,-,'./01-2
$ $
%&& " ∑ ( ) ( & ! ( "( ) ) " ∑ ( ) ( & ( #" ( # $ ) )
("% ("%

The ‘best’ line has minimum error between line and data points

This is called the least squares approach, since we minimize the square of the error.
*'+,-,'./01-2'"'*
$
minimize %&& " ∑ ( ) ( & ( #" ( # $ ) )
("%

time to pull out the calculus... finding the minimum of a function

1) derivative describes the slope
2) slope = zero is a minimum
==> take the derivative of the error with respect to # and $ , set each to zero

*
∂%&&
----------- " & $
∂# ∑ " ( ( ) ( & #" ( & $ ) " !
("%
*
∂%&&
----------- " & $
∂$ ∑ ( ) ( & #" ( & $ ) " !
("%

Numerical Methods Lecture 5 - Curve Fitting Techniques page 91 of 102

CGN 3421 - Computer Methods Gurley

Solve for the # and $ so that the previous two equations both = 0
re-write these two equations
$
# ∑ "( # $ ∑ "( " ∑ ( "( )( )
# ∑ " ( # $3* " ∑ )(
put these into matrix form

* ∑ "(
$ " ∑ )(
$ #
∑ "( ∑ "( ∑ ( "( )( )
what’s unknown?
we have the data points ( " , ) ) for ( " %, 4445'* , so we have all the summation terms in the matrix
( (

so unknows are # and $

Good news, we already know how to solve this problem
remember Gaussian elimination ??

* ∑ "( ∑ )(
+ " , , " $ , - "
$
∑ "( ∑ "( # ∑ ( "( )( )
so
+, " -

using built in Mathcad matrix inversion, the coefficients # and $ are solved
>> X = A-1*B

Note: + , - , and , are not the same as # , $ , and "

Let’s test this with an example:

i 1 2 3 4 5 6

" 0 0.5 1.0 1.5 2.0 2.5

) 0 1.5 3.0 4.5 6.0 7.5

First we find values for all the summation terms

* " 6
$
∑ "( " 748 , ∑ )( " $$48 , ∑ "( " %(478 , ∑ "( )( " )%4$8
Now plugging into the matrix form gives us:

Numerical Methods Lecture 5 - Curve Fitting Techniques page 92 of 102

CGN 3421 - Computer Methods Gurley

6 748 $ " $$48 $ $

748 %(478 # )%4$8
Note: we are using ∑ "( , NOT ( ∑ "( )

$ " (*. 6 748 3 $$48 or use Gaussian elimination...

# 748 %(478 )%4$8

The solution is $ " ! ===> ! ( " ) " (" # !

# (
This fits the data exactly. That is, the error is zero. Usually this is not the outcome. Usually we have data
that does not exactly fit a straight line.
Here’s an example with some ‘noisy’ data

x = [0 .5 1 1.5 2 2.5], y = [-0.4326 -0.1656 3.1253 4.7877 4.8535 8.6909]

6 748 $ " $!498:( , $ " (*. 6 748 3 $!498:( , $ " & !4:78
748 %(478 # )%4689) # 748 %(478 )%4689) # (486%

so our fit is ! ( " ) " (486%'" & !4:78

Here’s a plot of the data and the curve fit:

So...what do we do when a straight line is not

suitable for the data set?

Profit

paid labor hours

Straight line will not predict diminishing returns that data shows
Curve fitting - higher order polynomials
Numerical Methods Lecture 5 - Curve Fitting Techniques page 93 of 102
CGN 3421 - Computer Methods Gurley

Example #1:

Fit a second order polynomial to the following data

i 1 2 3 4 5 6

" 0 0.5 1.0 1.5 2.0 2.5

) 0 0.25 1.0 2.25 4.0 6.25

Since the order is 2 ( / " $ ), the matrix form to solve is

$
* ∑ "( ∑ "( #! ∑ )(
∑ "( ∑ "( ∑ "(
$ ( # "
% ∑ "( )(
$
$ ( ) #$ ∑ "( )(
∑ "( ∑ "( ∑ (
"
Now plug in the given data.
Before we go on...what answers do you expect for the coefficients after looking at the data?
* " 6
∑ "( " 748 , ∑ )( " %(478
$
∑ "( " %(478 , ∑ "( )( " $94%$8
( $
∑ "( " $94%$8 ∑ "( )( " 6%4%978
)
∑ "( " 6%4%978

#!
6 748 %(478 %(478
748 %(478 $94%$8 # % " $94%$8
%(478 $94%$8 6%4%978 # 6%4%978
$

$ $
Note: we are using ∑ "( , NOT ( ∑ " ( ) . There’s a big difference

#!
6 748 %(478 %(478
using the inversion method # " (*. 748 %(478 $94%$8 3 $94%$8
%
#$ %(478 $94%$8 6%4%978 6%4%987

Numerical Methods Lecture 5 - Curve Fitting Techniques page 97 of 102

CGN 3421 - Computer Methods Gurley

or use Gaussian elimination gives us the solution to the coefficients

#!
! $
#% " ! ===> ! ( " ) " ! # !3" # %3"
#$ %
This fits the data exactly. That is, f(x) = y since y = x^2

Example #2: uncertain data

Now we’ll try some ‘noisy’ data

x = [0 .0 1 1.5 2 2.5]
y = [0.0674 -0.9156 1.6253 3.0377 3.3535 7.9409]
The resulting system to solve is:

#!
6 748 %(478 %84%!:(
# % " (*. 748 %(478 $94%$8 3 ($4$9()
#$ %(478 $94%$8 6%4%978 7%4$76

#!
& !4%9%$
giving: #% " & !4($$%
#$ %4(8(7

So our fitted second order function is:

$
! ( " ) " & !4%9%$ & !4($$%"3 # %4(8(73"

Example #3 : data with three different fits

In this example, we’re not sure which order will fit

well, so we try three different polynomial orders
Note: Linear regression, or first order curve fitting is
just the general polynomial form we just saw, where
we use j=1,

• 2nd and 6th order look similar, but 6th has a

‘squiggle to it. We may not want that...

Numerical Methods Lecture 5 - Curve Fitting Techniques page 98 of 102