Non-linear

regression
 Non-linear Linear
3.5 12

3 10
2.5 8
2
6
1.5
4
1
0.5 2

0 0
0.5 1 1.5 2 2.5 3 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5

2
 Regrssion

Linear Non-linear

Intrinsically Intrinsically
Simple multiple linear non-linear

3
Intrinsically linear Intrinsically non-linear

The models that can be transformed into

the models that can not be transformed into
linear models after applying some suitable
linear models are called intrinsically non
transformation
linear models.

4
Least square method

5
 Y-Values
A B 6

1 2 5

2 4 4

3 5 3

4 4 2

5 5 1

3 4 0
0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5

6
X Y X-X` Y-Y` (X-X`)(Y-Y`)

1 2 -2 -2 4 4

2 4 -1 0 1 0

3 5 0 1 0 0

4 4 1 0 1 0

5 5 2 1 4 2

10 8

7
 Y = b0 + b1 X

𝒃 𝟏=
∑ ( 𝑿 − 𝑿 )( 𝒀 −𝒀 ) =0.6
∑ ¿¿¿

4 = b0 + 0.6 * 3 B0 = 2.2
Using Direct Substitution to find b0:

Y = 2.2 + 0.6 X

8
Non-linear regression MODELS

Polynomial
Logistic

growth
Exponential

9
Polynomial model
1 10
……. an l

Parabola /
Logistic quadratic

𝟐
𝒂𝟎+𝒂𝟏 𝑿 +𝒂𝟐 𝑿
Exponential

11
 No. of points

(n) a0 +

() a0 + Solve to find a0, a1 and a2

Logistic

() a0 +

Exponential

12
Example: fit a second order polynomial to the given
data:

Xi Yi Xi^2 Yi^2 Xi^3 Xi^4 (Xi)^2*Yi

0 2.1 0 4.41 0 0 0

1 59.29 1 1 7.7
1 7.7
4 184.9 8 Logistic
16 54.4
2 13.6
9 739.8 27 81 244.8
3 27.2
16 1672.8 64 256 654.4
4 40.9
3733.2
5 61.1 25 125 625 1527.5
1

15 152.6 55 6594.5 225 979 2488.8

13
 Y-Values
70

60

50

40

30

20

10

0
0 1 2 3 4 5 6

14
 6 15 55 152.6
(n) a0 +

15 55 225 763
() a0 + Logistic

55 225 979 2488.8

() a0 +

Exponential

15
After solving the 3 equations we find that:

• a0 = 2.47
• a1 = 2.35
• a2 = 1.86

l l

16
Model order selection

17
Just Right! overfitting

18
19
Problem: Suppose you are given a dataset containing measurements of a certain phenomenon.
You want to fit polynomial regression models of varying degrees to this data and determine the
most appropriate model order using the Bayesian Information Criterion (BIC). The dataset
consists of the following observations:

51 99 88 64 108 34 X

5 14 8 11 17 5 Y

20
18
Y-Values
16

14

12

10

0
30 40 50 60 70 80 90 100 110 120

21
Solution:
^Y-Y ^Y Y
0.721 0.84 4.16 5

4.123 2.03 14.96 17

we get Y^ using Least square
6.06 2.46 8.544 11 method to find the line
equation which is going to
16.36 4.04- 12.04 8
be: y=0.146X–0.8
0.12 0.346 13.65 14

2.67 1.635- 6.63 5

30.075 :SSE

22
 BIC=n*ln(sse/n) + k ln(n)

BIC for order 1: 6 ln(30.75/6) + 2 ln(6) = 13.38

BIC for order 2: 6 ln(30.75/6) + 3 ln(6) = 15.18

BIC for order 3: 6 ln(30.75/6) + 4 ln(6) = 16.97

Least BIC is the best order #

23
Exponential model
2
Example:
Many patients get concerned when a test involves injection of a radioactive material. For
example for scanning a gallbladder, a few drops of Technetium- 99m isotope is used. Half of the
Technetium-99m would be gone in about 6 hours. It, however, takes about 24 hours for the
radiation levels to reach what we are exposed to in day-to-day activities. Below is given the
relative intensity of radiation as a function of time.
Table. Relative intensity of radiation as a function of time

9 7 5 3 1 0 T(hrs)

0.355 0.44 0.562 0.708 0.891 1 Y

7

Y=A

25
Y=A Ln (Y) = ln (A Ln (Y) = ln (A) + bT Z = a0 + a1 T

Z a0 a1

Ln (A) = a0 B = a1

A=

26
a1 =

a0 = Z ` - a1 T`

27
9 7 5 3 1 0 T(hrs)

0.355 0.44 0.562 0.708 0.891 1 Y

7
Ti^2 Ti Zi Zi = ln(yi) Yi Ti
0 0 0 1 0

1 -0.1154 0.1154- 0.891 1

9 -1.0359 0.3453- 0.708 3

25 -2.8813 0.5763- 00.562 5

49 -5.6364 0.805- 0.447 7

81 -9.3207 1.0356- 0.355 9

165 18.9899 2.8778- 25

28
a1 =

a0 = Z ` - a1 T` = (-2.8778/6) – 0.115 * (25/6) = 0.0002615

Average of T = sum(T)/6

Average of Z = sum(Z)/6

29
A= A = 0.999

B = a1 B=

Y = 0.999

30
Logarithmic model 3
Example:
The following table shows the meat consumption of 10 families in kilograms (Y) and their
average monthly income in dinars Determine regression equations for meat consumption in
terms of monthly income, then estimating meat consumption for a family with an average
income of 250 dinars

Family 1 2 3 4 5 6 7 8 9 10
Monthly income 250 30 290 255 26 25 200 220 270 300
In dinar(X) 0 0 0
Consumption 45 42 44 52 50 55 40 40 50 60
In kilograms(Y)

Y = a + lin b

32
Family X Yi Lin =(xi) Lin X Y LinX=(xi yi)

1 250 45 62500 11.0429 5.5215 248.4657

2 300 42 90000 11.4076 5.7038 239.5589

3 290 44 84100 11.3398 5.6699 249.4748

4 255 52 65025 11.0825 5.5413 288.1457

5 260 50 67600 11.1214 5.5607 278.0341

6 250 55 62500 11.0429 5.5215 303.6804

7 200 40 40000 10.5966 5.2983 211.9327

8 220 40 48400 10.7873 5.3936 215.7451

9 270 50 72900 11.1968 5.5984 279.9211

10 300 60 90000 11.4076 5.7038 342.2269

TOTAL 2595 478 111.0254 55.5127 2657.1853

33
b=
b=
b= -0.0187 =-0.02
a=
a= 47.9
Y = a + LinX
Y = 47.9 + (– 0.02)(Lin250)
Y = 47.9 – 0.11
Y = 47.8

34
Growth Model 4
Example:
The following table shows the meat consumption of 10 families in kilograms (Y) and their
average monthly income in dinars Determine regression equations for meat consumption in
terms of monthly income, then estimating meat consumption for a family with an average
income of 250 dinars

Family 1 2 3 4 5 6 7 8 9 10
Monthly income 250 30 290 255 26 25 200 220 270 300
In dinar(X) 0 0 0
Consumption 45 42 44 52 50 55 40 40 50 60
In kilograms(Y)

Y = a + b(1/x)

36
Family X Yi 1/x=xi (1/x)y=xi yi (1/x)2

1 250 45 0.00400 0.18000 0.000016

2 300 42 0.00333 0.14000 0.000011

3 290 44 0.00345 0.15172 0.000012

4 255 52 0.00392 0.20392 0.000015

5 260 50 0.00385 0.19231 0.000015

6 250 55 0.00400 0.22000 0.000016

7 200 40 0.00500 0.20000 0.000025

8 220 40 0.00455 0.18182 0.000021

9 270 50 0.00370 0.18519 0.000014

10 300 60 0.00333 0.20000 0.000011

TOTAL 2595 478 0.03913 1.85496 0.000156

37
b=
b=
b= – 5151.3 ≈ – 5151
a=
a=
Y = 67.957 – 5151.3(1/x)s
Y = 67.957 – (5151.3 ÷ 250)
Y = 67.957 – 20.605
Y = 47.352 ≈ 47.4

38