Hypothesis Testing in the Multiple regression

model
• Testing that individual coefficients take a specific value such as
zero or some other value is done in exactly the same way as
with the simple two variable regression model.

• Now suppose we wish to test that a number of coefficients or

combinations of coefficients take some particular value.

• In this case we will use the so called “F-test”

• Suppose for example we estimate a model of the form

Yi = a + b1 X i1 + b2 X i 2 + b3 X i 3 + b4 X i 4 + b5 X i 5 + ui

• We may wish to test hypotheses of the form {H0: b1=0 and

b2=0 against the alternative that one or more are wrong} or
{H0: b1=1 and b2-b3=0 against the alternative that one or more
are wrong} or {H0: b1+b2=1 and a=0 against the alternative that
one or more are wrong}
• This lecture is inference in this more general set up.
• We will not outline the underlying statistical theory for this. We
will just describe the testing procedure.
 Definitions

• The Unrestricted Model: This is the model without any of the

restrictions imposed. It contains all the variables exactly as in
the regression of the previous page

• The Restricted Model: This is the model on which the

restrictions have been imposed. For example all regressors
whose coefficients have been set to zero are excluded and any
other restriction has been imposed.
 Example 1

• Suppose we want to test that :H0: b1=0 and b2=0 against the
alternative that one or more are wrong in:
Yi = a + b1 X i1 + b2 X i 2 + b3 X i 3 + b4 X i 4 + b5 X i 5 + ui
• The above is the unrestricted model

• The Restricted Model would be

Yi = a + b3 X i 3 + b4 X i 4 + b5 X i 5 + ui
 Example 2

• Suppose we want to test that : H0: b1=1 and b2-b3=0 against the
alternative that one or more are wrong :
Yi = a + b1 X i1 + b2 X i 2 + b3 X i 3 + b4 X i 4 + b5 X i 5 + ui

• The above is the unrestricted model

• The Restricted Model would be
Yi = a + X i1 + b2 X i 2 − b2 X i 3 + b4 X i 4 + b5 X i 5 + ui
• Rearranging we get a model that uses new variables as functions
of the old ones:

(Yi − X i1 ) = a + b2 ( X i 2 − X i 3 ) + b4 X i 4 + b5 X i 5 + ui
• Inference will be based on comparing the fit of the restricted and
unrestricted regression.

• The unrestricted regression will always fit at least as well as

the restricted one. The proof is simple: When estimating the
model we minimise the residual sum of squares. In the
unrestricted model we can always choose the combination of
coefficients that the restricted model chooses. Hence the
restricted model can never do better than the unrestricted one.

• So the question will be how much improvement in the fit do we

get by relaxing the restrictions relative to the loss of precision
that follows. The distribution of the test statistic will give us a
measure of this so that we can construct a decision rule.
 Further Definitions

• Define the Unrestricted Residual Residual Sum of Squares (URSS)

as the residual sum of squares obtained from estimating the
unrestricted model.
• Define the Restricted Residual Residual Sum of Squares (RRSS) as
the residual sum of squares obtained from estimating the restricted
model.
• Note that according to our argument above RRSS ≥ URSS

• Define the degrees of freedom as N-k where N is the sample size and
k is the number of parameters estimated in the unrestricted model (I.e
under the alternative hypothesis)
• Define by q the number of restrictions imposed (in both our examples
there were two restrictions imposed
 The F-Statistic

• The Statistic for testing the hypothesis we discussed is

( RRSS − URSS ) / q
F=
URSS /( N − K )

• The test statistic is always positive. We would like this to be

“small”. The smaller the F-statistic the less the loss of fit due to
the restrictions

• Defining “small” and using the statistic for inference we need to

know its distribution.
 The Distribution of the F-statistic

• As in our earlier discussion of inference we distinguish two

cases:
Normally Distributed Errors

– The errors in the regression equaion are distributed

normally. In this case we can show that under the null
hypothesis H0 the F-statistic is distributed as an F
distribution with degrees of freedom (q,N-k) .
– The number of restrictions q are the degrees of freedom of
the numerator.
– N-K are the degrees of freedom of the denominator.
• Since the smaller the test statistic the better and since the test
α positive we only have one critical value.
statistic is always
• For a test at the level of significance we choose a critical
value of F1−α , ( q, N − k )

• If the test statistic is below the critical value we accept the null
hypothesis.
• Otherwise we reject.
 Examples

• Examples of Critical values for 5% tests in a regression model

with 6 regressors under the alternative
– Sample size 18. One restriction to be tested: Degrees of
freedom 1, 12: F1− 0 . 05 , (1 ,12 ) = 4 . 75

– Sample size 24. Two restrictions to be tested: degrees of

freedom 2, 18: F1 − 0 . 05 , ( 2 ,18 ) = 3 . 55

– Sample size 21. Three restrictions to be tested: degrees of

freedom 3, 15: F1− 0 .05 ,( 3 ,15 ) = 3 . 29
 Inference with non-normal errors

• When the regression errors are not normal (but satisfy all the
other assumptions we have made) we can appeal to the central
limit theorem to justify inference.
• In large samples we can show that the q times the F statistic is
distributed as a random variable with a
α
qF ~χ 2
q distribution
 Examples

• Examples of Critical values for 5% tests in a regression model

with 6 regressors under the alternative. Inference based on large
samples:

– One restriction to be tested: Degrees of freedom 1. :

χ 12− 0 .05 ,1 = 3 . 84
– Two restrictions to be tested: degrees of freedom 2:
χ 12− 0 .05 , 2 = 5 .99
– Three restrictions to be tested: degrees of freedom 3:
χ 12−0.05 , 3 = 7 .81
 Example: The Demand for butter:
Hypothesis to be tested: Butter and margarine advertising do not change
demand and income elasticity of butter is one: Three restrictions
Unrestricted Model

. regr lbp lpbr lpsmr lryae ltba lrma

Source | SS df MS Number of obs = 51

-------------+------------------------------ F( 5, 45) = 11.74
Model | .357443231 5 .071488646 Prob > F = 0.0000
Residual | .273965407 45 .00608812 R-squared = 0.5661
-------------+------------------------------ Adj R-squared = 0.5179
Total | .631408639 50 .012628173 Root MSE = .07803

------------------------------------------------------------------------------
log butter purchases lbp | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
log price of butter lpbr | -.7297508 .1540721 -4.74 0.000 -1.040068 -.4194336
log price of margarine lpsmr | .7795654 .3205297 2.43 0.019 .1339856 1.425145
log real income lryae | .9082464 .510288 1.78 0.082 -.1195263 1.936019
log butter advertising ltba | -.0167822 .0133142 -1.26 0.214 -.0435984 .0100339
log margarine advertising lrma | -.0059832 .0166586 -0.36 0.721 -.0395353 .027569
Constant _cons | 6.523365 .8063481 8.09 0.000 4.899296 8.147433
------------------------------------------------------------------------------
 Restricted Model
lbp = a + b1 lpbr + b 2 lpsmr + 1 × lryae + 0 × ltba + 0 × lrma + u

( lbp − lryae ) = a + b1 lpbr + b 2 lpsmr + u

. gen lbpry=lbp-lryae

. regr lbpry lpbr lpsmr

Source | SS df MS Number of obs = 51

-------------+------------------------------ F( 2, 48) = 43.74
Model | .523319203 2 .261659601 Prob > F = 0.0000
Residual | .287162961 48 .005982562 R-squared = 0.6457
-------------+------------------------------ Adj R-squared = 0.6309
Total | .810482164 50 .016209643 Root MSE = .07735

------------------------------------------------------------------------------
New dep var lbpry | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
log price of butter lpbr | -.7481124 .14332 -5.22 0.000 -1.036277 -.4599483
log price of margarine lpsmr | .782316 .2466846 3.17 0.003 .2863234 1.278309
Constant _cons | 6.255797 .5969626 10.48 0.000 5.055523 7.456071
------------------------------------------------------------------------------
 The Test

• The value of the test statistic is

( 0.287 − 0.274 ) / 3
F = = 0.71
0.274 /( 51 − 6)

• The critical value for a 5% test wit (3,45) degrees of freedom is

2.81
• We accept the null hypothesis since 0.71<2.81.
A Large sample example: Testing for seasonality in
fuel expenditure

 

    

                   
  ! "  #    $ %  & $ $
' ' 
  
 
  ( ) &  $ ' '  
& '  ' $ '     *  + !  $
$ $ $ $ 








  








     $ 
$ $ & ' ,   ,      






 



   -  . / 0   $
$  ,  
1 
 
$ ( $          
$ $ &   ) & & (  /   2  
$ ' , ( ( 


 3 ! 
   
 4 
 
2 
* +   5 (  6 4 
7 8 
9 
 
: 
2    
 
$ & & , ( ,  
$ $ $  & )  & ,
 ) $
$ $ $ 
$ & ' $ ( '  
$ & $ ) ( ( , 
  
$ $ ,    ' 
$ $ &  &  & 
)  $
$ $ $ 
$ $ '    & 
$ $ (  &   
  
$ $ ) $ &  , 
$ $ &    ' &
' & $
& ( ) 
$ $  $   
$ $ & $ & &  
   
$ $ ( (  &  
$ $ &  )  ,
 ' $
$ $ $ 
$ & ) ( ' ( , 
$ $ , ( ,  & 
;   
& & ,   $  
$ $ ' $ (  ' 
 ' $
$ $ $ 
& & $  $ ' 
& ) )  '  & 

 The Restricted Model: Excludes the Seasonal Indicator

. regress wfuel logex

Source | SS df MS Number of obs = 4785
-------------+------------------------------ F( 1, 4783) = 270.14
Model | .37905073 1 .37905073 Prob > F = 0.0000
Residual | 6.71140481 4783 .001403179 R-squared = 0.0535
-------------+------------------------------ Adj R-squared = 0.0533
Total | 7.09045554 4784 .001482119 Root MSE = .03746

------------------------------------------------------------------------------
Share of Fuel in budget wfuel | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
log real expenditure logex | -.0118527 .0007211 -16.44 0.000 -.0132665 -.0104389
Constant _cons | .1160574 .0030032 38.64 0.000 .1101697 .1219451
------------------------------------------------------------------------------

The Chi squared test statistic: (6.71- 6.54)/(6.54/4785)= 124.38

Critical Value for 5% test and three degrees of freedom 7.81
Hypothesis rejected since 124.38>7.81
Alternative form of the F-statistic using the R
squared
• So long as the Total sum of squares is kept the same between
models we can also write the F-statistic as

(R − R ) / q 2 2
F = U R

(1 − R U ) /( N − k )
2

• where U refers to the unrestricted model and R to the restricted

model
• This will not work if we compute the R squared with different
dependent variables in each case (e.g. because of
transformations.
 Heteroskedasticity

• Heteroskedasticity means that the variance of the errors is not

constant across observations.

• In particular the variance of the errors may be a function of

explanatory variables.

• Think of food expenditure for example. It may well be that the

“diversity of taste” for food is greater for wealthier people than
for poor people. So you may find a greater variance of
expenditures at high income levels than at low income levels.
• Heteroskedasticity may arise in the context of a “random
coefficients model.
• Suppose for example that a regressor impacts on individuals in a
different way
Y i = a + ( b1 + ε i ) X i 1 + u i

Y i = a + b1 X i 1 + ε i X i 1 + u
 • Assume for simplicity that å and u are independent.
• Assume that å and X are independent of each other.
• Then the error term has the following properties:

E (ε i X i + u i | X ) = E (ε i X i | X ) + E (u i | X ) = E (ε i | X ) X i = 0

Var(ε i X i + ui | X ) = Var(ε i X i | X ) + Var(ui | X ) = X σ ε + σ 2

2 2
i

• Where σ ε is the variance of å

2
 Implications of Heteroskedasticity

• Assuming all other assumptions are in place, the assumption

guaranteeing unbiasedness of OLS is not violated.
Consequently OLS is unbiased in this model

• However the assumptions required to prove that OLS is efficient

are violated. Hence OLS is not BLUE in this context

• We can devise an efficient estimator by reweighing the data

appropriately to take into account of heteroskedasticity