Econometrics for

Management (MGMT3071)

Chapter FOUR:
Discrete Choice Models
(non-linear regression analysis)

Teklebirhan Alemnew (Assistant Professor)

[email protected]
AAU, 2023

By: Teklebirhan A. 1
 Outline
1. Introduction
2. Linear Regression Model (LPM)
3. Non-linear Regression models
3.1. The logit and Probit models
3.2. Multinomial logit and Probit
models
3.3. Ordered logit and Probit
models

By: Teklebirhan A. 2
1. Introduction

By: Teklebirhan A. 3
 Cont…
Determinates of employees satisfaction
in AAU.
Factors affecting loan repayment in AAU.
Factors affecting Turnover Intention in
CBE.
Determinants of women’s willingness to
practice family planning utilities in the
case of AA town.
Determinants of Household’s Saving
behavior in AA town.
By: Teklebirhan A. 4
 Cont…
In the above examples, response
(dependent) variable is qualitative/
categorical/discrete.
Models dealing with such kind of binary
responses are called binary/ Discrete
choice models.
Technically, it is possible to estimate the
binary choices using OLS.
If OLS is used to estimate qualitative
response variable, the resulting model is
called linear probability model (LPM).
By: Teklebirhan A. 5
Cont…

By: Teklebirhan A. 6
 Cont…

 Both Yi and Ui take only two values with

respective probabilities and such
variables are called Bernoulli variables
and follow Bernoulli distributions.
distributions
 But, OLS assumes that both Yi and Ui are
normally distributed.
By: Teklebirhan A. 7
Cont…

By: Teklebirhan A. 8
Cont…

By: Teklebirhan A. 9
 Cont…
 The very difference between linear
regression model (LRM) and binary
regression model (BRM) is that
 In the case of LRM, regression analysis
deals with the prediction of the
average value of the response
variable from the given values of
explanatory variables.
 This is because, the response variable
is continuous/quantitative in LRM.
By: Teklebirhan A. 10
Cont…

By: Teklebirhan A. 11
2. Linear Probability Model

By: Teklebirhan A. 12
 Cont…
b) The predicted probability may lie
outside the natural interval (0≤𝑝≤1)

By: Teklebirhan A. 13
 Cont…
(Not
Always)

e) The true relationship between a binary

outcome and a continuous explanatory
variable is inherently nonlinear
 This means the functional form of the LPM is
generally not correctly specified, which can
lead to biased estimates of some
parameters of interest
By: Teklebirhan A. 14
 Cont…
 Due to the above five basic limitations of
using OLS on qualitative response variable
model, non-linear regression models were
developed
 Example:

By: Teklebirhan A. 15
 Cont…
 Violation of the axiom of Probability
reg grade gpa pc ase
predict yhat
scatter grade yhat||lfit grade yhat

 Violation of the assumption of

Heteroscedasticity
rvfplot, yline(0)
hettest

By: Teklebirhan A. 16
 Cont…
 Violation of the assumption of
normality
predict r, resid
pnorm r
qnorm r
mvtest norm r
kdensity r, normal
histogram r, kdensity normal
 The coefficient of determination is not

dependable which is 0.4 2

By: Teklebirhan A. 17
 3. Nonlinear Regression
Models
 In the LPM, the heteroskedasticity
problem is less worrying as it can be
easily handled.
 We need to resort to other methods to
account for the other shortcomings.
 In particular, we need a model 0 which

satisfies

By: Teklebirhan A. 18
 Cont…
 Non-linear regression model includes
a)The Logit Model
b)The Probit Model
c)Multinomial Logit and Probit Model
(MNL & MNP)
d)Ordered Logit and Probit Model.

By: Teklebirhan A. 19
3.1. The Logit and Probit
Models

Non-linearity
By: Teklebirhan A. 20
Cont…

By: Teklebirhan A. 21
Cont…

By: Teklebirhan A. 22
 Cont…

Satisfied

Not-
Satisfied

By: Teklebirhan A. 23
 Cont…
 Take the ratio of the probability of an
event occurring (Pi) to the probability
of an event not happening (1-Pi) and
the resulting ratio is called odds ratio.

By: Teklebirhan A. 24
 Cont…
 Take the natural log of the above odds
ratio and the resulting equation is
called Logit.

 Where, Li is called Logit which is

linearly related with Xi is explanatory
variables
By: Teklebirhan A. 25
 Cont…
Pi

Cumulative Normal Distribution Function

Pi =1

Logistic Distribution Function

By: Teklebirhan A. 26
Cont…

By: Teklebirhan A. 27
 Cont…
 Example on Logit and Probit
 Suppose that we want to examine
the effect of routine weekly
exercises on the performance of
students.
 To this end, suppose we gave routine
exercises to second year section A
students and at the end of the
semester, we found average scores in
exercise (ASE) for each student.
By: Teklebirhan A. 28
Cont…

By: Teklebirhan A. 29
 Cont…
A) Logit Interpretation of Logit
Model
logit grade gpa ase pc

By: Teklebirhan A. 30
 Cont…
Interpretation:
 As GPA increases by one point, the log of
the odds ratio increases by 2.8 and
statistically significant.
 A student who owned PC, the log of the
odds ratio increases by 2.4 and
statistically significant.

By: Teklebirhan A. 31
 Cont…
B) Odds Ratio Interpretation of
Logit Model
logit grade gpa ase pc, or

By: Teklebirhan A. 32
 Cont…
Interpretation:
 As GPA increases by one point, the odds
of getting A is 16.87 times the odds of
getting other grades (B, C, D, F)
 A student who owned PC, the odds of
getting A is 10.8 times the odds of getting
other grades (B, C, D, F)

By: Teklebirhan A. 33
 Cont…
C) Probability (mfx) Interpretation
of the logit model

By: Teklebirhan A. 34
 Cont…
Interpretation:
 As GPA increases by one point, the
probability of getting grade A increases by
53%.
 A student who owned PC, the probability of
getting grade A increases by 45.6%.

By: Teklebirhan A. 35
 Cont…
D) Probit Estimation
probit grade gpa ase pc

NB: the interpretation way is similar

36
with Logit estimation By: Teklebirhan A.
 Cont…
E) Probability Interpretation of
Probit Model

By: Teklebirhan A. 37
 Cont…
Logit/Probit Model Diagnostic Tests
Multicollinearity Test
vif
 Heteroscedasticity Test
hettest
Model specification/omitted variable
Test
linktest

By: Teklebirhan A. 38
 Cont…
 Multicollinearity Test
. vif

Variable VIF 1/VIF

ase 1.19 0.840735

gpa 1.18 0.850226
pc 1.01 0.987262

Mean VIF 1.13

 Heteroscedasticity Test
. hettest

Breusch-Pagan / Cook-Weisberg test for heteroskedasticity

Ho: Constant variance
Variables: fitted values of grade

chi2(1) = 2.53
Prob > chi2 = 0.1117

39
By: Teklebirhan A.
 Cont…
 Link Test/Model specification test

grade Coef. Std. Err. z P>|z| [95% Conf. Interval]

_hat .9551764 .383456 2.49 0.013 .2036165 1.706736

_hatsq -.0453861 .1881828 -0.24 0.809 -.4142177 .3234455
_cons .0817277 .6074585 0.13 0.893 -1.108869 1.272324

 The insignificant hat square shows

that the model has no error on its
formula and no omission of significant
40
variable. By: Teklebirhan A.
 3.2 Multinomial Logit and
Probit Model
 The probability that a particular consumer will
choose a particular alternative is given by the
probability that the utility of that alternative to
that consumer is greater than the utility to that
consumer of all other alternatives.

 Then the consumer picks the alternative that

maximizes his or her utility.

 MNL model is a simple extension to the

logit model when the dependent variable
can take more than two categorical values.
By: Teklebirhan A. 41
 Cont…
 For instance, in Addis Ababa, a person
may have the following choice of means
of transportation to go to work place.
 Car, Bus, Train
 A person may have three voting options:
 Labor party
 Conservative party
 liberal democrat party
 A respondent is provided with more than
two alternatives and s/he is expected to
choose one. By: Teklebirhan A. 42
 Cont…
 There is no order within the categories of Y
(any of a chosen categories can be the
baseline for comparison).

 In Multinomial Logit Model (MLM), a response

variable with K categories will generate K-1
equations.

 That means, in multinomial logit model we

have K-1 equations instead of one equation.

 That is why Multinomial logit models are

called multi-equation models.
By: Teklebirhan A. 43
 Cont…
 Each of these K-1 equations is a binary
logistic regression comparing a group with
the reference group.

 The choice of reference category is

arbitrary.

 Example: if our dependent variable Y= 1, 2,

3 then our reference category is 1, then we
will have two logit equations,
 First equation : Y=2 versus Y=1
 Second equation: By: Teklebirhan A. 44
Y=3 versus Y=1
 Cont…
 The probabilities for all the categories of Y(all the
possible outcomes for our dependent variable) add
to 1 or 100%.
 That means, P1 + P2 + P3=1

 The multinomial logit is equivalent to running a

series of separate binary logit models to find the
coefficients, but these would not give us a single
overall measures of our model.

 Multinomial logistic regression simultaneously

estimates the K-1 logit coefficients through MLE.
 Hence, if the first category is the reference, then
45
for m= 2, …, M. By: Teklebirhan A.
 Cont…

 Hence, for each case, there will be M-1

predicted log odds, one for each category
relative to the reference category.

 When there are more than 2 groups,

computing probabilities is a little more
complicated than it was in logistic
regression. For j = 2, …, M,
By: Teklebirhan A. 46
Cont…

By: Teklebirhan A. 47
 Cont…
 In other words, you take each of the M-
1 log odds you computed and
exponentiate it.

 Once you have done that the

calculation of the probabilities is
straightforward.
 Note that, when M=2, the mlogit and
logistic regression models (and for that
matter the ordered logit model)
become one and the same.
By: Teklebirhan A. 48
Cont…

By: Teklebirhan A. 49
 3.3 Ordered Logit Model
 The ordered logit model is also known as the
proportional odds model.
model
 The terms parallel lines model and parallel
regressions model are also sometimes used.

 In ordered logit model, there is observed

ordinal variable, Y which in turn, is a function
of another variable, Y*, that is not measured.

 That means, in ordered logit model, there is a

continuous, unmeasured latent variable Y*,
whose values determine the value of ordinal
variable Y.
By: Teklebirhan A. 50
 Cont…
 The continuous latent variable Y* has
various threshold points (cut points).

 Your value on the observed variable Y

depends on whether or not you have
crossed a particular threshold.

 These cut points (thresholds) are

represented by k which is the Greek
small letter kappa.
By: Teklebirhan A. 51
 Cont…
 For example, when the number of
categories are three (M=3)

 For example, it might be that if your

score on the unobserved latent variable
Y* was 10 utils or less, your score on Y
would be low(1); if your Y* score was b/n
10 and 25 utils, Y would be medium (2);
and if your score on latent variable (Y*)
52
was above 25, Y would be high(3).
By: Teklebirhan A.
 Cont…
 Put another way, you can think of Y as
being a collapsed version of Y*.
 Example Y* can take on an infinite
range of values which might then be
collapsed into 3 categories of Y. So,
what does Y* equal? How do you
estimate this model?
 In the population, the continuous
latent variable Y* is equal to

By: Teklebirhan A. 53
 Cont…
 Note that there is a random
disturbance term, which, in this case,
has a standard logistic distribution
(mean of 0 and variance of 3.29).
This reflects the fact that relevant
variables may be left out of the
equation, or variables may not be
perfectly measured.
 The ordered logit model estimates part
of the above:
By: Teklebirhan A. 54
Cont…

By: Teklebirhan A. 55
Cont…

By: Teklebirhan A. 56
Cont…

By: Teklebirhan A. 57
End!

By: Teklebirhan A. 58