BINARY RESPONSE MODELS

Trương Đăng Thụy

[email protected]
 OUTLINE
▪ estimation
▪ interpretation
▪ hypothesis testing: LR/Wald
Linear Probability Model test
Logit Model ▪ marginal effects
Probit Model ▪ prediction
▪ predicted probability

2
 BINARY DEPENDENT VARIABLE
▪ Sometimes the dep var under consideration is binary:
▪ Whether loan application is approved
▪ Whether borrower can repay loan
▪ Whether a person has credit card
▪ ...

3
 BINARY OUTCOME
▪ Consider a model where the dependent variable 𝑌 is
binary 0, 1. The density function is
Pr 𝑌𝑖 = 1 = 𝐹 𝑋𝑖 𝛽
Pr 𝑌𝑖 = 0 = 1 − 𝐹 𝑋𝑖 𝛽

4
 MODELLING BINARY OUTCOMES
We have several ways of modelling depvar with binary outcome
▪ linear probability model: 𝐹 𝑋𝑖 , 𝛽 = 𝑋𝑖 𝛽
1
▪ logit: 𝐹 𝑋𝑖 , 𝛽 = Λ 𝑋𝑖 𝛽 = −𝑋𝑖 𝛽
1+𝑒
1 𝑡−𝜇 2
𝑋𝑖 𝛽 𝑋𝑖 𝛽 1 −2 𝜎
▪ probit: 𝐹 𝑋𝑖 , 𝛽 = Φ 𝑋𝑖 𝛽 = ‫׬‬−∞ 𝜙 𝑋𝑖 𝛽 = ‫׬‬−∞ 𝜎 2𝜋 𝑒 𝑑𝑡
−𝑋𝑖 𝛽
▪ Gumbel model: 𝐹 𝑋𝑖 , 𝛽 =𝑒 −𝑒
𝑋𝑖 𝛽
▪ complementary log-log model: 𝐹 𝑋𝑖 , 𝛽 = 1 − 𝑒 −𝑒

▪ and many other variations.

5
6
EXAMPLE DATA
 EXAMPLE: COVID-19 VACCINE PURCHASE
DATA IS MADE AVAILABLE BY EEPSEA
▪ Problem: the decision to vaccinate for oneself with a hypothetical COVID-19 vaccine
▪ Data: a survey of 377 individuals in HCMC in 2020
▪ Dep var: dself : 1 = decide to vaccinate for oneself
▪ efficacy80 : 1 = the efficacy is 80%, 0 = 50%
▪ duration3 : 1 = effectiveness duration is 3 years, 0 = 1 year
▪ priceUS (USD/2-dose vaccine): price of vaccine.
▪ pbenefit : 1 = respondent was provided information on the externality of vaccination
▪ hhincomeUS (USD/month): total monthly household income
▪ hhsize (members): household size
▪ age (years): age of respondent
▪ edu (categorical): education attainment, 1 = under primary school, 2 = primary, 3 = secondary, 4 = highschool, 5 =
College, 6 = University or higher.
▪ male : gender of respondent, 1 = male, 0 = female
▪ risk (ordinal): perceived risk of COVID-19 infection: 1 = “Very unlikely”, 2 = “Unlikely”, 3 = “Neither”, 4 = “Likely”, 5 =
“Very likely”.
▪ Data file: https://kinhteluong.online/esdata/iu/EMP4.csv
7
DATA AND SUMMARY STATISTICS

8
SUMMARY STATISTICS

9
10
LINEAR PROBABILITY MODEL
 OLS WITH BINARY
DEP VAR:
THE LINEAR
PROBABILITY MODEL
▪ If 𝑦 is a binary variable (0/1),
and we apply OLS, then the
model is called Linear
Probability Model (LPM)

11
12
 DISADVANTAGES OF LPM
▪ Fitted value of Pr 𝑦 = 1 may be out of [0,1].
▪ Assume that Pr 𝑦 = 1 linearly correlate with 𝑋
regardless the initial value of 𝑋
▪ 𝜀 has unequal variance, resulting in unreliability
of hypothesis testing.
▪ Violate the assumption that 𝜀 is normally
distributed.

13
14
THE LOGIT MODEL
 THE LOGIT MODEL
▪ Logit model assume ui follows logistic distribution

▪ Probability for 𝑌 = 1:
1
Pr 𝑌𝑖 = 1 = 𝑃𝑖 =
1 + 𝑒 −𝛽𝑋𝑖

▪ With −∞ < 𝑍𝑖 < +∞, then 0 < Pr(𝑌𝑖 = 1) < 1.

15
 optional

THE LOGIT MODEL

▪ The odd ratio in this case is the ratio between probability of
default and probability of non-default:
𝑃𝑖 1 1 + 𝑒 −𝛽𝑋𝑖 𝛽𝑋𝑖
= = 𝑒
1 − 𝑃𝑖 1 + 𝑒 −𝛽𝑋𝑖 𝑒 −𝛽𝑋𝑖
▪ Taking log of both sides, we obtain the logit:
𝑃𝑖
ln = 𝛽𝑋𝑖
1−𝑃𝑖
▪ LPM assumes Pi linearly correlates with 𝑋𝑖 , the Logit model
assumes the logit linearly correlates with 𝑋𝑖 .
16
 PROPERTIES OF LOGIT MODEL
▪ Pi varies from 0 to 1 while the logit varies from – to +.
▪ Although Li is a linear function of Xi, the probability is not.
▪ Interpretation of estimated coefficients:
▪ j is the change in log-odd ratio when Xj increase by 1 unit,
▪ j shows the direction of change in Pi.

▪ In LPM, the marginal effect of Xj is constant. In the Logit

model, the marginal effect of Xj changes.

17
 ESTIMATION METHOD
▪ Maximum Likelihood (ML)
▪ ML seeks j such that log 𝐿 is maximized
𝑛

log 𝐿 = ෍ 𝑌𝑖 log 𝑃𝑖 + 1 − 𝑌𝑖 log 1 − 𝑃𝑖

𝑖=1
1
where 𝑃𝑖 = , 𝑌𝑖 is the observed choice.
1+𝑒 −𝛽𝑋𝑖

18
 INTERPRETATION OF THE COEFFICIENTS
▪ j is the change in log-odd ratio when Xj increase by 1 unit,
𝑃𝑖
𝐿𝑖 = ln = 𝛽𝑋𝑖 + 𝑢𝑖
1 − 𝑃𝑖
▪ j shows the direction of change in Pi.
1
𝑃𝑖 =
1 + 𝑒 −𝛽𝑋𝑖
▪ Coefficient 𝛽 only indicates the direction of the effect. It
says nothing about the magnitude of the effect.

19
20
 HYPOTHESIS TESTING AFTER LOGIT:
LIKELIHOOD RATIO TEST
The procedure:
1
▪ Estimate the full model: 𝑃𝑖 = , then obtain the Log-likelihood value 𝐿𝐿𝐹 . (Note
1+𝑒 −𝛽𝑋𝑖
that log-likelihood = - deviance/2.)
▪ Suppose we test the null hypothesis: H0: 𝛽1 = 𝛽2 = 0 (could be one or more coef.)
▪ Impose the null hypothesis to the full model, we have the restricted model in which
the variables with 𝛽1 and 𝛽2 are removed.
▪ Estimate the restricted model to obtain the log-likelihood value 𝐿𝐿𝑅 .
▪ The test statistic = 2 𝐿𝐿𝐹 − 𝐿𝐿𝑅 , follow 𝜒 2 distribution with df = number of
coefficients tested.

21
LRTEST FOR OVERALL SIGNIFICANCE

22
HYPOTHESIS TESTING AFTER LOGIT:
LIKELIHOOD RATIO TEST

23
HYPOTHESIS TESTING AFTER LOGIT:
WALD CHI-SQUARED TEST

24
 MARGINAL EFFECTS
▪ If we want to know: when 𝑋 increases by 1 unit, then how
much 𝑃 changes (marginal effect)

𝜕𝑃𝑖 𝜕 1 1
= −𝛽𝑋
= 𝛽𝑖
𝜕𝑋𝑖 𝜕𝑋𝑖 1 + 𝑒 𝑖 1 + 𝑒 −𝛽𝑋𝑖
▪ Marginal effect in the logit model is not constant. It varies
with 𝑋.

25
 PARTIAL
EFFECTS
AFTER LOGIT
- FOR THE
AVERAGE
OBSERVATION

26
 AVERAGE
PARTIAL
EFFECTS
AFTER LOGIT

27
PREDICTION

28
PREDICTED PROBABILITY

29
PREDICTED PROBABILITY

30
MARGINAL EFFECTS AT SPECIFIC POINTS

31
 LOGISTIC
REGRESSION
WITH ODDS
RATIO
 ODD RATIO
ESTIMATE IS
IDENTICAL TO
EXPNENTIAL
OF LOGIT
ESTIMATE
34
THE PROBIT MODEL
 THE PROBIT MODEL
▪ In the Logit model, u follows logistic distribution

1
Pr 𝑌𝑖 = 1 = 𝑃𝑖 =
1 + 𝑒 −𝛽𝑋𝑖
▪ In the Probit model, u follows normal distribution

𝛽𝑋𝑖
1 2 /2
Pr 𝑌𝑖 = 1 = 𝑃𝑖 = න 𝑒 −𝑧 𝑑𝑧
2𝜋
−∞
where F is the cumulative distribution function (CDF) of the normal
distribution.
35
ESTIMATING
PROBIT
MODEL IN R

36
OVERALL SIGNIFICANCE AFTER PROBIT

37
LR TEST FOR JOINT SIGNIFICANCE

38
WALD TEST FOR JOINT SIGNIFICANCE

39
 PARTIAL
EFFECTS
AFTER
PROBIT –
FOR THE
AVERAGE OBS

40
AVERAGE
PARTIAL
EFFECTS
AFTER
PROBIT

41
 PREDICTED PROBABILITY
AND CORRECT PREDICTION

42
 PREDICT
PROBABILITY
AFTER
PROBIT

43
 PREDICT
PROBABILITY
AFTER
PROBIT

44
 MARGINAL EFFECTS
AT A SPECIFIC DATA POINT

45
 LOGIT OR PROBIT?
▪ Pi approaches 0 and 1 slower in the Logit, compared
to the Probit model.
▪ No obvious reason of choosing between the two
models.
▪ However Logit is preferred for its simplicity in
computing the marginal effects.

46
LOGIT OR PROBIT?
COMPARING THE
COEFFICIENTS

47
 LOGIT OR
PROBIT?
COMPARING
THE
PREDICTED
PROBABILITIES