Lecture 1: Introduction and Review of

Statistics

Stock and Watson Chapter 1-2

 9

Course outline

Lecture 1: Introduction and Review of Statistics (S&W Ch 1-2)

Lecture 2: Review of Statistics (S&W Ch 2-3)
Lecture 3: Review of Statistics & Ordinary Least Squares (S&W Ch 3-4)
Lecture 4: Linear regression with one regressor (S&W Ch 4)
Lecture 5: Hypothesis tests & confidence intervals–One regressor (S&W Ch 5)
Lecture 6: Linear regression with multiple regressors (S&W Ch 6)
Lecture 7: Hypothesis tests & conf. intervals–Multiple regressors (S&W Ch 7)
Lecture 8: Nonlinear regression (S&W Ch 8)
Lecture 9: Internal and external validity (S&W Ch 9)
Lecture 10: Panel data (S&W Ch 10)
Lecture 11: Binary dependent variables (S&W Ch 11)
Review of Statistics
 11

Review of Statistics

Today we will discuss:

• A random variable and its probability distribution

• Measures of the shape of a probability distribution

• Mean, variance, skewness and kurtosis

• Two random variables and their joint distribution

• Joint distribution, marginal distribution, conditional distribution
• Law of iterated expectations
• Means, variances and covariances of sums of random variables

• Often used probability distributions in econometrics

• Normal, Chi-Squared, Student t and F-distributions
 12

A random variable

Some definitions:

Outcomes are the mutually exclusive potential results of a random

process
• Your grade on the exam, the number of days it will snow
next week

Random variable is a numerical summary of a random outcome

• The number of days it will snow next week is random
and takes on a numerical value (0,1,2,3,4,5,6 or 7).

• There are two types of random variables:

Discrete random variable takes on discrete number of values, like 0,1,2,...

Continuous random variable takes on a continuum of possible values
 13

Probability distribution of a discrete random variable

• Each outcome of a discrete random variable occurs with a certain

probability

A Probability distribution of a discrete random variable is the list of possible

values of the variable and the probability that each value will
occur.

• Let random variable S be the number of days it will snow in the last
week of January

Probability distribution of S
Outcome 0 1 2 3 4 5 6 7
Probability 0.20 0.25 0.20 0.15 0.10 0.05 0.04 0.01
 14

Cumulative distribution of a discrete random variable

A cumulative probability distribution is the probability that the random

variable is less than or equal to a particular value

• The probability that it will snow less than or equal to s days,

F (s) = Pr (S ≤ s) is the cumulative probability distribution of S
evaluated at s

• A cumulative probability distribution is also referred to as a cumulative

distribution or a CDF.

(cumulative) Probability distribution of S

Outcome 0 1 2 3 4 5 6 7
Probability 0.20 0.25 0.20 0.15 0.10 0.05 0.04 0.01
CDF 0.20 0.45 0.65 0.80 0.90 0.95 0.99 1
 15

Probability distribution of a continuous random variable

• Tomorrow’s temperature is an example of a continuous random variable

• The CDF is defined similar to a discrete random variable.

• A probability distribution that lists all values and the probability of each
value is not suitable for a continuous random variable.

• Instead the probability is summarized in a probability density function

(PDF/ density)

.08 1

Cumulative distribution function

.8
Probability density

.06 Area to the left of the red

line: Pr(T <= -5) = 0.5
.6
.04
.4

.02
.2

0 0
-30 -20 -10 0 10 -30 -20 -10 0 10 20
Tomorrow's temperature Tomorrow's temperature
 16

Measures of the shape of a probability distribution

Expected value

The expected value or mean of a random variable is the average value over
many repeated trails or occurrences.

Suppose a discrete random value Y takes on k possible values

k
X
E (Y ) = yi · Pr (Y = yi ) = µY
i=1

Number of days it will snow in the last week of January (S)

Outcome 0 1 2 3 4 5 6 7
Probability 0.20 0.25 0.20 0.15 0.10 0.05 0.04 0.01

E (S) = 0·0.2+1·0.25+2·0.2+3·0.15+4·0.1+5·0.05+6·0.04+7·0.01 = 2.06

Expected value of a continuous random variable

Z ∞
E (Y ) = y · f (y )dy = µY
−∞
 17

Measures of the shape of a probability distribution

Variance

The variance of a random variable Y is the expected value of the square of

the deviation of Y from its mean.

• The variance is a measure of the spread of a probability distribution.

• Suppose a discreet random variable Y takes on k possible values

h k
i X
2 2
σY2 = Var (Y ) = E (Y − µY ) = (yi − µY ) · Pr (Y = yi )
i=1

Number of days it will snow in the last week of January (S)

Outcome 0 1 2 3 4 5 6 7
Probability 0.20 0.25 0.20 0.15 0.10 0.05 0.04 0.01

Var (S) = (0 − 2.06)2 · 0.2 + (1 − 2.06)2 · 0.25 + (2 − 2.06)2 · 0.2 + (3 − 2.06)2 · 0.15
+(4 − 2.06)2 · 0.1 + (5 − 2.06)2 · 0.05 + (6 − 2.06)2 · 0.04 + (7 − 2.06)2 · 0.01
2.94
p
The standard deviation σY = Var (Y ) has the same units as Y
 18

Mean and variance of a Bernoulli random variable

A Bernoulli random variable is a binary random variable with two possible

outcomes, 0 and 1.

• For example, let B be a random variable which equals 1 if you pass the
exam and 0 if you don’t pass
(
1 with probability p
B=
0 with probability (1 − p)

• The expected value of B is

k
X
E (B) = µB = bi · Pr (B = bi ) = 1 · p + 0 · (1 − p) = p
i=1

• The variance of B is
Pk
Var (B) = σB2 = (bi − µB )2 · Pr (B = bi )
i=1
= (0 − p)2 · (1 − p) + (1 − p)2 · p
= p(1 − p)
p
• The standard deviation of B is σB = p(1 − p)
 19

Mean and variance of a linear function of a random variable

• In this course we will consider random variables (say X and Y ) that are
related by a linear function

Y =a+b·X

• Suppose E (X ) = µX and Var (X ) = σX2

• This implies that the expected value of Y equals

E (Y ) = µY = E (a + b · X ) = a + b · E (X ) = a + b · µX

• The variance of Y equals

h i
Var (Y ) = σY2 = E (Y − µY )2
h i
= E ((a + bX ) − (a + bµX ))2

E b2 (X − µX )2
 
=

b2 E (X − µX )2
 
=

= b2 · σX2
 20

Measures of the shape of a probability distribution: Skewness

Skewness is a measure of the lack of symmetry of a distribution
h i
E (Y − µY )3
Skewness =
σY3

Negative Skew Positive Skew

• For a symmetric distribution positive values of (Y − µY )3 are offset by

negative values (equally likely) and skewness is 0
• For a negatively (positively) skewed distribution negative (positive)
values of (Y − µY )3 are more likely and the skewness is negative
(positive).
• Skewness is unit free.
 21

Measures of the shape of a probability distribution: Kurtosis

Kurtosis is a measure of how much mass is in the tails of a distribution

h i
E (Y − µY )4
Kurtosis =
σY4

.4 .4

.3 .3

.2 .2
.

.
.1 .1

0 0
-4 -2 0 2 4 -10 -5 0 5 10
Low kurtosis High kurtosis

• If a random variable has extreme values “outliers” the kurtosis will be

high

• The Kurtosis is unit free and cannot be negative

 22

Two random variables and their joint distribution

• Most of the interesting questions in economics involve 2 or more random

variables

• Answering these questions requires understanding the concepts of joint,

marginal and conditional probability distribution.

The joint probability distribution of two random variables X and Y can be

written as Pr (X = x, Y = y )

• Let Y equal 1 if it snows and 0 if it does not snow.

• Let X equal 1 if it is very cold and 0 if it is not very cold.

Joint probability distribution of X and Y

Very cold (X = 1) Not very cold (X = 0) Total
Snow (Y = 1) 0.15 0.07 0.22
No snow (Y = 0) 0.15 0.63 0.78

Total 0.30 0.70 1.00

 23

Two random variables and the marginal distributions

The marginal probability distribution of a random variable is just another
name for its probability distribution

• The marginal distribution of Y can be computed from the joint

distribution of X and Y by adding up the probabilities of all possible
outcomes for which Y takes a specific value
l
X
Pr (Y = y ) = Pr (X = xi , Y = y )
i=1

• The probability that it will snow

Pr (Y = 1) = Pr (X = 1, Y = 1) + Pr (X = 0, Y = 1) = 0.22

Joint probability distribution of X and Y

Very cold (X = 1) Not very cold (X = 0) Total
Snow (Y = 1) 0.15 0.07 0.22
No snow (Y = 0) 0.15 0.63 0.78

Total 0.30 0.70 1.00

 24

Two random variables and the conditional distribution

The conditional distribution is the distribution of a random variable

conditional on another random variable taking on a specific
value.

• The conditional probability that it snows given that is it very cold

0.15
Pr (Y = 1 | X = 1) = = 0.5
0.3
• In general the conditional distribution of Y given X is
Pr (X = x, Y = y )
Pr (Y = y | X = x) =
Pr (X = x)
• The conditional expectation of Y given X is
k
X
E (Y | X = x) = yi Pr (Y = yi |X = x)
i=1

The expected value of snow given that it is very cold equals

E (Y | X = 1) = 1 · Pr (Y = 1 |X = 1) + 0 · Pr (Y = 0 |X = 1)
1 · 0.5 + 0 · 0.5 = 0.5
 25

The law of iterated expectations

Law of iterated expectations states that the mean of Y is the weighted
average of the conditional expectation of Y given X , weighted
by the probability distribution of X .
X
E (Y ) = E [E (Y | X )] = E (Y | X = xi ) · Pr (X = xi )
i

Men Women
∙ Pr (IQ|G=male) (IQ|G=female)

If we use the second circle to compute E (IQ):

E (IQ) = E [E (IQ | G)]

P
= i E (IQ | G = gi ) Pr (G = gi )

= E (IQ | G = m) · Pr (G = m) + E (IQ | G = f ) · Pr (G = f )
 26

Independence

Independence: Two random variables X and Y are independent if the

conditional distribution of Y given X does not depend on X

Pr (Y = y | X = x) = Pr (Y = y )

• If X and Y are independent this also implies

Pr (X = x, Y = y ) = Pr (X = x) · Pr (Y = y |X = x)
(see slide 23)
= Pr (X = x) · Pr (Y = y )

Mean independence: The conditional mean of Y given X equals the

unconditional mean of Y

E (Y | X ) = E (Y )

• For example if the expected value of snow (Y ) does not depend on

whether it is very cold (X )

E (Y | X = 1) = E (Y | X = 0) = E (Y )
 27

Covariance

The covariance is a measure of the extend to which two random variables X

and Y move together,

Cov (X , Y ) = σXY = E [(X − µX ) · (Y − µY )]

Pk Pl
= i=1 j=1 (xj − µX )(yi − µY ) · Pr (X = xj , Y = yi )

Very cold (X = 1) Not very cold (X = 0) Total

Snow (Y = 1) 0.15 0.07 0.22
No snow (Y = 0) 0.15 0.63 0.78
Total 0.30 0.70 1.00

Example: the covariance between snow (Y) and it being very cold (X):
Cov (X , Y ) = (1 − 0.3) (1 − 0.22) · 0.15
+ (1 − 0.3) (0 − 0.22) · 0.15
+ (0 − 0.3) (1 − 0.22) · 0.07
+ (0 − 0.3) (0 − 0.22) · 0.63
= 0.084
 28

Correlation

• The units of the covariance of X and Y are the units of X multiplied by

the units of Y

• This makes it hard to interpret the size of the covariance.

The correlation between X and Y is unit free:

Cov (X , Y ) σXY
Corr (X , Y ) = p =
Var (X )Var (Y ) σX σY

• A correlation is always between -1 and 1 and X and Y are uncorrelated

if Corr (X , Y ) = 0

• If the conditional mean of Y does not depend on X, X and Y are

uncorrelated

if E (Y | X ) = E (Y ) , then Cov (X , Y ) = 0 & Corr (X , Y ) = 0

• If X and Y are uncorrelated this does not necessarily imply mean

Independence!
 29

Means, Variances and covariances of sums of random variables

Let
Z = aX + bY
The mean of Z equals

E (Z ) = E (aX + bY ) = aE (X ) + bE (Y )

The variance of Z equals

n o
Var (Z ) = Var (aX + bY ) = E [(aX + bY ) − (aµX + bµY )]2
n o
= E [a (X − µX ) + b (Y − µY )]2
 
= E a2 (X − µX )2 + b2 (Y − µY )2
+2ab (X − µX ) (Y − µY )
h i h i
a2 E (X − µX )2 + b2 E (Y − µY )2
=
+2abE [(X − µX ) (Y − µY )]

= a2 Var (X ) + b2 Var (Y ) + 2abCov (X , Y )

 30
[email protected]. Printing is for personal, private use only. No part of this book may be reproduced or transmitted without publish
Means, Variances and covariances of sums of random variables
n. Violators will be prosecuted.

Carefully study Key concept 2.3!

Examples of often used probability distributions in
Econometrics
 32

The Normal distribution

The most often encountered probability density function in econometrics is
the Normal distribution:
 
1 1 y − µ
fY (y ) = √ exp −
σ 2π 2 σ

• A normal distribution with mean µ and standard deviation σ is denoted

as N (µ, σ)
.

µ-1.96*σ µ µ-1.96*σ
.
 33
BY: [email protected]. Printing is for personal, private use only. No part of this book may be reproduced or transmitted without
The Normal distribution
sion. Violators will be prosecuted.

A standard normal distribution N (0, 1) has µ = 0 and σ = 1

A random variable with a N (0, 1) distribution is often denoted by Z and the
CDF is denoted by Φ (z) = Pr (Z ≤ z)
 34

The Normal distribution

To look up probabilities of a general normally distributed random variable

Y ∼ N (µ, σ)

we must first standardize Y to obtain the standard normal random variable Z

(Y − µ)
Z =
σ

• For example let Y ∼ N(5, 2)

 
(Y −5) (0−5)
Pr (Y ≤ 0) = Pr 2
≤ 2

= Pr (Z ≤ −2.5)
= 0.0062
 35

The Chi-Squared distribution

The chi-squared distribution is the distribution of the sum of m squared
independent standard normal random variables
• Let Z1 , Z2 , ..., Zm be m independent standard normal random variables
• The sum of the squares of these random variables has a chi-squared
distribution with m degrees of freedom
m
X
Zi2 ∼ χ2m
i=1
.

Chi-squared distribution with 5 degrees of freedom

 36

The Chi-Squared distribution

ion to Econometrics, Update, Global Edtion

• The chi-squared distribution is used when testing hypotheses in
econometrics

• Appendix table 3 shows the 90th 95th and 99th percentiles of the
Y: [email protected]. Printing is for personal, private use only. No part of this book may be reproduced or transmitted wi
χ2 -distribution
on. Violators will be prosecuted.
P 
3
• For example Pr i=1 Zi2 ≤ 7.81 = 0.95
 37

The Student t distribution

Let Z be a standard normal random variable and W a Chi-Squared
distributed random variable with m degrees of freedom
The Student t-distribution with m degrees of freedom is the distribution the
Z
random variable √W
/m

2.5% 95% 2.5%

-2.57 2.57
Student t distribution with 5 degrees of freedom

• The t distribution has fatter tails than the standard normal distribution.
• When m ≥ 30 it is well approximated by the standard normal
distribution.
 38

The Student t distribution

• The Student t distribution is often used when testing hypotheses in

on to Econometrics, Update, Global Edtion
econometrics P
• Appendix Table 2 shows selected percentiles of the tm distribution

• For example with m = 5;

[email protected]. Printing is for personal, private use only. No part of this book may be reproduced or transmitted wit
n. Violators will be prosecuted.
Pr (t < −2.57) = 0.025; Pr (|t| > 2.57) = 0.05; Pr (t > 2.57) = 0.025
 39

The F-distribution
Let W a chi-squared random variable with m degrees of freedom and V a
chi-squared random variable with n degrees of freedom.

The F-distribution with m and n degrees of freedom Fm,n is the

W
distribution of the random variable V//nm
.

3.52 5.19
F distribution with m=4 and n=5 degrees of freedom
tion to Econometrics, Update, Global Edtion 40
Page 1
The F-distribution
The 90th, 95th & 99th percentiles of the Fm,n distribution are shown in Table 5
Y: [email protected]. Printing is for personal, private use only. No part of this book may be reproduced or transmitted without publi
ion. Violators will be prosecuted.
• For example with m = 4 and n = 5;

Pr (F > 3.52) = 0.10; Pr (F > 5.19) = 0.05

on to Econometrics, Update, Global Edtion Page 1 o

[email protected]. Printing is for personal, private use only. No part of this book may be reproduced or transmitted without publishe
n. Violators will be prosecuted.
 41

The F-distribution
• A special case of the F distribution which is often used in econometrics
Introduction to Econometrics, Update, Global Edtion Page 1 of 1
is when Fm,n can be approximated by Fm,∞
• In this limiting case the denominator is the mean of infinitely many
squared standard normal random variables, which equals 1.
PRINTED BY: [email protected]. Printing is for personal, private use only. No part of this book may be reproduced or transmitted without publisher's
prior permission. Violators will be prosecuted.
• Appendix Table 4 shows the 90th, 95th and 99th percentiles of the Fm,∞
distribution.