Scribd
Upload
5 views

Mstat Note8 Expectation f23

Uploaded by

junmokim123
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
5 views

Mstat Note8 Expectation f23

Uploaded by

junmokim123
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 45

Expectation

Math & Stat for Data Science


Graduate School of Data Science
Seoul National University
Expectation
• Suppose researchers are interested in the height of
Korean population
• How can we summarize the data?
Expectation
• Now researchers are interested in both height and
weight
• How we can capture the relationship between these
two?
Expectation

• One-number summary of the distribution


• Can be approximated by sample mean (for IID samples)
Expectation (Binary Dist)
• X~ Bernoulli(p)
• E(X) = p
Expectation (Binary Dist)
• X~Binomial(n,p)
• E(X) = np
Expectation (Binary Dist)
• X~Poisson(𝜆)
• E(X) = 𝜆
Expectation

• Expected value of transformed variables can be


easily calculated using the above theorem.
• For binary variables (or multivariate variables)
Example

E(r(X))?
Example

E(r(X, Y))?
Properties of Expectation

• Very useful property!


• Do not require independence of X!!
• Example: mean of Binomial (n, p)?
Properties of Expectation

• Requires independence of X
• Example: Mean of XY, where X and Y are independent and
X~Bernoulli(p1) and Y~ Bernoulli(P2)
Variance and Covariance

Normal, mu=0, sigma=1

• Variance
0.4
0.3

represents the
Density

0.2

spread of
0.1

distribution
0.0

-4 -2 0 2 4

x
Variance – Important properties
Variance
• Example: Variance of binomial(n, p)?
Variance
• Sample mean and variance
Covariance and Correlation

• Indicate the strength of linear relationship between


two random variables X and Y
Covariance and Correlation
corr=0 corr=0.8
3

2
2

1
1
0
Y

0
-1

-1
-2

-2
-3

-3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3

X X
Covariance and Correlation

Correlation?
8
6
4
Y

2
0
-2

-3 -2 -1 0 1 2 3

X
Covariance and Correlation
Covariance and Correlation
Important RVs
Multivariate &
Conditional Expectation &
Monte-Carlo approach
Multivariate RVs
Mean
Vector

Covariance
Matrix
Multivariate RVs

• Very useful to identify the mean and variance of


linear combination of random variables
Multivariate RVs
$
• Example: Suppose 𝑋! , 𝑋" , 𝑋# have the following
mean and variance
0 1 0.5 0.5
𝜇 = 1 , Σ = 0.5 1 0.5 ,
2 0.5 0.5 1

• Let 𝑌 = 𝑋! + 0.5𝑋" + 0.5𝑋#. Mean and variance of Y?


Conditional Expectation
Conditional Expectation
• Given Y, what is the expected values of X?
• Ex. Given Height=180 cm, what is the expected value of
Weight

• Important: E(X|Y=y) & E(r(X,Y)|Y=y) are functions of


y. So E(X|Y) is a random variable of Y
• Ex. X ~ Uniform(0,1), and Y~Uniform(x,1) given X=x.
Then E(Y | X) = (1+x)/2
Conditional Expectation
Conditional Variance

Conditional Variance is also a RV


Conditional Expectation and
Variance
• Example: Suppose height of male ~ N(173, 52) and
female ~ N(163, 42). The numbers of males and
females are the same
• Mean height?
Conditional Expectation and
Variance
• Example: Suppose height of male ~ N(173, 52) and
female ~ N(163, 42). The numbers of males and
females are the same
• Variance?
Monte-Carlo simulation
• Suppose we want to calculate E(f(x)) and V(f(x)), where
distribution of x is know and easily sampled
• EX. x ~ N(0,1) and f(x) = x3

• In many situations, it is difficult to get them analytically

• Monte Carlo approach can be used


• Simulate x, B times, 𝑥! , … , 𝑥"
• Estimate E(f(x)) as the sample mean
1 $
𝐸 𝑓 𝑥 ≈ 2 𝑓(𝑥! )
𝑛 !"#
Monte-Carlo simulation
• EX. Suppose that (height,weight) in Korean Male
follows MVN with
173 5 % 10 ,
𝜇= ,Σ =
68 10 4%

• Mean and variance of BMI (kg/m2)?


Monte-Carlo simulation: BMI
MGF
Moment Generating Function

• Have all the information on the distribution


• Useful to derive the distribution of sum of random
variables
• Moment calculation:
Moment Generating Function
MGF
• Example:
• Let X ~ Binomial(n, p). MGF of X?
MGF
• Example:
• Let X ~ Binomial(n1, p), and Y~ Binomial (n2, p). MGF of
X+Y?
MGF

• MGF is widely used when to derive the


distribution!
MGF
MGF
• There are slightly different versions of similar
functions
• Cumulant Generating Function

• Characteristic function
MGF
• Using MGF (CGF and characteristic functions),
distribution function can be estimated
• This kind of technique can be very useful…
Summary
• Expectation
• One-number summary of the distribution
• Linear operator
• Variance
• Represent the spread of distribution
• Covariance & Correlation
• Indicate the (linear) relationship between two RV

• Conditional Expectation
• Moment generating function

You might also like