LECTURE 3: Independence

• Independence of two events

• Conditional independence
• Independence of a collection of events

• Pairwise independence
• Reliability
• The king 's sibling puzzle

1
 A model based on conditional probabilities
E (fl. I 1-1, ) ~ p:: f().it } T, )
• 3 tosses of a biased coin : P ( H ) = p, p eT) = 1 - p
f( Hz) : E(H,) f (l~ .. ) J.I ,)
1(\1,\ f./,) l (H)-I H, C1 ~~,) +- NT,) f.( Hz. 1T,)
-=p
• M ult iplicatio n rule: P (T /-IT) = (1- p) p(l- p)
.:mJ HfIT

HTH
, - p • T ota l probabi lity:
P ( l head) = 3 p(1 _1')7.
p niH
, . p • Bayes rul e:
p
I- p E(H niCleoJ,)
p
P(first t oss is H 11 head) = . .-:.._~'-~~--
.:.
•
!?. (l. (lead.)
I. p rrr ,

2
Independence of two events
n / A '\ 13
•
• Intuitive "definition": P CB I A) = P CB)
- occurrence of A provides no new inform ati on about B
1Ctj() ~) ::.f (t/) .f( Bill) =- f (A) f (s) "- /
p eA) > 0 P CB) > 0

In depend ent?
Definition of independence: p eA n B) = p eA) . PCB)

P(A0g)=0
f. (.'\ )!~. ( (3) ? 0
- Symm etri c w ith resp ect t o A and B

im p li es p eA I B) = p eA)

- appli es even if p eA) = 0

3
Independence of event complements

Definition of independence: P CA n B) = P CA) . P CB)

-
- -
• If A a nd B are indepe nd ent. then A and B e are independent.
Intuitive argument - Formal proof

A A :0 (I-Ing) u (A ()~ t)
1 (ll):: .E (4'1~) + f (1If)~(-)
:: tr4H (~) t f (.4~ 12/)
.£ (AI)P;~) ,,£ (11) -.H4)!(I»:: 1(4) (I - p. (1?»)

C
= f (A) .E (a )

4
Conditional independence

• Conditional independence, given C,

is defined as ind ependence under the probability law PC· I C)

n Assume A and B are ind ependent

/ A
'­ ./
A
"­ C
B

• If we are told that C occurred,

are A and B independent? fIIo

5
Conditioning may affect independence
• Two unfair coins, A and B:
P(H I coin A) = 0.9, P(H I coin B) = 0.1

• choose either coin with equal probability

• Are coin tosses independent?
0.9 tvO.
./
0.1

Coin A

Compare: " ( )
O. 1
0.9
P(toss 11 = H) =irA) £. (!-I" )A) ... .En~h flu 8»
0.1
= 0." d). 9 r 0.5' • o. I 'Z <!). 5"
O. 1 P(toss 11 = H I first 10 tosses are heads)
0.5
0.9

Coin B
O. 1

0.9

6
Independence of a collection of events

• Intuitive "definition": Inform ati on on so me of th e events

does not change pro babiliti es relat ed t o th e rem aining events

A, )A1 '''. i '" J e '(J ::::;> f. (A,!> I) A'1' ) :. f. (.q ') nA: \ A I U (,q2 l) t); )) •

1 (A:;) ::. 1 (4 ') 1A,1\ At) .: J (4; 1AI nA; ) '" .£ (A:; M,' 1)42 )
Definition: Events Al l A 2,"" A n are ca lled independent if:

for an y distin ct indi ces i , j , ... ) m

n = 3:

p eA , n A2) = p eA,) . P (A2)

p eA , n A3) = p eA,) . P (A3)

pairwise ind ependence
P (A 2 n A 3 ) = P (A 2 ) . P (A3)

7
Independence vs. pairwise independence
\-11
• Two independent fair cOin tosses
HH; HT
'4 'N
- H 1: First toss is H

- H 2: Second toss is H
'N
H~ TT
P( H , ) = P( H 2 ) = 1( 2

• C : the two tosses had the same result -: 1 H H," T~

.Y (Ii,n c) ,,1 (l-II()H~)" '/'1 .e (H/H (c.) " i.f =* l+ I) C: i .. ,hf..
H., c ~ ,.. <iff..
.r (11, () l-I'/. ~ <.) "f (H I-I) :. '1"1 '- d'
'- Itt·
1(Il,)i(U,JiC<.)::>'18 /
•

.1' C(. \ H,) :: f ()./2 \ I-} I ) = f.().)2.) .= 'h :: f Cc.)

.i ( c. \ ).J, cl)..)'/,) :::: i =fr 'f Cc..') = 'h

H l. H 2, a nd C ar e p airw ise ind ep endent, but n ot ind ep endent

8
Reliability U; : (, t h u "i t u p
Pi : pro bability th at unit i is " up" (./, ,Uy, .• . ,U", ~v>JQ.fcvoJe",t
inde pendent un its ,... . i 1:)'
r- • • u '" ~ I: 01.0 w .....

~ F: t ",o.(I.)?Il.1AJe'IA. f
probab ili t y th at syst em is " u p" 7

f (Sylte..... 1A-f') =1. el).., n(,(t n U,»

:-1((./,) t(U.)f (U~) ~ p, P. f.
I
Pl
1 (~r?.j." ... " "'1') : f (U, u U. u U'lo)

I P2 I
-:: I - .e (r:-, () F2 11 F3 ")
"1>3
" I- f ((:,) H ~2.) f (I=~)
" I _ (I - P,) ( t - )? ) (1 - F1 )
9
The king's sibling
•

• The king comes from a family of two children . /

What is the probability that h is Siblin'g is female?

~O7> ()",ve f~ecede"'ce !(l)c)'):o£.(<t{re);; ,/~

~ ... dl/. rell>de... t

~s SG­
_ -,.c.
,/q ,;;!._
/q
•

• /;; ee

10
MIT OpenCourseWare
https://ocw.mit.edu

Resource: Introduction to Probability

John Tsitsiklis and Patrick Jaillet

The following may not correspond to a particular course on MIT OpenCourseWare, but has been provided by the author as an individual learning resource.

For information about citing these materials or our Terms of Use, visit: https://ocw.mit.edu/terms.