Scribd
Upload
14 views

L1.5 Independence

The document discusses the concept of independence between events in probability. It defines independent events as those where the probability of their intersection is equal to the product of their individual probabilities. Conditional independence is defined as events being independent given some third conditioning event. Several examples are provided to illustrate independence, conditional independence, and how the two concepts differ.

Uploaded by

anurag.211ee215
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
14 views

L1.5 Independence

The document discusses the concept of independence between events in probability. It defines independent events as those where the probability of their intersection is equal to the product of their individual probabilities. Conditional independence is defined as events being independent given some third conditioning event. Several examples are provided to illustrate independence, conditional independence, and how the two concepts differ.

Uploaded by

anurag.211ee215
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 6

Statistical Foundation for Electrical Engineers

(EE343)

Unit-1: Probability Models and Axioms

Lecture 1.5: Independence

Krishnan C.M.C
Assistant Professor, E&E,
NITK Surathkal
Independence and Conditional Probability

Event A: roll a die Event B: flip a coin


and get a Six and get a Head

P(roll gives a six given coin flip gives a head)? 𝑃(𝐴|𝐵) = 𝑃(𝐴)

P(coin flip gives a head given roll gives a six given)? 𝑃(𝐵|𝐴) = 𝑃(𝐵)

Here it is intuitive

We know: But of A is independent of B then 𝑃 𝐴 𝐵 = 𝑃(𝐴)


𝑃 𝐴∩𝐵
𝑃 𝐴𝐵 = ⇒ 𝑃 𝐴 ∩ 𝐵 = 𝑃 𝐴 𝑃(𝐵)
𝑃 𝐵
It is a symmetric relation

hence we can say A and B are independent of each other

EE343, Dept. of E & E, NITK Surathkal


Conditional Independence
Two events A and B conditioned on a third event C are independent
of each other

𝑃 (𝐴 ∩ 𝐵 𝐶) = 𝑃 𝐴 𝐶 𝑃(𝐵|𝐶) 1

A and B are conditionally independent (on the conditioning event C)


Alternate View
𝑃 𝐴∩𝐵
From 𝑃 𝐴 𝐵 = 𝑃 𝐴∩𝐵 ∩𝐶
𝑃 𝐵 ⇒ 𝑃 (𝐴 ∩ 𝐵) 𝐶 = Multiplication rule
𝑃(𝐶)
𝑃 𝐶 𝑃 𝐵 𝐶 𝑃(𝐴|𝐵 ∩ 𝐶)
=
𝑃(𝐶)
= 𝑃 𝐵 𝐶 𝑃(𝐴|𝐵 ∩ 𝐶) 2

1 and 2 𝑃 𝐴 𝐶 = 𝑃(𝐴|𝐵 ∩ 𝐶)

This means: Event A given C has no effect whether B has occurred.

EE343, Dept. of E & E, NITK Surathkal


Independence: Some important Aspects
[1] Two events A and B are independent if
𝑃 𝐴 ∩ 𝐵 = 𝑃 𝐴 𝑃(𝐵) If 𝑃 𝐵 > 0 (non-zero) 𝑃 𝐴|𝐵 = 𝑃 𝐴

[2] If two events A and B are independent then 𝐴 and 𝐵𝐶 are also independent

[3] Two events 𝐴 and 𝐵 are said to be conditionally independent,


given another event 𝐶 with 𝑷(𝐶) > 0, if
𝑃 𝐴 ∩ 𝐵|𝐶 = 𝑃 𝐴|𝐶 𝑃(𝐵|𝐶)
[4] Independence DOES NOT imply conditional independence and visa versa
𝑷 𝑨 ∩ 𝑩 = 𝑷 𝑨 𝑷(𝑩) DOES NOT imply 𝑷 𝑨 ∩ 𝑩|𝑪 = 𝑷 𝑨|𝑪 𝑷(𝑩|𝑪)

[5] Independence of a collection of events

𝑷 ሩ 𝑨𝒊 = ෑ 𝑷 𝑨𝒊 For every subset 𝑆 of 1,2, … 𝑛


𝒊∈𝑺 𝒊∈𝑺
Ex: 𝑛 = 3
𝐴1 𝐴2 𝐴1 𝐴3 𝐴2 𝐴3 𝐴2
(pairwise independence) 𝐴3
𝐴1
EE343, Dept. of E & E, NITK Surathkal
Independence: Some important Aspects
[4] Independence DOES NOT imply conditional independence and visa versa
𝑷 𝑨 ∩ 𝑩 = 𝑷 𝑨 𝑷(𝑩) DOES NOT imply 𝑷 𝑨 ∩ 𝑩|𝑪 = 𝑷 𝑨|𝑪 𝑷(𝑩|𝑪)

Ex: Consider two independent fair coin tosses, in which all four
possible outcomes are equally likely. Let
• 𝐻1 = 1st toss is a head , 𝐻1
• 𝐻2 = 2nd toss is a head , 𝐻 𝐻 𝐻2
• 𝐷 = {The two tosses have different results} 𝐷 𝐻 𝑇
𝑇 𝐻
We can verify that 𝐻1 and 𝐻2 are independent 𝑇 𝑇
2 1 2 1 1
𝑃 𝐻1 = = 𝑃 𝐻2 = = 𝑃 𝐻1 ∩ 𝐻2 =
4 2 4 2 4
⇒ 𝑷 𝑯𝟏 ∩ 𝑯𝟐 = 𝑷 𝑯𝟏 𝑷 𝑯𝟐

Are they conditionally independent? (conditioned on 𝐷)


1 1 0
𝑃 𝐻1 |𝐷 = 𝑃 𝐻2 |𝐷 = 𝑃 𝐻1 ∩ 𝐻2 |𝐷 =
2 2 2
⇒ 𝑷 𝑯𝟏 ∩ 𝑯𝟐 |𝑫 ≠ 𝑷 𝑯𝟏 |𝑫 𝑷 𝑯𝟐 |𝑫
NOT conditionally independent
EE343, Dept. of E & E, NITK Surathkal
Independence: Some important Aspects
[5] Independence of a collection of events

𝑷 ሩ 𝑨𝒊 = ෑ 𝑷 𝑨𝒊 For every subset 𝑺 of 𝟏, 𝟐, … 𝒏


𝒊∈𝑺 𝒊∈𝑺
Ex: Consider two independent fair coin tosses, in which all four
possible outcomes are equally likely. Let
• 𝐻1 = 1st toss is a head , 𝐻1
• 𝐻2 = 2nd toss is a head , 𝐻 𝐻 𝐻2
• 𝐷 = {The two tosses have different results} 𝐷 𝐻 𝑇
𝑇 𝐻
𝐻1 𝐻2 𝐻1 𝐷 𝐻2 𝐷 𝐻2 𝑇 𝑇
(pairwise independence) 𝐷3
𝐻1

From earlier: 𝑃 𝐻1 = 1/2 𝑃 𝐻2 = 1/2 𝑃 𝐷 = 1/2

𝑃 𝐻1 ∩ 𝐻2 = 1/4 𝑃 𝐻1 ∩ 𝐷 = 1/4 𝑃 𝐻2 ∩ 𝐷 = 1/4 𝑃 𝐻1 ∩ 𝐻2 ∩ 𝐷 = 0


This implies
𝑃 𝐻1 ∩ 𝐻2 = 𝑃 𝐻1 𝑃(𝐻2 )
But
𝑃 𝐻1 ∩ 𝐷 = 𝑃 𝐻1 𝑃(𝐷) 𝑃 𝐻1 ∩ 𝐻2 ∩ 𝐷 ≠ 𝑃 𝐻1 𝑃 𝐻2 P(D)
𝑃 𝐻2 ∩ 𝐷 = 𝑃 𝐻2 𝑃(𝐷) EE343, Dept. of E & E, NITK Surathkal

You might also like