Scribd
Upload
59 views

EC381/MN308 Probability and Some Statistics Lecture 3 - Outline

This document provides an outline for Lecture 3 of the course EC381/MN308 Probability and Some Statistics. The lecture will cover the topics of independence, conditional probability, and Bayes' theorem. It includes examples and definitions of independence, conditional probability, Bayes' theorem, and the total probability theorem. It also discusses independence versus conditional independence and pairwise independence versus mutual independence of multiple events. The document is an outline for students to understand the key concepts that will be covered in Lecture 3.

Uploaded by

Magdy Adel
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
59 views

EC381/MN308 Probability and Some Statistics Lecture 3 - Outline

This document provides an outline for Lecture 3 of the course EC381/MN308 Probability and Some Statistics. The lecture will cover the topics of independence, conditional probability, and Bayes' theorem. It includes examples and definitions of independence, conditional probability, Bayes' theorem, and the total probability theorem. It also discusses independence versus conditional independence and pairwise independence versus mutual independence of multiple events. The document is an outline for students to understand the key concepts that will be covered in Lecture 3.

Uploaded by

Magdy Adel
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

1/30/2008

EC381/MN308 Probability and Some Statistics


Yannis Paschalidis [email protected], http://ionia.bu.edu/

Lecture 3 - Outline
1. Independence. 2. Conditional probability. 3. Bayes Theorem.

Dept. of Manufacturing Engineering Dept. of Electrical and Computer Engineering Center for Information and Systems Engineering

EC381 2007/2008

EC381 2007/2008

1.3 Two Events: A, B


Independence & Conditional Probability A. Independence
Two events A and B are said to be independent if, and only if, the probability of the event AB=AB is the product of the individual probabilities: P [AB] = P [A] P [B] 1 P [A] P [B] C B 0 P [AB]

B. Conditional Probability
The probability of observing an event A, given that another event B has been observed, is called the conditional probability, and is defined as:
A AB

assuming that P [B] > 0. The quantity P [A|B] is called the probability of A given B. Similarly, Therefore,

If A and B are independent, then

so that
EC381 2007/2008 EC381 2007/2008

i.e. it is irrelevant to A whether B is measured or not.

Properties of Conditional Probabilities:


i) P [A|B] [0,1] ii) P [A|A] = 1 iii) If A = A1 A2 countable and mutually exclusive, then P [A|B] = P [A1|B] + P [A2|B] + Proof:
A1 A2 A3

C. Bayes Theorem

Proof:

Bayes Theorem provides a technique for evaluating the likelihood of cause, based on the observation of effect.
For example, for diagnosis, want to compute P [B|A], what is the likelihood of cause, given the observed effect. (Use in quality control.)

A
EC381 2007/2008 EC381 2007/2008

1/30/2008

Models based on Conditional Probability


Compound experiments:
e.g., first pick a factory, then observe the product; first pick a test to perform, then generate the test result.

D. Total Probability Theorem


Given countable B1, B2, , which are mutually exclusive and collectively exhaustive events, and event A:

Conditional probability and Bayes theorem provide a basis for generating a probability law, with the h l of help fi inference: f
e.g., first observe the product, then determine the probability it was produced in a given factory. Given an observed bit sequence 00010101 with possible errors, then determine the probability that the true bit sequence is 00110101?

Proof: follows from the p partition p property p y for events.


B 3
A

B 1

B 2

B 4

Divide (that is, condition) and Conquer idea


EC381 2007/2008 EC381 2007/2008

Example 1
Experiment: roll two 6-sided dice, all 36 outcomes equally likely Events:
B: {(x,y): min(x,y) = 3} (red triangles in figure) M = {(x,y): x = 3} (blue circles in figure) Determine P (M | B) P (MB) = P [{(3,3),(3,4),(3,5),(3,6)}] = 4/36 = 1/9 P (B) = P [{(3,3),(3,4),(3,5),(3,6),(4,3),(5,3),(6,3)] = 7/36 P (M | B) = P (MB)/P(B) = (1/9)/(7/36) = 4/7
y = second die
6 5 4 3 2 1 1 2 3 4 5 6

Example 2
You test positive for a disease. The test has a 5% false alarm rate and 0% misdetection. The unconditional probability for having the disease is 1/1000. What is the probability you actually have the disease ?
M B

x = first die
EC381 2007/2008

P [have disease | test +] P [have [h di disease test t t +] = P [test +] P [test + | have disease]P [have disease] = P [test + have disease] + P [test + no disease] 1 1/1000 = 1/1000 + P [test + | no disease]P [no disease] 1/1000 = = 0.0196 1/1000 + (5/100) (1 1/1000)
EC381 2007/2008

Example 3: Parts from two factories


2 Factories, making identical parts
Probability that part is OK from factory 1 is 0.9 Probability that part is OK from factory 2 is 0.8 Factory 1 makes 70% of parts sold at store, Factory 2 makes 30%

Example 4: Communication System


Experiment: Noisy channel changes binary signals randomly
Binary bit are transmitted randomly: Bit 0 transmitted with probability 0.9 Bit 1 transmitted with probability 0.1 After bit goes through channel: Output is correct bit with probability 0.95 Output is incorrect bit with probability 0.05
Factory 2

P [F1]=0.7
Factory 1 0.1

P [F2]=0.3

P [G|F1]=0.9
OK

P [G|F2]=0.8
OK 0.24

0.2

Bad 0.07

Bad

Question: If bit 0 was received, what is probability that a 0 was the original bit? Answer: Event a Event b Event A Event B
Bit 0 transmitted. Bit 1 transmitted. Bit 0 received. Bit 1 received. P [a]= 0.9 P [b]= 0.1 P [A|a]=0.95, P [A|b]=0.05 P [B|a]=0.05, P [B|b]=0.95

F1
0.63

0.7

F2
0 24 0.24

0.3

0.63

0.06

G1

G2

Q: What is the probability that, if the part you bought is OK, it was made in Factory 1? Event G: Part is OK, G = G1 G2 Event F1 : Part made in Factory 1. Want P(F1|G)

Determine P [a|A]

EC381 2007/2008

EC381 2007/2008

1/30/2008

Example 5: Monty Hall Game Show


We have a prize in one of 3 envelopes
You pick one, sealed. I know which envelope has the prize. I open one of two remaining envelopes, and show you that it is empty I give you choice of swapping your envelope for the remaining envelope. Should you swap?

Example 6
3 factories making a product (e.g., 1 k resistors). 70% of product from factory 1 meets spec, 80% from factory 2, 85% from factory 3. Factory 1 produces 40% of total product, Factory 2 produces 30%, Factory 3 produces 30% What is probability that a sample product meets spec? Event G: Product good. Fi: product comes from factory i.

P [G] =
If we stick with original choice we win with probability 1/3. If we switch:
We loose iff we pick the right envelope initially, hence, we win with probability (1-1/3)=2/3 !
EC381 2007/2008 EC381 2007/2008

i=1

3 X

P [G | Fi]P [Fi ] = 0.775.

1.4 Multiple Events


Independence & Conditional Probability

B. Conditioning can affect independence


A. Conditional Independence
Given C, A and B are said to be conditionally independent if: P [AB|C] = P [A|C] P [B|C]

A. Multiplication Rule
P [A B] = P [B|A] P [A] P [A B C] = P [C|A B] P [B|A] P [A]

Proof:
Two events that are not independent may become independent conditioned on a third event being true. Two independent events may become dependent when a third event is observed as true.

EC381 2007/2008

EC381 2007/2008

C. Independence of multiple events


Events A, B , and C are mutually independent if and only if any pair of these events are

Sequential Experiments
Experiment is sequence of sub-experiments
Select factory, then generate product Put bits in channel, let channel corrupt each bit

independent, and:
P [ABC] = P [A] P [B] P [C]

Can generate outcomes sequentially, using a tree


Simplifies greatly when sub-experiments have some form of independence of events

A1, A2 , , An are mutually independent if and only if every set of n-1 events taken from A1, A2 , , An are independent and
P [A1 A2 An] = P [A1] P [A2] P [An]
EC381 2007/2008 EC381 2007/2008

1/30/2008

Example 1: Independence vs pairwise independence


Two independent samples of a fair coin
S = {HH, HT, TH, TT}, each atom has probability Event A = outcome of first sample is H P [A] = Event B = outcome of second sample is H P [B] = Event C = outcome of first and second samples are equal P [C] = P [A B] = P [{HH}] = = P [A] P [B] P [A C] = P [{HH}] = = P [A] P [C] pairwise independent P [B C] = P [{HH}] = = P [B] P [C] P [A B C] = P [{HH}] = , not P [A] P [B] P[C] (!!!)
A T HT TT T H T C

Example 2 : Independence vs pairwise independence


3 machines Bi making resistors
Machines get selected as first step Selected machine generates resistor, Acceptable or Not, as a subexperiment Tree Diagram
P [B1] = P [A|B1] =

HH

H
EC381 2007/2008

EC381 2007/2008

Example 1.24, Fig. 1.2 (p. 25) in book

You might also like