Lecture 2

probability and Random Processes

By Tafari Lemma

由NordriDesign提供
www.nordridesign.com
 Lecture Outline
• Random Experiment
• Sample Space & Events
• Discrete and Continuous Sample Spaces
• Mutually Exclusive Events
• Union & Intersection
• Complementary Events
• Conditional Probability
• Total Probability
• Independent Events
• Bayes’ Law

2
 Introduction
• In dealing with probabilities mathematically, it is more
convenient to express probabilities as fractions.

• Thus, we measure the probability of the occurrence of some

event by a number between zero and one.

• The more likely the event, the closer the number is to one; and
the more unlikely the event, the closer the number is to zero.

• An event that cannot occur has a probability of zero, and an

event that is certain to occur has a probability of one.

3
 Random Experiment

Definition: Random Experiment

Examples:

• Time to reboot a server (in seconds)

• Measuring the clock frequency in a motherboard (in GHz)

• Measuring the temperature evolution in a graphics processing

unit (GPU) while performing a benchmark task (in deg. Celsius)

4
 Sample Space

Definition: Sample Space

Examples:

• Time to reboot a server (in seconds)

• Measuring the clock frequency in a motherboard (in GHz)

• Age (in terms of number of birthdays) of a person

• Measuring the temperature evolution in a GPU while

performing a benchmark task (in deg. Celsius)

5
 Sample Space (S)
• The sample space is represented by the symbol S.
• There are 2 possible outcomes with the sample space of tossing a
coin; S = {Head, Tail}
• There are 6 possible outcomes with the sample space of rolling a
dice; S = {1, 2, 3, 4, 5, 6}
• There are 52 possible outcomes with the sample space of drawing a
card; S = {2♠, 2♣, 2♦, 2♥, 3♠, 3♣, 3♦, 3♥, ..., A♠, A♣, A♦, A♥}

6
Discrete and Continuous Sample Spaces

Examples:

• Time to reboot a server

• Measuring the clock frequency in a motherboard

• Age (in terms of number of birthdays) of a person

•Measuring the temperature evolution in a GPU while

performing a benchmark task

7
 More Examples
The choice between discrete or continuous sample spaces
might depend on the particular objective of the study…

• Messaging in a communication system:

•Three time sensitive messages are sent. These will either
be on time, or delayed

What is the sample space?

8
 Events

Definition: Events

9
 Events
Example:

Events are just sets contained in the sample space !!!

10
 Event Operations

Definition: Mutually Exclusive Events (Disjoint Events)

18

11
Venn Diagrams

19

12
 Interpretation of Probability

The concept of probability is a very abstract one:

•Idea: quantify the “likelihood” or “chance” of the outcome
of an experiment…

• One can have a subjective interpretation of probability

13
 Interpretation of Probability
• A frequentist interpretation of probability:

Relative Frequency Definition: The probability of an event A is defined as

nA
P ( A) = lim
n→ n

where n(A) is the number of occurrences of A and n is the total number of

trials.

• We can use the relative frequency definition to derive set of axioms

14
 Axioms of Probability
For any event A, we assign a number P(A), called the probability of
the event A. This number satisfies the following three conditions
that act the axioms of probability.

(i) P( A)  0 (Probability is a nonnegative number)

(ii) P() = 1 (Probability of the whole set is unity)
(iii) If A  B =  , then P( A  B ) = P( A) + P( B ).

15
 Axioms of Probability
Example: Random experiment – Flipping a coin

If we have a fair coin then

In general, if you have N possible outcomes:

16
 Basic Properties
Just from the axioms we can deduce a number of simple, but
useful properties:

17
 Addition Rules
Actually, the probability of an event is just a way to measure it !!
- you can think of it as the generalized volume of the set.

If the events are mutually exclusive then (and only then!):

18
 Conditional Probability
This is a very important concept. In most cases events “share”
information, and so we want to see how can we take this into
account

Example: taking a card out of a shuffled deck

19
 Conditional Probability

Definition: Conditional Probability

20
 Multiplication Rules
Multiplication Rule:

Total Probability Rule:

21
Total Probability Rule for Multiple Events

22
Independence of Events
Sometimes two events might not be “related”:

•Given that the outcome of the experiment is in A might not

affect the probability that this same outcome is also in B…
Example: Take a card out of a shuffled deck

This means that B doesn’t give any probabilistic

information about A !!!
23
Definition of Independence
Definition: Independence

Independence is a truly probabilistic concept, and not a set

relationship !!!

24
Properties

25
 Independence is IMPORTANT
Independence is a key assumption in most probability models, e.g.:

•We assume each memory chip from a certain manufacturer fails

independently from one another
•Processes arriving to a server from different users are assume
to arrive independently from one another

Example: There are three elevators in the MetaForum, but one is

currently broken. The other two fail independently with probability
0.2 and 0.1 respectively.

What is the probability I cannot take one of the elevators to get to the
7th floor?

A – 0.1 B – 0.2 C – 0.3 D – 0.02

26
 A Simple Example
We can think of this setting as a circuit:

Elevator 1

Elevator 2

27
 Another Example
Example: A harddrive backup unit uses redundancy to ensure
reliable operation. It consists of two redundant data storage units
(each with probability of failure of 1/100) and two independent
power supply units each with probabilities of failure respectively
1/50 (electric grid unit) and 1/120 (battery unit).

What is the overall probability of failure of such system?

Data Units Power Units

28
 Bayes’s Rule
In many situations in practice what we can measure (estimate) are
conditional probabilities. It is rather useful to be able to relate
various conditional probabilities to one another.

Despite it’s simplicity, this is a very powerful result, and forms the
basis of many inference procedures used nowadays (e.g. both the GPS
system and the communication encoding used in cellphones rely on
Bayes’s rule).

29
Bayes’s Rule

30
Example
Two boxes B1 and B2 contain 100 and 200 light bulbs respectively.
The first box (B1) has 15 defective bulbs and the second 5. Suppose
a box is selected at random and one bulb is picked out.
(a) What is the probability that it is defective?
(b) Suppose we test the bulb and it is found to be defective. What
is the probability that it came from box 1?

31
Example
Two boxes B1 and B2 contain 100 and 200 light bulbs respectively.
The first box (B1) has 15 defective bulbs and the second 5. Suppose
a box is selected at random and one bulb is picked out.
(a) What is the probability that it is defective?
Solution: Note that box B1 has 85 good and 15 defective bulbs.
Similarly box B2 has 195 good and 5 defective bulbs.
Let D = “Defective bulb is picked out”.
Then
15 5
P ( D | B1 ) = = 0.15, P ( D | B2 ) = = 0.025.
100 200

32
Cont’d
Since a box is selected at random, they are equally likely.
1
P ( B1 ) = P ( B2 ) = .
2
Thus B1 and B2 form a partition
And we obtain P( D) = P( D | B1 ) P( B1 ) + P( D | B2 ) P ( B2 )
1 1
= 0.15  + 0.025  = 0.0875.
2 2
Thus, there is about 9% probability that a bulb picked at random is
defective.

33
 Cont’d
(b) Suppose we test the bulb and it is found to be defective. What is the
probability that it came from box 1? P( B1 | D) = ?

P ( D | B1 ) P ( B1 ) 0.15  1 / 2
P ( B1 | D ) = = = 0.8571.
P( D) 0.0875

Notice that initially P( B1 ) = 0.5; then we picked out a box at random and
tested a bulb that turned out to be defective. Can this information shed
some light about the fact that we might have picked up box 1?

From above P( B1 | D) = 0.857  0.5, and indeed it is more likely

at this point that we must have chosen box 1 in favor of box 2. (Recall box1
has six times more defective bulbs compared to box2).

34
 Exercise
1. Three switches connected in parallel operate independently.
Each switch remains closed with probability p. (a) Find the
probability of receiving an input signal at the output. (b) Find
the probability that switch S1 is open given that an input signal
is received at the output.

35
 Exercise
1. Binary Symmetric Channel. This channel is said to be discrete in that it
is designed to handle discrete messages. The channel is memoryless in
the sense that the channel output at any time depends only on the
channel input at that time. The channel is symmetric, which means that
the probability of receiving symbol 1 when 0 is sent is the same as the
probability of receiving symbol 0 when symbol 1 is sent.

36
 Cont’d
The probabilities of sending binary symbols 0 and 1 is

The conditional probabilities of error is

Compute
a) The probability of receiving symbol 0
b) The probability of receiving symbol 1
c)

37
Thank You !!!