Probability & Random Process

Chapter I
 Basics
Probability & Random Process

2
 PROBABILITY THEORY
Probability theory deals with the study of random
phenomena, which under repeated experiments yield different
outcomes that have certain underlying patterns about them.

Laplaces Classical Definition: The Probability of an event A

is defined a priori without actual experimentation as
Number of outcomes favorable to A
P( A) ,
Total number of possible outcomes
provided all these outcomes are equally likely.

3
 Example1: A spinner has 4 equal sectors colored yellow, blue,
green and red. What are the chances of landing on blue after
spinning the spinner? What are the chances of landing on red?
Solution:

The chances of landing on blue are 1

in 4, or one fourth

The chances of landing on red are 1 in

4, or one fourth

4
 Example 2: A single 6-sided die is rolled. What is the probability
of each outcome? What is the probability of rolling an even
number? of rolling an odd number?

Solution: The possible outcomes of this

experiment are 1, 2, 3, 4, 5 and 6.
P(1) = Number of outcomes favourable to 1 = 1
Total number of possible outcomes 6

P(even) = Number of outcomes favourable to even = 3 = 1

Total number of possible outcomes 6 2

5
Example 3: A glass jar contains 6 red, 5 green, 8 blue and 3 yellow
marbles. If a single marble is chosen at random from the jar, what is
the probability of choosing a red marble? a green marble? a blue
marble? a yellow marble?

P(red) = Number of outcomes favourable to red = 6 = 3

Total number of possible outcomes 22 11

P(blue) = 8 = 4 P(green) = 5 P(yellow) = 3

22 11 22 22

6
 Relative Frequency Definition: The probability of an
event A is defined as

nA
P ( A) lim
n n

where nA is the number of occurrences of A and n is the

total number of trials.
axiomatic probability theory specifies a set of axioms for
a well defined mathematical model of physical
experiments with random outcome.
7
 Review of Set Notation
Since probability theory relies on the concept of set theory

heavily, a review of set theory is given in the following

sections.

A set is collection of objects, which are the elements of the sets.

If every element in A also belongs to B, we say that A is subset

of B,

If A is subset of B and B is subset of A, A = B

If A is subset of B but AB, A is proper subset of B

universal set that contains all objects

8
 has subsets A, B, C,. Recall that if A is a subset
of , then A implies . From A and B, we
can generate other related subsets A B, A B, A, B,
A B | A or B
etc.

A B | A and B
and

A | A

A
A B A B A

A B A B A

9 Fig.1.1
 A and B are said to be mutually exclusive (M.E), if
A B ,
Complement of A , A = { : A}
The set difference operation is defined by B \ A B A
i.e., { | B and A}

B Fig. 1.2
A A B

A B B \ A A B

10
 A partition of is a collection of mutually exclusive

subsets of such that their union is .

A1
A2
Ai Ai Aj , and A .
i (1-5)
Aj An i 1

11
 Set Operation obey the following relations
Commutative property
A B B A and A B B A
Associative

A B C A B C and A B C A B C

Distributive

A B C A B A C and
A B C A B A C

12
 De-Morgans Laws:

A B A B ; A B A B

A B A B A B A B

A B A B A B

Fig.1.3

13
 Probabilistic Model
Probabilistic model: is a mathematical description of an uncertain
situation.
To model systems that yield uncertain or random measurements, we
let denote the set of all possible distinct, measurements that could
be observed. The set is called the sample space.
Finite/infinite or Countable/ uncountable sets

Elements or points in the sample space are called outcomes.

1 , 2 ,, k ,
Collections of outcomes are called events.

14
 Cont
We can take dice rolling; with the possible outcomes

as
,1,2,3,4,5,6

Any Combination of the above outcomes can be

considered as an event.

15
 Example: Consider the experiment where two coins are
simultaneously tossed. The various elementary events
are

1 ( H , H ), 2 ( H , T ), 3 (T , H ), 4 (T , T )
and
1 , 2 , 3 , 4 .

The subset A 1 , 2 , 3 is the same as Head

has occurred at least once and qualifies as an event.
16
 Probability law:
assigns a non negative number P(A) to a set A of possible
outcomes (also called events) that encodes our
knowledge about the collective likelihood of the
elements of A.

17
 A mathematical model for tossing of a fair die
Sample space modeling the possible results is:

1,2,3,4,5,6.

Now define an event

For example, if we are interested in the probability of the

die showing a face with an even number of dots, this
event can be represented as
E={2,4,6}

18
 The probability of an event is given as
E
P( E )

|E| denotes number of element in E or the cardinality of

the event under consideration.
|| is cardinality of the subspace.
With this definition

P(E)= 3/6 =
The probability law must satisfy basic properties
called axioms of probability
19
 Axioms of Probability
For any event A, we assign a number P(A), called the
probability of the event A. This number satisfies the
following three conditions that act the axioms of
probability.
(i) P( A) 0 (Probabili ty is a nonnegativ e number)
(ii) P() 1 (Probabili ty of the whole set is unity)
(iii) If A B , then P( A B ) P( A) P( B ).

(Note that (iii) states that if A and B are mutually exclusive (M.E.) events, the
probability of their union is the sum of their probabilities.)
20
 The following conclusions follow from these
axioms:
a. Since A A , we have using (ii)
P( A A) P() 1.
But A A , and using (iii),
P( A A) P( A) P( A) 1 or P( A) 1 P( A).

b. Similarly, for any A, A .

Hence it follows that P A P( A) P( ) .

But A A, and thus P 0.

21
 c. Suppose A and B are not mutually exclusive
(M.E.)?
How does one compute P( A B ) ?
Let us write A + B in terms of two disjoint
function
A B A AB,
where A and AB are clearly M.E.
events. A B

22
Thus using axiom (iii)

P( A B ) P( A AB ) P( A) P( AB ).
To compute P ( AB ), we can express B as
B AB AB

P( AB ) P( B ) P( AB)

P( A B ) P( A) P( B ) P( AB).

23
 d. If B is subset of A

P(A) > P(B)

Write A in terms of disjoint sets

A B AB
P( A) P( B) P(A B)
A
B
Since both probabilities are
non-zero values

P(A) > P(B)

24
 Conditional probability

Conditional probability provides us with a way to reason

about the outcome of an experiment based on partial
information
P( A B)
P( A | B)
P( B)

Where P(A|B) is the conditional probability of event A given

that event B has already occurred.

25
 Example: We toss a fair coin three successive times. We
Wish to find the conditional probability P(A/B) when A
B are the events.
A = {more heads than tails come up}
B ={1st toss is a head}
Solution: Select the possible sample space
= {HHH, HHT , HTH , HTT, THH, THT, TTH ,TTT}
B = {HHH, HHT , HTH , HTT} A = {HHH, HHT , HTH , THH}
P(A/B) = P(A n B)/P(B)
A n B = {HHH, HHT , HTH}
P(A n B) = 3/8 P(B) = 4/8 = 1/2 P(A/B) = 3/8 = 3/4
26 1/2
 Total probability and Bayes Theorem
Total probability Theorem

Let A1 , A2..Am be disjoint events that form a partition of the

sample space and assume P(Ai) > 1 for i = 1,2 m. Then for any
Event B, we have
P( B) P( A1 B) ..... P( Am B)
P( B) P( A1 ) P( B / A1 ) ..... P( Am ) P( B / Am )
A1
B

A2 A3
27
 Bayes Rule

Let A1 , A2..Am be disjoint events that form a partition of the

sample space and assume P(Ai) > 1 for i = 1,2 m. Then for any
Event B, such that P(B) > 0
P( Ai ) P( B / Ai )
P( Ai / B)
P( B)
P( B) P( A1 ) P( B / A1 ) ..... P( Am ) P( B / Am )
Since,

P( Ai ) P( B / Ai )
P( Ai / B)
P( A1 ) P( B / A1 ) ..... P( Am ) P( B / Am )

28
 Example: we have four boxes
Box1 = 2000 component , 5 % defective
Box2 = 500 component , 40 % defective
Box3 = 1000 component , 10 % defective
Box4 = 1000 component , 10% defective
We select one box at random and we remove single component at
random
a. What is the probability that the selected component is defective?
Box1 = 1900g, 100d Box2 = 300g, 200d
Box3 = 900g, 100d Box4 = 900g, 100d
P(D) = P(B1)P(D/B1) + P(B2)P(D/B2) + P(B3)P(D/B3) + P(B4)P(D/B4)

P(B1)= P(B2)=P(B3)=P(B4)= 1/4

P(D/B1)= 0.05, P(D/B2)= 0.4 ,P(D/B3)= 0.1, P(D/B4)= 0.1
P(D) = (1/4)*0.05 + (1/4)*0.4 + (1/4)*0.1 + (1/4)*0.1 = 0.1625
29
 (b) Suppose we the selected component is found to be

defective. What is the probability that it came from

box 2? P( B2 | D) ?

P( D | B2 ) P( B2 ) 0.4 0.25
P( B2 | D) 0.615.
P ( D) 0.1625

Notice that initially P( B2 ) 0.25; then we picked out

a box at random and tested a bulb that turned out to
be defective. Can this information shed some light
about the fact that we might have picked up box
30 2?
 Independence

Two events A and B are said to be statistically independent, or just

independent, if

P( A B) P( A) P( B)
A is independent of B if occurrence of B provides no information to
the probability that A has occurred
P( A | B) P( A)
For more than two events

P( Ai A j Ak ) P( Ai ) P( A j ) P( Ak )

31
 Example: If two experiments ,rolling a die and tossing a

coin are done simultaneously where the events are

independent. Then, the probability of getting head and six
dots

P ( H 6) P ( H ) P (6)
P ( H 6) 1 / 2 *1 / 6 1 / 12

32
 Example 1: The probability that a cell in a wireless system is
overloaded is 1/3. Given that it is overloaded, the probability
of a blocked call is 0.3. Given that it is not overloaded, the
probability of a blocked call is 0.1. Find the conditional
probability that the system is overloaded given that your call
is blocked.

33
 Soln:

1 2
P (O) P (O ) c

3 3
P ( B | O ) 0.3
P ( B | O ) 0.1
c

P (O | B ) ?
P ( B | O) P (O )
P (O | B )
P ( B | O ) P (O) P( B | O ) P (O )
c c

1
0.3 *
P (O | B ) 3 0.6
1 2
0.3 * 0.1*
34 3 3
 2) Suppose box 1 containers x white balls and y black
balls, and box 2 containers w white balls and z black.
One ball of unknown color is transferred from box 1 to
box 2 and then a ball is drawn from the second box.
What is the probability that it will be white?

35
 Soln: If a ball is not transferred from box 1 to box 2 then

w z
P (W ) P( B)
w z w z

36
A { the drawn ball is white}
B {the transferr ed ball is white}
C {the transferr ed ball is black}
P(A) {B and the probabilit y of event A
given the transferr ed ball is white} or
{C and the probabilit y of event A
given the transferr ed ball is black}
P( A) P( A / B ) * P( B ) P( A / C ) * P(C )
x y
(( ) * ( w 1) /( w z 1)) (( ) * ( w /( w z 1)))
x y x y

37
 3. Ten percent of the population suffer from a certain disease.
Each time a person with the disease is tested for the disease,
there is a 90% chance that the test will show a positive result.
Each time a person without the disease is tested, there is a 1%
chance of it showing a positive result. A person is chosen at
random from the population.
(a) Determine the probability that the test gives a positive
result for the chosen person.
(b) Determine the probability that the chosen person has the
disease given that the test was positive.

38
 Soln. Let P be the event that the test is positive and
H be the event that the person has the disease.
(a) By the Theorem of Total Probability
P(P) = P(P|H)P(H) + P(P|Hc)P(Hc) = 99/1000

(b) By Bayes Theorem

P(H|P) = P(P|H)P(H) = 10/11
P(P)

39