Chapter One

1. Overview of Basic of Probability Theory

1.1. Introduction
Personally, in our daily lives we are faced with a lot of decision-making that involve
uncertainty. Perhaps you may ask you self to analyse one of the following situations:

- What is the chance for me to score “A” in statistics?

- What is the likelihood that your (our/my) weekend picnic be successful?

- Etc

In such situations we use the concept of probability in our daily life without detailed and
actual knowledge of the concept in other words we use it intuitively.

Probability is:

 Numerical measure of the chance of likelihood that a particular event will occur
 A quantitative measure of uncertainty
 A measure of the strength of belief in the occurrence of an uncertain event
 Measured by a number between 0 and 1 (or between 0% and 100%)

1.2. Definition of Basic Concepts in Probability

 Experiment: is the Process that leads to the occurrence one of several possible
outcomes , e.g.:
 Coin toss : Heads, Tails
 Throw die : 1, 2, 3, 4, 5, 6

Note: Each trial of an experiment has a single observed outcome.

The precise outcome of a random experiment is unknown before a trial.

 Sample Space: - is the set of all possible outcomes that may occur as a result of a
particular experiment
 Event:- is Collection of outcomes having a common characteristics
Example

If someone takes three shots at a target and we care only whether each shot is a hit or a
miss, describe a suitable sample space, the elements of the sample space that

1
constitute event M that the person will miss the target three times in a row, and the
elements of event N that the person will hit the target once and miss it twice

Solution If we let 0 and 1 represent a miss and a hit, respectively, the eight
possibilities
S= {(0, 0, 0), (1, 0, 0), (0, 1, 0), (0, 0, 1), (1, 1, 0), (1, 0, 1), (0, 1, 1), (1, 1, 1)}

M = { (0,0,0) }

N= {(1, 0, 0), (0, 1, 0), (0, 0, 1)}

Events can be:

Simple Event: – is a subset of sample space that has exactly one sample point. It can also be
called as element or fundamental event.
Compound Event: – is a subset of sample space that has two or more sample points.
Complement Event: – the complement of event A is denoted by A’. A’ is the event that has
all the points in a sample space that are not in A.
E.g. rolling a die: S = {1, 2, 3, 4, 5, 6}
Event A = {1, 3, 5}
Complement event of A, A’ = {2,4, 6}.
Impossible Event: – is a subset of sample space that contains none of the Points.
E.g. Rolling a die: S = {1, 2, 3, 4, 5, 6}
E = {7} or E = {0}
Independent Events:-Two events are said to be independent when the happening of one
event doesn’t affect the happening of the other.
E.g. rolling a die
Dependent Events: - Two events are said to be dependent when the happening (or
occurrence) and non-occurrence of an event affects the happening of another event.
Mutually Exclusive Event: - Events are said to be mutually exclusive if one and only of
them can take place at a time.
Collectively Exhaustive Events/Lists: - When a set of events for an experiment includes
every possible outcome the set is said to be collectively exhaustive event/list.
E.g. flipping a fair coin twice: S = {HH, HT, TH, TT}
Once looking the basic concepts, we pass to formally give definitions for probability.

2
 1.3. APPROACHES IN PROBABILITY
1.3.1. Objective Probability
Classic Probability
It is probability based on the symmetry of games of chance or similar situations. This
probability is based on the idea that certain occurrences are equally likely. eg. The numbers
1,2,3,4,5,and 6 on fair die are equally likely to occur i.e they do have equal chance of
occurrence.
Long-term Relative Frequency Probability
The probability of an event happening in the long-term is determined by observing what
fraction of the time similar events happened in the past. We often think of a probability in
terms of the percentage of the time the event would occur in many repetition of the
experiment. Suppose that A is an event that might occur when a particular experiment is
performed then the probability that the event A will occur, P (A), can be interpreted to be the
number that would be approached by the relative frequency of the event A If we perform the
experiment an indefinitely large number of times.

E.g. When we say that the probability of obtaining a head when we toss a coin is 0.5 we are
saying that, when we repeatedly toss the coin an indefinitely large number of times, we will
obtain a head 50% of the repetition.
In terms of formula
Probability of an event happening = Number of times occurred in past
Total number of observation

If a truck operator experienced 5 accidents out of 50 truck last year, then the probability that
a truck will have an accident next year can be 5/50 = 0.10
1.3.2. Subjective Probability
Subjective probability, probabilities of the occurrence of an event determined based on an educated
guess or estimate. This guess is based on the person's professional and life experience and evaluation
of a solution.

For example, a physician might say that because of his diagnosis, there is a 30% chance that
the patient will need an operation

Both classic and long-term relative frequency probabilities are objective in the sense that no
personal judgment is involved.

3
 1.4. Postulates and Basic Theorems of Probability
1.4.1. Postulates of Probability
i) Given a sample space, S, of a random experiment, the probability of the entire
sample space is 1. i.e. P (S) = 1
ii) The probability of an event ranges from 0 to 1. i.e. 0 ≤P (A) ≤1
Where: A is any event in a random experiment
P (A) is the probability of A
iii) If two events A and B are mutually exclusive (disjoint events), then the
probability of either A or B or both is
P (A or B) = P (AUB) = P (A) + P (B) … Addition rule
1.4.2. Basic theorems of Probability
Theorem 1.1. If two events A and B are not mutually exclusive, then the occurrence
of either event A or B is given by the probability:
P (A or B) = P (AUB) = P (A) + P (B) – P (A n B)
Theorem 1.2. If A is an event from a sample space, S, and A’ it its complement then:
P (A) + P (A’) = 1

Proof: Making use of the definition of a complement, according to which A and A' are
mutually exclusive and A U A' = S. Thus, we write

1 = P(S) (by Postulate 1)

= P (A U A')

= P (A) + P (A') (by Postulate 3)

And it follows that P (A') = 1 – P ( A ) .

Example 1: If A and B are the events that the consumer Union will rate a car stereo good or
poor is given by; P(A)=0.24, P(B)=0.35, then determine the following probabilities
a) P (A′), b) P (AUB) c) P(A ∩)
Answer
a) From the theorem ii of probability, P(A′)=1-P(A)
=1-0.24=0.76
b) Since the two events are mutually exclusive events (consumer can either rate the car as
good or poor, they cannot rate the car poor and good at the same time)
P (AUB) = P (A) + P(B)= 0.24 + 0.35=0.59
c) We have said that event A and B are mutually exclusive events, so, P (A∩)B)= P (∅) =0

4
 Example 2: The probability that it will rain in Jigjiga on a particular day is 0.27, the
probability there will be thunderstorm on that day is 0.24 and that there will be a rain as
well as a thunderstorm is 0.15. What is the probability that there will be a rain or a
thunderstorm in Jigjiga on such day?
Answer:
Let A represents the event that there is rain and B there will be thunderstorm
P (A) =0.27, P (B) = 0.24, and P (A∩B) =0.15
Since the two events are not mutually exclusive events, we use
P (AUB) = P (A) + P (B) – P (A ∩) B)
= 0.27+0.24-0.15
= 0.36

Self-test One
Consider randomly selecting a card from a standard deck of well-shuffled 52 playing cards,
a) what is the probability that a randomly drawn card is Jack or Queen
b) What is the probability that a randomly selected card is an Ace or Red
Theorem 1.3. If events A1, A2, ------An are mutually exclusive i.e. no two of the events have
any sample space outcome in common. The probability that one of them will occur equals the
sum of their individual probabilities; symbolically,
P (A1UA2U-----UAn) = P (A1) + P (A2) +-----+P (An)
Example 3: If we twice flip a balanced coin, what is the probability of getting at least one
head?
Solution: The sample space is S = {HH, HT, TH, TT), where H and T denote head and tail.
Since we assume that the coin is balanced, these outcomes are equally likely and we assign to
each sample point the probability Letting A denote the event that we will get at least one
head, we get A = {HH, HT, TH} and

P (A) = P (HH) + P (HT) + P (TH)

=1/4+1/4+1/4 =3/4

Example 4: A die is loaded in such a way that each odd number is twice as likely to occur
as each even number. Find P (G), where G is the event that a number greater than 3 occurs
on a single roll of the die.
Solution: The sample space is S = {1, 2, 3, 4, 5, 6). Hence, if we assign probability w to
each even number and probability 2w to each odd number, we find that 2w + w + 2w + w +
2w + w = 9w = 1 in accordance with Postulate 1. It follows that w = 1/9 and

5
 P (G) = 1/9+ 2/9 + 1/9= 4/9

Theorem 1.4.
1.4. If an experiment can result in any one of N different equally likely
outcomes, and if n of these outcomes together constitute event A, then the probability
of event A is
P (A) = n/N
Theorem 1.5. P ( ∅) = 0 for any sample space S.

Proof: Since S and ∅ are mutually exclusive and S U ∅ = S in accordance with the
definition of the empty set∅, it follows that;
P(S) = P(S U∅)
= P(S) + P (∅) (by Postulate 3)

And, hence, that P (∅) = 0

Theorem1.6. If A and B are events in a sample space S and A c B, then P (A)≤ P (B).
Proof: Since A c B, we can write B = A U ( A ′ ∩ B )

Then, since A and A' ∩ B are mutually exclusive, we get

P (B) = P (A) + P (A ' ∩ B) (by Postulate 3)
P (B) ≥P (A) (by Postulate 2)
Theorem 1.7. If A, B, and C are any three events in a sample space S, then
P (A U B U C) = P (A) + P (B) + P (C) - P (A ∩ B) - P (A ∩ C) - P ( B ∩ C ) + P

(A∩B∩C)

Proof. Writing AUBUC as AU (BUC) and separately dealing with P [A U (B U

C)] and P (B U C), we get

P (A U B U C) = P [A U (B U C)]

= P (A) + P (B U C) - P [A∩ (BU C)]

=P (A) +P (B) +P(C)-P (B∩C) -P [A∩ (BUC)]

And now focusing on P [ A ∩ ( B U C ) ]

P [A∩ (BUC)] = P [(A∩B) U (A∩C)]

=P (A ∩ B ) + P ( A ∩ C ) - P [ ( A ∩ B ) ∩ ( A ∩ C ) ]

=P (A ∩ B ) + P ( A ∩ C ) - P ( A ∩ B ∩ C )

6
Hence

P (A U B U C) = P (A) + P (B) + P (C) - P (A ∩ B) - P (A ∩ C) - P ( B ∩ C ) + P ( A ∩ B ∩ C )

E x a m p l e 5 : If a person visits his dentist, suppose that the probability that

he will have his teeth cleaned is 0.44, the probability that he will have a cavity
filled is 0.24, the probability that he will have a tooth extracted is 0.21, the
probability that he will have his teeth cleaned and a cavity filled is 0.08, the
probability that he will have his teeth cleaned and a tooth extracted is 0.11, the
probability that he will have a cavity filled and a tooth extracted is 0.07, and
the probability that he will have his teeth cleaned, a cavity filled, and a tooth
extracted is 0.03. What is the probability that a person visiting his dentist will
have at least one of these things done to him?

Solution If C is the event that the person will have his teeth cleaned, F is the event that
he will have a cavity filled, and E is the event that he will have a tooth extracted, we are
given P(C) = 0.44, P ( F ) = 0.24, P ( E ) = 0.21, P(C ∩ F ) = 0.08, P(C ∩ E ) = 0.11,
P ( F ∩ E ) = 0.07, and P(C ∩ F ∩ E ) = 0.03, and substitution into the formula of
theorem 1.7 yields

P ( C U F U E ) = 0.44 + 0.24 + 0.21 - 0.08 - 0.11 - 0.07 + 0.03

= 0.66

1.5. Conditional Probability

Definition: If A and B are any two events in a sample space S and P(A) ≠0, the conditional

PA ∩ B
probability of B given A is P(B\A)=
P( A)
Example 6: A consumer research organization has studied the services under warranty
provided by the 50 new-car dealers in a certain city, and its findings are summarized in the
following table.

7
 Good service under Poor service under
warranty warranty

10 years or more 16 4

Less than 10 years 10 20

 If a person randomly selects one of these new-car dealers, what is the probability that
he gets one who provides good service under warranty?

 Also, if a person randomly selects one of the dealers who has been in business for 10
years or more, what is the probability that he gets one who provides good service
under warranty
Solution:
If we let G denote the selection of a dealer who provides good service under warranty, and
if we let n(G) denote the number of elements in G and n(S) the number of elements in the
whole sample space, we get
n(G) 16+10
P(G)= = =0.52
n(S) 50
 Second question is about dealers in business for 10 or more yeas which provides
goods services

Let T denotes the selection of a dealer who has been in business 10 years or more

Thus, n(T∩G)=16, n(T)=20

P (T ∩G)
P(T\G)= =0.8
P(T )
Theorem 1.8. If A and B are any two events in a sample space S and P(A) ≠0, P(B) ≠0 then
P(A ∩ B)= P(A/B) * P(B) or
= P(B/A) * P(A)
Example 7: If we randomly pick two television tubes in succession from a shipment of 240
television tubes of which 15 are defective, what is the probability that they will both be
defective?

Solution:

8
 Assume equal probabilities for each selection (which its what we mean by "randomly"
picking the tubes)
Let A represents the event that the first tube is defective
B ‘’ 2nd ‘’
Then P(A)=15/240, P(B)=14/239

Thus, the probability that both tubes will be defective, P (A∩B)= P(A)*P(B\A)

15
∗14
= 240 =0.00366
239
Exercises
1. Find the probabilities of randomly drawing two aces in succession from an ordinary deck
of 52 playing cards if we sample
a) without replacement
b) b) with replacement
2. A manufacturer of airplane parts knows from past experience that the probability is 0.80
that an order will be ready for shipment on time, and it is 0.72 that an order will be ready for
shipment on time and will also be delivered on time. What is the probability that such an
order will be delivered on time given that it was ready for shipment on time?
1.6. Independent Events
Definition: Two events A and B with positive probabilities are said to be statistically
independent if and only if P (A\ B) = P (A). Equivalently, P (B \ A) = P (B) and P(A n B) =
P(A) * P (B).
Note: If two events are not independent, they are said to be dependent
Theorem 1.9. If A and B are independent, then A and B' are also independent.

Proof: Since A = (A n B) U (A n B'), A n B and A n B' are mutually exclusive

And A and B are independent by assumption

Then we have
P (A)= P(A n B) U P(A n B')
= P(A n B) + P(A n B')
It follows that, P(A n B')= P(A)-P(A)*P(B)
=P(A)[1-P(B)]
=P(A)*P(B‘)

9
Example 8: A coin is tossed three times. If A is the event that a head occurs on each of the
first two tosses, B is the event that a tail occurs on the third toss, and C is the event that
exactly two tails occur in the three tosses, show that;

• events A and B are independent;

• events B and C are dependent
Solution: S={HHH,HHT,HTH,THH,HTT,THT,TTH,TTT}
A={HHH,HHT}
B={HHT,HTT,THT,TTT}
C={HTT,THT,TTH}
A∩B={HHT}, B∩C={HTT,THT}
The assumption that the eight possible outcomes are all equi-probable yields P(A) =1/4
P(B)=1/2 P(C)=3/8 P(A ∩ B)=1/8 and P(B ∩ C)=1/4
 Since P(A) *P(B) = ¼*1/2=1/8 = P(A ∩ B), events A and B are independent.
 Since P(B) * P(C) = ½*3/8=3/16≠ P(B ∩ C), events B and C are not independent

1.7. BAYES’ THEOREM

Theorem 1.10. . If the events B1, B2, ..., and Bk constitute a partition of the sample space S
and P(Bi) ≠0 for i = 1, 2, . . . , k, then for any event A in S
k
P(A)=∑ P ( Bi )∗P ( A {B¿¿ i)
i=1

Example: The completion of a construction job may be delayed because of a strike. The
probabilities are 0.60 that there will be a strike, 0.85 that the construction job will be
completed on time if there is no strike, and 0.35 that the construction job will be completed
on time if there is a strike. What is the probability that the construction job will be
completed on time?
Solution: If A is the event that the construction job will be completed on time and B is the
event that there will be a strike, we are given P(B) = 0.60, P(A\B') = 0.85, and P(A\B) =
0.35.
Then, P(A)=P(B) *P(A\B) + P(B') * P(A\B‘)
Then, substituting the given numerical values, we get
P (A) = (0.60) (0.35) + (1 - 0.60) (0.85)
= 0.55

10
Example 9: The members of a consulting firm rent cars from three rental agencies: 60
percent from agency 1, 30 percent from agency 2, and 10 percent from agency 3. If 9 percent
of the cars from agency 1 need a tune-up, 20 percent of the cars from agency 2 need a tune-
up, and 6 percent of the cars from agency 3 need a tune-up, what is the probability that a
rental car delivered to the firm will need a tune-up

Solution If A is the event that the car needs a tune-up, and B1, B2, and B3 are the events that
the car comes from rental agencies 1, 2, or 3, we have P(B1) = 0.60, P(B2) = 0.30, P(B3) =
0.10, and P(A\B1) = 0.09, P(A\B2) = 0.20, and P(A\B3) = 0.06. Then;

P(A)=P(B1) *P(A\B1) + P(B2) * P(A\B2 ) +P(B3)* P(A\B3 )

P(A) = (0.60) (0.09) + (0.30) (0.20) + (0.10) (0.06)

= 0.12

Theorem 1.11. If B1, B2... Bk constitute a partition of the sample space S and P(Bi) ≠ 0 for i =
1, 2…..k, then for any event A in S such that P(A) ≠0,
P (Br\A) =
P( Br ¿ *P(A" {B} rsub {r} ¿ } over {sum from {i=1} to {k} {P left ({B} rsub {i} right ) *P(A\ {B} rsub {i} )}
, for r=1,2,…k
Example 10: In above example, what is the probability that it came from rental agency 2?
Solution:
P(B2) *  P(A\B 2
P(B2\A)=
P(B1) * P(A\B1)  + P(B2) *  P(A\B2  ) +P(B3)*  P(A\B3  ) 
(0.3)(0.2)
= ¿=0.5
( 0.6 ) 0.3 ¿
Exercise: In a certain state, 25 percent of all cars emit excessive amounts of pollutants. If the
probability is 0.99 that a car emitting excessive amounts of pollutants will fail the state's
vehicular emission test, and the probability is 0.17 that a car not emitting excessive amounts
of pollutants will nevertheless fail the test, what is the probability that a car that fails the test
actually emits excessive amounts of pollutants?
Solution: Let A denotes the event that a car emit excessive amounts of pollutants and B a
car will fail the state's vehicular emission test, then
P (A) =0.25, P (B\A) =0.99, P (A′) =0.75, P (B\A′) =0.17

P ( A )∗P( B ¿) (0.25)(0.99) 0.2475

P (B\A) = ' = (0.25∗0.99)(0.75∗0.17) = 0.2475+0.1275 =0.66
P ( A )∗P ( B ¿ )∗P ( A )∗P ¿ ¿

11