Lecture notes on Biostatistics (EaBC 636) Chapter 2: Probability

CHAPTER 2
2. Probability
2.1. Introduction
Probability: is the method of quantifying the uncertainty in a random phenomenon.
Probability theory is the foundation upon which the logic of inference is built. It helps us to
cope up with uncertainty. In general, probability is the chance of an outcome of an
experiment. It is the measure of how likely an outcome is to occur.

Knowledge about probabilities comes from the relative frequency of a large number of
trials.

People use the term probability many times each day. For example,
 Physician says that a patient has a 50-50 chance of surviving a certain operation.
 Another physician may also say that she is 95% certain that a patient has a particular
disease.
 A third physician may also say that the probability that an adult American male dies
from lung cancer during one year is 9 in 100,000 for a non-smoker, but is 190 in
100,000 for a smoker.
2.2. Definition and some concepts (Random experiment, Sample Space, Event, Equally
Likely Outcomes, and Mutually Exclusive Events)
1. Experiment: Any process of observation or measurement or any process, which
generates well defined outcome.
2. Probability Experiment: It is an experiment that can be repeated any number of times
under similar conditions and it is possible to enumerate the total number of outcomes
without predicting an individual out come. It is also called random experiment.
Example:
If a fair die is rolled once it is possible to list all the possible outcomes i.e.1, 2, 3, 4, 5, 6.
However, it is not possible to predict which outcome will occur.
An experiment generates an outcome through some random process.
No. Experiment Outcome
1. Weather Rains, Does not rain
2. Lottery Win, Lose
3. Football game of a given team Win, Lose
4. Growth status of a given plant Healthily grown, Not Healthily grown, Not grow
5. Diabetic Status Yes, No
6. Smoking Status Yes, No
3. Outcome: The result of a single trial of a random experiment.
4. Sample Space: Set of all possible outcomes of a probability experiment. It should consist
of mutually exclusive and exhaustive set of outcomes.
5. Event: It is a subset of sample space. It is a statement about one or more outcomes of a
random experiment .They are denoted by capital letters.
Example: Considering the above experiment let A be the event of odd numbers, B be the
event of even numbers, and C be the event of number 8.
 A  1,3,5
B  2,4,6
C    or empty space or impossible event

Page 1 of 13
 Lecture notes on Biostatistics (EaBC 636) Chapter 2: Probability
Remark:
If S (sample space) has n members then there are exactly 2n subsets or events.
6. Equally Likely Events: Events which have the same chance of occurring.
7. Complement of an Event: the complement of an event A means non-occurrence of A
and is denoted by 𝐴′ , 𝑜𝑟 𝐴𝑐 , 𝑜𝑟 𝐴̅; which contains those points of the sample space, which
don’t belong to A.
8. Elementary Event: an event having only a single element or sample point.
9. Mutually Exclusive Events: Two events which cannot happen at the same time.
10. Independent Events: Two events are independent if the occurrence of one does not
affect the probability of the other occurring.
11. Dependent Events: Two events are dependent if the first event affects the outcome or
occurrence of the second event in a way the probability is changed.

Example: What is the sample space for the following experiment?

a) Toss a die one time.
b) Toss a coin two times.
c) A light bulb is manufactured. It is tested for its life length by time.

Solution
a) S={1,2,3,4,5,6}
b) S={(HH),(HT),(TH),(TT)}
c) S={t /t≥0}
 Sample space can be

 Countable ( finite or infinite)

 Uncountable.

2.3. Methods of Counting

In order to calculate probabilities, we have to know
 The number of elements of an event
 The number of elements of the sample space.
That is, in order to judge what is probable, we have to know what is possible.

- In order to determine the number of outcomes, one can use several rules of counting.
- The addition rule
- The multiplication rule
- Permutation rule
- Combination rule

- To list the outcomes of the sequence of events, a useful device called tree diagram is
used.
Example:
A student goes to the nearest snack to have a breakfast. He can take tea, coffee, or milk
with bread, cake and sandwich. How many possibilities does he have?

Page 2 of 13
Lecture notes on Biostatistics (EaBC 636) Chapter 2: Probability
Solutions:
Tea Bread
Cake
Sandwich

Coeffee Bread
Cake  There are nine(9) possibilities
Sandwich

Milk Bread
Cake
Sandwich

The addition rule

- When two events, A and B, are non-mutually exclusive, the probability that A or B will
occur is:
P(A⋃B) =P(A)+P(B) - P(A⋂B)

Example:
In a mathematics class of 30 students, 17 are boys and 13 are girls. on a unit test, 4 boys
and 5 girls made an A grade. If a student is chosen at random from the class, what is the
probability of choosing a girl or an A grade student.

Solution
P(girl ⋃ A grade) =P(girl)+P(A grade) - P(girl ⋂ A grade)
13 9 5 22 − 5 17
= + − = = = 0.57
30 30 30 30 30

- When two events, A and B, are mutually exclusive, the probability that A or B will occur
is the sum of the probabilities of each event. i.e.,
P(A⋃B) =P(A)+P(B)
Example:
A single 6 sided die is rolled. What is the probability of rolling a 2 or a 5?

Solution
Possibilities
(1) The number rolled can be a 2
(2) The number rolled can be a 5
These events are mutually exclusive since they cannot occur at the same time.
1 1
𝑃(2) = 𝑃(5) =
6 6
1 1
𝑃(2 ⋃ 5) = 𝑃(2) + 𝑃(5) = +
6 6
2 1
= = = 0.33
6 3

Page 3 of 13
Lecture notes on Biostatistics (EaBC 636) Chapter 2: Probability

The Multiplication Rule:

If a choice consists of k steps of which the first can be made in n1 ways, the second can be
made in n2 ways, …, the kth can be made in nk ways, then, the whole choice can be made in
(n1*n2*n3*...*nk) ways.

Example:
The digits 0, 1, 2, 3, and 4 are to be used in 4 digit identification card. How many different
cards are possible if
a) Repetitions are permitted.
b) Repetitions are not permitted.

Solutions
a) 1st digit 2nd digit 3rd digit 4th digit
5 5 5 5
There are four steps
1. Selecting the 1st digit, this can be made in 5 ways.
2. Selecting the 2nd digit, this can be made in 5 ways.
3. Selecting the 3rd digit, this can be made in 5 ways.
4. Selecting the 4th digit, this can be made in 5 ways.
 5 * 5 * 5 * 5  625 different cards are possible.

b) 1st digit 2nd digit 3rd digit 4th digit

5 4 3 2
There are four steps
1. Selecting the 1st digit, this can be made in 5 ways.
2. Selecting the 2nd digit, this can be made in 4 ways.
3. Selecting the 3rd digit, this can be made in 3 ways.
4. Selecting the 4th digit, this can be made in 2 ways.
 5 * 4 * 3 * 2  120 different cards are possible.
Permutation
An arrangement of n objects in a specified order is called permutation of the objects.
Permutation Rules:
1. The number of permutations of n distinct objects taken all together is n!
Where n! n * (n  1) * (n  2) * ..... * 3 * 2 *1
2. The arrangement of n objects in a specified order using r objects at a time is called
the permutation of n objects taken r objects at a time. It is written as nPr and the
formula is:
n!
Pr 
(n  r )!
n

3. The number of permutations of n objects in which k1 are alike k2 are alike ---- etc
is:
n!
n Pr 
k1!*k2 * ... * kn

Page 4 of 13
Lecture notes on Biostatistics (EaBC 636) Chapter 2: Probability
Example:
1. Suppose we have a letters A,B, C, D
a) How many permutations are there taking all the four?
b) How many permutations are there two letters at a time?
2. How many different permutations can be made from the letters in the word
“CORRECTION”?
Solutions:
1. a) Here n  4, there are four disnict object
 There are 4! 24 permutatio ns.
b) Here n  4, r  2
4! 24
 There are 4 P2    12 permutatio ns.
(4  2)! 2
2. Here n  10
Of which 2 are C , 2 are O, 2 are R ,1E ,1T ,1I ,1N
 K1  2, k 2  2, k 3  2, k 4  k 5  k 6  k 7  1
U sin g the 3 rd rule of permutatio n , there are
10!
 453600 permutatio ns.
2!*2!*2!*1!*1!*1!*1!
Exercises:
1. Six different statistics books, seven different physics books, and 3 different
Economics books are arranged on a shelf. How many different arrangements are
possible if;
i. The books in each particular subject must all stand together
ii. Only the statistics books must stand together
2. If the permutation of the word WHITE is selected at random, how many of the
permutations
i. Begins with a consonant?
ii. Ends with a vowel?
iii. Has a consonant and vowels alternating?

Combination
A selection of objects without regard to order is called combination.

Example:
Given the letters A, B, C, and D list the permutation and combination for selecting two
letters.
Solutions:
Permutation Combination
AB BA CA DA AB BC
AC BC CB DB AC BD
AD BD CD DC AD DC

Note that:
In permutation AB is different from BA but in combination AB is the same as BA.

Page 5 of 13
Lecture notes on Biostatistics (EaBC 636) Chapter 2: Probability
Combination Rule
The number of combinations of r objects selected from n objects is denoted by
n
n C r or   and is given by the formula:
r 
 n n!
  
 r  (n  r )!*r!
Examples:
1. In how many ways a committee of 5 people be chosen out of 9 people?

Solutions:
n9, r 5
n n! 9!
     126 ways
 r  (n  r )!*r! 4!*5!
2. Among 15 clocks there are two defectives. In how many ways can an inspector chose
three of the clocks for inspection so that:
a) There is no restriction.
b) None of the defective clock is included.
c) Only one of the defective clocks is included.
d) Two of the defective clock is included.
Solutions:
n  15 of which 2 are defective and 13 are non  defective .
r 3
a) If there is no restriction select three clocks from 15 clocks and this can be
done in :
n  15 , r  3
n n! 15!
     455 ways
 r  (n  r )!*r! 12!*3!
b) None of the defective clocks is included.
This is equivalent to zero defective and three non defective, which can be done in:
 2  13 
  *    286 ways.
0 3 
c) Only one of the defective clocks is included.
This is equivalent to one defective and two non defective, which can be done in:
 2  13 
  *    156 ways.
1   2 
d) Two of the defective clock is included.
This is equivalent to two defective and one non defective, which can be done in:

 2  13 
  *    13 ways.
 2  1 

Page 6 of 13
Lecture notes on Biostatistics (EaBC 636) Chapter 2: Probability
Exercises:
1. Out of 5 Mathematician and 7 Statistician a committee consisting of 2
Mathematician and 3 Statistician is to be formed. In how many ways this can
be done if
a) There is no restriction
b) One particular Statistician should be included
c) Two particular Mathematicians can not be included on the committee.
2. If 3 books are picked at random from a shelf containing 5 novels, 3 books of
poems, and a dictionary, in how many ways this can be don if
a) There is no restriction.
b) The dictionary is selected?
c) 2 novels and 1 book of poems are selected?

2.4. Approaches in Probably definition (Classical and Axiomatic)

There are four different conceptual approaches to the study of probability theory. These
are:
 The classical approach.
 The frequentist approach.
 The axiomatic approach.
 The subjective approach.
1: The classical approach
This approach is used when:
- All outcomes are equally likely.
- Total number of outcome is finite, say N.
Definition:
If a random experiment with N equally likely outcomes is conducted and out of these NA
outcomes are favorable to the event A, then the probability that event A occur denoted
P(A) is defined as:
N No. of outcomes favourable to A n( A)
P( A)  A  
N Total number of outcomes n( S )
Examples:
1. A fair die is tossed once. What is the probability of getting
a) Number b) An odd number? c) An even number? d) Number 8?
4?

Solutions:

First identify the sample space, say S

S  1, 2, 3, 4, 5, 6  N  n(S )  6
a) Let A be the event of number 4
A  4  N A  n( A)  1
n( A)
P( A)  1 6
n( S )

Page 7 of 13
Lecture notes on Biostatistics (EaBC 636) Chapter 2: Probability
b) Let A be the event of odd numbers
A  1,3,5  N A  n( A)  3
n( A)
P( A)   3 6  0.5
n( S )
c) Let A be the event of even numbers
A  2,4,6  N A  n( A)  3
n( A)
P( A)   3 6  0.5
n( S )
d) Let A be the event of number 8
A   N A  n( A)  0
n( A)
P( A)  0 60
n( S )
2. A box of 80 candles consists of 30 defective and 50 non defective candles. If 10
of this candles are selected at random, what is the probability
a) All will be defective.
b) 6 will be non defective
c) All will be non defective
Solutions:
 80 
Total selection     N  n( S )
10 
a) Let A be the event that all will be defective.
 30   50 
Total way in which A occur    *    N A  n( A)
10   0 
 30   50 
 * 
n( A) 10   0 
 P( A)    0.00001825
n( S )  80 
 
10 
b) Let A be the event that 6 will be non defective.
 30   50 
Total way in which A occur    *    N A  n( A)
4  6 
 30   50 
 * 
n( A)  4   6 
 P ( A)    0.265
n( S )  80 
 
10 
c) Let A be the event that all will be non defective.
 30   50 
Total way in which A occur    *    N A  n( A)
 0  10 
 30   50 
 * 
n( A)  0  10 
 P( A)    0.00624
n( S )  80 
 
10 

Page 8 of 13
Lecture notes on Biostatistics (EaBC 636) Chapter 2: Probability
Exercises:
1. What is the probability that a waitress will refuse to serve alcoholic beverages to
only three minors if she randomly checks the I.D’s of five students from among
ten students of which four are not of legal age?
2. If 3 books are picked at random from a shelf containing 5 novels, 3 books of
poems, and a dictionary, what is the probability that
a) The dictionary is selected?
b) 2 novels and 1 book of poems are selected?
Short coming of the classical approach:
- This approach is not applicable when the total number of outcomes is infinite.
2: The Frequentist Approach
This is based on the relative frequencies of outcomes belonging to an event.
Definition: The probability of an event A is the proportion of outcomes favourable to A
in the long run when the experiment is repeated under same condition.
N
P( A)  lim A
N  N

Example:
If records show that 60 out of 100,000 bulbs produced are defective. What is the
probability of a newly produced bulb to be defective?
Solution:
Let A be the event that the newly produced bulb is defective.
N 60
P( A)  lim A   0.0006
N  N 100,000
3: Axiomatic Approach:
Let E be a random experiment and S be a sample space associated with E. With each
event A a real number called the probability of A satisfies the following properties called
axioms of probability or postulates of probability.
(1) P(A) ≥ 0
(2) P(S) = 1, S is the sure event
(3) If A and B are mutually exclusive events, the probability that one or the other
occur equals the sum of the two probabilities.
i. e. P(AUB ) = P(A) + P(B)
(4) P(A') = 1- P(A)
(5) 0 ≤ P(A) ≤ 1
(6) P(ø) =0, ø is the impossible event.
Remark:
Venn-diagrams can be used to solve probability problems.

AUB AnB A
In general
p( A  B)  p( A)  p( B)  p( A  B)

Page 9 of 13
Lecture notes on Biostatistics (EaBC 636) Chapter 2: Probability
2.5. Some Probability Rules
2.5.1. Definition of Probability of an Event with Equally Likely outcomes

Probability is the result of mapping an event into a number that in some sense indicates
how likely or often the event will occur. One way to do that is to use the notion of equally
likely outcomes. Calculation of probabilities in real life is very difficult. One helpful
method of calculation is to create a sample space with equally likely outcomes and
assign probabilities to the outcomes based on this assumption.

Define
The outcomes in a sample space are said to be “equally likely” if they will all occur
approximately equally often in the long run if the random experiment is repeated many,
many times.

Examples of experiments with equally likely outcomes are:

 Draw a random sample of size n from a population or urn. The assumption that
the sample is drawn at random means that all samples of size n have an equal
chance of being chosen (much of statistical analysis depends on the assumption
that samples are chosen randomly).
 Flip a fair coin n times and observe the sequence of heads and tails that results.
 Roll n dice, die 1, die 2, die 3, . . . , die n, and observe the ordered sequence of
numbers on the uppermost faces.

Define
- The sample space for a chance experiment is the set of all possible distinct
outcomes.
- An event is a subset of the sample space.
- Let n(A) be the number of ways event A can occur in the sample space where n
total equally likely outcomes are possible.
- Define the probability of event A occurring as:
𝑛(𝐴)
𝑃(𝐴) =
𝑛
Example:
- Consider a toss of two dice where each side is equally likely to land facing up.
- Illustrate the sample space for a single toss of 2 die.
- Compute the probability of the following events: The sum of the two dice, Sd, are
1,2,3,4,5,6,7,8,9,10,11,12, and 13.

Show:
- P(Sd=1) = 0, - P(Sd =5) = 4/36, - P(Sd =8) = 5/36, - P(Sd =11) =
- P(Sd =2) = 1/36, - P(Sd =6) = 5/36, - P(Sd =9) = 4/36, 2/36, P(Sd =12)
- P(Sd =3) = 2/36, - P(Sd =7) = 6/36, - P(Sd =10) = = 1/36, P(Sd
- P(Sd =4) = 3/36, 3/36, =13) = 0

Page 10 of 13
Lecture notes on Biostatistics (EaBC 636) Chapter 2: Probability
2.5.2. Properties of Probability of an Event
Definitions: Let S be a sample space and let A and B be any two events from S. Then
 The union of the events A and B, denoted AB , is the event consisting of all
outcomes that belong to A or B or both.
 The intersection of the events A and B, denoted A  B or sometimes by the shorter
AB, is the event consisting of all outcomes common to both A and B.
 The complement of an event A, denoted Ac , is the collection of all outcomes that are
not in A.
 The event A is a subset of B, denoted A  B , if every outcome in A is also contained
in B.
 The empty set or null set, denoted Ø, is the event, which consists of no outcomes.
 The events A and B are disjoint or mutually exclusive if A and B cannot happen
simultaneously. Thus A and B are disjoint if AB = Ø.
 Pr(A) or P(A) is used to denote the probability of event A.

Example:
Shuffle a standard deck of 52 cards and randomly select one card from the deck. Then the
sample space S consists of each of the 52 cards in the deck. Some possible events to
consider:
A = the card is a heart
B = the card is a face card
C = the card is the king of hearts
D = the card is black
Describe the following related events:
Ac =
AB =
AB =
The union of A and B is an event consisting of 22 outcomes (all 13 of the hearts plus the
king, queen, and jack from each of the others suits).
Note that C  A and C  B.
List two pairs of events from above that are mutually exclusive.
One of the most basic probability rules is the complement rule, which asserts that the
probability of
the complement of an event equals one minus the probability of the event. i.e.,
P(Ac) = 1 - P(A)
The addition rule asserts that the probability of the union of two events can be calculated
by adding
the individual event probabilities and then subtracting the probability of their
intersection:
P(A B) = P(A) + P(B) - P(AB)
If AB = , then it follows that P(AB) = P(A) + P(B) . This is known as the addition
rule for disjoint events; it is a special case of the addition rule since if AB = ,
P(AB) = P() = 0 . We use the notation P(B|A) to denote the conditional probability
of event B given event A.

Page 11 of 13
Lecture notes on Biostatistics (EaBC 636) Chapter 2: Probability
2.5.3. Mutually and non-mutually exclusive Events
Define:
- Events are mutually exclusive if and only if the occurrence of one event did not
imply the occurrence of the other events. i.e., If two events are such that they can
never occur simultaneously, then, they are said to be mutually exclusive events.
- Non-mutually exclusive events are events that can both be true at the same time.
That means, two events are said to be non-mutually exclusive events if the
intersection of these event sets is not the null set.

Example:
(1) Consider the examples of events in a sample space S described with a Venn diagram.

- Events A and B are mutually exclusive outcome

because no single outcome can satisfy both
events.

- Event C and D are non-mutually exclusive, since

an outcome exits that can satisfy both events.
(2) In Tossing a die one time, suppose event A is the event when the die throws up an
even number. Event B is the event when the die throws up a prime number.
A = {2,4,6}
B = {2, 3, 5}
Thus, we see that the number 2 is common to both the subsets of the sample space.
Therefore, it is possible that both the events A and B occur at the same time if the die
throws up number 2. Thus, we can say that events A and B are non-mutually exclusive
events

2.6. Conditional probability and Independence

In some cases the occurrence of one event (or knowledge that one event has occurred)
may increase or decrease the probability that another event has or will occur. This
relationship between events is modeled with conditional probabilities.

The conditional probability for event A given event B has occurred, where P(A) and P(B)
> 0, is defined by:
𝑃(𝐴 ∩ 𝐵)
𝑃(𝐴|𝐵) =
𝑃(𝐵)

and the conditional probability for event B given event A has occurred is defined by:

𝑃(𝐴 ∩ 𝐵)
𝑃(𝐵|𝐴) =
𝑃(𝐴)

The above relationships can be used to show:

P(A∩B) = P(B│A)P(A) = P(A│B)P(B)

Page 12 of 13
Lecture notes on Biostatistics (EaBC 636) Chapter 2: Probability
Independent Events
Events A and B are statistically independent if and only if P(A∩B) = P(A)P(B)

Example:
Show that if events A and B are statistically independent, then,
(i) P(A│B) = P(A)
(ii) P(B│A) = P(B)
Solution
𝑃(𝐴 ∩ 𝐵)
(𝑖)𝑃(𝐴|𝐵) =
𝑃(𝐵)
since events A and B are statistically independent, then P(A∩B) = P(A)P(B)
𝑃(𝐴)𝑃(𝐵)
=> 𝑃(𝐴|𝐵) = = 𝑃(𝐴)
𝑃(𝐵)
𝑃(𝐴 ∩ 𝐵)
(𝑖𝑖)𝑃(𝐵|𝐴) =
𝑃(𝐴)
since events A and B are statistically independent, then P(A∩B) = P(A)P(B)
𝑃(𝐴)𝑃(𝐵)
=> 𝑃(𝐵|𝐴) = = 𝑃(𝐵)
𝑃(𝐴)
Determine which events are statistically independent and Show:
 P(A) = 0.25,  P(A|C) = 0.25,  P(A|D) = 0.0625, and
 P(B) = 0.25,  P(B|C) = 0,  P(D|B) = 0.5
 P(C) = 0.25,  P(D|C) = 0.125,
 P(D) = 0.5,
Solution
Events A and C are statistically independent because
 P(A|C) = P(A) = 0.25
 P(C|A) = P(C) = 0.25

Events B and D are statistically independent because

 P(D|B) = P(D) = 0.5

Events B and C are mutually exclusive events because

𝑃(𝐵 ⋂ 𝐶)
𝑃(𝐵|𝐶) = =0
𝑃(𝐶)
=> implies that 𝑃(𝐵 ⋂ 𝐶) = 0, which is, these events are mutually exclusive

Page 13 of 13