Unit – II - Basic Probability

Topics:

1. Basic terminology related to probability.

2. Independent events.
3. Mutually exclusive and exhaustive events.
4. Addition theorem.
5. Multiplication theorem.
6. Bayes theorem.
7. Discrete and continuous random variables.
8. Probability distributions and their properties: expectation, variance, MGF,
CGF, characteristic function.

I. Basic terminology related to probability.

Probability is a measure that is associated with how certain we are of outcomes of a

particular experiment or activity. An experiment is a planned operation carried out under
controlled conditions. If the result is not predetermined, then the experiment is said to be
a probability experiment. Flipping one fair coin twice is an example of an experiment.

A result of an experiment is called an outcome. The sample space of an experiment is the set
of all possible outcomes. Three ways to represent a sample space are: to list the possible
outcomes, to create a tree diagram, or to create a Venn diagram. The uppercase letter S is used
to denote the sample space. For example, if you flip one fair coin, S = {H, T} where H = heads
and T = tails are the outcomes.

Terms Related to Probability

The list of the terms related to probability are as follows.

1. Experiment: An activity whose outcomes are not known is an experiment. Every

experiment has a few favourable outcomes and a few unfavourable outcomes.
For example, the tossing of a coin, picking a card from a deck of cards, rolling
dice, etc.
2. Random Experiment: A random experiment is an experiment for which the set
of possible outcomes is known, but which particular outcome will occur on a
particular execution of the experiment cannot be said prior to performing the
experiment. Tossing a coin, rolling a die, and drawing a card from a deck are all
examples of random experiments.
3. Trial: The numerous attempts in the process of an experiment are called trials. In
other words, any particular performance of a random experiment is called a trial.
For example, tossing a coin is a trial.
4. Event: A trial with a clearly defined outcome is an event. For example, getting a
tail when tossing a coin is termed as an event.
5. Random Event: An event that cannot be easily predicted is a random
event. For such events, the probability value is very less. For example, the
throwing of dice or a coin, or selecting a card from a group of cards, etc.
6. Favourable outcomes: The favourable outcomes as the name signifies are
the outcomes in which an individual is interested.
7. Possible Outcome: The list of all the outcomes in an experiment can be referred
to as possible outcomes. In tossing a coin, the possible outcomes are heads or
tails.
8. Equally likely Outcomes: An experiment in which each of the outcomes has an
equal probability, such outcomes are referred to as equally likely outcomes. In the
process of rolling a six-faced dice, the probability of getting any number is equal.
1
P(any number)= 6

9. Sample Space: It is the set of outcomes of all the trials in an experiment. On

rolling a dice, the possible outcomes are 1, 2, 3, 4, 5, and 6. These outcomes make
up the sample space. S = {1, 2, 3, 4, 5, 6}
10. Event: The possible subsets of the sample space linked with a random experiment
are termed as the event of an experiment. Events are categorized based on their
occurrence as follows;
11. Equally likely events: Any set of events is said to be equally likely if they have
an equal possibility to occur. Consider an example: When a dice is thrown the
six (occurrences of 1 to 6) events are equally likely.

12. Complementary Events: Complementary events occur when there are just two
outcomes, and one event is exactly opposite to another event. For an event with
probability P(A), its compliment is P(𝐴̅).
P(A)+ P(𝐴̅)=1
In an examination, the event of success and the event of failure are
complementary events. P(Success)+P(Failure)=1.
 13. Dependent Events: If the occurrence of one event influences or affects the
probability of another event then the two events are said to be dependent.
14. Impossible Event: The event that cannot happen is called an impossible event. For
example, in tossing a coin it is impossible to get both head and tail at an equal time.

Example 1: A football team plays 120 matches and wins 80 matches. What is the
probability of the team winning the next match?

Example 2:

Tom's teacher wrote each English alphabet on a different piece of paper and jumbled all those
in a box. He asked Tom to randomly pick a piece of paper. What is the probability of having
a vowel written on that piece of paper?
Example 3:

Sam takes two coins and flips them both at once. What is the probability of getting heads
on both the coins?

Example 4:

Mary had a jar containing 8 red balls, 5 blue balls, and 7 green balls. She called one of her
friends and asked them to pick a ball from the jar. What is the probability that the ball which
is picked is either a red or a blue ball?
Example 5:
What is the chance that a leap year selected at random will contain 53 Sundays?
Solution:
In a leap year (which consists of 366 days) there are 52 complete weeks and 2 days over.

The following are the Possible combinations for these two 'over' days:

Sunday and Monday, (ii) Monday and Tuesday, (iii) Tuesday and Wednesday (iv)

Wednesday and Thursday, (v) Thursday and Friday, (vi) Friday and Saturday and (vii)

Saturday and Sunday.

In order that a leap year selected at random should contain 53 Sundays, one of the two

'over’ days must be Sunday. Since out of the above 7 possibilities, 2 such as (i) and (vii),

are favorable to this event,

𝟐
Required Probability = .
𝟕
 II. Independent Events:

A set of events is said to be independent if the occurrence of an event is not

influenced/ affected by the occurrence of the rest of the events. Independent
events occur in different sample spaces. Suppose if A and B both are independent
events in different sample spaces then P(A∩B) = P(A)P(B).

III. Mutually Exclusive Events:

Two events such that the happening of one event prevents the happening of
another event are referred to as mutually exclusive events. In other words, two
events are said to be mutually exclusive events, if they cannot occur at the same
time. For example, tossing a coin can result in either heads or tails. Both cannot be
seen at the same time.

IV. Exhaustive events

A set of events is said to be exhaustive if at least one of them
surely occurs whenever the experiment is executed. Consider the same above
example of throwing the dice, here all the events from 1 to 6 are exhaustive.

The Addition Law of Probability - Simple Case

If two events A and B are mutually exclusive, then

The Addition Law of Probability - General Case

Multiplication Theorem on Probability

For two events A and B,

Multiplication Theorem for Independent Events

The multiplication theorem on probability for dependent events can be extended for
the independent events.

If the events A and B are independent, then, P(B | A) = P(B). The multiplication theorem
reduces to

P(A ∩ B) = P(A) P(B).

Examples on Addition and Multiplication theorem:

1. A bag contains 20 balls, 3 are colored red, 6 are colored green, 4 are colored blue, 2 are
colored white and 5 are colored yellow. One ball is selected at random. Find the probabilities
of the following events.

(a) the ball is either red or green.

(b) the ball is not blue.
(c) the ball is either red or white or blue.
 Solution:

Addition Theorem
Statement of the addition theorem on probability:
If A and B are any two events of a random experiment and P is a probability function then the
probability of happening of at least one of the events is defined as
P(A∪B)=P(A)+P(B)−P(A∩B).
Note:(1) Two events A and B are independent events if the equation.
i.e. P(A∩B)=P(A)×P(B).
(2). If two events A and B are mutually exclusive, then A∩B is a null set. Then
i.e. P(A∪B)=P(A)+P(B).
(3). If A and B are two mutually exhaustive, then the probability of their union is 1.
i.e. P(AUB)=1.

Example 1: A card is drawn at random from a pack of 52 cards. Find the probability that the
drawn card is either club or a queen.

Solution 2: Let A: Event of drawing a card of club.

Let B: Event of drawing a queen card.

The probability of drawing a card of club P(A) = 13/52.

The probability of drawing a queen card P(B) = 4/52.

As, one of the queen is club, so, the events are not mutually exclusive.

So, the probability of drawing queen of club = P(A∩B)=1/52

P(A∪B)=P(A)+P(B)−P(A∩B)

=13/52+4/52−1/52

=16/52

=4/13

Example 2: If George and James, two students, have a probability of solving a problem
that is 1/2 and 1/3, respectively, then what is the likelihood that the problem will be
solved?

Solution:

Let’s say that A and B represent the odds that George and James will be able to solve the
problem, respectively.

Therefore, P(A) equals 1/2 and P(B) equals 1/3.

If at least one of them is able to find a solution to the problem, then it can be considered
solved.

P(A∩B)= P(A) * P(B)= 1/2 * 1/3.

Therefore, we need to locate P(AUB).

According to an extension of the addition theorem for probability, we obtain

P(A∪B)=P(A)+P(B)−P(A∩B)

P(AUB) = 1/2 +.1/3 – 1/2 * 1/3 = 1/2 +1/3-1/6 = (3+2-1)/6 = 4/6 = 2/3

Example 3: Two dice are tossed once. Find the probability of getting an even number on first
dice or a total of 8.

Solution: An even number can be got on a die in 3 ways because any one of 2, 4, 6, can
come. The other die can have any number. This can happen in 6 ways.

∴ P (an even number on Ist die) =

A total of 8 can be obtained in the following cases:

{(2,6),(3,5),(4,4),(5,3),(6,2)}

∴ P (a total of 8) =

∴ Total Probability =
Example 4: What is the probability of drawing either a king or a queen in a
single draw from a well shuffled pack of 52 cards?

Solution: Total number of cards = 52

Number of king cards = 4
Probability of drawing a king card = 4/52
Number of queen cards = 4
Probability of drawing a queen card= 4/52
Both the events of drawing a king and a queen are mutually exclusive
⇒ P (A ∪ B) = P (A) + P (B)
Therefore, probability of drawing either a king or a queen = 4/52 + 4/52 = 2/13

Example 5: A number is chosen from the first 100 natural numbers. Find the probability that it
is a number of 4 or 6.

Solution: Given that one number is selected from the first 100 natural numbers, n(S) = 100,

Let A be the event of selecting a number which is multiple of 4

Here A = {4, 8, 12 . . . . ., 100}

⇒ n(A) = 25;

Let B be the event of selecting a number which is multiple of 6;

B = {6, 12, 18, . . . . , 96}

⇒ n(B) = 16;

Here A, B are not exlcusive

(A ⋂ B) is the event of selecting a number which is a multiple of 4 and 6

i.e., it is a multiple of 12.

A ⋂ B = {12, 24, . . . . . , 96}

⇒ n(A ⋂ B) = 8;

P(A ⋃ B) = P(A) + P(B) – P(A ⋂ B)

=

Example 6: Two dice are rolled together. Find the probability of getting a
doublet or sum of faces as 4.

Solution: When two dice are rolled together, there will be 6×6 = 36 outcomes.

Let S be the sample space. Then n (S) = 36

Let A be the event of getting a doublet and B be the event of getting face sum 4.

Then A = {(1,1),(2,2),(3,3),(4,4),(5,5),(6,6)}

B = {(1,3),(2,2),(3,1)}

Therefore, A Ո B = {(2,2)}

Then, n (A) = 6 , n (B) = 3 , n (A ∩ B) = 1.

Therefore, P (getting a doublet or a total of 4) = P (A U B)

P (A U B) = P (A) + P (B ) − P (A ∩ B)

= 6/36 + 3/36 – 1/36 = 8/36 = 2/9

Hence, the required probability is 2/9.

Multiplication Theorem Examples

Example 1: An jar contains 20 red and 10 blue balls. Two balls are drawn from a bag one after
the other without replacement. What is the probability that both the balls are drawn are red?

Solution: Let A and B denote the events that the first and the second balls are drawn are red
balls.

We have to find P(A∩B) or P(AB):

P(A) = P(red balls in first draw) = 20/30

Now, only 19 red balls and 10 blue balls are left in the bag.

The probability of drawing a red ball in the second draw too is an example of conditional
probability where the drawing of the second ball depends on the drawing of the first ball.

Hence Conditional probability of B on A will be,

P(B|A) = 19/29
By multiplication rule of probability,
P(A∩B) = P(A) × P(B|A)

Hence required probability is 38/87.

Example 2: 15 black and 10 white balls are contained in a box. After each other, two balls
are taken out of the box, and they are not changed in between games. What is the
probability that the two drawn balls will both be white?

Solution: Let P and Q represent the first and second draws, respectively, in which a white
ball is drawn from the box without replacement.

White ball in first draw P(P)=P=10/25

Event P has just occurred

P (Q) is a conditional expression, so

P(Q│P)=9/24

Now that we have used the multiplication rule, we can say that

P(Q|P)=9/24×9/24=3/20

Example 3: What is the probability of getting a 5 and then a 2 with the normal 6-sided dice?

Solution:

Sample space = {1, 2, 3, 4, 5, 6}

Total events = 6

Probability of getting a 5= P(A) = 1/6

Probability of getting a 2 = P(B)= 1/6

Applying the multiplication rule of probability for independent events,

We have, P(A ∩ B) = P(A) P(B | A),

so if the events A and B are independent, then, P(B | A) = P(B), and thus, the above
theorem reduces to P(A ∩ B) = P(A) P(B)

P(A∩B) = P(A) × P(B|A) = P(A) P(B)

P(getting a 5 and then a 2 ) = (1/6).(1/6) = 1/36.

Therefore, the probability of getting a 5 and then a 2 with the normal 6-sided die is 1/36.

Example 4: Two cards are selected without replacing the first card from the deck. Find
the probability of selecting a king and then selecting a queen.

Solution:

Total events = 52

Since the first card is not replaced, the events are dependent.

Probability of selecting a king = P(K) = 4/52

Probability of getting a queen = P(Q|K) = 4/51 (one card drawn first has not been replaced)

P(K∩Q) = P(K) × P(Q|K)

P(a king & a then queen) = P(K).P(Q|K)

=4/52 . 4/51 = 16/2652 = 1/166.

Therefore, the probability of selecting a king and then selecting a queen is 1/166.

Bayes Theorem
Bayes’ Theorem is used to determine the conditional probability of an event. It was named after an
English statistician, Thomas Bayes who discovered this formula in 1763.
➢ It is used to find the probability of an event, based on prior knowledge of conditions that might
be related to that event.
➢ For example, if we want to find the probability that a white marble drawn at random came
from the first bag, given that a white marble has already been drawn, and there are three bags
each containing some white and black marbles, then we can use Bayes’ Theorem.
➢ Def-1: Conditional probability: The probability of an event (A), given that another (B) has
already occurred.
➢ Def-2: Conditional probability is known as the possibility of an event or outcome happening,
based on the existence of a previous event or outcome. It is calculated by multiplying the
probability of the preceding event by the renewed probability of the succeeding, or conditional,
event.
➢
 ➢ Bayes theorem, in simple words, determines the conditional probability of event A given that
event B has already occurred based on the following:

• Probability of B given A
• Probability of A
• Probability of B

Bayes Theorem Formula:

If A and B are two events, then the formula for the Bayes theorem is given by:

Where:

• P(A) – the probability of event A.

• P(B) – the probability of event B.
• P(B) is never equal to zero.
• P(A|B) – the probability of event A occurring, given event B has occurred.
• P(B|A) – the probability of event B occurring, given event A has occurred.
• While writing the conditional probability of A given B, i.e. P(A|B), the symbol “|” represents
the word “given”. This means the event after “|” is already occurred.

Bayes theorem for more than two events:

Let E1, E2, E3,….En is a set of events related to a sample space S, where all the
events E1, E2, E3,…….En have a non-zero probability of occurrence, and they
form a partition of Sample space. Suppose A is the event-related with sample
space S, then according to Bayes theorem.

Note:
(1). The theorem has become a useful element in the implementation of machine learning.
(2). The events A and B are independent events (i.e., the probability of the outcome of an
event (A) does not depend on the probability of the outcome of the event (B)).
Example 1: A Bag-I contains 4 white and 6 black balls while another Bag-II contains 4
white and 3 black balls. One ball is drawn at random from one of the bags, and it is found to
be black. Find the probability that it was drawn from Bag I.
Solution:

Example 2: A man is known to speak the truth 2 out of 3 times. He throws a dice and reports
that the number obtained is a four. Find the probability that the number obtained is actually a
four.
Solution:
Example 3: In a bolt factory machines A, B and C manufacture respectively 25%, 35% and
40% of the total. Of their output 5,4, 2 percent are defective bolts. A bolt is drawn at random
from the product and is found to be defective. What are the probabilities that it was
manufactured by machines A, B and C?

Example 4: The contents of urns I, II and III are as follows:

I white, 2 black and 3 red balls,

2 white, 1 black and 1 red balls, and
4 white, 5 black and 3 red balls.

One urn is chosen at random and two balls drawn. They happen to be white and red. What is
the probability that they come from urns I, II or III ?
Example 5: Suppose 15 men out of 300 men and 25 women out of 1000 are good
researchers. A researcher is chosen at random. Find the probability that a male person
is selected. Assume that there are equal numbers of men and women.

Solution:

Let there be 1000 men and 1000 women.

Let E1 and E2 be the events of choosing a man and a woman respectively. Then,

P(E1) = 1000/2000 = 1/2 , and P(E2) = 1000/2000 = 1/2

Let A be the event of choosing a researcher. Then,

P(A|E1) = 50/1000 = 1/20, and P(A|E2) = 25/1000 = 1/40

The probability of selecting a male person, given that the person selected is a good researcher

P(A|E1) × P(E1)
P(E1/A) =
P(A|E1) × P(E1) + P(A|E2) × P(E2)

= (1/2 × 1/20) /{(1/2 × 1/20) + (1/2 × 1/40)}

= 2/3

Hence the required probability is 2/3.

Practice Problems:

Solve the following problems using Bayes Theorem.

1. A bag contains 5 red and 5 black balls. A ball is drawn at random, its colour is noted, and again
the ball is returned to the bag. Also, 2 additional balls of the colour drawn are put in the bag.
After that, the ball is drawn at random from the bag. What is the probability that the second ball
drawn from the bag is red?
2. Of the students in the college, 60% of the students reside in the hostel and 40% of the students
are day scholars. Previous year results report that 30% of all students who stay in the hostel
scored A Grade and 20% of day scholars scored A grade. At the end of the year, one student is
chosen at random and found that he/she has an A grade. What is the probability that the student
is a hosteler?
3. From the pack of 52 cards, one card is lost. From the remaining cards of a pack, two cards are
drawn and both are found to be diamond cards. What is the probability that the lost card is a
diamond?
 Random Variables(RV)
Defn-1: A random variable is a variable whose value is unknown or a function that assigns
values to each of an experiment's outcomes. A random variable can be either discrete (having
specific values) or continuous (any value in a continuous range).

Defn-2: A random variable is a numerically valued variable that takes on different

values with given probabilities.
✓ Random variables are often denoted by capital letters, say X, Y, E or Z.
✓ It is defined as a real number X connected with the outcome of a random
experiment E.
Examples:
• The return on an investment in one year,
• The price of an equity,
• The number of customers entering a store,
• The sales volume of a store on a particular day,
• The turnover rate at your organization next year.

Example-1: If E consists of three tosses of a coin, we may consider random variable X which
denotes the number of heads(H) (0, 1, 2 or 3).

Example-2: Suppose 2 dice are rolled and the random variable, X, is used to represent
the sum of the numbers. Then, the smallest value of X will be equal to 2 (1 + 1), while
the highest value would be 12 (6 + 6). Thus, X could take on any value between 2 to
12 (inclusive).

Types of Random Variables

As data can be of two types, discrete and continuous hence, there can be two types of
random variables. A discrete random variable can take on an exact value while the
value of a continuous random variable will fall between some particular interval.
Discrete Random Variable:
A random variable that can take on a finite number of distinct values is known as
a Discrete Random variable.
In other words, a real-valued function defined on a discrete sample space is called a
discrete random variable.
In the case of discrete random variables, we usually talk of values at a point. Generally,
it represents counted data.
For example, marks scored by students in a class for a particular subject can be represented
using a discrete random variable.

Continuous Random Variable:

A random variable that can take on an infinite number of possible values is known as
a continuous random variable. Such a variable is defined over an interval of values rather
than a specific value.

For example: The temperature of a city for a week.

The calculation of Rainfall in a city for a week.

Basic Concept: Integer or rational numbers are discrete, while real numbers are
continuous.

Probability Distribution of a Random Variable

The concept of probability distribution is equivalent to the frequency distribution.

It is used to show how probabilities are distributed over the values of a given random variable.
It depicts how the total probability of one is distributed among various values that a random
variable can take.

Probability Mass Function (Discrete Random Variable)

The probability mass function gives the probability that a discrete random variable will be
exactly equal to a specific value. The probability mass function is only used for discrete
random variables.
The probability mass function is also known as a frequency function. It can be represented
numerically as a table, in graphical form, or analytically as a formula.

Example-1
Suppose a fair coin is tossed twice and the sample space is recorded as
S = [HH, HT, TH, TT].
The probability of getting heads needs to be determined.
It can be seen that the random variable, X, represents the number of heads in the coin tosses.
This shows that X can take the values 0 (no heads), 1 (1 head), and 2 (2 heads).
The probabilities of each outcome can be calculated by dividing the number of favorable
outcomes by the total number of outcomes.
This gives us the following probabilities.

P(X = 0) = 1 / 4 = 0.25

P(X = 1) = 2 / 4 = 0.5

P(X = 2) = 1 / 4 = 0.25

Properties of Probability Mass Function

Probability Density Function (Continuous random variable)

The probability density function of a continuous random variable can be defined as a function
that gives the probability that the value of the random variable will fall between a range of
values.

Properties of Probability Density Function:

Example-1:
Suppose the probability density function of a continuous random variable, X, is given by 4x3,
where x ∈ [0, 1].
The probability that X takes on a value between 1/2 and 1 needs to be determined. This can
be done by integrating 4x3 between 1/2 and 1.
Thus, the required probability is 15/16.

Distribution Function or Cumulative Probability Function:

Example of Probability Mass function and distribution function:

Example of Probability density function and distribution function:
 Mean and Variance of a Random Variable:

For a discrete random variable:

Example 5:
Two cards are drawn successively with replacement from a well shuffled pack of 52 cards.
Find the mean and variance of the number of aces.
For a continuous random variable:

The mean and variance of a continuous random variable can be determined with the help of
the probability density function, f(x).
Example:2
Some Properties of Expectation
Moment Generating Function
Example 2: Let X be a discrete random variable with a probability mass function
𝑃𝑋 (𝑥) represented as:

Note: 𝑅𝑋 = {0,1}.
Example 3: Let X be a random variable with moment generating function
 Characteristic Function

1. There are random variables for which the moment generating function does not
exist on any real interval with positive length.
2. Therefore, the moment generating function does not exist for this random variable
on any real interval with positive length. If a random variable does not have a
well-defined MGF, we can use the characteristic function (c.f.) defined as
Example 1:

Let X be a discrete random variable with a probability mass function

Where 𝑅𝑋 = {0,1,2}. Derive the characteristic function of X.

Example 2:

Use the characteristic function found in the previous exercise to derive the variance of X.

Solution:
 Cummulant Generating Function

1. In probability theory and statistics, the cumulants 𝒌𝒏 of a probability distribution are

a set of quantities that provide an alternative to the moments of the distribution. Any
two probability distributions whose moments are identical will have identical
cumulants as well, and vice versa.
2. The first cumulant is the mean, the second cumulant is the variance, and the third
cumulant is the same as the third central moment. But fourth and higher order
cumulants are not equal to central moments.
3. In some cases, theoretical treatments of problems in terms of cumulants are simpler
than those using moments.

The n-th cumulant is:

𝑑𝑛 𝐾(𝑡)
𝑘𝑛 = 𝑎𝑡 𝑡 = 0.
𝑑𝑡 𝑛

The relationship between the moments and cumulants are represented as:

@End of Unit – 2@
Prepared by:
Dr. Amlan Kanti Halder
Assistant Prof. of Mathematics
School of Technology
Woxsen University, Hyderabad, Telangana, India

**************************************************************************