Introduction to probability

1 / 70
 Table of contents
1 About This Course
2 Experiment, Outcomes and Events
3 Definitions of Probability
4 Some Applications of Probability

2 / 70
 Textbook

(1) Oliver C. Ibe, Fundamentals of Applied

Probability and Random Processes, 1st
edition
(2) Walpole et al, Probability and Statistics for
Engineers and Scientists, 9th edition.

3 / 70
 Grade

• Progress score: 20%

• Quiz: 10%
• Homework: 5%
• Attendance: 5%
• Midterm exam: 30%
• Final exam: 50%

4 / 70
 3 parts of this course
• Probability
Theory of the randomness
• Statistics
the art of learning from data
• Random process
Probability with time line

5 / 70
 Probability Part
Chapter 1, 2, 3, 4 in textbook (1)
• Elements of probability
• Probability space and event
• Rules to compute probability (addition rule,
conditional probability, multiple rule, total
rule, Bayes’s rule)
• Random variable
• Probability distribution and mathematical
expectation
• Special probability distributions 6 / 70
 Statistics Part
Chapter 8, 9 in textbook (1)
• Descriptive statistics
• Inference statistics
• parameter estimation
• hypothesis testing
• Linear regression
• Least square estimators of coefficients
• Inference about the estimators.
7 / 70
 Random process Part
Section 12.7 in textbook (1)

Markov chain
• Transition probability
• State classification
• Stationary distribution
8 / 70
PART 1: Probability

9 / 70
 What is Probability?
• Probability is the mathematics of chance -
a discipline in mathematics which deals
with phenomena whose outcome is
affected by random events and therefore
they can not be predicted with certainty
• Probability is a numerical measure of the
likelihood that a specific event will occur -
Measure of belief
10 / 70
 Example
What is the chance that head occur when tossing
a fair coin?

Probability that result of tossing a coin is head?

The events whose probabilities we wish to com-

pute all arise as outcomes of experiments.

11 / 70
 Example
What is the chance that head occur when tossing
a fair coin?

Probability that result of tossing a coin is head?

The events whose probabilities we wish to com-

pute all arise as outcomes of experiments.

11 / 70
 Example
What is the chance that head occur when tossing
a fair coin?

Probability that result of tossing a coin is head?

The events whose probabilities we wish to com-

pute all arise as outcomes of experiments.

11 / 70
 Session Objectives
• Understand and describe sample spaces
and events for random experiments with
graphs, tables, lists, or tree diagrams
• Interprete and use operations on events
such as unions, intersections, complement
• Interpret and use probabilities of
outcomes to calculate probabilities of
events in finite sample spaces
12 / 70
 Table of contents
1 About This Course
2 Experiment, Outcomes and Events
3 Definitions of Probability
4 Some Applications of Probability

13 / 70
 Random experiment
• An experiment is any process of trial and
observation

• An experiment whose outcome is uncertain

before it is performed is called a random
experiment

14 / 70
 Sample space

• Each result is called an elemetary

outcome or sample point

• The set of all possible outcomes is called

the sample space denoted by Ω or S or U

15 / 70
 Examples
Sample space of experiment
• Toss a coin
Ω = {Head, Tail}
• Roll a dice
Ω = {1, 2, 3, 4, 5, 6}
• Gender of a unborn baby
Ω = {male, female}
16 / 70
 Example - Sequential model
Two tosses of a coin
Tree diagram of sample space

H T

H T H T
Sample space
Ω = {(H, H), (H, T ), (T, H), (T, T )}
17 / 70
 Examples
• Sample space of rolling 2 dice
Ω = {(x, y) : x, y = 1, . . . , 6}
Finite sample space
• Sample space of measuring the thickness a
connector
Ω = (0, ∞)
Uncountable sample space
18 / 70
 Examples
• Sample space of rolling 2 dice
Ω = {(x, y) : x, y = 1, . . . , 6}
Finite sample space
• Sample space of measuring the thickness a
connector
Ω = (0, ∞)
Uncountable sample space
18 / 70
 Examples
• Sample space of rolling 2 dice
Ω = {(x, y) : x, y = 1, . . . , 6}
Finite sample space
• Sample space of measuring the thickness a
connector
Ω = (0, ∞)
Uncountable sample space
18 / 70
 Examples
• Sample space of rolling 2 dice
Ω = {(x, y) : x, y = 1, . . . , 6}
Finite sample space
• Sample space of measuring the thickness a
connector
Ω = (0, ∞)
Uncountable sample space
18 / 70
 Practice

Consider the experiment of tossing a coin un-

til “Tails” appear for the first time.
Suggest a suitable sample space that describes
the outcomes of this experiment.

19 / 70
 Different sample spaces for the same experiment

• A car store has 2 salespersons

• The store stock 2 cars for sales
• If we are interested in the number of cars
which will be sold by each of the two sales-
persons during next week then the sample
space is the set of pairs (i, j) where i and j
are the number of cars sold by the first and
second salesperson

20 / 70
• There are 2 cars available for sales
=⇒ i + j ≤ 2
• Arrive at the sample space
Ω1 = {(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (2, 0)}

21 / 70
• if the store owner is only interested in the
total number of cars sold during next week,
then we could use as a sample space the set
Ω2 = {0, 1, 2}

22 / 70
 Events
• An event is the occurrence of either a
prescribed outcome or any one of a number
of possible outcomes of an experiment.
• An event is a subset of sample space
• Event A occurs or appears if the
elementary outcome is an element in A

23 / 70
 Events
• An event is the occurrence of either a
prescribed outcome or any one of a number
of possible outcomes of an experiment.
• An event is a subset of sample space
• Event A occurs or appears if the
elementary outcome is an element in A

23 / 70
 Example
• Toss a coin 3 times
• Observe the outcome HHT
• The event that there is exactly 1 tail
A = {HHT, HT H, T HH}
has occured
• But the
B = {HHH, T T T }
event has not occured
24 / 70
 Examples
• Roll 1 dice. The sample space is
Ω = {1, 2, 3, 4, 5, 6}
• event = having an odd face
A = {1, 3, 5}

• Roll 2 dice, event = sum of 2 faces is 6

B = {(1, 5), (2, 4), (3, 3), (4, 2), (5, 1)}
25 / 70
 Examples
• Roll 1 dice. The sample space is
Ω = {1, 2, 3, 4, 5, 6}
• event = having an odd face
A = {1, 3, 5}

• Roll 2 dice, event = sum of 2 faces is 6

B = {(1, 5), (2, 4), (3, 3), (4, 2), (5, 1)}
25 / 70
 Examples
• Roll 1 dice. The sample space is
Ω = {1, 2, 3, 4, 5, 6}
• event = having an odd face
A = {1, 3, 5}

• Roll 2 dice, event = sum of 2 faces is 6

B = {(1, 5), (2, 4), (3, 3), (4, 2), (5, 1)}
25 / 70
 Examples
• Roll 1 dice. The sample space is
Ω = {1, 2, 3, 4, 5, 6}
• event = having an odd face
A = {1, 3, 5}

• Roll 2 dice, event = sum of 2 faces is 6

B = {(1, 5), (2, 4), (3, 3), (4, 2), (5, 1)}
25 / 70
 Example
If we measure the lifetime of an electronic com-
ponent, such as a chip, the sample space con-
sists of all nonnegative real numbers. That is
Ω = {x : x ≥ 0} = [0, ∞)
The event that the lifetime is not more than 7
hours is defined as follows
E = {x : 0 ≤ x ≤ 7} = [0, 7]
26 / 70
 Practice
John throws a die and subsequently he tosses a
coin.
1 Suggest a suitable sample space that
describes the outcomes of this experiment.
2 Let A be the event that “the outcome of the
coin toss is Heads.” Which elements of the
sample space are included in the event A?

27 / 70
 Practice - Go home
Bill wants to return hom at Place I from Place A

In order to minimize the distance he has to walk,

he moves either downwards (e.g. from Place A
to Place D) or to the right 28 / 70
1 Give a sample space for the different routes
Bill can follow to return home.
2 Write down explicitly the following events
as subsets of the sample space given in (1):
1 he passes through Place E on his way back
home
2 he does not pass through Place D 29 / 70
 New from Old Events
From two events, we can define new events us-
ing set operation

• Complement
• Intersection
• Mutual exclusive
• Union

30 / 70
 Complement
The complement of event A, denoted by Ac or Ā
or A′ is the subset containing all the elements of
Ω that are not in A.

31 / 70
 Example

Light bulb lifetime:

E = bulb last more than 3 hours,
E ′ = bulb last less than or equal 3 hours

32 / 70
 Example
Measurements of the thickness of a plastic con-
nector might be modeled with the sample space
Ω = R+ the set of positive real numbers. Let
A = {x|x ≥ 10}
Then,
A′ = {x|x < 10}

33 / 70
 Example
Measurements of the thickness of a plastic con-
nector might be modeled with the sample space
Ω = R+ the set of positive real numbers. Let
B = {x|8 < x < 15}.
be the event that the random selected connector
has thickness between 8 and 15
B ′ =?
34 / 70
 Intersection
Intersection of A and B, denoted by AB or A ∩
B, is the subset of all elements that are in both
A and B
AB or A ∩ B : both A and B occurs

35 / 70
 Example - thickness of plastic
connector

A = {x|x ≥ 10}, B = {x|8 < x < 15}.

then
A ∩ B = {x|10 ≤ x < 15}

36 / 70
 Mutually exclusive
Two events A and B are called mutually exclu-
sive or disjoint if
A∩B =∅
(A and B have no common element)

A and B never occurs simultaneously 37 / 70

 Example

• Roll a 6-sided dice

• A = {1, 2}
• B = {4, 6}
• A ∩ B = ∅ so A and B are mutually
exclusive

38 / 70
 Union
Union of A and B, denoted byA ∪ B or A + B.
is the set of all elements that are in A or in B or
in both
A ∪ B= either A or B or both occurs.

39 / 70
 Example

Let A = {a, b, c} and B = {b, c, d, e}

then A ∪ B = {a, b, c, d, e}

40 / 70
 Example
Let
• P : the event that an employee selected at
random from an oil drilling company
smokes cigarettes.
• Q: the event that the employee selected
drinks alcoholic beverages.
Then the event P ∪ Q is the set of all employees
who either drink or smoke or do both
41 / 70
 Example - thickness of plastic
connector

A = {x|x ≥ 10}, B = {x|8 < x < 15}.

then
A ∪ B = {x|x > 8}

42 / 70
 Example
• A = {1, 3, 5}, B = {1, 2, 3}
• AB = {1, 3} (in both A and B)
• A ∪ B = {1, 2, 3, 5} (in A or in B or in
both)
• A ∩ B ′ = {5} (in A but not in B)
• B ∩ A′ = {2} (in B but not in A)

43 / 70
 Practice
Consider the sample space
Ω = {(x, y) : −5 ≤ x ≤ 5, −3 ≤ y ≤ 7}
and the events
1 B = AC
A = {(x, y) ∈ Ω : x = y}, 2 AC ⊂ B
B = {(x, y) ∈ Ω : x2 = y 2, 3 A and C are
C = {(x, y) ∈ Ω : x + y = 0}} disjoint
4 A = {(x, x) : 44 / 70
 Properties for Operations
1 A ∪ B = B ∪ A, A ∩ B = B ∩ A
2 A ∪ Ac = Ω, A ∩ Ac = ∅
3 (A ∩ B) ∪ C = (A ∪ C) ∩ (B ∪ C)
4 (A ∪ B) ∩ C = (A ∩ C) ∪ (B ∩ C)
5 If A ⊂ B then B c ⊂ Ac and A ∩ B = A,
A∪B =B

45 / 70
 De Morgan’s laws
′ ′ ′
1 (A ∪ B) =A
| ∩
{z B }
no event occurs
| {z }
complement of at least one event occurs

′ ′ ′
2 (A ∩ B) = A
| ∪
{z B }
at least one event not occurs
| {z }
complement of all events occur

46 / 70
 Table of contents
1 About This Course
2 Experiment, Outcomes and Events
3 Definitions of Probability
4 Some Applications of Probability

47 / 70
 Probability
• A probability is a numerical measure of the
likelihood that a specific event will occur
• Probability of an event, denoted by P(event),
is a number between 0 and 1
• The larger the probability, the more con-
fident we are that the event will occur.
• Probability of an event is equal to 0: we can
almost be sure that this event cannot occur
• Probability of an event is equal to 1: this48 / 70
 Definitions of Probability
There are several ways to define probability
1 Axiomatic definition
2 Relative-frequency definition
3 Classical definition- based on counting tech-
nique

49 / 70
 Axiom of Probability
P (A), called the probability of event A on a sam-
ple space Ω of a random experiment, is defined
such that the following hold:
1 0 ≤ P (A) ≤ 1
2 P (Ω) = 1 (Normalization)
3 If A1, A2 . . . are mutually exclusive then
P (A1 ∪ A2 ∪ . . . ) = P (A1) + P (A2) + . . .
50 / 70
 Example

Suppose the probability of an event A is larger

by 0.5 than the probability of its complement,
that is P (A) = P (Ac) + 0.5.
Find P (A)

51 / 70
 Solution
Because A and Ac are mutually exclusive and
Ω = A ∪ Ac, we have
1 = P (Ω) = P (A ∪ Ac) = P (A) + P (Ac)
Combining this condition with the assumption
P (A) = P (Ac) + 0.5, we obtain P (A) = 0.75

52 / 70
 Practice
When a salesman visits a certain town, he stays
in one of three available hotels, H1, H2, H3. Let
Ai be the event that he stays in Hotel Hi. It is
known that
P (A3)
P (A1 ∪ A2) = 3P (A3) and 3P (A2) = 1 −
2
Find P (Ai) - the probability that he stays in Ho-
tel Hi , for i = 1, 2, 3.
53 / 70
 Practice

For two disjoint events A and B in a sample

space, suppose
P (A ∪ B) = 0.5 and 3P (Ac) + 2P (B) = 2.3
Find P (A) and P (B).

54 / 70
Probability law on finite sample
space
• Sample space Ω = {s1, s2, . . . , sn}
• Assign each outcome si with a probability
p(si) which satisfies
• 0 ≤ p(si) ≤ 1
• p(s1) + p(s2) + · · · + p(sn) = 1
• Probability of an event
P (A) = p(si)
X

si ∈A
55 / 70
 Example
Suppose there is a coin for which the chance to
show head is twice more likely than the chance
to show tail.
• Ω = {H, T } with P (H) = 2P (T )
• Normalization: P (H) + P (T ) = 1
• P ({H}) = 2/3, P ({T }) = 1/3

56 / 70
 Example
Suppose there is a coin for which the chance to
show head is twice more likely than the chance
to show tail.
• Ω = {H, T } with P (H) = 2P (T )
• Normalization: P (H) + P (T ) = 1
• P ({H}) = 2/3, P ({T }) = 1/3

56 / 70
 Example
Suppose there is a coin for which the chance to
show head is twice more likely than the chance
to show tail.
• Ω = {H, T } with P (H) = 2P (T )
• Normalization: P (H) + P (T ) = 1
• P ({H}) = 2/3, P ({T }) = 1/3

56 / 70
 Example
Suppose there is a coin for which the chance to
show head is twice more likely than the chance
to show tail.
• Ω = {H, T } with P (H) = 2P (T )
• Normalization: P (H) + P (T ) = 1
• P ({H}) = 2/3, P ({T }) = 1/3

56 / 70
 Example
Consider a sample space Ω = {a, b, c, d} with
p(a) = 0.1, p(b) = 0.5, p(c) = 0.3 and p(d) =
0.1. Let
A = {a, b, d}
then
P (A) = p(a)+p(b)+p(d) = 0.1+0.5+0.1 = 0.7

57 / 70
 Equally likely outcomes
Classical Defintion
• If p(x) is the same for all x in Ω then we
say that Ω has equally likely outcomes.
• If Ω has equally likely outcomes then

number of elements in A
P (A) =
number of elements in Ω
58 / 70
 Equally likely outcomes
Classical Defintion
• If p(x) is the same for all x in Ω then we
say that Ω has equally likely outcomes.
• If Ω has equally likely outcomes then

number of elements in A
P (A) =
number of elements in Ω
58 / 70
 Example
• Toss a fair coin
1
P (H) = P (T ) =
2
• Flip a fair dice
1
P (i) =
6
for i = 1, 2, 3, 4, 5, 6
59 / 70
 Example
• Toss a fair coin
1
P (H) = P (T ) =
2
• Flip a fair dice
1
P (i) =
6
for i = 1, 2, 3, 4, 5, 6
59 / 70
 Example
Roll a fair dice, event = having an odd face
A = {1, 3, 5}
Then
3 1
P (A) = =
6 2

60 / 70
 Example
Roll a fair dice, event = having an odd face
A = {1, 3, 5}
Then
3 1
P (A) = =
6 2

60 / 70
 Example

A statistics class for engineers consists of 25 in-

dustrial, 10 mechanical, 10 electrical, and 8 civil
engineering students. If a person is randomly
selected by the instructor to answer a question,
find the probability that the student chosen is an
industrial engineering major.

61 / 70
 Solution
• Each student is an elementary outcome
which is equally likely chosen
• Sample space
Ω = {I1, . . . , I25, M1, . . . , M10, E1, . . . , E10, C1, . . . , C8}
• Let I be the event that the student chosen is
an industrial engineering major then
I = {I1, . . . , I25}. So
n(I) 25 25
P (I) = = =
n(Ω) 25 + 10 + 10 + 8 53 62 / 70
 Practice

A fair coin is tossed twice. What is the probabil-

ity that at least 1 head occurs?

63 / 70
 Practice

Select randomly an emission from a set emis-

sions which come from three suppliers and are
classified for conformance to air-quality specifi-
cations.

64 / 70
A = emission is from supplier 1, B = emission
conforms to specifications
1 Determine the number of samples in
A′ ∩ B, B ′, and A ∪ B
2 Compute probability of the events in the
previous part 65 / 70
 Practice
A large New York store that sells toys is going
to hold a draw and the winner will receive a free
one-week holiday. The store has 6000 tickets for
sale to its customers during a week in the pre-
Christmas season. If the tickets are numbered
from 1 to 6000, what is the probability that the
number on the winning ticket is a multiple of 2
or 5?

66 / 70
 Table of contents
1 About This Course
2 Experiment, Outcomes and Events
3 Definitions of Probability
4 Some Applications of Probability

67 / 70
 Channel Noise

Noise is an unwanted signal which leads to error

in the message received. One of the performance
issues is the probability that a received message
is not corrupted by noise. Probability plays an
important role in evaluating the performance
of noisy communication channels 68 / 70
 Information Theory
The message generated by the source conveys
some information. One of the objectives of in-
formation theory is to quantify the information
content of messages.

The information content of a message A, I(A),

is given by
1
I(A) = log2( ) 69 / 70
 Reliability Engineering
Reliability theory is concerned with the duration
of the useful life of components and systems of
components. System failure times are unpre-
dictable. Thus, the time until a system fails,
which is referred to as the time to failure of the
system, is usually modeled by a probabilistic func-
tion.

70 / 70