Scribd
Upload
15 views65 pages

Part1_Probability and RV Fundamentals

Chapter 5 covers probability and random processes with a focus on their application in digital communication systems. It discusses key concepts such as conditional probabilities, Bayes' rule, and Gaussian random variables, emphasizing the role of probability in modeling uncertainty in communication. The chapter also introduces random variables, their distributions, and the relationship between multiple random variables in the context of communication links.

Uploaded by

Nanda Mitra
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
15 views65 pages

Part1_Probability and RV Fundamentals

Chapter 5 covers probability and random processes with a focus on their application in digital communication systems. It discusses key concepts such as conditional probabilities, Bayes' rule, and Gaussian random variables, emphasizing the role of probability in modeling uncertainty in communication. The chapter also introduces random variables, their distributions, and the relationship between multiple random variables in the context of communication links.

Uploaded by

Nanda Mitra
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 65

Chapter 5

Probability and Random Processes

Part I
Focused coverage of probability and random variables
(with examples related to communications)

Topics:
Conditional Probabilities
Law of total probability
Bayes’ rule and posterior probabilities
Gaussian random variables
Joint Gaussianity
Components of a Digital
Communication System

2
Basic idea
• Transmitter picks one out of M signals to send

• Receiver gets noisy, possibly distorted signal

• Receiver must guess which transmitted signal was sent

• We use probabilistic modeling and inference to derive algorithms


Role of Probability
• Provides models for uncertainty
• Receiver does not know what bits/symbols the
transmitter has sent, otherwise there would be no
information transfer  models them as random
• Uncontrolled sources of uncertainty modeled as random
• receiver noise, interference
• Once we have a statistical model, can develop systematic
framework for inference
Outline
• Review of probability and random variables

• Lectures review concepts most relevant to comm


• Conditional probabilities and Bayes’ rule
• Gaussian random variables

• Random processes
• Just enough coverage for noise modeling
Probability Basics
• Sample Space (): The set of all possible outcomes of an experiment
Example: Tossing a coin:
Rolling a die:

• Event: A subset of the sample space to which we can assign a probability


Example: Set of odd numbered outcomes when rolling a die {1,3,5}

• Probability Measure (P): It is a function that assigns probability to events

• For an event A,.

• [⋃]
Conditional Probability
• , assuming that we already know that the outcome of the experiment
is in B

• Outcomes corresponding to this probability must therefore belong to


the intersection A∩B. We therefore define the conditional probability
as

[|]

• Conditional probabilities behave just the same as regular


probabilities, since all we are doing is restricting the sample space to
the event being conditioned on
Law of Total Probability
• For two events A and B,

• In the above, we have decomposed an event of interest, A, into a disjoint union of two
events, (A ∩ B) and (A ∩ ).
• The sets B and form a partition of the entire sample space; that is, they are disjoint,
and their union equals Ω.

• If are mutually exclusive events


and , then
= 0.25
= 0.1
Baye’s Rule
• Given and a priori probability P[B]

𝑃 [ 𝐴 ∩ 𝐵]
𝑃 [ B | 𝐴 ]= =𝑃 [ A | 𝐵 ] 𝑃 ¿ ¿
𝑃 [ 𝐴]

a posteriori probability
Random Variables (RVs)
• A random variable is a mapping from the sample space to a set of real numbers
• A random variable assigns a real number to each outcome of a random experiment

• Example: Flip a coin

• Discrete RV: is a discrete RV if it takes a finite number of values. If takes values ,


… then its probability distribution is characterized by its probability mass
function (PMF), as defined below

and

Example: Bernoulli RV, Binomial RV, Poisson RV etc.


• Continuous RV: is a continuous RV if
for each

• Probability Density Function (PDF) When

• Properties of PDF:
• Cumulative Distribution Function (CDF)
p(t)
X

X X

• For a continuous RV

• Properties of CDF: CDF of different cont. RVs

• is non-decreasing in

• is right continuous, i.e.

• Complementary Cumulative Distribution Function (CCDF)


Find its CDF and CCDF.
More than one random variable

• For example, a model of a received sample in a communication link may involve a


randomly chosen transmitted bit, a random channel gain, and a random noise sample.
• In general, we are interested in multiple random variables defined on a “common
probability space,” means simply that we can, in principle, compute the probability of
events involving all of these random variables.
Multiple Random Variables
• Let are RVs defined in a common probability space
• n-dimensional random vector

• Joint CDF:

• Joint PDF from joint CDF

• Joint PMF for discrete RV

• Marginal distributions
Conditional Density (Conditional PDF and PMF)
• Conditional density of given is defined as

, where
Bayes’ rule for conditional density

• Given the conditional density of Y given X, the


conditional density for X given Y is given by
Cond. Probability and Baye’s rule
with discrete and continuous RVs
𝑃 [ 𝑋=0,𝑌 ∊ (𝑦 − ε , 𝑦+ε)]
lim
ε→0 𝑃 [𝑌 ∊ (𝑦 − ε , 𝑦+ε)]

𝑃 [ 𝑌 ∊ ( 𝑦 −ε , 𝑦+ε )|𝑋=0 ] 𝑃[ 𝑋=0]


¿ lim
ε→0 𝑃[𝑌 ∊(𝑦−ε , 𝑦+ε)]

( 2ε ) 𝑝 𝑌 ∨ 𝑋 [ 𝑦| 𝑋=0 ] 𝑃 [ 𝑋=0]
¿ lim
ε→ 0 ( 2ε ) 𝑝 𝑌 [ 𝑦 ]
• Hence, we get

•
Functions of Random Variables

• is also a RV if is a RV

• How to find CDF and PDF of RV


• Method 1 (Find the CDF first):

• The CDF of is given by

Where
Example: and . Find the
CDF and PDF of
Method 2 (Find the PDF directly)

• For differentiable and continuous RVs, let


is satisfied for ,…, .

We can write
• Example:

a) Find the joint PDF


b) Repeat the above when
c) Find the marginal PDFs and
d) Let . Show that is exponential RV with mean 2.
e) Whether Z is independent of?
(c) Verify that

Part d and e are left for you


Homework
• Example: Let Find the PDF of if and are independent RVs.
Mean, Variance and Standard
Deviation
Independent RVs
Independent RVs
• Exception: For Jointly Gaussian Random variables,
if uncorrelated then they are also independent RVs

You might also like