Lecture 1 Introduction to

probability and Random Process

By Tafari Lemma

由NordriDesign提供
www.nordridesign.com
Topics of this lecture

• Application

• Basic set theory

• Factorial

• Permutation

• Combination

1
Introduction
Why study probability and random Processes?
• The theory of probability provides powerful tools to explain, model,
analyze and design technology
• Many systems are very complex (e.g. an operating system)
• It is not easy to model complex systems

What can we do with probabilistic models?

• Use randomness to model various events that can happen.

• Use stochastic models to describe how users/clients interact
with a system
• Quantify the reliability of a system

2
Applications

-Communication, information, and control systems

-Signal processing

-Machine learning

- Biostatistics, bioinformatics, and related fields

- Econometrics and mathematical finance

- Queueing theory and other applications

3
 Communications, information, and control systems

• An example: In design of a mobile handset, we face several “stochastic” or “probabilistic”

issues:
- The signal: only statistical properties may be known.
- The channel characteristic is often time varying.
- Interfering signals coming from other users are noise like.
- Thermal noise at an antenna and RF amplifier.
- Algorithms to recover the signal are often probabilistic.

• Signals (as well as noise) are often represented as Gaussian processes

• Use of statistical estimation and decision theory in modern communications.
• Filtering and prediction have been mathematically formulated based on probability theory
Poisson processes to represent call or packet arrivals.
• Information theory is a branch of applied probability theory.

4
Signal processing

• Processing: filtering, compression, feature extraction, classification.

• Signals are often characterized as random processes.

• Statistical inference theory has been applied.

• Markov process representation.

e.g., Markov model representation of a written English text (Shannon)

5
Machine learning

• Probabilistic reasoning and the Bayesian statistical approach play an

important role.

• Hidden Markov model (HMM), Bayesian network, artificial neural

network (ANN), etc.

• Example: Speech recognition technology

– HMM at phoneme level

– The Viterbi algorithm for sequence estimation

– The expectation-maximization (EM) algorithm to estimate model

parameters.

6
Biostatistics, bioinformatics, and related fields

• Probabilistic formulation and statistical analysis are important

• Related fields include computational biology and epidemiology.
• Examples of well used methods:
– Principal component analysis (PCA)
– Singular value decomposition (SVD)
– Maximum-likelihood estimation
– Hypothesis testing and statistical decision approach
– Receiver operating characteristic (ROC)
• Random processes used in bioinformatics
– Brownian motion
– Diffusion process

7
Econometrics and mathematical finance

• In econometrics, a time series or a discrete-time random process is used

to represent data.
– Autoregressive (AR) model
– Autoregressive moving average (ARMA) model
– Vector autoregression (VAR)
• Numerical Bayesian method
– Markov chain Monte Carlo (MCMC) simulation
• In mathematical finance, a continuous-time random processes is used.
– Brownian motion
– Geometric Brownian motion (GBM)

8
Queueing theory and other applications

• When multiple users contend for a resource simultaneously,

congestion occurs.

• Users may be put in a queue, or rejected (or lost).

• The Poisson process plays an important role

• queueing network models and loss network models

9
Introduction to Set Theory

• A Set is any well defined collection of “objects.”

• The elements of a set are the objects in a set.

• Notation. Usually we denote sets with upper-case letters,

elements with lower-case letters. The following notation is
used to show set membership

• xA means that x is a member of the set A

• xA means that x is not a member of the set A.

10
Ways of Describing Sets
• Give a verbal description

A= 1,2,3,4,5,6
• Give a verbal description
“A is the set of all integers from 1 to 6, inclusive”
• Give a mathematical inclusion rule

A= Integers x 1  x  6

11
 Some Special Sets
• The Null Set or Empty Set. This is a set with no elements,
often symbolized by


• The Universal Set. This is the set of all elements currently
under consideration, and is often symbolized by

12
 Membership Relationships

• Subset.
A  B “A is a subset of B”
• We say “A is a subset of B” if x  A  x  B, i.e., all the
members of A are also members of B.

• Proper Subset
A B “A is a proper subset of B”
• We say “A is a proper subset of B” if all the members of A
are also members of B, but in addition there exists at least
one element c such that c  B but c  A .
13
 Combining Sets

• Set Union
A B
• “A union B” is the set of all elements that are in A, or B, or
both.
• Set Intersection
A B
• “A intersect B” is the set of all elements that
are in both A and B.

14
 Set Complement

A
• “A complement,” or “not A” is the set of all elements not in A.

15
Set Difference

A− B
• The set difference “A minus B” is the set of elements that
are in A, with those that are in B subtracted out. Another
way of putting it is, it is the set of elements that are in A,
and not in B, so A − B = A  B

16
Examples

 = {1,2,3,4,5,6}
A = {1,2,3}
B = {3,4,5,6}

17
Solution (Example)

A  B = {1,2,3,4,5,6}
A  B = {3}
B − A = {4,5,6}
B = {1,2}

18
Venn Diagrams

• Venn Diagrams use topological areas to stand for sets.

19
Mutually Exclusive

We say that two sets A and B are mutually exclusive if

A  B =  , that is, the sets have no elements in
common.

20
Some review Questions

A  A=?

A = ?
A − A=?
21
Some review Questions

A = ?
 = ?

22
Some review Questions

If A  B then A  B = ?

If A  B then A  B = ?

23
Factorial

n! = n factorial is defined as n(n −1) × . . . × 2 ×1

Example: Evaluate 5 factorial.

Solution

5! = 5 × 4 × 3 × 2 × 1 = 120

24
Permutations

A permutation is a particular arrangement. The number of

permutations of n things taken k at a time is,

nPk = n(n −1) × . . . × (n − k +1)

It represents the number of ways of selecting k items of n,

where the order of selection is important.

25
 Permutations

Permutations in terms of Factorials

An alternative formula expressing permutations in terms of factorials is

given by,

nPk = n(n −1) × . . . × (n − k +1)

nPk

nPk

Where
P stands for permutations
n is the number of distinct objects you wish to arrange
k is the number of slots or spaces. 26
 Permutations

Example: How many ways can you arrange the letters A,

B, and C?

Solution

When n = k, nPn = n!

3P3 = 3!/(3-3)!

3P3 =6

27
Permutations

Example:
1.How many ways can you arrange the
letters ‘ BOOK ’ ?

2. How many ways can you arrange the

letters ‘ EXCESSIVE ’ ?

26
 Permutations

Example: How many ways can you assign 5 Nurses to 5 different

tasks?

Solution

When n = k, nPn = n!

5P5 = 5!/(5-5)!

5P5 = 120

29
 Combinations

• In permutations the arrangement of the items is important.

Each unique sequence is another permutation. Thus, ABC
was different from BCA and both were different from CBA.

• In combinations however, ABC, BCA, and CBA are all the

same. They are all part of the same combination.

30
 Combinations

The number of combinations of n things taken k at a time is,

It represents the number of ways of selecting k objects out of n where the

order of selection does not matter.

31
 Combinations

Combinations in terms of factorials

An alternative formula expressing Combinations in terms of factorials is given

by,

Note that combinations are equal to the permutations with the following
relationship

nC k = nPk/k!

32
 Combinations

Example: How many different groups of 3 can be selected from 7 people?

Say these people are named A, B, C, D, E, F, G.
[Note that once you select, say, B, D, and E, the six different arrangements you
can make from them are irrelevant.]

33
 Exercise

1. Suppose we are going to use the symbols {a,

b, c, *, @, #, %} to form a five character
‘Password’ having no repeated character. How
many different passwords are possible?

2. How many ways can you arrange the letters ‘

ABCCCDDDEE ’ ?

34
Thank You !!!

35