Intelligent Systems

SCE408
Representing Uncertainty
Chapter 13.1 – 13.5
 Propositional Logic
• A symbol in Propositional Logic (PL) is a symbolic
variable whose value must be either True or False,
and which stands for a natural language statement that
could be either true or false

– A = “Ahmed has chest pain”

– B = “Ahmed is depressed”
– C = “It is raining”
Each statement assigned letter such as A,B,C

Not Proposition
What time is now?

3
 Propositional Logic
• Two types:

– Simple proposition:
• Single variable such as A
• Called Atomic sentence

– Complex proposition:
• Multi variable proposition A and B or C
• Called Complex sentence

4
 Logic Operators
➢ A= today is Sunday A B ¬A A∧B A ∨B A ⇔ B
➢ B= today is national day T T F T T T
T F F F T F
❑ Operators
F T T F T F
• Not :¬
F F T F F T
• E.g. ¬A → today is not Sunday
• And : ∧
• E.g. A ∧ B → today is Sunday and today is national day
• Both should be true
• Or : ∨
• E.g. A ∨ B → today is Sunday or today is national day
• Either true or both
• Bi-Conditional /Equivalence : ⇔
• E.g. A ⇔ B → today is Sunday then today is national day
and If today is national day then today is Sunday
5
 Logic Operators
• A= bring Tyson A B A⇒B
T T T
• B= defeat Tyson T F F
F T T

• Operators F F T

• Conditional : ⇒
• E.g. A ⇒ B → If you bring Tyson, I will defeat him
• A implies B

➢ Mean: ¬A ∨ B

• only false if Tyson came but you don’t defeat him

6
 Logic Operators

• A= you take this class A B A⇔B

T T T
• B= you get a grade T F F
F T F
F F T
• Operators
• Bi-Conditional /Equivalence : ⇔
• E.g. A ⇔ B → you get a grade if and only if (iff)
you take this class

7
 Example
• Convert English statement to PL

8
 Propositional Logic Syntax

Sentence → AtomicSentence | ComplexSentence

AtomicSentence → True | False | Symbol
Symbol → P|Q|R|...
ComplexSentence → ¬ Sentence
| ( Sentence ∧ Sentence )
| ( Sentence ∨ Sentence )
| ( Sentence ⇒ Sentence )
| ( Sentence ⇔ Sentence )
BNF (Backus-Naur Form) grammar for Propositional Logic

9
 Uncertainty in the World
• An agent can often be uncertain about the state of the world/domain
since there is often ambiguity and uncertainty.

• Plausible/probabilistic inference
• I’ve got this evidence; what’s the chance this conclusion is true?
• Ex:
✓I’ve got a sore neck; how likely am I to have meningitis?
✓A mammogram test is positive; what’s the probability that the
patient has breast cancer?
 Uncertainty
• Say we have a rule:
if toothache then problem is cavity

• But not all patients have toothaches due to cavities, so we

could set up rules like:
if toothache and ¬gum-disease and ¬filling and ...
then problem = cavity

• This gets complicated; the better method is:

if toothache then problem is cavity with 0.8 probability
or P(cavity | toothache) = 0.8
✓ the probability of cavity is 0.8 given toothache is observed

11
 Example of Uncertainty
• Assume a camera and vision system is used to estimate the
curvature of the road ahead.
• There's uncertainty about which way it curves!
– limited pixel resolution, noise in image.
– algorithm for “road detection” is not perfect.

• This uncertainty can be represented with a simple probability

model:
P(road curves to left | E) = 0.6
P(road goes straight | E) = 0.3
P(road curves to right | E) = 0.1
– where the probability of an event is a measure of agent’s belief
in the event given the evidence E
12
 Probability Theory

• Probability theory serves as a formal means for

– Representing and reasoning with uncertain
knowledge
– Modeling degrees of belief in a proposition (event,
conclusion, diagnosis, etc.)

• Probability is the “language” of uncertainty

– A key modeling method in modern AI

13
 Probability Notation
• A representation that is similar to Propositional
Logic but is more expressive in being able to
represent degrees of belief as probabilities.

• A proposition (event) is:

– A=a which means variable A takes value a
e.g., RoadCurvature = left
– either true or false in the world

– agent has a degree of belief in the proposition

14
 Sample Space
• A space of events in which we assign probabilities
• Events can be binary, multi-valued, or continuous
• Events are mutually exclusive.

• Examples
– Coin flip/tossing: {head, tail}
– Die roll: {1,2,3,4,5,6}
– English words: a dictionary
– High temperature tomorrow: {-100, …, 100}

15
 Random Variable
• A variable, X, whose domain is a sample space,
and whose value is (somewhat) uncertain.

• Examples:
X = coin flip outcome
X = first word in tomorrow’s NYT newspaper
X = tomorrow’s high temperature
• For a given task, the user defines a set of
random variables for describing the world
16
 Random Variable
• Random Variables (RV):
– are capitalized (usually) e.g., Sky, Weather, Temperature.
– refer to attributes of the world whose "status" is unknown.
– have one and only one value at a time.

– have a domain of values that are possible states of the world:

• Boolean: domain = <true, false>
Cavity = true (often abbreviated as cavity )
Cavity = false (often abbreviated as ¬ cavity )

• Discrete: domain is countable (includes Boolean)

values are mutually exclusive and exhaustive
e.g. Sky domain = <clear, partly cloudy, overcast>
Sky = clear abbreviated as clear
Sky ≠ clear also abbreviated as ¬clear

• Continuous: domain is real numbers

18
19
 Probability for Discrete Events
• An agent’s uncertainty is represented by
P(A=a) or simply P(a)
– the agent’s degree of belief that variable A takes on
value a given no other information relating to A

– a single probability called an unconditional or prior

probability

22
 The Axioms of Probability
1. 0 ≤ P(A) ≤ 1
2. P(true) = 1, P(false) = 0
3. P(A ∨ B) = P(A) + P(B) – P(A ∧ B)

Note: Here P(A) means P(A=a) for some value a

and P(A ∨ B) means P(A=a ∨ B=b)

23
 The Axioms of Probability
▪ 0 ≤ P(A) ≤ 1
▪ P(true) = 1, P(false) = 0
▪ P(A ∨ B) = P(A) + P(B) – P(A ∧ B)

The fraction of A can’t

be smaller than 0 Sample
space

24
 The Axioms of Probability
▪ 0 ≤ P(A) ≤ 1 The fraction of A can’t
▪ P(true) = 1, P(false) = 0 be bigger than 1
▪ P(A ∨ B) = P(A) + P(B) – P(A ∧ B)

Sample
space

25
 The Axioms of Probability
▪ 0 ≤ P(A) ≤ 1
▪ P(true) = 1, P(false) = 0
▪ P(A ∨ B) = P(A) + P(B) – P(A ∧ B)

Sample
Valid sentence: e.g., “X=head or X=tail” space

26
 The Axioms of Probability
▪ 0 ≤ P(A) ≤ 1
▪ P(true) = 1, P(false) = 0
▪ P(A ∨ B) = P(A) + P(B) – P(A ∧ B)

Sample
space

Invalid sentence:
e.g., “X=head AND X=tail”

27
 The Axioms of Probability
▪ 0 ≤ P(A) ≤ 1
▪ P(true) = 1, P(false) = 0
▪ P(A ∨ B) = P(A) + P(B) – P(A ∧ B)

Sample
A space

28
 Probability for Discrete Events
• Probability for more complex events, A
▪ P(A = “head or tail”) = 0.5 + 0.5 = 1 fair coin
▪ P(A = “even number”) = 1/6 + 1/6 + 1/6 = 0.5
fair 6-sided die
▪ P(A = “two dice rolls sum to 10”) = 1/36 +
1/36 + 1/36 = 3/36

29
 Quiz
▪ P(A = “two dice rolls even numbers”) =?
▪ P(A=“two dice rolls sum to 10”) =?
▪ P(A = “two dice rolls even numbers” ∧ A=“two dice rolls sum to 10”) =?

30
 Probability Distributions
❑ Given A is a RV taking values in〈a1, a2, … , an〉

e.g., if A is Sky, then a is one of <clear, partly_cloudy, overcast>

• P(a) represents a single probability where A=a

e.g., if A is Sky, then P(a) means any one of

P(clear), P(partly_cloudy), P(overcast)

• P(A) represents a probability distribution

– the set of values: 〈P(a1), P(a2), …, P(an)〉

– If A takes n values, then P(A) is a set of n probabilities

e.g., if A is Sky, then P(Sky) is the set of probabilities:

〈P(clear), P(partly_cloudy), P(overcast)〉

– Property: ∑ P(ai) = P(a1) + P(a2) + ... + P(an) = 1

✓ sum over all values in the domain of variable A is 1 because the domain is
mutually exclusive and exhaustive

31
 Probability Table
• Weather
sunny cloudy rainy
200/365 100/365 65/365

• P(Weather = sunny) = P(sunny) = 200/365

• P(Weather) = 〈200/365, 100/365, 65/365〉

• For now we’ll be satisfied with obtaining the

probabilities by counting frequencies from data

32
34
Full Joint Probability Distribution

Weather
sunny cloudy rainy
Temp hot 150/365 40/365 5/365
cold 50/365 60/365 60/365

• P(Temp=hot, Weather=rainy) = P(hot, rainy) =

5/365 = 0.014

✓ The full joint probability distribution table for n

random variables, each taking k values, has kn
entries.
35
 Full Joint Probability Distribution
Bird Flier Young Probability
T T T 0.0
T T F 0.2
T F T 0.04
T F F 0.01
F T T 0.01
F T F 0.01
F F T 0.23
F F F 0.5

3 Boolean random variables ⇒ 23 – 1 = 7 “degrees

of freedom” (DOF) or “independent values” Sums to 1
36
 Full Joint Probability Distribution

– P(Bird=T) = P(bird) = 0.0 + 0.2 + 0.04 + 0.01 = 0.25

P(bird, ¬flier) =
P(bird ∨ flier) =

37
 Unconditional / Prior Probability
• One’s uncertainty or original assumption about
an event prior to having any data about it or
anything else in the domain.
➢P(Coin = heads) = 0.5
➢P(Bird = T) = 0.0 + 0.2 + 0.04 + 0.01 = 0.22
• Compute by marginalization
We can write the following general marginalization rule for any
sets of variables Y and Z:

39
 Marginal Probability
Weather
sunny cloudy rainy
Temp
hot 150/365 40/365 5/365
cold 50/365 60/365 60/365
∑ 200/365 100/365 65/365

P(Weather) = 〈200/365, 100/365, 65/365〉

Probability distribution for r.v. Weather

P(Weather=Sunny) = P(Weather=Sunny, Temp=hot) + P(Weather=Sunny, , Temp=cold)

P(Weather=Sunny) = 150/365 + 50/356

40
 Marginal Probability
Weather
sunny cloudy rainy ∑
Temp
hot 150/365 40/365 5/365 195/365
cold 50/365 60/365 60/365 170/365

P(Temp) = 〈195/365, 170/365〉

This is nothing but P(B) = ∑i=1…kP(B ∧ A=ai), if

A can take k values

41
 Conditional Probability
• Conditional probabilities
– formalizes the process of accumulating evidence
and updating probabilities based on new evidence
– specifies the belief in a proposition (event, conclusion,
diagnosis, etc.) that is conditioned on a proposition
(evidence, feature, symptom, etc.) being true
• P(a | e): conditional probability of A=a
given E=e evidence is all that is known true
– P(a | e) = P(a ∧ e) / P(e) = P(a, e) / P(e)
– conditional probability can viewed as the joint probability
P(a, e) normalized by the prior probability, P(e)
https://www.mathsisfun.com/data/probability-events-conditional.html
42
 Conditional Probability
• Conditional probabilities behave exactly like
standard probabilities; for example:
0 ≤ P(a | e) ≤ 1
➢ conditional probabilities are between 0 and 1
inclusive
• P(a1 | e) + P(a2 | e) + ... + P(ak | e) = 1
conditional probabilities sum to 1 where a1, …, ak are all
values in the domain of random variable A
• P(¬a | e) = 1 − P(a | e)
negation for conditional probabilities
43
 Conditional Probability

• P(conjunction of events | e)
➢P(a ∧ b ∧ c | e) or as P(a, b, c | e)
is the agent’s belief in the sentence a ∧ b ∧ c
conditioned on e being true
• P(a | conjunction of evidence)
➢P(a | e ∧ f ∧ g) or as P(a | e, f, g)
is the agent’s belief in the sentence a conditioned on
e ∧ f ∧ g being true

44
 Conditional Probability
The conditional probability P(A=a | B=b) is the fraction
of time A=a, within the region where B=b

P(A=a), e.g. P(1st word on a random page = “San”) = 0.001

P(B=b), e.g. P(2nd word = “Francisco”) = 0.0008

A

P(A=a | B=b), e.g. P(1st=“San” | 2nd =“Francisco”) = ?

(possibly: San, Don, Pablo …)

45
 Conditional Probability
Weather
sunny cloudy rainy ∑
Temp
hot 150/365 40/365 5/365 195/365
cold 50/365 60/365 60/365 170/365
P(Temp) = 〈195/365, 170/365〉

P(sunny) = 200/365= 0.54

P(hot) = 195/365=0.53 All=365

P(sunny ,hot) = 150/365=0.41

Hot=195 Cold=170
P(sunny \ hot) = ?
Sunny ? Cloudy ? Rainy ? Sunny ? Cloudy ? Rainy ?

46
 Full Joint Probability Distribution
Bird (B) Flier (F) Young (Y) Probability
T T T 0.0
T T F 0.2
T F T 0.04
T F F 0.01
F T T 0.01
F T F 0.01
F F T 0.23
F F F 0.5

3 Boolean random variables ⇒ 23 – 1 = 7

“degrees of freedom” or “independent values” Sums to 1
47
 Computing Conditional Probability

P(¬B|F) = ?

P(F) = ?

Note: P(¬B|F) means P(B=false | F=true)

and P(F) means P(F=true)
48
 Computing Conditional Probability

P(¬B|F) = P(¬B, F)/P(F)

= (P(¬B, F, Y) + P(¬B, F, ¬Y))/P(F)
= (0.01 + 0.01)/P(F)

P(F) = P(F, B, Y) + P(F, B, ¬Y) + P(F, ¬B, Y) +

P(F, ¬B, ¬Y)
= 0.0 + 0.2 + 0.01 + 0.01
= 0.22 Marginalization

49
 Conditional Probability with
Multiple Evidence

P(¬B | F, ¬Y) = P(¬B, F, ¬Y) / P(F, ¬Y)

= P(¬B, F, ¬Y) / (P(¬B, F, ¬Y) + P(B, F, ¬Y))
= .01 /(.01 + .2)
= 0.048

50
 Conditional Probability
• In general, the conditional probability is
P ( A = a, B ) P ( A = a, B )
P( A = a | B) = =
P( B)  P( A = ai , B)
all ai

• We can have everything conditioned on

some other event(s), C, to get a
conditionalized version of conditional
probability: P( A, B | C )
P ( A | B, C ) =
P( B | C )

‘|’ has low precedence.

This should read: P(A | (B,C))
51