CS 188 Introduction to

Summer 2021 Artificial Intelligence Written HW 3

Due: Wednesday 07/28/2021 at 11:59pm (submit via Gradescope).
Policy: Can be solved in groups (acknowledge collaborators) but must be written up individually
Submission: Your submission need not follow this template exactly, but you must tag where each question
begins in your writeup when submitting this HW on Gradescope.

First name Raahim

Last name Muzaffar

SID 3033186067

Collaborators

For staff use only:

Q1. Quadcopter: Spectator /30
Q2. Quadcopter: Data Analyst /36
Q3. Quadcopter: Pilot /27
Total /93

1
Q1. [30 pts] Quadcopter: Spectator
Flying a quadcopter can be modeled using a Bayes Net with the following variables:

• W (weather) ∈ {clear, cloudy, rainy}

• S (signal strength) ∈ {strong, medium, weak}
• P (true position) = (x, y, z, θ) where x, y, z each can take on values ∈ {0, 1, 2, 3, 4} and θ can take on values
∈ {0°, 90°, 180°, 270°}
• R (reading of the position) = (x, y, z, θ) where x, y, z each can take on values ∈ {0, 1, 2, 3, 4} and θ can take
on values ∈ {0°, 90°, 180°, 270°}
• C (control from the pilot) ∈ {forward, backward, rotate left, rotate right, ascend, descend} (6 controls in total)
• A (smart alarm to warn pilot if that control could cause a collision) ∈ {bad, good}

(a) Representation
(i) [2 pts] What is Np , where Np is the domain size of the variable P ? Please explain your answer.

500
Answer: Np =

5^3 * 4 = 500

Explanation:
(ii) [2 pts] Please list all of the Conditional Probability Tables that are needed in order to represent the Bayes
Net above. Note that there are 6 of them.

P(W), P(P), P(S|W), P(R|S, P), P(C | R), P(A|C, P)

2
 (iii) [1 pt] What is the size of the Conditional Probability Table for R? You may use Np in your answer.

750000

Now, assume that we look at this setup from the perspective of Spencer – a spectator who can observe A and W.
Spencer observes A=bad and W =clear, and he now wants to infer the signal strength. In BN terminology, he wants
to calculate P (S|A = bad, W = clear).

(b) [5 pts] Inference by Enumeration

If Spencer chooses to solve for this quantity using inference by enumeration, what is the biggest factor that
must be calculated along the way, and what is its size? You may use Np in your answer. Please show your
work.
Biggest factor:

f(W = clear, S, P, R, C, A = bad)

Size of factor:

18 * Np^2

(c) [5 pts] Inference by Variable Elimination: Order 1

Spencer chooses to solve for this quantity by performing variable elimination in the order of R − P − C. Answer
the following prompts to work your way through this procedure.

(1a) First, we need to eliminate R. Which factors (from the 6 CPTs above) are involved?
P(R|S, P), P(C|R)

(1b) Describe how you eliminate the variable of interest by multiplication of those factors. What conditional
probability factor results from this step?
P(r|S, P) * P(C|r) = f(C|S, P)

(2a) Second, we need to eliminate P . Which factors are involved?

P(A = bad| C, P), P(P), f(C|S, P)

(2b) Describe how you eliminate the variable of interest by multiplication of those factors. What conditional
probability factor results from this step?

We eliminate p from P(A = bad| C, P) * P(P) * f(C|S, P) = f(A = bad, C|S)

3
(3a) Third, we need to eliminate C. Which factor/s are involved?

f(A=bad, C|S)

(3b) Describe how you eliminate the variable of interest by multiplication of those factors. What conditional
probability factor results from this step?

We eliminate c from f(A = bad, C|S) = f(A= bad | S)

(4) List the 3 conditional probability factors that you calculated as a result of the 3 elimination steps above,
along with their domain sizes. You may use Np in your answer. Which factor is the biggest? Is this bigger or
smaller than the biggest factor from the “inference by enumeration” approach?

f(C|S, P), f(A = bad, C|S), f(A = bad|S) . Largest factor is f(C|S, P) (18 *Np) which is larger than the
biggest factor from the inference by enumeration.

(5) List the 1 unused conditional probability factor from the 3 that you calculated above, and also list the 2
resulting conditional probability factors from the 6 original CPTs.

f( A = bad | S), P(S|W = clear), P(W = clear)

(6) Finally, let’s solve for the original quantity of interest: P (S|A = bad, W = clear). After writing the equa-
tions to show how to use the factors from (5) in order to solve for f (S|A = bad, W = clear), don’t forget to
write how to turn that into a probability P (S|A = bad, W = clear).
Hint: use Bayes Rule, and use the 3 resulting factors that you listed in the previous question.

P(S|A = bad, W = clear) = P(A=bad, S, W = clear) / = f(A = bad, S, W = clear) = f(A = bad, W = clear)
P(A=bad, W=clear)

4
(d) [5 pts] Inference by Variable Elimination: Order 2
Spencer chooses to solve for this quantity by performing variable elimination, but this time, he wants to do
elimination in the order of P − C − R. Answer the following prompts to work your way through this procedure.

(1a) First, we need to eliminate P . Which factors (from the 6 CPTs above) are involved?

P(P), P(R|S, P), P(A = bad|C, P)

(1b) Show the multiplication of those factors and eliminate the variable of interest. What conditional proba-
bility factor results from this step?

f(R, A = bad| S, C)

(2a) Second, we need to eliminate C. Which factors are involved? Recall that you might want to use the factor
that resulted from the previous step, but you should not reuse factors that you already used.
P(C|R), f(R, A = bad|S, C)

(2b) Show the multiplication of those factors and eliminate the variable of interest. What conditional proba-
bility factor results from this step?

f(R, A = bad|S)

(3a) Third, we need to eliminate R. Which factor/s are involved? Recall that you might want to use the factor
that resulted from the previous step, but you should not reuse factors that you already used.
f(R, A = bad | S)

(3b) Show the multiplication of those factors and eliminate the variable of interest. What conditional proba-
bility factor results from this step?
f( A = bad|S)

(4) List the 3 conditional probability factors that you calculated as a result of the 3 elimination steps above,
along with their domain sizes. You may use Np in your answer. Which factor is the biggest? Is this bigger or
smaller than the biggest factor from the “inference by enumeration” approach? Is this bigger or smaller than
the biggest factor from R-P-C elimination order?

5
(e) D-Separation
(i) [5 pts] Which variable (in addition to W and A), if observed, would make S ⊥ ⊥ P ? Please shade in the
variable that fits the criteria and run the D-separation algorithm on the resulting graph to support your
answer. If no such variable exists, please provide an explanation for why each candidate fails.

No candidate exists:

S and P fail because if we shade either of them, it does not make S - A - P inactive.

R fails because if we shade in R, it makes S - R - P an active path that only has 1 active triple

C fails because if we shade in C, it makes S - c - P an active path that only has 1 active triple

6
(ii) [5 pts] Ivan claims that there exist two unshaded variables (which are NOT directly connected) such that
if you know the value of all other variables, then those two variables are guaranteed to be independent.
Is Ivan right?
# Yes, and I will shade in all but two variables in the graph below, and run D-separation algorithm to
prove that those two variables are guaranteed to be independent conditioned on all other variables in the
graph.
# No, there is no such two variables, and I will provide a proof below.

7
Q2. [36 pts] Quadcopter: Data Analyst
In this question, we look at the setup from the previous problem, but we consider the quadcopter flight over time.
Here, flight can be considered in discrete time-steps: t ∈ 0, 1, 2, ..., N − 1 with, for example, Pt representing the true
position P at discrete time-step t. Suppose the weather (W) does not change throughout the quadcopter flight.

One key thing to note here is that there are edges going between time t and time t + 1: The true position at time
t + 1 depends on the true position at time t as well as the control input from time t.

Let’s look at this setup from the perspective of Diana, a data analyst who can only observe the output from a
data-logger, which stores R (reading of position) and C (control from the pilot).

(a) Hidden Markov Model

(i) [4 pts] List all the hidden variables and observed variables in this setup. In a few sentences, how is this
setup different from the vanilla Hidden Markov Model you saw in lecture? You should identify at least 2
major differences.

Hidden variables:

W, Si∀i, Pi∀i, Ai∀i

Observed variables:

Ri∀i, Ci∀i

Differences:

There is a hidden variable (A) at the tail of R and C

At all time steps, there are multiple hidden and observation variables

8
(ii) [3 pts] As a data analyst, Diana’s responsibility is to infer the true positions of the quadcopter throughout
its flight. In other words, she wants to find a list of true positions p0 , p1 , p2 , ..., pN −1 that are the most
likely to have happened, given the recorded readings r0 , r1 , r2 , ..., rN −1 and controls c0 , c1 , c2 , ..., cN −1 .

Write down the probability that Diana tries to maximize in terms of a joint probability, and interpret
the meaning of that probability. Note that the objective that you write below is such that Diana is solving
the following problem: maxp0 ,p1 ,...,pN (maximization objective).

Maximization objective:
P(P0 = p0, P1 = p1, ..., Pn = pn, R0 = r0, Rn = rn , C0 = c0, Cn = cn

Explanation:

This is the probability of the positions being there and the observations of the controls/readings also being there.

(iii) [3 pts] Morris, a colleague of Diana’s, points out that maximizing the joint probability is the same as
maximizing a conditional probability where all evidence (r0 , r1 , r2 , . . . and c0 , c1 , c2 , . . . ) are moved to
#
the right of the conditional bar. Is Morris right?
# Yes, and I will provide a proof/explanation below.
# No, and I will provide a counter example below.

Yes. maxpP(p, r, c) is equivalent to maxpP(p|r, c) since P(r, c) is independent of p (our optimization variable) and therefore
counts as a constant.

9
(b) The Markov Property
(i) [5 pts] In this setup, conditioned on all observed evidence, does the sequence P0 , P2 , ...., PN −2 follow the
Markov property? Please justify your answer.

No it does not since there is an active path (Pt-2, Rt-2, St-2) - (W)-(Pt,Rt,St)

(ii) [5 pts] In this setup, conditioned on all observed evidence, does the sequence S0 , S2 , ...., SN −2 follow the
Markov property? Please justify your answer.

Since St+1 is not independent of St-1, it doesn't follow Markov property.

(c) Forward Algorithm Proxy

Conner, a colleague of Diana’s, would like to use this model (with the Rt and Ct observations) to perform
something analogous to the forward algorithm for HMMs to infer the true positions. Let’s analyze below the
effects that certain decisions can have on the outcome of running the forward algorithm.
Note that when we say to not include some variable in the algorithm, we mean that we marginalize/sum
out that variable. For example, if we doPnot want to include W in the algorithm, then we replace P (St |W )
everywhere with P (St ), where P (St ) = W P (St |W )P (W ).
(i) [4 pts] He argues that since W (weather) does not depend on time, and is not something he is directly
interested in, he does not need to include it in the forward algorithm. What effect does not including W
in the forward algorithm have on (a) the accuracy of the resulting belief state calculations, and on (b) the
efficiency of calculations? Please justify your answer.
Accuracy:
Accuracy becomes worse since we are less informed than if W was included.

Efficiency:

Efficiency is better since we only include P(St) instead of P(St|W)

(ii) [3 pts] He also argues that he does not need to include hidden state A (smart alarm warning) in the
forward algorithm. What effect does not including A in the forward algorithm have on (a) the accuracy of
the resulting belief state calculations, and on (b) the efficiency of calculations? Please justify your answer.
Accuracy:
Not including A has no effect on the accuracy since A is downstream of the observations and doesn't impact any
future timesteps.

10
 Efficiency:
A has no effect on efficiency, since it isn't included

(iii) [3 pts] Last but not least, Conner recalls that for the forward algorithm, one should calculate the belief
at time-step t by conditioning on evidence up to t − 1, instead of conditioning on evidence from the entire
trajectory (up to N − 1). Let’s assume that some other algorithm allows us to use evidence from the full
trajectory (t = 0 to t = N − 1) in order to infer each belief state. What is an example of a situation
(in this setup, with the quadcopter variables) that illustrates that incorporating evidence from the full
trajectory can result in better belief states than incorporating evidence only from the prior steps?

# If the signal strength is bad before t − 1, but gets better later.

# If the signal strength is good up to t − 1, and the signal is lost later.
# There isn’t such example because using evidence up to t − 1 gives us the optimal belief.

(d) Policy Reconstruction

Emily, another colleague of Diana’s, would like to use this model to reconstruct the pilot’s policy from data.
Let’s analyze below the effects that certain decisions can have on the outcome of doing policy reconstruction.
(i) [2 pts] Emily states that the probabilistic model for the pilot’s policy is entirely captured in one Condi-
tional Probability Table from the Bayes Net Representation. Which table do you think this is, and explain
why this table captures the pilot’s policy.
Table:
P(C|R)

Explanation:
The probability of control when given a reading of a position is what a policy looks like in this table representation.

(ii) [4 pts] Emily argues that if we were given a lot of data from the data logger, we could reconstruct the
probabilistic model for the pilot’s policy. Is she right?
# Yes, and I will provide an overview of how to reconstruct the pilot’s policy from the data.
# No, and I will provide a list of reasons for why we cannot reconstruct the policy.

11
Q3. [27 pts] Quadcopter: Pilot
In this question, we look at same setup from Question 2, but we now look at it from the perspective of Paul, a
quadcopter pilot who can observe W (weather), R (reading of position), C (control from the pilot), and
A (smart alarm warning). As before, suppose weather (W) does not change throughout the quadcopter’s flight.

(a) Forward Algorithm: The real deal

(i) [2 pts] Now that the only hidden states are St and Pt , is this graph a well-behaving HMM (where Et+1 ⊥
⊥
Et | Xt+1 and Xt+1 ⊥ ⊥ Et | Xt , recall that X is the hidden variable and E is the evidence variable)?
Please explain your reasoning. A subset of your responses from Q2 might apply here.
The graph is not a well-behaving HMM since the observed state Ct affects the hidden state Pt+1

(ii) [4 pts] What is the time-elapsed update from time-step t to time-step t+1? Be sure to include all hidden
states and observed states, and show how to assemble the update from the conditional probability tables
corresponding to the graph. Denote B(St , Pt ) = P (St , Pt | W, R0:t , C0:t , A0:t ). Find B ′ (St+1 , Pt+1 ) =
P (St+1 , Pt+1 | W, R0:t , C0:t , A0:t ) Hint: you can follow the notation from lecture notes.
P
# B ′ (St+1 , Pt+1 ) = P maxst pt P (St+1 | W ) ∗ P (Pt+1 | pt , ct ) ∗ B(St , Pt )
P
# B ′ (St+1 , Pt+1 ) = Pst Ppt P (St+1 | St ) ∗ P (Pt+1 | pt ) ∗ B(St , Pt )
# B ′ (St+1 , Pt+1 ) = Pst Ppt P (Rt+1 |St+1 , Pt+1 )P (St+1 | St ) ∗ P (Pt+1 | pt ) ∗ B(St , Pt )
# B ′ (St+1 , Pt+1 ) = Pst Ppt P (St+1 | St ) ∗ P (Pt+1 | pt )P (Pt+1 | ct ) ∗ B(St , Pt )
# B ′ (St+1 , Pt+1 ) = Pst pt P (St+1 | W ) ∗ P (Pt+1 | pt , ct ) ∗ B(St , Pt )
# B ′ (St+1 , Pt+1 ) = st maxpt P (St+1 | W ) ∗ P (Pt+1 | pt , ct ) ∗ B(St , Pt )

12
 (iii) [4 pts] What is the observation update at time-step t + 1? Be sure to include all hidden states and ob-
served states, and show how to assemble the update from the conditional probability tables corresponding
to the graph. Denote B ′ (St+1 , Pt+1 ) = P (St+1 , Pt+1 | W, R0:t , C0:t , A0:t ), find B(St+1 , Pt+1 )
# B(St+1 , Pt+1 ) = P (Rt+1 | St+1 , Pt+1 ) ∗ P (Ct+1 | Rt+1 ) ∗ P (At+1 | Ct+1 ) ∗ B ′ (St+1 , Pt+1 )
# B(St+1 , Pt+1 ) = P (Rt+1 | St+1 , Pt+1 ) ∗ P (Ct+1 | Rt+1 ) ∗ B ′ (St+1 , Pt+1 )
# B(St+1 , Pt+1 ) = P (Rt+1 | St+1 , Pt+1 ) ∗ P (Ct+1 | Rt+1 ) ∗ P (At+1 | Ct+1 , Pt+1 ) ∗ B ′ (St+1 , Pt+1 )
# B(St+1 , Pt+1 ) ∝ P (Rt+1 | St+1 , Pt+1 ) ∗ P (Ct+1 | Rt+1 ) ∗ P (At+1 | Ct+1 , Pt+1 ) ∗ B ′ (St+1 , Pt+1 )
# B(St+1 , Pt+1 ) ∝ P (Rt+1 | St+1 , Pt+1 ) ∗ P (Ct+1 | Rt+1 ) ∗ B ′ (St+1 , Pt+1 )

(b) Consider a simpler scenario where we only track the 2D position (x, y) of the quadcopter. Paul, the pilot, wants
to infer the quadcopter’s true position P as accurately as possible.
• x, y each can take on values ∈ {0, 1, 2}.
• We have four controls: forward, backward, left, and right.
• Let variable ER be Paul’s estimate of the current position, and this variable depends on the reading R.
The utility is based on the difference between the estimate of current position ER and the actual position
P : U (P, ER ) = −||P − ER ||x − ||P − ER ||y , in dollars.
• We consider only one time step. In that time step the reading R is (1, 0) and that the weather is cloudy.
• Under cloudy weather, the signal strength can take on 2 values with equal probability: weak and strong.
The signal strengths correspond to the following errors in readings:
– Weak: The reading R returns a random number (for each position element) sampled uniformly from
the domain of possible positions.
– Strong: The reading R is identical to the true position.
Answer the following questions:
(i) [2 pts] Among the hidden variables S and P , Which variable should intuitively have the greatest VPI?
Explain your answer. You should not do any calculations for this part.

P should have the greatest VPI since our utility

Paul’s coworker offers to tell him the signal strength (S) in exchange for some cash.
(ii) [3 pts] Suppose the signal strength is strong. Given the current reading R, what is the Maximum Expected
Utility after knowing this information of S?

VPI(S) = MEU(S) - MEU(θ).

MEU(S) = 1/3

13
 (iii) [3 pts] Suppose the signal strength is weak. Given the current reading R, what is the Maximum Expected
Utility after knowing this information of S?

(iv) [3 pts] Considering the possibility of both signal strength, how much should Paul should pay to know this
information of S?

(c) (i) [3 pts] Suppose your coworker only tells you the signal strength with probability q, and with probability
1 − q, they don’t tell you the signal strength even after payment. How much would you be willing to pay
in this scenario? Your result should contain q.

14
(ii) [3 pts] How much would you pay to know the true position (P)?

15