Scribd
Upload
9 views

NPTEL NLP Assignment 11.Bin

The document is an assignment for a Natural Language Processing course consisting of multiple-choice questions (MCQs) related to summarization methods, image classification metrics, and AI reliability. It includes questions on macro averaged precision, recall, accuracy, micro averaged precision, and the probability of an article being reliable after being flagged by an AI model. Additionally, it assesses understanding of ROUGE-1 recall in the context of summarization.

Uploaded by

d36078067
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
9 views

NPTEL NLP Assignment 11.Bin

The document is an assignment for a Natural Language Processing course consisting of multiple-choice questions (MCQs) related to summarization methods, image classification metrics, and AI reliability. It includes questions on macro averaged precision, recall, accuracy, micro averaged precision, and the probability of an article being reliable after being flagged by an AI model. Additionally, it assesses understanding of ROUGE-1 recall in the context of summarization.

Uploaded by

d36078067
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 9

Natural Language Processing

Assignment 11

Type of Question: MCQ

Number of Questions: 7 Total Marks: 10 (4*1 + 3*2) [Q5, Q6, Q7 carries two marks each]

Question 1:
Your teacher recommended you to read the book “Introduction to Deep Learning’. After
reading the book, you want to summarize it. What kind of summarization method would you
use for this purpose?
1. Abstractive single document summarization
2. Abstractive multi document summarization
3. Extractive single document summarization
4. Extractive multi document summarization

a. 1, 2
b. 3, 4
c. 1, 3
d. 2, 4

Answer: c
Solution
For question 2-5, use the data given in Table 1.
Suppose you have trained an image classifier with 5 classes - cat, dog, lion, tiger, and deer.
Consider the confusion matrix shown in Table 1.

Question 2:
What is the macro averaged precision?

a. 0.6696
b. 0.6078
c. 0.6433
d. None of the above

Answer: c

Question 3:
What is the macro averaged recall?

a. 0.6464
b. 0.6540
c. 0.6190
d. None of the above

Answer: a
Solution:
Question 4:
What is the accuracy of your classifier?

a. 0.6421
b. 0.6536
c. 0.6319
d. None of the above

Answer: a
Solution:

Question 5:
What is the micro averaged precision?

a. 0.6915
b. 0.6421
c. 0.6245
d. None of the above

Answer: b
Question 6:
It is estimated that 30% of articles generated by an AI system are unreliable. Another AI model
is used to filter these unreliable articles. The AI model claims that it can correctly detect 95%
of unreliable articles, with a false positive rate (classifying a reliable article as unreliable) of
5%. If an article is flagged as unreliable by the AI model, what is the probability that it is actually
a reliable article?
a. 0.019
b. 0.119
c. 0.109
d. None of the above
Answer: c

Solution:
Let, A = Event that a content is detected as fake
B = Event that a generated text is fake
P(B) = 0.2 = 0.3
P(B’′) = 0.8= 0.7
P(A|B) = 0.98=0.95
P(A|B′) = 0.03=0.05
P (B′|A) = P (A|B′)P (B′)/ P (A)

= P (A|B′)P (B′)/ (P (A|B)P (B) + P (A|B′)P (B′))

= (0.05 × 0.7)/(0.95 × 0.3 + 0.05 × 0.7)

≈ 0.109

Question 7:
Consider the system-generated summary (S) and the reference summary (R) as follows:

S: The AI model processes language by leveraging neural networks and large datasets for
training.
R: The AI model uses neural networks and extensive datasets to understand language patterns.

What is the ROUGE-1 recall for the given summary with respect to the reference?

a. 0.625
b. 0.600
c. 0.571
d. None of the above

Answer: d

Unigrams in S: The, AI, model, processes, language, by, leveraging, neural, networks, and, large,
datasets, for, training.
Unigrams in R: The, AI, model, uses, neural, networks, and, extensive, datasets, to, understand,
language, patterns.

Overlapping unigrams: The, AI, model, neural, networks, and, datasets, language.
Count of overlapping unigrams = 8.
Total unigrams in R = 13.

ROUGE-1 recall = 8/13 ≈ 0.615

You might also like