UNIT-V

Coreference Resolution: Coreference Resolution

comes with NLP and it tries to find all linguistic expressions in a given text that refer to
the same real-world entity. This is how it works.

Suppose you have to find the pronouns in a sentence and replace them with relevant
nouns. Coreference resolution can be used to do that. It finds and groups the words which
refer to the same entities and replaces pronouns with noun phrases.

I Gave my book to Abdul because he said that he wants to write the Assignment.

Coreference resolution is using in a variety of NLP tasks such as,

• Text understanding

• Document summarization

• Information extraction

• Sentiment analysis

Different types of References:

Anaphora and cataphora

Anaphora: is the use of the expression whose interpretation depends upon another
expression in context (its antecedent)

The Movie was horrible so that it couldn’t be enjoyable.

Cataphora: When the referring expression is pointing forward then it is cataphora.

Despite of his difficulty rama, went ahead to help him.

Coreference Vs Anaphora:

Not all anaphoric relations are Coreference.

We went to see a movie last night. The tickets were expensive.

The relation is anaphoric but not coreference.

Text Coherence : As we have previously discussed, the coherent discourse in NLP aims
to find the coherence relation among the discourse text.

Hobb’s algorithm is one of the several approaches for pronoun resolution. The algorithm
is mainly based on the syntactic parse tree of the sentences. Hobbs Algorithm is one of the
techniques used for Pronoun Resolu on.

Consider two sentences:

Sentence 1(S1): Jack is an engineer.

Sentence 2 (S2): Jill likes him.

So here, we have the syntactic parse tree of the two sentences as shown.
ti
 The algorithm starts with the target pronoun and walks up the parse tree to the root node
‘S’. For each noun phrase or ‘S’ node that it finds, it does the breadth rst le to right
search of the node’s children to the left of the target. So in our example, the algorithm
starts with the parse tree of the sentence 2 and climbs up to the root node S2. Then it does
a breadth first search to find the noun phrase (NP). Here the algorithm, finds its first noun
phrase for noun ‘Jill’. [Source : https://medium.com/analytics-vidhya/hobbs-algorithm-pronoun-
resolution-7620aa1af538]

Binding theory states that: A re exive can refer to the subject of the most immediate
clause in which it appears, whereas a nonre exive cannot co-refer this subject. Words
such as himself, herself, themselves, etc. are known as reflexive.

Let’s understand this with an example.

• John bought himself a new car.

Here, himself refers to John. Whereas if the sentence is

• John bought him a new car.


fl
fl
fi
ft
So according to the binding theory constraint, ‘him’ in our example will not refer to Jill.

Hence the algorithm now starts the search in the syntax tree of the previous sentence. And
hence the subject Jack in the sentence, Jack is an engineer, is explored before the object
engineer and finally Jack is the resolved referent for the pronoun him.

Discourse:

When we are dealing with Natural Language Processing, the provided language consists
of structured, collective, and consistent groups of sentences, which are termed discourse
in NLP. Discourse Analysis is extracting the meaning out of the corpus or text. Discourse
Analysis is very important in Natural language Processing and helps train the NLP model
better.

Coherence in terms of Discourse in NLP means making sense of the words or making
meaningful connections and correlations. There is a lot of connection between the
coherence and the discourse structure. The coherent rela on tells us that there is some
sort of connection present between the words.

Cohesion: A close relationship based on grammar or meaning between two parts of a

sentence.
Discourse Structure:
So far, we have discussed discourse and coherence, but we have not discussed the
structure of the discourse in NLP. Let us now look at the structure that discourse in NLP
must have. Now, the structure of the discourse depends on the type of segmentation
applied to the discourse.
Algorithms for Discourse Segmentation
We have different algorithms for Unsupervised Discourse Segmenta on and Supervised
Discourse Segmenta on.
Unsupervised Discourse Segmentation
The class of unsupervised segmentation is also termed or represented as linear
segmentation. The unsupervised discourse segmenta on can also be performed with the
help of lexicon cohesion. The lexicon cohesion indicates the relationship among similar
units, for example, synonyms.
ti
ti
ti
ti
 Supervised Discourse Segmentation
But in the supervised discourse segmentation, we only deal with the training data set
having a labeled boundary
Text Coherence
As we have previously discussed, the coherent discourse in NLP aims to find the
coherence relation among the discourse text.
Coherence Relations:
Suppose we have two kinds of related sentences, namely: S0 and S1.
Result
We can say that the second statement, i.e., S1 can be the cause of the first statement, i.e.,
S0. For example, Rahul is late. He will be punished.
In the above example, we can say that the first statement, S0, i.e., Rahul is late, has
caused the second statement, i.e., S1, i.e., He will be punished.
Explanation
Similar to the result, we can say that the first statement, i.e., S0 can be the cause of the
second statement, i.e., S1. For example, Rahul fought with his friend. He was drunk.
Parallel
By the term parallel, we mean that the assertion from the statement S0, i.e., p(a1, a2, …),
and the assertion from the statement S1, i.e. p(b1, b2, …), the ai and bi is similar for all
the values of I.
In simpler terms, it shows us that the sentences are parallel. For example, He wants food.
She wants money. Both of the statements are parallel as there is a sense of want in both
sentences.
Elaboration (Additional Information)
Elabora on means that proposition P is inferring from both the assertions S0 and S1. For
example, Rahul is from Delhi. Rohan is from Mumbai.
Occasion
The occasion takes place when the change in the state is inferred from the first assertion
S0, the final state is inferred from the statement S1, and vice-versa. Let us take an
example to understand the relationship occasion better. For example, Rahul took the
money. he gave it to Rohan.
Let us consider the following phrases and serially number them.
ti
 • S1:
Rahul went to the bank to deposit money.
• S2:
He then went to Rohan's shop.
• S3
He wanted a phone.
• S4
He did not have a phone.
• S5:
He also wanted to buy a laptop from Rohan's shop.
Now the entire discourse can be represented using the below hierarchal discourse
structure.


 Word Senses and WordNet
Word sense: A sense (or word sense) is a discrete representation of one aspect of the
meaning of a word.


WordNet:
• A large online thesaurus —a database that represents word senses with versions in
many languages.
• It also represents relations between senses.
A sense (or word sense) is a discrete representation of one aspect of the meaning of a
word.
• We represent each sense with a superscript:
mouse1 and mouse2 , bank1 and bank2
mouse1 : .... a mouse controlling a computer system in 1968.
mouse2 : .... a quiet animal like a mouse
bank1 : ...a bank can hold the investments in a custodial account ...
bank2 : ...as agriculture burgeons on the east bank, the river ...
Here are the glosses for two senses of bank:
1. financial institution that accepts deposits and channels the money into
lending activities
2. sloping land (especially the slope beside a body of water).
Thesauruses:
➢ They define a sense through its relationship with other senses.

Semantic Role Labeling:

Semantic role labeling (SRL) is a natural language processing (NLP) technique that
involves identifying the syntactic and semantic roles of words in a sentence.
SRL involves identifying the roles of words in a sentence, such as the subject, object, and
verb. The subject is the entity that performs the action, while the object is the entity that
receives the action. The verb is the action itself. For example, in the sentence “John ate the
apple,” “John” is the subject, “ate” is the verb, and “the apple” is the object.
Sometimes, the subject and object are not explicitly stated in the sentence, making it
difficult to determine their roles. In addition, the same word can have different roles
depending on the context in which it is used.
One popular approach to SRL is the FrameNet approach, which uses a database of frames,
which are structured representations of common situations and events, and their associated
semantic roles.
FrameNet maps argument structure for frames, which are evoked by a lexical unit
 Another approach to SRL is the PropBank approach, which uses a database of syntactic
frames, which are templates that describe the structure of a sentence and the roles of its
constituent words.
PropBank is an annotation of syntactically parsed, or treebanked, structures with
`predicate-argument' structures.

[ARG0 John] broke [ARG1 the window]

[ARG1 The window] broke
As this example shows, the arguments of the verbs are labeled as numbered arguments:
Arg0, Arg1, Arg2 and so on.
The second task of the PropBank annotation involves assigning functional tags to all
modifiers of the verb, such as manner (MNR), locative (LOC), temporal (TMP) and
others:
Mr. Bush met him privately, in the White House, on Thursday.
Rel: met
Arg0: Mr. Bush
Arg1: him


ArgM-MNR: privately
ArgM-LOC: in the White House
ArgM-TMP: on Thursday.

Semantic Roles:

Semantic/Thematic roles Verbs describe events or states (‘eventualities’):

Tom broke the window with a rock.
The window broke.
The window was broken by Tom/by a rock.
Thematic roles refer to participants of these events: Agent (who performed the action):
Tom Patient (who was the action performed on): window Tool/Instrument (what was used
to perform the action): rock
 Word-sense disambiguation in NLP

Natural Language Processing (NLP), a branch of artificial intelligence which studies the
ability of computers to interpret and “understand” the human language. It is an open
problem in computational linguistics concerned with identifying which sense of a word is
used in a sentence.
Dictionary-based or Knowledge-based Methods. As the name suggests, for
disambiguation, these methods primarily rely on dictionaries

Lesk’s Algorithm: A simple method for word-sense disambiguation:


 The Lesk de ni on, on which the Lesk algorithm is based is “measure overlap between
sense definitions for all words in context”.
Perhaps one of the earliest and still most commonly used methods for word-sense
disambiguation today is Lesk’s Algorithm, proposed by Michael E. Lesk in 1986. Lesk’s
algorithm is based on the idea that words that appear together in text are related somehow,
and that the relationship and corresponding context of the words can be extracted through
the definitions of the words of interest as well as the other words used around it.

Supervised Methods:

For disambiguation, machine learning methods make use of sense-annotated corpora to

train. These methods assume that the context can provide enough evidence on its own to
disambiguate the sense.

The context is represented as a set of “features” of the words. It includes the information
about the surrounding words also. Support vector machine and memory-based learning are
the most successful supervised learning approaches to WSD.

Semi-supervised Methods

Due to the lack of training corpus, most of the word sense

disambiguation algorithms use semi-supervised learning methods. It is because semi-
supervised methods use both labelled as well as unlabeled data. These methods require
very small amount of annotated text and large amount of plain unannotated text. The
technique that is used by semi-supervised methods is bootstrapping from seed data.

fi
ti
What are the challenges in word sense disambiguation?

WSD faces a lot of challenges and problems.

The most common problem is the difference between various dictionaries or text corpus.
Different dictionaries have different meanings for words, which makes the sense of the
words to be perceived as different. A lot of text information is out there and often it is not
possible to process everything properly.
[Source: https://www.engati.com/glossary/word-sense-disambiguation]

Word sense disambiguation has many applications in various text processing

and NLP fields.
➢ WSD can also be used in Text Mining and Information Extraction tasks.

➢ WSD can be used alongside Lexicography.

➢ Similarly, WSD can be used for Information Retrieval purposes.