Algorithms for Reference

Resolution
 Anaphora resolution

• Finding in a text all the referring expressions that

have one and the same denotation
– Pronominal anaphora resolution
– Anaphora resolution between named entities
– Full noun phrase anaphora resolution
Review: What Factors Affect Reference Resolution?
• Lexical factors
– Reference type: Inferability, discontinuous set,
generics, one anaphora, pronouns,…
• Discourse factors:
– Recency
– Focus/topic structure, digression
– Repeated mention
• Syntactic factors:
– Agreement: gender, number, person, case
– Parallel construction
– Grammatical role
 – Selectional restrictions
• Semantic/lexical factors
– Verb semantics, thematic role
• Pragmatic factors
 Reference Resolution

• Given these types of constraints, can we construct

an algorithm that will apply them such that we can
identify the correct referents of anaphors and other
referring expressions?
 Issues

• Which constraints/features can/should we make

use of?
• How should we order them? I.e. which override
which?
• What should be stored in our discourse model?
I.e., what types of information do we need to keep
track of?
• How to evaluate?
 Three Algorithms

• Lappin & Leas ‘94: weighting via recency and

syntactic preferences
• Hobbs ‘78: syntax tree-based referential search
• Centering (Grosz, Joshi, Weinstein, ‘95 and
various): discourse-based search
 Lappin & Leass ‘94

• Weights candidate antecedents by recency and

syntactic preference (86% accuracy)
• Two major functions to perform:
– Update the discourse model when an NP that evokes a
new entity is found in the text, computing the salience
of this entity for future anaphora resolution
– Find most likely referent for current anaphor by
considering possible antecedents and their salience
values
• Partial example for 3P, non-reflexives
 Saliency Factor Weights

• Sentence recency (in current sentence?) 100

• Subject emphasis (is it the subject?) 80
• Existential emphasis (existential prednom?) 70
• Accusative emphasis (is it the dir obj?) 50
• Indirect object/oblique comp emphasis 40
• Non-adverbial emphasis (not in PP,) 50
• Head noun emphasis (is head noun) 80
• Implicit ordering of arguments:
subj/exist pred/obj/indobj-oblique/dem.advPP
On the sofa, the cat was eating bonbons.
sofa: 100+80=180
cat: 100+80+50+80=310
bonbons: 100+50+50+80=280
• Update:
– Weights accumulate over time
– Cut in half after each sentence processed
– Salience values for subsequent referents accumulate for
equivalence class of co-referential items (exceptions,
e.g. multiple references in same sentence)
The bonbons were clearly very tasty.
sofa: 180/2=90
cat: 310/2=155
bonbons: 280/2 +(100+80+50+80)=450
– Additional salience weights for grammatical role
parallelism (35) and cataphora (-175) calculated when
pronoun to be resolved
– Additional constraints on gender/number agrmt/syntax
They were a gift from an unknown admirer.
sofa: 90/2=45
cat: 155/2=77.5
bonbons: 450/2=225 (+35) = 260….
 Reference Resolution
• Collect potential referents (up to four sentences
back): {sofa,cat,bonbons}
• Remove those that don’t agree in number/gender
with pronoun {bonbons}
• Remove those that don’t pass intra-sentential
syntactic coreference constraints
The cat washed it. (itcat)
• Add applicable values for role parallelism (+35) or
cataphora (-175) to current salience value for each
potential antecedent
• Select referent with highest salience; if tie, select
closest referent in string
 A Different Aproach: Centering Theory

• (Grosz et al 1995) examines interactions between

local coherence and the choice of referring
expressions

– A pretty woman entered the restaurant. She sat at the

table next to mine…

– A woman entered the restaurant. They like ice cream.

 Centering theory: Motivation

• (Grosz et al 1995) examine interactions between

local coherence and the choice of referring
expressions
– Pronouns and definite descriptions are not equivalent
with respect to their effect on coherence
– Different inference demands on the hearer/reader.
 Centering theory: Definitions

• The centers of an utterance are discourse entities serving to

link the utterance to other utterances
– Forward looking centers: a ranked list
– A backward looking center: the entity currently ‘in focus’ or
salient
• Centers are semantic objects, not words, phrases, or
syntactic forms but
– They are realized by such in an utterance
– Their realization can give us clues about their likely salience
 More Definitions

• More on discourse centers and utterances

– Un: an utterance
– Backward-looking center Cb(Un): current focus after Un
interpreted
– Forward-looking centers Cf(Un): ordered list of potential
focii referred to in Un
• Cb(Un+1) is highest ranked member of Cf(Un)
• Cf may be ordered subj<exist. Prednom<obj<indobj-
oblique<dem. advPP (Brennan et al)
• Cp(Un): preferred (highest ranked) center of Cf(Un)
 Transitions from Un to Un+1
Cb(Un+1)=Cb(Un) Cb(Un+1)Cb(Un)
or Cb(Un) undef

Cb(Un+1)= Continue Smooth-Shift

Cp(Un+1)

Cb(Un+1) Retain Rough-Shift

Cp(Un+1)
 Rules
• If any element of Cf(Un) is pronominalized in Un+1,
then Cb(Un+1) must also be
• Preference: Continue > Retain > Smooth-Shift >
Rough-Shift
• Algorithm
– Generate Cb and Cf assignments for all possible
reference assignments
– Filter by constraints (syntactic coreference, selectional
restrictions,…)
– Rank by preference among transition orderings
 Example
U1:George gave Harry a cookie. U2:He baked the cookie
Thursday. U3: He ate the cookie all up.
• One
– Cf(U1): {George,cookie,Harry}
– Cp(U1): George
– Cb(U1): undefined
• Two
– Cf(U2): {George,cookie,Thursday}
– Cp(U2): George
– Cb(U2): George
– Continue (Cp(U2)=Cb(U2); Cb(U1) undefined
• Three
– Cf(U3): {George?,cookie}
– Cp(U3): George?
– Cb(U3): George?
– Continue (Cp(U3)=Cb(U3); Cb(U3)= Cb(U2)
• Or, Three
– Cf(U3): {Harry?,cookie}
– Cp(U3): Harry?
– Cb(U3): Harry?
– Smooth-Shift (Cp(U3)=Cb(U3); Cb(U3)  Cb(U2)

The winner is…..George!

Centering Theory vs. Lappin & Leass

• Centering sometimes prefers an antecedent Lappin

and Leass (or Hobbs) would consider to have low
salience
– Always prefers a single pronominalization strategy:
prescriptive, assumes discourse coherent
– Constraints too simple: grammatical role, recency,
repeated mention
– Assumes correct syntactic information available as
input
 Evaluation
• Centering only now being specified enough to be
tested automatically on real data
– Specifying the Parameters of Centering Theory: A
Corpus-Based Evaluation using Text from Application-
Oriented Domains (Poesio et al., ACL 2000)
• Walker ‘89 manual comparison of Centering vs.
Hobbs ‘78
– Only 281 examples from 3 genres
– Assumed correct features given as input to each
– Centering 77.6% vs. Hobbs 81.8%
– Lappin and Leass’ 86% accuracy on test set from
computer training manuals
 Rule-based vs. Statistical Approaches

• Rule-based vs statistical
– (Kennedy & Boguraev 1996), (Lappin & Leass 1994)
vs (Ge, Hale & Charniak 1998)
• Performed on full syntactic parse vs on shallow
syntactic parse
– (Lap 1994), (Ge 1998) vs (Ken 1996)
• Type of text used for the evaluation
– (Lap 1994) computer manual texts (86% accuracy)
– (Ge 1998) WSJ articles (83% accuracy)
– (Ken 1996) different genres (75% accuracy)