Scribd
Upload
8 views

Lin 13

The document discusses parsing, which involves determining if a string of symbols can be generated by a context-free grammar. It also discusses treebanks, which are corpora with parsed text used for testing parsers, and provides an example of evaluating parser performance metrics on a treebank.

Uploaded by

Brock Ternov
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
8 views

Lin 13

The document discusses parsing, which involves determining if a string of symbols can be generated by a context-free grammar. It also discusses treebanks, which are corpora with parsed text used for testing parsers, and provides an example of evaluating parser performance metrics on a treebank.

Uploaded by

Brock Ternov
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 8

Spurious Ambiguity

• Most parse trees of most NL sentences make no


sense.

19
Parsing
• Given a string of non-terminals and a CFG,
determine if the string can be generated by the
CFG.
– Also return a parse tree for the string
– Also return all possible parse trees for the string
• Must search space of derivations for one that
derives the given string.
– Top-Down Parsing: Start searching space of
derivations for the start symbol.
– Bottom-up Parsing: Start search space of reverse
deivations from the terminal symbols in the string.
Parsing Example

VP

Verb NP
book that flight
book Det Nominal

that Noun

flight
Treebanks
• English Penn Treebank: Standard corpus for
testing syntactic parsing consists of 1.2 M words
of text from the Wall Street Journal (WSJ).
• Typical to train on about 40,000 parsed sentences
and test on an additional standard disjoint test set
of 2,416 sentences.
• Chinese Penn Treebank: 100K words from the
Xinhua news service.
• Other corpora existing in many languages, see the
Wikipedia article “Treebank”

85
First WSJ Sentence

( (S
(NP-SBJ
(NP (NNP Pierre) (NNP Vinken) )
(, ,)
(ADJP
(NP (CD 61) (NNS years) )
(JJ old) )
(, ,) )
(VP (MD will)
(VP (VB join)
(NP (DT the) (NN board) )
(PP-CLR (IN as)
(NP (DT a) (JJ nonexecutive) (NN director) ))
(NP-TMP (NNP Nov.) (CD 29) )))
(. .) ))
86
Parsing Evaluation Metrics
• PARSEVAL metrics measure the fraction of the
constituents that match between the computed and
human parse trees. If P is the system’s parse tree and T
is the human parse tree (the “gold standard”):
– Recall = (# correct constituents in P) / (# constituents in T)
– Precision = (# correct constituents in P) / (# constituents in P)
• Labeled Precision and labeled recall require getting the
non-terminal label on the constituent node correct to
count as correct.
• F1 is the harmonic mean of precision and recall.

87
Computing Evaluation Metrics

Correct Tree T Computed Tree P


S S
VP
VP
Verb NP VP
book Det Nominal Verb NP
the Nominal PP book Det Nominal PP
Noun Prep NP Noun Prep NP
the
flight through Proper-Noun flight through Proper-Noun
Houston Houston
# Constituents: 12 # Constituents: 12
# Correct Constituents: 10
Recall = 10/12= 83.3% Precision = 10/12=83.3% F1 = 83.3%
Treebank Results
• Results of current state-of-the-art systems on the
English Penn WSJ treebank are slightly greater than
90% labeled precision and recall.

89

You might also like