Scribd
Upload
6 views

CFG & PCFG

Uploaded by

225003012
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
6 views

CFG & PCFG

Uploaded by

225003012
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 15

CFG & PCFG

CFG & PCFG


Context-Free Grammar (CFG) and Probabilistic Context-Free Grammar (PCFG) are
related formalisms used in linguistics and natural language processing (NLP) to model
the syntax of languages, particularly human languages like English.
CFG PCFG
• CFG is a formal grammar that describes • PCFG is an extension of CFG that
the syntax or structure of a language using incorporates probabilities into the
production rules. In CFGs, each grammar. In PCFGs, each production
production rule is equally likely, and there rule is associated with a probability that
is no notion of probability associated with represents the likelihood of choosing
the rules. that rule during language generation
• CFGs are commonly used in syntactic • By considering the probabilities of
parsing, where they help identify the different parse trees or sentence
syntactic structure of a sentence but do not structures, PCFGs can be used to rank
provide information about the likelihood and choose the most likely
or probability of different parse trees. interpretations of a sentence.
CFG & PCFG Example
CFG Production: PCFG Production (with probabilities):
•S → NP VP •S → NP VP [0.6]
•NP → Det N •NP → Det N [0.8]
•VP → V NP •VP → V NP [0.7]
•Det → "the" •Det → "the" [0.9]
•N → "cat" •N → "cat" [0.6]
•V → "chased" •V → "chased" [0.5]

In the PCFG example, each production rule has a probability associated with
it, indicating how likely it is to be chosen during the generation or parsing
process. This allows PCFGs to capture the likelihood of different sentence
structures or parse trees, which is valuable in various NLP tasks like
machine translation and speech recognition.
CFG & PCFG
The simplest probabilistic model for recursive embedding is a PCFG, a Probabilistic
(sometimes also called Stochastic) Context Free Grammar
Is simply a CFG with probabilities added to the rules, indicating how likely different
rewritings are.
PCFGs are the simplest and most natural probabilistic model for tree structures, the
mathematics behind them is well understood, the algorithms for them are a natural
development of the algorithms employed with HMMs.
PCFGs provide a sufficiently general computational device that they can simulate various
other forms of probabilistic conditioning
Some Features of PCFGs
The predictive power of a PCFG as measured by entropy tends to be greater than that for
a finite state grammar (i.e., an HMM) with the same number of parameters. (For such
comparisons, we compute the number of parameters as follows. A V terminal, n
nonterminal PCFG has n3 + nV parameters, while a K state M output HMM has K2 + MK
parameters. While the exponent is higher in the PCFG case, the number of nonterminals
used is normally quite small.
Inside & Outside Probability

• The Probability of a String: Using inside probabilities


• In general, we cannot efficiently calculate the probability of a string by simply
summing the probabilities of all possible parse trees for the string, as there will be
exponentially many of them.
• An efficient way to calculate the total probability of a string is by the inside
algorithm, a dynamic programming algorithm based on the inside probabilities:
The Probability of a String: Using inside probabilities
Notations
•

• Definition:

• Computed recursively, base case:

• Induction:
Chomsky Normal Form
• Chomsky Normal Form (CNF) grammars only
have unary and binary rules of the form
N →N N
j r s
For syntactic categories

Nj → w k For lexical categories

• The parameters of a PCFG in CNF are


i r s
n3 matrix of parameters
P (N → N N G) (when n nonterminals)
i k nV matrix of parameters n3+nV
P (N → w G ) (when n nonterminals and parameters
V terminals )
P(k N j → N r N s )+ P(N i → wk ) =
∑
1
r ,s
∑
• Any CFG can be represented by a weakly equivalent CFG in CNF
– “weakly equivalent” : “generating the same language”
• But do not assign the same phrase structure to each sentence
CYK Algorithm
• CYK (Cocke-Younger-Kasami) algorithm
– A bottom-up parser using the dynamic programming table
– Assume the PCFG is in Chomsky normal form (CNF)
• Definition
– w1…wn: an input string composed of n words
– wij: a string of words from words i to j
– π[i, j, a]: a table entry holds the maximum probability for a constituent with
non-terminal index a

N
a spaning words wi…wj

w1 …….wi ………..wj ……. wn


Inside probability example
• Consider the following PCFG fragment
NP→DET N 0.8 NP→N 0.2
DET→a 0.6 DET→the 0.4
N→apple 0.8 N→orange 0.2
CYK Algorithm

set to zero
Finding the most
Likely parse for a
sentence
A
on the word-span

m-word input B C
string
n non-terminals
begin m m+1
end
O(m3n3)

bookkeeping
Inside Probability Example
astronomers saw stars with ears
0 1 2 3 45
1 2 3 4 5

astronomers saw stars with ears


Inside Probability Example
astronomers saw stars with ears
Inside Probability Example
fruit fly like a banana
1 2 3 4 5

fruit fly like a banana

You might also like