Parsing – Earley

Algorithm
◦ Subject: ML&NLP

◦ Instructor:
◦ Dr. K. Vimal Kumar
◦ Department of CSE & IT,
◦ Jaypee Institute of Information
Technology, Noida
Introduction to Earley Parser
◦ Dynamic programming approach created in 1970 by Earley.

◦ Earley algorithm basically integrates both the top-down and bottom-up approach.

◦ Top-down approach is used to various possible parse trees as per the grammar.

◦ Bottom-up approach is used to removes ambiguity in the parsing structure.

◦ Partial parse tree structure is stored in a special data structure called as chart. Hence,
Earley algorithm is also called as chart parsing.

2
Introduction to Earley Parser
◦ The chart is populated with the help of dotted grammatical rules.
◦ The dotted grammatical rules makes use of following three functions,
◦ Predictor
◦ Scanner
◦ Completer

3
Introduction to Earley Parser
◦ The chart is populated with the help of dotted grammatical rules.
◦ The dotted grammatical rules makes use of following three functions,
◦ Predictor [dot is placed on the left of a NT – top-down]
◦ For example,
◦ S →. 𝑁𝑃 𝑉𝑃 − 𝑠𝑖𝑔𝑛𝑖𝑓𝑖𝑒𝑠 𝑡ℎ𝑎𝑡 𝑡ℎ𝑒 𝑝𝑎𝑟𝑠𝑖𝑛𝑔 ℎ𝑎𝑠 𝑛𝑜𝑡 𝑠𝑡𝑎𝑟𝑡𝑒𝑑 𝑎𝑛𝑑 𝑖𝑡 𝑒𝑥𝑝𝑒𝑐𝑡𝑠 𝑎 𝑁𝑃
◦ It also includes every rules that has NP on the LHS.
◦ S → 𝑁𝑃. 𝑉𝑃 − 𝑠𝑖𝑔𝑛𝑖𝑓𝑖𝑒𝑠 𝑡ℎ𝑎𝑡 𝑡ℎ𝑒 𝑝𝑎𝑟𝑠𝑖𝑛𝑔 ℎ𝑎𝑠 𝑠𝑡𝑎𝑟𝑡𝑒𝑑 𝑎𝑛𝑑 𝑖𝑡 𝑒𝑥𝑝𝑒𝑐𝑡𝑠 𝑎 𝑉𝑃
◦ It includes every rules that has VP on the LHS.
◦ Scanner
◦ Completer

4
Introduction to Earley Parser
◦ The chart is populated with the help of dotted grammatical rules.
◦ The dotted grammatical rules makes use of following three functions,
◦ Predictor [dot is placed on the left of a NT]
◦ Scanner [dot is placed on the left of a terminal – Bottom-up]
◦ For example,
◦ S →. det 𝑁𝑂𝑀 − 𝑠𝑖𝑔𝑛𝑖𝑓𝑖𝑒𝑠 𝑡ℎ𝑎𝑡 𝑡ℎ𝑒 𝑝𝑎𝑟𝑠𝑖𝑛𝑔 ℎ𝑎𝑠 𝑠𝑡𝑎𝑟𝑡𝑒𝑑 𝑎𝑛𝑑 𝑖𝑡 𝑒𝑥𝑝𝑒𝑐𝑡𝑠 𝑎 𝑑𝑒𝑡𝑒𝑟𝑚𝑖𝑛𝑒𝑟
◦ It also doesn’t include any new rules.
◦ Completer

5
Introduction to Earley Parser
◦ The chart is populated with the help of dotted grammatical rules.
◦ The dotted grammatical rules makes use of following three functions,
◦ Predictor [dot is placed on the left of a NT]
◦ Scanner [dot is placed on the left of a terminal – Bottom-up]
◦ Completer [dot is placed at the end of a grammar rule – Bottom-up]
◦ For example,
◦ S → 𝑁𝑃 𝑉𝑃. −𝑠𝑖𝑔𝑛𝑖𝑓𝑖𝑒𝑠 𝑡ℎ𝑎𝑡 𝑡ℎ𝑒 𝑝𝑎𝑟𝑠𝑖𝑛𝑔 𝑖𝑠 𝑐𝑜𝑚𝑝𝑙𝑒𝑡𝑒 𝑓𝑜𝑟 𝑉𝑃
◦ It also includes every rules that had dot on left of S but with a shifting of dot next to S.

6
Introduction to Earley Parser
S  NP VP statement Det  that | a | an |my
S  Aux NP VP question Noun  book|flight|
S  VP command meal| elephant |
NP  Det NOM pyjamas
NP  Proper-Noun | Pronoun Proper-Noun Houston
NOM  Noun | Noun NOM | NOM PP Verb  book | include |
VP  Verb | Verb NP | VP PP | Verb NP PP prefer | shot
PP  Prep NP Aux  does
Pronoun  I Prep  from | to | on | in

VP . V NP
S  . VP

Bring that book

7
Introduction to Earley Parser
S  NP VP statement Det  that | a | an |my
S  Aux NP VP question Noun  book|flight|
S  VP command meal| elephant |
NP  Det NOM pyjamas
NP  Proper-Noun | Pronoun Proper-Noun Houston
NOM  Noun | Noun NOM | NOM PP Verb  book | include |
VP  Verb | Verb NP | VP PP | Verb NP PP prefer | shot
PP  Prep NP Aux  does
Pronoun  I Prep  from | to | on | in

S  . VP VP V . NP
NP . Det Nom
V
Bring that book
8
Introduction to Earley Parser
S  NP VP statement Det  that | a | an |my
S  Aux NP VP question Noun  book|flight|
S  VP command meal| elephant |
NP  Det NOM pyjamas
NP  Proper-Noun | Pronoun Proper-Noun Houston
NOM  Noun | Noun NOM | NOM PP Verb  book | include |
VP  Verb | Verb NP | VP PP | Verb NP PP prefer | shot
PP  Prep NP Aux  does
Pronoun  I Prep  from | to | on | in

S  . VP VP V . NP
NP Det . Nom
Nom  . Noun
V Det
Bring that book
9
Introduction to Earley Parser
S  NP VP statement Det  that | a | an |my
S  Aux NP VP question Noun  book|flight|
S  VP command meal| elephant |
NP  Det NOM pyjamas
NP  Proper-Noun | Pronoun Proper-Noun Houston
NOM  Noun | Noun NOM | NOM PP Verb  book | include |
VP  Verb | Verb NP | VP PP | Verb NP PP prefer | shot
PP  Prep NP Aux  does
Pronoun  I Prep  from | to | on | in

S  . VP VP V . NP
NP Det . Nom
Nom  Noun .
V Det Noun
Bring that book 10
Introduction to Earley Parser
S  NP VP statement Det  that | a | an |my
S  Aux NP VP question Noun  book|flight|
S  VP command meal| elephant |
NP  Det NOM pyjamas
NP  Proper-Noun | Pronoun Proper-Noun Houston
NOM  Noun | Noun NOM | NOM PP Verb  book | include |
VP  Verb | Verb NP | VP PP | Verb NP PP prefer | shot
PP  Prep NP Aux  does
Pronoun  I Prep  from | to | on | in

S  . VP VP V . NP NP Det Nom .

Nom  Noun .
V Det Noun
Bring that book
11
Introduction to Earley Parser
S  NP VP statement Det  that | a | an |my
S  Aux NP VP question Noun  book|flight|
S  VP command meal| elephant |
NP  Det NOM pyjamas
NP  Proper-Noun | Pronoun Proper-Noun Houston
NOM  Noun | Noun NOM | NOM PP Verb  book | include |
VP  Verb | Verb NP | VP PP | Verb NP PP prefer | shot
PP  Prep NP Aux  does
Pronoun  I Prep  from | to | on | in
VP V NP .

S  . VP NP Det Nom .

Nom  Noun .
V Det Noun
Bring that book 12
Introduction to Earley Parser

S  VP .

VP V NP .

NP Det Nom .

Nom  Noun .
V Det Noun
Bring that book

13
Earley Parser - Example
◦ Consider the parsing of string “baaba” for the grammar below,
◦ 𝑆 → 𝐴𝐵 | 𝐵𝐶
◦ 𝐴 → 𝐵𝐴 | 𝑎
◦ 𝐵 → 𝐶𝐶 | 𝑏 Chart-0
◦ 𝐶 → 𝐴𝐵 | 𝑎
𝛾 →. 𝑆 0,0
◦ Augment the grammar as below,
𝑆 →. 𝐴𝐵 0,0
◦ 𝛾→𝑆 𝑆 →. 𝐵𝐶 0,0
◦ 𝑆 → 𝐴𝐵 | 𝐵𝐶 𝐴 →. 𝐵𝐴 0,0
◦ 𝐴 → 𝐵𝐴 | 𝑎
𝐴 →. 𝑎 0,0
𝐵 →. 𝐶𝐶 0,0
◦ 𝐵 → 𝐶𝐶 | 𝑏
𝐵 →. 𝑏 0,0
◦ 𝐶 → 𝐴𝐵 | 𝑎 𝐶 →. 𝐴𝐵 0,0
𝐶 →. 𝑎 (0,0)
14
Earley Parser - Example
◦ Consider the parsing of string “baaba” for the grammar below,
◦ 𝑆 → 𝐴𝐵 | 𝐵𝐶
◦ 𝐴 → 𝐵𝐴 | 𝑎 Chart-0 Chart-1
◦ 𝐵 → 𝐶𝐶 | 𝑏
𝛾 →. 𝑆 0,0 𝐵 → 𝑏. 0,1
◦ 𝐶 → 𝐴𝐵 | 𝑎
𝑆 →. 𝐴𝐵 0,0 𝑆 → 𝐵. 𝐶 0,1
𝑆 →. 𝐵𝐶 0,0 𝐴 → 𝐵. 𝐴 0,1
𝐴 →. 𝐵𝐴 0,0 𝐴 →. 𝐵𝐴 1,1
𝐴 →. 𝑎 0,0 𝐴 →. 𝑎 1,1
𝐵 →. 𝐶𝐶 0,0 𝐶 →. 𝐴𝐵 1,1
𝐵 →. 𝑏 0,0 𝐶 →. 𝑎 (1,1)
𝐶 →. 𝐴𝐵 0,0 𝐵 →. 𝐶𝐶 1,1
𝐶 →. 𝑎 (0,0) 𝐵 →. 𝑏 1,1

15
Earley Parser - Example Chart-2

◦ Consider the parsing of string “baaba” for the grammar below,

𝐴 → 𝑎. 1,2
𝐶 → 𝑎. (1,2)
◦ 𝑆 → 𝐴𝐵 | 𝐵𝐶 𝑆 → 𝐵𝐶. 0,2
◦ 𝐴 → 𝐵𝐴 | 𝑎 Chart-1 𝐵 → 𝐶. 𝐶 1,2
◦ 𝐵 → 𝐶𝐶 | 𝑏
𝐴 → 𝐵𝐴. 0,2
𝐵 → 𝑏. 0,1 𝐶 → 𝐴. 𝐵 1,2
◦ 𝐶 → 𝐴𝐵 | 𝑎 𝛾 → 𝑆. 0,2
𝑆 → 𝐵. 𝐶 0,1
𝐴 → 𝐵. 𝐴 0,1 𝑆 → 𝐴. 𝐵 0,2
𝐴 →. 𝐵𝐴 1,1 𝐶 → 𝐴. 𝐵 0,2
𝐴 →. 𝑎 1,1 𝐵 →. 𝐶𝐶 2,2
𝐶 →. 𝐴𝐵 1,1 𝐵 →. 𝑏 2,2
𝐶 →. 𝑎 1,1 𝐶 →. 𝐴𝐵 2,2
𝐵 →. 𝐶𝐶 1,1 𝐶 →. 𝑎 (2,2)
𝐵 →. 𝑏 1,1 𝐴 →. 𝐵𝐴 2,2
𝐴 →. 𝑎 2,2
16
Earley Parser - Example
Chart-2
◦ String=“baaba”
𝐴 → 𝑎. 1,2 Chart-3
◦ 𝑆 → 𝐴𝐵 | 𝐵𝐶 𝐶 → 𝑎. (1,2)
◦ 𝐴 → 𝐵𝐴 | 𝑎 𝑆 → 𝐵𝐶. 0,2 𝐴 → 𝑎. 2,3
𝐵 → 𝐶. 𝐶 1,2 𝐶 → 𝑎. 2,3
◦ 𝐵 → 𝐶𝐶 | 𝑏
𝐴 → 𝐵𝐴. 0,2 𝐵 → 𝐶. 𝐶 2,3
◦ 𝐶 → 𝐴𝐵 | 𝑎 𝐶 → 𝐴. 𝐵 1,2 𝐵 → 𝐶𝐶. 1,3
𝛾 → 𝑆. 0,2 𝐶 → 𝐴. 𝐵 2,3
𝑆 → 𝐴. 𝐵 0,2 𝐶 →. 𝐴𝐵 3,3
𝐶 → 𝐴. 𝐵 0,2 𝐶 →. 𝑎 (3,3)
𝐵 →. 𝐶𝐶 2,2 𝐵 →. 𝐶𝐶 3,3
𝐵 →. 𝑏 2,2 𝐵 →. 𝑏 (3,3)
𝐶 →. 𝐴𝐵 2,2 𝐴 →. 𝐵𝐴 3,3
𝐶 →. 𝑎 (2,2) 𝐴 →. 𝑎 3,3
𝐴 →. 𝐵𝐴 2,2 𝐴 → 𝐵. 𝐴 1,3
𝐴 →. 𝑎 2,2 17
Earley Parser - Example
◦ String=“baaba”
Chart-3 Chart-4
◦ 𝑆 → 𝐴𝐵 | 𝐵𝐶
◦ 𝐴 → 𝐵𝐴 | 𝑎 𝐴 → 𝑎. 2,3 𝐵 → 𝑏. 3,4
𝐶 → 𝑎. 2,3 𝐶 → 𝐴𝐵. 2,4
◦ 𝐵 → 𝐶𝐶 | 𝑏
𝐵 → 𝐶. 𝐶 2,3 𝐴 → 𝐵. 𝐴 3,4
◦ 𝐶 → 𝐴𝐵 | 𝑎 𝐵 → 𝐶. 𝐶 2,4
𝐵 → 𝐶𝐶. 1,3
𝐶 → 𝐴. 𝐵 2,3 𝐴 →. 𝐵𝐴 4,4
𝐶 →. 𝐴𝐵 3,3 𝐴 →. 𝑎 4,4
𝐶 →. 𝑎 (3,3) 𝐶 →. 𝐴𝐵 4,4
𝐵 →. 𝐶𝐶 3,3 𝐶 →. 𝑎 (4,4)
𝐵 →. 𝑏 (3,3) 𝐵 →. 𝐶𝐶 4,4
𝐴 →. 𝐵𝐴 3,3 𝐵 →. 𝑏 (4,4)
𝐴 →. 𝑎 3,3
𝐴 → 𝐵. 𝐴 1,3
18
 Chart-5
Earley Parser - Example 𝐴 → 𝑎. 4,5
𝐶 → 𝑎. 4,5
◦ String=“baaba”
𝐴 → 𝐵𝐴. 3,5
Chart-4 𝐶 → 𝐴. 𝐵 4,5
◦ 𝑆 → 𝐴𝐵 | 𝐵𝐶 𝐵 → 𝐶𝐶. 2,5
◦ 𝐴 → 𝐵𝐴 | 𝑎 𝐵 →. 𝑏 3,4 𝐵 → 𝐶. 𝐶 4,5
◦ 𝐵 → 𝐶𝐶 | 𝑏 𝐶 → 𝐴𝐵. 2,4 𝐵 →. 𝐶𝐶 5,5
𝐴 → 𝐵. 𝐴 3,4 𝐵 →. 𝑏 (5,5)
◦ 𝐶 → 𝐴𝐵 | 𝑎 𝑆 → 𝐴𝐵. 0,5
𝐵 → 𝐶. 𝐶 2,4
𝐴 →. 𝐵𝐴 4,4 𝐶 → 𝐴𝐵. 0,5
𝐴 →. 𝑎 4,4 𝐴 → 𝐵. 𝐴 2,5
𝐶 →. 𝐴𝐵 4,4 𝐶 →. 𝐴𝐵 5,5
𝐶 →. 𝑎 (4,4) 𝐶 →. 𝑎 5,5
𝐵 →. 𝐶𝐶 4,4 𝛾 → 𝑆. 0,5
𝐵 →. 𝑏 (4,4) 𝐵 → 𝐶. 𝐶 0,5
𝐴 →. 𝐵𝐴 5,5
𝐴 →. 𝑎 5,5
19
Earley Parser – Example-2
◦ Using Earley Parser, parse the Sentence – “Book the flight to Houston.”
◦ Grammar G is,

S  NP VP statement Det  the | a | an |my

S  VP command Noun  book | flight
NP  Det NOM | Proper-Noun Proper-Noun Houston
NOM  Noun | NOM noun| NOM PP Verb  book | include |
VP  Verb | Verb NP | VP PP | Verb NP PP Prep  from | to | on | in
PP  Prep NP

20
Earley Parser – Example-2
◦ Using Earley Parser, parse the Sentence – “Book the flight to Houston.”
◦ Augmented Grammar G is,

𝛾→S Det  the

S  NP VP statement Noun  book | flight
S  VP command Pr-Noun Houston
NP  Det NOM | Pr-Noun Verb  book
NOM  Noun | NOM noun| NOM PP Prep  to
VP  Verb | Verb NP | VP PP | Verb NP PP
PP  Prep NP

21
Earley Parser – Example-2
◦ Using Earley Parser, parse the Sentence – “Book the flight to Houston.”

Chart-0

𝛾 → .S (0,0) – P

S  .NP VP (0,0) - P
S  .VP (0,0) – P
NP  .Det NOM (0,0) - P
NP  .Pr-Noun (0,0) – P
VP  .Verb (0,0) -P
VP  .Verb NP (0,0) -P
VP  .VP PP (0,0) -P
VP  .Verb NP PP (0,0) –P
Det  .the (0,0) - S

Pr-Noun .Houston (0,0) – S

Verb  .book (0,0) – S

22
Earley Parser – Example-2
Chart-1
◦ Sentence – “Book the flight to Houston.” Verb  book. (0,1) – C

Chart-0 VP  Verb. (0,1) - C

𝛾 → .S (0,0) – P
VP  Verb. NP (0,1) - P
VP  Verb. NP PP (0,1) – P
S  .NP VP (0,0) - P
S  VP. (0,1) – C
S  .VP (0,0) – P
VP  VP. PP (0,1) – P
NP  .Det NOM (0,0) - P
NP  .Pr-Noun (0,0) – P NP  .Det NOM (1,1) – P
NP  .Pr-Noun (1,1) – P
VP  .Verb (0,0) -P
VP  .Verb NP (0,0) -P 𝛾 → S. (0,1) – C
VP  .VP PP (0,0) -P
VP  .Verb NP PP (0,0) –P PP  .Prep NP (1,1) – P
Det  .the (0,0) - S Det  .the (1,1) – S

Pr-Noun .Houston (0,0) – S Pr-Noun .Houston (1,1) – S

Verb  .book (0,0) – S Prep  to (1,1) - S

23
Earley Parser – Example-2
Chart-1
◦ Sentence – “Book the flight to Houston.” Verb  book. (0,1) – C

Chart-2 VP  Verb. (0,1) - C

Det  the. (1,2) – C
VP  Verb. NP (0,1) - P
VP  Verb. NP PP (0,1) – P
NP  Det .NOM (1,2) – P S  VP. (0,1) – C
NOM  .Noun (2,2) – P VP  VP. PP (0,1) – P
NOM  .NOM noun (2,2) – P
NP  .Det NOM (1,1) – P
NOM  .NOM PP (2,2) – P
NP  .Pr-Noun (1,1) – P
Noun  .book (2,2) – S
Noun  .flight (2,2) – S 𝛾 → S. (0,1) – C
PP  .Prep NP (1,1) – P
Det  .the (1,1) – S

Pr-Noun .Houston (1,1) – S

Prep  .to (1,1) - S

24
Earley Parser – Example-2
Chart-3
Noun  flight. (2,3) – C
◦ Sentence – “Book the flight to Houston.”
NOM  Noun. (2,3) – C
Chart-2
NP  Det NOM. (1,3) – C
Det  the. (1,2) – C NOM  NOM. noun (2,3)- P
NOM  NOM. PP (2,3)- P
NP  Det .NOM (1,2) – P
VP  Verb NP. (0,3) - C
NOM  .Noun (2,2) – P VP  Verb NP. PP (0,3) – P
NOM  .NOM noun (2,2) – P
NOM  .NOM PP (2,2) – P S  VP. (0,3) – C
VP  VP .PP (0,3) – P
Noun  .book (2,2) – S
Noun  .flight (2,2) – S Noun  .book (3,3) – S
Noun  .flight (3,3) – S
𝛾 → S. (0,3) – C

PP  .Prep NP (3,3) – P
Prep  .to (3,3) - S
25
Earley Parser – Example-2
Chart-3
Noun  flight. (2,3) – C
◦ Sentence – “Book the flight to Houston.”
NOM  Noun. (2,3) – C
Chart-4
NP  Det NOM. (1,3) – C
Prep  to. (3,4) – C NOM  NOM. noun (2,3)- P
NOM  NOM. PP (2,3)- P
PP  Prep .NP (3,4) – P
VP  Verb NP. (0,3) - C
NP  .Det NOM (4,4) - P VP  Verb NP. PP (0,3) – P
NP  .Pr-Noun (4,4) – P
S  VP. (0,3) – C
Det  .the (4,4) – S VP  VP .PP (0,3) – P
Pr-Noun .Houston (4,4) – S Noun  .book (3,3) – S
Noun  .flight (3,3) – S
𝛾 → S. (0,3) – C

PP  .Prep NP (3,3) – P
Prep  .to (3,3) - S
26
 Chart-5
Pr-Noun Houston. (4,5) – C

Earley Parser – Example-2 NP  Pr-Noun. (4,5) - C

PP  Prep NP. (3,5) – C

VP  VP PP. (0,5) – C
◦ Sentence – “Book the flight to Houston.” NOM  NOM PP. (2,5) – C
VP  Verb NP PP. (0,5) - C
Chart-4
S  VP. (0,5) – C
Prep  to. (3,4) – C VP  VP. PP (0,5) – P
PP  Prep .NP (3,4) – P 𝛾 → S. (0,5) – C
NP  .Det NOM (4,4) - P PP  .Prep NP (5,5) – P
NP  .Pr-Noun (4,4) – P
Prep  .to (5,5) – S
Det  .the (4,4) – S
NP  Det NOM. (1,5) – C
Pr-Noun .Houston (4,4) – S
NOM  NOM. noun (2,5) – P
NOM  NOM. PP (2,5) – P

Noun  .book (5,5) – S

Noun  .flight (5,5) – S
VP  Verb NP. (0,5) – C
VP  Verb NP. PP (0,5) - P 27
Earley Parser – Example-2

28
References

1. Daniel Jurafsky and James H. Martin. 2009. Speech and Language Processing (2nd
Edition). Prentice-Hall, Inc., USA.
2. Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural Language Processing with
Python (1st. ed.). O'Reilly Media, Inc.

29