SlideShare a Scribd company logo

Learning Multilingual Semantic Parsers for Question Answering over Linked Data - A comparison of neural and probabilistic graphical model architectures

S
shakimov

The document summarizes a PhD dissertation defense talk on learning multilingual semantic parsers for question answering over linked data. It discusses comparing neural and probabilistic graphical model architectures for semantic parsing to map natural language to formal meaning representations. The talk outlines introducing dependency parse tree-based approaches, evaluating different model architectures, and addressing challenges in building multilingual question answering systems over structured knowledge bases.

Technology
1 of 122
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
Learning Multilingual Semantic Parsers for 
 Question Answering over Linked Data A comparison of neural and probabilistic graphical model architectures PhD Dissertation Defense Talk March 2019 Sherzod Hakimov Semantic Computing Group, CITEC Bielefeld University
Outline 1. Introduction 2. Motivation & Problem 3. Preliminaries 4. Dependency parse tree-based Semantic Parsing Approach 5. Evaluation of different architectural choices for “simple question” answering 6. Discussion 7. Conclusion 2
Introduction 3
Introduction 44 Human-machine interaction Give me the route to Jahnplatz
Introduction 55 Human-machine interaction Give me the route to Jahnplatz ?
Introduction 66 Human-machine interaction ? Knowledge Base 6
7 Give me the route to Jahnplatz What is Semantic Parsing? • mapping natural language sentence to a detailed meaning representation
8 Give me the route to Jahnplatz route($LOC, “Jahnplatz”) What is Semantic Parsing? • mapping natural language sentence to a detailed representation of meaning representation route(StartLocation, EndLocation)
9 Give me the route to Jahnplatz route($LOC, “Jahnplatz”) What is Semantic Parsing? • mapping natural language sentence to a detailed representation of meaning representation • meaning representation can be modelled using a formal language that
10 Give me the route to Jahnplatz route($LOC, “Jahnplatz”) What is Semantic Parsing? • mapping natural language sentence to a detailed representation of meaning representation • meaning representation can be modelled using a formal language, e.g. lambda calculus • an ontology with properties, classes, entities, etc. (route, create_calendar_event, set_alarm) • supports automated execution or reasoning
11 Give me the route to Jahnplatz Query Knowledge Base Answer Why do we need Semantic Parsers? route($LOC, “Jahnplatz”)
12 Give me the route to Jahnplatz Query Knowledge Base Answer Why do we need Semantic Parsers? route($LOC, “Jahnplatz”)
Motivation • Building semantic parsers with application on Question Answering 13 Linked Data Cloud
Motivation • Building semantic parsers with application on Question Answering • Building multilingual solutions that can be applied for multiple languages 14 Which German politicians were born in Bielefeld? Which metal has a liquid form? Welche deutschen Politiker wurden in Bielefeld geboren? Welches Metall hat eine flüssige Form? ¿Qué políticos alemanes nacieron en Bielefeld? ¿Qué metal tiene una forma líquida?
Motivation • Building semantic parsers with application on Question Answering • Building multilingual solutions that can be extended for other languages • Comparison and evaluation of different model architectures 15
Motivation • Building semantic parsers with application on Question Answering • Building multilingual solutions that can be extended for other languages • Comparison and evaluation of different model architectures • Highlight the challenges of building Question Answering systems 16
• based on structured content from Wikipedia • more than 130 languages supported • 760 classes, 1105 object & 1622 data type properties • ca. 9 million resources 17
dbr:Dan_Brown dbo:author dbo:author dbr:Tom_Hanks dbo:starring dbo:starring dbo:basedOn dbo:basedOn dbo:Writer dbo:Book dbo:Film dbo:Actor DBpedia Ontology & Data 18
dbr:Dan_Brown dbo:author Question Answering on RDF Data dbr:Inferno_(novel) 19
dbr:Dan_Brown dbo:author Question Answering on RDF Data dbr:Inferno_(novel)Triple: dbr:Inferno_(novel) dbo:author dbr:Dan_Brown 20
dbr:Dan_Brown dbo:author Question Answering on RDF Data dbr:Inferno_(novel) Dan Brown is the author of Inferno Triple: Natural Language: dbr:Inferno_(novel) dbo:author dbr:Dan_Brown 21
dbr:Dan_Brown dbo:author Question Answering on RDF Data dbr:Inferno_(novel) Dan Brown is the author of Inferno Who is the author of Inferno?Natural Language: Question format dbr:Inferno_(novel) dbo:author dbr:Dan_BrownTriple: Natural Language: 22
dbr:Dan_Brown dbo:author Question Answering on RDF Data dbr:Inferno_(novel) Dan Brown is the author of Inferno dbr:Inferno_(novel) dbo:author dbr:Dan_BrownTriple: Natural Language: Who is the author of Inferno?Natural Language: SPARQL Query: Question format SELECT ?x WHERE {dbr:Inferno_(novel) dbo:author ?x} 23
dbr:Dan_Brown dbo:author Question Answering on RDF Data dbr:Inferno_(novel)Who is the author of Inferno? Semantic Parser SELECT ?x WHERE {dbr:Inferno_(novel) dbo:author ?x} 24
Research Questions How to map natural language phrases into knowledge base entries for multiple languages? Which linguistic resources can be used? 25 dbr:Dan_Brown dbo:author Who is the author of Inferno? dbr:Inferno_(novel) Who wrote Inferno? Who is the writer of Inferno? SELECT ?x WHERE {dbr:Inferno_(novel) dbo:author ?x}
Research Questions How to map natural language phrases into knowledge base entries for multiple languages? Which linguistic resources can be used? 26 dbr:Dan_Brown dbo:author Who is the author of Inferno? dbr:Inferno_(novel) Who wrote Inferno? Who is the writer of Inferno? SELECT ?x WHERE {dbr:Inferno_(novel) dbo:author ?x} Lexical Gap: write -> dbo:author
Research Questions How to disambiguate URIs when multiple candidates are retrieved from mapping natural language tokens into knowledge base entries? 27 When was Inferno released? SELECT ?x WHERE {dbr:Inferno_(novel) dbo:releaseDate ?x} dbr:Inferno_(2016_film) dbr:Inferno_(novel)
Research Questions How to use syntactic information of a natural language question together with semantic representations of entries in a knowledge base? 28 Who wrote Inferno? dbr:Dan_Brown dbo:author dbr:Inferno_(novel) SELECT ?x WHERE { dbr:Inferno_(novel) dbo:author ?x } wrote
 (VERB) Who
 (PRON) Inferno
 (PROPN) nsubj dobj
Research Questions What are the advantages and the disadvantages of a multilingual QA system vs. a monolingual system built for each language? 29 Who is the author of Inferno? dbr:Dan_Brown dbo:author dbr:Inferno_(novel) Wer ist der Autor von Inferno? ¿Quién es el autor de Inferno? SELECT ?x WHERE { dbr:Inferno_(novel) dbo:author ?x }
Research Questions What effort is required to adapt our QA pipelines to another language? 30 Who is the author of Inferno? dbr:Dan_Brown dbo:author dbr:Inferno_(novel) Qui est l'auteur de Inferno? Infernoning muallifi kim? SELECT ?x WHERE { dbr:Inferno_(novel) dbo:author ?x }
Preliminaries 31
Preliminaries • Logical Form - DUDES, formalism for specifying meaning representations for dependency tree structures 32
Preliminaries • Logical Form - DUDES, formalism for specifying meaning representations for dependency tree structures • Semantic Composition - acquiring the meaning representations using the syntax of questions 33
Logical Form 34 • DUDES - Dependency-based Underspecified Discourse Representation Structures (Cimiano et al [1]) [1] Cimiano, P., 2009, Flexible semantic composition with DUDES. In Proceedings of the Eighth International Conference on Computational Semantics (pp. 272-276). Association for Computational Linguistics.
Logical Form 35 • DUDES - Dependency-based Underspecified Discourse Representation Structures (Cimiano et al [1]) • Formalism for specifying meaning representation • Flexible semantic composition w.r.t order of application • Build on semantic dependencies e.g. suitable for working with dependency-based syntactic analysis
 [1] Cimiano, P., 2009, Flexible semantic composition with DUDES. In Proceedings of the Eighth International Conference on Computational Semantics (pp. 272-276). Association for Computational Linguistics.
DUDES v : is the main variable vs : the projection variables l : is the label of the main DRS drs : is a DRS (Discourse Representation Structure) slots : is a set of semantic dependencies 36
Semantic Composition with DUDES Who created Wikipedia? Input: a natural language question and its dependency parse tree 37
Semantic Composition with DUDES Who created Wikipedia? Input: a natural language question and its dependency parse tree dbr:Wikipedia dbo:author ?x Output: a meaning representation based on certain domain 38
Semantic Composition with DUDES Each node gets a pair of assignments: DUDES Type + Knowledge base ID Oracle 39
Semantic Composition with DUDES - Bottom Up 40
Semantic Composition with DUDES - Bottom Up 41 1 2
Semantic Composition with DUDES - Bottom Up 42 2
Semantic Composition with DUDES - Bottom Up 43 2
Semantic Composition with DUDES - Bottom Up 44 2
Semantic Composition with DUDES - Bottom Up 45 x
Semantic Composition with DUDES 46
Logical Form into SPARQL 47
Dependency parse tree-based Semantic Parsing Approach 48
Dependency parse tree-based Semantic Parsing Approach • multilingual semantic parsing approach: English, German & Spanish [1] 49 [1] Hakimov S, Jebbara S, Cimiano P. AMUSE: Multilingual Semantic Parsing for Question Answering over Linked Data. In Proceedings of the 16th International Semantic Web Conference (ISWC), 2017
Dependency parse tree-based Semantic Parsing Approach • multilingual semantic parsing approach: English, German & Spanish [1] • uses language-independent dependency parse trees from Universal Dependencies 50 [1] Hakimov S, Jebbara S, Cimiano P. AMUSE: Multilingual Semantic Parsing for Question Answering over Linked Data. In Proceedings of the 16th International Semantic Web Conference (ISWC), 2017
Dependency parse tree-based Semantic Parsing Approach • multilingual semantic parsing approach: English, German & Spanish [1] • uses language-independent dependency parse trees from Universal Dependencies • combines different types of lexical information: DBpedia Ontology labels, the M-ATOLL[2] lexicon & word embeddings 51 [1] Hakimov S, Jebbara S, Cimiano P. “AMUSE: Multilingual Semantic Parsing for Question Answering over Linked Data”. ISWC 2017 [2] Walter S, Unger C, and Cimiano P. “M-ATOLL: A Framework for the Lexicalization of Ontologies in Multiple Languages”. ISWC 2014 [3] Hakimov S, Walter S, Unger C, and Cimiano P. “Applying semantic parsing to question answering over linked data: Addressing the lexical gap”. NLDB 2015
Dependency parse tree-based Semantic Parsing Approach Oracle SELECT ?x WHERE { dbr:Wikipedia dbo:author ?x. } Semantic Composition 52
Dependency parse tree-based Semantic Parsing Approach Oracle SELECT ?x WHERE { dbr:Wikipedia dbo:author ?x. } Semantic Composition Structure Learning 53
Dependency parse tree-based Semantic Parsing Approach • inference • factor graph model 54
Inference • Metropolis-Hastings: exploring huge search space (ca. 10 million resources, 2000 properties) • Linking to Knowledge Base (L2KB) •objective : compare set of URIs to the expected set of URIs • Query Construction (QC) •objective : compare the constructed query to the expected query 55 Input: initial state
L2KB Sampling Explore the edges and assign Knowledge Base IDs based on lemmas of nodes Inverted index: Ontology labels, lexicon from M-ATOLL & word embeddings 56
L2KB Sampling Explore the edges and assign Knowledge Base IDs based on lemmas of nodes Check the triple pattern- ?x dbo:author dbr:Wikipedia : Slot 2, dbr:Wikipedia dbo:author ?x : Slot1 Inverted index: Ontology labels, lexicon from M-ATOLL & word embeddings 57
58
59
60
QC Sampling Assign DUDES with Return Variable and KB ID to nodes and assign remaining slots 61
Inference 62 m steps
Model Representation 63 Observed variables: dependency parse tree Hidden variables: KB IDs, slot, DUDE types
Model Representation 64 Observed variables: dependency parse tree Hidden variables: KB IDs, slot, DUDE types
Model Representation 65 Observed variables: dependency parse tree Hidden variables: KB IDs, slot, DUDE types Child_Node [POS: PROPN, DUDES: Resource] Parent_Node [POS: VERB, DUDES:Property] Sample Feature
Evaluation Dataset: Question Answering over Linked Data (QALD), 6th challenge English, German, Spanish, Italian, French, Dutch, Romanian, Farsi 350 for train, 100 for test Unger, Christina, Axel-Cyrille Ngonga Ngomo, and Elena Cabrio (2016). “6th open challenge on question answering over linked data (qald-6)”. In: Semantic Web Evaluation Challenge. 66
Evaluation DBP: lexicon from DBpedia Ontology & WordNet
 M-ATOLL: lexicon induced by the M-ATOLL (Walter et al. 2014)
 Embed: lexicon added using pre-trained word embeddings (Mikolov et al. 2013) Walter, Sebastian, Christina Unger, and Philipp Cimiano. “M-ATOLL: A Framework for the Lexicalization of Ontologies in Multiple Languages”. ISWC 2014 Mikolov, Tomas et al. “Distributed representations of words and phrases and their compositionality”. NIPS 2013 67
Evaluation DBP: lexicon from DBpedia Ontology & WordNet
 M-ATOLL: lexicon induced by the M-ATOLL (Walter et al. 2014)
 Embed: lexicon added using pre-trained word embeddings (Mikolov et al. 2013)
 Dict: manually defined lexicon Walter, Sebastian, Christina Unger, and Philipp Cimiano. “M-ATOLL: A Framework for the Lexicalization of Ontologies in Multiple Languages”. ISWC 2014 Mikolov, Tomas et al. “Distributed representations of words and phrases and their compositionality”. NIPS 2013 68
Evaluation DBP: lexicon from DBpedia Ontology & WordNet
 M-ATOLL: lexicon induced by the M-ATOLL (Walter et al. 2014)
 Embed: lexicon added using pre-trained word embeddings (Mikolov et al. 2013)
 Dict: manually defined lexicon Walter, Sebastian, Christina Unger, and Philipp Cimiano. “M-ATOLL: A Framework for the Lexicalization of Ontologies in Multiple Languages”. ISWC 2014 Mikolov, Tomas et al. “Distributed representations of words and phrases and their compositionality”. NIPS 2013 69
Evaluation of Lexicon 70 English German Spanish
Evaluation of different architectural choices for “simple question” answering 71
Outline • SimpleQuestions dataset, 74k samples, Freebase data • Question: “Who wrote Mildred Pierced?” • Fact: mildred_pierced, book.written_work.author, stuart_kaminsky • Answer: mildred_pierced, book.written_work.author, ?x • Systematic comparison of different model architectures 72 Hakimov S, Jebbara S, Cimiano P. “Evaluating Architectural Choices for Deep Learning Approaches for Question Answering over Knowledge Bases”. ICSC 2019
Named Entity Recognition • Used by all models to predict the entity span • Character & word embeddings • Trained using weak supervision: inference is correct if the expected entity has been found 73
Candidate Retrieval Candidate pairs: subject and predicate 74 Surface form frequencies
Model1: BiLSTM-Softmax 75 Model2: BiLSTM-KB Model3: BiLSTM-Binary Model4: Fasttext [1] Architectures [1] A. Joulin, E. Grave, P. Bojanowski, T. Mikolov, "Bag of Tricks for Efficient Text Classification", 2016, arxiv.org
Model1: BiLSTM-Softmax 76 Model2: BiLSTM-KB Model3: BiLSTM-Binary Model4: Fasttext [1] Architectures [1] A. Joulin, E. Grave, P. Bojanowski, T. Mikolov, "Bag of Tricks for Efficient Text Classification", 2016, arxiv.org
Evaluation 77 Named Entity Linking Named Entity Recognition Predicate & Answer Prediction
Evaluation 78 Named Entity Linking Named Entity Recognition 0,82 Predicate & Answer Prediction
Evaluation 79 Named Entity Linking Named Entity Recognition 0,82 Predicate & Answer Prediction
Evaluation 80 Named Entity Linking Named Entity Recognition 0,82 Predicate & Answer Prediction
Discussion 81
Discussion • Manual Effort • Syntax and Semantics • Multilinguality • Cross-domain Transferability • Training Data Size and Search Space 82
Discussion 83 Systems Manual Effort Syntax & Semantics Multilinguality Cross-domain transferability Training Data & 
 Number of KB IDs CCG-based
 (Chapter 6) CCG combination rules
 manual lexicon • learned in tandem • CCG for syntax • lambda calculus for semantics manual effort is required manual effort is required 600 training instances 750 entities Dependency- based
 (Chapter 7) feature templates syntax is given DUDES as a formalism an adaptable solution • a dependency parser required. • e.g. biomedical domain 300 training instances <= 10 mil. entities
 >= 2000 predicates BiLSTM-Softmax
 (Chapter 8) - • word & char embed. for lexical & contextual info • semantics is limited to a single predicate and a subject entity an adaptable solution only word & char embed. an adaptable solution only word & char embed. >= 75K instances
 <= 2 mil. entities
Research Questions 84
Research Questions 85 • Ontology lexicalisations, e.g. M-ATOLL (Walter et al. 2014) • Ontology labels, e.g. DBpedia labels • Dictionaries • WordNet synsets • Induced from contextual embeddings of words RQ1: How to map natural language phrases into knowledge base entries for multiple languages? Which linguistic resources can be used?
Research Questions 86 • Supervised models with objective for disambiguation • CCG-based model • uses lexical and syntactic information as features • Dependency tree-based model • syntactic dependency between words, lexical similarity, ontology restrictions • Neural network-based model • ranking objective of predicates RQ2: How to disambiguate URIs when multiple candidates are retrieved from mapping natural language tokens into knowledge base entries?
Research Questions 87 • Semantic Parsing • bottom-up composition • CCG-based model • learns the syntax and semantics together • Dependency tree-based model • learns composing semantics based on dependency trees RQ3: How to use syntactic information of a natural language question together with semantic representations of entries in a knowledge base?
Research Questions 88 RQ4: What are the advantages and the disadvantages of a multilingual QA system vs. a monolingual system built for each language? • Advantages • Multilingual: broader coverage • Monolingual: higher performance, e.g. Xser (Xu et al. 2014) 0.7 F1 on QALD-4 • Disadvantages • Multilingual: lower performance, e.g. AMUSE 0.3 F1 on QALD-6 • Monolingual: need expertise, e.g. CCG rules, lexicon
Research Questions 89 • CCG-based model • grammar rules, manually defined lexicon • language-specific • Dependency parse tree-based model • dependency parse tree generator • lexicon • Neural network-based model • depends on the training data RQ5: What effort is required to adapt our QA pipelines to another language?
Conclusion • Address the lexical gap for QA systems • Incorporate ontology lexicalizations to reduce the lexical gap • Use Universal Dependencies to build language-independent QA pipeline • Multilingual semantic parsing for Question Answering • Evaluate different QA models under the certain conditions • Highlight importance of building blocks of a pipeline for a fair comparison 90
CCG-based Semantic Parsing Approach 91
GENLEX Barack Obama is married to Michelle Obama [1] Zettlemoyer, Luke S and Michael Collins (2005). “Learning to Map Sentences to Logical Form : Structured Classification with Probabilistic Categorial Grammars”. In: 21st Conference on Uncertainty in Artificial Intelligence [2] Hakimov, Sherzod et al. (2015). “Applying semantic parsing to question answering over linked data: Addressing the lexical gap”. In: International Conference on Applications of Natural Language to Information Systems 92
CCG & GENLEX Rules 93
Semantic Composition Barack Obama is married to Michelle Obama 94
Evaluation QALD-4 95
Lexicon During sampling, compute cosine similarity of words into Ontology labels of properties
 Vectors for multiple words are summed, e.g. V(population) + V(total) 96
Lexicon Restriction Class 97
Dataset - GeoQuery 600 train, 280 test instances 98
Dataset - QALD 99
Dataset-QALD 100
Dataset - SimpleQuestions Train: 75,910 Validation: 10,845 Test: 21,687 Total: 108,442 101
Dataset Complexity Lexical Overlap, tokens exist in both 102
Semantic Composition •recursively computing the meaning of each node from the meanings of its child nodes •build the meaning representation bottom-up ComposeSemantics(dependency-parse-tree) If parse-tree is a terminal node (word) then return an atomic lexical meaning for the word. Else For each child, subtreei, of parse-tree Create its MR by calling ComposeSemantics(subtreei) Return an MR by properly combining the resulting MRs for its children into an MR for the overall parse-tree. 103
Features 104
Feature Templates 105 L2KB QC
L2KB Features 106
QC Features 107
Model Representation 108 Observed variables: dependency parse tree Hidden variables: KB IDs, slot, DUDE types • States can be ranked by • objective score : compare to ground truth • model score: computed using feature weights • Training procedures • switch between model & objective score after every iteration
Chapter 8 109
Model Score 110
Model1: BiLSTM-Softmax • Softmax layer that predicts predicates seen during training • Encoding layer: word & character • BiLSTM: two LSTM layers (backward, forward) 111
Model2: BiLSTM-KB • Learn embedding of predicates in KB • Encoding layer: word & character • BiLSTM: two LSTM layers (backward, forward) • Output layer computes cosine similarity to all predicates and chooses the closest 112
Model3: BiLSTM-Binary • Encoding layer: encodes input question with word & character embeddings • Encoding layer: encodes input predicate with word & character embeddings • Output layer: binary decision 113
Model4: Fasttext • Document classification tool developed by Facebook* • Uses word & character embeddings • Softmax layer that predicts the expected predicate 114 * http://fasttext.cc
Factor Graph 115
116 Generative Models -> computing joint probability distribution on p(y|x)
 HMM -> y_t depends on y_t-1 and x_t how output label y_t generates input vector x Discriminative Models -> computing conditional probability distribution over inputs x and outputs y
 CRF -> doesn’t have any limitation like that how feature vector x gets assignment y_t
Discussion 117
Manual Effort • CCG-based model • define CCG grammar rules, hand-crafted lexicon for domain independent phrases • Dependency parse tree-based model • feature functions • Neural network-based model (BiLSTM-Softmax) • not required 118
Syntax and Semantics • CCG-based model • syntax and semantics is learned in tandem • CCG for syntax and the lambda calculus for semantics • syntax guides the semantics of the sentences • Dependency parse tree-based model • syntax is given and the semantics is learned • DUDES as a formalism for semantics, syntax is based on dependency trees from Universal Dependencies • Neural network-based model (BiLSTM-Softmax) • syntactic information is learned, e.g. word and character embeddings provide contextual information • semantics is based on a single subject and the predicate, simpler task 119
Multilinguality • CCG-based model • CCG grammar rules needs to be extended • Dependency parse tree-based model • a multilingual solution • Neural network-based model (BiLSTM-Softmax) • can be adapted to other languages, e.g. word & characters as features 120
Cross-domain Transferability • CCG-based model • manual effort is required: CCG rules, lexicon • Dependency parse tree-based model • dependency parse trees e.g. biomedical domain • Neural network-based model (BiLSTM-Softmax) • can be easily adapted 121
Training Data Size and Search Space • CCG-based model • Dependency parse tree-based model • Neural network-based model (BiLSTM-Softmax) • heavily depends on the data 122

More Related Content

What's hot (20)

PPTX
Natural Language Processing in R (rNLP)
fridolin.wild
 
PPTX
Named entity recognition (ner) with nltk
Janu Jahnavi
 
PPTX
Ir 1 lec 7
alaa223
 
PPTX
Text Mining Infrastructure in R
Ashraf Uddin
 
PPT
Chapter 10 Data Mining Techniques
Houw Liong The
 
PPT
Copy of 10text (2)
Uma Se
 
PPT
Web and text
Institute of Technology Telkom
 
PDF
Text Indexing / Inverted Indices
Carlos Castillo (ChaTo)
 
PDF
TermPicker: Enabling the Reuse of Vocabulary Terms by Exploiting Data from th...
JohannWanja
 
PPTX
RDF2Vec: RDF Graph Embeddings for Data Mining
Petar Ristoski
 
PPTX
DS2014: Feature selection in hierarchical feature spaces
Petar Ristoski
 
PPTX
SWT Lecture Session 8 - Rules
Mariano Rodriguez-Muro
 
PPTX
SWT Lecture Session 3 - SPARQL
Mariano Rodriguez-Muro
 
ODP
Information Extraction from the Web - Algorithms and Tools
Benjamin Habegger
 
PDF
Link Discovery Tutorial Part I: Efficiency
Holistic Benchmarking of Big Linked Data
 
PDF
Link Discovery Tutorial Part V: Hands-On
Holistic Benchmarking of Big Linked Data
 
PDF
Lecture: Question Answering
Marina Santini
 
PDF
Link Discovery Tutorial Part II: Accuracy
Holistic Benchmarking of Big Linked Data
 
PPTX
Performance of graph query languages
Athiq Ahamed
 
PPTX
Enriching the semantic web tutorial session 1
Tobias Wunner
 
Natural Language Processing in R (rNLP)
fridolin.wild
 
Named entity recognition (ner) with nltk
Janu Jahnavi
 
Ir 1 lec 7
alaa223
 
Text Mining Infrastructure in R
Ashraf Uddin
 
Chapter 10 Data Mining Techniques
Houw Liong The
 
Copy of 10text (2)
Uma Se
 
Web and text
Institute of Technology Telkom
 
Text Indexing / Inverted Indices
Carlos Castillo (ChaTo)
 
TermPicker: Enabling the Reuse of Vocabulary Terms by Exploiting Data from th...
JohannWanja
 
RDF2Vec: RDF Graph Embeddings for Data Mining
Petar Ristoski
 
DS2014: Feature selection in hierarchical feature spaces
Petar Ristoski
 
SWT Lecture Session 8 - Rules
Mariano Rodriguez-Muro
 
SWT Lecture Session 3 - SPARQL
Mariano Rodriguez-Muro
 
Information Extraction from the Web - Algorithms and Tools
Benjamin Habegger
 
Link Discovery Tutorial Part I: Efficiency
Holistic Benchmarking of Big Linked Data
 
Link Discovery Tutorial Part V: Hands-On
Holistic Benchmarking of Big Linked Data
 
Lecture: Question Answering
Marina Santini
 
Link Discovery Tutorial Part II: Accuracy
Holistic Benchmarking of Big Linked Data
 
Performance of graph query languages
Athiq Ahamed
 
Enriching the semantic web tutorial session 1
Tobias Wunner
 

Similar to Learning Multilingual Semantic Parsers for Question Answering over Linked Data - A comparison of neural and probabilistic graphical model architectures (20)

PDF
Deep Learning for Information Retrieval
Roelof Pieters
 
PDF
The web of interlinked data and knowledge stripped
Sören Auer
 
PPTX
From Semantics to Self-supervised Learning for Speech and Beyond (Opening Ke...
linshanleearchive
 
PPTX
semantic web & natural language
Nurfadhlina Mohd Sharef
 
PDF
Getting Started with Knowledge Graphs
Peter Haase
 
PDF
Schema-agnositc queries over large-schema databases: a distributional semanti...
Andre Freitas
 
PPT
DODDLE-OWL: A Domain Ontology Construction Tool with OWL
Takeshi Morita
 
PDF
Multilingual qa
shakimov
 
PPTX
Identifying Topics in Social Media Posts using DBpedia
Óscar Muñoz García
 
PPT
Understanding Natural Language Queries over Relational Databases
Ashis Kumar Chanda
 
PPTX
Neural Text Embeddings for Information Retrieval (WSDM 2017)
Bhaskar Mitra
 
PDF
Deep Learning for Natural Language Processing: Word Embeddings
Roelof Pieters
 
PDF
Scalable Cross-lingual Document Similarity through Language-specific Concept ...
Carlos Badenes-Olmedo
 
PPT
Understanding Natural Language Queries over Relational Databases
Ashis Chanda
 
PPTX
A Panorama of Natural Language Processing
Ted Xiao
 
PDF
Approach to leverage Websites to APIs through Semantics
Ioannis Stavrakantonakis
 
PPTX
LSI latent (par HATOUM Saria et DONGO ESCALANTE Irvin Franco)
rchbeir
 
PDF
What you Can Make Out of Linked Data
Marco Fossati
 
PDF
ESWC 2014 Tutorial part 3
Miriam Fernandez
 
PDF
Aussenac semanticsnl pwebsem2017-v4
Nathalie Aussenac-Gilles
 
Deep Learning for Information Retrieval
Roelof Pieters
 
The web of interlinked data and knowledge stripped
Sören Auer
 
From Semantics to Self-supervised Learning for Speech and Beyond (Opening Ke...
linshanleearchive
 
semantic web & natural language
Nurfadhlina Mohd Sharef
 
Getting Started with Knowledge Graphs
Peter Haase
 
Schema-agnositc queries over large-schema databases: a distributional semanti...
Andre Freitas
 
DODDLE-OWL: A Domain Ontology Construction Tool with OWL
Takeshi Morita
 
Multilingual qa
shakimov
 
Identifying Topics in Social Media Posts using DBpedia
Óscar Muñoz García
 
Understanding Natural Language Queries over Relational Databases
Ashis Kumar Chanda
 
Neural Text Embeddings for Information Retrieval (WSDM 2017)
Bhaskar Mitra
 
Deep Learning for Natural Language Processing: Word Embeddings
Roelof Pieters
 
Scalable Cross-lingual Document Similarity through Language-specific Concept ...
Carlos Badenes-Olmedo
 
Understanding Natural Language Queries over Relational Databases
Ashis Chanda
 
A Panorama of Natural Language Processing
Ted Xiao
 
Approach to leverage Websites to APIs through Semantics
Ioannis Stavrakantonakis
 
LSI latent (par HATOUM Saria et DONGO ESCALANTE Irvin Franco)
rchbeir
 
What you Can Make Out of Linked Data
Marco Fossati
 
ESWC 2014 Tutorial part 3
Miriam Fernandez
 
Aussenac semanticsnl pwebsem2017-v4
Nathalie Aussenac-Gilles
 
Ad

Recently uploaded (20)

PDF
Presentation - Vibe Coding The Future of Tech
yanuarsinggih1
 
PDF
How Startups Are Growing Faster with App Developers in Australia.pdf
India App Developer
 
PPTX
WooCommerce Workshop: Bring Your Laptop
Laura Hartwig
 
PDF
Complete JavaScript Notes: From Basics to Advanced Concepts.pdf
haydendavispro
 
PDF
Using FME to Develop Self-Service CAD Applications for a Major UK Police Force
Safe Software
 
PDF
Timothy Rottach - Ramp up on AI Use Cases, from Vector Search to AI Agents wi...
AWS Chicago
 
PDF
SWEBOK Guide and Software Services Engineering Education
Hironori Washizaki
 
PDF
July Patch Tuesday
Ivanti
 
PDF
CIFDAQ Market Insights for July 7th 2025
CIFDAQ
 
PPTX
OpenID AuthZEN - Analyst Briefing July 2025
David Brossard
 
PPTX
✨Unleashing Collaboration: Salesforce Channels & Community Power in Patna!✨
SanjeetMishra29
 
PDF
Log-Based Anomaly Detection: Enhancing System Reliability with Machine Learning
Mohammed BEKKOUCHE
 
PPTX
Webinar: Introduction to LF Energy EVerest
DanBrown980551
 
PDF
CIFDAQ Token Spotlight for 9th July 2025
CIFDAQ
 
PDF
Fl Studio 24.2.2 Build 4597 Crack for Windows Free Download 2025
faizk77g
 
PDF
LLMs.txt: Easily Control How AI Crawls Your Site
Keploy
 
PDF
Why Orbit Edge Tech is a Top Next JS Development Company in 2025
mahendraalaska08
 
PPT
Interview paper part 3, It is based on Interview Prep
SoumyadeepGhosh39
 
PDF
Chris Elwell Woburn, MA - Passionate About IT Innovation
Chris Elwell Woburn, MA
 
PDF
Learn Computer Forensics, Second Edition
AnuraShantha7
 
Presentation - Vibe Coding The Future of Tech
yanuarsinggih1
 
How Startups Are Growing Faster with App Developers in Australia.pdf
India App Developer
 
WooCommerce Workshop: Bring Your Laptop
Laura Hartwig
 
Complete JavaScript Notes: From Basics to Advanced Concepts.pdf
haydendavispro
 
Using FME to Develop Self-Service CAD Applications for a Major UK Police Force
Safe Software
 
Timothy Rottach - Ramp up on AI Use Cases, from Vector Search to AI Agents wi...
AWS Chicago
 
SWEBOK Guide and Software Services Engineering Education
Hironori Washizaki
 
July Patch Tuesday
Ivanti
 
CIFDAQ Market Insights for July 7th 2025
CIFDAQ
 
OpenID AuthZEN - Analyst Briefing July 2025
David Brossard
 
✨Unleashing Collaboration: Salesforce Channels & Community Power in Patna!✨
SanjeetMishra29
 
Log-Based Anomaly Detection: Enhancing System Reliability with Machine Learning
Mohammed BEKKOUCHE
 
Webinar: Introduction to LF Energy EVerest
DanBrown980551
 
CIFDAQ Token Spotlight for 9th July 2025
CIFDAQ
 
Fl Studio 24.2.2 Build 4597 Crack for Windows Free Download 2025
faizk77g
 
LLMs.txt: Easily Control How AI Crawls Your Site
Keploy
 
Why Orbit Edge Tech is a Top Next JS Development Company in 2025
mahendraalaska08
 
Interview paper part 3, It is based on Interview Prep
SoumyadeepGhosh39
 
Chris Elwell Woburn, MA - Passionate About IT Innovation
Chris Elwell Woburn, MA
 
Learn Computer Forensics, Second Edition
AnuraShantha7
 
Ad

Learning Multilingual Semantic Parsers for Question Answering over Linked Data - A comparison of neural and probabilistic graphical model architectures

  • 1. Learning Multilingual Semantic Parsers for 
 Question Answering over Linked Data A comparison of neural and probabilistic graphical model architectures PhD Dissertation Defense Talk March 2019 Sherzod Hakimov Semantic Computing Group, CITEC Bielefeld University
  • 2. Outline 1. Introduction 2. Motivation & Problem 3. Preliminaries 4. Dependency parse tree-based Semantic Parsing Approach 5. Evaluation of different architectural choices for “simple question” answering 6. Discussion 7. Conclusion 2
  • 7. 7 Give me the route to Jahnplatz What is Semantic Parsing? • mapping natural language sentence to a detailed meaning representation
  • 8. 8 Give me the route to Jahnplatz route($LOC, “Jahnplatz”) What is Semantic Parsing? • mapping natural language sentence to a detailed representation of meaning representation route(StartLocation, EndLocation)
  • 9. 9 Give me the route to Jahnplatz route($LOC, “Jahnplatz”) What is Semantic Parsing? • mapping natural language sentence to a detailed representation of meaning representation • meaning representation can be modelled using a formal language that
  • 10. 10 Give me the route to Jahnplatz route($LOC, “Jahnplatz”) What is Semantic Parsing? • mapping natural language sentence to a detailed representation of meaning representation • meaning representation can be modelled using a formal language, e.g. lambda calculus • an ontology with properties, classes, entities, etc. (route, create_calendar_event, set_alarm) • supports automated execution or reasoning
  • 11. 11 Give me the route to Jahnplatz Query Knowledge Base Answer Why do we need Semantic Parsers? route($LOC, “Jahnplatz”)
  • 12. 12 Give me the route to Jahnplatz Query Knowledge Base Answer Why do we need Semantic Parsers? route($LOC, “Jahnplatz”)
  • 13. Motivation • Building semantic parsers with application on Question Answering 13 Linked Data Cloud
  • 14. Motivation • Building semantic parsers with application on Question Answering • Building multilingual solutions that can be applied for multiple languages 14 Which German politicians were born in Bielefeld? Which metal has a liquid form? Welche deutschen Politiker wurden in Bielefeld geboren? Welches Metall hat eine flüssige Form? ¿Qué políticos alemanes nacieron en Bielefeld? ¿Qué metal tiene una forma líquida?
  • 15. Motivation • Building semantic parsers with application on Question Answering • Building multilingual solutions that can be extended for other languages • Comparison and evaluation of different model architectures 15
  • 16. Motivation • Building semantic parsers with application on Question Answering • Building multilingual solutions that can be extended for other languages • Comparison and evaluation of different model architectures • Highlight the challenges of building Question Answering systems 16
  • 17. • based on structured content from Wikipedia • more than 130 languages supported • 760 classes, 1105 object & 1622 data type properties • ca. 9 million resources 17
  • 19. dbr:Dan_Brown dbo:author Question Answering on RDF Data dbr:Inferno_(novel) 19
  • 20. dbr:Dan_Brown dbo:author Question Answering on RDF Data dbr:Inferno_(novel)Triple: dbr:Inferno_(novel) dbo:author dbr:Dan_Brown 20
  • 21. dbr:Dan_Brown dbo:author Question Answering on RDF Data dbr:Inferno_(novel) Dan Brown is the author of Inferno Triple: Natural Language: dbr:Inferno_(novel) dbo:author dbr:Dan_Brown 21
  • 22. dbr:Dan_Brown dbo:author Question Answering on RDF Data dbr:Inferno_(novel) Dan Brown is the author of Inferno Who is the author of Inferno?Natural Language: Question format dbr:Inferno_(novel) dbo:author dbr:Dan_BrownTriple: Natural Language: 22
  • 23. dbr:Dan_Brown dbo:author Question Answering on RDF Data dbr:Inferno_(novel) Dan Brown is the author of Inferno dbr:Inferno_(novel) dbo:author dbr:Dan_BrownTriple: Natural Language: Who is the author of Inferno?Natural Language: SPARQL Query: Question format SELECT ?x WHERE {dbr:Inferno_(novel) dbo:author ?x} 23
  • 24. dbr:Dan_Brown dbo:author Question Answering on RDF Data dbr:Inferno_(novel)Who is the author of Inferno? Semantic Parser SELECT ?x WHERE {dbr:Inferno_(novel) dbo:author ?x} 24
  • 25. Research Questions How to map natural language phrases into knowledge base entries for multiple languages? Which linguistic resources can be used? 25 dbr:Dan_Brown dbo:author Who is the author of Inferno? dbr:Inferno_(novel) Who wrote Inferno? Who is the writer of Inferno? SELECT ?x WHERE {dbr:Inferno_(novel) dbo:author ?x}
  • 26. Research Questions How to map natural language phrases into knowledge base entries for multiple languages? Which linguistic resources can be used? 26 dbr:Dan_Brown dbo:author Who is the author of Inferno? dbr:Inferno_(novel) Who wrote Inferno? Who is the writer of Inferno? SELECT ?x WHERE {dbr:Inferno_(novel) dbo:author ?x} Lexical Gap: write -> dbo:author
  • 27. Research Questions How to disambiguate URIs when multiple candidates are retrieved from mapping natural language tokens into knowledge base entries? 27 When was Inferno released? SELECT ?x WHERE {dbr:Inferno_(novel) dbo:releaseDate ?x} dbr:Inferno_(2016_film) dbr:Inferno_(novel)
  • 28. Research Questions How to use syntactic information of a natural language question together with semantic representations of entries in a knowledge base? 28 Who wrote Inferno? dbr:Dan_Brown dbo:author dbr:Inferno_(novel) SELECT ?x WHERE { dbr:Inferno_(novel) dbo:author ?x } wrote
 (VERB) Who
 (PRON) Inferno
 (PROPN) nsubj dobj
  • 29. Research Questions What are the advantages and the disadvantages of a multilingual QA system vs. a monolingual system built for each language? 29 Who is the author of Inferno? dbr:Dan_Brown dbo:author dbr:Inferno_(novel) Wer ist der Autor von Inferno? ¿Quién es el autor de Inferno? SELECT ?x WHERE { dbr:Inferno_(novel) dbo:author ?x }
  • 30. Research Questions What effort is required to adapt our QA pipelines to another language? 30 Who is the author of Inferno? dbr:Dan_Brown dbo:author dbr:Inferno_(novel) Qui est l'auteur de Inferno? Infernoning muallifi kim? SELECT ?x WHERE { dbr:Inferno_(novel) dbo:author ?x }
  • 32. Preliminaries • Logical Form - DUDES, formalism for specifying meaning representations for dependency tree structures 32
  • 33. Preliminaries • Logical Form - DUDES, formalism for specifying meaning representations for dependency tree structures • Semantic Composition - acquiring the meaning representations using the syntax of questions 33
  • 34. Logical Form 34 • DUDES - Dependency-based Underspecified Discourse Representation Structures (Cimiano et al [1]) [1] Cimiano, P., 2009, Flexible semantic composition with DUDES. In Proceedings of the Eighth International Conference on Computational Semantics (pp. 272-276). Association for Computational Linguistics.
  • 35. Logical Form 35 • DUDES - Dependency-based Underspecified Discourse Representation Structures (Cimiano et al [1]) • Formalism for specifying meaning representation • Flexible semantic composition w.r.t order of application • Build on semantic dependencies e.g. suitable for working with dependency-based syntactic analysis
 [1] Cimiano, P., 2009, Flexible semantic composition with DUDES. In Proceedings of the Eighth International Conference on Computational Semantics (pp. 272-276). Association for Computational Linguistics.
  • 36. DUDES v : is the main variable vs : the projection variables l : is the label of the main DRS drs : is a DRS (Discourse Representation Structure) slots : is a set of semantic dependencies 36
  • 37. Semantic Composition with DUDES Who created Wikipedia? Input: a natural language question and its dependency parse tree 37
  • 38. Semantic Composition with DUDES Who created Wikipedia? Input: a natural language question and its dependency parse tree dbr:Wikipedia dbo:author ?x Output: a meaning representation based on certain domain 38
  • 39. Semantic Composition with DUDES Each node gets a pair of assignments: DUDES Type + Knowledge base ID Oracle 39
  • 40. Semantic Composition with DUDES - Bottom Up 40
  • 41. Semantic Composition with DUDES - Bottom Up 41 1 2
  • 42. Semantic Composition with DUDES - Bottom Up 42 2
  • 43. Semantic Composition with DUDES - Bottom Up 43 2
  • 44. Semantic Composition with DUDES - Bottom Up 44 2
  • 45. Semantic Composition with DUDES - Bottom Up 45 x
  • 47. Logical Form into SPARQL 47
  • 49. Dependency parse tree-based Semantic Parsing Approach • multilingual semantic parsing approach: English, German & Spanish [1] 49 [1] Hakimov S, Jebbara S, Cimiano P. AMUSE: Multilingual Semantic Parsing for Question Answering over Linked Data. In Proceedings of the 16th International Semantic Web Conference (ISWC), 2017
  • 50. Dependency parse tree-based Semantic Parsing Approach • multilingual semantic parsing approach: English, German & Spanish [1] • uses language-independent dependency parse trees from Universal Dependencies 50 [1] Hakimov S, Jebbara S, Cimiano P. AMUSE: Multilingual Semantic Parsing for Question Answering over Linked Data. In Proceedings of the 16th International Semantic Web Conference (ISWC), 2017
  • 51. Dependency parse tree-based Semantic Parsing Approach • multilingual semantic parsing approach: English, German & Spanish [1] • uses language-independent dependency parse trees from Universal Dependencies • combines different types of lexical information: DBpedia Ontology labels, the M-ATOLL[2] lexicon & word embeddings 51 [1] Hakimov S, Jebbara S, Cimiano P. “AMUSE: Multilingual Semantic Parsing for Question Answering over Linked Data”. ISWC 2017 [2] Walter S, Unger C, and Cimiano P. “M-ATOLL: A Framework for the Lexicalization of Ontologies in Multiple Languages”. ISWC 2014 [3] Hakimov S, Walter S, Unger C, and Cimiano P. “Applying semantic parsing to question answering over linked data: Addressing the lexical gap”. NLDB 2015
  • 52. Dependency parse tree-based Semantic Parsing Approach Oracle SELECT ?x WHERE { dbr:Wikipedia dbo:author ?x. } Semantic Composition 52
  • 53. Dependency parse tree-based Semantic Parsing Approach Oracle SELECT ?x WHERE { dbr:Wikipedia dbo:author ?x. } Semantic Composition Structure Learning 53
  • 54. Dependency parse tree-based Semantic Parsing Approach • inference • factor graph model 54
  • 55. Inference • Metropolis-Hastings: exploring huge search space (ca. 10 million resources, 2000 properties) • Linking to Knowledge Base (L2KB) •objective : compare set of URIs to the expected set of URIs • Query Construction (QC) •objective : compare the constructed query to the expected query 55 Input: initial state
  • 56. L2KB Sampling Explore the edges and assign Knowledge Base IDs based on lemmas of nodes Inverted index: Ontology labels, lexicon from M-ATOLL & word embeddings 56
  • 57. L2KB Sampling Explore the edges and assign Knowledge Base IDs based on lemmas of nodes Check the triple pattern- ?x dbo:author dbr:Wikipedia : Slot 2, dbr:Wikipedia dbo:author ?x : Slot1 Inverted index: Ontology labels, lexicon from M-ATOLL & word embeddings 57
  • 58. 58
  • 59. 59
  • 60. 60
  • 61. QC Sampling Assign DUDES with Return Variable and KB ID to nodes and assign remaining slots 61
  • 63. Model Representation 63 Observed variables: dependency parse tree Hidden variables: KB IDs, slot, DUDE types
  • 64. Model Representation 64 Observed variables: dependency parse tree Hidden variables: KB IDs, slot, DUDE types
  • 65. Model Representation 65 Observed variables: dependency parse tree Hidden variables: KB IDs, slot, DUDE types Child_Node [POS: PROPN, DUDES: Resource] Parent_Node [POS: VERB, DUDES:Property] Sample Feature
  • 66. Evaluation Dataset: Question Answering over Linked Data (QALD), 6th challenge English, German, Spanish, Italian, French, Dutch, Romanian, Farsi 350 for train, 100 for test Unger, Christina, Axel-Cyrille Ngonga Ngomo, and Elena Cabrio (2016). “6th open challenge on question answering over linked data (qald-6)”. In: Semantic Web Evaluation Challenge. 66
  • 67. Evaluation DBP: lexicon from DBpedia Ontology & WordNet
 M-ATOLL: lexicon induced by the M-ATOLL (Walter et al. 2014)
 Embed: lexicon added using pre-trained word embeddings (Mikolov et al. 2013) Walter, Sebastian, Christina Unger, and Philipp Cimiano. “M-ATOLL: A Framework for the Lexicalization of Ontologies in Multiple Languages”. ISWC 2014 Mikolov, Tomas et al. “Distributed representations of words and phrases and their compositionality”. NIPS 2013 67
  • 68. Evaluation DBP: lexicon from DBpedia Ontology & WordNet
 M-ATOLL: lexicon induced by the M-ATOLL (Walter et al. 2014)
 Embed: lexicon added using pre-trained word embeddings (Mikolov et al. 2013)
 Dict: manually defined lexicon Walter, Sebastian, Christina Unger, and Philipp Cimiano. “M-ATOLL: A Framework for the Lexicalization of Ontologies in Multiple Languages”. ISWC 2014 Mikolov, Tomas et al. “Distributed representations of words and phrases and their compositionality”. NIPS 2013 68
  • 69. Evaluation DBP: lexicon from DBpedia Ontology & WordNet
 M-ATOLL: lexicon induced by the M-ATOLL (Walter et al. 2014)
 Embed: lexicon added using pre-trained word embeddings (Mikolov et al. 2013)
 Dict: manually defined lexicon Walter, Sebastian, Christina Unger, and Philipp Cimiano. “M-ATOLL: A Framework for the Lexicalization of Ontologies in Multiple Languages”. ISWC 2014 Mikolov, Tomas et al. “Distributed representations of words and phrases and their compositionality”. NIPS 2013 69
  • 71. Evaluation of different architectural choices for “simple question” answering 71
  • 72. Outline • SimpleQuestions dataset, 74k samples, Freebase data • Question: “Who wrote Mildred Pierced?” • Fact: mildred_pierced, book.written_work.author, stuart_kaminsky • Answer: mildred_pierced, book.written_work.author, ?x • Systematic comparison of different model architectures 72 Hakimov S, Jebbara S, Cimiano P. “Evaluating Architectural Choices for Deep Learning Approaches for Question Answering over Knowledge Bases”. ICSC 2019
  • 73. Named Entity Recognition • Used by all models to predict the entity span • Character & word embeddings • Trained using weak supervision: inference is correct if the expected entity has been found 73
  • 74. Candidate Retrieval Candidate pairs: subject and predicate 74 Surface form frequencies
  • 75. Model1: BiLSTM-Softmax 75 Model2: BiLSTM-KB Model3: BiLSTM-Binary Model4: Fasttext [1] Architectures [1] A. Joulin, E. Grave, P. Bojanowski, T. Mikolov, "Bag of Tricks for Efficient Text Classification", 2016, arxiv.org
  • 76. Model1: BiLSTM-Softmax 76 Model2: BiLSTM-KB Model3: BiLSTM-Binary Model4: Fasttext [1] Architectures [1] A. Joulin, E. Grave, P. Bojanowski, T. Mikolov, "Bag of Tricks for Efficient Text Classification", 2016, arxiv.org
  • 77. Evaluation 77 Named Entity Linking Named Entity Recognition Predicate & Answer Prediction
  • 78. Evaluation 78 Named Entity Linking Named Entity Recognition 0,82 Predicate & Answer Prediction
  • 79. Evaluation 79 Named Entity Linking Named Entity Recognition 0,82 Predicate & Answer Prediction
  • 80. Evaluation 80 Named Entity Linking Named Entity Recognition 0,82 Predicate & Answer Prediction
  • 82. Discussion • Manual Effort • Syntax and Semantics • Multilinguality • Cross-domain Transferability • Training Data Size and Search Space 82
  • 83. Discussion 83 Systems Manual Effort Syntax & Semantics Multilinguality Cross-domain transferability Training Data & 
 Number of KB IDs CCG-based
 (Chapter 6) CCG combination rules
 manual lexicon • learned in tandem • CCG for syntax • lambda calculus for semantics manual effort is required manual effort is required 600 training instances 750 entities Dependency- based
 (Chapter 7) feature templates syntax is given DUDES as a formalism an adaptable solution • a dependency parser required. • e.g. biomedical domain 300 training instances <= 10 mil. entities
 >= 2000 predicates BiLSTM-Softmax
 (Chapter 8) - • word & char embed. for lexical & contextual info • semantics is limited to a single predicate and a subject entity an adaptable solution only word & char embed. an adaptable solution only word & char embed. >= 75K instances
 <= 2 mil. entities
  • 85. Research Questions 85 • Ontology lexicalisations, e.g. M-ATOLL (Walter et al. 2014) • Ontology labels, e.g. DBpedia labels • Dictionaries • WordNet synsets • Induced from contextual embeddings of words RQ1: How to map natural language phrases into knowledge base entries for multiple languages? Which linguistic resources can be used?
  • 86. Research Questions 86 • Supervised models with objective for disambiguation • CCG-based model • uses lexical and syntactic information as features • Dependency tree-based model • syntactic dependency between words, lexical similarity, ontology restrictions • Neural network-based model • ranking objective of predicates RQ2: How to disambiguate URIs when multiple candidates are retrieved from mapping natural language tokens into knowledge base entries?
  • 87. Research Questions 87 • Semantic Parsing • bottom-up composition • CCG-based model • learns the syntax and semantics together • Dependency tree-based model • learns composing semantics based on dependency trees RQ3: How to use syntactic information of a natural language question together with semantic representations of entries in a knowledge base?
  • 88. Research Questions 88 RQ4: What are the advantages and the disadvantages of a multilingual QA system vs. a monolingual system built for each language? • Advantages • Multilingual: broader coverage • Monolingual: higher performance, e.g. Xser (Xu et al. 2014) 0.7 F1 on QALD-4 • Disadvantages • Multilingual: lower performance, e.g. AMUSE 0.3 F1 on QALD-6 • Monolingual: need expertise, e.g. CCG rules, lexicon
  • 89. Research Questions 89 • CCG-based model • grammar rules, manually defined lexicon • language-specific • Dependency parse tree-based model • dependency parse tree generator • lexicon • Neural network-based model • depends on the training data RQ5: What effort is required to adapt our QA pipelines to another language?
  • 90. Conclusion • Address the lexical gap for QA systems • Incorporate ontology lexicalizations to reduce the lexical gap • Use Universal Dependencies to build language-independent QA pipeline • Multilingual semantic parsing for Question Answering • Evaluate different QA models under the certain conditions • Highlight importance of building blocks of a pipeline for a fair comparison 90
  • 92. GENLEX Barack Obama is married to Michelle Obama [1] Zettlemoyer, Luke S and Michael Collins (2005). “Learning to Map Sentences to Logical Form : Structured Classification with Probabilistic Categorial Grammars”. In: 21st Conference on Uncertainty in Artificial Intelligence [2] Hakimov, Sherzod et al. (2015). “Applying semantic parsing to question answering over linked data: Addressing the lexical gap”. In: International Conference on Applications of Natural Language to Information Systems 92
  • 93. CCG & GENLEX Rules 93
  • 94. Semantic Composition Barack Obama is married to Michelle Obama 94
  • 96. Lexicon During sampling, compute cosine similarity of words into Ontology labels of properties
 Vectors for multiple words are summed, e.g. V(population) + V(total) 96
  • 98. Dataset - GeoQuery 600 train, 280 test instances 98
  • 101. Dataset - SimpleQuestions Train: 75,910 Validation: 10,845 Test: 21,687 Total: 108,442 101
  • 102. Dataset Complexity Lexical Overlap, tokens exist in both 102
  • 103. Semantic Composition •recursively computing the meaning of each node from the meanings of its child nodes •build the meaning representation bottom-up ComposeSemantics(dependency-parse-tree) If parse-tree is a terminal node (word) then return an atomic lexical meaning for the word. Else For each child, subtreei, of parse-tree Create its MR by calling ComposeSemantics(subtreei) Return an MR by properly combining the resulting MRs for its children into an MR for the overall parse-tree. 103
  • 108. Model Representation 108 Observed variables: dependency parse tree Hidden variables: KB IDs, slot, DUDE types • States can be ranked by • objective score : compare to ground truth • model score: computed using feature weights • Training procedures • switch between model & objective score after every iteration
  • 111. Model1: BiLSTM-Softmax • Softmax layer that predicts predicates seen during training • Encoding layer: word & character • BiLSTM: two LSTM layers (backward, forward) 111
  • 112. Model2: BiLSTM-KB • Learn embedding of predicates in KB • Encoding layer: word & character • BiLSTM: two LSTM layers (backward, forward) • Output layer computes cosine similarity to all predicates and chooses the closest 112
  • 113. Model3: BiLSTM-Binary • Encoding layer: encodes input question with word & character embeddings • Encoding layer: encodes input predicate with word & character embeddings • Output layer: binary decision 113
  • 114. Model4: Fasttext • Document classification tool developed by Facebook* • Uses word & character embeddings • Softmax layer that predicts the expected predicate 114 * http://fasttext.cc
  • 116. 116 Generative Models -> computing joint probability distribution on p(y|x)
 HMM -> y_t depends on y_t-1 and x_t how output label y_t generates input vector x Discriminative Models -> computing conditional probability distribution over inputs x and outputs y
 CRF -> doesn’t have any limitation like that how feature vector x gets assignment y_t
  • 118. Manual Effort • CCG-based model • define CCG grammar rules, hand-crafted lexicon for domain independent phrases • Dependency parse tree-based model • feature functions • Neural network-based model (BiLSTM-Softmax) • not required 118
  • 119. Syntax and Semantics • CCG-based model • syntax and semantics is learned in tandem • CCG for syntax and the lambda calculus for semantics • syntax guides the semantics of the sentences • Dependency parse tree-based model • syntax is given and the semantics is learned • DUDES as a formalism for semantics, syntax is based on dependency trees from Universal Dependencies • Neural network-based model (BiLSTM-Softmax) • syntactic information is learned, e.g. word and character embeddings provide contextual information • semantics is based on a single subject and the predicate, simpler task 119
  • 120. Multilinguality • CCG-based model • CCG grammar rules needs to be extended • Dependency parse tree-based model • a multilingual solution • Neural network-based model (BiLSTM-Softmax) • can be adapted to other languages, e.g. word & characters as features 120
  • 121. Cross-domain Transferability • CCG-based model • manual effort is required: CCG rules, lexicon • Dependency parse tree-based model • dependency parse trees e.g. biomedical domain • Neural network-based model (BiLSTM-Softmax) • can be easily adapted 121
  • 122. Training Data Size and Search Space • CCG-based model • Dependency parse tree-based model • Neural network-based model (BiLSTM-Softmax) • heavily depends on the data 122