Ontology-Backed Chatbots

A Project Report

submitted by

SUHAS PAI

in partial fulfilment of the requirements

for the award of the degree of

BACHELOR OF TECHNOLOGY &

MASTER OF TECHNOLOGY

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING

INDIAN INSTITUTE OF TECHNOLOGY, MADRAS.
May 2022
 THESIS CERTIFICATE

This is to certify that the thesis entitled Ontology-Backed Chatbots, submitted

by Suhas Pai (CS17B116), to the Indian Institute of Technology, Madras, for the

award of the degree of Bachelors of Technology and Master of Technology, is a

bona fide record of the research work carried out by him under my supervision.

The contents of this thesis, in full or in parts, have not been submitted to any other

Institute or University for the award of any degree or diploma.

Dr. P Sreenivasa Kumar

Research Guide
Professor
Dept. of Computer Science and Engineering
IIT-Madras, 600 036

Place: Chennai

Date: May 2022

 ACKNOWLEDGEMENTS

I was introduced to the exciting field of ontologies in one of the elective courses I

took, and the entire journey has been a whirlwind since then. Firstly, I would like

to thank Prof. P Sreenivasa Kumar for his unwavering faith in me and guidance

this past year. He has been a constant source of support and motivation and has

pushed me to do better work. I would also like to thank my faculty advisor, Prof.

Mitesh Khapra, for his help and support during my time here.

I want to thank the developer of VOnDA, Prof. Bernd Kiefer, who spent

countless hours helping me understand the system. The project would not be

what it is without his guidance and patience. I would also like to thank Suresh

Saharan for his valuable suggestions during the course of the project.

I am incredibly thankful to my family, who’ve supported me every step of the

way and have had unwavering faith in my abilities, even when I did not.

I would not have made it through the five years at IIT Madras without my

friends. I express gratitude and sincere thanks to the three-member groups that I

was a part of, Gridlock, Kekw and Crypto.

i
 ABSTRACT

KEYWORDS: Ontologies ; Chatbots; Dialogue Acts.

Chatbots have been developed and deployed for a very long time now. However,

it is often the case that they are unable to maintain natural language conversations

for a considerable amount of time. Ontologies can be used as both a database and

as a way to guide these conversations. This thesis is a preliminary study on the

available systems for building chatbots and how they can be extended to fit the

above use case. It outlines the development of an ontology-backed chatbot built

using the VOnDA core system. The VOnDA system uses dialogue speech acts to

guide the conversation, hence working as a meta-ontology. It has a special reasoner

that allows for ontologies to be stored in the form of n-tuples instead of triples.

This helps in maintaining dialogue history and resolving contexts. One of the main

parts of any chatbot is the NLU module. We have tried two approaches for this, a

grammar-based parser and a deep learning-backed module. The grammar-based

parser allows only a small input space, while the deep learning-backed module

helps maintain a more natural conversation, which was chosen in the final chatbot.

ii
 TABLE OF CONTENTS

ACKNOWLEDGEMENTS i

ABSTRACT ii

LIST OF TABLES v

LIST OF FIGURES vii

ABBREVIATIONS viii

1 INTRODUCTION 1
1.1 Ontologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.1 Description Logics . . . . . . . . . . . . . . . . . . . . . . . 2
1.1.2 Ontology Storage Formats . . . . . . . . . . . . . . . . . . . 3
1.1.3 Querying Ontologies . . . . . . . . . . . . . . . . . . . . . . 4
1.2 Chatbots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2.1 Dialogue Management System . . . . . . . . . . . . . . . . 6
1.2.2 NLU Module . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2.3 NLG Module . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.2.4 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . 8

2 Related Work 10
2.1 OntoVPA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.1.2 Example Workflow . . . . . . . . . . . . . . . . . . . . . . . 12
2.1.3 Critical Review . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2 Ontology Based Conversation System . . . . . . . . . . . . . . . . 13
2.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

iii
 2.2.2 Conversation Patterns . . . . . . . . . . . . . . . . . . . . . 13
2.2.3 Generating Training Examples . . . . . . . . . . . . . . . . 13
2.2.4 Dialogue Management . . . . . . . . . . . . . . . . . . . . . 14
2.2.5 Critical Review . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.2.6 Final System . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3 Ontology-Based Chatbot 16
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.1.1 RUDI Language . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.1.2 RUDI Example Code Walkthrough . . . . . . . . . . . . . . 17
3.1.3 Context History . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.1.4 Dynamic Interpretation Theory(DIT)++ Dialogue Act . . 18
3.1.5 Domain Ontology . . . . . . . . . . . . . . . . . . . . . . . 20
3.1.6 NLU and NLG Modules . . . . . . . . . . . . . . . . . . . . 21
3.1.7 Structure and Workflow . . . . . . . . . . . . . . . . . . . . 22
3.2 Work Done . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.2.1 Ontologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.2.2 VOnDA Core Module . . . . . . . . . . . . . . . . . . . . . 27
3.2.3 Anaphora Resolution . . . . . . . . . . . . . . . . . . . . . . 29
3.2.4 SRGS Grammar Module . . . . . . . . . . . . . . . . . . . . 33
3.2.5 Rasa NLU module . . . . . . . . . . . . . . . . . . . . . . . 35
3.2.6 CPlanner NLG Module . . . . . . . . . . . . . . . . . . . . 41
3.2.7 Final System . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.3 Domain invariance - Developing a new system . . . . . . . . . . . 43

4 Conclusions 45
4.1 Results and Conclusions . . . . . . . . . . . . . . . . . . . . . . . . 45
4.1.1 User interaction . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.1.2 Chatbot Evaluation . . . . . . . . . . . . . . . . . . . . . . . 48
4.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

A Example RUDI rules 51

 LIST OF TABLES

2.1 Default templates for the conversation Quamar et al. [2020] . . . . 14

3.1 Object Properties in the domain ontology . . . . . . . . . . . . . . 25

3.2 Intents modelled in the chatbot . . . . . . . . . . . . . . . . . . . . 38

v
 LIST OF FIGURES

1.1 RDF Triple . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2 Data formatted using the RDF/XML syntax . . . . . . . . . . . . . 3
1.3 Sample SPARQL Query . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 A typical Dialogue Management System . . . . . . . . . . . . . . . 6

2.1 A part of the OntoVPA domain-generic ontology Wessel et al. [2019] 11

3.1 Sample Rudi file Kiefer et al. [2019] . . . . . . . . . . . . . . . . . . 17

3.2 DIT++ Dialogue Act ontology hierarchical structure Bunt . . . . . 19
3.3 Sample n-tuple domain ontology, zero timestamped . . . . . . . . 19
3.4 VOnDA objects can be manipulated as per requirements . . . . . 21
3.5 Querying the HFC database . . . . . . . . . . . . . . . . . . . . . . 21
3.6 VOnDA system architecture Kiefer et al. [2019] . . . . . . . . . . . 23
3.7 Graphical representation of the domain ontology . . . . . . . . . . 24
3.8 Rules for small talk conversation . . . . . . . . . . . . . . . . . . . 28
3.9 Java Function that queries the HFC database . . . . . . . . . . . . 29
3.10 Example conversation . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.11 The department name gets resolved in this conversation . . . . . 30
3.12 VOnDA rule involving Anaphora Resolution . . . . . . . . . . . . 31
3.13 Example conversation involving a list being resolved . . . . . . . 32
3.14 Typical SRGS Grammar rule . . . . . . . . . . . . . . . . . . . . . . 33
3.15 Example of an intent with it’s training examples . . . . . . . . . . 36
3.16 Synonyms can be provided for entities . . . . . . . . . . . . . . . . 36
3.17 Lookup tables help in entity extraction . . . . . . . . . . . . . . . . 36
3.18 The Rasa NLU configuration pipeline . . . . . . . . . . . . . . . . 39
3.19 Typical Rasa NLU output . . . . . . . . . . . . . . . . . . . . . . . 39
3.20 Typical CPlanner NLG rule . . . . . . . . . . . . . . . . . . . . . . 41

vi
3.21 Final System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

4.1 How easy was it to use the system on a scale from 1 (very hard) to
5 (very easy)? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.2 How natural was the conversation on a scale from 1 (very unnatural)
to 5 (very natural)? . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.3 How well did the bot understand you on a scale from 1 (very bad)
to 5 (very good)? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

vii
 ABBREVIATIONS

IITM Indian Institute of Technology, Madras

RTFM Read the Fine Manual

DL Description Logic

HOD Head of Department

RDF Resource Description Framework

OWL Web Ontology Language

GPT Generative Pre-trained Transformer

DMS Dialogue Management System

NLU Natural Language Understanding

NLG Natural Language Generation

SPARQL SPARQL Protocol and RDF Query Language

viii
 CHAPTER 1

INTRODUCTION

1.1 Ontologies

Machines can process large amounts of data much faster than a human brain.

However, the human brain is excellent at performing reasoning tasks on data,

which machines find hard to do. This has motivated the use and development of

ontologies.

An ontology is essentially a representation consisting of formal naming and

definition of various categories, relationships, and properties of data and entities

in a particular domain.

Typical components of ontologies include - Individuals, Classes, Attributes,

Relations, Function terms, Restrictions, Rules, Axioms, and Events.

Classes(aka concepts) are essentially groups or collections of objects, usually

following certain distinct properties, that allow objects to be classified as a member

of the class. For example, in the University domain, a class Professor could be

defined as one who teaches a course.

Individuals are ground instances of a concept. For example, in the university

domain, ”Prof.Ramamurthy” can be an instance of the concept ”Professor.”

Relations describe the interaction between various concepts and often help

form a hierarchy amongst them. For example, in the university domain, ”is senior
to” can be a relation amongst Professors that describes their seniority compared to

one another.

Axioms are assertions that provide a logical base for ontologies. For example,

a rule could state that all Professors have to be under the age of 65.

1.1.1 Description Logics

One of the main ways to describe ontologies is to use Description Logics(DL).

It provides the basic formalism for most ontology formats. Different DLs have

varied levels of expressive power. DLs are made up of two distinct parts, the

Terminological Box (TBox) and the Assertional Box (ABox). The TBox consists

of rules and statements that describe relations between various concepts. On the

other hand, the ABox is used to assert that individuals are members of certain

concepts, and also provide their specific relationships with other individuals in

the domain.

For example, a statement in the TBox of a University domain might be of

the form ”Every HOD has to be a Professor.” Similarly, the ABox in the same

domain will consist of statements of the form ”Raman is a HOD.” The advantage

of ontologies is the fact that reasoners can be run on them. A reasoner here would

therefore conclude that ”Raman is a Professor” as a result of the two statements.

Another type of statement that ABoxs assert is the relationship assertions between

two individuals. Consider a relation defining a research supervisor. Then, an

assertion in the ABox will be of the form ”Raman is the research supervisor of

Sumanth.”

2
 Figure 1.1: RDF Triple

Figure 1.2: Data formatted using the RDF/XML syntax

1.1.2 Ontology Storage Formats

Ontologies are stored mainly in the Resource Description Framework (RDF) format

or the Web Ontology Language (OWL) format. These are structured data formats

that store information as data triples. One of the main reasons to use these formats

is to take advantage of the uniformity and standardization it provides. Also,

machines find it easier to work with such standard data, and reasoners are able to

directly take these triples to draw inferences. Figure 1.1 shows how every assertion

can be viewed as an edge in a graph. A collection of these statements can thought

to be representing a directed graph. Figure 1.2 shows how data can be succintly

expressed using RDF.

3
 Figure 1.3: Sample SPARQL Query

1.1.3 Querying Ontologies

SPARQL Pérez et al. [2006] is a semantic query language for retriving and ma-

nipulating data stored in the RDF triple format. It consists of features that allow

querying triple patters, disjunctions, conjunctions, and optional patterns.

In Figure 1.3, the query, written in the Terse RDF Triple Language(Turtle) syntax,

extracts all state capital cities, along with the respective states. All pairs of RDF

objects in the domain that satisfy the WHERE clauses are returned successfully.

1.2 Chatbots

A chatbot is a software application used to conduct conversation online, usually

via text, replacing the need to provide direct contact with a human agent.

They are designed and implemented in a way that would emulate a human as

a conversational partner and require large amounts of effort, tuning, and testing,

to be able to pass the industry standard Turing test adequately, an accomplishment

that hasn’t been achieved so far.

Chatbots are prevalent in almost every domain and usually provide users with

answers to various frequently asked questions. Some chatbots can be incredibly

4
complex, for example, those powered by the famed GPT-3 Brown et al. [2020], an

autoregressive language model with over 100 billion parameters. However, most

chatbots usually end up performing keyword parsing and generating responses

by just querying databases for frequently used phrases. One of the most common

problems with chatbots is their inability to sustain a long conversation and their

ineptitude at being able to infer information correctly.

Chatbots involve a flow of dialogue between two or more agents. It is important

to understand the various nuances that exist in a natural language conversation.

A conversation space consists of three building blocks.

• Intents: Intents express the purpose or goal of the user utterance. Usually,
machine learning systems are used to identify intents in the sentence. Train-
ing examples can be auto-generated and labelled to the appropriate intent.
These intents depend on the domain itself, and the possible questions that
may be asked by the user.
For example : If the domain is medicine, then an intent might be of the form
– Find the side-effects of medicine X.
– Find all the substitutes of medicine X.

• Entities: Real-world objects relevant to a particular domain. If the domain

is medicine, the names of medicines, side effects, etc., can be considered
as entities. These need to be provided as a set of predefined entities with
examples and domain-specific synonyms (the domain vocabulary) relevant
within the context of the application.

• Dialogue: Structural representation of the flow of conversation and is used

to specify all possible conversation interactions that the conversation agent is
required to support. There are different ways of representing such a structure,
e.g., a tree, a state machine, etc.

Keeping these building blocks in mind, there are three main modules that

form the backbone of every conventional chatbot, namely, Dialogue Management

System(DMS), Natural Language Understanding(NLU) module, and the Natural

Language Generation(NLG) module.

5
 Figure 1.4: A typical Dialogue Management System

1.2.1 Dialogue Management System

A dialogue management system in a chatbot is responsible for maintaining the

state and flow of the conversation.

As shown in figure 1.4, the Dialogue Management System(DMS) works closely

in co-ordination with the Natural Language Understanding (NLU) and Natural

Language Generation (NLG) modules.

The input to the dialogue manager is usually some truncated version of the

natural language input. That is, the NLU captures some intents in the input

sentences, and this is passed on to the dialogue manager. Consider a ticket booking

system - If the input sentence is “Book a flight from India to the USA on 21st June

2022”, then the input to the dialog manager may look something like “FLIGHT(to

= USA, from = India, data = 21/06/2022)”. The dialog manager’s task is to take this

input, and update various variables so that the next part of the conversation can

be effectively carried out. It usually also sends a similar frame to the NLG module.

For example, in this case, the dialogue manager can send a confirmation frame of

6
the form - Confirm(FlightBooking) to the NLG module, which then generates the

natural language version of this.

There are mainly three types of dialogue management systems, categorized

based on the level of control the user has on the conversation Lee et al. [2010] :

• System-driven: In system-driven dialogue management, the system usually

asks clarifying questions in order to fill up slots for the user queries ap-
propriately. After all the slots are filled, the system queries the underlying
database to provide an appropriate response. The responses generated are
usually single-word or certain phrases that appropriately answer the user’s
question. A big advantage of this type of dialogue control is the fact that the
user’s conversation space is limited. Hence, a small vocabulary and gram-
mar will suffice for building the system. However, a downside to these types
of systems is that the conversation might seem extremely robotic.

• User-driven: In a user-driven system, the user has the freedom to direct

the conversation as he wants, with the system asking clarifying questions
in case some intents aren’t clear. The advantage of allowing this is that the
conversation feels more natural, and there is no restriction on the natural
language used. However, developing user-driven dialogue management
systems often requires the use of advanced NLU modules that can recognize
a large vocabulary and grammar and also requires complex dialogue flows
to be implemented.

• Mixed initiative dialogue - In a mixed initiative dialogue, the system is sup-

posed to control major portions of the dialogue, but it has enough flexibility
to allow the end-user to change the track of the dialogue/ switch the topic.
Therefore, mixed-initiative systems involve more complex mechanisms to
handle features like turn-taking.

1.2.2 NLU Module

NLU is a sub-topic of Natural Language Processing (NLP) that deals with machines

understanding natural language text. Regardless of the language being handled,

NLU modules try to understand the structure, semantics, discourse, and intents of

the input messages to be able to provide an appropriate response. NLU modules

usually consist of classifiers that aim to handle the above-mentioned tasks.

7
 Most classifiers use deep learning methods that require large number of training

examples. However, other types of statistical methods are used as well. Most

popular classifiers utilize:

• Rules based pattern matching

• Machine learning classification algorithms such as decision trees, naı̈ve Bayes,
and logistic regression.
• Deep learning such as artificial neural networks

One of the main tasks of classifiers is to identify intents. For example, if a

user tells the chatbot, ”I want to book a train from Bangalore to Chennai,” then

the intent classifier should ideally recognize the ”book train” as the intent, with

Bangalore and Chennai as the slots. This classification is usually passed on to the

dialogue management module so that various states and variables can be updated.

1.2.3 NLG Module

NLG is a software process that produces natural language output. NLG modules

can be viewed to be doing the opposite of NLU models - the latter mainly deals with

understanding natural language and converting it into machine-friendly formats,

while the former deals with converting machine-friendly formats into natural

language. NLG modules range from either simple template-based models, to

complex systems that have a deep understanding of the natural language grammar.

Statistical methods like deep learning can also be used in these modules.

1.2.4 Problem Statement

Implementation-wise, there are two types of chatbots prevalent in the world. The

first kind is one that’s built using simple rules, usually consisting of a set of

8
small answers, and that depends on primitive keyword matching to make sense

of user inputs. The other type of chatbots are ones that are backed by NLP and

other natural language technologies. These take longer to create but are more

sophisticated in nature. They use statistical methods like deep learning to evolve

and grow over time.

When it comes to chatbots backed by ontologies - we expect them to perform

better than conventional bots because ontologies are a more evolved version of

databases. The inferences that the reasoners make can be used to answer questions

intelligently and provide more insights than a regular database.

We’re interested in developing/implementing a framework for a chatbot in

which both the domain-specific and the domain-independent parts of the conver-

sation are guided by ontologies and to see how it compares to normal chatbots.

The main goal is to see if it’s possible to hold long conversations in a particular

domain. In conventional chatbots, either no history of the conversation is main-

tained, or, in the case of statistical approaches, long conversations are not possible

due to the limitations of technology.

The domain that has been chosen for the chatbot is the University FAQ domain

- information about various programs, selection criteria, institute facilities, etc.

Chapter 2 discusses the related work that has been done in the field of ontology-

backed chatbots. Chapter 3 discusses the novel method that has been implemented

in our chatbot. Chapter 4 goes on to discuss the results and findings of the work,

and also the scope for improvements in the future.

9
 CHAPTER 2

Related Work

This chapter details the various works related to ontology-backed chatbots, looking

at the current state-of-the-art systems and various other ideas to draw inspiration

from.

2.1 OntoVPA

Despite the paper having technical details only at a surface level, it is important

to look the the OntoVPA Wessel et al. [2019] system, since it is one of the few DMs

that employ ontologies to drive the conversation itself.

2.1.1 Introduction

OntoVPA (Ontology-based Virtual Personal Assistant) is a DMS that employs

ontologies and ontology-based rules to implement dialogue and domain repre-

sentation, and also other dialogue system core capabilities like state tracking, slot

filling, anaphora resolution, and response computation.

A meta-ontology is used based on the classes and acts involved in the speech

act theory Searle [1969]. During run time, these classes and relation types are

instantiated based on the conversation, and ontology-based rules over this meta-

ontology guide the dynamics of the conversation. Figure 2.1 shows the various

classes in the meta-ontology.

Figure 2.1: A part of the OntoVPA domain-generic ontology Wessel et al. [2019]

11
 In OntoVPA models, ontologies facilitate the following:

• Domain Model Representation: This is the domain-specific ontology, con-

sisting of both the TBox and the ABox.

• Dialogue Ontology: The meta-ontology that guides the domain-agnostic

parts of the conversation, which is built based on speech act theory Searle
[1969]. The TBox consists of various dialogue act classes, and other speech
rules. The ABox gets instantiated during runtime, and forms the dialogue
representation.

• Reasoning to Compute VPA’s Responses: Ontology-based rules are used to

compute the VPA’s responses.

2.1.2 Example Workflow

• User asks a question : What are the courses in the CS program

• Intent classifier identifies the intent in the user query. E.g. : FindPOIIntent.
Here Point of Interest(POI) is CS Program, and in the dynamic dialogue Abox,
it gets filled as the dialogue entity. Similarly, courses can be recognized as
something related to a CS program, as specified in the domain ontology.

• Once an intent is completely specified, the domain ontology is queried.

• SPAQRL rule engine is used to create the VPA responses

2.1.3 Critical Review

While the results shown by OntoVPA are definitely exciting, it is a product built

for commercial use, and none of the technical details are available, and the code is

not open-sourced. Efforts to contact the creators went in vain. The original paper

provided just a brief overview of the chatbot and did not contain a lot of important

information - the dialogue management system in use, the NLU modules, the

training methods, etc. It would be naive to try and extend the research done in the

paper with this little knowledge. However, it provided us a basic understanding

of how meta-ontologies can be used to drive a conversation.

12
2.2 Ontology Based Conversation System

2.2.1 Introduction

The domain described in the paper, authored by Quamar et al. Quamar et al. [2020]

is the healthcare domain, which would involve conversations revolving around

medicines, diseases, etc. This paper focuses mainly on identifying the right intents

in each of the user’s sentences.

All key concepts in the ontology are found by running a centrality analysis

on the ontology graph and ranking the concepts according to a centrality score.

Then, statistical segregation is used to identify the top-k concepts. Once these key

concepts are found, dependent concepts are found for these key concepts. For

example, if Medicine is a key concept, then precaution, side-effects, etc. can be the

dependent concepts.

2.2.2 Conversation Patterns

The conversation is restricted to a set of template patterns as shown in Table 2.1.

Apart from this, specific intent patterns can be added as required.

2.2.3 Generating Training Examples

Deep learning is used to recognize intents. For this reason, training examples

are automatically generated from the ontology, once all the key and dependent

concepts have been identified. This is done by using graph algorithms. For

example, consider the lookup pattern: Show me ⟨ dependent concept ⟩ for ⟨ @Key

13
 Table 2.1: Default templates for the conversation Quamar et al. [2020]

Pattern Description Generalized Pattern Example

Name
Lookup Looking up information Show me the ⟨X⟩ for Show me the precau-
pattern for a key concept based on ⟨Y⟩. X is a dependent tions for Crocin
the dependent concept concept of Y
Direct Re- Getting information for Show me the ⟨X⟩ that What drug treats
lationship queries which consists of ⟨Y⟩’s ⟨Z⟩. X, Z are key fever?
Pattern two concepts and a direct concepts, Y is a rela-
relationship between tionship.
Indirect Getting information for N/A Give me the dosage
Rela- queries which consists of of crocin that treats
tionship indirect relationships - i.e fever.
Patterns they are related via multi- Crocin and Fever are
ple functions key concepts, dosage
and treats are the two
functions that relate
them.

Concept ⟩ . The pattern consists of three parts: (1) The initial/start phrase: Show

me, (2)⟨ dependent concept ⟩ , and (3) ⟨ @Key Concept ⟩ . The domain ontology is

traversed and all subgraphs containing pairs of key and dependent concepts such

as“Drug” and “Precaution”,“Drug” and “Dosage”,“Drug” and “Adverse Effects”,

etc. are identified. For each identified dependent concept, different instance values

of the key concepts are taken from the knowledge base and corresponding training

examples with different paraphrases of the initial phrase are created.

2.2.4 Dialogue Management

A dialogue tree is used to facilitate conversation in this system. The dialogue tree

essentially provides the complete space of user conversation that can be under-

stood and provided a response to by the system. It is built using the IBM Watson

Assistant IBM.

14
 The dialogue tree is built in three steps:

• The dialogue logic table is created. This table consists of the template for
various intents, their required entities, optional entities, agent elicitation in
case an entity is missing, and the response to be provided for the particular
intent.

• The dialogue tree is built from a structure of nodes, using the IBM Watson
Assistant Platform. These nodes consist of the information provided in the
dialogue logic table, and have appropriate trigger conditions based on the
same.

• Additional nodes are created to facilitate generic conversation that are domain-
agnostic. These can be used across various domains and use-cases.

2.2.5 Critical Review

While the paper makes some really novel contributions, it doesn’t exactly fit the

problem statement that we are trying to solve. Again, the Ontology store is being

used more like a database, and the conversation is guided entirely by a separate

dialogue management system, without any meta-ontology as an acting hand -

which is what we are interested in.

However, the process of identifying key and dependent concepts might come

in handy, in case we try a model which relies on deep learning and would require

training examples.

2.2.6 Final System

The next chapter discusses the VOnDA Kiefer et al. [2019] system that was used to

build the final chatbot and goes into detail explaining the system, along with the

various choices made through the course of its implementation.

15
 CHAPTER 3

Ontology-Based Chatbot

VOnDA Kiefer et al. [2019] is the system we have extended and fit for our chosen

University FAQ domain. This chapter details the work done through the course

of the implementation of the chatbot, along with the various challenges faced, and

the architectural choices made to tackle them.

3.1 Introduction

VOnDA Kiefer et al. [2019] is a framework to implement dialogue management

functionality in any system. One of the biggest advantages of VOnDA is that it is

built around the RDF/OWL syntax - both it’s specification and memory layer follow

it, making it very easy to add in external data sources like domain ontologies.

VOnDA systems use the HFC reasoner Krieger [2013], as it supports n-tuples

as opposed to the general triples found in most ontologies.

It is domain-independent, and is made for systems that require long-term

memory, and have high user adaptivity. Low-error rates are prioritized, since the

framework is used in systems in healthcare etc, and it cannot afford to go wrong.

VOnDA has strived to provide support for the following design principles:

• Uniform specification of dialogue semantics, that can be modified as per

requirements
• Ability to allow large-scale data storage, both in terms of long conversations,
and also data from various sensors
 Figure 3.1: Sample Rudi file Kiefer et al. [2019]

• Easy and clear way to specify dialogue management rules, while allowing
access to the underlying database which is stored as RDF n-tuples
• Allowing for easy integration of standard Java classes with the entire system

3.1.1 RUDI Language

The rules for dialogue management are written in the RUDI language. In syntax,

it is very similar to C++/Java. However, the language has features that make

it extremely useful to write dialogue specifications. It allows for easy access to

RDF objects and their properties. Appropriate operators are provided that help

facilitate easy checking of various ontological relations.

3.1.2 RUDI Example Code Walkthrough

In Figure 3.1, the rules aim to handle user greeting in the conversation. Line 1

states that the rule should get triggered only if the greeting hasn’t already hap-

pened before. This history is stored in the HFC database. Lines(3-8) specifies that

the system must make a greeting, if the user hasn’t made a greeting. It does this by

emitting a dialogue act called InitialGreeting, and appropriately fills the param-

eters, in this case, the user name. Lines(11-13) specify that if a greeting has been

17
recieved during the session, the system has to greet back with the user’s name.

3.1.3 Context History

The dialogue management module in VOnDA is implemented using a rule-based

approach. The RUDI language can be thought of as a more powerful version of

the if-then statements in other programming languages. The information state in

VOnDA is maintained through the RDF database in the form of n-tuples. The

HFC reasoner is specially built to handle these n-tuples. The advantage of using n-

tuples over standard triples is that timestamps can be attached, allowing a complete

history of interaction to be maintained in the database. This makes more sense

considering that VOnDA was built keeping the context of the robotic sensors in

mind. In the vanilla version of VOnDA, n=4, as can be seen in Figure 3.3.

3.1.4 Dynamic Interpretation Theory(DIT)++ Dialogue Act

The entire dialogue management module is guided by an ontology of speech acts

known as the DIT++ dialogue acts Bunt [2009].

This ontology contains the definition of various dialogue acts and semantic

frames. It serves as a meta-ontology, and takes care of the domain-agnostic parts

of the conversation. Along with the definitions of the dialogue acts, it also consists

of the definition of the semantic relation between these dialogue acts. All this

together, provides the basis for understanding of user utterances.

As shown in Figure 3.2, the hierarchical relations in the ontology represent the

specificity of each of the dialogue acts. In other words, the preconditions of a

dialogue act that is deeper in the tree, are a super set of its ancestors. For example,

18
 Figure 3.2: DIT++ Dialogue Act ontology hierarchical structure Bunt

Figure 3.3: Sample n-tuple domain ontology, zero timestamped

the preconditions for a utterance to be classified as a choice question, is stronger

than the preconditions required for it to be classified as a plain question.

As these dialogue acts form the backbone of the entire dialogue model, it is

crucial to understand these properly. They can be divided into two main groups:

• Information-transfer functions: These consists of all functions whose main

goal is to either provide information to the user(aka addressee), or seek
information from the user. It can be further divided into
– Info-seeking: These are functions which help the speaker gain informa-
tion from the addressee. For example, in questions sentences, the user
tries aims to get some information from the chatbot.
– Info-providing: These are functions which help the speaker give in-
formation to the addressee. For example, when a question is asked, a
chatbot can answer the question, and this sentence can be categorized
under the Info-providing class.

19
 • Action-discussion functions: These functions consist of an action in context,
along with a particular frequency of performing this action. It can be further
divided into:
– Commissives: The speaker is committed to performing a certain action,
and informs the addressee of the same. For example, consider a chatbot
assuring the user that it will get a query right in the future. These sort
of promises can be classified under the Commissives class.
– Directives: The speaker wants the addressee to perform a certain action
and may offer the instructions to do so. For example, in some cases, if
the chatbot feels that all the necessary context information hasn’t been
provided by the user for a particular query, the bot can request the
user to provide the same. This sentence can be categorized under the
Directives class

3.1.5 Domain Ontology

Apart from the meta-ontology, the main driving force of the conversation is the

domain ontology, which provides all the domain knowledge required to answer

various domain-specific user queries. VOnDA converts all ontology triples to n-

tuples, since the HFC reasoner is used. These n-tuples usually consist of the triples

along with an associated timestamp.

In most cases, the domain ontology is static. That is, the TBox, and the ground

information does not change with time. Hence, the way to represent this in the

HFC database, is to format the domain ontology by attaching a timestamp of ”0”

to it, as can be seen in Figure 3.3. However, dynamic data can be added to the

ABox during the course of the chatbot conversation.

The domain ontology can be integrated with the system in two ways. First, it

can be treated similar to a Java object, in the sense that dot (.) operator can be used

to access the particular object’s data properties, directly from the RUDI rule file.

In Figure 3.4, a rule for creating a new Professor object results in the creation of an

object in the HFC database. The object’s datatype properties can be accessed and

20
 Figure 3.4: VOnDA objects can be manipulated as per requirements

manipulated using the various features that the RUDI language supports.

Figure 3.5: Querying the HFC database

In some cases, it is easier to directly query the domain ontology using the HFC

query language. Currently, this functionality does not exist in the RUDI language,

and hence, plain Java functions have to be used for this. The output can be used

in the RUDI rules file by invoking the Java function appropriately. An example is

provided in Figure 3.5. It is a piece of Java code to get all the courses offered by

the CS department. The list of courses returned can be used appropriately in the

dialogue rule file.

3.1.6 NLU and NLG Modules

The DIT++ dialogue act hierarchy is being used in the NLU and NLG units. The

dialogue act object consists of a dialogue act token, a frame and a list of key-value

pairs as arguments to the frame, for example, (Offer(Transporting, what=tool,

to=workbench)). Any NLU/NLG units that can be used in VOnDA will have to

conform to these specifications.

21
 In the vanilla implementation of VOnDA, the NLU module uses Speech Recog-

nition Grammar Specification (SRGS) Burke [2007] to define a grammar for the

user utterances.

3.1.7 Structure and Workflow

Figure 3.6 shows the high-level system architecture. There are five main compo-

nents.

At the base of the entire system is the RDF n-tuple store. This consists of various

speech acts and frame semantics, following the DIT++ Dialogue Act structure. It

also consists of the domain-specific ontology. The HFC reasoner allows for rea-

soning and querying over the n-tuple store during runtime. Extension reasoning

services are added over the general reasoning rules to facilitate n-tuple reasoning.

The next module is the DM module. This is written in the RUDI language and

forms the backbone of the entire chatbot conversation. The dialogue Information

State(IS) can be updated through either new application data, or dialogue acts rec-

ognized by the NLU module in the user utterances. When this happens, different

rules get triggered in the DM module. These rules can either further trigger other

rules or result in the selection of some actions like providing a response to the user.

All rules are repeatedly applied till a fixed point is reached, i.e., there are no new

updates to the IS. As a part of the rules, the HFC database can be queried to get

the domain knowledge to be used in responses to the user utterances.

The Action Selection module comes into play when there are multiple actions to

be taken as a result of an update to the IS. In vanilla implementation, a randomized

method is used to select a particular action. This can, however be extended to use

22
 Figure 3.6: VOnDA system architecture Kiefer et al. [2019]

smarter probabilistic methods for choosing actions.

The other two modules are the NLU and the NLG modules. The NLU module,

as mentioned in 3.1.6 recognizes user intents and the appropriate slots and passes

it to the DM module. The NLG module picks up the output from the DM module

to generate the system response.

Speech Recognition modules can be integrated with the system, but for the

purpose of the project, only natural language input is being considered.

3.2 Work Done

3.2.1 Ontologies

The graphical representation of the domain ontology can be seen in Figure 3.7.

It consists of various entities, relations, and TBox rules for these entities. The

23
 Figure 3.7: Graphical representation of the domain ontology

ontology has been setup keeping in mind the main purpose of it, to function as a

datastore for various Frequently Asked Questions (FAQs) regarding the university

IIT Madras, various departments, professors, courses etc.

How the domain ontology helps the system

Ontologies provide us with certain advantages over the same data being repre-

sented in a traditional database with all the ABox data. One of the main advantages

is that a reasoner can be used to draw various inferences.

Consider the following relation isPrerequisiteFor, which essentially states the

24
 Table 3.1: Object Properties in the domain ontology

Name Domain Range Description

advises Person Person Used to describe the project
guides for various stu-
dents, professors etc.
belongsTo Professor Department Used to describe the de-
partment of a professor
enrolledIn Student Program Used to model the student
enrollment in programs
hasAdmissionReqs Programs Qualifications or Used to model the ad-
StandardizedTests mission criteria for various
programs
hasFacility Department InstituteFacilities Used to model lab facilities
provided by a department
hasPrerequisite Course Course Used to model course pre-
requisites. Transitive in na-
ture.
hasPublished Professors Publicaition Used to model professor’s
publications.
hasScholarship Programs Financial Scholarship Used to model various
scholarships for programs
headsDepartment Professor Department Used to model the HOD for
a department
isAdvisedBy Person Person Inverse of advises
isPrerequisiteFor Course Course Inverse of hasPrerequisite
isSpecializedIn Professors ResearchFields Used to model a profes-
sor’s area of research
offers Department Course or Programs Used to model course or
programs offered by a de-
partment
teaches Professors Courses Used to model course pro-
fessors
taughtBy Courses Professors Inverse of teaches

25
course prerequisites. Now consider the following triples:

CS1100 isPrerequisiteFor CS3700

CS3700 isPrerequisiteFor CS6852

Then, CS6852 has two prerequisites, infact - CS1100 and CS3700.

Just by making the relation isPrerequisiteFor a transitive relation, this can be

inferred automatically by the reasoner. Now, if the same thing had to be done using

a database, we would require some sort of recursion because the prerequisite depth

is unknown and may not be a fixed quantity as the data evolves.

Furthermore, verification of the database can be done using the reasoners.

For example, if a student is defined as a person who takes at least three courses

and is enrolled in some program, then there cannot be any false information in

the database for a person identified as a student because the reasoner checks for

consistency.

How the meta ontology is used

The meta ontology consists only of concepts, and the sub-class object property.

Essentially, the tree shown in Figure 3.2 is modelled in the meta-ontology, and

this is used in the VOnDA rules to handle subsumption conditions. This means

that if there was a rule that would get triggered when a Question dialogue act

is recieved, then the same rule would also get triggered when a Check Question

dialogue act is received. The meta ontology helps guide the underlying parts of

the conversation. For example, if user input is a sentence with a “request” as the

meta ontology classification, then the bot knows that it’s next response should

26
be an “inform” sentence since the user has requested for some information. This

can be extended to other types of exchanges between the user and the bot. If the

user thanks the bot for providing some information with a “thank” sentence, then

again, the bot knows that it has been a successful transfer of information and that

it can ask the user for their next query.

3.2.2 VOnDA Core Module

VOnDA rules have been written for the core module; this involves writing triggers

for various intents and other user responses, like small talk, as shown in Figure

3.8.

For each intent that is modeled, a specific rule has to be written corresponding

to it, which takes care of the actions that have to be performed when the rule

gets triggered. In most cases, this involves some anaphora resolution, and once

the slots are appropriately filled, a call is made to a java function that makes the

appropriate queries in the HFC database and returns the correct query response.

For example, in the Figure 3.10, course information is obtained from the function

answerCourseInfo .

These java functions usually consist of a query to the HFC reasoner. The query

format is very similar to SPARQL queries, and the only difference is the n-tuple

formatted has to be accounted for. In the example function shown in Figure 3.9 ,

the query is:

select ?b where <dept uri> <univ:hasFacility> ?a ? & ?a <rdfs:label> ?b ?

Basically this is choosing the label names of all the department facilities offered by

the department with a given department uri ( unique resource identifier ).

27
Figure 3.8: Rules for small talk conversation

28
 Figure 3.9: Java Function that queries the HFC database

Once the query is returned, it is formatted appropriately, and the flow of control

returns to the VOnDA rule, which made the initial call. The VOnDA rule then

emits a dialogue act which consists of the result in one of the slots.

3.2.3 Anaphora Resolution

Introduction

Anaphora Resolution refers to the use of an expression whose meaning depends

on the context provided either before or after the particular expression.

Consider the sentences in the Figure 3.10. If we consider the second sentence

on its own, its meaning does not become clear. It has to be clubbed with the first

sentence so that the meaning of the word “he” gets attached to Prof.Raman.

This is one of the types of anaphora resolution. It is a complex problem in

29
 Figure 3.10: Example conversation

Figure 3.11: The department name gets resolved in this conversation

the field of Natural Language Processing(NLP), as there can be multiple levels of

reference being used in any sentence.

In VOnDA, the anaphora resolution of the simplest kind has been implemented.

The chatbot system can handle references to object pronouns or personal pronouns

made in the conversation previously.

In Figure 3.11, the chatbot resolved “the department” to the CS department,

because the context history is maintained.

Methodology

Every time a user makes an utterance, the sentence is analyzed, and when an

appropriate rule gets triggered, the code ensures that the entities recognized in the

30
Figure 3.12: VOnDA rule involving Anaphora Resolution

31
 Figure 3.13: Example conversation involving a list being resolved

sentence are timestamped in the database.

Consider the rule that answers questions about a particular course, as seen in

Figure 3.12. There can be two cases.

• If the course name is mentioned, then the “else” clause gets triggered. Here,
we create a new dynamic object in the database of the type Courses. We
assign the name to it. This dynamic object will be the most recent Courses
object in the database. In the future, if any reference is made to a course
without the name being mentioned, we can assume that the course being
talked about is this particular course.

• If the course name is not mentioned in the utterance, it means that somewhere
in the conversation before, the user has mentioned a course name, and he
is currently talking about that particular course. In this case, we go to the
database, look up the latest dynamic course object, and get its name.

Limitations

Anaphora Resolution has not been implemented in cases where the context exists

in the same sentence. “What courses do I teach, if my name is Prof. Raman?” will

not be recognized by the chatbot.

Furthermore, in some cases, the answers might be a list of values. This context

is currently not being maintained.

32
 Figure 3.14: Typical SRGS Grammar rule

In Figure 3.13, the chatbot will not be able to answer the user, since the list

of courses are not being maintained in the dynamic ontology database. This is

something that can be worked on in the next version of the chatbot.

3.2.4 SRGS Grammar Module

The vanilla implementation of the VOnDA system consists of using SRGS grammar

for the NLU module. SRGS grammar is essentially a syntax for grammar represen-

tation. In the project, it has been expressed through the augmented Backus–Naur

form (ABNF). It has the expressive power of context-free grammar. This form

of language expression is sufficient for the vast majority of speech recognition

applications.

A rule exists for each type of expression that the chatbot wants to recognize. A

typical rule looks is shown in Figure 3.14. This rule consists of two parts

• The first part has the sentence that the rule recognizes. Anything in square
brackets is considered an optional part of the rule. A sentence of the form
“what it the course X” and a sentence of the form “what is the course X
about” both get recognized in this rule because “about” is an optional word.
This allows for some flexibility in the input space. The “$$” part refers to a
wildcard; that is, any word is allowed to be matched there. Because a course
name is expected, the $$ will catch the entirety of the course’s name here.

• The second part of the rule has the representation of the sentence that the
chatbot uses in the system to trigger various VOnDA rules. In this case, based
on the DIT++ ontology, the user’s input can be classified as a “Request”
dialogue act. The proposition, in this case, is “Courses”. The theme is

33
 CourseInfo . The dialogue act theme and the proposition are chosen by the
developer. That is, there is no hard and fast rule that these particular dialogue
acts/themes/propositions must be used. However, whatever is used in the
rule here, must tie-up with the VOnDA rules for dialogue management as
well.

Advantages

• Relatively easy to code - Since the NLU is a rule-based grammar, there is no

concept of learning. This makes it very straightforward to code. For every
type of query that the chatbot aims to answer, there has to be a grammar rule
that is able to capture all possible utterances for that particular type of query

• Easy to debug - The code execution is in a very linear fashion, and debugging
becomes extremely easy as a result

Limitations

• The input space is extremely restricted. It allows for almost no flexibility

in terms of spelling errors and extra wrong words being used. For exam-
ple: “what courses are offered in in the CS department” will generate an
error message since it contains the word “in” twice. The grammar is very
unforgiving to such errors.

• Questions of the type “Tell me about X” cannot be answered very easily.

X has to be manually resolved. Furthermore, having these type of rules
complicates things more with other rules. If there is a rule of the type “Tell
me more about X department”, then there is confusion about what to do
when a sentence of the type “Tell me about CS department” comes up.

• It is hard to understand when a sentence is wrong and when it doesn’t belong

in the input space. For example: “Tell me about the prof in the CS department
whose main area of expertise is Artificial intelligence” is a query that is not
supported by the grammar rules. However, even a wrong sentence is not
supported by the rules. Letting the user know when something is a wrong
query as opposed to a query not supported by the chatbot becomes difficult.

• Small talk: Accommodating human small talk becomes tough. Often, words
like thanks, please, etc. might be added at the end of sentences. It’s not easy
to write grammar rules for this.

34
3.2.5 Rasa NLU module

Rasa Bocklisch et al. [2017] is an open-source machine learning framework to au-

tomate text-and voice-based conversation. Based on lower-level machine learning

libraries like Tensorflow and spaCy, Rasa Open Source provides natural language

processing software that’s easy to customize and use in different settings. It’s a full

toolset for extracting the important keywords, or entities, from user messages, as

well as the meaning or intent behind those messages. The output is a standardized,

machine-readable version of the user’s message, which is used to determine the

chatbot’s next action. For this project, we will be using only the Rasa NLU mod-

ule and not be involving the Rasa Core module - which essentially takes care of

the chatbot’s next response, maintaining history, etc. It uses an Recurrent Neural

Network(RNN) based model to maintain history and predict actions. Generally, a

chatbot is developed by using both the Rasa NLU and the Rasa Core modules to-

gether. However, the VOnDA core module provides us the benefit of not needing

training examples, since it is mainly a rule based system, while the Rasa Core’s

RNN requires a large number of training examples to accurately predict various

actions.

Training the NLU module

NLU training data consists of example user utterances categorized by intent. Train-

ing examples can also include entities. Entities are structured pieces of information

that can be extracted from a user’s message.

Training Examples are provided for each intent that the developer aims to

capture, as shown in Figure 3.15. The [] brackets are used to specify the name of

the entity, and the () brackets are used to specify the entity it has to be mapped to.

35
 Figure 3.15: Example of an intent with it’s training examples

Figure 3.16: Synonyms can be provided for entities

Synonyms are provided for phrases that have to be mapped to a same specific

phrase, as shown in Figure 3.16

Regex patterns are also allowed, and this allows for easier entity extraction

Lookup tables are used to aid in easier entity extraction. Basically, the model

has some idea of what are the different types of examples available for a given

entity, as shown in Figure 3.17.

Entities are structured pieces of information that can be extracted from a user’s

message. Entities are annotated in training examples with the entity’s name. In

addition to the entity name, you can annotate an entity with synonyms, roles, or

groups. In the above Figure 3.15, Professors is an entity that is annotated.

Figure 3.17: Lookup tables help in entity extraction

36
Intents Modelled

Table 3.2 consists of all the intents that have been modeled for the chatbot using

the Rasa NLU module. For each of these intents, training examples have been

generated in a format as described in Section 3.2.5. Python code was used to au-

tomatically generate these training examples. However, this method is extremely

naive. For each intent, the code goes through all possible pairs of entities along

with different pairs of phrases to generate training examples. This is unfeasible if

the number of training examples required is large, but one of the main advantages

of Rasa NLU is that it does not require a large training corpus.

Why Rasa NLU?

While starting from scratch, it’s often helpful to start with pre-trained word em-

beddings. Pre-trained word embeddings are helpful as they already encode some

kind of linguistic knowledge. For example, if there is a sentence like “I want to

buy apples” in the training data, and Rasa is asked to predict the intent for “get

pears”, the model already knows that the words “apples” and “pears” are very

similar. This is especially useful if you don’t have enough training data.

Integrating Rasa NLU with VOnDA

VOnDA requires dialogue acts and propositions for the core to work. This is

the first attempt in combining Rasa NLU with VOnDA. Most previous attempts

usually employed different grammar based NLU modules, as opposed to a deep

learning based module. One of the main parts of integrating the Rasa NLU is to

convert the output format from the NLU intent and entity classifier into useable

37
 Table 3.2: Intents modelled in the chatbot

Intent Name Intent Description Intent Example

prof course info Questions regarding What course does Prof.
courses taught by a profes- Abishek Sinha teach?
sor
course info Course details, additional Give me some information
information about the database course?
course prof info Course professor details Who teaches the database
course?
course prereq info Details about the course What are the prerequisite
prerequisites courses for cs3700 ?
prof research info Details about professors’ What are Prof Sinha’s areas
areas of research of research?
dept facilities info Details about department What are the labs in the CS
facilities department?
dept courses info Details about courses of- What are all the courses in
fered in a department the cs department?
dept profs info Details about professors Which profs work in the CS
working in a particular de- department?
partment
dept ug programs info Details about undergradu- What are the ug programs
ate programs in a depart- offered in the CS depart-
ment ment?
dept pg programs info Details about postgraduate What are the pg programs
programs in a department offered in the CS depart-
ment?
dept programs info Details about all programs What are the programs of-
in a department fered in the CS department
dept info General information about Give me information about
a department CS department
program info Details about a particular What is the CS btech pro-
program gram
program admission info Details about admission re- What are the minimum
quirements for a program requisites to join cs btech
program?
prof publication info Details about a professors’ What research work has
publication Prof Anirudh done?
student researchers info Details about students per- Who are the students per-
forming research forming research?
prof guidees info Details about people being Who works under Prof.
guided by a prof Kumar
students info Details about various stu- Details of students enrolled
dents enrolled in different in BTech program
programs

38
 Figure 3.18: The Rasa NLU configuration pipeline

Figure 3.19: Typical Rasa NLU output

VOnDA dialogue acts.

The typical Rasa NLU output is shown in Figure 3.19. The output format has

to be reworked for it to be usable in VOnDA.

First, a Rasa NLU server is started locally. This server allows POST requests

with a given string and returns the entity intents recognized by the Rasa model.

In the VOnDA code, once the user input is received, a POST request is made to

this local server, and the entity intent output is received.

39
 It is then converted into a dialogue act appropriately.

The VOnDA code has a function that converts a given string into a dialogue

act object that is used internally.

A separate function is written for each intent to convert the Rasa NLU output

into a string that satisfies the dialogue act string format.

When the Rasa NLU output is returned, based on the intent, the appropriate

function is called, and a formatted string is returned, which is converted into a

dialogue act object.

The rest of the code remains the same, as the VOnDA rules deal only with

dialogue act objects, and hence no other change is needed.

Advantages of Rasa NLU

• Compared to a normal machine learning module, Rasa requires far fewer

training examples because the pipeline can make use of pre-trained models,
and since we are dealing with English, most of the words used in the domain
already exist. If they don’t, providing training examples for them alone is
enough to make the model work well.

• The strictness and the rigidity that was in the SRGS grammar have been
improved upon in this version. Additional words that don’t change the
meaning of the sentence can still be parsed, and this allows for a larger user
input space.

• Small typos, stray characters, and extra characters are more or less taken care
of by the NLU model. “What coursee does Anirudh teach” has a small typo,
and stuff like this is detected, and intent recognition is still done despite the
error.

Limitations

• Multiple intents cannot be modeled in the same sentence using Rasa NLU un-
less we explicitly mention so. For example, ”who does the prof who teaches
CS3700 advise” consists of two different queries ”who teaches CS3700” and

40
 ”who does this prof advise.” However, Rasa NLU doesn’t output a split
probability for these intents and just gives 0 probability for one of them.

• Overfitting is an issue while training. Since the number of training examples

is less, the NLU module tends to overfit intents to specific phrases in the
training examples. Consider the sentence ”tell me about the course CS3700”
in the training example. Now, the model overfits ”tell me about” to this
particular intent, and any sentence starting with ”tell me about” gets auto-
matically categorized as this intent unless explicitly specified in the training
example.

3.2.6 CPlanner NLG Module

CPlanner NLG is being used as the NLG module for the system. Essentially, it is a

template-based NLG module. It has a set of rules for different dialogue acts, and

depending on the VOnDA core module’s output, these rules get triggered.

Figure 3.20: Typical CPlanner NLG rule

A typical rule is shown in Figure 3.20. It states that if the incoming object con-

sists of an ”Inform” and ”Department” dialogue act, and has attributes dept name

and dept facilities , then the output should be df , the variable associated with

the dept facilities attribute.

3.2.7 Final System

Figure 3.21 shows the workflow of the entire chatbot system in place. Once the user

types in a sentence, it is analyzed by the Rasa NLU model, which then outputs the

most likely intent that categorizes the given sentence, along with entities, if they

41
Figure 3.21: Final System

42
exist. Then, the intents and entities get converted into dialogue acts and are fed into

the VOnDA rule system. Based on various rules and trigger conditions, different

actions are taken, and this may involve querying the HFC reasoner, performing

anaphora resolution, and emitting other dialogue acts. The n-tuple store’s dynamic

data is reset after every chat session, since it contains information only relevant

to a particular chat session. Provisions have been made to store every user chat

session into a local file, in case further analysis has to be performed.

3.3 Domain invariance - Developing a new system

The methodology followed in developing a new chatbot system using the Rasa

NLU and VOnDA core modules are invariant of the domain. This section outlines

the main steps in developing a new chatbot system from scratch, given the domain

ontology. Note that there is no special requirements for how the domain ontology

has to be modelled; it can be manipulated in the VOnDA and Java code depending

on it’s structure.

• Conversion of the domain ontology into an n-tuple format: The Raptor tool
allows for this conversion and zero timestamps the ontology. It creates a .nt
file from a given .owl file. This conversion is necessary to comply with the
format required by the HFC reasoner.

• Training the Rasa NLU module: The Rasa NLU module has to be trained
with various intents and entities. The training examples can be generated
automatically or, in the event of a pre-existing dataset, can be annotated
appropriately as per the required Rasa format.

• Conversion of Rasa intents into VOnDA dialogue acts: Once the Rasa NLU
gives an output, it has to be converted into a dialogue act frame for it to be
used in a VOnDA system. This can be done with the help of Java functions,
using simple string formatting.

• Develop RUDI rules: For each intent that has to be supported by the chatbot,
a VOnDA rule has to be written, along with other generic rules that guide
the conversation.

43
• Develop helper functions: RUDI rules often make calls to the HFC reasoner
and database, and these are supported by Java helper functions.

• Write the NLG rules: The CPlanner is a template-based NLG module, and
each outgoing dialogue act will require a separate rule in this module.

• Modify the meta-ontology: The meta-ontology, as generic as it is, might not

have every classification required to hold a conversation in any domain. This
can be modified as per requirements based on how the chatbot performs.

44
 CHAPTER 4

Conclusions

4.1 Results and Conclusions

25 students were invited to interact with the chatbot, as a part of the user survey.

This helped us understand the drawbacks of the bot, and various improvements

that could be made.

4.1.1 User interaction

User interactions were stored in logs, to perform analysis.

Successful conversation - An example

Bot: Hello!

User: Hi

Bot: Whats your name

User: Im Suhas

Bot: How can I help you, Suhas

User: Who are the profs in the CS department?

Bot: These are the profs working in the department:

1. P sreenivasa kumar

2. Rupesh Nasre

3. Shweta Agarwal
4. N.S. Narayanaswamy

5. C Chandra Sekhar

6. Arun Rajkumar

7. Balaram Ravindran

User: What course does sreenivasa teach?

Bot: Sreenivasa teaches the ontologies course

User: Tell me more about the course

Bot: This is an introductory course about ontologies. In computer science

and information science, an ontology encompasses a representation,

formal naming, and definition of the categories, properties, and

relations between the concepts, data, and entities that substantiate

one, many, or all domains of discourse.

User: What are itsss prerequsites?

Bot: The ontologies course has the following prerequisites:

1. CS1100 - Introduction to programming

2. CS3700 - Database Systems

User: Thanks

User: What are the papers he has published? (Notice he refers to Sreenivasa

sir)

Bot: Sreenivasa has published the following papers:

1. ERONTO: a tool for extracting ontologies from extended E/R diagrams

2. A heuristic for placement of limited range wavelength converters in all-

optical networks

3. Converter placement in all-optical networks using genetic algorithms

User: What are the other courses in this department? (Notice the this

refers to CS department, since it was the last spoken about department)

46
Bot: cs department offers the following courses: 1. CS1100 - Introduction

to Programming 2. CS6852 - Theory and Applications of Ontologies 3.

CS6023 - GPU Programming 4. CS1200 - Discrete Mathematics 5. CS2800 -

Design and Analysis of Algorithms 6. CS3700 - Introduction to Database

Systems 7. CS6700 - Reinforcement Learning 8. CS6910 - Deep Learning

User: Thanks

Bot: Anything else?

User: Tell me about the Btech program in this department

Bot: The Btech program is a 4-year course for students and exposes them to

varied aspects of computer science

User: How can I join? What are the admission requirements?

Bot: The BTech in CS program has the following admission criteria:

1. JEE Advance Pass

2. 12th Standard Pass

User: Thanks

Bot: Anything else?

User: no, thats it.

Bot: Have a nice day

Unsuccessful query examples

User: What are the prerequisites of the ontologies course?

Bot: The ontologies course has the following prerequisites:

1. CS1100 - Introduction to programming

2. CS3700 - Database Systems

User: Tell me about the first course

Bot: No course with the name first exists

User: cs6852

47
Bot: I was unable to understand that. Could you rephrase the sentence?

User: cs6852 course info

Bot: Info about the course cs6852: To discuss topics in the<br>emerging

areas of Semantic Web Technology, Ontologies for Knowledge

Representation<br>and their Applications.

4.1.2 Chatbot Evaluation

The 25 users who interacted with the chatbot system were asked to fill out a survey

form. Figures 4.1,4.2, 4.3, detail the various responses received from the users.

Figure 4.1: How easy was it to use the system on a scale from 1 (very hard) to 5
(very easy)?

Figure 4.2: How natural was the conversation on a scale from 1 (very unnatural)
to 5 (very natural)?

48
Figure 4.3: How well did the bot understand you on a scale from 1 (very bad) to 5
(very good)?

A few other metrics were tracked from the log files of the conversation.

Average Session Length : 21 exchanges

Average Questions per session : 9.45

Average non-response/error rate : 24%

4.2 Future Work

Since this is just the first two versions of the chatbot to be developed, there is a lot

of scope for improvement in the chatbot as a whole. A few of them include:

• Domain Ontology expansion: The domain ontology can be expanded to

include a complete database of the institute. Currently, since the chatbot is a
proof of concept model, only a subset of the database has been modeled as
the domain ontology

• More use of the meta ontology in the VOnDA rules: Currently, the chatbot
is set up more as an information-providing bot, and there are very few rules
that handle small talk and other parts of the conversation.

• Differentiating between queries that cannot be answered due to lack of do-

main knowledge and queries that cannot be answered because the bot is
unable to understand them, despite having the necessary knowledge. For
example, A query of “How far away is the Sun” is out of the university

49
 domain knowledge. But a query of the type “what course does the profes-
sor who guides Suhas teach?” is a more complicated query, which the NLU
cannot understand, but has enough knowledge to respond to.

• Expanding the intent list : Currently, only 20 intents are being supported. A
full-fledged bot can be made by expanding the intent list.

• Improving training examples: The training examples were generated with

a very naive method by cross-matching different phrases and entities. This
can be improved as described in 2.2.3. This will improve the NLU module.

• Distributing the chatbot on a website: Ideally, students should be able to in-

teract with the chatbot. This would require us to implement a multithreaded
chatbot, so that different users will be able to interact simultaneously on a
website.

50
 APPENDIX A

Example RUDI rules

A few VOnDA RUDI rules are provided below. The complete set of rules, along

with the domain ontology, can be found here.

answer_courses:

/** Provides description about the course */

if (lastDA()>= #Request(Courses) && lastDA().theme=="CourseInfo")

if(lastDA().what == "objectPronoun" || (Boolean)hfcutils.

checkObjectPronoun(lastDA().what))

course_resolved_name = (String)hfcutils.resolveObjectPronoun("<univ:

Courses>");

course_resolved_info = (String)hfcutils.answerCourseInfo(

course_resolved_name);

propose("return_course_info"){

emitDA(#Inform(Courses, cname={course_resolved_name}, info={

course_resolved_info}));

else

course_first_name = lastDA().what;

course_name = (String)hfcutils.cleanCourseNames(course_first_name);
 courseObj = new Courses;

courseObj.name = course_name;

course_info = (String)hfcutils.answerCourseInfo(course_name);

propose("return_course_info"){

emitDA(#Inform(Courses, cname={course_name}, info={course_info}));

lastDAprocessed();

answer_course_instructor:

/** Provides info about which prof is teaching the course **/

if (lastDA()>= #Request(Courses) && lastDA().theme=="CourseTeacherInfo")

if(lastDA().what == "objectPronoun" || (Boolean)hfcutils.

checkObjectPronoun(lastDA().what))

course_resolved_name = (String)hfcutils.resolveObjectPronoun("<univ:

Courses>");

course_resolved_teacher_info = (String)hfcutils.answerCourseTeacherInfo

(course_resolved_name);

prof_name = (String)hfcutils.getProfNameFromOutput(

course_resolved_teacher_info);

if(prof_name!= "No such course exists.")

prof = new Professors;

52
 prof.name = prof_name;

propose("return_course_teacher_info")

emitDA(#Inform(Courses, cname={course_resolved_name},

course_teacher_info={course_resolved_teacher_info}));

else

course_first_name = lastDA().what;

course_name = (String)hfcutils.cleanCourseNames(course_first_name);

courseObj = new Courses;

courseObj.name = course_name;

course_teacher_info = (String)hfcutils.answerCourseTeacherInfo(

course_name);

prof_name = (String)hfcutils.getProfNameFromOutput(course_teacher_info)

if(prof_name!= "No such course exists.")

prof = new Professors;

prof.name = prof_name;

propose("return_course_teacher_info")

emitDA(#Inform(Courses, cname={course_name}, course_teacher_info={

course_teacher_info}));

53
 }

lastDAprocessed();

// Program related rules

answer_program_info:

if (lastDA()>= #Request(Programs) && lastDA().theme=="ProgramInfo")

if(lastDA().pronoun && lastDA().pronoun == "True")

dept_resolved_name = (String)hfcutils.resolveObjectPronoun("<univ:

Department>");

program_resolved_name = (String)hfcutils.resolveObjectPronoun("<univ:

Programs>");

program_resolved_info = (String)hfcutils.answerProgramInfo(

dept_resolved_name, program_resolved_name);

propose("return_program_info")

emitDA(#Inform(Programs, program_info = {program_resolved_info}));

else

54
if(lastDA().dept == "objectPronoun" || (Boolean)hfcutils.

checkObjectPronoun(lastDA().dept))

dept_resolved_name = (String)hfcutils.resolveObjectPronoun("<univ:

Department>");

program_name = lastDA().program_type;

program_resolved_info = (String)hfcutils.answerProgramInfo(

dept_resolved_name, program_name);

propose("return_program_info")

emitDA(#Inform(Programs,program_info={program_resolved_info}));

else

dept_name = lastDA().dept;

dept = new Department;

dept.name = dept_name;

program_name = lastDA().program_type;

program = new Programs;

program.name = program_name;

program_info = (String)hfcutils.answerProgramInfo(dept_name,

program_name);

propose("return_program_info")

55
 emitDA(#Inform(Programs, program_info={program_info}));

lastDAprocessed();

56
 REFERENCES
Bocklisch, T., J. Faulkner, N. Pawlowski, and A. Nichol (2017). Rasa: Open source
language understanding and dialogue management. URL https://arxiv.org/abs/
1712.05181.

Brown, T. B., B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan,

P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan,
R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler,
M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford,
I. Sutskever, and D. Amodei (2020). Language models are few-shot learners.

Bunt, H. (). Iso/dit dialogue annotation and its semantics. http://www2.lpl-aix.fr/

˜otim/documents/OTIM_HarryBunt_Wshop24mai2011.pdf. Accessed: 2021-12-06.
Bunt, H. (2009). The dit++ taxanomy for functional dialogue markup. Journal of Philosoph-
ical Logic.

Burke, D., Speech Recognition Grammar Specification (SRGS). 2007, 143–157.

IBM (). Ibm watson assistant. https://www.ibm.com/products/watson-assistant. Ac-

cessed: 2022-04-27.

Kiefer, B., A. Welker, and C. Biwer (2019). Vonda: A framework for ontology-based
dialogue management.

Krieger, H.-U., An efficient implementation of equivalence relations in owl via rule and
query rewriting. 2013.

Lee, C., S. Jung, K. Kim, D. Lee, and G. Lee (2010). Recent approaches to dialog manage-
ment for spoken dialog systems. JCSE, 4, 1–22.

Pérez, J., M. Arenas, and C. Gutiérrez (2006). Semantics and complexity of SPARQL.
CoRR, abs/cs/0605124. URL http://arxiv.org/abs/cs/0605124.

Quamar, A., C. Lei, D. Miller, F. Ozcan, J. Kreulen, R. J. Moore, and V. Efthymiou,

An ontology-based conversation system for knowledge bases. In Proceedings of the 2020
ACM SIGMOD International Conference on Management of Data, SIGMOD ’20. Association
for Computing Machinery, New York, NY, USA, 2020. ISBN 9781450367356. URL
https://doi.org/10.1145/3318464.3386139.

Searle, J. R., Speech Acts: An Essay in the Philosophy of Language. Cambridge University
Press, 1969.

Wessel, M., G. Acharya, J. Carpenter, and M. Yin, OntoVPA—An Ontology-Based Dialogue

Management System for Virtual Personal Assistants: 8th International Workshop on Spoken
Dialog Systems. 2019. ISBN 978-3-319-92107-5, 219–233.

57