Natural Language

Processing
KOE-088
Unit2
Prepared by Sandhya Avasthi, AP, Department of CSE
Topics Covered in Unit 2
✓Introduction to semantics
✓Linking syntax and semantics
✓Ambiguity resolution, Semantic Representation
✓Strategies for knowledge representation
✓Knowledge representation
✓Reasoning
✓Applications
✓Machine translation
✓Database interface
 Introduction to semantics
• Semantics is the study of linguistic meaning
• It examines what meaning is, how words get their meaning, and how the
meaning of a complex expression depends on its parts.
• Part of this process involves the distinction between sense and reference
• Semantics contrasts with syntax, which studies the rules that dictate how to
create grammatically correct sentences, and pragmatics which investigates
how people use language in communication.
Lexical Semantics
• Lexical semantics is branch of semantics that studies word meaning
• It examines whether words have one or several meanings and in
what lexical relations they stand to one another.
• Phrasal semantics exploring the phenomenon
of compositionality or how new meanings can be created by
arranging words.
• Formal semantics relies on logic and mathematics to provide
precise frameworks of the relation between language and meaning.
• Cognitive semantics examines meaning from a psychological
perspective and assumes a close relation between language ability
and the conceptual structures to understand the world.
Semantic processing
• Permit us to reason about their truth (i.e., their relationship to some world)
• Permit us to answer questions based on their content
• Permit us to perform inference (answer questions and determine the truth of
things we don’t already know to be true)

Real life scenario where semantic processing is needed:-

• Answering an essay question on an exam.
• Deciding what to order at a restaurant by reading a menu.
• Learning to use a new piece of software by reading the manual.
• Realizing that you’ve been insulted. Following a recipe.
Semantic Analysis
• Semantic analysis is the process of taking in some linguistic
input and assigning a meaning representation to it.
• There a lot of different ways to do this that make more or less (or no)
use of syntax
• compositional rule-to-rule approach
• Compositional Analysis
• Create a logical representation that accounts for all the entities, roles
and relations present in a sentence.
Representational Schemes
• In general use of First Order Logic (FOL) can be used as
representational framework
 Four meaning Representations
• For example sentence I have a car
• Meaning representation languages are
- First Order Predicate Calculus
-graph in the center illustrates a Semantic Network
-the third row contains a Conceptual Dependency diagram
-frame-based representation,
• These representations can be viewed from at least two distinct perspectives in all
four of these approaches: as representations of the meaning of the particular
linguistic input I have a car, and as representations of the state of affairs in some
world.
Four meaning Representations
Why Meaning Representations are needed?
Assuming that we have a computer system that accepts spoken language queries from
tourists and construct appropriate responses by using a knowledge base of relevant
domain knowledge.

Meaning representation will help in ……..

▪ Verifiability
▪ Unambiguous Representations
▪ Canonical Form
▪ Inference and Variables
▪ Expressiveness
 First Order Logic(FOL)
• Allows for…
• The analysis of truth conditions
• Allows us to answer yes/no questions
• Supports the use of variables
• Allows us to answer questions through the use of variable
binding
• Supports inference
• Allows us to answer questions that go beyond what we know
explicitly
Meaning Structure of Language
• These include a variety of conventional form-meaning associations,
word-order regularities, tense systems, conjunctions and quantifiers,
and a fundamental predicate-argument structure.
• Natural languages convey meaning through the use of
• Predicate-argument structures
• Variables
• Quantifiers
• Compositional semantics
 Predicate-Argument Structure
• This asserts that specific relationships hold among the various concepts
underlying constituent words and phrases that make up sentences.
• Events, actions and relationships can be captured with representations
that consist of predicates and arguments to those predicates.
• Languages display a division of labor where some words and constituents
(typically) function as predicates and some as arguments.
• Predicates
• Primarily Verbs, VPs, Sentences
• Sometimes Nouns and NPs
• Arguments
• Primarily Nouns, Nominals, NPs, PPs
 First Order Predicate Calculus/Logic
• It is a flexible, well-understood, and computationally tractable approach to
the representation of knowledge that satisfies many of the requirements
• It provides a sound computational basis for the verifiability, inference, and
expressiveness requirements.
• Most attractive feature of FOPC is the fact that it makes very few specific
commitments as to how things ought to be represented
Elements of FOPC
▪ Terms- It is a device for representing objects, each device thought of as a way of
naming, or pointing to, an object in world under consideration
▪ Constants- Refer to specific objects in world being de- CONSTANTS scribed. Such
constants are depicted as either single capitalized letters such as A and B or
single capitalized words that are often reminiscent of proper nouns such as
Maharani and Harry
▪ Functions-correspond to concepts that which are often ex- FUNCTIONS pressed
in English as genitives such as the location of Maharani or Maharani’s locaation.
▪ Variable- mechanism for referring to , objects. Variables, which are normally
depicted as single lower-case letters, give us the ability to make assertions and
draw inferences about objects without having to make reference to any particular
named object
Example of an FOPC representation
• FOPC is organized around notion of the predicate. Predicates are symbols
that refer to, or name, the relations that hold among some fixed number of
objects in a given domain.
• Example Sentence -> Maharani serves vegetarian food
• FOPC representation-> serves ( Maharani, vegetarian food)
One-place predicate that is used, not to relate multiple objects, but rather
to assert a property of a single object.

Example sentence Maharani is a restaurant

FOPC Restaurant( Maharani)
Logical Connectives
• Larger composite representations can also be put together through
the use of logical connectives
• logical connectives give us the ability to create larger
representations by conjoining logical formulas using one of three
operators
Example -> I only have five dollars and I don’t have a lot of time.
Semantics of FOPC/FOL
• Various objects, properties, and relations represented in a FOPC knowledge base
acquire their meanings by virtue of their correspondence to objects, properties, and
relations out in the external world being modeled by the knowledge base.
• FOPC sentences can be assigned a value of True or False based on whether
propositions they encode are in accord with the world or not.
• Example-> Ay Caramba is near ICSI
• Capturing meaning of example in FOPC involves identifying Terms and Predicates
that correspond to grammatical elements in the sentence
• Creating logical formulas that capture the relations implied by the words and syntax
of the sentence
• FOPC form -> Near(LocationOf(AyCaramba, LocationOf(ICSI)
• this sentence can be assigned a value of True or False based on whether or not the
real Ay Caramba is actually close to ICSI or not
 Variables and Quantifiers
• Variables are used in two ways in FOL: to refer to particular anonymous
objects and to refer generically to all objects in a collection.
• These two uses are made possible through the use of operators known as
quantifiers two operators that are basic to FOPC are the ex-
QUANTIFIERS existential quantifier, which is denoted by ∃ ( there exists)
and ∀ ( for all)

FOL-
Inference through Modus ponens
• Inference — Ability to add valid new propositions to a knowledge base, or to
determine the truth of propositions not explicitly contained within a knowledge
base.
• Modus ponens is a familiar form of inference that MODUS corresponds to what
• PONENS is informally known as if-then reasoning

• where a and b should be taken as FOPC formulas

Use of modus ponens Example

Modus Ponens can be used in two ways:-

• Forward chaining- as individual facts are added in knowledge base , modus
ponens is used to fire all applicable implication rules
• Backward chaining- modus ponens is run in reverse to prove specific proposition
called queries. Prolog programming language is a backward chaining system that
implements this strategy
 Backward Chaining
1. First step is to see if the query formula is true by determining if it is present in the
knowledge base
2. If not, then next step is to search for applicable implication rules present in the
knowledge base.
3. An applicable rule is one where the consequent of rule matches the query formula
4. If there are any rule such rule , then the query can be proved if the antecedent of any
one of them can be shown to be true.
Example:-
To verify truth of proposition Serve(Leaf, VegetarianFood)

1. Since it is not present in the knowledge base, a search for applicable rule is initiated
resulting in rule
2. After substituting constant Leaf for ‘x’, antecedent of rule VegetarianRestraurant(Leaf)
truthness is checked, which is TRUE.
3. Hence given proposition is TRUE.
 Event and State Representations
• States are conditions or properties that remain unchanged
• Events denote changes in some state of affairs
• The representations of both can involve a host of participants, props,
times and locations.
• it consists of of single predicates with as many arguments as needed
to incorporate all the roles associated with example.

Sentence – Leaf serves vegetarian fare

FOL - Serves( Leaf, Vegetarian Fare)

 Events(1)
• Representations for events consists of single predicates with as many arguments as
are needed to incorporate all the roles associated with an example.
• For example, the representation for making a reservation consists of a single
predicate with arguments for the person making the reservation
• restaurant, day, time, and the number of people in the party, as in the following:-
 Events(2)
There are Four problems that are there with this approach:-
➢Determining correct number of roles for any given event.
➢Representing facts about the roles associated with an event.
➢Ensuring all correct inferences can be derived from the representation of an event.
➢Ensuring that no incorrect inferences can be derived from the representation of an
event.
 Representing Events

• This representation is costly.

• These predicates can’t be tied together, but example 14.22 to 14.28 have logical relationship.
Event Representation Using Event Variable
• Add an event variable as the first argument to the representation of any
event. The variable ‘e’ give handle on the event in the questions.
• To make additional assertions about this event, this variable can be used.
• This is known as Davidsonian representation.

Example 1 – I ate a turkey sandwich for lunch at my desk.

FOL Representation-
∃𝑒 𝐸𝑎𝑡𝑖𝑛𝑔(𝑒, 𝑆𝑝𝑒𝑎𝑘𝑒𝑟, 𝑇𝑢𝑟𝑘𝑒𝑦𝑆𝑎𝑛𝑤𝑖𝑐ℎ, 𝐿𝑢𝑛𝑐ℎ, 𝐷𝑒𝑠𝑘)

Example 2 – I ate a turkey sandwich for lunch at my desk on Tuesday.

FOL Representation-
∃𝑒 𝐸𝑎𝑡𝑖𝑛𝑔 𝑒, 𝑆𝑝𝑒𝑎𝑘𝑒𝑟, 𝑇𝑢𝑟𝑘𝑒𝑦𝑆𝑎𝑛𝑤𝑖𝑐ℎ, 𝐿𝑢𝑛𝑐ℎ, 𝐷𝑒𝑠𝑘 ∧ Time(e, Tuesday
Representing Time
• The representation of time information in a useful form is the domain of
temporal logic and Tense logic.
• Theory of time holds that it flows inexorably forward and that events are
associated with either points or internal in time.
• Order distinct events by situating them on timeline

FOL Representation-
 English Tense Logic

Reference Point- the current moment in time is equated with time of utterance and is used as
reference point when the event occurred (before, at or after).
Semantics of various logical connectives
Aspect
• This concerns a cluster of related topics including whether an event
has ended or is ongoing.
• Whether it is conceptualized as happening at a point in time or over
some interval.
• Based on these notion event expressions is divided into four general
classes….
Stative- I know my departure gate
Activity- John is flying
Accomplishment- Sally booked her flight
Achievement- She found her gate
Stative expressions
• Represents the notion of an event participant having a particular
property or being in a state, at a given point of time.
• These expressions can be thought of as capturing an aspect of a world
at a single point in time.
• Participant can be seen as experiencing something.
Example-
I like flight 840 arriving at 10:06
I need the cheapest fare
I want to go first class
 Activity Expressions
• These describes events undertaken by a participant and have no particular
end point.
• Unlike statives activities are seen as occurring over some span of time and
therefore not associated with single points in time.
Example-
She drove a Mazda
I live in Brooklyn
She lives in New Delhi.
She drove her SUV for an hour
Description Logics(1)
• Some other representational schemes are semantic network and frames.
• Frames are also known as slot-filter representations.
Semantic Networks- objects are represented as nodes in a graph with
relations between objects being represented by named links.
Frame-based systems- objects are represented as feature structures.
Features are called slots and the values, or fillers of these slots can either be
atomic values or other embedded frames.
• When using Description Logics to model an application domain, emphasis is
given on representation of knowledge about categories, individuals that belong
to those categories and relationship that can hold among these individuals.
• Set of categories, or concepts is called it terminology.
Description Logics(2)
• Portion of knowledge base that contain terminology is called TBox
• Facts about individuals is called ABox.
• Terminology is arranged into a hierarchical organization called an ontology
• Ontology captures the subset/superset relations among the categories.
• After specifying categories of interest , next step is to arrange categories into
hierarchy.
• There are two ways to do it---
-directly assert relations between categories that are related
-provide complete definitions of the concepts, then rely on it to infer
What is Ontology?(1)
An ontology specifies a rich description of the
• Terminology, concepts, nomenclature
• Properties explicitly defining concepts
• Relations among concepts ( hierarchical and lattice)
• Rules distinguishing concepts, refining definitions and relations ( constraints,
restrictions and regular expressions)
All these related to a specific domain.
What is Ontology?(2)
• Ontologies provide a common vocabulary for use by independently developed
resources, processes, services
• Agreements among organizations sharing common services can be made with regard
to their usage; the meaning of relevant concepts can be expressed unambiguously
• By composing/mapping ontologies and mediating terminology across participating
events, resources and services, independently developed services can work together
to share information and process consistent accurately and completely.
• Ontologies also ensure
- valid conversations among agents to collect, process, fuse and exchange
information
-accurate searching by ensuring context using concept definitions and relations
instead of/in addition to statistical relevance of keywords.
Semantic Network
• A semantic network or net is a graph structure for representing knowledge in
patterns of interconnected nodes and arcs.
• it is a declarative graphic representation that can be used to represent
knowledge and support automated systems for reasoning about knowledge.
• It is a directed or undirected graph consisting of vertices, which
represent concepts, and edges, which represent semantic
relations between concepts
• Six types are definitional, assertional, implicational, executable, learning and
hybrid semantic networks
Semantic Network for financial Product
• financial products can be described
by their duration, risk level, and
other characteristics
• a mortgage is a financial product
that is long-term and has a
dependency on other financial
products such as home insurance,
life insurance, and current
accounts.
• An example of a long-term loan is a
mortgage, and an example of a
low-risk investment is a
government bond.
1. Definitional networks
▪ It emphasize the subtype or is-a relation between a concept type and a newly defined
subtype.
▪ It is called a generalization or subsumption hierarchy
▪ It supports the rule of inheritance for copying properties defined for a supertype to all
of its subtypes.
▪ Since definitions are true by definition, the information in these networks is often
assumed to be necessarily true.
2. Assertional networks
• are designed to assert propositions.
• Unlike definitional networks, the information in an assertional network is assumed to be
contingently true, unless it is explicitly marked with a modal operator.
• Some assertional networks have been proposed as models of the conceptual structures
underlying natural language semantics.
Semantic Network Types
3. Implicational networks use implication as the primary relation for
connecting nodes. They may be used to represent patterns of beliefs,
causality, or inferences.
4. Executable networks include some mechanism, such as marker
passing or attached procedures, which can perform inferences, pass
messages, or search for patterns and associations.
5. Learning networks build or extend their representations by acquiring
knowledge from examples. The new knowledge may change the old
network by adding and deleting nodes and arcs or by modifying
numerical values, called weights, associated with the nodes and arcs.
6. Hybrid networks combine two or more of the previous techniques,
either in a single network or in separate, but closely interacting
networks.
 Pipeline of Semantic Analysis

• syntactic analysis of an input sentence will form the input to a semantic analyzer.
• An input is first passed through a parser to derive its syntactic analysis.
• This analysis is then passed as input to a semantic analyzer to produce a meaning
representation.
• Note that although this diagram shows a parse tree as input, other syntactic
representations such as feature structures, or lexical dependency diagrams, can be
used.
Compositional Analysis
• Principle of Compositionality
• The meaning of a whole is derived from the meanings of the parts
• What parts?
• The constituents of the syntactic parse of the input
• What could it mean for a part to have a meaning?
Example AyCaramba serves meat.

• A semantic analyzer given this tree as input proceed by first retrieving a meaning
representation from subtree corresponding to the verb serves
• Analyzer next extracts meaning representations corresponding to two noun phrases in the
sentence.
• Then using the representation acquired from the verb as a template, the noun phrase meaning
representations can be used to bind the appropriate variables in the verb representation
• thus producing the meaning representation for the sentence as a whole