Natural Language

Processing
Pushpak Bhattacharya
Aditya Joshi

Chapter 1
Introduction

Copyright © 2023 by Wiley India Pvt. Ltd.

Chapter 1 Introduction
• 1.1 Language and Linguistics
• 1.2 Ambiguity and Layers of NLP
• 1.3 Grammar, Probability and Data
• 1.4 Generations of NLP
• 1.5 Scope of the book

2 CH1 Introduction Copyright © 2023 by Wiley India Pvt. Ltd.

 Learning Objectives

• Understand the relationship between linguistics, probability, and data in the context
of Natural Language Processing (NLP)

• Describe three generations of NLP

• List typical NLP problems

• Conceptualize applications of NLP

3 CH1 Introduction Copyright © 2023 by Wiley India Pvt. Ltd.

“Languages allow humans to record and reproduce ideas and are recorded in multiple
forms using spoken words or gestures/signs, written words, or in the digitized form
such as video, audio, or text”

• This work explores the field of natural language processing (NLP): computational
techniques that process languages used by humans (i.e., ‘natural’ languages)

• ‘Human language technology’ highlights that it is a technology while keeping the

‘human’ in focus

• ‘Computational linguistics’ is an alternative name, which identifies the collaboration

between linguistics and computer science that is at the heart of NLP

4 CH1 Introduction Copyright © 2023 by Wiley India Pvt. Ltd.

Language and Linguistics

• Birds and animals create sounds to communicate with one another

• Communication allows exchange of ideas and, as a result, improves the species’

ability to survive

• This extends even to humans

• One of the key reasons for the advancement of the human race is language

• Language is a set of well-agreed codes to convey ideas

5 CH1 Introduction Copyright © 2023 by Wiley India Pvt. Ltd.

• In the case of spoken texts, sounds are the codes

• In the case of written texts, symbols are the codes that carry meaning and, hence,
convey ideas

• Scripts of languages fall into three categories depending on how they treat vowels

• They are: alphabet, abugida, and abjad

• In case of ‘alphabet’, all consonants and vowels are written explicitly

• For the ‘abjad’ category, as in Arabic, vowels are mostly dropped completely

6 CH1 Introduction Copyright © 2023 by Wiley India Pvt. Ltd.

• Sign languages are a detailed set of gestures conveyed using hands and/or facial
expressions to convey a detailed meaning

• They are used by people who may not be able to produce conventional sounds
in a language

• Sign language can be codified via specialized scripts

• With this diversity of language, “Linguistics” is the scientific study of languages

• The layers of NLP align with several sub-fields of linguistics

Metaphorically speaking, linguistics is the eye of NLP!

7 CH1 Introduction Copyright © 2023 by Wiley India Pvt. Ltd.

Ambiguity and Layers of NLP
• Language is inherently ambiguous

• A word, a phrase, a sentence can have more than one meaning

• Ambiguity can broadly be of two kinds: lexical and structural

• The former arises primarily due to multiple meanings of lexical units, specifically
words

• The latter arises from how phrases are linked to other sentential units

8 CH1 Introduction Copyright © 2023 by Wiley India Pvt. Ltd.

Lexical Ambiguity
Example:

Question: Why should one never date a tennis player?

Answer: Love means nothing to them (Ambiguity of ‘love’ and ‘nothing’)

• The ambiguity arises due to a possible meaning of ‘love’ as a zero score in tennis and
the word ‘nothing’

• The word ‘love’ as zero has a specific technical meaning in tennis

9 CH1 Introduction Copyright © 2023 by Wiley India Pvt. Ltd.

Dependency Ambiguity
Let us see an interesting sentence which appeared in The Times of India:
‘Maharashtra reports increased COVID-19 cases’

• There are five words in this sentence, with the intended meaning as ‘it is reported by
the Government of Maharashtra that COVID-19 cases have increased’

Fig 1.1 Structural ambiguity

10 CH1 Introduction Copyright © 2023 by Wiley India Pvt. Ltd.

Pragmatic Ambiguity
Example: The following is a conversation between a passenger and a chatbot

Fig 1.2 Example of multimodal sarcasm

11 CH1 Introduction Copyright © 2023 by Wiley India Pvt. Ltd.

NLP handles many research problems, some of which are as follows:

i. Machine translation (e.g., Google Translate)

ii. Information extraction (e.g., OpenIE)

iii. Information retrieval (all the search engines)

iv. Question-answering systems (e.g., Alexa)

v. Multimodal NLP (e.g., emotion recognition based on facial expressions and spoken
words)

vi. Sentiment and emotion analysis: (e.g., the recent ChatGPT tool)

12 CH1 Introduction Copyright © 2023 by Wiley India Pvt. Ltd.

 Fig 1.3 Visualizing NLP along three dimensions

13 CH1 Introduction Copyright © 2023 by Wiley India Pvt. Ltd.

Grammar, Probability, and Data
• Computation in NLP involves mechanisms that encode the structure of language

• While structure brings a sense of determinism to language, it often misses the point
that language is not deterministic or fossilized

• Language is non-deterministic implies:

i. No grammar, as far as we know, can capture all and only the phenomena of the
language

ii. Given the left-hand side (LHS) of a production rule, multiple right-hand sides (RHS)
are possible

14 CH1 Introduction Copyright © 2023 by Wiley India Pvt. Ltd.

• Lexical and structural ambiguities can cause more than one derivation

• It is to be noted that language behaviour of a community evolves over time

• As a result, new and unexpected language phenomena are witnessed, while some
fade away

• This potentially explains why NLP turns to “probability”

• Similar to Machine Learning, In the context of NLP, this maps to:

“Use a dataset of textual examples as the ‘training data’ (in the sense of past data), to
learn a model that makes predictions on the ‘test data’ (in the sense of future data)”

15 CH1 Introduction Copyright © 2023 by Wiley India Pvt. Ltd.

• Ambiguity resolution in the context of NLP is grounded on two key principles:

i. Language constructs have multiple meanings (i.e., ambiguity)

ii. NLP must choose among possible meanings by assigning numerical scores to them

• These scores come from probability, which is in turn based on data

16 CH1 Introduction Copyright © 2023 by Wiley India Pvt. Ltd.

 Fig 1.4 Relationship between grammar,
probability, and data

17 CH1 Introduction Copyright © 2023 by Wiley India Pvt. Ltd.

 Generations of NLP

• Two forces propelled entry and harnessing of data in NLP:

i. Rules many times proved woefully inadequate to capture the full spectrum and
complexity of language phenomena

ii. Language data in machine-processable form kept on increasing and machine

learning opened up new vistas

18 CH1 Introduction Copyright © 2023 by Wiley India Pvt. Ltd.

• Probability showed a way of teasing the regularities out of the data

• Three ways of computing probability from language data emerged:

i. Maximum likelihood
ii. Maximum entropy
iii. Bayesian probability

• In general, disambiguation in NLP at any layer amounts to choosing the best among
possible labels, strings, trees, or graphs

• Deep learning introduced a continuum in which similar linguistic units were close to
each other because:
i. It could handle data sparsity
ii. it could learn vectors that are similar to each other

19 CH1 Introduction Copyright © 2023 by Wiley India Pvt. Ltd.

• However, NLP witnessed a paradigm shift through the use of deep towards the so-
called deep neural networks (DNN)

• DNNs interweave two operations:

i. Learning meaning representations of language units

ii. Learning to solve particular NLP tasks. DNN for NLP relies on layers of neural
networks learning representations of text

20 CH1 Introduction Copyright © 2023 by Wiley India Pvt. Ltd.

 Fig 1.5 Three generations of NLP and their characteristics

21 CH1 Introduction Copyright © 2023 by Wiley India Pvt. Ltd.

 Fig 1.6 Interaction between different components in the three generations of NLP
22 CH1 Introduction Copyright © 2023 by Wiley India Pvt. Ltd.
Scope of the Book

• This book takes a historical perspective of NLP via the three generations when
describing the foundations and applications of NLP

• Representations of NLP is introduced in the next chapters

• Several problems in NLP such as POS tagging, parsing, natural language inference,
sentiment analysis, question-answering is discussed

• It introduces resources, business cases, linguistic foundations, and research avenues

to account for the applications, foundations, and innovations that drive today’s NLP

23 CH1 Introduction Copyright © 2023 by Wiley India Pvt. Ltd.

 Thank you

23 CH1 Introduction Copyright © 2023 by Wiley India Pvt. Ltd.