What is Natural Language Processing?

| IBM 24/11/2022 09:18

Skip to content
Cloud Products Contact
Solutions
us: Log inPricing
: https://cloud.ibm.com/login
Docs Support

IBM Cloud Learn Hub What is Natural Language Processing?

Natural Language Processing

(NLP)

By: IBM Cloud Education

2 July 2020

Artificial intelligence

What is
natural
Natural Featured
products
language
processing? Language IBM
Watson

NLP tasks Processing Assistant

NLP tools
and
(NLP) IBM
Watson
Discovery
approaches

Natural language
processing strives
https://www.ibm.com/cloud/learn/natural-language-processing Page 1 sur 17
What is Natural Language Processing? | IBM 24/11/2022 09:18

NLP use to build machines

cases
that understand
Natural and respond to
language
processing
text or voice data—
and IBM and respond with
Watson
text or speech of
their own—in much
the same way
humans do.

What is
natural
language
processing?
Natural language processing
(NLP) refers to the branch of
computer science—and more
specifically, the branch of
artificial intelligence or AI—
concerned with giving
computers the ability to
understand text and spoken
https://www.ibm.com/cloud/learn/natural-language-processing Page 2 sur 17
What is Natural Language Processing? | IBM 24/11/2022 09:18

words in much the same way

human beings can.

NLP combines computational

linguistics—rule-based modeling
of human language—with
statistical, machine learning, and
deep learning models. Together,
these technologies enable
computers to process human
language in the form of text or
voice data and to ‘understand’
its full meaning, complete with
the speaker or writer’s intent
and sentiment.

NLP drives computer programs

that translate text from one
language to another, respond to
spoken commands, and
summarize large volumes of text
rapidly—even in real time.
There’s a good chance you’ve
interacted with NLP in the form
of voice-operated GPS systems,
digital assistants, speech-to-text
dictation software, customer
service chatbots, and other
consumer conveniences. But
NLP also plays a growing role in
enterprise solutions that help
streamline business operations,
increase employee productivity,
and simplify mission-critical
business processes.

https://www.ibm.com/cloud/learn/natural-language-processing Page 3 sur 17

What is Natural Language Processing? | IBM 24/11/2022 09:18

NLP tasks
Human language is filled with
ambiguities that make it
incredibly difficult to write
software that accurately
determines the intended
meaning of text or voice data.
Homonyms, homophones,
sarcasm, idioms, metaphors,
grammar and usage exceptions,
variations in sentence structure
—these just a few of the
irregularities of human language
that take humans years to learn,
but that programmers must
teach natural language-driven
applications to recognize and
understand accurately from the
start, if those applications are
going to be useful.

Several NLP tasks break down

human text and voice data in
ways that help the computer
make sense of what it's
ingesting. Some of these tasks
include the following:

– Speech recognition, also

called speech-to-text, is the
task of reliably converting
voice data into text data.
Speech recognition is

https://www.ibm.com/cloud/learn/natural-language-processing Page 4 sur 17

What is Natural Language Processing? | IBM 24/11/2022 09:18

required for any application

that follows voice
commands or answers
spoken questions. What
makes speech recognition
especially challenging is the
way people talk—quickly,
slurring words together, with
varying emphasis and
intonation, in different
accents, and often using
incorrect grammar.

– Part of speech tagging, also

called grammatical tagging,
is the process of determining
the part of speech of a
particular word or piece of
text based on its use and
context. Part of speech
identifies ‘make’ as a verb in
‘I can make a paper plane,’
and as a noun in ‘What make
of car do you own?’

– Word sense disambiguation

is the selection of the
meaning of a word with
multiple meanings  through
a process of semantic
analysis that determine the
word that makes the most
sense in the given context.
For example, word sense
disambiguation helps
distinguish the meaning of
the verb 'make' in ‘make the
grade’ (achieve) vs. ‘make a

https://www.ibm.com/cloud/learn/natural-language-processing Page 5 sur 17

What is Natural Language Processing? | IBM 24/11/2022 09:18

bet’ (place).

– Named entity
recognition, or NEM,
identifies words or phrases
as useful entities. NEM
identifies ‘Kentucky’ as a
location or ‘Fred’ as a man's
name.

– Co-reference resolution is
the task of identifying if and
when two words refer to the
same entity. The most
common example is
determining the person or
object to which a certain
pronoun refers (e.g., ‘she’ =
‘Mary’),  but it can also
involve identifying a
metaphor or an idiom in the
text  (e.g., an instance in
which 'bear' isn't an animal
but a large hairy person).

– Sentiment
analysis attempts to extract
subjective qualities—
attitudes, emotions,
sarcasm, confusion,
suspicion—from text.

– Natural language
generation is sometimes
described as the opposite of
speech recognition or
speech-to-text; it's the task
of putting structured

https://www.ibm.com/cloud/learn/natural-language-processing Page 6 sur 17

What is Natural Language Processing? | IBM 24/11/2022 09:18

information into human

language. 

See the blog post “NLP vs. NLU

vs. NLG: the differences between
three natural language
processing concepts” for a
deeper look into how these
concepts relate.

NLP tools
and
approaches
Python and the Natural
Language Toolkit (NLTK)

The Python programing language

provides a wide range of tools
and libraries for attacking
specific NLP tasks. Many of
these are found in the Natural
Language Toolkit, or NLTK, an
open source collection of
libraries, programs, and
education resources for building
NLP programs.

The NLTK includes libraries for

https://www.ibm.com/cloud/learn/natural-language-processing Page 7 sur 17

What is Natural Language Processing? | IBM 24/11/2022 09:18

many of the NLP tasks listed

above, plus libraries for
subtasks, such as sentence
parsing, word segmentation,
stemming and lemmatization
(methods of trimming words
down to their roots), and
tokenization (for breaking
phrases, sentences, paragraphs
and passages into tokens that
help the computer better
understand the text). It also
includes libraries for
implementing capabilities such
as semantic reasoning, the
ability to reach logical
conclusions based on facts
extracted from text.

Statistical NLP, machine

learning, and deep
learning
The earliest NLP applications
were hand-coded, rules-based
systems that could perform
certain NLP tasks, but couldn't
easily scale to accommodate a
seemingly endless stream of
exceptions or the increasing
volumes of text and voice data.

Enter statistical NLP, which

combines computer algorithms
with machine learning and deep
learning models to automatically
extract, classify, and label
elements of text and voice data

https://www.ibm.com/cloud/learn/natural-language-processing Page 8 sur 17

What is Natural Language Processing? | IBM 24/11/2022 09:18

and then assign a statistical

likelihood to each possible
meaning of those elements.
Today, deep learning models and
learning techniques based on
convolutional neural networks
(CNNs) and recurrent neural
networks (RNNs) enable NLP
systems that 'learn' as they work
and extract ever more accurate
meaning from huge volumes of
raw, unstructured, and
unlabeled text and voice data
sets. 

For a deeper dive into the

nuances between these
technologies and their learning
approaches, see “AI vs. Machine
Learning vs. Deep Learning vs.
Neural Networks: What’s the
Difference?”

NLP use
cases
Natural language processing is
the driving force behind machine
intelligence in many modern
real-world applications. Here are

https://www.ibm.com/cloud/learn/natural-language-processing Page 9 sur 17

What is Natural Language Processing? | IBM 24/11/2022 09:18

a few examples:

– Spam detection: You may

not think of spam detection
as an NLP solution, but the
best spam detection
technologies use NLP's text
classification capabilities to
scan emails for language
that often indicates spam or
phishing. These indicators
can include overuse of
financial terms,
characteristic bad grammar,
threatening language,
inappropriate urgency,
misspelled company names,
and more. Spam detection is
one of a handful of NLP
problems that experts
consider 'mostly solved'
(although you may argue
that this doesn’t match your
email experience).

– Machine translation: Google
Translate is an example of
widely available NLP
technology at work. Truly
useful machine translation
involves more than replacing
words in one language with
words of another.  Effective
translation has to capture
accurately the meaning and
tone of the input language
and translate it to text with
the same meaning and

https://www.ibm.com/cloud/learn/natural-language-processing Page 10 sur 17

What is Natural Language Processing? | IBM 24/11/2022 09:18

desired impact in the output

language. Machine
translation tools are making
good progress in terms of
accuracy. A great way to test
any machine translation tool
is to translate text to one
language and then back to
the original. An oft-cited
classic example: Not long
ago, translating “The spirit is
willing but the flesh is weak”
from English to Russian and
back yielded “The vodka is
good but the meat is rotten.”
Today, the result is “The
spirit desires, but the flesh is
weak,” which isn’t perfect,
but inspires much more
confidence in the English-to-
Russian translation.

– Virtual agents and

chatbots: Virtual agents
such as Apple's Siri and
Amazon's Alexa use speech
recognition to recognize
patterns in voice commands
and natural language
generation to respond with
appropriate action or helpful
comments. Chatbots
perform the same magic in
response to typed text
entries. The best of these
also learn to recognize
contextual clues about
human requests and use

https://www.ibm.com/cloud/learn/natural-language-processing Page 11 sur 17

What is Natural Language Processing? | IBM 24/11/2022 09:18

them to provide even better

responses or options over
time. The next enhancement
for these applications is
question answering, the
ability to respond to our
questions—anticipated or
not—with relevant and
helpful answers in their own
words.

– Social media sentiment

analysis: NLP has become
an essential business tool
for uncovering hidden data
insights from social media
channels. Sentiment
analysis can analyze
language used in social
media posts, responses,
reviews, and more to extract
attitudes and emotions in
response to products,
promotions, and events–
information companies can
use in product designs,
advertising campaigns, and
more.

– Text summarization: Text

summarization uses NLP
techniques to digest huge
volumes of digital text and
create summaries and
synopses for indexes,
research databases, or busy
readers who don't have time
to read full text. The best

https://www.ibm.com/cloud/learn/natural-language-processing Page 12 sur 17

What is Natural Language Processing? | IBM 24/11/2022 09:18

text summarization
applications use semantic
reasoning and natural
language generation (NLG)
to add useful context and
conclusions to summaries.

Natural
language
processing
and IBM
Watson
– IBM has innovated in the
artificial intelligence space
by pioneering NLP-driven
tools and services that
enable organizations to
automate their complex
business processes while
gaining essential business
insights. These tools include:

▪ Watson Discovery -
Surface high-quality
answers and rich insights

https://www.ibm.com/cloud/learn/natural-language-processing Page 13 sur 17

What is Natural Language Processing? | IBM 24/11/2022 09:18

from your complex

enterprise documents -
tables, PDFs, big data
and more - with AI
search. Enable your
employees to make more
informed decisions and
save time with real-time
search engine and text
mining capabilities that
perform text extraction
and analyze relationships
and patterns buried in
unstructured data.
Watson Discovery
leverages custom NLP
models and machine
learning methods to
provide users with AI that
understands the unique
language of their industry
and business. Explore
Watson Discovery

▪ Watson Natural Language

Understanding (NLU) -
Analyze text in
unstructured data
formats including HTML,
webpages, social media,
and more. Increase your
understanding of human
language by leveraging
this natural language tool
kit to identify concepts,
keywords, categories,
semantics, and emotions,
and to perform text

https://www.ibm.com/cloud/learn/natural-language-processing Page 14 sur 17

What is Natural Language Processing? | IBM 24/11/2022 09:18

classification, entity
extraction, named entity
recognition (NER),
sentiment analysis, and
summarization. Explore
Watson Natural Language
Understanding

▪ Watson Assistant -
Improve the customer
experience while
reducing costs. Watson
Assistant is an AI chatbot
with an easy-to-use
visual builder so you can
deploy virtual agents
across any channel, in
minutes.  Explore Watson
Assistant

▪ Purpose-built for
healthcare and life
sciences domains, IBM
Watson Annotator for
Clinical Data extracts key
clinical concepts from
natural language text,
like conditions,
medications, allergies
and procedures. Deep
contextual insights and
values for key clinical
attributes develop more
meaningful data.
Potential data sources
include clinical notes,
discharge summaries,
clinical trial protocols

https://www.ibm.com/cloud/learn/natural-language-processing Page 15 sur 17

What is Natural Language Processing? | IBM 24/11/2022 09:18

and literature data.

– For more information on how

to get started with one of
IBM Watson's natural
language processing
technologies, visit the IBM
Watson Natural Language
Processing page. 

Sign up for an IBMid and create

your IBM Cloud account.

https://www.ibm.com/cloud/learn/natural-language-processing Page 16 sur 17

What is Natural Language Processing? | IBM 24/11/2022 09:18

Why IBM Cloud Products and Learn about  Resources

Solutions
Why IBM Cloud What is Hybrid Cloud? Get started

Hybrid Cloud approachCloud Paks What is Cloud Docs

Trust and security Cloud pricing Computing? Architectures

Open Cloud View all products What is Confidential IBM Garage

View all solutions Computing?
Data centers Training and
What is a Data Lake? Certifications
Case studies
What is a Data Partners
Warehouse?
Cloud blog
What is Artificial
Hybrid Cloud careers
Intelligence (AI)?
My Cloud account
What is Machine
Learning?

What is DevOps?

What is Microservices?

Let’s talk

https://www.ibm.com/cloud/learn/natural-language-processing Page 17 sur 17