Natural Language Processing glossary for Coders
1 like357 views
If you are getting started with Natural Language Processing, these slides will give you a basic understanding of the capabilities of NLP.
Ad
More Related Content
What's hot (20)
Viewers also liked (20)
Ad
Similar to Natural Language Processing glossary for Coders (20)
Ad
Recently uploaded (20)
Natural Language Processing glossary for Coders
- 1. NATURAL LANGUAGE PROCESSING 10 KEY CONCEPTS FOR CODERS
- 2. 1. SENTENCE SEGMENTATION • Sentence segmentation is the process of splitting a document of text into individual sentences.
- 3. 2. TOKENIZATION • Tokenization is the process of splitting a document into individual words. • Tokenization and sentence splitting usually go hand in hand. • For example, Stanford’s CoreNLP expects tokenization to be done before sentence segmentation.
- 4. 3. PART OF SPEECH TAGGING • Part of speech (POS) tagging inspects your text and decides if the individual words are nouns, adjectives, verbs, adverbs etc. There are a lot of POS tags (over 35 according to this list on StackOverflow). • http://stackoverflow.com/a/1833718/7337349
- 5. 4. STEMMING • Stemming is the process of getting the “root” from a word • For example, the words organize, organized and organizing are all derived from organize, and when you do a search for the word organize, you would expect to also get the other forms of the word since they represent the same idea. • A very important thing to note (especially if you end up using stemming in your NLP projects) is that the stem of a word does not have to be and often isn’t a dictionary word. • E.g the stem of the word “saw” is just “s”.
- 6. 5. LEMMATIZATION • Lemmatization is similar to stemming in that it tries to get the root of a word, except that it tries to regularize the word to end up with a dictionary word. • E.g. the lemma of the word “saw” is either “see” or “saw” based on whether the token was a verb or a noun.
- 7. 6. NAMED ENTITY RECOGNITION • Named Entity Recognition or NER is simply the process of extracting nouns from your text. • For example, using NER, you could automatically detect all the occurrences of a brand name in a person’s Twitter feed.
- 8. 7. PARSE TREES • Syntax or constituent parse trees • Is concerned with how words combine to form constituents of a sentence • Dependency parse tree • Is concerned with the relationship between the words in a sentence Source: http://www.nltk.org/book/ch08.html
- 9. 8. COREFERENCE RESOLUTION • Coreference occurs when two or more expressions in a text refer to the same person or thing. • For example, in the sentence “Bill said he would come”, the word “he” refers to Bill. • Coreference resolution is the ability to resolve the co-reference to find what it is referring to.
- 10. 9. POLARITY DETECTION • This is a fancy term for deciding whether a piece of text conveys a positive or a negative sentiment. Imagine if you are writing a program for figuring out whether a tweet says something positive or negative about a brand.
- 11. 10. INFORMATION EXTRACTION • Information Extraction is the process of extracting facts (about the world) from text information. • For example, if you saw the sentence “Nigeria is a country in Africa” you should be able to answer the question “India is a country in ______”.
- 12. TO LEARN MORE ABOUT NATURAL LANGUAGE PROCESSING http://www.miningbusinessdata.com