Introduction to Named Entity Recognition
- 1. Named Entity Recognition Tomer Lieber Moti Goldklang 5.5.2020
- 2. What is Named Entity Recognition (NER)? ● A common task of Natural Language Processing (NLP) ● Find and classify entities in text into predefined categories ● Popular categories are person names, organizations and locations ● Usages: machine translation, information retrieval, question-answering...
- 3. Example Paris Hilton is an American singer that borned in New York 20 years ago. Or
- 4. Another Example Danny bought a chocolate snack of Mars yesterday in Tel Aviv. Or
- 5. How to solve Named Entity Recognition task? ● Statistical learning models: Maximum Entropy model, Hidden Markov Models. ● Machine Learning models: Support Vector Machines, Voted perceptron. ● Deep Learning models: Recurrent neural network (RNN), Long short term Memory (LSTM), Bidirectional LSTM ● Tokens as words or letters
- 6. The Classes (labels) ● IO encoding (PER, LOC, OTHER) Alex is going to Los Angeles ● IOB encoding (B-PER, I-PER, B-LOC, I-LOC, OTHER) Alex is going to Los Angeles PER O O O LOC LOC B-PER O O O B-LOC I-LOC
- 7. Traditional Machine Learning ● Feature extraction ○ Word length ○ Location in the sentence ○ Previous/next word ○ Previous word label ○ Linguistics ○ Substrings ○ Regular expressions matches ● Train/Test via “traditional” models (usually trees)
- 9. Neural Networks & LSTM ● Feature extraction? ● Embeddings ● LSTM
- 10. The evaluation method - F1 score ● β is chosen such that recall is considered β times as important as precision. ● precision is the percentage of named entities found by the learning system that are correct ● recall is the percentage of named entities present in the corpus that are found by the system.
- 11. The evaluation problem First Bank Chicago announced an important message last week...
- 12. The CoNLL-2003 Shared Task ● An academic conference held a competition to find a perfect method to detect and classify entities in a given text (english and german). ● Sixteen groups have participated in the competition. ● Each group received a training file, a development file, a test file and a large file with unannotated data (Reuters news stories and a German newspaper). ● They employed a wide variety of machine learning techniques as well as system combination. ● The performance in this task was measured by a variant of the F1 score.
- 13. The CoNLL-2003 Shared Task - Results
- 14. The CoNLL-2003 Shared Task - Progress
- 15. References ● NLP Progress - Named Entity Recognition ● Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition ● Contextual String Embeddings for Sequence Labeling ● F1 Score ● Tel Aviv Meetup: Deep Learning for Named Entity Recognition - Kfir Bar
- 16. Questions?
Editor's Notes
- #4: Classify entity and not token: put attention that we classify Paris Hilton together as a one entity and not as two token because we want to hold the connection between the tokens.
- #12: The measure behave a bit weird when there are boundary errors (which are common). This counts as both fp a fn, therefore select nothing would have been better. There are some other methods like MUC that give partial credit according to complex rules, but it also has its disadvantages.
- #13: The categories are person names, organizations, locations and others. The shared task organizers were especially interested in approaches that made use of resources other than the supplied training data, for example gazetteers and unannotated data The learning methods were trained with the training data. The development data could be used for tuning the parameters of the learning methods.