An Introduction to Natural Language Processing
Download as PPTX, PDF0 likes323 views
Learn about how Natural Language Processing in AI can be used and how it applies to you in the real world. You can learn about NLP concepts, Pre-processing steps, Vectorization Methods, Generative and Unsupervised methods. All the resource is available for you to grow your knowledge and skills about Natural Language Processing webinar!
![[AAAI 2019 tutorial] End-to-end goal-oriented question answering systems](https://cdn.slidesharecdn.com/ss_thumbnails/aaai2019tutorialend-to-endgoal-orientedquestionansweringsystems-190128122117-thumbnail.jpg?width=560&fit=bounds)
![[KDD 2018 tutorial] End to-end goal-oriented question answering systems](https://cdn.slidesharecdn.com/ss_thumbnails/kdd2018tutorialend-to-endgoal-orientedquestionansweringsystems-180821081440-thumbnail.jpg?width=560&fit=bounds)
Ad
More Related Content
What's hot (18)
Similar to An Introduction to Natural Language Processing (20)
Ad
More from Tyrone Systems (20)
Ad
Recently uploaded (20)
An Introduction to Natural Language Processing
- 1. An Introduction to Natural Language Processing Monday | 1st June, 2020 LIVE WEBINAR Presented by
- 2. Agenda 1. About Tyrone • World’s high performing AI platform system – A100 • Get Development, Training, Inference in one • New Tyrone Launches • Tyrone Kubyts™ 2. Introduction to NLP Concepts • Why NLP Is required • Significance of using NLP • Source of Text Data 3. NLP Pre-processing Steps • Overview of Pre-processing steps in NLP • Prepare the data for Vectorization 4. NLP Vectorization Methods • Simple to advanced methods 5. Generative and Unsupervised Methods in NLP • Topic Modelling • Unsupervised techniques to analyse the text data 6. Case Studies • Topic modelling to identify the important topics in the discussion • Customer sentiment analysis
- 3. Tyrone Systems at a Glance
- 4. NVIDIA HGX A100 PERFORMANCE New Tensor Core for AI & HPC New Multi-instance GPU New Hardware Engines Increase in GPU interconnect bandwidth Increase in GPU memory Increase in memory bandwidth Speedup in AI performance
- 5. 54 Billion XTORS 3rd Gen Tensor cores Sparsity Acceleration Multi Instance GPU 3rd GEN NVLINK & NVSwitch
- 6. NVIDIA A100 Greatest Generational Leap – 20X Volta 54B XTOR | 826mm2 | TSMC 7N | 40GB Samsung HBM2 | 600 GB/s NVLink Peak Vs Volta FP32 TRAINING 312 TFLOPS 20X INT8 INFERENCE 1,248 TOPS 20X FP64 HPC 19.5 TFLOPS 2.5X MULTI INSTANCE GPU 7X GPUs
- 7. New tf32 tensor cores on A100 20X Higher FLOPS for AI, Zero Code Change 20X Faster than Volta FP32 | Works like FP32 for AI with Range of FP32 and Precision of FP16 No Code Change Required for End Users | Supported on PyTorch, TensorFlow and MXNet Frameworks Containers
- 8. Most flexible ai platform with MULTI-INSTANCE GPU (MIG) Optimize GPU Utilization, Expand Access to More Users with Guaranteed Quality of Service Up To 7 GPU Instances In a Single A100: Simultaneous Workload Execution With Guaranteed Quality Of Service: All MIG instances run in parallel with predictable throughput & latency Flexibility to run any type of workload on a MIG instance Right Sized GPU Allocation: Different sized MIG instances based on target workloads Amber GPU Mem GPU GPU Mem GPU GPU Mem GPU GPU Mem GPU GPU Mem GPU GPU Mem GPU GPU Mem GPU
- 9. ONE SYSTEM FOR ALL ai INFRASTRUCTURE AI Infrastructure Re-Imagined, Optimized, and Ready for Enterprise AI-at-Scale any job | any size | any node | anytime Analytics Training Inference Flexible AI infrastructure that adapts to the pace of enterprise • One universal building block for the AI data center • Uniform, consistent performance across the data center • Any workload on any node - any time • Limitless capacity planning with predictably great performance with scale
- 10. Game-changing performance for innovators 9x Mellanox ConnectX-6 200Gb/s Network Interface 450GB/sec Peak Bi-directional Bandwidth Dual 64-core AMD Rome CPUs and 1TB RAM 3.2X More Cores to Power the Most Intensive AI Jobs 8x NVIDIA A100 GPUs with 320GB Total GPU Memory 12 NVLinks/GPU 600GB/sec GPU-to-GPU Bi-directional Bandwidth 15TB Gen4 NVME SSD 4.8TB/sec Bi-directional Bandwidth 2X More than Previous Generation NVSwitch 6x NVIDIA NVSwitches 25GB/sec Peak Bandwidth 2X Faster than Gen3 NVME SSDs
- 11. 2U GPU server up to 4 NVIDIA HGX ™ A100 GPU Camarero DAS7TGVQ-24RT Tyrone NVIDIA A100 based SERVERS • Supports 4x A100 40GB SXM4 GPUs • Supports CPU TDP up to 280W • Dual AMD EPYC™ 7002 Series Processors in up to 128 Cores • Flexible storage with 4 hotswap for SAS, SATA or NVMe • PCI-E Gen 4 NVLink for fast GPU-GPU connections • 32 DIMM Slots that allow up to 8TB of 3200Mhz DDR4 memory • 4 Hot-swap heavy duty fans • 2x 2200W Redundant Power Supplies, Titanium Level PCI-E Gen 4 NEW LAUNCH NVIDIA NVLink
- 12. 4U GPU server up to 8 NVIDIA HGX ™ A100 GPU Tyrone NVIDIA A100 based SERVERS NVIDIA NVLink & NVSwitch NEW LAUNCH • Supports up to 8 double-width GPUs, • Supports CPU TDP up to 280W • Dual AMD EPYC™ 7002 Series Processors in up to 128 Cores • Flexible storage with 4 hotswap for SAS, SATA or NVMe • PCI-E Gen 4 NVLink for fast GPU-GPU connections • 32 DIMM Slots that allow up to 8TB of 3200Mhz DDR4 memory • 4 Hot-swap heavy duty fans • 2x 2000W Redundant Power Supplies, Titanium Level
- 13. 4U GPU server up to 8 NVIDIA HGX ™ A100 GPU Tyrone NVIDIA A100 based SERVERS NVIDIA NVLink COMING SOON • Supports Intel Xeon • Supports NVLink • 8 x NVIDIA Tesla A100 SXM4
- 14. Delivers 4XFASTER TRAINING than other GPU-based systems Your Personal AI Supercomputer Power-on to Deep Learning in Minutes Pre-installed with Powerful Deep Learning Software Extend workloads from your Desk-to-Cloud in Minutes
- 15. Run Multiple Applications simultaneously Tyrone KUBYTS™ Cloud Flow Architecture Revolutionizing Deep Learning CPU-GPU Environment KUBYTS™ Compatible Workstations WITH TYRONE KUBYTS™ CLIENT KUBYTS™ has a repository of : 50 containerized applications 100s of Containers 10X20X30X40X50X60X70X SPEED
- 16. Agenda 1. About Tyrone • World’s high performing AI platform system – A100 • Get Development, Training, Inference in one • New Tyrone Launches • Tyrone Kubyts™ 2. Introduction to NLP Concepts • Why NLP Is required • Significance of using NLP • Source of Text Data 3. NLP Pre-processing Steps • Overview of Pre-processing steps in NLP • Prepare the data for Vectorization 4. NLP Vectorization Methods • Simple to advanced methods 5. Generative and Unsupervised Methods in NLP • Topic Modelling • Unsupervised techniques to analyse the text data 6. Case Studies • Topic modelling to identify the important topics in the discussion • Customer sentiment analysis
- 17. What is NLP NLP is a branch of data science that consists of systematic processes for analyzing, understanding, and deriving information from the text data in a smart and efficient manner. By utilizing NLP and its components, one can organize the massive chunks of text data, perform numerous automated tasks and solve a wide range of problems such as – automatic summarization, machine translation, named entity recognition, relationship extraction, sentiment analysis, speech recognition, and topic segmentation etc. Lexical Analysis Syntactic Analysis (Parsing) Semantic Analysis Discourse integration Pragmatic Analysis • It involves identifying and analyzing the structure of words. • Lexical analysis is dividing the whole chunk of txt into paragraphs, sentences, and words. • It involves analysis of words in the sentence for grammar and arranging words in a manner that shows the relationship among the words. • It draws the exact meaning or the dictionary meaning from the text. The text is checked for meaningfulness. • Discourse integration is considered as the larger context for any smaller part of NL structure. NL is so complex and, most of the time, sequences of text are dependent on prior discourse. • It involves deriving those aspects of language which require real world knowledge.
- 18. Source of Raw Text Any Source of free text Web pages Chat contents / Chatbot Twitter / Social Media Feedback comments Movie reviews Product Descriptions Citations Customer Complaints News Feeds
- 19. Important Libraries for NLP (python) • Scikit-learn: Machine learning in Python • Natural Language Toolkit (NLTK): The complete toolkit for all NLP techniques. • Pattern : A web mining module for the with tools for NLP and machine learning. • TextBlob : Easy to use nlp tools API, built on top of NLTK and Pattern. • SpaCy : Industrial strength N LP with Python and Cython. • Gensim : Topic Modelling • Stanford Core NLP : NLP services and packages by Stanford NLP Group.
- 20. Text Processing • Since, text is the most unstructured form of all the available data, various types of noise are present in it and the data is not readily analyzable without any pre-processing. The entire process of cleaning and standardization of text, making it noise-free and ready for analysis is known as text preprocessing. • It is predominantly comprised of three steps: • Noise Removal • Lexicon Normalization • Object Standardization Raw Text Stop words , URLS , Punctuation , Mentions etc Lemmatization , Stemming Spell Checks Word Standardization Clean Text
- 21. Removing Stop Words/Punctuations etc The process of converting data to something a computer can understand is referred to as pre-processing. One of the major forms of pre-processing is to filter out useless data. In natural language processing, useless words (data), are referred to as stop words. What are Stop words? A stop word is a commonly used word (such as “the”, “a”, “an”, “in”)
- 22. Stemming and Lemmatization • There are two kinds of suffixes that we will consider removing. • Inflectional: "cats", "calling", "quickest" • Derivational: "realize", "hopeless", "requirement" • Stemming and lemmatization (sometimes called Soft Stemming) are text marking processes that mark segments (almost always tokens) with canonical or reduced forms. In natural language processing (NLP) it is generally used to combine words with common lemmas or stems when the meaning of the inflection or derivation is not important to the problem at hand. • The two techniques differ in what kind of word is returned, as well as implementation. A stem is not necessarily a "word", it can be part of a word like "includ" for "include" and "including". A lemma is the head-word of a dictionary entry. In English, stemming is usually implemented with a sequence of transformation rules, and lemmatization is implemented with a mapping from word to lemma. For example, runs, running, ran are all forms of the word run, therefore run is the lemma of all these words. Because lemmatization returns an actual word of the language, it is used where it is necessary to get valid words.
- 23. Part of Speech Tagging • POS tagging is the process of marking up a word in a corpus to a corresponding part of a speech tag, based on its context and definition. • Most POS are divided into sub-classes. POS Tagging simply means labeling words with their appropriate Part-Of-Speech. • POS tagging is a supervised learning solution that uses features like the previous word, next word, is first letter capitalized etc. NLTK has a function to get pos tags and it works after tokenization process.
- 24. Vector representation of Text To use a machine learning algorithm or a statistical technique on any form of text, it is prescribed to transform the text into some numeric or vector representation. This numeric representation should depict significant characteristics of the text. There are many such techniques, for example, occurrence, term-frequency, TF-IDF, word co-occurrence matrix, word2vec and GloVe.
- 25. Bag of Words • A bag-of-words model, or BoW for short, is a way of extracting features from text for use in modeling, such as with machine learning algorithms. • The approach is very simple and flexible • A bag-of-words is a representation of text that describes the occurrence of words within a document. It involves two things: • A vocabulary of known words. • A measure of the presence of known words. It was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness, “best” “times” “worst” “age” “wisdom” “foolishness” “best” = 1 “times” = 1 “worst” = 0 “age” = 0 “wisdom” = 0 “foolishness” = 0
- 26. TF – IDF Vectorization TF-IDF is an abbreviation for Term Frequency-Inverse Document Frequency and is a very common algorithm to transform text into a meaningful representation of numbers. The technique is widely used to extract features across various NLP applications • TF: Term Frequency, which measures how frequently a term occurs in a document. Since every document is different in length, it is possible that a term would appear much more times in long documents than shorter ones. Thus, the term frequency is often divided by the document length (aka. the total number of terms in the document) as a way of normalization: • TF(t) = (Number of times term t appears in a document) / (Total number of terms in the document). • IDF: Inverse Document Frequency, which measures how important a term is. While computing TF, all terms are considered equally important. However it is known that certain terms, such as "is", "of", and "that", may appear a lot of times but have little importance. Thus we need to weigh down the frequent terms while scale up the rare ones, by computing the following: • IDF(t) = log_e(Total number of documents / Number of documents with term t in it). Consider a document containing 100 words wherein the word cat appears 3 times. The term frequency (i.e., tf) for cat is then (3 / 100) = 0.03. Now, assume we have 10 million documents and the word cat appears in one thousand of these. Then, the inverse document frequency (i.e., idf) is calculated as log(10,000,000 / 1,000) = 4. Thus, the Tf-idf weight is the product of these quantities: 0.03 * 4 = 0.12.
- 27. Topic Modelling • it is a process to automatically identify topics present in a text object and to derive hidden patterns exhibited by a text corpus. • Topic Modelling is different from rule-based text mining approaches that use regular expressions or dictionary based keyword searching techniques. It is an unsupervised approach used for finding and observing the bunch of words (called “topics”) in large clusters of texts. • Topics can be defined as “a repeating pattern of co-occurring terms in a corpus”. A good topic model should result in – “health”, “doctor”, “patient”, “hospital” for a topic – Healthcare, and “farm”, “crops”, “wheat” for a topic – “Farming”. • Topic Models are very useful for the purpose for document clustering, organizing large blocks of textual data, information retrieval from unstructured text and feature selection. For Example – New York Times are using topic models to boost their user – article recommendation engines.
- 28. Word Embedding • Word Embedding is a language modeling technique used for mapping words to vectors of real numbers. It represents words or phrases in vector space with several dimensions. Word embeddings can be generated using various methods like neural networks, co-occurrence matrix, probabilistic models, etc. • Word2Vec consists of models for generating word embedding. These models are shallow two layer neural networks having one input layer, one hidden layer and one output layer. Word2Vec utilizes two architectures
- 29. Advanced Language Models and Tranformers ELMo ULMFit BERT Transformer
- 30. Case Studies • Using Topic Modeling technique to identify underlying topics in the corpus • Customer sentiment analysis
- 31. Q&A Session Hirdey Vikram [email protected] India (North) Niraj [email protected] India (South) Vivek [email protected] India (East) Navin [email protected] India (West) Anupriya [email protected] Singapore Arun [email protected] UAE Agam [email protected] Indonesia Contact our team if you have any further questions after this webinar [email protected] to our AI Experts