Scribd
Upload
2 views

Let's Learn NLP in 5 minutes (Part 7)

The document provides an overview of Word2Vec, a technique that transforms words into vectors to capture their meanings and relationships. It describes two primary models of Word2Vec: Continuous Bag of Words (CBOW), which predicts a target word from context words, and Skip-Gram, which predicts context words from a target word. The document also highlights the differences between these models in terms of their objectives, focus, speed, and data requirements.

Uploaded by

Nirranjan J
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
2 views

Let's Learn NLP in 5 minutes (Part 7)

The document provides an overview of Word2Vec, a technique that transforms words into vectors to capture their meanings and relationships. It describes two primary models of Word2Vec: Continuous Bag of Words (CBOW), which predicts a target word from context words, and Skip-Gram, which predicts context words from a target word. The document also highlights the differences between these models in terms of their objectives, focus, speed, and data requirements.

Uploaded by

Nirranjan J
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 8

Manidweep Sharma Save Post

Let’s Learn
NLP
in 5 minutes
PART 7

Swipe for more


Manidweep Sharma Save Post

Word2Vec

"king" "queen" "king" "dog"

Vectors nearby Vectors far apart

Word2Vec turns words into compact number patterns


(vectors) that capture their meanings and how they relate to
each other.

Unlike One-Hot Encoding or Bag of Words, Word2Vec


understands how words relate to each other in context.

Swipe for more


Types of Word2Vec
Word2Vec has two primary models that generate word
embeddings, each with its unique way of training and
understanding context.

The following are the two types:

1. Continuous Bag of Words (CBOW) Model


2. Skip-Gram Model

Word2Vec

CBOW Skip-Gram
Predict the target word using its Predict the surrounding (context)
surrounding (context) words. words given a target word.

Swipe for more


Manidweep Sharma Save Post

Continuous Bag of Words


(CBOW)
The model takes a group of words (the context) and predicts the
target word that is most likely to fit in that context.

How CBoW works?


Input:
A set of context words within a specified window size (e.g., words
surrounding the target word in a sentence).
Output:
The central word (target word) is predicted based on these context
words.

Example:
Sentence: "I love learning NLP every day."

Output/Target word: "learning"

Window size = 2 (meaning 2 words before and 2 words after target word)

Context words for "learning" with window size = 2:

["I", "love", "NLP", "every"]


["I", "love", "NLP", "every"] learning
predicts

Swipe for more


Manidweep Sharma Save Post

Skip-Gram Model
The Skip-gram model works in the opposite way to CBOW. It takes a
target word as input and predicts the context words that are most
likely to surround it.

How Skip-Gram works?


Input:
A single target word (the word for which context words are
predicted).
Output:
The context words (words surrounding the target word in a specific
window size) are predicted.

Example:
Sentence: "I love learning NLP every day."

Output/Context words: ["I", "love", "NLP", "every"]

Input/Target word : “learning”

learning ["I", "love", "NLP", "every"]


predicts

Swipe for more


Manidweep Sharma Save Post

Differences between
Skip-Gram and CBOW

Feature Skip-Gram CBOW

Objective Predict context words Predict the target word

Focus Works well for rare words Works well for frequent words

Speed Slower Faster

Data Requirement Smaller datasets Larger datasets

Swipe for more


Manidweep Sharma Save Post

Implementation
CBOW Implementation:

Skip-Gram Implementation:

Swipe for more


Manidweep Sharma

If you
find this
helpful, please
like and share
it with your
friends
for PART 8

@manidweepsharma

You might also like