HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY

Part-Of-Speech Tagging, Sequence Labeling

Lecturer: Dr. Bùi Thanh Hùng

Data Science Department
Faculty of Information Technology
Industrial University of Ho Chi Minh city
Email: [email protected]
Website: https://sites.google.com/site/hungthanhbui1980/
 Acknowledgement
• The contents of these slides have origin from
University of Texas at Austin
• We greatly appreciate support from Prof. Raymond J.
Mooney for kindly sharing these materials.

2
 Part Of Speech Tagging

• Annotate each word in a sentence with a part-of-

speech marker.
• Lowest level of syntactic analysis.

John saw the saw and decided to take it to the table.

NNP VBD DT NN CC VBD TO VB PRP IN DT NN
• Useful for subsequent syntactic parsing and word
sense disambiguation.

3
 English POS Tagsets
• Original Brown corpus used a large set of 87 POS
tags.
• Most common in NLP today is the Penn Treebank set
of 45 tags.
– Tagset used in these slides.
– Reduced from the Brown set for use in the context of a
parsed corpus (i.e. treebank).
• The C5 tagset used for the British National Corpus
(BNC) has 61 tags.

4
 English Parts of Speech

• Noun (person, place or thing)

– Singular (NN): dog, fork
– Plural (NNS): dogs, forks
– Proper (NNP, NNPS): John, Springfields
– Personal pronoun (PRP): I, you, he, she, it
– Wh-pronoun (WP): who, what
• Verb (actions and processes)
– Base, infinitive (VB): eat
– Past tense (VBD): ate
– Gerund (VBG): eating
– Past participle (VBN): eaten
– Non 3rd person singular present tense (VBP): eat
– 3rd person singular present tense: (VBZ): eats
– Modal (MD): should, can
– To (TO): to (to eat)
5
 English Parts of Speech (cont.)
• Adjective (modify nouns)
– Basic (JJ): red, tall
– Comparative (JJR): redder, taller
– Superlative (JJS): reddest, tallest
• Adverb (modify verbs)
– Basic (RB): quickly
– Comparative (RBR): quicker
– Superlative (RBS): quickest
• Preposition (IN): on, in, by, to, with
• Determiner:
– Basic (DT) a, an, the
– WH-determiner (WDT): which, that
• Coordinating Conjunction (CC): and, but, or,
• Particle (RP): off (took off), up (put up)

6
 Closed vs. Open Class
• Closed class categories are composed of a small,
fixed set of grammatical function words for a given
language.
– Pronouns, Prepositions, Modals, Determiners, Particles,
Conjunctions
• Open class categories have large number of words
and new ones are easily invented.
– Nouns (Googler, textlish), Verbs (Google), Adjectives
(geeky), Abverb (automagically)

7
 Ambiguity in POS Tagging
• “Like” can be a verb or a preposition
– I like/VBP candy.
– Time flies like/IN an arrow.
• “Around” can be a preposition, particle, or adverb
– I bought it at the shop around/IN the corner.
– I never got around/RP to getting a car.
– A new Prius costs around/RB $25K.

8
 POS Tagging Process
• Usually assume a separate initial tokenization process that
separates and/or disambiguates punctuation, including
detecting sentence boundaries.
• Degree of ambiguity in English (based on Brown corpus)
– 11.5% of word types are ambiguous.
– 40% of word tokens are ambiguous.
• Average POS tagging disagreement amongst expert human
judges for the Penn treebank was 3.5%
– Based on correcting the output of an initial automated tagger, which
was deemed to be more accurate than tagging from scratch.
• Baseline: Picking the most frequent tag for each specific word
type gives about 90% accuracy
– 93.7% if use model for unknown words for Penn Treebank tagset.

9
 POS Tagging Approaches

• Rule-Based: Human crafted rules based on lexical

and other linguistic knowledge.
• Learning-Based: Trained on human annotated
corpora like the Penn Treebank.
– Statistical models: Hidden Markov Model (HMM),
Maximum Entropy Markov Model (MEMM), Conditional
Random Field (CRF)
– Rule learning: Transformation Based Learning (TBL)
– Neural networks: Recurrent networks like Long Short Term
Memory (LSTMs)
• Generally, learning-based approaches have been found
to be more effective overall, taking into account the
total amount of human expertise and effort involved.

10
 Classification Learning
• Typical machine learning addresses the problem of
classifying a feature-vector description into a fixed
number of classes.
• There are many standard learning methods for this
task:
– Decision Trees and Rule Learning
– Naïve Bayes and Bayesian Networks
– Logistic Regression / Maximum Entropy (MaxEnt)
– Perceptron and Neural Networks
– Support Vector Machines (SVMs)
– Nearest-Neighbor / Instance-Based

11
 Beyond Classification Learning
• Standard classification problem assumes individual
cases are disconnected and independent (i.i.d.:
independently and identically distributed).
• Many NLP problems do not satisfy this assumption
and involve making many connected decisions, each
resolving a different ambiguity, but which are
mutually dependent.
• More sophisticated learning and inference techniques
are needed to handle such situations in general.

12
 Sequence Labeling Problem

• Many NLP problems can viewed as sequence

labeling.
• Each token in a sequence is assigned a label.
• Labels of tokens are dependent on the labels of
other tokens in the sequence, particularly their
neighbors (not i.i.d).

foo bar blam zonk zonk bar blam

13
 Information Extraction

• Identify phrases in language that refer to specific types of

entities and relations in text.
• Named entity recognition is task of identifying names of
people, places, organizations, etc. in text.
people organizations places
– Michael Dell is the CEO of Dell Computer Corporation and lives
in Austin Texas.
• Extract pieces of information relevant to a specific
application, e.g. used car ads:
make model year mileage price
– For sale, 2002 Toyota Prius, 20,000 mi, $15K or best offer.
Available starting July 30, 2006.

14
 Semantic Role Labeling
• For each clause, determine the semantic role played
by each noun phrase that is an argument to the verb.
agent patient source destination instrument
– John drove Mary from Austin to Dallas in his Toyota Prius.
– The hammer broke the window.
• Also referred to a “case role analysis,” “thematic
analysis,” and “shallow semantic parsing”

15
 Bioinformatics
• Sequence labeling also valuable in labeling genetic
sequences in genome analysis.
extron intron
– AGCTAACGTTCGATACGGATTACAGCCT

16
Problems with Sequence Labeling as Classification

• Not easy to integrate information from category of

tokens on both sides.
• Difficult to propagate uncertainty between decisions
and “collectively” determine the most likely joint
assignment of categories to all of the tokens in a
sequence.

17
 Probabilistic Sequence Models
• Probabilistic sequence models allow integrating
uncertainty over multiple, interdependent
classifications and collectively determine the most
likely global assignment.
• Two standard models
– Hidden Markov Model (HMM)
– Conditional Random Field (CRF)

18
 Markov Model / Markov Chain
• A finite state machine with probabilistic state
transitions.
• Makes Markov assumption that next state only
depends on the current state and independent of
previous history.

19
 Sample Markov Model for POS
0.1

Det Noun
0.5
0.95
0.9
stop
0.05 Verb
0.25
0.1
PropNoun 0.8
0.4
0.5 0.1
0.25
0.1
start
20
 Sample Markov Model for POS
0.1

Det Noun
0.5
0.95
0.9
stop
0.05 Verb
0.25
0.1
PropNoun 0.8
0.4
0.5 0.1
0.25
0.1
start
P(PropNoun Verb Det Noun) = 0.4*0.8*0.25*0.95*0.1=0.0076 21
 Hidden Markov Model

• Probabilistic generative model for sequences.

• Assume an underlying set of hidden (unobserved,
latent) states in which the model can be (e.g. parts of
speech).
• Assume probabilistic transitions between states over
time (e.g. transition from POS to another POS as
sequence is generated).
• Assume a probabilistic generation of tokens from
states (e.g. words generated for each POS).

22
 Sample HMM for POS
the cat 0.1
a the dog
a
the a the car bed
that pen apple
0.5
0.95
Det Noun 0.9 bit
stop
ate saw
0.05 played
Tom 0.25 hit gave
0.1 John Mary Verb
Alice 0.8
0.4 Jerry
0.5 PropNoun 0.1
0.25
0.1
start
23
 Sample HMM Generation
the cat 0.1
a the dog
a
the a the car bed
that pen apple
0.5
0.95
Det Noun 0.9 bit
stop
ate saw
0.05 played
Tom 0.25 hit gave
0.1 John Mary Verb
Alice 0.8
0.4 Jerry
0.5 PropNoun 0.1
0.25
0.1
start
24
 Sample HMM Generation
the cat 0.1
a the dog
a
the a the car bed
that pen apple
0.5
0.95
Det Noun 0.9 bit
stop
ate saw
0.05 played
Tom 0.25 hit gave
0.1 John Mary Verb
Alice 0.8
0.4 Jerry
0.5 PropNoun 0.1

0.1
start
25
 Sample HMM Generation
the cat 0.1
a the dog
a
the a the car bed
that pen apple
0.5
0.95
Det Noun 0.9 bit
stop
ate saw
0.05 played
Tom 0.25 hit gave
0.1 John Mary Verb
Alice 0.8
0.4 Jerry
0.5 PropNoun 0.1
0.25
0.1
start John
26
 Sample HMM Generation
the cat 0.1
a the dog
a
the a the car bed
that pen apple
0.5
0.95
Det Noun 0.9 bit
stop
ate saw
0.05 played
Tom 0.25 hit gave
0.1 John Mary Verb
Alice 0.8
0.4 Jerry
0.5 PropNoun 0.1
0.25
0.1
start John
27
 Sample HMM Generation
the cat 0.1
a the dog
a
the a the car bed
that pen apple
0.5
0.95
Det Noun 0.9 bit
stop
ate saw
0.05 played
Tom 0.25 hit gave
0.1 John Mary Verb
Alice 0.8
0.4 Jerry
0.5 PropNoun 0.1
0.25
0.1
start John bit
28
 Sample HMM Generation
the cat 0.1
a the dog
a
the a the car bed
that pen apple
0.5
0.95
Det Noun 0.9 bit
stop
ate saw
0.05 played
Tom 0.25 hit gave
0.1 John Mary Verb
Alice 0.8
0.4 Jerry
0.5 PropNoun 0.1
0.25
0.1
start John bit
29
 Sample HMM Generation
the cat 0.1
a the dog
a
the a the car bed
that pen apple
0.5
0.95
Det Noun 0.9 bit
stop
ate saw
0.05 played
Tom 0.25 hit gave
0.1 John Mary Verb
Alice 0.8
0.4 Jerry
0.5 PropNoun 0.1
0.25
0.1
start John bit the
30
 Sample HMM Generation
the cat 0.1
a the dog
a
the a the car bed
that pen apple
0.5
0.95
Det Noun 0.9 bit
stop
ate saw
0.05 played
Tom 0.25 hit gave
0.1 John Mary Verb
Alice 0.8
0.4 Jerry
0.5 PropNoun 0.1
0.25
0.1
start John bit the
31
 Sample HMM Generation
the cat 0.1
a the dog
a
the a the car bed
that pen apple
0.5
0.95
Det Noun 0.9 bit
stop
ate saw
0.05 played
Tom 0.25 hit gave
0.1 John Mary Verb
Alice 0.8
0.4 Jerry
0.5 PropNoun 0.1
0.25
0.1
start John bit the apple
32
 Sample HMM Generation
the cat 0.1
a the dog
a
the a the car bed
that pen apple
0.5
0.95
Det Noun 0.9 bit
stop
ate saw
0.05 played
Tom 0.25 hit gave
0.1 John Mary Verb
Alice 0.8
0.4 Jerry
0.5 PropNoun 0.1
0.25
0.1
start John bit the apple
33
 Formal Definition of an HMM
• A set of N +2 states S={s0,s1,s2, … sN, sF}
– Distinguished start state: s0
– Distinguished final state: sF
• A set of M possible observations V={v1,v2…vM}
• A state transition probability distribution A={aij}

aij = P(qt +1 = s j | qt = si ) 1  i, j  N and i = 0, j = F

N

a
j =1
ij + aiF = 1 0  i  N
• Observation probability distribution for each state j
B={bj(k)}

• Total parameter set λ={A,B}

b j (k ) = P(vk at t | qt = s j ) 1 j  N 1 k  M
34
 HMM Generation Procedure
• To generate a sequence of T observations: O = o1 o2
… oT

Set initial state q1=s0

For t = 1 to T
Transit to another state qt+1=sj based on transition
distribution aij for state qt
Pick an observation ot=vk based on being in state qt using
distribution bqt(k)

35
 Three Useful HMM Tasks

• Observation Likelihood: To classify and order

sequences.
• Most likely state sequence (Decoding): To tag each
token in a sequence with a label.
• Maximum likelihood training (Learning): To train
models to fit empirical training data.

36
 HMM: Observation Likelihood
• Given a sequence of observations, O, and a model
with a set of parameters, λ, what is the probability
that this observation was generated by this model:
P(O| λ) ?
• Allows HMM to be used as a language model: A
formal probabilistic model of a language that assigns
a probability to each string saying how likely that
string was to have been generated by the language.
• Useful for two tasks:
– Sequence Classification
– Most Likely Sequence

37
 Sequence Classification

• Assume an HMM is available for each category

(i.e. language).
• What is the most likely category for a given
observation sequence, i.e. which category’s HMM
is most likely to have generated it?
• Used in speech recognition to find most likely
word model to have generate a given sound or
phoneme sequence.
O
ah s t e n

? ?
Austin P(O | Austin) > P(O | Boston) ? Boston 38
 Most Likely Sequence

• Of two or more possible sequences, which one

was most likely generated by a given model?
• Used to score alternative word sequence
interpretations in speech recognition.

O1
? dice precedent core

? vice president Gore

O2
Ordinary English
P(O2 | OrdEnglish) > P(O1 | OrdEnglish) ?
39
 HMM: Observation Likelihood
Naïve Solution
• Consider all possible state sequences, Q, of length T
that the model could have traversed in generating the
given observation sequence.
• Compute the probability of a given state sequence
from A, and multiply it by the probabilities of
generating each of given observations in each of the
corresponding states in this sequence to get P(O,Q| λ)
= P(O| Q, λ) P(Q| λ) .
• Sum this over all possible state sequences to get P(O|
λ).
• Computationally complex: O(TNT).

40
 HMM: Observation Likelihood
Efficient Solution
• Due to the Markov assumption, the probability of
being in any state at any given time t only relies on
the probability of being in each of the possible states
at time t−1.
• Forward Algorithm: Uses dynamic programming to
exploit this fact to efficiently compute observation
likelihood in O(TN2) time.
– Compute a forward trellis that compactly and implicitly
encodes information about all possible state paths.

41
 Forward Trellis

s1 • • •
s2 • • •
• • • • •
s0 • • • •
sF
• • • •
• • • • •

sN • • •

t1 t2 t3 tT-1 tT

• Continue forward in time until reaching final time

point and sum probability of ending in final state.
42
 Forward Probabilities

• Let t(j) be the probability of being in state j after

seeing the first t observations (by summing over
all initial paths leading to j).

 t ( j ) = P(o1 , o2 ,...ot , qt = s j |  )

43
 Forward Step

• Consider all possible ways of

s1 a1j
getting to sj at time t by coming
s2 a2j from all possible states si and
• a2j determine probability of each.
sj
•
• aNj • Sum these to get the total
sN
probability of being in state sj at
time t while accounting for the
t-1(i) t(i)
first t −1 observations.
• Then multiply by the probability
of actually observing ot in sj.

44
 Computing the Forward Probabilities
• Initialization
1 ( j ) = a0 j b j (o1 ) 1  j  N

• Recursion
N 
 t ( j ) =   t −1 (i )aij b j (ot ) 1  j  N , 1  t  T
 i =1 

• Termination N
P(O |  ) =  T +1 ( sF ) =   T (i)aiF
i =1

45
 Forward Computational Complexity
• Requires only O(TN2) time to compute the probability
of an observed sequence given a model.
• Exploits the fact that all state sequences must merge
into one of the N possible states at any point in time
and the Markov assumption that only the last state
effects the next one.

46
 Most Likely State Sequence (Decoding)

• Given an observation sequence, O, and a model, λ,

what is the most likely state sequence,Q=q1,q2,…qT,
that generated this sequence from this model?
• Used for sequence labeling, assuming each state
corresponds to a tag, it determines the globally best
assignment of tags to all tokens in a sequence using a
principled approach grounded in probability theory.

John gave the dog an apple.

47
 Most Likely State Sequence

• Given an observation sequence, O, and a model, λ,

what is the most likely state sequence,Q=q1,q2,…qT,
that generated this sequence from this model?
• Used for sequence labeling, assuming each state
corresponds to a tag, it determines the globally best
assignment of tags to all tokens in a sequence using a
principled approach grounded in probability theory.

John gave the dog an apple.

Det Noun PropNoun Verb

48
 Most Likely State Sequence

• Given an observation sequence, O, and a model, λ,

what is the most likely state sequence,Q=q1,q2,…qT,
that generated this sequence from this model?
• Used for sequence labeling, assuming each state
corresponds to a tag, it determines the globally best
assignment of tags to all tokens in a sequence using a
principled approach grounded in probability theory.

John gave the dog an apple.

Det Noun PropNoun Verb

49
 Most Likely State Sequence

• Given an observation sequence, O, and a model, λ,

what is the most likely state sequence,Q=q1,q2,…qT,
that generated this sequence from this model?
• Used for sequence labeling, assuming each state
corresponds to a tag, it determines the globally best
assignment of tags to all tokens in a sequence using a
principled approach grounded in probability theory.

John gave the dog an apple.

Det Noun PropNoun Verb

50
 Most Likely State Sequence

• Given an observation sequence, O, and a model, λ,

what is the most likely state sequence,Q=q1,q2,…qT,
that generated this sequence from this model?
• Used for sequence labeling, assuming each state
corresponds to a tag, it determines the globally best
assignment of tags to all tokens in a sequence using a
principled approach grounded in probability theory.

John gave the dog an apple.

Det Noun PropNoun Verb

51
 Most Likely State Sequence

• Given an observation sequence, O, and a model, λ,

what is the most likely state sequence,Q=q1,q2,…qT,
that generated this sequence from this model?
• Used for sequence labeling, assuming each state
corresponds to a tag, it determines the globally best
assignment of tags to all tokens in a sequence using a
principled approach grounded in probability theory.

John gave the dog an apple.

Det Noun PropNoun Verb

52
 Most Likely State Sequence

• Given an observation sequence, O, and a model, λ,

what is the most likely state sequence,Q=q1,q2,…qT,
that generated this sequence from this model?
• Used for sequence labeling, assuming each state
corresponds to a tag, it determines the globally best
assignment of tags to all tokens in a sequence using a
principled approach grounded in probability theory.

John gave the dog an apple.

Det Noun PropNoun Verb

53
 HMM: Most Likely State Sequence
Efficient Solution

• Obviously, could use naïve algorithm based on

examining every possible state sequence of length T.
• Dynamic Programming can also be used to exploit
the Markov assumption and efficiently determine the
most likely state sequence for a given observation
and model.
• Standard procedure is called the Viterbi algorithm
(Viterbi, 1967) and also has O(N2T) time complexity.

54
 Viterbi Scores

• Recursively compute the probability of the most

likely subsequence of states that accounts for the
first t observations and ends in state sj.

vt ( j ) = max P(q0 , q1 ,..., qt −1 , o1 ,..., ot , qt = s j |  )

q0 , q1 ,...,qt −1

• Also record “backpointers” that subsequently

allow backtracing the most probable state
sequence.
▪ btt(j) stores the state at time t-1 that maximizes the
probability that system was in state sj at time t
(given the observed sequence).
55
 Computing the Viterbi Scores
• Initialization
v1 ( j ) = a0 j b j (o1 ) 1  j  N
• Recursion
N
vt ( j ) = max vt −1 (i)aij b j (ot ) 1  j  N , 1  t  T
i =1
• Termination
N
P* = vT +1 ( sF ) = max vT (i)aiF
i =1

Analogous to Forward algorithm except take max instead of sum

56
 Computing the Viterbi Backpointers
• Initialization
bt1 ( j ) = s0 1  j  N
• Recursion
N
bt t ( j ) = argmax vt −1 (i )aij b j (ot ) 1  j  N , 1  t  T
i =1
• Termination
N
qT * = bt T +1 ( sF ) = argmax vT (i )aiF
i =1

Final state in the most probable state sequence. Follow

backpointers to initial state to construct full sequence.
57
 Viterbi Backpointers

s1 • • •
s2 • • •
• • • • •
s0 • • • •
sF
• • • •
• • • • •

sN • • •

t1 t2 t3 tT-1 tT

58
 Viterbi Backtrace

s1 • • •
s2 • • •
• • • • •
s0 • • • •
sF
• • • •
• • • • •

sN • • •

t1 t2 t3 tT-1 tT

Most likely Sequence: s0 sN s1 s2 …s2 sF

59
 HMM Learning
• Supervised Learning: All training sequences are
completely labeled (tagged).
• Unsupervised Learning: All training sequences are
unlabelled (but generally know the number of tags,
i.e. states).
• Semisupervised Learning: Some training sequences
are labeled, most are unlabeled.

60
 Supervised HMM Training

• If training sequences are labeled (tagged) with the

underlying state sequences that generated them,
then the parameters, λ={A,B} can all be estimated
directly.
Training Sequences
John ate the apple
A dog bit Mary
Mary hit the dog Supervised
John gave Mary the cat. HMM
. Training
.
.
Det Noun PropNoun Verb

61
 Supervised Parameter Estimation

• Estimate state transition probabilities based on tag

bigram and unigram statistics in the labeled data.
C (qt = si , q t +1 = s j )
aij =
C (qt = si )
• Estimate the observation probabilities based on
tag/word co-occurrence statistics in the labeled data.
C (qi = s j , oi = vk )
b j (k ) =
C (qi = s j )
• Use appropriate smoothing if training data is sparse.

62
 Learning and Using HMM Taggers
• Use a corpus of labeled sequence data to easily
construct an HMM using supervised training.
• Given a novel unlabeled test sequence to tag, use the
Viterbi algorithm to predict the most likely (globally
optimal) tag sequence.

63
 Evaluating Taggers
• Train on training set of labeled sequences.
• Possibly tune parameters based on performance on a
development set.
• Measure accuracy on a disjoint test set.
• Generally measure tagging accuracy, i.e. the
percentage of tokens tagged correctly.
• Accuracy of most modern POS taggers, including
HMMs is 96−97% (for Penn tagset trained on about
800K words) .
– Generally matching human agreement level.

64
 Unsupervised
Maximum Likelihood Training
Training Sequences

ah s t e n
a s t i n
oh s t u n
eh z t en HMM
. Training
.
. Austin

65
 Maximum Likelihood Training
• Given an observation sequence, O, what set of
parameters, λ, for a given model maximizes the
probability that this data was generated from this
model (P(O| λ))?
• Used to train an HMM model and properly induce its
parameters from a set of training data.
• Only need to have an unannotated observation
sequence (or set of sequences) generated from the
model. Does not need to know the correct state
sequence(s) for the observation sequence(s). In this
sense, it is unsupervised.

66
 Bayes Theorem
P( E | H ) P( H )
P( H | E ) =
P( E )

Simple proof from definition of conditional probability:

P( H  E )
P( H | E ) = (Def. cond. prob.)
P( E )
P( H  E )
P( E | H ) = (Def. cond. prob.)
P( H )
P( H  E ) = P( E | H ) P( H )

P( E | H ) P( H )
QED: P( H | E ) =
P( E )
 Maximum Likelihood vs.
Maximum A Posteriori (MAP)

• The MAP parameter estimate is the most

likely given the observed data, O.
P(O |  ) P( )
MAP = argmax P( | O) = argmax
  P(O)
argmax P(O |  ) P( )

• If all parameterizations are assumed to be equally
likely a priori, then MLE and MAP are the same.
• If parameters are given priors (e.g. Gaussian or
Lapacian with zero mean), then MAP is a principled
way to perform smoothing or regularization.
 HMM: Maximum Likelihood Training
Efficient Solution
• There is no known efficient algorithm for finding the
parameters, λ, that truly maximizes P(O| λ).
• However, using iterative re-estimation, the Baum-
Welch algorithm (a.k.a. forward-backward) , a
version of a standard statistical procedure called
Expectation Maximization (EM), is able to locally
maximize P(O| λ).
• In practice, EM is able to find a good set of
parameters that provide a good fit to the training data
in many cases.

69
 EM Algorithm

• Iterative method for learning probabilistic

categorization model from unsupervised data.
• Initially assume random assignment of examples to
categories.
• Learn an initial probabilistic model by estimating
model parameters  from this randomly labeled data.
• Iterate following two steps until convergence:
– Expectation (E-step): Compute P(ci | E) for each example
given the current model, and probabilistically re-label the
examples based on these posterior probability estimates.
– Maximization (M-step): Re-estimate the model
parameters, , from the probabilistically re-labeled data.

70
 EM
Initialize:
Assign random probabilistic labels to unlabeled data
Unlabeled Examples
+ −
+ −

+ −
+ −

+ −

71
 EM
Initialize:
Give soft-labeled training data to a probabilistic learner

+ −
+ −

+ −
Prob.
+ − Learner
+ −

72
 EM
Initialize:
Produce a probabilistic classifier

+ −
+ −

+ −
Prob. Prob.
+ − Learner Classifier
+ −

73
 EM
E Step:
Relabel unlabled data using the trained classifier

+ −
+ −
Prob. Prob.
+ −
Learner Classifier + −

+ −

74
 EM
M step:
Retrain classifier on relabeled data

+ −
+ −
Prob. Prob.
+ −
Learner Classifier + −

+ −

Continue EM iterations until probabilistic labels on

unlabeled data converge.
75
 Sketch of Baum-Welch (EM) Algorithm
for Training HMMs

Assume an HMM with N states.

Randomly set its parameters λ=(A,B)
(making sure they represent legal distributions)
Until converge (i.e. λ no longer changes) do:
E Step: Use the forward/backward procedure to
determine the probability of various possible
state sequences for generating the training data
M Step: Use these probability estimates to
re-estimate values for all of the parameters λ

See textbook for detailed equations for E and M steps

76
 EM Properties
• Each iteration changes the parameters in
a way that is guaranteed to increase the
likelihood of the data: P(O|).
• Anytime algorithm: Can stop at any time
prior to convergence to get approximate
solution.
• Converges to a local maximum.
 Semi-Supervised Learning
• EM algorithms can be trained with a mix of
labeled and unlabeled data.
• EM basically predicts a probabilistic (soft)
labeling of the instances and then iteratively
retrains using supervised learning on these
predicted labels (“self training”).
• EM can also exploit supervised data:
– 1) Use supervised learning on labeled data to
initialize the parameters (instead of initializing them
randomly).
– 2) Use known labels for supervised data instead of
predicting soft labels for these examples during
retraining iterations.
 Semi-Supervised EM

Unlabeled Examples
Training Examples
+ + −
+ −
- Prob. Prob.
- + −
Learner Classifier + −
+
+ + −

79
 Semi-Supervised EM

Training Examples
+ + −
+ −
- Prob. Prob.
- + −
Learner Classifier + −
+
+ + −

80
 Semi-Supervised EM

Training Examples
+
- Prob. Prob.
- Classifier
+
Learner
+
+ −
+ −

+ −
+ −

+ −

81
 Semi-Supervised EM

Unlabeled Examples
Training Examples
+ + −
+ −
- Prob. Prob.
- + −
Learner Classifier + −
+
+ + −

82
 Semi-Supervised EM

Training Examples
+ + −
+ −
- Prob. Prob.
- + −
Learner Classifier + −
+
+ + −

Continue retraining iterations until probabilistic

labels on unlabeled data converge.
83
 Semi-Supervised Results
• Use of additional unlabeled data improves on
supervised learning when amount of labeled
data is very small and amount of unlabeled data
is large.
• Can degrade performance when there is
sufficient labeled data to learn a decent model
and when unsupervised learning tends to create
labels that are incompatible with the desired
ones.
– There are negative results for semi-supervised POS
tagging since unsupervised learning tends to learn
semantic labels (e.g. eating verbs, animate nouns)
that are better at predicting the data than purely
syntactic labels (e.g. verb, noun).
 Conclusions
• POS Tagging is the lowest level of syntactic
analysis.
• It is an instance of sequence labeling, a
collective classification task that also has
applications in information extraction, phrase
chunking, semantic role labeling, and
bioinformatics.
• HMMs are a standard generative probabilistic
model for sequence labeling that allows for
efficiently computing the globally most probable
sequence of labels and supports supervised,
unsupervised and semi-supervised learning.