POS Tagging Methods

There are three broad families of POS tagging approaches: rule-based, stochastic (probabilistic), and hybrid/neural. Modern taggers (like the one used by NLTK's pos_tag) are trained statistical/neural models, but understanding the classical approaches builds crucial intuition.

1. Rule-Based Tagging

Uses a large set of hand-written linguistic rules plus a dictionary of possible tags per word.

Rule example:
  IF a word ends in "-ing" AND follows a verb THEN tag it VBG
  IF a word is preceded by "the"/"a" THEN it is likely a NN (noun)

• Pros: transparent, interpretable, no training data required.
• Cons: brittle — cannot generalize to unseen patterns; rule sets become huge and hard to maintain.

2. Stochastic (Probabilistic) Tagging

Uses probabilities learned from a large tagged corpus to pick the most likely tag sequence. The most common stochastic approach is the Hidden Markov Model (HMM).

Hidden Markov Model (HMM) for POS Tagging

An HMM models POS tagging as finding the tag sequence T = t1, t2, ..., tn that maximizes the probability of the tag sequence given the word sequence W = w1, w2, ..., wn:

argmax P(T | W)  =  argmax P(W | T) · P(T)          (Bayes' Rule)

Approximated using two simplifying assumptions:
  P(T) ≈ Π P(t_i | t_{i-1})        -- transition probability (tag given previous tag)
  P(W | T) ≈ Π P(w_i | t_i)        -- emission probability (word given its tag)

The Viterbi Algorithm

Trying every possible tag sequence is computationally infeasible for long sentences. The Viterbi algorithm is a dynamic-programming algorithm that efficiently finds the single most probable tag sequence in O(n × T²) time (n = sentence length, T = number of tags), by keeping only the best path to each state at each step instead of recomputing from scratch.

import nltk
nltk.download('averaged_perceptron_tagger')

# NLTK's built-in pos_tag() is a pre-trained statistical (perceptron-based) tagger --
# conceptually similar in spirit to an HMM/Viterbi-tagged model, but trained
# with a discriminative averaged-perceptron classifier for higher accuracy.
from nltk import pos_tag, word_tokenize
print(pos_tag(word_tokenize("The dog barked loudly")))
# [('The', 'DT'), ('dog', 'NN'), ('barked', 'VBD'), ('loudly', 'RB')]

Training a Simple HMM-style Bigram Tagger with NLTK

import nltk
from nltk.corpus import treebank
from nltk.tag import UnigramTagger, BigramTagger, DefaultTagger

train_data = treebank.tagged_sents()[:3000]
test_data = treebank.tagged_sents()[3000:3200]

default_tagger = DefaultTagger('NN')                       # fallback tag
unigram_tagger = UnigramTagger(train_data, backoff=default_tagger)
bigram_tagger = BigramTagger(train_data, backoff=unigram_tagger)  # backoff chain

print(bigram_tagger.evaluate(test_data))   # accuracy, e.g. 0.87
print(bigram_tagger.tag(['The', 'market', 'fell', 'sharply']))

Comparing the Approaches

POS tagging is a sequence labeling problem — the same class of problem as Named Entity Recognition, which we study later in this unit, and it reuses the same HMM/Viterbi machinery.