Siksha Sarovar

Siksha Sarovar (sikshasarovar.com) is a free educational web application that helps students in India learn programming and prepare for academic and competitive exams. The platform offers structured coding courses (C, C++, Python, Java, HTML, CSS, PHP, Power BI, AI, Machine Learning, Data Science), complete university curriculum notes for BCA/MCA students with previous year question papers, Class 10 and Class 12 CBSE/HBSE school notes, and dedicated preparation material for SSC, UPSC, Banking, Railway and other government exams. Browsing the site is completely free and requires no account. Users may optionally sign in with Google solely to save their learning progress, quiz scores and personal preferences across devices.

Privacy Policy | Terms of Service | Contact Siksha Sarovar | About Siksha Sarovar

v4.0.9 · PWA
Siksha Sarovar logo
Siksha Sarovar
Your Learning Universe

Siksha Sarovar is a free e-learning platform for coding courses, BCA university notes and competitive exam preparation. Optional Google sign-in saves your learning progress across devices.

Initializing knowledge base…
Compiling modules 0%

Unit 2 — Named Entity Recognition (NER): Concepts & Techniques

Lesson 17 of 39 in the free Natural Language Processing notes on Siksha Sarovar, written by Rohit Jangra.

Named Entity Recognition (NER)

Named Entity Recognition (NER) identifies and classifies "named entities" in text into predefined categories — people, organizations, locations, dates, monetary values, and more.

"Riya joined Google in Bengaluru on 5 August 2026 for ₹18,00,000."
   [PERSON]    [ORG]    [LOC]         [DATE]              [MONEY]

Common Entity Types

TypeExamples
PERSONRiya, Elon Musk
ORG (Organization)Google, IIT Delhi
GPE / LOC (Geo-political entity / Location)Delhi, India, Mount Everest
DATE / TIME5 August 2026, 10 AM
MONEY₹18,00,000, $500
PERCENT25%
PRODUCTiPhone 17

Why NER Is Hard

  1. Ambiguity — "Washington" could be a person (George Washington), a place (Washington D.C.), or an organization.
  2. Boundary detection — is "New York Stock Exchange" one entity or three?
  3. Novel/unseen entities — new company/product names appear constantly; a dictionary-only approach fails.
  4. Case sensitivity issues — many NER systems rely on capitalization, which fails on lowercase text (social media, ASR transcripts).

NER as a Sequence Labeling Problem — BIO Tagging

NER is typically framed the same way as POS tagging (previous lessons): assign a label to every token, using the BIO scheme:

PrefixMeaning
B-TYPEBeginning of an entity of that type
I-TYPEInside (continuation) of an entity
OOutside — not part of any entity
Token:  Riya    joined  Google  in   Bengaluru
Tag:    B-PER   O       B-ORG   O    B-LOC
Token:  New     York    Stock   Exchange  opened
Tag:    B-ORG   I-ORG   I-ORG   I-ORG     O

NER Approaches

ApproachHow it worksExample
Rule-based / GazetteerDictionary lookup + hand-written regex patternsList of all known country names
Statistical (HMM/CRF)Sequence models trained on tagged corpora, similar to POS tagging's HMM (previous lesson)Conditional Random Fields (CRF)
Neural / Transformer-basedDeep contextual models (BiLSTM-CRF, BERT-based)spaCy's default pipeline, BERT-NER

CRF (Conditional Random Field) — Brief Intuition

Where an HMM models P(word | tag) and P(tag | previous tag) separately (generative), a CRF directly models P(tag sequence | word sequence) (discriminative) and can use arbitrary overlapping features of the input (capitalization, prefixes/suffixes, surrounding words) — which is why CRFs historically outperformed HMMs for NER.

Useful CRF features for NER:
  - Is the word capitalized?
  - Is the word in a gazetteer (known-entity list)?
  - What is the previous/next word?
  - What is the word's POS tag?
  - Does the word contain digits (helps detect DATE, MONEY)?

We put NER into practice with a production-grade library in the next lesson.