Sentiment Analysis — Hands-on with Python
Lexicon-Based Sentiment with VADER (Great for Social Media Text)
VADER (Valence Aware Dictionary and sEntiment Reasoner) is a lexicon-and-rule-based sentiment tool tuned specifically for short, informal text (tweets, reviews) — it natively handles negation, punctuation emphasis ("!!!"), capitalization ("GREAT" vs "great"), and degree modifiers ("very good").
import nltk
from nltk.sentiment import SentimentIntensityAnalyzer
nltk.download('vader_lexicon')
sia = SentimentIntensityAnalyzer()
texts = [
"The movie was absolutely fantastic!",
"The movie was not good at all.",
"The movie was okay, nothing special.",
"The movie was GREAT!!! Loved it :)"
]
for t in texts:
scores = sia.polarity_scores(t)
print(t, "->", scores)
# The movie was absolutely fantastic! -> {'neg': 0.0, 'neu': 0.406, 'pos': 0.594, 'compound': 0.6239}
# The movie was not good at all. -> {'neg': 0.379, 'neu': 0.621, 'pos': 0.0, 'compound': -0.3865}
# The movie was okay, nothing special.-> {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}
# The movie was GREAT!!! Loved it :) -> {'neg': 0.0, 'neu': 0.238, 'pos': 0.762, 'compound': 0.8955}
compoundis the single overall normalized score, ranging from -1 (most negative) to +1 (most positive).- Notice VADER correctly flips "not good" to negative, and boosts "GREAT!!!" with capitalization + exclamation marks.
def classify_sentiment(compound):
if compound >= 0.05:
return "Positive"
elif compound <= -0.05:
return "Negative"
return "Neutral"
for t in texts:
score = sia.polarity_scores(t)['compound']
print(t, "->", classify_sentiment(score))
Lexicon-Based Sentiment with TextBlob (Simple API)
from textblob import TextBlob
text = "The battery life is excellent but the camera is quite disappointing."
blob = TextBlob(text)
print(blob.sentiment)
# Sentiment(polarity=0.05, subjectivity=0.65)
# polarity: -1 (negative) to +1 (positive)
# subjectivity: 0 (objective/factual) to 1 (subjective/opinion)
ML-Based Sentiment Classifier (Reusing the Naïve Bayes Pipeline)
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.naive_bayes import MultinomialNB
from sklearn.pipeline import Pipeline
reviews = [
"amazing product loved it", "terrible quality very disappointed",
"excellent value for money", "worst purchase ever made",
"great performance highly recommend", "not worth the price at all"
]
sentiments = ["positive", "negative", "positive", "negative", "positive", "negative"]
pipeline = Pipeline([
('tfidf', TfidfVectorizer()),
('classifier', MultinomialNB())
])
pipeline.fit(reviews, sentiments)
print(pipeline.predict(["great quality highly recommend"])) # ['positive']
print(pipeline.predict(["very disappointed with this"])) # ['negative']
Aspect-Based Sentiment (Simplified Rule-Based Approach)
from nltk.sentiment import SentimentIntensityAnalyzer
sia = SentimentIntensityAnalyzer()
review = "The camera is disappointing but the battery life is excellent."
aspects = {"camera": "camera is disappointing",
"battery": "battery life is excellent"}
for aspect, clause in aspects.items():
score = sia.polarity_scores(clause)['compound']
sentiment = "Positive" if score > 0 else "Negative" if score < 0 else "Neutral"
print(f"{aspect}: {sentiment} ({score})")
# camera: Negative (-0.4939)
# battery: Positive (0.5719)
Real-World Deployment Considerations
| Consideration | Why it matters |
|---|---|
| Domain adaptation | A model trained on movie reviews may perform poorly on product reviews — sentiment words carry different weight per domain |
| Multilingual/code-mixed text | Hinglish reviews ("mast product hai") need specialized lexicons or multilingual models |
| Class imbalance | Real-world data often skews heavily positive or negative; accuracy alone can be misleading (use precision/recall/F1) |
| Sarcasm detection | Still an open research challenge; often requires dedicated sarcasm-detection models as a pre-filter |