Supervised and Unsupervised Learning

Machine Learning (ML) is the branch of AI in which systems learn patterns from data rather than being explicitly programmed with rules. In data analytics, ML is what powers predictive analytics (supervised) and descriptive analytics (unsupervised).

Supervised Learning

The algorithm learns from labelled training data — every input example carries the correct output. The goal is a function f(X) → Y that generalises to unseen data.

Training data:
   X (features/inputs)              Y (label/target)
   ────────────────────             ────────────────
   hours=5, attendance=80%    ->    Pass
   hours=2, attendance=45%    ->    Fail
   hours=8, attendance=92%    ->    Pass

Learn f  ->  then predict for a NEW student with hours=6, attendance=75%

Unsupervised Learning

The algorithm receives only inputs, no labels, and must discover structure on its own.

Reinforcement Learning (Brief)

An agent interacts with an environment, taking actions and receiving rewards or penalties, learning a policy that maximises cumulative reward. Used in game playing (AlphaGo), robotics, dynamic pricing, and recommendation sequencing. It needs neither labels nor a fixed dataset — it needs an environment to experiment in.

Comparison — The Standard Exam Table

Semi-Supervised and Self-Supervised Learning

• Semi-supervised — a small labelled set plus a large unlabelled set. Common in practice, because labelling is the bottleneck (e.g. 1,000 labelled medical images plus 100,000 unlabelled ones).
• Self-supervised — labels generated automatically from the data's own structure (e.g. predicting a masked word). This is how modern language models are pre-trained.

The Supervised Learning Workflow

The cardinal rule: never evaluate a model on data it was trained on. Training accuracy is always optimistic; only performance on held-out data predicts real-world behaviour.

Overfitting and Underfitting

The bias-variance trade-off:

   Total Error = Bias² + Variance + Irreducible Error

   Bias     = error from wrong assumptions (model too simple)
   Variance = error from sensitivity to the particular training sample

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import PolynomialFeatures
from sklearn.pipeline import make_pipeline
from sklearn.metrics import mean_squared_error

np.random.seed(42)
X = np.sort(np.random.uniform(0, 10, 30)).reshape(-1, 1)
y = 2 * X.ravel() + 5 + np.random.normal(0, 3, 30)

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

for degree in [1, 3, 15]:
    model = make_pipeline(PolynomialFeatures(degree), LinearRegression())
    model.fit(X_train, y_train)
    train_rmse = np.sqrt(mean_squared_error(y_train, model.predict(X_train)))
    test_rmse  = np.sqrt(mean_squared_error(y_test,  model.predict(X_test)))
    print(f"Degree {degree:2d}: train RMSE = {train_rmse:6.3f}, test RMSE = {test_rmse:8.3f}")
# Degree  1: train RMSE =  2.708, test RMSE =    2.906   <- good fit
# Degree  3: train RMSE =  2.605, test RMSE =    3.297
# Degree 15: train RMSE =  1.183, test RMSE =  108.402   <- OVERFITTING: memorised the noise

Train-Test Split and Cross-Validation

from sklearn.model_selection import KFold, cross_val_score

# Simple split
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)
print(f"Train: {len(X_train)}, Test: {len(X_test)}")

# k-fold cross-validation — every observation is used for testing exactly once
model = LinearRegression()
scores = cross_val_score(model, X, y, cv=5, scoring="r2")
print("Fold R² scores:", scores.round(3))
print(f"Mean R² = {scores.mean():.3f} (+/- {scores.std():.3f})")

Why cross-validation beats a single split: one random split can be lucky or unlucky. Averaging over k folds gives a far more stable estimate, and uses every record for both training and testing.

The next lessons dive into each task type: classification, then regression, then the specific algorithms named in the syllabus.