Probability Distributions

A probability distribution describes how probability is spread across the possible values of a random variable. Knowing the distribution lets you compute the likelihood of any range of outcomes — which is exactly what hypothesis testing needs.

1. Binomial Distribution

Models the number of successes in n independent trials, each with the same success probability p.

   P(X = k) = C(n, k) · p^k · (1 − p)^(n − k)

              n!
   C(n,k) = ────────
            k!(n−k)!

   Mean  = n·p
   Variance = n·p·(1 − p)

Conditions (remember as BINS): Binary outcomes, Independent trials, Number of trials fixed, Same probability each trial.

Worked example. A machine produces 5% defective items. In a batch of 10, what is the probability of exactly 2 defectives?

n = 10, k = 2, p = 0.05

C(10,2) = 10!/(2!·8!) = 45
P(X = 2) = 45 × (0.05)² × (0.95)⁸
         = 45 × 0.0025 × 0.6634
         = 0.0746   ->  about 7.46%

Mean number of defectives = 10 × 0.05 = 0.5

2. Poisson Distribution

Models the number of events occurring in a fixed interval of time/space, when events are rare and independent.

              e^(−λ) · λ^k
   P(X = k) = ──────────────
                    k!

   λ (lambda) = average number of events per interval
   Mean = Variance = λ          <- the Poisson signature

Worked example. A helpdesk receives on average 3 calls per hour. P(exactly 5 calls in an hour)?

λ = 3, k = 5
P(X = 5) = e^(−3) × 3⁵ / 5!
         = 0.049787 × 243 / 120
         = 0.1008   ->  about 10.08%

Poisson is the natural model for: website hits per minute, defects per metre of cable, accidents per month, arrivals in a queue.

3. Normal (Gaussian) Distribution

The most important continuous distribution — the classic bell curve.

                  1              −(x − μ)²
   f(x) = ─────────────── · exp( ───────── )
            σ√(2π)                  2σ²

   Notation: X ~ N(μ, σ²)

Properties:

1. Symmetric and bell-shaped about μ
2. Mean = Median = Mode = μ
3. Total area under the curve = 1
4. Asymptotic — the tails approach but never touch the x-axis
5. Fully described by just two parameters: μ and σ
6. Follows the empirical rule: 68% within ±1σ, 95% within ±2σ, 99.7% within ±3σ

Standard Normal Distribution and Z-Scores

   Z ~ N(0, 1)          standard normal: mean 0, standard deviation 1

         x − μ
   z = ─────────         converts ANY normal variable to standard normal
           σ

Worked example. Marks are normally distributed with μ = 65, σ = 10. What percentage of students score above 80?

z = (80 − 65) / 10 = 1.5

From the standard normal table, P(Z < 1.5) = 0.9332
P(Z > 1.5) = 1 − 0.9332 = 0.0668

=> about 6.68% of students score above 80

What percentage score between 55 and 75?

z₁ = (55 − 65)/10 = −1.0      z₂ = (75 − 65)/10 = +1.0
P(−1 < Z < 1) = 0.8413 − 0.1587 = 0.6826    ->  68.26%   (the empirical rule ✓)

Central Limit Theorem (CLT)

The theorem that makes inferential statistics work.

Regardless of the shape of the population distribution, the sampling distribution of the sample mean approaches a normal distribution as the sample size increases (typically n ≥ 30), with mean μ and standard error σ/√n.

   Sampling distribution of x̄:   x̄ ~ N( μ,  σ²/n )

   Standard Error (SE) = σ / √n

This is why z-tests and t-tests are valid even on non-normal populations — and why quadrupling your sample size only halves the standard error.

import numpy as np
import matplotlib.pyplot as plt

np.random.seed(42)

# Population is heavily SKEWED (exponential), nothing like a bell curve
population = np.random.exponential(scale=10, size=100000)
print("Population skewness: strongly right-skewed, mean =", round(population.mean(), 2))

# Take 1000 samples of size 30 and record each sample MEAN
sample_means = [np.random.choice(population, 30).mean() for _ in range(1000)]

print("Mean of sample means:", round(np.mean(sample_means), 2))    # ≈ population mean
print("Theoretical SE:", round(population.std() / np.sqrt(30), 3))
print("Observed SD of sample means:", round(np.std(sample_means), 3))   # matches SE

# The distribution of sample means is approximately NORMAL — that's the CLT

Python — Working with Distributions

from scipy import stats

# BINOMIAL: exactly 2 defectives in 10 items, p = 0.05
print("P(X=2)  =", round(stats.binom.pmf(k=2, n=10, p=0.05), 4))     # 0.0746
print("P(X<=2) =", round(stats.binom.cdf(k=2, n=10, p=0.05), 4))     # 0.9885

# POISSON: exactly 5 calls when λ = 3
print("P(X=5)  =", round(stats.poisson.pmf(k=5, mu=3), 4))           # 0.1008
print("P(X>=5) =", round(1 - stats.poisson.cdf(k=4, mu=3), 4))       # 0.1847

# NORMAL: marks ~ N(65, 10²)
print("P(X>80)     =", round(1 - stats.norm.cdf(80, loc=65, scale=10), 4))   # 0.0668
print("P(55<X<75)  =", round(stats.norm.cdf(75, 65, 10) - stats.norm.cdf(55, 65, 10), 4))  # 0.6827

# Inverse: what mark is the 90th percentile?
print("90th percentile mark =", round(stats.norm.ppf(0.90, loc=65, scale=10), 2))  # 77.82

# Checking whether real data is normally distributed
import pandas as pd

marks = pd.Series(np.random.normal(65, 10, 200))

print("Skewness:", round(marks.skew(), 3))       # near 0 for normal
print("Kurtosis:", round(marks.kurtosis(), 3))   # near 0 (excess) for normal

# Shapiro-Wilk normality test
stat, p = stats.shapiro(marks)
print(f"Shapiro-Wilk: W = {stat:.4f}, p = {p:.4f}")
# p > 0.05  ->  cannot reject normality; treating the data as normal is reasonable

# Visual check — Q-Q plot (points on the line => normal)
# stats.probplot(marks, dist="norm", plot=plt); plt.show()

Distribution Summary Table

Choosing the Right Distribution

With distributions in hand, we can finally ask formal questions of the data — but first, we explore it. The next lesson begins Exploratory Data Analysis.