ANOVA: Comparing Multiple Groups Rigorously

1. The Problem with Multiple T-Tests

If testing 3 algorithms (A, B, C), running 3 T-tests results in Family-wise Error Rate Inflation.

Study Deep: Post-Hoc Analysis (The "Who" Question)

A significant ANOVA result only tells you that "at least one group is different." It doesn't tell you which one.

• Next Step: You must perform a Post-Hoc Test (like Tukey's HSD).
• Why?: These tests perform pairwise comparisons while strictly controlling the "False Positive" rate, ensuring that your discovery of a difference between Group A and Group C is actually valid.

1. The Problem with Multiple T-Tests

• df_between = k - 1 (k = number of groups)
• df_within = N - k (N = total observations)
• MSB (Mean Square Between) = SSB / df_between
• MSW (Mean Square Within) = SSW / df_within

The F-Ratio Formula: F = MSB / MSW

• If F is close to 1: Signal ≈ Noise. Fail to reject H₀.
• If F > Critical Value: Signal is much larger than Noise. Reject H₀. The group means are significantly different.

4. Post-Hoc Analysis

A significant ANOVA only tells you at least one group is different. It doesn't tell you which one. You must follow up with a Post-Hoc test:

• Tukey's HSD (Honestly Significant Difference): Best for all pairwise comparisons when group sizes are equal.
• Bonferroni Correction: Extremely conservative. Divides α by the number of comparisons. Good for a small number of planned comparisons.

5. Variations of ANOVA

• One-Way ANOVA: 1 independent categorical variable (e.g., Server Type: AWS, Azure, GCP) affecting continuous Y (Response time).
• Two-Way ANOVA: 2 independent variables (e.g., Server Type AND Time of Day). This allows us to test for Interaction Effects (e.g., does AWS perform better only at night?).
• Repeated Measures ANOVA: Same subjects measured across multiple conditions (analogous to the Paired T-test).

6. Complete Worked ANOVA Example (Exam Style)

Problem: Three teaching methods A, B, C were applied to 3 groups of students. Test marks are:

• Method A: 60, 62, 65
• Method B: 70, 72, 75
• Method C: 80, 82, 85

Step 1: Grand Mean All 9 scores: 60+62+65+70+72+75+80+82+85 = 631; Grand Mean = 631/9 ≈ 70.11

Step 2: Group Means

• x̄_A = (60+62+65)/3 = 62.33
• x̄_B = (70+72+75)/3 = 72.33
• x̄_C = (80+82+85)/3 = 82.33

Step 3: SSB (Between Groups) SSB = 3×(62.33-70.11)² + 3×(72.33-70.11)² + 3×(82.33-70.11)² = 3×(60.5) + 3×(4.9) + 3×(149.3) = 643.6

Step 4: SSW (Within Groups) SSW = (60-62.33)²+(62-62.33)²+(65-62.33)² + (70-72.33)²+(72-72.33)²+(75-72.33)² + (80-82.33)²+(82-82.33)²+(85-82.33)² = (5.4+0.1+7.1) + (5.4+0.1+7.1) + (5.4+0.1+7.1) = 38.0

Step 5: F-Statistic

• MSB = SSB/(k-1) = 643.6/2 = 321.8
• MSW = SSW/(N-k) = 38.0/6 = 6.33
• F = MSB/MSW = 321.8/6.33 ≈ 50.8

Step 6: Compare to F critical F_critical (df1=2, df2=6, α=0.05) = 5.14

Since F = 50.8 > 5.14 → Reject H₀ Conclusion: Teaching methods have significantly different effects on student performance.

7. ANOVA Summary Table Template