24 · The t-distribution and t-tests
When σ is unknown — almost always — the t-distribution accounts for the extra uncertainty of estimating σ from data. Heavier tails, then Normal as n grows.
When σ is UNKNOWN — which is almost always — use the t-distribution: heavier tails for small samples, identical to Normal as n grows.
Without this:
Without it, you'd over-confidently reject H₀ with small samples and unknown variance.
Think of the t-test as the Z-test's grown-up cousin — it handles the case where σ is unknown, which in practice means almost always.
Why a different distribution?
When you estimate σ from the data using the sample standard deviation s, you introduce extra uncertainty — especially for small n. William Gosset (publishing as "Student" in 1908) showed that the correct null distribution for the test statistic
t = (x̄ − μ₀) / (s / √n)
is the t-distribution with ν = n − 1 degrees of freedom, not the Standard Normal.
Key properties of the t-distribution:
- Symmetric around 0, like N(0,1), but with heavier tails for small ν.
- As ν → ∞, t(ν) → N(0, 1) exactly. For ν ≥ 30 the difference is already tiny.
- The heavier tails reflect the extra uncertainty from estimating σ — more probability mass in the tails means the critical values are larger, making it harder to reject H₀ (which is correct — you have less information).
Three flavours of t-test:
| Test | scipy function | Use when |
|------|---------------|----------|
| One-sample | ttest_1samp(sample, popmean) | Is x̄ significantly different from μ₀? |
| Independent two-sample (Welch's) | ttest_ind(a, b, equal_var=False) | Are two independent groups different? σ's may differ. |
| Paired | ttest_rel(before, after) | Same subjects measured twice — much more powerful when pairing is real. |
Python (in browser)
The t-distribution has heavier tails than the Normal for small df — reflecting the extra uncertainty from estimating σ. As df grows it converges exactly to N(0,1). The critical values show this: t(df=2) needs ±4.30 to reject at α=0.05, while Normal needs only ±1.96.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
One-sample t-test on the same IQ data as the Z-test lesson. The t-test gives a slightly larger p-value because the t-distribution is more conservative — appropriate when σ is estimated, not known.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Welch's t-test for two independent groups. Setting equal_var=False is the safe default — it adjusts degrees of freedom when group variances differ, which is virtually always the case in real data.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Paired t-test vs independent t-test on the same data. The paired test is far more powerful because it computes the difference within each subject, eliminating person-to-person variability. The independent test wastes that information and may fail to detect a real effect.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Z vs t decision flowchart, and when to go further to non-parametric tests.
You have 'before' and 'after' measurements on the SAME 30 people who took a training course. Which test should you use?
- **t-statistic:** t = (x̄ − μ₀) / (s/√n). Follows t(ν=n−1) when σ is unknown. Heavier tails than Normal for small n.
- Three scipy functions: ttest_1samp (one-sample), ttest_ind with equal_var=False (Welch's two-sample), ttest_rel (paired).
- **Paired > independent** whenever the same subjects are measured twice — pairing eliminates between-subject variance and dramatically increases power.
Comparing two models' accuracies on the SAME test set is a paired t-test (each row gives a paired result). A/B test analyses default to t-tests since true variance is unknown.
If you remove it: You'd lose paired-test power and overstate variance — both lead to bad statistical decisions.