19 · Central Limit Theorem
No matter the distribution — if it has finite variance and n is large enough, the sample mean converges to a Normal. This single fact is why statistics works.
The sample mean of ANY distribution with finite variance approaches a Normal as n grows — this is why Normal is everywhere.
Without this:
Without the CLT, you couldn't compute a confidence interval on any sample mean, A/B test result, or model accuracy.
The Central Limit Theorem (CLT) is arguably the most important theorem in applied statistics. It states:
If X₁, X₂, …, Xₙ are independent and identically distributed (iid) random variables from ANY distribution with finite mean μ and finite variance σ², then the standardised sample mean converges in distribution to the Standard Normal as n → ∞:
(X̄ − μ) · √n / σ → N(0, 1)
Equivalently, X̄ ~ N(μ, σ²/n) approximately for large n.
Required conditions:
- The variables must be iid (or at least uncorrelated with bounded moments).
- The distribution must have finite variance (σ² < ∞). Heavy-tailed distributions with power-law tails where α ≤ 2 violate this and the CLT does not apply.
Rule of thumb: n ≥ 30 is usually enough for the approximation to be accurate. Very skewed distributions may need n ≥ 100.
Standard error of the mean: SE = σ / √n
The SE is the standard deviation of the sampling distribution of X̄. It shrinks as n grows — quadrupling the sample size halves the SE. This is why larger studies give narrower confidence intervals.
Why is this so powerful? The shape of the underlying distribution does not matter. Whether your data comes from a Uniform, Exponential, Log-normal, or Binomial — the sample mean will look Normal if n is large enough. This universality is what lets us build hypothesis tests and confidence intervals using Normal tables for almost any kind of data.
Python (in browser)
Four histograms of 10,000 sample means from Uniform(0, 10) at n = 2, 10, 30, 100. Even starting from a flat (non-Normal) distribution, the sampling distribution of the mean rapidly converges to a bell curve. At n = 30 it is already nearly indistinguishable from Normal.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Exponential(λ=1) has skewness = 2 — extremely right-skewed. Yet by n = 30 the distribution of sample means is already nearly symmetric and bell-shaped. The CLT applies to ANY distribution with finite variance, regardless of shape.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
With σ = 10 and n = 100, the theoretical SE is exactly 10/√100 = 1.0. The empirical standard deviation of 10,000 simulated sample means matches this closely, and the Shapiro-Wilk test confirms normality of the means.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
The CLT is powerful but not universal. Heavy-tailed distributions (Cauchy, Pareto with α ≤ 2) have infinite variance, breaking the CLT's guarantee. For such data, bootstrap CIs or median-based summaries are safer choices.
If σ = 20 and n = 400, what is the standard error of the sample mean?
- **CLT statement:** (X̄ − μ)·√n / σ → N(0, 1). Requires iid draws with finite variance. Rule of thumb: n ≥ 30.
- **Standard error:** SE = σ/√n — the std dev of the sample mean. Quadrupling n halves the SE.
- The CLT does NOT apply to distributions with infinite variance (Cauchy, Pareto α ≤ 2). Use bootstrap CIs instead.
Every confidence interval on a model metric (accuracy, AUC, loss) is a CLT application. Bootstrap distributions of any statistic approach Normal. A/B test significance bottoms out at CLT.
If you remove it: You can't say 'this model's accuracy is 92% ± 1.2%' — that interval is the CLT in action.