18 · Power law and Pareto
Winner-take-all distributions with infinite variance — why 20% of users generate 80% of revenue and why the sample mean is meaningless.
Power-law tails (P(X > x) ∝ x^-α) are the signature of 'winner-take-all' phenomena — Pareto is the workhorse.
Without this:
Without it, you'd predict a 'mean income' that's nowhere near a typical income, because power-law tails wreck the mean.
A power-law distribution is one where the survival function (the complementary CDF) decays as a power of x:
P(X > x) ∝ x^(−α) for x ≥ x_min, α > 0
The exponent α (alpha) is the tail index — the smaller α is, the heavier the tail.
The Pareto distribution is the canonical continuous power-law distribution: PDF: f(x) = α · x_min^α / x^(α+1) for x ≥ x_min
Pareto(b=α) in scipy uses shape parameter b = α. The default x_min = 1.
Mean and variance:
- E[X] = α · x_min / (α − 1) when α > 1; infinite when α ≤ 1
- Var(X) = x_min² · α / ((α − 1)² · (α − 2)) when α > 2; infinite when α ≤ 2
This is critical: when α ≤ 2, variance is infinite — the sample variance keeps growing as you collect more data and never converges. When α ≤ 1, even the mean is infinite.
The 80/20 rule (Pareto principle): For a Pareto distribution with α ≈ 1.16, the top 20% of x-values account for 80% of the total "wealth". This is the famous Pareto principle — 80% of effects from 20% of causes.
Log-log diagnostic: If P(X > x) ∝ x^(−α), then on a log-log plot: log P(X > x) = −α · log(x) + const → a straight line with slope −α
If your data's survival function is linear on a log-log plot, you have a power law.
Python (in browser)
Pareto(α=1.5): theoretical mean = 3.0, but the sample mean is noisy and the max value is very large. With α ≤ 2 the variance is infinite — the sample mean converges very slowly (if at all) and the spread of the distribution is dominated by rare extreme values.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
The Pareto survival function P(X > x) is curved and hard to interpret on a linear scale (left). On a log-log scale (right) it becomes a straight line with slope −α = −1.5 — the definitive diagnostic for a power law. Open /tmp/pareto.png to view.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
The 80/20 rule emerges near α≈1.16. With α=1.5, the top 20% still hold a large fraction. With α=3.0 the tail is lighter and the concentration is less extreme. Compare to the Normal distribution where wealth is much more evenly spread.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Power laws are everywhere a competitive or network-effect process operates: wealth, city size, word frequency, earthquake magnitude, follower count. Recognising a power-law tail prevents you from trusting the sample mean and nudges you toward median-based or log-transformed statistics.
Why does the mean of a Pareto(α=1) distribution diverge?
- Pareto PDF ∝ x^(−α−1); survival function P(X > x) ∝ x^(−α). When α ≤ 2: variance is infinite. When α ≤ 1: even the mean is infinite.
- Log-log plot of the survival function: if it is linear with slope −α, you have a power law. This is the standard diagnostic.
- Never use the mean to summarise power-law data when α ≤ 2 — it is dominated by extreme outliers and never stabilises. Use the median or log-transform.
Long-tail recommendation problems (most items have few interactions, a few items dominate); word frequencies in NLP follow Zipf (a discrete power law); training-set imbalance in classification often has power-law class frequencies.
If you remove it: You'd model 'average user behavior' but real systems are dominated by power-user behavior.