15 · Standard Normal and the Z-score
Subtract the mean, divide by σ — every Normal collapses to N(0,1). The z-score is the universal language of 'how many standard deviations away?'
Subtract the mean, divide by the standard deviation, and you've converted any Normal RV to the Standard Normal — the same scale everyone uses.
Without this:
Without z-scores, you can't compare features measured on different scales, look up probabilities in a Z-table, or interpret 'σ-away-from-mean' threshold rules.
The Standard Normal distribution N(0, 1) is the Normal with mean 0 and standard deviation 1. It is the reference distribution: every N(μ, σ²) can be converted to it via the z-score transformation.
The z-score: z = (x − μ) / σ
If X ~ N(μ, σ²), then Z = (X − μ) / σ ~ N(0, 1).
The z-score answers: "How many standard deviations is x above (positive) or below (negative) the mean?"
Why standardize?
- Comparability — A height of 185 cm and a salary of €4000 are on incompatible scales. After z-scoring, both are in units of standard deviations, and you can compare them.
- Probability lookup — Once you have z, you can compute any probability from the Standard Normal CDF, Φ(z) = P(Z ≤ z), without caring about μ and σ.
- Critical values — Common significance thresholds translate to fixed z-values (see cheat sheet below).
Percentile ↔ z-score:
Φ(z) = P(Z ≤ z) gives the percentile of z. Conversely, Φ⁻¹(p) (the quantile function, norm.ppf) gives the z-score at percentile p.
The most famous critical value: z = 1.96 corresponds to the 97.5th percentile, so P(−1.96 ≤ Z ≤ 1.96) = 0.95 exactly — the basis of 95% confidence intervals.
Python (in browser)
z = (x − μ) / σ = (115 − 100) / 15 = 1.0. Φ(1.0) ≈ 0.8413 — an IQ of 115 is at the 84th percentile, meaning it exceeds about 84% of all scores. The z-score converts any Normal measurement to a universal 'standard deviations above mean' scale.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Z-scoring (raw − mean) / std produces a column with mean ≈ 0 and std ≈ 1. This is exactly what scikit-learn's StandardScaler does to each feature column before passing data to gradient-descent-based models. No information is lost — it is a linear rescaling.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
norm.ppf(p) is the inverse CDF — it answers 'what z-score puts p of the mass below it?' ppf(0.975) = 1.9600 is the critical value for 95% two-sided confidence intervals. ppf(0.995) = 2.5758 is the z for 99% CIs. These are constants you'll see throughout inferential statistics.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Memorize the big three: z=1.645 (90% one-sided), z=1.960 (95% two-sided), z=2.576 (99% two-sided). These four numbers underlie virtually all frequentist confidence intervals in published research.
What's the z-score for a value 2.5 standard deviations above the mean?
- **z-score**: z = (x − μ) / σ. If X ~ N(μ, σ²) then Z ~ N(0,1). The z-score is the number of standard deviations x sits above (z > 0) or below (z < 0) the mean.
- Critical values: z=1.645 (90%), z=1.960 (95%), z=2.576 (99%). These underlie all frequentist confidence intervals.
- StandardScaler = z-scoring. Feature normalization before gradient descent is z-score standardization applied column-wise.
Feature normalization (StandardScaler) is z-scoring. Batch normalization is dynamic z-scoring per mini-batch. P-values and confidence intervals all bottom out at z-score lookups.
If you remove it: Models with mixed-scale features train badly — gradient descent zigzags along ill-conditioned axes.