10 · The distribution zoo
Six distributions cover 95% of what you'll see in ML — and each one models a specific kind of real-world process.
Six distributions cover 95% of what you'll see in ML — and each one models a specific kind of real-world process.
Without this:
Without this tour, you can't choose the right likelihood for your model — and the wrong likelihood gives the wrong loss.
Every probability model starts with a question: "what kind of random process generated this data?" The answer points you to a distribution.
Think of distributions as pre-built mathematical models for common data-generating processes. Once you name the right distribution, you get:
- A PMF or PDF for computing exact probabilities
- A CDF for computing interval probabilities
- Closed-form formulas for mean, variance, entropy
- Ready-made
scipy.statsobjects for simulation and fitting
Here is the field guide for the six distributions that cover almost every ML task. The deep-dive lessons (Chapters 4 and 5) will revisit each one with full derivations — this is the map, not the territory.
| Distribution | Type | Models | Key parameter(s) | |---|---|---|---| | Bernoulli | Discrete | Single yes/no trial | p ∈ [0,1] | | Binomial | Discrete | Count of successes in n trials | n, p | | Poisson | Discrete | Rare event counts in fixed interval | λ > 0 | | Normal | Continuous | Sum of many small effects | μ, σ | | Uniform | Continuous | No preference within range | a, b | | Exponential | Continuous | Waiting time between Poisson events | λ > 0 |
Python (in browser)
Six distributions in one view. Top row: Bernoulli (binary), Binomial (count of successes), Poisson (rare event counts). Bottom row: Normal (bell curve), Uniform (flat), Exponential (decay/waiting). Open /tmp/dist_zoo.png to view the full grid.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
The 6-row field guide maps each distribution to a real-world context and a machine learning use case. Recognize the process → name the distribution → use the right likelihood.
A decision guide for picking a distribution. The key question is always: what generated the data? Binary? → Bernoulli. Counting rare events? → Poisson. Continuous and symmetric? → Normal. Waiting times? → Exponential.
You want to model the number of customer support tickets received per day at a call center. Which distribution is most appropriate?
- **Bernoulli** models a single yes/no trial; **Binomial** extends it to n trials. Both are the foundation of binary classification.
- **Poisson** models rare event counts per fixed interval. If your target is a count and mean ≈ variance, reach for Poisson first.
- **Normal** is the universal default for continuous targets (justified by the Central Limit Theorem). MSE loss assumes Gaussian residuals — swap it for binary cross-entropy or Poisson loss when the assumption breaks.
Picking a distribution is picking a loss function. Logistic regression assumes Bernoulli; linear regression assumes Gaussian residuals; count models assume Poisson.
If you remove it: Without knowing which distribution models your target, you'll default to MSE for everything — and lose accuracy on count or skewed data.