1 · What is statistics?
Descriptive vs inferential statistics — and why every step of an ML pipeline is one or the other.
Statistics is the science of turning noisy data into trustworthy conclusions.
Without this:
Without it, ML degenerates into curve-fitting that looks good on training data and fails on real-world inputs.
Statistics is the discipline of collecting, organizing, analyzing, and interpreting data to draw conclusions about the world.
In machine learning, statistics is not optional background theory — it's the operating system underneath everything you do. Every train/test split you write is, at its core, an inference question: does the accuracy this model achieved on 20% of the data tell us how it will behave on all future inputs? The answer requires statistics to answer honestly.
Statistics has two main branches:
- Descriptive statistics: summarise the data you already have. Mean, median, standard deviation, histograms — tools that tell you what your dataset looks like.
- Inferential statistics: use a sample to make claims about a larger population you haven't fully observed. Confidence intervals, hypothesis tests, Bayesian updates — tools that let you generalise beyond your dataset.
Both branches live in every ML project.
Python (in browser)
Three canonical descriptive statistics: mean (centre), median (robust centre), and sample standard deviation (spread).
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Now the inferential angle. Suppose those 7 scores are a random sample from a population of many students. We want to claim something about the population mean — a number we cannot measure directly. A 95 % confidence interval is an inferential statement: "we're 95 % confident the true population mean lies in this range."
Python (in browser)
The t-interval is one of the simplest inferential tools: it generalises from 30 observed scores to a claim about the population.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Quick cheat sheet: every common stats tool maps to one of the two branches.
Is computing a sample mean descriptive or inferential?
Probability Spaces give the formal foundation: a sample space Ω, an event algebra, and a probability measure P. Everything in inferential statistics is built on top of these three objects.
- Statistics has two branches: **descriptive** (summarise what you have) and **inferential** (generalise to what you haven't seen).
- A confidence interval is an inferential statement — it makes a claim about a population parameter from sample data.
- In ML, EDA and reporting are descriptive; A/B tests, generalization bounds, and metric confidence intervals are inferential.
Every step of the ML pipeline is one or the other: EDA, feature engineering, and reporting are descriptive; A/B testing, generalization bounds, confidence intervals on metrics, and Bayesian inference are inferential.
If you remove it: You can train a model but you can't tell whether its 92% accuracy is real or luck — confidence intervals on metrics are inferential by construction.