6 · MSE, MAE, RMSE, R²
Four numbers that say very different things about your regressor — learn when to reach for each.
Four numbers say very different things about your regressor — pick the one that matches your error cost.
Without this:
Without knowing the difference, you'll report MSE on a price target and unwittingly hide model failures on cheap items.
Once a regression model is trained you need a single number that summarises how wrong it is. The four most common choices are:
MSE — Mean Squared Error MSE = (1/n) · Σ(yᵢ − ŷᵢ)²
Errors are squared before averaging, so large errors are punished disproportionately. MSE has a clean probabilistic interpretation: it is the negative log-likelihood of the predictions under a Gaussian noise assumption. The natural gradient of MSE brings you to the OLS solution.
RMSE — Root Mean Squared Error RMSE = √MSE
Taking the square root brings the metric back to the same units as the target (e.g., dollars, degrees, kilometres). RMSE is the field default — Kaggle leaderboards, academic papers, and sklearn's default for regression all use it.
MAE — Mean Absolute Error MAE = (1/n) · Σ|yᵢ − ŷᵢ|
Absolute rather than squared errors make MAE far more robust to outliers. The corresponding noise model is Laplace (double-exponential) rather than Gaussian. The minimiser of expected MAE is the conditional median, not the conditional mean.
R² — Coefficient of Determination R² = 1 − SS_res / SS_tot where SS_res = Σ(yᵢ − ŷᵢ)², SS_tot = Σ(yᵢ − ȳ)²
R² measures the fraction of variance in y that the model explains. A perfect model gives R² = 1. A model that always predicts the mean gives R² = 0. A model worse than predicting the mean gives R² < 0 — yes, R² can be negative.
MAPE — Mean Absolute Percentage Error (brief mention) MAPE = (100/n) · Σ|yᵢ − ŷᵢ| / |yᵢ|
Useful when relative errors matter (e.g., 10% off on a $1 000 item is worse than 10% off on a $10 item). Breaks down completely when yᵢ ≈ 0 (division by near-zero explodes).
Python (in browser)
Notice RMSE > MAE on clean data because squaring amplifies even moderate errors. On this dataset they are fairly close — the gap widens dramatically once outliers appear.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
A single extreme outlier can multiply MSE by an order of magnitude while barely moving MAE. This is the core difference between the two metrics.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
R² = 0 for a model that always predicts the training mean. Any model performing worse than that baseline — for example, predicting a wildly wrong constant — gives a negative R².
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
This rubric won't be memorised in one read — bookmark it and return whenever you reach for an evaluation metric.
If you switch from MSE to MAE for training your regressor, the model will focus more on:
- RMSE = √MSE brings the error back to the target's units — it is the field default for regression evaluation.
- MAE is robust to outliers because it does not square errors; its probabilistic equivalent is Laplace noise.
- R² can be negative: any model worse than 'always predict the mean' gives R² < 0.
- Choose RMSE for symmetric errors, MAE for outlier-heavy targets, R² for cross-dataset comparison, MAPE when ratio errors matter (but not when y ≈ 0).
RMSE is the default leaderboard metric for Kaggle regression competitions. R² is the universal 'how good?' score in linear regression reports. MAE shows up in price prediction, ETA prediction, and any setting where outliers shouldn't dominate.
If you remove it: You'd report a single accuracy number with no error-cost context — a meaningless statement that fools stakeholders.