7 · Loss functions: which one for which task
MSE, MAE, Huber, BCE, CCE — the menu of loss functions and the probabilistic assumption hidden inside each.
Picking the loss = picking the noise model: MSE for Gaussian regression, MAE for robust regression, BCE for binary, CCE for multi-class, Huber/quantile for special cases.
Without this:
Wrong loss = wrong objective = your model optimizes the wrong thing.
Every loss function is secretly a negative log-likelihood under some probabilistic model:
| Loss | Implicit noise model | When to use | |------|---------------------|-------------| | MSE | Gaussian errors | Regression, clean data | | MAE | Laplace errors | Regression, outlier-robust | | Huber| Gaussian + Laplace | Best of both worlds | | BCE | Bernoulli | Binary classification | | CCE | Categorical | Multi-class classification |
This connection is not academic trivia — it tells you exactly what distributional assumption you're making and whether it matches your data. Choosing MSE when errors are heavy-tailed (few massive outliers) means those outliers dominate the loss and the model chases them.
Before writing a single line of network code, ask: "What is the probabilistic model of my outputs?"
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Loss selection cheat sheet — regression, classification, and the MLE connection
Which loss for predicting house prices where some are mansions (outliers)?
- Every loss function = a negative log-likelihood under some probabilistic assumption. Choosing wrong = optimizing the wrong thing.
- MSE punishes outliers quadratically; MAE treats all errors linearly; Huber combines both — use Huber when your data has occasional large errors.
- BCE for binary classification, CCE for multi-class — using MSE for classification tasks produces non-calibrated probabilities and slow convergence.
- PyTorch: `nn.MSELoss`, `nn.L1Loss`, `nn.HuberLoss`, `nn.BCEWithLogitsLoss`, `nn.CrossEntropyLoss` — each pairs with the right architecture output (logits vs. probabilities).
Picking the right loss is the difference between a model that learns and one that doesn't. Stats track ch7 covered MLE; every loss IS an MLE under some distribution.
If you remove it: You'd default to MSE everywhere — which silently fails for classification, and silently overweights outliers in regression. The wrong loss is the most insidious source of poor model performance.