3 · Imbalanced datasets + SMOTE
When 99% of labels are negative, accuracy is a lie — learn to spot it and fix it.
When 99% of your labels are negative, accuracy is a lie — and your model will learn 'predict negative for everything' as a near-perfect cheat.
Without this:
Without rebalancing or appropriate metrics, your fraud/disease/churn classifier achieves 99% accuracy and catches zero positives.
In a balanced dataset, class proportions are roughly equal (say 50/50). In the real world, the interesting class is almost always rare:
- Fraud transactions: ~0.1% of all transactions.
- Cancer diagnoses: often < 1% of screened population.
- Churn events: typically 2–5% of a subscriber base.
- Click-throughs: often < 0.5% of impressions.
A naive classifier that predicts "majority class for every sample" achieves 99% accuracy on a 99/1 dataset — and is completely useless. This is the accuracy paradox, and it's the first pitfall every ML engineer falls into.
The fix has two parts:
- Switch to the right metrics: precision, recall, F1, balanced accuracy, ROC-AUC, PR-AUC.
- Rebalance the data or the loss: class weights, oversampling (SMOTE), undersampling.
Python (in browser)
The majority-class baseline achieves 99% accuracy by predicting zero positives — a perfect demonstration of why accuracy is meaningless on imbalanced data.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
`class_weight='balanced'` penalises misclassification of the minority class proportionally more — recall jumps at the cost of some precision.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
SMOTE-by-hand: for each minority point, pick a random neighbour, then sample a new point along the connecting line at a random fraction between 0 and 1.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
The hierarchy: class weights → SMOTE → undersampling. Use the simplest intervention that achieves adequate minority recall.
On a 99/1 imbalanced dataset, a model that always predicts 'majority' achieves what accuracy?
- The **accuracy paradox**: on a 99/1 dataset, a model that predicts 'majority always' achieves 99% accuracy and catches zero positives — accuracy is the wrong metric.
- Use **balanced accuracy**, **F1**, **ROC-AUC**, and **PR-AUC** for imbalanced problems. PR-AUC is most honest for rare-event scenarios.
- Fix hierarchy: try `class_weight='balanced'` first → SMOTE second → undersampling only if the majority class is massive.
- SMOTE synthesises new minority points by interpolating between existing minority neighbours. **Always apply it AFTER the train/test split, only to the training set.**
Fraud detection (~0.1% positives), medical diagnosis of rare diseases, churn prediction, click-through rate prediction. Every production binary classifier on rare events has SOME form of imbalance handling — class weights, focal loss, or resampling.
If you remove it: Your model 'works' on training metrics but is operationally useless because it never flags the cases you actually care about.