58 · Split, cross-validate, search
A single split is noisy; cross-validation averages over folds, and hyperparameter search automates trying configurations — all operating on the Pipeline as one object.
A single train/test split gives one noisy number; cross-validation re-trains on several folds and averages, giving a far more reliable estimate. Hyperparameter search (`GridSearchCV`, `RandomizedSearchCV`) automates trying many configurations — and because the Pipeline is a single estimator, all of this operates on the whole preprocessing-plus-model object at once.
Without this:
Judge a model on one arbitrary split and you might report a score that's lucky or unlucky by several points; tune hyperparameters by hand and you'll explore a tiny corner of the configuration space, missing better models entirely.
One split is a coin flip
A single train_test_split carves your data into one training portion and one test portion. The resulting score depends heavily on which rows happened to land in test. Re-run with a different random_state and the number can swing by several percentage points — that's noise, not signal.
Cross-validation removes most of that noise. K-fold CV splits the data into k equal folds, then trains k separate models: each one holds out a different fold for testing and trains on the other k − 1. Averaging the k scores gives a far more stable estimate, and the spread (standard deviation) tells you how sensitive the model is to the particular split.
Stratification matters when classes are imbalanced. StratifiedKFold (and train_test_split(..., stratify=y)) preserves the class proportions in every fold, so a 90/10 dataset doesn't accidentally produce a fold with almost no minority examples.
Hyperparameter search automates the tedium of trying configurations:
GridSearchCV— exhaustively tries every combination in a grid, scoring each with cross-validation.RandomizedSearchCV— samples a fixed number of random configurations, which scales far better when the grid is large.
The crucial point: all of these accept a Pipeline as their estimator. You hand them the whole preprocessing-plus-model object, and they re-fit everything inside each fold — so cross-validation stays leakage-free automatically.
Python (in browser)
On a 90/10 imbalanced set, `stratify=y` forces both the train and test splits to mirror the overall class-1 proportion. Without it, the random split can drift, leaving the test set with an unrepresentative class balance — and an unreliable score.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
`cross_val_score` re-fits the entire Pipeline on each of five stratified folds and returns one score per fold. The mean is your headline estimate; the standard deviation quantifies how much a single split could have misled you.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
`GridSearchCV` scores every value in `param_grid` with cross-validation. The key `model__C` uses the `step__param` namespace — `model` is the pipeline step, `C` is its hyperparameter. After fitting, `best_params_` and `best_score_` report the winner.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
`RandomizedSearchCV` samples `n_iter` configurations instead of trying them all. Here we pass a plain list of candidate `C` values (Pyodide-safe — no `scipy.stats` distributions) and draw 5 at random. It's the go-to when an exhaustive grid would be too expensive.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
In a `GridSearchCV` over a Pipeline, what does the key `model__C` mean?
- A single split is noisy; k-fold cross-validation re-trains on several folds and averages, giving a stable estimate plus a standard deviation that flags split sensitivity.
- `stratify=y` / `StratifiedKFold` preserves class proportions in every fold — essential for imbalanced data so no fold loses the minority class.
- `GridSearchCV` tries every combination; `RandomizedSearchCV` samples `n_iter` configs for scalability. Both target Pipeline steps via the `step__param` double-underscore syntax.
Every model-selection workflow in scikit-learn runs on this trio: split honestly, cross-validate to estimate, and search to tune. `GridSearchCV`/`RandomizedSearchCV` over a Pipeline is the standard pattern for producing a tuned, leakage-free model.
If you remove it: Without cross-validation you'd trust one lucky-or-unlucky split; without automated search you'd hand-tune a handful of configurations and almost certainly leave performance on the table.