15 · ML forecasting
Turn forecasting into supervised learning: lag and rolling features become X, the next value becomes y, and a gradient-boosting regressor learns the mapping — then forecasts recursively over the horizon.
Any regressor (gradient boosting, random forest) can forecast once you frame it as supervised learning: lag + rolling features → next value. Multi-step forecasts come either recursively (feed predictions back as lags) or directly (one model per horizon).
Without this:
Without the supervised reframing you're locked into ARIMA-family models; with it, every gradient-boosting and tree trick you learned in the supervised track applies to time series — nonlinearity, feature importance, exogenous variables for free.
ARIMA and exponential smoothing are linear, hand-tuned models. But you already own a far bigger toolbox from the supervised track — gradient boosting, random forests, regularized regression. The trick to unlock them for forecasting is a reframing: turn the series into a supervised learning table.
For each time step t, build a feature row from the past only:
- Lag features —
y[t-1],y[t-2], … (the value 1, 2, … steps ago). - Rolling features — a moving average or std over the last
ksteps (smoothed recent level and volatility).
The target is y[t] itself. Now it's a plain regression problem, and any regressor learns the mapping. We use GradientBoostingRegressor, which captures nonlinear interactions a linear AR model can't.
The catch is multi-step forecasting. The horizon is several steps ahead, but to predict t+2 you'd need y[t+1] as a lag — which you don't have yet. Two strategies:
- Recursive — predict
t+1, then feed that prediction back as the lag to predictt+2, and so on. One model, but errors compound. - Direct — train a separate model for each horizon (
h=1,h=2, …), each predicting straight from features known today. No compounding, but more models.
Below we build lag + rolling features, train a gradient-boosting regressor respecting time order (no shuffling, train on the earlier portion), then forecast the test horizon recursively and compare MAE against the naive last-value baseline.
Python (in browser)
Forecasting as supervised learning: lag+rolling features train a GradientBoostingRegressor, then a recursive loop forecasts the horizon and beats the naive last-value baseline.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Notice the structure of the recursive loop: it keeps a running history list, and after each prediction it appends the prediction itself so the next step's lags are built from forecasts, not from the actuals we're pretending not to see. That faithfully simulates production, where you only have your own past outputs to lean on.
Which regressor should you reach for? Tree ensembles (gradient boosting, random forests) are the workhorses of tabular forecasting: they handle nonlinear trend and interaction between lags, need little scaling, and — crucially — give you feature importances so you can see whether lag1, the weekly lag7, or the rolling mean is carrying the signal. They also accept exogenous features for free (price, weather, promotions as extra columns), which classical ARIMA can only do awkwardly via SARIMAX. The ceiling, though, is that trees cannot extrapolate a trend beyond the range they saw in training — for a strongly trending series you often detrend first (forecast the differenced series, then cumulate) or add a time index feature with care.
Python (in browser)
Feature importances from a random forest reveal that lag1 and the weekly lag7 carry the forecasting signal — interpretability you don't get from raw ARIMA coefficients.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
In recursive multi-step forecasting, how do you build the lag features for the step you're predicting at horizon h > 1?
- Reframe forecasting as supervised learning: lag + rolling features are X, the next value is y — then any regressor (gradient boosting, random forest) applies.
- Build features from the past only and split by time; recursive multi-step forecasting feeds predictions back in as lags (errors compound), while direct trains one model per horizon.
- Tree ensembles bring nonlinearity, feature importances, and free exogenous variables — but cannot extrapolate a trend beyond the training range, so detrend strongly-trending series first.
Tree-based forecasters (LightGBM/XGBoost on lag features) win most retail-demand and energy-load competitions and power production forecasting at scale because they ingest exogenous drivers easily.
If you remove it: Without the supervised reframing you're stuck with linear ARIMA-family models and lose the entire gradient-boosting toolbox, nonlinearity, and easy exogenous features.