7 · AR & MA models
The two atoms of classical forecasting: AR regresses today on past VALUES, MA regresses today on past ERRORS. Everything in the ARIMA family is built from these two.
An AR(p) model predicts the next value as a weighted sum of the p most recent VALUES; an MA(q) model predicts it as a weighted sum of the q most recent forecast ERRORS (shocks). They capture different kinds of memory, and combining them gives ARMA.
Without this:
Without AR and MA you can't read what a model is actually doing — you'd treat ARIMA as a black box instead of knowing whether it remembers past levels, past surprises, or both.
Classical forecasting builds everything from two simple ideas about memory.
Autoregressive — AR(p). The next value is a weighted sum of the most recent values, plus a fresh shock:
y_t = c + φ₁·y_{t-1} + φ₂·y_{t-2} + … + φ_p·y_{t-p} + ε_t
It's literally a linear regression of the series on lagged copies of itself (hence auto-regressive). The coefficients φ say how strongly the recent past pulls the present. If φ₁ = 0.7, today is mostly 70% of yesterday plus noise.
Moving Average — MA(q). The next value is the mean plus a weighted sum of the most recent errors (the surprises the model made), plus a fresh shock:
y_t = μ + ε_t + θ₁·ε_{t-1} + θ₂·ε_{t-2} + … + θ_q·ε_{t-q}
This is a different memory: it remembers shocks, not levels. A one-off spike (a flash sale) ripples forward for exactly q steps and then vanishes. (Confusingly, this "moving average" has nothing to do with a rolling-mean smoother — it's a regression on past residuals.)
The practical difference shows up in forecasts. An AR model's forecast decays smoothly toward the mean over many steps — it has long, fading memory of past levels. An MA(q) model's forecast returns to the mean after exactly q steps, because beyond q lags there are no known errors left to lean on. Below we generate a true AR(1) series, fit both an AR(1) and an MA(1) with statsmodels, recover the coefficients, and watch the two forecast shapes differ.
Python (in browser)
Fitting AR(1) and MA(1) with statsmodels ARIMA. The recovered phi≈0.7 matches the true AR coefficient; AR forecasts fade slowly while the MA(1) forecast jumps to the mean after one step.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
What is the key difference between an AR(p) and an MA(q) term?
- AR(p): y_t = c + φ₁y_{t-1} + … + φ_p y_{t-p} + ε_t — a regression on the series' own past VALUES.
- MA(q): y_t = μ + ε_t + θ₁ε_{t-1} + … + θ_q ε_{t-q} — a regression on past ERRORS (shocks), not a rolling mean.
- AR forecasts decay smoothly toward the mean; an MA(q) forecast returns to the mean after exactly q steps.
AR and MA terms are the building blocks of ARMA, ARIMA, SARIMA, and SARIMAX — the workhorses of classical demand and finance forecasting.
If you remove it: Without understanding AR vs MA you can't read PACF/ACF plots to choose orders, and ARIMA becomes an opaque knob you tune blindly.