Boost Time Series Forecasting with Autocorrelated Error Adjustment – A 5‑Line PyTorch Trick

Background

Time series forecasting aims to predict the value at time t using a sliding window of past observations. Formally, given a window of data

, the model predicts the next value

.

Paper Improvement

The paper observes that standard maximum‑likelihood training assumes independent errors across time steps, which contradicts many real‑world series where errors are autocorrelated. It introduces a learnable parameter ρ that models first‑order autocorrelated residuals, effectively transforming the input into a differenced form and adjusting the model’s output accordingly.

The modified prediction equation becomes:

and the loss is still computed with a standard regression loss (e.g., MSE).

Code Implementation

Standard forecasting code (PyTorch):

## x: input features X
prd_y = model(X)  # model prediction
loss = criterion(y, prd_y)  # usually MSE
loss.backward()  # back‑propagation

Adjusted forecasting with learnable autocorrelation:

## rho: learnable parameter
## bs: batch size
# prepend mean for differencing convenience
inp = torch.cat([X.mean(axis=0)[None].repeat(bs, 1, 1), x], dim=1)
# first‑order differencing with learnable rho
inp = inp[:, 1:] - rho * inp[:, :-1]  # input correction
prd_y = model(inp)  # model prediction
prd_y += rho * x[:, -1]  # output correction
loss = criterion(y, prd_y)  # same loss as before
loss.backward()

The authors note that a separate optimizer can be used to update ρ, though the example omits this detail.

Empirical results on many datasets show an average relative improvement of about 17% when the autocorrelation correction is applied.

Other Useful Time‑Series Tricks

Use first‑order or second‑order differencing as input features.

Treat the loss from a differenced prediction as an additional objective in a multi‑task weighted loss.

Include cumulative‑sum (cumsum) prediction loss as another auxiliary objective.

Combine forecasts from different window sizes or horizons as model ensembles, similar to techniques used in quantitative stock selection.

These tricks, together with the autocorrelation adjustment, form a practical toolbox for improving neural‑network‑based time‑series forecasting.