Skip to content
Time-Series Forecasting Survey: Autocorrelation Modeling Unifies 10 Years of Progress

Time-Series Forecasting Survey: Autocorrelation Modeling Unifies 10 Years of Progress

A TPAMI 2026 survey uses autocorrelation modeling as a unified lens to categorize ten years of time-series forecasting advances, covering backbone architectures (RNN, CNN, Transformer, Mamba, LLM alignment) and loss functions (likelihood-based, shape alignment, distribution alignment, conditional generation), plus future research directions.

Machine Heart
Machine Heart
Machine Heart

Introduction

Over the past decade, time-series forecasting has seen widespread application in finance, energy, and manufacturing, with diffusion models and time-series foundation models emerging in succession. The broad application demand and active method innovation have jointly driven rapid progress in the field. Against this background, the team led by Lin Zhuchen at Peking University released a survey paper that systematically reviews the research progress of the last ten years.

Paper Metadata

Title: Deep Time-Series Forecasting in 10 Years: A Survey

arXiv: https://arxiv.org/abs/2603.19899

IEEE: https://ieeexplore.ieee.org/document/11731513

GitHub Repository: https://github.com/Master-PLC/Awesome-TSF-Papers

1. Autocorrelation Modeling Is the Key to Time-Series Forecasting

In a time series, observations of the same variable at different time points are often interrelated; this autocorrelation property is a key feature distinguishing time series from general tabular data. For example, today's temperature is usually related to yesterday's. As shown in Figure 1, different time-series patterns correspond to different autocorrelation structures. How to effectively model this autocorrelation structure is the central challenge of time-series forecasting.

Figure 1: Common time-series patterns and their autocorrelation functions. Trend series show strong positive correlation; periodic series exhibit alternating positive/negative correlation with lag; seasonal series show significant peaks at integer multiples of the seasonal period.
Figure 1: Common time-series patterns and their autocorrelation functions. Trend series show strong positive correlation; periodic series exhibit alternating positive/negative correlation with lag; seasonal series show significant peaks at integer multiples of the seasonal period.

In the statistical modeling era, classic methods such as ARIMA explicitly modeled autocorrelation through lag terms. In the deep learning era, the challenge persists: autocorrelation exists both in the input historical sequence and in the future label sequence that serves as the supervision signal. Accordingly, the paper identifies two key questions from the autocorrelation perspective:

How to design the backbone architecture to model autocorrelation in the input sequence?

How to design the loss function to model autocorrelation in the label sequence?

2. Taxonomy Centered on Autocorrelation Modeling

Figure 2 presents the paper's literature classification framework. The first level splits into two main lines — backbone architecture and loss function — corresponding to the two key questions above. The second level further refines each line by specific modeling strategy. The two lines receive roughly equal coverage, both centered on autocorrelation modeling but focusing on different objects: backbone architecture focuses on the input sequence (X), while loss function focuses on the label sequence (Y) . This framework provides a unified perspective for tracing the evolution of both categories of methods.

Figure 2: Literature classification framework centered on autocorrelation modeling.
Figure 2: Literature classification framework centered on autocorrelation modeling.

3. Backbone Architecture Lineage: Modeling Autocorrelation in the Input Sequence

Non-Transformer Architectures

Figure 3: Overview of representative non-Transformer architectures.
Figure 3: Overview of representative non-Transformer architectures.

The paper first reviews traditional forecasting models.

RNN was once the go-to choice for sequence modeling: its hidden state updates recursively over time, naturally fitting the autocorrelation property. However, this mechanism brings two limitations. First, traditional RNNs rely on the previous step's result for state updates, making parallel computation along the time dimension difficult. Second, the finite-dimensional hidden state must continuously compress historical information, making long-range dependencies hard to preserve stably. Recently, selective state-space models such as Mamba have attracted attention; they can be understood from these two aspects: parallel scanning improves long-sequence processing efficiency, and HiPPO enhances memory units to achieve theoretically lossless memory.

CNN was initially used mainly for image processing; when applied to time series, 1D convolution aggregates information from adjacent time steps to capture local autocorrelation relationships . However, the receptive field of a single standard convolution layer is limited. How to capture longer-span historical information became a focus of subsequent methods. One line of work expands the effective receptive field in the time domain — e.g., dilated convolution, large-kernel separable convolution — to cover longer history while controlling computational cost. Another line uses frequency-domain filtering to obtain input representations — e.g., FiLM, FilterNet — based on the convolution theorem, where frequency-domain filtering is equivalent to full-receptive-field time-domain convolution.

DLinear demonstrated the effectiveness of simple linear models, achieving performance comparable to complex models on some benchmark tasks. However, linear mappings struggle to fully capture nonlinear, non-stationary, and multi-period-interleaved autocorrelation characteristics . Around this limitation, follow-up methods developed along two directions. One transforms data to reduce modeling difficulty: DLinear uses decomposition to separate components and process them individually; RLinear uses normalization to mitigate distribution shifts. The other enhances model expressiveness: TiDE and TSMixer introduce MLPs to model nonlinearity; TimeMixer further enhances expressiveness through multi-scale modeling.

Transformer Architectures

Figure 4: Overview of representative Transformer architectures. Circular nodes represent model input and output; colored blocks indicate main modifications that adapt Transformer to time-series forecasting.
Figure 4: Overview of representative Transformer architectures. Circular nodes represent model input and output; colored blocks indicate main modifications that adapt Transformer to time-series forecasting.

The paper then surveys Transformer-based forecasting models. Transformers use self-attention to model autocorrelation; however, directly using a single time-point observation as input often fails to capture temporal semantic information. For instance, a single temperature value carries limited information; only by combining changes from adjacent time steps can one judge whether it is a warming trend or a short-term anomaly. To solve this, the Tokenizer groups consecutive time steps into a token, allowing the model to first acquire local context and then learn long-range dependencies between segments. This design later became almost standard in Transformer time-series models.

Building on this, researchers further asked: Can the powerful sequence modeling ability of LLMs be used for time-series autocorrelation modeling? To achieve this, the input sequence representation must be aligned with the LLM's input representation . The survey summarizes four alignment paradigms:

Query paradigm: rewrites time-series information into text prompts and directly uses the LLM for prediction, but performance is limited by the ability to express numerical information.

Reprogramming paradigm: maps time-series segments into the word embedding space, adapting the LLM without fine-tuning.

Fine-tuning paradigm: updates a small number of parameters to adapt the pre-trained model to time-series data.

Pre-training paradigm: directly trains dedicated foundation models on large-scale time-series data, such as Timer-XL and Chronos , demonstrating zero-shot forecasting capability.

4. Loss Function Lineage: Modeling Autocorrelation in the Label Sequence

Existing surveys often treat loss functions as implementation details, mentioning them only in passing. This paper argues that the loss function determines the optimization direction during training and deserves equal attention. This issue is particularly prominent in time-series forecasting: future time points are not isolated from each other; whether the loss function can effectively model the autocorrelation property of the label sequence is a major challenge . Take mean squared error (MSE) as an example: it computes and aggregates prediction errors pointwise. Under a probabilistic interpretation, if one further assumes that prediction errors at each step are mutually independent, minimizing MSE is equivalent to maximizing likelihood. However, once the label sequence exhibits autocorrelation, the independence assumption no longer holds, and MSE becomes a biased loss relative to the true likelihood function. Therefore, how to model label-sequence autocorrelation in the loss function is a distinctive problem in time-series forecasting. Around this question, existing methods can be organized into four categories.

Category 1: Likelihood-Based Losses with Accurate Construction

These methods still use the likelihood function as the loss but strive to construct it accurately, following two paths:

Label transformation: Since MSE implicitly assumes uncorrelated labels, a natural approach is to first transform labels into a decorrelated space and then compute the loss there. Representative methods include FreDF 's discrete Fourier transform and Time-o1 's PCA whitening; both ensure the resulting label components are decorrelated, making MSE's decorrelation assumption closer to reality.

Covariance modeling: Instead of transforming labels, this path explicitly estimates the conditional covariance matrix of the labels and uses it to compute the full negative log-likelihood as the training objective. For example, QDF employs meta-learning to obtain the conditional covariance matrix.

Figure 6: Computation process of likelihood-based loss functions.
Figure 6: Computation process of likelihood-based loss functions.

Category 2: Shape Alignment Losses

The basic assumption is: autocorrelation manifests in the waveform of the label sequence , so the loss function should focus on whether the predicted waveform matches the true waveform , rather than the magnitude of pointwise errors. Dynamic Time Warping (DTW) is a classic method for measuring waveform difference, but it is non-differentiable and cannot be used directly for gradient optimization. SoftDTW applies a soft relaxation to DTW, making it everywhere differentiable and thus supporting end-to-end training. Such losses are especially effective in scenarios requiring phase invariance: for example, in ECG signals, a prediction that lags by half a beat but has the correct waveform shape is far more valuable than a prediction with distorted shape.

Figure 7: Computation process of shape-alignment loss functions.
Figure 7: Computation process of shape-alignment loss functions.

Category 3: Distribution Alignment Losses

The core idea: autocorrelation properties are embodied in the conditional distribution of the label sequence ; therefore, constructing a loss that minimizes the distance between the predicted sequence's conditional distribution and the label sequence's conditional distribution incorporates label autocorrelation into the training objective. The key lies in the choice of probability distribution distance. Existing approaches fall into two groups: one directly uses existing distribution distances as the loss — e.g., DistDF uses Wasserstein distance, KMB-DF uses MMD distance; the other borrows from adversarial learning, adaptively learning the distribution distance — training a discriminator that cannot distinguish predicted sequences from label sequences, thereby achieving distribution alignment.

Figure 8: Computation process of distribution-alignment loss functions.
Figure 8: Computation process of distribution-alignment loss functions.

Category 4: Conditional Generation Losses

These methods no longer directly regress future values but treat forecasting as a generation task, leveraging the inference mechanism of generative models to model autocorrelation. There are two paths: one is diffusion generation represented by CSDI , which generates future sequences through stepwise denoising; the other is autoregressive generation represented by DeepAR , where each generation step depends on previous outputs, explicitly decomposing the future into a chain of dependencies. The core advantage of the conditional generation paradigm is that it fundamentally avoids autocorrelation bias — the generation process itself assumes a sequential dependency structure, eliminating the need for an independence assumption like that in MSE.

Figure 9: Computation process of conditional-generation loss functions.
Figure 9: Computation process of conditional-generation loss functions.

5. Future Outlook: Research Directions Around Autocorrelation Modeling

The above review shows that autocorrelation modeling is a central thread running through time-series forecasting research, driving the proposal and improvement of numerous methods. On this basis, the paper outlines future research directions from four aspects; two are highlighted here:

First, addressing real-world autocorrelation modeling challenges. Real-world time-series data often suffer from measurement noise, distribution drift, and missing observations. For measurement noise , one can explore denoising learning for continuous labels . For distribution drift , techniques such as domain adaptation, domain generalization, and test-time adaptation can be introduced to balance responsiveness to distribution changes with robustness. For missing observations , the common practice is to impute first and then predict, but imputation errors may distort the temporal structure and degrade prediction quality. Therefore, direct prediction from incomplete observations and fusion of multi-sampling-rate data are equally worthy of study.

Second, transferring autocorrelation modeling methods to variable dependency modeling. In multivariate forecasting, correlations between different variables or channels also affect prediction performance. The vast majority of methods used for autocorrelation modeling (e.g., self-attention, label transformation) can be directly extended to the variable dimension ; however, how to design network architectures and loss functions that simultaneously capture both autocorrelation and variable dependency remains a large open research space.

Conclusion

In recent years, various forecasting models have developed rapidly, with new methods constantly emerging. With the unified perspective of autocorrelation modeling, this paper attempts to answer two interrelated questions: What is the core challenge of deep time-series forecasting? How do existing methods address it? Autocorrelation is no longer an abstract term in statistics textbooks but a crucial object that must be modeled when designing forecasting methods in the deep learning era. Whether designing better architectures or constructing more reasonable loss functions, one cannot bypass the same question: have we better understood and modeled the intrinsic dependency structure of time series? The understanding and modeling of autocorrelation remains the distinctive and core problem of this field.

Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Sign in to view source
Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactadmin@besthub.devand we will review it promptly.

deep learningtime-series forecastingloss functionsurveyautocorrelation modelingbackbone architecture
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.