How Pathformer Redefines Multi-Scale Time Series Forecasting with Adaptive Pathways

Opening

The paper Pathformer: Multi-Scale Transformers With Adaptive Pathways For Time Series Forecasting was accepted at ICLR 2024. It proposes an adaptive multi‑scale time‑series forecasting model built on a Pathways architecture, modeling both time resolution and time distance, and achieves state‑of‑the‑art results on Alibaba Cloud datasets and public benchmarks with good generalization and transferability.

Background

In real‑world scenarios, time series such as CPU, GPU, and memory demand exhibit distinct patterns at daily, monthly, and seasonal scales. Multi‑scale modeling extracts temporal features and dependencies across these scales, requiring consideration of two aspects:

Time resolution : the size of each time segment used for modeling (e.g., fine‑grained vs. coarse‑grained patches).

Time distance : the temporal gap between steps, influencing local detail extraction versus long‑range global correlation.

Challenges

The existing Transformer‑based multi‑scale approaches face two main challenges:

Incomplete multi‑scale modeling: focusing solely on time resolution fails to capture diverse temporal dependencies, while emphasizing time distance alone is limited by data partitioning.

Fixed multi‑scale process: a single scale configuration cannot accommodate series that require different granularities (e.g., rapid fluctuations vs. long‑term trends), and manually tuning scales for each dataset is time‑consuming.

Solution

We introduce Pathformer , an adaptive multi‑scale Transformer that integrates both time resolution and time distance through a novel multi‑scale Transformer module and adaptive pathways.

(1) Multi‑scale Transformer module : The input series is divided into patches of varying sizes, representing different resolutions. A dual‑attention mechanism is applied:

Intra‑patch attention captures local details within each patch.

Inter‑patch attention captures global relationships across patches.

(2) Adaptive Pathways : A multi‑scale router selects appropriate patch sizes based on the input dynamics, activating specific Transformer components. An aggregator then combines the extracted features via weighted aggregation, enabling the model to adapt its scale configuration on the fly.

Experiments and Results

Extensive experiments on three real‑world clusters of Alibaba Cloud’s native big‑data service MaxCompute and on public datasets show that Pathformer significantly outperforms existing time‑series forecasting models, demonstrating strong generalization and transfer capabilities across different workloads.

Application

The algorithm has been integrated into the Feitian Big Data AI Control Platform (ABM) as an algorithm service, supporting intelligent operation scenarios such as resource recommendation.