Paper Review: HGTS‑Former – A Hierarchical Hypergraph Transformer for Multivariate Time‑Series Analysis

Background

Multivariate time‑series (MTS) analysis is widely used in fusion, stock prediction, weather monitoring and other domains. Existing Transformer‑based models (e.g., iTransformer, Timer‑XL) capture global relations with self‑attention but struggle to model high‑order dependencies. Graph‑based methods (e.g., CrossGNN, MSHyper, Ada‑MSHyper) can encode higher‑order interactions but are limited by binary edges or restricted receptive fields.

Problem Definition

Traditional GNNs model only binary relations, limiting high‑order interaction modeling. Hypergraph models capture higher‑order relations but suffer from limited receptive fields when stacking layers and weak dynamic/heterogeneous relation modeling. The goal is a hierarchical hypergraph‑based Transformer with global receptive field and adaptive attention to model intra‑variable patterns and inter‑variable dynamics.

Method

Input Representation

InstanceNorm normalizes the input tensor X \in \mathbb{R}^{B \times C \times L} (batch size B, variable count C, length L). The series is split into non‑overlapping blocks X_p and embedded into a shared feature space.

Time Representation Enhancement (MHSA)

Multi‑head self‑attention (MHSA) enriches each block’s temporal representation. Rotational Position Encoding (RoPE) injects sequential order. Linear layers W_Q, W_K, W_V, W_P compute queries, keys, values and projections.

Hierarchical Hypergraph Aggregation

Intra‑HyperGraph

Hyper‑edge generation uses a learnable query Q to capture intra‑variable patterns. Cosine similarity between nodes and Q yields a confidence matrix M_{conf}, passed through a sigmoid and TOPK to produce adjacency and mask matrices. Hyperparameter \alpha down‑weights contributions from non‑hyperedge nodes.

Cross‑attention (with mask) aggregates node features into hyper‑edge features, followed by LayerNorm and a feed‑forward network (FFN).

Inter‑HyperGraph

Hyper‑edges from the intra‑graph become nodes for the inter‑hypergraph. A global query Q_G derived from the raw series guides generation and aggregation of inter‑variable hyper‑edges.

Edge‑to‑Node Conversion

Attention converts hyper‑edge features back to node features, with residual connections and an FFN to enhance node representations.

Loss Function

For forecasting and interpolation, a linear output head followed by inverse normalization (RevIN) restores the original scale. The training objective is mean squared error (MSE) between predictions and ground‑truth Y over horizon T.

Experiments

Datasets and Metrics

Eight MTS datasets: ETTh1, ETTh2, ETTm1, ETTm2, ECL, Traffic, Weather, Solar‑Energy (7–862 variables, sampling 10 min–1 h). Evaluation metrics are MSE and MAE.

Implementation Details

Hyper‑parameters explored: number of layers L (1–4), model dimension d_{model} (256–1024), feed‑forward dimension d_{ff} (1024–2048), block length P (16–96), number of hyper‑edges (3–8).

Forecasting Results

Across 32 tasks (8 datasets × 4 horizons), HGTS‑Former achieves 15 first‑place MSE scores and 27 first‑place MAE scores, surpassing iTransformer, PatchTST, TimeMixer++ and other baselines. The advantage is especially pronounced for long horizons (336/720 steps), where error growth is far smaller.

Interpolation Results

On six datasets with random masking rates of 12.5 %–50 %, HGTS‑Former consistently yields lower MSE and MAE than baselines. Example: on ETTm1, MSE = 0.040 vs. TimeMixer++ = 0.041 (‑2.4 %); on ECL, MSE = 0.051 vs. 0.109 (‑53.2 %).

Ablation Study

Removing MHSA+RoPE raises average MSE/MAE by 0.047/0.037. Removing the intra‑hypergraph adds 0.013/0.006, and removing the inter‑hypergraph adds 0.011/0.007, confirming that each component is essential.

Parameter Sensitivity

Varying look‑back length, \alpha, and layer count shows HGTS‑Former is robust to hyper‑parameter changes; increasing layers continuously improves performance without saturation.

Visualization and Efficiency

Hypergraph visualizations demonstrate effective aggregation of intra‑variable patterns and dynamic inter‑variable relations. The model contains 10.38 M parameters, trains at 0.0133 s per iteration, and occupies 738 MB GPU memory, indicating practical deployment feasibility.

References

Paper: https://arxiv.org/pdf/2508.02411<br/>Code: https://github.com/Event-AHU/Time_Series_Analysis