How MSH‑LLM Fuses Multi‑Scale Hypergraphs with Large Language Models for Time‑Series Analysis

Background Time‑series analysis is crucial for forecasting, imputation, and classification in domains such as retail, traffic, economics, weather, and healthcare. While pretrained large language models (LLMs) have succeeded in NLP and CV, directly applying them to time‑series tasks is limited because existing methods ignore the multi‑scale structures inherent in both natural language and time‑series data.

Problem Definition The authors identify two main challenges: (1) a mismatch between the multi‑scale semantic spaces of language and time‑series, leading to sparse semantic information in the latter; and (2) pretrained LLMs lack intrinsic knowledge of time‑series patterns and reasoning abilities. The goal is to design a method that jointly captures multi‑scale semantics and aligns the two modalities.

Method

MSH‑LLM re‑programs viewable embeddings of LLMs (e.g., LLaMA, GPT‑2) for general time‑series analysis while explicitly modeling multi‑scale structures. The architecture consists of four key components:

3.1 Multi‑Scale Extraction (ME) Module

Given an input time‑series X^{1}, reversible instance normalization is first applied. Multi‑scale temporal features are then extracted at each scale s using an aggregation function (e.g., 1‑D convolution or average pooling):

Here X^{s} denotes the subsequence at scale s, Agg is the aggregation function, and θ^{s‑1} are learnable parameters for the previous scale.

For textual prototypes, token embeddings U from a pretrained LLM are linearly projected to a small set of prototypes U^{1} ( V' << V). Subsequent scales are obtained by further linear mappings:

3.2 Hyperedge Mechanism

Multi‑scale temporal features are treated as nodes. For each scale s, two learnable embeddings are initialized: hyperedge embeddings and node embeddings. The number of hyperedges at scale s is M^{s}. Similarity scores are computed to build a scale‑specific adjacency matrix, followed by a tanh non‑linear transformation and a ReLU to prune weak connections. Sparsity is enforced with a threshold η (TopK):

The hyperedge feature for the i ‑th hyperedge e_{i}^{s} is obtained by aggregating neighboring node features N(e_{i}^{s}):

3.3 Cross‑Modal Alignment (CMA) Module

For a given scale s, textual prototypes U^{s} and hyperedge features E^{s} are projected to query, key, and value vectors and aligned via multi‑head cross‑attention. The outputs of all heads Z_{k}^{s} are aggregated to produce the aligned representation for scale s:

3.4 Mixed Prompt (MoP) Mechanism

Three types of learnable prompts are introduced:

Learnable prompts : scale‑specific vectors P^{s} of length L^{s} trained from the loss between LLM outputs and true task labels.

Data‑related prompts : composed of data description π, task description τ, and data statistics μ.

Capability‑enhancing prompts : include logical reasoning φ, affective control φ, and time‑series reasoning ψ.

3.5 Output Projection

The concatenated learnable prompts and aligned features are fed into the LLM. After a final linear projection and instance de‑normalization, the model produces the prediction.

Experiments

4.1 Experimental Setup The method is evaluated on 27 real‑world datasets covering long/short‑term forecasting, classification, few‑shot and zero‑shot learning. LLaMA‑7B serves as the base LLM; experiments are repeated three times and averaged. Adam optimizer is used with learning rates selected from {1e‑3, 5e‑3, 1e‑4}. The total number of scales S is set to 3, and 1‑D convolution is the aggregation function. Hyper‑parameters are searched automatically with the NNI toolkit.

4.2 Datasets Long‑term forecasting and few‑shot learning use seven standard datasets (ETTh1, ETTh2, ETTm1, ETTm2, Weather, Traffic, Electricity). Short‑term forecasting and zero‑shot learning use the M4 dataset, while M3 and M4 are used for zero‑shot evaluation. Time‑series classification is evaluated on ten multivariate datasets from the UEA archive.

4.3 Results

Long‑term forecasting : MSH‑LLM achieves the best results on all seven datasets, reducing MSE/MAE by an average of 4.10%/3.72% vs. LLM4TS, 8.54%/6.45% vs. the latest Transformer‑based methods, and 7.48%/5.58% vs. linear baselines.

Short‑term forecasting : On M4, MSH‑LLM slightly outperforms AutoTimes and significantly surpasses other baselines, attributed to the hyperedge mechanism that enriches multi‑scale semantic information while suppressing irrelevant noise.

Time‑series classification : MSH‑LLM attains an average accuracy of 75.38%, exceeding the advanced LLM4TS method FPT (74%).

Few‑shot learning : With only 5% or 10% of training data, MSH‑LLM consistently yields the best performance; on 5% data, MSE and MAE drop by 10.47% and 6.74% on average compared to other LLM4TS variants.

Zero‑shot learning : On M3 and M4, MSH‑LLM reduces SMAPE by an average of 10.23% over all baselines.

4.4 Parameter Study

On the Traffic dataset, the best performance is achieved with η = 4. Smaller values fail to capture inter‑group interactions, while larger values introduce noise. The optimal number of scales is 3; fewer scales limit expressive power, and more scales cause over‑parameterization and over‑fitting.

4.5 Ablation Study

LLM choice : Variants using the first 12 Transformer layers of LLaMA‑7B, replacing LLaMA‑7B with GPT‑2‑Small, or using only the first 6 layers of GPT‑2‑Small all underperform the default MSH‑LLM (32 layers), confirming that scaling laws apply to frozen LLMs in cross‑modal alignment.

MoP mechanism : Removing learnable prompts, data‑related prompts, capability‑enhancing prompts, or the entire MoP module degrades performance, demonstrating the effectiveness of each prompt component and the overall MoP design.

4.6 Visualization

MoP visualization : t‑SNE on the Traffic dataset shows that outputs with MoP form distinct clusters, whereas outputs without MoP are scattered, indicating that MoP activates the LLM’s ability to capture multi‑scale temporal patterns.

Hyperedge embedding visualization : t‑SNE on the ETTh1 dataset reveals that, as training progresses, hyperedge embeddings at different scales form clear clusters. Early training focuses on coarse patterns, while later stages capture finer temporal dynamics.