A Quick Survey of Recent Time Series Representation Learning Papers

I. Contrastive Learning Paradigm

Contrastive learning creates positive and negative sample pairs via data augmentation, encouraging positive pairs to be close and negatives far in feature space, thereby extracting valuable features from different augmented views.

TS2Vec[1] builds positive pairs by randomly cropping two subsequences that share a common part, then applies a timestamp‑level binary mask after an input projection. The premise is that representations of the same timestamps in both views should be consistent. TS2Vec also leverages hierarchical contrastive learning across multiple time granularities (hour, day, week) by pooling representations from coarse to fine granularity, enabling learning of representations from timestamp level up to sequence level.

TF‑C[2] improves cross‑domain generalization by enforcing “time‑frequency consistency”: in feature space, the time‑domain and frequency‑domain representations of the same series are similar (forming positive pairs), while those of different series are dissimilar (forming negative pairs). TF‑C constrains contrastive loss in time, frequency, and joint time‑frequency spaces and introduces a novel frequency‑domain augmentation for time‑series data.

II. Mask Reconstruction Paradigm

Mask reconstruction forces the model to recover corrupted parts of the data, learning fine‑grained semantic information. Inspired by the successes of BERT and MAE in NLP and computer vision, this self‑supervised paradigm has been applied to time series, typically using a Transformer encoder as backbone.

PatchTST[3] offers supervised and self‑supervised versions. In the representation‑learning stage, the series is divided into non‑overlapping uniform segments; a binary mask is applied at the segment level. A Transformer encoder with a linear head reconstructs the masked segments, using MSE as the reconstruction loss. When fine‑tuned on downstream forecasting tasks, PatchTST surpasses existing supervised methods, demonstrating the strong potential of mask‑reconstruction for time‑series representation.

III. Summary and Outlook

Future work must still address data heterogeneity and domain differences. Understanding intrinsic time‑series characteristics is key to building high‑quality, feature‑complete large‑scale datasets—a focus of Data‑centric AI—and to raising the upper bound of model capability. Further analysis of the strengths and limitations of each paradigm will help combine their advantages. The community hopes to eventually see a universal “generalist” model for time‑series data.