Paper Review: Kronos – A Temporal Foundation Model for Financial Market Language

Background

Foundation models have driven advances in natural language and computer vision, inspiring time‑series foundation models (TSFMs). Financial markets rely on K‑line data (OHLCVA) as a core language, but generic TSFMs perform poorly on financial tasks because financial data exhibit low signal‑to‑noise ratios, strong non‑stationarity, high‑order dependencies, and constitute less than 1 % of typical pre‑training corpora. Key financial tasks such as volatility prediction and synthetic data generation are also under‑represented.

Problem Definition

Insufficient adaptability of generic TSFMs to financial data.

Missing coverage of essential financial tasks (volatility prediction, synthetic data generation).

Difficulty of modeling continuous high‑dimensional K‑line streams, requiring an efficient discretization method that preserves critical dynamics.

Method

K‑line Tokenization

The tokenizer is built on a Transformer auto‑encoder consisting of an encoder E_{enc}, a binary spherical quantizer Q, and a decoder E_{dec}. Each continuous K‑line vector x_{i,t}\in\mathbb{R}^6 is mapped by binary spherical quantization (BSQ) to a k -bit binary code b_t. The code is split into a coarse sub‑token b_t^{c} and a fine sub‑token b_t^{f}. Three losses guide learning: L_{coarse} trains b_t^{c} to capture low‑resolution structure. L_{fine} trains b_t^{f} to encode residual information. L_{quant} regularizes the quantization process.

Autoregressive Pre‑training

A causal‑attention decoder Transformer E_{ar} models temporal and cross‑asset relationships via autoregressive prediction. Hierarchical prediction first generates the coarse sub‑token, then conditions the fine sub‑token on the coarse output. Coarse and fine sub‑tokens are embedded separately, concatenated, and linearly projected to form a fused input v_i. The Transformer processes the sequence, and separate linear heads predict the distributions of coarse and fine sub‑tokens.

Model Configurations and Pre‑training Data

Kronos provides three model variants to suit different compute budgets: small (24.7 M parameters), base (102.3 M), and large (499.2 M). Pre‑training uses 12 billion K‑line records collected from 45 global exchanges, covering seven time granularities. A data‑cleaning pipeline removes missing values and filters low‑quality segments to ensure high data quality.

Experiments

Tasks and Datasets

Five tasks are evaluated: price‑sequence prediction, return prediction, volatility prediction, synthetic K‑line generation, and investment simulation. Datasets span stocks from nine major exchanges, cryptocurrency data from Binance, and over 1,000 FX pairs, with timeframes ranging from 1‑minute to weekly. Training data end in June 2024; testing starts in July 2024.

Main Results

Price‑sequence prediction: RankIC improves by 93 % over the leading TSFM and by 87 % over the best non‑pre‑trained baseline; the large model achieves an average RankIC of 0.0267.

Volatility prediction: MAE is reduced by 9 % (large model MAE 0.033) and R² increases to 0.262.

Synthetic K‑line generation: Discriminator score is 22 % higher (large model 0.208); t‑SNE and KDE visualizations show strong distribution overlap with real data.

Investment simulation: Annualized excess return and information ratio both exceed all baselines, confirming the practical profitability of the learned signals.

Ablation Studies

Discrete modeling (Kronos) significantly outperforms continuous regression (Direct‑AR) and probabilistic modeling (Prob‑AR).

Hierarchical sub‑token generation outperforms parallel generation (Kronos‑Parallel).

Expanding the vocabulary size 2^{k} improves reconstruction quality and prediction accuracy; for example, k=20 reduces MAE to 0.024.

Monte‑Carlo rollout with ten sampled predictions raises IC by 15 %.