AlphaCFG: Grammar‑Guided, Interpretable Alpha‑Factor Discovery Framework

Background

Alpha factors map historical market features (price, volume) to future return predictions and are essential for asset management and quantitative trading. Discovering such factors requires searching a huge combinatorial space for symbolic functions that are both predictive and interpretable.

Related Work

Existing approaches fall into three categories: heuristic/expert‑driven methods that lack scalability; data‑driven learning methods that capture nonlinear patterns but suffer from limited interpretability and over‑fitting; and formulaic methods that use predefined operators to produce human‑readable expressions. Formulaic methods still suffer from missing language representations (low‑efficiency search) and semantic redundancy (wasted learning effort).

Problem Definition

The task is framed as symbolic regression: find explicit mathematical expressions that fit market data while remaining interpretable. Two main shortcomings of prior work are identified:

Absence of a formal language representation leads to inefficient search in an unbounded space.

Semantic redundancy causes systematic waste during learning and search.

Method

3.1 Grammar‑Constrained Alpha Factors

AlphaCFG introduces two formal languages, α‑Syn and α‑Sem , which combine context‑free grammars (CFG) with finance‑specific knowledge.

3.1.1 Syntactically Valid Alpha Language (α‑Syn)

The grammar enforces prefix notation, eliminating operator‑precedence ambiguities. Production rules generate expressions where Expr denotes recursively expandable non‑terminals, Op denotes prefix operators, and TermSym denotes terminal symbols (features or constants). This guarantees each operator receives the correct number of operands.

3.1.2 Semantically Interpretable Alpha Language (α‑Sem)

Building on α‑Syn, domain‑specific semantic constraints are embedded: rolling‑window limits, non‑triviality, numerical validity, and time‑series consistency. The generation rules incorporate these constraints.

3.1.3 Length‑Bounded Grammar (α‑Sem‑k)

A length counter k with upper bound K is assigned to each expression. Each production rule incurs an incremental cost Δk; a rule may be applied only if k+Δk ≤ K, preventing unbounded recursion.

3.2 Alpha Space Structure

Each grammar defines a language: L_{syn}, L_{sem}, and L_{≤K}. These languages are nested; L_{≤K} is finite yet expressive, enabling feasible search. Alpha discovery becomes a search for high‑quality leaf nodes within the tree‑structured space L_{sem}^{≤K} induced by α‑Sem‑k.

3.3 Reinforcement‑Guided Alpha Language Tree Search

The problem is cast as a Tree‑Structured Language Markov Decision Process (TSL‑MDP) and solved with a syntax‑aware Monte‑Carlo Tree Search (MCTS) guided by reinforcement learning.

3.3.1 Decision Making on Large Trees

State S is a (partial or complete) set of alpha expressions; action set A consists of grammar production rules. Transition P(s'|s,a) deterministically applies rule a to the leftmost non‑terminal, extending the expression. Reward R(s,a) is zero except when s' is a complete expression, in which case it equals the Information Coefficient (IC) evaluated on market data.

3.3.2 RL‑Guided MCTS

Two neural networks—a policy network and a value network—are driven by a Tree‑LSTM encoder. In each iteration, I rounds of syntax‑aware MCTS are performed, using the current policy and value networks to guide search. The MCTS components are:

Selection: Adaptive PUCT‑style selection with branch factor b and normalization constant b_{ref}.

Expansion & Evaluation: At frontier nodes, all applicable α‑CFG production rules generate child states; the value network V(s) evaluates nodes, while the policy network yields a distribution P(s,a) over valid productions.

Backpropagation: Evaluation results V(s) are back‑propagated, updating Q(s,a) and visit counts N(s,a).

3.3.3 Syntax‑Aware Representation Learning

The Tree‑LSTM encoder provides syntax‑aware representations, avoiding costly roll‑out evaluations in classic MCTS. It has two heads: a policy head predicting production‑rule distributions and a value head estimating terminal rewards. Joint training uses a diversity‑aware value target, where similarity sim is a normalized structural similarity based on maximum common subtree matching.

Experiments

4.1 Experimental Setup

Datasets: CSI 300 constituents (China A‑share) and S&P 500 constituents (US). Training period 2010‑01‑01 to 2017‑12‑31, validation 2018‑01‑01 to 2019‑12‑31, test 2021‑01‑01 to 2024‑12‑31 (2020 excluded).

Baselines: Grammar‑constrained methods (α‑Syn, α‑Sem, α‑Sem‑k), Reverse Polish Notation (RPN), state‑of‑the‑art factor mining baselines (AlphaGen, AlphaQCM), symbolic regression baseline (GPlearn), and common ML models (XGBoost, LightGBM, LSTM, ALSTM, TCN, Transformer).

Evaluation metrics: Predictive relevance (IC, RankIC, ICIR, RankICIR) and back‑test performance (Maximum Drawdown, Sharpe ratio).

4.2 Experimental Results

Search space comparison: More constrained, grammar‑defined spaces converge faster and yield higher‑quality factors. RPN eventually reaches performance close to α‑Sem but converges markedly slower.

Comparison with existing factor mining methods: On both CSI 300 and S&P 500 test sets, AlphaCFG achieves the best scores on all relevance metrics directly tied to IC. Ablation studies confirm the indispensability of syntactic constraints, semantic constraints, and length control. In back‑testing, AlphaCFG consistently attains high Sharpe ratios and low maximum drawdowns, delivering the highest overall profitability among all compared methods.

Improving traditional alpha factors: Applying α‑Sem‑k + MCTS to the GTJA‑191 and Alpha101 factor libraries improves the absolute IC of many classic but recently underperforming factors on the test data, demonstrating the framework’s effectiveness in enhancing existing alpha signals.

Conclusion

AlphaCFG provides a unified, grammar‑driven framework for discovering syntactically valid and financially interpretable alpha factors. By integrating syntax‑aware MCTS with reinforcement‑learning‑guided policy and value networks, it achieves superior search efficiency and trading profitability compared with existing baselines, and can be extended to broader symbolic factor discovery tasks such as asset pricing and portfolio construction.