How AlphaPROBE Leverages DAGs for Efficient Alpha‑Factor Mining

Extracting predictive signals from noisy, high‑dimensional market data is a core challenge in quantitative finance, where alpha‑factor mining aims to transform raw data into expressions that forecast future asset returns. Existing automated methods fall into two paradigms: Decoupled Factor Generation (DFG) and Iterative Factor Evolution (IFE). Both lack a global structural view, leading to redundant searches and limited diversity.

Problem Definition : The task is to select an optimal set of factors \(A = \{f_1, f_2, \dots, f_N\}\) that maximizes a portfolio‑level utility \(J(A)\) (e.g., Sharpe ratio, Information Coefficient). Current methods either treat factor generation as independent events (DFG) or focus only on local parent‑child relationships (IFE), ignoring the overall factor evolution network.

Method : AlphaPROBE reconstructs factor mining as a strategic navigation and creation process on a DAG \(G = (F, E)\), where \(F\) are discovered factors (nodes) and \(E\) are directed edges representing factor lineage. The problem splits into two inter‑linked sub‑tasks:

Strategic Retrieval : Find the most promising parent factor \(F_p^*\) whose potential offspring \(F_{new}\) maximizes expected quality. This is formalized as a Bayesian ranking problem where the posterior \(P(F_{new}\mid D) \propto P(F_{new})\,P(D\mid F_{new})\) balances individual factor value and its contribution to the factor pool.

Target Generation : Using the selected parent’s full ancestry trajectory \(T(F_p^*)\) and a generation function \(G\), produce a set of novel, high‑quality child factors \(\{F_{c,1}, \dots, F_{c,k}\}\).

The framework consists of a closed‑loop system with two core components:

Bayesian Factor Retriever : Ranks factors by combining a prior term \(P(F_{new})\) (quality, depth penalty, retrieval penalty) with a likelihood term \(P(D\mid F_{new})\) that evaluates diversity in value, semantics, and syntax. The prior uses risk‑adjusted performance \(Qual(F)\), depth \(depth(F)\), and retrieval count \(k(F)\) with hyper‑parameters \(\gamma\) and \(\omega\).

DAG‑aware Factor Generator : After a parent is chosen, three agents operate sequentially:

Analyst agent designs diverse, context‑aware modification strategies based on the parent’s full evolution path.

Executor agent translates each strategy into concrete candidate factor expressions.

Verifier agent checks syntax and predefined constraints, discarding invalid candidates before adding the valid ones to the DAG, thereby expanding the graph for the next retrieval round.

Dynamic Factor Integrator : Inspired by AlphaForge, it periodically selects recent effective factors and aggregates them into a “mega‑factor” \(F_y\) for portfolio construction.

Experimental Setup : Experiments use three Chinese stock pools—CSI 300, CSI 500, and CSI 1000—split into training (2010‑2020), validation (2021‑2022‑06), and testing (2022‑07‑2025‑06). Evaluation metrics include prediction‑ability measures (IC, ICIR, RIC, RICIR) and portfolio‑construction metrics (annualized return, max drawdown, Sharpe ratio). Baselines cover expert‑collected factor pools (Alpha158), DFG methods (AlphaGen, AlphaForge, AlphaQCM, AlphaSAGE), and IFE methods (GP, AlphaAgent, R&D‑Agent(Q)). All models share the same backbone large language model (Deepseek V3.1) and embedding model (Qwen‑3‑Embedding‑4B). Hyper‑parameters such as factor pool capacity (50), factor length limit (40), depth penalty \(\gamma = 0.05\), and retrieval penalty \(\omega = 0.10\) are kept identical across baselines.

Main Results : AlphaPROBE consistently outperforms all baselines on the three datasets, achieving the highest IC, RIC, and annualized return, while also delivering superior ICIR, RICIR, Sharpe ratio, and lower max drawdown. Back‑testing on CSI 300 shows AlphaPROBE’s cumulative returns dominate competitors, especially during market stress periods (late‑2023 bear market, April‑2025 tariff shock), where it exhibits better risk control and faster recovery.

Ablation Studies demonstrate that:

Replacing the retriever with random selection drastically degrades performance, confirming the importance of global factor relationships.

Heuristic retrievers that ignore structural context yield sub‑optimal results, highlighting the value of the DAG‑encoded strategic information.

Monte‑Carlo Tree Search (local view) underperforms AlphaPROBE, validating the superiority of a global topology perspective.

Removing the prior term or topological penalties reduces performance, showing that both factor quality and graph position are crucial for effective retrieval.

Omitting likelihood components or non‑leaf factor considerations harms results, indicating that pool‑level context and ancestor lineage provide essential non‑redundant information.

Substituting the DAG‑aware generator with a simple chain‑of‑thought generator lowers performance, proving that DAG structure encodes richer interaction cues for evolution search.

Parameter Sensitivity : Varying \(\gamma\) and \(\omega\) within a broad range (0.05‑0.15) leaves performance stable; extreme values break the balance between exploration and exploitation, causing degradation.

Visualization : A case study shows AlphaPROBE starting from a simple handcrafted seed factor (e.g., Div(Sub(Less($open, $close), $low), $open)), identifying high‑potential parents, and generating more complex, high‑quality offspring such as TsCorr(Sub($close, $open), Sub($low, TsMin($low, 5)), 5), illustrating a clear optimization trajectory.

Efficiency Analysis : Compared with baseline LLM‑based models, AlphaPROBE’s retriever leverages global topological information to select the most promising factors early, achieving comparable or better results with fewer training iterations, confirming its training efficiency.

Overall, AlphaPROBE demonstrates that incorporating a global DAG‑based topology into factor discovery yields more accurate predictions, more stable returns, and higher training efficiency compared with existing DFG and IFE approaches.