AQuA: Dual Isolated AI Systems Enable Leak-Proof Recursive Self-Improvement in Quant Research
The AQuA framework deploys two independent LLM-driven research systems—one for symbolic factor discovery, another for trainable model development—each confined in sealed sandboxes that freeze data splits, features, labels, and evaluators, achieving 0.190 combined IC in crypto factors and 0.0843 per-stock IC with 2.50 Sharpe in US equities through recursive self-improvement.
Abstract
This paper studies recursive self-improvement in quantitative investment research: whether an autonomous system can use evidence from prior experiments to improve hypotheses and candidates in later iterations. The authors propose AQuA , comprising two independent, LLM-driven research systems: one for symbolic factor discovery and another for trainable model development .
The two systems share no agents, memory, candidate spaces, or research state; each closes its own research loop by retaining validated evidence and using it to guide subsequent proposals. Each system also employs its own sealed sandbox , fixing data splits, feature and label definitions, and evaluators, while allowing the model to act only through constrained factor expressions or configuration diffs.
The factor system is a manager-mediated multi-agent pipeline that discovers and combines factors in the crypto domain, yielding a combined Information Coefficient (IC) of ~0.190 . The model system is a configuration-driven loop running on a hybrid time-series architecture, achieving a per-stock IC of +0.0843 on US equities, which translates into a threshold long-short strategy with an out-of-sample Sharpe ratio of +2.50 under two-way costs, with positive returns every year from 2021 to 2025.
Background
Quantitative research searches large factor and model spaces where small methodological errors become "plausible but irreproducible" backtests. A feature that peeks at future data, a strategy selected on the test set, or a result that only holds in a favorable regime can all fail out-of-sample. Hence quant research relies on frozen data splits, hold-out evaluation , and skepticism toward anomalously strong results.
LLMs can now propose hypotheses, write experiments, and correct their search based on empirical feedback. However, existing quant agents typically focus on either factor discovery or model development alone; more critically, unconstrained agents contaminate the very evidence their later iterations depend on . A code-generating agent may inadvertently introduce look-ahead or preprocessing bugs, using information unavailable at prediction time; a reviewer agent may miss the bug because the code looks semantically reasonable. If the resulting leak yields a high score, the experiment is stored as a successful precedent and propagated to later iterations—recursive improvement thus amplifies both genuine discoveries and undetected errors.
Prompt-level instructions and model-based review cannot provide reliable integrity boundaries: repeated access to a fixed hold-out set leads to adaptive overfitting, and LLM agents have been observed to exploit mismatched objectives, tests, evaluators, or reward mechanisms. AQuA's approach is to make "leak-inducing actions" unavailable to the agent: each system freezes the data pipeline, splits, labels, and evaluators before autonomous iteration begins; the agent cannot write arbitrary experiment code but can only emit programs in a restricted domain-specific language (DSL) that has no operations to modify or bypass the sealed data paths or evaluators.
Problem Statement
The problem: in quantitative research—a domain prone to data leakage and backtest overfitting—how can an LLM-driven autonomous research system achieve recursive self-improvement at the research-process level while ensuring that what improves is not the "definition of success" but the research process itself ?
Neither system shares agents, memory, candidate spaces, or outputs. In each system, validated experiments update a local research state that guides subsequent proposals, while the experimental contract used to judge those proposals stays fixed. The key design is "asymmetric freedom" : the agent retains exploration freedom within its DSL, but the evaluator sits outside the adaptive surface—the data path and evaluator are inaccessible to the agent rather than relying on its judgment.
Formally, each iteration goes through five phases: propose hypothesis (candidate factors in Part I / model configs in Part II) → build or train the concrete artifact → evaluate on hold-out data → validate against look-ahead bias and regime dependence → surviving artifacts are selected or combined into that part's running output → persistent research state updates write this round's evidence into that part's storage for the next round's proposals to consult. The recursive iteration can be written as:
R_{t+1}(p) = U_p(R_t(p), H_t(p), C_t(p), E_t(p)), (H_{t+1}(p), C_{t+1}(p)) ~ P_p(· | R_{t+1}(p))where U_p turns experimental results into reusable research knowledge, and P_p uses that knowledge to guide the next proposal; Part I does not update Part II's state, and vice versa.
Method
3.1 Sealed Sandbox & Constrained Iteration
Because the LLM both proposes hypotheses and writes the experiments, the core failure mode is data leakage. The authors block it structurally: the sandbox is fixed before iteration starts. Data splits D, feature and label definitions F, L, and evaluator V are all sealed and human-written; the model never edits them. Every model action is a specification θ in a constrained space Θ, compiled by a harness and scored by the sealed evaluator:
s_k = V(C(θ_k); S), θ_k ∈ Θ, S = (D, F, L, V) sealedThe definition of Θ guarantees that no θ can modify S—leakage cannot enter through generation because generation cannot touch the sealed components. Sealing the data path closes "generation leakage," but a second channel remains: "selection leakage" —a search that can read the final reported metrics will, after enough iterations, learn to select for that metric. The authors close this channel by separating the "metric optimized during the loop" from the "metric reported": during search the harness returns only scores on a predefined validation slice; if a part designates a final test window, that window is scored exactly once after the configuration is frozen, never returned to the agent, and never used for candidate ranking. In Part II that window is the untouched 2021–2025 period, so the reported test IC is out-of-sample with respect to the entire search.
Part I's registry is a standard vocabulary of formulaic alpha operators: leaves are raw fields (open, high, low, close, volume, vwap, returns); internal nodes fall into three categories— cross-sectional operators (rank, z-score, industry neutralization), time-series operators (lag, diff, rolling correlation/covariance, rolling rank, rolling std, linear decay weighting), and element-wise arithmetic and conditionals . A factor is an expression tree combining these operators, e.g.:
f = rank(corr(r_1d, v_1d, 20)) − rank(std(r_1d, 20))Each time-series operator reads only its trailing window; each cross-sectional operator reads only the current timestamp, so causality is closed under composition : any expression the model assembles is causal by construction and cannot introduce primitives that look into future data. Part II's specification is a DSL config covering the entire experiment: data split chosen from the frozen set, sampler, architecture blocks, loss, and optimizer; a config diff defines exactly one comparable model variant.
Part I: Autonomous Factor Discovery
Part I is a multi-agent system that turns a research objective into validated factors. An AI Manager mediates every run: it reads the research strategy, cumulative memory, and historical run logs, writes a plan, and assigns concrete tasks to each downstream agent; agents never call each other directly—every handoff goes through the Manager, making runs auditable and reproducible. The pipeline passes through six specialist agents in sequence: Data Steward (loads and aligns market data, returns quality report and atomic features) → Visual Analyst (searches history for representative events and distills them into event portraits) → Idea Miner (proposes candidate factors from portraits) → Factor Evaluator (scores every candidate) → Backtest Engineer (simulates trading) → Research Librarian (records results).
Factors enter the pipeline as "proposals" rather than raw expressions : the Idea Miner must state the hypothesis, the underlying economic mechanism, the expected predictive direction, and the conditions that would falsify it. Evaluation follows the same contract for all proposals: compute IC on forward-return labels, monthly stability, hold-out split behavior, regime behavior, turnover, expression complexity, correlation with the existing factor pool, and controlled comparison against simple baselines built from price, volume, open interest, basis, and aggressive flow changes. The Backtest Engineer then turns passing signals into simple quantile portfolios, testing both the proposed direction and its opposite, keeping the cleaner trading interpretation—direction calibration matters because a formula can have predictive content even if its initial sign is wrong.
The system improves across runs via memory: after each run the Research Librarian writes a structured record (objective, event type, visual observations, proposed mechanism, evaluated factors, selected signals, backtest summary, updated beliefs). Beliefs attach confidence to a mechanism in a market context, e.g., "rebounds after open-interest collapse without aggressive flow confirmation are more likely to reverse." This forms a three-layer feedback loop : direction calibration within a single backtest, belief-state updates from failed proposals within a single run, and cross-run guidance where the Manager reads cumulative memory to steer the next search toward mechanisms with accumulated evidence, away from repeatedly failing mechanisms, and to refine promising mechanisms with new event definitions, holding periods, or conditioning variables. Subsequent runs do not start from a blank prompt; they inherit prior observations, avoid repeated dead ends, and refine promising mechanisms with new event definitions, holding periods, or conditioning variables.
A complete worked iteration example: the run asks "Do weak price rebounds after open-interest collapse predict reversals?" The Manager plans: find deleveraging events, check post-liquidation price and flow, test whether rebound quality distinguishes continuation from failure. The Visual Analyst returns event portraits separating "clean forced deleveraging patterns" (OI down, volume up, price only briefly rebounds) from "healthy resets" (aggressive flow and basis recover with price). The Idea Miner proposes the mechanism: rebounds unconfirmed by aggressive flow are more likely to fail. The evaluator clears the factor against price, OI, and flow baselines; its skill concentrates in the event window the mechanism targets. The strongest factor in the series achieves single-factor IC of ~0.026–0.037. A later run changes the objective to "volume expansion in quiet regimes" and reuses the same mechanism suite without touching the evaluator—the loop explores new mechanisms by re-planning, not by re-coding. Deployed expressions remain confidential throughout.
Part I's output is a combined factor signal built from surviving factors: the combined validation IC rises as cycles accumulate and reuse evidence, reaching ~0.190 (combined factor Spearman IC on the crypto 5-minute universe, not directly comparable to Part II's per-stock IC convention). Each discovered mechanism is an economically grounded signal with absolute IC typically around 0.03—the strength comes not from any single rule but from combining a mechanism-driven factor library into a stronger aggregate signal.
Part II: Autonomous Model Development
Part II develops trading models through a configuration-driven loop : each iteration starts from a single config diff against the sealed sandbox (changes to architecture, loss, sampler, or optimizer), the harness compiles the diff into a training run, the sealed evaluator scores it, and the result is written into a knowledge base that the next hypothesis reads. One diff yields exactly one variant, and the data path is fixed, so two variants differ only on the changed knob—directly comparable and leak-free search.
The predictor is a hybrid model : a deep convolutional front-end—multiple 1D convolution blocks with varied kernel sizes and dilation rates run over the input history, building rich local representations of each stock's recent price-volume dynamics at each time step; these representations feed a temporal modeling stage instantiated from a sequence-model family (LSTM, Mamba state-space, Transformer attention; attention used in experiments); a cross-sectional stage then mixes information across the stock panel at each time step, with a gated fusion branch and a pooling readout that outputs per-stock scores. The entire model is a point in the configuration space: convolutional front-end, sequence family, cross-sectional mixer, and fusion are all operators in a registry; a new model is a new config, not new code.
The task is US equity intraday prediction : forecast each stock's 30-minute forward return. Data is split temporally: 2010–2019 training, 2020 as an embargo gap (untouched by training or selection), 2021–2025 as the untouched test window. Model selection is driven solely by an internal validation slice carved from the end of the training window. Model inputs are short histories of pure price-volume features, standardized per stock. No single feature carries signal alone: representative price-volume features' single-feature ICs and ridge linear combinations all top out at ~0.03 magnitude (ridge reaches +0.025); predictable signal lives in their joint, nonlinear, and temporal structure .
The evaluation engine uses the same sealed evaluator for all runs, reporting three quantities on hold-out data: per-stock time-series IC, per-stock R² (mean of cross-sectional IC²), and threshold long-short Sharpe under two-way turnover cost (2 bps). Model comparison (same data, same evaluator): linear ridge +0.0251, LGB +0.0397, xLSTM +0.0434, LSTM +0.0535, GRU +0.0613, hybrid model strongest at +0.0843 —an absolute improvement of +0.0230 over the strongest baseline GRU, a 37.5% relative gain .
From signal to strategy: per-stock scores are rebalanced at a fixed cadence into a dollar-neutral threshold long-short portfolio (long above upper threshold, short below lower threshold), paying two-way turnover cost (2 bps). Two construction steps lift out-of-sample Sharpe: industry neutralization removes common industry exposure, raising Sharpe to +2.15 (train and hold-out values nearly equal, indicating no overfit in construction); a causal volatility-targeting overlay scales daily exposure to an expanding-median target using an online trailing volatility estimate, further lifting to +2.50. Fully causal walk-forward evaluation (every model and strategy parameter fixed using only data available before the next test segment) still holds ~+2.0 Sharpe.
Experiments
Experimental Setup
Part I : Formulaic factor discovery on the crypto 5-minute universe; evaluator reports combined factor IC (~0.190). Part II : US equity intraday 30-minute return prediction, 2010–2019 train, 2020 validation, 2021–2025 hold-out test; evaluation reports per-stock raw IC, per-stock R², and threshold long-short Sharpe at 2 bps two-way cost.
Main Results
Part II hold-out window (2021–2025) headline results: per-stock raw IC +0.0843 , per-stock R² 1.20%; threshold long-short portfolio Sharpe +2.15 (before volatility overlay), +2.50 (after), +2.00 (fully causal walk-forward). Calendar-year Sharpe@2bp: 2021 +1.7, 2022 +3.5, 2023 +1.9, 2024 +1.8, 2025 +2.7— every year positive , including the 2022 index drawdown, so the strategy is not propped up by a single regime. The volatility-targeted, dollar-neutral long-short portfolio's cumulative returns climb steadily through 2021–2025, persistently making new highs, ending significantly above the contemporaneous Nasdaq-100 buy-and-hold benchmark and without its 2022 drawdown.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
