Predicting ML Solution Quality Without Execution: LLMs Achieve 61.5% Accuracy (ACL 2026 Highlight)
A joint study by Zhejiang University and Ant Group shows that large language models can forecast the effectiveness of machine‑learning pipelines without executing them, reaching 61.5% prediction accuracy, cutting search time by sixfold and improving final performance by 6%.
Autonomous machine‑learning agents traditionally follow a Generate‑Execute‑Feedback loop, where the most time‑consuming step is executing training; a single experiment can take nine hours, and an agent may generate ten candidate solutions but can only afford to run one.
To break this execution bottleneck, the authors borrow ideas from world‑model research and embed execution priors into the model, allowing immediate predictive inference to replace costly physical runs.
The paper defines a "Data‑centric Solution Preference" task and constructs a large corpus containing 18,438 paired comparisons. Experiments demonstrate that, when provided with validated data‑analysis reports, a large language model predicts the better solution with 61.5% accuracy and exhibits reliable confidence calibration.
This predictive framework is applied to ForeAgent, an agent that employs a "Predict‑then‑Verify" cycle. Compared with a baseline that relies on execution, ForeAgent converges six times faster and achieves a 6% improvement in final performance.
The research received the ACL 2026 SAC Highlight award, and both the code and dataset have been open‑sourced for the community.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.