Machine Learning Algorithms & Natural Language Processing
Nov 19, 2026 · Artificial Intelligence
Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering – GPT-6 Astra Tops at 53% Pass Rate
Researchers from CAS, Tsinghua, PKU, and Southeast University introduce Doc2Harness, the first deterministic benchmark evaluating LLMs' ability to build coding agent harnesses from a contract skeleton and dev doc; 19 models tested, GPT-6 Astra achieves highest pass rate of 53.05% on 328 test cases, revealing significant room for improvement in long-horizon engineering.
Doc2HarnessLLM benchmarkcoding agents
0 likes · 25 min read
