How Code Serves as the Harness for AI Agents: Insights from UIUC, Meta, and Stanford

Code Reshapes the Execution Base

Large language models (LLMs) are stateless; to turn them into persistent agents a surrounding software layer—called a Harness—is required. The Harness comprises tools, a sandbox, memory, verifiers, and an execution loop, and code is the optimal medium for building it.

Code possesses three decisive traits: it is executable, allowing model outputs to become objectively verifiable actions; it is inspectable, leaving structured execution traces for debugging; and it is stateful, enabling progress to be persisted across steps. Thus code forms the "floor" and "steering wheel" of an agent, while the LLM functions as a memory‑less brain.

Beyond traditional source code, the authors define code broadly as any executable or machine‑checkable artifact, including:

Conventional code and rules: programs, scripts, configuration files.

Rigorous contracts: formal specifications, proof scripts, API schemas, tool definitions.

Environment and evaluation standards: code repositories, simulators, test cases.

Execution by‑products: execution traces and logs generated or consumed by the system.

Physical state, human intent, and model inference can be perceived, serialized, verified, or executed through code, but they should not be conflated with "code interfaces".

Code is the only structured entity that LLMs can read, run, inspect, and leave state records for when interacting with the external world.

Mechanisms that Keep the System Running

Once interfaces are established, agents face long‑duration tasks. The Harness’s operation determines whether agents can self‑correct and continue when errors occur.

Planning controls task direction. With large codebases, agents must decompose goals to avoid dead‑ends. Modern planning has evolved from textual outlines to system‑level entity control, including dependency graphs, solution‑space searches, and workflow orchestration.

Memory addresses the limited context window of LLMs. Real software generates massive logs and test reports; feeding all of them to a prompt would crash the model. The system therefore decides which clues stay in short‑term memory, which code snippets are retrieved via semantic search, and which historical lessons are stored as long‑term experience, while compressing irrelevant error output.

Tool Invocation dramatically expands an agent’s capability frontier. Agents can call search engines, modify files via the command line, or run test scripts. A robust Harness strictly manages tool lifecycles, standardizes parameters, and inserts human‑in‑the‑loop safeguards before dangerous operations.

All these components form a verification‑centric control loop: planning becomes an explicit contract, modifications run in isolated sandboxes, and compilers, static analysers, and tests act as impartial judges. The Harness decides whether to proceed, roll back, retry, or request human intervention.

Foundations of Multi‑Agent Collaboration

Single agents hit limits when projects span thousands of lines of code. Multi‑agent systems (MAS) distribute responsibilities: some decompose architecture, others implement functions, some hunt bugs, and others oversee global quality. Communication shifts from free‑form chat to a shared code repository and execution logs.

Organizational structures vary: waterfall‑style handoffs, agile cycles of code‑write‑test‑fix, or even automated rewiring where failing nodes are pruned and communication networks are restructured. The biggest risk is state desynchronization among agents.

Researchers mitigate this by introducing a global blackboard pattern and structured context scheduling, ensuring all participants operate on a fully aligned, execution‑result‑driven code world, with strict compilation and safety metrics governing progress.

Deployment Scenarios and Future Challenges

Code‑centric Harnesses already demonstrate strong impact in real applications. Programming assistants have evolved from simple completion to repository‑level collaboration, reading code, running tests, and fixing bugs within isolated sandboxes.

GUI/OS agents such as Claude Computer Use translate visual web and desktop interactions into precise execution scripts, automating clicks and inputs.

In scientific research, hypotheses, literature, and robotic instructions are encoded into execution graphs, enabling autonomous chemical simulations and lab‑automation.

Even personal recommendation systems and embodied agents rely on continuously updated preference memories and skill libraries.

However, powerful execution introduces new challenges: evaluating safety, compliance, and efficiency of autonomous tool use; avoiding false confidence where all tests pass but logic remains flawed; and establishing immutable permission boundaries that force critical irreversible actions to pause for human‑in‑the‑loop review.

Reference materials:

https://arxiv.org/pdf/2605.18747

https://github.com/YennNing/Awesome-Code-as-Agent-Harness-Papers