Control Inversion in Agent Tool Calling: Orchestration Loops Move Into Model-Generated Code

Control Flow Changes Owner

The classic agent loop works like this: the model outputs a structured tool call, the host parses, validates, executes it, stitches the result back into the message list, and triggers the next inference round. Control flow lives in the host process; the model is just a function call inside the loop body.

Programmatic tool calling flips this. Tools are exposed as typed functions; the model writes code that calls them, and that code runs in a sandbox. In a single model turn it can write loops, branches, use asyncio.gather to fire multiple calls in parallel, sort and filter results, and hand back an aggregated conclusion.

The immediate consequence: retry policies, fan-out width, error handling, and result merging move from Go or Python source into model-generated code. You lose static guarantees about execution shape, but gain the ability for the model to express far more complex orchestration in one turn. Whether that trade-off pays off depends on what your tools look like.

Anthropic's interface makes this explicit. Tool definitions gain an allowed_callers field declaring who may invoke the tool. ["direct"] means direct calls only (the default if omitted); ["code_execution_20260120"] means calls only from inside the code execution container; both can be enabled together. Official guidance recommends choosing one rather than opening both, because it gives the model clearer guidance.

The bridge mechanism is pause-resume. When generated code calls a tool function, the container pauses; the API returns a tool_use block to your application, you execute it and send the result back, and the container resumes from the pause point. Intermediate results never enter the model context — only what the code finally prints goes back. Each tool_use block carries a caller field indicating whether the call was direct or triggered by code execution. This field proves far more useful for debugging and auditing than it first appears.

Another Way Down the Same Path

Anthropic implements this at the API layer; LangChain's Deep Agents takes a different route, building it as middleware. Developers pass a whitelist (e.g., ptc=["task"]) to CodeInterpreterMiddleware; allowed tools appear as async functions under the sandbox's tools namespace, invoked with await. The middleware mounts an eval tool that maintains a QuickJS context, runs the model's TypeScript, and feeds the final expression result back into context.

The difference isn't just engineering taste. API-layer binding ties you to a model provider, but the pause-resume protocol is guaranteed by the service. The middleware approach is provider-agnostic — open-source models work too — but you must handle sandbox lifecycle, memory caps, per- eval timeouts, max programmatic calls per turn, and max result size yourself.

LangChain also splits agent context into three planes, a decomposition more valuable than the feature itself. Message history is what the model reasons over right now — expensive and attention-constrained. The filesystem holds durable artifacts, at the cost of serialization and reconstruction. Interpreter state holds live values that don't yet need to become model input: arrays, hash maps, counters, queues, helper functions — surviving across calls like REPL variables.

This third class of state previously had no natural home. It was either forced into message history (expensive) or serialized to disk (slow). Programmatic calling gives it a natural place. The early observation of "up to 35% token savings on partial tasks" mostly comes not from reduced reasoning but from eliminating data shuffling.

Where the Value Lies

The model's only exit to affect the external world becomes the tool declarations themselves.

Generated code runs in a sandbox with no network, no filesystem, no shell. Writing a file, sending a request, or mutating a record must go through an explicitly declared tool function. GPT-5.6's programmatic calling executes generated JavaScript in a V8 sandbox with no network access; tools are the sole egress. If you don't expose delete_record, the generated code simply cannot delete records.

This is a much stronger guarantee than prompting the model not to delete records. But a boundary must be stated clearly: Anthropic's documentation explicitly says allowed_callers is not a hard security boundary at the API layer. It strongly guides the model toward the code execution path, yet clients must still handle possible direct tool_use calls. Treating allowed_callers as authorization control mistakes guidance for enforcement.

Cost control primitives shift accordingly. Old limits on turn count or token count were fairly intuitive. Now a model can launch forty tool calls in one turn, so turn-based rate limiting is useless, and token-based cost curves lag far behind. The real control knobs are: max programmatic calls per turn, per-execution memory ceiling and wall-clock timeout, and max single-result size. These must be explicitly configured at the gateway or runtime layer; defaults are unreliable.

Container lifecycle enters your design. Anthropic's container ID must be passed between steps, and tool results must be returned while the container lives — a window of roughly four and a half minutes. Long-running tasks therefore need explicit "rebuild from where, recover to which step" logic after container expiry; you cannot assume the execution environment persists.

A subtler point: LangChain's middleware supports snapshotting interpreter state between turns, but snapshots store only serializable data, not live handles. If orchestration code stashes database connections, file descriptors, or timers in interpreter state, those references break after snapshot restore. What state may contain becomes a checklist you must review when writing recovery logic.

Debugging Gains an Unrecorded Artifact

The generated orchestration code itself becomes a new runtime artifact — and by default it isn't recorded.

When a turn that made thirty tool calls fails, you have only the final error and the last tool_use. To diagnose whether the loop condition was wrong, a return structure was misread, or the eighth fan-out hit a rate limit, you need to see that code. The caller field tells you whether a tool was invoked directly or programmatically, but not why the code was written that way.

The practical fix is to feed eval inputs and outputs into tracing: full generated code text, every tool bridge's inputs and outputs, container IDs and lifecycle events, per-execution latency and peak memory. These data must not enter model context — that's what the design avoids — but they must enter your observability pipeline. Aligned with existing span trees, a programmatic calling turn appears as a wide fan-out tree under a single span, not a dozens-deep chain.

What the Data Says and Where It Doesn't Work

An August arXiv evaluation compared programmatic calling against native JSON calling on BFCL v4 across 14 models. A few headline figures are worth noting: 11 of 14 models matched or beat the JSON baseline under programmatic calling. The GPT-5.6 series scored 10.6% higher than the JSON baseline. Under parallel fan-out conditions, 13 of 14 models matched or improved. Under context pollution conditions, the JSON baseline dropped 2.3% on average while programmatic calling remained stable.

Interpret these numbers with restraint. They come from an abstract — no variance, no significance testing, no absolute values for the programmatic side, and "remained stable" is qualitative. They support "this path is worth trying" but not "switch everything over." The context pollution observation does align with the mechanism: intermediate results never enter the window, so the model doesn't lose accuracy from window clutter.

Limitations are clearly listed. Three tool categories cannot be used programmatically: web search, web scraping, and tools provided via MCP connectors. The computer_use and browser_use toolsets accept only "direct". Tools with strict: true or disable_parallel_tool_use: true are unsupported. Custom tools whose input_schema contains a recursive $ref return a 400 error Circular $ref detected — they must be made direct-only or have the cycle broken. tool_choice cannot force a tool down the programmatic path.

Viewed together, the boundary is fairly sharp. Programmatic calling suits tools with well-structured I/O, good idempotency, and no dependence on server-side session state. Query, retrieval, and read tools fit naturally. Write operations with side effects that require ordering need extra thought before inclusion: two writes to the same resource inside one asyncio.gather will race — concurrency order is decided by the scheduler, not by you.

Four Questions I'd Ask First

Whether to enable this capability in my own agent, I'd judge in this order:

Tool shape fit. If most tools are "one call per turn, result shown directly to user," programmatic calling adds complexity for little gain. If patterns like "fan out N ways, filter, then aggregate" exist, the payoff is dramatic.

Model capability. This path hard-depends on code generation quality. If the model can't write correct asyncio.gather logic, the benefit turns negative immediately. The three models that didn't win likely fell here. When selecting models, treat "can write correct orchestration code" as an independent capability test — don't infer from general coding leaderboards.

Control knobs configured. Max programmatic calls, execution timeout, memory ceiling, result size ceiling, container lifecycle handling — all five must be explicitly set before deploying to any environment with real side effects. Canary on read-only tools first; only after generated orchestration code accumulates enough samples in traces should you consider opening write operations.

Observability wired. Generated code, bridge calls, container events — if these don't reach tracing, a post-mortem degrades into guesswork. This cost is routinely underestimated, yet it decides whether the system survives in production.