Slashing Token Costs in Multi-Agent Workflows: Context Lifecycle & Batch Editing

Background: DevFlow Multi-Agent Workflow

The lightweight cloud team built DevFlow, a multi-agent pipeline where a requirement passes through Architect, Developer, Code Reviewer, Test Engineer, Knowledge Engineer, and Leader agents. Each stage adds context—requirements, design docs, source snippets, tool outputs, stage status—so the context window grows. Developer and Test Engineer invoke the model most frequently. In one run, Developer entered with ~120K tokens of context but only needed to make four small edits (modify handler, add import, modify another handler, update report). If split across four model calls, each call re-sends the entire 120K context, multiplying cost.

How Token Cost Amplifies

Total input tokens ≈ Σ (existing context at round i + new content at round i)

Two factors drive cost: (1) single-round context length, and (2) number of model rounds. They compound: a 5K exploration snippet that persists through 10+ subsequent rounds becomes part of every later input. Splitting one logical edit into four rounds doesn't just add three tool calls—it re-sends the full context three extra times. The team identified key culprits: long-lived exploration context, low-frequency templates loaded too early, unnecessary Main-agent handoffs between stages, and fine-grained edit tool calls inside high-frequency stages.

Optimizing Context Lifecycle

3.1 Principle

Information valuable in one stage need not be retained in all later stages.

3.2 Code Exploration → Short-Lived Agent

Instead of letting the main Developer agent search code (dumping raw search results and source snippets into the long-lived session), a temporary Code Explorer agent performs exploration with strict limits: read-only, not part of the permanent team; uses code index to narrow scope; caps search/read calls and per-file snippet length; returns only a structured summary (~500 words); raw exploration artifacts die with the agent. The main flow receives only the conclusion (modules, interfaces, call chains, impact, risks).

3.3 Low-Frequency Templates → Progressive Loading

Developer needs change-report.md and API doc templates only at the final delivery step, yet they were embedded in the main prompt and carried through code reading, editing, verification, and fixing. The fix: keep only paths, triggers, and process skeleton in the main prompt; load developer-deliverables.md (report + API templates) after implementation and verification complete. Skills similarly moved detailed templates, edge-case rules, and low-frequency scenarios into assets/ and references/, loaded on demand. The key is not merely splitting files but deciding when each piece enters context.

The point is not “where content lives” but “when content enters context.”

3.4 Reducing Main-Agent Handoffs

Normal flow previously bounced through Main at every stage transition (Architect → Main → Developer → Main → Code Reviewer …). Since the next stage is usually deterministic, the team changed to direct handoff: Architect → Developer → Code Reviewer → Test Engineer → Knowledge Engineer → Leader. Main now handles only failures, retries, human gates, protocol errors, and workflow completion. This removes unnecessary Main model calls and prevents Main's context from growing with each hop.

Optimizing Model Round-Trips Inside Stages

4.1 Response-Level Batching

Many independent operations (reads, writes, verifications) can be determined simultaneously. The team added a prompt rule: when multiple independent operations are ready, submit them in the same model response . Examples: Architect generates tech-design.md and execution-plan.md via two write_to_file calls in one response; Test Engineer runs unit tests, e2e scripts, registration, and report updates together; multiple independent reads/searches grouped. This is response-level batching —not dependent on host parallelism, but on expressing multiple ready actions in one model turn.

4.2 Tool-Level Batching for In-File Edits

The native replace_in_file(file_path, old_str, new_str) handles one replacement per call. When Developer knows four edits upfront, it would still make four sequential calls, each re-sending the 120K context. The team considered adopting Codex's apply_patch but found it unreliable when migrated to CodeBuddy via MCP: models struggled with JSON escaping plus patch DSL syntax (missing + in Add File, lost leading spaces in Update File, malformed @@ headers, line-prefix errors breaking whole transaction). They also realized new files work fine with write_to_file, moves/deletes are rare, and the high-cost scenario is multiple precise edits in existing files—where CodeBuddy already understands old_str/new_str semantics.

4.3 Prompt Alone Isn't Deterministic

Global rules, tool descriptions, and prompts encouraged batching (research first, merge known edits, avoid single-location edits, submit independent edits together). This raised probability but didn't change the tool's capability boundary. The team settled on a three-layer approach: prompt guidance, response-level batching for independent ops, and a new tool for multi-edit transactions.

replace_batch: Tool-Level Batch Editing

6.1 Protocol

A simple JSON array of replacements, each with file_path, old_str, new_str (multi-line). No new DSL; model just packs the same old_str/new_str pairs it would have sent sequentially into one array.

{
  "replacements": [
    {
      "file_path": "/absolute/path/handler.go",
      "old_str": "old code block 1",
      "new_str": "new code block 1"
    },
    {
      "file_path": "/absolute/path/handler.go",
      "old_str": "old code block 2",
      "new_str": "new code block 2"
    },
    {
      "file_path": "/absolute/path/another_handler.go",
      "old_str": "old code block 3",
      "new_str": "new code block 3"
    }
  ]
}

6.2 Snapshot-Based Execution

All replacements reference the original file snapshot at call start. The tool: (1) reads original files; (2) locates every old_str in the original; (3) verifies each matches exactly once; (4) checks for overlapping ranges; (5) builds new content from back to front to avoid offset shifts. Adjacent/overlapping edits must be merged into a single larger replacement by the model.

6.3 Transactional Guarantees

Unlike sequential replace_in_file calls where a mid-sequence failure leaves partial writes, replace_batch validates the entire batch before any write: absolute paths, existing UTF-8 text files, exact single matches, no overlaps, size/count limits, write all to temp files with fsync, re-check for concurrent changes, then atomically replace; on any failure, roll back from original snapshots.

6.4 Batch Size Heuristics

Bigger batches aren't always better. Batch boundary should be: can these edits be determined and validated against the same code state? If later edits need compile/test results, actual diffs, newly read code, or code-review feedback, they belong in the next batch. Forcing dependent edits together lowers success rate and widens blast radius.

6.5 Converge Before First Write

Tool support doesn't guarantee model behavior. The team redefined Developer's pre-write research completion criteria: core implementation, direct callers, imports/constants/helpers, interface contracts, config/init logic, and any associated edits deducible from current state. Only when the implementation path and impact scope converge does the first batch write occur. Batches are logical, not per-file: if two handlers' edits are simultaneously determinable, they go in one batch. Rhythm becomes: research & converge → batch write → check diff → verify → next batch based on new evidence.

6.6 Hook Enforcement with Fail-Open

A PreToolUse Hook intercepts replace_in_file / Edit calls and redirects to replace_batch. Before blocking, it health-checks the MCP server (initialize → tools/list → confirm replace_batch exposed) with a 2-second timeout. On success: reject single-edit tools, enforce batch. On failure/timeout: fail-open, allow native tools. Rationale: batch editing is a cost optimization, not a safety boundary; task completion trumps token savings when the batch tool is unavailable.

Results

Tested on a real medium-sized requirement (6 HTTP interface changes) across full DevFlow pipeline with Claude Opus 5 and GLM 5.2 (GLM averaged over two runs each before/after).

Claude Opus 5

Full flow tokens down ~25.69%.

Developer: 6.05M → 4.44M (-26.58%).

Test Engineer: 13.37M → 8.69M (-35.05%, ~4.69M tokens saved).

GLM 5.2

Full flow tokens down ~41.95%.

Developer: 6.23M → 2.31M (-62.94%).

Test Engineer: 5.11M → 2.53M (-50.47%).

Both models show largest drops in the high-frequency read/write/test/fix stages where response-level batching and replace_batch apply. Absolute numbers differ widely between models, so the team treats percentages as directional evidence, not fixed guarantees.

Retrospective: Harness Design Principles

The optimization didn't cut necessary analysis—it adjusted two levers:

Information lifecycle: ephemeral exploration agents, stage-gated template loading, avoid long-term residency of stage-specific data.

Action expression granularity: response-level batching for independent ops; tool-level batching ( replace_batch) for multi-edit transactions with snapshot isolation, pre-validation, transactional write, and rollback.

The runtime engine handles snapshot, exact-match, pre-flight, transactional write, rollback. Hooks handle default routing, health-check, fail-open. The core reusable principle:

Models excel at deciding what to do; once a set of actions is decided, tools and runtime should execute them reliably.

Prompt suits goals, conditions, principles. For behaviors the model already supports (multi-read/write in one response), prompt guidance suffices. When behavior impacts cost, quality, or stability, push it down to tool protocol, hooks, and execution engine.

Reference implementation: https://github.com/Tencent/LoopForge (Tencent open-source).