How Pi’s Rewritten Harness Verifies Long‑Running Agent Actions After 50 Hours

Pi recently rewrote its Harness runtime to address a core difficulty of long‑running agents: after a crash or restart, the system must still be able to explain which actions have already been performed and which have not.

Typical failure scenario

An agent that upgrades a database might edit code, run a large test suite, and emit tens of thousands of characters of log output. If the container is reclaimed mid‑test, the chat history only shows “running tests” and the full log is lost, leaving no reliable evidence of whether the code change, the test run, or the deployment command succeeded.

Context vs. evidence

The author stresses that the model’s Context should be a working set for inference, not the sole source of truth for execution facts. Full logs must be retained externally and fetched on demand.

DSH’s permanently‑lossy pruner

External contributor adamteale’s PR #8172 proposes a permanently‑lossy pruner that keeps only the beginning and end of an overly long tool result, discarding the middle. While this saves Context space, a 30 KB test log with a critical error at character 12 000 would become invisible to the model.

The PR’s actual solution is prune + spill: the complete tool result is written to disk, while Context stores a short preview plus a file path and offset range. When the model needs details, it can grep, sed or read the stored file.

Self‑tested benchmark

Using GLM‑5.3 and DeepSeek V4 Flash, the author ran 19 sessions and observed:

Uncached pre‑fill reduced by 72 %–88 %.

Per‑request Context usage dropped by 26 %–35 %.

Persisted transcript size shrank by 81 %–93 %.

These numbers come from the unmerged PR and are not official Pi benchmarks.

Claude schema mismatch

Issue #6278 reported that Claude sometimes generates fields ( new_text_x, type, in_file, closeenough) not defined in Pi’s edit schema, causing ~20 % of edit sessions to fail. Maintainer mitsuhiko added a compatibility fix by removing additionalProperties: false and allowing only oldText and newText fields.

The discussion suggests that a model trained on a different harness (e.g., Claude Code) may still emit familiar schema shapes when switched to Pi’s harness.

Transactional agent run

Pi’s Harness v2 treats each agent run as a recoverable transaction with three stages:

intent commit → external effect → settlement commit

When the model proposes a tool call, the harness first records the tool name, parameters, call ID, and replay policy (the intent). Only after this record is stable does it execute the external effect. Upon completion, the result, usage, and next state are written as a settlement.

If a crash occurs before settlement, the system can see the state effect_pending instead of a vague “tool call in progress”.

Replay policies

replay: safe

– read‑only queries (e.g., grep) can be safely re‑executed. replay: never – irreversible actions (file deletion, deployment, messaging) are not re‑run; instead a synthetic “interrupted” result is recorded for higher‑level handling.

This mirrors the “record intent, charge, then settle” pattern in payment systems, but only guarantees local state recovery; external services must still provide idempotency.

Separated persistence layers

Pi splits persisted data into three categories:

Entry – append‑only conversation tree (messages, tool results, compressed summaries).

Register – mutable runtime state (current lane cursor, operation phase).

Usage Ledger – token count, cost, and retry metrics.

The Session Tree is shared, while each Lane (analogous to a Git branch’s worktree) maintains its own execution pointer. Multiple lanes can coexist under one session, allowing parallel code edits and test runs without rewriting history.

Context compression only affects the next model request; the original entries and usage ledger remain untouched, ensuring evidence is never lost.

Plugin state management

Compared with DeepSeek Harness (DSH), Pi pushes complexity into short‑loop extensions, whereas DSH delegates composition to a Cordis runtime that handles temporal and spatial composability of plugins. In Pi, every plugin (including the model adapter and the agent loop) must declare its replay policy, and hooks that cause external side‑effects must be idempotent.

Open issues and checklist

Although many components are in place, the full AgentHarness API (e.g., prompt(), resume(), compact(), navigateTree()) still throws HarnessNotImplemented. The author suggests a checklist for building long‑running agents:

Can compressed tool output be retrieved in full later?

Does each side‑effect‑producing tool declare a safe replay policy?

How does the system handle a window where the effect is done but settlement is missing?

Where are the boundaries for conversation, runtime state, usage, and telemetry storage?

After a plugin is unloaded or the process restarts, who explains the remaining state?

These questions apply to any system that expects agents to run overnight, perform CI pipelines, or handle production tickets.

Conclusion

Pi’s Harness v2 is moving toward a persistable, recoverable agent workflow runtime that separates evidence from inference, records intent and settlement, and defines explicit replay policies. While it solves local transaction recovery, exactly‑once guarantees for external effects still rely on traditional idempotency mechanisms.