PI's Agent Runtime v2: Crash Recovery as the First Constraint

Design Philosophy: Crash Recovery as First Constraint

earendil-works' PI is an open-source agent harness and coding agent with 95k+ GitHub stars. Its v2 runtime rewrite is documented in harness-v2.md, a 3,446-line design doc that makes "can it continue after a crash?" the system's first constraint. The only backward-compatibility requirement is that v3 JSONL session files must open and restore to idle; format, API, and storage layer can be completely redesigned without migration scripts or schema versioning.

Intent Records and Provisioned IDs

The core mechanism for durable operations: before any side-effecting action, write an intent record naming what will happen and the provisioned ID the result will occupy. After the effect, append the result as an entry with exactly that ID. No multi-record atomicity is needed; each record and entry is durable alone. A crash between intent and result leaves the intent unfulfilled; recovery decides per intent type: complete it, retry it, or close it with a synthetic result. An intent is fulfilled iff an entry with its provisioned ID exists. If an ID exists but content mismatches the promise, it is flagged as corruption.

An accepted prompt is a durable operation. After a crash, a new process restores the session. It resumes the run from the last safe boundary. Every state that a crash can produce is recoverable.

Four-Layer Session State

The persistent state splits into four parts:

tree – the conversation itself, entries linked by parentId. Contains messages, model/thinking/tool activations, compaction summaries, branch summaries. Shared, append-only, never modified or deleted after write.

lanes – where work actually happens. A lane is a name plus a leaf entry; subsequent work extends from that leaf. Every session has a main lane; applications can create more using stable keys like Slack thread IDs.

lane operation logs – record execution steps: operation started, step attempted, tool started, message queued, operation finished. Unread during normal execution; exists solely so a new process can pick up a lane's unfinished work after a crash.

global facts – session-scoped key-value pairs, latest write wins (e.g., session name, entry labels).

tree (shared, append-only)     lanes
a ── b ── c ── d               main           → d   (op log: …)
     └── e ── f                 slack:171943…  → f   (op log: …)

global facts: name = "Refactor auth", label(b) = "checkpoint-1"

A lane resembles a git branch checked out in its own worktree: name bound to a position, new work advances it, can move to any existing entry without rewriting history. Tree holds only conversation content, no lane or orchestration state. An entry's parent chain never changes. A lane's leaf moves only by appending an entry or explicit navigation. Only one open operation per lane at a time; two concurrent opens = corruption.

Three Operations: Run, Compaction, Navigation

Run – an accepted prompt that runs through all automatic continuations (tool calls, steering, follow-ups, auto-compaction) until no pending work remains.

Compaction – replaces old context with a summary entry.

Navigation – moves a lane's leaf to an existing entry, optionally leaving a branch summary.

A run is a sequence of turns; a turn = assistant step + full tool batch triggered by that message. A step is the smallest retryable unit: produces an assistant message, compaction summary, or branch summary. Its retry count is persisted on disk, not kept in memory.

Persisted Retry Counts Prevent Infinite Loops

Persisting retry counts prevents a classic failure: request fails → process crashes → restart resets in-memory counter → fails again → infinite loop burning tokens. Once written to a step_attempt record, the count survives crashes; the retry limit applies across any number of restarts.

a crash-restart loop cannot reset it.

Context Growth Only at Tail with Deferred Writes

Across a lane's requests, provider context must only grow at the tail. Inserting into the middle invalidates the KV cache from that point, multiplying token cost. Therefore, when a step needs to write into the conversation mid-execution, it records a deferred write applied at the next checkpoint, appended at the tail.

Across the requests of a lane, provider context only grows at the tail. An insertion before the previous request's tail invalidates the provider's KV cache from that point on and multiplies token cost.

R   step_attempt           request in flight, context ends at user message U
    session.appendMessage(M) caller resolves here
R   write_deferred         full payload, provisioned id
E   assistant message A    provider cached [.., U, A]
E   message M              checkpoint applies the write; tail append

Compaction is the sole deliberate exception: a full cache invalidation traded for a smaller context. The doc explicitly acknowledges this cost trade-off.

Overflow Handling: One Recovery per Conversational Input

When a model response stops with length reason, two distinct cases exist: (1) output hit the configured maxTokens – genuine stop, unrecoverable; (2) output cut off before that limit – context window full or provider truncation, recoverable by compaction + retry. PI distinguishes them by comparing actual generated tokens against the original configured limit , not the possibly clamped value sent to the provider.

One recovery per conversational input. An overflow compaction may start only when no overflow-reason compaction step_attempt is newer than this run's newest consumed conversational message.

If the retried request overflows again, PI writes an "abandoned" error entry and fails the run. Only a new user input (or steering) resets the "already recovered once" flag.

Five Crash Points in Tool Calls and Replay Safety

A tool call is split into five crash points:

E   assistant message, calls c1, c2
X1  before before_tool               nothing durable for c1
H   before_tool(c1)
X2  decision made, nothing written     same as X1
R   tool_started(c1)
X3  tool executing
H   after_tool(c1)
X4  hook interrupted                   same durable state as X3
E   tool result c1
X5  result durable                     c1 finished

X1/X2: nothing durable → re-run whole path. X5: result entry exists → skip. The danger zone is X3/X4: tool_started persisted, side effect may or may not have occurred. PI's solution: each tool declares replay: "never" | "safe", snapshotted into the tool_started record at execution time. Recovery re-executes only if both the snapshotted value and the current tool declaration say "safe"; otherwise writes a synthetic "interrupted" result. Dual confirmation handles tool implementation drift over time.

The tool's declared replay safety, snapshotted at execution time. Recovery re-executes an unfinished call only when this field AND the current tool declaration both say "safe"; otherwise it writes a synthetic "interrupted" result.

Suspended Lanes Indistinguishable from Crashed Lanes

For non-instant providers (batch API, background: true), the handle-bearing message is persisted, the lane suspends, and prompt() returns "suspended". A different process later calls resume() to fulfill the handle, appending the real result. In storage, a suspended lane looks identical to a crashed one: an open operation whose newest entry is a deferred assistant message with no successor. Recovery logic treats both as suspended; resume() only checks whether the handle has been fulfilled.

The suspended lane is indistinguishable from a crashed one in storage: an open operation whose newest entry is a deferred assistant message with no successor.

Cost Durability Independent of Result Durability

Every provider request writes a usage record immediately upon settlement, before any classification or retry decision. Retryable steps may produce responses that never become entries (failed attempts, exhausted retries, discarded overflow responses). If cost tracking depended on result durability, those costs would vanish with the discarded responses. Hence usage is a separate, unconditional record.

cost durability must not depend on result durability.

The doc admits one irreducible window: if the transport layer dies between response settlement and usage write, that cost is lost.

Race Conditions Resolved via Lane Mutation Line

Long-running operations naturally race with new inputs. Each lane has a single FIFO promise queue called the lane mutation line . Any "check-state-then-decide" operation must run as a job on this queue: validate, at most one durable write, update in-memory state. Provider requests, tool execution, hooks, and backoff are not allowed inside the job. Because jobs run sequentially, two concurrent operations on a lane have exactly two possible histories – [A, B] or [B, A] – both defined outcomes; no interleaved third history exists.

Because jobs run one at a time, two concurrent operations on a lane have exactly two possible histories — [A, B] or [B, A] — and both are defined outcomes. No third, interleaved history exists.

A 12-line race catalog enumerates pairs ( prompt() vs prompt(), steer vs run finish, abort vs queue consumption) with their two legal histories and enabling mechanisms. One case admits sorting cannot solve it: abort hitting an in-flight provider call or tool side effect – handled same as crash: intent record + replay policy.

Recovery Self-Checks: Writer/Reducer Drift Detection

restore

is read-only, starts with indexed lookup ( findOpenOperations returning unfinished ops in reverse chronological order, scoped to the lane). The standout self-check: after every resume(), the harness re-reduces the entire storage state from scratch and compares it against the in-memory state maintained during runtime. Any mismatch = corruption, immediate fault. This catches writer/reducer drift at the moment it happens, not one crash later. Recovery itself is idempotent: existing provisioned IDs are skipped; a crash during recovery simply leaves fewer items to recover.

writer/reducer drift is caught the moment it happens instead of one crash later.

Exhaustive Crash-Point Testing with Manual Drive Mode

Section 19 defines three test layers. The third layer uses drive: "manual" mode: the harness pauses before every side effect, describes the pending action to test code, which decides when to proceed. Crash simulation: call close() at a boundary, reopen the same backend, call resume(). Crash points are not hand-picked:

drive each section 6 trace in manual mode, snapshot the backend after every executeAction() , then reopen every snapshot and resume() — and run recovery twice per snapshot, proving half-completed recovery is safe. New effects added to a trace get crash coverage automatically.

Production and tests run the same code; only the drive mode differs. Manual mode exists so tests can step through execution one action at a time.

Costs and Non-Goals

A typical tool-involved turn writes: user message, step_attempt, assistant message, tool_started, tool result, another step_attempt – every item a durable write before the model reply finishes. Storage inherits real complexity: SQLite branch cache has four cases, worst-case copies an entire path; JSONL handles torn tails (last line half-written). Hard single-writer constraint: only one process may write a session at a time; concurrency via multiple lanes, not multiple processes. Default does not even promise fsync; power loss survival is not guaranteed (may be added later as explicit capability). Non-goals: no migration (except v3 compat), no multi-writer, no replication. The design bets on workloads where a crash-recovery cost of hours far exceeds per-step write overhead – not on simple Q&A interactions.

Implementation as Small Work Packages

The spec is broken into dozens of small work packages – F0, R0 – R3, J0 – J6, H0 – H8 – each small enough for one person to own. The repo shows records like "Reserved: I2 by @vegarsti". The delivery method mirrors the design: many small pieces, distributed among many people, each can be picked up and continued from the record if dropped.

Source:

https://github.com/earendil-works/pi/blob/harness-v2/j4/packages/agent/docs/harness-v2.md