Why Long-Term LLM Agents Fail: The Same Design Choice Behind Two Deaths

Problem

Long‑term agents that autonomously execute multi‑step tasks (e.g., managing a 500‑shelf warehouse or solving Linux CTF challenges) exhibit two symptoms when the number of steps grows: latency increases and the agent eventually “dies” from context poisoning. The root cause identified by Badhe, Tiwari, and Chung is the design choice of appending every observation, action, and intermediate reasoning trace to an ever‑growing dialogue history.

Method (SKILL.state)

SKILL.state replaces the ever‑increasing history with an explicit, mutable execution state. At each step the model receives only three inputs:

Immutable skill specification P Current structured state Σ<sub>t</sub> Latest observation O<sub>t</sub> After producing a verified state update, the intermediate reasoning is discarded. Prompt size therefore remains constant while the state itself grows.

Analogy: the traditional approach is a diary that the agent rereads from the beginning before every decision; SKILL.state maintains a single “current state table” that is updated and then discarded.

Complexity

Traditional methods incur a prompt length that grows linearly with the step count, leading to cumulative token consumption of O(T²) . SKILL.state keeps the per‑step prompt size fixed, yielding cumulative token consumption of O(T) .

Experimental Setup

Two layers of benchmarks were used.

Self‑built benchmark: SkillExecBench

Warehouse Management : 500 independent shelves with actions Store, Ship, Move, Wait. Tests long‑term maintenance of independent state variables.

Software Repository : Nested Git graph with branches, commits, pull requests, CI status. Actions include CherryPick, Merge, RunTests, CreateRelease, Rollback. Tests complex structural reasoning.

Public benchmarks

InterCode CTF : 100 Linux bash CTF challenges covering reverse engineering, forensics, cryptography, binary exploitation. Agents execute commands in Docker containers.

Sierra τ‑Bench : Simulated enterprise customer‑service scenarios (Retail and Airline). Agents converse with simulated users, query SQLite databases, and perform transactional operations while respecting business constraints.

Four baselines were compared:

ReAct (full history)

Memory (3‑step sliding window + periodic summarization)

Stateful/LangGraph (structured state injected alongside history)

SKILL.state

Budget‑matched control groups included sliding‑window truncation, hard‑limit summarization, and LLMLingua statistical compression.

Models: Gemini‑3‑Flash, Gemma‑4‑31B‑it, Qwen‑3‑8B‑it. All experiments used temperature 0 and top‑p 1.

Results

Warehouse Management (T = 200)

SKILL.state achieved 0.94 accuracy with 122 k tokens. Memory baseline achieved 0.84 accuracy with 6.1 M tokens.

Warehouse Management (T = 100)

SKILL.state used 65 k tokens; Stateful used 1.06 M tokens (16.2× more).

Software Repository (T = 100)

SKILL.state reached 0.78 accuracy with 90 k tokens; Prompt baseline reached 0.53 accuracy with 1.85 M tokens.

Noise‑robustness (Warehouse, T = 50)

Three levels of irrelevant telemetry were injected. Prompt baseline accuracy fell from 0.68 to 0.53, while SKILL.state remained above 0.97 because noise was filtered out during state‑patch generation.

State‑recovery

When external changes (e.g., hidden shelf items moved) occurred, baselines hallucinated 5–8 steps, whereas SKILL.state recovered instantly with zero‑step lag, updating the structured state directly from the new observation.

Public benchmarks

InterCode CTF: SKILL.state pass@1 = 54.2 % (7.8 pp higher than the strongest baseline, 12.4 pp over Stateful). Token consumption 60.4 % lower than ReAct and 65.9 % lower than Stateful. The state schema used five static fields (discovered_flags, tested_hypotheses, active_files, working_dir, cmd_summary) shared across all 100 tasks.

τ‑Bench Retail: 58.3 % pass rate with the lowest token consumption among all methods.

τ‑Bench Airline: 32.4 % pass rate; prompt size stayed around 2.8 k tokens per step, whereas baselines exceeded 11 k tokens per step.

Budget‑matching experiment

All methods were limited to the same 1 800 token/step budget on the Warehouse task (T = 100). Results: sliding‑window truncation 0.18 accuracy, LLMLingua 0.22 accuracy, SKILL.state 0.94 accuracy. Structured state preserved precise relational dependencies that statistical compressors destroyed.

Open‑Source Model Analysis

Error analysis on Gemma‑4‑31B (T = 100, overall score 0.42) showed:

68 % of errors were premature overwrites or deletions of existing keys rather than in‑place merges.

20 % stemmed from schema or type‑conversion misunderstandings (e.g., mismatched nested list/dict expectations).

12 % were JSON syntax mistakes (missing separators, trailing commas).

The authors suggest integrating syntax‑constrained decoding to eliminate formatting errors.

Limitations

Tasks without a fixed schema where the state structure must be discovered dynamically.

Scenarios where an early observation’s importance is missed and never added to the state.

Objectives defined directly on the interaction history (e.g., audit, debugging, explanation), where the history itself is the required output.

Discussion

Although some view SKILL.state as a simple token‑saving trick, budget‑matched controls demonstrate that the benefit arises from replacing the “record everything” abstraction with a “current‑where‑we‑are” abstraction. This shift fundamentally changes runtime behavior rather than merely discarding information.

Paper Link

https://arxiv.org/abs/2608.26263