LightRSI TokenPilot: Slash LLM Agent Context Costs Up to 87% with Smart Cleanup

Problem: Context Explosion in Long-Running Agent Sessions

When using an agent for coding, a single task often involves multiple rounds of file reading, tool calls, code edits, and test runs. As the session grows, the accumulated history — previously read files, repeated test logs, completed tasks — continues to be carried in subsequent requests. This redundant context increases inference overhead without adding value for the current step.

Solution: TokenPilot — Cache-Efficient Context Management

TokenPilot is the context management component of the LightRSI project (aimed at low-cost recursive self-improvement of agents). It tackles redundancy from three angles while preserving useful history and cache reuse:

1. Stabilize Reusable Prefixes for Cache Hits

Agent requests often contain recurring instructions and structures. Dynamic fields like timestamps or temporary IDs can break prefix matching on caching-enabled model backends. TokenPilot stabilizes suitable dynamic fields and layouts, reducing unnecessary variations so the backend can reuse cached computations more often.

2. Summarize Tool Outputs at Ingestion

Raw tool results — verbose web pages with heavy markup, lengthy test logs — enter the context and are then repeatedly resent. TokenPilot condenses structural noise and overlong outputs at the point they enter the context, while archiving the original content for on-demand restoration when the agent needs details.

3. Manage Old Task History by Future Value

Completed tasks may still leave behind useful artifacts (recently read docs, located file positions). TokenPilot, where supported by the integrated tool, retains such information based on estimated future utility and evicts it only when conditions indicate it is no longer needed.

Tool Integrations and Visualization

The project now provides adapters for six coding agents: OpenClaw, Claude Code, Codex, pi, DeepSeek Harness (DSH), and OpenCode . The latter two (pi and OpenCode) are new additions that run via internal extensions/plugins without an extra request proxy.

After installation, users can launch a local browser dashboard with: lightrsi visual The dashboard offers three views:

Prefix Stability: Cache audit and prefix stability metrics showing what changed between requests (Figure 1).

Tool Output Summarization: Before/after size comparison, summarization logs, and active rules (Figure 2).

History Management: Cleanup status and records for old task context (Figure 3).

Visualization is shared across OpenClaw, Claude Code, Codex, pi, and OpenCode; DSH uses native plugins and status commands. Automatic history cleaning for pi and OpenCode is off by default and must be enabled explicitly. Panel metrics reflect optimization progress, not direct billing savings.

Context Cleaner: User-Driven Task-Level Cleanup

Beyond automation, Context Cleaner lets users manually curate context by task. It lists identifiable context segments with descriptions, occupancy, keep/clean suggestions, and protected items. Users review and explicitly confirm which tasks to clean — no action runs without confirmation. Ongoing tasks, unresolved issues, system instructions, and content failing safety checks are protected. Cleaner currently supports OpenClaw, Codex, Claude Code, and DSH (not yet pi or OpenCode). Approved plans execute on the next normal request in the same session; executed cleanups are irreversible, but pending plans can be cancelled. Project files remain untouched.

Experimental Results

Evaluated on PinchBench and Claw-Eval under continuous multi-task sessions (same session reused), TokenPilot reduced inference costs significantly:

PinchBench: Baseline cost $7.24 → TokenPilot cost $2.79 (≈61% reduction)

Claw-Eval: Baseline cost $81.52 → TokenPilot cost $10.58 (≈87% reduction)

Task scores: PinchBench improved from 79.2 to 81.3; Claw-Eval dipped from 63.4 to 60.8, indicating a trade-off that should be weighed per workload. Macro cache hit rates rose from 38.7% to 79.2% (PinchBench) and from 67.2% to 83.1% (Claw-Eval). These figures come from specific model, task, and billing configurations and do not represent uniform measurements across all six integrated platforms. Real-world gains depend on backend cache support, tool output volume, and inter-task relatedness.

Future Direction

LightRSI will continue expanding tool integrations, refining the visualization and cleaning experience, and developing lighter ways to leverage accumulated execution traces for subsequent agent improvement. Details and results will follow in upcoming releases.

Resources

Project: https://github.com/zjunlp/LightRSI Paper: https://arxiv.org/abs/2606.17016 (TokenPilot: Cache-Efficient Context Management for LLM Agents, EMNLP 2026)

Reproduction:

https://github.com/Xubqpanda/TokenPilot