Lossless Context Management (LCM): Handling Unlimited Agent Tasks with Finite Windows

LLM‑based agents face a fundamental limitation: the model’s context window is finite while real‑world tasks can be unbounded. Although window sizes have grown from 4 K to 1 M tokens, larger windows do not equate to better memory.

Existing approaches

Truncate

Simply discarding the oldest messages keeps the token count low but is lossy and irreversible, especially problematic for tool‑generated content.

Summarization

Models can compress older dialogue into summaries according to rules (e.g., context proportion, number of turns, tool calls). The quality of the summary depends on the model.

RAG

Embedding historical information in a vector database and retrieving the most similar fragments works well for static knowledge bases but fails for highly coherent conversational histories, often returning redundant or structurally important fragments such as “who said what, what was responded, and what was finally decided.”

Lossless Context Management (LCM)

LCM paper: https://papers.voltropy.com/LCM

LCM introduces a “structured control flow” for context handling, analogous to structured programming replacing unrestricted GOTO.

Architecture

Immutable Store records every user message, AI reply, and tool result unchanged, providing a single source of truth.

Active Context is the view presented to the model, composed of recent raw messages mixed with LLM‑generated summary nodes.

Summaries are derived caches; the original data remain intact, allowing any summary to be expanded back to its source, which is the meaning of “lossless” in the paper.

Hierarchical DAG

As dialogue progresses, older messages are compressed into summary nodes forming a directed acyclic graph:

D0 leaf nodes : finest‑grained summaries covering the most recent minutes.

D1 nodes : aggregate multiple D0 nodes to cover several hours.

D2 nodes : aggregate D1 nodes to cover days.

The model sees the active context consisting of system prompts, the highest‑level summary node (full history overview), and recent raw messages (detail).

Each summary node stores a pointer to its source messages, accessible via three tools: lcm_grep – full‑text search. lcm_describe – view node metadata. lcm_expand – expand a summary back to the original text.

Three‑level compression

L1 (detailed summary) : standard LLM summarization that retains details.

L2 (aggressive compression) : if L1 does not reduce token count, switch to bullet‑point mode aiming to halve tokens.

L3 (deterministic truncation) : if L2 still fails, truncate to a fixed length without invoking the LLM.

Zero overhead for short tasks

LCM sets soft and hard thresholds. Below the soft threshold, the immutable store is passive and the model sees raw dialogue with no latency. When the soft threshold is exceeded, compression runs asynchronously between turns, causing no user‑visible delay. Only extreme cases that hit the hard threshold incur synchronous blocking.

Large‑scale data processing

LCM defines two operators, LLM‑Map and Agentic‑Map , to offload massive dataset processing to the engine:

# LCM style: model declares intent, engine executes
tool_call("llm_map",
    input_path="dataset.jsonl",
    prompt="Extract key entities from each record...",
    output_schema={...},
    concurrency=16)

The engine tracks each entry’s state in persistent storage, uses pessimistic locking for exactly‑once semantics, retries failures, and validates outputs with JSON Schema. Input and output files remain on disk, outside the model’s active context, so the model never sees the raw dataset, only the aggregated summary.

Conclusion

LCM defines “lossless” as the ability to retrieve any historical state, though it cannot guarantee the model will always query history correctly. As agent tasks grow longer and more complex, robust context management remains a challenging component of AI system design.