MIT Study: How Self‑Generated History Pollutes LLM Context and Degrades Multi‑Turn Chats

Testing an Unquestioned Assumption

The authors extracted real, noisy conversations from WildChat and ShareLM—actual user‑AI interactions rather than synthetic benchmarks—and evaluated four models (Qwen3‑4B, DeepSeek‑R1‑8B, GPT‑OSS‑20B, and GPT‑5.2) under two conditions.

# Condition A – Standard (what every chatbot does today)
context = [user_1, assistant_1, user_2, assistant_2, …]

# Condition B – Omit assistant replies (never tried before)
context = [user_1, user_2, user_3, …]
# Remove all previous AI responses, keep only human messages.
# Then compare quality. That’s the whole experiment.

Removing prior assistant responses does not affect response quality on a large fraction of turns. Omitting assistant‑side history can reduce cumulative context lengths by up to 10×.

The study found that context length shrank roughly tenfold while response quality remained almost unchanged. In multi‑turn dialogs, 36.4% of prompts are completely self‑contained, and about 70% of turns either need no history or can reconstruct context from user messages alone.

Mechanism of Context Pollution

During a follow‑up, the model sees the new question together with the full text of all its previous replies, including any errors, hallucinations, or stylistic biases. Because the model has no marker distinguishing its own output from external ground‑truth information, early mistakes are reinforced in later turns—a phenomenon the authors name “context pollution.”

Most Dialogues Do Not Require All History

Analysis shows that in typical conversations, the stored assistant history is often irrelevant noise or, worse, a source of distortion. Approximately 70% of turns either contain no useful signal from prior replies or are harmed by them.

Selective Filtering, Not Blind Deletion

The paper does not advocate deleting all history indiscriminately. Performance varies across models: open‑source inference models (DeepSeek‑R1‑8B, GPT‑OSS‑20B) show little quality change when assistant history is removed, whereas the stronger closed‑source GPT‑5.2 suffers a modest drop, indicating that more capable models can extract useful signals from their own past.

To address this, the researchers trained a classifier that decides per turn whether retaining the assistant’s previous output is beneficial. This adaptive omission improves both response quality and context efficiency.

Implications for AI Agents

Current AI agents (e.g., code assistants, web‑browsing bots) store the full interaction trajectory—tool calls, intermediate reasoning, every reply—causing linear growth of context and forcing systems like Cursor or Claude Code to compress or truncate after hitting limits. The study suggests a paradigm shift: rather than asking “when to prune?”, we should ask “why store these replies at all?” and retain only indispensable context.

The industry’s recent focus on ever‑larger context windows (128K, 1M tokens) overlooks the fact that much of the added content provides little value and can even be harmful.

Related Work

Microsoft (2025) reported that LLM performance drops by ~25 percentage points in multi‑turn under‑specified dialogs, coining the term “lost in conversation.” Chroma Research (2025) identified “context rot,” where performance becomes erratic as input length grows, affecting models such as GPT‑4.1, Claude 4, and Gemini 2.5. Another study on “context branching” observed that polluted context leads developers to accept seemingly reasonable but actually incorrect solutions.

Conclusion

For everyday users of AI tools—coding assistants or research agents—starting a new conversation can be more beneficial than persisting with a 20‑turn dialogue that has accumulated errors. For system builders, the default architecture of stacking every turn into a single window wastes compute, adds latency, and actively degrades output quality. The next frontier in agent design lies in dynamic, selective omission rather than merely expanding context capacity.