Why LLMs Remember Yet Forget: The Cost of Evolving User Intent

1. Static Benchmarks Miss Critical Ability Gaps

LLMs are increasingly deployed as collaborative agents for tasks such as vibe coding, deep research, and iterative document editing, where users continuously refine or change requirements. Most mainstream benchmarks are single‑turn and assume the full request is given up front, so they fail to capture performance on evolving intent.

2. Core Modeling: Three Ways User Intent Changes

The authors formalize the user intent at turn t as a four‑tuple I_t = (f_t, C_t, C^rev_t, y_t), where f_t is the desired function (e.g., find_restaurant), C_t the full set of parameters, C^rev_t the subset already revealed to the agent, and y_t the verifiable answer.

Based on this representation they define three intent‑transition types that may co‑occur within a single turn:

Argument Reveal : the intent stays the same but parameters are disclosed incrementally (e.g., “find a restaurant in New York” → “by the way, I’m vegetarian”).

Argument Revision : previously revealed parameters are changed, altering the intent (e.g., “New York” → “change to Brooklyn”).

Function Switch : the task changes while retaining some parameters (e.g., “find a restaurant” → “book a table at that restaurant”).

Reveal is naturally limited by the number of parameters, whereas Revision and Switch can be stacked arbitrarily, enabling long‑range, arbitrarily complex evolving‑intent dialogues.

3. Method Framework: Anchor the Final Turn and Generate History Backwards

The most expensive part of multi‑turn data annotation is labeling the entire conversation. The authors invert the process: instead of constructing dialogues forward, they start from the last turn (the anchor) and work backwards.

Intent Extraction : From any verifiable single‑turn (question, answer) pair, an LLM extracts the function and parameters, fixing the last‑turn intent as the anchor.

Retrospective Expansion : For Revision, synthetic “counterfactual” parameters are generated (e.g., first say “New York”, then revise to “Brooklyn”). For Switch, a predecessor function chain is recursively built (e.g.,

find_appointment_location → find_restaurant → book_restaurant

), ensuring parameter overlap for context inheritance.

Situated Simulation : A scheduler places the various transition events within a fixed number of turns T, obeying five consistency rules (last turn anchored, first turn non‑empty, unique parameters per turn, task must be explicit before switching, counterfactuals precede revisions). A renderer then translates each incremental intent ΔI_t into natural language, prefixed with phrases like “Wait, I forgot to mention…”.

4. Main Experiment: Single‑Turn Scores Do Not Transfer to Multi‑Turn Reliability

The core setup uses six intent transitions (two of each type) across seven dialogue turns. Accuracy drops dramatically compared with the single‑turn baseline.

Key numbers (accuracy change):

GPT‑5.5: GSM8K 99.0 → 80.5 (‑18.7 %); BIRD‑SQL 80.0 → 71.0.

DeepSeek V3.2: BIRD‑SQL 76.0 → 53.0 (‑30.3 %); BrowseComp+ 36.0 → 15.0 (‑58.3 %).

Mistral Large 3: BrowseComp+ 17.0 → 5.0 (‑70.6 %).

SWE‑Bench: GPT‑5.1, Grok‑4.20, Mistral Large 3 fall from 72 %/84 %/56 % to 0 % because the models exhaust tool‑call budgets (100 calls/turn) on irrelevant operations (e.g., sed/grep/find) and time out.

The authors explain that evolving intent forces the model to selectively focus on the updated intent while discarding stale information, a much harder task than single‑turn reasoning.

5. Ablation Studies: Which Transition Is Most Harmful?

5.1 More Transitions, Larger Drop

Scaling the number of each transition type on GSM8K shows monotonic degradation for all four models; Function Switch has the steepest decline because it requires the largest belief‑state update, discarding much accumulated context.

5.2 Complex Combinations Amplify Confusion

When all three transition types are combined (six total transitions), performance continuously worsens; Function Switch remains the hardest case.

5.3 After a Switch, Old Context Is “Forgotten”

If evaluation follows a Switch immediately, models perform reasonably; however, inserting a few Reveal or Revision steps after the Switch causes a further accuracy drop, indicating that the agent cannot fuse pre‑switch context with post‑switch updates.

6. Mitigation: Memory Mechanisms Help but Do Not Close the Gap

The authors test two recap strategies to aid belief tracking:

Prompt Recap : prepend a reminder before each turn, prompting the model to review the dialogue.

Oracle Recap : explicitly restate the full revealed intent up to the current turn (an upper bound).

Both improve performance; Oracle Recap raises Function Switch accuracy from 65 % to 75 % for GPT‑5.5 on BIRD‑SQL, yet even the best “oracle” setting fails to reach single‑turn levels.

The conclusion for agent developers is that passive context recaps are insufficient; future systems must actively infer the current user goal while suppressing irrelevant context.

7. Supplementary Analysis

Difficulty amplification: using BIRD‑SQL’s optional external‑knowledge hints, single‑turn hard questions lose only 3.6 % accuracy versus easy ones, but under multi‑turn evolution the gap widens to 9.0 %.

Intent‑tracking probe: after each turn, an independent judge asks the model “What is the user’s intent now?” Reveal and Revision achieve near‑perfect scores (96–99 %), whereas Function Switch drops to 82 % when it appears twice, confirming that task switching destabilizes the belief state.

https://arxiv.org/pdf/2607.20734