MemAdapter: Post-Retrieval Calibration Stops Memory-Induced Sycophancy in LLM Agents

Introduction: Memory-Induced Sycophancy

Long-term memory enables LLM agents to retain user preferences and history across tasks and sessions, forming the basis of personalized service. However, the article identifies a critical flaw: historical beliefs and preferences are repeatedly retrieved into subsequent reasoning, even when they are outdated, incorrect, or conflict with current evidence. The agent then blindly accommodates these memories, a phenomenon termed Memory-Induced Sycophancy . Unlike prior approaches that filter "bad memories" during extraction, retrieval, or organization (i.e., before reasoning), this work argues the problem lies not in what is stored but in how memory is used. The solution is a calibration mechanism that operates after retrieval but during reasoning .

Pre-Experiment: Two Counter-Intuitive Findings

The authors conducted controlled experiments using only factually correct, highly relevant memories. Two key findings emerged:

Finding 1: Correct and relevant memories still mislead the model. Adding such memories reduced overall accuracy from 96.0% to 77.3%. Worse, 19.4% of originally correct samples were flipped to incorrect answers, and every flip aligned with the memory's direction.

Finding 2: The appropriate influence of a memory depends on the current task. For example, knowing "the user is a cardiologist" is appropriate for adjusting terminology depth when explaining guidelines, but becomes sycophancy if it causes the agent to ignore patient evidence when selecting a treatment plan.

Conclusion: Memory safety is not a property of the memory content alone, but a relational property of memory × context .

Method: MemAdapter Three-Step Framework

MemAdapter is a post-retrieval calibration framework that plugs into existing memory systems without modifying them. It operates in three stages:

Counterfactual Induction: For a given memory, systematically vary future task conditions to observe when its "appropriate contribution" changes qualitatively. This induces conditional usage boundaries — what the memory can support, what it may influence, and what it must never justify.

Context-Aware Reflection: Match the induced boundaries against the current request and available evidence. Through self-reflection, generate a task-specific "usage instruction" for each memory.

Evidence-Driven Reasoning: While generating the answer, produce an internal supporting trace that verifies segment-by-segment whether sources exist and usage complies with the instructions. Only verified output is returned to the user.

Main Experiments: Three Benchmarks × Five Memory Systems

Experiments covered MemSyco-Bench , PersistBench , and MemTrapBench benchmarks, paired with five memory systems: A-MEM , Mem0 , NaiveRAG , MemoryBank , and LightMem (backbone: DeepSeek-V4-Flash), compared against four intervention baselines.

MemSyco-Bench ("when to use memory" and "how to use memory" dimensions): MemAdapter achieved state-of-the-art across all five systems . On Mem0, "when to use" score jumped from 44.35 to 87.89 (+43.54).

PersistBench (sycophancy failure rate): Failure rates dropped across the board; on MemoryBank, from 82.0 down to 51.0.

MemTrapBench (reasoning solidification and belief distortion scores): Both metrics improved fully.

Trade-off: Some systems showed increased "beneficial memory failure rate", indicating a clear precision-recall style trade-off.

Cross-Backbone Generalization

MemAdapter remained effective when the backbone model was swapped to GPT-5.6-sol and Qwen3-8B .

On Qwen3-8B, NaiveRAG average accuracy surged from 36.86 to 86.67 (+49.8). GPT showed minor variance on the "personalized memory usage" dimension, suggesting performance depends on the interaction between backbone and memory system.

Ablation Study: Every Module Contributes

Using NaiveRAG average accuracy as the metric, modules were added incrementally:

Direct generation: 70.25

+ Counterfactual Induction only: 84.26 ( +14.01, largest single contribution )

+ Reflection module: 85.16

Full MemAdapter: 89.94 ( +19.69 total )

References

MemAdapter: Counterfactual Adaptation Against Memory-Induced Sycophancy
https://arxiv.org/pdf/2610.05162
github.com/DEEP-JLU/MemAdapter