Data Party THU

Jun 7, 2026 · Artificial Intelligence

When Long Prompts Cause Forgetting: Understanding Generalization in In‑Context Continual Learning

The paper introduces a theoretical framework for In‑Context Continual Learning, showing how shared attention in large language models creates bias, variance, and a novel interference term that explains why longer prompts can lead to forgetting, and provides concrete guidelines for prompt design based on task similarity, context length, and order.