REMORY: 1.9B Memory Module Beats Context Compression for 3-Day Autonomous Research

Overcoming Context Compression with REMORY

Long-running agents hit context-window limits and trigger compression (e.g., /compact) that summarizes history. This creates a sawtooth pattern of context growth and loss, causing tool errors and repeated outputs. The article shows a sawtooth chart for Qwen3.8-27B on Terminal-Bench 2.1 Core War, where red and yellow dots mark tool errors and duplicate generations after each compression.

REMORY Architecture: Fitting the Memory Residual

When compression occurs, the raw context and its summary are fed into REMORY via cross-attention. REMORY outputs a set of continuous residual tokens (soft memory) that are concatenated (⊕) with the summary and passed to the main model. This residual captures information lost during summarization.

Two-Stage Self-Supervised Training

Training is fully self-supervised with the base model frozen. Stage 1 uses content reconstruction to teach the memory module to retain source information. Stage 2 conditions on the summary and minimizes KL divergence to learn the continuation behavior that the full history would support. Gradients flow through the frozen main model, updating only ~1.9B REMORY parameters. Training completes on two H800 GPUs.

Benchmark Results

Terminal-Bench 2.1 (Qwen3.8-27B): Tool errors drop from 933 to 568; duplicate outputs from 459 to 377. Average token cost per task falls from $4.29 to $2.86 (~33% reduction).

BrowseComp: Tool errors decrease 49.7% (25,244 → 12,698); duplicate outputs decrease 29.8% (25,801 → 18,105). Fewer retries shift generation budget toward task progress.

Extension to Stronger Models

REMORY also benefits larger and MoE models. GLM-5.3-Flash scores rise from 84.3 to 87.6 on Terminal-Bench 2.1 and from 84.9 to 89.0 on BrowseComp, with simultaneous reductions in errors and duplicates under identical execution settings.

Real-World Autonomous Research Collaboration

Fudan University scholars collaborated with a REMORY-driven agent over multi-day sessions. The agent continuously read papers, built experiments, reproduced baselines, analyzed results, improved methods, and revised manuscripts. Each compression cycle's data was collected to retrain the memory module, enabling fully self-supervised iterative updates. This demonstrates that residual memory preserves cross-stage evidence, failed paths, and prior insights so the agent can resume research after each /compact.

Resources

Paper: https://arxiv.org/pdf/2610.11287
Hugging Face: https://huggingface.co/papers/2610.11287
GitHub: https://github.com/fudan-generative-vision/Remory
Model on Galaxy Platform: https://aistudio.ai4s.com.cn/partner/research-plaza/galaxy-model/model/Qwen3.8-27B-REMORY-1.9B