MemSkill: Evolving Memory Skills for Self-Improving Agents

Paper Overview

Title: MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents

Authors: Haozhen Zhang, Quanyu Long, Jianzhu Bao, Tao Feng, Weizhi Zhang, Haodong Yue, Wenya Wang

Affiliations: Nanyang Technological University, University of Illinois Urbana-Champaign, University of Illinois Chicago, Tsinghua University

Venue: NeurIPS 2026

arXiv: https://arxiv.org/abs/2602.02474 GitHub:

https://github.com/ViktorAxelsen/MemSkill

Core Idea: Memory Operations as Evolvable Skills

Recursive Self-Improvement (RSI) has sparked interest in agents that continuously improve from interaction experience. Existing memory systems rely on fixed operations (add, update, delete, skip) that cannot adapt their memory construction strategy based on task feedback. MemSkill addresses this by encapsulating memory operations into Memory Skills — natural-language descriptions of purpose, applicability conditions, execution steps, and constraints — making them learnable, reusable, and evolvable.

Figure 1: Comparison of per-turn operation vs. segment-level skill-guided construction

Method Framework: Learning Skill Selection and Evolving the Skill Bank

The framework starts from four initial memory operations. A Controller selects Top-K skills for each text segment; an Executor feeds the segment, existing memory, and selected skills to an LLM to generate memory updates. Downstream task performance provides reward to update the Controller via reinforcement learning. A Designer periodically analyzes failure cases collected in a Hard Case Buffer, revises existing skills, and adds new ones. The Controller then trains on the updated skill bank, forming a self-evolution loop.

Figure 2: Skill learning and evolution loop

Controller and Executor: Skill Selection and Memory Construction

The lightweight Controller sequentially picks Top-K skills per segment. The Executor combines the current segment, existing memory, and chosen skills in a single LLM call, reducing the overhead of per-turn, per-operation processing. During training, the constructed memory is used for downstream QA or interaction tasks; task performance serves as reward to update the Controller via RL.

Designer: Continuous Skill Bank Refinement

Failure cases are stored in a Hard Case Buffer. The Designer periodically samples representative hard cases, identifies missing or erroneous memory rules, and revises or adds skills. The Controller then continues training on the expanded skill bank, closing the loop from task feedback to skill improvement to selection policy update.

Experimental Results: Comprehensive Evaluation on Long-Term Dialogue and Embodied Interaction

Experiments use LLaMA and Qwen as base models across three benchmarks: LoCoMo (long-term conversation), LongMemEval (long-term memory evaluation), and ALFWorld (embodied interaction). Results show MemSkill achieves the highest L-J score and ALFWorld average success rate on both base models.

Table 1: Main experimental results

Notably, Qwen results come from direct transfer of skills learned on LLaMA; LongMemEval reuses the skill bank from LoCoMo — no retraining required.

Transfer Evaluation: From Dialogue Memory to Multi-Hop QA

Skills evolved on LoCoMo are transferred zero-shot to HotpotQA (document-based multi-hop QA). With 7 skills selected, MemSkill outperforms MemoryOS and A-MEM across three context lengths. Using 7 skills beats 3 or 5, indicating that combining more skills within a range improves transfer.

Figure 3: LoCoMo skill transfer to HotpotQA results

Ablation and Efficiency

Ablation variants (random skill selection, disabling Designer, only revising old skills without adding new ones) all underperform the full framework, confirming contributions of both skill selection and skill evolution, with new skills providing extra gains. Efficiency comparison (Table 3) shows MemSkill reduces token usage and LLM calls by nearly 70% compared to LightMem while improving answer quality, demonstrating the benefit of segment-level construction.

Table 2 and Table 3: Ablation results (left) and inference overhead (right)

Case Analysis: Task-Specific Memory Skills

Skills evolved on LoCoMo emphasize temporal relations, activity details, and strict adherence to explicitly stated information. Skills from ALFWorld focus on action constraints, object locations, and state updates. This demonstrates MemSkill's ability to evolve domain-specific memory skills tailored to different task demands.

Figure 4: Representative skills in dialogue and interaction tasks

Conclusion and Future Work

MemSkill enables memory systems to continuously evolve based on user feedback during long-term interaction, gradually adapting to individual needs. Future work aims to extend optimization signals from benchmark QA pairs to real user feedback — follow-up questions, corrections, satisfaction signals — to guide skill adjustments in content and granularity. Evaluation must also shift from static QA to long-term interaction benchmarks with continuous feedback and preference changes to validate sustained personalization.