SkillHone: Tencent WeChat's Continuous Skill Evolution Framework for LLM Agents

LLM agents often call skills that reference missing scripts or outdated documentation, and the same failures recur because the debugging experience stays trapped in chat history rather than being captured in the skill repository.

To solve this, the Tencent WeChat team proposes SkillHone , a harness framework that treats the entire skill repository as the modification target. SkillHone can update stale documentation, repair broken scripts, and add missing references. It integrates into daily workflows: after installation and model configuration, the agent loads relevant skills; when a reproducible failure occurs, the system records an issue and regression test, then enters a repair queue. DeepSeek Harness performs fixes on an isolated branch, runs associated tests, and returns a local pull request. Users review file changes, execution logs, and test results in a workspace before merging.

The core design is persistent decision history . Beyond version diffs that show what files changed, the history retains the problem diagnosis, candidate solutions, evaluation evidence, and the final accept/reject rationale. Subsequent agents can inspect whether a similar fix was already attempted and why it was kept or discarded.

An evaluation-driven process separates concerns using two repositories: a skill repository (optimization side) and an evaluation repository (evaluation side). The optimization side modifies skills but receives only sanitized feedback; the evaluation side sees test targets, validators, and execution traces but cannot modify skills. This permission separation lets the optimizer learn from failure feedback while preventing it from directly reading answers to rewrite skills.

A public reproduction case demonstrates the workflow: the Phoenix tracing Skill indexed four documents that were not shipped with the package, so the agent could not access the promised guidance. The article links to the public issue, the merged fix, and a fixed-version sample for inspection.

Experiments use the GAIA text-only validation subset and WebWalkerQA English set. Table 1 reports average accuracy; the commercial retrieval baseline uses a hosted search service, while other methods run on the open web without pre-integrated search tools.

Table 2 shows ablation results with the same execution model. Removing decision history reduces accuracy by 13.4 and 10.9 percentage points on the two benchmarks, confirming the history's critical contribution.

Internal tool analysis across seven scenarios shows six improvements and one unchanged , with an average gain of 18.8 percentage points over the initial skills.

Unlike frozen-benchmark-driven optimization pipelines, SkillHone's daily maintenance mode starts directly from real failures without requiring a benchmark first; a separate evaluation process can be activated when broader capability measurement is needed. The framework is compatible with CLI-callable agents such as Codex , Claude Code , and Cursor , ensuring that repair outcomes and the reasoning behind them persist for future work.