Can Single-Agent Skill Systems Outperform Multi-Agent Architectures?

Background

Claude Skills are operations constrained by a pattern, with a semantic descriptor, a clearly defined input‑output signature, and an execution strategy. They are the fundamental building blocks for modern AI agents.

Empirical Findings from Recent Analyses

Security scan of 42,447 public skill packages (31,132 after deduplication) revealed 8,126 confirmed vulnerabilities (precision 86.7 %, recall 82.5 %, F1 84.6 %). Data‑leakage was the most common issue.

When a single‑agent skill set exceeds 50–100 skills, accuracy drops from >95 % to ~20 %, indicating a systemic phase‑transition rather than a gradual decline.

Even with only 20 skills, high semantic similarity (e.g., “Calculate Sum”, “Compute Total”, “Sum Numbers”) reduces accuracy to 37–70 %.

Single‑Agent Skill System (SAS)

The proposed SAS treats each skill as a triple:

δ – semantic descriptor (the skill name used for selection).

π – execution strategy (how the skill is performed).

ξ – execution backend (internal reasoning or external tool call).

Key Insight

The communication graph of a multi‑agent system can be transformed into implicit constraints between skills; for example, the output format of skill A must be consumable by skill B, which becomes a format requirement in SAS.

Experimental Validation: Efficiency Gains

Three representative multi‑agent architectures (AutoGen‑style, MetaGPT‑style, and a generic MAS) were compiled into SAS and evaluated on benchmark tasks (including HotpotQA). Results show near‑lossless accuracy (average +0.7 % improvement, HotpotQA +4 %) while achieving:

Token consumption reduced by an average of 53.7 % (peak reduction 58.4 %).

Latency reduced by an average of 49.5 % (peak reduction 60.9 %).

API calls per task collapsed from 3–4 calls to a single call.

Cognitive Capacity Limits (Experiment H1)

Skill‑selection accuracy was measured for skill libraries of size |S| = 5 → 200 using GPT‑4o‑mini and GPT‑4o. The results exhibit a non‑linear phase transition:

|S| ≤ 20 → accuracy > 95 %.

|S| ≈ 50 → rapid accuracy decline.

|S| > 100 → accuracy plateaus around 20 %.

This behavior aligns with Hick’s law and working‑memory limits, defining a critical capacity threshold κ of roughly 50–100 skills.

Semantic Confusion as Primary Cause (Experiment H2)

When the skill set contains semantically overlapping entries, accuracy collapses even if the total number of skills is modest. With 20 non‑conflicting skills, selection accuracy reaches 100 %; adding highly similar competitors drops accuracy to 7–63 %.

Hierarchical Routing Solution (Experiment H4)

Three routing strategies were compared:

Flat selection (baseline): choose directly from the full skill list.

Naïve domain hierarchy: first select a broad domain (e.g., math, writing), then a specific skill.

Confusion‑aware hierarchy: group semantically similar skills, select the group first, then the concrete skill.

For |S| > 60, the confusion‑aware hierarchy restores accuracy from ~45 % to 83–85 % (a 37–40 % relative gain) and ensures that each decision point presents fewer than κ options.

SkillScan Security Analysis

SkillScan implements a three‑stage pipeline:

Collection : crawled two public skill marketplaces, obtaining 42,447 skill packages (anonymized).

Deduplication/Filtering : removed duplicates, invalid entries, and pure‑document packages, leaving 31,132 skills for analysis.

Detection Engine : static analysis (AST parsing, regex, dependency graph), LLM‑based semantic classification (GPT‑4o fine‑tuned), followed by manual verification.

The engine identified 8,126 confirmed vulnerabilities (precision 86.7 %, recall 82.5 %, F1 84.6 %). Four attacker groups and 14 distinct vulnerability patterns were catalogued.

Implications

When skill libraries are kept below the cognitive capacity threshold κ or organized with a hierarchical routing scheme, a single‑agent skill system can match or surpass the performance of multi‑agent collaborations while dramatically reducing token usage, latency, and API calls. Systematic security vetting (e.g., SkillScan) is essential to mitigate the high prevalence of vulnerabilities in publicly shared skills.

https://arxiv.org/pdf/2601.04748
When Single-Agent with Skills Replace Multi-Agent Systems and When They Fail

https://arxiv.org/pdf/2601.10338
Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale