Avoiding Skill Hell: Writing Agent Skills That Remain Predictable, Not Outdated Wikis

Skill Hell

When downloadable Skills become abundant, agents face four concrete problems:

Routing : deciding which Skill to trigger and when.

Context : determining which content should be resident and which should be loaded on demand.

Execution : defining the exact steps of a process and the condition for completion.

Maintenance : identifying outdated, duplicated, or no‑op rules.

Simply moving team knowledge into a Markdown file is insufficient; a disciplined interface design is required.

Filesystem layout

my-skill/
  SKILL.md
  references/
  scripts/
  assets/

This layout enables progressive disclosure: only metadata is loaded initially; scripts or assets are fetched when needed.

Three‑layer split

description

: routing trigger words, scope, and boundaries. Judgement criterion: the agent can decide whether to trigger without reading the main file. SKILL.md: main flow, branches, and short completion standards. Judgement criterion: every run uses these steps. references/, scripts/, assets/: detailed references, templates, test cards, deterministic scripts. Judgement criterion: only needed for specific branches or steps.

Example: Refund Verification

# Refund Verification

Use this skill to verify refund‑related changes before reporting completion.

## Steps
1. Identify the changed refund paths.
   Completion criterion: every modified refund handler, service, and test file is listed.
2. Run the refund test subset.
   Completion criterion: command, exit code, and failing cases are captured.
3. Check external side effects.
   Completion criterion: no production credentials, live payment endpoints, or irreversible writes are used.
4. Produce evidence.
   Completion criterion: final response includes changed files, commands run, results, and unresolved risks.

## References
- `references/refund-test-cards.md`
- `references/refund-gotchas.md`
- `scripts/check_refund_events.py`

Each step specifies concrete evidence (commands, exit codes, file lists, risk checks) so the agent can verify completion autonomously and a human can audit the result.

Evidence‑rich steps

Weak step contracts such as “Understand the task” or “Test your work” lack verifiable criteria. A robust completion criterion must satisfy two conditions:

The agent can autonomously determine whether the step succeeded.

A human can quickly audit the recorded evidence.

Concrete example for a test step:

Completion criterion:
- `npm test -- billing/refund` has been run.
- The final response includes the exact command, exit code, and failing test names if any.
- If the command cannot run, the blocker and next human action are recorded.

Leading words

Stable, well‑defined terms (e.g., vertical slice , red‑green‑refactor ) compress semantics and let models reuse priors. Invented jargon without clear definitions adds overhead and should be avoided.

Deletion difficulty

Skills tend to accumulate:

duplication : the same rule appears in multiple places.

sediment : old rules linger without verification.

sprawl : the file becomes long even though each line is still active.

no‑op : rules that do not change agent behavior.

Pruning should start with no‑ops: remove a rule only if its deletion does not degrade agent behavior. Convert useful but vague rules into actionable checks (e.g., replace “Follow security best practices” with an explicit checklist of external inputs, file writes, network calls, and permission changes).

External dependencies

When a Skill is shared, treat it like a software dependency. Audit the following aspects (as recommended by Anthropic and OpenAI):

Trigger scope – is the description too broad?

File access – does the Skill read unnecessary directories, secrets, or user data?

Script behavior – does any script perform network requests, deletions, writes, or install dependencies?

External URLs – does the Skill fetch remote content and execute it?

Permission boundaries – does the Skill require production, payment, email, or database write permissions?

Evidence requirements – are there tests, logs, or exit criteria beyond a generic “completed”?

Maintenance metadata – author, version, last update, license, and change log.

Trigger mechanisms

Skills can be invoked automatically (model‑invoked) or manually (user‑invoked). The trade‑off is context cost versus human effort:

Automatic trigger : the agent matches the description against the current context. Suitable for high‑frequency, low‑risk tasks with stable trigger words (e.g., generating a commit message). Cost: the description occupies context and may cause mis‑trigger.

Manual trigger : a human explicitly names the Skill. Suitable for low‑frequency, high‑impact tasks that require human judgment (e.g., security audit). Cost: the human must remember the Skill and invoke it at the right moment.

Router Skill : a dedicated Skill that maps user intents to concrete Skills, reducing the cognitive load on humans but requiring its own maintenance.

Step‑by‑step workflow to create a first Skill

Select a high‑frequency, well‑bounded task that repeats in the last two weeks.

Write a description that lists precise trigger words and explicitly states when the Skill should not be used.

Draft SKILL.md with 5–8 main steps, each paired with a concrete completion criterion.

Move long references, templates, and test cards into references/.

Implement deterministic checks as scripts in scripts/ instead of relying on model intuition.

Run the Skill on three real tasks, recording:

Whether it triggered correctly.

If the agent skipped required pre‑checks.

Whether evidence (commands, outputs, risks) was captured.

Context size (did the main file pull in too much material?).

Security concerns (file access, network calls).

Prune no‑ops and duplicate rules; decide whether automatic triggering is safe.

Health‑check checklist for existing Skills

Trigger range : Is the description overly broad?

File access : Does the Skill request unnecessary files or secrets?

Script behavior : Do scripts perform network requests, deletions, or privileged actions?

External dependencies : Are remote URLs fetched and executed?

Permission boundaries : Does the Skill need production or payment permissions?

Evidence : Are there concrete tests, logs, or exit criteria?

Maintenance status : Are author, version, and change log clearly documented?

Symptom diagnosis table (text version)

Agent does not trigger Skill – suspect an overly generic description. Check trigger words and real task prompts.

Skill triggers but runs off‑track – likely missing or weak completion criteria. Review SKILL.md and its completion criterion entries.

Agent reads too much material – main file probably contains reference material. Move such content to references/.

Output claims completion without evidence – output contract is too weak. Ensure final response lists commands, exit codes, and any unresolved risks.

File grows continuously – no disciplined pruning. Identify duplication, sediment, sprawl, and no‑op entries and remove them.

Conclusion

A Skill should be a tiny, well‑defined interface that makes a process predictable, auditable, and maintainable. Keep the description as a routing contract, the SKILL.md short and behavior‑changing, steps evidence‑rich, and prune no‑ops, duplicates, and expired content. Treat external Skills as software dependencies and audit them for file access, script behavior, external calls, and permission boundaries. By following the three‑layer split, concrete evidence, and disciplined pruning, teams can prevent “Skill Hell” and keep agent behavior stable while leveraging reusable Skills.