12 Pitfalls I Learned While Building AI Skills Over Six Months

1. A Skill Is Not Just About Writing Prompts

The author initially assumed that writing prompts was enough to create a functional Skill, but quickly realized that a Skill requires a full workflow design: trigger conditions, execution boundaries, error handling, and output standards. Missing any of these leads to failure, as illustrated by a colleague’s Skill that unintentionally executed an SQL statement on its first day.

2. More Skills Aren’t Always Better

Enthusiasm led to installing many Skills that were rarely used; less than a third proved useful, while the rest caused accidental triggers and increased token consumption for loading metadata. The author now screens new Skills with three questions about frequency, time saved, and potential conflicts.

3. Writing Descriptions Carelessly Leads to Trouble

A one‑line description like "a code review tool" caused the Agent to misinterpret when to invoke the Skill. The author refined the description to a detailed instruction: "Review code changes in the current diff, checking logical errors, test coverage, and destructive changes. Trigger automatically when the diff contains modifications." This clearer description halved mis‑trigger rates.

"Review code changes in the current diff, checking logical errors, test coverage, and destructive changes. Trigger automatically when the diff contains modifications."

4. Using Skills Doesn’t Always Save Money

While Skills load incrementally, loading many of them consumes a noticeable amount of token budget. Over‑optimizing descriptions to save tokens can backfire: a overly terse publishing Skill sent drafts without confirmation, costing far more than the saved tokens.

5. Where Should Knowledge Reside? Depends on Its Stability

Static knowledge (e.g., team coding standards) belongs inside the Skill, whereas frequently changing data (e.g., database schema that changes multiple times a week) should be fetched from an external source at runtime.

6. A Skill Is More Than a Markdown File

Markdown only tells the model *how* to act; the real work is performed by attached scripts, templates, and toolchains. Deterministic operations are delegated to scripts, while the model handles judgment and creative tasks, keeping each component within its strengths.

7. Over‑ or Under‑Granular Skill Splitting Causes Issues

Bundling all functionality into one large Skill makes trigger logic tangled and hard to modify. Conversely, splitting a single function into many tiny Skills forces the Agent to spend excessive time routing. The recommended approach is to split by call boundaries: coarse grouping for user‑initiated Skills, fine granularity for automatically triggered ones.

8. Skills Can’t Bridge External Systems on Their Own

Skills define the sequence of actions but do not provide connectivity. Integration with external systems requires MCP, APIs, SDKs, or scripts. If an Agent cannot reach a database, the failure is due to missing connectivity layers, not a flawed Skill.

9. Tool Declarations Aren’t a Security Safeguard

Some platforms let Skills declare allowed tools, but enforcement depends on the host. Sensitive operations such as database writes, messaging, or file deletions must still rely on permission systems and human confirmation; the author enforces manual approval for any write operation.

10. The Same Skill May Behave Differently Across Platforms

A Skill that runs smoothly in Claude Code failed in Codex because the two platforms use different trigger mechanisms (frontmatter vs. YAML config). Therefore, Skills must be authored with awareness of the target platform’s execution model and trigger design.

11. Skills Without Acceptance Criteria Are Incomplete

Initially the author listed only procedural steps, resulting in outputs that diverged from expectations. Adding explicit acceptance criteria—defining completion, quality thresholds, and failure conditions—dramatically improved output consistency. Example criteria include capturing full source content, distinguishing author opinions from analysis, providing actionable conclusions, flagging conflicts with existing knowledge, and limiting length to 1500 characters.

Must capture the full original content and store it.

Must differentiate author viewpoints from the analyst’s commentary.

Must deliver actionable conclusions, not just a summary.

If content conflicts with known knowledge, it must be explicitly highlighted.

Length must stay within 1500 characters; longer outputs should be split.

12. Passing Once Doesn’t Mean Stable Use

Running a Skill successfully a single time is insufficient. The author now creates evaluation suites covering normal, boundary, and error scenarios for each core Skill. After any modification, the suite is executed to detect regressions, treating Skills like code that requires testing to avoid unexpected failures.

In summary, building AI Skills is less about writing them and more about designing robust, maintainable, and continuously validated workflows.