Why Prompt Caching Is Everything for Claude Code

If you have used Claude Code, the seamless code‑collaboration experience relies on a single concept: prompt caching.

Queue prompts for stability

Prompt caching works like a strict prefix‑matching game; the API reuses computation when the request start matches exactly. Claude Code places the most stable content at the front, arranging the request as a layered structure:

Global static system prompts and tool definitions (highest cache hit rate).

Project‑specific rules from CLAUDE.md (reusable within the same project).

Current session context (unchanged during the round).

Dynamic conversation messages (added last).

This static‑then‑dynamic layout maximizes cross‑session cache sharing, but even tiny changes—such as inserting a timestamp, reordering tools, or tweaking a tool’s parameters—break the prefix and cause a cache miss.

To update information without invalidating the cache, Claude Code injects a <system‑reminder> tag into the next user or tool message, delivering fresh data while keeping the prompt prefix intact.

Do not change model or tools

Switching models mid‑conversation seems cost‑effective, but because the cache is bound to a specific model, changing from Opus to Haiku discards the existing cache and forces a full resend of the context, increasing cost.

The recommended solution is to keep the main model and delegate cheaper tasks to a sub‑agent. Claude Code’s Explore agents use Haiku for low‑cost exploration while the primary Opus model remains unchanged.

Similarly, adding or removing tools alters the cache prefix. Instead of removing a tool, Claude Code marks tools for deferred loading using a lightweight stub with defer_loading: true. The full tool definition loads only when the model selects it, preserving cache integrity.

Plan mode as a tool

Plan mode is implemented as a tool rather than a new set of tools. When entering Plan mode, the model receives a system message instructing it to explore the codebase without editing files and to call an ExitPlanMode tool when done. Because the tool set does not change, the cache remains safe.

This design also lets the model autonomously decide to enter planning when needed, without any cache interruption.

Compress context safely

When the context window fills up, Claude Code performs a cache‑safe fork for compaction. It creates a new request that copies the entire parent conversation, appends a compression command as the final user message, and sends it with the identical system prompt, tool set, and prefix. Only the added compression token is new, allowing the massive cached prefix to be fully reused.

The mechanism requires a reserved “compression buffer” to ensure enough window space for the command and generated summary. Anthropic has since baked this compression feature directly into their API.

Cache‑first best practices

The team’s practical rules include:

Never modify the static prompt prefix; use regular messages for updates (e.g., date changes, Plan mode switches).

Avoid mid‑conversation model or tool swaps; simulate state changes with tools and deferred loading.

Monitor cache‑hit rates like online‑rate metrics and treat misses as incidents.

All forked operations must share the parent’s prefix and parameters to stay cache‑safe.

Claude Code was designed from day one around prompt caching, and the article suggests that anyone building a new AI agent should start with the same principle.