Why Are AI Agent Bills Soaring? Token‑Saving Techniques to Cut Costs

Cost Structure of AI Coding Agents

A typical request to agents such as CodeBuddy, Cursor, Codex or Gemini CLI contains a large amount of system‑injected context. The token breakdown often looks like:

System Prompt          5K
Project description   10K
Skill definitions      20K
Tool / MCP definitions 30K
Session history       100K
Code files             50K
User question          0.1K

The user question is usually less than 1 % of the total tokens. The total cost can be approximated as:

Total Cost ≈ Fixed Prefix + Session History + Runtime Retrieval + Tool Round‑trip + Model Output

Fixed Prefix : System prompt, skill definitions, tool/MCP definitions, stable background documents.

Semi‑fixed Context : Project description, repo map, memory, long‑term constraints.

Dynamic Context : Chat history, code snippets, retrieval results, tool returns, the new user question.

The heavy‑lifting cost is the background that the model must process each round, not the short user query.

Five Cost Categories (Beyond Token Count)

Fixed Prefix

Semi‑fixed Context

Dynamic Context

Tool Round‑trip (definition → parameter generation → result → re‑processing)

Retry Cost (failed first attempts that trigger a full re‑send of context)

Prompt Caching

Prompt caching stores the processed result of the stable prefix so that repeated requests with the same prefix avoid re‑processing.

Fixed Prefix → Cache Hit → No Re‑processing

Cacheable items include the system prompt, tool/MCP definitions, skill definitions, long background documents and stable few‑shot examples. Keeping the prefix stable maximises cache hits and reduces repeated token spend.

Five‑Layer Optimization Model (Cheapest → Most Expensive)

Layer 1 – Usage habits (eliminate meaningless history and waste tokens).

Layer 2 – Model routing (ensure cheap models handle cheap work).

Layer 3 – Context engineering (avoid sending the same prefix repeatedly).

Layer 4 – Code graph (prevent re‑searching code each round).

Layer 5 – Agent architecture (avoid funneling all tasks through a single massive context).

Layer 1 – Usage Habits

One Session = One Goal

Open a separate session for each distinct objective (bug fix, refactor, documentation, etc.). A long‑running session that mixes unrelated tasks accumulates history and makes every subsequent round more expensive.

Compress Long Sessions

Use the /compact command to collapse the full history into a reusable state:

1 把完整历史
2 压成可继续工作的状态

Treat Chat History as a Database

Persist long‑term information (project docs, summaries, memory files, repo maps, task lists) outside the agent. The session should only carry the current work state.

Cut Redundant Output

Ask the agent to return only the essential artifact (diff, steps, JSON, table) instead of verbose explanations. Example command:

直接给结论，不要复述问题，必要时再展开

Skill Cost Management

Each skill adds description, instructions, examples and trigger logic to the context. Keep high‑frequency, stable skills resident; load low‑frequency, heavy skills on demand.

Limit MCP Over‑installation

Every additional MCP adds a tool definition and selection overhead. Over‑installing integrations (GitHub, Notion, Jira, Slack, Kubernetes, Docker, etc.) inflates context size, slows decision making and raises error probability.

Prefer CLI Over MCP When Possible

When a mature CLI exists, invoke it directly instead of wrapping it in an MCP. Example hierarchy:

1 能 CLI 解决 → 尽量 CLI
2 能脚本解决 → 尽量脚本
3 最后才考虑 MCP

Typical CLI commands: gh pr create, kubectl get pods, docker ps, git diff.

Use Full Paths for Files

Providing an absolute or @‑prefixed path lets the agent locate the file instantly, avoiding costly search loops.

1 看一下 src/config/config.go 的问题
2 帮我修改 @/Users/yourname/project/utils.py

State Intent in One Shot

Give the complete intent in a single request to avoid repeated context reconstruction:

1 看 @src/order/service.go 的 CreateOrder 函数，
2 找出潜在的 bug，修复，并为修复后的函数写单测。

Layer 2 – Model Routing

Match Task Complexity

Complex tasks → strong model

Simple tasks → cheap model

Repetitive tasks → stable model

Upgrade Only When Needed

1 便宜模型先跑
2 ↓
3 判断复杂度
4 ↓
5 需要才升级

Example: a cheap model performs initial classification, then a strong model refines the result.

Adjust Budget Knobs

Modern models expose knobs such as reasoning effort , thinking budget , verbosity and max output tokens . Tuning these reduces token consumption for tasks that do not require deep reasoning.

Bind Models to Skills/Commands

Specify the model in a skill file or slash command so that cheap models are automatically selected for routine operations:

---
context: fork
model: deepseek-v4-pro
---

Layer 3 – Context Compression Tools

RTK – Reduce Terminal Output Tokens

RTK filters command output (removing comments, ANSI codes, progress bars) and aggregates similar results, achieving 60‑99 % compression.

# macOS (recommended)
brew install rtk

# Linux / WSL
curl -fsSL https://raw.githubusercontent.com/rtk-ai/rtk/master/install.sh | sh

Enable globally for CodeBuddy:

rtk init -g --agent codebuddy
rtk init -g --hook-only

Common commands:

rtk gain               # view savings
rtk git status
rtk test cargo test
rtk vitest run

Caveman – Compress Model Output Tokens

Caveman provides four modes (lite, full, ultra, wenyan) to trim the model’s reply while preserving essential information.

/caveman lite   # keep articles, remove filler
/caveman        # default, drop articles, fragment sentences
/caveman ultra  # extreme compression using A→B→C chains
/caveman wenyan # classical Chinese style (high token efficiency)

Example (full mode):

普通输出 (69 tokens): "The reason your React component is re‑rendering is likely because you're creating a new object reference on each render cycle..."
Caveman 输出 (19 tokens): "New object ref each render. Inline object prop = new ref = re‑render. Wrap in useMemo."

Installation (CodeBuddy integration via the studyzy fork):

git clone https://github.com/studyzy/caveman cd caveman && ./install.sh  # select [2] CodeBuddy Code

Activate inside CodeBuddy with /caveman; the mode persists for the session and can be stopped with stop caveman.

Conclusion

Understanding that the bulk of token spend comes from system‑added context enables a layered optimisation approach: start with disciplined usage habits, leverage prompt caching, route tasks to appropriate models, and finally compress both input (RTK) and output (Caveman) tokens. Applying these steps can dramatically lower the cost of AI coding agents while preserving functionality.