Why AI Coding Agent Bills Soar and 5 Token‑Saving Techniques to Cut Costs

1. Where the bill comes from

Most users first try to shorten their prompts, but the real cost driver is the large amount of system‑provided context that the model must ingest each request.

A typical request contains:

System Prompt          5K
Project description    10K
Skill definition       20K
Tool / MCP definition   30K
Conversation history   100K
Code files             50K
User question          0.1K

The user question is usually less than 1 % of the total tokens; the expensive part is the fixed prefix, semi‑fixed context, and dynamic context that the system automatically adds.

Expensive part = system‑added context, not the user’s sentence.

The total cost can be approximated as:

Total cost ≈ Fixed prefix + Conversation history + Runtime retrieval + Tool round‑trip + Model output

Fixed prefix : System Prompt, Skill definition, Tool/MCP definition, stable background documents.

Semi‑fixed context : Project description, Repo map, memory, long‑term constraints.

Dynamic context : Chat history, code snippets, retrieval results, tool returns, the new user question.

The user’s question is merely the “trigger” and not the cost mainstay.

1.1 The question is cheap

Optimizing only the prompt length misses the main cost source because the model receives the whole context bundle.

1.2 The “it remembers” illusion

AI coding agents are stateless; the apparent memory comes from the agent layer re‑injecting history, rules, and tool definitions each round, making longer sessions increasingly expensive.

1.3 Five cost categories

Beyond prompt length, costs fall into five groups:

Fixed prefix

Semi‑fixed context

Dynamic context

Tool round‑trip processing

Retry overhead (failed first attempts)

Tool round‑trip cost includes parsing tool definitions, generating parameters, receiving results, and re‑injecting them.

1.4 Prompt cache as the foundation

Prompt cache stores the processing result of the stable prefix so that repeated requests with the same prefix avoid re‑processing, saving repeated‑cost tokens.

Fixed prefix → Cache hit → No need to process from scratch

Cacheable items include System Prompt, Tool/MCP definition, Skill definition, long background documents, and stable few‑shot prefixes.

1.5 Five‑layer model

The optimization framework is organized into five layers, from cheapest to most complex:

Usage habits – eliminate meaningless history and waste tokens.

Model routing – ensure cheap models handle cheap work.

Context engineering – avoid sending the same prefix repeatedly.

Code graph – prevent re‑searching code each round.

Agent architecture – avoid funneling all tasks into a single massive context.

All layers share the goal of reducing irrelevant context, letting cheap models do cheap work, and reserving expensive models for high‑value reasoning.

2. Usage habits – the cheapest yet most overlooked optimizations

2.1 One session, one goal

Never treat an AI agent as a never‑ending chat; start a fresh session for each distinct task (bug fix, refactor, documentation, etc.) to avoid accumulating costly history.

2.2 Compress long sessions

Use the /compact command to collapse history into a reusable state, preventing the session from becoming a liability.

2.3 Treat chat history as a database

Store long‑term artifacts (project docs, summary files, memory files, repo map, task lists) outside the conversation and only load what is needed for the current step.

2.4 Cut output waste

Ask the model to return only the essential result (e.g., diff, steps, JSON) to avoid verbose explanations that consume output tokens and increase retry chances.

2.5 Skill cost awareness

Each Skill carries description, instructions, and examples that become part of the context when resident. Keep high‑frequency, stable Skills resident and load low‑frequency, heavy Skills on demand.

2.6 MCP overload

Adding many MCPs (GitHub, Notion, Jira, etc.) inflates the per‑round payload and decision space, leading to higher token usage and slower selection.

2.7 Prefer CLI over MCP when possible

When a mature CLI exists (e.g., gh pr create, kubectl get pods, docker ps), use it directly instead of wrapping it in an MCP, saving the definition payload.

2.8 Use full file paths

Reference files with absolute or relative paths prefixed by @ so the agent can read them directly without a search round, eliminating an entire token‑heavy lookup chain.

2.9 State intent in a single message

Provide a complete, concrete instruction in one turn rather than a fragmented dialogue; this prevents multiple rounds of context re‑injection and token waste.

3. Model routing – don’t let expensive models do cheap work

3.1 Match task complexity, not price

Complex tasks → strong model

Simple tasks → cheap model

Repetitive tasks → stable model

Use a cascade: cheap model first, then upgrade only if needed.

3.2 Upgrade chain example

1. Run cheap model
2. ↓
3. Assess complexity
4. ↓
5. Upgrade to stronger model only if required

3.3 Budget knobs

Many models expose knobs such as reasoning effort, thinking budget, verbosity, and max output tokens to control token consumption directly.

3.4 Bind models to Skills/Commands

Declare the model to use in a Skill’s metadata (e.g., model: deepseek-v4-pro) so that cheap Skills automatically run on cheap models without manual switching.

4. Context compression – shrink the inevitable payload

4.1 RTK – terminal output compression

RTK filters, aggregates, truncates, and deduplicates command output before it reaches the model, achieving 60‑99 % compression for typical Git and test commands.

# macOS (recommended)
brew install rtk

# Linux / WSL
curl -fsSL https://raw.githubusercontent.com/rtk-ai/rtk/master/install.sh | sh

Common commands after installation:

rtk init -g --agent codebuddy   # enable globally for CodeBuddy
rtk gain                         # view saved statistics
rtk git diff                     # compressed Git diff (60‑80% reduction)
rtk test cargo test               # compressed test output (90‑99.6% reduction)

4.2 Caveman – output token compression

Caveman reduces the model’s reply tokens with four modes (lite, full, ultra, wenyan). Example:

Question: Why does a React component keep re‑rendering?

Full output (69 tokens): "The reason your React component is re‑rendering is likely because you're creating a new object reference on each render cycle..."
Caveman output (19 tokens): "New object ref each render. Inline object prop = new ref = re‑render. Wrap in useMemo."

Installation:

git clone https://github.com/studyzy/caveman && cd caveman && ./install.sh

5. Putting it all together

By first eliminating unnecessary context (usage habits), then routing tasks to appropriately sized models, and finally applying context‑compression tools (RTK for input, Caveman for output), developers can dramatically lower token bills for AI coding agents.

The core mental model: Cost = amount of context the system must carry for each request . Reduce that context at every layer, and the bill will shrink accordingly.