Why Giving Agents the Strongest Model by Default Is Wasteful: Databricks Introduces Joint Model‑and‑Harness Selection

Databricks moves the routing of Coding Agents from a simple "pick model" approach to a task‑aware "pick Model + Harness" strategy, focusing cost optimization on the total expense of completing a task rather than on token price alone.

1. Model cheap ≠ task cheap

Traditional LLM pricing compares input and output token costs per million, but a Coding Agent also reads code, searches repositories, calls tools, and iterates. The actual cost equals "price per token × actual consumption," where consumption depends on model inference efficiency, harness context management, and the number of reasoning rounds.

Databricks' internal benchmark shows Sonnet 5 is 1.7× cheaper per token than Opus 4.8, yet its average cost per task is $2.09 versus $1.94 for Opus 4.8, with lower completion rates (81 % vs 87 %). The higher cost stems from longer runtimes and roughly 1.9× token usage.

Thus, the goal is not the cheapest token or the strongest model, but the lowest total cost to meet quality requirements.

2. Why Smart Routing routes by task

Smart Routing does not switch models on every request. Instead, at task start it evaluates complexity, modification scope, code evidence, fault type, and locality, then selects a model and harness that remain fixed for the task. Frequent model changes would break prompt‑cache hits and raise costs.

The router first uses a cheap, low‑latency model to read the task description and metadata, infers task type, and then defaults to a mid‑capability model: simple tasks stay on cheap models, complex tasks upgrade to stronger ones.

Benchmark results: internal tests save 35 % cost; a public coding benchmark saves 56 % while achieving performance comparable to Claude Opus 5.

3. Harness matters as much as the model

The harness determines how much context is sent each round, when tools are invoked, and when context is compressed. Swapping only the harness while keeping the same model can change per‑task cost by more than 2× in some configurations.

Pi Harness reduces the context sent to the model by about threefold and completes tasks in fewer rounds, cutting token consumption dramatically.

Omnigent acts as a meta‑harness that unifies different harnesses, models, and agents, allowing developers to switch harnesses at runtime without code changes.

4. Agent cost is now measured per completed task

In agent workflows, the initial prompt is only a small fraction of the final context; code, search results, tool outputs, skills, system info, and history dominate token usage. Adjusting harness and cache settings can halve generated tokens and cost without degrading quality.

Therefore, evaluating "cost per million tokens" is insufficient. The meaningful metric is the total expense to finish a task, its success rate, and whether a more expensive model is required.

Smart Routing’s evolution:

Before: Prompt → single model

Later: Prompt → router → multiple models

Now: Task → Smart Routing → Model + Harness; sub‑agents may be routed again.

Current limitations: the router decides based on the task’s initial information, which may be incomplete. Databricks is exploring delayed routing after several rounds and model switches during context compaction.

Overall, Smart Routing turns agent cost optimization into a task‑level decision process: assess the task, then choose the appropriate model, harness, and when to upgrade to more expensive capabilities, aiming to minimize the cost of completing the task rather than buying the cheapest token.