HiRoute Update: Per-Step Model Routing with Three-Question Decision Engine

01 Every Step Re-Judged: Complex Tasks Contain Simple Work

The common practice assigns models by scenario: strong model for coding, cheap model for documentation. But difficulty varies step by step — complex scenarios contain many simple steps, and simple scenarios can suddenly present hard problems.

A technical selection experiment compared two approaches: research six candidates, verify official docs, write a decision memo. Delegating research to a cheap model and key judgments to the flagship model matched the all-flagship quality at ~1/12 the cost. Using only the cheap model produced acceptable research but made major errors in 2 of 3 key judgments.

HiRoute's division: scenario defines candidate models and acceptance criteria; each step's model is decided at that step. Switching occurs only on new user message, context compression, or rebuild; within a round, tool calls stay on the same model, and only the first request after a switch misses cache.

See practical case: https://mp.weixin.qq.com/s?__biz=MzU0MzkyMTgzNg==∣=2247490561&idx=1&sn=ac0cc95f5702da73c3261d85729af6ef&scene=21#wechat_redirect and switching and cache cost analysis: https://mp.weixin.qq.com/s?__biz=MzU0MzkyMTgzNg==∣=2247490576&idx=1&sn=ec7f1a40703bba11d1a073c9b03e531e&scene=21#wechat_redirect.

A real session recording shows DeepSeek Harness (left) vs HiRoute (right). Routing plan sets DeepSeek V4.1 Flash as new model, GPT-6 Astra as flagship. User never manually switches models:

Write tests, discuss design trade-offs — not simple enough, flagship handles all 21 calls with ~90% cache hit.

Write usage docs — simple enough, delegated to new model.

User says it reads like a feature intro. Previous step scored 0.32; issue relates to next rewrite, so low-score protection hands off to flagship.

Rewritten version scores 0.83, problem solved; next step exporting data returns to new model.

02 New Models: Evaluate While Using in Your Own Work

Benchmarks and leaderboards help initial screening but cannot answer "will it work in my work?" Scale AI's SWE-Bench Pro (Sep 2024) shows top models solving 70%+ on SWE-bench Verified drop to ~23% on public Pro tasks, and further to 17.8% on private startup codebases. The gap between public and private tasks, built with the same methodology, highlights that your codebase is unseen private code for the model.

New models arrive too fast for dedicated eval sets per model. HiRoute's approach: evaluate while using. Assign sufficiently simple steps to the new model, review results each step, let flagship take over on failure, gradually increase delegation as evidence accumulates. This update completes that loop:

Start from Homepage : Homepage "New Models" shows recommended new models. Select new model and familiar flagship, map to your Agent; changes listed before publish. Agent model names unchanged, no Agent config changes needed.

Continuous Trust Adjustment : Trust level = difficulty threshold; further right delegates more simple work to new model. Beyond five presets (Trial, Familiar, Trust, Delegate, Entrust), any position can be set.

Flagship Takes Over Anytime : If new model's step score falls below competence floor and the issue still affects subsequent work, low-score protection hands off to flagship; if issue becomes irrelevant, revert to difficulty-based selection.

Most direct evidence of step quality is in the user's next message: explicit rejection = failure evidence; continuing without comment ≠ success. Scoring now recognizes explicit approval, distinguishes partial vs full pass, and credits partially usable progress.

After some usage, Model Performance Report aggregates phase scores, new model share, and cost estimates. Single scores have noise; trust adjustment requires sufficient evidence:

Suggest Increase : Since last adjustment, ≥30 complete scoring phases from ≥3 sessions, ≥60% coverage, ≥85% above competence line.

Suggest Decrease : In ≥30 complete scoring phases, <70% meet standard.

Increase threshold is intentionally higher than decrease: occasional successes don't justify more delegation; clear problems warrant timely pullback. Suggestions are advisory; publishing doesn't change routing until confirmed. In "Agent" → "HiRoute Management", daily Agents can read reports and adjustment suggestions; periodic review via Agent's own scheduled tasks.

03 System One: Three Questions Every Round

Naïve approach: give task, history, routing policy to decision model, ask "which model for this step?" Unstable — must simultaneously weigh task difficulty, past performance, issue resolution, policy compliance; any misweight breaks conclusion.

System One splits into three independent questions answered in one request:

Previous Step · Look Back : How did it go? Outputs competence score, feeds Model Performance Report.

Current Step · Right Now : How difficult? Outputs simplicity probability; only sufficiently simple work can go to economy model.

Next Step · Look Ahead : Will previous step's problem carry into next step? Judges whether a known, unresolved issue still applies — not predicting next step success.

Decision model only answers questions; thresholds, group selection, execution handled by HiRoute. Adjusting trust changes thresholds, not questions. From this update, third question becomes default: if previous step failed and problem persists, flagship takes over; if resolved or irrelevant, protection lifts.

Mechanism published as paper with Chinese/English datasets, raw model answers, offline reproduction tools. Experiment: two methods read identical task context and full routing policy — one directly picks model group, other answers three questions then HiRoute combines judgments.

Key finding: two decision models' direct-selection errors were opposite directions. In math scenarios, Jev 1.14 was over-bold (6/24 protection-needed correct), OpenAI Decisions over-conservative (2/16 release-needed correct). With three questions: Jev 1.14 got all 24 protection and all 16 release correct; OpenAI Decisions improved release to 14/16, protection 22/24. Opposite biases both significantly reduced by same decomposition.

Cost and latency acceptable: per Jev billing, ~$0.084–0.086 per 1k three-question decisions; median decision latency 0.5–1s (excluding connection setup), affordable per round.

Note: data measures "whether model group selected per declared policy", not task completion rate or actual savings. Audit subset uses post-output review; all models and methods share same exclusion list; gains vary by decision model and scenario; OpenAI results from research adapter in experiment, not representative of built-in OpenAI connection.

Full data, exclusion list, limitations in paper [2]: https://github.com/higress-group/HiRoute/blob/a7b19b4b7d519dac27c927658e60202dad9be29f/experiments/system-one-routing-paper/README.md

Decision model integration more flexible:

New OpenAI Decisions added; built-in connections also support Bailian, OpenRouter Jev, TypeSafe, and compatible endpoints.

Reuse Existing API Connections : OpenAI, OpenRouter, Bailian Token Plan and Workspace connections can share keys with regular models; TypeSafe and other compatible endpoints still allow separate URL and key.

Decision Context Allocated by Model Window : Known models use official specs, custom limits allowed. Current input and three questions always fully retained, then as many recent complete steps as fit.

04 Task Routing: Each Task to the Right Agent

Model routing decides "which model for this step"; task routing decides "which execution Agent for this job". Each routing plan defines suitable work types and can bind an execution Agent: Codex CLI, Claude Code, Qoder CLI, Pi, DeepSeek Harness, or OpenCode. During execution, that Agent's each round still follows plan's model routing between economy and flagship groups.

This update:

Hermes can connect its model requests to HiRoute, and enable task routing to delegate published plans.

OpenCode can act as daily Agent delegating tasks, and also serve as execution Agent.

Execution Agent availability checked against local installed capabilities; failure reasons preserved; CLI directly flags missing delegation config (e.g., plan's use-case description).

DeepSeek Harness team Oct 31 released "Super-agent Design Exploration: Toward More Autonomous Agent Systems" and open-sourced prototype dsh-superagent-prototype [3]: https://github.com/deepseek-ai/dsh-superagent-prototype. Two-layer architecture: resident Router handles external communication, global coordination, task dispatch (may not execute tasks); Workers execute single tasks in isolated contexts, return results to Router for unified reply. Router stays responsive, not blocked by long tasks. Motivation: human can monitor ~3–5 Agent sessions simultaneously; attention is bottleneck.

Similar architecture achievable with HiRoute's existing capabilities, no need to build separate Super Agent routing mechanism:

Personal Agent as Router : Hermes can enable task routing directly in HiRoute; OpenClaw etc., if they can run local CLI, can call same commands.

Delegate with hiroute worker exec : Pass task, working directory, acceptance criteria. Add --no-wait to return immediately on acceptance; Router resumes conversation; multiple tasks run in parallel.

Auto Report on Completion : Hermes example — after submit, background process waits via built-in notify_on_complete; on task end, process exits, Hermes wakes, retrieves result, reports in conversation.

Specify Execution Agent : Light research → plan bound to Pi; engineering code → plan bound to Qoder CLI; code review → plan bound to Codex CLI. Delegate by naming plan or let Router auto-select per plan's declared purpose.

# Delegate research to Pi-bound plan, return immediately
hiroute worker exec --plan <PLAN_ID> --cwd /work/mq-research --no-wait \
  -- "Collect official docs and release notes for three message queues, organize into comparison table"
# Check progress anytime; retrieve result after completion
hiroute worker status --run <RUN_ID>
hiroute worker result --run <RUN_ID>

Router only understands intent, dispatches tasks, aggregates results; each execution Agent's model cost decided per-round by its plan. Full Hermes + HiRoute Super Agent build process in "Build a DeepSeek-style Super Agent with Hermes and HiRoute" [4]: https://hiroute.ai/news/hermes-super-agent/

05 macOS, Windows, Linux All Have GUI

macOS : Desktop app, supports macOS 15+, Apple Silicon and Intel.

Windows : Desktop first release. x64 installer per-user, bundles CLI, background service, WebView2 bootstrapper; no Python or admin rights required. Upgrades wait for in-flight calls and Worker tasks to finish.

Linux : x86_64 and ARM64, installs to user directory, no sudo. Browser opens same UI as desktop. Headless servers supported; hiroute console open generates login link.

curl -fsSL https://hiroute.ai/install.sh | sh -s -- --console-mode loopback --console-port 5838
hiroute service start --output json
hiroute console open

06 More Improvements

Subscription and API Together

When adding Codex or Claude subscription, can also provide corresponding OpenAI or Anthropic API Key. One model selection, one save, both sides' models imported simultaneously.

Subscription-included API credits usable: eligible ChatGPT Pro/Business, Claude Max/Ultra/Team receive API credits each billing cycle, unused credits don't roll over (see OpenAI [5]: https://developers.openai.com/api/docs/guides/chatgpt-api-credits and Anthropic [6]: https://support.claude.com/en/articles/17154008-monthly-api-credits-for-max-ultra-and-team-plans official docs).

Provide the key from the org that claimed credits, then in model details create route for subscription model; HiRoute discovers same-named model in API, both added as candidates. Subscription and API same-named models assemble into one model: Agent uses single model name; HiRoute uses subscription first, falls back to API when subscription temporarily unavailable.

Set Spending Limit per API Key

Can set one-time quota, calendar-period replenishment, or multiple keys sharing one quota. HiRoute reserves before forwarding, settles after completion. Amount estimated from HiRoute-forwarded traffic, not provider's account balance. Suitable for managing credits: set monthly-replenishing quota equal to credit amount for that key; when exhausted, key skipped.

Proxy per Connection

Each API connection can choose direct, inherit background service's env proxy, or manually specify HTTP(S) proxy and bypass rules; decision models sharing that connection follow same setting.

Local Storage More Space-Efficient

Long sessions no longer repeatedly save full request snapshots and protocol logs; only retain content needed for history view, scoring, and model attribution. Local storage defaults to max 20 GB, adjustable or disableable; 7-day retention still applies.

Model Metadata Updated Separately

Official model specs (context windows, etc.) and homepage new model recommendations published via HiRoute's model catalog independently, no app upgrade required.

Local-First, Open Source

HiRoute runs on your computer or server, Apache-2.0 license.

07 Getting Started

Add Models : Add supported subscription or API, pick economy and flagship models.

Configure Division : In "Models → Decision Model" connect decision service, choose trust level, publish and map to Agent.

Work as Usual : Continue in original Agent; check HiRoute for model switches and Model Performance Reports. For task delegation, see Intelligent Task Routing [7]: https://hiroute.ai/docs/task-routing/ to choose execution Agent per plan.

HiRoute isn't perfect; decision models can misjudge. Welcome to try in your scenarios and tell us where it judged wrong.

Related links:

[1] HiRoute : https://github.com/higress-group/HiRoute

[2] Paper : https://github.com/higress-group/HiRoute/blob/a7b19b4b7d519dac27c927658e60202dad9be29f/experiments/system-one-routing-paper/README.md

[3] dsh-superagent-prototype : https://github.com/deepseek-ai/dsh-superagent-prototype

[4] Build a DeepSeek-style Super Agent with Hermes and HiRoute : https://hiroute.ai/news/hermes-super-agent/

[5] OpenAI : https://developers.openai.com/api/docs/guides/chatgpt-api-credits

[6] Anthropic : https://support.claude.com/en/articles/17154008-monthly-api-credits-for-max-ultra-and-team-plans

[7] Intelligent Task Routing : https://hiroute.ai/docs/task-routing/

[8] Download HiRoute : https://hiroute.ai/download/

[9] Documentation : https://hiroute.ai/docs/

[10] GitHub : https://github.com/higress-group/HiRoute