Practical Multi‑Model Routing with Embabel: Mixing DeepSeek and Claude

Why One Model Is Not Enough

In a typical enterprise AI workflow—intent parsing, knowledge retrieval, deep analysis, and report generation—each step has vastly different requirements for capability, cost, and latency, forming an "impossible triangle" where no single model can excel at all three.

Embabel Multi‑Model Capabilities Overview

Embabel defines three core concepts: @Action (typed operations with Java records), @AchievesGoal (the planner’s target), and a classic GOAP planner that re‑evaluates after each step.

Default model – withDefaultLlm(): global fallback for ordinary steps.

Specified model – withLlm(LlmOptions.withModel(...)): hard‑wired at the action level.

Role alias – withLlmByRole("reviewer"): configuration‑driven routing (the main focus of this article).

Fallback chain – LlmOptions.withFirstAvailableLlmOf(...): automatic downgrade when the primary model fails.

Routing Architecture Design

Overall Layering

用户请求
   ↓
┌─────────────────────────────────────┐
│ L1 任务分解层：Embabel 规划器          │  把目标拆成动作序列
├─────────────────────────────────────┤
│ L2 角色分配层：动作 → 角色映射          │  每个动作声明角色别名
│   （cheap / best / writer…）           │
├─────────────────────────────────────┤
│ L3 模型解析层：角色 → 具体模型          │  application.yml 集中配置
│   （cheap→DeepSeek，best→Claude）      │
├─────────────────────────────────────┤
│ L4 兜底降级层：withFirstAvailableLlmOf │  主模型故障自动切换
└─────────────────────────────────────┘

The design isolates concerns: business code only chooses a role (e.g., "cheap"), while cost‑optimisation teams change the underlying model in application.yml without touching code.

Routing Strategy: Four‑Dimensional Decision Matrix

For each action the planner asks four questions:

Task type : extraction/formatting vs. reasoning/creation → cheap for the former, best for the latter.

Complexity : highly structured output vs. open‑ended generation → cheap for structured, best for open.

Cost budget : frequency × token count → high‑frequency steps forced to cheap.

Latency requirement : real‑time vs. async tolerant → avoid best for strict real‑time.

Applying the matrix to the research‑brief generator yields the following role‑to‑model mapping:

understandRequest (structured extraction) → role cheap → DeepSeek

research (high‑frequency retrieval) → role cheap → DeepSeek

deepAnalysis (cross‑document reasoning) → role best → Claude

writeBrief (final report) → role best → Claude

One‑sentence principle: give Claude the high‑intelligence steps and DeepSeek the high‑throughput steps.

Implementation Details

Environment and Dependencies

<dependencies>
  <!-- OpenAI‑compatible (DeepSeek) -->
  <dependency>
    <groupId>com.embabel.agent</groupId>
    <artifactId>embabel-agent-starter-openai</artifactId>
    <version>1.0.0</version>
  </dependency>
  <!-- Anthropic (Claude) -->
  <dependency>
    <groupId>com.embabel.agent</groupId>
    <artifactId>embabel-agent-starter-anthropic</artifactId>
    <version>1.0.0</version>
  </dependency>
  <dependency>
    <groupId>com.embabel.agent</groupId>
    <artifactId>embabel-agent-starter-shell</artifactId>
    <version>1.0.0</version>
  </dependency>
</dependencies>

Environment variables:

export DEEPSEEK_API_KEY=sk-xxxx
export ANTHROPIC_API_KEY=sk-ant-xxxx

Role Routing Configuration (Core)

spring:
  ai:
    openai:
      base-url: https://api.deepseek.com/v1 # DeepSeek uses OpenAI‑compatible API
      api-key: ${DEEPSEEK_API_KEY}
      chat:
        options:
          model: deepseek-chat
    anthropic:
      api-key: ${ANTHROPIC_API_KEY}

    embabel:
      models:
        default-llm: deepseek-chat # default cheap model
        llms:
          cheap: deepseek-chat   # high‑throughput role
          best: claude-sonnet-4-5 # high‑intelligence role

Note: YAML indentation is critical; a misplaced level prevents the Agent from locating the role configuration.

Action‑Level Routing Code

@Agent(description = "Multi‑model routing research brief generator")
public class RoutedResearchAgent {

  // ===== Cheap step: intent parsing =====
  @Action
  public ResearchRequest understandRequest(UserInput input, AI ai) {
    return ai.withLlmByRole("cheap")
             .withId("parse-request")
             .creating(ResearchRequest.class)
             .fromPrompt("Extract research topic and audience: %s", input.getContent());
  }

  // ===== Cheap step: web research =====
  @Action
  public ResearchFindings research(ResearchRequest req, AI ai) {
    return ai.withLlmByRole("cheap")
             .withId("web-research")
             .withToolGroup(CoreToolGroups.WEB)
             .creating(ResearchFindings.class)
             .fromPrompt("Topic: %s, Audience: %s. Return verified facts and sources.",
                         req.topic(), req.audience());
  }

  // ===== Best step: deep analysis =====
  @Action
  public DeepAnalysis analyze(ResearchFindings findings, AI ai) {
    return ai.withLlmByRole("best")
             .withId("deep-analysis")
             .creating(DeepAnalysis.class)
             .fromPrompt("Based on the facts, infer three core trends, point out the weakest evidence, and give architects actionable advice. Facts: %s", findings.facts());
  }

  // ===== Best step: write brief with fallback =====
  @AchievesGoal(description = "Produce a technically sound brief")
  @Action
  public ResearchBrief writeBrief(ResearchRequest req, DeepAnalysis analysis, AI ai) {
    return ai.withLlm(LlmOptions.withFirstAvailableLlmOf("claude-sonnet-4-5", "deepseek-chat"))
             .withId("brief-writer")
             .creating(ResearchBrief.class)
             .fromPrompt("Write a technical brief on topic: %s, using analysis: %s. Do not fabricate information.",
                         req.topic(), analysis.insights());
  }
}

Runtime Verification

mvn spring-boot:run

After entering a research topic, the log shows the call chain:

parse-request(deepseek) → web-research(deepseek) → deep-analysis(claude) → brief-writer(claude)

Four IDs map to two models, confirming that mixed routing is active.

Effect Evaluation: Three‑Dimensional Comparison

Methodology: cost estimated from public pricing (≈8K input tokens, 2K output tokens per task), quality scored blind by three reviewers on a 5‑point scale, latency measured as median of multiple runs. Figures illustrate magnitude relationships rather than precise benchmarks.

Cost per run : Claude ¥1.0, DeepSeek ¥0.02, Mixed ¥0.3 (‑70%).

End‑to‑end latency : Claude 45 s, DeepSeek 22 s, Mixed 30 s.

Parsing / retrieval quality : Claude 4.6, DeepSeek 4.5, Mixed 4.5 (DeepSeek sufficient).

Analysis / final‑report quality : Claude 4.7, DeepSeek 3.9, Mixed 4.6 (Claude fallback).

Overall score : Claude 4.7, DeepSeek 4.1, Mixed 4.6.

Conclusions :

Mixed routing achieves ~95 % of the quality of an all‑Claude setup while spending only ~30 % of the cost.

Cheap steps (intent parsing, retrieval) show negligible quality loss, confirming that using the flagship model for them is wasteful.

Latency improves over the all‑Claude configuration because the high‑throughput DeepSeek steps offset Claude’s slower phases.

Lessons Learned

Four Pitfalls to Anticipate

YAML indentation : a wrong level makes the Agent fail to resolve model aliases.

Routing‑benefit inversion : if most requests truly need the strong model, the savings may not outweigh the added complexity; always profile task distribution first.

Over‑granular role granularity : one role per action inflates configuration; keep roles to 3‑5 logical tiers (cheap, best, writer, coder, etc.).

Ignoring fallback chains : with multiple vendors, any outage can break the whole pipeline; always attach withFirstAvailableLlmOf to critical actions.

Three Best Practices

Roles as intelligence tiers : treat a role as an abstract "intelligence level"; business code references the tier, while model choice lives in configuration.

Tag every call : use withId() from day one to enable cost attribution and quality tracing.

Pre‑deployment quality gate : run a blind‑evaluation on a fixed test set comparing all‑strong vs. mixed routing before scaling up.

Applicable Scenario Boundaries

Suitable for mixed routing : multi‑step pipelines with divergent quality/cost requirements, high call volume, and clear strong vs. weak steps (e.g., intelligent客服, report generation, code review).

Not suitable for mixed routing : single request‑response calls where routing overhead outweighs benefits, ultra‑low‑latency paths, or domains where every step must meet the highest quality standard (e.g., medical diagnosis assistance).

Interaction : What proportion of "cheap" versus "best" traffic does your system have? Share the numbers and I’ll estimate the potential savings.

Stay tuned for the next Embabel post: exporting an Agent as an MCP service so that your Agent can be invoked by AI systems worldwide.