5 Agent Orchestration Patterns to Elevate Spring AI from Single‑Turn Q&A to Multi‑Step Decision Making

Clarifying a common misconception

Spring AI does not provide a built‑in @Agent annotation or an Agent class. The official "Building Effective Agents" guide extracts five orchestration patterns and supplies reference implementations in the spring-ai-examples repository (e.g., ChainWorkflow, RoutingWorkflow). These patterns are assembled from existing components such as ChatClient, @Tool, and standard Java control flow.

Chain workflow – linear pipeline

The simplest pattern splits a complex task into a series of steps, each driven by a dedicated system prompt. The output of one step becomes the input of the next.

public String chain(String userInput) {
    String response = userInput;
    for (String prompt : systemPrompts) {
        // process each step sequentially
        String input = prompt + "

上一步结果：
" + response;
        response = chatClient.prompt().user(input).call().content();
    }
    return response;
}

Customer‑service example: extract the real request, look up the knowledge base, then polish the answer—trading a little latency for noticeably higher accuracy.

Routing workflow – triage then specialize

When a request may belong to different domains (billing, technical, general), a single monolithic prompt performs poorly. Routing first classifies the request, then forwards it to a domain‑specific “expert” prompt.

Map<String, String> routes = Map.of(
    "billing", "你是账单专家，专处理扣费、退款问题……",
    "technical", "你是技术支持，专解决产品报错、使用故障……",
    "general", "你是通用客服，处理其他咨询……"
);
String type = chatClient.prompt()
    .user("判断下面问题属于 billing/technical/general，只回类型词：
" + input)
    .call().content().trim();
String systemPrompt = routes.getOrDefault(type, routes.get("general"));
return chatClient.prompt().system(systemPrompt).user(input).call().content();

This pattern is highlighted as the most suitable for typical customer‑service routing tasks.

Parallelization workflow – independent tasks in parallel

If subtasks do not depend on each other, they can be executed concurrently and aggregated afterward.

List<String> results = new ParallelizationWorkflow(chatClient).parallel(
        "给这条工单分类，输出：投诉 / 咨询 / 退货",
        List.of(工单1, 工单2, 工单3, 工单4),
        4 // concurrency level
);

Use cases include batch classification of hundreds of tickets or simultaneous quality checks (compliance, tone, accuracy) on a single response.

Orchestrator‑Workers workflow – dynamic task decomposition

Unlike static parallelization, this pattern lets the model decide how many sub‑tasks to create. An orchestrator analyses the request, splits it into dynamic tasks, and dispatches workers to run them in parallel before merging the results.

用户：「我要退货，但订单显示已签收，钱也想原路退」
    │
    └────▼──── 编排者拆解 ────┐
        子任务1：核对签收状态
        子任务2：查退货政策   ← 工人并行处理
        子任务3：发起原路退款工单
    └────────┬──────────────┘
            ▼ 合并成最终答复

Ideal for open‑ended tasks where the number of steps cannot be predetermined; the trade‑off is higher token usage and longer execution chains.

Evaluator‑Optimizer workflow – generate‑evaluate‑refine loop

When a clear quality metric exists, the AI can self‑evaluate and regenerate until the metric is satisfied.

String result = new EvaluatorOptimizerWorkflow(chatClient)
        .loop("起草一条退货拒绝话术，要求：礼貌、给替代方案、不超 3 句", 3);
// Internally: generate → evaluate → (if not good) generate again, up to 3 rounds.

In customer‑service this automatically polishes sensitive scripts (e.g., denial or complaint replies) to meet compliance and tone standards, reducing manual review.

Choosing the right pattern

Fixed steps with dependencies → Chain

Input needs categorisation → Routing

Batch of independent homogeneous tasks → Parallelization

Dynamic sub‑task creation → Orchestrator‑Workers

Clear quality criteria, iterative refinement → Evaluator‑Optimizer

Official guidance: if a single‑turn + @Tool solves the problem, avoid adding an Agent layer. Multi‑step orchestration trades higher accuracy for added latency and cost, so start simple and only increase complexity when necessary.

Technical references

Building Effective Agents: https://docs.spring.io/spring-ai/reference/api/effective-agents.html
spring-ai-examples: https://github.com/spring-projects/spring-ai-examples
ChatClient API: https://docs.spring.io/spring-ai/reference/api/chatclient.html