Java AI Framework Selection: Infrastructure vs. Orchestration Layers Compared

Preface: The Layer Confusion Trap

The article opens with a real-world scenario: a team building an intelligent customer service system debates between Spring AI ("Spring official, stable"), LangChain4j ("richest ecosystem"), and AgentScope ("Alibaba production-verified"). All three arguments are valid yet address different problems because the frameworks operate at different architectural layers. The core pitfall is comparing feature lists instead of identifying which layer your scenario requires.

Layer Classification: Rod Johnson's Analogy

Rod Johnson (Spring founder) introduced a sharp analogy when launching Embabel:

Infrastructure Layer (equivalent to Servlet API): Spring AI, LangChain4j, Solon AI — solve "how to connect LLMs, manage embeddings, integrate vector databases, handle tool calling."

Advanced Application Framework Layer (equivalent to Spring MVC): Embabel — solves "how to make agents reliably, explainably, and deterministically complete multi-step tasks in complex business processes."

Higher Enterprise Layers : AgentScope (enterprise distributed agent runtime), Spring AI Alibaba (enterprise multi-agent/workflow in Spring), Solon AI (cross-version/framework AI embedding).

These layers are complementary, not competitive. The first selection step is not "which framework" but "which layer do I need?"

Visual Overview: Two-Layer Architecture

The article provides a flowchart (Mermaid syntax) illustrating the two-layer structure:

flowchart TB
  subgraph L1[Infrastructure Layer<br>How to integrate AI]
    A1[Spring AI<br>Spring ecosystem AI abstraction]
    A2[LangChain4j<br>JVM AI toolbox]
    A3[Solon AI<br>Full-scenario AI framework]
  end
  subgraph L2[Agent Orchestration Layer<br>How to organize AI work]
    B1[Spring AI Alibaba<br>Enterprise multi-agent & workflow]
    B2[AgentScope<br>Enterprise distributed Agent]
    B3[Embabel<br>Deterministic GOAP orchestration]
  end
  L1 --> L2

Layer 1: Infrastructure — How to Integrate AI

3.1 Spring AI 2.0: Spring's Official Answer

Released GA on June 12, 2026, built on Spring Boot 4.1 and Spring Framework 7.0. Key changes from 1.x:

Clear ChatClient/ChatModel boundary : ChatClient is the high-level user-facing API with built-in Advisor chain; ChatModel is the low-level model abstraction for framework developers.

Tool calling reconstruction : 1.x embedded private tool execution loops per ChatModel (OpenAI, Ollama, Anthropic each had own), causing code duplication, inconsistent behavior, and bugs. 2.0 moves the execution loop into the Advisor chain for unified handling.

Code example (Spring AI 2.0 tool calling):

// Spring AI 2.0 tool calling
String response = ChatClient.create(chatModel)
    .prompt("明天是星期几？")
    .tools(new DateTimeTools())
    .call()
    .content();

The Advisor chain supports loop re-entry — tool calling Advisor can re-enter downstream chain. Same mechanism enables structured output retry loops, evaluation loops, etc. When to use : Spring Boot 4.x projects, team knows Spring ecosystem, need fast AI integration. Trade-off: requires Java 21+ (2.0) / 17+ (1.x).

3.2 LangChain4j: JVM AI Toolbox

Evolving since early 2023; v1.19.0 as of September 2026. Richest model (30+ out-of-box) and vector store support in Java ecosystem. Core difference: Spring AI targets "integration efficiency"; LangChain4j targets "scenario capability" .

Spring AI wins on native Spring integration: declarative @AiService, auto-configuration, Spring Boot Starters.

LangChain4j wins on breadth/flexibility: framework-neutral (Quarkus, Micronaut, plain Java), mature Agent mechanism for multi-step logic.

Code example (LangChain4j declarative service):

// LangChain4j declarative AI service
interface Assistant {
  @SystemMessage("你是一个Java技术顾问，请用中文回答")
  String chat(@UserMessage String question);
}

Assistant assistant = AiServices.create(Assistant.class, model);
String answer = assistant.chat("什么是虚拟线程？");

MCP protocol reached stability in 1.19 (July 28, 2026 spec: Streamable HTTP stateless, protocol-level session removed). When to use : Need maximum model/vector store coverage or non-Spring stack. Trade-off: less concise syntax, higher learning curve.

3.3 Solon AI: Full Scenario, Full Version Compatibility

Core Solon subproject, positioned as "full-scenario Java AI development framework." Two unique traits:

Java 8–26 compatibility — rare among Java AI frameworks (Spring AI 2.0 needs 21+, LangChain4j needs 17+).

Framework-agnostic embedding — not bound to Solon; drops into Spring Boot, Vert.x, Quarkus, JFinal.

Code example (Solon AI builder API):

// Solon AI builder API
ChatModel chatModel = ChatModel.of("http://127.0.0.1:11434/api/chat")
    .provider("ollama")
    .model("qwen2.5:1.5b")
    .build();

AssistantMessage result = chatModel.prompt("杭州今天天气怎么样？")
    .options(op -> op.toolAdd(new WeatherTools()))
    .call()
    .getMessage();

Provides three-tier agent architecture: SimpleAgent → ReActAgent → TeamAgent. When to use : Projects stuck on Java 8/11, or need a framework-independent AI capability layer.

Layer 2: Agent Orchestration — Controlling Multi-Step AI Work

4.1 Spring AI Alibaba: Enterprise Multi-Agent & Workflow

Co-maintained by Spring and Alibaba open source communities; open-sourced September 2024, 10,000+ GitHub stars. Built on Spring AI, adds upward abstraction and feature enhancement:

Spring AI provides base abstractions (model access, function calling, MCP, memory, RAG, observability).

Spring AI Alibaba adds Agent Framework (ReactAgent core), Graph workflow engine, MCP dual-end support, enterprise governance .

One-sentence distinction: Spring AI solves "how to call models"; Spring AI Alibaba solves "how to make multiple agents collaborate on complex business flows."

Three-layer architecture:

Agent Framework — ReactAgent-centered development framework.

Graph — Low-level workflow & multi-agent coordination, runtime foundation for Agent Framework.

Augmented LLM — Based on Spring AI atomic abstractions: models, tools, MCP, vector stores.

Code example (creating a ReactAgent):

// Spring AI Alibaba create Agent
DashScopeApi dashScopeApi = DashScopeApi.builder()
    .apiKey(System.getenv("AI_DASHSCOPE_API_KEY"))
    .build();
ChatModel chatModel = DashScopeChatModel.builder()
    .dashScopeApi(dashScopeApi)
    .build();

ReactAgent agent = ReactAgent.builder()
    .name("weather_agent")
    .model(chatModel)
    .instruction("You are a helpful weather forecast assistant.")
    .build();

agent.call("what is the weather in Hangzhou?");

Version 1.1.2.0 key upgrades:

Agent Skills — Progressive disclosure: system prompt injects only skill list; full skill content loaded on demand, reducing token consumption and scaling capability.

Parallel multi-agent execution — LlmRouting/Supervisor patterns can route to multiple sub-agents concurrently for parallel domain queries and result aggregation.

When to use : Spring Boot projects building enterprise multi-agent systems or workflow orchestration (intelligent customer service, approval flows, data pipelines). Deep Alibaba Cloud integration (Qwen, Bailian, Higress AI Gateway, Nacos).

4.2 AgentScope Java 2.0: Enterprise Distributed Agent Foundation

GA June 2026; Alibaba's most widely used internal agent framework (Java & Python), running in 10+ core business lines. Core approach: ReActAgent reasoning kernel + Harness engineering layer .

Developers can stay with lightweight ReAct loop or opt into Workspace, persistent memory, Session, Sandbox, Skill, Subagent for production deployment.

Code example (HarnessAgent):

// AgentScope HarnessAgent
var agent = HarnessAgent.builder()
    .name("coder")
    .model("dashscope:qwen-max")
    .workspace(Paths.get(".agentscope/workspace"))
    .filesystem(new DockerFilesystemSpec())
    .isolationScope(IsolationScope.USER)
    .build();

agent.call(msg, RuntimeContext.builder()
    .sessionId("demo")
    .userId("alice")
    .build()).block();

Three differentiating capabilities: Identity persistence, context control, state recovery .

Workspace = Agent persona & long-term memory, auto-injected each turn.

Auto context compression, large tool results spilled to disk, ContextOverflow fallback retry.

Same sessionId restores full conversation across processes; sandbox state snapshots.

When to use : Enterprise multi-agent distributed systems needing long-running stability, multi-tenant isolation, tool permission control.

4.3 Embabel: Taking Planning Back from LLMs

Rod Johnson's JVM-native agent framework, 1.0.0 GA August 2026. Johnson: "Since founding Spring, never been this sure a new project is necessary."

Core design: GOAP (Goal-Oriented Action Planning) — a game-AI planning algorithm. Agents receive a set of Actions with preconditions and effects; planner searches action sequences satisfying the goal. Planning does not depend on LLMs.

Contrast: Most agent frameworks let LLMs decide "what next." LLMs hallucinate and are non-deterministic. Embabel hands planning to deterministic GOAP — computes optimal path from preconditions/effects and explains why each step was chosen.

Built on Spring AI, reuses Spring AI's model connectivity (@Tool, ChatClient, etc.). When to use : Scenarios requiring auditable, deterministic multi-step orchestration — financial compliance, medical decision-making, supply chain scheduling.

Unified Comparison Table

Framework | Layer | Core Positioning | Java Version | Framework Dependency | Best For ---|---|---|---|---|--- Spring AI | Infrastructure | Spring ecosystem AI abstraction | 21+ | Strong Spring Boot | Spring Boot projects fast AI integration LangChain4j | Infrastructure | JVM AI toolbox | 17+ | Framework-neutral | Rich model/vector store support, non-Spring Solon AI | Infrastructure | Full-scenario AI framework | 8–26 | Embeddable anywhere | Java 8/11 projects, framework-agnostic AI layer Spring AI Alibaba | Orchestration | Enterprise multi-agent & workflow | 17+ | Based on Spring AI | Spring Boot enterprise multi-agent/workflow AgentScope | Orchestration | Enterprise distributed Agent | 17+ | Framework-neutral | Enterprise multi-agent production deployment Embabel | Orchestration | Deterministic GOAP orchestration | 17+ | Based on Spring AI | Auditable, deterministic process orchestration

Decision Logic: Three Questions

6.1 Question 1: What is Your Tech Stack?

(Refer to decision diagram image: sz_mmbiz_jpg/HUV4yMdu0rcA0XtOqcLYMyrGltMff8KPPib7qEXlGk3g90n9O7lvkag0koLvByCUWGtx3wmEF8P2FMpMqG9ibK9LCDMwfeb5gxbHlOVVVYF9U)

6.2 Question 2: What Does Your Scenario Need?

(Refer to decision diagram image: sz_mmbiz_jpg/HUV4yMdu0rfOaNuBicVHkEIz3ALzjUvI6ich0JyRL8bGz4qGRAOhHGiaymerIN8wwzmfEILGE6M4sqVAtadhYqcMY7Gv5Sl29S05vH5mIgLbms)

6.3 Three Key Judgments

Judgment 1: Java version? Java 8/11 → only Solon AI works.

Judgment 2: Monolith or distributed? Simple monolith/microservice → Spring AI or LangChain4j sufficient. Enterprise distributed multi-agent → Spring AI Alibaba or AgentScope engineering layer required.

Judgment 3: Need determinism? Finance, healthcare, government (audit/compliance) → Embabel's GOAP planning more reliable because every decision is traceable and explainable.

Pros/Cons Summary

Spring AI 2.0

Pros : Official Spring, deep Spring Boot 4.x integration; elegant Advisor chain, composable tool calling; mature auto-config & observability.

Cons : Hard dependency on Spring Boot 4.x & Java 21+; no Java 8/11/17 support; awkward outside Spring.

Fit : Spring Boot 4.x quick AI integration.

LangChain4j

Pros : 30+ models ready, widest vector store coverage; framework-neutral; mature MCP support.

Cons : Verbose syntax, steeper learning curve; fast iterations, API stability weaker than Spring AI.

Fit : Need model/vector store breadth or non-Spring stack.

Solon AI

Pros : Only framework supporting Java 8–26 ; embeddable in Spring Boot, Vert.x, Quarkus; three-tier agent architecture.

Cons : Smaller ecosystem/community; primarily Chinese docs.

Fit : Java 8/11 projects, or framework-agnostic AI layer.

Spring AI Alibaba

Pros : Spring AI + Alibaba Cloud full stack ; Agent Framework + Graph workflow engine; Agent Skills progressive disclosure cuts tokens; parallel multi-agent execution; deep Qwen/Bailian/Higress/Nacos integration.

Cons : Relatively new (Sept 2024), ecosystem building; Spring-centric, awkward outside Spring.

Fit : Spring Boot enterprise multi-agent systems, workflow orchestration.

AgentScope

Pros : Validated across 10+ Alibaba business lines; Harness layer gives Workspace, persistent memory, sandbox isolation, permission control; distributed deployment support.

Cons : Many concepts, steep learning curve; over-engineered for simple scenarios.

Fit : Enterprise multi-agent distributed systems.

Embabel

Pros : GOAP deterministic planning, no LLM dependency; explainable, auditable; builds on Spring AI, reuses its ecosystem.

Cons : Core in Kotlin (Java-compatible); new, ecosystem nascent.

Fit : Financial compliance, medical decisions, deterministic orchestration needs.

Final Recommendation: Three-Step Selection

Identify your layer : "Integrate AI" (infrastructure) vs. "Orchestrate AI work" (orchestration) — answers differ completely.

List constraints : Java version, framework, team familiarity, data compliance — cuts options in half.

Pinpoint core scenario : RAG, multi-agent collaboration, workflow orchestration, deterministic orchestration, simple integration — each maps to different framework strength.

Worst selection method : Comparing feature lists. "Spring AI has 15 vector stores, LangChain4j has 30, so LangChain4j wins" — meaningless. You'll likely use one; the other 14/29 are baggage. Real question : "What capability does my project need most, and which framework is strongest there?" Right tool = fit, not feature count.

Reference Resources

Spring AI Docs : https://docs.spring.io/spring-ai

LangChain4j Docs : https://docs.langchain4j.dev

Solon AI Docs : https://solon.noear.org

Spring AI Alibaba Docs : https://java2ai.com

Spring AI Alibaba GitHub : https://github.com/alibaba/spring-ai-alibaba

AgentScope Java Docs : https://java.agentscope.io

Embabel Docs : https://docs.embabel.com