Skip to content
Spring AI 2.1: Decision Models, Session API & Stateless MCP for Production Agents

Spring AI 2.1: Decision Models, Session API & Stateless MCP for Production Agents

Spring AI 2.1 introduces Decision Models for structured outputs, a new message-part model preserving reasoning traces, Session API replacing ChatMemory with event-sourced memory and recall, tool confirmation and rate limits for safe agent loops, stateless MCP 2026-07-28 protocol, and bring-your-own-vector support across 11 vector stores.

java1234
java1234
java1234

What Is Spring AI

Spring AI is the official Spring framework for building AI applications in Java. Its core idea: write AI applications the Spring way. It provides a unified model abstraction across OpenAI, Anthropic, Google GenAI, Amazon Bedrock, Ollama, DeepSeek, Mistral, and others — switching models is mostly a matter of changing a starter and a few configuration lines. The ChatClient streaming API feels like WebClient or RestClient, so Spring developers have near-zero learning curve. The Advisor mechanism lets you plug in memory, RAG, safety guardrails, and logging like interceptors. Built-in support covers tool calling, structured output, RAG, vector databases, MCP, and observability. From 1.0 GA (2025) to 2.0 on Spring Boot 4, and now 2.1, the framework has moved from "works" to "works well" and is now targeting production-grade agents.

2.1 Feature Overview

The release includes over 430 commits in five months. Key features:

MCP 2026-07-28 : Supports the new stateless MCP protocol on both server and client sides

Decision Models : New model API for yes/no, single-choice, and scoring decisions with probabilities

Message Parts : Messages become ordered lists of parts (text, reasoning, tool calls, media) preserving order

Session API : Replaces ChatMemory with append-only event log, turn-based compaction, and recall tool

Tool Confirmation & Limits : Sensitive operations require human approval; per-tool and total call limits prevent runaway loops

Bring-Your-Own Vectors : 11 vector stores now support upserting pre-computed embeddings and querying with custom vectors

Security : 13 CVEs fixed

Spring AI 2.1 is built on Spring Boot 4.2; both must be upgraded together. The BOM coordinates are:

<parent>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-parent</artifactId>
  <version>4.2.0</version>
</parent>
<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.springframework.ai</groupId>
      <artifactId>spring-ai-bom</artifactId>
      <version>2.1.0</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

Decision Models: Let the Model Do Multiple Choice

This is arguably the most practical addition. Business logic often needs fixed answers: is this comment spam? Which category does this ticket belong to? Is the user satisfied? Previously you had to prompt a chat model for free text and parse JSON/regex, which broke when the model hallucinated extra words. DecisionModel solves this: you provide the allowed options, the model picks one, and returns a probability for each option. Your code decides the confidence threshold for automatic action.

Three question types are supported: YesNoQuestion, ChoiceQuestion, and ScoreQuestion. The example below shows a customer-support triage service that simultaneously decides whether a message reports a defect, classifies the topic, and scores satisfaction — all in one call. It runs locally with Ollama and the d1-omni:600m model, no API key required.

@Service
public class TicketTriageService {
    private final DecisionModel decisionModel;
    public TicketTriageService(DecisionModel decisionModel) {
        this.decisionModel = decisionModel;
    }
    public String triage(String message) {
        DecisionResponse response = decisionModel.call(DecisionPrompt.builder()
            .text(message)
            .question("defect", YesNoQuestion.builder()
                .text("Does the customer report a defect?")
                .whenYes("Part of the item does not work as it should")
                .whenNo("The item works, and the message is about something else")
                .build())
            .question("topic", ChoiceQuestion.builder()
                .text("What is the message about?")
                .option("delivery", "物流、配送进度")
                .option("product", "收到的商品有问题")
                .option("account", "登录、地址或支付信息")
                .option("other", "以上都不是")
                .build())
            .question("satisfaction", ScoreQuestion.of(
                "How satisfied is the customer?", "unhappy", "neutral", "happy"))
            .build());
        boolean defect = response.getYesNo("defect").isYes(0.9);
        String topic = response.getChoice("topic").getValue();
        int satisfaction = response.getScore("satisfaction").getNearestLevel();
        return "缺陷=" + defect + ",分类=" + topic + ",满意度等级=" + satisfaction;
    }
}

Configuration is minimal:

spring:
  ai:
    ollama:
      init:
        pull-model-strategy: when_missing

Three implementations exist: OpenAiDecisionModel, OllamaDecisionModel, and SystemOneDecisionModel. The framework already builds higher-level features on top, such as DecisionDocumentReranker (RAG re-ranking), DecisionDocumentFilter (prompt-injection filtering), and DecisionToolIndex (tool retrieval).

Reminder: decision models are still ML models, not rule engines. For ultra-high-stakes decisions, keep a human in the loop.

Richer Message Model: Reasoning No Longer Lost

Pre-2.0, a message was a flat text plus tool-call and media lists. Modern models interleave reasoning, tool calls, and media in complex sequences. The old structure couldn't preserve order or capture thinking traces. In 2.1, AssistantMessage, UserMessage, and ToolResponseMessage become ordered lists of parts: TextPart, ReasoningPart, ToolCallPart, ToolResultPart, MediaPart, and fallback UnknownPart. Anthropic's thinking signatures and similar opaque blobs are retained verbatim.

The example below sends a text prompt with two images and prints the model's reasoning parts via the new getReasoning() accessor.

@Service
public class ChartCompareService {
    private final ChatModel chatModel;
    public ChartCompareService(ChatModel chatModel) {
        this.chatModel = chatModel;
    }
    public String compare(Media firstChart, Media secondChart) {
        UserMessage question = UserMessage.builder()
            .part(TextPart.of("这两张图表有什么变化?"))
            .part(MediaPart.of(firstChart))
            .part(MediaPart.of(secondChart))
            .build();
        AssistantMessage answer = chatModel.call(new Prompt(question)).getResult().getOutput();
        for (ReasoningPart reasoning : answer.getReasoning()) {
            System.out.println("推理过程:" + reasoning.text());
        }
        return answer.getText();
    }
}

Legacy getters like getText() and getToolCalls() remain as views over the part list, so existing code doesn't break.

2.1 also adds OpenAiResponsesChatModel targeting OpenAI's /v1/responses endpoint. From GPT-5.4 onward, the Chat Completions API no longer supports combining tool calls with reasoning effort; agents must use the Responses API. Switching is a single config line: spring.ai.openai.chat.api=responses Every Generation now carries a unified FinishReasonCategory ( COMPLETED, TRUNCATED, TOOL_CALL, FILTERED, etc.), eliminating vendor-specific stop / end_turn / STOP if-else chains.

Agent Foundations: Session, Tool Confirmation & Call Limits

Models only generate text; the agent loop (iteration, tools, memory, permissions, checks) lives outside. 2.1 adds several critical pieces to that shell.

5.1 Session API: Replacing ChatMemory

ChatMemory

simply stored a message list per conversation ID. It struggled with long dialogues and agent runs. The new Session API persists conversations as an append-only event log and adds three capabilities:

Turn-based compaction : when history grows, it compresses at turn boundaries, never splitting a tool call from its result. Original log is untouched; a boundary marker is appended.

Recall retrieval : compacted content stays in the log. The model can search it via the built-in conversation_search tool.

JDBC persistence : JdbcSessionRepository supports PostgreSQL, MySQL, MariaDB, SQL Server, and H2.

Add the starter spring-ai-starter-model-session-jdbc and configure a datasource:

spring:
  datasource:
    url: jdbc:mysql://localhost:3306/db_spring_ai?useUnicode=true&characterEncoding=utf8&serverTimezone=Asia/Shanghai
    username: root
    password: 123456

Then register the session advisor and recall tool:

@Configuration
public class ChatClientConfig {
    @Bean
    public ChatClient chatClient(ChatModel chatModel, SessionService sessionService) {
        return ChatClient.builder(chatModel)
            .defaultAdvisors(SessionAdvisors.builder(sessionService)
                .compaction(new TurnCountTrigger(20), WindowCompactionStrategy.builder().lastTurns(10).build())
                .build())
            .defaultTools(SessionRecallTools.builder(sessionService).build())
            .build();
    }
}

Each request must pass the session ID:

@RestController
@RequestMapping("/api/chat")
public class ChatController {
    private final ChatClient chatClient;
    public ChatController(ChatClient chatClient) { this.chatClient = chatClient; }
    @GetMapping
    public String chat(@RequestParam String sessionId, @RequestParam String message) {
        return chatClient.prompt()
            .user(message)
            .advisors(SessionAdvisors.session(sessionId))
            .toolContext(Map.of(SessionRecallTools.SESSION_ID_CONTEXT_KEY, sessionId))
            .call()
            .content();
    }
}

The framework intentionally provides no default session to avoid silently mixing different users' histories. The advisor is stateless, so any instance in a clustered deployment can continue a session. ChatMemory classes are deprecated and slated for removal in 3.0.0; new projects should adopt Session directly.

5.2 Tool Confirmation: Human-in-the-Loop for Sensitive Actions

Booking flights, deleting files, sending emails — you don't want the model to execute these autonomously. Annotate the tool with requiresConfirmation = true:

@Component
public class BookingTools {
    private final BookingService bookingService;
    public BookingTools(BookingService bookingService) { this.bookingService = bookingService; }
    @Tool(description = "Book a flight", requiresConfirmation = true)
    public String bookFlight(String flightNumber) { return bookingService.book(flightNumber); }
    @Tool(description = "Get the status of a flight")
    public String flightStatus(String flightNumber) { return bookingService.status(flightNumber); }
}

The confirmation flow is stateless: the framework pauses, returns a response listing pending tool calls with IDs; the UI presents them to the user; the user approves or denies each. On resume, you supply ToolCallDecisions and the manager executes allowed calls, feeds denials back to the model with the denial reason, and continues the conversation.

ToolCallDecisions decisions = ToolCallDecisions.of(
    ToolCallDecision.allow("call_1"),
    ToolCallDecision.deny("call_2", "用户取消了这次预订"));
ToolExecutionResult result = toolCallingManager.executeToolCalls(prompt, reply, decisions);
String answer = chatClient.prompt()
    .messages(result.conversationHistory())
    .call()
    .content();

Because pause/resume are ordinary request/response cycles, the conversation can survive a service restart as long as you persist the history. For third-party MCP tools you don't control, use

ToolCallConfirmationPolicy.forTools("bookFlight", "cancelBooking")

to require confirmation by name.

5.3 Tool Call Limits: Prevent Runaway Loops

Models can enter infinite tool-call loops, burning tokens and money. 2.1 enforces defaults: max 40 calls per tool per turn, 150 total calls per turn. Adjust via spring.ai.tools.limits.* or programmatically:

ToolCallingManager manager = ToolCallingManager.builder()
    .maxCallsPerTool(20)
    .build();

5.4 Unified Guardrail Refusal Handling

Guardrail rejections are now surfaced as GuardrailRefusal; vendor safety filters throw ContentFilteredException. A single catch block handles both:

try {
    Order order = chatClient.prompt().user(message).call().entity(Order.class);
} catch (ContentFilteredException ex) {
    return "抱歉,这个问题我没法回答。";
}

MCP 2026-07-28: Stateless Protocol

The 2026-07-28 MCP revision removes handshakes and sessions. Each request carries the protocol version and client capabilities, making server-side horizontal scaling trivial. Enable the modern protocol with one property; existing @McpTool, @McpResource annotations work unchanged. spring.ai.mcp.server.protocol=MODERN Example tool:

@Component
public class WeatherMcpTools {
    @McpTool(name = "weather", description = "查询城市天气")
    public String weather(@McpToolParam(description = "城市名称") String city) {
        return city + ":晴,18℃";
    }
}

Clients can mix legacy and modern connections; the framework merges tools from both. Modern clients are stateless and cheap to create (default PER_REQUEST scope), so per-user isolation becomes natural.

spring:
  ai:
    mcp:
      client:
        streamable-http:
          connections:
            billing:
              url: http://localhost:8081
            tickets:
              url: http://localhost:8080
              protocol: MODERN

Vector Stores: Bring Your Own Vectors

Previously VectorStore.add() always used the store's configured embedding model. Real projects often compute embeddings in nightly batch jobs or separate pipelines. 2.1 adds upsert() to write pre-computed vectors (idempotent, safe for retries) and similaritySearch() overloads that accept a query embedding directly.

// Write pre-computed vectors; same ID overwrites, safe for job retries
vectorStore.upsert(List.of(new EmbeddedDocument(document, embedding)));
// Query with your own embedding
List<Document> results = vectorStore.similaritySearch(queryEmbedding,
    SearchRequest.builder().topK(5).similarityThreshold(0.7).build());

Supported stores: pgvector, Qdrant, Redis, Elasticsearch, OpenSearch, Neo4j, MongoDB Atlas, Typesense, MariaDB, Chroma, Couchbase (11 total). Additionally, TokenTextSplitter now supports chunk overlap, a welcome improvement for RAG pipelines.

Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Sign in to view source
Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactadmin@besthub.devand we will review it promptly.

JavaMCPAgentSpring BootSpring AIVector StoreSession APIDecision Models
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.