Spring AI Overhauls Memory: Replacing ChatMemory with Session

Why the old ChatMemory fails

Each LLM call is stateless; the application must feed the full conversation history. The default MessageWindowChatMemory keeps a fixed‑size window (20 messages) and discards the oldest entries, including the initial SystemMessage. This works for simple Q&A but breaks when an Agent starts invoking tools.

Tool‑calling introduces a four‑message sequence: user question, tool request, tool result, final reply. If the sliding window truncates in the middle of this sequence, the model receives an incomplete context (e.g., only the tool result and reply), leading to nonsensical output or protocol errors. Moreover, the production‑grade JdbcChatMemoryRepository only stores user and assistant text messages, so function‑calling messages disappear after a restart, erasing the reasoning trace.

Spring AI 2.0 moves the tool‑calling loop into the Advisor chain, making the old memory mechanism incompatible: placing it outside the loop loses intermediate reasoning, while inserting it inside causes JDBC persistence failures.

Session redesign

Spring AI introduces Session – an immutable metadata object (session ID, user ID, TTL) that carries no payload – and SessionEvent, an immutable record for every user input, model reply, tool call, or tool result. Each event carries a UUID, timestamp, and session ID and is only ever appended.

A Turn groups a UserMessage with all subsequent events (tool calls, intermediate reasoning, final reply) until the next UserMessage. Compression now operates on whole Turns, guaranteeing that a complete tool‑calling round is either kept or fully discarded.

Automatic compression

Compression is split into two pluggable components:

Triggers decide *when* to compress: TurnCountTrigger: fires after a configurable number of Turns (e.g., 20). TokenCountTrigger: fires based on estimated token usage, useful for cost‑sensitive scenarios. CompositeCompactionTrigger: combines multiple conditions with OR/AND logic.

Strategies decide *how* to truncate: SlidingWindowCompactionStrategy: hard cut on event count, aligning the cut to the nearest Turn boundary. TurnWindowCompactionStrategy: retains the most recent N Turns, discarding whole Turns together. TokenCountCompactionStrategy: accumulates Turns from newest to oldest until a token budget is reached. RecursiveSummarizationCompactionStrategy: invokes the model to summarise older Turns into a synthetic user‑assistant message, then stacks summaries.

Getting started

Add the session starter dependency:

<dependency>
  <groupId>org.springaicommunity</groupId>
  <artifactId>spring-ai-starter-session-jdbc</artifactId>
  <version>${spring-ai-session.version}</version>
</dependency>

Configure a SessionMemoryAdvisor and replace the old MessageChatMemoryAdvisor:

SessionMemoryAdvisor advisor = SessionMemoryAdvisor.builder(sessionService)
    .defaultUserId("alice")
    .compactionTrigger(new TurnCountTrigger(20))
    .compactionStrategy(TurnWindowCompactionStrategy.builder().maxTurns(10).build())
    .build();

ChatClient client = ChatClient.builder(chatModel)
    .defaultAdvisors(advisor)
    .build();

String answer = client.prompt()
    .user("帮我review一下当前的系统架构")
    .advisors(a -> a.param(SessionMemoryAdvisor.SESSION_ID_CONTEXT_KEY, "session-abc"))
    .call()
    .content();

When a session ID is supplied, the session is created automatically. Loading history, appending new events, and triggering compression are all handled inside the advisor, so application code stays clean.

Multi‑Agent support

Agents that run parallel sub‑tasks can now isolate their logs using a Branch label on SessionEvent (e.g., orch.researcher). Applying EventFilter.forBranch(...) to an advisor ensures each sub‑agent only sees its own branch and ancestor events, eliminating cross‑agent noise.

Compressed summary events remain on the root branch, visible to all agents. The SessionEventTools component wraps the immutable log as a conversation_search tool, allowing the model to query older details that were trimmed from the active context.

Roadmap

According to the Spring AI roadmap, Session will officially replace ChatMemory in the 2.1 release, expected around November 2026.