Deep Dive into Spring AI: Advanced ChatClient, Prompt Templates, and Function Calling

1. Core Design Patterns of Spring AI

Spring AI follows the same design philosophy as Spring Framework: interface‑driven programming, dependency injection, and convention over configuration. Its architecture revolves around several key patterns:

1. Strategy pattern: provider‑agnostic

This is the soul of Spring AI. To support different APIs such as OpenAI, Ollama, or Tongyi Qianwen, Spring AI defines standard domain models like ChatModel and EmbeddingModel. Concrete implementations include OpenAiChatModel and OllamaChatModel. Switching models only requires configuration changes, no Java code modifications.

2. Fluent API: ChatClient

To reduce the complexity of using a ChatModel, Spring AI provides a higher‑level abstraction – ChatClient. Example code shows chainable calls:

String content = chatClient
    .prompt("你好，你是什么大模型")
    .system("作为技术架构师，回答要体现架构思想")
    .call()
    .content();

Chainable calls give high readability; business code only depends on the ChatClient interface.

3. Advisor chain

Inspired by Servlet Filter and Spring AOP, Spring AI defines an Advisors mechanism to handle cross‑cutting concerns such as memory, safety, and logging. MessageChatMemoryAdvisor: automatically manages conversation history. PromptChatMemoryAdvisor: optimizes the context window.

Custom advisors: implement sensitive‑word filtering, token‑count monitoring, etc.

Since Spring AI 1.1.0‑M4, recursive advisors are introduced, supporting repeated execution of the advisor chain, tool‑calling loops, and output‑validation retries. They use chain.copy(this).nextCall(...) and require explicit termination and retry limits to avoid infinite loops.

2. Prompt Template Deep Dive

2.1 PromptTemplate basics

PromptTemplate

is the core tool for building dynamic prompts; it supports placeholders that are filled at runtime.

@RestController
public class PromptController {
    private final ChatClient chatClient;
    public PromptController(ChatClient.Builder builder) {
        this.chatClient = builder.build();
    }
    @GetMapping("/template")
    public String templateDemo() {
        // create system message template
        SystemPromptTemplate systemTemplate = new SystemPromptTemplate("你是一位风格{style}的演说家");
        Message systemMessage = systemTemplate.createMessage(Map.of("style", "幽默诙谐"));
        // create user message template
        PromptTemplate userTemplate = new PromptTemplate("请介绍一下自己的{store}");
        Message userMessage = userTemplate.createMessage(Map.of("store", "情感经历"));
        List<Message> messages = List.of(systemMessage, userMessage);
        return chatClient.prompt(new Prompt(messages)).call().content();
    }
}

2.2 Injecting templates as resource files

Spring AI allows prompts to be stored in *.st resource files and injected with @Value into a PromptTemplate.

# src/main/resources/templates/user-message.st
讲两个冷笑话，笑话的主题分别是{subject1}和{subject2}

@Value("classpath:/templates/user-message.st")
private Resource promptResource;

@GetMapping("/resource")
public String resourceTemplate() {
    PromptTemplate template = new PromptTemplate(promptResource);
    Prompt prompt = template.create(Map.of(
        "subject1", "程序员",
        "subject2", "产品经理"));
    return chatClient.prompt(prompt).call().content();
}

2.3 Few‑shot prompting

ChatClient supports embedding examples inside the prompt to guide the model, which is useful for tasks that require a specific output format or handling edge cases.

public String fewShotDemo(ChatClient chatClient) {
    String pizzaOrder = chatClient.prompt("""
        Parse a customer's pizza order into valid JSON

        EXAMPLE 1:
        I want a small pizza with cheese, tomato sauce, and pepperoni.
        JSON Response:
        {
            "size": "small",
            "type": "normal",
            "ingredients": ["cheese", "tomato sauce", "pepperoni"]
        }

        EXAMPLE 2:
        Can I get a large pizza with tomato sauce, basil and mozzarella.
        JSON Response:
        {
            "size": "large",
            "type": "normal",
            "ingredients": ["tomato sauce", "basil", "mozzarella"]
        }

        Now, I would like a large pizza, with the first half cheese and mozzarella.
        And the other tomato sauce, ham and pineapple.
        """)
        .options(ChatOptions.builder()
            .temperature(0.1)
            .maxTokens(250)
            .build())
        .call()
        .content();
    return pizzaOrder;
}

Few‑shot prompting is especially effective when the task definition is ambiguous or when a strict output format is required.

2.4 Output parsing and structured responses

Beyond plain text, Spring AI can map LLM responses directly to Java objects.

public enum Sentiment { POSITIVE, NEUTRAL, NEGATIVE }

public Sentiment classifySentiment(ChatClient chatClient) {
    Sentiment result = chatClient.prompt("""
        Classify movie reviews as POSITIVE, NEUTRAL or NEGATIVE.
        Review: \"Her\" is a disturbing study revealing the direction
        humanity is headed if AI is allowed to keep evolving, unchecked.
        Sentiment:
        """)
        .options(ChatOptions.builder()
            .temperature(0.1)
            .maxTokens(5)
            .build())
        .call()
        .entity(Sentiment.class);
    return result;
}

When combined with system prompts that ask the model to return structured data, the entity() method becomes powerful. Spring AI 1.1.0‑M4 also adds StructuredOutputValidationAdvisor, which validates the model output against a JSON Schema and retries on failure, with a configurable maximum retry count.

3. Function Calling Deep Dive

3.1 Declarative tool definition

Spring AI provides a convenient API to define tools. Methods annotated with @Tool are automatically exposed to the LLM.

import org.springframework.ai.tool.annotation.Tool;
import org.springframework.ai.tool.annotation.ToolParam;
import org.springframework.stereotype.Component;
import java.time.LocalDateTime;

@Component
public class DateTimeTools {
    @Tool(description = "获取用户时区的当前日期和时间")
    String getCurrentDateTime() {
        return LocalDateTime.now().toString();
    }

    @Tool(description = "为指定时间设置闹钟，时间格式为 ISO-8601")
    String setAlarm(@ToolParam(description = "ISO-8601 格式的时间") String time) {
        System.out.println("⏰ 闹钟已设置为 " + time);
        return "闹钟已设置为 " + time;
    }
}

The @Tool annotation supports the following parameters: name: tool name (defaults to method name), must be unique. description: detailed description to help the model decide when to call. returnDirect: whether the tool result is returned directly to the client without passing back to the model. resultConverter: custom result converter.

3.2 Tool invocation flow

┌─────────────────────────────────────────────────────────────────┐
│               Spring AI Tool Invocation Flow                │
├─────────────────────────────────────────────────────────────────┤
│ 1. User registers tools in ChatClient (.tools(new DateTimeTools())) │
│   ▼                                                             │
│ 2. Spring AI converts method signatures to JSON Schema          │
│   ▼                                                             │
│ 3. Model decides to call a tool, returns tool name and arguments│
│   ▼                                                             │
│ 4. Spring AI parses request and executes the corresponding Java method │
│   ▼                                                             │
│ 5. Tool result is returned to the model, which generates final response │
└─────────────────────────────────────────────────────────────────┘

3.3 Adding tools to ChatClient

Runtime‑only tools (effective for a single request):

ChatClient chatClient = ChatClient.create(chatModel);
String response = chatClient.prompt("明天是星期几？")
    .tools(new DateTimeTools()) // only for this request
    .call()
    .content();

Default tools (apply to all ChatClient instances built from the same builder):

ChatClient chatClient = ChatClient.builder(chatModel)
    .defaultToolCallbacks(ToolCallbacks.from(new DateTimeTools()))
    .build();

If the model needs real‑time data (e.g., current date), it will automatically invoke the tool; otherwise it falls back to a generic answer.

3.4 Multi‑tool collaboration

Spring AI can invoke multiple tools within a single request to accomplish complex tasks.

// User asks: "Set an alarm 10 minutes from now"
// Model flow:
// 1. Call getCurrentDateTime to obtain now
// 2. Compute time 10 minutes later
// 3. Call setAlarm with the computed time

String response = ChatClient.create(chatModel)
    .prompt("Can you set an alarm 10 minutes from now?")
    .tools(new DateTimeTools())
    .call()
    .content();

4. Testing and Verification

Example curl commands to test the endpoints:

# Test Prompt Template
curl http://localhost:8080/prompt/template

# Test Few‑shot Prompt
curl http://localhost:8080/prompt/fewshot

# Test Tool Invocation
curl "http://localhost:8080/tools/current-time"

5. Next Episode Preview

Java Technology Stack AI Application Development (Part 3): Retrieval‑Augmented Generation (RAG) in Java, covering vector‑database integration, document loading and chunking, and a full RAG pipeline.