Building Java AI Agents with Spring AI: A Hands‑On Guide

Spring AI provides a native Java framework for building AI agents that integrates directly with Spring Boot, avoiding the need for a Python stack.

Why Java developers need Spring AI

Seamless Spring Boot integration

Supports OpenAI, Azure, Tongyi Qianwen, Zhipu, Ollama and other model providers

Unified API abstraction for chat, embeddings and vector stores

Production‑grade features such as retry, circuit‑breaker, rate‑limiting and monitoring

Spring AI Architecture

┌─────────────────────────────────────────────────────────────────┐
│                     Spring AI Architecture                     │
├─────────────────────────────────────────────────────────────────┤
│                                                               │
│   ┌─────────────────────────────────────────────────────┐   │
│   │                     Business Code                  │   │
│   │   ChatClient / PromptTemplate / Tool Calls          │   │
│   └─────────────────────────────────────────────────────┘   │
│                     │                                       │
│   ┌─────────────────────────────────────────────────────┐   │
│   │                Spring AI Core Abstractions        │   │
│   │   ChatModel │ EmbeddingModel │ VectorStore │ Tool │   │
│   └─────────────────────────────────────────────────────┘   │
│                     │                                       │
│   ┌─────────────────────────────────────────────────────┐   │
│   │                AI Model Adapters                  │   │
│   │   OpenAI │ Azure │ Tongyi Qianwen │ Zhipu │ Ollama │   │
│   └─────────────────────────────────────────────────────┘   │
│                                                               │
└─────────────────────────────────────────────────────────────────┘

Environment Setup and Quick Start

Create a Maven project with Spring Boot 3.2.4 and add the Spring AI starter, Chroma vector store and Lombok. The pom.xml snippet below shows the required coordinates and the Spring Milestones repository.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>3.2.4</version>
    </parent>
    <groupId>com.teaching</groupId>
    <artifactId>spring-ai-agent</artifactId>
    <version>1.0.0-SNAPSHOT</version>
    <properties>
        <java.version>17</java.version>
        <spring-ai.version>1.0.0-M4</spring-ai.version>
    </properties>
    <repositories>
        <repository>
            <id>spring-milestones</id>
            <name>Spring Milestones</name>
            <url>https://repo.spring.io/milestone</url>
        </repository>
    </repositories>
    <dependencies>
        <!-- Spring AI core -->
        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-openai-spring-boot-starter</artifactId>
            <version>${spring-ai.version}</version>
        </dependency>
        <!-- Web support -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <!-- Vector store (Chroma) -->
        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-chroma-store</artifactId>
            <version>${spring-ai.version}</version>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
    </dependencies>
</project>

The application.yml configures OpenAI credentials, model options, embedding model, server port and logging level.

spring:
  application:
    name: spring-ai-agent
  ai:
    openai:
      api-key: ${OPENAI_API_KEY:your-api-key}
      base-url: ${OPENAI_BASE_URL:https://api.openai.com/v1}
      chat:
        options:
          model: ${MODEL_NAME:gpt-4}
          temperature: 0.7
          max-tokens: 2000
      embedding:
        options:
          model: text-embedding-ada-002
    server:
      port: 8080
    logging:
      level:
        com.teaching: DEBUG

Prompt Templates

The PromptService shows two templates: a simple style‑injection template and a few‑shot Q&A template.

public String formatTemplate(String concept, String style) {
    String template = """
        你是一个{style}风格的讲解专家。
        请用{style}的风格解释：{concept}

        要求：
        1. 语言要{style}
        2. 举例要贴近生活
        3. 控制在100字以内
        """;
    PromptTemplate promptTemplate = new PromptTemplate(template);
    String prompt = promptTemplate.create(Map.of("style", style, "concept", concept)).getContents();
    return chatModel.call(prompt);
}

public String fewShotQA(String question) {
    String template = """
        以下是一些问答示例：

        问题：什么是 Java？
        答案：Java 是一种面向对象的编程语言，由 Sun Microsystems 公司于 1995 年发布。

        问题：什么是 Spring Boot？
        答案：Spring Boot 是 Spring 框架的扩展，用于简化 Spring 应用的初始搭建和开发过程。

        问题：{question}
        答案：
        """;
    PromptTemplate promptTemplate = new PromptTemplate(template);
    String prompt = promptTemplate.create(Map.of("question", question)).getContents();
    return chatModel.call(prompt);
}

Tool Calling (Function Calling)

Three example tools are provided. WeatherTool: returns simulated weather data for a few Chinese cities. CalculatorTool: evaluates a JavaScript expression using ScriptEngine. OrderQueryTool: looks up a static map of order information.

// WeatherTool.java (excerpt)
public String getName() { return "getWeather"; }
public String getDescription() { return "获取指定城市的天气信息，参数：city（城市名称）"; }
public BiFunction<Map&lt;String, Object&gt;, Map&lt;String, Object&gt;, Object> getCallback() {
    return (request, context) -> {
        String city = (String) request.get("city");
        if (city == null) city = (String) request.get("city_name");
        return switch (city) {
            case "北京" -> "晴，25°C，湿度45%，东南风2级，紫外线强";
            case "上海" -> "多云，22°C，湿度65%，东北风3级";
            case "深圳" -> "雨，28°C，湿度85%，南风4级";
            case "广州" -> "阴，26°C，湿度70%，微风";
            default -> city + "天气：晴，20°C";
        };
    };
}

Agent Configuration

The AgentConfig bean registers the tools and creates a ChatClient with a system prompt that defines the assistant’s role and workflow.

@Configuration
public class AgentConfig {
    @Bean
    public ToolCallbacks tools(WeatherTool weatherTool, CalculatorTool calculatorTool, OrderQueryTool orderQueryTool) {
        return ToolCallbacks.from(weatherTool, calculatorTool, orderQueryTool);
    }

    @Bean
    public ChatClient agentClient(ChatClient.Builder builder, ToolCallbacks tools) {
        return builder
            .defaultSystem("""
                你是一个智能助手，可以调用工具来帮助用户解决问题。

                可用工具：
                - getWeather: 查询天气
                - calculate: 数学计算
                - queryOrder: 查询订单

                工作流程：
                1. 分析用户需求
                2. 选择合适的工具
                3. 调用工具获取结果
                4. 给出最终答案

                注意：不要编造信息，所有数据必须来自工具。
                """)
            .defaultTools(tools)
            .build();
    }
}

Retrieval‑Augmented Generation (RAG)

RagService

loads a knowledge file, splits it into token chunks, stores the chunks in a vector store and provides a similarity‑search method that returns the top‑K matching documents.

@Service
public class RagService {
    private final VectorStore vectorStore;
    @Value("classpath:/knowledge.txt")
    private Resource knowledgeResource;

    public RagService(VectorStore vectorStore) { this.vectorStore = vectorStore; }

    @PostConstruct
    public void init() {
        TikaDocumentReader reader = new TikaDocumentReader(knowledgeResource);
        List<Document> documents = reader.get();
        TokenTextSplitter splitter = new TokenTextSplitter();
        List<Document> chunks = splitter.apply(documents);
        vectorStore.accept(chunks);
        System.out.println("✅ 知识库加载完成，共 " + chunks.size() + " 个文档块");
    }

    public String search(String query, int topK) {
        List<Document> results = vectorStore.similaritySearch(query, topK);
        if (results.isEmpty()) return "未找到相关信息";
        StringBuilder context = new StringBuilder();
        for (int i = 0; i < results.size(); i++) {
            context.append(i + 1).append(". ").append(results.get(i).getContent()).append("

");
        }
        return context.toString();
    }
}

Production‑Grade Agent Service

ProductionAgentService

adds Resilience4j annotations for retry, circuit‑breaker and rate‑limiting. It first retrieves relevant knowledge via RAG, builds an enhanced prompt that includes the retrieved context, and then calls the agent. Fallback methods return user‑friendly messages when the service is unavailable or throttled.

@Service
@Slf4j
public class ProductionAgentService {
    private final ChatClient agentClient;
    private final RagService ragService;

    public ProductionAgentService(ChatClient agentClient, RagService ragService) {
        this.agentClient = agentClient;
        this.ragService = ragService;
    }

    @Retry(name = "agentRetry", fallbackMethod = "fallbackResponse")
    @CircuitBreaker(name = "agentCircuitBreaker", fallbackMethod = "fallbackResponse")
    @RateLimiter(name = "agentRateLimiter", fallbackMethod = "rateLimitFallback")
    public String chatWithRAG(String userInput) {
        String context = ragService.search(userInput, 3);
        String enhancedPrompt = String.format("""
            参考以下知识库内容回答问题：

            知识库：
            %s

            问题：%s

            请基于知识库内容回答，如果知识库中没有相关信息，请如实告知。
            """, context, userInput);
        return agentClient.prompt(enhancedPrompt).call().content();
    }

    public String fallbackResponse(String userInput, Throwable t) {
        log.error("Agent 调用失败: {}", t.getMessage());
        return "服务繁忙，请稍后重试。如有紧急问题，请联系人工客服。";
    }

    public String rateLimitFallback(String userInput, Throwable t) {
        return "请求过于频繁，请稍后再试。";
    }
}

Spring AI vs LangChain Comparison

生态成熟度 : LangChain ★★★★★, Spring AI ★★★☆☆

社区活跃度 : LangChain ★★★★★, Spring AI ★★★☆☆

企业集成 : LangChain ★★☆☆☆, Spring AI ★★★★★

类型安全 : LangChain ★★☆☆☆, Spring AI ★★★★★

性能 : LangChain ★★★★☆, Spring AI ★★★★★

学习曲线 : LangChain 平缓, Spring AI 中等

适用场景 : LangChain 原型、研究, Spring AI 生产、企业应用