Building Java AI Agents with LangChain4j: A Hands‑On Guide

Why LangChain4j?

Spring AI provides basic model calls, RAG and function calling, but scenarios that require multi‑step reasoning, tool composition and conversation memory need a framework with native agent orchestration and richer memory strategies. LangChain4j fills this gap with built‑in agent abstractions, pluggable memory implementations and fine‑grained component composition.

Core capability comparison

Agent orchestration: Spring AI – basic tool calling; LangChain4j – native AiServices with declarative definition.

Memory management: Spring AI – MessageChatMemoryAdvisor; LangChain4j – built‑in ChatMemory supporting window, token and persistent options.

Tool definition: both use @Tool; LangChain4j adds a more flexible callback mechanism.

Community ecosystem: Spring AI – official Spring project with rapid growth; LangChain4j – >4k GitHub stars and production‑grade validation.

Learning curve: Spring AI – low for Spring developers; LangChain4j – medium because of richer concepts, though the organization remains clear.

Quick start

Project dependencies

<!-- pom.xml -->
<properties>
    <langchain4j.version>1.0.0-beta2</langchain4j.version>
</properties>

<dependencies>
    <!-- LangChain4j core -->
    <dependency>
        <groupId>dev.langchain4j</groupId>
        <artifactId>langchain4j</artifactId>
        <version>${langchain4j.version}</version>
    </dependency>
    <!-- OpenAI integration -->
    <dependency>
        <groupId>dev.langchain4j</groupId>
        <artifactId>langchain4j-open-ai</artifactId>
        <version>${langchain4j.version}</version>
    </dependency>
    <!-- Spring Boot starter (auto‑configuration) -->
    <dependency>
        <groupId>dev.langchain4j</groupId>
        <artifactId>langchain4j-spring-boot-starter</artifactId>
        <version>${langchain4j.version}</version>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
    <dependency>
        <groupId>org.projectlombok</groupId>
        <artifactId>lombok</artifactId>
        <optional>true</optional>
    </dependency>
</dependencies>

Basic configuration (application.yml)

langchain4j:
  chat-model:
    provider: openai
    api-key: ${OPENAI_API_KEY}
    base-url: ${OPENAI_BASE_URL:https://api.openai.com/v1}
    model-name: ${MODEL_NAME:gpt-4}
    temperature: 0.7
    max-tokens: 2000

First agent – interface as agent

import dev.langchain4j.service.SystemMessage;
import dev.langchain4j.service.UserMessage;
import dev.langchain4j.service.V;

public interface CustomerAgent {
    @SystemMessage("""
        你是一个专业的客服助手，专门处理订单查询、退款和售后问题。
        请友好、专业地帮助用户。
        """)
    String chat(@UserMessage @V("question") String question);
}

Bean configuration

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import dev.langchain4j.service.AiServices;
import dev.langchain4j.model.language.ChatLanguageModel;

@Configuration
public class AgentConfig {
    @Bean
    public CustomerAgent customerAgent(ChatLanguageModel model) {
        return AiServices.builder(CustomerAgent.class)
                         .chatLanguageModel(model)
                         .build();
    }
}

Defining tools

import dev.langchain4j.agent.tool.Tool;
import dev.langchain4j.agent.tool.P;
import org.springframework.stereotype.Component;
import java.util.Map;

@Component
public class OrderTools {
    private static final Map<String, String> ORDERS = Map.of(
        "1001", "已发货，预计明天送达，快递单号 SF123456",
        "1002", "处理中，预计48小时内发货",
        "1003", "已签收"
    );

    @Tool("查询订单状态，参数为订单号")
    public String queryOrder(@P("订单号") String orderId) {
        String result = ORDERS.get(orderId);
        return result != null ? "订单 " + orderId + "：" + result : "未找到订单 " + orderId;
    }

    @Tool("获取退款政策")
    public String getRefundPolicy() {
        return "7天无理由退货，15天质量问题换货，1年质保免费维修";
    }
}

Agent with tools and memory

import dev.langchain4j.service.SystemMessage;
import dev.langchain4j.service.UserMessage;

public interface SmartCustomerAgent {
    @SystemMessage("""
        你是专业客服助手，可以调用工具处理用户问题。
        工具列表：
        - queryOrder：查询订单
        - getRefundPolicy：退款政策
        请根据用户问题选择合适的工具。
        """)
    String chat(@UserMessage String question);
}

import org.springframework.context.annotation.Bean;
import dev.langchain4j.service.AiServices;
import dev.langchain4j.memory.ChatMemory;
import dev.langchain4j.memory.MessageWindowChatMemory;
import dev.langchain4j.model.language.ChatLanguageModel;

@Bean
public SmartCustomerAgent smartAgent(ChatLanguageModel model, OrderTools tools) {
    ChatMemory chatMemory = MessageWindowChatMemory.builder()
                                                .maxMessages(10)
                                                .build();
    return AiServices.builder(SmartCustomerAgent.class)
                     .chatLanguageModel(model)
                     .tools(tools)
                     .chatMemory(chatMemory)
                     .build();
}

Controller

import org.springframework.web.bind.annotation.*;
import java.util.Map;

@RestController
@RequestMapping("/api/agent")
public class AgentController {
    private final SmartCustomerAgent agent;

    public AgentController(SmartCustomerAgent agent) {
        this.agent = agent;
    }

    @PostMapping("/chat")
    public Map<String, String> chat(@RequestBody Map<String, String> request) {
        String response = agent.chat(request.get("message"));
        return Map.of("response", response);
    }
}

Memory management deep dive

Message window memory

Retains the most recent N messages; older messages are discarded.

ChatMemory memory = MessageWindowChatMemory.builder()
                                          .maxMessages(10)
                                          .build();

Token window memory

Controls context size by token count, suitable for production environments.

ChatMemory memory = TokenWindowChatMemory.builder()
                                        .maxTokens(2000)
                                        .tokenCountEstimator(new OpenAiTokenizer())
                                        .build();

Persistent memory

Using ChatMemoryStore, memory can be stored in Redis or a database for cross‑session persistence.

ChatMemory memory = MessageWindowChatMemory.builder()
                                          .maxMessages(10)
                                          .chatMemoryStore(new RedisChatMemoryStore(redisTemplate))
                                          .build();

Complete LangChain4j agent flow

用户请求
    │
    ▼
AiServices 动态代理
    │
    ▼
ChatMemory 加载历史
    │
    ▼
SystemMessage + UserMessage
    │
    ▼
LLM 决策（是否调用工具）
    │
    ├── 是 → 执行 Tool（Java 方法） → 结果返回 LLM
    ▼
生成最终响应

Testing the agent

# Test order query
curl -X POST http://localhost:8080/api/agent/chat \
  -H "Content-Type: application/json" \
  -d '{"message":"查询订单1001"}'

# Test multi‑turn conversation
curl -X POST http://localhost:8080/api/agent/chat \
  -H "Content-Type: application/json" \
  -d '{"message":"我要退款"}'