Introducing Spring AI: A Java Framework for Building Generative AI Applications

Spring AI is a Java AI framework inspired by Python projects LangChain and LlamaIndex, aiming to bring generative AI capabilities to the Java ecosystem and integrate seamlessly with Spring Boot, Spring Cloud, and other Spring components.

The core functions of Spring AI include model management (loading and unloading local or remote models), unified model inference interfaces, extensibility for custom models, and tight integration with the broader Spring ecosystem.

Installation and configuration are straightforward: add the

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-core</artifactId>
    <version>1.0.0</version>
</dependency>

to your Maven project.

To connect a model, configure properties for remote services such as DeepSeek or a local Ollama instance, e.g.:

// DeepSeek configuration
spring.ai.openai.api-key=sk-xxxxxx
spring.ai.openai.base-url=https://api.deepseek.com
spring.ai.openai.chat.options.model=deepseek-chat

// Ollama configuration
spring.ai.ollama.chat.model:deepseek-r1
spring.ai.ollama.base-url:http://localhost:11434

Example code for a minimal chat application includes a configuration class that creates a ChatClient bean and a REST controller that streams responses from the model:

@Configuration
public class CommonConfig {
    @Bean
    public ChatClient chatClient(OllamaChatModel model) {
        return ChatClient.builder(model).build();
    }
}

@RestController
@RequestMapping("/ai")
public class ChatController {
    @Resource
    private ChatClient chatClient;

    @GetMapping(value = "/chat", produces = "text/html; charset=utf-8")
    public Flux<String> chat(String msg) {
        return chatClient.prompt()
                         .user(msg)
                         .stream()
                         .content();
    }
}

Spring AI also provides an implementation of the Model Context Protocol (MCP) as a Java SDK, enabling standardized AI model integration. Adding MCP support involves including the spring-ai-starter-mcp-server-webmvc dependency, defining tool services (e.g., a weather service), registering them, and starting the server.

@Service
public class WeatherService {
    @Tool(description = "Get weather forecast by city name")
    public String getWeatherByCity(@ToolParam(description = "City name") String city) {
        if (Objects.isNull(city)) {
            return "Error: city name cannot be empty!";
        }
        Map<String, String> mockData = Map.of(
            "Xi'an", "Sunny",
            "Beijing", "Light rain",
            "Shanghai", "Heavy rain"
        );
        return mockData.getOrDefault(city, "Error: city not found!");
    }
}

@Bean
public ToolCallbackProvider weatherTools(WeatherService weatherService) {
    return MethodToolCallbackProvider.builder().toolObjects(weatherService).build();
}

Start the MCP server with a standard Spring Boot main method and invoke tools via HTTP using the MCP client:

public static void main(String[] args) {
    SpringApplication.run(ServerApplication.class, args);
}

// Example client call
var client = McpClient.sync(new HttpClientSseClientTransport("http://localhost:8080")).build();
CallToolResult result = client.callTool(new CallToolRequest("getWeatherByCity", Map.of("city", "Shanghai")));
System.out.println("Shanghai weather: " + result);

The article concludes with a link to a community resource for further learning, but the technical content above provides a comprehensive guide to using Spring AI for Java‑based generative AI development.