Mastering Structured Output in Spring AI: Getting Precise JSON from Large Language Models

Why Structured Output Matters

In intelligent‑agent applications the downstream system expects a program‑friendly JSON object. A weather query should return {"city":"北京","weather":"晴","temperature":22} instead of a natural‑language sentence.

Spring AI Structured Output Mechanism

Spring AI uses the Function Calling infrastructure. When a Java class is supplied to ChatClient, Spring AI generates a JSON Schema that describes field names, types and required constraints. Models that support function calling (e.g., qwen2.5:7b) use the schema to shape their output. The response is automatically deserialized into the target Java object. If deserialization fails Spring AI retries up to three times before throwing an exception.

Basic Example – ChatClient Returns a POJO

1. Define the output entity

package com.badao.ai.output;

public class WeatherInfo {
    private String city;
    private String weather;
    private int temperature;
    // No‑arg constructor, getters and setters required for Jackson
    public WeatherInfo() {}
    public String getCity() { return city; }
    public void setCity(String city) { this.city = city; }
    public String getWeather() { return weather; }
    public void setWeather(String weather) { this.weather = weather; }
    public int getTemperature() { return temperature; }
    public void setTemperature(int temperature) { this.temperature = temperature; }
}

2. Call the model with entity()

package com.badao.ai.service;

import com.badao.ai.config.AgentGraphConfig;
import com.badao.ai.output.WeatherInfo;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.stereotype.Service;

@Service
public class AgentService {
    private final AgentGraphConfig.AgentWorkflow workflow;
    private final ChatClient chatClient;

    public AgentService(AgentGraphConfig.AgentWorkflow workflow, ChatClient chatClient) {
        this.workflow = workflow;
        this.chatClient = chatClient;
    }

    public String ask(String question) {
        return workflow.execute(question);
    }

    public WeatherInfo askWeatherStructured(String city) {
        String prompt = "请查询城市\"%s\"的天气，并以 JSON 格式返回，包含 city、weather、temperature 三个字段。".formatted(city);
        return chatClient.prompt()
                         .user(prompt)
                         .call()
                         .entity(WeatherInfo.class);
    }
}

3. Expose a REST endpoint

@PostMapping("/weather")
public WeatherInfo weather(@RequestBody String city) {
    return agentService.askWeatherStructured(city);
}

The entity() call forces the model to emit a valid JSON object; invalid JSON triggers a retry via RetryTemplate (default three attempts).

Advanced Example – Structured Returns from Tools

Annotate a tool method with @Tool. Spring AI automatically generates a JSON Schema from the method’s return type.

package com.badao.ai.tools;

import com.badao.ai.output.WeatherInfo;
import org.springframework.ai.tool.annotation.Tool;
import org.springframework.ai.tool.annotation.ToolParam;
import org.springframework.stereotype.Component;

@Component
public class WeatherTool {
    @Tool(name = "get_weather", description = "查询指定城市的实时天气，返回结构化数据")
    public WeatherInfo getWeather(@ToolParam(description = "城市名称") String city) {
        WeatherInfo info = new WeatherInfo();
        info.setCity(city);
        info.setWeather("晴");
        info.setTemperature(22);
        return info;
    }
}

The generated schema is sent to the model when the tool is invoked, so the model knows the exact JSON shape to produce.

Tool registration is performed in AgentRagConfig via .defaultTools(weatherTool):

package com.badao.ai.config;

import com.badao.ai.tools.WeatherTool;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.client.advisor.vectorstore.QuestionAnswerAdvisor;
import org.springframework.ai.chat.model.ChatModel;
import org.springframework.ai.vectorstore.SearchRequest;
import org.springframework.ai.vectorstore.VectorStore;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class AgentRagConfig {
    @Bean
    public ChatClient chatClient(ChatModel chatModel, VectorStore vectorStore, WeatherTool weatherTool) {
        return ChatClient.builder(chatModel)
                         .defaultAdvisors(
                             QuestionAnswerAdvisor.builder(vectorStore)
                                 .searchRequest(SearchRequest.builder()
                                     .similarityThreshold(0.7)
                                     .topK(3)
                                     .build())
                                 .build())
                         .defaultTools(weatherTool) // register @Tool
                         .build();
    }
}

In the graph‑agent workflow, when no relevant documents are found the service calls weatherTool.getWeather(city), obtains a WeatherInfo object and formats a textual fallback for the final answer.

Common Issues and Mitigations

entity() throws “No suitable converter” – caused by the model returning invalid JSON or mismatched field names. Mitigation: add explicit format instructions or use an OutputParser.

Tool’s structured fields ignored – the model may overlook the tool description. Mitigation: emphasize in the prompt “please answer strictly using the tool’s returned data”.

Ollama model lacks strict JSON Schema support – qwen2.5:7b has limited schema handling. Mitigation: use BeanOutputParser together with prompt constraints and fallback handling.

Complex nested fields fail deserialization – simple bean deserializer insufficient. Mitigation: apply @JsonNaming or provide a custom deserializer.