Build a Generative AI RAG App with Spring AI in Minutes

Introduction

Spring AI is a Java framework that simplifies the development of applications with generative AI capabilities, avoiding unnecessary complexity. Inspired by Python projects such as LangChain and LlamaIndex, it aims to make AI‑enabled services accessible to developers across many programming languages.

Key Features

Support for major model providers including OpenAI, Microsoft, Amazon, Google, and HuggingFace.

Model types such as chat and text‑to‑image, with more types under development.

Automatic mapping of AI model outputs to POJOs.

Integration with major vector‑store providers like Azure Vector Search, Chroma, Milvus, Neo4j, PostgreSQL/PGVector, Pinecone, Qdrant, Redis, and Weaviate.

Portable API across vector‑store providers, including a SQL‑like metadata filter API.

Function calling support.

Spring Boot auto‑configuration and starters for AI models and vector stores.

ETL framework for data engineering.

These capabilities enable common use cases such as document Q&A and chat interfaces.

Environment Requirements

- SpringBoot 3.2.4
- JDK 21
- Gradle 8.7
- OpenAI API Key
- Docker Compose
- Git

Project Initialization

You can clone a ready‑made demo repository:

git clone https://github.com/flyeric0212/eric-spring-ai-rag-demo

Or create a new project with Spring CLI:

# macOS installation of spring-clirew tap spring-cli-projects/spring-cli
brew install spring-cli

spring boot new --from ai --name eric-spring-ai-rag-demo

Alternatively, use Spring Initializr (https://start.spring.io/) in an IDE such as IntelliJ IDEA.

OpenAI Credential Configuration

Replace the placeholder values with your own OpenAI API key and optional proxy URL:

export OPENAI_API_KEY=sk-1234567890abcdef1234567890abcdef
export OPENAI_BASE_URL=https://api.openai.com/

Spring AI Project Configuration

spring:
  application:
    name: spring-ai-rag-demo

  datasource:
    url: jdbc:postgresql://localhost:5432/postgres
    username: postgres
    password: postgres

  ai:
    openai:
      api-key: ${OPENAI_API_KEY}
      # Note: OpenAiApi will splice "/v1/chat/completions"
      base-url: ${OPENAI_BASE_URL}
      chat:
        options:
          model: gpt-3.5-turbo
          max-tokens: 1024
          temperature: 0.7
      embedding:
        options:
          model: 'text-embedding-ada-002'
    vectorstore:
      pgvector:
        index-type: hnsw
        distance-type: cosine_distance
        dimensions: 1536

Demo Execution

Start the application: ./gradlew bootRun Test OpenAI connectivity:

GET http://localhost:8080/ai/generate
{
  "question": "Tell me a joke",
  "answer": "Why did the scarecrow win an award? Because he was outstanding in his field!"
}

Load data into the vector store (PDF parsing, chunking, embedding): POST http://localhost:8080/data/load Sample log shows PDF pages being processed and document chunks being created before embeddings are stored.

Verify vector data in PGVector database (screenshot omitted).

RAG Query Testing

Default RAG‑enabled query:

GET http://localhost:8080/rag
{
  "question": "What is the largest trend of 2023?",
  "answer": "The largest trend of 2023 is the rise of Generative AI, powered by Large Language Models such as GPT‑3 and GPT‑4..."
}

Chinese example:

GET http://localhost:8080/rag?question=开源大模型食用指南主讲了什么？
{
  "question": "开源大模型食用指南主讲了什么？",
  "answer": "...（详细中文答案）"
}

Disable RAG:

GET http://localhost:8080/rag?rag=false
{
  "question": "What is the largest trend of 2023?",
  "answer": "I'm sorry, but I don't have information on future trends..."
}

Additional Resources

Official Spring AI project page: https://spring.io/projects/spring-ai

RAG and Spring AI tutorial: https://levelup.gitconnected.com/rag-and-spring-ai-querying-your-own-documents-with-open-ai-54b404eb7d08