Quarkus Embraces AI: Declarative Services, Tools, RAG, and MCP in Java

Introduction to Quarkus

Quarkus is a cloud-native Java framework designed for fast startup, low memory footprint, and GraalVM native compilation. It consolidates common Java ecosystem capabilities — REST, reactive programming, messaging, security, data access — into a consistent extension model where adding an extension and a few configuration lines is sufficient.

With the arrival of AI, Quarkus brings the same philosophy to large language model integration. Instead of building a separate Python service and calling it via HTTP, developers can declare an AI service directly inside a Quarkus application and inject it like any other CDI bean.

Why Quarkus for AI

The core of Quarkus's AI support is the Quarkus LangChain4j extension, which wraps LangChain4j capabilities into Quarkus's extension system. Key features include:

Declarative AI Service : An interface with annotations enables chat, summarization, classification.

Multi-model integration : OpenAI, Azure OpenAI, Ollama, Hugging Face, etc.; switching models is mainly a configuration change.

Tools / Function Calling : The model can invoke your Java methods.

RAG : Combines document retrieval with the LLM so answers are grounded in your own data.

MCP : Uses the Model Context Protocol to discover and call external tools via a standard protocol.

Observability : Logs, metrics, and distributed tracing follow Quarkus's existing Micrometer/OpenTelemetry setup.

Native Image : AI applications can be compiled to GraalVM native images, preserving fast startup.

AI Service, RAG, Tools, and MCP form the four foundational pillars of this capability set.

A typical invocation flow involves the user request, the AI service, optional tool calls, optional RAG retrieval, and the model response. Although the diagram looks complex, the code reduces to a few annotations.

Declarative AI Service: Chat by Writing an Interface

This is the easiest entry point. No manual prompt construction or HTTP client management — just define an interface.

/**
 * Customer assistant that answers user questions in natural language.
 */
@RegisterAiService
public interface CustomerAssistant {

    /**
     * Chat with the user, returning the model-generated reply.
     */
    @SystemMessage("You are an e-commerce support agent. Keep answers concise; do not fabricate uncertain information.")
    String chat(@UserMessage String question);
}

Configuration lives in application.properties. Local development can use Ollama; production can switch to OpenAI without changing Java code:

# Local development with Ollama
quarkus.langchain4j.ollama.chat-model.model-name=llama3.1
quarkus.langchain4j.ollama.chat-model.temperature=0.2

The REST layer injects the assistant like any other service:

/**
 * Exposes a chat endpoint.
 */
@Path("/chat")
public class ChatResource {

    @Inject
    CustomerAssistant assistant;

    /**
     * Receives a user question and returns the AI reply.
     */
    @POST
    public String ask(String question) {
        return assistant.chat(question);
    }
}

At build time Quarkus generates the model invocation, JSON mapping, and CDI wiring — consistent with its compile-time approach.

Tools: Let the Model Call Your Business Methods

Pure chat quickly hits a ceiling. When a user asks "Has my order shipped?", the model doesn't know your warehouse state. Expose business methods as tools; the model decides when to call them.

/**
 * Order query tool, available for the model to invoke during conversation.
 */
@ApplicationScoped
public class OrderTools {

    @Inject
    OrderRepository orderRepository;

    /**
     * Look up logistics status by order number.
     */
    @Tool("Query current logistics status by order number")
    public String findOrderStatus(String orderNo) {
        Order order = orderRepository.findByOrderNo(orderNo);
        if (order == null) {
            return "Order not found";
        }
        return "Order " + orderNo + " current status: " + order.getStatus();
    }
}

Attach the toolbox to the AI service:

/**
 * Order assistant that can query real orders.
 */
@RegisterAiService
public interface OrderAssistant {

    @SystemMessage("""You are an order assistant. Only call the tool when you need to check a real order; do not fabricate logistics information.""")
    @ToolBox(OrderTools.class)
    String chat(String question);
}

When the user says "Check if A2026001 has shipped", the model invokes findOrderStatus, then formulates a natural-language answer. Business rules (inventory deduction, permission checks) remain in Java code; the model only decides when to use a tool and how to phrase the result.

RAG: Retrieve First, Then Speak

Models have a training cutoff and cannot see internal documents. RAG works by chunking documents, embedding them, storing the vectors; at query time, relevant chunks are retrieved and injected into the prompt.

Quarkus allows a custom RetrievalAugmentor CDI bean or the Easy RAG configuration approach. A common pattern:

/**
 * Document Q&A assistant; answers must be based on retrieved content.
 */
@RegisterAiService
@ApplicationScoped
@SystemMessage("You are a document assistant. Prefer answers grounded in retrieved material; if the material lacks the answer, say you don't know.")
public interface DocsAssistant {

    /**
     * Answer a question using the knowledge base.
     */
    String ask(String question);
}

The retriever bean wires an embedding model and an embedding store (Redis, PgVector, Chroma, Neo4j, etc.):

/**
 * Augments the user question with vector search results.
 */
@ApplicationScoped
public class DocsRetrievalAugmentor implements Supplier<RetrievalAugmentor> {

    @Inject
    EmbeddingModel embeddingModel;

    @Inject
    EmbeddingStore<TextSegment> embeddingStore;

    @Override
    public RetrievalAugmentor get() {
        EmbeddingStoreContentRetriever retriever = EmbeddingStoreContentRetriever.builder()
                .embeddingModel(embeddingModel)
                .embeddingStore(embeddingStore)
                .maxResults(3)
                .build();
        return DefaultRetrievalAugmentor.builder()
                .contentRetriever(retriever)
                .build();
    }
}

Document Q&A, internal knowledge bases, and policy lookup all follow this pattern.

MCP: Plug External Capabilities into the Model

Tools are ideal for encapsulating your own code. When tools live in other systems, writing adapters for each becomes tedious. MCP (Model Context Protocol) standardizes this: servers expose tools, clients discover and invoke them.

Quarkus can act as both an MCP server (exposing existing endpoints) and an MCP client (calling external tools). Example client-side usage:

/**
 * Travel assistant that calls an external weather service via MCP.
 */
@RegisterAiService
public interface TravelAssistant {

    @SystemMessage("""
        You are a travel assistant. When users ask about weather,
        first look up the location, then query the temperature.
        Do not answer from memory.
    """)
    @McpToolBox("weather")
    String chat(String question);
}

Configuration points to the MCP server:

quarkus.langchain4j.mcp.weather.transport-type=streamable-http
quarkus.langchain4j.mcp.weather.url=http://localhost:8081/mcp

Local methods use @ToolBox, remote capabilities use @McpToolBox — a unified annotation style that keeps the learning curve low for Java developers.

Cloud-Native Benefits Extend to AI

Adding AI doesn't forfeit Quarkus's original strengths:

Dev UI lets you inspect model configuration, test conversations, and verify tool scanning during development.

Runtime logs, metrics, and traces continue via Micrometer and OpenTelemetry.

Native Image compilation works for AI services, keeping cold starts and memory usage minimal — valuable for serverless, scale-to-zero, and edge scenarios where AI services no longer need to be heavy, always-on processes.

Recommended practical combination for real projects:

Develop with Ollama running a local model to iterate on prompts and tools.

Use RAG for knowledge-intensive questions; avoid stuffing all company docs into the system message.

Implement business actions as Tools, keeping real rules in Java.

Adopt MCP for cross-system capabilities, avoiding per-service glue code.

Before production, finalize model switching, timeouts, rate limiting, and observability.