Build Java LLM Applications with LangChain4j: A Hands‑On Guide

1. Introduction

In this tutorial we explore LangChain , a framework for developing applications powered by large language models (LLMs). We start by reviewing basic LLM concepts that will help you follow the rest of the guide.

2. Background

2.1. Large Language Models

LLMs are probabilistic models of natural language that can generate sequences of words. They are typically massive neural networks with billions of parameters, pretrained on vast amounts of unlabelled data using self‑supervised and weakly supervised learning, then fine‑tuned or prompt‑engineered for specific tasks.

LLMs can perform translation, summarisation, content generation, and many other NLP tasks, making them useful for question answering and similar applications.

2.2. Prompt Engineering

Prompt engineering is a fast way to guide an LLM to perform a specific task by providing a structured textual description of the goal. It enables contextual learning, improves safety, and allows integration of domain knowledge and external tools.

Techniques such as chain‑of‑thought prompting break a problem into intermediate steps before producing a final answer.

2.3. Word Embeddings

Representing words as real‑valued vectors (embeddings) captures semantic meaning and improves model performance. Common algorithms include Word2Vec and GloVe.

Embeddings can be stored in vector databases for semantic search, providing richer context to LLMs.

3. Building an LLM Stack with LangChain

Effective prompting, contextual awareness, and model reasoning are key to leveraging LLMs. LangChain offers modules for templating prompts, calling models, and feeding user‑specific data from various sources.

The framework also supports chaining multiple models and recalling past interactions.

4. LangChain for Java (LangChain4j)

LangChain was launched in 2022 as a Python project and later expanded to JavaScript/TypeScript. Although there is no official Java version, the community provides LangChain4j , compatible with Java 8+ and Spring Boot 2/3.

Dependencies can be added from Maven Central:

<dependency>
  <groupId>dev.langchain4j</groupId>
  <artifactId>langchain4j</artifactId>
  <version>0.23.0</version>
</dependency>

5. LangChain Building Blocks

5.1. Model I/O

LangChain provides templated prompts and dynamic model input handling, as well as output parsers.

PromptTemplate promptTemplate = PromptTemplate.from("Tell me a {{adjective}} joke about {{content}}..");
Map<String, Object> variables = new HashMap<>();
variables.put("adjective", "funny");
variables.put("content", "computers");
Prompt prompt = promptTemplate.apply(variables);

5.2. Memory

Memory stores past interactions so the model can reference earlier conversation turns.

ChatMemory chatMemory = TokenWindowChatMemory.withMaxTokens(300, new OpenAiTokenizer(GPT_3_5_TURBO));
chatMemory.add(userMessage("你好，我叫 Kumar"));
AiMessage answer = model.generate(chatMemory.messages()).content();
System.out.println(answer.text()); // 你好 Kumar！今天我能为您做些什么？
chatMemory.add(answer);

5.3. Retrieval

Retrieval‑augmented generation (RAG) fetches relevant external data and feeds it to the LLM.

Document document = FileSystemDocumentLoader.loadDocument("simpson's_adventures.txt");
DocumentSplitter splitter = DocumentSplitters.recursive(100, 0, new OpenAiTokenizer(GPT_3_5_TURBO));
List<TextSegment> segments = splitter.split(document);
EmbeddingModel embeddingModel = new AllMiniLmL6V2EmbeddingModel();
List<Embedding> embeddings = embeddingModel.embedAll(segments).content();
EmbeddingStore<TextSegment> store = new InMemoryEmbeddingStore<>();
store.addAll(embeddings, segments);
Embedding questionEmbedding = embeddingModel.embed("Who is Simpson?").content();
List<EmbeddingMatch<TextSegment>> relevant = store.findRelevant(questionEmbedding, 3, 0.7);

6. Complex LangChain Applications

6.1. Chains

Chains sequence multiple components, simplifying development, debugging, and maintenance.

ConversationalRetrievalChain chain = ConversationalRetrievalChain.builder()
  .chatLanguageModel(chatModel)
  .retriever(EmbeddingStoreRetriever.from(embeddingStore, embeddingModel))
  .chatMemory(MessageWindowChatMemory.withMaxMessages(10))
  .promptTemplate(PromptTemplate.from("Answer the following question ... {{question}}

Base your answer on the following information:
{{information}}"))
  .build();
String answer = chain.execute("Who is Simpson?");

6.2. Agents

Agents treat the LLM as a reasoning engine, deciding which actions to take and in what order, often with tool access.

public class AIServiceWithCalculator {
  static class Calculator {
    @Tool("Calculates the length of a string")
    int stringLength(String s) { return s.length(); }
    @Tool("Calculates the sum of two numbers")
    int add(int a, int b) { return a + b; }
  }
}
interface Assistant { String chat(String userMessage); }
Assistant assistant = AiServices.builder(Assistant.class)
  .chatLanguageModel(OpenAiChatModel.withApiKey(OPENAI_API_KEY))
  .tools(new Calculator())
  .chatMemory(MessageWindowChatMemory.withMaxMessages(10))
  .build();
String answer = assistant.chat("What is the sum of the numbers of letters in the words \"language\" and \"model\"?");
System.out.println(answer);

7. Conclusion

This tutorial covered the essential elements for building LLM‑driven applications, highlighted the value of LangChain as a technology stack, and demonstrated core components of the Java implementation LangChain4j.

Translation: JavaGuide Original source: www.baeldung.com/java-langchain-basics