LangChain4j RAG Chunking Strategies: Recursive Splitting, Overlap Windows & Parent-Child Retrieval

Why Fixed-Size Chunking Fails

Fixed-size chunking (e.g., every 500 characters) arbitrarily severs semantic units. A critical causal logic spanning characters 499–505 gets split across two chunks, destroying the complete meaning. Embedding models then encode fragmented halves, causing retrieval to miss the relevant text entirely.

Three-Tier Semantic Chunking Implementation

Tier 1: Recursive Punctuation-Based Splitting

Instead of rigid character counts, split by natural language boundaries in descending priority: double newlines (paragraphs) → single newlines → Chinese periods (。) → commas. This preserves the "breathing rhythm" of the text and keeps basic business semantics intact.

Tier 2: Tokenizer-Aware Overlapping Windows

Even recursive splitting can cut context at chunk boundaries. Solution: add 10–20% token overlap (e.g., 50 tokens overlap on 500-token chunks). Crucially, splitting must use the same tokenizer as the embedding model (e.g., OpenAI's GPT-4 tokenizer) because 500 Chinese characters may equal 300–600 tokens. LangChain4j's

DocumentSplitters.recursive(maxTokens, overlapTokens, tokenizer)

handles recursive fallback and overlap calculation automatically.

import dev.langchain4j.data.document.Document;import dev.langchain4j.data.document.DocumentSplitter;import dev.langchain4j.data.document.splitter.DocumentSplitters;import dev.langchain4j.model.openai.OpenAiTokenizer;public class DocumentProcessService {  public List<TextSegment> processWithOverlap(Document document) {    // 1. Define tokenizer (OpenAI example; use HuggingFace tokenizer for private deployment)    Tokenizer tokenizer = new OpenAiTokenizer("gpt-4");    // 2. Create recursive splitter with overlap    int maxTokens = 500;     // max tokens per chunk    int overlapTokens = 50;  // ~10% overlap    DocumentSplitter splitter = DocumentSplitters.recursive(      maxTokens,      overlapTokens,      tokenizer    );    // 3. Execute splitting    return splitter.split(document);  }}

Tier 3: Parent-Child Document Mapping

Retrieval faces a dilemma: large chunks dilute vector specificity; small chunks lack context for generation. The parent-child pattern resolves this: small child chunks handle retrieval, large parent chunks provide context to the LLM.

Ingestion: Split source into parent chunks (e.g., by paragraph). Store each parent in Redis with a unique parent_id. Further split each parent into child chunks (e.g., sentences). Embed child chunks and store in Qdrant with parent_id in payload metadata.

Retrieval: Embed user query, search Qdrant for top-k child chunks, extract unique parent_id s, fetch full parent texts from Redis, return assembled parents to the LLM.

Ingestion code:

public void ingestParentChild(String largeText) {  List<String> parentChunks = splitIntoParagraphs(largeText);  for (String parentText : parentChunks) {    String parentId = UUID.randomUUID().toString();    // Store parent in Redis    redisTemplate.opsForValue().set("doc:parent:" + parentId, parentText);    // Split parent into child sentences    List<String> childChunks = splitIntoSentences(parentText);    List<TextSegment> childSegments = new ArrayList<>();    for (String childText : childChunks) {      Metadata metadata = new Metadata();      metadata.put("parent_id", parentId);      childSegments.add(TextSegment.from(childText, metadata));    }    // Embed and store children in Qdrant    embeddingStore.addAll(embeddingModel.embedAll(childSegments).content(), childSegments);  }}

Custom retriever implementing ContentRetriever:

@Component@RequiredArgsConstructorpublic class ParentChildRetriever implements ContentRetriever {  private final EmbeddingStore<TextSegment> qdrantStore;  private final EmbeddingModel embeddingModel;  private final StringRedisTemplate redisTemplate;  @Override  public List<Content> retrieve(Query query) {    // 1. Embed query    Embedding queryEmbedding = embeddingModel.embed(query.text()).content();    // 2. Search child chunks (top 5)    List<EmbeddingMatch<TextSegment>> matches = qdrantStore.findRelevant(queryEmbedding, 5);    // 3. Deduplicate parent_ids    Set<String> parentIds = matches.stream()        .map(match -> match.embedded().metadata().getString("parent_id"))        .collect(Collectors.toSet());    // 4. Fetch full parents from Redis    List<Content> finalContents = new ArrayList<>();    for (String parentId : parentIds) {      String parentText = redisTemplate.opsForValue().get("doc:parent:" + parentId);      if (parentText != null) {        finalContents.add(Content.from(parentText));      }    }    // 5. Return complete, context-rich parents    return finalContents;  }}

Metadata Injection for Global Context

Chunks like "Zhang San sentenced to three years" lose case year, crime type, and source. During extraction (e.g., via Apache NiFi processing PDFs), prepend document title, chapter, and page number to each chunk text or store in metadata. Example enriched chunk:

[《2023 Criminal Law Classic Cases》 - Robbery Chapter - Page 12] Zhang San was sentenced to three years imprisonment.

This gives every chunk standalone global semantics.