Use JVM Native Vector API to Remove an External Vector Store in RAG

01 Understand the JVM Vector API

The JDK incubating module jdk.incubator.vector offers SIMD vector operations, but it is still experimental and must be enabled at runtime with --add-modules jdk.incubator.vector.

static float dot(float[] a, float[] b) {
    var s = FloatVector.SPECIES_PREFERRED;
    float sum = 0f;
    int i = 0;
    for (; i < s.loopBound(a.length); i += s.length()) {
        sum += FloatVector.fromArray(s, a, i)
                     .mul(FloatVector.fromArray(s, b, i))
                     .reduceLanes(VectorOperators.ADD);
    }
    for (; i < a.length; i++) {
        sum += a[i] * b[i]; // scalar fallback for tail
    }
    return sum;
}

02 What integrallis/vectors Provides

The integrallis/vectors library wraps the low‑level Vector API into three functional parts:

Distance kernel : ready‑made dot‑product, L2, and cosine calculations that automatically fall back to scalar code when SIMD is unavailable.

Index : supports FLAT (exact, small corpora), HNSW (default for larger sets), IVF_FLAT, IVF_PQ, and quantization options SQ8/SQ4, PQ, RaBitQ.

Persistence : uses MemorySegment and commit() to write vectors to disk, avoiding re‑embedding after a restart.

03 Maven Dependency

<dependency>
    <groupId>com.integrallis</groupId>
    <artifactId>vectors</artifactId>
    <version>0.1.7</version>
</dependency>

04 Configure Index and Persistence

Example configuration (YAML or properties) to switch from the default FLAT index to HNSW and enable automatic commit after each addition:

java-vectors:
  index-type: HNSW
  storage-path: /var/lib/pigai/vectors/kb
  commit-after-add: true

05 Integrate with Spring AI

Spring AI already provides EmbeddingModel, VectorCollection, and VectorStore. After adding the starter dependency, you can autowire the store and use it directly:

@Autowired
VectorStore vectorStore;

public void ingest() {
    vectorStore.add(List.of(
        new Document("浏览器报错 404，请检测您输入的路径是否正确",
            Map.of("author", "lengleng", "product", "PigAI")),
        new Document("host 报错请检查环境",
            Map.of("author", "lengleng", "product", "PigAI"))));
}

public List<Document> ask(String question) {
    return vectorStore.similaritySearch(
        SearchRequest.builder()
            .query(question)
            .topK(5)
            .filterExpression("author == 'lengleng'")
            .build());
}

The filterExpression syntax is identical to that used with pgvector or Qdrant.

06 Connect LangChain4j

Add the LangChain4j starter for vectors:

<dependency>
    <groupId>com.integrallis</groupId>
    <artifactId>vectors-langchain4j</artifactId>
    <version>0.1.7</version>
</dependency>

Build a VectorCollection, create an EmbeddingStore, add a segment, and perform a similarity search with metadata filtering:

VectorCollection collection = VectorCollection.builder()
    .dimension(embeddingModel.dimension())
    .metric(SimilarityFunction.COSINE)
    .indexType(IndexType.HNSW)
    .storagePath(Path.of("/var/lib/pigai/vectors/kb"))
    .build();

EmbeddingStore<TextSegment> store = JavaVectorsEmbeddingStore.builder(collection)
    .commitAfterAdd(true)
    .build();

TextSegment segment = TextSegment.from(
    "PigAI 知识库检索走进程内向量集合",
    Metadata.from("author", "lengleng"));
store.add(embeddingModel.embed(segment).content(), segment);

var matches = store.search(
    EmbeddingSearchRequest.builder()
        .queryEmbedding(embeddingModel.embed("进程内怎么查").content())
        .maxResults(5)
        .filter(metadataKey("author").isEqualTo("lengleng"))
        .build());

07 Required JVM Startup Flags

--add-modules jdk.incubator.vector
--enable-native-access=ALL-UNNAMED

These flags activate the incubating Vector API and allow native memory access needed by MemorySegment.