How Milvus 3.0 Packs 1,031 Patches in One Row with Struct + EmbList, Eliminating Chunk Flattening

1. Why flattening chunks is not viable

Before Milvus 3.0, handling a document with many token vectors followed three paths: flattening each token vector into separate rows (which breaks entity boundaries and requires de‑duplication), serializing the whole vector set into a JSON field (which prevents the vector engine from indexing the structure and adds costly deserialization), or using named vectors (which each need an independent index and still require a post‑processing join). Milvus maintainer xiaofan‑luan confirmed the memory and compute overhead of ColBERT‑style models in issue #31920.

2. Why Struct lives only in ARRAY element_type

The design doc docs/design-docs/design_docs/20260306-struct.md states that Struct can only be used as the element_type of a DataType.ARRAY. The user doc array-of-structs.md reinforces that a StructArray cannot be added to an existing collection and that the field is stored as a regular ARRAY<T> column. Each sub‑field becomes an independent column, preserving existing columnar storage without introducing a new physical type.

3. EmbList + MAX_SIM for late interaction

Milvus introduces ten placeholder types (e.g., EmbListFloatVector) in plan.proto. The function is_emb_list_placeholder checks whether the placeholder belongs to the EmbList family. In Plan.cpp, if the field data type is VECTOR_ARRAY, the metric type must match the placeholder type; otherwise a DataTypeInvalid error is thrown. This drives the three logical branches:

MAX_SIM metric + EmbList placeholder → row‑level (embedding‑list) search.

Non‑MAX_SIM metric + normal placeholder → element‑level search.

Metric and placeholder mismatch → error.

MAX_SIM metrics (FLOAT, COSINE, IP, L2, etc.) are defined in internal/core/src/common/Types.h and supported by existing single‑vector indexes (AUTOINDEX, HNSW, IVF_FLAT, DISKANN). The MAX_SIM formula is score(q, d) = Σ_{i=1..m} max_{j=1..n} sim(e_qi, e_dj), which pushes the late‑interaction aggregation down to the distance layer, eliminating the need for a separate re‑rank step.

4. Row‑level vs element‑level identity

The hybrid search classifier in internal/proxy/struct_hybrid_search.go distinguishes three kinds: hybridSubSearchStructEmbList – embedding‑list (row‑level) search. hybridSubSearchStructElement – element‑level search. hybridSubSearchNormal – normal scalar/vector fields.

Row‑level search returns a single (primary_key) identity per document, while element‑level search returns a triple (primary_key, parent_struct_field, element_index). Hybrid searches lock to row‑level if any sub‑search uses EmbList.

5. Match family and same‑element constraints

Scalar sub‑fields inside a struct array (e.g., clip_id, element_embedding) support nested filtering via ElementFilterExpr and MatchExpr defined in plan.proto. The element_filter must appear at the end of the expression tree; otherwise entity‑level filters are applied before element filters, causing semantic misalignment.

6. The six‑row offset bug

Issue #51414 reproduces a silent error where a nullable VECTOR_ARRAY field loaded as a sealed segment misaligns offsets, returning incorrect IDs ( [3, 4] instead of [2, 3, 5]). The root cause is an off‑by‑one in ArrayOffsets.cpp:BuildFromColumn that reports row_count=6, total_elements=3, leading to truncated element data during brute‑force search. Workarounds include avoiding nullable struct arrays in sealed segments, using indexed paths (HNSW/IVF) instead of brute‑force, or replacing null rows with empty arrays.

7. Takeaways

Zero storage changes : Struct is stored as a plain ARRAY<T>, and EmbList flattens vectors with offsets, reusing existing knowhere indexes.

Late interaction in the search path : MAX_SIM + EmbList pushes token‑by‑token max‑similarity aggregation to the distance layer, a design shared by Qdrant and LanceDB.

Candidate identity split : Row‑level searches yield document‑level IDs; element‑level searches yield (pk, parent_field, element_index). Hybrid searches are locked to row‑level if any sub‑search uses EmbList.

The six‑row bug illustrates the trade‑offs of extending a mature system; Milvus 2.6.4 introduced Array‑of‑Structs + MAX_SIM, while 3.x continues to refine nullable vector ID mapping (see issues #51381, #49999).

Note : This analysis is based on Milvus 3.0 source code, design documents, and GitHub issues. Benchmarks are not included.