How Milvus 3.0 Embeds Fine‑Grained Ranking: L0 Early Prune, L2 Re‑Rank, and Native XGBoost Scoring

Milvus users who enable the L0 XGBoost FunctionChain encounter a bug where search_iterator() returns the first page on the second page; the distance and raw XGBoost error remain correct, but pagination logic fails. The community hypothesizes that the proxy’s search iterator v2 uses the last returned score as last_bound, causing the cursor to point to a model output instead of a proper ANN score.

The official fix rejects the buggy continuation by making function rerank and search iterator mutually exclusive (PR #51334), highlighting a broader design choice: the new Function Chain API replaces the old function_score and ranker parameters, which only supported predefined scoring formulas.

1. Why the old entry was insufficient

The previous post‑processing entry could not express a multi‑stage retrieval pipeline such as computing freshness, combining ANN distance, freshness and popularity scores, optionally applying an external re‑rank model, rewriting the final score, and then sorting and cutting candidates. Because the old entry lacked ordered, composable steps, users often fetched all results to the client for custom processing.

Milvus 3.0’s release notes state the goal is to move the multi‑stage pipeline into the engine, reducing over‑fetching and eliminating dependence on external post‑processing services. A JSON‑string approach was rejected due to late parsing failures, weak typing, SDK inconsistencies, and ambiguous numeric types, leading to the adoption of a structured protobuf FunctionChain definition in schemapb.

The typed chain guarantees deterministic execution order, nested parameter types (bool, int64, double, string, array, object, bytes), explicit field dependencies, and a unified $score semantics that can be overridden by map operators.

2. Three operators, one chain

The first public release provides only three operators: map, sort, and limit. map evaluates an expression and writes the result to a column (temporary variable or system virtual column $score); sort orders by a column with optional tie‑break columns; limit trims candidates with optional offset.

Chains execute exactly as sent—no implicit tail operators are added. Sorting is explicit, and the engine no longer infers direction from the vector metric type; users must declare the order when $score is overridden.

from pymilvus import FunctionChain, FunctionChainStage
from pymilvus.function_chain import col, fn

chain = (
    FunctionChain(FunctionChainStage.L2_RERANK, name="fresh_popular_rerank")
    .map("freshness", fn.decay(col("published_at"), function="exp", origin=current_time, scale=86400, offset=0, decay=0.5))
    .map("$score", fn.num_combine(col("$score"), col("freshness"), col("popularity"), mode="weighted", weights=[0.7, 0.2, 0.1]))
    .map("$score", fn.round_decimal(col("$score"), decimal=4))
    .sort(col("$score"), desc=True, tie_break_col=col("$id"))
    .limit(10)
)

This chain first computes an exponential decay freshness factor, then combines the original score, freshness, and popularity with weighted averaging, rounds the result, sorts descending by the new $score, and finally returns the top ten.

3. L0 and L2: two rescoring stages

L0 runs on the QueryNode at the segment level before merging. Only map is allowed; the system automatically appends a Sort($score desc, $id) reduce contract to guarantee each segment’s output is already sorted. L0 and L2 are mutually exclusive with the legacy boost‑score entry, and attempting to use both yields the error “boost score and L0 rerank function chain cannot be used together”.

L2 runs on the Proxy after all shard results are merged. It supports the full map / sort / limit set and follows the rerankOperator path in search_pipeline.go. Only the ordinary SearchRequest.function_chains path uses L2, and the current release only supports the L2_RERANK stage; L1 is reported as “not supported yet”.

The design rationale is that L0 sacrifices precision for scalability—early pruning reduces network traffic—while L2 prioritizes precision with full‑result re‑ranking. L0’s restriction to map prevents per‑segment sorting that would break global order after merging.

4. Native XGBoost scoring

In the first release, XGBoost is the only model executable at L0. The design splits the workflow into four layers:

Expression layer ( xgboost_expr.go) validates required fields: a model_resource must be provided, output can be raw or default, at least one feature column is required, and the number of features must match the model.

Resource layer registers the model as a FileResource identified by a path ending in .ubj (binary JSON).

Cache layer ( xgboost_cache.go) caches models keyed by {resource.ID}:{resource.Path}, loads lazily, uses singleflight to avoid duplicate loads, and manages lifecycle with lease/refcount.

Bridge layer uses CGO and the Arrow C Data Interface to pass Arrow arrays to C++.

The C++ side ( xgboost_model_c.cpp) parses the UBJ model with nlohmann::json::from_ubjson, supports only reg:squarederror and binary:logistic objectives, and only the gbtree booster (single tree, no parallel trees, no categorical splits). Prediction iterates trees, adds leaf values, applies sigmoid for logistic output, and respects default handling of missing features.

5. Compatibility with the legacy path

The new chain coexists with the old function_score and legacy reranker by translating the legacy entry into an internal chain representation. Validation rules (in function_chain_validator.go) enforce mutual exclusion between function_score and function_chains, reject hybrid search with function chains, limit each stage to a single occurrence, and restrict system inputs/outputs ( $id, $score) at L2.

Column pruning is performed: only columns referenced by the chain are retained, reducing data movement. Input planning walks the required inputs, resolves schema field IDs for non‑system fields, and treats temporary variables as outputs of previous operators.

Conclusion

Milvus 3.0’s Function Chain moves fine‑grained ranking from the client into the engine, introduces a typed, ordered pipeline, and separates early (L0) and post‑reduction (L2) rescoring to balance scale and accuracy. While the design choices are sound, the first draft leaves open issues such as limited XGBoost booster support, lack of cache capacity management, and the incompatibility of hybrid search with function chains. Future releases that solidify UDF support and address these gaps will determine the long‑term robustness of the approach.