Why Milvus 3.0 Pagination Breaks After Reranking: When Cursor Scores Cross Worlds

An issue (#51306) reported that when a Milvus collection uses an L0 XGBoost Function Chain for reranking, the search_iterator() returns identical IDs on the second page. The scores are numerically correct (within 1e-5 of local XGBoost predictions), yet pagination repeats the first page.

01 How the Cursor Enables Pagination

According to the Milvus documentation ( with-iterators.md), a single query returns at most 16,384 entities; for larger topK the SearchIterator should be used, looping next() with a batch_size. The documentation does not explain how the next page's starting point is determined. Community root-cause analysis in PR #51334 reveals the mechanism: internal/proxy/task_search.go 's getLastBound reads the Scores field of the previous page's last result and uses that score as last_bound for the next request.

The next ANN search carries this last_bound to segcore, where CachedSearchIterator::IsValid filters candidates with dist > last_bound (details per PR #51334 analysis; not line-by-line verified).

In plain terms: the system treats the last result's score as a gatekeeper, allowing only candidates with a higher score (i.e., better match) on the next page. This relies on two implicit assumptions:

Scores are monotonic (earlier results have higher scores).

The score from the previous page and the distance computed on the next page are the same kind of quantity in the same dimension —both are ANN distances (COSINE in [-1,1], L2 in [0,+∞)).

These assumptions hold by default, so the cursor works in a closed loop where scores are comparable.

02 $score Is Designed to Be Rewritten

Milvus 3.0's Function Chain (design draft 2026-06-24) introduces a system virtual column $score ("not a collection field"). It is readable and writable: at runtime it is initialized from the current result's distance; map("$score", expr) can overwrite the entire score register; sort orders by the rewritten score; finally the rewritten $score is serialized back into the result's score / distance field, so SDK users still see a normal hit distance.

Serializing back into the distance field is the trigger : upstream changes $score and puts it back into the distance slot. What the user thinks is a distance may not be a distance at all. It can be a weighted combination (design doc example):

.map("$score", fn.num_combine(col("$score"), col("freshness"), col("popularity"), mode="weighted", weights=[0.7, 0.2, 0.1]))

It can be a decay score; in the XGBoost scenario it is directly the model output. The XGBoost design doc ( 20260708-xgboost-function-chain.md) states: "use native model prediction as the rerank score", and with output=raw no transformation is applied—the raw tree margin is returned. After reranking, the "distance" you receive has a dimension determined by your model, completely detached from the vector index's metric.

The design doc further states: "Milvus does not infer ordering direction from vector metric type after a chain sort is present"—once a sort appears in the chain, sort direction is no longer inferred from the vector metric. This confirms that score semantics being user-defined is an accepted design fact, not a bug byproduct.

The rewritten score is consumed as the sorting key at both L0 (QueryNode, per segment, only map allowed, auto-appends Sort($score desc, $id asc) reduce contract via appendL0RerankReduceContract in l0_function_chain.go) and L2 (Proxy, after global reduce, rerankOperator executes user chain map / sort / limit per search_pipeline.go). In both layers the final user-visible score is the rewritten value. This leads to a key judgment: score is an interface, not an implementation detail . Milvus no longer promises that score equals ANN distance.

03 Failure Chain: Cursor Reads a Score from the Wrong World

Combining the previous two sections explains the bug. In the scenario of issue #51306 (L0 XGBoost rerank):

First page finishes; the iterator takes the last result's score as last_bound (per PR #51334). That score is an XGBoost raw prediction, magnitude ~20.

The next page's ANN continuation search operates in the COSINE space [-1,1] (per PR #51334 analysis).

The continuation filter dist > last_bound (i.e., dist > 20) is evaluated in a space whose maximum is 1. The condition excludes nothing—PR #51334 calls it "the continuation filter excludes nothing". ANN returns the same top batch again; after reranking the same first page is reproduced.

The cursor faithfully executes its instruction; its only mistake is treating a score from a new world as a coordinate in the old world. The mechanism is correct; the input is wrong.

The control group in the issue reproduction makes this clear: the only variable changed is the score in the results (without rerank, two pages are disjoint; with rerank, they are identical, issue #51306).

This is not limited to L0. PR #51334 analysis shows L2 function chain, function_score rerank, and iterator v1 (client-side pagination on returned distances) all suffer the same class of pollution. Any scenario where the cursor takes a boundary from a result score that has been rewritten will fail.

04 Why Milvus Chose to Reject the Combination Rather Than Fix Pagination

Given the clear root cause, why not fix the cursor to handle rewritten scores? The community did propose a proper fix in PR #51334: correctly supporting "rerank + iterator" requires carrying both the original ANN distance and the rewritten score through the entire reduce chain—a "feature-level change for #51192". The short-term alternative is simpler: explicitly reject the combination in the Proxy, making function_chains or function_score with iterator return ParameterInvalid instead of silently returning duplicate pages.

Milvus chose the latter, codifying it in three consistent places: Search API combination limits, L1 compatibility notes, and validation rule 18—all stating that Function rerank and Search Iterator (legacy or v2) are mutually exclusive, L1 likewise, and neither supports order_by; L1 rejects order_by because "both define ordering behavior". Implementation landed in PR #51347 (merged 2026-07-17), point 2: "search iter v2 reject chain", modifying internal/proxy/task_search.go.

Note the sequence: first came issue #51306 (silent duplicate pages), then PR #51347 (explicit rejection). The same combination went from an undetectable bad behavior to a clear "not allowed".

The author argues this is not laziness. Fixing pagination would mean changing the reduce contract to propagate original distances alongside rewritten scores, altering the merge semantics of the entire Function Chain feature. This trade-off aligns with the design doc's emphasis on "explicit over implicit": the chain executes exactly what the user specifies; the system does not secretly inject limit, group_by, or similar public operators (the internal normalization Sort auto-appended at L0/L1 is an internal merge behavior, not a user-chain operator), so it also won't secretly paginate for you. Rejecting the combination is a one-line validation, semantically self-consistent, and aligns with existing guards for group_by / offset / order_by. This is a responsible decision, not a technical retreat.

Deeper down, two generations of design hold opposite attitudes toward "score comparability". The old hybrid search rerank system was extremely wary of incomparable scores: WeightedRanker explicitly noted different score distributions (IP can reach [-∞,+∞], L2 is [0,+∞)) and required arctan normalization to [0,1] before comparison; decay rerank multiplied normalized similarity by decay score; RRF ignored scores entirely, using only ranks (metric-agnostic). The old world's solution to "scores not comparable" was to bypass: normalize or don't read scores. Function Chain does the reverse—it opens scores for user rewriting. Pagination happens to be the system corner most dependent on the "scores are comparable" assumption. Opposite directions colliding at pagination was almost inevitable.

The story continues. In issue #52319 (2026-08-07) maintainers state the current status as "function chains currently reject search-iterator-v2 and order_by ", and list "define the intended semantics instead of blanket rejection" as future work—acknowledging the current blanket rejection and leaving "what pagination score semantics should be" for the future. (L1 was merged 2026-08-21 via PR #52376, but the design doc's combination restrictions for L1 × iterator and L1 × order_by remain.)

05 Boundaries: What You Can and Cannot Combine

Converging the constraints into an actionable checklist.

Forbidden Combinations (explicit in design doc and merged implementation):

Function rerank ( function_chains, and SDK ranker corresponding to function_score) × search_iterator (legacy and v2): rejected.

Function rerank × order_by: rejected, because both define ordering behavior.

L1 × search_iterator, L1 × order_by: similarly rejected.

The commonality is clear: search_iterator and order_by are interfaces that consume ordering semantics . The cursor needs the previous page's score as the next page's coordinate; order_by needs the score as a sort key. Both assume "score is a controllable, comparable quantity". Once Function Chain rewrites the score into user-defined semantics, these interfaces lose their footing. By contrast, the design doc allows group_by with L1 map / sort / limit ("Search-level group-by may be combined with L1 map, sort, or limit, matching L2 compatibility"). The reason (inferred) is that group_by does not consume the same ordering assumption—the doc only gives the combination matrix, no explanation; the same rejection decision incidentally draws a clear line between interfaces that consume scores and those that don't.

Viable Paths , by evidence level:

If your result volume stays within the single-search limit, use normal search which runs the rerankOperator reduce → rerank → pick pipeline. offset is applied in the reduce phase ( search_pipeline.go comment: "Search performs Offset in the reduce phase"). It does not rely on previous-page scores as coordinates, so it is immune to this failure. This is based on source-code and doc reading, not tested in your environment.

If you truly need to page through a large collection (>16,384, the very reason iterator exists), the current master forbids function_chains with iterator. Options: (a) drop server-side rerank, use plain iterator to fetch candidates, then rerank and paginate in your application layer; (b) keep rerank but constrain it to a volume that fits in a single search, avoiding cross-page cursor dependence. These are suggestions based on source and docs, not tested conclusions.

If you receive ParameterInvalid instead of duplicate pages, your version already includes PR #51347's rejection logic. This is good news: the system surfaces "score semantics undefined" explicitly rather than letting you debug duplicate data for hours.

Finally, back to that 1e-5. The most striking line in issue #51306: scores are all correct, pages are all duplicates. Two individually correct components composed to produce a wrong result—what's missing isn't a bug-fix patch, but a contract on score semantics: who guarantees that the pagination cursor receives a quantity it recognizes as comparable? Milvus 3.0 makes an explicit choice on that contract: after reranking, no guarantee. So don't rely on automatic pagination after reranking—that is the most valuable legacy of this bug.