Breaking the Hourglass Phenomenon of Residual Quantization: Enhancing the Upper Bound of Generative Retrieval

Generative search and recommendation systems increasingly rely on numeric identifiers such as Residual‑Quantized Semantic IDs (RQ‑SID) to improve efficiency and generalization, especially in e‑commerce; however, these identifiers suffer from a "sandglass" phenomenon where middle‑layer codebook tokens become overly concentrated, leading to path sparsity and long‑tail distributions that limit overall performance.

The paper reviews background methods (DSI, NCI, TIGER, GDR, GenRet) and highlights TIGER’s use of residual quantization to capture hierarchical semantic information, which is particularly effective for product‑centric data.

A task definition example illustrates how user attributes, interaction history, and query keywords are used to predict the most likely purchased item using SID‑based models.

The RQ‑VAE process for generating SIDs is described, emphasizing its ability to encode semantic structure via residual quantization.

Extensive visualizations of the sandglass effect show that the second codebook layer contains a dense cluster of tokens, confirmed by low entropy, high Gini coefficient, and large standard deviation, indicating severe imbalance.

Analysis of uniform versus non‑uniform embedding distributions reveals that residual quantization amplifies non‑uniformity in the second layer, creating long‑tail token dominance and sparse routing paths.

Empirical studies across models (LLaMA2, Baichuan2, Qwen1.5) demonstrate that head‑token test sets achieve markedly higher performance than tail‑token sets, confirming the practical impact of the phenomenon.

Two mitigation strategies are proposed: (1) heuristically removing the second layer to eliminate long‑tail effects, and (2) an adaptive top‑K token removal approach that dynamically trims low‑importance tokens; experiments on LLaMA show both methods improve metrics, with the adaptive strategy yielding the best results.

The conclusion summarizes the systematic exploration of RQ‑SID limitations, the identification of the sandglass bottleneck, and the effectiveness of the proposed solutions, providing a foundation for future model optimizations.

Future work includes optimizing SID generation with temporal and statistical features, unifying sparse and dense representations for LLMs, and achieving loss‑less end‑to‑end search pipelines.