How JD’s PODM‑MI Model Revolutionizes E‑Commerce Search Ranking

Algorithm engineer Hui Mu at JD Retail shares his journey from theory to practice, highlighting the gap between optimal and suitable solutions in large‑scale e‑commerce.

From Search Optimization to Dynamic Re‑ranking

In the JD main‑site search project, traditional algorithms favored top‑selling items, neglecting long‑tail products. Recognizing that users are in different decision stages—browsing vs buying—the team asked whether algorithms could dynamically understand user intent.

They introduced the PODM‑MI re‑ranking framework, which models user preference with a Gaussian distribution, adjusts diversity‑accuracy trade‑offs via covariance changes, incorporates mutual‑information lower‑bound optimization, and adds a utility‑matrix fusion module to balance product relevance and diversity.

Experiments showed significant improvements in the UC‑VR metric and an annual order increase of over ten million.

Discovering the “Hourglass” Bottleneck in Generative Retrieval

While building semantic identifiers (SID) for billions of products using RQ‑VAE, the team observed a “sand‑glass” distribution: dense middle layer causing low code table utilization and training difficulty.

Analysis revealed that coarse‑grained clustering followed by residual quantization amplified long‑tail effects, creating the bottleneck.

Two lightweight solutions were proposed: (1) remove bottleneck nodes from the middle layer after full SID generation; (2) introduce an adaptive threshold to dynamically prune overly concentrated high‑frequency nodes. Both methods improved offline recall significantly.

Impact and Outlook

The work resulted in a SIGIR paper titled “A Preference‑oriented Diversity Model Based on Mutual‑information in Re‑ranking for E‑commerce Search” and highlighted the importance of bridging business needs with systematic technical solutions.

Artificial Intelligence

E‑commerce search

Re‑ranking models

Generative retrieval

Large‑scale systems