How Generative Pretraining Overcomes Discriminative Model Bottlenecks in Ad Ranking

The article presents UserLLM, a three‑stage framework—generative pre‑training, discriminative SFT, and CTR fusion—that leverages ultra‑long user behavior sequences to address the compression and scaling limits of traditional discriminative ranking models, demonstrating significant offline gains and validated scaling‑law behavior across token, layer, and width dimensions.

UserLLMad rankinggenerative pretraining

0 likes · 17 min read

How Generative Pretraining Overcomes Discriminative Model Bottlenecks in Ad Ranking

Alibaba International Intelligent Technology

Nov 21, 2025 · Artificial Intelligence

Can Recommendation Ranking Models Follow the LLM Scaling Law? Introducing GPSD for Scalable Transformers

The paper analyzes why enlarging discriminative recommendation models leads to severe over‑fitting, proposes a two‑stage Generative Pretraining for Scalable Discriminative Recommendation (GPSD) that freezes sparse embeddings, and demonstrates across massive e‑commerce data, public benchmarks, and production A/B tests that the approach restores the classic scaling‑law behavior where larger models consistently improve performance.

generative pretrainingonline A/B testrecommendation

0 likes · 15 min read

Can Recommendation Ranking Models Follow the LLM Scaling Law? Introducing GPSD for Scalable Transformers