Xiaohongshu Tech REDtech
Jul 29, 2024 · Artificial Intelligence
Scaling Laws for Dense Retrieval: Empirical Study of Model Size, Training Data, and Annotation Quality
The award‑winning study shows that dense retrieval performance follows precise power‑law scaling with model size, training data quantity, and annotation quality, introduces contrast entropy for evaluation, validates joint scaling formulas on MS MARCO and T2Ranking, and uses cost models to guide budget‑optimal resource allocation.
annotation qualitycontrast entropydense retrieval
0 likes · 13 min read