Enhancing Recommendation Models with Scaling Law via HCNet and MemoNet: A Memory‑Based Feature‑Combination Approach

The talk begins by highlighting the scaling‑law property of large language models (LLMs), where model performance continuously improves as parameters increase, a property absent in current recommendation models that are over‑parameterized yet show limited gains from larger embeddings.

Inspired by LLMs, the authors introduce an independent memory mechanism—HCNet—to store, learn, and recall any feature‑combination representation. HCNet uses a Multi‑Hash Codebook to map high‑order feature combinations to codewords, followed by three stages: multi‑hash addressing, memory restoring (Linear Memory Restoring and Attentive Memory Restoring), and feature shrinking via Global Attentive Shrinking to keep the output dimension comparable to first‑order features.

MemoNet integrates HCNet with a DNN backbone, feeding both first‑order embeddings and recovered high‑order embeddings into the network. Experiments on Avazu, KDD12, and Criteo datasets show that increasing codebook size yields consistent AUC improvements, confirming that MemoNet exhibits scaling‑law behavior.

Additional studies examine the impact of hash‑function count, compare LMR vs. AMR strategies (AMR excels with few codewords, LMR with many), and evaluate key‑feature identification methods (Feature Number Ranking and Field Attention Ranking) that reduce memory usage while preserving most of the performance gain.

Online A/B testing demonstrates a 9.73% CTR lift and 6.86% increase in user dwell time when HCNet is deployed as a plug‑in, with a 10.33% reduction in response latency using only four key features.

The authors conclude that HCNet provides an efficient, flexible memory module for CTR prediction, MemoNet shows preliminary scaling‑law properties, and future work will explore more powerful memory mechanisms and richer feature‑combination modeling.