How Meta’s HSTU Architecture Scales Recommendation Systems Beyond Decades of Deep Models

Motivation

Industrial recommender systems for billions of users face three fundamental bottlenecks: (1) heterogeneous high‑cardinality features lack explicit structure, (2) the item/attribute vocabulary grows to billions of dynamic IDs, and (3) compute cost for training and inference exceeds the budget of the largest LLMs (e.g., GPT‑3).

Generative Recommendation (GR) Formulation

GR reframes both retrieval and ranking as a sequential transduction problem. A user’s entire interaction history—item IDs, timestamps, action types, and slowly changing sparse attributes—is serialized into a single time‑series token sequence. Retrieval learns a probability distribution over the full item vocabulary conditioned on the user representation derived from this sequence. Ranking is performed in a target‑aware fashion by interleaving candidate items into the same sequence; a single forward pass scores all candidates, with a mask distinguishing positive‑item tokens from undefined attribute tokens.

Unified Feature Space

All categorical (sparse) features are merged into the longest time‑series (the main series). Auxiliary slowly‑changing features are compressed to the earliest record of each continuous segment and inserted into the main series, keeping overall length manageable. Dense statistical features are omitted; the model learns implicit statistics end‑to‑end from the long sequence.

HSTU Encoder Architecture

HSTU (Hierarchical Sequential Transduction Unit) follows the Transformer skeleton but replaces the standard QKV self‑attention with three pointwise sub‑layers:

Pointwise projection : a single‑layer MLP with SiLU activation compresses long‑term user history.

Pointwise spatial aggregation : a custom attention‑bias mechanism drops the softmax normalization, preserving the intensity of frequent actions.

Pointwise transformation : a second MLP performs feature crossing and representation conversion.

These changes reduce the number of linear layers from six to two, enable operator fusion, and lower per‑layer memory to ~14 d (bfloat16), allowing deeper networks.

Performance Optimizations

Streaming Training

Standard self‑attention costs O(N²) with sequence length N, which is infeasible for billions of tokens. By training in a generative fashion, the encoder cost is amortized across all candidates, reducing complexity to O(N) per training step.

Sparsity Optimization

A GPU‑friendly attention kernel groups GEMM operations by token density, delivering 2–5× higher throughput. A stochastic‑length (SL) sampler truncates overly long sequences with probability α, cutting token count by up to 84 % while keeping Normalized Entropy (NE) degradation below 0.2 %.

Memory Optimization

Reducing linear layers from six to two and fusing operators halves memory usage, enabling >2× deeper models.

Compute Amortization (M‑FALCON)

M‑FALCON modifies attention masks and bias terms so that m candidates share identical attention operations. Candidates are processed in micro‑batches, making inference cost scale linearly with candidate count. Reported speedups reach up to 700× (285× for very deep models) and 2.48× higher QPS on the same hardware.

Evaluation

Datasets & Metrics

Experiments use public benchmarks (MovieLens, Amazon Reviews) and Meta’s internal streaming logs. Retrieval is measured with log‑perplexity; ranking uses Normalized Entropy (NE) and NDCG@10. Training runs on 64–256 NVIDIA H100 GPUs with up to 100 billion samples.

Results

On public datasets, GR improves NDCG@10 by 20.3 %–65.8 % over SASRec.

Online A/B tests show 12.4 % ranking lift and 6.2 % retrieval lift, for an overall 18.6 % gain.

HSTU achieves 15.2× training and 5.6× inference speedups versus FlashAttention‑2 Transformers, while using 50 % less HBM.

M‑FALCON enables 1.5×–2.48× higher throughput despite a 285× increase in model complexity.

Scaling Law for Recommendation Systems

GR exhibits power‑law scaling of performance with compute, analogous to LLMs. A 1.5 trillion‑parameter model requires ~1000× the training compute of prior DLRMs, matching GPT‑3/LLaMA‑2 regimes. Performance (NE) consistently improves as sequence length and embedding dimension increase.

Conclusion

Meta’s generative recommendation model, powered by the HSTU encoder and M‑FALCON inference, replaces the decade‑old heterogeneous‑feature pipeline, delivering up to 18.6 % online improvement and demonstrating that LLM‑style scaling laws hold for industrial recommender systems. The approach reduces reliance on massive hand‑engineered feature stacks, lowers memory and compute requirements, and provides a concrete path for scaling recommender models to trillion‑parameter regimes.

Code example

增大，也能减少约64%-84%的tokens。the NEs指标不会变差，变差幅度不超过0.002 (0.2%)。
如上图所示，NE变差幅度基本不超过0.002。甚至有些情况下指标还更好(左图alpha=1.8、1.9；右图alpha=1.7的时候)。
下图(a)展示了不同序列长度和alpha下的指标。(b,c) 是相比FlashAttention优化的Transformers，HSTU在训练和推理阶段效率能分别提升15.2x、5.6x倍。此外，由于memory的节省，相比于Transformers，HSTU可以再叠深2层网络。