How Much Can Large Language Models Remember? ICML 2026 Finds ~3.6 bits per Parameter

The paper asks a fundamental question: how much of the training text truly remains encoded in a model’s parameters? It argues that simply observing a model reproduce a string does not prove memorization, because models can generate unseen text by extrapolating learned patterns.

Non‑expected memory definition : The authors adopt an information‑compression viewpoint. If a reference model approximates the true data distribution, the reduction in coding length of a sample when using the target model measures the amount of sample‑specific information the target has retained. This “non‑expected memory” is estimated via arithmetic coding combined with model likelihood, turning the intractable Kolmogorov complexity into a practical metric.

Experimental setup : Researchers train GPT‑class Transformers of various scales (≈500 K to 1.5 B parameters) on synthetic random sequences, bfloat16 and fp32 precision variants, and real‑world text (FineWeb). They also evaluate member‑inference attacks.

Key findings :

Across all settings, the empirical capacity is about 3.5–3.6 bits per parameter . Linear scaling between capacity and parameter count is observed, with a plateau when data size exceeds the model’s capacity.

On random data, non‑expected memory rises with model size then plateaus, defining the model’s “capacity”.

When training data surpasses capacity, a double‑descent appears: test loss first increases then decreases as the model shifts from memorizing individual samples to learning shared patterns.

Precision matters little for capacity: bfloat16 yields ~3.51 bits/param, fp32 ~3.83 bits/param.

Member‑inference risk scales with the ratio Capacity(θ)/|D|. Larger models memorize more samples, but increasing dataset size reduces per‑sample identifiability, eventually approaching random guessing.

A scaling law predicts member‑inference F1 given model capacity and dataset size; experiments with GPT2‑Medium (≈124 M) and GPT2‑XL (1.5 B) confirm the prediction, with maximal error near the steepest sigmoid region.

Rare or high‑TF‑IDF tokens correlate with higher non‑expected memory, indicating elevated leakage risk for atypical samples.

The authors caution that the 3.6 bits/parameter figure is an empirical lower bound for the specific GPT architecture and training regime, not a universal physical limit. Model capacity depends jointly on parameter count, data scale, sample rarity, and the chosen reference distribution.

Overall, the work reframes the memorization question: instead of asking whether a model memorizes training data, we should quantify *what* information is retained, *how much*, and at which data scales memorization gives way to generalization, with direct implications for privacy‑preserving AI.