Diffusion LLMs Leak Less Than Autoregressive Models, But Finer Sampling Steps Increase Extraction Risk
Aalborg University researchers show diffusion language models leak less training data than autoregressive models at equal scale, but increasing sampling steps during generation significantly raises exact memorization extraction risk, as demonstrated on Enron email completion tasks with LLaDA-8B and matched 1.1B models.
Generalized Probabilistic Extraction Framework
To measure memorization in diffusion language models, the authors propose a generalized probabilistic extraction framework: given partially visible text, the model attempts multiple completions, and the probability of recovering all masked original content is measured. This framework supports arbitrary masking positions, different recovery orders, and subsumes traditional prefix continuation as a special case.
Probability Estimation Validation
The feasibility of probability estimation is validated on 200 samples: estimates from 1,024 random mask samples closely match empirical extraction probabilities from 100,000 random generations, providing a practical way to reduce evaluation cost.
Theoretical Characterization of Sampling Steps
Under the assumption that adding correct context does not decrease the recovery probability of remaining content, the authors prove that fixing the recovery order and splitting the original batch into finer steps does not decrease the probability of full recovery. This provides a theoretical basis for the observed increase in extraction success with more sampling steps.
Empirical Results: Sampling Steps Increase Extraction Success
Experiments on 200 mask settings with LLaDA-8B show that as generation steps become finer (1-step, 2-step, 5-step, 10-step, token-by-token), the exact recovery success rate increases overall. The model parameters remain unchanged; only the opportunity to extract memorized content changes.
Comparison with Autoregressive Models
To compare fairly, the team trained a 1.1B-parameter diffusion model and a 1.1B-parameter autoregressive model with aligned dataset, tokenizer, and compute budget. Both were adapted to the Enron email corpus. Given 100 tokens preceding a target email or phone number, models attempted completion under a 10,000-query budget across 3,000 email samples.
Key results: at the 50% extraction probability threshold, the diffusion model succeeded on only 16 samples, while the autoregressive model succeeded on 213 samples. This indicates substantially lower leakage for diffusion models at matched scale.
LLaDA-8B Case Study: Sampling Setting Drastically Changes Risk
For the larger LLaDA-8B model, switching from 1-step recovery to token-by-token recovery increased the number of samples reaching the 50% threshold from 9 to 179 — nearly a 20-fold increase. This demonstrates that sampling configuration alone can create large privacy risk differences for the same model.
Control Experiment: Training vs. Unseen Data
To distinguish memorization from lucky guesses, the authors compared reconstruction probabilities for 10,000 Enron training windows against 10,000 unseen TREC 2007 spam windows. Training data showed systematically higher reconstruction probabilities, confirming that the extraction metric captures memorization effects.
Implications for Privacy Evaluation
Evaluate generation settings, not just model architecture. The same parameters can exhibit different training-data extraction risks under different sampling step counts.
Single-query failure does not guarantee safety. Query budget, success probability, and masking strategy must be reported.
Diffusion models' lower leakage warrants further study. Whether this holds for native bidirectional infilling, instruction tuning, and preference optimization remains to be verified.
The paper concludes that training determines what a model sees, but the generation process determines how easily that content can reappear. The work is by Xiaoyu Luo and Wenrui Yu (co-first authors), Qiongxiu Li (corresponding author), and Johannes Bjerva from Aalborg University. Paper: https://arxiv.org/abs/2603.02333
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
