Skip to content
Diffusion LLMs Leak Less Than Autoregressive Models, But Finer Sampling Steps Increase Extraction Risk

Diffusion LLMs Leak Less Than Autoregressive Models, But Finer Sampling Steps Increase Extraction Risk

Aalborg University researchers show diffusion language models leak less training data than autoregressive models at equal scale, but increasing sampling steps during generation significantly raises exact memorization extraction risk, as demonstrated on Enron email completion tasks with LLaDA-8B and matched 1.1B models.

Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing

Generalized Probabilistic Extraction Framework

To measure memorization in diffusion language models, the authors propose a generalized probabilistic extraction framework: given partially visible text, the model attempts multiple completions, and the probability of recovering all masked original content is measured. This framework supports arbitrary masking positions, different recovery orders, and subsumes traditional prefix continuation as a special case.

Probability Estimation Validation

The feasibility of probability estimation is validated on 200 samples: estimates from 1,024 random mask samples closely match empirical extraction probabilities from 100,000 random generations, providing a practical way to reduce evaluation cost.

Figure 2: Probability estimation vs. actual extraction probabilities (original paper Figure 2).
Figure 2: Probability estimation vs. actual extraction probabilities (original paper Figure 2).

Theoretical Characterization of Sampling Steps

Under the assumption that adding correct context does not decrease the recovery probability of remaining content, the authors prove that fixing the recovery order and splitting the original batch into finer steps does not decrease the probability of full recovery. This provides a theoretical basis for the observed increase in extraction success with more sampling steps.

Empirical Results: Sampling Steps Increase Extraction Success

Experiments on 200 mask settings with LLaDA-8B show that as generation steps become finer (1-step, 2-step, 5-step, 10-step, token-by-token), the exact recovery success rate increases overall. The model parameters remain unchanged; only the opportunity to extract memorized content changes.

Figure 3: Finer sampling steps increase exact recovery success rate relative to 1-step generation; shaded areas show 95% confidence intervals (original paper Figure 3).
Figure 3: Finer sampling steps increase exact recovery success rate relative to 1-step generation; shaded areas show 95% confidence intervals (original paper Figure 3).

Comparison with Autoregressive Models

To compare fairly, the team trained a 1.1B-parameter diffusion model and a 1.1B-parameter autoregressive model with aligned dataset, tokenizer, and compute budget. Both were adapted to the Enron email corpus. Given 100 tokens preceding a target email or phone number, models attempted completion under a 10,000-query budget across 3,000 email samples.

Key results: at the 50% extraction probability threshold, the diffusion model succeeded on only 16 samples, while the autoregressive model succeeded on 213 samples. This indicates substantially lower leakage for diffusion models at matched scale.

Table 1: Number of samples reaching extraction probability thresholds under 10,000-query budget (original paper Table 1).
Table 1: Number of samples reaching extraction probability thresholds under 10,000-query budget (original paper Table 1).

LLaDA-8B Case Study: Sampling Setting Drastically Changes Risk

For the larger LLaDA-8B model, switching from 1-step recovery to token-by-token recovery increased the number of samples reaching the 50% threshold from 9 to 179 — nearly a 20-fold increase. This demonstrates that sampling configuration alone can create large privacy risk differences for the same model.

Control Experiment: Training vs. Unseen Data

To distinguish memorization from lucky guesses, the authors compared reconstruction probabilities for 10,000 Enron training windows against 10,000 unseen TREC 2007 spam windows. Training data showed systematically higher reconstruction probabilities, confirming that the extraction metric captures memorization effects.

Figure 4: Reconstruction probability distributions for Enron training data (blue) vs. unseen TREC data (orange) (original paper Figure 4).
Figure 4: Reconstruction probability distributions for Enron training data (blue) vs. unseen TREC data (orange) (original paper Figure 4).

Implications for Privacy Evaluation

Evaluate generation settings, not just model architecture. The same parameters can exhibit different training-data extraction risks under different sampling step counts.

Single-query failure does not guarantee safety. Query budget, success probability, and masking strategy must be reported.

Diffusion models' lower leakage warrants further study. Whether this holds for native bidirectional infilling, instruction tuning, and preference optimization remains to be verified.

The paper concludes that training determines what a model sees, but the generation process determines how easily that content can reappear. The work is by Xiaoyu Luo and Wenrui Yu (co-first authors), Qiongxiu Li (corresponding author), and Johannes Bjerva from Aalborg University. Paper: https://arxiv.org/abs/2603.02333

Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Sign in to view source
Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactadmin@besthub.devand we will review it promptly.

autoregressive modelsdiffusion language modelsmemorizationNeurIPS 2026privacy leakagesampling stepsAalborg UniversityLLaDA-8B
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.