Skip to content
Diffusion vs. Autoregressive LLMs: Which Leaks More Training Data? Sampling Steps Dramatically Increase Memorization Risk

Diffusion vs. Autoregressive LLMs: Which Leaks More Training Data? Sampling Steps Dramatically Increase Memorization Risk

Aalborg University researchers show diffusion language models leak less training data than autoregressive models at equal scale, but finer sampling steps sharply raise exact memorization risk — LLaDA-8B's extraction success jumps 20× when moving from one-step to token-by-token generation.

Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing

From Next-Token Prediction to Arbitrary Infilling: Measuring Memorization in Diffusion LMs

Large language models sometimes reproduce training data verbatim, exposing personal information such as emails and phone numbers. As diffusion language models scale, a key question arises: when generation is no longer left-to-right, does memorization manifest differently? Autoregressive models predict the next token given prefix context; masked diffusion LMs instead fill in masked spans over multiple denoising steps, enabling parallel generation and altering extraction pathways.

Aalborg University's NeurIPS 2026 paper

Characterizing Memorization in Diffusion Language Models: Generalized Extraction and Sampling Effects

(arXiv: https://arxiv.org/abs/2603.02333) systematically investigates this.

Generalized Probabilistic Extraction Framework

Standard memorization evaluation provides a prefix and checks if the model completes it identically. Diffusion models, however, can infill arbitrary masked positions using bidirectional context. Which positions are masked, which tokens are recovered first, and how many per step all affect outcomes. The authors propose a generalized framework: given partially visible text, allow multiple generation attempts and measure the probability of recovering all masked original content. This framework supports arbitrary masking patterns, recovery orders, and subsumes traditional prefix completion.

They validate probabilistic estimation feasibility: on 200 samples, extraction probabilities estimated from 1,024 random mask samples closely match empirical frequencies from 100,000 generations, offering a practical evaluation shortcut.

Figure 2: Probabilistic estimates vs. large-scale generation extraction probabilities (original paper Figure 2).
Figure 2: Probabilistic estimates vs. large-scale generation extraction probabilities (original paper Figure 2).

Why Finer Sampling Steps Increase Leakage

Imagine filling a familiar text: completing all blanks at once gives limited per-position clues; filling a few correctly first provides new context that makes the rest easier. The paper formalizes this: under the assumption that adding correct context never reduces the recovery probability of remaining content, fixing the recovery order and splitting a batch into finer steps does not decrease the probability of full reconstruction.

The condition is critical: the result concerns correct context and specific step refinement, not a universal claim that more steps always mean worse leakage under any sampling strategy.

Experiments confirm the trend. Using 200 qualifying mask configurations, the team tested one-step, two-step, five-step, ten-step, and token-by-token recovery with large-scale generation. As steps become finer, exact reconstruction success rates rise overall.

Figure 3: Finer sampling steps increase exact reconstruction success relative to one-step generation; shaded areas show 95% confidence intervals (original paper Figure 3).
Figure 3: Finer sampling steps increase exact reconstruction success relative to one-step generation; shaded areas show 95% confidence intervals (original paper Figure 3).

Model parameters are unchanged; only the opportunity to extract memorized content varies. Sampling steps thus affect not just speed and quality but also training data extractability.

Diffusion vs. Autoregressive: Controlled Comparison

To compare fairly, the team trained 1.1B-parameter diffusion and autoregressive models with aligned datasets, tokenizers, and compute budgets, then adapted both on the Enron email corpus. Evaluation: given 100 tokens preceding a target email or phone number, the model attempts completion. Prompts containing other email addresses were filtered to reduce extra clues.

Under a 10,000-query budget across 3,000 email completion samples:

At the 50% extraction probability threshold, the diffusion model succeeded on only 16 samples, while the autoregressive model succeeded on 213.

For LLaDA-8B, switching from one-step to token-by-token recovery increased threshold-meeting samples from 9 to 179 — nearly a 20× jump.

Table 1: Samples reaching success probability thresholds under query budget (adapted from original paper Table 1).
Table 1: Samples reaching success probability thresholds under query budget (adapted from original paper Table 1).

These results reflect aligned prefix completion; they do not prove diffusion models are universally safer, especially in their native bidirectional infilling scenarios.

Memorization vs. Generalization: Controlled Comparison

Successful completion doesn't prove memorization — common formats and language patterns could enable guessing. The team compared 10,000 Enron training windows against 10,000 unseen TREC 2007 Spam windows from the same domain. Reconstruction probability distributions for training data were systematically higher, confirming the evaluation captures training-exposure effects.

Figure 4: Reconstruction probability distributions for Enron training data (blue) vs. unseen TREC data (orange) (original paper Figure 4).
Figure 4: Reconstruction probability distributions for Enron training data (blue) vs. unseen TREC data (orange) (original paper Figure 4).

Lower extraction rates do not imply absence of memorization; they may indicate memories are harder to elicit under current prompts and sampling conditions.

Three Implications for Privacy Evaluation

Evaluate generation settings, not just model weights. Identical parameters can exhibit different training data extraction risks under different sampling steps.

Single-query failure ≠ inextractability. Query budget, success probability, and masking strategy must be reported.

Diffusion models' lower leakage warrants further study, but boundaries need verification. Whether the property holds after instruction tuning, preference optimization, and in native bidirectional infilling remains open.

This work introduces an often-overlooked variable into model safety assessment: training determines what the model sees, but the generation process governs how easily that content resurfaces.

Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Sign in to view source
Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactadmin@besthub.devand we will review it promptly.

autoregressive modelsdiffusion language modelsmemorizationNeurIPS 2026privacy leakagesampling stepsAalborg UniversityLLaDA-8B
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.