Can DP‑SGD’s Toughest Clip Threshold Auto‑Adjust? Inside the SlaClip Method

Researchers from the University of Southampton and Guangzhou University introduced SlaClip, an adaptive gradient‑clipping method for differential‑privacy stochastic gradient descent (DP‑SGD), which was accepted as an ICML 2026 Spotlight paper.

DP‑SGD achieves privacy by clipping each per‑sample gradient to a preset ℓ₂ norm threshold and adding calibrated Gaussian noise. Choosing a fixed clipping threshold is difficult: a threshold that is too low over‑clips useful gradients, while a threshold that is too high enlarges the sensitivity bound, allowing noise to drown the signal.

Existing adaptive methods such as Adap‑Clip track the proportion of gradients that are not clipped and adjust the threshold toward a target ratio (e.g., 50%). However, they incur two problems: (1) estimating the unclipped proportion requires additional privacy queries, consuming extra privacy budget or adding more noise; (2) a fixed target ratio may become suboptimal as the gradient‑norm distribution shifts during training, causing the threshold to keep decreasing.

SlaClip asks whether the information needed to adapt the clipping threshold can be obtained without extra privacy queries. Its key observation is that the “slack” – the difference between a gradient’s ℓ₂ norm and the clipping threshold – naturally encodes partial information about the gradient‑norm distribution.

In each iteration, SlaClip appends a K‑dimensional vector to every per‑sample gradient and encodes the slack into this vector. The extended gradient still satisfies the same ℓ₂ norm constraint, preserving the global ℓ₂ sensitivity of DP‑SGD while providing a Slack Indicator that can be used to adjust the clipping bound.

After aggregation, noise addition, and normalization, the Slack Indicator can be interpreted as a noisy, binned estimate of the cumulative distribution function (CDF) of gradient norms. This CDF reveals not only how many gradients are clipped but also how close remaining gradients are to the threshold and the proportion of small‑norm gradients, enabling dynamic adjustment of the target unclipped ratio.

Experimental design follows a fair‑comparison protocol: for each method, dataset, and privacy budget, hyper‑parameters are searched over the same pool via grid search. Results show that SlaClip consistently achieves competitive or best accuracy across multiple datasets and privacy budgets. Heat‑map analyses of learning‑rate and initial‑threshold sensitivity demonstrate that SlaClip yields a wider high‑accuracy region and is less sensitive to hyper‑parameter choices than traditional adaptive clipping methods.

In summary, SlaClip (1) requires no additional privacy queries, (2) is a plug‑and‑play method with low computational overhead, and (3) provides richer information than simple clipped/unclipped statistics, improving stability and performance of DP‑SGD training.