Differentially Private Stochastic Gradient Descent (DP-SGD) is a widely used approach to protecting privacy when training AI models. During training, this method adds carefully calibrated random noise, making it harder to determine whether a particular individual's data was included in the training dataset. However, there is a trade-off: adding more noise strengthens privacy protection but can also reduce the accuracy of the resulting model.
Lower noise limit for privacy
The researchers show mathematically that DP-SGD has a fundamental limitation under standard worst-case assumptions about potential attackers. They derive a lower bound on the amount of noise required to provide strong privacy guarantees. Although this lower bound decreases as the number of training updates increases, it does so extremely slowly: even with very large numbers of updates, the required noise remains substantial. Their experiments confirm that these noise levels can significantly reduce model accuracy under realistic training conditions.

