Fractional-Order Differential Privacy

DP-SGD is a standard tool for training machine learning models with differential privacy, but the Gaussian noise required to protect individual records can destabilize optimization and significantly reduce accuracy, especially under tight privacy budgets and small sampling rates. We propose Fractional-DP-SGD, a simple plug-in modification that preserves the core DP-SGD privacy mechanism—per-example gradient clipping and Gaussian noise injection—while changing only the update dynamics. Instead of updating parameters directly with the current noisy gradient, Fractional-DP-SGD applies a finite-window, normalized fractional-memory filter to the sequence of previously released DP gradients. This introduces a power-law temporal aggregation that behaves like a long-memory smoother, shaping the injected noise over time and potentially reducing the effective variance of parameter updates while keeping privacy accounting unchanged. We provide a formal privacy guarantee using the post-processing property of differential privacy together with standard Rényi DP accounting under subsampling, and we analyze the additional computational and memory costs introduced by the finite window. Finally, we outline an extensive experimental protocol comparing Fractional-DP-SGD against DP-SGD and widely used DP optimizers such as DP-Adam, DP-Adagrad, and DP-FTRL across multiple datasets and models. Our evaluation includes systematic ablations over fractional order, window length, kernel family, clipping norm, and noise multiplier to isolate when and why fractional memory improves stability and utility at matched privacy.