Pith. sign in

REVIEW 4 major objections 4 minor 55 references

D2P2-SGD: private training at non-private SGD speed

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

D2P2-SGD combines time-decreasing privacy noise with random projection to improve the accuracy of differentially private SGD, with convergence rates matching ordinary SGD.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection The random-projection DP idea is worth engaging, but the privacy proof's sensitivity calculation ignores the projection and the theorem as written does not go through. the 4 major comments →

arxiv 2509.09485 v2 pith:N3ZMG22J submitted 2025-09-11 cs.LG

Balancing Utility and Privacy: Dynamically Private SGD with Random Projection

Zhanhong Jiang , Md Zahid Hasan , Nastaran Saadati , Aditya Balu , Chao Liu , Soumik Sarkar This is my paper
classification cs.LG MSC 68P2790C1590C2590C26
keywords differential privacystochastic gradient descentrandom projectiondynamic privacy noisegradient clippingconvex optimizationnon-convex optimizationsub-linear convergence
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces D2P2-SGD, a stochastic optimizer that trains models under differential privacy by combining three mechanisms: per-sample gradient clipping that needs no threshold tuning, a Gaussian noise scale that decreases over the course of training, and random projection of gradients into a low-dimensional space before noise is added. The authors claim this design preserves the (ε,δ)-differential privacy guarantee while achieving sub-linear convergence rates of O(1/√K + lnK/K^{1.5}) for both convex and non-convex objectives, matching the best available rate of standard SGD. If correct, the method would let practitioners train private models with less accuracy loss than static-noise differentially private SGD, and with noise cost tied to the projected dimension rather than the full parameter dimension. The paper also reports experiments on several image datasets suggesting the optimizer outperforms static and dynamic baselines.

Core claim

The central claim is that dynamic privacy noise and random projection can be combined without sacrificing the convergence rate of SGD. Theorem 1 states that D2P2-SGD is (ε,δ)-differentially private whenever the base noise variance satisfies σ²_ε ≥ C₂ B² K² ln(1/δ)/(n² ε²), with B the mini-batch size, n the dataset size, and K the number of iterations. Theorems 2 and 3 respectively give expected suboptimality bounds for convex and non-convex objectives; with step size α = O(1/√K), these become O(1/√K + lnK/K^{1.5}) for convex functions, and the same rate plus a clipping-bias term for non-convex functions. The mechanism works by projecting the clipped mini-batch gradient onto a random low-dime

What carries the argument

The update rule is x_{k+1} = x_k − α A_k( (1/√p) A_k^T g_k + ε_k ), where g_k is the mini-batch gradient after per-sample clipping, A_k is a d×p Gaussian random matrix, and ε_k ~ N(0, σ²_{ε,k} I_p) with σ²_{ε,k} = σ²_ε/k. The privacy argument treats the sequence of projected noisy gradients as a composition of subsampled Gaussian mechanisms and relies on the unit l2 sensitivity of the clipped per-sample gradients. The convergence analysis controls the distortion introduced by the random projection through its sampling variance σ_A², yielding error terms that scale with p^{3/2} and p d² σ_A² and vanish at the claimed rates.

Load-bearing premise

The privacy proof assumes that random projection leaves the l2 sensitivity of the mini-batch gradient at 1; for a Gaussian projection matrix, the sensitivity of (1/√p)A^T g actually scales with the spectral norm of A/√p, which typically exceeds 1, so the stated (ε,δ) guarantee depends on an assumption that is likely false.

What would settle it

Calculate the l2 sensitivity of the released quantity (1/√p)A^T g for a Gaussian matrix A with d=1000, p=100 and two neighboring datasets whose clipped gradients differ by a unit vector; the sensitivity concentrates around (√d+√p)/√p ≈ 4.16, not 1. If this exceeds the value used in the privacy accountant, Theorem 1's (ε,δ) bound cannot hold as stated.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Differentially private training can asymptotically match the convergence rate of ordinary SGD, up to logarithmic factors, for both convex and non-convex objectives.
  • The injected noise variance depends on the projection dimension p rather than the parameter dimension d, so private training of very large models becomes less expensive.
  • The decreasing noise schedule lets practitioners shift more privacy cost to early iterations, effectively spending the privacy budget where it matters least for final utility.
  • The static-noise and non-projected versions of private SGD appear as special cases, so the analysis provides a unified convergence and privacy treatment of these variants.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A cautionary extension: the stated privacy guarantee assumes the projected gradient keeps unit l2 sensitivity, but for a Gaussian projection the sensitivity scales with the spectral norm of A_k/√p; unless the projection is normalized or the sensitivity is recomputed, the (ε,δ) bound in Theorem 1 may not hold as written.
  • A practical variant the paper does not explore is fixing the random projection matrix across iterations, which would cut the per-iteration cost from O(dp) to an amortized O(dp/K) and might admit a different privacy analysis.
  • The same dynamic-noise scheduler could replace random projection with other dimensionality reducers, such as quantization or sketching, yielding a family of private optimizers whose noise cost tracks the compressed dimension rather than the parameter count.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces D2P2-SGD, a differentially private SGD variant that combines (i) a dynamic (time-varying) Gaussian noise schedule for DP, (ii) per-sample gradient normalization as automatic clipping, and (iii) random projection of the mini-batch gradient into a lower-dimensional space. The main theoretical claims are an (ε,δ)-DP guarantee (Theorem 1), sub-linear convergence rates for convex objectives (Theorem 2, Corollary 1) and non-convex objectives (Theorem 3, Corollary 2), and a formal trade-off among utility, privacy, and projection dimension. Experiments on FashionMNIST, SVHN, CIFAR-10, and other datasets are reported to support the claims.

Significance. If the results were correct, the paper would offer a useful combination of dynamic privacy budgeting and random projection, with a potentially practical reduction in the dimension of injected noise. The paper is clearly written, includes code, and reports experiments across a wide set of datasets and baselines. The automatic-clipping mechanism from Bu et al. is competently integrated into the algorithm. However, the central theoretical results are not established: the privacy proof ignores the effect of the random projection on sensitivity, the convergence theorems contain an incorrect linear-algebra identity, and the stated convergence rates do not withstand substitution of the privacy-required noise magnitude. These are load-bearing issues that affect the main contributions of the paper.

major comments (4)
  1. [Appendix A.3, Lemma 5; Theorem 1 (Section 3.2)] The privacy proof assumes the sensitivity of the noisy query is 1, but that is false for the actual mechanism. Line 6 of Algorithm 1 adds Gaussian noise to q_k = (1/√p) A_k^T g_k, not to g_k. For two neighboring datasets differing in one sample, the ℓ2 sensitivity of q_k is (1/(B√p)) ||A_k^T (ĝ_s − ĝ_s')||, which scales with the spectral norm of A_k. For A_k with iid N(0,1) entries, ||A_k||_2 = Θ(√d + √p) with high probability, so the sensitivity is Θ(√(d/p)), not 1. Lemma 5's bound in Eq. (11) and the resulting σ_ε^2 bound in Theorem 1 are therefore unsupported. The post-processing argument in §3.2 does not apply, because the noise is added after projection; post-processing preserves DP only if q_k + ε_k is already a DP release, which is exactly what is in question. A valid analysis would need to condition on the realized A_k, set the noise variance proportional to ||A_k||^2/p, and fold
  2. [Theorems 2 and 3, Corollaries 1 and 2] The stated convergence rates are inconsistent with the privacy requirement. The noise term in Theorem 2 (Eq. (2)) is O(α p^{1.5} lnK σ_ε^2 / K). Theorem 1 requires σ_ε^2 = Ω(B^2 K^2 ln(1/δ)/(n^2 ε^2)). Substituting this into Eq. (2) with α = O(1/√K) gives a noise term Ω(B^2 p^{1.5} √K lnK ln(1/δ)/(n^2 ε^2)), which diverges as K grows. The same issue appears in Theorem 3 (Eq. (3)). Thus Corollaries 1 and 2 do not follow from the privacy theorem; they treat σ_ε as a constant independent of K, which is incompatible with Theorem 1. The claim of 'matching the best available rate' is therefore not supported for the privacy-compliant algorithm.
  3. [Appendix A.4, proof of Theorem 2; also used in Theorem 3] The proof incorrectly computes E[‖A_k ε_k‖²]. For a d×p matrix A_k with iid N(0, σ_A²) entries, E[A_k^T A_k] = d σ_A² I_p, not p σ_A² I_p. Therefore E[‖A_k ε_k‖²] = d p σ_A² σ_ε,k², not p² σ_A² σ_ε,k². This error propagates into Eq. (2) and Eq. (3): the additive-noise term should scale as d√p rather than p^{3/2}. Since the paper's motivation is that random projection reduces the influence of the ambient dimension d on the injected noise, this is not a cosmetic typo; it changes the central dimension-privacy-utility trade-off claimed in Section 3.2.
  4. [Appendix A.5 and A.8] The dimension-privacy-utility comparison is not a valid implication of Theorem 1. Theorem 1 states that σ_ε^2 = C₂ B² K² ln(1/δ)/(n² ε²) for universal constants C₁, C₂. In Appendix A.5, however, C₂ is set to nε/(p^{5/2} K lnK √ln(1/δ)), which depends on ε, K, p, and n and is therefore not a constant. The subsequent 'privacy error' terms in Eqs. (23)–(24) and (35)–(36) are constructed by this choice and do not follow from the theorem. This undermines the quantitative conclusions about how projection reshapes the privacy-utility trade-off.
minor comments (4)
  1. [Theorem 1 statement and proof] The theorem statement says 'for any ε ≤ C₁B²K/n²', but the proof concludes using ε = C₁B²K²/n². The exponent of K is inconsistent and should be corrected.
  2. [Theorem 3 statement] The condition reads 'If the step size α ≤ 1/(2L), and, then ...'; there is a stray 'and'.
  3. [Algorithm 1 and Lemma 5] Lemma 5 refers to the 'mini-batch sum' and sets sensitivity Δ=1, while Algorithm 1 uses the mini-batch average g_k = (1/B)∑ ĝ_s. The factor of B (and the factor 2 arising from sample replacement) should be made explicit so that the privacy accountant's normalization is clear.
  4. [Appendix A.8, Eq. (35)] The non-convex section writes E[f(¯x_K) − f*] even though Theorem 3 bounds the expected gradient norm; this is a confusing reuse of the convex notation.

Circularity Check

0 steps flagged

No circularity: the derivation chain is self-contained given standard external results; the dynamic noise schedule is a design choice, not a fitted input.

full rationale

The paper's utility theorems (Theorems 2 and 3, Corollaries 1 and 2) are derived from standard smoothness and bounded-variance assumptions (Assumptions 1 and 2), convexity, Jensen's inequality, and explicit random-projection moment identities such as E[A_k A_k^T]=pσ_A^2 I and E||A_k A_k^T||^2 ≤ p^2 d^2 σ_A^4. None of these identities presuppose the convergence rates being proven; the rates follow from substituting α=O(1/√K) into the derived bounds. The privacy guarantee (Theorem 1) is obtained by composing subsampled Gaussian mechanisms using the Rényi differential privacy accountant of Abadi et al. (2016) (Lemmas 3–5), an external prior result, and the time-varying schedule σ_{ε,k}^2=σ_ε^2/k is a stated algorithm design whose effect on the privacy accountant is computed algebraically rather than fitted to the target bound. The cited automatic clipping (Bu et al. 2024) and low-dimensional SGD (Kasiviswanathan 2021) are independent prior works, not self-citations, and the paper discloses its relationship to them (e.g., D2P2-SGD degenerates to D2P-SGD and PrivSGD in special cases). The only notable weakness is in Lemma 5, which claims sensitivity Δ=1 for the mini-batch sum, while Algorithm 1 actually adds noise to the projected query (1/√p)A_k^T g_k; this is a potential correctness gap in applying an external sensitivity bound, not a circular definition or a fitted-parameter-as-prediction, so it does not raise the circularity score. The paper's own Limitation section concedes that high-privacy regimes are challenging, an honest scope caveat rather than a circular justification. Thus no circular step is present.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 0 invented entities

The central claim rests on smoothness and bounded variance assumptions, plus JL-based projection properties and the moments accountant. The privacy proof also implicitly assumes the projection does not change the l2 sensitivity, which is not established and is in fact false for Gaussian projection matrices.

free parameters (6)
  • step size α = 0.01 in experiments; O(1/√K) in theory
    Learning rate. The convergence bounds require α ≤ 1/(2L).
  • clipping stability constant γ = 0.01
    Prevents division by zero in the automatic clipping; contributes to the bias term in non-convex bound.
  • projection variance σ_A² = 1 in experiments
    Variance of entries of the random projection matrix A; affects projection error and privacy.
  • projection dimension p = Set by dimensionality reduction rate, e.g., 0.3d for 70% reduction
    Controls the tradeoff between noise reduction and projection error.
  • batch size B = 256 to 1024
    Affects privacy amplification and convergence bias.
  • noise scale σ_ε = 3.0 in main experiments; ablated from 1 to 10
    Must satisfy the DP bound in Theorem 1.
axioms (4)
  • domain assumption Gradient smoothness: f is L-smooth
    Assumption 1(a); standard for convergence analysis of gradient-based methods.
  • domain assumption Bounded stochastic gradient variance: E||∇f(x;s)-∇f(x)||² ≤ σ²
    Assumption 2; strong condition, as the authors note it is violated by quadratic objectives.
  • standard math Johnson-Lindenstrauss lemma and concentration of random projections
    Used to justify that random projection approximately preserves gradient geometry, and for E[A A^T]=pσ_A² I.
  • standard math Moments accountant of Abadi et al. (2016)
    Used for privacy composition; assumed to hold for the projected gradients without modification.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Balancing Utility and Privacy: Dynamically Private SGD with Random Projection." pith.science (2026). https://pith.science/paper/N3ZMG22J

@misc{pith2026250909485,
  author       = {Pith},
  title        = {Pith review of: Balancing Utility and Privacy: Dynamically Private SGD with Random Projection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N3ZMG22J}},
  note         = {Machine review of arXiv:2509.09485}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Stochastic optimization is a pivotal enabler in modern machine learning, producing effective models for various tasks. However, several existing works have shown that model parameters and gradient information are susceptible to privacy leakage. Although Differentially Private SGD (DPSGD) addresses privacy concerns, its static noise mechanism impacts the error bounds for model performance. Additionally, with the exponential increase in model parameters, efficient learning of these models using stochastic optimizers has become more challenging. To address these concerns, we introduce the Dynamically Differentially Private Projected SGD (D2P2-SGD) optimizer. In D2P2-SGD, we combine two important ideas: (i) dynamic differential privacy (DDP) with automatic gradient clipping and (ii) random projection with SGD, allowing dynamic adjustment of the tradeoff between utility and privacy of the model. It exhibits provably sub-linear convergence rates across different objective functions, matching the best available rate. The theoretical analysis further suggests that DDP leads to better utility at the cost of privacy, while random projection enables more efficient model learning. Extensive experiments across diverse datasets show that D2P2-SGD remarkably enhances accuracy while maintaining privacy. Our code is available here.

Figures

Figures reproduced from arXiv: 2509.09485 by Aditya Balu, Chao Liu, Md Zahid Hasan, Nastaran Saadati, Soumik Sarkar, Zhanhong Jiang.

Figure 1
Figure 1. Figure 1: D2P2-SGD method involves five steps. For more technical details, please see Algorithm 1. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Comparison among methods for SVHN data: on the right side, the privacy loss is shown for static [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Comparison among methods for FashionMNIST data: on the right side, the privacy loss is shown [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Accuracy vs. standard deviation 2 4 6 8 10 1 2 3 4 5 Fin al DPSGD D2P-SGD DP2-SGD D2P2-SGD (a) SVHN 2 4 6 8 10 1 2 3 4 5 6 Fin al DPSGD D2P-SGD DP2-SGD D2P2-SGD (b) FashionMNIST [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Privacy loss vs. standard deviation random projection enables more efficient model learning. Similar conclusions can be made from [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Impacts of parameters in D2P2-SGD maximum privacy losses of the dynamic mechanism end up with respectively 2.45 (for SVHN) and 2.75 (for FashionMNIST). Compared to values with the static DP mechanism (1.06 and 0.95, respectively), the privacy loss of D2P2-SGD grows sharply. However, given the bound for ε in Theorem 1, as long as the constant C1 (≥ 314, see Appendix A.9) is selected properly, D2P2-SGD still… view at source ↗
Figure 7
Figure 7. Figure 7: Comparison among different methods for CIFAR10 data with ResNet20: on the right side, the [PITH_FULL_IMAGE:figures/full_fig_p025_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Comparison among different methods for CIFAR10 data with reduction rate being 0.3, with CNN: [PITH_FULL_IMAGE:figures/full_fig_p025_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Comparison among different methods for CIFAR10 data with reduction rate being 0.1, with CNN: [PITH_FULL_IMAGE:figures/full_fig_p025_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Comparison among different methods for CIFAR10 data with CNN: training loss and testing [PITH_FULL_IMAGE:figures/full_fig_p026_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Comparison among different methods for KMNIST data with CNN: training loss and testing [PITH_FULL_IMAGE:figures/full_fig_p026_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Comparison among different methods for EMNIST data with CNN: training loss and testing [PITH_FULL_IMAGE:figures/full_fig_p026_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Comparison among different methods for MNIST data with CNN: training loss and testing [PITH_FULL_IMAGE:figures/full_fig_p027_13.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

55 extracted references · 15 linked inside Pith

  1. [1]

    Deep learning with differential privacy

    Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pp.\ 308--318, 2016

    2016

  2. [2]

    Database-friendly random projections

    Dimitris Achlioptas. Database-friendly random projections. In Proceedings of the twentieth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, pp.\ 274--281, 2001

    2001

  3. [3]

    cpsgd: Communication-efficient and differentially-private distributed sgd

    Naman Agarwal, Ananda Theertha Suresh, Felix Xinnan X Yu, Sanjiv Kumar, and Brendan McMahan. cpsgd: Communication-efficient and differentially-private distributed sgd. Advances in Neural Information Processing Systems, 31, 2018

    2018

  4. [4]

    Private empirical risk minimization: Efficient algorithms and tight error bounds

    Raef Bassily, Adam Smith, and Abhradeep Thakurta. Private empirical risk minimization: Efficient algorithms and tight error bounds. In 2014 IEEE 55th annual symposium on foundations of computer science, pp.\ 464--473. IEEE, 2014

    2014

  5. [5]

    A critical review on the use (and misuse) of differential privacy in machine learning

    Alberto Blanco-Justicia, David S \'a nchez, Josep Domingo-Ferrer, and Krishnamurty Muralidhar. A critical review on the use (and misuse) of differential privacy in machine learning. ACM Computing Surveys, 55 0 (8): 0 1--16, 2022

    2022

  6. [6]

    The johnson-lindenstrauss transform itself preserves differential privacy

    Jeremiah Blocki, Avrim Blum, Anupam Datta, and Or Sheffet. The johnson-lindenstrauss transform itself preserves differential privacy. In 2012 IEEE 53rd Annual Symposium on Foundations of Computer Science, pp.\ 410--419. IEEE, 2012

    2012

  7. [7]

    Optimization methods for large-scale machine learning

    L \'e on Bottou, Frank E Curtis, and Jorge Nocedal. Optimization methods for large-scale machine learning. SIAM review, 60 0 (2): 0 223--311, 2018

    2018

  8. [8]

    Automatic clipping: Differentially private deep learning made easier and stronger

    Zhiqi Bu, Yu-Xiang Wang, Sheng Zha, and George Karypis. Automatic clipping: Differentially private deep learning made easier and stronger. Advances in Neural Information Processing Systems, 36, 2024

    2024

  9. [9]

    Model compression

    Cristian Buciluǎ, Rich Caruana, and Alexandru Niculescu-Mizil. Model compression. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, pp.\ 535--541, 2006

    2006

  10. [10]

    Ethical machine learning in healthcare

    Irene Y Chen, Emma Pierson, Sherri Rose, Shalmali Joshi, Kadija Ferryman, and Marzyeh Ghassemi. Ethical machine learning in healthcare. Annual review of biomedical data science, 4: 0 123--144, 2021

    2021

  11. [11]

    Understanding gradient clipping in private sgd: A geometric perspective

    Xiangyi Chen, Steven Z Wu, and Mingyi Hong. Understanding gradient clipping in private sgd: A geometric perspective. Advances in Neural Information Processing Systems, 33: 0 13773--13782, 2020

    2020

  12. [12]

    Robust quantization: One model to rule them all

    Brian Chmiel, Ron Banner, Gil Shomron, Yury Nahshan, Alex Bronstein, Uri Weiser, et al. Robust quantization: One model to rule them all. Advances in neural information processing systems, 33: 0 5308--5317, 2020

    2020

  13. [13]

    A comprehensive survey on model compression and acceleration

    Tejalal Choudhary, Vipul Mishra, Anurag Goswami, and Jagannathan Sarangapani. A comprehensive survey on model compression and acceleration. Artificial Intelligence Review, 53: 0 5113--5155, 2020

    2020

  14. [14]

    Beyond uniform lipschitz condition in differentially private optimization

    Rudrajit Das, Satyen Kale, Zheng Xu, Tong Zhang, and Sujay Sanghavi. Beyond uniform lipschitz condition in differentially private optimization. In International Conference on Machine Learning, pp.\ 7066--7101. PMLR, 2023

    2023

  15. [15]

    A concentration theorem for projections

    Sanjoy Dasgupta, Daniel Hsu, and Nakul Verma. A concentration theorem for projections. arXiv preprint arXiv:1206.6813, 2012

    Pith/arXiv arXiv 2012

  16. [16]

    Dynamic differential-privacy preserving sgd

    Jian Du, Song Li, Xiangyi Chen, Siheng Chen, and Mingyi Hong. Dynamic differential-privacy preserving sgd. arXiv preprint arXiv:2111.00173, 2021

    Pith/arXiv arXiv 2021

  17. [17]

    Differential privacy

    Cynthia Dwork. Differential privacy. In International colloquium on automata, languages, and programming, pp.\ 1--12. Springer, 2006

    2006

  18. [18]

    Differential privacy: A survey of results

    Cynthia Dwork. Differential privacy: A survey of results. In International conference on theory and applications of models of computation, pp.\ 1--19. Springer, 2008

    2008

  19. [19]

    Rqp-sgd: Differential private machine learning through noisy sgd and randomized quantization

    Ce Feng and Parv Venkitasubramaniam. Rqp-sgd: Differential private machine learning through noisy sgd and randomized quantization. arXiv preprint arXiv:2402.06606, 2024

    Pith/arXiv arXiv 2024

  20. [20]

    Semi-supervised learning using deep generative models and auxiliary tasks

    Jhosimar Arias Figueroa. Semi-supervised learning using deep generative models and auxiliary tasks. In NeurIPS Workshop on Bayesian Deep Learning, 2019

    2019

  21. [21]

    Differentially private next-token prediction of large language models

    James Flemings, Meisam Razaviyayn, and Murali Annavaram. Differentially private next-token prediction of large language models. arXiv preprint arXiv:2403.15638, 2024

    Pith/arXiv arXiv 2024

  22. [22]

    Handbook of convergence theorems for (stochastic) gradient methods

    Guillaume Garrigos and Robert M Gower. Handbook of convergence theorems for (stochastic) gradient methods. arXiv preprint arXiv:2301.11235, 2023

    Pith/arXiv arXiv 2023

  23. [23]

    Stochastic first-and zeroth-order methods for nonconvex stochastic programming

    Saeed Ghadimi and Guanghui Lan. Stochastic first-and zeroth-order methods for nonconvex stochastic programming. SIAM journal on optimization, 23 0 (4): 0 2341--2368, 2013

    2013

  24. [24]

    Artificial intelligence and machine learning in finance: Identifying foundations, themes, and research clusters from bibliometric analysis

    John W Goodell, Satish Kumar, Weng Marc Lim, and Debidutta Pattnaik. Artificial intelligence and machine learning in finance: Identifying foundations, themes, and research clusters from bibliometric analysis. Journal of Behavioral and Experimental Finance, 32: 0 100577, 2021

    2021

  25. [25]

    Choosing public datasets for private machine learning via gradient subspace distance

    Xin Gu, Gautam Kamath, and Zhiwei Steven Wu. Choosing public datasets for private machine learning via gradient subspace distance. arXiv preprint arXiv:2303.01256, 2023

    Pith/arXiv arXiv 2023

  26. [26]

    Differential privacy and machine learning: a survey and review

    Zhanglong Ji, Zachary C Lipton, and Charles Elkan. Differential privacy and machine learning: a survey and review. arXiv preprint arXiv:1412.7584, 2014

    Pith/arXiv arXiv 2014

  27. [27]

    Extensions of lipschitz mappings into a hilbert space

    William B Johnson, Joram Lindenstrauss, et al. Extensions of lipschitz mappings into a hilbert space. Contemporary mathematics, 26 0 (189-206): 0 1, 1984

    1984

  28. [28]

    Sgd with low-dimensional gradients with applications to private and distributed learning

    Shiva Prasad Kasiviswanathan. Sgd with low-dimensional gradients with applications to private and distributed learning. In Uncertainty in Artificial Intelligence, pp.\ 1905--1915. PMLR, 2021

    1905

  29. [29]

    u chemann, Maria Bannert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan G \

    Enkelejda Kasneci, Kathrin Se ler, Stefan K \"u chemann, Maria Bannert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan G \"u nnemann, Eyke H \"u llermeier, et al. Chatgpt for good? on opportunities and challenges of large language models for education. Learning and individual differences, 103: 0 102274, 2023

    2023

  30. [30]

    Revisiting gradient clipping: Stochastic bias and tight convergence guarantees

    Anastasia Koloskova, Hadrien Hendrikx, and Sebastian U Stich. Revisiting gradient clipping: Stochastic bias and tight convergence guarantees. In International Conference on Machine Learning, pp.\ 17343--17363. PMLR, 2023 a

    2023

  31. [31]

    Gradient descent with linearly correlated noise: Theory and applications to differential privacy

    Anastasiia Koloskova, Ryan McKenna, Zachary Charles, John Rush, and H Brendan McMahan. Gradient descent with linearly correlated noise: Theory and applications to differential privacy. Advances in Neural Information Processing Systems, 36, 2023 b

    2023

  32. [32]

    Optimality of the johnson-lindenstrauss lemma

    Kasper Green Larsen and Jelani Nelson. Optimality of the johnson-lindenstrauss lemma. In 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS), pp.\ 633--638. IEEE, 2017

    2017

  33. [33]

    Convergence and privacy of decentralized nonconvex optimization with gradient clipping and communication compression

    Boyue Li and Yuejie Chi. Convergence and privacy of decentralized nonconvex optimization with gradient clipping and communication compression. IEEE Journal of Selected Topics in Signal Processing, 2025

    2025

  34. [34]

    Differentially private language models for secure data sharing

    Justus Mattern, Zhijing Jin, Benjamin Weggenmann, Bernhard Schoelkopf, and Mrinmaya Sachan. Differentially private language models for secure data sharing. arXiv preprint arXiv:2210.13918, 2022

    Pith/arXiv arXiv 2022

  35. [35]

    Efficient deep learning: A survey on making deep learning models smaller, faster, and better

    Gaurav Menghani. Efficient deep learning: A survey on making deep learning models smaller, faster, and better. ACM Computing Surveys, 55 0 (12): 0 1--37, 2023

    2023

  36. [36]

    Differentially private model compression

    Fatemehsadat Mireshghallah, Arturs Backurs, Huseyin A Inan, Lukas Wutschitz, and Janardhan Kulkarni. Differentially private model compression. Advances in Neural Information Processing Systems, 35: 0 29468--29483, 2022

    2022

  37. [37]

    A survey of regularization strategies for deep models

    Reza Moradi, Reza Berangi, and Behrouz Minaei. A survey of regularization strategies for deep models. Artificial Intelligence Review, 53 0 (6): 0 3947--3986, 2020

    2020

  38. [38]

    Random projection and its applications

    Mahmoud Nabil. Random projection and its applications. arXiv preprint arXiv:1710.03163, 2017

    Pith/arXiv arXiv 2017

  39. [39]

    Explicit regularization in overparametrized models via noise injection

    Antonio Orvieto, Anant Raj, Hans Kersting, and Francis Bach. Explicit regularization in overparametrized models via noise injection. In International Conference on Artificial Intelligence and Statistics, pp.\ 7265--7287. PMLR, 2023

    2023

  40. [40]

    How to dp-fy ml: A practical guide to machine learning with differential privacy

    Natalia Ponomareva, Hussein Hazimeh, Alex Kurakin, Zheng Xu, Carson Denison, H Brendan McMahan, Sergei Vassilvitskii, Steve Chien, and Abhradeep Guha Thakurta. How to dp-fy ml: A practical guide to machine learning with differential privacy. Journal of Artificial Intelligence Research, 77: 0 1113--1201, 2023

    2023

  41. [41]

    Enabling fast differentially private sgd via just-in-time compilation and vectorization

    Pranav Subramani, Nicholas Vadivelu, and Gautam Kamath. Enabling fast differentially private sgd via just-in-time compilation and vectorization. Advances in Neural Information Processing Systems, 34: 0 26409--26421, 2021

    2021

  42. [42]

    Large language models in medicine

    Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. Large language models in medicine. Nature medicine, 29 0 (8): 0 1930--1940, 2023

    1930

  43. [43]

    On differentially private subspace estimation without distributional assumptions

    Eliad Tsfadia. On differentially private subspace estimation without distributional assumptions. arXiv preprint arXiv:2402.06465, 2024

    Pith/arXiv arXiv 2024

  44. [44]

    Resnets ensemble via the feynman-kac formalism to improve natural and robust accuracies

    Bao Wang, Zuoqiang Shi, and Stanley Osher. Resnets ensemble via the feynman-kac formalism to improve natural and robust accuracies. Advances in Neural Information Processing Systems, 32, 2019 a

    2019

  45. [45]

    Private model compression via knowledge distillation

    Ji Wang, Weidong Bao, Lichao Sun, Xiaomin Zhu, Bokai Cao, and S Yu Philip. Private model compression via knowledge distillation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp.\ 1190--1197, 2019 b

    2019

  46. [46]

    Protect privacy from gradient leakage attack in federated learning

    Junxiao Wang, Song Guo, Xin Xie, and Heng Qi. Protect privacy from gradient leakage attack in federated learning. In IEEE INFOCOM 2022-IEEE Conference on Computer Communications, pp.\ 580--589. IEEE, 2022 a

    2022

  47. [47]

    Differentially private sgd with non-smooth losses

    Puyu Wang, Yunwen Lei, Yiming Ying, and Hai Zhang. Differentially private sgd with non-smooth losses. Applied and Computational Harmonic Analysis, 56: 0 306--336, 2022 b

    2022

  48. [48]

    A theory to instruct differentially-private learning via clipping bias reduction

    Hanshen Xiao, Zihang Xiang, Di Wang, and Srinivas Devadas. A theory to instruct differentially-private learning via clipping bias reduction. In 2023 IEEE Symposium on Security and Privacy (SP), pp.\ 2170--2189. IEEE, 2023

    2023

  49. [49]

    Robust regression and lasso

    Huan Xu, Constantine Caramanis, and Shie Mannor. Robust regression and lasso. Advances in neural information processing systems, 21, 2008

    2008

  50. [50]

    Opacus: User-friendly differential privacy library in pytorch

    Ashkan Yousefpour, Igor Shilov, Alexandre Sablayrolles, Davide Testuggine, Karthik Prasad, Mani Malek, John Nguyen, Sayan Ghosh, Akash Bharadwaj, Jessica Zhao, et al. Opacus: User-friendly differential privacy library in pytorch. arXiv preprint arXiv:2109.12298, 2021

    Pith/arXiv arXiv 2021

  51. [51]

    Differentially private fine-tuning of language models

    Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin A Inan, Gautam Kamath, Janardhan Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, et al. Differentially private fine-tuning of language models. arXiv preprint arXiv:2110.06500, 2021

    Pith/arXiv arXiv 2021

  52. [52]

    Differentially private sgd without clipping bias: An error-feedback approach

    Xinwei Zhang, Zhiqi Bu, Zhiwei Steven Wu, and Mingyi Hong. Differentially private sgd without clipping bias: An error-feedback approach. arXiv preprint arXiv:2311.14632, 2023

    Pith/arXiv arXiv 2023

  53. [53]

    idlg: Improved deep leakage from gradients

    Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. idlg: Improved deep leakage from gradients. arXiv preprint arXiv:2001.02610, 2020

    Pith/arXiv arXiv 2001

  54. [54]

    Bypassing the ambient dimension: Private sgd with gradient subspace identification

    Yingxue Zhou, Zhiwei Steven Wu, and Arindam Banerjee. Bypassing the ambient dimension: Private sgd with gradient subspace identification. arXiv preprint arXiv:2007.03813, 2020

    Pith/arXiv arXiv 2007

  55. [55]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.