REVIEW 2 major objections 5 minor 51 references
Dyn-D$^2$P: Dynamic Differentially Private Decentralized Learning with Provable Utility Guarantee
T0 review · 2 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read This paper claims that an exponentially decaying clipping bound and an exponentially growing per-step privacy budget let a decentralized learning system shrink its injected noise over time while holding a fixed total privacy budget, and…
desk verdict Genuinely new utility analysis for dynamic clipping/noise in decentralized DP, but the privacy theorem silently requires mu_k <= 1 and the utility bound leaves T2 unquantified; worth peer review with repair requests. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrying mechanism is the noise schedule: at iteration $k$ each node clips its stochastic gradient at $C_k=C_0(\rho_c)^{-k/K}$, assigns itself a per-step Gaussian-DP budget $\mu_k=\mu_0(\rho_\mu)^{k/K}$, and adds Gaussian noise with standard deviation $\sigma_k=C_k/\mu_k$, so noise decays at the combined rate $(\rho_c\rho_\mu)^{-k/K}$. The privacy accounting is the GDP composition formula $\mu_{\mathrm{tot}}=(1/J)\sqrt{\sum_{k=0}^{K-1}(e^{\mu_k^2}-1)}$, and Proposition 4 turns that into a single noise-scale condition $\tilde{\sigma}$. The utility analysis then writes the error as a privacy-noise term $T_1$ plus a clipping-bias term $T_2$, with consensus error controlled by the Push-Sum averaging protocol's propagation constant $q$.
What would settle it
For a concrete case, choose $J$, $\rho_\mu$, $K$, and $\mu_{\mathrm{tot}}$ as in the paper's experiments, solve equation (8) for $\mu_0$, and compute $\max_k \mu_0(\rho_\mu)^{k/K}$; if the maximum exceeds $1$, the proof of Proposition 4 does not apply at that schedule, so the noise scale from (10) may not deliver the claimed privacy guarantee.
Extended reading notes
Core claim
For its general Algorithm 2 with arbitrary clipping bounds $C_k$ and noise multipliers $\tilde{\sigma}_k$, the paper proves (Theorem 1) an upper bound on the average squared gradient norm at the node-average model, with the error split into a privacy-noise term $T_1$ and a clipping-bias term $T_2$. Choosing $C_k=\Theta((\rho_c)^{-k/K})$ and $\tilde{\sigma}_k=\Theta((\rho_c\rho_\mu)^{-k/K})$, together with step size $\gamma=1/(\sqrt{n}J\mu_{\mathrm{tot}})$ and $\gamma K=\sqrt{n}J\mu_{\mathrm{tot}}$, the bound becomes $O(1/((1-q)^2\sqrt{n}J\mu_{\mathrm{tot}}))+T_2$ whenever $J\mu_{\mathrm{tot}}>\sqrt{n}$. This is the paper's claimed first model-utility guarantee for differentially private decentralized non-convex optimization with dynamic gradient clipping and dynamic noise, and it makes the network dependence explicit through the information-propagation constant $q$.
Load-bearing premise
The load-bearing premise is that every per-step privacy budget satisfies $\mu_k \le 1$, because the composition bound's inequality $e^{\mu_k^2}-1<2\mu_k^2$ only holds in that range, and the paper never states or verifies this condition for its exponentially growing schedules.
Editorial extensions
If this is right
- Under the Corollary 1 scheduling, the utility error floor decreases as $1/\sqrt{n}$, so adding nodes genuinely improves accuracy under a fixed privacy budget.
- The explicit factor $1/(1-q)^2$ means faster-mixing network topologies are provably better for private decentralized training with the same total privacy cost.
- Letting $n=1$ recovers the centralized dynamic-DP rate $O(1/(J\mu_{\mathrm{tot}}))$, showing the decentralized analysis contains the single-node result as a special case.
- The bias term $T_2$ persists because clipping makes gradients biased, and the paper's design keeps the clipping probability roughly constant as gradient norms decay, rather than eliminating the bias.
Reading between the lines
- The privacy proof's inequality $e^{\mu_k^2}-1<2\mu_k^2$ is only valid for $\mu_k\le 1$, so any concrete schedule from Algorithm 1 should be checked against that condition; without that check, the claimed DP guarantee for large $J$ is unverified.
- Because Theorem 1 allows arbitrary $C_k$ and $\tilde{\sigma}_k$ sequences, it provides a ready template for evaluating adaptive clipping heuristics that set the bound from current gradient norms instead of an exponential schedule.
- A sharper treatment of the clipping-bias term $T_2$ would be needed to obtain a fully vanishing error; in the strong-privacy regime the bias may dominate, and the paper's own experiments show the largest gains at small $\epsilon$ without isolating how much of those gains come from reduced noise versus bias changes.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Dyn-D2P, a differentially private decentralized learning algorithm for non-convex stochastic optimization over time-varying directed networks. The algorithm combines Push-Sum consensus with per-iteration gradient clipping and Gaussian noise, using a decaying clipping bound C_k and an increasing per-step GDP privacy budget μ_k so that the injected noise level decreases over training. The authors provide a general utility result (Theorem 1) for arbitrary clipping and noise schedules, then specialize to exponential schedules in Corollary 1, obtaining a bound of the form O(1/((1-q)^2 √(n) J μ_tot)) plus a bias term T2 from clipping. Experiments on CIFAR-10 and FashionMNIST show that Dyn-D2P outperforms fixed-noise decentralized DP baselines, especially under strong privacy. The paper claims to be the first to give a utility guarantee for decentralized non-convex DP optimization with dynamic clipping bounds and noise levels, with an explicit network dependence.
Significance. If the central results are correct, this is a meaningful contribution: it extends dynamic clipping/noise scheduling from centralized DP-SGD to the decentralized setting, provides explicit network-dependent rates, and demonstrates empirical gains. The 1/√n scaling in Corollary 1 is a useful qualitative result, and the general Theorem 1 covers arbitrary schedules. The paper also includes two ablation variants (Dyn[C]-D2P and Dyn[μ]-D2P) and a fixed-noise baseline, and the experiments support the main qualitative claims. The proofs follow standard descent-plus-consensus arguments and are largely coherent. However, the privacy proof of Proposition 4 contains a domain condition that is neither stated nor verified for the schedules used in Corollary 1; this is a load-bearing gap that must be addressed before the utility guarantee can be considered fully supported.
major comments (2)
- [Appendix C, Proposition 4] The proof of Proposition 4 uses the inequality e^x − 1 < 2x for 0 ≤ x ≤ 1, with x = μ_k^2, but the statement of Proposition 4, the description of Algorithm 2, and Corollary 1 do not state or verify that every per-step GDP parameter μ_k = C_k/(σ̃ σ̃_k) satisfies μ_k ≤ 1 (or at least μ_k^2 below the validity threshold of the inequality). For the exponential schedules of Corollary 1, with C_k = Θ((ρ_c)^{-k/K}), σ̃_k = Θ((ρ_c ρ_μ)^{-k/K}), γ = 1/(√n J μ_tot), and K = n J^2 μ_tot^2, the per-step budget satisfies μ_k = (J μ_tot) ρ_μ^{k/K} / √(2Σ_{s=0}^{K-1} ρ_μ^{2s/K}) up to constants. For n = 1 and ρ_μ = 10, this exceeds 1 at later iterations (approximately 1.52 at k = K−1), so e^{μ_k^2} − 1 > 2μ_k^2 for those steps and the composition bound in (10) may not hold. Consequently, the (ε,δ)-DP guarantee claimed in Proposition 4 and the utility bound in Corollary 1 built on (10) are not established as stated. The authors should either add the condition μ_k ≤ 1 for all k and verify it for the schedules used in Corollary 1, or replace the relaxed inequality with the exact composition formula (as already done in (8) for Algorithm 1).
- [Corollary 1, Eq. (12)] The headline utility bound in Corollary 1 is O(1/((1-q)^2 √n J μ_tot)) + T2, where the bias term T2 is left entirely unquantified. Since T2 = 2E[(1/K)Σ Λ‖∇f(x̄_k)‖ (1/n)Σ P_k^i(C_k)] is nonnegative and can be of constant order, the stated 1/√n scaling does not by itself provide a convergence guarantee to a stationary point unless the clipping probabilities P_k^i(C_k) are controlled. The authors should either prove a bound on T2 under an explicit condition on the clipping schedule (e.g., a bound on the probability of clipping as a function of C_k), or clearly state that the result is a bias-variance trade-off and that the 1/√n scaling applies only to the non-bias terms. Without this, the claim of a "provable utility guarantee" is weaker than the notation suggests.
minor comments (5)
- [Algorithm 2, lines 5–6] In Algorithm 2, the noise is drawn as N_i^k ∼ N(0, σ̃^2 I_d) and then multiplied by σ̃_k in the update, so the actual injected noise has variance σ̃^2 σ̃_k^2. The notation is consistent with the text "σ_k = σ̃·σ̃_k", but reading line 5 alone is confusing because σ̃^2 is not the full variance; please clarify by writing N_i^k ∼ N(0, I_d) and scaling by σ̃ σ̃_k in the update, or by defining the noise in one place.
- [Abstract and Section 6] The abstract and conclusion state the contribution as the 'first model utility analysis for differentially private decentralized non-convex optimization with dynamic gradient clipping bounds and noise levels.' Given that the proposition underlying the privacy guarantee currently requires an unstated condition, the authors should either temper this claim until the condition is verified or explicitly state the condition in the theorem statements.
- [Appendix C, step (a)] The inequality e^x − 1 < 2x is stated for '0 ≤ x ≤ 1', but the proof does not explain why μ_k^2 falls in this range. Even when the condition μ_k ≤ 1 is added, the authors should also justify it for the schedules in Algorithm 1, where μ_k is computed via the exact composition formula (8) rather than the relaxed bound (10).
- [Throughout] There are several typos: 'unblanceness' should be 'unbalancedness'; 'Cauchy-Swarchz' should be 'Cauchy-Schwarz'; and in the pseudocode of Algorithm 1, 'Calculates the clipping bound by: C_k = C_0·(ρ_c)^{-k/K}' uses a superscript minus sign that renders awkwardly. A careful proofreading pass is recommended.
- [Appendix B, Eq. (35)–(36)] The Θ notation in Corollary 1 hides constants that depend on ρ_c and ρ_μ, and the step from (35) to (36) uses the fact that (1/K)(ρ_μ^2−1)/(ρ_μ^{2/K}−1) is O(1) for fixed ρ_μ. This is correct, but the dependence on ρ_μ and ρ_c is not made explicit in the final O(·) statement; the authors should state whether the hidden constants remain uniformly bounded over the range of ρ_μ, ρ_c used in the experiments.
Circularity Check
No circularity found: the 1/sqrt(n) rate is an explicit step-size/horizon choice, and the one self-citation is peripheral; a missing mu_k <= 1 privacy condition is a soundness issue, not circularity.
full rationale
Walking the derivation chain, I find no step in which a predicted quantity is defined in terms of the target, or in which a fitted parameter is renamed as a prediction. The utility bound in Theorem 1 is derived from the consensus lemma (Lemma 2, adapted from Assran et al. 2019) and standard smoothness/descent arguments; no conclusion is assumed. Corollary 1 substitutes the geometric schedules C_k = Theta((rho_c)^{-k/K}) and sigma_tilde_k = Theta((rho_c * rho_mu)^{-k/K}) and then selects gamma = 1/(sqrt(n) * J * mu_tot) with gamma * K = sqrt(n) * J * mu_tot. That selection is an explicit theoretical tuning of step size and horizon, not a fit to data; the resulting O(1/((1-q)^2 * sqrt(n) * J * mu_tot)) rate follows algebraically from equations (37)-(38) in Appendix B. The hyperparameters rho_c, rho_mu, and C_0 are user-chosen and cancel or remain hidden in the O(1) factor; they do not enter as fitted constants. The self-citation [Zhu et al., 2024] appears only in a background sentence about decentralized SGD and is not load-bearing. The privacy accounting in Proposition 4 uses the GDP composition cited from Du et al. 2021 and the standard GDP-to-DP conversion of Dong/Bu; it is not circular. I flag one non-circular soundness gap: the proof of Proposition 4 (Appendix C) uses e^x - 1 < 2x 'for 0 <= x <= 1' but neither states nor verifies mu_k = C_k/(sigma_tilde * sigma_tilde_k) <= 1 for the exponential schedules of Corollary 1; the schedule can violate this condition for small n or large rho_mu. This is a missing condition, not a circular reduction, so it does not raise the circularity score.
Assumptions & free parameters
free parameters (4)
- C0 =
7 (ResNet-18), 4 (shallow CNN)
- rho_c =
1/rho_c in {0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8}
- rho_mu =
1/rho_mu in {0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8}
- gamma =
0.05 (ResNet-18), 0.03 (shallow CNN)
assumptions (5)
- domain assumption Mixing matrices P_k are column-stochastic (Assumption 1).
- domain assumption The time-varying graph is B-strongly connected with diameter Delta (Assumption 2).
- domain assumption Each local loss f_i is L-smooth (Assumption 3).
- domain assumption Per-sample stochastic gradients are uniformly bounded by Lambda.
- ad hoc to paper Per-step GDP parameter satisfies mu_k <= 1 for the inequality e^{mu_k^2} - 1 < 2 * mu_k^2.
Cite this review
Pith. "Pith review of Dyn-D$^2$P: Dynamic Differentially Private Decentralized Learning with Provable Utility Guarantee." pith.science (2026). https://pith.science/paper/OP5R73XM
@misc{pith2026250506651,
author = {Pith},
title = {Pith review of: Dyn-D$^2$P: Dynamic Differentially Private Decentralized Learning with Provable Utility Guarantee},
year = {2026},
howpublished = {\url{https://pith.science/paper/OP5R73XM}},
note = {Machine review of arXiv:2505.06651}
}
abstract
Most existing decentralized learning methods with differential privacy (DP) guarantee rely on constant gradient clipping bounds and fixed-level DP Gaussian noises for each node throughout the training process, leading to a significant accuracy degradation compared to non-private counterparts. In this paper, we propose a new Dynamic Differentially Private Decentralized learning approach (termed Dyn-D$^2$P) tailored for general time-varying directed networks. Leveraging the Gaussian DP (GDP) framework for privacy accounting, Dyn-D$^2$P dynamically adjusts gradient clipping bounds and noise levels based on gradient convergence. This proposed dynamic noise strategy enables us to enhance model accuracy while preserving the total privacy budget. Extensive experiments on benchmark datasets demonstrate the superiority of Dyn-D$^2$P over its counterparts employing fixed-level noises, especially under strong privacy guarantees. Furthermore, we provide a provable utility bound for Dyn-D$^2$P that establishes an explicit dependency on network-related parameters, with a scaling factor of $1/\sqrt{n}$ in terms of the number of nodes $n$ up to a bias error term induced by gradient clipping. To our knowledge, this is the first model utility analysis for differentially private decentralized non-convex optimization with dynamic gradient clipping bounds and noise levels.
Figures
Figures from the paper (7 more)
Reference graph
Works this paper leans on
-
[1]
Deep learning with differential privacy
[Abadi et al., 2016] Martin Abadi, Andy Chu, Ian Goodfel- low, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on com- puter and communications security, pages 308–318,
work page 2016
-
[6]
LEASGD: an Efficient and Privacy-Preserving Decentralized Algorithm for Distributed Learning
[Cheng et al., 2018] Hsin-Pai Cheng, Patrick Yu, Haojing Hu, Feng Yan, Shiyu Li, Hai Li, and Yiran Chen. LEASGD: an efficient and privacy-preserving decentral- ized algorithm for distributed learning. arXiv preprint arXiv:1811.11124,
work page Pith review arXiv 2018
-
[8]
[Dong et al., 2019] Jinshuo Dong, Aaron Roth, and Wei- jie J Su. Gaussian differential privacy. arXiv preprint arXiv:1905.02383,
arXiv 2019
-
[9]
Dynamic differential-privacy preserving sgd
[Du et al., 2021] Jian Du, Song Li, Xiangyi Chen, Siheng Chen, and Mingyi Hong. Dynamic differential-privacy preserving sgd. In International Conference on Machine Learning. PMLR,
work page 2021
-
[10]
Our data, ourselves: Privacy via distributed noise generation
[Dwork et al., 2006] Cynthia Dwork, Krishnaram Kentha- padi, Frank McSherry, Ilya Mironov, and Moni Naor. Our data, ourselves: Privacy via distributed noise generation. In Annual international conference on the theory and ap- plications of cryptographic techniques , pages 486–503. Springer,
work page 2006
-
[15]
Towards practical differentially private convex optimiza- tion
[Iyengar et al., 2019] Roger Iyengar, Joseph P Near, Dawn Song, Om Thakkar, Abhradeep Thakurta, and Lun Wang. Towards practical differentially private convex optimiza- tion. In 2019 IEEE Symposium on Security and Privacy (SP), pages 299–316. IEEE,
work page 2019
-
[16]
Gossip-based computation of aggregate information
[Kempe et al., 2003] David Kempe, Alin Dobra, and Jo- hannes Gehrke. Gossip-based computation of aggregate information. In 44th Annual IEEE Symposium on Foun- dations of Computer Science,
work page 2003
-
[18]
Learning multiple lay- ers of features from tiny images
[Krizhevsky, 2009] Alex Krizhevsky. Learning multiple lay- ers of features from tiny images. Master’s thesis, Univer- sity of Toronto,
work page 2009
Show all 51 references
-
[20]
Convergence and privacy of decentralized nonconvex optimization with gradient clipping and communication compression
[Li and Chi, 2025] Boyue Li and Yuejie Chi. Convergence and privacy of decentralized nonconvex optimization with gradient clipping and communication compression. IEEE Journal of Selected Topics in Signal Processing,
2025
-
[22]
Asynchronous federated learning with dif- ferential privacy for edge intelligence
[Li et al., 2019b] Yanan Li, Shusen Yang, Xuebin Ren, and Cong Zhao. Asynchronous federated learning with dif- ferential privacy for edge intelligence. arXiv preprint arXiv:1912.07902,
1912 arXiv
-
[23]
SoteriaFL: A unified framework for private feder- ated learning with communication compression
[Li et al., 2022] Zhize Li, Haoyu Zhao, Boyue Li, and Yue- jie Chi. SoteriaFL: A unified framework for private feder- ated learning with communication compression. Advances in Neural Information Processing Systems, 35:4285–4300,
2022
-
[24]
Can decentral- ized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient de- scent
[Lian et al., 2017] Xiangru Lian, Ce Zhang, Huan Zhang, Cho-Jui Hsieh, Wei Zhang, and Ji Liu. Can decentral- ized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient de- scent. Advances in Neural Information Processing Sys-...
2017
-
[26]
Loss-privacy tradeoff in federated edge learning
[Liu et al., 2022] Tianyu Liu, Boya Di, Bin Wang, and Lingyang Song. Loss-privacy tradeoff in federated edge learning. IEEE Journal of Selected Topics in Signal Pro- cessing, 16(3):546–558,
2022
-
[27]
Learning differen- tially private recurrent language models
[McMahan et al., 2017b] H Brendan McMahan, Daniel Ra- mage, Kunal Talwar, and Li Zhang. Learning differen- tially private recurrent language models. arXiv preprint arXiv:1710.06963,
-
[28]
R´enyi differential privacy
[Mironov, 2017] Ilya Mironov. R´enyi differential privacy. In 2017 IEEE 30th computer security foundations symposium (CSF), pages 263–275. IEEE,
2017
-
[29]
Pytorch: Tensors and dy- namic neural networks in python with strong gpu acceler- ation
[Paszke et al., 2017] Adam Paszke, Sam Gross, Soumith Chintala, and Gregory Chanan. Pytorch: Tensors and dy- namic neural networks in python with strong gpu acceler- ation. PyTorch: Tensors and dynamic neural networks in Python with strong GPU acceleration, 6(3):67,
2017
-
[30]
Privacy enhanced matrix factor- ization for recommendation with local differential privacy
[Shin et al., 2018] Hyejin Shin, Sungwook Kim, Junbum Shin, and Xiaokui Xiao. Privacy enhanced matrix factor- ization for recommendation with local differential privacy. IEEE Transactions on Knowledge and Data Engineering , 30(9):1770–1782,
2018
-
[31]
D2: Decentralized training over de- centralized data
[Tang et al., 2018] Hanlin Tang, Xiangru Lian, Ming Yan, Ce Zhang, and Ji Liu. D2: Decentralized training over de- centralized data. In International Conference on Machine Learning, pages 4848–4856. PMLR,
2018
-
[32]
Tailoring gradient methods for differentially private distributed optimization
[Wang and Nedic, 2024] Yongqiang Wang and Angelia Nedic. Tailoring gradient methods for differentially private distributed optimization. IEEE Transactions on Automatic Control, 69(2):872–887,
2024
-
[34]
Efficient privacy- preserving stochastic nonconvex optimization
[Wang et al., 2019a] Lingxiao Wang, Bargav Jayaraman, David Evans, and Quanquan Gu. Efficient privacy- preserving stochastic nonconvex optimization. arXiv e- prints, pages arXiv–1910,
1910
-
[35]
Beyond inferring class representatives: User-level privacy leakage from federated learning
[Wang et al., 2019b] Zhibo Wang, Mengkai Song, Zhifei Zhang, Yang Song, Qian Wang, and Hairong Qi. Beyond inferring class representatives: User-level privacy leakage from federated learning. In IEEE INFOCOM 2019-IEEE conference on computer communications , pages 2512–
2019
-
[36]
On differentially private stochas- tic convex optimization with heavy-tailed data
[Wang et al., 2020] Di Wang, Hanshen Xiao, Srinivas De- vadas, and Jinhui Xu. On differentially private stochas- tic convex optimization with heavy-tailed data. In Inter- national Conference on Machine Learning, pages 10081– 10091. PMLR,
2020
-
[37]
Gradient leakage attack resilient deep learning
[Wei and Liu, 2021] Wenqi Wei and Ling Liu. Gradient leakage attack resilient deep learning. IEEE Transactions on Information Forensics and Security, 17:303–316,
2021
-
[38]
Federated learning with dif- ferential privacy: Algorithms and performance analysis
[Wei et al., 2020] Kang Wei, Jun Li, Ming Ding, Chuan Ma, Howard H Yang, Farhad Farokhi, Shi Jin, Tony QS Quek, and H Vincent Poor. Federated learning with dif- ferential privacy: Algorithms and performance analysis. IEEE Transactions on Information Forensics and Security, 15:...
2020
-
[39]
Securing distributed sgd against gradient leakage threats
[Wei et al., 2023] Wenqi Wei, Ling Liu, Jingya Zhou, Ka-Ho Chow, and Yanzhao Wu. Securing distributed sgd against gradient leakage threats. IEEE Transactions on Parallel and Distributed Systems,
2023
-
[41]
Fashion-mnist: a novel image dataset for bench- marking machine learning algorithms
[Xiao et al., 2017] Han Xiao, Kashif Rasul, and Roland V ollgraf. Fashion-mnist: a novel image dataset for bench- marking machine learning algorithms. arXiv preprint arXiv:1708.07747,
2017 arXiv
-
[42]
A(DP)ˆ2SGD: Asynchronous decentralized parallel stochastic gradient descent with differential privacy
[Xu et al., 2022] Jie Xu, Wei Zhang, and Fei Wang. A(DP)ˆ2SGD: Asynchronous decentralized parallel stochastic gradient descent with differential privacy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(11):8036–8047,
2022
-
[43]
Decentralized parallel sgd with privacy preserva- tion in vehicular networks
[Yu et al., 2021] Dongxiao Yu, Zongrui Zou, Shuzhen Chen, Youming Tao, Bing Tian, Weifeng Lv, and Xiuzhen Cheng. Decentralized parallel sgd with privacy preserva- tion in vehicular networks. IEEE Transactions on Vehicu- lar Technology, 70(6):5211–5220,
2021
-
[44]
Differentially private federated tempo- ral difference learning
[Zeng et al., 2021] Yiming Zeng, Yixuan Lin, Yuanyuan Yang, and Ji Liu. Differentially private federated tempo- ral difference learning. IEEE Transactions on Parallel and Distributed Systems, 33(11):2714–2726,
2021
-
[45]
Efficient private erm for smooth objec- tives
[Zhang et al., 2017] Jiaqi Zhang, Kai Zheng, Wenlong Mou, and Liwei Wang. Efficient private erm for smooth objec- tives. In Proceedings of the 26th International Joint Con- ference on Artificial Intelligence, pages 3922–3928,
2017
-
[46]
Optimizing the numbers of queries and replies in convex federated learn- ing with differential privacy
[Zhou et al., 2023] Yipeng Zhou, Xuezheng Liu, Yao Fu, Di Wu, Jessie Hui Wang, and Shui Yu. Optimizing the numbers of queries and replies in convex federated learn- ing with differential privacy. IEEE Transactions on De- pendable and Secure Computing,
2023
-
[47]
Deep leakage from gradients
[Zhu et al., 2019] Ligeng Zhu, Zhijian Liu, and Song Han. Deep leakage from gradients. Advances in neural infor- mation processing systems, 32,
2019
-
[48]
R-FAST: Robust fully-asynchronous stochastic gradient tracking over general topology
[Zhu et al., 2024] Zehan Zhu, Ye Tian, Yan Huang, Jinming Xu, and Shibo He. R-FAST: Robust fully-asynchronous stochastic gradient tracking over general topology. IEEE Transactions on Signal and Information Processing over Networks, 10:665–678,
2024
-
[49]
Parallelized stochastic gra- dient descent
[Zinkevich et al., 2010] Martin Zinkevich, Markus Weimer, Lihong Li, and Alex Smola. Parallelized stochastic gra- dient descent. Advances in neural information processing systems, 23,
2010
-
[50]
Then, we have kX l=0 λk−lvl !2 ⩽ 1 1−λ kX l=0 λk−l vl 2
Let vk ∞ k=0 be a non-negative sequence andλ∈ (0, 1). Then, we have kX l=0 λk−lvl !2 ⩽ 1 1−λ kX l=0 λk−l vl 2 . (15) Proof. Using Cauchy-Swarchz inequality, we have kX l=0 λk−lvl !2 = kX l=0 λ k−l 2 λ k−l 2 vl !2 ⩽ kX l=0 λ k−l 2 2 · kX l=0 λ k−l 2 vl 2 ⩽ 1 1−λ kX l=0 λk−l vl ...
2019
-
[51]
Given a privacy parameter µtot calculated according to (ϵ,δ ) based on (4), we use it to bound the above ˆµtot, i.e., ˆµtot < 1 J vuut2 K−1X k=0 µ2 k = 1 J vuut2 K−1X k=0 C2 k ˜σ2· ˜σ2 k ⩽µtot, and we obtain ˜σ ⩾ 1 Jµtot vuut2 K−1X k=0 C2 k ˜σ2 k , which is sufficient to guara...
2019
-
[2003]
Decentralized deep learning with arbitrary communication compression
[Koloskova et al., 2019] Anastasiia Koloskova, Tao Lin, Se- bastian Urban Stich, and Martin Jaggi. Decentralized deep learning with arbitrary communication compression. In Proceedings of the 8th International Conference on Learn- ing Representations,
2019
-
[2006]
The algorithmic foundations of differential privacy
[Dwork et al., 2014] Cynthia Dwork, Aaron Roth, et al. The algorithmic foundations of differential privacy. Founda- tions and Trends® in Theoretical Computer Science, 9(3– 4):211–407,
2014
-
[2009]
Distributed training of deep learn- ing models: A taxonomic perspective
[Langer et al., 2020] Matthias Langer, Zhen He, Wenny Ra- hayu, and Yanbo Xue. Distributed training of deep learn- ing models: A taxonomic perspective. IEEE Transactions on Parallel and Distributed Systems , 31(12):2802–2818,
2020
-
[2014]
Adap DP- FL: Differentially private federated learning with adaptive noise
[Fu et al., 2022] Jie Fu, Zhili Chen, and Xiao Han. Adap DP- FL: Differentially private federated learning with adaptive noise. In 2022 IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), pages 656–663. IEEE,
2022
-
[2015]
Deep residual learning for image recog- nition
[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778,
2016
-
[2016]
Differentially pri- vate learning with adaptive clipping
[Andrew et al., 2021] Galen Andrew, Om Thakkar, Brendan McMahan, and Swaroop Ramaswamy. Differentially pri- vate learning with adaptive clipping. Advances in Neural Information Processing Systems, 34:17455–17466,
2021
-
[2017]
Asynchronous decentralized parallel stochastic gra- dient descent
[Lian et al., 2018] Xiangru Lian, Wei Zhang, Ce Zhang, and Ji Liu. Asynchronous decentralized parallel stochastic gra- dient descent. In International Conference on Machine Learning, pages 3043–3052. PMLR,
2018
-
[2018]
Towards decentralized deep learning with differen- tial privacy
[Cheng et al., 2019] Hsin-Pai Cheng, Patrick Yu, Haojing Hu, Syed Zawad, Feng Yan, Shiyu Li, Hai Li, and Yiran Chen. Towards decentralized deep learning with differen- tial privacy. In International Conference on Cloud Com- puting, pages 130–145. Springer,
2019
-
[2019]
Deep learning with gaussian differential privacy
[Bu et al., 2020] Zhiqi Bu, Jinshuo Dong, Qi Long, and Wei- jie J Su. Deep learning with gaussian differential privacy. arXiv preprint arXiv:1911.11607,
2020 arXiv
-
[2020]
Understanding gradient clipping in private sgd: A geometric perspective
[Chen et al., 2020] Xiangyi Chen, Steven Z Wu, and Mingyi Hong. Understanding gradient clipping in private sgd: A geometric perspective. Advances in Neural Information Processing Systems, 33:13773–13782,
2020
-
[2021]
Stochastic gradient push for distributed deep learning
[Assran et al., 2019] Mahmoud Assran, Nicolas Loizou, Nicolas Ballas, and Mike Rabbat. Stochastic gradient push for distributed deep learning. In International Conference on Machine Learning, pages 344–353. PMLR,
2019
-
[2022]
Escaping from saddle points—online stochastic gra- dient for tensor decomposition
[Ge et al., 2015] Rong Ge, Furong Huang, Chi Jin, and Yang Yuan. Escaping from saddle points—online stochastic gra- dient for tensor decomposition. InConference on Learning Theory, pages 797–842. PMLR,
2015
-
[2023]
The value of collaboration in convex machine learning with differential privacy
[Wu et al., 2020] Nan Wu, Farhad Farokhi, David Smith, and Mohamed Ali Kaafar. The value of collaboration in convex machine learning with differential privacy. In2020 IEEE Symposium on Security and Privacy (SP) , pages 304–317. IEEE,
2020
-
[2024]
Differentially private empirical risk minimization revis- ited: Faster and more general
[Wang et al., 2017] Di Wang, Minwei Ye, and Jinhui Xu. Differentially private empirical risk minimization revis- ited: Faster and more general. Advances in Neural In- formation Processing Systems, 30,
2017
-
[2025]
Differentially private meta- learning
[Li et al., 2019a] Jeffrey Li, Mikhail Khodak, Sebastian Cal- das, and Ameet Talwalkar. Differentially private meta- learning. arXiv preprint arXiv:1909.05830,
1909 arXiv
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.