Pith. sign in

REVIEW 2 major objections 5 minor 83 references

Delayed Momentum Aggregation: Communication-efficient Byzantine-robust Federated Learning with Partial Participation

T0 review · 2 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read By aggregating cached momentum from non-sampled clients alongside fresh momentum from sampled clients, D-Byz-SGDM ensures the robust aggregator always sees fewer than half Byzantine clients and attains the optimal O(cδζ²/p) convergence rate

desk verdict Delayed momentum aggregation is a fresh, plausible idea, but the lower bound only satisfies the paper's own assumptions for p near 1, so the advertised tightness is unproven in the regime they test. read the letter →

arxiv 2509.02970 v3 pith:PGB47YRA submitted 2025-09-03 cs.LG math.OC

Kaoru Otsuka , Yuki Takezawa , Makoto Yamada This is my paper
classification cs.LGmath.OC
keywords Byzantine-robustfederatedlearningpartialparticipationdelayedmomentumaggregationclientsamplingrobustnon-convexoptimizationdataheterogeneitySGD
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

In federated learning only a subset of clients can be contacted each round, and when that subset happens to contain more malicious than honest clients, standard robust-aggregation defenses fail. This paper argues the failure is avoidable with a simple principle: the server should aggregate not only the momentum of the clients sampled this round, but also the most recent momentum it already has cached for every client that was not sampled. The resulting optimizer, D-Byz-SGDM, always hands the robust aggregator a full set of client momentum vectors, so the Byzantine fraction it sees is the global fraction δ < 1/2 rather than a sampled fraction that can exceed 1/2. The paper proves convergence to an O(cδζ²/p) neighborhood of a stationary point and proves a lower bound Ω(δζ²/p) for any algorithm in this setting, claiming the rate is optimal. Because the cached values require no extra communication, the method costs no more per round than ordinary partial-participation federated learning.

What carries the argument

Delayed momentum aggregation: the server applies the robust aggregator to the union of fresh momentum from sampled clients and cached momentum from non-sampled clients, keeping one momentum vector per client. This is the mechanism that makes the effective Byzantine fraction seen by the aggregator always δ < 1/2; without it, a round's sample can be dominated by Byzantine clients and no robust aggregator can help. The proof also relies on the (δ,c)-robust aggregator property, a Lyapunov function combining objective gap, momentum error, and gradient norm, and a mimic-attack lower-bound construction.

What would settle it

Run the paper's World 1/World 2 lower-bound functions with p=0.5 and δ=0.2 and compute (1/G)Σ_{i∈G} ||∇f_i(x)−∇f(x)||²; it exceeds ζ², so the claimed Ω(δζ²/p) optimality bound does not apply in that regime. Alternatively, exhibit any algorithm that under the paper's assumptions achieves expected squared gradient error o(δζ²/p) at p=0.5, δ=0.2; that would directly refute the lower bound.

Watch

Extended reading notes

Core claim

The paper's central discovery is that partial participation itself—not just the adversarial clients—causes Byzantine robustness to fail, and that the server can neutralize this by never discarding the last momentum received from a client. Concretely, D-Byz-SGDM maintains a per-client momentum vector; in each round each client is included independently with probability p, sampled clients refresh their momentum with a local stochastic gradient, and non-sampled clients keep their cached vector. The robust aggregator then operates on the union of both groups, i.e. on n vectors of which exactly δn are Byzantine, so the aggregator's minority-guarantee assumption holds in every round regardless of

Load-bearing premise

The optimality result depends on a lower-bound construction whose two worlds satisfy the paper's own ζ²-heterogeneity bound only when p² + δp − 1 ≥ 0; at small participation rates such as p=0.5 the Ω(δζ²/p) lower bound is not established, and the whole analysis additionally assumes bounded client gradients.

Editorial extensions

If this is right

  • Existing Byzantine-robust aggregators can be deployed under partial participation by adding the server-side momentum cache; the convergence guarantee requires no change in clients' communication.
  • At full participation p=1 the rate recovers prior full-participation results; at δ=0 it matches standard partial-participation rates, so the mechanism is safe in benign settings.
  • The lower bound implies any algorithm that sees each honest client's fresh update only with probability p must pay a residual error of at least Ω(δζ²/p) in heterogeneous, Byzantine environments; this price cannot be removed by better aggregation rules.
  • The method is robust for any Bernoulli sampling probability p, including regimes where the sampled set is, with high probability, a Byzantine majority; the cached vectors ensure this never reaches the aggregator.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The optimality claim is proven only for participation rates satisfying p² + δp − 1 ≥ 0; the paper's own lower-bound construction violates its heterogeneity assumption for small p (e.g. p=0.5, δ=0.2), so extending Ω(δζ²/p) to the full range of p is a natural next step rather than an established result.
  • The same cached-momentum idea should transfer to non-Bernoulli client-selection schemes such as importance sampling or power-of-choice, where the effective Byzantine fraction seen by the aggregator may need a different but analogous calculation; this is a direct testable extension the paper names as future work.
  • The abstract's claim of robustness at 10% participation does not match the experiments reported, which use p=0.5; a reader wanting to use the method in a 10%-participation regime should check that regime directly, since the formal upper bound holds for all p but the lower-bound optimality does not.
  • The empirical finding that delayed momentum acts as an implicit regularizer in non-Byzantine non-IID training suggests the method may improve plain federated learning even without adversaries; on the reported benchmark this appears, but larger benchmarks would test whether the effect survives.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper studies Byzantine-robust federated optimization under partial participation. The proposed optimizer, D-Byz-SGDM, applies a robust aggregation rule to the union of fresh momentum from sampled clients and cached momentum from non-sampled clients, so that the aggregator always sees the global Byzantine fraction δ < 1/2. Under L-smoothness, bounded stochastic variance, ζ²-heterogeneity, bounded gradients, and a (δ,c)-robust aggregator, the authors prove convergence to a first-order stationary point at rate O(cδζ²/p + ...). They also give a two-world lower bound of Ω(δζ²/p) and claim information-theoretic optimality in δ, ζ², and p. Experiments on MNIST compare the method with FedAvg, FedAvg-M, and a heuristic momentum extension of Byz-VR-MARINA-PP across five robust aggregators and six attacks, using p = 0.5.

Significance. The algorithmic idea is simple and communication-neutral: the server reuses cached per-client momentum for non-sampled clients, so no extra communication is required, and the robust aggregator always sees the global Byzantine fraction. If the convergence analysis were fully verified, this would be a useful extension of prior full-participation Byzantine-robustness results to partial participation, recovering those results at p = 1. The experimental evaluation is broad, covering multiple aggregators and attacks. However, the claimed optimality is currently not established: the lower-bound construction violates the paper's own heterogeneity assumption in the small-p regime used in the experiments, and the key aggregation-error lemma in the upper-bound proof is not derived in sufficient detail. I see no circularity; the lower bound is a genuine two-world construction, though flawed as written.

major comments (2)
  1. [Appendix G.1 (Theorem 7)] The heterogeneity verification does not hold for the participation rates used in the paper. The construction defines biased clients as {1,...,pδn}, so the fraction of biased clients is pδ, not δ, but the displayed computation weights the biased term by δn/n. Using the definitionally correct weights pδ and 1−pδ, the averaged squared deviation is ((1−pδ)^2/p^2 + δ(1−pδ)/p)ζ^2 = (1−pδ)ζ^2/p^2, which is ≤ζ^2 only when p^2+δp−1≥0. For δ=0.2 this requires p≥0.905, whereas Section 5 and Fig. 1 use p=0.5; there the deviation is 3.6ζ^2, violating Assumption 4. Thus Theorem 7's Ω(δζ^2/p) lower bound is not proven for small p, and the claim that O(cδζ^2/p) is information-theoretically optimal is unsupported as stated.
  2. [Appendix F.1, Lemma 14] The derivation of the heterogeneous aggregation error is not verifiable as written. After the expansion over participation patterns, the proof jumps to a bound of the form (1−αp)E_i||hat m^{t-1}_i−hat m^{t-1}||^2 + αζ^2 + 3αp(1−p)·4L^2η^2B^2(...), without showing the summation over subsets. The subsequent unrolling to ζ^2/p (1−(1−αp)^t) is also omitted. Since this lemma is the source of the leading cδζ^2/p residual in Theorem 6, the central convergence claim cannot be checked from the appendix as written. Please provide the complete derivation or a precise reference to a full proof.
minor comments (5)
  1. [Abstract vs Section 5] The abstract advertises a 10% partial participation rate and names the optimizer DeMoA, but the body uses p=0.5 in all experiments and calls the method D-Byz-SGDM. Please reconcile these statements.
  2. [Assumption 5, Remark] The remark states that the bounded-gradient assumption B is not essential and can be removed with more refined techniques, but no proof or precise citation is given. Since B appears in the square-root and staleness terms of Theorem 6, please either provide the removal argument or soften the remark.
  3. [Appendix B.3] The sentence 'configured n=25 clients with 5 Byzantine clients (20' appears truncated; please correct. Also check Table 1 for formatting issues such as 'p = 0.5 same'.
  4. [References] References [11] and [12] appear to be duplicate entries of the same paper; please consolidate.
  5. [Theorem 7 scope] Theorem 7 is stated for deterministic strongly convex functions, while the upper bound is nonconvex and stochastic. The claimed optimality 'in its dependence on δ, ζ², and p' should clarify that it applies to the non-vanishing residual term only, not to the transient or stochastic terms.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation; lower-bound assumption gap is a correctness issue, not circularity.

full rationale

The derivation chain is self-contained: the upper bound (Theorems 6 and 16) is obtained from the robust-aggregator property (Assumption 1, imported from Karimireddy et al. [45], an external source) and standard smoothness/variance/heterogeneity assumptions, with no parameter fitted to data and no term defined as the quantity to be predicted. The delayed-momentum guarantee that the aggregator always sees the global Byzantine fraction δ is a definitional feature of the algorithm: the aggregator input is the union of all n cached/fresh client momenta, so the algorithm's own construction ensures the minority condition; it is not a prediction derived from a fit. The lower bound (Theorem 7, Appendix G) is a classical two-world indistinguishability construction: World 1 has pδn biased honest clients and World 2 has Byzantine clients mimicking them, and the bound δζ²/(4p) follows from the distance between the two optima and the pigeonhole principle, independent of the upper-bound analysis. This is not circular. The paper itself flags Assumption 5 as strong and non-essential (Sec. 4.1 Remark), and the G.1 heterogeneity verification appears algebraically inconsistent for small p (e.g., p=0.5, δ=0.2, as used in experiments), but that is an assumption-satisfaction/correctness gap in the lower-bound proof, not a reduction of the conclusion to its own inputs. The only self-citation (Takezawa et al. [73]) appears in the related-work discussion and is not load-bearing. No circular step meets the quoted-reduction bar.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

The central claim rests on standard FL assumptions plus the robust aggregator property. No new physical or mathematical entities are introduced. The main unstated load-bearing condition is the heterogeneity verification in the lower bound construction, which is not satisfied for small p.

assumptions (6)
  • domain assumption Assumption 1: (δ, c)-Robust Aggregator property
    Used throughout to characterize the aggregator's bias bound. It is an assumption about the aggregation rule, not proved in this paper.
  • domain assumption Assumption 2: L-smoothness and lower boundedness of objectives
    Standard smoothness assumption invoked in Lemma 8 and the descent lemma.
  • domain assumption Assumption 3: bounded variance and unbiased stochastic gradients
    Standard noise assumption used in momentum error bounds.
  • domain assumption Assumption 4: ζ²-heterogeneity bound
    Controls the average deviation of local gradients from the global gradient; central to the O(cδζ²/p) term.
  • domain assumption Assumption 5: bounded gradients (||∇f_i(x)|| ≤ B)
    Strong assumption used to bound staleness in the momentum analysis; the paper admits it is not essential but relies on it.
  • domain assumption Independent Bernoulli sampling with probability p
    The algorithm and analysis assume each client is sampled independently with probability p; this is the partial participation model.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Delayed Momentum Aggregation: Communication-efficient Byzantine-robust Federated Learning with Partial Participation." pith.science (2026). https://pith.science/paper/PGB47YRA

@misc{pith2026250902970,
  author       = {Pith},
  title        = {Pith review of: Delayed Momentum Aggregation: Communication-efficient Byzantine-robust Federated Learning with Partial Participation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PGB47YRA}},
  note         = {Machine review of arXiv:2509.02970}
}
read the original abstract

Partial participation is essential for communication-efficient federated learning at scale, yet existing Byzantine-robust methods typically assume full client participation. In the partial participation setting, a majority of the sampled clients may be Byzantine, once Byzantine clients dominate, existing methods break down immediately. We introduce delayed momentum aggregation, a principle where the central server aggregates cached momentum from non-sampled clients along with fresh momentum from sampled clients. This principle ensures Byzantine clients remain a minority from the server's perspective even when they dominate the sampled set. We instantiate this principle in our optimizer DeMoA. We analyze the convergence rate of DeMoA, showing that DeMoA is Byzantine-robust under partial participation. Experiments show that, with 20% Byzantine ratio and only 10% partial participation rate, DeMoA achieves the best accuracy even when existing methods fail empirically.

Figures

Figures reproduced from arXiv: 2509.02970 by the authors.

Figure 1
Figure 1. Training dynamics with centered clipping (cp), n = 25, δ = 0.2, p = 0.5 across six attacks. D-Byz-SGDM outperformed all baselines, while FedAvg/FedAvg-M diverged when Byzantine majority was sampled. See Appendix C for other aggregators. 5. Experiments We evaluate D-Byz-SGDM under various Byzantine attacks with partial participation (p = 0.5) by training an MLP on MNIST across IID and non-IID data partitions. We comp… view at source ↗
Figure 2
Figure 2. (non-Byzantine) Federated Learning with Partial Participation Training dynamics across optimizers with n = 20, δ = 0, and p = 0.5. Byz-VR-MARINA-PP under￾performed in all cases, while D-Byz-SGDM surpassed FedAvg-M under both IID and non-IID partitions, suggesting benefits from mitigating heterogeneity-induced drift. 5.2. Baseline Performance without Byzantine Clients We also examined the non-Byzantine setting (δ = 0… view at source ↗
Figure 3
Figure 3. avg (simple mean) under Byzantine attacks with partial participation. 2.5 5.0 7.5 10.0 Epoch 0 20 40 60 80 100 Validation Accuracy (%) ALIE 2.5 5.0 7.5 10.0 Epoch 0 20 40 60 80 100 Bit-Flipping 2.5 5.0 7.5 10.0 Epoch 0 20 40 60 80 100 INFINITY 2.5 5.0 7.5 10.0 Epoch 0 20 40 60 80 100 IPM 2.5 5.0 7.5 10.0 Epoch 0 20 40 60 80 100 Label-Flipping 2.5 5.0 7.5 10.0 Epoch 0 20 40 60 80 100 IID Mimic 2.5 5.0 7.5 10.0 Epoch … view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: cm (coordinate-wise median) under Byzantine attacks with partial participation. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_4.png]
Figure 5
Figure 5. Figure 5: krum / Multi-Krum under Byzantine attacks with partial participation. 2.5 5.0 7.5 10.0 Epoch 0 20 40 60 80 100 Validation Accuracy (%) ALIE 2.5 5.0 7.5 10.0 Epoch 0 20 40 60 80 100 Bit-Flipping 2.5 5.0 7.5 10.0 Epoch 0 20 40 60 80 100 INFINITY 2.5 5.0 7.5 10.0 Epoch 0 …
Figure 6
Figure 6. Figure 6: rfa (Robust Federated Averaging) under Byzantine attacks with partial participation. Appendix D. Notation Summary for Convergence Analysis The table 3z summarizes the key notations used in the convergence analysis of D-Byz-SGDM in both homogeneous and heterogeneous set…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

83 extracted references · 73 canonical work pages

  1. [1]

    Dhillon, and Ufuk Topcu

    Anish Acharya, Abolfazl Hashemi, Prateek Jain, Sujay Sanghavi, Inderjit S. Dhillon, and Ufuk Topcu. Robust training in high dimensions via block coordinate geometric median descent. In International Conference on Artificial Intelligence and Statistics, 2022

    2022

  2. [2]

    Optimal gradient compression for distributed and federated learning

    Alyazeed Albasyoni, Mher Safaryan, Laurent Condat, and Peter Richt ´arik. Optimal gradient compression for distributed and federated learning. ArXiv preprint, abs/2010.03246, 2020

    arXiv 2010

  3. [3]

    Alghunaim

    Sulaiman A. Alghunaim. Local exact-diffusion for decentralized optimization and learning. IEEE Transactions on Automatic Control, 69(11):7371–7386, 2024

    2024

  4. [4]

    QSGD: communication-efficient SGD via gradient quantization and encoding

    Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, and Milan V ojnovic. QSGD: communication-efficient SGD via gradient quantization and encoding. In Advances in Neural Information Processing Systems, 2017

    2017

  5. [5]

    Byzantine stochastic gradient descent

    Dan Alistarh, Zeyuan Allen-Zhu, and Jerry Li. Byzantine stochastic gradient descent. In Advances in Neural Information Processing Systems, 2018

    work page 2018

  6. [6]

    Byzantine-resilient non-convex stochastic gradient descent

    Zeyuan Allen-Zhu, Faeze Ebrahimianghazani, Jerry Li, and Dan Alistarh. Byzantine-resilient non-convex stochastic gradient descent. In International Conference on Learning Representa- tions, 2021. 10 DELAYED MOMENTUM AGGREGATION

    work page 2021

  7. [7]

    Fixing by mixing: A recipe for optimal byzantine ML under heterogeneity

    Youssef Allouah, Sadegh Farhadkhani, Rachid Guerraoui, Nirupam Gupta, Rafael Pinot, and John Stephan. Fixing by mixing: A recipe for optimal byzantine ML under heterogeneity. In International Conference on Artificial Intelligence and Statistics, 2023

    work page 2023

  8. [8]

    Byzantine-robust federated learning: Impact of client sub- sampling and local updates

    Youssef Allouah, Sadegh Farhadkhani, Rachid Guerraoui, Nirupam Gupta, Rafael Pinot, Geo- vani Rizk, and Sasha V oitovych. Byzantine-robust federated learning: Impact of client sub- sampling and local updates. In International Conference on Machine Learning, 2024

    work page 2024

Show all 83 references
  1. [9]

    A little is enough: Circumventing defenses for distributed learning

    Gilad Baruch, Moran Baruch, and Yoav Goldberg. A little is enough: Circumventing defenses for distributed learning. In Advances in Neural Information Processing Systems, 2019

    2019

  2. [10]

    signsgd with majority vote is communication efficient and fault tolerant

    Jeremy Bernstein, Jiawei Zhao, Kamyar Azizzadenesheli, and Anima Anandkumar. signsgd with majority vote is communication efficient and fault tolerant. In International Conference on Learning Representations, 2019

    2019

  3. [12]

    Machine learning with adversaries: Byzantine tolerant gradient descent

    Peva Blanchard, El Mahdi El Mhamdi, Rachid Guerraoui, and Julien Stainer. Machine learning with adversaries: Byzantine tolerant gradient descent. In Advances in Neural Information Processing Systems, 2017

    2017

  4. [13]

    Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H

    Kallista A. Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H. Brendan McMa- han, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. Practical secure aggregation for privacy-preserving machine learning. InACM SIGSAC Conference on Computer and Com- munications ...

    2017

  5. [14]

    Curtis, and Jorge Nocedal

    L ´eon Bottou, Frank E. Curtis, and Jorge Nocedal. Optimization methods for large-scale ma- chine learning. SIAM Rev., 60(2):223–311, 2018

    2018

  6. [15]

    Charles, and Dimitris S

    Lingjiao Chen, Hongyi Wang, Zachary B. Charles, and Dimitris S. Papailiopoulos. DRACO: byzantine-resilient distributed training via redundant gradients. In International Conference on Machine Learning, 2018

    2018

  7. [16]

    Optimal client sampling for federated learning

    Wenlin Chen, Samuel Horv ´ath, and Peter Richt ´arik. Optimal client sampling for federated learning. Trans. Mach. Learn. Res., 2022

    2022

  8. [17]

    Momentum benefits non-iid federated learning simply and provably

    Ziheng Cheng, Xinmeng Huang, Pengfei Wu, and Kun Yuan. Momentum benefits non-iid federated learning simply and provably. In International Conference on Learning Representa- tions, 2024

    2024

  9. [18]

    Client selection in federated learning: Con- vergence analysis and power-of-choice selection strategies

    Yae Jee Cho, Jianyu Wang, and Gauri Joshi. Client selection in federated learning: Con- vergence analysis and power-of-choice selection strategies. ArXiv preprint, abs/2010.01243, 2020

    2010 arXiv

  10. [19]

    Tehila Dahan and Kfir Y . Levy. Weight for robustness: A comprehensive approach towards op- timal fault-tolerant asynchronous ML. InAdvances in Neural Information Processing Systems, 2024. 11 DELAYED MOMENTUM AGGREGATION

    2024

  11. [20]

    Fault tolerant ML: efficient meta-aggregation and syn- chronous training

    Tehila Dahan and Kfir Yehuda Levy. Fault tolerant ML: efficient meta-aggregation and syn- chronous training. In International Conference on Machine Learning, 2024

    2024

  12. [21]

    AGGREGATHOR: byzantine machine learning via robust gradient ag- gregation

    Georgios Damaskinos, El-Mahdi El-Mhamdi, Rachid Guerraoui, Arsany Guirguis, and S´ebastien Rouault. AGGREGATHOR: byzantine machine learning via robust gradient ag- gregation. In Proceedings of Machine Learning and Systems, 2019

    2019

  13. [22]

    Alexandre d’Aspremont, Damien Scieur, and Adrien B. Taylor. Acceleration methods. Found. Trends Optim., 5(1-2):1–245, 2021

    2021

  14. [23]

    Deepesh Data and Suhas N. Diggavi. Byzantine-resilient SGD in high dimensions on hetero- geneous data. In IEEE International Symposium on Information Theory, 2021

    2021

  15. [24]

    Deepesh Data and Suhas N. Diggavi. Byzantine-resilient high-dimensional SGD with local iterations on heterogeneous data. In International Conference on Machine Learning, 2021

    2021

  16. [25]

    On the ineffectiveness of variance reduced optimization for deep learning

    Aaron Defazio and L ´eon Bottou. On the ineffectiveness of variance reduced optimization for deep learning. In Advances in Neural Information Processing Systems, 2019

    2019

  17. [26]

    Collaborative learning in the jungle (decentralized, byzantine, heterogeneous, asynchronous and nonconvex learning)

    El-Mahdi El-Mhamdi, Sadegh Farhadkhani, Rachid Guerraoui, Arsany Guirguis, L ˆe-Nguyˆen Hoang, and S´ebastien Rouault. Collaborative learning in the jungle (decentralized, byzantine, heterogeneous, asynchronous and nonconvex learning). In Advances in Neural Information Process...

    2021

  18. [27]

    Byzantine machine learning made easy by resilient averaging of momentums

    Sadegh Farhadkhani, Rachid Guerraoui, Nirupam Gupta, Rafael Pinot, and John Stephan. Byzantine machine learning made easy by resilient averaging of momentums. InInternational Conference on Machine Learning, 2022

    2022

  19. [28]

    Momentum provably improves error feedback! In Advances in Neural Information Processing Systems, 2023

    Ilyas Fatkhullin, Alexander Tyurin, and Peter Richt ´arik. Momentum provably improves error feedback! In Advances in Neural Information Processing Systems, 2023

    2023

  20. [29]

    Clustered sampling: Low- variance and improved representativity for clients selection in federated learning

    Yann Fraboni, Richard Vidal, Laetitia Kameni, and Marco Lorenzi. Clustered sampling: Low- variance and improved representativity for clients selection in federated learning. In Interna- tional Conference on Machine Learning, 2021

    2021

  21. [30]

    A general theory for client sampling in federated learning

    Yann Fraboni, Richard Vidal, Laetitia Kameni, and Marco Lorenzi. A general theory for client sampling in federated learning. In International Workshop on Trustworthy Federated Learning. Springer, 2022

    2022

  22. [31]

    Handbook of convergence theorems for (stochastic) gradient methods

    Guillaume Garrigos and Robert M Gower. Handbook of convergence theorems for (stochastic) gradient methods. ArXiv preprint, abs/2301.11235, 2023

    2023 arXiv

  23. [32]

    Stochastic first- and zeroth-order methods for nonconvex stochastic programming

    Saeed Ghadimi and Guanghui Lan. Stochastic first- and zeroth-order methods for nonconvex stochastic programming. SIAM J. Optim., 23(4):2341–2368, 2013

    2013

  24. [33]

    Byzfl: Research framework for robust federated learning, 2025

    Marc Gonz ´alez, Rachid Guerraoui, Rafael Pinot, Geovani Rizk, John Stephan, and Franc ¸ois Ta¨ıani. Byzfl: Research framework for robust federated learning, 2025

    2025

  25. [34]

    Variance reduction is an antidote to byzantines: Better rates, weaker assumptions and communication compression as a cherry on the top

    Eduard Gorbunov, Samuel Horv´ath, Peter Richt´arik, and Gauthier Gidel. Variance reduction is an antidote to byzantines: Better rates, weaker assumptions and communication compression as a cherry on the top. In International Conference on Learning Representations, 2023. 12 DEL...

    2023

  26. [35]

    Fast federated learning in the presence of arbitrary device unavailability

    Xinran Gu, Kaixuan Huang, Jingzhao Zhang, and Longbo Huang. Fast federated learning in the presence of arbitrary device unavailability. In Advances in Neural Information Processing Systems, 2021

    2021

  27. [36]

    New proximal point algorithms for convex minimization.SIAM J

    Osman G ¨uler. New proximal point algorithms for convex minimization.SIAM J. Optim., 2(4): 649–664, 1992

    1992

  28. [37]

    Federated learning with compression: Unified analysis and sharp guarantees

    Farzin Haddadpour, Mohammad Mahdi Kamani, Aryan Mokhtari, and Mehrdad Mahdavi. Federated learning with compression: Unified analysis and sharp guarantees. In International Conference on Artificial Intelligence and Statistics, 2021

    2021

  29. [38]

    Byzantine-robust decentralized learning via clippedgossip

    Lie He, Sai Praneeth Karimireddy, and Martin Jaggi. Byzantine-robust decentralized learning via clippedgossip. arXiv preprint arXiv:2202.01545, 2022

    2022 arXiv

  30. [39]

    Samuel Horv ´ath, Dmitry Kovalev, Konstantin Mishchenko, Peter Richt´arik, and Sebastian U. Stich. Stochastic distributed learning with gradient quantization and double-variance reduc- tion. Optim. Methods Softw., 38(1):91–106, 2023

    2023

  31. [40]

    Xiaowen Jiang, Anton Rodomanov, and Sebastian U. Stich. Stabilized proximal-point methods for federated optimization. In Advances in Neural Information Processing Systems, 2024

    2024

  32. [41]

    Brendan McMahan, Brendan Avent, Aur ´elien Bellet, Mehdi Bennis, Ar- jun Nitin Bhagoji, Kallista A

    Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aur ´elien Bellet, Mehdi Bennis, Ar- jun Nitin Bhagoji, Kallista A. Bonawitz, Zachary Charles, Graham Cormode, Rachel Cum- mings, Rafael G. L. D’Oliveira, Hubert Eichner, Salim El Rouayheb, David Evans, Josh Gardner, Zachary Ga...

    2021

  33. [42]

    Mime: Mimicking centralized stochastic algo- rithms in federated learning

    Sai Praneeth Karimireddy, Martin Jaggi, Satyen Kale, Mehryar Mohri, Sashank J Reddi, Se- bastian U Stich, and Ananda Theertha Suresh. Mime: Mimicking centralized stochastic algo- rithms in federated learning. ArXiv preprint, abs/2008.03606, 2020

    2008 arXiv

  34. [43]

    Reddi, Sebastian U

    Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank J. Reddi, Sebastian U. Stich, and Ananda Theertha Suresh. SCAFFOLD: stochastic controlled averaging for federated learning. In International Conference on Machine Learning, 2020

    2020

  35. [44]

    Learning from history for byzantine robust optimization

    Sai Praneeth Karimireddy, Lie He, and Martin Jaggi. Learning from history for byzantine robust optimization. In International Conference on Machine Learning, 2021

    2021

  36. [45]

    Byzantine-robust learning on hetero- geneous datasets via bucketing

    Sai Praneeth Karimireddy, Lie He, and Martin Jaggi. Byzantine-robust learning on hetero- geneous datasets via bucketing. In International Conference on Learning Representations , 2022. 13 DELAYED MOMENTUM AGGREGATION

    2022

  37. [46]

    Distributed learning with compressed gradients

    Sarit Khirirat, Hamid Reza Feyzmahdavian, and Mikael Johansson. Distributed learning with compressed gradients. ArXiv preprint, abs/1806.06573, 2018

    2018 arXiv

  38. [47]

    Anastasia Koloskova, Nicolas Loizou, Sadra Boreiri, Martin Jaggi, and Sebastian U. Stich. A unified theory of decentralized SGD with changing topology and local updates. In Interna- tional Conference on Machine Learning, 2020

    2020

  39. [48]

    Stich, and Martin Jaggi

    Anastasia Koloskova, Sebastian U. Stich, and Martin Jaggi. Sharper convergence guarantees for asynchronous SGD for distributed and federated learning. In Advances in Neural Informa- tion Processing Systems, 2022

    2022

  40. [49]

    Gasnikov, and Gesualdo Scutari

    Dmitry Kovalev, Aleksandr Beznosikov, Ekaterina Borodich, Alexander V . Gasnikov, and Gesualdo Scutari. Optimal gradient sliding and its application to optimal distributed opti- mization under similarity. In Advances in Neural Information Processing Systems, 2022

    2022

  41. [50]

    Shostak, and Marshall C

    Leslie Lamport, Robert E. Shostak, and Marshall C. Pease. The byzantine generals problem. In Concurrency: the Works of Leslie Lamport, pages 203–226. 2019

    2019

  42. [51]

    First-order and stochastic optimization methods for machine learning

    Guanghui Lan. First-order and stochastic optimization methods for machine learning . Springer, 2020

    2020

  43. [52]

    Giannakis, and Qing Ling

    Liping Li, Wei Xu, Tianyi Chen, Georgios B. Giannakis, and Qing Ling. RSA: byzantine- robust stochastic aggregation methods for distributed learning from heterogeneous datasets. In AAAI Conference on Artificial Intelligence, 2019

    2019

  44. [53]

    Federated optimization in heterogeneous networks

    Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. In Proceedings of Machine Learn- ing and Systems, 2020

    2020

  45. [54]

    PAGE: A simple and optimal probabilistic gradient estimator for nonconvex optimization

    Zhize Li, Hongyan Bao, Xiangliang Zhang, and Peter Richt ´arik. PAGE: A simple and optimal probabilistic gradient estimator for nonconvex optimization. In International Conference on Machine Learning, 2021

    2021

  46. [55]

    A universal catalyst for first-order opti- mization

    Hongzhou Lin, Julien Mairal, and Za ¨ıd Harchaoui. A universal catalyst for first-order opti- mization. In Advances in Neural Information Processing Systems, 2015

    2015

  47. [56]

    Byzantine ro- bustness and partial participation can be achieved at once: Just clip gradient differences

    Grigory Malinovsky, Peter Richt ´arik, Samuel Horv ´ath, and Eduard Gorbunov. Byzantine ro- bustness and partial participation can be achieved at once: Just clip gradient differences. In Advances in Neural Information Processing Systems, 2024

    2024

  48. [57]

    Communication-efficient learning of deep networks from decentralized data

    Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Ag¨uera y Arcas. Communication-efficient learning of deep networks from decentralized data. In International Conference on Artificial Intelligence and Statistics, 2017

    2017

  49. [58]

    The hidden vulnerability of distributed learning in byzantium

    El Mahdi El Mhamdi, Rachid Guerraoui, and S ´ebastien Rouault. The hidden vulnerability of distributed learning in byzantium. In International Conference on Machine Learning, 2018

    2018

  50. [59]

    Distributed momentum for byzantine-resilient stochastic gradient descent

    El Mahdi El Mhamdi, Rachid Guerraoui, and S ´ebastien Rouault. Distributed momentum for byzantine-resilient stochastic gradient descent. In International Conference on Learning Rep- resentations, 2021. 14 DELAYED MOMENTUM AGGREGATION

    2021

  51. [60]

    Bach, Mathieu Even, and Blake E

    Konstantin Mishchenko, Francis R. Bach, Mathieu Even, and Blake E. Woodworth. Asyn- chronous SGD beats minibatch SGD under arbitrary delays. In Advances in Neural Informa- tion Processing Systems, 2022

    2022

  52. [61]

    Distributed learning with compressed gradient differences

    Konstantin Mishchenko, Eduard Gorbunov, Martin Tak ´aˇc, and Peter Richt ´arik. Distributed learning with compressed gradient differences. Optimization Methods and Software , pages 1–16, 2024

    2024

  53. [62]

    Renato D. C. Monteiro and Benar Fux Svaiter. An accelerated hybrid proximal extragradient method for convex optimization and its implications to second-order methods.SIAM J. Optim., 23(2):1092–1125, 2013

    2013

  54. [63]

    Lectures on convex optimization

    Yurii Nesterov. Lectures on convex optimization. Springer, 2018

    2018

  55. [64]

    Billion-scale federated learning on mobile clients: a submodel design with tunable privacy

    Chaoyue Niu, Fan Wu, Shaojie Tang, Lifeng Hua, Rongfei Jia, Chengfei Lv, Zhihua Wu, and Guihai Chen. Billion-scale federated learning on mobile clients: a submodel design with tunable privacy. In Annual International Conference on Mobile Computing and Networking , 2020

    2020

  56. [65]

    Woodworth, Brian Bullins, and Nati Srebro

    Kumar Kshitij Patel, Lingxiao Wang, Blake E. Woodworth, Brian Bullins, and Nati Srebro. Towards optimal communication complexity in distributed non-convex optimization. In Ad- vances in Neural Information Processing Systems, 2022

    2022

  57. [66]

    Kakade, and Za ¨ıd Harchaoui

    Krishna Pillutla, Sham M. Kakade, and Za ¨ıd Harchaoui. Robust aggregation for federated learning. IEEE Trans. Signal Process., 70:1142–1154, 2022

    2022

  58. [67]

    Distributed stochastic gradient tracking methods

    Shi Pu and Angelia Nedi ´c. Distributed stochastic gradient tracking methods. Mathematical Programming, 187(1):409–457, 2021

    2021

  59. [68]

    Charles, and Dimitris S

    Shashank Rajput, Hongyi Wang, Zachary B. Charles, and Dimitris S. Papailiopoulos. DETOX: A redundancy-based framework for faster and more robust gradient aggregation. In Advances in Neural Information Processing Systems, 2019

    2019

  60. [69]

    Communication compression for byzantine robust learning: New efficient algorithms and im- proved rates

    Ahmad Rammal, Kaja Gruntkowska, Nikita Fedin, Eduard Gorbunov, and Peter Richt ´arik. Communication compression for byzantine robust learning: New efficient algorithms and im- proved rates. In International Conference on Artificial Intelligence and Statistics, 2024

    2024

  61. [70]

    On the byzantine robustness of clustered federated learning

    Felix Sattler, Klaus-Robert M ¨uller, Thomas Wiegand, and Wojciech Samek. On the byzantine robustness of clustered federated learning. In IEEE International Conference on Acoustics, Speech and Signal Processing, 2020

    2020

  62. [71]

    Ordered momentum for asynchronous SGD

    Chang-Wei Shi, Yi-Rui Yang, and Wu-Jun Li. Ordered momentum for asynchronous SGD. In Advances in Neural Information Processing Systems, 2024

    2024

  63. [72]

    Stich, Jean-Baptiste Cordonnier, and Martin Jaggi

    Sebastian U. Stich, Jean-Baptiste Cordonnier, and Martin Jaggi. Sparsified SGD with memory. In Advances in Neural Information Processing Systems, 2018

    2018

  64. [73]

    Momentum track- ing: Momentum acceleration for decentralized deep learning on heterogeneous data

    Yuki Takezawa, Han Bao, Kenta Niwa, Ryoma Sato, and Makoto Yamada. Momentum track- ing: Momentum acceleration for decentralized deep learning on heterogeneous data. Transac- tions on Machine Learning, 2022. 15 DELAYED MOMENTUM AGGREGATION

    2022

  65. [74]

    Vincent Poor

    Jianyu Wang, Qinghua Liu, Hao Liang, Gauri Joshi, and H. Vincent Poor. Tackling the objec- tive inconsistency problem in heterogeneous federated optimization. In Advances in Neural Information Processing Systems, 2020

    2020

  66. [75]

    A unified analysis of federated learning with arbitrary client participation

    Shiqiang Wang and Mingyue Ji. A unified analysis of federated learning with arbitrary client participation. In Advances in Neural Information Processing Systems, 2022

    2022

  67. [76]

    Fall of empires: Breaking byzantine- tolerant SGD by inner product manipulation

    Cong Xie, Oluwasanmi Koyejo, and Indranil Gupta. Fall of empires: Breaking byzantine- tolerant SGD by inner product manipulation. In Uncertainty in Artificial Intelligence, 2019

    2019

  68. [77]

    Zeno: Distributed stochastic gradient descent with suspicion-based fault-tolerance

    Cong Xie, Sanmi Koyejo, and Indranil Gupta. Zeno: Distributed stochastic gradient descent with suspicion-based fault-tolerance. InInternational Conference on Machine Learning, 2019

    2019

  69. [78]

    Federated optimization under intermittent client availability

    Yikai Yan, Chaoyue Niu, Yucheng Ding, Zhenzhe Zheng, Shaojie Tang, Qinya Li, Fan Wu, Chengfei Lyu, Yanghe Feng, and Guihai Chen. Federated optimization under intermittent client availability. INFORMS Journal on Computing, 36(1):185–202, 2024

    2024

  70. [79]

    Achieving linear speedup with partial worker participation in non-iid federated learning

    Haibo Yang, Minghong Fang, and Jia Liu. Achieving linear speedup with partial worker participation in non-iid federated learning. In International Conference on Learning Repre- sentations, 2021

    2021

  71. [80]

    BASGD: buffered asynchronous SGD for byzantine learning

    Yi-Rui Yang and Wu-Jun Li. BASGD: buffered asynchronous SGD for byzantine learning. In International Conference on Machine Learning, 2021

    2021

  72. [81]

    Bartlett

    Dong Yin, Yudong Chen, Kannan Ramchandran, and Peter L. Bartlett. Byzantine-robust dis- tributed learning: Towards optimal statistical rates. In International Conference on Machine Learning, 2018

    2018

  73. [82]

    Parallel restarted SGD with faster convergence and less communication: Demystifying why model averaging works for deep learning

    Hao Yu, Sen Yang, and Shenghuo Zhu. Parallel restarted SGD with faster convergence and less communication: Demystifying why model averaging works for deep learning. In AAAI Conference on Artificial Intelligence, 2019

    2019

  74. [83]

    Dhople, Wotao Yin, and Yang Liu

    Xinwei Zhang, Mingyi Hong, Sairaj V . Dhople, Wotao Yin, and Yang Liu. Fedpd: A federated learning framework with adaptivity to non-iid data. IEEE Transactions on Signal Processing, 69:6055–6070, 2021

    2021

  75. [84]

    true gradient momentum

    Banghua Zhu, Lun Wang, Qi Pang, Shuai Wang, Jiantao Jiao, Dawn Song, and Michael I. Jordan. Byzantine-robust federated learning with optimal statistical rates. In International Conference on Artificial Intelligence and Statistics, 2023. 16 DELAYED MOMENTUM AGGREGATION Appendix...

    2023

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.