Distributionally Robust Federated Learning with Client Drift Minimization

Chaouki Ben Issaid; Mehdi Bennis; Mounssif Krouka

REVIEW 5 major objections 4 minor 42 references

Distributionally Robust Federated Learning with Client Drift Minimization

T0 review · 5 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DRDM, a federated algorithm that optimizes for the worst-performing client while penalizing local drift, is proved to converge to the robust optimum at rate O(1/T^{3/8}) with O(T^{3/4}) communication rounds.

desk verdict The main theorem's O(1/T^{3/8}) rate fails because the bound retains a T-independent term; the algorithm idea is reasonable but the theory is not established. read the letter →

arxiv 2505.15371 v1 pith:HYNNLBWN submitted 2025-05-21 cs.LG

Mounssif Krouka , Chaouki Ben Issaid , Mehdi Bennis This is my paper

classification cs.LG MSC 90C4790C2568T05

keywords federatedlearningdistributionallyrobustoptimizationclientdriftnon-iiddatacommunicationefficiencymin-maxdynamicregularizationconvexconvergence

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper presents DRDM, a federated-learning algorithm for settings where clients' data distributions differ, which optimizes the worst-case mixture of client losses rather than the average. The algorithm adds a dynamic regularization penalty to each client's local objective to counteract client drift, and it updates the mixture weights at the server only once every $\tau$ local steps using a randomized snapshotting scheme. For convex smooth losses with partial client participation, the paper proves that the duality gap between the output and the robust optimum decays as $O(1/T^{3/8})$ in the total number of local updates, while the number of communication rounds is reduced to $O(T^{3/4})$. Experiments on MNIST, Fashion-MNIST, and Kuzushiji-MNIST under Zipf and Dirichlet heterogeneity show that DRDM raises worst-case test accuracy and reduces accuracy spread across clients relative to FedAvg, SCAFFOLD, SCAFF-PD, and DRFA, and an energy analysis shows that the local step count can be chosen to minimize total energy at a given SNR and bandwidth.

What carries the argument

The machinery is the drift-corrected local objective. Each participating client minimizes $f_i(w) - \langle \nabla f_i(w_i^{(s\tau)}), w\rangle + \frac{\mu}{2}\|w - \bar{w}^{(s)}\|^2$, where $\nabla f_i(w_i^{(s\tau)})$ is the exact local gradient at the start of the round and $\bar{w}^{(s)}$ is the server's global model; this makes the stationary point of the modified objective track the global robust optimum. The server maintains a virtual global sequence $w^{(t)} = \frac{1}{m}\sum_{i\in D} w_i^{(t)} - c_t/\mu$ with a control variate $c_t$, and a randomized snapshotting step (Algorithm 3) constructs an unbiased estimate of the cumulative gradient in $\lambda$ so that the dual weights update only every $\tau$ iterations. The proof bounds three deviation terms, $\delta(t)$, $\beta(t)$, and $\varphi(t)$, that measure how far local iterates sit from the virtual sequence, then combines one-step progress bounds for $w$ (Lemma 6) and for $\lambda$ (Lemma 7) to obtain the duality-gap estimate.

What would settle it

Implement DRDM on a convex logistic-regression problem with controlled Dirichlet heterogeneity and a known optimum, but replace the full-batch gradient in line 5 of Algorithm 2 with a mini-batch estimate; if the measured primal-dual gap does not decay at roughly $T^{-3/8}$ or the iterates diverge, the theorem's load-bearing assumption is violated.

Watch

Extended reading notes

Core claim

This paper's central claim is that client drift, the divergence of local iterates produced by heterogeneous data, can be corrected inside a distributionally robust objective without sacrificing communication efficiency. For convex smooth local objectives in which $F(w,\lambda)$ is linear in $\lambda$, and under gradient boundedness, bounded domains, and bounded variance (Assumptions 1–4), Theorem 1 states that Algorithm 1 with $\tau = T^{1/4}/\sqrt{m}$, $\eta = 1/(4L\sqrt{T})$, $\gamma = 1/T^{5/8}$, and $\mu = 2L\sqrt{N/m}$ returns $\hat{w}$ and $\hat{\lambda}$ satisfying $\max_{\lambda\in\Lambda}\mathbb{E}[F(\hat{w},\lambda)] - \min_{w\in W}\mathbb{E}[F(w,\hat{\lambda})] \leq O((D_W^2+G_w^2)/\sqrt{T} + (D_\Lambda^2 + G_\lambda^2/m^{1/2} + \sigma_\lambda^2/m^{3/2})/T^{3/8} + (\sigma_w^2+\Gamma)/(m\sqrt{T}))$. The dominant term gives the advertised $O(1/T^{3/8})$ rate, and the paper interprets the terms as coming from smoothness and gradient bounds, from the infrequent dual updates, and from heterogeneity as measured by the gradient-dissimilarity constant $\Gamma$. Because each of the $O(T^{3/4})$ communication rounds contains $\tau$ local steps, the round count is $O(T^{3/4})$. In the same paper, experiments on three image datasets with CNN, linear, and fine-tuned ResNet18 models report that DRDM achieves the best worst-case test accuracy and lowest cross-client standard deviation among the baselines across the heterogeneity grid.

Load-bearing premise

The proof requires that each participating client compute the exact full-batch gradient of its local loss once per round and feed that exact gradient into the drift-correction term; if the correction term were built from stochastic mini-batch gradients, the deterministic drift bounds used in Lemmas 3 and 5 would fail and the $O(1/T^{3/8})$ rate would not follow.

Editorial extensions

If this is right

With the stated hyperparameters, any target duality gap $\epsilon$ is reached in $T = O(\epsilon^{-8/3})$ local updates and in $O(\sqrt{m}\,\epsilon^{-2})$ communication rounds, so the method trades a slower per-iteration rate for a $T^{1/4}$-fold reduction in server-client synchronization.
The parameter choices decouple: $\tau$ controls communication cost, $\gamma$ controls the dual-update frequency, and $\mu$ controls the drift-correction strength; the bounds indicate they can be retuned for other heterogeneity levels $\Gamma$ without changing the dominant rate.
On the three benchmark datasets and across Zipf and Dirichlet heterogeneity settings, DRDM's worst-case test accuracy is reported higher than FedAvg, SCAFFOLD, SCAFF-PD, and DRFA, with lower cross-client standard deviation.
The number of local steps can be selected to meet a target worst-case accuracy with minimal energy: low SNR and narrow bandwidth favor more local steps (e.g., $\tau=25$–$30$), while high SNR favors fewer (e.g., $\tau=15$ or less).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: Because the theorem uses an exact full-batch gradient in the correction term, a practical variant that reuses a stale or variance-reduced estimate of $\nabla f_i$ is a natural test; the expected price is an extra variance term in the bound, likely degrading the rate in $\sigma_w$ unless the control variate is refreshed frequently.
Editorial inference: Since the dual update is a Monte Carlo estimate of the cumulative gradient, replacing the uniform snapshot $t'$ with a low-variance or importance-sampled one could sharpen the $\sigma_\lambda^2/(m^{3/2}T^{3/8})$ term and possibly push the communication-round count below $O(T^{3/4})$; this is not analyzed in the paper.
Editorial inference: The reported energy savings omit the computation cost of the full-batch gradient required each round, so for large models the optimal $\tau$ under low SNR could shift to smaller values than Figure 4 suggests; a fair energy comparison would charge that cost.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Here's my read on the DRDM paper. The headline: the main convergence theorem doesn't hold as stated. After plugging the prescribed hyperparameters into the bound, a term proportional to 1/m remains that doesn't decay with T, so the claimed O(1/T^{3/8}) duality-gap rate isn't established. The culprit is the 144 ητ² term from Lemma 5: with η=1/(4L√T) and τ=T^{1/4}/√m, the product ητ² = 1/(4Lm), a constant. That term is dropped when the proof moves from the explicit bound to big-O notation. This isn't a minor typo; the bound doesn't tend to zero.

What the paper does well: the algorithm is a sensible hybrid — DRFA's DRO objective with FedDyn's drift-correction regularizer and a randomized snapshot for the dual update. That combination is new as far as I know, and the partial-participation treatment is an honest attempt at a realistic setting. The writing is clear and the experiments span three datasets with multiple heterogeneity levels.

The soft spots beyond the theorem: the proof assumes each participating client computes an exact full-batch gradient once per round for the correction term, which is impractical and load-bearing for the drift bounds. The experiments omit FedDyn, the closest baseline, and give no error bars. In some configurations (e.g., Fashion-MNIST, α=0.1) SCAFFOLD beats DRDM on average accuracy, so the empirical advantage is not universal.

Net: the idea is reasonable, the paper is readable, but the main guarantee fails. The proof may be fixable by choosing hyperparameters so ητ² decays (e.g., τ≈T^{1/4}, η≈T^{-3/4}), but as written the theorem is wrong. I wouldn't cite it in current form. It does warrant a serious referee: the flaw is specific and identifiable, and the fix is plausible, so this is a manuscript where peer review could turn it into something useful.

Send it to review. If the authors close the gap, it's a decent contribution to the DRO-FL literature; if not, it should be rejected. For a reading group, it's a useful example of verifying that constants actually vanish — but not a paper I'd build on.

Best, [Your name]

Referee Report

5 major / 4 minor

Summary. The paper proposes DRDM, a federated learning algorithm that combines a distributionally robust optimization (DRO) objective with FedDyn-style dynamic regularization to mitigate client drift, and a randomized snapshotting scheme for the server-side dual variable. The authors claim a convergence rate of O(1/T^{3/8}) for the primal-dual gap under convex smooth local objectives with partial participation, and they support the method with experiments on MNIST, Fashion-MNIST, and Kuzushiji-MNIST, plus an energy-consumption analysis that suggests adaptive selection of the local-step count. The theoretical section provides a series of lemmas bounding client drift and one-step progress, culminating in Theorem 1. However, the central convergence claim is not established because the proof of Theorem 1 contains a load-bearing algebraic error: after substituting the chosen hyperparameters, the bound in (52) contains terms that do not vanish as T grows, so the claimed O(1/T^{3/8}) rate does not follow. Additional technical issues in the proofs of Lemmas 3 and 6 further undermine the derivation.

Significance. If the convergence result were correct, DRDM would be a meaningful contribution to robust and communication-efficient federated learning, combining drift correction with worst-case client optimization and providing a non-asymptotic duality-gap bound. The paper also makes a useful attempt to connect local-step selection with communication energy. Strengths include the detailed proof structure, the explicit unbiasedness of the dual stochastic gradient estimator (Lemma 2), and a broad set of experiments over three datasets and several heterogeneity levels. However, because the main theorem's proof is invalid as written, the central theoretical claim is currently unsupported; the energy analysis also contains a unit/symbol inconsistency that affects the quantitative conclusions. The potential significance is real, but the manuscript does not yet deliver a sound version of its core result.

major comments (5)

[Appendix G, Eq. (52); Theorem 1] The bound in (52) contains the term (36/m)(σ_w^2 + σ_w^2/m + Γ + 2G_w^2 + 2μ^2D_W^2), which is independent of T. This term arises from substituting τ=T^{1/4}/√m and η=1/(4L√T) into the φ(t) contribution, since ητ²=1/(4Lm). With μ=2L√(N/m), the μ²D_W² part is also T-independent and strictly positive in general. The theorem's stated O(1/T^{3/8}) rate therefore does not follow from the proof, and the right-hand side of the proven inequality does not tend to zero as T→∞. This is a direct algebraic gap in the central claim.
[Appendix E, Eq. (41)] Lemma 6's one-step progress bound relies on the inequality ∥a+b∥² ≤ ∥a∥²+∥b∥² in the decomposition of the global update after Eq. (41). This inequality is not valid unless the cross term 2⟨a,b⟩ is nonpositive, which is not established. The missing cross term can be positive, so the bound on E∥w^{(t+1)}−w∥² is not justified, invalidating Lemma 6 and the telescoped inequality (15) used in Theorem 1.
[Appendix B, proof of Lemma 3, Eq. (26)] The proof bounds E∥μ w_i^{(r)}∥² by μ²D_W² using only the bounded diameter Assumption 3. Since D_W is the diameter of W, ∥w∥ is not necessarily bounded by D_W unless 0∈W, and the paper does not assume 0∈W. Without this additional assumption, the bound on δ(t) is invalid, and the error propagates to Lemma 5 and the final theorem.
[Section IV, Eq. (18)] The energy formula E = T×τ Σ E_p + T×Σ E_t is inconsistent with the paper's definition of T as the total number of local iterations and τ as local steps per round, because the number of communication rounds is then T/τ. The processing energy should scale as T ΣE_p and the transmission energy as (T/τ)ΣE_t; the expression given has the τ factors reversed. This undermines the 'optimal number of local steps' analysis in Figure 4.
[Algorithm 2 and footnote 1] The algorithm and the convergence analysis assume each participating client computes the exact full-batch gradient ∇f_i(w_i^{sτ}) once per round for the correction term, and the drift bounds in Lemmas 3 and 5 treat this gradient as deterministic. The paper does not discuss the computational cost or feasibility of this exact-gradient computation, and the energy model in Eq. (18) ignores it. As a result, the practical applicability of the method under the stated assumptions is not established.

minor comments (4)

[Section IV-B and Tables 1–6] The experimental tables report only scalar means over 10 Monte Carlo runs, with no error bars, standard deviations, or significance tests; the claim that DRDM 'consistently outperforms' baselines would be more convincing with uncertainty quantification.
[Throughout the paper] The notation alternates between w((s+1)τ)_i, w_i^{(s+1)τ}, and w^{t+1}_i for the same quantities, and the symbol T is used both for total local iterations in the theory and in the energy equation (18) without clarifying whether the meaning changes; a consistent notational table would help.
[Appendix F, proof of Lemma 7] In the proof of Lemma 7, the line 'E∥λ(s+1)−λ∥² = E∥λ(s)−λ∥² + ...' omits the expectation on the final term ∥λ(0)−λ∥² in the telescoped bound (17), and the reference to 'Lemma 1' in the proof of Lemma 2 appears to be a typo for Lemma 2.
[Figure 4] The 'optimal' number of local steps in Figure 4 is computed from the energy formula and an accuracy-versus-τ relationship inferred from Figure 3; without a validated model for how worst-case accuracy depends on τ, bandwidth, and SNR, the optimality claim is illustrative rather than demonstrated.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: DRDM's convergence proof is a self-contained derivation from stated assumptions; the claimed rate has an independent algebraic gap, not a circularity.

full rationale

The central claim, Theorem 1's O(1/T^{3/8}) duality-gap bound, is derived from Assumptions 1--4 through a chain of lemmas that bound the drift terms delta, beta, and phi and then combine one-step progress bounds for the primal and dual variables. No parameter in the bound is fitted to the experimental data; the hyperparameters tau, eta, gamma, and mu are substitution values chosen inside the proof, and the experimental results are not used to define or normalize the theoretical quantities. The algorithm borrows dynamic regularization from FedDyn and randomized snapshotting from DRFA, but those are external prior methods, and the authors' own prior work (references [5], [6], and [30]) is not load-bearing for the convergence argument. The min-max duality gap is the standard agnostic-federated-learning gap, and the proof does not define it in terms of the algorithm's outputs in a way that forces the conclusion. The energy discussion in Section IV is an empirical, illustrative calculation of how many local steps are needed for a target accuracy; it is not a fitted parameter renamed as a prediction and does not feed back into the proof. A separate mathematical gap does exist: after substituting the stated hyperparameters into bound (51), the term 144eta*tau^2*(sigma_w^2 + sigma_w^2/m + Gamma + 2G_w^2 + 2mu^2D_W^2) becomes 36/m times the same parenthesized expression, which is independent of T, yet the O(...) statement in Theorem 1 and equation (52) drops it. That is an algebraic correctness concern in the proof, not an equivalence of input and output, so it does not constitute circularity and does not raise the circularity score.

Assumptions & free parameters 5 free parameters · 7 assumptions · 0 invented entities

The central guarantee depends on the prescribed hyperparameters and four standard assumptions, plus the non-standard requirement of an exact full-batch gradient per client per round. No new physical entities are introduced.

free parameters (5)

local steps tau = 10 in experiments; T^{1/4}/sqrt(m) in theory
Chosen by hand in experiments; the theoretical rate is stated only for this specific value.
learning rate eta = 1/(4L sqrt(T)) in theory; unspecified in experiments
Prescribed in Theorem 1; not fitted to data.
dual step size gamma = 1/T^{5/8} in theory; unspecified in experiments
Prescribed in Theorem 1.
regularization coefficient mu = 2L sqrt(N/m) in theory; unspecified in experiments
Prescribed in Theorem 1.
per-client processing/transmission energy E_p, E_t
Used in equation (18) for the energy analysis; no concrete values or models are given.

assumptions (7)

domain assumption Local objectives f_i and global F are L-smooth (Assumption 1).
Standard smoothness requirement invoked throughout Lemmas 3-6.
domain assumption Gradients are bounded: E||nabla f_i(w)|| <= G_w and E||nabla_lambda F|| <= G_lambda (Assumption 2).
Used to bound T2 and T3 in Lemma 6 and the dual variance.
domain assumption Domains W and Lambda have bounded diameters D_W, D_Lambda (Assumption 3).
Used in telescoping sums and dual progress bound.
domain assumption Stochastic gradients have bounded variance sigma_w^2, sigma_lambda^2 (Assumption 4).
Used in Lemmas 1-5.
domain assumption Each f_i is convex and F is linear in lambda (Theorem 1 statement).
The proof relies on convexity for T1 in Lemma 6 and on linearity for the dual progress bound.
ad hoc to paper Clients compute the exact full-batch gradient nabla f_i(w_i^{s tau}) once per round for the correction term.
Equation (4) and Algorithm 2 line 5 require this exact gradient; the theory does not cover stochastic correction gradients, and the cost of this computation is not included in the energy analysis.
standard math Client subsets and snapshot times t', U are sampled with unbiasedness properties used in Lemma 1 and Lemma 2.
Unbiasedness of the client and snapshot estimators is asserted in equations (8) and (9).

how reviews work

0 comments

Cite this review

Pith. "Pith review of Distributionally Robust Federated Learning with Client Drift Minimization." pith.science (2026). https://pith.science/paper/HYNNLBWN

@misc{pith2026250515371,
  author       = {Pith},
  title        = {Pith review of: Distributionally Robust Federated Learning with Client Drift Minimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HYNNLBWN}},
  note         = {Machine review of arXiv:2505.15371}
}

read the original abstract

Federated learning (FL) faces critical challenges, particularly in heterogeneous environments where non-independent and identically distributed data across clients can lead to unfair and inefficient model performance. In this work, we introduce \textit{DRDM}, a novel algorithm that addresses these issues by combining a distributionally robust optimization (DRO) framework with dynamic regularization to mitigate client drift. \textit{DRDM} frames the training as a min-max optimization problem aimed at maximizing performance for the worst-case client, thereby promoting robustness and fairness. This robust objective is optimized through an algorithm leveraging dynamic regularization and efficient local updates, which significantly reduces the required number of communication rounds. Moreover, we provide a theoretical convergence analysis for convex smooth objectives under partial participation. Extensive experiments on three benchmark datasets, covering various model architectures and data heterogeneity levels, demonstrate that \textit{DRDM} significantly improves worst-case test accuracy while requiring fewer communication rounds than existing state-of-the-art baselines. Furthermore, we analyze the impact of signal-to-noise ratio (SNR) and bandwidth on the energy consumption of participating clients, demonstrating that the number of local update steps can be adaptively selected to achieve a target worst-case test accuracy with minimal total energy cost across diverse communication environments.

Figures

Figures reproduced from arXiv: 2505.15371 by the authors.

**Figure 2.** FIGURE 2: Results of [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗

**Figure 3.** FIGURE 3: Worst-case test accuracy versus the number of [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 5.** Figure 5: FIGURE 5: Results of [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]

**Figure 6.** Figure 6: FIGURE 6: Results of [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

42 extracted references · 22 canonical work pages

[1]

Advances and open problems in federated learning,

P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, and at al, “Advances and open problems in federated learning,”F oundations and Trends® in Machine Learning, vol. 14, no. 1–2, pp. 1–210,

work page
[2]

Federated machine learning: Concept and applications,

Q. Yang, Y . Liu, T. Chen, and Y . Tong, “Federated machine learning: Concept and applications,”ACM Trans. Intell. Syst. Technol., vol. 10, no. 2, Jan. 2019. [Online]. Available: https://doi.org/10.1145/3298981

doi:10.1145/3298981 2019
[3]

A survey on federated learning systems: Vision, hype and reality for data privacy and protection,

Q. Li, Z. Wen, Z. Wu, S. Hu, N. Wang, Y . Li, X. Liu, and B. He, “A survey on federated learning systems: Vision, hype and reality for data privacy and protection,”IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 4, pp. 3347–3366, 2023. FIGURE 4: Number of local steps to achieve 80%worst-case test accuracy with minimum total energy cost, ...

work page 2023
[4]

Federated learning: Challenges, methods, and future directions,

T. Li, A. K. Sahu, A. Talwalkar, and V . Smith, “Federated learning: Challenges, methods, and future directions,”IEEE Signal Processing Magazine, vol. 37, no. 3, pp. 50–60, 2020

2020
[5]

Communication- efficient second-order newton-type approach for decentralized learn- ing,

M. Krouka, A. Elgabli, C. B. Issaid, and M. Bennis, “Communication- efficient second-order newton-type approach for decentralized learn- ing,” in2023 IEEE Wireless Communications and Networking Con- ference (WCNC), 2023, pp. 1–7

work page 2023
[6]

Communication- efficient split learning based on analog communication and over the air aggregation,

M. Krouka, A. Elgabli, C. b. Issaid, and M. Bennis, “Communication- efficient split learning based on analog communication and over the air aggregation,” in2021 IEEE Global Communications Conference (GLOBECOM), 2021, pp. 1–6

work page 2021
[7]

Personalized Federated Learning for Intelligent IoT Applications: A Cloud-Edge Based Framework ,

Q. Wu, K. He, and X. Chen, “ Personalized Federated Learning for Intelligent IoT Applications: A Cloud-Edge Based Framework ,” IEEE Open Journal of the Computer Society, vol. 1, no. 01, pp. 35–44, Jan. 2020. [Online]. Available: https://doi.ieeecomputersociety. org/10.1109/OJCS.2020.2993259

arXiv 2020
[8]

Model-Contrastive Federated Learning ,

Q. Li, B. He, and D. Song, “ Model-Contrastive Federated Learning ,” in2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos, CA, USA: IEEE Computer Society, Jun. 2021, pp. 10 708–10 717. [Online]. Available: https://doi.ieeecomputersociety.org/10.1109/CVPR46437.2021.01057

arXiv 2021

Show all 42 references

[9]

Federated visual classification with real-world data distribution,

T.-M. H. Hsu, H. Qi, and M. Brown, “Federated visual classification with real-world data distribution,” inComputer Vision – ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part X. Berlin, Heidelberg: Springer-Verlag, 2020, p. 76–92. [Online]....

2020 doi
[10]

Federated learning with non-iid data: A survey,

Z. Lu, H. Pan, Y . Dai, X. Si, and Y . Zhang, “Federated learning with non-iid data: A survey,”IEEE Internet of Things Journal, vol. 11, no. 11, pp. 19 188–19 209, 2024

2024
[11]

Optimizing federated learning on non-iid data with reinforcement learning,

H. Wang, Z. Kaplan, D. Niu, and B. Li, “Optimizing federated learning on non-iid data with reinforcement learning,” inIEEE INFOCOM 2020 - IEEE Conference on Computer Communications, 2020, pp. 1698– 1707

2020
[12]

Local sgd converges fast and communicates little,

S. U. Stich, “Local sgd converges fast and communicates little,”
[13]

Communication-efficient learning of deep networks from decentralized data,

H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” 2023. [Online]. Available: https://arxiv.org/abs/ 1602.05629

2023 arXiv
[14]

Federated optimization in heterogeneous networks,

T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V . Smith, “Federated optimization in heterogeneous networks,” inProceedings of Machine Learning and Systems, I. Dhillon, D. Papailiopoulos, and V . Sze, Eds., vol. 2, 2020, pp. 429–450. 11 M. Krouka et al.: Distribut...

2020
[15]

Practical federated gradient boosting decision trees,

Q. Li, Z. Wen, and B. He, “Practical federated gradient boosting decision trees,” 2019. [Online]. Available: https://arxiv.org/abs/1911. 04206

2019
[16]

Federated learning with matched averaging,

H. Wang, M. Yurochkin, Y . Sun, D. Papailiopoulos, and Y . Khazaeni, “Federated learning with matched averaging,” inInternational Conference on Learning Representations, 2020. [Online]. Available: https://openreview.net/forum?id=BkluqlSFDS

2020
[17]

Federated bayesian optimization via thompson sampling,

Z. Dai, B. K. H. Low, and P. Jaillet, “Federated bayesian optimization via thompson sampling,” inProceedings of the 34th International Conference on Neural Information Processing Systems, ser. NIPS ’20. Red Hook, NY , USA: Curran Associates Inc., 2020

2020
[18]

The oarf benchmark suite: Characterization and implications for federated learning systems,

S. Hu, Y . Li, X. Liu, Q. Li, Z. Wu, and B. He, “The oarf benchmark suite: Characterization and implications for federated learning systems,”ACM Trans. Intell. Syst. Technol., vol. 13, no. 4, Jun. 2022. [Online]. Available: https://doi.org/10.1145/3510540

2022 doi
[19]

Fedml: A research library and benchmark for federated machine learning,

C. He, S. Li, J. So, X. Zeng, M. Zhang, H. Wang, X. Wang, P. Vepakomma, A. Singh, H. Qiu, X. Zhu, J. Wang, L. Shen, P. Zhao, Y . Kang, Y . Liu, R. Raskar, Q. Yang, M. Annavaram, and S. Avestimehr, “Fedml: A research library and benchmark for federated machine learning,” 2020. ...

2020 arXiv
[20]

Federated learning: Strategies for improving communication efficiency,

J. Kone ˇcn´y, H. B. McMahan, F. X. Yu, P. Richtarik, A. T. Suresh, and D. Bacon, “Federated learning: Strategies for improving communication efficiency,” inNIPS Workshop on Private Multi-Party Machine Learning, 2016. [Online]. Available: https://arxiv.org/abs/ 1610.05492

2016 arXiv
[21]

On the convergence of fedavg on non-iid data,

X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang, “On the convergence of fedavg on non-iid data,” inInternational Conference on Learning Representations, 2020. [Online]. Available: https://openreview.net/forum?id=HJxNAnVtDS

2020
[22]

Federated learning based on dynamic regularization,

D. A. E. Acar, Y . Zhao, R. M. Navarro, M. Mattina, P. N. Whatmough, and V . Saligrama, “Federated learning based on dynamic regularization,” 2021. [Online]. Available: https://arxiv.org/ abs/2111.04263

2021 arXiv
[23]

Scaffold: Stochastic controlled averaging for federated learning,

S. P. Karimireddy, S. Kale, M. Mohri, S. J. Reddi, S. U. Stich, and A. T. Suresh, “Scaffold: Stochastic controlled averaging for federated learning,” 2021. [Online]. Available: https://arxiv.org/abs/1910.06378

2021 arXiv
[24]

Tackling the objective inconsistency problem in heterogeneous federated op- timization,

J. Wang, Q. Liu, H. Liang, G. Joshi, and H. V . Poor, “Tackling the objective inconsistency problem in heterogeneous federated op- timization,” inAdvances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33. Cur...

2020
[25]

Stochastic controlled averaging for federated learning with communication compression,

X. Huang, P. Li, and X. Li, “Stochastic controlled averaging for federated learning with communication compression,” inThe Twelfth International Conference on Learning Representations, 2024. [Online]. Available: https://openreview.net/forum?id=jj5ZjZsWJe

2024
[26]

Ditto: Fair and robust federated learning through personalization,

T. Li, S. Hu, A. Beirami, and V . Smith, “Ditto: Fair and robust federated learning through personalization,” inInternational Conference on Machine Learning, 2020. [Online]. Available: https://api.semanticscholar.org/CorpusID:235446706

2020
[27]

Agnostic federated learning,

M. Mohri, G. Sivek, and A. T. Suresh, “Agnostic federated learning,” inProceedings of the 36th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, K. Chaudhuri and R. Salakhutdinov, Eds., vol. 97. PMLR, 09–15 Jun 2019, pp. 4615–4625. [O...

2019
[28]

Robust fed- erated learning: the case of affine distribution shifts,

A. Reisizadeh, F. Farnia, R. Pedarsani, and A. Jadbabaie, “Robust fed- erated learning: the case of affine distribution shifts,” inProceedings of the 34th International Conference on Neural Information Processing Systems, ser. NIPS ’20. Red Hook, NY , USA: Curran Associates Inc., 2020

2020
[29]

Distributionally robust federated averaging,

Y . Deng, M. M. Kamani, and M. Mahdavi, “Distributionally robust federated averaging,” 2021. [Online]. Available: https: //arxiv.org/abs/2102.12660

2021 arXiv
[30]

DR-DSGD: A distributionally robust decentralized learning algorithm over graphs,

C. B. Issaid, A. Elgabli, and M. Bennis, “DR-DSGD: A distributionally robust decentralized learning algorithm over graphs,” Transactions on Machine Learning Research, 2022. [Online]. Available: https://openreview.net/forum?id=VcXNAr5Rur

2022
[31]

Fair resource allocation in federated learning,

T. Li, M. Sanjabi, A. Beirami, and V . Smith, “Fair resource allocation in federated learning,” inInternational Conference on Learning Representations, 2020. [Online]. Available: https: //openreview.net/forum?id=ByexElSYDr

2020
[32]

Clustered federated learn- ing: Model-agnostic distributed multitask optimization under privacy constraints,

F. Sattler, K.-R. M ¨uller, and W. Samek, “Clustered federated learn- ing: Model-agnostic distributed multitask optimization under privacy constraints,”IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 8, pp. 3710–3722, 2021

2021
[33]

Communication- efficient local decentralized sgd methods,

X. Li, W. Yang, S. Wang, and Z. Zhang, “Communication- efficient local decentralized sgd methods,” 2021. [Online]. Available: https://arxiv.org/abs/1910.09126

2021 arXiv
[34]

Scaff-pd: Communication efficient fair and robust federated learning,

Y . Yu, S. P. Karimireddy, Y . Ma, and M. I. Jordan, “Scaff-pd: Communication efficient fair and robust federated learning,” 2023. [Online]. Available: https://arxiv.org/abs/2307.13381

2023 arXiv
[35]

The mnist database of handwritten digit images for machine learning research,

L. Deng, “The mnist database of handwritten digit images for machine learning research,”IEEE Signal Processing Magazine, vol. 29, no. 6, pp. 141–142, 2012

2012
[36]

Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,

H. Xiao, K. Rasul, and R. V ollgraf, “Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,” 2017. [Online]. Available: https://arxiv.org/abs/1708.07747

2017 arXiv
[37]

Deep learning for classical japanese literature,

T. Clanuwat, M. Bober-Irizar, A. Kitamoto, A. Lamb, K. Yamamoto, and D. Ha, “Deep learning for classical japanese literature,” 2018

2018
[38]

Deep residual learning for image recognition,

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” 2015. [Online]. Available: https://arxiv.org/abs/ 1512.03385

2015 arXiv
[39]

Large-scale analysis of zipf’s law in english texts,

I. Moreno-S ´anchez, F. Font-Clos, and A. Corral, “Large-scale analysis of zipf’s law in english texts,”PLOS ONE, vol. 11, pp. 1–19, 01 2016. [Online]. Available: https://doi.org/10.1371/journal.pone.0147073

2016 doi
[40]

The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator,

J. Pitman and M. Yor, “The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator,”The Annals of Probability, vol. 25, no. 2, pp. 855 – 900, 1997. [Online]. Available: https://doi.org/10.1214/aop/1024404422 12 APPENDIX In this section, we present the p...

1997
[2019]

Available: https://arxiv.org/abs/1805.09767

[Online]. Available: https://arxiv.org/abs/1805.09767

arXiv
[2021]

Available: http://dx.doi.org/10.1561/2200000083

[Online]. Available: http://dx.doi.org/10.1561/2200000083

doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Advances and open problems in federated learning,

P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, and at al, “Advances and open problems in federated learning,”F oundations and Trends® in Machine Learning, vol. 14, no. 1–2, pp. 1–210,

work page

[2] [2]

Federated machine learning: Concept and applications,

Q. Yang, Y . Liu, T. Chen, and Y . Tong, “Federated machine learning: Concept and applications,”ACM Trans. Intell. Syst. Technol., vol. 10, no. 2, Jan. 2019. [Online]. Available: https://doi.org/10.1145/3298981

doi:10.1145/3298981 2019

[3] [3]

A survey on federated learning systems: Vision, hype and reality for data privacy and protection,

Q. Li, Z. Wen, Z. Wu, S. Hu, N. Wang, Y . Li, X. Liu, and B. He, “A survey on federated learning systems: Vision, hype and reality for data privacy and protection,”IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 4, pp. 3347–3366, 2023. FIGURE 4: Number of local steps to achieve 80%worst-case test accuracy with minimum total energy cost, ...

work page 2023

[4] [4]

Federated learning: Challenges, methods, and future directions,

T. Li, A. K. Sahu, A. Talwalkar, and V . Smith, “Federated learning: Challenges, methods, and future directions,”IEEE Signal Processing Magazine, vol. 37, no. 3, pp. 50–60, 2020

2020

[5] [5]

Communication- efficient second-order newton-type approach for decentralized learn- ing,

M. Krouka, A. Elgabli, C. B. Issaid, and M. Bennis, “Communication- efficient second-order newton-type approach for decentralized learn- ing,” in2023 IEEE Wireless Communications and Networking Con- ference (WCNC), 2023, pp. 1–7

work page 2023

[6] [6]

Communication- efficient split learning based on analog communication and over the air aggregation,

M. Krouka, A. Elgabli, C. b. Issaid, and M. Bennis, “Communication- efficient split learning based on analog communication and over the air aggregation,” in2021 IEEE Global Communications Conference (GLOBECOM), 2021, pp. 1–6

work page 2021

[7] [7]

Personalized Federated Learning for Intelligent IoT Applications: A Cloud-Edge Based Framework ,

Q. Wu, K. He, and X. Chen, “ Personalized Federated Learning for Intelligent IoT Applications: A Cloud-Edge Based Framework ,” IEEE Open Journal of the Computer Society, vol. 1, no. 01, pp. 35–44, Jan. 2020. [Online]. Available: https://doi.ieeecomputersociety. org/10.1109/OJCS.2020.2993259

arXiv 2020

[8] [8]

Model-Contrastive Federated Learning ,

Q. Li, B. He, and D. Song, “ Model-Contrastive Federated Learning ,” in2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos, CA, USA: IEEE Computer Society, Jun. 2021, pp. 10 708–10 717. [Online]. Available: https://doi.ieeecomputersociety.org/10.1109/CVPR46437.2021.01057

arXiv 2021