Pith. sign in

REVIEW 29 cited by

On the Convergence of FedAvg on Non-IID Data

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1907.02189 v4 pith:BL7FECYS submitted 2019-07-04 stat.ML cs.LGmath.OC

Xiang Li , Kaixuan Huang , Wenhao Yang , Shusen Wang , Zhihua Zhang This is my paper · ORCID
classification stat.MLcs.LGmath.OC
keywords convergencedatafedavgratedevicedeviceslearningnon-iid
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
abstract

Federated learning enables a large amount of edge computing devices to jointly learn a model without data sharing. As a leading algorithm in this setting, Federated Averaging (\texttt{FedAvg}) runs Stochastic Gradient Descent (SGD) in parallel on a small subset of the total devices and averages the sequences only once in a while. Despite its simplicity, it lacks theoretical guarantees under realistic settings. In this paper, we analyze the convergence of \texttt{FedAvg} on non-iid data and establish a convergence rate of $\mathcal{O}(\frac{1}{T})$ for strongly convex and smooth problems, where $T$ is the number of SGDs. Importantly, our bound demonstrates a trade-off between communication-efficiency and convergence rate. As user devices may be disconnected from the server, we relax the assumption of full device participation to partial device participation and study different averaging schemes; low device participation rate can be achieved without severely slowing down the learning. Our results indicate that heterogeneity of data slows down the convergence, which matches empirical observations. Furthermore, we provide a necessary condition for \texttt{FedAvg} on non-iid data: the learning rate $\eta$ must decay, even if full-gradient is used; otherwise, the solution will be $\Omega (\eta)$ away from the optimal.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 29 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 1,012 citations worldwide. Full citation record

  1. FedMeNF: Privacy-Preserving Federated Meta-Learning for Neural Fields

    cs.LG 2025-08 conditional novelty 7.0 of 10

    MDIR detects LLM weight homology from embedding matrices alone using polar decomposition and permutation matching, achieving perfect AUC and accuracy on LeaFBench and reconstructing layer-level transformations.

  2. FedACT: Federated Adaptive Coordinate Trust Modulation for Robust Transformer Training under Data Heterogeneity

    cs.LG 2026-07 conditional novelty 6.5 of 10

    Global-aware coordinate trust modulation after corrected AdamW updates improves federated Transformer and LLM training under data heterogeneity over strong adaptive baselines.

  3. Robust Federated Learning Under Real-World Client Churn

    cs.LG 2026-07 conditional novelty 6.0 of 10

    FeLiX reduces wall-clock time-to-target accuracy in federated learning by up to 2.37x using lightweight availability tiers, fresh-utility client selection, and informativeness-aware aggregation without requiring oracu...

  4. Achieving Linear Speedup for Composite Federated Learning

    cs.LG 2026-02 conditional novelty 6.0 of 10

    FedNMap provably achieves communication complexity O(1/(nQ ε⁴)) for nonconvex composite federated learning with nonsmooth regularizers, the first such linear-speedup guarantee.

  5. Degree of Staleness-Aware Data Updating in Federated Learning

    cs.LG 2025-08 unverdicted novelty 6.0 of 10

    DUFL is a payment-based incentive mechanism that jointly balances data staleness and data volume via a Stackelberg game and derives a closed-form optimal client data update strategy.

  6. Federated Learning Enhanced by Feature Reconstruction for Semantic Communication Module Updates of Agents

    eess.SP 2025-08 conditional novelty 6.0 of 10

    FedSFR lets poorly connected clients upload compact encoder features, which the server uses in a feature-reconstruction step, improving the stability and efficiency of federated training for vector-quantized image sem...

  7. Towards Collaborative Fairness in Federated Learning Under Imbalanced Covariate Shift

    cs.LG 2025-07 conditional novelty 6.0 of 10

    FedAKD improves collaborative fairness under imbalanced covariate shift by distilling only each client's correctly predicted samples into the global model.

  8. Adaptive collaboration for online personalized distributed learning with heterogeneous clients

    stat.ML 2025-07 conditional novelty 6.0 of 10

    An adaptive gradient-similarity criterion dynamically selects collaboration partners in personalized federated learning, provably recovering the oracle-optimal sample complexity of All-for-one without knowing client h...

  9. FedCLAM: Client Adaptive Momentum with Foreground Intensity Matching for Federated Medical Image Segmentation

    eess.IV 2025-06 conditional novelty 6.0 of 10

    FedCLAM improves federated medical segmentation by weighting client updates with validation-loss progress and aligning predicted and ground-truth foreground intensities.

  10. PNCS:Power-Norm Cosine Similarity for Diverse Client Selection in Federated Learning

    cs.LG 2025-06 conditional novelty 6.0 of 10

    A client selection rule based on L4-norm cosine similarity and a rotation queue improves federated learning accuracy modestly over three baselines on CIFAR-10 and Fashion-MNIST.

  11. Mitigating Non-IID Drift in Zeroth-Order Federated LLM Fine-Tuning with Transferable Sparsity

    cs.LG 2025-06 reject novelty 6.0 of 10

    Sparse zeroth-order federated fine-tuning with shared seeds and GradIP-based early stopping matches or beats full-parameter ZO while using far less communication.

  12. Adaptive Federated LoRA in Heterogeneous Wireless Networks with Independent Sampling

    cs.LG 2025-05 reject novelty 6.0 of 10

    An adaptive federated LoRA scheduler jointly tunes client sampling probabilities and LoRA sketching ratios to minimize wall-clock fine-tuning time, with experiments reporting 2.8 to 4.2 times speedups over prior methods.

  13. Hybrid Batch Normalisation: Resolving the Dilemma of Batch Normalisation in Federated Learning

    cs.LG 2025-05 conditional novelty 6.0 of 10

    Hybrid Batch Normalisation improves federated learning accuracy by combining local batch statistics with global statistics, and derives those global statistics from the pre-update global model.

  14. DeCAF: Decentralized Consensus-And-Factorization for Low-Rank Adaptation of Foundation Models

    cs.LG 2025-05 reject novelty 6.0 of 10

    A truncated-SVD consensus step for decentralized LoRA is claimed to reach O(1/sqrt T) convergence, matching decentralized SGD, with supporting CLIP and LLAMA2-7B experiments.

  15. Channel-Adaptive Robust Aggregation for Over-the-Air Federated Learning in Heterogeneous Networks

    cs.LG 2026-07 conditional novelty 5.0 of 10

    Adaptive channel- and readiness-triggered aggregation with dual-purpose precoding yields O(1/T) OTA-FL convergence and higher CIFAR accuracy under stragglers and noise than fixed-schedule baselines.

  16. Enhancing Model Privacy in Federated Learning with Random Masking and Quantization

    cs.LG 2025-08 reject novelty 5.0 of 10

    FedQSN hides part of the server model with random masks and quantizes the remainder to give clients a degraded proxy, reporting a large global-vs-proxy performance gap with modest loss in the final global model.

  17. When Secure Aggregation Falls Short: Achieving Long-Term Privacy in Asynchronous Federated Learning for LEO Satellite Networks

    cs.CR 2025-08 conditional novelty 5.0 of 10

    Long-term privacy leakage in asynchronous federated learning over LEO satellite networks is kept bounded by fixed jointly-visible satellite partitions used with secure aggregation.

  18. Discerning and quantifying high frequency activities in EEG under normal and epileptic conditions

    nlin.CD 2025-08 unverdicted novelty 5.0 of 10

    The authors claim that gamma-band EEG activity at roughly 50 Hz and 76 Hz shows distinct bi-stable dynamics in epilepsy, enabling 94-95% accuracy in separating normal from ictal states.

  19. On the Convergence and Complexity of Proximal Gradient and Accelerated Proximal Gradient Methods under Adaptive Gradient Estimation

    math.OC 2025-07 conditional novelty 5.0 of 10

    Adaptive gradient accuracy yields optimal iteration complexity for (accelerated) proximal gradient methods with biased estimates, and query complexity claims for unbiased estimates.

  20. Federated Learning for Commercial Image Sources

    cs.CV 2025-07 conditional novelty 5.0 of 10

    The authors present a new 31-class, 8-source image classification dataset for federated learning and show that Fed-Cyclic and Fed-Star beat FedAvg and RingFed on it.

  21. SFedKD: Sequential Federated Learning with Discrepancy-Aware Multi-Teacher Knowledge Distillation

    cs.LG 2025-07 conditional novelty 5.0 of 10

    SFedKD uses discrepancy-weighted multi-teacher knowledge distillation and greedy teacher selection to reduce catastrophic forgetting in sequential federated learning.

  22. Cooperative Gradient Coding

    cs.DC 2025-07 reject novelty 5.0 of 10

    A gradient coding scheme for federated learning that avoids dataset duplication and adds a fallback decoder that recovers the model from incomplete messages.

  23. Avoid Forgetting by Preserving Global Knowledge Gradients in Federated Learning with Non-IID Data

    cs.LG 2025-05 conditional novelty 5.0 of 10

    FedProj combines client-side gradient projection onto a global-knowledge loss with server-side ensemble distillation and outperforms existing federated learning methods on non-IID image and NLP benchmarks.

  24. Decentralized Federated Learning by Partial Message Exchange

    cs.LG 2026-03 reject novelty 4.0 of 10

    PaME combines random coordinate exchange with a growing-penalty schedule, claiming linear convergence under two mild assumptions, but its key parameter condition is never satisfied by its own experiments and the limit...

  25. Variational Gaussian Mixture Manifold Models for Client-Specific Federated Personalization

    cs.LG 2025-09 conditional novelty 4.0 of 10

    VGM2 personalizes federated learning by exchanging compact Bayesian summaries of same-class and different-class distance distributions instead of model weights.

  26. A Systematic Survey of Model Extraction Attacks and Defenses: State-of-the-Art and Perspectives

    cs.CR 2025-08 conditional novelty 4.0 of 10

    The paper classifies model extraction attacks and defenses into attack, defense, and computing environment categories and surveys their current state.

  27. PPFL-RDSN: Privacy-Preserving Federated Learning-based Residual Dense Spatial Networks for Encrypted Lossy Image Reconstruction

    cs.LG 2025-06 reject novelty 4.0 of 10

    A federated, privacy-preserving RDSN framework for encrypted image reconstruction whose local differential privacy mechanism is not actually differentially private because it releases low-frequency DCT coefficients wi...

  28. Communication Efficient Adaptive Model-Driven Quantum Federated Learning

    cs.LG 2025-06 conditional novelty 4.0 of 10

    mdQFL cuts quantum federated learning communication cost by training one device per cluster, while reporting maintained server accuracy and improved local accuracy.

  29. RoadFed: A Multimodal Federated Learning System for Improving Road Safety

    cs.CE 2025-02 reject novelty 4.0 of 10

    A multimodal federated learning system with quantization and local differential privacy is reported to detect road hazards at 96.42% accuracy with 0.035 s latency and up to 1000x lower communication cost than baselines.

Pith tools