REVIEW 29 cited by
On the Convergence of FedAvg on Non-IID Data
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
Federated learning enables a large amount of edge computing devices to jointly learn a model without data sharing. As a leading algorithm in this setting, Federated Averaging (\texttt{FedAvg}) runs Stochastic Gradient Descent (SGD) in parallel on a small subset of the total devices and averages the sequences only once in a while. Despite its simplicity, it lacks theoretical guarantees under realistic settings. In this paper, we analyze the convergence of \texttt{FedAvg} on non-iid data and establish a convergence rate of $\mathcal{O}(\frac{1}{T})$ for strongly convex and smooth problems, where $T$ is the number of SGDs. Importantly, our bound demonstrates a trade-off between communication-efficiency and convergence rate. As user devices may be disconnected from the server, we relax the assumption of full device participation to partial device participation and study different averaging schemes; low device participation rate can be achieved without severely slowing down the learning. Our results indicate that heterogeneity of data slows down the convergence, which matches empirical observations. Furthermore, we provide a necessary condition for \texttt{FedAvg} on non-iid data: the learning rate $\eta$ must decay, even if full-gradient is used; otherwise, the solution will be $\Omega (\eta)$ away from the optimal.
Forward citations
Cited by 29 Pith papers
-
FedMeNF: Privacy-Preserving Federated Meta-Learning for Neural Fields
MDIR detects LLM weight homology from embedding matrices alone using polar decomposition and permutation matching, achieving perfect AUC and accuracy on LeaFBench and reconstructing layer-level transformations.
-
FedACT: Federated Adaptive Coordinate Trust Modulation for Robust Transformer Training under Data Heterogeneity
Global-aware coordinate trust modulation after corrected AdamW updates improves federated Transformer and LLM training under data heterogeneity over strong adaptive baselines.
-
Robust Federated Learning Under Real-World Client Churn
FeLiX reduces wall-clock time-to-target accuracy in federated learning by up to 2.37x using lightweight availability tiers, fresh-utility client selection, and informativeness-aware aggregation without requiring oracu...
-
Achieving Linear Speedup for Composite Federated Learning
FedNMap provably achieves communication complexity O(1/(nQ ε⁴)) for nonconvex composite federated learning with nonsmooth regularizers, the first such linear-speedup guarantee.
-
Degree of Staleness-Aware Data Updating in Federated Learning
DUFL is a payment-based incentive mechanism that jointly balances data staleness and data volume via a Stackelberg game and derives a closed-form optimal client data update strategy.
-
Federated Learning Enhanced by Feature Reconstruction for Semantic Communication Module Updates of Agents
FedSFR lets poorly connected clients upload compact encoder features, which the server uses in a feature-reconstruction step, improving the stability and efficiency of federated training for vector-quantized image sem...
-
Towards Collaborative Fairness in Federated Learning Under Imbalanced Covariate Shift
FedAKD improves collaborative fairness under imbalanced covariate shift by distilling only each client's correctly predicted samples into the global model.
-
Adaptive collaboration for online personalized distributed learning with heterogeneous clients
An adaptive gradient-similarity criterion dynamically selects collaboration partners in personalized federated learning, provably recovering the oracle-optimal sample complexity of All-for-one without knowing client h...
-
FedCLAM: Client Adaptive Momentum with Foreground Intensity Matching for Federated Medical Image Segmentation
FedCLAM improves federated medical segmentation by weighting client updates with validation-loss progress and aligning predicted and ground-truth foreground intensities.
-
PNCS:Power-Norm Cosine Similarity for Diverse Client Selection in Federated Learning
A client selection rule based on L4-norm cosine similarity and a rotation queue improves federated learning accuracy modestly over three baselines on CIFAR-10 and Fashion-MNIST.
-
Mitigating Non-IID Drift in Zeroth-Order Federated LLM Fine-Tuning with Transferable Sparsity
Sparse zeroth-order federated fine-tuning with shared seeds and GradIP-based early stopping matches or beats full-parameter ZO while using far less communication.
-
Adaptive Federated LoRA in Heterogeneous Wireless Networks with Independent Sampling
An adaptive federated LoRA scheduler jointly tunes client sampling probabilities and LoRA sketching ratios to minimize wall-clock fine-tuning time, with experiments reporting 2.8 to 4.2 times speedups over prior methods.
-
Hybrid Batch Normalisation: Resolving the Dilemma of Batch Normalisation in Federated Learning
Hybrid Batch Normalisation improves federated learning accuracy by combining local batch statistics with global statistics, and derives those global statistics from the pre-update global model.
-
DeCAF: Decentralized Consensus-And-Factorization for Low-Rank Adaptation of Foundation Models
A truncated-SVD consensus step for decentralized LoRA is claimed to reach O(1/sqrt T) convergence, matching decentralized SGD, with supporting CLIP and LLAMA2-7B experiments.
-
Channel-Adaptive Robust Aggregation for Over-the-Air Federated Learning in Heterogeneous Networks
Adaptive channel- and readiness-triggered aggregation with dual-purpose precoding yields O(1/T) OTA-FL convergence and higher CIFAR accuracy under stragglers and noise than fixed-schedule baselines.
-
Enhancing Model Privacy in Federated Learning with Random Masking and Quantization
FedQSN hides part of the server model with random masks and quantizes the remainder to give clients a degraded proxy, reporting a large global-vs-proxy performance gap with modest loss in the final global model.
-
When Secure Aggregation Falls Short: Achieving Long-Term Privacy in Asynchronous Federated Learning for LEO Satellite Networks
Long-term privacy leakage in asynchronous federated learning over LEO satellite networks is kept bounded by fixed jointly-visible satellite partitions used with secure aggregation.
-
Discerning and quantifying high frequency activities in EEG under normal and epileptic conditions
The authors claim that gamma-band EEG activity at roughly 50 Hz and 76 Hz shows distinct bi-stable dynamics in epilepsy, enabling 94-95% accuracy in separating normal from ictal states.
-
On the Convergence and Complexity of Proximal Gradient and Accelerated Proximal Gradient Methods under Adaptive Gradient Estimation
Adaptive gradient accuracy yields optimal iteration complexity for (accelerated) proximal gradient methods with biased estimates, and query complexity claims for unbiased estimates.
-
Federated Learning for Commercial Image Sources
The authors present a new 31-class, 8-source image classification dataset for federated learning and show that Fed-Cyclic and Fed-Star beat FedAvg and RingFed on it.
-
SFedKD: Sequential Federated Learning with Discrepancy-Aware Multi-Teacher Knowledge Distillation
SFedKD uses discrepancy-weighted multi-teacher knowledge distillation and greedy teacher selection to reduce catastrophic forgetting in sequential federated learning.
-
Cooperative Gradient Coding
A gradient coding scheme for federated learning that avoids dataset duplication and adds a fallback decoder that recovers the model from incomplete messages.
-
Avoid Forgetting by Preserving Global Knowledge Gradients in Federated Learning with Non-IID Data
FedProj combines client-side gradient projection onto a global-knowledge loss with server-side ensemble distillation and outperforms existing federated learning methods on non-IID image and NLP benchmarks.
-
Decentralized Federated Learning by Partial Message Exchange
PaME combines random coordinate exchange with a growing-penalty schedule, claiming linear convergence under two mild assumptions, but its key parameter condition is never satisfied by its own experiments and the limit...
-
Variational Gaussian Mixture Manifold Models for Client-Specific Federated Personalization
VGM2 personalizes federated learning by exchanging compact Bayesian summaries of same-class and different-class distance distributions instead of model weights.
-
A Systematic Survey of Model Extraction Attacks and Defenses: State-of-the-Art and Perspectives
The paper classifies model extraction attacks and defenses into attack, defense, and computing environment categories and surveys their current state.
-
PPFL-RDSN: Privacy-Preserving Federated Learning-based Residual Dense Spatial Networks for Encrypted Lossy Image Reconstruction
A federated, privacy-preserving RDSN framework for encrypted image reconstruction whose local differential privacy mechanism is not actually differentially private because it releases low-frequency DCT coefficients wi...
-
Communication Efficient Adaptive Model-Driven Quantum Federated Learning
mdQFL cuts quantum federated learning communication cost by training one device per cluster, while reporting maintained server accuracy and improved local accuracy.
-
RoadFed: A Multimodal Federated Learning System for Improving Road Safety
A multimodal federated learning system with quantization and local differential privacy is reported to detect road hazards at 96.42% accuracy with 0.035 s latency and up to 1000x lower communication cost than baselines.
Discussion (0). Sign in to comment.