REVIEW 60 cited by
Communication-Efficient Learning of Deep Networks from Decentralized Data
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Communication-Efficient Learning of Deep Networks from Decentralized Data
read the original abstract
Modern mobile devices have access to a wealth of data suitable for learning models, which in turn can greatly improve the user experience on the device. For example, language models can improve speech recognition and text entry, and image models can automatically select good photos. However, this rich data is often privacy sensitive, large in quantity, or both, which may preclude logging to the data center and training there using conventional approaches. We advocate an alternative that leaves the training data distributed on the mobile devices, and learns a shared model by aggregating locally-computed updates. We term this decentralized approach Federated Learning. We present a practical method for the federated learning of deep networks based on iterative model averaging, and conduct an extensive empirical evaluation, considering five different model architectures and four datasets. These experiments demonstrate the approach is robust to the unbalanced and non-IID data distributions that are a defining characteristic of this setting. Communication costs are the principal constraint, and we show a reduction in required communication rounds by 10-100x as compared to synchronized stochastic gradient descent.
Forward citations
Cited by 60 Pith papers
-
What's in a Smoothness Constant? Tighter Rates for Local SGD with Bounded Second-order Heterogeneity
Local SGD provably improves over Mini-batch SGD under bounded second-order heterogeneity in the general convex setting, with nearly tight upper and lower bounds.
-
LOSCAR-SGD: Local SGD with Communication-Computation Overlap and Delay-Corrected Sparse Model Averaging
LOSCAR-SGD combines local updates, sparse model averaging, and communication-computation overlap with a delay-corrected merge rule, providing convergence rates for smooth non-convex objectives under worker heterogeneity.
-
PMF-CL: Pareto-Minimal-Forgetting Continual Learner for Conflicting Tasks
PMF-CL derives Pareto-minimal-forgetting algorithms for linear/basis-function regression and quadratic-bounded losses like logistic regression, achieving static O(d²) memory for d-parameter models.
-
Can Quantum Federated Learning Withstand Circuit-Level Backdoors?
Introduces the CULT threat model with four circuit-level attacks on quantum federated learning and shows they degrade accuracy on MNIST and CIFAR-10 even when defenses like Krum are used.
-
Ringmaster LMO: Asynchronous Linear Minimization Oracle Momentum Method
Ringmaster LMO extends delay-thresholding from ASGD to LMO-based momentum updates, providing convergence guarantees under (L0, L1)-smoothness and time-complexity bounds that recover optimal rates in the Euclidean case.
-
On Hyperparameters and Backdoor-Resistance in Horizontal Federated Learning
Benign clients' training hyperparameters act as a backdoor-defense lever: choosing higher learning rates, more local epochs, and smaller batch sizes substantially reduces backdoor attack success in horizontal federate...
-
Beyond Assumptions: Measuring Federated Learning over Real 5G Networks
Real 5G testbed experiments show consistent stragglers in 70% of federated learning trials due to communication delays, challenging common wireless FL assumptions.
-
Prefix-Tuning: Optimizing Continuous Prompts for Generation
Prefix-tuning matches or exceeds fine-tuning on NLG tasks by optimizing a continuous prefix using 0.1% of parameters while keeping the LM frozen.
-
Autonomous Collaborative Learning Among an Ensemble of Tsetlin Machines with Consensus-Based Inference
A two-layer Tsetlin Machine ensemble with gossip-based vote sharing matches centralized accuracy on several benchmarks without exchanging raw data.
-
Sarus: Privacy-Preserving Multi-Vendor Perception Fusion via Homomorphic Encryption
Sarus is an HE-based framework that fuses vendors' Gaussian-moment detection summaries in encrypted form, with linear-scaling server fusion and near-identical output to plaintext fusion.
-
Mobius Learning: Cyclic Depth Folding in Transformers
Möbius Learning, which cyclically shifts block order across data streams, achieves lower validation loss than fixed-order looped training at loop depths 6, 10, and 15 in a 124M-parameter GPT-2 experiment.
-
PRoVeFL: Private Robust and Verifiable Aggregation in Federated Learning
Multi-server multi-key FHE with a shared random mask lets PRoVeFL run complex Byzantine-robust FL aggregation privately and verifiably, with large reported speedups over Prio and ELSA.
-
Counterfactual Methods for Detecting Unfairness in Anti-Money Laundering Algorithms
Models that improve most from added country and behaviour features also show larger path-specific fairness violations on synthetic AML data, illustrating an accuracy–fairness trade-off.
-
Distributionally Robust Linear Regression With Block Lewis Weights
Algorithm for group distributionally robust linear regression using block Lewis weights to achieve (1+ε) optimality in Õ(min{rank(A), m}^{1/3} ε^{-2/3}) linear-system solves.
-
Development and Design of FLKit: A Structured Onboarding Toolkit for Federated Learning in Health and Life Sciences
FLKit is a new toolkit with four lifecycle stages, eleven role-specific entry points, a glossary, FL Story template, and tool directory to support federated learning projects in health and life sciences.
-
HADES: Privacy-Preserving Federated Learning via Selective Feature Encryption and Hybrid Model Fusion
HADES selectively encrypts privacy-sensitive features identified by PCA in federated learning, trains hybrid encrypted and plaintext networks, and fuses them to match vanilla FL accuracy with reduced overhead and bett...
-
Substrate Asymmetry in User-Side Memory: A Diagnostic Framework
User memory in LLMs factors into three orthogonal axes where parametric adapters and retrieval show opposite strengths, with causal evidence from attention interventions and an alignment tax on RLHF models.
-
PMF-CL: Pareto-Minimal-Forgetting Continual Learner for Conflicting Tasks
PMF-CL derives Pareto-optimal solutions for continual learning on conflicting tasks, yielding memory-efficient algorithms for linear regression and quadratically bounded losses with static O(d^2) memory.
-
Scalable Multimodal Beam Alignment in V2X: An Anti-Imbalance Graph Learning Approach
A multimodal graph learning method for V2X beam alignment cuts overhead by over 90% and outperforms prior federated learning baselines under label and modality imbalance.
-
Decoupled DiLoCo for Resilient Distributed Pre-training
Decoupled DiLoCo enables asynchronous distributed pre-training with zero global downtime under simulated failures while preserving competitive performance on text and vision tasks.
-
Federated Learning: An approach with Hybrid Homomorphic Encryption
Pairing the PASTA stream cipher with BFV homomorphic encryption in federated learning cuts client upload by about 2000x and keeps MNIST accuracy within 1.3% of plaintext, but makes server aggregation roughly 15,000x m...
-
Recovering Clinical Utility Under Differential Privacy: Empirical Validation of Adaptive Federated Aggregation on Heterogeneous Cardiovascular Datasets
FedCVR achieves 79.2% F1 and 0.96 AUC on five real heart-disease datasets under client-level DP, beating a no-DP FedAvg baseline — but not a DP-trained FedAvg baseline.
-
Discovering Collaboration from Novelty: Random Network Distillation for Clustered Federated Learning
Random Network Distillation enables pre-training discovery of client clusters in federated learning via local novelty signals, supporting autonomous grouping under non-IID data without a priori cluster count.
-
EH-FedSAG: Variance-Reduced Federated Learning with Energy-Aware Participation in Energy-Harvesting IoT
EH-FedSAG achieves higher test accuracy and lower training variance than EH-FedAvg in simulations of energy-harvesting federated learning for both homogeneous and heterogeneous data, with larger gains under scarce energy.
-
C2FL: Clustered Continual Federated Learning under Spatial and Temporal Drift
C2FL proposes spatial clustering plus continual learning techniques inside federated learning to maintain performance under combined spatial heterogeneity and temporal drift.
-
FPLIER: Federated Pathway-Level Information Extractor
FPLIER performs federated PLIER training via secure aggregation that is algebraically equivalent to centralized training, with membership-inference risk shown to decrease as the rank of the expression matrix increases.
-
Federated Naive Bayes with Real Mixture of Gaussians and Institutional Governance Regularization for Network Intrusion Detection
A federated intrusion detection method combines hybrid Naive Bayes classifiers as a mixture of Gaussians and uses a governance-derived Institutional Coherence Index to regularize server-side weights via Nelder-Mead op...
-
BiFedKD: Bidirectional Federated Knowledge Distillation Framework for Non-IID and Long-Tailed ECG Monitoring
BiFedKD improves ECG classification accuracy by 3.52% and Macro-F1 by 9.93% on MIT-BIH while cutting communication overhead 40% and computation cost 71.7% versus baseline federated methods.
-
Rennala MVR: Improved Time Complexity for Parallel Stochastic Optimization via Momentum-Based Variance Reduction
Rennala MVR improves time complexity over Rennala SGD for smooth nonconvex stochastic optimization in heterogeneous parallel systems under a mean-squared smoothness assumption.
-
On the Tradeoffs of On-Device Generative Models in Federated Predictive Maintenance Systems
Experiments on real industrial time series show that partial model sharing improves diffusion model performance in bandwidth-limited non-IID settings, while full sharing stabilizes GAN training but offers less robustn...
-
Overcoming data scarcity through multi-center federated learning for organs-at-risk segmentation in pediatric upper abdominal radiotherapy
Federated learning on 310 CT scans from two centers yields pediatric OAR segmentation models with better cross-center robustness than local models for nine evaluated structures.
-
SplitFT: An Adaptive Federated Split Learning System For LLMs Fine-Tuning
SplitFT adapts cut-layer selection and reduces LoRA rank per client in federated split learning to improve efficiency and performance when fine-tuning LLMs on heterogeneous devices and data.
-
Scalable and Private Federated Learning Using Distributed Differential Privacy and Secure Aggregation
A blacklist/whitelist-guided prompt optimization plus diffusion pipeline produces de-identified chest X-rays that retain enough pathology for competitive report-generation training while cutting patient-identity class...
-
Scalable and Private Federated Learning Using Distributed Differential Privacy and Secure Aggregation
DDP-SA combines client-side Laplace noise perturbation with full-threshold additive secret sharing to let federated learning servers reconstruct only aggregated noisy gradients without exposing individual client updates.
-
Secure, Verifiable, and Scalable Multi-Client Data Sharing via Consensus-Based Privacy-Preserving Data Distribution
CPPDD is a new consensus-based protocol for privacy-preserving multi-client data sharing that achieves unanimous-release confidentiality, linear scalability, and high-probability malicious deviation detection.
-
FedRP: A Communication-Efficient Approach for Differentially Private Federated Learning Using Random Projection
FedRP claims to preserve FedAvg-level accuracy while sending only a few numbers per client per round and providing an (epsilon, delta)-DP guarantee.
-
Taming Volatility: Stable and Private QUIC Classification with Federated Learning
Buffering client data in federated QUIC classification suppresses training-time volatility, yet the headline 95.2 percent F1 is measured on a buffered test set that hides real-time traffic swings.
-
Controlled Periodic Synchronization for Efficient Data-Parallel Training
Periodic gradient+parameter synchronization with SlowMo beats DDP by 2.44 pp (K=4) on a WAN while cutting average wall-clock time by 13.8%, but only under a fixed LR=0.1 protocol.
-
Joint Channel Estimation and Dynamics-Aware Grouping for Time-Varying RIS-Assisted OTA Federated Learning
A GRU-based channel predictor plus mobility-aware user grouping reduces CSI and over-the-air aggregation error in RIS-assisted federated learning under imperfect, time-varying channels.
-
FoggyTrust: Robust Federated Learning with Hierarchical Trust Networks
FoggyTrust is a hierarchical extension of FLTrust that localizes trust computation to fog nodes and combines it with heterogeneity-aware optimizers, reporting over 50% gains on CIFAR-10 under Krum and Trim attacks.
-
FLFL: Federated Latent Factor Learning for Private Recovery of Spatio-Temporal Signals
FLFL extends latent factor learning into a federated framework that recovers missing spatio-temporal signals in wireless sensor networks by sharing gradients and enforcing spatio-temporal regularization.
-
ScaleAcross: Designing Multi-Data-Center Infrastructure for Geo-Distributed AI Training
Presents an EVPN-VXLAN emulation framework with ECMP, BFD, and queue-pair traffic distribution for studying AllReduce and Parameter Server patterns in geo-distributed AI training.
-
SwarmHarness: Skill-Based Task Routing via Decentralized Incentive-Aligned AI Agent Networks
SwarmHarness is a proposed decentralized protocol for compute sharing among AI agents via DHT registry, load-aware routing, and credit incentives that penalize non-contributors.
-
Position: Life-Logging Video Streams Make the Privacy-Utility Trade-off Inevitable
Life-logging video streams create an inevitable privacy-utility trade-off that is a foundational challenge for always-on AI systems.
-
The Impact of Federated Learning on Distributed Remote Sensing Archives
FedProx outperforms FedAvg for deeper models under data heterogeneity, BSP reaches near-centralized accuracy at high communication cost, and LeNet gives the best accuracy-communication trade-off on the UC Merced dataset.
-
Pseudoconvex Problems in Operational Decision Systems: Algorithms for Joint Learning and Optimization
Iterative joint learning-optimization framework with convergent algorithms for pseudoconvex objectives in operational decision systems.
-
Pre-Deployment Complexity Estimation for Federated Perception Systems
A pre-deployment complexity score for federated learning, built from entropy, sparsity, and intrinsic dimensionality plus client frequencies, predicts accuracy and communication rounds on three MNIST variants.
-
A Robust Framework for Secure Cardiovascular Risk Prediction: An Architectural Case Study of Differentially Private Federated Learning
On synthetic cardiac data, FedCVR — a re-implementation of FedAdam with server-side momentum — is reported to reach F1 0.78 / AUC 0.96 under DP (ε≈13.4), beating stateless and other adaptive baselines, though the pape...
-
Standardized Methods and Recommendations for Green Federated Learning
A phase-aware carbon-accounting method for federated learning shows that client efficiency tiers and coordination idle time can inflate total CO2e by 8-22x, while GPU choice changes runtime more than energy.
-
Multi-Worker Selection based Distributed Swarm Learning for Edge IoT with Non-i.i.d. Data
Introduces M-DSL algorithm for distributed swarm learning that selects workers using a new non-i.i.d. degree metric to improve convergence and accuracy under data heterogeneity, with theoretical analysis and experimen...
-
FEDEXCHANGE: Bridging the Domain Gap in Federated Object Detection for Free
FEDEXCHANGE improves cross-domain federated object detection by server-side clustering and exchanging client decoder models, achieving higher mAP in some domains at no extra local compute.
-
Centralized vs. Federated Learning for Educational Data Mining: A Comparative Study on Student Performance Prediction with SAEB Microdata
Federated FedProx DNN on SAEB data achieved 61.23% peak accuracy vs 63.96% for centralized XGBoost, but the gap is undercut by a confounded comparison and peak-round reporting.
-
FedEve: On Bridging the Client Drift and Period Drift for Cross-device Federated Learning
FedEve uses a Kalman filter to combine server momentum (prediction) with client updates (observation) to offset period drift and client drift in cross-device federated learning.
-
Developing a Transferable Federated Network Intrusion Detection System
A federated CNN with a two-step preprocessing stage and Block-Based Smart Aggregation achieves superior transferability and local detection rates for network intrusion detection.
-
Strategic Incentivization for Locally Differentially Private Federated Learning
A token system where tokens expire and global models cost tokens forces strategic federated learning clients to adopt the server's acceptable privacy level.
-
Split and Aggregation Learning for Foundation Models Over Mobile Embodied AI Network (MEAN): A Comprehensive Survey
The paper surveys split and aggregation learning for foundation models in 6G networks to improve efficiency, resource use, and data privacy in distributed AI.
-
Strategies for Improving Communication Efficiency in Distributed and Federated Learning: Compression, Local Training, and Personalization
A PhD dissertation showing unified compression theory, personalized accelerated local training, and pruning methods that reduce communication costs in federated learning and maintain accuracy in LLM pruning.
-
The Role of Artificial Intelligence in the SKA Era
This review chapter maps SKA data volume, complexity, and interpretability challenges onto deep learning, generative models, reinforcement learning, and federated learning for source detection, calibration, and discovery.
-
Machine Unlearning: A Comprehensive Survey
A survey classifying machine unlearning into centralized (exact and approximate), distributed/irregular data, verification, and privacy/security categories with technique overviews.
-
Data Aggregation Techniques for Internet of Things
Proposes three approaches for IoT data aggregation: D2D-based clustering for energy efficiency in stationary/mobile nodes, a scheme to improve quality of uncertain raw data, and a prediction-based framework for massiv...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.