REVIEW 7 cited by
Federated Learning of a Mixture of Global and Local Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We propose a new optimization formulation for training federated learning models. The standard formulation has the form of an empirical risk minimization problem constructed to find a single global model trained from the private data stored across all participating devices. In contrast, our formulation seeks an explicit trade-off between this traditional global model and the local models, which can be learned by each device from its own private data without any communication. Further, we develop several efficient variants of SGD (with and without partial participation and with and without variance reduction) for solving the new formulation and prove communication complexity guarantees. Notably, our methods are similar but not identical to federated averaging / local SGD, thus shedding some light on the role of local steps in federated learning. In particular, we are the first to i) show that local steps can improve communication for problems with heterogeneous data, and ii) point out that personalization yields reduced communication complexity.
Forward citations
Cited by 7 Pith papers
-
Robust Decentralized Optimization under Node Failures via Adaptive Regularization
Legacy-GT lets a departing node hand a gradient-anchored quadratic surrogate and tracker correction to one neighbor, yielding geometrically decaying residual bias instead of permanent drop-and-forget bias.
-
Collaborative and Efficient Fine-tuning: Leveraging Task Similarity
CoLoRA shares a low-rank adapter pair across users plus a small personal matrix, improving fine-tuning for similar tasks and providing a recovery guarantee.
-
DICE: Data Influence Cascade in Decentralized Learning
DICE defines and approximates multi-hop data influence in decentralized learning, showing that influence is shaped by data, topology, and loss curvature.
-
SPIRE: Conditional Personalization for Federated Diffusion Generative Models
SPIRE adds per-client embeddings to a shared diffusion backbone, enabling parameter-efficient personalization in federated learning, with new-client KID improvements on MNIST, CIFAR-10, and CelebA.
-
Decoding FL Defenses: Systemization, Pitfalls, and Remedies
Many FL defenses are evaluated on overly easy datasets and attacks, and this paper demonstrates with case studies that those easy settings can make weak defenses look strong.
-
Optimization Methods and Software for Federated Learning
A thesis that packages the author's published federated learning work, whose main new theoretical result is an improved complexity bound for error-feedback compression.
-
Strategies for Improving Communication Efficiency in Distributed and Federated Learning: Compression, Local Training, and Personalization
A PhD dissertation showing unified compression theory, personalized accelerated local training, and pruning methods that reduce communication costs in federated learning and maintain accuracy in LLM pruning.
Discussion (0). Continue with ORCID to comment.