REVIEW 8 cited by
SCAFFOLD: Stochastic Controlled Averaging for Federated Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Federated Averaging (FedAvg) has emerged as the algorithm of choice for federated learning due to its simplicity and low communication cost. However, in spite of recent research efforts, its performance is not fully understood. We obtain tight convergence rates for FedAvg and prove that it suffers from `client-drift' when the data is heterogeneous (non-iid), resulting in unstable and slow convergence. As a solution, we propose a new algorithm (SCAFFOLD) which uses control variates (variance reduction) to correct for the `client-drift' in its local updates. We prove that SCAFFOLD requires significantly fewer communication rounds and is not affected by data heterogeneity or client sampling. Further, we show that (for quadratics) SCAFFOLD can take advantage of similarity in the client's data yielding even faster convergence. The latter is the first result to quantify the usefulness of local-steps in distributed optimization.
Forward citations
Cited by 8 Pith papers
-
FedFFT: Taming Client Drift in Federated SAM via Spectral Perturbation Filtering
Low-frequency components of client-side SAM perturbations carry most inter-client disagreement; high-pass filtering them yields more consistent federated updates and higher accuracy under non-IID data.
-
Sarus: Privacy-Preserving Multi-Vendor Perception Fusion via Homomorphic Encryption
Sarus is an HE-based framework that fuses vendors' Gaussian-moment detection summaries in encrypted form, with linear-scaling server fusion and near-identical output to plaintext fusion.
-
Federated Lightweight Fine-Tuning
A federated fine-tuning method transmits only 1,280 latent floats per round and reaches near-FedAvg accuracy by exploiting the exact averaging identity of affine mapping networks.
-
Differentially Private Federated Low Rank Adaptation Beyond Fixed-Matrix
FedASK uses double sketching and server-side SVD to aggregate federated LoRA updates under differential privacy, allowing both A and B matrices to be updated instead of freezing one.
-
FedQuad: Federated Stochastic Quadruplet Learning to Mitigate Data Heterogeneity
A quadruplet-based loss for federated learning that aims to reduce representational collapse under data heterogeneity, with mixed empirical support.
-
FedEve: On Bridging the Client Drift and Period Drift for Cross-device Federated Learning
FedEve uses a Kalman filter to combine server momentum (prediction) with client updates (observation) to offset period drift and client drift in cross-device federated learning.
-
A Multi-Objective Optimization framework for Decentralized Learning with coordination constraints
A weighted-sum scalarization of agent and coordinator objectives yields a FedAvg-like decentralized algorithm with O(1/sqrt(T)) convergence under convexity and bounded heterogeneity.
-
DP-FedLoRA: Privacy-Enhanced Federated Fine-Tuning for On-Device Large Language Models
DP-FedLoRA clips and adds Gaussian noise to per-client LoRA matrices in federated LLM fine-tuning, claiming unbiased updates and bounded variance, but the privacy calibration and experiments have significant gaps.
Discussion (0). Sign in to comment.