REVIEW 2 cited by
On Second-order Optimization Methods for Federated Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We consider federated learning (FL), where the training data is distributed across a large number of clients. The standard optimization method in this setting is Federated Averaging (FedAvg), which performs multiple local first-order optimization steps between communication rounds. In this work, we evaluate the performance of several second-order distributed methods with local steps in the FL setting which promise to have favorable convergence properties. We (i) show that FedAvg performs surprisingly well against its second-order competitors when evaluated under fair metrics (equal amount of local computations)-in contrast to the results of previous work. Based on our numerical study, we propose (ii) a novel variant that uses second-order local information for updates and a global line search to counteract the resulting local specificity.
Forward citations
Cited by 2 Pith papers
-
pFedSOP : Accelerating Training Of Personalized Federated Learning Using Second-Order Optimization
pFedSOP combines Gompertz-weighted local/global gradients with a rank-one Fisher Information Matrix update to speed up personalized federated learning, but the convergence proof is invalid and the update reduces to no...
-
Accelerated Training of Federated Learning via Second-Order Methods
A survey that categorizes second-order federated learning methods and argues they reduce communication rounds, based on results borrowed from the cited papers rather than new experiments.
Discussion (0). Sign in to comment.