REVIEW 1 cited by
A survey of deep learning optimizers -- first and second order methods
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
Deep Learning optimization involves minimizing a high-dimensional loss function in the weight space which is often perceived as difficult due to its inherent difficulties such as saddle points, local minima, ill-conditioning of the Hessian and limited compute resources. In this paper, we provide a comprehensive review of $14$ standard optimization methods successfully used in deep learning research and a theoretical assessment of the difficulties in numerical optimization from the optimization literature.
Forward citations
Cited by 1 Pith paper
-
Accelerated Training of Federated Learning via Second-Order Methods
A survey that categorizes second-order federated learning methods and argues they reduce communication rounds, based on results borrowed from the cited papers rather than new experiments.
Discussion (0). Continue with ORCID to comment.