REVIEW 2 cited by
On the curvature of the loss landscape
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
One of the main challenges in modern deep learning is to understand why such over-parameterized models perform so well when trained on finite data. A way to analyze this generalization concept is through the properties of the associated loss landscape. In this work, we consider the loss landscape as an embedded Riemannian manifold and show that the differential geometric properties of the manifold can be used when analyzing the generalization abilities of a deep net. In particular, we focus on the scalar curvature, which can be computed analytically for our manifold, and show connections to several settings that potentially imply generalization.
Forward citations
Cited by 2 Pith papers
-
Spectral Collapse Drives Loss of Plasticity in Deep Continual Learning
Continual learning agents lose plasticity because of Hessian spectral collapse at task boundaries, and combining effective rank maximization with L2 weight decay (L2-ER) prevents this collapse.
-
Learning by solving differential equations
Runge-Kutta optimizers adapted with momentum, preconditioning, or adaptive learning rates can close the large-batch generalization gap and match Adam on small MLP workloads.
Discussion (0). Continue with ORCID to comment.