Understanding over-parameterized deep networks by geometrization
read the original abstract
A complete understanding of the widely used over-parameterized deep networks is a key step for AI. In this work we try to give a geometric picture of over-parameterized deep networks using our geometrization scheme. We show that the Riemannian geometry of network complexity plays a key role in understanding the basic properties of over-parameterizaed deep networks, including the generalization, convergence and parameter sensitivity. We also point out deep networks share lots of similarities with quantum computation systems. This can be regarded as a strong support of our proposal that geometrization is not only the bible for physics, it is also the key idea to understand deep learning systems.
This paper has not been read by Pith yet.
Forward citations
Cited by 2 Pith papers
-
Deep network as memory space: complexity, generalization, disentangled representation and interpretability
Deep networks are framed as memory spaces whose complexity is defined by a Fisher metric, with the least action principle linking this complexity to generalization and disentanglement for better interpretability.
-
Gauge theory and twins paradox of disentangled representations
Authors propose a fibre bundle gauge theory model for disentangled representations and connect it to the relativity twins paradox.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.