For ReLU networks with width at least two in input and hidden layers, an open set of parameters is identifiable, implying functional dimension equals parameter count minus hidden neurons.
Identifiable equivariant networks are layerwise equivariant, 2026 a
4 Pith papers cite this work. Polarity classification is still indexing.
4
Pith papers citing it
abstract
We investigate the relation between end-to-end equivariance and layerwise equivariance in deep neural networks. We prove the following: For a network whose end-to-end function is equivariant with respect to group actions on the input and output spaces, there is a parameter choice yielding the same end-to-end function such that its layers are equivariant with respect to some group actions on the latent spaces. Our result assumes that the parameters of the model are identifiable in an appropriate sense. This identifiability property has been established in the literature for a large class of networks, to which our results apply immediately, while it is conjectural for others. The theory we develop is grounded in an abstract formalism, and is therefore architecture-agnostic. Overall, our results provide a mathematical explanation for the emergence of equivariant structures in the weights of neural networks during training -- a phenomenon that is consistently observed in practice.
citation-role summary
method 1
citation-polarity summary
years
2026 4verdicts
UNVERDICTED 4roles
method 1polarities
use method 1representative citing papers
Data symmetries generically do not induce conserved quantities in NN training for analytic non-polynomial losses, but can for MSE with tensorizable networks.
Develops algebraic geometry tools for monomial neural networks and proves the singular locus of neurovarieties is contained in the architectural degeneracy locus for fully connected networks with non-increasing widths and scalar output under layerwise regularity assumptions.
For large monomial activation degree, critical points in deep fully-connected networks coincide exactly with subnetwork configurations where neurons are inactive or redundant.
citing papers explorer
-
Algebraic Networks and Architectural Degenerations
Develops algebraic geometry tools for monomial neural networks and proves the singular locus of neurovarieties is contained in the architectural degeneracy locus for fully connected networks with non-increasing widths and scalar output under layerwise regularity assumptions.