REVIEW 5 cited by
On the Expressive Power of Deep Polynomial Neural Networks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
On the Expressive Power of Deep Polynomial Neural Networks
read the original abstract
We study deep neural networks with polynomial activations, particularly their expressive power. For a fixed architecture and activation degree, a polynomial neural network defines an algebraic map from weights to polynomials. The image of this map is the functional space associated to the network, and it is an irreducible algebraic variety upon taking closure. This paper proposes the dimension of this variety as a precise measure of the expressive power of polynomial neural networks. We obtain several theoretical results regarding this dimension as a function of architecture, including an exact formula for high activation degrees, as well as upper and lower bounds on layer widths in order for deep polynomials networks to fill the ambient functional space. We also present computational evidence that it is profitable in terms of expressiveness for layer widths to increase monotonically and then decrease monotonically. Finally, we link our study to favorable optimization properties when training weights, and we draw intriguing connections with tensor and polynomial decompositions.
Forward citations
Cited by 5 Pith papers
-
A Complete Symmetry Classification of Shallow ReLU Networks
A complete classification of symmetries in shallow ReLU networks is achieved by using the non-differentiability of ReLU.
-
TwistedMerge: Certified Higher-Order Diagnostics and Abstention for Model Merging
Cycle inconsistency in model merging is not automatically a cohomological obstruction: TwistedMerge certifies a class only after frozen-complex, centrality, closure, and statistical gates, and finds no natural central class.
-
Landscape analysis for shallow neural networks: Complete classification of critical points for cubic activation and affine target functions
For cubic-activation shallow networks with affine targets, the squared-loss landscape has no local maxima; every critical point is a global minimizer, a rigid non-global local minimum, or a saddle, and zero loss is ac...
-
Algebraic Representability as the Limiting Regime of Grokking: An Exactly Solvable Model with Holomorphic Activations
A task ma+nb mod p is representable by a z^k holomorphic network iff m+n=k; non-representable tasks cannot be memorised at any width.
-
A Complete Symmetry Classification of Shallow ReLU Networks
Shallow ReLU networks admit a complete classification of parameter symmetries obtained by exploiting ReLU non-differentiability rather than analytic activation assumptions.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.