arxiv: 2605.09609 · v1 · submitted 2026-05-10 · 💻 cs.LG · math.AG

Recognition: no theorem link

Minimal Filling Architectures of Polynomial Neural Networks: Counterexamples, Frontier Search, and Defects

Kevin Dao , Jose Israel Rodriguez

Authors on Pith no claims yet

Pith reviewed 2026-05-12 02:15 UTC · model grok-4.3

classification 💻 cs.LG math.AG

keywords polynomial neural networksminimal filling architecturesunimodal conjecturecounterexamplefrontier searchdefectpower activation functions

0 comments

The pith

A minimal filling architecture for polynomial neural networks need not have unimodal hidden-layer widths.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper supplies a counterexample to the claim that every smallest network able to realize every possible input-output map, for fixed input and output widths, must have hidden widths that rise and then fall. The authors locate such an architecture by systematically exploring the space of candidate widths and then confirm that it is indeed minimal and filling by means of recursive bounds on the dimension of the represented function space together with explicit symbolic verification. The same example contains sub-networks whose defect, the shortfall between represented dimension and target dimension, is markedly larger than the small defects seen in earlier cases. If this holds, the unimodal conjecture cannot be used as a general design rule for minimal polynomial networks.

Core claim

We exhibit a counterexample to the minimal unimodal conjecture for polynomial neural networks with power activation functions. With input and output widths held fixed, there exists a minimal filling architecture whose sequence of hidden widths is not unimodal. The architecture was discovered by frontier search and certified by recursive dimension bounds combined with symbolic computation. Several of its subarchitectures display large defect, unlike the predominantly small-defect behavior of previously examined examples.

What carries the argument

Frontier search over candidate width sequences, certified by recursive dimension bounds on the represented function space and by direct symbolic computation.

If this is right

Minimal filling architectures for polynomial networks can have non-unimodal hidden widths.
The unimodal conjecture does not hold in general for power activations.
Subarchitectures of a minimal filling network can exhibit large defect.
Frontier search combined with dimension bounds can locate counterexamples that symbolic checks then confirm.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Network design rules that assume unimodal widths may miss some of the smallest architectures.
Large-defect subnetworks inside a minimal architecture may point to regions where further width reductions are still possible.
The same search-and-certify method could be applied to other activation families or to deeper networks to test related conjectures.

Load-bearing premise

The architecture located by the search is genuinely minimal, fills the target function space, and truly fails to be unimodal.

What would settle it

An explicit computation showing either that the dimension of functions realized by this width sequence falls short of the target dimension or that some strictly smaller architecture with unimodal widths already fills the space.

Figures

Figures reproduced from arXiv: 2605.09609 by Jose Israel Rodriguez, Kevin Dao.

**Figure 2.** Figure 2: A MFA with d0 = dL = 2 (r = 2) References [1] Y. Arjevani, J. Bruna, J. Kileel, E. Polak, and M. Trager, Geometry and Optimization of Shallow Polynomial Networks, SIAM J. Appl. Algebra Geom., 10 (2026), pp. 174–209. [DOI]. [2] M.-C. Brandenburg, G. Loho, and G. Montufar, The real tropical geometry of neural networks for binary classification, Transactions on Machine Learning Research, (2024). [URL]. [3] G.… view at source ↗

read the original abstract

We provide a counterexample to the minimal unimodal conjecture for polynomial neural networks (PNNs) with power activation functions. Fixing the input and output widths, the conjecture states that any minimal filling architecture has unimodal widths for the hidden layers. We found a counterexample via a frontier search and certified it using recursive dimension bounds and symbolic computation. Notably, several subarchitectures of this example exhibit large defect, in contrast with the predominantly small-defect behavior observed in prior examples.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper supplies a concrete, certified counterexample to the minimal unimodal conjecture for polynomial neural networks with power activations.

read the letter

The main takeaway is that the authors located an architecture that fills the required dimensions but does not have unimodal hidden-layer widths, and they verified this with recursive dimension bounds plus symbolic computation. Several of its subarchitectures also carry large defects, unlike the small-defect pattern seen before. This directly contradicts the conjecture for the fixed input-output widths they consider. The example itself is new; it was not already known from the cited prior work on the topic. The combination of frontier search to find candidates and then formal bounds to certify minimality and non-unimodality is a practical way to handle these questions. The certification step gives the claim a measurable degree of rigor that pure search papers sometimes lack. The result stays narrow. It applies only to power activations and the specific widths examined, so it refines a small corner of algebraic neural-network theory without changing how most networks are built or trained. The search procedure is described at a high level in the abstract, and one could reasonably ask for more detail on how exhaustive the frontier exploration was, though the subsequent symbolic checks mitigate that concern. Readers who work on exact characterizations of polynomial networks or on algebraic machine learning will find the counterexample and the defect observations useful. Someone outside that subfield will not get much from it. The paper is coherent on its own terms and engages the existing conjecture directly, so it deserves a serious referee rather than a desk rejection. I would send it out for review.

Referee Report

1 major / 2 minor

Summary. The manuscript provides a counterexample to the minimal unimodal conjecture for polynomial neural networks (PNNs) with power activation functions. For fixed input and output widths, the conjecture claims that every minimal filling architecture has unimodal hidden-layer widths. The authors locate a counterexample by frontier search and certify that it is minimal and non-unimodal by means of recursive dimension bounds together with symbolic computation. They further report that several of its subarchitectures display large defects, in contrast to the small-defect pattern seen in earlier examples.

Significance. A verified counterexample would refute the minimal unimodal conjecture and thereby sharpen the understanding of which width sequences can realize minimal filling architectures in PNNs. The combination of automated frontier search with rigorous, machine-checkable certification constitutes a methodological contribution that could be reused for related conjectures. The observation of large defects in subarchitectures supplies new empirical material on defect behavior.

major comments (1)

[Certification procedure] Certification procedure (the section describing recursive dimension bounds and symbolic computation): the manuscript states that the discovered architecture is minimal and non-unimodal, yet the explicit recursive bounds, the symbolic expressions that were checked, and the termination criteria of the computation are not reproduced in sufficient detail for an independent reader to reconstruct the verification. Because this certification is the sole warrant for the central claim, the omission is load-bearing.

minor comments (2)

[Frontier search] The description of the frontier-search algorithm would benefit from a concise pseudocode block or a reference to the exact implementation parameters (depth limit, branching factor, pruning rule) used to locate the counterexample.
[Defect analysis] Table or figure that lists the widths of the reported counterexample and its subarchitectures should include the corresponding defect values so that the “large defect” claim can be inspected at a glance.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the positive evaluation of the significance of our counterexample and the methodological aspects of the work. We address the single major comment below and will incorporate the requested clarifications in a revised manuscript.

read point-by-point responses

Referee: [Certification procedure] Certification procedure (the section describing recursive dimension bounds and symbolic computation): the manuscript states that the discovered architecture is minimal and non-unimodal, yet the explicit recursive bounds, the symbolic expressions that were checked, and the termination criteria of the computation are not reproduced in sufficient detail for an independent reader to reconstruct the verification. Because this certification is the sole warrant for the central claim, the omission is load-bearing.

Authors: We agree that the current manuscript does not supply the certification details at a level sufficient for independent reconstruction. In the revised version we will expand the relevant section to include: (i) the complete set of recursive dimension bounds applied at each layer, (ii) the precise symbolic expressions that were verified by the computer-algebra system, and (iii) the termination criteria used both for the frontier search and for the subsequent minimality check. We will also add a short algorithmic outline and, if space permits, a supplementary file containing the raw computation log. revision: yes

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The paper reports a computational search for a counterexample to the minimal unimodal conjecture for PNNs, certified by recursive dimension bounds and symbolic computation. This process is a direct enumeration and verification procedure rather than a derivation that reduces by construction to fitted inputs, self-definitional relations, or load-bearing self-citations. No equations or steps in the described argument equate a claimed prediction or uniqueness result to its own inputs; the certification step supplies independent evidence external to any ansatz or prior fit within the paper itself.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review supplies no explicit free parameters, axioms, or invented entities. The result rests on prior definitions of polynomial neural networks, filling architectures, and the unimodal conjecture itself.

pith-pipeline@v0.9.0 · 5369 in / 1055 out tokens · 32665 ms · 2026-05-12T02:15:09.376253+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

36 extracted references · 36 canonical work pages

[2]

Brandenburg, G

M.-C. Brandenburg, G. Loho, and G. Montufar , The real tropical geometry of neural networks for binary classification , Transactions on Machine Learning Research, (2024). https://openreview.net/forum?id=I7JWf8XA2w URL

work page 2024
[4]

Finkel, J

B. Finkel, J. I. Rodriguez, C. Wu, and T. Yahl , A ctivation degree thresholds and expressiveness of polynomial neural networks , Algebraic Statistics, 16 (2025), pp. 113--130. https://doi.org/10.2140/astat.2025.16.113 DOI

work page doi:10.2140/astat.2025.16.113 2025
[5]

O’Connor, and Kevin McGuinness

M. Goyal, R. Goyal, and B. Lall , I mproved P olynomial N eural N etworks with N ormalised A ctivations , in 2020 International Joint Conference on Neural Networks (IJCNN), 2020, pp. 1--8. https://doi.org/10.1109/IJCNN48605.2020.9207535 DOI

work page doi:10.1109/ijcnn48605.2020.9207535 2020
[6]

Henry, G

N. Henry, G. L. Marchetti, and K. Kohn , Geometry of lightning self-attention: Identifiability and dimension , in International Conference on Learning Representations, Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu, eds., vol. 2025, 2025, pp. 14400--14416. https://proceedings.iclr.cc/paper_files/paper/2025/file/259e59fe23ebd09252647fed42949182-Paper-Conferen...

work page 2025
[7]

F. A. Hossain and T. Rahman , A T raining F ramework for O ptimal and S table T raining of P olynomial N eural N etworks , 2025. https://arxiv.org/abs/2505.11589 arXiv: 2505.11589

work page arXiv 2025
[8]

Kileel, M

J. Kileel, M. Trager, and J. Bruna , O n the E xpressive P ower of D eep P olynomial N eural N etworks , in Advances in Neural Information Processing Systems, vol. 32, 2019. https://proceedings.neurips.cc/paper/2019/file/a0dc078ca0d99b5ebb465a9f1cad54ba-Paper.pdf URL

work page 2019
[10]

Kubjas, J

K. Kubjas, J. Li, and M. Wiesmann , G eometry of P olynomial N eural N etworks , Algebraic Statistics, 15 (2024), pp. 295--328. https://doi.org/10.2140/astat.2024.15.295 DOI

work page doi:10.2140/astat.2024.15.295 2024
[11]

G. L. Marchetti, V. Shahverdi, S. Mereta, M. Trager, and K. Kohn , P osition: A lgebra U nveils D eep L earning -- A n I nvitation to N euroalgebraic G eometry , in Proceedings of the 42nd International Conference on Machine Learning, PMLR, 2025. https://doi.org/10.48550/arXiv.2501.18915 DOI

work page doi:10.48550/arxiv.2501.18915 2025
[13]

Usevich, R

K. Usevich, R. Borsoi, C. D\' e rand, and M. Clausel , Identifiability of deep polynomial neural networks , in Advances in Neural Information Processing Systems, vol. 38, Curran Associates, Inc., 2025, pp. 81809--81858. https://openreview.net/forum?id=MrUsZfQ9pC URL

work page 2025
[14]

Zhang, G

L. Zhang, G. Naitzat, and L.-H. Lim , Tropical geometry of deep neural networks , in Proceedings of the 35th International Conference on Machine Learning, J. Dy and A. Krause, eds., vol. 80 of Proceedings of Machine Learning Research, PMLR, 10--15 Jul 2018, pp. 5824--5832. https://proceedings.mlr.press/v80/zhang18i.html URL

work page 2018
[15]

J. Zhou, H. Qian, X. Lu, Z. Duan, H. Huang, and Z. Shao , Polynomial activation neural networks: M odeling, stability analysis and coverage BP -training , Neurocomputing, 359 (2019), pp. 227--240. https://doi.org/10.1016/j.neucom.2019.06.004 DOI

work page doi:10.1016/j.neucom.2019.06.004 2019
[16]

Geometry of Lightning Self-Attention: Identifiability and Dimension , url =

Henry, Nathan and Marchetti, Giovanni Luca and Kohn, Kathl\'. Geometry of Lightning Self-Attention: Identifiability and Dimension , url =. International Conference on Learning Representations , editor =

work page
[17]

, TITLE =

Lim, Lek-Heng and Nelson, Bradley J. , TITLE =. Notices Amer. Math. Soc. , FJOURNAL =. 2023 , NUMBER =. doi:10.1090/noti2666 , url =

work page doi:10.1090/noti2666 2023
[18]

Transactions on Machine Learning Research , issn=

The Real Tropical Geometry of Neural Networks for Binary Classification , author=. Transactions on Machine Learning Research , issn=. 2024 , url=

work page 2024
[19]

Proceedings of the 35th International Conference on Machine Learning , pages =

Tropical Geometry of Deep Neural Networks , author =. Proceedings of the 35th International Conference on Machine Learning , pages =. 2018 , editor =

work page 2018
[21]

Kohn, Kathl\'en and Merkh, Thomas and Mont\'ufar, Guido and Trager, Matthew , TITLE =. SIAM J. Appl. Algebra Geom. , FJOURNAL =. 2022 , NUMBER =. doi:10.1137/21M1441183 , url =

work page doi:10.1137/21m1441183 2022
[22]

Kohn, Kathl\'en and Mont\'ufar, Guido and Shahverdi, Vahid and Trager, Matthew , TITLE =. SIAM J. Appl. Algebra Geom. , FJOURNAL =. 2024 , NUMBER =. doi:10.1137/23M1565504 , url =

work page doi:10.1137/23m1565504 2024
[23]

and Moschoglou, Stylianos and Bouritsas, Giorgos and Deng, Jiankang and Panagakis, Yannis and Zafeiriou, Stefanos , title =

Chrysos, Grigorios G. and Moschoglou, Stylianos and Bouritsas, Giorgos and Deng, Jiankang and Panagakis, Yannis and Zafeiriou, Stefanos , title =. IEEE Trans. Pattern Anal. Mach. Intell. , month1 = aug, pages =. 2022 , issue_date1 =. doi:10.1109/TPAMI.2021.3058891 , abstract =

work page doi:10.1109/tpami.2021.3058891 2022
[24]

Identifiability of Deep Polynomial Neural Networks , url =

Usevich, Konstantin and Borsoi, Ricardo and D\'. Identifiability of Deep Polynomial Neural Networks , url =. Advances in Neural Information Processing Systems , editor2 =

work page
[25]

SIAM Journal on Applied Algebra and Geometry , volume =

Arjevani, Yossi and Bruna, Joan and Kileel, Joe and Polak, Elzbieta and Trager, Matthew , title =. SIAM Journal on Applied Algebra and Geometry , volume =. 2026 , doi =. https://doi.org/10.1137/25M1732994 , abstract =

work page doi:10.1137/25m1732994 2026
[26]

Langley , title =

P. Langley , title =. Proceedings of the 17th International Conference on Machine Learning (ICML 2000) , address =. 2000 , pages =

work page 2000
[27]

T. M. Mitchell. The Need for Biases in Learning Generalizations. 1980

work page 1980
[28]

M. J. Kearns , title =

work page
[29]

Machine Learning: An Artificial Intelligence Approach, Vol. I. 1983

work page 1983
[30]

R. O. Duda and P. E. Hart and D. G. Stork. Pattern Classification. 2000

work page 2000
[31]

Suppressed for Anonymity , author=

work page
[32]

Newell and P

A. Newell and P. S. Rosenbloom. Mechanisms of Skill Acquisition and the Law of Practice. Cognitive Skills and Their Acquisition. 1981

work page 1981
[33]

A. L. Samuel. Some Studies in Machine Learning Using the Game of Checkers. IBM Journal of Research and Development. 1959

work page 1959
[34]

and Trager, M

Kileel, J. and Trager, M. and Bruna, J. , title =. Advances in Neural Information Processing Systems , volume =. 2019 , url =

work page 2019
[35]

and Li, J

Kubjas, K. and Li, J. and Wiesmann, M. , title =. Algebraic Statistics , volume =. 2024 , doi =

work page 2024
[36]

Algebraic Statistics , year =

Finkel, Bella and Rodriguez, Jose Israel and Wu, Chenxi and Yahl, Thomas , title =. Algebraic Statistics , year =

work page
[37]

arXiv preprint arXiv:2511.19703 , year =

Massarenti, Alessio and Mella, Massimiliano , title=. arXiv preprint arXiv:2511.19703 , year =. doi:10.48550/arXiv.2511.19703 , url =

work page doi:10.48550/arxiv.2511.19703
[38]

Proceedings of the 42nd International Conference on Machine Learning , year =

Marchetti, Giovanni Luca and Shahverdi, Vahid and Mereta, Stefano and Trager, Matthew and Kohn, Kathl\'. Proceedings of the 42nd International Conference on Machine Learning , year =

work page
[39]

Neurocomputing , volume =

Zhou, Jun and Qian, Huimin and Lu, Xinbiao and Duan, Zhaoxia and Huang, Haoqian and Shao, Zhen , title =. Neurocomputing , volume =. 2019 , issn =

work page 2019
[40]

2020 , volume=

Goyal, Mohit and Goyal, Rajan and Lall, Brejesh , booktitle=. 2020 , volume=

work page 2020
[41]

2025 , eprint=

Geometry and Optimization of Shallow Polynomial Networks , author=. 2025 , eprint=

work page 2025