REVIEW 2 major objections 5 minor 63 references
Reducing Symmetry Increase in Equivariant Neural Networks
T0 review · 2 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read For equivariant networks on symmetric inputs, the minimum possible symmetry gain is fixed by the feature space, and generic networks achieve it almost everywhere.
desk verdict A genuinely useful and mostly rigorous framework for computing symmetry increase in equivariant nets, held back by a density-vs-measure gap in the 'most equivariant maps' claim. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the symmetry infimum $I_G(Y,H)$, the unique smallest orbit type appearing in the fixed-point subspace $Y^H$; it is what the feature space algebraically forces an equivariant map to reach. The argument is carried by three devices: the operator $p_Y$ that enlarges a subgroup by the kernel of the feature representation and thereby separates designed, unavoidable increases from accidental ones; an orbit-type test based on Michel's Criterion, which compares fixed-point-space dimensions at adjacent supergroups and becomes sufficient for high-multiplicity representations where every isotypic component has multiplicity exceeding $\dim G$, with a trace formula computing those dimensions; and a genericity argument on Whitney regular orbit-type stratifications showing that, in a $C^\infty$-dense equivariant family, transversality makes the dimension of the set of too-symmetric outputs strictly smaller than the data stratum, so almost-isovariance holds almost everywhere. A counterexample in the appendix shows that orbit-type inclusion alone is not enough for full isovariance, which is why the multiplicity condition $r>\max_j\dim M_j$ carries real weight.
What would settle it
Train a $C^\infty$-expressive equivariant network on a compact $G$-invariant submanifold whose orbit types satisfy $(p_Y(H))\in O_G(Y)$, and compute the Hausdorff measure of the set of inputs where $\rho_Y(G_x)\neq \rho_Y(G_{f(x)})$; if this set has positive measure for any approximant with enough multiplicity, the generic almost-isovariance claim of Theorem 5.2 is false.
Extended reading notes
Core claim
The paper's central claim is that the degradation has a universal lower bound. For a representation $Y$ of a compact Lie group $G$ and a closed subgroup $H$ of $G$, the fixed-point subspace $Y^H$ contains points of several orbit types, and Theorem 3.1 asserts that among them there is a unique minimal one, the symmetry infimum $I_G(Y,H)$. Equivariance forces the isotropy subgroup of the output to contain that of the input, and generically it should be exactly the image of $I_G(Y,H)$ under the kernel-projection operator $p_Y$ rather than something larger. Theorem 5.2 makes this precise: if the data is supported on a finite union of compact smooth $G$-invariant submanifolds and the map family has $C^\infty$ approximation capability, then dense approximants are almost isovariant relative to $Y$, with $\rho_Y(G_x)=\rho_Y(G_{f(x)})$ holding on every stratum except a subset of measure zero; if the feature space contains more than $\max_j \dim M_j$ copies of each needed component, full relative isovariance can be achieved. Thus the paper claims that the expressive collapse of equivariant networks on symmetric data is predictable and, in principle, avoidable by design.
Load-bearing premise
The proofs that generic equivariant maps realize the infimum assume the data lies exactly on a finite union of compact, smooth, $G$-invariant submanifolds of the input space; real point clouds and molecules are discrete, finite, and noisy, so the almost-everywhere statement applies to that idealization rather than to raw data.
Editorial extensions
If this is right
- A fixed feature space has a fixed minimal symmetry gain: every equivariant network built on that feature space must lose at least the information erased by $I_G(Y,H)$, regardless of architecture or training.
- The algorithm gives a pre-training check: compute $I_G(Y,H)$ for the input symmetries present in a dataset, and the degeneracies, such as full, axial, or half collapse for $k$-fold objects, are known before any experiment.
- Because almost isovariance is generic for expressive maps, sufficiently capable equivariant models should exhibit exactly the predicted infimum behavior on symmetric inputs, not a worse collapse; the paper's TFN, HEGNN, and QM9 experiments are consistent with this.
- Increasing the multiplicity of feature components above the dimension of the data manifold removes the remaining exceptional inputs and yields full relative isovariance, providing a constructive remedy.
- Feature-selection guidelines follow: orientation-dependent tasks should include components containing the projected orbit type $p_Y(H)$, and general tasks should avoid components whose infimum is the full symmetry group, since those components are annihilated.
Reading between the lines
- This suggests a compositional design principle that goes beyond the paper's stated guidelines: because the infimum of a direct sum can only be smaller than or equal to the infimum of each summand, adding features induces a partial order on symmetry loss, and bottleneck-type subgroups determine when new components can introduce new orbit types.
- The same orbit-type stratification should predict degeneracies in any smooth equivariant map, not just neural networks, so learned equivariant simulators or generative models on rotationally symmetric data should display the same collapse near symmetric states.
- A testable extension is to perturb data off the manifold: if almost-isovariance is the mechanism, adding noise should break the almost-everywhere guarantee at finite sample size, with the observed degeneracies controlled by the local dimension of the data distribution.
- The closed-form infimum tables for $SO(3)$ and $O(3)$ could seed an architecture search that scores candidate irrep selections by whether $I_G(Y,H)$ equals $p_Y(H)$ for the molecular point groups present in a dataset, and uses that score as a feature-selection regularizer.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies the phenomenon of symmetry increase in equivariant neural networks: for a G-equivariant map f:X→Y, the isotropy subgroup of x is contained in that of f(x). The authors prove (Thm. 3.1) that for every closed H⊆G the fixed-point subspace X^H has a unique minimal orbit type, defining a symmetry infimum, and give necessary conditions for the existence of (relative) isovariant maps (Thms. 3.2–3.3). For SO(3)/O(3) feature spaces they propose a high-multiplicity sufficiency condition for Michel's criterion (Prop. 4.2), algorithms to test orbit types and compute the infimum, and detailed tables (Appendix E) predicting full/axial/half degenerations for k-fold inputs. Under the manifold hypothesis, they prove C∞-density of TFNs in smooth equivariant maps (Thm. 5.1) and a genericity result (Thm. 5.2) asserting that, for any target map, there exist arbitrarily close approximating equivariant maps that are almost isovariant on the data manifold, with full isovariance at sufficiently high multiplicity. Experiments on synthetic k-fold point clouds and QM9 polarizability are offered as validation.
Significance. If the results hold, the paper provides a unifying and computable framework for an important failure mode of ENNs. The symmetry-infimum concept is natural and the derivation is largely self-contained; the proofs in Appendix B–D are detailed, the SO(3)/O(3) tables are consistent with the Michel/Ihrig-Golubitsky/Linehan-Stedman criteria, the C∞-density theorem extends Dym & Maron in a useful way, and the experiments—especially the sharp 2D/3D k-fold classification in Table 21—align with the tables. The paper also ships a reproducibility commitment with code. My reservations concern the gap between the proved density statement and the advertised 'for most equivariant maps' claim, and a missing chain argument in the general version of Prop. 4.2; both are fixable and do not undermine the core infimum calculus.
major comments (2)
- [§5.2, Theorem 5.2, Prop. D.20] Theorem 5.2 and Prop. D.20 establish only a C∞-dense (Baire-category) statement: there is a dense subset G⊂F on which the relevant transversality and dimension bounds hold, so for every target f and ϵ one can find a good approximating map g. This does not imply that 'most' parameters of a TFN, in the measure-theoretic sense relevant to random initialization or gradient descent, are almost isovariant; an open dense subset of a finite-dimensional parameter space can have arbitrarily small Lebesgue measure, and §D.4 (Remark after Prop. D.19) explicitly notes that the openness of the transversality property does not generally hold. The abstract's 'for most equivariant maps' and §5.2's 'a significant portion of maps within a dense parameterization' are therefore stronger than what is proved. Since contribution (iii) is the basis for the practical claim that symmetry increase is predictable in actual ENNs, the paper should either prove a measure-theoretic typicality statement or replace 'most'/'significant portion' by 'topologically generic/existence' throughout.
- [§4.1, Prop. 4.2, Algorithms 1–2] The proof of Prop. 4.2 passes from the Michel condition on adjacent supergroups to the assertion 'for any closed subgroup H′ of G containing H, dim V^{H′}<dim V^H' without justification. This step is nontrivial for compact Lie groups whose subgroup lattice is infinite (e.g., tori): one must show that any proper supergroup H′ contains an adjacent supergroup K of H with dim V^K = dim V^{H′} (or otherwise derive the inequality by a chain argument). As written, the sufficiency of the adjacent-subgroup check—and hence the correctness of Algo. 1 for arbitrary compact G—is not fully established. The concrete SO(3)/O(3) computations in §E are explicit and checked, so the paper's applications survive, but the claimed algorithm's generality should be restricted to those groups, or the missing chain argument should be supplied.
minor comments (5)
- [§6.2] The threshold-based interpretation of Fig. 5—values above 10^-3 indicate distinguishable and below 10^-6 indistinguishable—is a numerical heuristic; it should be stated as such and ideally supported by a sensitivity analysis or explicit numerical-error bound.
- [§6.3] Several point groups in Fig. 6 and Table 22 have very small sample sizes (e.g., Td with 5, C3h with 4, D3h with 15); the text should explicitly acknowledge that the QM9 evidence for these groups is anecdotal rather than a statistical validation.
- [Theorem 5.1, Eq. (10)] The notation D^k is used in Eq. (10) but the surrounding text sometimes writes 'Dkf'; please define D^k before first use and make the notation uniform.
- [§4.1, Eq. (7)] Equation (7) contains a garbled symbol 'L∞ l0=0' and should be typeset as a direct sum over l0, e.g., ⊕_{l0=0}^{∞}.
- [Algorithm 2] The return value 'min(O)' relies on the partial order of orbit types; please state explicitly at that point that the minimum is with respect to the orbit-type ordering and is unique by Theorem 3.1.
Circularity Check
No significant circularity: the symmetry-infimum and genericity claims are derived from external mathematical criteria, and the experiments test rather than define the predicted constants.
full rationale
The paper's load-bearing derivation is self-contained and does not reduce to its own inputs. Theorem 3.1 (uniqueness of the minimal orbit type in a fixed-point subspace) is proved from external results by Azzi, Desmorat, Grivaux, and Kolev (2023, 2026), not from the authors' own prior work. The orbit-type computations in Sec. 4 use Michel (1980), Ihrig and Golubitsky (1984), and Linehan and Stedman (2001), all independent published criteria, and the paper explicitly notes where its high-multiplicity tables agree with Linehan and Stedman's single-representation tables. The density/genericity argument in Sec. 5 builds on Bierstone (1977), Dym and Maron (2021), and Pinkus (1999), again external and not fitted to the experimental results. The only self-citations, chiefly Cen et al. (2024, 2025), appear as architectural context (HEGNN), historical comparison, or experimental backbone selection; none supplies the uniqueness theorem, the orbit-type criterion, or the density conclusion. The empirical sections precompute the predicted degeneration patterns from the tables and then test them on randomly initialized TFN/HEGNN embeddings and QM9 error profiles; the clear 1e-3/1e-6 separation in Sec. 6.2 is observed, not used to define the infimum entries. The skeptic's objection that C-infinity density does not establish measure-theoretic typicality is a genuine rigor concern—and the paper itself remarks after Prop. D.19 that the relevant openness does not generally hold—but this concerns the strength of the interpretation of 'most equivariant maps', not circularity, since the asserted density still follows from independent transversality theorems rather than from the paper's conclusions. No step was found in which an equation, fitted parameter, or self-citation is the source of the claimed prediction by construction.
Assumptions & free parameters
assumptions (5)
- domain assumption G is a compact Lie group acting linearly on finite-dimensional real vector spaces.
- domain assumption The data is supported on a finite union of compact, connected, smooth, G-invariant submanifolds of the input space.
- ad hoc to paper For the sufficiency of Michel's criterion, every non-zero isotypic component has multiplicity m(V,V_i) > dim G.
- standard math The orbit-type stratification of representation spaces satisfies Whitney's conditions (Field 2007, Prop. 3.9.2; Bierstone 1977).
- standard math TFN with polynomial filters is universal over equivariant polynomial maps (Dym & Maron 2021, Lem. 4) and MLPs approximate functions with all derivatives (Pinkus 1999, Thm. 4.1).
Cite this review
Pith. "Pith review of Reducing Symmetry Increase in Equivariant Neural Networks." pith.science (2026). https://pith.science/paper/Q5KDG7XI
@misc{pith2026260812010,
author = {Pith},
title = {Pith review of: Reducing Symmetry Increase in Equivariant Neural Networks},
year = {2026},
howpublished = {\url{https://pith.science/paper/Q5KDG7XI}},
note = {Machine review of arXiv:2608.12010}
}
read the original abstract
Equivariant Neural Networks (ENNs) have empowered numerous applications in scientific fields. Despite their remarkable capacity for representing geometric structures, ENNs suffer from degraded expressivity when processing symmetric inputs: the output representations are invariant to transformations that extend beyond the input's symmetries. The mathematical essence of this phenomenon is that a symmetric input, after being processed by an equivariant map, experiences an increase in symmetry. While prior research has documented symmetry increase in specific cases, a rigorous understanding of its underlying causes and general reduction strategies remains lacking. In this paper, we provide a detailed and in-depth characterization of symmetry increase together with a principled framework for its reduction: (i) For any given feature space and input symmetry group, we prove that the increased symmetry admits an infimum determined by the structure of the feature space; (ii) Building on this foundation, we develop a computable algorithm to derive this infimum, and propose practical guidelines for feature design to prevent harmful symmetry increases. (iii) Under standard regularity assumptions, we demonstrate that for most equivariant maps, our guidelines effectively reduce symmetry increase. To complement our theoretical findings, we provide visualizations and experiments on both synthetic datasets and the real-world QM9 dataset. The results validate our theoretical predictions.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Scaling Learning Algorithms Towards
Bengio, Yoshua and LeCun, Yann , booktitle =. Scaling Learning Algorithms Towards
-
[2]
and Osindero, Simon and Teh, Yee Whye , journal =
Hinton, Geoffrey E. and Osindero, Simon and Teh, Yee Whye , journal =. A Fast Learning Algorithm for Deep Belief Nets , volume =
-
[3]
2016 , publisher=
Deep Learning , author=. 2016 , publisher=
2016
-
[4]
International conference on machine learning , pages=
On the expressive power of geometric graph neural networks , author=. International conference on machine learning , pages=. 2023 , organization=
work page 2023
-
[5]
Topology from the differentiable viewpoint , isbn =
Milnor, John Willard , year =. Topology from the differentiable viewpoint , isbn =
- [6]
- [7]
-
[8]
Symmetry defects and broken symmetry
Michel, Louis , year=. Symmetry defects and broken symmetry. Configurations Hidden Symmetry , journal=
Show all 63 references
-
[9]
Pattern selection with
Ihrig, E and Golubitsky, Martin , journal=. Pattern selection with. 1984 , publisher=
1984
-
[10]
Linehan, M. J. and Stedman, G. E. , journal=. Little groups of irreps of. 2001 , publisher=
2001
-
[11]
1972 , publisher=
Introduction to compact transformation groups , author=. 1972 , publisher=
1972
-
[13]
arXiv preprint arXiv:2301.08599v2 , year=
On the isotropy stratification of a real representation of a compact Lie group , author=. arXiv preprint arXiv:2301.08599v2 , year=
-
[14]
The Annals of Mathematics , author =
Equivariant Embeddings in Euclidean Space , volume =. The Annals of Mathematics , author =. doi:10.2307/1970055 , pages =
-
[15]
2003 , url =
Group actions on manifolds , author =. 2003 , url =
2003
-
[16]
Group theory and its application to physical problems , publisher =
Hamermesh, Morton , year =. Group theory and its application to physical problems , publisher =
-
[17]
, year =
Golubitsky, Martin and Stewart, Ian and Schaeffer, David G. , year =. Singularities and Groups in Bifurcation Theory , volume =
-
[18]
Canadian Journal of Mathematics , volume=
On the symmetries of spherical harmonics , author=. Canadian Journal of Mathematics , volume=. 1954 , publisher=
1954
-
[19]
Introduction to Number Theory , rights =
Keng, Hua Loo , urldate =. Introduction to Number Theory , rights =. 1982 , langid =
1982
-
[20]
and Gusein-Zade, S.M
Arnold, V.I. and Gusein-Zade, S.M. and Varchenko, A.N. , year =. Singularities of Differentiable Maps: Classification of Critical Points, Caustics and Wave Fronts , volume =
-
[21]
Transactions of the American Mathematical Society , volume=
General position of equivariant maps , author=. Transactions of the American Mathematical Society , volume=
-
[22]
Local Properties of Analytic Varieties , isbn =
Whitney, Hassler , editor =. Local Properties of Analytic Varieties , isbn =. Hassler Whitney Collected Papers , publisher =. 1992 , langid =. doi:10.1007/978-1-4612-2972-8_33 , keywords =
1992 doi
-
[23]
Feldman, E. A. , urldate =. The Geometry of Immersions. I , volume =. 1965 , note =. doi:10.2307/1994017 , pages =
1965 doi
-
[24]
Stability of transversality to a stratification implies Whitney (a)-regularity , volume =
Trotman, David , year=. Stability of transversality to a stratification implies Whitney (a)-regularity , volume =. doi:10.1007/BF01410081 , pages =
-
[25]
Real and complex singularities, Oslo , author=
Counterexamples in stratification theory: two discordant horns , pages=. Real and complex singularities, Oslo , author=
-
[26]
, year =
Lee, John M. , year =. Introduction to Smooth Manifolds , volume =
-
[27]
Stable mappings and their singularities , isbn=
Golubitsky, Martin and Guillemin, Victor , year=. Stable mappings and their singularities , isbn=
-
[28]
Handbook of geometry and topology of singularities I , pages=
Stratification theory , author=. Handbook of geometry and topology of singularities I , pages=. 2020 , publisher=
2020
-
[29]
Archiv der Mathematik , volume=
The weak isovariant Borsuk-Ulam theorem for compact Lie groups , author=. Archiv der Mathematik , volume=. 2003 , doi =
2003
-
[30]
Differential topology , isbn=
Guillemin, Victor and Pollack, Alan , year=. Differential topology , isbn=
-
[31]
Acta numerica , volume=
Approximation theory of the MLP model in neural networks , author=. Acta numerica , volume=. 1999 , publisher=
1999
-
[32]
On the Universality of Rotation Equivariant Point Cloud Networks , booktitle =
Dym, Nadav and Maron, Haggai , year =. On the Universality of Rotation Equivariant Point Cloud Networks , booktitle =
-
[33]
2007 , publisher=
Lie groups: an approach through invariants and representations , author=. 2007 , publisher=
2007
-
[34]
Advances in Neural Information Processing Systems , author =
Are High-Degree Representations Really Unnecessary in Equivariant Graph Neural Networks? , volume =. Advances in Neural Information Processing Systems , author =. 2024 , langid =
2024
-
[35]
Symmetry Breaking and Equivariant Neural Networks , booktitle =
Kaba, Sékou-Oumar and Ravanbakhsh, Siamak , year =. Symmetry Breaking and Equivariant Neural Networks , booktitle =
-
[36]
Physical Review Research , volume=
Finding symmetry breaking order parameters with euclidean neural networks , author=. Physical Review Research , volume=. 2021 , publisher=
2021
-
[37]
arXiv preprint arXiv:1802.08219 , year=
Tensor field networks: Rotation-and translation-equivariant neural networks for 3D point clouds , author=. arXiv preprint arXiv:1802.08219 , year=
-
[38]
arXiv preprint arXiv:2207.09453 , year=
e3nn: Euclidean neural networks , author=. arXiv preprint arXiv:2207.09453 , year=
-
[39]
Aroyo, M. I. , urldate =. International Tables for Crystallography: Space-group symmetry , volume =. 2016 , edition =
2016
-
[40]
Chen, J. Q. and Ping, Jialun and Wang, Fan , note =. Group representation theory for physicists , isbn =. 2022 , edition =
2022
-
[41]
1970 , location =
General topology , isbn =. 1970 , location =
1970
-
[42]
Scientific data , volume=
Quantum chemistry structures and properties of 134 kilo molecules , author=. Scientific data , volume=. 2014 , publisher=
2014
-
[43]
2025 , url =
pointgroup: Python library to determine the point symmetry group of molecular geometries , author =. 2025 , url =
2025
-
[44]
arXiv preprint arXiv:1903.02428 , year=
Fast graph representation learning with PyTorch Geometric , author=. arXiv preprint arXiv:1903.02428 , year=
1903 arXiv
-
[45]
Artificial Intelligence Chemistry , pages=
A Method For Predicting Molecular Point Group Based On Graph Neural Networks , author=. Artificial Intelligence Chemistry , pages=. 2025 , publisher=
2025
-
[46]
arXiv preprint arXiv:2207.02862 , year=
Verifying the union of manifolds hypothesis for image data , author=. arXiv preprint arXiv:2207.02862 , year=
-
[47]
2021 , eprint=
Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges , author=. 2021 , eprint=
2021
-
[48]
Deep Learning in Drug Design , pages=
Geometric graph learning for drug design , author=. Deep Learning in Drug Design , pages=. 2026 , publisher=
2026
-
[49]
Universally Invariant Learning in Equivariant
Jiacheng Cen and Anyi Li and Ning Lin and Tingyang Xu and Yu Rong and Deli Zhao and Zihe Wang and Wenbing Huang , booktitle=. Universally Invariant Learning in Equivariant
-
[50]
Forty-first International Conference on Machine Learning , year=
Improving Equivariant Graph Neural Networks on Large Geometric Graphs via Virtual Nodes Learning , author=. Forty-first International Conference on Machine Learning , year=
-
[51]
arXiv preprint arXiv:2506.19482 , year=
Fast and Distributed Equivariant Graph Neural Networks by Virtual Node Learning , author=. arXiv preprint arXiv:2506.19482 , year=
-
[52]
International Conference on Learning Representations , year=
Frame Averaging for Invariant and Equivariant Network Design , author=. International Conference on Learning Representations , year=
-
[53]
International Conference on Machine Learning , pages=
Faenet: Frame averaging equivariant gnn for materials modeling , author=. International Conference on Machine Learning , pages=. 2023 , organization=
2023
-
[54]
2025 , title =
Aykent, Sarp and Xia, Tian , booktitle =. 2025 , title =
2025
-
[55]
International Conference on Machine Learning , year=
E (n) equivariant graph neural networks , author=. International Conference on Machine Learning , year=
-
[56]
Jinjia Feng and Zhewei Wei and Taifeng Wang and Zongyang Qiu , booktitle=. Tetra
-
[57]
The Fourteenth International Conference on Learning Representations , year=
Beyond Structure: Invariant Crystal Property Prediction with Pseudo-Particle Ray Diffraction , author=. The Fourteenth International Conference on Learning Representations , year=
-
[58]
Size-Generalizable
Zongzhao Li and Jiacheng Cen and Wenbing Huang and Taifeng Wang and Le Song , booktitle=. Size-Generalizable
-
[59]
Advances in Neural Information Processing Systems , volume=
Equivariant Spatio-Temporal Attentive Graph Networks to Simulate Physical Dynamics , author=. Advances in Neural Information Processing Systems , volume=
-
[60]
The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=
Geometric Mixture Models for Electrolyte Conductivity Prediction , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=
-
[61]
Nature Communications , volume=
Powder diffraction crystal structure determination using generative models , author=. Nature Communications , volume=. 2025 , publisher=
2025
-
[62]
arXiv preprint arXiv:2503.10471 , year=
Siamese foundation models for crystal structure prediction , author=. arXiv preprint arXiv:2503.10471 , year=
-
[63]
arXiv preprint arXiv:2602.04734 , year=
DMFlow: Disordered Materials Generation by Flow Matching , author=. arXiv preprint arXiv:2602.04734 , year=
-
[64]
Frontiers of Computer Science , volume=
A survey of geometric graph neural networks: Data structures, models and applications , author=. Frontiers of Computer Science , volume=. 2025 , publisher=
2025
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.