Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

On the Ability of Deep Networks to Learn Symmetries from Data: A Neural Kernel Theory

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Deep networks trained from data alone cannot learn symmetries their architecture lacks, a neural kernel theory shows.

desk verdict A genuinely useful spectral error formula for kernel regression on cyclic datasets, with a real but addressable gap between the exact circulant theory and the rotated-MNIST validation. read the letter →

arxiv 2412.11521 v2 pith:XDZFECGU submitted 2024-12-16 cs.LG

Andrea Perin , Stephane Deny This is my paper · ORCID
classification cs.LG
keywords deeplearningsymmetryneuraltangentkernelregressionspectralerrorcirculantmatricesequivariantnetworksrotated-MNIST
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish a precise condition under which conventional deep networks generalize a partially observed symmetry to unseen class poses: they do so only when a simple spectral error, the ratio of the inverse highest-frequency eigenvalue to the average inverse spectrum of the neural kernel, is small. That ratio is small exactly when classes are well separated in kernel space and orbits are densely sampled, so generalization is driven by local data geometry rather than by any learned non-local group structure. Because standard architectures have no mechanism for the kernel to adapt to the dataset's symmetry, the paper concludes that conventional networks cannot learn symmetries that were not embedded a priori. This matters because it turns an observed empirical failure, networks trained on rotated-MNIST with one pose missing per class, into a quantitative prediction about when the failure will and will not occur.

What carries the argument

The central object is the spectral error $\epsilon_s = \lambda_N^{-1}/\langle\lambda^{-1}\rangle$ of Proposition 4, derived for a circulant Gram matrix over an interleaved two-class cyclic-orbit dataset. Because a circulant matrix is diagonalized by the discrete Fourier transform, the difficulty of kernel regression on a missing orbit point reduces to a ratio of inverse frequency powers: the contribution of the class-alternating highest frequency against all frequencies. The paper uses this identity as a lens across RBF kernels, dot-product MLP kernels, and convolutional kernels, and extends it in Theorem 15 to arbitrary finite groups via non-commutative Fourier analysis.

What would settle it

Run exact NTK regression on rotated-MNIST orbit pairs without circularizing the Gram matrix and compare the prediction error to Eq. 1 per orbit pair; if the discrepancy grows with the degree to which the two orbits fail to be interleaved, for example when seeds lie near the rotation stabilizer, then the circularity approximation is load-bearing.

Watch

Extended reading notes

Core claim

On a dataset of two interleaved cyclic-group orbits with alternating labels, kernel regression on a held-out orbit point has error $\epsilon_s = \lambda_N^{-1}/\langle\lambda^{-1}\rangle$, where $\lambda_N$ is the eigenvalue of the highest Fourier frequency of the circulant Gram matrix and $\langle\lambda^{-1}\rangle$ is the average inverse eigenvalue over all frequencies. The paper shows this formula accounts for the generalization of MLPs, CNNs, and a ViT, both in the infinite-width NTK limit and for finite-width trained networks on rotated-MNIST, and that it recovers exact invariance only when the architecture is fully equivariant to the data symmetry, such as a translation-invariant global-average-pooling CNN on translated images. The central conclusion is that conventional deep networks lack a mechanism to learn symmetries that are not already built into their kernel or architecture, and that apparent symmetry learning is actually successful local interpolation in kernel space.

Load-bearing premise

The exact spectral error formula requires the Gram matrix to be circulant over perfectly interleaved orbits; for real MNIST orbit pairs the paper obtains this by diagonal-wise averaging, an approximation the authors state is not justified a priori, so the rotated-MNIST agreement depends on that post-hoc symmetrization being faithful.

Editorial extensions

If this is right

  • Increasing the number of sampled angles in a class orbit monotonically lowers the predicted error, so denser symmetry sampling produces gradual improvement without a phase transition or grokking.
  • Increasing class separation in kernel space increases the highest-frequency eigenvalue and lowers the error; separable classes need less orbit density to generalize.
  • Equivariant architectures only guarantee perfect generalization when their built-in invariance matches the data symmetry; a global-average-pooling CNN gives zero spectral error on translations but not on rotations.
  • The same spectral error formula applies to any finite group, with representation-theoretic Fourier coefficients replacing scalar DFT coefficients, so the conclusion is not an artifact of cyclic rotations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: a testable extension is to measure the spectral error directly on non-circularized data, because if real image orbits violate the interleaving assumption, the theory's quantitative predictions should degrade in a measurable way.
  • Editorial inference: augmentation and self-supervised invariance losses effectively raise orbit density or class separation in kernel space, so comparing the spectral ratios they induce could predict which pretraining strategy will transfer a given symmetry.
  • Editorial inference: since the frozen-kernel premise is essential to the argument, genuine symmetry learning should be sought in feature-learning regimes where the kernel changes during training; the paper's conclusion does not rule out such regimes, but leaves them unexplored.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies supervised classification on datasets with cyclic-group symmetry where some classes are fully observed over a group orbit while others are only partially observed. In the infinite-width NTK limit, the authors treat training as kernel regression and derive the spectral error formula ε_s = λ_N^{-1}/⟨λ^{-1}⟩ (Eq. 1) for a two-class interleaved dataset with a circulant Gram matrix, interpreting the numerator and denominator as class separation and orbit density in kernel space. They extend the formula to general finite groups (Theorem 15), apply it to rotated-MNIST via a diagonal-wise circularization approximation, extend it to multi-seed and multi-class settings through pairwise averaging and a one-vs-many heuristic, and analyze equivariant CNN architectures. The empirical comparisons across MLP, CNN, and ViT lead to the conclusion that conventional deep networks lack a mechanism to learn symmetries that are not embedded in their architecture a priori.

Significance. The exact part of the paper is elegant and reproducible: Eq. 1 is derived rather than fitted, the RBF circular-dataset experiments in Sec. 4.3 verify it exactly, and the non-abelian extension in App. C broadens the reach of the theory. The broad qualitative match between the spectral theory and finite-width network behavior on rotated-MNIST, together with the released code, makes this a useful contribution to the neural-kernel literature. However, the rotated-MNIST and multi-class validations, which carry the headline negative conclusion, rest on approximations that are acknowledged but not quantified. If those approximations are shown to be faithful, the paper provides an interpretable criterion for symmetry generalization; without such a check, the central claim for real data remains a plausible conjecture rather than a demonstrated consequence of the exact theory.

major comments (3)
  1. [Sec. 4.4, 'Approximation: Circularity'] The application of Eq. 1 to rotated-MNIST replaces the true non-circulant Gram matrix K_NTK with the diagonal-wise averaged matrix \tilde{K}_NTK, a procedure the authors state 'is not justified a priori'. Because Prop. 4 and App. B prove Eq. 1 only for circulant Gram matrices on exactly interleaved orbits, the agreement in Figs. 4B and 5A-C is a correlation between the spectral error of a modified regression problem and the exact error of the original problem, not a test of Eq. 1 on the actual Gram matrix. This is load-bearing, since the same approximation enters the multi-seed and multi-class validation (Fig. 6) and therefore the paper's headline conclusion about rotated-MNIST. Please quantify the approximation error (e.g., the relative Frobenius norm ||K - \tilde{K}|| / ||K|| and the L2 difference between the leave-one-out predictors under K and \tilde{K}) and either prove a bound for dot-product kernels on non-interleaved orbits or provide a controlled dataset where the circulant assumption is satisfied exactly. The GAP-CNN discussion in Sec. 4.6.3, where the authors say the circularization 'may reshape its structure too drastically', makes clear that the approximation's fidelity is architecture-dependent and cannot be assumed from the MLP agreement alone.
  2. [Sec. 4.5] The multi-seed extension replaces the exact two-orbit problem with an arithmetic average of spectral errors over all cross-class orbit pairs, introduced as a 'simplifying assumption'; the text immediately notes that the resulting error 'does not capture well the magnitude of the exact error anymore'. No derivation or error bound is given for this linear averaging, and the subsequent multi-class extension relies on a one-vs-many heuristic. These steps are needed to connect the exact theorem to the realistic rotated-MNIST experiments in Fig. 6, so the conclusion that finite-width networks display no symmetry-learning mechanism on this dataset is not a direct corollary of Eq. 1. Please either derive the averaging approximation, or demonstrate its validity by comparing the average of pairwise exact NTK errors to the exact multi-orbit NTK error on the same datasets, ideally for more than two classes.
  3. [Sec. 4.6 and App. E] Propositions 11-14, which state that FC-CNN and GAP-CNN kernels are circulant or constant over translation and rotation orbits, are proved for the NNGP kernel, while the main-text experiments and spectral-error computations use the NTK; the authors write that 'essentially the same proofs should hold for the NTK' but do not provide them. Since Eq. 1 applies only to the exact Gram matrix, the empirical CNTK comparisons in Sec. 4.6.3 and App. I are only justified if the NTK Gram matrix inherits the circulant structure. Please supply the NTK versions of these proofs, or state explicitly which of the CNTK figures would be affected if the NTK matrix deviates from circulancy.
minor comments (5)
  1. [Sec. 4.4, p. 15] The phrase 'anex-post circularization procedure' should read 'an ex-post circularization procedure'.
  2. [Sec. 4.6.3] The references to 'Prop. 3', 'Prop. 4', 'Prop. 1', and 'Prop. 2' should point to Props. 11-14.
  3. [Fig. 5E and Sec. 4.6.3] The caption says 'the assumptions of the theory are too crude to capture empirical phenomenology', while the main text describes the agreement as 'worse, although still reasonable'; these two characterizations should be reconciled.
  4. [App. B, Eq. (6)] The label formula µ_i = µ ω^{iN} + (µ0 − µ)δ_i0 implicitly assumes the missing point is at index 0; this is stated nearby but could be made more prominent for readers applying the formula to other missing points.
  5. [Sec. 4.4, 'Adjustment: Symmetrization'] The symmetrized NTK error averages over removing a point from either orbit, but the individual class-specific errors are not reported; reporting them would help assess whether the symmetrization hides a systematic asymmetry between the two orbits.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the spectral error formula is independently derived, and the Section 4.4 circulantization is an explicitly acknowledged approximation, not a fitted input or self-referential construction.

full rationale

The paper's central result, the spectral error formula in Eq. (1), is derived in Proposition 4 and Appendix B from Gaussian/kernel regression conditioning on a circulant Gram matrix. No free parameter is fitted to the empirical target, and the derivation is self-contained given the stated circulant and interleaved-label assumptions. The rotated-MNIST application in Section 4.4 is the only place where a circularity concern could arise: the true NTK Gram matrix over two MNIST orbits is not circulant, and the authors construct a circulant Ktilde by diagonal-wise averaging, explicitly writing that this circularization 'is not justified a priori.' This is an acknowledged approximation rather than a circular reduction, because the spectral error is not defined to be the exact NTK error; it is computed from Ktilde while the comparison target is the exact error from the true K, and the agreement is checked empirically across many random orbit pairs. Notably, the authors also report a failure case for the GAP-CNN, showing that the approximation is not trivially forced. The paper's headline conclusion that conventional networks lack a mechanism to learn unembedded symmetries is a consequence of the infinite-width NTK model assumption of a frozen, architecture-determined kernel, and the authors explicitly list limits outside the NTK regime; it is not obtained by defining the conclusion into the premises. Self-citations to prior work by the same group (e.g., Abbas and Deny 2023; Ollikka et al. 2025) appear only as supporting empirical background and are not load-bearing for the derivation, which rests on external kernel-theory results such as Jacot et al. and Neal. Thus there is no circularity warranting a nonzero score.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The exact theory rests on standard GP regression, circulant matrix diagonalization, and Schur orthogonality. The application to real data adds two non-derived assumptions: the circularization of non-interleaved orbit Gram matrices and the linear pairwise averaging across seeds and classes. No free parameters or invented entities are introduced.

assumptions (4)
  • domain assumption Kernel regression in the infinite-width limit captures the behavior of trained finite-width networks.
    The entire framework equates supervised training with GP regression over the frozen NTK (Section 4, App. A). Finite-width equivalence is not guaranteed; the authors validate this empirically but cannot prove it.
  • ad hoc to paper The Gram matrix of the neural kernel over a two-orbit MNIST dataset is approximately circulant after diagonal-wise averaging.
    Sec. 4.4 introduces this circularization and states it is not justified a priori, only empirically supported.
  • ad hoc to paper Multi-seed and multi-class errors are well approximated by averaging pairwise orbit spectral errors.
    Sec. 4.5 states: 'we make the simplifying assumption that pairs of orbits interact linearly'; this is not derived.
  • standard math DFT diagonalizes circulant matrices and Schur orthogonality block-diagonalizes group-circulant kernels.
    Used in App. B and App. C for deriving the spectral error formula and Theorem 15.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On the Ability of Deep Networks to Learn Symmetries from Data: A Neural Kernel Theory." pith.science (2026). https://pith.science/paper/XDZFECGU

@misc{pith2026241211521,
  author       = {Pith},
  title        = {Pith review of: On the Ability of Deep Networks to Learn Symmetries from Data: A Neural Kernel Theory},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XDZFECGU}},
  note         = {Machine review of arXiv:2412.11521}
}
read the original abstract

Symmetries (transformations by group actions) are present in many datasets, and leveraging them holds considerable promise for improving predictions in machine learning. In this work, we aim to understand when and how deep networks -- with standard architectures trained in a standard, supervised way -- learn symmetries from data. Inspired by real-world scenarios, we study a classification paradigm where data symmetries are only partially observed during training: some classes include all transformations of a cyclic group, while others -- only a subset. In the infinite-width limit, where kernel analogies apply, we derive a neural kernel theory of symmetry learning. The group-cyclic nature of the dataset allows us to analyze the Gram matrix of neural kernels in the Fourier domain; here we find a simple characterization of the generalization error as a function of class separation (signal) and class-orbit density (noise). This characterization reveals that generalization can only be successful when the local structure of the data prevails over its non-local, symmetry-induced structure, in the kernel space defined by the architecture. We extend our theoretical treatment to any finite group, including non-abelian groups. Our framework also applies to equivariant architectures (e.g., CNNs), and recovers their success in the special case where the architecture matches the inherent symmetry of the data. Empirically, our theory reproduces the generalization failure of finite-width networks (MLP, CNN, ViT) trained on partially observed versions of rotated-MNIST. We conclude that conventional deep networks lack a mechanism to learn symmetries that have not been explicitly embedded in their architecture a priori. Our framework could be extended to guide the design of architectures and training procedures able to learn symmetries from data.

Figures

Figures reproduced from arXiv: 2412.11521 by the authors.

Figure 1
Figure 1. Common deep network architectures fail to extrapolate symmetries from a partially observed version of rotated-MNIST. A: A conceptual sketch of the learning task. For samples in the leave-out class (digit “5” in this example), the upright pose is not included in the training set (empty circles). For samples in the remaining classes, all poses are included in the training set (full circles). A model that can generaliz… view at source ↗
Figure 2
Figure 2. Case study: Analysis of the generalization behavior of the Gaussian kernel on a circular-symmetric dataset. A: A circular dataset is made of two sets of interleaved points in R 3 belonging to two classes (denoted in red and blue). One point (in black) is left out during training, and we regress on it. B: The pairwise distance matrix between the points is circulant (above), leading to a circulant kernel matrix (below… view at source ↗
Figure 3
Figure 3. Further geometric interpretations of the Gaussian kernel generalization behavior on a circular-symmetric dataset. A: We progressively increase the separation ∆ between two circular classes of points (respectively in blue and red). The leave-out point used for testing is in black. B: The prediction error (in black) of the RBF kernel on the leave￾out point decreases as a function of ∆, as predicted by our spectral for… view at source ↗
Figures from the paper (15 more)
Figure 4
Figure 4. Figure 4: Analysis of the prediction behavior of a MLP on pairs of orbits from rotated-MNIST. A: We take samples of two different MNIST digits and generate their rotation orbits. The task is to predict the label value of a leave-out point in one of the two orbits. We use angle s…
Figure 5
Figure 5. Figure 5: Spectral error matches exact NTK error across various architectures and for finite-width networks, on rotated-MNIST orbit pairs: (A) a MLP with 1 hidden layer, (B) a finite-width 1-hidden layer MLP trained with Adam, (C) a MLP with 5 hidden layers, (D) a ConvNet with a…
Figure 6
Figure 6. Figure 6: Application of the spectral theory to a MLP trained on a subsection of rotated-MNIST comprising multiple seeds per class and multiple classes. A: For datasets comprised of two classes of rotated-MNIST, comprising multiple seeds each (Nseed = 13), we compare the average…
Figure 7
Figure 7. Figure 7: Analysis of the prediction behavior of a fully connected convolutional network on pairs of orbits from rotated-MNIST. Same caption as [PITH_FULL_IMAGE:figures/full_fig_p024_7.png]
Figure 8
Figure 8. Figure 8: 2D sketch of the geometric interpretation, showing the effect of the whitening [PITH_FULL_IMAGE:figures/full_fig_p043_8.png]
Figure 9
Figure 9. Figure 9: Comparison between the errors as given by Eq. 13 and by standard numerical [PITH_FULL_IMAGE:figures/full_fig_p054_9.png]
Figure 10
Figure 10. Figure 10: Comparison of spectral and exact NTK errors across a range of values for [PITH_FULL_IMAGE:figures/full_fig_p064_10.png]
Figure 11
Figure 11. Figure 11: MLP, 5 hidden layers. 125 150 175 h∏°1i 1.2 1.4 1.6 1/Ω Nrot =8 °1.0 °0.5 0.0 0.5 log "s 2.5 5.0 7.5 ∏N 0.00 0.25 0.50 0.75 ¢2 £10°2 Nrot =8 °1.0 °0.5 0.0 0.5 log "s 0 1 2 Emp. error (train) 0.0 0.5 1.0 1.5 2.0 Spectral error Nrot =8 0 1 2 Num. correct 1.2 1.4 1.6 1.8…
Figure 12
Figure 12. Figure 12: Trained MLP. 65 [PITH_FULL_IMAGE:figures/full_fig_p065_12.png]
Figure 13
Figure 13. Figure 13: Comparison of spectral and exact NTK errors for the multiple seeds case. We compare the spectral error, averaged over all the pairings of orbits for a given number of pairs, and the symmetrized NTK prediction error, over a range of number of rotations in the orbits, N…
Figure 14
Figure 14. Figure 14: Our spectral method correctly reproduces the generalization behavior of a trained finite-width MLP on rotated-MNIST, as we vary the number of sampled angles. On a version of rotated-MNIST comprising all 10 classes and 13 seed images per class, we compare (A) a cross-e…
Figure 15
Figure 15. Figure 15: CNTK, Global Average Pooling, on rotation orbit. [PITH_FULL_IMAGE:figures/full_fig_p067_15.png]
Figure 16
Figure 16. Figure 16: CNTK, Global Average Pooling, on rotation orbit. Kernel size (4, 4), strides (4, [PITH_FULL_IMAGE:figures/full_fig_p067_16.png]
Figure 17
Figure 17. Figure 17: CNTK, Fully Connected, on translation orbit. [PITH_FULL_IMAGE:figures/full_fig_p068_17.png]
Figure 18
Figure 18. Figure 18: CNTK, Global Average Pooling, on translation orbit. [PITH_FULL_IMAGE:figures/full_fig_p068_18.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Zero-Shot Visual Generalization in Robot Manipulation

    cs.RO 2025-05 conditional novelty 6.0 of 10

    Disentangled latent representations with associative memory, paired with diffusion policies, provide zero-shot visual generalization on manipulation tasks, and a learned canonicalization step adds robustness to planar...

Reference graph

Works this paper leans on

126 extracted references · 78 canonical work pages · cited by 1 Pith paper

  1. [1]

    Progress and limitations of deep networks to recognize objects in unusual poses

    Amro Abbas and St\' e phane Deny. Progress and limitations of deep networks to recognize objects in unusual poses. In Conference on Artificial Intelligence (AAAI), 2023

    2023

  2. [2]

    Git Re-Basin : Merging models modulo permutation symmetries

    Samuel Ainsworth, Jonathan Hayase, and Siddhartha Srinivasa. Git Re-Basin : Merging models modulo permutation symmetries. In International Conference on Learning Representations (ICLR), 2023

    2023

  3. [3]

    Strike (with) a pose: Neural networks are easily fooled by strange poses of familiar objects

    Michael A Alcorn, Qi Li, Zhitao Gong, Chengfei Wang, Long Mai, Wei-Shinn Ku, and Anh Nguyen. Strike (with) a pose: Neural networks are easily fooled by strange poses of familiar objects. In Conference on Computer Vision and Pattern Recognition (CVPR), 2019

    2019

  4. [4]

    Symmetry-adapted representation learning

    Fabio Anselmi, Georgios Evangelopoulos, Lorenzo Rosasco, and Tomaso Poggio. Symmetry-adapted representation learning. Pattern Recognition, 2019

    2019

  5. [5]

    Data symmetries and learning in fully connected neural networks

    Fabio Anselmi, Luca Manzoni, Alberto d’Onofrio, Alex Rodriguez, Giulio Caravagna, Luca Bortolussi, and Francesca Cairoli. Data symmetries and learning in fully connected neural networks. IEEE Access, 2023

    2023

  6. [6]

    On exact computation with an infinitely wide neural net

    Sanjeev Arora, Simon S Du, Wei Hu, Zhiyuan Li, Russ R Salakhutdinov, and Ruosong Wang. On exact computation with an infinitely wide neural net. In Advances in Neural Information Processing Systems (NeurIPS), 2019

    2019

  7. [7]

    Unified theoretical framework for wide neural network learning dynamics

    Yehonatan Avidan, Qianyi Li, and Haim Sompolinsky. Unified theoretical framework for wide neural network learning dynamics. Physical Review E, 2025

    2025

  8. [8]

    Why do deep convolutional networks generalize so poorly to small image transformations? Journal of Machine Learning Research (JMLR), 2019

    Aharon Azulay and Yair Weiss. Why do deep convolutional networks generalize so poorly to small image transformations? Journal of Machine Learning Research (JMLR), 2019

    2019

Show all 126 references
  1. [9]

    Breaking the curse of dimensionality with convex neural networks

    Francis Bach. Breaking the curse of dimensionality with convex neural networks. Journal of Machine Learning Research (JMLR), 2017

    2017

  2. [10]

    Explaining neural scaling laws

    Yasaman Bahri, Ethan Dyer, Jared Kaplan, Jaehoon Lee, and Utkarsh Sharma. Explaining neural scaling laws. Proceedings of the National Academy of Sciences (PNAS), 2024

    2024

  3. [11]

    A cookbook of self-supervised learning

    Randall Balestriero, Mark Ibrahim, Vlad Sobal, Ari Morcos, Shashank Shekhar, Tom Goldstein, Florian Bordes, Adrien Bardes, Gregoire Mialon, Yuandong Tian, Avi Schwarzschild, Andrew Gordon Wilson, Jonas Geiping, Quentin Garrido, Pierre Fernandez, Amir Bar, Hamed Pirsiavash, Yan...

    2023

  4. [12]

    Developmental changes in children’s object insertions during play

    Valerie Bambha, Aaron Beckner, Nikita Shetty, Annika Voss, Jinlin Xie, Eunice Yiu, Vanessa LoBue, Lisa Oakes, and Marianella Casasola. Developmental changes in children’s object insertions during play. Journal of Cognition and Development, 2022

    2022

  5. [13]

    B-spline CNNs on Lie groups

    Erik J Bekkers. B-spline CNNs on Lie groups. arXiv preprint, 2021

    2021

  6. [14]

    Learning invariances in neural networks from training data

    Gregory Benton, Marc Finzi, Pavel Izmailov, and Andrew G Wilson. Learning invariances in neural networks from training data. In Advances in Neural Information Processing Systems (NeurIPS), 2020

    2020

  7. [15]

    Self-consistent dynamical field theory of kernel evolution in wide neural networks

    Blake Bordelon and Cengiz Pehlevan. Self-consistent dynamical field theory of kernel evolution in wide neural networks. In Advances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  8. [16]

    Spectrum dependent learning curves in kernel regression and wide neural networks

    Blake Bordelon, Abdulkadir Canatar, and Cengiz Pehlevan. Spectrum dependent learning curves in kernel regression and wide neural networks. In International Conference on Machine Learning (ICML), 2020

    2020

  9. [17]

    Addressing the topological defects of disentanglement via distributed operators

    Diane Bouchacourt, Mark Ibrahim, and Stéphane Deny. Addressing the topological defects of disentanglement via distributed operators. arXiv preprint, 2021

    2021

  10. [18]

    Does equivariance matter at scale? arXiv preprint, 2024

    Johann Brehmer, Sönke Behrends, Pim de Haan, and Taco Cohen. Does equivariance matter at scale? arXiv preprint, 2024

    2024

  11. [19]

    Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković

    Michael M. Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković. Geometric deep learning: Grids , groups, graphs, geodesics, and gauges. arXiv preprint, 2021

    2021

  12. [20]

    Spectral bias and task-model alignment explain generalization in kernel regression and infinitely wide neural networks

    Abdulkadir Canatar, Blake Bordelon, and Cengiz Pehlevan. Spectral bias and task-model alignment explain generalization in kernel regression and infinitely wide neural networks. Nature Communications, 2021

    2021

  13. [21]

    Vinod Kumar Chauhan, Jiandong Zhou, Ping Lu, Soheila Molaei, and David A. Clifton. A brief review of hypernetworks in deep learning. Artificial Intelligence Review, 2024

    2024

  14. [22]

    Deep reasoning networks for unsupervised pattern de-mixing with constraint reasoning

    Di Chen, Yiwei Bai, Wenting Zhao, Sebastian Ament, John Gregoire, and Carla Gomes. Deep reasoning networks for unsupervised pattern de-mixing with constraint reasoning. In International Conference on Machine Learning (ICML), 2020

    2020

  15. [23]

    Chirikjian and Alexander B

    Gregory S. Chirikjian and Alexander B. Kyatkin. Engineering Applications of Noncommutative Harmonic Analysis. CRC Press, 2021

    2021

  16. [24]

    On the global convergence of gradient descent for over-parameterized models using optimal transport

    L\' e na\" c Chizat and Francis Bach. On the global convergence of gradient descent for over-parameterized models using optimal transport. In Advances in Neural Information Processing Systems (NeurIPS), 2018

    2018

  17. [25]

    Lee, and Haim Sompolinsky

    SueYeon Chung, Daniel D. Lee, and Haim Sompolinsky. Classification and geometry of general perceptual manifolds. Physical Review X, 2018

    2018

  18. [26]

    Group equivariant convolutional networks

    Taco Cohen and Max Welling. Group equivariant convolutional networks. In International Conference on Machine Learning (ICML), 2016

    2016

  19. [27]

    Gauge equivariant convolutional networks and the icosahedral CNN

    Taco Cohen, Maurice Weiler, Berkay Kicanaoglu, and Max Welling. Gauge equivariant convolutional networks and the icosahedral CNN . In International Conference on Machine Learning (ICML), 2019 a

    2019

  20. [28]

    A general theory of equivariant CNNs on homogeneous spaces

    Taco S Cohen, Mario Geiger, and Maurice Weiler. A general theory of equivariant CNNs on homogeneous spaces. In Advances in Neural Information Processing Systems (NeurIPS), 2019 b

    2019

  21. [29]

    Lee, and Haim Sompolinsky

    Uri Cohen, SueYeon Chung, Daniel D. Lee, and Haim Sompolinsky. Separability and geometry of object manifolds in deep neural networks. Nature Communications, 2020

    2020

  22. [30]

    Representing closed transformation paths in encoded network latent space

    Marissa Connor and Christopher Rozell. Representing closed transformation paths in encoded network latent space. Conference on Artificial Intelligence (AAAI), 2020

    2020

  23. [31]

    Learning internal representations of 3D transformations from 2D projected inputs

    Marissa Connor, Bruno Olshausen, and Christopher Rozell. Learning internal representations of 3D transformations from 2D projected inputs. Neural Computation, 2024

    2024

  24. [32]

    Lagrangian neural networks

    Miles Cranmer, Sam Greydanus, Stephan Hoyer, Peter Battaglia, David Spergel, and Shirley Ho. Lagrangian neural networks. In ICLR Workshop on Integration of Deep Neural Models and Differential Equations, 2020

    2020

  25. [33]

    Learning transport operators for image manifolds

    Benjamin Culpepper and Bruno Olshausen. Learning transport operators for image manifolds. In Advances in Neural Information Processing Systems (NeurIPS), 2009

    2009

  26. [34]

    Convit: Improving vision transformers with soft convolutional inductive biases

    St \'e phane D'Ascoli, Hugo Touvron, Matthew L Leavitt, Ari S Morcos, Giulio Biroli, and Levent Sagun. Convit: Improving vision transformers with soft convolutional inductive biases. In International Conference on Machine Learning (ICML), 2021

    2021

  27. [35]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at...

    2021

  28. [36]

    Equivariant neural rendering

    Emilien Dupont, Miguel Bautista Martin, Alex Colburn, Aditya Sankar, Josh Susskind, and Qi Shan. Equivariant neural rendering. In International Conference on Machine Learning (ICML), 2020

    2020

  29. [37]

    Revisiting spatial invariance with low-rank local connectivity

    Gamaleldin Elsayed, Prajit Ramachandran, Jonathon Shlens, and Simon Kornblith. Revisiting spatial invariance with low-rank local connectivity. In International Conference on Machine Learning (ICML), 2020

    2020

  30. [38]

    Topological obstructions and how to avoid them

    Babak Esmaeili, Robin Walters, Heiko Zimmermann, and Jan-Willem van de Meent. Topological obstructions and how to avoid them. In Advances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  31. [39]

    Matthew Farrell, Blake Bordelon, Shubhendu Trivedi, and Cengiz Pehlevan. Capacity of group-invariant linear readouts from equivariant representations: How many objects can be linearly classified under all possible views? In International Conference on Learning Representations ...

    2022

  32. [40]

    Model-agnostic meta-learning for fast adaptation of deep networks

    Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning (ICML), 2017

    2017

  33. [41]

    Generalizing convolutional neural networks for equivariance to Lie groups on arbitrary continuous data

    Marc Finzi, Samuel Stanton, Pavel Izmailov, and Andrew Gordon Wilson. Generalizing convolutional neural networks for equivariance to Lie groups on arbitrary continuous data. In International Conference on Machine Learning (ICML), 2020

    2020

  34. [42]

    Deep learning versus kernel learning: An empirical study of loss landscape geometry and the time evolution of the neural tangent kernel

    Stanislav Fort, Gintare Karolina Dziugaite, Mansheej Paul, Sepideh Kharaghani, Daniel M Roy, and Surya Ganguli. Deep learning versus kernel learning: An empirical study of loss landscape geometry and the time evolution of the neural tangent kernel. In Advances in Neural Inform...

    2020

  35. [43]

    Learning and leveraging world models in visual representation learning

    Quentin Garrido, Mahmoud Assran, Nicolas Ballas, Adrien Bardes, Laurent Najman, and Yann LeCun. Learning and leveraging world models in visual representation learning. arXiv preprint, 2024

    2024

  36. [44]

    Disentangling feature and lazy training in deep neural networks

    Mario Geiger, Stefano Spigler, Arthur Jacot, and Matthieu Wyart. Disentangling feature and lazy training in deep neural networks. Journal of Statistical Mechanics: Theory and Experiment, 2020

    2020

  37. [45]

    Deep symmetry networks

    Robert Gens and Pedro Domingos. Deep symmetry networks. In Advances in Neural Information Processing Systems (NeurIPS), 2014

    2014

  38. [46]

    Probing transfer learning with a model of synthetic correlated datasets

    Federica Gerace, Luca Saglietti, Stefano Sarao Mannelli, Andrew Saxe, and Lenka Zdeborová. Probing transfer learning with a model of synthetic correlated datasets. Machine Learning: Science and Technology, 2022

    2022

  39. [47]

    Emergent equivariance in deep ensembles

    Jan E Gerken and Pan Kessel. Emergent equivariance in deep ensembles. In International Conference on Machine Learning (ICML), 2024

    2024

  40. [48]

    Modeling the influence of data structure on learning in neural networks: The hidden manifold model

    Sebastian Goldt, Marc M\'ezard, Florent Krzakala, and Lenka Zdeborov\'a. Modeling the influence of data structure on learning in neural networks: The hidden manifold model. Physical Review X, 2020

    2020

  41. [49]

    Hamiltonian neural networks

    Samuel Greydanus, Misko Dzamba, and Jason Yosinski. Hamiltonian neural networks. In Advances in Neural Information Processing Systems (NeurIPS), 2019

    2019

  42. [50]

    David J. Gross. The role of symmetry in fundamental physics. Proceedings of the National Academy of Sciences (PNAS), 1996

    1996

  43. [51]

    The Lie derivative for measuring learned equivariance

    Nate Gruver, Marc Anton Finzi, Micah Goldblum, and Andrew Gordon Wilson. The Lie derivative for measuring learned equivariance. In International Conference on Learning Representations (ICLR), 2023

    2023

  44. [52]

    Dey, Soham Mukherjee, Shreyas N

    Mustafa Hajij, Ghada Zamzmi, Theodore Papamarkou, Nina Miolane, Aldo Guzmán-Sáenz, Karthikeyan Natesan Ramamurthy, Tolga Birdal, Tamal K. Dey, Soham Mukherjee, Shreyas N. Samaga, Neal Livesay, Robin Walters, Paul Rosen, and Michael T. Schaub. Topological deep learning: Going b...

    2023

  45. [53]

    Towards a definition of disentangled representations

    Irina Higgins, David Amos, David Pfau, Sebastien Racaniere, Loic Matthey, Danilo Rezende, and Alexander Lerchner. Towards a definition of disentangled representations. arXiv preprint, 2018

    2018

  46. [54]

    Deep networks always grok and here is why

    Ahmed Imtiaz Humayun, Randall Balestriero, and Richard Baraniuk. Deep networks always grok and here is why. In International Conference on Machine Learning (ICML), 2024

    2024

  47. [55]

    Robust self-supervised learning with Lie groups

    Mark Ibrahim, Diane Bouchacourt, and Ari Morcos. Robust self-supervised learning with Lie groups. arXiv preprint, 2022

    2022

  48. [56]

    Morcos, and Diane Bouchacourt

    Mark Ibrahim, Quentin Garrido, Ari S. Morcos, and Diane Bouchacourt. The robustness limits of so TA vision models to natural variation. Transactions on Machine Learning Research (TMLR), 2023

    2023

  49. [57]

    van der Ouderaa, Gunnar R\" a tsch, Vincent Fortuin, and Mark van der Wilk

    Alexander Immer, Tycho F.A. van der Ouderaa, Gunnar R\" a tsch, Vincent Fortuin, and Mark van der Wilk. Invariance learning in deep neural networks with differentiable laplace approximations. In Advances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  50. [58]

    Neural tangent kernel: Convergence and generalization in neural networks

    Arthur Jacot, Franck Gabriel, and Clement Hongler. Neural tangent kernel: Convergence and generalization in neural networks. In Advances in Neural Information Processing Systems (NeurIPS), 2018

    2018

  51. [59]

    How DNN s break the curse of dimensionality: Compositionality and symmetry learning

    Arthur Jacot, Seok Hoan Choi, and Yuxiao Wen. How DNN s break the curse of dimensionality: Compositionality and symmetry learning. In International Conference on Learning Representations (ICLR), 2025

    2025

  52. [60]

    Spatial transformer networks

    Max Jaderberg, Karen Simonyan, Andrew Zisserman, and koray kavukcuoglu. Spatial transformer networks. In Advances in Neural Information Processing Systems (NeurIPS), 2015

    2015

  53. [61]

    Symmetry breaking and equivariant neural networks

    S \'e kou-Oumar Kaba and Siamak Ravanbakhsh. Symmetry breaking and equivariant neural networks. In NeurIPS Workshop on Symmetry and Geometry in Neural Representations, 2023

    2023

  54. [62]

    Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint, 2020

    2020

  55. [63]

    Anderson Keller and Max Welling

    T. Anderson Keller and Max Welling. Topographic vaes learn equivariant capsules. In Advances in Neural Information Processing Systems (NeurIPS), 2021 a

    2021

  56. [64]

    Anderson Keller and Max Welling

    T. Anderson Keller and Max Welling. Topographic vaes learn equivariant capsules. In Advances in Neural Information Processing Systems (NeurIPS), 2021 b

    2021

  57. [65]

    The formation and transformation of the perceptual world

    Ivo Kohler. The formation and transformation of the perceptual world. Psychological issues, 1963

    1963

  58. [66]

    LeCun, L

    Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998

    1998

  59. [67]

    Deep neural networks as Gaussian processes

    Jaehoon Lee, Jascha Sohl-dickstein, Jeffrey Pennington, Roman Novak, Sam Schoenholz, and Yasaman Bahri. Deep neural networks as Gaussian processes. In International Conference on Learning Representations (ICLR), 2018

    2018

  60. [68]

    How diffusion models learn to factorize and compose

    Qiyao Liang, Ziming Liu, Mitchell Ostrow, and Ila Fiete. How diffusion models learn to factorize and compose. In Advances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  61. [69]

    When does compositional structure yield compositional generalization? A kernel theory

    Samuel Lippl and Kim Stachenfeld. When does compositional structure yield compositional generalization? A kernel theory. In International Conference on Learning Representations (ICLR), 2025

    2025

  62. [70]

    When and how convolutional neural networks generalize to out-of-distribution category–viewpoint combinations

    Spandan Madan, Timothy Henry, Jamell Dozier, Helen Ho, Nishchal Bhandari, Tomotake Sasaki, Frédo Durand, Hanspeter Pfister, and Xavier Boix. When and how convolutional neural networks generalize to out-of-distribution category–viewpoint combinations. Nature Machine Intelligence, 2022

    2022

  63. [71]

    In-distribution adversarial attacks on object recognition models using gradient-free search

    Spandan Madan, Tomotake Sasaki, Hanspeter Pfister, Tzu-Mao Li, and Xavier Boix. In-distribution adversarial attacks on object recognition models using gradient-free search. arXiv preprint, 2025

    2025

  64. [72]

    Harmonics of learning: Universal fourier features emerge in invariant networks

    Giovanni Luca Marchetti, Christopher J Hillar, Danica Kragic, and Sophia Sanborn. Harmonics of learning: Universal fourier features emerge in invariant networks. In Conference on Learning Theory (COLT), 2024

    2024

  65. [73]

    A mean field view of the landscape of two-layer neural networks

    Song Mei, Andrea Montanari, and Phan-Minh Nguyen. A mean field view of the landscape of two-layer neural networks. Proceedings of the National Academy of Sciences (PNAS), 2018

    2018

  66. [74]

    Gabriel Mel and Surya Ganguli. A theory of high dimensional regression with arbitrary correlations between input features and target functions: Sample complexity, multiple descent curves and a hierarchy of phase transitions. In International Conference on Machine Learning (ICML), 2021

    2021

  67. [75]

    Symmetry-induced disentanglement on graphs

    Giangiacomo Mercatali, Andre Freitas, and Vikas Garg. Symmetry-induced disentanglement on graphs. In Advances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  68. [76]

    Smeulders

    Artem Moskalev, Anna Sepliarskaia, Erik J Bekkers, and Arnold W.M. Smeulders. On genuine invariance learning without weight-tying. In Proceedings of 2nd Annual Workshop on Topology, Algebra, and Geometry in Machine Learning (TAG-ML), 2023

    2023

  69. [77]

    Radford M. Neal. Bayesian Learning for Neural Networks. Springer-Verlag, 1996

    1996

  70. [78]

    E. Noether. Invariante variationsprobleme. Nachrichten von der Gesellschaft der Wissenschaften zu Göttingen, Mathematisch-Physikalische Klasse, 1918

    1918

  71. [79]

    Ensembles provably learn equivariance through data augmentation

    Oskar Nordenfors and Axel Flinth. Ensembles provably learn equivariance through data augmentation. arXiv preprint, 2024

    2024

  72. [80]

    Alemi, Jascha Sohl-Dickstein, and Samuel S

    Roman Novak, Lechao Xiao, Jiri Hron, Jaehoon Lee, Alexander A. Alemi, Jascha Sohl-Dickstein, and Samuel S. Schoenholz. Neural tangents: Fast and easy infinite neural networks in python. In International Conference on Learning Representations (ICLR), 2020

    2020

  73. [81]

    A comparison between humans and AI at recognizing objects in unusual poses

    Netta Ollikka, Amro Kamal Mohamed Abbas, Andrea Perin, Markku Kilpel \"a inen, and Stephane Deny. A comparison between humans and AI at recognizing objects in unusual poses. Transactions on Machine Learning Research (TMLR), 2025

    2025

  74. [82]

    Neural anisotropy directions

    Guillermo Ortiz-Jimenez, Apostolos Modas, Seyed-Mohsen Moosavi, and Pascal Frossard. Neural anisotropy directions. In Advances in Neural Information Processing Systems (NeurIPS), 2020

    2020

  75. [83]

    Breaking the symmetry: Mirror discrimination for single letters but not for pictures in the visual word form area

    Felipe Pegado, Kimihiro Nakamura, Laurent Cohen, and Stanislas Dehaene. Breaking the symmetry: Mirror discrimination for single letters but not for pictures in the visual word form area. NeuroImage, 2011

    2011

  76. [84]

    Suppression of mirror generalization for reversible letters: Evidence from masked priming

    Manuel Perea, Carmen Moret-Tatay, and Victoria Panadero. Suppression of mirror generalization for reversible letters: Evidence from masked priming. Journal of Memory and Language, 2011

    2011

  77. [85]

    Equivariant representation learning in the presence of stabilizers

    Luis Armando P \'e rez Rey, Giovanni Luca Marchetti, Danica Kragic, Dmitri Jarnikov, and Mike Holenderski. Equivariant representation learning in the presence of stabilizers. In Machine Learning and Knowledge Discovery in Databases: Research Track, 2023

    2023

  78. [86]

    Disentangling by subspace diffusion

    David Pfau, Irina Higgins, Alex Botev, and S\' e bastien Racani\` e re. Disentangling by subspace diffusion. In Advances in Neural Information Processing Systems (NeurIPS), 2020

    2020

  79. [87]

    Grokking: Generalization beyond overfitting on small algorithmic datasets

    Alethea Power, Yuri Burda, Harri Edwards, Igor Babuschkin, and Vedant Misra. Grokking: Generalization beyond overfitting on small algorithmic datasets. arXiv preprint, 2022

    2022

  80. [88]

    Dynamic routing between capsules

    Sara Sabour, Nicholas Frosst, and Geoffrey E Hinton. Dynamic routing between capsules. In Advances in Neural Information Processing Systems (NeurIPS), 2017

    2017

  81. [89]

    An analytical theory of curriculum learning in teacher-student networks

    Luca Saglietti, Stefano Mannelli, and Andrew Saxe. An analytical theory of curriculum learning in teacher-student networks. In Advances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  82. [90]

    Sophia Sanborn, Christian A Shewmake, Bruno Olshausen, and Christopher J. Hillar. Bispectral neural networks. In International Conference on Learning Representations (ICLR), 2023

    2023

  83. [91]

    Saxe, James L

    Andrew M. Saxe, James L. McClelland, and Surya Ganguli. A mathematical theory of semantic development in deep neural networks. Proceedings of the National Academy of Sciences (PNAS), 2019

    2019

  84. [92]

    u gelgen, Frederik Tr \

    Lukas Schott, Julius Von K \"u gelgen, Frederik Tr \"a uble, Peter Vincent Gehler, Chris Russell, Matthias Bethge, Bernhard Sch \"o lkopf, Francesco Locatello, and Wieland Brendel. Visual representation learning does not generalize strongly within the same domain. In Internati...

    2022

  85. [93]

    Laion-5b: An open large-scale dataset for training next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. Laion...

    2022

  86. [94]

    The pitfalls of simplicity bias in neural networks

    Harshay Shah, Kaustav Tamuly, Aditi Raghunathan, Prateek Jain, and Praneeth Netrapalli. The pitfalls of simplicity bias in neural networks. In Advances in Neural Information Processing Systems (NeurIPS), 2020

    2020

  87. [95]

    Shepard and Jacqueline Metzler

    Roger N. Shepard and Jacqueline Metzler. Mental rotation of three-dimensional objects. Science, 1971

    1971

  88. [96]

    Investigating the nature of 3D generalization in deep neural networks

    Shoaib Ahmed Siddiqui, David Krueger, and Thomas Breuel. Investigating the nature of 3D generalization in deep neural networks. arXiv preprint, 2023

    2023

  89. [97]

    Geometry of the loss landscape in overparameterized neural networks: Symmetries and invariances

    Berfin Simsek, Fran c ois Ged, Arthur Jacot, Francesco Spadaro, Clement Hongler, Wulfram Gerstner, and Johanni Brea. Geometry of the loss landscape in overparameterized neural networks: Symmetries and invariances. In International Conference on Machine Learning (ICML), 2021

    2021

  90. [98]

    Revisiting weakly supervised pre-training of visual perception models

    Mannat Singh, Laura Gustafson, Aaron Adcock, Vinicius De Freitas Reis, Bugra Gedik, Raj Prateek Kosaraju, Dhruv Mahajan, Ross Girshick, Piotr Dollar, and Laurens Van Der Maaten. Revisiting weakly supervised pre-training of visual perception models. In Conference on Computer Vi...

    2022

  91. [99]

    Olshausen

    Jascha Sohl-Dickstein, Ching Ming Wang, and Bruno A. Olshausen. An unsupervised algorithm for learning Lie group transformations. arXiv preprint, 2017

    2017

  92. [100]

    Neural representational geometry underlies few-shot concept learning

    Ben Sorscher, Surya Ganguli, and Haim Sompolinsky. Neural representational geometry underlies few-shot concept learning. Proceedings of the National Academy of Sciences (PNAS), 2022

    2022

  93. [101]

    Stanley and Risto Miikkulainen

    Kenneth O. Stanley and Risto Miikkulainen. Evolving neural networks through augmenting topologies. Evolutionary Computation, 2002

    2002

  94. [102]

    Revisiting unreasonable effectiveness of data in deep learning era

    Chen Sun, Abhinav Shrivastava, Saurabh Singh, and Abhinav Gupta. Revisiting unreasonable effectiveness of data in deep learning era. In International Conference on Computer Vision (ICCV), 2017

    2017

  95. [103]

    Symmetry perception by deep networks: Inadequacy of feed-forward architectures and improvements with recurrent connections

    Shobhita Sundaram, Darius Sinha, Matthew Groth, Tomotake Sasaki, and Xavier Boix. Symmetry perception by deep networks: Inadequacy of feed-forward architectures and improvements with recurrent connections. arXiv preprint, 2022

    2022

  96. [104]

    Noether s learning dynamics: Role of symmetry breaking in neural networks

    Hidenori Tanaka and Daniel Kunin. Noether s learning dynamics: Role of symmetry breaking in neural networks. In Advances in Neural Information Processing Systems (NeurIPS), 2021

    2021

  97. [105]

    Probing equivariance and symmetry breaking in convolutional networks

    Sharvaree Vadgama, Mohammad Mohaiminul Islam, Domas Buracas, Christian Shewmake, Artem Moskalev, and Erik Bekkers. Probing equivariance and symmetry breaking in convolutional networks. arXiv preprint, 2025

    2025

  98. [106]

    Camargo, and Ard A

    Guillermo Valle-Perez, Chico Q. Camargo, and Ard A. Louis. Deep learning generalizes because the parameter-function map is biased towards simple functions. In International Conference on Learning Representations (ICLR), 2019

    2019

  99. [107]

    van der Linden, Alejandro Garc\' a-Castellanos, Sharvaree Vadgama, Thijs P

    Putri A. van der Linden, Alejandro Garc\' a-Castellanos, Sharvaree Vadgama, Thijs P. Kuipers, and Erik J. Bekkers. Learning symmetries via weight-sharing with doubly stochastic tensors. In Advances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  100. [108]

    Tycho F. A. van der Ouderaa, Alexander Immer, and Mark van der Wilk. Learning layer-wise equivariances automatically using gradients. In Advances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  101. [109]

    Tycho F. A. van der Ouderaa, Mark van der Wilk, and Pim de Haan. Noether's razor: Learning conserved quantities. In Advances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  102. [110]

    van der Ouderaa and Mark van der Wilk

    Tycho F.A. van der Ouderaa and Mark van der Wilk. Learning invariant weights in neural networks. In Conference on Uncertainty in Artificial Intelligence, 2022

    2022

  103. [111]

    Learning invariances using the marginal likelihood

    Mark van der Wilk, Matthias Bauer, ST John, and James Hensman. Learning invariances using the marginal likelihood. In Advances in Neural Information Processing Systems (NeurIPS), 2018

    2018

  104. [112]

    Approximately equivariant networks for imperfectly symmetric dynamics

    Rui Wang, Robin Walters, and Rose Yu. Approximately equivariant networks for imperfectly symmetric dynamics. In International Conference on Machine Learning (ICML), 2022

    2022

  105. [113]

    Coordinate independent convolutional networks -- isometry and gauge equivariant convolutions on riemannian manifolds

    Maurice Weiler, Patrick Forré, Erik Verlinde, and Max Welling. Coordinate independent convolutional networks -- isometry and gauge equivariant convolutions on riemannian manifolds. arXiv preprint, 2021

    2021

  106. [114]

    Compositional generalization from first principles

    Thadd\" a us Wiedemer, Prasanna Mayilvahanan, Matthias Bethge, and Wieland Brendel. Compositional generalization from first principles. In Advances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  107. [115]

    Justin N. Wood. Newborn chickens generate invariant object representations at the onset of visual object experience. Proceedings of the National Academy of Sciences (PNAS), 2013

    2013

  108. [116]

    Worrall, Stephan J

    Daniel E. Worrall, Stephan J. Garbin, Daniyar Turmukhambetov, and Gabriel J. Brostow. Harmonic networks: Deep translation and rotation equivariance. In Conference on Computer Vision and Pattern Recognition (CVPR), 2017

    2017

  109. [117]

    Greg Yang and Edward J. Hu. Tensor programs IV : Feature learning in infinite-width neural networks. In International Conference on Machine Learning (ICML), 2021

    2021

  110. [118]

    Tensor programs VI : Feature learning in infinite depth neural networks

    Greg Yang, Dingli Yu, Chen Zhu, and Soufiane Hayou. Tensor programs VI : Feature learning in infinite depth neural networks. In International Conference on Learning Representations (ICLR), 2024 a

    2024

  111. [119]

    Latent space symmetry discovery

    Jianke Yang, Nima Dehmamy, Robin Walters, and Rose Yu. Latent space symmetry discovery. In International Conference on Machine Learning (ICML), 2024 b

    2024

  112. [120]

    Dynamics of concept learning and compositional generalization

    Yongyi Yang, Core Francisco Park, Ekdeep Singh Lubana, Maya Okawa, Wei Hu, and Hidenori Tanaka. Dynamics of concept learning and compositional generalization. In International Conference on Learning Representations (ICLR), 2025

    2025

  113. [121]

    Yeh, Yuan-Ting Hu, Mark Hasegawa-Johnson, and Alexander Schwing

    Raymond A. Yeh, Yuan-Ting Hu, Mark Hasegawa-Johnson, and Alexander Schwing. Equivariance discovery by learned parameter-sharing. In International Conference on Artificial Intelligence and Statistics (AISTATS), 2022

    2022

  114. [122]

    Barlow Twins : Self -supervised learning via redundancy reduction

    Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stephane Deny. Barlow Twins : Self -supervised learning via redundancy reduction. In International Conference on Machine Learning (ICML), 2021

    2021

  115. [123]

    Discovering viewpoint-invariant relationships that characterize objects

    Richard Zemel and Geoffrey E Hinton. Discovering viewpoint-invariant relationships that characterize objects. In Advances in Neural Information Processing Systems (NeurIPS), 1990

    1990

  116. [124]

    Understanding deep learning (still) requires rethinking generalization

    Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning (still) requires rethinking generalization. Commun. ACM, 2021

    2021

  117. [125]

    Meta-learning symmetries by reparameterization

    Allan Zhou, Tom Knowles, and Chelsea Finn. Meta-learning symmetries by reparameterization. In International Conference on Learning Representations (ICLR), 2021

    2021

  118. [126]

    Neural networks fail to learn periodic functions and how to fix it

    Liu Ziyin, Tilman Hartwig, and Masahito Ueda. Neural networks fail to learn periodic functions and how to fix it. In Advances in Neural Information Processing Systems (NeurIPS), 2020

    2020

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.