REVIEW 2 major objections 5 minor 26 references
A Group Theoretic Analysis of the Symmetries Underlying Base Addition and Their Learnability by Neural Networks
T0 review · 2 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read A tiny recurrent network can learn to add arbitrarily long numbers when the carry function is chosen and presented to expose its symmetry.
desk verdict A real empirical bridge between carry-function structure and RNN learnability, but the headline correlations rest on a depth-4 truncation that needs a convergence check. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the carry function $f$ on pairs of digits in $Z_b$, viewed as a 2-cocycle in the group extension from $Z_b$ to $Z_{b^2}$; the k-digit analogue is the depth-$k$ carry table $F_k$, whose entry says what is carried from digit $k$ to digit $k+1$ for every pair of $k$-digit numbers. Choosing a different section of the extension changes $f$ by a coboundary, which yields the alternative carry functions; a carry is $k$-equivariant when the resulting $k$-digit addition remains associative, meaning the cocycle condition survives to that depth. The paper's structural measures are computed on these tables: box-counting dimension of the border between distinct carried values, frequency of non-zero carries, and associativity fraction over sampled triplets. For training, the equivalent machinery is the interleaved problem format, which presents digits least-to-most significant so the recurrent network must store and carry a scalar value across time steps, making the carry symmetry accessible through weight sharing over time.
What would settle it
Recompute the three structural measures at depths five, six, or higher; if their rank order across carry functions changes substantially, or if the Spearman correlations with maximum test accuracy on 6-digit (or longer) numbers drop toward zero or reverse, the paper's claim that these statistics capture learning-relevant complexity would be falsified.
Extended reading notes
Core claim
The central claim is that even a tiny neural network can achieve radical generalization in addition, provided the input format exposes the per-place carry operation and the carry function itself has the right symmetry structure. Concretely, the paper argues that a carry function for base $b$ is a normalized 2-cocycle of $Z_b$ with coefficients in $Z_b$; two carry functions are equivalent when they differ by a coboundary, so the set of valid carries for two-digit addition has size $b^{b-2}$, of which $\varphi(b)$ are Single Value carries (carrying a fixed unit) and the rest are Multiple Value carries. The paper further claims that these classes separate by structure: at depth four, Single Value carry tables have box-counting dimension near 1, Multiple Value tables stay above 1.25, and a subset of Low Dimensional Multiple Value tables keep full associativity to depth 4 or 5. Training a one-layer GRU and LSTM on all 3-digit pairs with an interleaved least-to-most-significant format, the paper reports near-perfect accuracy for Single Value carries on 6-digit numbers and high accuracy through 10 digits, with maximum 6-digit test accuracy across bases 3 to 5 ranked by fractal dimension (Spearman -0.872), carry frequency (-0.656), and associativity fraction (0.887). The discovery is the alignment between cohomological complexity measures and learnability.
Load-bearing premise
The structural measures are computed on carry tables only four digits deep, and the paper assumes that depth-4 values rank the carry functions in the same order as the true complexity a network faces when generalizing from three to six or ten digits.
Editorial extensions
If this is right
- A one-layer GRU trained only on 3-digit pairs can add 6-digit numbers almost perfectly and 10-digit numbers with high accuracy when the carry is Single Value, showing that radical generalization does not require a large model.
- Across bases 3 to 5, maximum 6-digit test accuracy is rank-correlated with carry-table structure: fractal dimension and carry frequency hurt, associativity helps.
- Because the standard 1-carry is the simplest Single Value carry and aligns with ordinal digit order, the paper explains the universal human choice of carry algorithm as a learnability optimum.
- The interleaved digit-by-digit input format is a critical inductive bias: it reduces addition to modular addition plus a remembered carry, and without it the task requires external memory or ordering machinery.
Reading between the lines
- A testable extension: if the depth-4 measures are proxies for asymptotic structure, then training curricula that expose the network to progressively longer numbers should yield the same rank ordering of carry functions, whereas a curriculum that fixes depth 3 may not.
- The group cohomology picture likely generalizes to other recursive symbolic operations, such as multiplication by a fixed modulus or carry in mixed-radix systems, where the load-bearing structure is again a cocycle condition; learning difficulty there may also be predictable from the same structural measures.
- The results suggest one reason large language models struggle with arithmetic in standard notation: the format hides the carry symmetry behind token ordering, and an explicit scratchpad or right-aligned interleaved encoding might give even small models algorithmic addition.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper formalizes base addition through group cohomology, treating a carry function as a 2-cocycle and studying the resulting group extensions of Z_b by Z_b. It defines Single Value and Multiple Value carry functions, introduces depth-k carry tables, and characterizes carry functions by three quantitative measures: the box-counting dimension of the table border, the frequency of carrying, and the associativity fraction, computed at depths 1-4 for bases 3-5. It then trains a 1-layer GRU (and an LSTM, in the appendix) on 3-digit addition in an interleaved digit-pair format, with one-hot or Gaussian-smoothed digit embeddings, for all carry functions in bases 3-5, and tests on 6-10 digit numbers. The main results are that Single Value carry functions are learned most effectively and generalize far out of distribution, and that maximum 6-digit test accuracy is rank-correlated with the structural measures (Spearman rho = -0.872, -0.656, 0.887 across bases 3-5). The discussion connects these findings to human arithmetic and to the universality of the 1-carry function.
Significance. If the result holds, the paper provides a clean demonstration that a very small recurrent network can discover a carry symmetry and generalize radically when the input format is interleaved and the carry function has sufficient structure. The group-theoretic framing is elegant, the experimental design is transparent, the code is openly available, and the structural measures are computed independently of the network, so the reported correlations are genuine empirical findings rather than constructions. The main limitations are that the quantitative measures are shown to separate the carry-function classes only at depth 4, and that the correlational analysis pools across bases that differ systematically in training-example count; both issues are fixable and do not undermine the core demonstration of radical generalization for Single Value carries.
major comments (2)
- [§3.2–3.4, Fig. 3; §4.2.3, Fig. 7] The correlational analysis uses the depth-4 values of fractal dimension, carry frequency, and associativity fraction as the operative complexity for learning to generalize from 3-digit training to 6-digit testing. For carry functions that are only finitely equivariant—and Section 5.1 explicitly leaves open which Multiple Value functions are ∞-equivariant—iterating Equation 2 to depth 4 is not a sample of a convergent asymptotic object. The paper provides no evidence that the rank ordering of carry functions by these measures is stable across depths; if the depth-5 or depth-6 ranking differs, the reported Spearman correlations (-0.872, -0.656, 0.887) may be an artifact of the chosen truncation. The authors should report the three measures at deeper depths (or at least demonstrate rank-order stability across k = 1 to 6 for the available bases) and recompute the correlations at each depth, ideally including the depths actually tested by the network.
- [§4.2.2–4.2.3, Fig. 6–7] The headline correlations pool all carry functions from bases 3, 4, and 5 into a single Spearman rank correlation. Section 4.2.2 (Fig. 6) shows that test accuracy increases systematically with base, and Section 5.2.2 attributes this at least in part to the larger number of training examples at larger bases. Since base may influence both the structural measures and learning, the pooled correlations could be inflated by a base-level confound. The authors should report the correlations separately within each base, or compute partial Spearman correlations controlling for base, and should separate between-class from within-class contributions to the correlations.
minor comments (5)
- [§3.4] The associativity fraction is said to be based on sampled triplets, but no sample size, number of trials, or confidence intervals are reported; this matters for the precision of claims such as an associativity fraction of exactly 1 up to depth 4.
- [§3.2] The border definition as entries with different values from their left and upper neighbors does not state how the first row and first column are treated; please clarify the boundary convention.
- [§4.2.1] The Gaussian convolution envelope used for semantic embeddings is described by example only; the standard deviation, normalization, and circular padding details should be specified for reproducibility.
- [Appendix C.2] Roughly 15% of learning curves could not be fit by the sigmoid and are excluded; the paper should state whether there is a systematic pattern among the excluded curves (for example, concentration in the Other Multiple Value class), since this could bias the robustness analysis.
- [§2.4.2 and Fig. 3] The Low Dimensional Multiple Value category is defined post hoc from the same quantitative measures used in Figure 7, so the visual separation into three groups in Figure 7 is partly by construction; the paper should state this explicitly and emphasize that the non-circular evidence is the continuous correlation with learning and the generalization gap in Figure 6.
Circularity Check
No significant circularity: structural measures are computed independently of network training; only the Low-Dimensional-Multiple-Value label is mildly self-referential, and the headline correlations do not depend on it.
-
self definitional
[Section 2.4.2 (definition); Section 4.2.2 and Figure 7 (same groupings)]
"In Section 3) we observe that there is a subset of Multiple Value carry functions that are notably lower in complexity than the remainder. We refer to these as Low Dimensional Multiple Value carry functions, that we characterize in Section 3)"
The category 'Low Dimensional Multiple Value' is introduced as the subset singled out by the same structural measures (fractal dimension, carry frequency, associativity) that Section 4 later reports as separating the learning curves. So the qualitative observation that learning separates 'into the same three groupings' is partly guided by the way the labels were assigned from those measures. However, the central quantitative claim is the Spearman correlation between continuous measure values and maximum test accuracy across all carry functions, which does not rely on the category labels and is therefore an independent empirical finding.
full rationale
The paper's derivation chain is largely self-contained and not circular. The group-cohomological construction of carry functions is derived in Appendix A.2 from standard results (Isaksen 2002; Brown 2012), and the three structural measures are computed directly from carry tables at depths 1-4, with no input from the neural network training or test results. The networks are trained only on digit-pair inputs and carry-specific answer sequences; they never receive the fractal dimension, carry frequency, or associativity fraction. Thus the strong correlations in Section 4.2.3 (-0.872, -0.656, 0.887) are genuine empirical relationships rather than quantities forced by construction. The self-citations (Segert 2024; Segert & Cohen 2022) are not load-bearing: the mathematical extension is re-derived in the appendix, and the network simulations are new and independently coded. The only mild self-referential element is the 'Low Dimensional Multiple Value' label being defined by thresholds on the same measures that later appear in the learning-curve groupings; that labeling is descriptive and does not drive the headline correlations. The depth-4 truncation concern raised in the reader's take is a legitimate external-validity risk about whether the measures have converged, but it is not circularity: even if depth-4 rankings were unstable, the correlations would be a real property of the depth-4 measures, not a construction artifact. Overall, the central claim has independent empirical content, so the circularity score is low.
Assumptions & free parameters
free parameters (4)
- Gaussian convolution envelope for semantic embeddings =
e.g., digit 1 in base 5 becomes (0.2, 0.5, 0.2, 0.05, 0.05)
- Truncation depth for fractal dimension estimation =
k = 1 to 4
- Thresholds defining Low Dimensional Multiple Value class =
fractal dimension in (1.25, 1.5), associativity fraction 1 up to depth 4, lower carry frequency
- Sigmoid fit parameters (upper asymptote, growth rate, critical point) =
not tabulated per function
assumptions (6)
- standard math Group axioms and existence of finite cyclic groups Zb
- domain assumption Isaksen's 2-digit cohomological construction of base addition is correct and extends iteratively to arbitrary digit length
- domain assumption The interleaved (column-wise, right-to-left) training format exposes the relevant symmetry and is an appropriate model of human learning
- domain assumption A one-layer GRU/LSTM with hidden dimension b can implement and learn the carry recursion given enough training
- ad hoc to paper Depth-4 estimates of the quantitative measures are representative of the asymptotic structure of carry functions
- ad hoc to paper Box-counting dimension of the carry-table border is the right measure of algorithmic complexity for learning
Cite this review
Pith. "Pith review of A Group Theoretic Analysis of the Symmetries Underlying Base Addition and Their Learnability by Neural Networks." pith.science (2026). https://pith.science/paper/E5ASE4OU
@misc{pith2026250710678,
author = {Pith},
title = {Pith review of: A Group Theoretic Analysis of the Symmetries Underlying Base Addition and Their Learnability by Neural Networks},
year = {2026},
howpublished = {\url{https://pith.science/paper/E5ASE4OU}},
note = {Machine review of arXiv:2507.10678}
}
read the original abstract
A major challenge in the use of neural networks both for modeling human cognitive function and for artificial intelligence is the design of systems with the capacity to efficiently learn functions that support radical generalization. At the roots of this is the capacity to discover and implement symmetry functions. In this paper, we investigate a paradigmatic example of radical generalization through the use of symmetry: base addition. We present a group theoretic analysis of base addition, a fundamental and defining characteristic of which is the carry function -- the transfer of the remainder, when a sum exceeds the base modulus, to the next significant place. Our analysis exposes a range of alternative carry functions for a given base, and we introduce quantitative measures to characterize these. We then exploit differences in carry functions to probe the inductive biases of neural networks in symmetry learning, by training neural networks to carry out base addition using different carries, and comparing efficacy and rate of learning as a function of their structure. We find that even simple neural networks can achieve radical generalization with the right input format and carry function, and that learnability is closely correlated with carry function structure. We then discuss the relevance this has for cognitive science and machine learning.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
author Anderson, P.W. , year 1978 . title Local moments and localized states . journal Science volume 201 , pages 307--316
work page 1978
-
[2]
author Beukers, A. , author Hamin, M. , author Norman, K.A. , author Cohen, J.D. , year 2024 . title When working memory may be just working, not memory. journal Psychological Review volume 131 , pages 563
work page 2024
-
[3]
author Brown, K. , year 2012 . title Cohomology of Groups . Graduate Texts in Mathematics, publisher Springer New York
work page 2012
-
[4]
, author Van Merri \"e nboer, B
author Cho, K. , author Van Merri \"e nboer, B. , author Gulcehre, C. , author Bahdanau, D. , author Bougares, F. , author Schwenk, H. , author Bengio, Y. , year 2014 . title Learning phrase representations using rnn encoder-decoder for statistical machine translation . journal arXiv preprint arXiv:1406.1078
arXiv 2014
-
[5]
author Cleeremans, A. , author McClelland, J.L. , year 1991 . title Learning the structure of event sequences. journal Journal of Experimental Psychology: General volume 120 , pages 235
work page 1991
-
[6]
author Dehaene, S. , year 1997 . title The Number Sense: How the Mind Creates Mathematics . publisher Oxford University Press , address New York
work page 1997
-
[7]
Your Context Is Not an Array: Unveiling Random Access Limitations in Transformers
author Ebrahimi, M. , author Panchal, S. , author Memisevic, R. , year 2024 . title Your context is not an array: Unveiling random access limitations in transformers . journal arXiv preprint arXiv:2408.05506
work page Pith review arXiv 2024
-
[8]
author Elman, J.L. , year 1990 . title Finding structure in time . journal Cognitive Science volume 14 , pages 179--211 . :https://doi.org/10.1207/s15516709cog1402\_1
Show all 26 references
-
[9]
, year 2014
author Falconer, K. , year 2014 . title Fractal Geometry: Mathematical Foundations and Applications . publisher John Wiley & Sons
2014
-
[10]
, author Finzi, M
author Goldblum, M. , author Finzi, M. , author Rowan, K. , author Wilson, A.G. , year 2023 . title The no free lunch theorem, kolmogorov complexity, and the role of inductive biases in machine learning . journal arXiv preprint arXiv:2304.05366
2023 arXiv
-
[11]
, author Schmidhuber, J
author Hochreiter, S. , author Schmidhuber, J. , year 1997 . title Long Short-Term Memory . journal Neural Computation volume 9 , pages 1735--1780 . :10.1162/neco.1997.9.8.1735, http://arxiv.org/abs/https://direct.mit.edu/neco/article-pdf/9/8/1735/813796/neco.1997.9.8.1735.pdf...
1997 doi
-
[12]
, author Kahana, M.J
author Howard, M.W. , author Kahana, M.J. , year 2002 . title A distributed representation of temporal context . journal Journal of mathematical psychology volume 46 , pages 269--299
2002
-
[13]
, year 2002
author Isaksen, D. , year 2002 . title A cohomological viewpoint on elementary school arithmetic . journal The American Mathematical Monthly
2002
-
[14]
, author Benna, M.K
author Ji-An, L. , author Benna, M.K. , author Mattar, M.G. , year 2024 . title Discovering cognitive strategies with tiny recurrent neural networks . journal bioRxiv https://www.biorxiv.org/content/early/2024/10/05/2023.04.12.536629, :10.1101/2023.04.12.536629, http://arxiv.o...
2024 doi
-
[15]
, author Sutskever, I
author Krizhevsky, A. , author Sutskever, I. , author Hinton, G.E. , year 2012 . title Imagenet classification with deep convolutional neural networks , in: booktitle Advances in Neural Information Processing Systems , publisher Curran Associates, Inc
2012
-
[16]
, author Bottou, L
author Lecun, Y. , author Bottou, L. , author Bengio, Y. , author Haffner, P. , year 1998 . title Gradient-based learning applied to document recognition . journal Proceedings of the IEEE volume 86 , pages 2278--2324 . :10.1109/5.726791
1998 doi
-
[17]
, author Poiitis, M
author Loukas, A. , author Poiitis, M. , author Jegelka, S. , year 2021 . title What training reveals about neural network complexity . journal Advances in Neural Information Processing Systems volume 34 , pages 494--508
2021
-
[18]
, author Jiang, Z
author Nogueira, R. , author Jiang, Z. , author Lin, J. , year 2021 . title Investigating the limitations of transformers with simple arithmetic tasks . journal arXiv preprint arXiv:2102.13019
2021 arXiv
-
[19]
, year 2023
author Piantadosi, S.T. , year 2023 . title The algorithmic origins of counting . journal Child Development volume 94 , pages 1472--1490
2023
-
[20]
, author Wang, H
author Qian, J. , author Wang, H. , author Li, Z. , author Li, S. , author Yan, X. , year 2022 . title Limitations of language models in arithmetic and symbolic induction . journal arXiv preprint arXiv:2208.05051
2022 arXiv
-
[21]
, year 2024
author Segert, S. , year 2024 . title Maximum Entropy, Symmetry, and Relational Bottleneck: Unraveling the Impact of Inductive Biases on Systematic Reasoning . Ph.D. thesis. Princeton University
2024
-
[22]
, author Cohen, J
author Segert, S. , author Cohen, J. , year 2022 . title A self-supervised framework for function learning and extrapolation . journal Transactions on Machine Learning Research https://openreview.net/forum?id=ILPFasEaHA
2022
-
[23]
, author Holyoak, K.J
author Webb, T. , author Holyoak, K.J. , author Lu, H. , year 2023 . title Emergent analogical reasoning in large language models . journal Nature Human Behavior
2023
-
[24]
, year 1992
author Wynn, K. , year 1992 . title Addition and subtraction by human infants . journal Nature volume 358 , pages 749--750
1992
-
[25]
, author Campbell, D
author Yang, Y. , author Campbell, D. , author Huang, K. , author Wang, M. , author Cohen, J. , author Webb, T. , year 2025 . title Emergent symbolic mechanisms support abstract reasoning in large language models . https://arxiv.org/abs/2502.20332, http://arxiv.org/abs/2502.20...
2025 arXiv
-
[26]
, author Nova, A
author Zhou, H. , author Nova, A. , author Larochelle, H. , author Courville, A. , author Neyshabur, B. , author Sedghi, H. , year 2022 . title Teaching algorithmic reasoning via in-context learning . journal arXiv preprint arXiv:2211.09066
2022 arXiv
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.