{"id":"c49dac5b-78db-4979-ac74-5442d3e715b0","arxiv_id":"2606.23044","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.5,"correctness_risk":"low","formal_verification":"none","parameter_count":3,"one_line_summary":"PFE makes modular arithmetic reduce to prime-channel selection; equivariant linear maps are block-diagonal by Schur, and CRT-predicted channels show >500× ablation specialization with perfect in-distribution accuracy.","lead":"Prime Fourier Embeddings encode integers as prime-indexed sine-cosine pairs so modular arithmetic becomes selecting the right prime channels. This offers a mathematically grounded way to reduce grokking and make arithmetic structure readable inside neural nets.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"The load-bearing gap is that Theorem 3.1 constrains only linear equivariant maps, while the trained model is a non-linear architecture whose routing is never shown to be forced by the theorem.","rationale":"The reader correctly isolates the weakest assumption: that GD on the non-linear architecture will approximately respect the linear equivariance assumed by Theorem 3.1. That gap is load-bearing because the paper’s central narrative is that the adelic/character construction plus Schur’s lemma supplies a principled, pre-structured basis whose block-diagonal property is then confirmed by ablation. If residual cross-prime coupling remains large after training, the ablations still show useful specialization but the theoretical prediction no longer accounts for it. The concrete finite-difference test above would settle the issue with a single post-training measurement and does not require new training runs. All other limitations listed by the reader (single seed, square-free only, missing baselines) are real but secondary; they do not undermine the logical link between theorem and experiment as directly as the linear-to-non-linear gap does. Hence the verdict remains CONDITIONAL and no stronger adjustment is warranted.","tokens_in":14775,"tokens_out":594,"duration_ms":5865,"concrete_test":"After training the full non-linear model on a two-factor composite (e.g., N=21), extract the effective linear map from the concatenated PFE input to the pre-softmax logits (or to the first linear layer of the classifier) by finite differences or by freezing all ReLUs to identity; measure the Frobenius norms of the off-block (cross-prime) submatrices. If those norms are not near zero relative to the on-block norms, the observed specialization is not explained by Theorem 3.1.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Theorem 3.1 states that any linear map W equivariant under the product action of the Z/p^{d+1}Z groups on the PFE blocks must be block-diagonal (one independent block per (p,d) pair). The empirical claims, however, rest on a concrete non-linear architecture: a shared per-prime MLP encoder (24\to64\to32, ReLU) followed by a two-layer classifier that concatenates the row representations. Remark 3.3 explicitly leaves open why gradient descent should converge to an equivariant solution. Consequently the large specialization ratios and perfect in-distribution accuracy demonstrate that the network can discover the predicted routing, but they do not establish that the representation-theoretic constraint is what forces that routing. Without a measurement of residual cross-prime linear coupling (or an ablation that freezes the encoder to pure linear maps), the bridge from Schur’s lemma to the observed channel specialization remains an empirical correlation rather than a consequence of the theorem.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper introduces Prime Fourier Embeddings (PFE), which map integers to prime-indexed (cos, sin) pairs at successive p-adic digit depths, motivated by the adelic character factorization of Q. Theorem 3.1 proves that any linear map equivariant under the product action of the groups Z/p^{d+1}Z on these blocks must be block-diagonal, with one independent block per (p,d) pair and zero cross-prime (and cross-depth) coupling, via Schur’s lemma applied to non-isomorphic characters of Z. For square-free composite moduli the Chinese Remainder Theorem further predicts which prime channels are task-relevant. Empirically, a fixed PFE front-end plus a shared per-prime MLP encoder and a mixing classifier, trained on (a+b) mod N, yields specialization ratios exceeding 500× under row ablation and perfect in-distribution test accuracy across the square-free composites tested.","tokens_in":14965,"tokens_out":1517,"duration_ms":25687,"significance":"If the framing holds, the work is a clean example of embedding design guided by representation theory and number theory rather than post-hoc interpretability: the block-diagonal constraint is stated before training, the CRT channel list is an external prediction, and both are checked by systematic ablations over many prime counts, moduli, and input ranges. The perfect in-distribution accuracy and extreme specialization ratios are strong empirical signals that pre-structuring modular arithmetic as prime-local selection can remove the usual representational bottleneck. The explicit use of Schur’s lemma, CRT, and adelic factorization, together with a large configuration sweep, are genuine strengths. The main scientific value is therefore as a falsifiable, theory-first account of when modular arithmetic becomes easy for a network—not as a generic claim that Schur’s lemma forces nonlinear training dynamics.","major_comments":[{"comment":"Theorem 3.1 constrains only linear equivariant maps W on the PFE blocks. The trained model is nonlinear: a shared per-prime encoder (24→64→32, ReLU) that already processes each prime row independently, followed by a classifier on the concatenation. Remark 3.3 correctly flags that gradient descent is not guaranteed to preserve equivariance, yet the abstract and §5 present the ablation ratios as confirmation of the theorem’s structural prediction. Without a residual cross-prime linear-coupling measurement, a purely linear equivariant baseline, or an ablation that freezes the encoder to linear maps, the bridge from Schur’s lemma to observed routing remains an empirical correlation. Please either (i) measure residual cross-prime coupling in the trained classifier weights, (ii) report a linear-only control, or (iii) reframe the empirical claims as confirmation of the CRT channel-selection pre","section":null},{"comment":"§4 Experimental setup: the encoder is defined to process each prime row independently before any mixing occurs. That architectural choice already enforces the between-prime independence that Theorem 3.1 derives for linear intertwiners. The load-bearing empirical content is therefore almost entirely the CRT selection result (which primes the classifier uses), not the discovery of block-diagonal structure. The manuscript should state this separation explicitly in §3–§5 and adjust the claim language (“both predictions are confirmed”) so that readers do not attribute the encoder’s independence to the representation-theoretic theorem.","section":null},{"comment":"§4: all runs use a single fixed seed (42) with no error bars; variance is asserted to be “negligible” without quantification. Specialization ratios are capped at 500× precisely when off-diagonal drops approach zero, so the headline “exceeding 500×” is sensitive to numerical noise in the denominator. At minimum, report multi-seed means and standard deviations for a representative subset of configurations (e.g., one two-factor and one three-factor modulus at r∈{500,4000}), or replace the capped ratio with a more stable statistic (e.g., absolute factor vs nonfactor drops with confidence intervals).","section":null},{"comment":"The paper’s central practical claim is that PFE turns modular arithmetic from a discovery problem into a selection problem (§5). That claim would be much stronger with head-to-head comparisons against the embeddings the introduction criticizes (FoNE base-10, xVal, learned embeddings) on the same (a+b) mod N tasks, measuring both final accuracy and sample/epoch efficiency or grokking delay. Conceptual discussion of misalignment is present; quantitative baselines are not. Without them, “perfect accuracy with PFE” is hard to calibrate against the status quo the paper aims to improve.","section":null}],"minor_comments":[{"comment":"Figures 6–9 captions repeatedly misspell “Experiment” as “Experiemnt”; fix throughout.","section":null},{"comment":"§2 Parameter choices: “excluding 2¹” is typeset awkwardly; state clearly that p=2 is excluded because sin(2πa/2)≡0 (as in footnote 1).","section":null},{"comment":"Theorem 3.1 proof: the argument that χ_{p,d} and χ_{q,d′} are non-isomorphic characters of the common group Z is correct, but a one-line reminder that the real 2D blocks are the real forms of these complex characters (and that Schur still forces the real intertwiners to vanish) would help readers who work only over R.","section":null},{"comment":"Remark 3.2 usefully distinguishes strong (between-prime) from weaker (within-prime depth) independence; consider elevating a short version of this distinction into the main theorem statement or abstract so the CRT prediction is not over-read as applying to depth levels.","section":null},{"comment":"Appendix B figures (10–21) are referenced but not described in the main text beyond “additional experimental figures”; a one-sentence pointer in §4 to what the per-prime profiles show would improve navigability.","section":null},{"comment":"Related work: FoNE is cited as Zhou et al., 2026 and Simon et al., 2026 appears in the impact statement; ensure arXiv/venue identifiers are stable at camera-ready time.","section":null}],"recommendation":"major_revision","confidential_remarks":"The algebraic core (characters of Z, Schur, CRT) is standard and correctly applied; I do not see an internal inconsistency. The main risk for the journal is overclaiming: the architecture already implements prime-local encoders, so the most novel empirical result is CRT channel selection, not a demonstration that Schur’s lemma forces nonlinear GD. If the authors tighten the theorem–experiment bridge and add multi-seed stats plus at least one baseline comparison, this is a solid contribution to theory-guided embedding design. Scope fit for a general ML venue is good; pure number-theory venues would find the experiments thin."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful core is a concrete embedding (PFE) that puts each prime into its own (cos, sin) channels, a short Schur argument that any linear equivariant map on those channels must be block-diagonal, and a CRT prediction of exactly which channels matter for square-free N. The ablations then show specialization ratios above 500x and perfect in-distribution accuracy on every square-free composite they tried. That combination is new relative to FoNE and the usual trigonometric embeddings; the adelic motivation is mostly packaging, but the prime indexing itself is the right design choice for modular tasks.\n\nThe algebra is standard and correctly applied: characters of Z, non-isomorphism across primes or depths, Schur forces the off-blocks to zero. The CRT channel list is classical number theory, not a free parameter. Empirically the heatmaps are consistent across the 35 + 50 configurations; once the input range is large enough the off-diagonal drops sit near zero. Single seed and no error bars are real but secondary given how large and stable the ratios are.\n\nThe stress-test concern is accurate and already flagged by the authors (Remark 3.3). Theorem 3.1 only constrains linear equivariant maps; the actual model is a shared per-prime MLP plus classifier. The experiments show the network can discover the predicted routing; they do not prove the theorem forces it. That is a genuine gap between theory and practice, not a fatal flaw. Other soft spots are minor: square-free moduli only, no head-to-head against FoNE on the same specialization metric, depth-within-prime still under-analyzed.\n\nThis is for people who care about numerical embeddings, grokking, or mechanistic interpretability of arithmetic. The math is solid, the data match the pre-stated prediction, and the citation pattern is appropriate. I would send it to referees; they will ask for the linear-vs-nonlinear bridge and a couple of baselines, but the paper already earns that conversation. Worth reading and worth citing if you work in the area.","headline":"Clean prime-channel embedding with a correct Schur block-diagonal theorem and strong CRT ablations; the open gap is only that the theorem constrains linear equivariant maps while the network is nonlinear.","tokens_in":15612,"tokens_out":519,"would_cite":true,"duration_ms":4784,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","11A07","20C15"],"pacs":[],"model":"grok-4.5","headline":"Prime Fourier Embeddings encode integers so modular arithmetic reduces to selecting the relevant prime channels rather than rediscovering algebraic structure.","keywords":["Prime Fourier Embeddings","modular arithmetic","Chinese Remainder Theorem","Schur's lemma","equivariant linear maps","character decomposition","neural number embeddings","block-diagonal structure"],"falsifier":"Train the same architecture on a square-free modulus at large input range and observe that ablating a non-factor prime drops accuracy comparably to ablating a true factor prime, or that the factor/nonfactor specialisation ratio stays near 1 instead of climbing past hundreds.","tokens_in":15605,"feed_emoji":"🔢","tokens_out":872,"duration_ms":17305,"temperature":0.7,"pith_summary":"Standard neural embeddings hide the prime-factor structure that modular arithmetic actually uses. This paper builds Prime Fourier Embeddings that place each prime’s periodic residue into its own (cos, sin) channels, taken from the harmonic analysis of the rationals. With that structure present from the start, any linear map that respects the natural group action must stay block-diagonal by prime, so the network’s job collapses to routing through the channels that divide the modulus. Ablations confirm the prediction: task-relevant primes produce accuracy drops hundreds of times larger than irrelevant ones, and every square-free composite tested reaches perfect in-distribution accuracy. A sympathetic reader cares because the work shows that the right mathematical embedding can turn a hard discovery problem into a simple selection problem.","feed_headline":"Prime embeddings turn modular arithmetic into channel selection","feed_subtitle":"Equivariant maps must stay block-diagonal by prime; ablations show 500× specialisation and perfect accuracy.","key_machinery":"The Block-Diagonal Decomposition Theorem (Theorem 3.1): each PFE block realises a distinct character of Z, so Schur’s lemma forces every equivariant linear map to have zero cross-prime (and cross-depth) coupling.","core_discovery":"Any linear map equivariant under the product group action on Prime Fourier Embeddings must be block-diagonal with one independent block per prime and per depth; this follows from Schur’s lemma applied to the non-isomorphic characters carried by each prime-indexed (cos, sin) block. For square-free composite moduli the Chinese Remainder Theorem further identifies exactly which prime blocks are task-relevant. Both structural predictions are confirmed by ablation: specialization ratios exceed 500\times and every tested square-free modulus reaches perfect test accuracy.","pith_inferences":["The same prime-channel construction may reduce grokking delay on other group-structured algorithmic tasks that currently force the network to invent the decomposition.","How gradient descent actually distributes weight across depth levels inside a single prime remains open; measuring that distribution would test whether deeper blocks simply subsume shallower ones.","Extending the basis to include the prime 2 (or p-adic characters of higher conductor) would show whether the block-diagonal guarantee survives the degenerate sine feature the authors currently exclude."],"forward_implications":["Modular arithmetic tasks become channel-selection problems once the embedding already isolates prime-local residues.","Targeted row ablations can verify or falsify the predicted block structure before any architectural changes are made.","Embedding design for arithmetic can be guided by adelic character factorisation rather than by base-10 or other heuristic frequencies.","Perfect in-distribution accuracy on square-free moduli is expected once the relevant prime channels are present and selected."],"fun_headline_variants":["Prime Fourier Embeddings turn modular ops into channel selection","Schur forces equivariant PFE maps to be prime-block-diagonal","CRT identifies task-relevant primes for composite modular PFE","Ablations show 500x prime-channel specialization in modular tasks","PFE pre-structures integers so modular arithmetic picks prime channels"],"cache_read_input_tokens":128,"weakest_assumption_plain":"Gradient descent on the actual non-linear network will still converge to solutions that approximately obey the linear equivariance the theorem assumes.","fun_headline_variants_meta":{"raw":{"variants":["Prime Fourier Embeddings turn modular ops into channel selection","Schur forces equivariant PFE maps to be prime-block-diagonal","CRT identifies task-relevant primes for composite modular PFE","Ablations show 500x prime-channel specialization in modular tasks","PFE pre-structures integers so modular arithmetic picks prime channels"]},"model":"grok-4.5","effort":"low","cost_usd":0.005446,"raw_usage":{"total_tokens":1457,"prompt_tokens":728,"num_sources_used":0,"completion_tokens":90,"cost_in_usd_ticks":54460000,"prompt_tokens_details":{"text_tokens":728,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":639,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":728,"tokens_out":90,"duration_ms":5528,"temperature":1.0,"reasoning_tokens":639,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-15T10:35:30.382365+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Train the same architecture on a square-free modulus at large input range and observe that ablating a non-factor prime drops accuracy comparably to ablating a true factor prime, or that the factor/nonfactor specialisation ratio stays near 1 instead of climbing past hundreds.","supporting_citations":[],"review_version":2}