{"id":"feb31099-d467-4d3c-855d-52391d573099","arxiv_id":"2506.13523","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"The reported speedups of Gaunt and matrix tensor products over the full Clebsch-Gordan tensor product come from reduced expressivity, and the only true per-expressivity speedup comes from fast spherical harmonic transforms.","lead":"This paper compares tensor product operations used in E(3)-equivariant neural networks, showing that the faster alternatives (Gaunt and matrix tensor products) pay for their speed with reduced expressivity. It also offers a faster spherical-grid implementation of the Gaunt tensor product and benchmarks all operations on CPU and GPU.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Expressivity normalization hinges on a dimension proxy that is not a comparable capacity measure; the paper's own Tetris/Appendix H examples show equal-dimension operations can have opposite task capabilities.","rationale":"The paper is careful, the asymptotic runtime derivations appear internally consistent, and the Tetris and MACE experiments are genuine empirical evidence. The reader's conditional verdict already targets the expressivity proxy, and my reading agrees that this is the load-bearing assumption. The formal statement 'when normalized for our expressivity measure' is true by construction, but the paper's contribution is the interpretation that faster TPOs are merely discarding degrees of freedom. Definition 2.2 is explicitly a proxy, and the paper itself provides two places where the proxy fails to capture practical expressivity: Appendix H's weighted-GTP example shows a strict functional deficiency despite comparable parameter counts, and the Tetris experiment shows a total failure on a task that requires a specific interactable path. These do not invalidate the asymptotic analysis, but they mean the headline claim should be read as conditional on the proxy being task-relevant, which Section 5.4 acknowledges. I therefore do not move the verdict; I would keep CONDITIONAL and ask that the paper either validate the proxy on tasks with controlled interaction requirements or soften the framing from 'expressivity' to 'parameter-count expressivity'.","tokens_in":26926,"tokens_out":17367,"duration_ms":196942,"concrete_test":"Use the paper's Tetris architecture (Section 6.2, Appendix I) at L=1. Build a dimension-matched CGTP by keeping only the single [1,1,1] path with one scalar weight, and train it on the chiral Tetris task. If this 1-parameter CGTP reaches 100% accuracy while GTP with several channels cannot, then expressivity dimension ranks a strictly weaker operation as no less expressive. Repeat at L=2 with a dimension-matched subset of CGTP paths that includes the parity-odd paths GTP lacks, comparing GTP with enough channels to match the CGTP parameter count; if GTP still fails while the dimension-matched CGTP succeeds, the Table 1 normalization is not a valid practical proxy.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in Section 4/Table 1 — that most TPOs have the same runtime per unit expressivity and that only S2FFT gives a true speedup — depends on Definition 2.2, which identifies expressivity with the dimension of the parameterized family of bilinear maps. Dimension is then used as the normalization in runtime/expressivity. The problem is that dimension is not a commensurable measure of functional capacity: two TPOs with the same expressivity dimension need not be able to realize the same interactions, so dividing by the dimension does not make the TPOs comparable. The paper's own Appendix H demonstrates this: weighted GTP has O(L) parameters, and a CGTP restricted to O(L) output paths also has O(L) parameters, yet Appendix H shows GTP cannot realize (x^(2)⊗_CG y^(2))^(2) + (x^(4)⊗_CG y^(4))^(2) without also producing unwanted cross terms, while a same-capacity CGTP can. The Tetris experiment in Section 6.2 is an even sharper illustration: for L=1, a CGTP restricted to a single [1,1,1] path and GTP have comparable expressivity dimension, yet the one-path CGTP can create the pseudoscalar needed for chirality while GTP provably cannot. Thus 'normalized for expressivity' does not establish that GTP/MTP buy speed only by removing degrees of freedom; it shows they remove specific, task-dependent degrees of freedom, and Section 5.4 concedes this ('may not be important to the task at hand'). The asymptotic runtime analysis is sound, but the headline conclusion is only as strong as the proxy, and the paper's own examples undercut it.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper analyzes the runtime and expressivity tradeoffs of several equivariant tensor product operations (TPOs) used in E(3)-equivariant neural networks: the Clebsch-Gordan tensor product (CGTP), the Gaunt tensor product (GTP), and the matrix tensor product (MTP). It introduces two measures, expressivity (the dimension of the space of constructible bilinear maps, Definition 2.2) and interactability (which irrep triples can interact, Definition 2.3), and uses them to argue that reported asymptotic speedups of GTP and MTP over sparse CGTP come from reduced expressivity. The paper proposes a simplified grid-based implementation of GTP, which it claims is a drop-in replacement for the original Fourier-based GTP and is about 30% faster in practice, and it presents extensive microbenchmarks on GPU and CPU. It also demonstrates experimentally that GTP cannot classify chiral 3D Tetris pieces because it lacks antisymmetric interactions, while CGTP can. The main theoretical findings are summarized in Table 1: after normalizing runtime by the expressivity measure, most TPOs have O(L^2) asymptotic runtime per unit expressivity, with only a variant using fast spherical transforms (S2FFT) achieving O(L log^2 L).","tokens_in":27158,"tokens_out":5302,"duration_ms":56210,"significance":"If the claims hold, this is a useful and timely contribution to the equivariant neural network literature. The paper provides the first systematic comparison of TPOs along both asymptotic and empirical axes, and its emphasis on the distinction between a true tensor product and a TPO clarifies an important conceptual point. The asymptotic analyses in Appendix E are careful, the expressivity and interactability definitions are explicit and mathematically well posed, and the Tetris experiment gives a clean behavioral demonstration of the interactability claims. The grid-based GTP implementation is simple and seems genuinely faster in practice, and the public code release supports reproducibility. The main significance hinges on whether the expressivity proxy is accepted as the right normalization for the headline 'no true speedup' conclusion; this is the central point that needs strengthening.","major_comments":[{"comment":"The headline conclusion that 'when normalized for expressivity, most TPOs have the same asymptotics as sparse CGTP' and that GTP/MTP 'do not truly speed up tensor products but rather remove degrees of freedom' relies entirely on Definition 2.2, which measures expressivity as the dimension of the parameterized family of bilinear maps. The paper itself labels this a proxy in Section 2.3, and Section 5.4 concedes that missing interactions may be irrelevant to a task. Moreover, Appendix H and the Tetris experiment (Section 6.2) demonstrate that two TPOs with the same expressivity dimension can have opposite task-level capabilities: for example, a one-path CGTP can realize the pseudoscalar needed for the chiral Tetris task while GTP cannot, even though both have comparable expressivity dimension. As written, the normalized-runtime comparison establishes a statement about a specific parameter-count proxy, not a general statement about practical speed per unit of usable expressivity. Please either scope the conclusion explicitly to this proxy, or provide additional evidence (for example, interactability-weighted counts or a task-based calibration) that supports the stronger practical claim.","section":"Definition 2.2; §4 / Table 1"},{"comment":"The paper calls the grid GTP a 'drop-in replacement' for the Fourier GTP, but I could not find any test of numerical output equivalence. The grid construction replaces exact spherical-harmonic transforms by quadrature (Appendix E.2.2), so the two implementations are not bitwise identical and may differ for finite L and for the chosen quadrature rule. Please report the maximum pointwise or normwise deviation between grid GTP and Fourier GTP on random inputs across a range of L, or provide an error bound derived from the quadrature rule. Without this, the 30% speedup claim is not tied to a demonstrated equivalence of the operation that is being sped up.","section":"§6.1 / Appendix E.2.2"}],"minor_comments":[{"comment":"The abstract and Section 6.1 report a 30% speedup for grid GTP, but the conclusion (Section 7) says it 'perfroms 20% faster in practice'; please reconcile the two numbers.","section":"Section 7 / Abstract / Section 6.1"},{"comment":"There is a typo in Section 2.5: 'this construction is cheap aaand so we focus' should read 'and'.","section":"Section 2.5"},{"comment":"Section 5.2 contains the placeholder phrase '(some details about the hardware here)', which should be replaced with an actual hardware description or removed.","section":"Section 5.2"},{"comment":"In Appendix L.2, 'we could not colleft hardware-counters for them' should read 'collect'.","section":"Appendix L.2"},{"comment":"In the caption of Table 4, the phrase 'for output settings' is slightly ambiguous; specifying that the three settings are SISO, SIMO, and MIMO would improve readability.","section":"Table 4 / Appendix E.4"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the journal and the empirical work is substantial. My main concern is that the central 'no true speedup after normalization' claim is stated more strongly than the expressivity proxy justifies; the authors can address this by carefully scoping the claim, which I think is feasible within a revision. The drop-in replacement claim also needs a numerical equivalence check before it can be accepted as stated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper's main claim—that GTP and MTP speedups over the Clebsch-Gordan tensor product come from lost expressivity, and that normalized per expressivity most TPOs are not asymptotically faster than sparse CGTP, with only an S2FFT variant genuinely faster—is mathematically sound as a formal statement. But it rests on a specific proxy, the dimension of constructible bilinearities, which is not a task-level capacity measure. The authors are honest about this, but the headline should not be read as 'per unit of actual expressivity.'\n\nWhat is new and done well: the expressivity and interactability definitions, the normalized runtime/expressivity table, the spherical-grid GTP implementation with real speedups, and the first systematic microbenchmarks with FLOPs and GPU utilization. The asymptotic derivations in Appendix E are careful and correct. The Tetris experiment is a clean demonstration that GTP cannot produce antisymmetric interactions, and it makes the interactability concept tangible. The MACE training run is a useful independent data point. Citations to Luo et al., Unke & Maennel, Cobb et al., and Healy et al. are appropriate; the O(L^5) CGTP runtime is correctly credited to Cobb et al.\n\nSoft spots, in rough order. First, the expressivity proxy. Dimension counts parameters, not which interactions are possible. The paper's own Appendix H and the Tetris example show two operations with comparable dimension can have very different capabilities. The paper qualifies this in Section 5.4, but Table 1's 'only true speedup' is easy to over-read. If the goal is to reproduce CGTP's interaction set, the speedup mostly disappears; stating that plainly would help. Second, the grid GTP is called a drop-in replacement for the Fourier GTP, but there is no numerical check that the outputs match. Quadrature error could matter. Third, the speedup number is inconsistent: abstract says 30%, conclusion says 20%. Fourth, the GPU microbenchmarks have no error bars.\n\nNone of these are load-bearing. The central analysis holds under the paper's own definitions, the math is sound, and the limitations are acknowledged. I would send this to a serious referee; it is a useful piece of analysis for anyone working with E(3)-equivariant models, and the issues are fixable with modest additions.","headline":"GTP/MTP speedups indeed come from lost expressivity, but the expressivity measure is a parameter count rather than task capacity, so the 'only true speedup' claim needs that caveat.","tokens_in":27756,"tokens_out":6245,"would_cite":true,"duration_ms":61476,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that the reported speedups of Gaunt and matrix tensor products over the Clebsch-Gordan tensor product are purchased by discarding expressive capacity, and that once runtime is normalized by expressivity, only a fast…","keywords":["E(3)-equivariant neural networks","tensor product operations","Clebsch-Gordan tensor product","Gaunt tensor product","matrix tensor product","expressivity","interactability","spherical harmonics"],"falsifier":"The chiral Tetris classification is the sharpest test: under the paper's selection rules, no GTP network of any width or depth can construct the antisymmetric $1_o \\otimes 1_o \\to 1_e$ path needed to tell mirror-image pieces apart, so a GTP model that reaches above-chance accuracy on the chiral pair would refute the claim that the missing interactions are structurally unreachable. A complementary check measures wall-clock time of the S2FFT variant against sparse CGTP at increasing $L$; if CGTP times fail to track $O(L^5)$ while S2FFT tracks $O(L^2 \\log^2 L)$, the expressivity-normalized asymptotics of Table 1 would need to be revisited.","tokens_in":26671,"feed_emoji":"⚖️","tokens_out":19438,"duration_ms":158063,"temperature":0.7,"pith_summary":"This paper tries to establish that the reported speedups of the Gaunt tensor product and the matrix tensor product over the standard Clebsch-Gordan tensor product are not free: the faster operations are not the same operation, and their asymptotic gains come with reduced capacity to construct distinct bilinear interactions. To make that precise, the authors define the expressivity of a tensor product operation as the dimension of the space of equivariant bilinear maps obtainable by composing the operation with equivariant linear layers, and interactability as the set of irrep triples it can couple. Under this measure GTP and MTP have expressivity $O(L)$ versus $O(L^3)$ for CGTP, and once runtime is divided by expressivity every alternative has the same asymptotics as sparse CGTP, leaving the fast spherical-harmonic transform variant as the only true speedup. The paper also contributes a simpler spherical-grid implementation of GTP that is a drop-in replacement and about 30% faster in training the MACE interatomic potential, plus microbenchmarks showing that asymptotic order, FLOP counts, GPU utilization, and wall-clock time disagree enough to make application-specific benchmarking necessary. The practical stakes are that choosing a tensor product changes what a model can learn, not merely how fast it runs.","feed_headline":"Only a fast spherical transform truly speeds up tensor products","feed_subtitle":"Gaunt and matrix tensor products drop interaction paths; normalized for expressivity, Clebsch-Gordan matches them.","key_machinery":"The load-bearing object is the tensor product operation (TPO), an equivariant bilinear map $T: X' \\times Y' \\to Z'$ used in place of the mathematical tensor product, together with a definition of expressivity as the dimension of the space of equivariant bilinear maps constructible by composing $T$ with equivariant linear layers on the inputs and output. The engine of the argument is the selection rule, a condition on irrep labels that a TPO must satisfy for an interaction to be nonzero, which determines interactability. For GTP, treating irreps up to degree $L$ as spherical-harmonic coefficients turns the operation into pointwise multiplication of two functions on the sphere $S^2$ followed by spherical-harmonic decomposition, and the resulting parity condition forces $\\ell_1+\\ell_2+\\ell_3$ to be even, excluding antisymmetric paths. For MTP, embedding irreps into tensor-product representations and multiplying the resulting matrices inherits the CGTP triangle condition but still emits only one channel per output irrep. The runtime analysis leans on the sparsity of the Clebsch-Gordan coefficients, where only paths with $m_1+m_2=m_3$ survive, cutting the naive CGTP complexity from $O(L^6)$ to $O(L^5)$.","core_discovery":"The paper's central claim is that most operations called tensor products in $E(3)$-equivariant networks are equivariant bilinear maps the authors rename tensor product operations (TPOs), and that substituting a TPO for the true tensor product trades away computable functions, not just speed. Measured by the dimension of constructible bilinearities, GTP and MTP keep only $O(L)$ of the $O(L^3)$ bilinear degrees of freedom that CGTP retains, and they merge the multiple paths yielding the same output irrep into a single channel. Consequently, when runtimes are normalized by expressivity, GTP and MTP fall back to the same $O(L^2)$ scaling as sparse CGTP, and the only genuinely faster option is GTP computed through a fast spherical-harmonic transform, at $O(L^2 \\log^2 L)$ runtime and $O(L \\log^2 L)$ per unit of expressivity. The selection-rule analysis shows GTP is inherently symmetric and cannot produce antisymmetric interactions such as the cross product, and the paper demonstrates the consequence with a chiral-classification experiment in which CGTP networks reach perfect accuracy while GTP networks cannot separate mirror-image pieces. It further shows the often-quoted $O(L^6)$ runtime for CGTP is not tight, since exploiting the sparsity of the Clebsch-Gordan coefficients brings it down to $O(L^5)$.","pith_inferences":["A testable extension of the paper's own logic: on tasks dominated by symmetric couplings GTP and MTP should match CGTP accuracy at lower cost, while on tasks with strong antisymmetric coupling they should lag regardless of width; the paper itself concedes the loss can be irrelevant when the missing interactions are not needed.","If the expressivity proxy is accepted, the practical Pareto frontier is nearly degenerate after normalization, which suggests the next real speedup for equivariant layers will come from faster spherical-harmonic transforms or GPU-native sparse CGTP kernels rather than from new bilinear interaction designs.","Interactability offers a quick theoretical screen: any proposed tensor product operation can be classified by which irrep triples it permits and how many independent bilinearities it constructs, letting designers check for expressivity loss before writing GPU code.","The paper's framing implies that benchmarking claims about tensor products should always be paired with an expressivity measure; otherwise a reported speedup may simply be a report of an operation that computes less."],"forward_implications":["A practitioner comparing GTP or MTP with CGTP by wall-clock time is comparing different operations; the speed advantage is real only for tasks whose needed interaction paths survive the TPO's selection rules.","The spherical-grid implementation of GTP is a drop-in replacement for the Fourier-basis implementation with identical asymptotics and about 30% faster training of the MACE interatomic potential.","Networks that must represent chiral or antisymmetric information cannot be built on GTP alone, because paths such as the cross product are structurally absent rather than merely expensive.","Asymptotic order does not predict wall-clock behavior: sparse CGTP has the fewest FLOPs but the slowest wall time, while GTP's higher FLOP count is offset by higher GPU utilization.","The commonly cited $O(L^6)$ cost of CGTP overstates the gap by an order of $L$; exploiting Clebsch-Gordan sparsity gives $O(L^5)$, narrowing the asymptotic gap the $O(L^3)$ alternatives advertise."],"supporting_citations":[{"why":"Introduced the Gaunt tensor product whose O(L^3) speedup claim the paper analyzes, re-implements, and benchmarks against.","marker":"(Luo et al., 2024)"},{"why":"Source of the commonly cited O(L^6) runtime for CGTP that the paper corrects to O(L^5) via Clebsch-Gordan sparsity.","marker":"(Passaro & Zitnick, 2023)"},{"why":"Introduced the matrix tensor product interaction analyzed here as a TPO with O(L) expressivity.","marker":"(Unke & Maennel, 2024)"},{"why":"Provides the fast spherical-harmonic transform that yields the only expressivity-normalized asymptotic speedup in the paper's Table 1.","marker":"(Healy et al., 2003)"},{"why":"The Gaunt coefficients whose selection rules, including the parity condition, determine which irrep triples GTP can couple.","marker":"(Gaunt, 1929)"},{"why":"Standard reference for Clebsch-Gordan coefficients and the CGTP selection rules used in the paper's analysis.","marker":"(Varshalovich et al., 1988)"},{"why":"The MACE interatomic potential in which the paper's spherical-grid GTP acts as a drop-in replacement that trains about 30% faster.","marker":"(Batatia et al., 2022)"},{"why":"The message-passing architecture whose design underlies the chiral Tetris experiment exposing GTP's missing antisymmetric path.","marker":"(Batzner et al., 2022)"},{"why":"Prior observation that exploiting Clebsch-Gordan sparsity gives an O(L^5) runtime for CGTP, adopted as the paper's sparse baseline.","marker":"(Cobb et al.)"}],"fun_headline_variants":["Speedups in tensor products often hide expressivity loss","Gaunt and matrix tensor products prune interaction paths","Only spherical transform delivers real speedup without loss","Faster tensor products: a trade-off with expressivity"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole comparison stands or falls on the premise that the dimension of the space of bilinear maps a tensor product operation can construct is a faithful proxy for what the operation contributes to a learned model, a modeling choice the paper states explicitly and qualifies by conceding that tasks which do not need the missing interactions lose nothing in practice.","fun_headline_variants_meta":{"raw":{"variants":["Speedups in tensor products often hide expressivity loss","Gaunt and matrix tensor products prune interaction paths","Only spherical transform delivers real speedup without loss","Faster tensor products: a trade-off with expressivity"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000575,"raw_usage":{"total_tokens":2780,"prompt_tokens":1078,"completion_tokens":1702,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":694,"completion_tokens_details":{"reasoning_tokens":1640}},"tokens_in":694,"tokens_out":1702,"duration_ms":11847,"temperature":1.0,"reasoning_tokens":1640,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:00:48.645154+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"The chiral Tetris classification is the sharpest test: under the paper's selection rules, no GTP network of any width or depth can construct the antisymmetric $1_o \\otimes 1_o \\to 1_e$ path needed to tell mirror-image pieces apart, so a GTP model that reaches above-chance accuracy on the chiral pair would refute the claim that the missing interactions are structurally unreachable. A complementary check measures wall-clock time of the S2FFT variant against sparse CGTP at increasing $L$; if CGTP times fail to track $O(L^5)$ while S2FFT tracks $O(L^2 \\log^2 L)$, the expressivity-normalized asymptotics of Table 1 would need to be revisited.","supporting_citations":[{"cited_title":"M., Rockmore, D","cited_arxiv_id":null,"evidence_quote":"Provides the fast spherical-harmonic transform that yields the only expressivity-normalized asymptotic speedup in the paper's Table 1."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The Gaunt coefficients whose selection rules, including the parity condition, determine which irrep triples GTP can couple."},{"cited_title":"A., Moskalev, A","cited_arxiv_id":null,"evidence_quote":"Standard reference for Clebsch-Gordan coefficients and the CGTP selection rules used in the paper's analysis."},{"cited_title":"P., Simm, G","cited_arxiv_id":null,"evidence_quote":"The MACE interatomic potential in which the paper's spherical-grid GTP acts as a drop-in replacement that trains about 30% faster."}],"review_version":2}