Pith. sign in

REVIEW 3 major objections 5 minor 24 references

ReLU$^k$ Neural de Rham Complexes

T0 review · 3 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fixed-neuron shallow ReLU^k networks contain a finite-dimensional de Rham subcomplex that is exact — the exterior derivative decouples neuron by neuron into Koszul complexes.

desk verdict A correct and genuinely new exactness theorem for fixed-neuron ReLU^k de Rham complexes, with a real but narrow gap between the theorem's hypothesis and the numerical experiments. read the letter →

arxiv 2607.22478 v1 pith:P6C7PZML submitted 2026-07-24 math.NA cs.NA

Kaibo Hu , Jindong Wang , Jinchao Xu This is my paper · ORCID
classification math.NAcs.NA MSC 41A3041A4655U1558A1065N2565N3068T07
keywords ReLUneuralnetworksdeRhamcomplexKoszulexactsequencestructure-preservingdiscretizationdifferentialformsmixedfiniteelementslinearizedshallow
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper shows that fixed-neuron shallow ReLU^k networks can be organized into a finite-dimensional de Rham complex — a chain of spaces of differential forms connected by the exterior derivative that satisfies the correct kernel–image identities. The construction assigns to each form degree p the span of the ridge functions σ_{k-p}(s_i) with constant p-form coefficients; because differentiating a ReLU power lowers its order by one and acts as multiplication by the fixed one-form ds_i, the derivative maps each space into the next. Under a linear independence condition on the lowest-order ridge family, the global complex is exact: every closed form is a derivative, with no spurious cohomology. The value is a mesh-free analogue of structure-preserving finite element spaces, with numerical experiments indicating stable mixed discretizations and spectra free of spurious modes.

What carries the argument

The machinery is the neuron-wise Koszul complex: for a fixed nonzero covector λ, the sequence Λ^0 → Λ^1 → ... → Λ^d given by wedge product with λ is exact, with homotopy provided by the contraction identity ι_v(λ∧α)+λ∧ι_v α = α. The proof isolates each neuron by the linear independence of the ridge functions, which also transfers from the lowest degree to all higher degrees via directional derivatives (Lemma 4.3). A geometric sufficient condition for the independence — distinct, non-antipodal neuron parameters with hyperplanes meeting the domain — is given in Lemma 4.4.

What would settle it

Take a quasi-uniform neuron set that avoids antipodal pairs and compute the numerical rank of the Gram matrix of {σ_{k-d}(s_i)} at dense sample points. A rank below n would violate the paper's assumption and break the exactness proof; a full rank for many random sets would support the assumption's genericity. The antipodal-pair example (4.3) shows the failure mode is real.

Watch

Extended reading notes

Core claim

The central discovery is that the exterior derivative acts independently on each neuron: d(σ_{k-p}(s_i) α_i) = (k-p) σ_{k-p-1}(s_i) ds_i ∧ α_i. Consequently the global complex 0→C^0→...→C^d→0 splits coefficient-wise into the classical Koszul complex of the one-form ds_i for each neuron. Provided the lowest-order ridge family {σ_{k-d}(s_i)} is linearly independent on the domain, the paper proves (Theorem 6.2) that the complex is exact in every degree, for any dimension d and any integer k≥d. Exactness gives the discrete kernel–range identities: every closed discrete form is the derivative of one at the previous level, ruling out spurious closed fields and ensuring the discrete spaces represen

Load-bearing premise

Everything rests on the assumption that the n ridge functions σ_{k-d}(s_i) are linearly independent on the domain; the paper supplies a sufficient geometric condition (distinct, non-antipodal neurons whose zero hyperplanes cross the domain), but does not prove that every quasi-uniform parameter set satisfies it, and if it fails the neuron-wise decoupling that drives the exactness proof collapses.

Editorial extensions

If this is right

  • If the exactness theorem holds, fixed-neuron shallow ReLU^k spaces contain a bona fide discrete de Rham complex on contractible domains, giving the correct kernel–range identities and eliminating algebraic spurious cohomology.
  • Mixed Galerkin discretizations built from the complex inherit the exactness structure, which is the algebraic backbone of stable mixed methods; the reported inf-sup constants and eigenvalue spectra support this.
  • Combined with the optimal approximation rates already known for fixed-neuron ReLU^k spaces, the complex provides a path to convergent, mesh-free structure-preserving discretizations of Hodge-type and Maxwell-type boundary value problems.
  • Because the proof is dimension-independent, the same construction yields grad–curl–div sequences in 3D and higher-degree form complexes in any dimension, and it opens the door to derived complexes for elasticity-like systems.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the linear independence assumption holds generically for quasi-uniform neuron sets — as the paper's geometric condition suggests but does not fully characterize — then exactness is a generic property rather than a special construction; a probabilistic or measure-theoretic argument would settle this.
  • The mechanism 'differentiation lowers activation order and acts as a fixed algebraic operation' is not unique to ReLU powers: cosine or Fourier ridge functions, where derivatives cycle back, could yield exact complexes with nontrivial cohomology representing the domain's topology, as the paper hints in its closing remarks.
  • Exactness gives an immediate algebraic test: the alternating sum of the dimensions of C^p must vanish, and since each C^p is spanned by n independent ridge forms, the dimension count is automatically consistent; any failure of independence would manifest as an anomalous dimension drop, which is computable for a given neuron set.
  • A bounded cochain projection onto the complex is still missing; if it exists, it would upgrade the algebraic exactness to full stability estimates (inf-sup constants uniformly bounded in n), turning the numerical evidence into a theorem.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper constructs finite-dimensional de Rham subcomplexes inside fixed-neuron shallow ReLU^k neural network spaces. For affine neurons s_i(x)=ω_i·x+b_i and k≥d, the authors define spaces C^p of p-forms with coefficients σ_{k-p}(s_i), noting that exterior differentiation lowers the ReLU degree and wedges with d s_i. The main result, Theorem 6.2, states that under linear independence of the lowest-order family {σ_{k-d}(s_i)}, the complex 0→C^0→⋯→C^d→0 is exact. The proof combines Lemma 4.3 (independence propagates upward via directional derivatives), Lemma 6.1 (Koszul homotopy for a nonzero covector), and a coefficient-wise decoupling argument. The paper also gives a sufficient geometric condition (Assumption 4.1 and Lemma 4.4) and presents numerical experiments for mixed Poisson, Hodge Laplacian, and eigenvalue problems. The conditional exactness theorem appears algebraically correct; the main gaps concern the scope of the numerical experiments and the weak/distributional interpretation at the k=d endpoint.

Significance. If the main theorem is accepted, the paper provides a clean and elementary construction of an exact de Rham subcomplex within a linearized shallow ReLU^k approximation space. This is a genuinely useful bridge between FEEC and neural-network approximation: exactness gives the discrete kernel–range identities that rule out algebraic spurious cohomology, and the proof is transparent, using only the ReLU derivative identity and the standard Koszul homotopy. No fitted constants or numerical tuning enter the theoretical argument, which is a strength. The paper is also careful to state that some ingredients of a full FEEC theory—uniform inf-sup bounds, bounded commuting projections, boundary compatibility—are not provided. However, the numerical section is currently not a rigorous validation of the theorem because it does not verify the key linear-independence hypothesis and uses k values outside the standing k≥d assumption. The central mathematical contribution is sound, but the applied claims require revision.

major comments (3)
  1. [§7 vs. Assumption 4.1 / Theorem 6.2] The numerical experiments are presented as supporting the exact neural complex, but they never verify the load-bearing hypothesis of Theorem 6.2, namely linear independence of {σ_{k-d}(s_i)}. Section 7 states only "quasi-uniform neuron parameters excluding antipodal pairs." This does not enforce Assumption 4.1's requirement that each hyperplane H_i intersect Ω. For Ω=(0,1)^d, many quasi-uniform neurons on S^d have H_i∩Ω=∅; for those neurons σ_{k-d}(s_i) is identically zero or a polynomial on Ω, which can make the lowest-order family rank-deficient. Without an explicit rank check or a neuron-generation procedure that filters neurons with H_i∩Ω≠∅, the reported convergence rates and the absence of spurious modes cannot be attributed to the exactness theorem. This is not a flaw in the conditional theorem, but it is a real gap between the theorem's hypothesis and the numerical evidence claime
  2. [§3 and §7: k≥d standing assumption] The paper assumes k≥d throughout §3 and in Theorem 6.2, but the numerical sections use smaller values: §7.1 uses k=1 in d=2 and §7.2 uses k=2 in d=3. These cases fall outside the theorem's scope. Moreover, in §7.1 the discrete pair (L^1_n)^2 × L^0_n is not the H(div)–L^2 part of the complex for k=1; it is the C^1–C^2 pair for the complex with parameter k=2. The displayed rates in §7.1 use the exponent k=1 and are therefore not the correct approximation exponents for the spaces actually used. A clarification of which complex parameter and which truncated subcomplex is used in each experiment is needed, along with corrected rate statements or a restriction to k≥d.
  3. [§2–§6, Lemma 4.3/4.4 and Theorem 6.2 for k=d] When k=d, the top space C^d contains σ_0(s_i), which is discontinuous (σ_0(t)=max{t,0}^0, an indicator-type function). Lemma 4.4 explicitly uses distribution derivatives and δ(s_i), and the exactness proof at C^d relies on representing top forms as ds_i∧ι_{v_i}α_i. However, the paper never states the ambient function-space setting in which the complex is exact: pointwise, L^2, or distributional. The directional-derivative argument in Lemma 4.3 and the final surjectivity step (6.26) need this clarification when k=d. This is a rigor gap in an endpoint case of the main theorem, not a fatal error, but it should be fixed.
minor comments (5)
  1. [Eq. (2.2)] The definition σ_k(t)=max{t,0}^k for k∈N_0 does not specify the value of σ_0 at t=0. Please state a convention, e.g., σ_0(t)=1 for t>0 and σ_0(t)=0 otherwise (or define the relevant equivalence class).
  2. [§7.1 and §7.2] The term "quasi-uniform" is used without definition. Since it is borrowed from [20], please either define it or give a precise reference to the definition used.
  3. [§7, Table 7.1] The inf-sup constant for k=2 at n=16 is 4.346e-01, which then jumps to above 9.6e-01 for larger n. A brief comment on small-n behavior would be helpful, although it does not affect the asymptotic claim.
  4. [Figures 7.1–7.4] The figure captions do not fully describe axes, markers, and reference lines. Adding axis labels and legends would improve reproducibility and readability.
  5. [§8] The discussion of possible future extensions (BGG, boundary-compatible spaces, nontrivial topology) is reasonable, but the sentence suggesting that the complex "rules out algebraic spurious cohomology" should be tied explicitly to the exactness theorem and its hypothesis, since the numerical evidence does not yet verify that hypothesis.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: Theorem 6.2 is a self-contained conditional exactness proof; the only self-citation ([20]) is used as approximation-theoretic motivation, not as an input to the exactness argument.

full rationale

The derivation chain is not circular. Theorem 6.2 is a conditional statement whose only hypotheses are k≥d and linear independence of {σ_{k-d}(s_i)}. Proposition 3.1 derives the differential inclusion from the elementary ReLU derivative identity, Lemma 4.3 lifts the lowest-order independence to all orders used in the complex, and the kernel/range steps at each degree are obtained coefficient-wise from the standard Koszul homotopy (Lemma 6.1). No parameter is fitted to data and no output is renamed as a prediction: the numerical sections solve Galerkin problems in the constructed fixed-neuron spaces and compare errors to rates quoted from the literature. The only author-overlapping citation is [20] (Liu–Mao–Xu, with Xu as a co-author), which supplies the integral representation and approximation rates (2.6)–(2.7); those rates are used as motivation and as benchmark slopes in Section 7, but they do not enter the exactness proof and are not load-bearing for the central constructed complex. The paper itself flags the missing uniform inf-sup stability estimate and the open question of a bounded cochain interpolation operator; these are analytical limitations, not circular inputs. The possible failure of Assumption 4.1 for the quasi-uniform neuron sets used numerically is an applicability/verification gap, not a circular step. Score 2 reflects only the minor, non-load-bearing self-citation; the central derivation is independently established.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The theorem is conditional on a linear-independence axiom; the geometric sufficient condition is an ad hoc nondegeneracy assumption. No fitted physical parameters enter the exactness proof. The only hand-chosen input is the neuron set, which matters for the numerical section.

free parameters (1)
  • fixed neuron parameters {θ_i=(ω_i,b_i)}_{i=1}^n ⊂ S^d = not specified; 'quasi-uniform parameters excluding antipodal pairs' (§7)
    The discrete spaces and all numerical results depend on this hand-chosen set; no generation algorithm or seed is given, so the numerical claims are not reproducible.
assumptions (5)
  • domain assumption Ω⊂R^d is bounded and contractible; s_i affine with ω_i≠0; k≥d
    Stated in §2-3; the whole construction is set on such domains.
  • ad hoc to paper Assumption 4.1: θ_i≠±θ_j and H_i∩Ω≠∅
    Used in Lemma 4.4 to prove linear independence; excludes antipodal pairs to avoid the ReLU-polynomial relation (4.3).
  • ad hoc to paper Linear independence of the lowest-order family {σ_{k-d}(s_i)}_{i=1}^n on Ω
    Theorem 6.2's premise; load-bearing because exactness decouples neurons only under unique representation.
  • domain assumption Distributional/weak exterior derivative identities for ReLU^k (σ_0 coefficients are discontinuous when k=d)
    Proposition 3.1 uses pointwise derivatives; for k=d, top-degree coefficients are σ_0, so d must be interpreted weakly; the paper does not state this.
  • standard math Koszul contraction identity and exactness of the λ∧ complex
    Lemma 6.1; standard material from Bott-Tu [22].

how reviews work

0 comments
Cite this review

Pith. "Pith review of ReLU$^k$ Neural de Rham Complexes." pith.science (2026). https://pith.science/paper/P6C7PZML

@misc{pith2026260722478,
  author       = {Pith},
  title        = {Pith review of: ReLU$^k$ Neural de Rham Complexes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P6C7PZML}},
  note         = {Machine review of arXiv:2607.22478}
}
abstract

We construct finite-dimensional de Rham subcomplexes generated by fixed-neuron shallow ReLU$^k$ neural networks, a class of spaces known to provide optimal approximation rates. For neurons of the form $s_i(x)=\omega_i\cdot x+b_i$, we introduce spaces of neural differential forms: differential $p$-forms whose coefficients are the ReLU$^k$ ridge functions $\sigma_{k-p}(s_i)$. These spaces are compatible with the exterior derivative because differentiating a ReLU power lowers its order by one, and for each fixed neuron, differentiation amounts to exterior multiplication by the fixed one-form $ d s_i$. Under a linear independence assumption on the lowest-order family $\{\sigma_{k-d}(s_i)\}_{i=1}^n$, the global complex decomposes into independent neuron-wise Koszul complexes. We prove exactness in arbitrary dimension and provide a geometric sufficient condition for the required linear independence. Numerical experiments based on the resulting complex provide evidence of stable discretizations and of convergence rates consistent with the underlying approximation theory, and exhibit no spurious modes in eigenvalue problems considered.

Figures

Figures reproduced from arXiv: 2607.22478 by the authors.

Figure 7
Figure 7. [PITH_FULL_IMAGE:figures/full_fig_p014_7.png] view at source ↗
Figure 7
Figure 7. [PITH_FULL_IMAGE:figures/full_fig_p016_7.png] view at source ↗
Figure 7
Figure 7. [PITH_FULL_IMAGE:figures/full_fig_p018_7.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 3 canonical work pages

  1. [19]

    Neural Networks165, 721–739 (2023)

    Longo, M., Opschoor, J.A., Disch, N., Schwab, C., Zech, J.: De Rham compatible deep neural network fem. Neural Networks165, 721–739 (2023)

    2023

  2. [20]

    arXiv preprint arXiv:2505.00351 (2025)

    Liu, X., Mao, T., Xu, J.: Integral representations of Sobolev spaces via ReLU k activation function and optimal error estimates for linearized networks. arXiv preprint arXiv:2505.00351 (2025)

    arXiv 2025

  3. [21]

    arXiv preprint arXiv:2603.28638 (2026)

    He, J., Liu, X., Tian, Z.: Divergence-free linearized neural networks: Integral representation and optimal approximation rates. arXiv preprint arXiv:2603.28638 (2026)

    arXiv 2026

  4. [1]

    Acta Numerica15, 1–155 (2006)

    Arnold, D.N., Falk, R.S., Winther, R.: Finite element exterior calculus, homolog- ical techniques, and applications. Acta Numerica15, 1–155 (2006)

    2006

  5. [2]

    Bulletin of the American Mathematical Society 47(2), 281–354 (2010)

    Arnold, D., Falk, R., Winther, R.: Finite element exterior calculus: from Hodge theory to numerical stability. Bulletin of the American Mathematical Society 47(2), 281–354 (2010)

    2010

  6. [3]

    CBMS-NSF Regional Conference Series in Applied Mathematics, vol

    Arnold, D.N.: Finite Element Exterior Calculus. CBMS-NSF Regional Conference Series in Applied Mathematics, vol. 93. Society for Industrial and Applied Math- ematics, Philadelphia, PA (2018). https://doi.org/10.1137/1.9781611975543

    doi:10.1137/1.9781611975543 2018

  7. [4]

    Springer Series in Computational Mathematics, vol

    Boffi, D., Brezzi, F., Fortin, M.: Mixed Finite Element Methods and Applications. Springer Series in Computational Mathematics, vol. 44. Springer, Berlin (2013). https://doi.org/10.1007/978-3-642-36519-5

    doi:10.1007/978-3-642-36519-5 2013

  8. [5]

    Numerische Mathematik139(2), 411–446 (2018) https://doi.org/10.1007/ s00211-017-0939-x

    Christiansen, S.H., Hu, J., Hu, K.: Nodal finite element de Rham com- plexes. Numerische Mathematik139(2), 411–446 (2018) https://doi.org/10.1007/ s00211-017-0939-x

    2018

Show all 24 references
  1. [6]

    Numerische Mathematik140(2), 327–371 (2018) https://doi.org/10.1007/s00211-018-0970-6

    Christiansen, S.H., Hu, K.: Generalized finite element systems for smooth dif- ferential forms and Stokes’ problem. Numerische Mathematik140(2), 327–371 (2018) https://doi.org/10.1007/s00211-018-0970-6

    2018 doi

  2. [7]

    Annals of Mathematics154(1), 97–113 (2001) https://doi.org/10.2307/3062111

    ˇCap, A., Slov´ ak, J., Souˇ cek, V.: Bernstein–Gelfand–Gelfand sequences. Annals of Mathematics154(1), 97–113 (2001) https://doi.org/10.2307/3062111

    2001 doi

  3. [8]

    Foundations of Com- putational Mathematics21(6), 1739–1774 (2021) https://doi.org/10.1007/ s10208-021-09498-9

    Arnold, D.N., Hu, K.: Complexes from complexes. Foundations of Com- putational Mathematics21(6), 1739–1774 (2021) https://doi.org/10.1007/ s10208-021-09498-9

    2021

  4. [9]

    Foun- dations of Computational Mathematics24, 1145–1184 (2024) https://doi.org/10

    ˇCap, A., Hu, K.: BGG sequences with weak regularity and applications. Foun- dations of Computational Mathematics24, 1145–1184 (2024) https://doi.org/10. 1007/s10208-023-09608-9 20

    2024

  5. [10]

    Mathe- matics of control, signals and systems2(4), 303–314 (1989)

    Cybenko, G.: Approximation by superpositions of a sigmoidal function. Mathe- matics of control, signals and systems2(4), 303–314 (1989)

    1989

  6. [11]

    Neural networks2(5), 359–366 (1989)

    Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural networks2(5), 359–366 (1989)

    1989

  7. [12]

    Analysis and Applications17(1), 19–55 (2019) https://doi.org/10.1142/S0219530518500203

    Schwab, C., Zech, J.: Deep learning in high dimension: Neural network expres- sion rates for generalized polynomial chaos expansions in UQ. Analysis and Applications17(1), 19–55 (2019) https://doi.org/10.1142/S0219530518500203

    2019 doi

  8. [13]

    Analysis and Applications18(5), 715–770 (2020) https://doi.org/10.1142/S0219530519410136

    Opschoor, J.A.A., Petersen, P.C., Schwab, C.: Deep ReLU networks and high- order finite element methods. Analysis and Applications18(5), 715–770 (2020) https://doi.org/10.1142/S0219530519410136

    2020 doi

  9. [14]

    SIAM/ASA Journal on Uncertainty Quantification11(1), 199–234 (2023) https://doi.org/10.1137/21M1462738

    Schwab, C., Zech, J.: Deep learning in high dimension: Neural network expression rates for analytic functions inl 2(Rd, γd). SIAM/ASA Journal on Uncertainty Quantification11(1), 199–234 (2023) https://doi.org/10.1137/21M1462738

    2023 doi

  10. [15]

    Journal of Computational Mathematics38(3), 502–527 (2020) https: //doi.org/10.4208/jcm.1901-m2018-0160

    He, J., Li, L., Xu, J., Zheng, C.: ReLU deep neural networks and linear finite elements. Journal of Computational Mathematics38(3), 502–527 (2020) https: //doi.org/10.4208/jcm.1901-m2018-0160

    2020 doi

  11. [16]

    Communications in Computational Physics28(5), 1707–1745 (2020) https://doi.org/10.4208/CICP

    Xu, J.: Finite neuron method and convergence analysis. Communications in Computational Physics28(5), 1707–1745 (2020) https://doi.org/10.4208/CICP. OA-2020-0191

    2020 doi

  12. [17]

    Applied and Computational Harmonic Analysis58, 1–26 (2022) https://doi.org/10.1016/j.acha.2021.12.005

    Siegel, J.W., Xu, J.: High-order approximation rates for shallow neural net- works with cosine and ReLU k activation functions. Applied and Computational Harmonic Analysis58, 1–26 (2022) https://doi.org/10.1016/j.acha.2021.12.005

    2022 doi

  13. [18]

    Foundations of Computational Mathe- matics24(2), 481–537 (2024)

    Siegel, J.W., Xu, J.: Sharp bounds on the approximation rates, metric entropy, and n-widths of shallow neural networks. Foundations of Computational Mathe- matics24(2), 481–537 (2024)

    2024

  14. [22]

    Graduate Texts in Mathematics, vol

    Bott, R., Tu, L.W.: Differential Forms in Algebraic Topology. Graduate Texts in Mathematics, vol. 82. Springer, New York (1982). https://doi.org/10.1007/ 21 978-1-4757-3951-0

    1982

  15. [23]

    In: Ciarlet, P.G., Lions, J.-L

    Babuˇ ska, I., Osborn, J.: Eigenvalue problems. In: Ciarlet, P.G., Lions, J.-L. (eds.) Finite Element Methods (Part 1). Handbook of Numerical Analysis, vol. 2, pp. 641–787. North-Holland, Amsterdam (1991). https://doi.org/10.1016/ S1570-8659(05)80042-0

    1991

  16. [24]

    IMA Journal of Numerical Analysis43(2), 663–691 (2023) 22

    Boffi, D., Guzm´ an, J., Neilan, M.: Convergence of Lagrange finite elements for the Maxwell eigenvalue problem in two dimensions. IMA Journal of Numerical Analysis43(2), 663–691 (2023) 22

    2023

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.