Pith. sign in

REVIEW 2 major objections 4 minor 99 references

Optimal Neural Network Approximation via Empirical Least Squares with Deterministic Samples

T0 review · 2 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proves that empirical least squares over deterministic collocation points achieves the optimal $n^{-(r-s)/d}$ approximation rate for linearized ReLU$^k$ networks on the sphere, with a sample size proportional to the number of…

desk verdict A genuinely new Bernstein inequality and a deterministic m~n sample bound for ReLU^k residual least squares on the sphere; the main caveat is an unproved stronger comparison theorem imported from the authors' prior work. read the letter →

arxiv 2608.06687 v1 pith:3JSZDG3K submitted 2026-08-07 math.NA cs.LGcs.NA

Xinliang Liu , Tong Mao , Jinchao Xu This is my paper · ORCID
classification math.NAcs.LGcs.NA MSC 41A2541A3065D1565N1265N35
keywords neuralnetworkapproximationempiricalleastsquaresdeterministiccollocationBernsteininequalityReLU^kactivationsphericalharmonicsSobolevspacesellipticspectralequations
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proves that for elliptic equations $L_\beta u = f$ on the sphere solved by linearized ReLU$^k$ networks, empirical least squares over deterministic collocation points reaches the same optimal approximation rate as the continuous formulation, provided the number of sampling points $m$ is proportional to the number of network parameters $n$ and the target has the parity compatible with the activation. The trial space consists of ridge functions $\sigma_k(\theta_j \cdot \eta)$ with parameters on the sphere, and $L_\beta$ is a positive elliptic spectral multiplier of order $\beta$. The key stability tool is a Bernstein inequality controlling high-order Sobolev norms of network functions by lower-order norms through the antipodal separation distance of the parameters. Because $m\asymp n$ suffices, the method bypasses heavy oversampling and quadrature. The paper also gives a near-optimal high-probability bound for i.i.d. uniform samples and, for $\beta=0$, an affine-network version on bounded Lipschitz domains through a constructive lifting.

What carries the argument

The load-bearing object is the Bernstein inequality for linearized ReLU$^k$ spaces on the sphere: for $0\le s<r<k+1/2$, every $v_n\in L_n^k$ satisfies $\|v_n\|_{H^r} \lesssim \underline{h}^{-(r-s)}\|v_n\|_{H^s}$, where $\underline{h}$ is the antipodal separation distance of the parameters. Its proof fills the spectral gaps of $\sigma_k$ with a surrogate kernel $\phi_k$, represents Sobolev seminorms as quadratic forms built from localized dyadic kernels, and establishes diagonal dominance of the resulting matrices under antipodal separation. Applied to the residual space $L_\beta L_n^k$, the same inequality gives the inverse estimate that turns empirical residual control into Sobolev error control. A companion comparison-network theorem supplies an approximant whose low-degree spherical harmonic coefficients match the target exactly, which is what lets the proof identify low-frequency parts of the residual before applying the Bernstein estimate.

What would settle it

Inspect the construction behind the comparison-network theorem: compute the low-degree spherical harmonic coefficients of the comparison network and compare them with the target's coefficients for all degrees up to $C_1 h^{-1}$. If the identity $\widehat{u_n}(\nu,\ell)=\widehat{u}(\nu,\ell)$ fails at any degree in that range, then the proof's step $P_q L_\beta u_n = P_q f$ no longer holds and the claimed rate is unsupported. An independent check would run the discrete least-squares problem with $m$ proportional to $n$ on a smooth parity-compatible right-hand side and test whether the empirical error decays at the predicted $n^{-(r-s)/d}$ rate over a wide range of $n$.

Watch

Extended reading notes

Core claim

Theorem 4 is the central claim: for $k>\beta+(d-1)/2$, $r\le (d+2k+1)/2-\beta$, and $0\le s<\min\{r,k+1/2-\beta\}$, if the network parameter set is antipodally quasi-uniform and the collocation set is quasi-uniform with $m\ge C_2 h^{-d}$, then the empirical residual minimizer satisfies $\|u-u_{n,m}\|_{H^{s+\beta}} \eqsim \|f-L_\beta u_{n,m}\|_{H^s}$, bounded above by $h^{r-s}$ times a Sobolev norm of $f$. With $m\asymp n$ this becomes the optimal rate $n^{-(r-s)/d}$ in the target norm. The paper's point is that the unregularized linear least-squares problem with deterministic points retains the full approximation power of the continuous problem. In the order-zero case $\beta=0$ the same estimate recovers ordinary discrete least-squares approximation, and the authors make clear that the bounded-domain version is an indirect lifting-based consequence, not a direct Euclidean-domain collocation theorem.

Load-bearing premise

The entire deterministic rate rests on a theorem from the authors' earlier work asserting that a target with the right parity can be approximated by a network whose low-frequency spherical coefficients match the target exactly; if that exact matching fails, the residual identity used in the proof breaks and the optimal-rate conclusion does not follow.

Editorial extensions

If this is right

  • For any positive elliptic spectral multiplier $L_\beta$ on the sphere, residual least squares with $m\asymp n$ deterministic quasi-uniform collocation points achieves the optimal rate $n^{-(r-s)/d}$ for parity-compatible data of Sobolev smoothness $r$.
  • The Bernstein inequality gives an inverse approximation theorem: if a function is approximated at rate $n^{-(r-s)/d}$ in $H^s$ by these networks, then it belongs to $H^\alpha$ for every $\alpha<r$.
  • For $\beta=0$, the theory covers ordinary discrete least-squares approximation, and the lifting construction yields affine ReLU$^k$ networks on bounded Lipschitz domains with the same rate.
  • I.i.d. uniform collocation points produce a near-optimal high-probability residual estimate, up to a logarithmic factor and an arbitrarily small smoothness loss.
  • The elliptic norm equivalence transfers residual estimates directly to solution errors: $\|u-u_{n,m}\|_{H^{s+\beta}}$ is comparable to the residual norm $\|f-L_\beta u_{n,m}\|_{H^s}$.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper explicitly restricts the bounded-domain construction to the order-zero case and does not claim a direct collocation theory on Euclidean domains; extending the Bernstein argument to affine networks with boundary residuals is a natural next step, but the paper leaves it open.
  • If the comparison-network identity can be established beyond antipodally quasi-uniform parameter sets, the same proof would carry the optimal-rate conclusion to other geometries; the paper does not assert this.
  • The explicit dependence of the sample threshold on $h^{-d}$ suggests that the optimal rate is tied to parameter geometry: poorly separated parameters force more collocation points, so the practical claim is about well-conditioned parameter sets.
  • The reported experiments cover finite refinement ranges; a sharper test of the theory would measure slopes at considerably larger $n$ to separate asymptotic rates from pre-asymptotic transients.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper develops deterministic and randomized discrete residual least-squares theory for elliptic spectral equations L_beta u = f on the unit sphere, using linearized ReLU^k network spaces with fixed, antipodally quasi-uniform parameter sets. The main deterministic result (Theorem 4) states that for any quasi-uniform collocation set with m >= C h^{-d} (so m proportionally n in the quasi-uniform case), the empirical residual minimizer achieves the continuous approximation rate h^{r-s}, equivalently n^{-(r-s)/d}. Theorem 5 gives a high-probability analogue for i.i.d. uniform collocation points, with a log factor and an arbitrarily small smoothness loss. The central technical novelty is a Bernstein inequality for ReLU^k network spaces (Theorem 1), with an explicit dependence on the antipodal separation distance, together with a residual-space version for L_beta L_n^k. Section 5 provides an affine-network interpretation for the order-zero case on bounded Lipschitz domains, and Section 6 reports numerical experiments consistent with the predicted rates.

Significance. If the main results are valid, the deterministic sample complexity m ~ n is a strong and interesting statement: it shows that equal-weight collocation with only a constant oversampling factor can match the optimal continuous Sobolev approximation rates for linearized ReLU^k networks, without cubature weights or polynomial exactness. The Bernstein inequality with explicit geometric constants is a useful independent contribution, and its inverse-approximation corollary is a meaningful addition to the literature. The paper is also commendably careful about the scope of the bounded-domain extension, explicitly stating that it does not prove a direct collocation theorem on general domains. The main caveat is that the deterministic proof relies on a strengthened comparison theorem imported from the authors' prior work, which is not proved in this manuscript; this limits the certainty of the headline claim until the gap is closed.

major comments (2)
  1. The deterministic rate in Theorem 4 depends on the exact low-degree identity (4.6), namely \hat u_n(ν,ℓ)=\hat u(ν,ℓ) for ν≤C_1 h^{-1}. The paper states that the cited theorem in [49] is weaker, and that the identity follows from [49, (4.12)–(4.13)], but the identity is not proved here. This is load-bearing: it is used to conclude P_q f = P_q L_β u_n, which underlies the decomposition in (4.23) and the equality in (4.25). As written, Theorem 4 is conditional on an unverified strengthening of an external result. I do not regard this as a counterexample, because the argument can likely be repaired by estimating \|P_q^c(L_β u_n - f)\|_{L^2} ≤ \|L_β u_n - f\|_{L^2} ≲ h^r \|f\|_{H^r} and carrying the extra term through (4.24)–(4.25), but the proof should present this repair or prove (4.6).
  2. The Bernstein inequality, which is the key analytical ingredient, relies on the asymptotic estimate (3.18) for ξ_α^{(ν)}(t), quoted from [49, Lemma 3.1] without proof. Since this estimate controls the localization estimate in Lemma 1 and hence the diagonal-dominance argument in Lemma 2, the main inequality is not fully self-contained. I suggest including a short derivation of (3.18) from gamma-function asymptotics, or at least reproducing the cited lemma in full, so that the novelty of Theorem 1 can be verified independently of the authors' prior preprint.
minor comments (4)
  1. The first branch in both displays reads "d/2 < r ≤ d/2", which is an empty range; it should read "d/p < r ≤ d/2" as in the statement of Theorem 4 and the abstract.
  2. The condition "d/p < r ≤ d/2, p > 2" should be read as "there exists p > 2 with d/p < r"; the quantifier over p should be made explicit in the theorem statement and in the abstract.
  3. The column heading "reference rate" for the cube experiments lists the spherical rates from Theorem 4 even though the text states they are not theoretical rates for Q^d; relabel the column as "spherical reference" to avoid confusion.
  4. The quasi-uniform point set in Theorem 7 is a set of spherical collocation points, not samples in Ω; please state this explicitly in the theorem hypothesis, since the surrounding text is clear but the theorem statement alone could be misread.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: Theorem 4 follows from internally derived Bernstein and sampling inequalities; the only external input is a parameter-free comparison theorem from the authors' prior work, a verification risk rather than a circular reduction.

full rationale

The derivation is not circular. The Bernstein inequality (Theorem 1) is proved in Section 3 via filtered spectral kernels, diagonal dominance (Lemma 2), and the auxiliary kernel phi_k; the cited decay estimate (3.18) from [49, Lemma 3.1] is a parameter-free asymptotic identity. Theorem 4 combines this with quasi-uniform sampling estimates (Lemmas 5-7) and the Marcinkiewicz-Zygmund inequality, all proved in the paper. The one external load-bearing result is Theorem 6, imported from the authors' preprint [49]; the paper explicitly says 'The cited theorem is stated in a weaker form than the one recorded here; the additional low-degree identity and explicit construction of u_n follows from [49, (4.12)-(4.13)]', and the proof of Theorem 4 uses identity (4.6) to get P_q(f-L_beta u_n)=0 at (4.23)/(4.25). This is a missing-proof/verification gap, not circularity: (4.6) is parameter-free, does not include the target rate, and is not fitted to the data. The numerical experiments are finite-range illustrations, not inputs to the theorems. No target claim reduces by construction to its own inputs.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The theory contains no fitted parameters; all estimates are deterministic with generic constants. The main load-bearing external inputs are the authors' own prior approximation theorem [49] and classical sampling inequalities. The parity and quasi-uniformity restrictions are explicit conditions on the problem class.

assumptions (5)
  • domain assumption Target f satisfies the parity condition f(eta) = (-1)^{k+1} f(-eta) (equation 1.2).
    Required throughout Section 4; the comparison network and the spectral gap filling argument rely on the parity class of sigma_k. Functions not of this form are not covered unless split as f_par + p with p in P_k(S^d).
  • domain assumption Network parameter sets are antipodally quasi-uniform and collocation sets are quasi-uniform (Definition 2).
    These geometric controls give h approximately ~h approximately n^{-1/d}, which converts the geometric Bernstein inequality into the final n^{-(r-s)/d} rate.
  • domain assumption Theorem 6 from [49]: existence of an approximation u_n with rate and low-degree identity (4.5)-(4.6).
    Cited from the authors' prior work, not reproved here; it is the comparison network on which the discrete stability proof acts.
  • domain assumption Asymptotic coefficient estimate xi_alpha^{(nu)}(t) approximately t^{2alpha-(d+2k+1)-nu} from [49, Lemma 3.1] (here equation 3.18).
    Used in Lemma 1 to derive the localized kernel bound; it encodes the ultraspherical coefficient decay of the auxiliary kernel phi_k.
  • standard math Sampling inequalities for quasi-uniform points: polynomial Marcinkiewicz-Zygmund (Lemma 7) from [59] and sampling inequality (Lemma 6) from [3].
    These classical results connect empirical ell2 norms on quasi-uniform grids to continuous Sobolev norms.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Optimal Neural Network Approximation via Empirical Least Squares with Deterministic Samples." pith.science (2026). https://pith.science/paper/3JSZDG3K

@misc{pith2026260806687,
  author       = {Pith},
  title        = {Pith review of: Optimal Neural Network Approximation via Empirical Least Squares with Deterministic Samples},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3JSZDG3K}},
  note         = {Machine review of arXiv:2608.06687}
}
abstract

We develop a rigorous theory of discrete residual least-squares approximation for elliptic spectral equations $\mathfrak L_\beta u=f$ using linearized ReLU$^k$ neural networks on the sphere, where $\mathfrak L_\beta$ is a positive elliptic spectral multiplier of order $\beta$. Given a parameter set $\Theta_n=\{\theta_{j}^*\}_{j=1}^n\subset\mathbb S^d$, we approximate $u$ in the linearized network space $L_n^k(\Theta_n)$ by the discrete residual on the collocation points $\{\eta_i^*\}_{i=1}^m$ \begin{equation*} u_{n,m}\in\arg\min_{v_n\in L_n^k(\Theta_n)}\frac1m\sum_{i=1}^m\left(f(\eta_i^*)-\mathfrak L_\beta v_n(\eta_i^*)\right)^2. \end{equation*} With $k>\frac{d-1}{2}+\beta$, for antipodally quasi-uniform network parameter sets and any quasi-uniform collocation points with $m\gtrsim n$, we prove that \begin{equation*} \|u-u_{n,m}\|_{\mathcal H^{\beta}(\mathbb S^d)}\eqsim\|f-\mathfrak L_\beta u_{n,m}\|_{\mathcal L^2(\mathbb S^d)}\lesssim n^{-\frac{r}{d}} \begin{cases} \|f\|_{\mathcal W^{r,p}(\mathbb S^d)},&\frac{d}{p}<r\leq \frac{d}{2},~p>2,\\ \|f\|_{\mathcal H^r(\mathbb S^d)},&r>\frac{d}{2}. \end{cases} \end{equation*} We also establish a high-probability residual estimate, up to a logarithmic factor and an arbitrarily small smoothness loss, for i.i.d.\ uniformly distributed collocation points. The key analytical ingredient is a Bernstein inequality for linearized ReLU$^k$ network spaces. If $\underline h$ denotes the antipodal separation distance of the network parameters, then \begin{equation*} \|v_n\|_{\mathcal H^r(\mathbb S^d)}\lesssim\underline h^{-(r-s)}\|v_n\|_{\mathcal H^s(\mathbb S^d)},\qquad 0\leq s<r<k+\tfrac12. \end{equation*}

Figures

Figures reproduced from arXiv: 2608.06687 by the authors.

Figure 1
Figure 1. Deterministic least-squares errors on S 2–S 5 with m = 8n. Each activation order satisfies k > d−1 2 ; dotted lines show the reference exponents from Theorem 4. The results are finite-range numerical evidence for the spherical theory, not an empirical proof of its asymptotic assumptions. Additional sampling parameters, random-collocation and oversampling studies, implementation checks, and solver diagnostics are rec… view at source ↗
Figure 2
Figure 2. Relative residual, equivalently operator-induced energy, errors for ( [PITH_FULL_IMAGE:figures/full_fig_p037_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

99 extracted references · 73 canonical work pages

  1. [49]

    X. Liu, T. Mao, and J. Xu. Integral representations of sobolev spaces via ReLU k activation function and optimal error estimates for linearized networks.arXiv preprint arXiv:2505.00351, 2025

    arXiv 2025

  2. [1]

    R. A. Adams and J. J. Fournier.Sobolev spaces. Elsevier, 2003

    2003

  3. [2]

    Arcang´ eli, M

    R. Arcang´ eli, M. C. L´ opez de Silanes, and J. J. Torrens. An extension of a bound for functions in sobolev spaces, with applications to (m, s)-spline interpolation and smoothing.Numerische Mathematik, 107(2):181–211, 2007

    2007

  4. [3]

    Arcang´ eli and J

    R. Arcang´ eli and J. J. Torrens. Sampling inequalities in sobolev spaces.Journal of Approximation Theory, 182:18–28, 2014

    2014

  5. [4]

    F. Bach. Breaking the curse of dimensionality with convex neural networks.The Journal of Machine Learning Research, 18(1):629–681, 2017

    2017

  6. [5]

    A. R. Barron. Universal approximation bounds for superpositions of a sigmoidal func- tion.IEEE Transactions on Information theory, 39(3):930–945, 1993

    1993

  7. [6]

    A. R. Barron. Approximation and estimation bounds for artificial neural networks. Machine learning, 14:115–133, 1994

    1994

  8. [7]

    A. R. Barron, A. Cohen, W. Dahmen, and R. A. DeVore. Approximation and learning by greedy algorithms.Annals of Statistics, 36(1):64–94, 2008

    2008

Show all 99 references
  1. [8]

    Bernstein, A

    J. Bernstein, A. Vahdat, Y. Yue, and M.-Y. Liu. On the distance between two neural networks and the stability of learning.Advances in Neural Information Processing Systems, 33:21370–21381, 2020. 38

    2020

  2. [9]

    S. N. Bernstein. Sur quelques propri´ et´ es des polynˆ omes de meixner et de leurs coeffi- cients (on some properties of meixner polynomials and their coefficients).Communi- cations of the Kharkov Mathematical Society, 13:1–78, 1912

    1912

  3. [10]

    Bondarenko, D

    A. Bondarenko, D. Radchenko, and M. Viazovska. Optimal asymptotic bounds for spherical designs.Annals of mathematics, pages 443–452, 2013

    2013

  4. [11]

    Bourgain and J

    J. Bourgain and J. Lindenstrauss. Projection bodies. InGeometric Aspects of Func- tional Analysis: Israel Seminar (GAF A) 1986–87, pages 250–270. Springer, 2006

    1986

  5. [12]

    S. C. Brenner and L. R. Scott.The Mathematical Theory of Finite Element Methods. Springer, 3rd edition, 2008

    2008

  6. [13]

    V. I. Burenkov.Sobolev Spaces on Domains. B. G. Teubner, 1998

    1998

  7. [14]

    J. Chen, X. Chi, W. E, and Z. Yang. Bridging traditional and machine learning-based algorithms for solving pdes: the random feature method.J Mach Learn, 1:268–98, 2022

    2022

  8. [15]

    Chkifa, A

    A. Chkifa, A. Cohen, G. Migliorati, F. Nobile, and R. Tempone. Discrete least squares polynomial approximation with random evaluations- application to parametric and stochastic elliptic pdes.ESAIM: Mathematical Modelling and Numerical Analysis, 49(3):815–837, 2015

    2015

  9. [16]

    Cohen and G

    A. Cohen and G. Migliorati. Optimal weighted least-squares methods.SMAI Journal of Computational Mathematics, 3:181–203, 2017

    2017

  10. [17]

    T. S. Cohen, M. Geiger, J. K¨ ohler, and M. Welling. Spherical CNNs. InInternational Conference on Learning Representations, 2018

    2018

  11. [18]

    Coroianu and D

    L. Coroianu and D. Costarelli. Best approximation results and inverse theorems for ReLU neural networks.arXiv preprint arXiv:2403.07335, 2024

    2024 arXiv

  12. [19]

    Dai and Y

    F. Dai and Y. Xu.Approximation theory and harmonic analysis on spheres and balls. Springer, 2013

    2013

  13. [20]

    P. J. Davis and P. Rabinowitz.Methods of Numerical Integration. Academic Press, 2nd edition, 1984

    1984

  14. [21]

    Defferrard, N

    M. Defferrard, N. Perraudin, T. Kacprzak, and R. Sgier. DeepSphere: Towards an equivariant graph-based spherical CNN. InICLR Workshop on Representation Learn- ing on Graphs and Manifolds, 2019

    2019

  15. [22]

    R. A. DeVore and V. A. Popov. Optimal nonlinear approximation.Manuscripta mathematica, 58(3):295–306, 1989

    1989

  16. [23]

    R. A. DeVore and V. N. Temlyakov. Some remarks on greedy algorithms.Advances in computational Mathematics, 5(1):173–187, 1996

    1996

  17. [24]

    W. E, C. Ma, and L. Wu. A priori estimates of the population risk for two-layer neural networks.Communications in Mathematical Sciences, 17(5):1407–1425, 2019. 39

    2019

  18. [25]

    W. E, C. Ma, and L. Wu. The barron space and the flow-induced function spaces for neural network models.Constructive Approximation, 55:259–292, 2022

    2022

  19. [26]

    W. E and S. Wojtowytsch. Representation formulas and pointwise properties for barron functions.Calculus of Variations and Partial Differential Equations, 61(2):46, 2022

    2022

  20. [27]

    W. E and B. Yu. A priori estimates and analysis of a numerical method for machine learning.Science China Mathematics, 62(11):2303–2330, 2019

    2019

  21. [28]

    El Alaoui and M

    A. El Alaoui and M. W. Mahoney. Fast randomized kernel ridge regression with statis- tical guarantees. InAdvances in Neural Information Processing Systems, volume 28, 2015

    2015

  22. [29]

    Z. Fang, H. Feng, S. Huang, and D.-X. Zhou. Theory of deep convolutional neural networks ii: Spherical analysis.Neural Networks, 131:154–162, 2020

    2020

  23. [30]

    L. Fej´ er. Untersuchungen ¨ uber fouriersche reihen.Mathematische Annalen, 58(1):51– 69, 1903

    1903

  24. [31]

    H. Feng, S. Huang, and D.-X. Zhou. Generalization analysis of CNNs for classification on spheres.IEEE Transactions on Neural Networks and Learning Systems, 2023

    2023

  25. [32]

    Filbir and H

    F. Filbir and H. N. Mhaskar. Marcinkiewicz–zygmund measures on manifolds.Journal of Complexity, 27(6):568–596, 2011

    2011

  26. [33]

    C. F. Gauss. Methodus nova integralium valores per approximationem inveniendi. Commentationes Societatis Regiae Scientiarum Gottingensis Recentiores, 3:29–76, 1814

  27. [34]

    E. H. Georgoulis. Inverse-type estimates onhp-finite element spaces and applications. Mathematics of Computation, 77(261):201–219, 2008

    2008

  28. [35]

    Guo and D.-X

    Z.-C. Guo and D.-X. Zou. Optimal approximation with relu neural networks.Neural Networks, 128:96–109, 2020

    2020

  29. [36]

    Hangelbroek, F

    T. Hangelbroek, F. Narcowich, C. Rieger, and J. Ward. An inverse theorem for compact lipschitz regions in R d using localized kernel bases.Mathematics of Computation, 87(312):1949–1989, 2018

    1949

  30. [37]

    D. Hsu, S. M. Kakade, and T. Zhang. Random design analysis of ridge regression. Foundations of Computational Mathematics, 14(3):569–600, 2014

    2014

  31. [38]

    Huang, Q.-Y

    G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew. Extreme learning machine: Theory and applications.Neurocomputing, 70(1-3):489–501, 2006

    2006

  32. [39]

    Huang, J

    S. Huang, J. Zhou, H. Feng, and D.-X. Zhou. Generalization analysis of pairwise learning for ranking with deep neural networks.Neural Computation, 35(6):1135– 1158, 2023

    2023

  33. [40]

    Ivanov, P

    K. Ivanov, P. Petrushev, and Y. Xu. Sub-exponentially localized kernels and frames induced by orthogonal expansions.Mathematische Zeitschrift, 264(2):361–397, 2010. 40

    2010

  34. [41]

    Jiang, J

    C. Jiang, J. Huang, K. Kashinath, Prabhat, P. Marcus, and M. Niessner. Spherical CNNs on unstructured grids. InInternational Conference on Learning Representations, 2019

    2019

  35. [42]

    L. K. Jones. A simple lemma on greedy approximation in hilbert space and convergence rates for projection pursuit regression and neural network training.The Annals of Statistics, 20(1):608–613, 1992

    1992

  36. [43]

    J. M. Klusowski and A. R. Barron. Approximation by combinations of relu and squared relu ridge functions with l1 and l0 controls.IEEE Transactions on Information Theory, 64(12):7649–7656, 2018

    2018

  37. [44]

    K ˚ urkov´ a and M

    V. K ˚ urkov´ a and M. Sanguineti. Bounds on rates of variable basis and neural network approximation.IEEE Transactions on Information Theory, 47(6):2659–2665, 2001

    2001

  38. [45]

    K ˚ urkov´ a and M

    V. K ˚ urkov´ a and M. Sanguineti. Comparison of worst case errors in linear and neural network approximation.IEEE Transactions on Information Theory, 48(1):264–275, 2002

    2002

  39. [46]

    Q. T. Le Gia and H. N. Mhaskar. Localized linear polynomial operators and quadrature formulas on the sphere.SIAM Journal on Numerical Analysis, 47(1):440–466, 2009

    2009

  40. [47]

    Lewicki and G

    G. Lewicki and G. Marino. Approximation of functions of finite variation by superpo- sitions of a sigmoidal function.Applied Mathematics Letters, 17(10):1147–1152, 2004

    2004

  41. [48]

    Liao and P

    Z. Liao and P. Ming. Deep nitsche method: Weak imposition of dirichlet boundary conditions for neural network solutions of pdes.Journal of Computational Physics, 482:112046, 2023

    2023

  42. [50]

    Maggioni and H

    M. Maggioni and H. N. Mhaskar. Diffusion polynomial frames on metric measure spaces.Applied and Computational Harmonic Analysis, 24(3):329–353, 2008

    2008

  43. [51]

    Magueresse and S

    A. Magueresse and S. Badia. Adaptive quadratures for the accurate integration of neural networks.Computers & Mathematics with Applications, 2024. In press; arXiv:2303.11617

    2024 arXiv

  44. [52]

    V. Maiorov. Pseudo-dimension and entropy of manifolds formed by affine-invariant dictionary.Advances in Computational Mathematics, 25:435–450, 2006

    2006

  45. [53]

    Y. Makovoz. Random approximants and neural networks.Journal of Approximation Theory, 85(1):98–109, 1996

    1996

  46. [54]

    Y. Makovoz. Uniform approximation by neural networks.Journal of Approximation Theory, 95(2):215–228, 1998

    1998

  47. [55]

    T. Mao, J. W. Siegel, and J. Xu. Approximation rates for shallow reluk neural networks on sobolev spaces via the radon transform.arXiv preprint arXiv:2408.10996, 2024. 41

    2024

  48. [56]

    J. Marzo. Marcinkiewicz–Zygmund inequalities and interpolation by spherical har- monics.Journal of Functional Analysis, 250(2):559–587, 2007

    2007

  49. [57]

    B. Maurey. Type et cotype dans les espaces munis de structures locales incondition- nelles.Seminaire Maurey-Schwartz, pages 1–25, 1973

    1973

  50. [58]

    Mhaskar and T

    H. Mhaskar and T. Mao. Tractability of approximation by general shallow networks. arXiv preprint arXiv:2308.03230, 2023

    2023 arXiv

  51. [59]

    Mhaskar, F

    H. Mhaskar, F. Narcowich, and J. Ward. Spherical marcinkiewicz-zygmund inequalities and positive quadrature.Mathematics of computation, 70(235):1113–1130, 2001

    2001

  52. [60]

    H. N. Mhaskar. Approximation properties of a multilayered feedforward artificial neural network.Advances in Computational Mathematics, 1:61–80, 1993

    1993

  53. [61]

    H. N. Mhaskar. Polynomial operators and local smoothness classes on the unit interval. Journal of Approximation Theory, 131(2):243–267, 2004

    2004

  54. [62]

    H. N. Mhaskar. On the representation of smooth functions on the sphere using finitely many bits.Applied and Computational Harmonic Analysis, 18(3):215–233, 2005

    2005

  55. [63]

    H. N. Mhaskar. Weighted quadrature formulas and approximation by zonal function networks on the sphere.Journal of Complexity, 22(3):348–370, 2006

    2006

  56. [64]

    H. N. Mhaskar. Local approximation using hermite functions. InProgress in Approx- imation Theory and Applicable Complex Analysis, pages 341–362. Springer, 2017

    2017

  57. [65]

    H. N. Mhaskar. A unified framework for harmonic analysis of functions on directed graphs and changing data.Applied and Computational Harmonic Analysis, 44(3):611– 644, 2018

    2018

  58. [66]

    H. N. Mhaskar. Function approximation with zonal function networks with activation functions analogous to the rectified linear unit functions.Journal of Complexity, 51:1– 19, 2019

    2019

  59. [67]

    H. N. Mhaskar. Kernel-based analysis of massive data.Frontiers in Applied Mathe- matics and Statistics, 6:30, 2020

    2020

  60. [68]

    H. N. Mhaskar and J. Prestin. On local smoothness classes of periodic functions. Journal of Fourier Analysis and Applications, 11(3):353–373, 2005

    2005

  61. [69]

    F. J. Narcowich, J. D. Ward, and H. Wendland. Sobolev bounds on functions with scattered zeros, with applications to radial basis function surface fitting.Mathematics of Computation, 74(250):743–763, 2005

    2005

  62. [70]

    F. J. Narcowich, J. D. Ward, and H. Wendland. Sobolev error estimates and a bernstein inequality for scattered data interpolation via radial basis functions.Constructive Approximation, 24:175–186, 2006

    2006

  63. [71]

    S. M. Nikolskii.Approximation of Functions of Several Variables and Imbedding The- orems. Springer, Berlin, 1975. 42

    1975

  64. [72]

    Parhi and R

    R. Parhi and R. D. Nowak. Near-minimax optimal estimation with shallow ReLU neural networks.IEEE Transactions on Information Theory, 69(2):1125–1140, 2023

    2023

  65. [73]

    Petrushev and Y

    P. Petrushev and Y. Xu. Localized polynomial frames on the interval with jacobi weights.Journal of Fourier Analysis and Applications, 11:557–575, 2005

    2005

  66. [74]

    P. P. Petrushev. Approximation by ridge functions and neural networks.SIAM Journal on Mathematical Analysis, 30(1):155–189, 1998

    1998

  67. [75]

    A. Pinkus. Approximation theory of the mlp model in neural networks.Acta numerica, 8:143–195, 1999

    1999

  68. [76]

    G. Pisier. Remarques sur un r´ esultat non publi´ e de B. Maurey.S´ eminaire d’Analyse fonctionnelle (dit” Maurey-Schwartz”), pages 1–12, 1981

    1981

  69. [77]

    Raissi, P

    M. Raissi, P. Perdikaris, and G. E. Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.Journal of Computational Physics, 378:686–707, 2019

    2019

  70. [78]

    J. A. Rivera, J. M. Taylor, ´A. J. Omella, and D. Pardo. On quadrature rules for solving pdes using neural networks.Computer Methods in Applied Mechanics and Engineering, 398:115261, 2022

    2022

  71. [79]

    Rudi and L

    A. Rudi and L. Rosasco. Generalization properties of learning with random features. InAdvances in Neural Information Processing Systems, volume 30, pages 3215–3225, 2017

    2017

  72. [80]

    Schneider

    R. Schneider. Zu einem problem von shephard ¨ uber die projektionen konvexer k¨ orper. Mathematische Zeitschrift, 101:71–82, 1967

    1967

  73. [81]

    J. W. Siegel and J. Xu. High-order approximation rates for shallow neural networks with cosine and ReLUk activation functions.Applied and Computational Harmonic Analysis, 58:1–26, 2022

    2022

  74. [82]

    J. W. Siegel and J. Xu. Optimal convergence rates for the orthogonal greedy algorithm. IEEE Transactions on Information Theory, 68(5):3354–3361, 2022

    2022

  75. [83]

    J. W. Siegel and J. Xu. Sharp bounds on the approximation rates, metric entropy, and n-widths of shallow neural networks.Foundations of Computational Mathematics, pages 1–57, 2022

    2022

  76. [84]

    J. W. Siegel and J. Xu. Characterization of the variation spaces corresponding to shallow neural networks.Constructive Approximation, 57(3):1109–1132, 2023

    2023

  77. [85]

    E. M. Stein.Singular integrals and differentiability properties of functions. Princeton university press, 1970

    1970

  78. [86]

    G. Strang. Variational crimes in the finite element method.Mathematical Aspects of the Finite Element Method, pages 689–710, 1972

    1972

  79. [87]

    Szeg¨ o.Orthogonal polynomials, volume 23 ofAmer

    G. Szeg¨ o.Orthogonal polynomials, volume 23 ofAmer. Math. Soc. Colloq. Publ.Amer. Math. Soc., Providence, 1975. 43

    1975

  80. [88]

    J. M. Taylor, J. A. Rivera, and D. Pardo. Stochastic quadrature for deep ritz methods. arXiv preprint arXiv:2501.01234, 2025

    2025 arXiv

  81. [89]

    V. N. Temlyakov. Greedy approximation.Acta Numerica, 17:235–409, 2008

    2008

  82. [90]

    J. A. Tropp. User-friendly tail bounds for sums of random matrices.Foundations of Computational Mathematics, 12(4):389–434, 2012

    2012

  83. [91]

    Vershynin.High-Dimensional Probability: An Introduction with Applications in Data Science, volume 47 ofCambridge Series in Statistical and Probabilistic Mathe- matics

    R. Vershynin.High-Dimensional Probability: An Introduction with Applications in Data Science, volume 47 ofCambridge Series in Statistical and Probabilistic Mathe- matics. Cambridge University Press, Cambridge, 2018

    2018

  84. [92]

    M. J. Wainwright.High-dimensional statistics: A non-asymptotic viewpoint, vol- ume 48. Cambridge university press, 2019

    2019

  85. [93]

    Wang and X

    W. Wang and X. Xie. Tensor neural network and its numerical integration.Journal of Computational Mathematics, 42(4):613–637, 2023

    2023

  86. [94]

    Wang and S

    Y. Wang and S. Dong. An extreme learning machine-based method for computational pdes in higher dimensions.Computer Methods in Applied Mechanics and Engineering, 418:116578, 2024

    2024

  87. [95]

    Y.-L. Wang, H. Li, and P. Li. Inverse approximation theorems of Leaky ReLU neural network classifier.arXiv preprint arXiv:2310.02116, 2023

    2023 arXiv

  88. [96]

    J. Xu. Finite neuron method and convergence analysis.Communications in Compu- tational Physics, 28(5):1707–1745, 2020

    2020

  89. [97]

    Y. Yang, S. Huang, H. Feng, and D.-X. Zhou. Spherical analysis of learning nonlinear functionals.Journal of Machine Learning Research, 2026

    2026

  90. [98]

    Yang and D.-X

    Y. Yang and D.-X. Zhou. Optimal rates of approximation by shallow relu k neural networks and applications to nonparametric regression.Constructive Approximation, pages 1–32, 2024

    2024

  91. [99]

    Zhou and K

    D.-X. Zhou and K. Jetter. Approximation with polynomial kernels and svm classifiers. Advances in Computational Mathematics, 25(1):323–344, 2006. 44

    2006

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.