Banach-valued random feature models, including random single-hidden-layer networks, universally approximate elements of Bochner spaces over non-compact domains with explicit approximation rates.
Risk Bounds for High-dimensional Ridge Function Combinations Including Neural Networks
4 Pith papers cite this work. Polarity classification is still indexing.
4
Pith papers citing it
abstract
Let $ f^{\star} $ be a function on $ \mathbb{R}^d $ with an assumption of a spectral norm $ v_{f^{\star}} $. For various noise settings, we show that $ \mathbb{E}\|\hat{f} - f^{\star} \|^2 \leq \left(v^4_{f^{\star}}\frac{\log d}{n}\right)^{1/3} $, where $ n $ is the sample size and $ \hat{f} $ is either a penalized least squares estimator or a greedily obtained version of such using linear combinations of sinusoidal, sigmoidal, ramp, ramp-squared or other smooth ridge functions. The candidate fits may be chosen from a continuum of functions, thus avoiding the rigidity of discretizations of the parameter space. On the other hand, if the candidate fits are chosen from a discretization, we show that $ \mathbb{E}\|\hat{f} - f^{\star} \|^2 \leq \left(v^3_{f^{\star}}\frac{\log d}{n}\right)^{2/5} $. This work bridges non-linear and non-parametric function estimation and includes single-hidden layer nets. Unlike past theory for such settings, our bound shows that the risk is small even when the input dimension $ d $ of an infinite-dimensional parameterized dictionary is much larger than the available sample size. When the dimension is larger than the cube root of the sample size, this quantity is seen to improve the more familiar risk bound of $ v_{f^{\star}}\left(\frac{d\log (n/d)}{n}\right)^{1/2} $, also investigated here.
verdicts
UNVERDICTED 4representative citing papers
A hybrid FEM and ELM framework for parameter-dependent PDEs derives existence, uniqueness, regularity, and error estimates for inverse problems in photoacoustic tomography.
Randomized neural networks require a sampling domain sized to target smoothness for optimal approximation, and an adaptive PIRaNN method with partition-of-unity refinement solves PDEs with limited local regularity.
A survey summarizing classical density results and quantitative approximation theory for feedforward networks and KANs, with emphasis on depth advantages.
citing papers explorer
-
Universal approximation property of Banach space-valued random feature models including random neural networks
Banach-valued random feature models, including random single-hidden-layer networks, universally approximate elements of Bochner spaces over non-compact domains with explicit approximation rates.
-
Solving Inverse Parametrized Problems via Finite Elements and Extreme Learning Networks
A hybrid FEM and ELM framework for parameter-dependent PDEs derives existence, uniqueness, regularity, and error estimates for inverse problems in photoacoustic tomography.
-
Adaptive Randomized Neural Networks with Locally Activation Function: Theory and Algorithm for Solving PDEs
Randomized neural networks require a sampling domain sized to target smoothness for optimal approximation, and an adaptive PIRaNN method with partition-of-unity refinement solves PDEs with limited local regularity.
-
Approximation Theory for Neural Networks: Old and New
A survey summarizing classical density results and quantitative approximation theory for feedforward networks and KANs, with emphasis on depth advantages.