Pith. sign in

REVIEW 2 cited by

Approximation Bounds for Transformer Networks with Application to Regression

T0 review · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fixed-depth Transformers with softmax attention approximate Hölder and Sobolev sequence functions at parameter rates epsilon^{-dxn/gamma} and epsilon^{-dxn}, and yield regression rates under beta-mixing data.

arxiv 2504.12175 v1 pith:JQ3XOKAU submitted 2025-04-16 stat.ML cs.LG

Yuling Jiao , Yanming Lai , Defeng Sun , Yang Wang , Bokai Yan This is my paper · ORCID
classification stat.MLcs.LG
keywords boundstransformerapproximationfunctionsnetworksolderregressionestablish
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper studies how many parameters a Transformer needs to approximate a smooth function that maps a sequence of n vectors in R^{dx} to another such sequence. For functions whose entries are Hölder continuous with smoothness gamma, the authors show that a fixed-depth Transformer with about epsilon^{-dxn/gamma} parameters can approximate the map to within epsilon in any Lp norm, including the uniform norm. The construction partitions the input cube into small cells, records the target values at grid points, and uses the attention layer to look up the correct cell. A key ingredient is the known fact that a single attention head can act as a contextual mapping, assigning distinct outputs to distinct cell identifiers. For the uniform norm, an extra horizontal shift step repairs errors near cell boundaries. The same approach yields epsilon^{-dxn} rates for Sobolev functions. The second part turns these approximation bounds into statistical guarantees for nonparametric regression with dependent observations. Using sliding windows from a stationary beta-mixing process, the empirical risk minimizer over Transformer classes achieves rates m^{-gamma/(gamma+dxn)} for geometric mixing and i.i.d. data, and a slower rate for algebraic mixing. The authors note these rates are suboptimal by a polynomial factor because the known VC dimension bounds for softmax networks grow quadratically in the parameter count. A final section gives a different proof strategy based on the Kolmogorov-Arnold representation theorem, but only for a generalized architecture where feed-forward layers may use per-token biases.
Extended reading notes

Core claim

Theorem 1: For a sequence-to-sequence map F on [0,1]^{dx by n} whose entries are gamma-Hölder, there is a fixed-depth standard Transformer with O(epsilon^{-dxn/gamma}) parameters that approximates F to within 4(dxn)^2 K_H epsilon in Lp for every p in [1, infinity], including p = infinity. If true, this gives the same parameter efficiency as fixed-depth FNNs and RNNs, for the original softmax attention architecture.

Load-bearing premise

Proposition 7 Step 3 invokes [33, Theorem 2] to assert that a single-head, single-size softmax self-attention layer can serve as a contextual mapping on the finite set of grid sequences {G+P}. This theorem is external to the paper and is the linchpin of the standard-architecture construction. If the contextual mapping requires extra conditions beyond distinctness of columns (for example, restrictions on the number of sequences or on norms of the embedded tokens), the fixed-depth width bound in Theorem 1 would need modification. The paper does not restate the theorem's hypotheses or prove it.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central results are proven from stated assumptions using external theorems from [33], [39], [51], and [2]. No free parameters are fitted to data. The dependence on [32] is self-referential but concerns a standard decomposition rather than the target rates.

assumptions (5)
  • domain assumption Contextual mapping theorem for one-layer softmax attention ([33, Theorem 2])
    Used in Proposition 7 Step 3 to map each grid cell to a unique token sequence; not proved in the paper.
  • standard math Mid-function ReLU representation and horizontal-shift lemmas ([39, Lemmas 3.1 and 3.3])
    Used in Proposition 8 to extend uniform approximation from outside the trifling region to the whole cube; adopted without proof.
  • standard math Kolmogorov-Arnold representation variant with Hölder transfer ([51, Theorem 2])
    Used in Theorem 4 to factor Hölder functions through a one-dimensional outer function.
  • domain assumption Excess risk decomposition for beta-mixing processes ([32, Theorem 5])
    Used in Theorem 3 to split excess risk into approximation, generalization, and dependence terms; cited from a prior paper by overlapping authors.
  • standard math VC dimension bound for algorithms with exponentials and arithmetic ([2, Theorem 8.14])
    Used in Lemma 13 to bound VC and pseudo-dimension of Transformer classes.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Approximation Bounds for Transformer Networks with Application to Regression." pith.science (2026). https://pith.science/paper/JQ3XOKAU

@misc{pith2026250412175,
  author       = {Pith},
  title        = {Pith review of: Approximation Bounds for Transformer Networks with Application to Regression},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JQ3XOKAU}},
  note         = {Machine review of arXiv:2504.12175}
}
abstract

We explore the approximation capabilities of Transformer networks for H\"older and Sobolev functions, and apply these results to address nonparametric regression estimation with dependent observations. First, we establish novel upper bounds for standard Transformer networks approximating sequence-to-sequence mappings whose component functions are H\"older continuous with smoothness index $\gamma \in (0,1]$. To achieve an approximation error $\varepsilon$ under the $L^p$-norm for $p \in [1, \infty]$, it suffices to use a fixed-depth Transformer network whose total number of parameters scales as $\varepsilon^{-d_x n / \gamma}$. This result not only extends existing findings to include the case $p = \infty$, but also matches the best known upper bounds on number of parameters previously obtained for fixed-depth FNNs and RNNs. Similar bounds are also derived for Sobolev functions. Second, we derive explicit convergence rates for the nonparametric regression problem under various $\beta$-mixing data assumptions, which allow the dependence between observations to weaken over time. Our bounds on the sample complexity impose no constraints on weight magnitudes. Lastly, we propose a novel proof strategy to establish approximation bounds, inspired by the Kolmogorov-Arnold representation theorem. We show that if the self-attention layer in a Transformer can perform column averaging, the network can approximate sequence-to-sequence H\"older functions, offering new insights into the interpretability of self-attention mechanisms.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. On Explicit Super-Expressive Approximation for Neural Networks

    cs.LG 2026-07 accept novelty 7.0 of 10

    Fixed-architecture networks of width O(D) and depth O(r) approximate Hölder functions with parameter magnitude log P = O(ε^{-2D/(r+γ)} log(1/ε)) via CRT encoding.

  2. Standard Transformers Achieve the Minimax Rate in Nonparametric Regression with $C^{s,\lambda}$ Targets

    stat.ML 2026-02 conditional novelty 5.0 of 10

    Standard Transformers attain the minimax optimal rate m^{-2γ/(2γ+dn)} (up to logs) for nonparametric regression of Hölder C^{s,λ} targets on [0,1]^{d×n}.

Reference graph

Works this paper leans on

71 extracted references · 62 canonical work pages · cited by 2 Pith papers

  1. [1]

    The generalization abili ty of online algorithms for dependent data

    Alekh Agarwal and John C Duchi. The generalization abili ty of online algorithms for dependent data. IEEE Transactions on Information Theory , 59(1):573–587, 2012

    work page 2012

  2. [2]

    Bartlett

    Martin Anthony and Peter L. Bartlett. Neural Network Learning: Theoretical Foundations . Cambridge University Press, 1999. 35

    work page 1999

  3. [3]

    Nearly-tight vc- dimension and pseudodimension bounds for piecewise linear neural networks

    Peter L Bartlett, Nick Harvey, Christopher Liaw, and Abba s Mehrabian. Nearly-tight vc- dimension and pseudodimension bounds for piecewise linear neural networks. Journal of Machine Learning Research, 20(63):1–17, 2019

    work page 2019

  4. [4]

    Vapnik-chervonenki s dimension of neural nets

    Peter L Bartlett and Wolfgang Maass. Vapnik-chervonenki s dimension of neural nets. The handbook of brain theory and neural networks , pages 1188–1192, 2003

    work page 2003

  5. [5]

    Birth of a transformer: A memory viewpoint

    Alberto Bietti, Vivien Cabannes, Diane Bouchacourt, Herv e Jegou, and Leon Bottou. Birth of a transformer: A memory viewpoint. Advances in Neural Information Processing Systems, 2023

    work page 2023

  6. [6]

    Nonparametric regression on low-dimensional manifolds using deep relu networks: Funct ion approximation and statistical recovery

    Minshuo Chen, Haoming Jiang, Wenjing Liao, and Tuo Zhao. Nonparametric regression on low-dimensional manifolds using deep relu networks: Funct ion approximation and statistical recovery. Information and Inference: A Journal of the IMA , 11(4):1203–1253, 2022

    work page 2022

  7. [7]

    Overcoming a theoretical limitation of self-attention

    David Chiang and Peter Cholak. Overcoming a theoretical limitation of self-attention. In Proceedings of the 60th Annual Meeting of the Association fo r Computational Linguistics (Volume 1: Long Papers) . Association for Computational Linguistics, 2022

    work page 2022

  8. [8]

    Approximation by superpositions of a si gmoidal function

    George Cybenko. Approximation by superpositions of a si gmoidal function. Mathematics of control, signals and systems , 2(4):303–314, 1989

    work page 1989

Show all 71 references
  1. [9]

    The bramble–hilbert lemma for convex domains

    Shai Dekel and Dany Leviatan. The bramble–hilbert lemma for convex domains. SIAM journal on mathematical analysis , 35(5):1203–1212, 2004

    2004

  2. [10]

    BERT: Pre-training of deep bidirectional transformers for language understan ding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understan ding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lang...

    2019

  3. [11]

    Nonlinear approximation

    Ronald A DeVore. Nonlinear approximation. Acta Numerica, 7:51–150, 1998

    1998

  4. [12]

    Constructive Approximation, volume 303

    Ronald A DeVore and George G Lorentz. Constructive Approximation, volume 303. Springer Science & Business Media, 1993

    1993

  5. [13]

    Semi-supervised deep sobolev regression: Estimation and variable selection by r equ neural network

    Zhao Ding, Chenguang Duan, Yuling Jiao, and Jerry Zhiji an Yang. Semi-supervised deep sobolev regression: Estimation and variable selection by r equ neural network. IEEE Trans- actions on Information Theory , 2025

    2025

  6. [14]

    Minimax estimation via wavelet shrinkage

    David L Donoho and Iain M Johnstone. Minimax estimation via wavelet shrinkage. The Annals of Statistics , 26(3):879–921, 1998

    1998

  7. [15]

    An image is worth 16x16 wo rds: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov , Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 wo rds: Transformers for image recognition ...

    2021

  8. [16]

    Inductive biases and variable creation in self-attention mechanisms

    Benjamin L Edelman, Surbhi Goel, Sham Kakade, and Cyril Z hang. Inductive biases and variable creation in self-attention mechanisms. In International Conference on Machine Learning. PMLR, 2022

    2022

  9. [17]

    Partial Differential Equations , volume 19

    Lawrence C Evans. Partial Differential Equations , volume 19. American Mathematical Soc., 2010

    2010

  10. [18]

    Attention enables zero approximation error

    Zhiying Fang, Yidong Ouyang, Ding-Xuan Zhou, and Guang Cheng. Attention enables zero approximation error. arXiv preprint arXiv:2202.12166 , 2022. 36

    2022 arXiv

  11. [19]

    Deep ne ural networks for estimation and inference

    Max H Farrell, Tengyuan Liang, and Sanjog Misra. Deep ne ural networks for estimation and inference. Econometrica, 89(1):181–213, 2021

    2021

  12. [20]

    Over-parameterized deep nonparametric regression for dependent data with its a pplications to reinforcement learning

    Xingdong Feng, Yuling Jiao, Lican Kang, Baqun Zhang, and Fan Zhou. Over-parameterized deep nonparametric regression for dependent data with its a pplications to reinforcement learning. Journal of Machine Learning Research , 24(383):1–40, 2023

    2023

  13. [21]

    On the rate of convergence of a classifier based on a transformer encoder

    Iryna Gurevych, Michael Kohler, and Gözde Gül Şahin. On the rate of convergence of a classifier based on a transformer encoder. IEEE Transactions on Information Theory , 68(12):8139–8155, 2022

    2022

  14. [22]

    Understanding sc aling laws with statistical and approximation theory for transformer neural networks on in trinsically low-dimensional data

    Alexander Havrilla and Wenjing Liao. Understanding sc aling laws with statistical and approximation theory for transformer neural networks on in trinsically low-dimensional data. Advances in Neural Information Processing Systems , 2024

    2024

  15. [23]

    D eep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. D eep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pa ttern recognition, pages 770–778, 2016

    2016

  16. [24]

    Minimal width for universal property of deep rnn

    Chang hoon Song, Geonho Hwang, Jun ho Lee, and Myungjoo K ang. Minimal width for universal property of deep rnn. Journal of Machine Learning Research , 24(121):1–41, 2023

    2023

  17. [25]

    M ultilayer feedforward networks are universal approximators

    Kurt Hornik, Maxwell Stinchcombe, and Halbert White. M ultilayer feedforward networks are universal approximators. Neural Networks, 2(5):359–366, 1989

    1989

  18. [26]

    Mixing time estimation in re- versible markov chains from a single sample path

    Daniel J Hsu, Aryeh Kontorovich, and Csaba Szepesvári. Mixing time estimation in re- versible markov chains from a single sample path. Advances in neural information processing systems, 2015

    2015

  19. [27]

    Fundamental limits of prompt tuning transformers: Univers ality, capacity and efficiency

    Jerry Yao-Chieh Hu, Wei-Po Wang, Ammar Gilani, Chenyan g Li, Zhao Song, and Han Liu. Fundamental limits of prompt tuning transformers: Univers ality, capacity and efficiency. In International Conference on Learning Representations , 2025

    2025

  20. [28]

    Approximation rate of th e transformer architecture for sequence modeling

    Haotian Jiang and Qianxiao Li. Approximation rate of th e transformer architecture for sequence modeling. Advances in Neural Information Processing Systems , 2024

    2024

  21. [29]

    Deep approximate policy iteration

    Yuling Jiao, Lican Kang, Jin Liu, Xiliang Lu, and Jerry Z hijian Yang. Deep approximate policy iteration. Annals of Statistics , 2025

    2025

  22. [30]

    Conve rgence analysis of flow match- ing in latent space with transformers

    Yuling Jiao, Yanming Lai, Yang Wang, and Bokai Yan. Conve rgence analysis of flow match- ing in latent space with transformers. arXiv preprint arXiv:2404.02538 , 2024

    2024 arXiv

  23. [31]

    Deep nonparametric regression on approximate manifolds: Nonasymptotic error bounds with polynomial prefactors

    Yuling Jiao, Guohao Shen, Yuanyuan Lin, and Jian Huang. Deep nonparametric regression on approximate manifolds: Nonasymptotic error bounds with polynomial prefactors. The Annals of Statistics , 51(2):691–716, 2023

    2023

  24. [32]

    Approximation bou nds for recurrent neural networks with application to regression

    Yuling Jiao, Yang Wang, and Bokai Yan. Approximation bou nds for recurrent neural networks with application to regression. arXiv preprint arXiv:2409.05577 , 2024

    2024

  25. [33]

    Are transformers with o ne layer self-attention using low- rank weight matrices universal approximators? In International Conference on Learning Representations, 2024

    Tokio Kajitsuka and Issei Sato. Are transformers with o ne layer self-attention using low- rank weight matrices universal approximators? In International Conference on Learning Representations, 2024

    2024

  26. [34]

    On the optimal memoriza tion capacity of transformers

    Tokio Kajitsuka and Issei Sato. On the optimal memoriza tion capacity of transformers. In International Conference on Learning Representations , 2025. 37

    2025

  27. [35]

    Polynomial bound s for vc dimension of sigmoidal and general pfaffian neural networks

    Marek Karpinski and Angus Macintyre. Polynomial bound s for vc dimension of sigmoidal and general pfaffian neural networks. Journal of Computer and System Sciences , 54(1):169– 176, 1997

    1997

  28. [36]

    Provab le memorization capacity of transformers

    Junghwan Kim, Michelle Kim, and Barzan Mozafari. Provab le memorization capacity of transformers. In International Conference on Learning Representations , 2023

    2023

  29. [37]

    On the rate of converg ence of fully connected deep neural network regression estimates

    Michael Kohler and Sophie Langer. On the rate of converg ence of fully connected deep neural network regression estimates. The Annals of Statistics , 49(4):2231–2249, 2021

    2021

  30. [38]

    Generalization bo unds for non-stationary mixing processes

    Vitaly Kuznetsov and Mehryar Mohri. Generalization bo unds for non-stationary mixing processes. Machine Learning, 106(1):93–117, 2017

    2017

  31. [39]

    Deep network approximation for smooth functions

    Jianfeng Lu, Zuowei Shen, Haizhao Yang, and Shijun Zhan g. Deep network approximation for smooth functions. SIAM Journal on Mathematical Analysis , 53(5):5465–5506, 2021

    2021

  32. [40]

    Nonparametric time series prediction throug h adaptive model selection

    Ron Meir. Nonparametric time series prediction throug h adaptive model selection. Machine Learning, 39:5–34, 2000

    2000

  33. [41]

    Rademacher com plexity bounds for non-iid processes

    Mehryar Mohri and Afshin Rostamizadeh. Rademacher com plexity bounds for non-iid processes. In Advances in Neural Information Processing Systems , 2008

    2008

  34. [42]

    Stability boun ds for stationary ϕ -mixing and β -mixing processes

    Mehryar Mohri and Afshin Rostamizadeh. Stability boun ds for stationary ϕ -mixing and β -mixing processes. Journal of Machine Learning Research , 11(2), 2010

    2010

  35. [43]

    Adaptive approxim ation and generalization of deep neural network with intrinsic dimensionality

    Ryumei Nakada and Masaaki Imaizumi. Adaptive approxim ation and generalization of deep neural network with intrinsic dimensionality. Journal of Machine Learning Research , 21(174):1–38, 2020

    2020

  36. [44]

    GPT-4 technical report

    OpenAI. GPT-4 technical report. arXiv preprint arXiv:2303.08774 , 2023

    2023 arXiv

  37. [45]

    Pro vable memorization via deep neural networks using sub-linear parameters

    Sejun Park, Jaeho Lee, Chulhee Yun, and Jinwoo Shin. Pro vable memorization via deep neural networks using sub-linear parameters. In Conference on learning theory . PMLR, 2021

    2021

  38. [46]

    Scalable diffusion mod els with transformers

    William Peebles and Saining Xie. Scalable diffusion mod els with transformers. In Proceed- ings of the IEEE/CVF International Conference on Computer V ision, pages 4195–4205, 2023

    2023

  39. [47]

    Atte ntion is turing-complete

    Jorge Pérez, Pablo Barceló, and Javier Marinkovic. Atte ntion is turing-complete. Journal of Machine Learning Research , 22(75):1–35, 2021

    2021

  40. [48]

    Promptin g a pretrained transformer can be a universal approximator

    Aleksandar Petrov, Philip Torr, and Adel Bibi. Promptin g a pretrained transformer can be a universal approximator. In International Conference on Machine Learning . PMLR, 2024

    2024

  41. [49]

    S tatistical spatially inhomoge- neous diffusion inference

    Yinuo Ren, Yiping Lu, Lexing Ying, and Grant M Rotskoff. S tatistical spatially inhomoge- neous diffusion inference. In Proceedings of the AAAI Conference on Artificial Intelligen ce, 2024

    2024

  42. [50]

    Nonparametric regression u sing deep neural networks with relu activation function

    Johannes Schmidt-Hieber. Nonparametric regression u sing deep neural networks with relu activation function. The Annals of Statistics , 48(4):1875, 2020

    2020

  43. [51]

    The kolmogorov–arnold repr esentation theorem revisited

    Johannes Schmidt-Hieber. The kolmogorov–arnold repr esentation theorem revisited. Neural Networks, 137:119–126, 2021

    2021

  44. [52]

    Predictive pac le arning and process decompositions

    Cosma Shalizi and Aryeh Kontorovich. Predictive pac le arning and process decompositions. In Advances in Neural Information Processing Systems , 2013. 38

    2013

  45. [53]

    Deep netwo rk approximation characterized by number of neurons

    Zuowei Shen, Haizhao Yang, and Shijun Zhang. Deep netwo rk approximation characterized by number of neurons. Communications in Computational Physics , 28(5):1768–1811, 2020

    2020

  46. [54]

    Optimal approximation rates for dee p relu neural networks on sobolev and besov spaces

    Jonathan W Siegel. Optimal approximation rates for dee p relu neural networks on sobolev and besov spaces. Journal of Machine Learning Research , 24(357):1–52, 2023

    2023

  47. [55]

    Fast learning f rom non-iid observations

    Ingo Steinwart and Andreas Christmann. Fast learning f rom non-iid observations. In Advances in Neural Information Processing Systems , 2009

    2009

  48. [56]

    Optimal global rates of convergence fo r nonparametric regression

    Charles J Stone. Optimal global rates of convergence fo r nonparametric regression. The Annals of Statistics , 10(4):1040–1053, 1982

    1982

  49. [57]

    Adaptivity of deep reLU network for learn ing in besov and mixed smooth besov spaces: optimal rate and curse of dimensionality

    Taiji Suzuki. Adaptivity of deep reLU network for learn ing in besov and mixed smooth besov spaces: optimal rate and curse of dimensionality. In International Conference on Learning Representations, 2019

    2019

  50. [58]

    Approximation and estimation ability of transform- ers for sequence-to-sequence functions with infinite dimen sional input

    Shokichi Takakura and Taiji Suzuki. Approximation and estimation ability of transform- ers for sequence-to-sequence functions with infinite dimen sional input. In International Conference on Machine Learning . PMLR, 2023

    2023

  51. [59]

    Approximation o f permutation invariant polynomi- als by transformers: Efficient construction in column-size

    Naoki Takeshita and Masaaki Imaizumi. Approximation o f permutation invariant polynomi- als by transformers: Efficient construction in column-size. arXiv preprint arXiv:2502.11467 , 2025

    2025 arXiv

  52. [60]

    Attention is all you ne ed

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you ne ed. In Advances in Neural Information Processing Systems , 2017

    2017

  53. [61]

    Inequalities for absolutely regul ar sequences: application to density estimation

    Gabrielle Viennet. Inequalities for absolutely regul ar sequences: application to density estimation. Probability Theory and Related Fields , 107:467–492, 1997

    1997

  54. [62]

    Understanding the expressive power and mechanisms of transformer for sequence modeling

    Mingze Wang and Weinan E. Understanding the expressive power and mechanisms of transformer for sequence modeling. In Advances in Neural Information Processing Systems , 2024

    2024

  55. [63]

    On expressive power of looped tr ansformers: Theoretical analysis and enhancement via timestep encoding

    Kevin Xu and Issei Sato. On expressive power of looped tr ansformers: Theoretical analysis and enhancement via timestep encoding. arXiv preprint arXiv:2410.01405 , 2024

    2024 arXiv

  56. [64]

    Nonparametric regress ion using over-parameterized shallow relu neural networks

    Yunfei Yang and Ding-Xuan Zhou. Nonparametric regress ion using over-parameterized shallow relu neural networks. Journal of Machine Learning Research , 25(165):1–35, 2024

    2024

  57. [65]

    Optimal rates of approx imation by shallow relu k neural networks and applications to nonparametric regression

    Yunfei Yang and Ding-Xuan Zhou. Optimal rates of approx imation by shallow relu k neural networks and applications to nonparametric regression. Constructive Approximation, pages 1–32, 2024

    2024

  58. [66]

    Error bounds for approximations with deep relu networks

    Dmitry Yarotsky. Error bounds for approximations with deep relu networks. Neural Net- works, 94:103–114, 2017

    2017

  59. [67]

    Density estimation in the l8 norm for dependent data with applications to the gibbs sampler

    Bin Yu. Density estimation in the l8 norm for dependent data with applications to the gibbs sampler. The Annals of Statistics , 21(2):711–735, 1993

    1993

  60. [68]

    Rates of convergence for empirical processes of s tationary mixing sequences

    Bin Yu. Rates of convergence for empirical processes of s tationary mixing sequences. The Annals of Probability , 22(1):94–116, 1994

    1994

  61. [69]

    Are transformers universal approximators of sequence -to-sequence functions? In International Conference on Learning Representations , 2019

    Chulhee Yun, Srinadh Bhojanapalli, Ankit Singh Rawat, S ashank Reddi, and Sanjiv Ku- mar. Are transformers universal approximators of sequence -to-sequence functions? In International Conference on Learning Representations , 2019. 39

    2019

  62. [70]

    O(n) connections are expressive enough: U niversal approximability of sparse transformers

    Chulhee Yun, Yin-Wen Chang, Srinadh Bhojanapalli, Anki t Singh Rawat, Sashank Reddi, and Sanjiv Kumar. O(n) connections are expressive enough: U niversal approximability of sparse transformers. Advances in Neural Information Processing Systems , 2020

    2020

  63. [71]

    Deep netwo rk approximation: Achieving arbitrary accuracy with fixed number of neurons

    Shijun Zhang, Zuowei Shen, and Haizhao Yang. Deep netwo rk approximation: Achieving arbitrary accuracy with fixed number of neurons. Journal of Machine Learning Research , 23(276):1–60, 2022. 40

    2022

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.