Pith. sign in

REVIEW 3 major objections 4 minor 72 references

Optimality and Adaptivity of Deep Neural Features for Instrumental Variable Regression

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A two-stage estimator that learns its features with deep neural networks attains the nearly minimax optimal convergence rate for nonparametric instrumental variable regression over Besov function classes, needing only as many Stage 1 as…

desk verdict First minimax-rate analysis of DFIV with a genuinely clever dynamic-cover proof, but the headline optimality claim rests on a maximal-smoothness example that is not a well-defined linear operator. read the letter →

arxiv 2501.04898 v1 pith:6CNTBNZE submitted 2025-01-09 stat.ML cs.LG

Juno Kim , Dimitri Meunier , Arthur Gretton , Taiji Suzuki , Zhu Li This is my paper · ORCID
classification stat.MLcs.LG MSC 62G0562G0862G2068T07
keywords instrumentalvariableregressiondeepneuralnetworksBesovspacesminimaxoptimalitytwo-stageestimationnonparametricadaptivityconditionalexpectationoperator
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proves that DFIV, a two-stage instrumental variable regression method that learns its features with deep neural networks, attains the statistically optimal convergence rate for nonparametric IV regression when the structural function lies in a Besov space, a class of generalized smoothness that can mix smooth and spiky or discontinuous regions. The proof shows that the regularity of the conditional distribution of the covariate given the instrument controls the difficulty of the first stage, and that when the projection operator has "maximal smoothness" the estimator reaches the information-theoretic limit. The paper also establishes two concrete advantages over fixed-feature (kernel or sieve) IV estimators: DFIV keeps the optimal rate on functions with low spatial homogeneity, where fixed features are provably suboptimal, and it requires only as many Stage 1 as Stage 2 samples, whereas kernel methods need far more of the former. If correct, these results give the first minimax guarantees for a deep-feature two-stage IV estimator and a rigorous explanation of its practical edge.

What carries the argument

The load-bearing object is the conditional expectation operator $T$, which maps the structural function to its projection $E[f_{\mathrm{str}}(X)|Z]$; its degree of contraction is quantified by link conditions with exponents $\gamma_1$ and $\gamma_0$ (Assumptions 3 and 5), and its regularity by a smoothness assumption on the image $T[U(B^s_{p,q}(X))]$ (Assumption 4). The identity that carries the argument is the maximal-smoothness equality $s'/d_z = (s+\gamma_0)/d_x$ of Lemma 3.2, which makes the Stage 1 smoothness $s'$ exactly match what the minimax lower bound requires; the paper notes that a strict inequality makes DFIV suboptimal. The technical centerpiece is a dynamic extended cover argument: because the Stage 1 estimator is a random function of the Stage 2 parameter, the proof constructs a data-dependent cover of the product of the two DNN classes and bounds the supremal Stage 1 error over it, producing the two-stage oracle bound that separates approximation from estimation error. Supporting this is a new approximation theorem showing sigmoid networks achieve the optimal Besov rate $N^{-s/d_x}$ and stay bounded in Besov norm, which is what keeps the Stage 2 network smooth enough for the reverse link condition to apply.

What would settle it

Simulate NPIV problems with a conditional density of $X$ given $Z$ constructed so that the inequality in Lemma 3.2 is strict, for instance by making $T$ smooth in $z$ at an exponent $s' < d_z(s+\gamma_0)/d_x$, and estimate DFIV's empirical $L^2$ rate with $m = n$ as $n$ grows; the theory predicts a rate strictly slower than $n^{-2s/(2s+2\gamma_1+d_x)}$. Conversely, a simulation with $T$ acting on B-splines as $2^{-\gamma_0 k}$, which saturates the equality, should reproduce the claimed near-optimal rate with an equal sample split, while a kernel IV estimator on the same data should show the strictly slower fixed-feature rate.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is a rate equality: under standard ill-posed inverse-problem assumptions, the DFIV estimator achieves the nearly minimax optimal mean squared error $n^{-2s/(2s+2\gamma_1+d_x)} (\log n)^{s/(s+\gamma_1)}$ for structural functions in a Besov space $B^s_{p,q}$, provided the Stage 1 and Stage 2 sample sizes scale as $m = \Omega(n)$, the link and reverse link exponents coincide ($\gamma_0 = \gamma_1$), and the projection operator $T$ has maximal smoothness in the sense of Lemma 3.2. The proof yields a general two-stage oracle bound in which the projected error separates into a Stage 2 approximation term, a supremal Stage 1 approximation term, and covering-entropy terms, and the Stage 1 term is tamed by a new dynamic-cover argument that handles the data-dependent first-stage estimator. Along the way the paper proves approximation guarantees for sigmoid networks that are rate-optimal in $L^2$ and also converge in Besov norm, which is what lets the smoothness of the Stage 2 network be controlled. In the regime $p < 2$, where the target is spatially inhomogeneous, the paper proves a strict separation: every linear (fixed-feature) IV estimator is lower-bounded by a strictly worse rate, while DFIV retains the optimal rate under an extended reverse link condition.

Load-bearing premise

The whole optimality conclusion rests on the projection operator $T$ having maximal smoothness, meaning the regularity of the conditional distribution of the covariate given the instrument lands exactly on the equality $s'/d_z = (s+\gamma_0)/d_x$ rather than below it; if the inequality is strict, the paper's own results concede that the DFIV rate is suboptimal.

Editorial extensions

If this is right

  • With a balanced split $m = \Omega(n)$, DFIV reaches the minimax rate for nonparametric IV regression over Besov classes; kernel IV requires $m/n \to \infty$, so within a fixed total sample budget DFIV's overall rate is optimal where kernel IV's is not.
  • For structural functions with low spatial homogeneity ($p < 2$), DFIV is strictly faster than every linear IV estimator, so the fixed-dictionary gap is not an artifact of a particular basis choice.
  • The two-stage oracle bound with the dynamic cover gives a general error decomposition, Stage 2 approximation error plus supremal Stage 1 approximation error plus covering entropies, that applies to any two-stage IV estimator with misspecified models in both stages.
  • The required Stage 2 smoothness control, a domain restriction or a Besov seminorm penalty, is a concrete algorithmic prescription: run DFIV and discard or penalize networks with too large a Besov seminorm, since the restricted subset still contains a rate-optimal solution.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The maximal-smoothness equality is a knife-edge condition on the conditional distribution of the covariate given the instrument that no finite sample can certify; a natural extension the paper does not pursue is an adaptive procedure that estimates $s'$ and $\gamma_0$ from data and interpolates between the optimal and the degraded rates.
  • The $m = \Omega(n)$ result suggests a simple practical design rule, split the available data roughly equally between the two stages when using DFIV, which also implies DFIV makes better use of a fixed dataset than kernel-based two-stage estimators.
  • Because the Stage 1 estimator in DFIV is a function of the Stage 2 parameter, the dynamic-cover technique appears transferable to other bilevel or iterative estimators, such as conditional-moment saddle-point methods, where the same parameter dependence has blocked rate analyses.
  • Whether the adaptivity advantage persists for other inhomogeneous function classes, and whether the balanced-split efficiency survives when the smoothness equality fails, are testable questions the paper leaves open.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper analyzes the deep feature instrumental variable (DFIV) estimator of Xu et al. (2021) for nonparametric instrumental variable regression. It proves upper bounds for the projected and full L2 risks when the structural function lies in a Besov space, under a link condition and a smoothness condition on the conditional expectation operator T. It also proves information-theoretic lower bounds and concludes that DFIV attains the nearly minimax optimal rate when T has maximal smoothness, defined by equality in Lemma 3.2, inequality (13). A further contribution is a separation result showing that DFIV outperforms linear (fixed-feature) IV estimators when the structural function has low spatial homogeneity (p < 2), under an extended reverse link condition. The proofs are extensive and introduce a dynamic-cover technique to control the supremal Stage 1 error, together with new sigmoid DNN approximation results in Besov norms.

Significance. If the main claims hold, this would be a substantial contribution: it would provide the first minimax convergence rates for a deep-feature two-stage IV estimator, characterize the Stage 1/Stage 2 sample split, and rigorously demonstrate an adaptivity advantage of neural features over fixed dictionaries. The dynamic cover argument for handling data-dependent Stage 1 estimators and the Besov-norm approximation theory for sigmoid DNNs are technically novel and likely useful beyond the specific DFIV setting. The lower bounds use information-theoretic arguments and are independent of the estimator, which is a methodological strength. However, the headline optimality result (Corollary 3.8) depends on an equality condition in Lemma 3.2 whose realizability is not demonstrated: the paper's only explicit candidate example of maximal smoothness is not a valid linear operator, and the lower-bound instance is not checked to lie in the restricted class where the upper bound is proved. These issues are load-bearing and prevent the paper, in its current form, from fully supporting the abstract's blanket optimality claim.

major comments (3)
  1. [Section 3.1, after Lemma 3.2] The example intended to show that equality can hold in (13) is not a well-defined linear operator when γ0 > 0. Since the B-spline spaces are nested, a coarse B-spline ω_{k,ℓ} is a finite linear combination of fine B-splines ω_{k+1,ℓ'}. Defining Tω_{k,ℓ} = 2^{-γ0 k} ω_{k,ℓ} assigns different scalings to the same function depending on the resolution at which it is expressed, which violates linearity unless γ0 = 0. Thus the paper does not demonstrate the existence of any genuine conditional-expectation operator with γ0 = γ1 > 0 that satisfies Assumption 4 and achieves equality in (13). Since Corollaries 3.4 and 3.8 rely on this equality, please either provide a valid smoothing example (e.g., a convolution operator whose kernel has polynomial Fourier decay) or state the optimality results as conditional on an existence assumption.
  2. [Appendix E.1, Propositions 3.3 and 3.7] The minimax lower bounds are stated under Assumptions 1(ii), 2, and 3 only, but the DFIV upper bounds require Assumption 4 and, for the optimality conclusion, equality in Lemma 3.2. The lower-bound construction reduces to an NPIR model with a known operator T and does not verify that the constructed T satisfies Assumption 4 or the maximal-smoothness equality. Consequently, the lower bound applies to a larger class of operators than the class for which DFIV is proved optimal, so the conclusion 'DFIV attains the minimax optimal rate' does not follow: a lower bound over a superset does not lower-bound the restricted class. Please verify that the lower-bound instances satisfy Assumption 4 and (13) with equality, or state a restricted minimax lower bound over the admissible class of operators.
  3. [Section 4, Assumption 6] The extended reverse link condition (Assumption 6) is essential for the p < 2 separation result (Corollary 4.2), but the only justification given is a finite-dimensional projection example whose parameters are not realized by a conditional-expectation operator in the NPIV model. As stated, Assumption 6 is an ad hoc condition on T that is not shown to hold for any concrete data-generating process with p < 2. Please provide a class of conditional distributions X|Z for which Assumption 6 holds together with the other assumptions, or explicitly frame the separation result as conditional on this additional unverified hypothesis.
minor comments (4)
  1. [Theorem 3.5] The sentence 'The log factors can be improved to log^{(s-γ0+γ1)/(s+γ1)}' is ambiguous: the displayed bound already contains a log(m ∧ n) factor, and it is unclear whether the proposed improvement changes the exponent of the log factor or removes it from one of the two terms. Please clarify the intended statement.
  2. [Abstract and Corollary 3.8] The abstract states that DFIV 'achieves the minimax optimal learning rate' without qualification, while the precise result in Corollary 3.8 requires γ0 = γ1 and maximal smoothness of T. Please add the qualification in the abstract to avoid overclaiming.
  3. [Figure 1] The label 'endogeneous variable' contains a typo; it should be 'endogenous variable'.
  4. [Section 2.3, Remark 2.3] The phrase 'the optimal θz is itself a function of the current θx' is slightly misleading: in the bilevel formulation, θz is a minimizer of the Stage 1 loss for a fixed θx, not a deterministic function. Consider rewording to 'depends on'.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: the DFIV rates are compared with independent information-theoretic lower bounds under explicit smoothness assumptions; the only caveat is a non-circular gap in exhibiting a valid maximal-smoothness operator.

full rationale

I walked the derivation chain. The DFIV upper bounds (Theorems 3.1 and 3.5) are stated in terms of assumed smoothness exponents s, s', link exponents γ0, γ1, and DNN class sizes; no parameter is fitted to data and then renamed a prediction. The lower bounds (Propositions 3.3 and 3.7) are Yang-Barron constructions over B-spline hypothesis classes and do not use the DFIV estimator, so they are independent information-theoretic benchmarks. Corollaries 3.4 and 3.8 are explicitly conditional on γ0 = γ1 and equality in Lemma 3.2 (maximal smoothness); this equality is the algebra that makes the stage-1 smoothness exponent s' convert the upper-bound exponent into the lower-bound exponent, not a fitted value. The paper honestly warns after Corollary 3.4 that if inequality (13) is strict, DFIV is suboptimal. Self-citations to Xu et al. (2021), Suzuki (2019), and Meunier et al. (2024) are used for algorithm definition, prior DNN approximation/covering results, and prior kernel-IV comparisons; the core sigmoid-DNN approximation is proved in Appendix B (Theorem B.8), so those citations are not the load-bearing reduction. A separate, non-circular correctness concern should be weighed: the only exhibited maximal-smoothness witness, in Appendix E.2, 'T maps B-splines to (scaled) B-splines' with Tω_{k,ℓ} = 2^{-γ0 k}ω_{k,ℓ}, is not a valid bounded linear operator when γ0 > 0 because the B-spline spaces are nested and linearity fails; moreover, the lower-bound class in §E.1 is not checked to satisfy Assumption 4. Thus Corollary 3.8 is a conditional statement whose maximal-smoothness condition is not demonstrated to be satisfiable by a genuine conditional-expectation operator. This affects support for the claimed minimax optimality, but it does not make the derivation circular.

Assumptions & free parameters 0 free parameters · 9 assumptions · 0 invented entities

All free parameters in the rates (s,p,q,s',γ0,γ1) are assumed properties of the data-generating process, not fitted from data. The only implementation-oriented idealization is the exact computability of the Besov penalty. Assumptions 3, 4, 5, and 6 are substantive conditions on the projection operator T; without them the stated rates do not hold.

assumptions (9)
  • domain assumption Assumption 1: noise η|Z is σ1-subgaussian; for lower bounds, variance is bounded below by σ0^2 and KL divergence is bounded by μ^2/(2σ0^2).
    Standard subgaussian noise assumption in nonparametric IV; needed for concentration inequalities in the upper bound and for the Yang-Barron lower bound.
  • domain assumption Assumption 2: fstr ∈ U(B^s_p,q(X)) with |fstr|≤C and, if s≤dx/p, P_X has bounded Lebesgue density.
    Defines the target class; the bounded density condition is needed when the structural function is discontinuous.
  • domain assumption Assumption 3 (link condition): ||T(f - Π^r_k f)||_{L2(P_Z)} ≲ 2^{-γ1 k} ||f - Π^r_k f||_{L2(P_X)}.
    Quantifies the contraction of the projection operator at high resolutions; standard in the mild ill-posed regime.
  • domain assumption Assumption 4 (smoothness of T): T[U(B^s_p,q(X))] ⊆ C_T U(B^{s'}_{p',q'}(Z)) for some s',p',q' with s'>dz(1/p'-1/2)_+.
    This is a strong regularity condition on the conditional distribution of X|Z; it controls the difficulty of Stage 1. The paper notes it is more general than Assumption 4* but still restrictive.
  • domain assumption Assumption 5 (reverse link condition): 2^{-γ0 k} ||f|| ≤ ||Tf|| for f in the span of B-splines of resolution k, with γ0≥γ1.
    Gives a lower bound on the operator's action; needed to convert projected rates into full L2 rates.
  • ad hoc to paper Assumption 6 (extended reverse link condition for p<2): the reverse link inequality holds for any subset S of B-splines up to resolution ⌈C* k⌉ of size O(2^{k dx}).
    Introduced specifically to prove the separation result Corollary 4.2; not a standard assumption in the NPIV literature.
  • ad hoc to paper The Besov seminorm regularizer R(θx) in (11) can be computed exactly and backpropagated, independent of data.
    The paper assumes exact computation 'for simplicity'; in practice this requires differentiating the network at mesh points and is an idealization.
  • standard math Adaptive B-spline recovery result of Dũng (2011, 2013) used in Lemma B.7.
    Cited external result for N-term approximation of Besov functions; underpins the DNN approximation theorem.
  • standard math Yang-Barron method and metric entropy bounds for Besov spaces (Giné and Nickl, Theorem 4.3.36).
    Used in the lower bound proofs and in Lemma 3.2 to relate the smoothness exponents.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Optimality and Adaptivity of Deep Neural Features for Instrumental Variable Regression." pith.science (2026). https://pith.science/paper/6CNTBNZE

@misc{pith2026250104898,
  author       = {Pith},
  title        = {Pith review of: Optimality and Adaptivity of Deep Neural Features for Instrumental Variable Regression},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6CNTBNZE}},
  note         = {Machine review of arXiv:2501.04898}
}
read the original abstract

We provide a convergence analysis of deep feature instrumental variable (DFIV) regression (Xu et al., 2021), a nonparametric approach to IV regression using data-adaptive features learned by deep neural networks in two stages. We prove that the DFIV algorithm achieves the minimax optimal learning rate when the target structural function lies in a Besov space. This is shown under standard nonparametric IV assumptions, and an additional smoothness assumption on the regularity of the conditional distribution of the covariate given the instrument, which controls the difficulty of Stage 1. We further demonstrate that DFIV, as a data-adaptive algorithm, is superior to fixed-feature (kernel or sieve) IV methods in two ways. First, when the target function possesses low spatial homogeneity (i.e., it has both smooth and spiky/discontinuous regions), DFIV still achieves the optimal rate, while fixed-feature methods are shown to be strictly suboptimal. Second, comparing with kernel-based two-stage regression estimators, DFIV is provably more data efficient in the Stage 1 samples.

Figures

Figures reproduced from arXiv: 2501.04898 by the authors.

Figure 1
Figure 1. Causal graph of IV. We study the nonparametric instrumental variable (NPIV) regression prob￾lem (Newey & Powell, 2003; Ai & Chen, 2003; Darolles et al., 2011). For random variables X, Y ,and ξ, we have Y = fstr(X) + ξ, E[ξ|X] 6= 0, (1) where X ∈ X is the endogenous variable, Y is the outcome, and ξ denotes unobserved confounding which affects both X and Y . The central object of interest is the structural function f… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

72 extracted references · 56 canonical work pages

  1. [1]

    Efficient estimation of models with conditional moment restrictions containing unknown functions

    Chunrong Ai and Xiaohong Chen. Efficient estimation of models with conditional moment restrictions containing unknown functions. Econometrica, 71 0 (6): 0 1795--1843, 2003

    work page 2003

  2. [2]

    Angrist and A

    J. Angrist and A. Krueger. Instrumental variables and the search for identification: From supply and demand to natural experiments. Journal of Economic Perspectives, 15: 0 69--85, 2001

    work page 2001

  3. [3]

    Identification and estimation of local average treatment effects, 1995

    Joshua Angrist and Guido Imbens. Identification and estimation of local average treatment effects, 1995

    1995

  4. [4]

    On gradient regularizers for MMD GANs

    Michael Arbel, Danica Sutherland, Mikolaj Binkowski, and Arthur Gretton. On gradient regularizers for MMD GANs . In Advances in Neural Information Processing Systems, 2018

    work page 2018

  5. [5]

    On deep learning as a remedy for the curse of dimensionality in nonparametric regression

    Benedikt Bauer and Michael Kohler. On deep learning as a remedy for the curse of dimensionality in nonparametric regression. The Annals of Statistics, 47 0 (4): 0 2261--2285, 2019

    work page 2019

  6. [6]

    Deep generalized method of moments for instrumental variable analysis

    Andrew Bennett, Nathan Kallus, and Tobias Schnabel. Deep generalized method of moments for instrumental variable analysis. In Advances in Neural Information Processing Systems, 2019

    work page 2019

  7. [7]

    Inference on strongly identified functionals of weakly identified functions

    Andrew Bennett, Nathan Kallus, Xiaojie Mao, Whitney Newey, Vasilis Syrgkanis, and Masatoshi Uehara. Inference on strongly identified functionals of weakly identified functions. In Conference on Learning Theory, 2023 a

    work page 2023

  8. [8]

    Minimax instrumental variable regression and l2 convergence guarantees without identification or closedness

    Andrew Bennett, Nathan Kallus, Xiaojie Mao, Whitney Newey, Vasilis Syrgkanis, and Masatoshi Uehara. Minimax instrumental variable regression and l2 convergence guarantees without identification or closedness. In Conference on Learning Theory, 2023 b

    work page 2023

Show all 72 references
  1. [9]

    Source condition double robust inference on functionals of inverse problems

    Andrew Bennett, Nathan Kallus, Xiaojie Mao, Whitney Newey, Vasilis Syrgkanis, and Masatoshi Uehara. Source condition double robust inference on functionals of inverse problems. arXiv preprint arXiv:2307.13793, 2023 c

    2023 arXiv

  2. [10]

    M. Z. Berkolaiko and I. Ya. Novikov. Unconditional bases in spaces of functions of anisotropic smoothness. Proc. Steklov Inst. Math., 204 0 (3): 0 27--41, 1994

    1994

  3. [11]

    Bissantz, T

    N. Bissantz, T. Hohage, A. Munk, and F. Ruymgaart. Convergence rates of general regularization methods for statistical inverse problems and applications. SIAM Journal on Numerical Analysis, 45 0 (6): 0 2610--2636, 2007

    2007

  4. [12]

    Semi-nonparametric IV estimation of shape-invariant Engel curves

    Richard Blundell, Dennis Kristensen, and Xiaohong Chen. Semi-nonparametric IV estimation of shape-invariant Engel curves . Econometrica, 75: 0 1613--1669, 2007

    2007

  5. [13]

    Linear inverse problems in structural econometrics estimation based on spectral decomposition and regularization

    Marine Carrasco, Jean-Pierre Florens, and Eric Renault. Linear inverse problems in structural econometrics estimation based on spectral decomposition and regularization. Handbook of Econometrics, 6: 0 5633--5751, 2007

    2007

  6. [14]

    Large sample sieve estimation of semi-nonparametric models

    Xiaohong Chen. Large sample sieve estimation of semi-nonparametric models. Handbook of Econometrics, 6: 0 5549--5632, 2007

    2007

  7. [15]

    Christensen

    Xiaohong Chen and Timothy M. Christensen. Optimal sup-norm rates and uniform inference on nonlinear functionals of nonparametric IV regression . Quantitative Economics, 9 0 (1): 0 39--84, 2018

    2018

  8. [16]

    On rate optimality for ill-posed inverse problems in econometrics

    Xiaohong Chen and Markus Reiss. On rate optimality for ill-posed inverse problems in econometrics. Econometric Theory, 27 0 (3): 0 497--521, 2011

    2011

  9. [17]

    On instrumental variable regression for deep offline policy evaluation

    Yutian Chen, Liyuan Xu, Caglar Gulcehre, Tom Le Paine, Arthur Gretton, Nando de Freitas, and Arnaud Doucet. On instrumental variable regression for deep offline policy evaluation. Journal of Machine Learning Research, 23 0 (302): 0 1--40, 2022

    2022

  10. [18]

    Darolles, Y

    S. Darolles, Y. Fan, J. P. Florens, and E. Renault. Nonparametric instrumental regression. Econometrica, 79 0 (5): 0 1541--1565, 2011

    2011

  11. [19]

    On the approximation of functions by tanh neural networks

    Tim De Ryck , Samuel Lanthaler, and Siddhartha Mishra. On the approximation of functions by tanh neural networks. Neural Networks, 143: 0 732--750, 2021

    2021

  12. [20]

    De V ore and Vasil A

    Ronald A. De V ore and Vasil A. Popov. Interpolation of B esov spaces. Transactions of the American Mathematical Society, 305 0 (1): 0 397--414, 1988

    1988

  13. [21]

    Minimax estimation of conditional moment models

    Nishanth Dikkala, Greg Lewis, Lester Mackey, and Vasilis Syrgkanis. Minimax estimation of conditional moment models. In Advances in Neural Information Processing Systems, 2020

    2020

  14. [22]

    Donoho and Iain M

    David L. Donoho and Iain M. Johnstone. Minimax estimation via wavelet shrinkage. The Annals of Statistics, 26 0 (3): 0 879--921, 1998

    1998

  15. [23]

    Optimal adaptive sampling recovery

    Dinh D \ u ng. Optimal adaptive sampling recovery. Advances in Computational Mathematics, 34: 0 1--41, 2011

    2011

  16. [24]

    Continuous algorithms in adaptive sampling recovery

    Dinh D \ u ng. Continuous algorithms in adaptive sampling recovery. Journal of Approximation Theory, 166: 0 136--153, 2013

    2013

  17. [25]

    Mathematical foundations of infinite-dimensional statistical models

    Evarist Gin\' e and Richard Nickl. Mathematical foundations of infinite-dimensional statistical models. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 2015

    2015

  18. [26]

    Conditional mean embeddings as regressors

    Steffen Gr \"u new \"a lder, Guy Lever, Luca Baldassarre, Sam Patterson, Arthur Gretton, and Massimilano Pontil. Conditional mean embeddings as regressors. In Proceedings of the 29th International Coference on International Conference on Machine Learning, pp.\ 1803--1810, 2012

    2012

  19. [27]

    Improved training of Wasserstein GANs

    Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. Improved training of Wasserstein GANs . In Advances in Neural Information Processing Systems, 2017

    2017

  20. [28]

    Horowitz

    Peter Hall and Joel L. Horowitz. Nonparametric methods for inference in the presence of instrumental variables. The Annals of Statistics, 33 0 (6): 0 2904--2929, 2005

    2005

  21. [29]

    Deep IV : a flexible approach for counterfactual prediction

    Jason Hartford, Greg Lewis, Kevin Leyton-Brown, and Matt Taddy. Deep IV : a flexible approach for counterfactual prediction. In International Conference on Machine Learning, 2017

    2017

  22. [30]

    On the minimax optimality and superiority of deep neural network learning over sparse parameter spaces

    Satoshi Hayakawa and Taiji Suzuki. On the minimax optimality and superiority of deep neural network learning over sparse parameter spaces. Neural Networks, 123: 0 343--361, 2020

    2020

  23. [31]

    Applied nonparametric instrumental variables estimation

    Joel L Horowitz. Applied nonparametric instrumental variables estimation. Econometrica, 79 0 (2): 0 347--394, 2011

    2011

  24. [32]

    A rigorous theory of conditional mean embeddings

    Ilja Klebanov, Ingmar Schuster, and Timothy John Sullivan. A rigorous theory of conditional mean embeddings. SIAM Journal on Mathematics of Data Science, 2 0 (3): 0 583--606, 2020

    2020

  25. [33]

    Approximating smooth functions by deep neural networks with sigmoid activation function

    Sophie Langer. Approximating smooth functions by deep neural networks with sigmoid activation function. Journal of Multivariate Analysis, 182, 2021

    2021

  26. [34]

    Optimal rates for regularized conditional mean embedding learning

    Zhu Li, Dimitri Meunier, Mattes Mollenhauer, and Arthur Gretton. Optimal rates for regularized conditional mean embedding learning. Advances in Neural Information Processing Systems, 35: 0 4433--4445, 2022

    2022

  27. [35]

    Towards optimal S obolev norm rates for the vector-valued regularized least-squares algorithm

    Zhu Li, Dimitri Meunier, Mattes Mollenhauer, and Arthur Gretton. Towards optimal S obolev norm rates for the vector-valued regularized least-squares algorithm. Journal of Machine Learning Research, 25 0 (181): 0 1--51, 2024 a

    2024

  28. [36]

    Regularized DeepIV with model selection

    Zihao Li, Hui Lan, Vasilis Syrgkanis, Mengdi Wang, and Masatoshi Uehara. Regularized DeepIV with model selection. arXiv preprint arXiv:2403.04236, 2024 b

    2024 arXiv

  29. [37]

    Provably efficient neural estimation of structural equation models: An adversarial approach

    Luofeng Liao, You-Lin Chen, Zhuoran Yang, Bo Dai, Mladen Kolar, and Zhaoran Wang. Provably efficient neural estimation of structural equation models: An adversarial approach. In Advances in Neural Information Processing Systems, 2020

    2020

  30. [38]

    Instrumental variable value iteration for causal offline reinforcement learning

    Luofeng Liao, Zuyue Fu, Zhuoran Yang, Yixin Wang, Mladen Kolar, and Zhaoran Wang. Instrumental variable value iteration for causal offline reinforcement learning. arXiv preprint arXiv:2102.09907, 2021

    2021 arXiv

  31. [39]

    Nonparametric instrumental regression via kernel methods is minimax optimal

    Dimitri Meunier, Zhu Li, Tim Christensen, and Arthur Gretton. Nonparametric instrumental regression via kernel methods is minimax optimal. arXiv preprint arXiv:2411.19653, 2024

    2024 arXiv

  32. [40]

    H. N. Mhaskar and Charles Micchelli. Approximation by superposition of sigmoidal and radial basis functions. Advances in Applied Mathematics, 13 0 (3): 0 350--373, 1992

    1992

  33. [41]

    Identification, doubly robust estimation, and semiparametric efficiency theory of nonignorable missing data with a shadow variable

    Wang Miao, Lan Liu, Eric Tchetgen Tchetgen, and Zhi Geng. Identification, doubly robust estimation, and semiparametric efficiency theory of nonignorable missing data with a shadow variable. arXiv preprint arXiv:1509.02556, 2015

    2015 arXiv

  34. [42]

    Nonparametric approximation of conditional expectation operators

    Mattes Mollenhauer and P \'e ter Koltai. Nonparametric approximation of conditional expectation operators. arXiv preprint arXiv:2012.12917, 2020

    2012 arXiv

  35. [43]

    Dual instrumental variable regression

    Krikamol Muandet, Arash Mehrjou, Si Kai Lee, and Anant Raj. Dual instrumental variable regression. In Advances in Neural Information Processing Systems, 2020

    2020

  36. [44]

    Regularization in H ilbert scales under general smoothing conditions

    Thamban Nair, Sergei V Pereverzev, and Ulrich Tautenhahn. Regularization in H ilbert scales under general smoothing conditions. Inverse Problems, 21 0 (6): 0 1851, 2005

    2005

  37. [45]

    M. Z. Nashed and Grace Wahba. Generalized inverses in reproducing kernel spaces: an approach to regularization of linear operator equations. SIAM Journal on Mathematical Analysis, 5 0 (6): 0 974--987, 1974

    1974

  38. [46]

    Newey and James L

    Whitney K. Newey and James L. Powell. Instrumental variable estimation of nonparametric models. Econometrica, 71 0 (5): 0 1565--1578, 2003

    2003

  39. [47]

    A measure-theoretic approach to kernel conditional mean embeddings

    Junhyung Park and Krikamol Muandet. A measure-theoretic approach to kernel conditional mean embeddings. In Advances in Neural Information Processing Systems, 2020

    2020

  40. [48]

    Functional bilevel optimization for machine learning

    Ieva Petrulionyte, Julien Mairal, and Michael Arbel. Functional bilevel optimization for machine learning. arXiv preprint arXiv:2403.20233, 2024

    2024 arXiv

  41. [49]

    Spectral representation for causal estimation with hidden confounders

    Tongzheng Ren, Haotian Sun, Antoine Moulin, Arthur Gretton, and Bo Dai. Spectral representation for causal estimation with hidden confounders. arXiv preprint arXiv:2407.10448, 2024

    2024 arXiv

  42. [50]

    A case for new neural network smoothness constraints

    Mihaela Rosca, Theophane Weber, Arthur Gretton, and Shakir Mohamed. A case for new neural network smoothness constraints. Proceedings of Machine Learning Research, 137: 0 21--32, Dec 2020

    2020

  43. [51]

    An unconditional basis in periodic spaces with dominating mixed smoothness properties

    H J Schmeisser. An unconditional basis in periodic spaces with dominating mixed smoothness properties. Analysis Mathematica, 13 0 (2): 0 153--168, 1987

    1987

  44. [52]

    Nonparametric regression using deep neural networks with ReLU activation function

    Johannes Schmidt-Hieber. Nonparametric regression using deep neural networks with ReLU activation function. The Annals of Statistics, 48 0 (4), 2020

    2020

  45. [53]

    Kernel instrumental variable regression

    Rahul Singh, Maneesh Sahani, and Arthur Gretton. Kernel instrumental variable regression. In Advances in Neural Information Processing Systems, 2019

    2019

  46. [54]

    Robust large margin deep neural networks

    Jure Sokoli\' c , Raja Giryes, Guillermo Sapiro, and Miguel Rodrigues. Robust large margin deep neural networks. Trans. Sig. Proc., 65 0 (16): 0 4265–4280, Aug 2017

    2017

  47. [55]

    Hilbert space embeddings of conditional distributions with applications to dynamical systems

    Le Song, Jonathan Huang, Alex Smola, and Kenji Fukumizu. Hilbert space embeddings of conditional distributions with applications to dynamical systems. In Proceedings of the 26th Annual International Conference on Machine Learning, pp.\ 961--968, 2009

    2009

  48. [56]

    Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionality

    Taiji Suzuki. Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionality . In International Conference on Learning Representations, 2019

    2019

  49. [57]

    Deep learning is adaptive to intrinsic dimensionality of model smoothness in anisotropic B esov space

    Taiji Suzuki and Atsushi Nitanda. Deep learning is adaptive to intrinsic dimensionality of model smoothness in anisotropic B esov space. In Advances in Neural Information Processing Systems, 2021

    2021

  50. [58]

    Theory of function spaces

    Hans Triebel. Theory of function spaces. Monographs in mathematics. Birkh\" a user Verlag, 1983

    1983

  51. [59]

    Finite sample analysis of minimax offline reinforcement learning: completeness, fast rates and first-order efficiency

    Masatoshi Uehara, Masaaki Imaizumi, Nan Jiang, Nathan Kallus, Wen Sun, and Tengyang Xie. Finite sample analysis of minimax offline reinforcement learning: completeness, fast rates and first-order efficiency. arXiv preprint arXiv:2102.02981, 2021

    2021 arXiv

  52. [60]

    Widths of embeddings in function spaces

    Jan Vyb\' i ral. Widths of embeddings in function spaces. Journal of Complexity, 24 0 (4): 0 545--570, 2008

    2008

  53. [61]

    Wainwright

    Martin J. Wainwright. High-dimensional statistics: a non-asymptotic viewpoint. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 2019

    2019

  54. [62]

    An instrumental variable approach for identification and estimation with nonignorable nonresponse

    Sheng Wang, Jun Shao, and Jae Kwang Kim. An instrumental variable approach for identification and estimation with nonignorable nonresponse. Statistica Sinica, pp.\ 1097--1116, 2014

    2014

  55. [63]

    Spectral representation learning for conditional moment models

    Ziyu Wang, Yucen Luo, Yueru Li, Jun Zhu, and Bernhard Sch \"o lkopf. Spectral representation learning for conditional moment models. arXiv preprint arXiv:2210.16525, 2022

    2022 arXiv

  56. [64]

    Wasserman

    L. Wasserman. All of Nonparametric Statistics. Springer, 2006

    2006

  57. [65]

    Learning deep features in instrumental variable regression

    Liyuan Xu, Yutian Chen, Siddarth Srinivasan, Nando de Freitas, Arnaud Doucet, and Arthur Gretton. Learning deep features in instrumental variable regression. In International Conference on Learning Representations, 2021

    2021

  58. [66]

    Information-theoretic determination of minimax rates of convergence

    Yuhong Yang and Andrew Barron. Information-theoretic determination of minimax rates of convergence. The Annals of Statistics, 27 0 (5): 0 1564--1599, 1999

    1999

  59. [67]

    Instrumental variable regression via kernel maximum moment loss

    Rui Zhang, Masaaki Imaizumi, Bernhard Schölkopf, and Krikamol Muandet. Instrumental variable regression via kernel maximum moment loss. Journal of Causal Inference, 11 0 (1): 0 1--42, 2023

    2023

  60. [68]

    Wavelet threshold estimation of a regression function with random design

    Shuanglin Zhang, Man-Yu Wong, and Zhongguo Zheng. Wavelet threshold estimation of a regression function with random design. Journal of Multivariate Analysis, 80 0 (2): 0 256--284, 2002

    2002

  61. [69]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  62. [70]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  63. [71]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  64. [72]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.