REVIEW 2 major objections 5 minor 1 cited by
RieszBoost: Gradient Boosting for Riesz Regression
T0 review · 2 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read This paper introduces RieszBoost, a gradient boosting algorithm that estimates the Riesz representer directly by minimizing the Riesz loss, matching or outperforming indirect plug-in estimators for ATE, ATT, ASE, and LASE.
desk verdict RieszBoost is a genuinely useful algorithmic idea for direct Riesz regression with boosted trees, but the empirical Riesz loss is unbounded below for continuous covariates, so the stated objective is not well-defined and the paper needs substantial revision before it can be taken as a rigorous method. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing identity is the Riesz representation of a continuous linear functional, $\Psi(P_0)=E[m(O,\mu_0)] = E[\alpha_0(W)\mu_0(W)]$, which turns the representer into the minimizer of the Riesz loss $E[-2m(O,\alpha)+\alpha(W)^2]$. The machinery is gradient boosting in function space, repeatedly fitting trees to the negative empirical gradient, plus a data-augmentation step that expands the predictor matrix with pseudo-data where the empirical gradient is nonzero; a target matrix and residual function encode the parts of the gradient that do not depend on the candidate function $\alpha$. This lets standard boosting libraries minimize the Riesz loss with minimal code changes.
What would settle it
Run RieszBoost on simulated data with continuous covariates and a known treatment assignment mechanism, without early stopping, and compare the fitted weights with the known inverse-probability weights as the number of boosting iterations grows; if the empirical Riesz loss keeps decreasing while the fitted weights move away from the truth, the central claim fails.
Extended reading notes
Core claim
The paper's central claim is that the Riesz representer $\alpha_0$, which encodes the weights needed for doubly robust estimation of a linear functional $\Psi(P_0)=E[m(O,\mu_0)]$, can be estimated directly by gradient boosting on the Riesz loss $E[-2m(O,\alpha)+\alpha(W)^2]$. The minimizer of this loss is $\alpha_0$ by the Riesz representation theorem, and the paper shows how to implement the minimization with tree boosting. The key implementation step is a data augmentation trick: build a predictor matrix from the observed $W$ plus pseudo-data, such as counterfactual treatments, where the empirical gradient is nonzero; build a target matrix recording the original and counterfactual exposure values; and define a residual function that combines these with the current prediction. This makes the Riesz loss compatible with standard gradient boosting code. Simulations show that the resulting RieszBoost estimates of $\alpha_0$ for ATE, ATT, ASE, and LASE are comparable to or better than indirect plug-in estimates, and that downstream efficient estimating equations estimates of the causal parameters have good coverage.
Load-bearing premise
The load-bearing assumption is that fitting trees to the finite-sample Riesz loss recovers the true weighting function, even though for continuous covariates that loss is unbounded below and the result can depend on early stopping.
Editorial extensions
If this is right
- For causal functionals whose Riesz representer has no tractable analytic form, such as shift effects with continuous treatments, RieszBoost provides an estimate directly from data.
- Substituting RieszBoost's representer into an efficient estimating equations estimator gave unbiased ATE and ATT estimates with 95% confidence interval coverage at or above the indirect method in simulations.
- Because tuning is done on the validation Riesz loss and the algorithm reuses standard boosting implementations, researchers can adopt it without new modeling machinery.
- RieszBoost estimates can be used in cross-fitted efficient estimating equations, TMLE, or double machine learning procedures without modification.
Reading between the lines
- An implicit consequence is that RieszBoost's practical success may depend on early stopping and tree smoothing, because with continuous covariates and no ties the empirical Riesz loss is unbounded below; formalizing this regularization could make the method more reliable.
- The same predictor and target augmentation could be automated from the functional $m(O,\mu)$, letting users apply RieszBoost to new estimands without hand-deriving residual functions.
- The method may be especially useful for longitudinal modified treatment policies and continuous exposures, where the analytical representer requires density-ratio estimation; the simulations only cover one-dimensional confounders, so high-dimensional behavior remains an open test.
- Combining RieszBoost with stochastic sub-sampling or other standard boosting extensions could reduce runtime and variance at large sample sizes.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes RieszBoost, a gradient boosting algorithm that directly estimates the Riesz representer of a linear functional by minimizing the Riesz loss. The method constructs predictor and target matrices to evaluate the loss at counterfactual points, and the authors derive residual functions for the ATE, ATT, ASE, and LASE functionals. Extensive simulations compare RieszBoost with plug-in estimators based on propensity scores or conditional densities; RieszBoost reports lower RMSE/MAE for the Riesz representer in the binary treatment cases and comparable coverage for all targets.
Significance. If validated, RieszBoost would be a practical and user-friendly contribution to causal inference with tabular data, sidestepping analytical derivation of Riesz representers and density estimation. The simulation evidence is encouraging, and the paper explicitly includes derivations, cross-fitting, and hyperparameter-tuning details. However, the paper's central claim is that the algorithm estimates the Riesz representer by minimizing the empirical Riesz loss; because this loss is unbounded below for continuous covariates, the optimization objective is not well defined. This is a load-bearing gap that prevents acceptance in its current form, although it is likely addressable with regularization or a restricted function class.
major comments (2)
- [Section 2.1, Eq. (4)] The empirical Riesz loss is unbounded below for continuous covariates with no ties, so the claimed minimization of L_n is not a well-defined optimization problem. In the ATE loss (Eq. 4), for a control unit j, the counterfactual value α(1,X_j) appears only in the linear term −2α(1,X_j) with no quadratic term; setting α(1,X_j)=t → ∞ sends L_n → −∞ while holding all other α values fixed. The same phenomenon occurs for the ASE and LASE losses (Section 2.3, Example 2, and Appendix B.3.2). The boosting function class of shallow trees can contain such unbounded directions by isolating a leaf with only counterfactual rows. Therefore, the algorithm's output depends on unanalyzed early stopping and tree-induced smoothing rather than on minimization of the stated loss. The authors should either add explicit regularization (e.g., a penalty on α), restrict the function space (e.g., bounded leaf values), or provide an analysis of implicit regularization in gradient boosting. As written, the theoretical motivation for RieszBoost is incomplete.
- [Section 2.3, Tuning paragraph] The recommended tuning criterion—the validation-set empirical Riesz loss—inherits the same unboundedness described above. Since the validation loss can be driven to −∞ by increasing model complexity or pushing counterfactual α values to extreme magnitudes, it cannot be used to select a stopping point or choose hyperparameters. The claim that 'tuning is relatively straightforward' and the suggested early-stopping procedure are not supported by a well-defined selection criterion. A bounded or regularized validation objective, or a theoretical analysis of early stopping in this setting, is needed.
minor comments (5)
- [Section 1] The text contains a duplicated word: 'formed by by first estimating µ0 and α0' should read 'formed by first estimating µ0 and α0.'
- [Section 3.2.1] Typos: 'esitmation' should be 'estimation', and 'assoicated' should be 'associated' in the sentence about avoiding conditional density estimation.
- [Section 2.3, Tuning formula] The validation loss formula writes α(O_i)^2, but since α is a function of W, this should be α(W_i)^2 to match the Riesz loss definition.
- [Section 2.3, predictor matrix construction] The mapping p(j) from rows of the predictor matrix to rows of the original data is not explicitly defined; the notation is introduced informally and would be clearer with a formal definition.
- [General] No code repository or reproducibility details are provided, which would be helpful for a methods paper depending on a new algorithmic procedure.
Circularity Check
No significant circularity: the Riesz loss is defined from the known functional and external Riesz representation theory, not from the target representer or fitted outputs; self-citations are background only.
full rationale
The paper's derivation chain is not circular. The Riesz loss l(O, alpha) = -2m(O, alpha) + alpha(W)^2 depends on the known functional m and on the candidate function alpha, not on the true Riesz representer alpha_0 or on fitted values. The claim that alpha_0 minimizes the population Riesz loss is imported from Chernozhukov et al. as an external mathematical result, and the paper does not redefine the functional in terms of the quantity it later estimates. RieszBoost applies standard gradient boosting to the empirical Riesz loss, with the data-augmentation step ensuring that gradients at counterfactual points are computed; nothing in this construction fits a parameter to a target and then relabels the fit as a prediction. The simulation benchmarks use known ground truth, cross-fitting, and separate estimation data, so the empirical performance claims are externally evaluated. The authors' self-citations ([34], [35]) appear only as background support for gradient boosting robustness and EEE motivation, and they do not carry the central claim; even if they were removed, the Riesz loss minimization logic and simulation comparisons would stand. The skeptical concern about the empirical Riesz loss being unbounded below for continuous covariates is a substantive correctness or identifiability issue, but it is not a circularity: the method's output being sensitive to early stopping does not mean the derivation reduces to its own inputs by construction.
Assumptions & free parameters
free parameters (2)
- Boosting hyperparameters (learning rate, iterations, tree depth) =
Grid searched in each simulation: lambda in {0.001, 0.01, 0.1, 0.25}, M in {10, 30, 50, 75, 100, 150, 200}, depth in…
- Kernel bandwidths for indirect ASE/LASE density estimation =
Grid searched: joint density bandwidth in {0.01, 1.2575, 2.505, 3.7525, 5}, marginal bandwidth in {0.01, 0.5075…
assumptions (4)
- standard math Riesz representation theorem and square-integrability of the candidate function space.
- domain assumption The linear functional m is such that the empirical Riesz loss gradient is nonzero only at observed and a finite set of pseudo-data points.
- domain assumption The outcome regression and Riesz representer estimates are consistent enough for the EEE estimator and variance estimator to be valid.
- ad hoc to paper The finite-sample empirical Riesz loss is a valid target for optimization by gradient boosting.
Cite this review
Pith. "Pith review of RieszBoost: Gradient Boosting for Riesz Regression." pith.science (2026). https://pith.science/paper/3PFVNNOM
@misc{pith2026250104871,
author = {Pith},
title = {Pith review of: RieszBoost: Gradient Boosting for Riesz Regression},
year = {2026},
howpublished = {\url{https://pith.science/paper/3PFVNNOM}},
note = {Machine review of arXiv:2501.04871}
}
read the original abstract
Answering causal questions often involves estimating linear functionals of conditional expectations, such as the average treatment effect or the effect of a longitudinal modified treatment policy. By the Riesz representation theorem, these functionals can be expressed as the expected product of the conditional expectation of the outcome and the Riesz representer, a key component in doubly robust estimation methods. Traditionally, the Riesz representer is estimated indirectly by deriving its explicit analytical form, estimating its components, and substituting these estimates into the known form (e.g., the inverse propensity score). However, deriving or estimating the analytical form can be challenging, and substitution methods are often sensitive to practical positivity violations, leading to higher variance and wider confidence intervals. In this paper, we propose a novel gradient boosting algorithm to directly estimate the Riesz representer without requiring its explicit analytical form. This method is particularly suited for tabular data, offering a flexible, nonparametric, and computationally efficient alternative to existing methods for Riesz regression. Through simulation studies, we demonstrate that our algorithm performs on par with or better than indirect estimation techniques across a range of functionals, providing a user-friendly and robust solution for estimating causal quantities.
Forward citations
Cited by 1 Pith paper
-
Outcome-adapted Automatic Debiased Machine Learning
Outcome-adapted representations that preserve outcome prediction while discarding Riesz information strictly reduce asymptotic variance of AutoDML estimators relative to using all covariates.
Reference graph
Works this paper leans on
-
[1]
Introductory Functional Analysis with Applications
Erwin Kreyszig. Introductory Functional Analysis with Applications . Wiley classics library. Wiley, New York?, wiley classics library ed. edition, 1989. Publication Title: Introductory functional analysis with applications
work page 1989
-
[2]
Robins, Andrea Rotnitzky, and Lue Ping Zhao
James M. Robins, Andrea Rotnitzky, and Lue Ping Zhao. Estimation of Regression Coefficients When Some Regressors Are Not Always Observed. Journal of the Amer- ican Statistical Association , 89(427):846–866, 1994. Publisher: [American Statistical Association, Taylor & Francis, Ltd.]
work page 1994
-
[3]
Neyman’s Repeated Sampling Approach to Completely Randomized Experiments
Guido W Imbens and Donald B Rubin. Neyman’s Repeated Sampling Approach to Completely Randomized Experiments. In Donald B. Rubin and Guido W. Imbens, edi- tors, Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction , pages 83–112. Cambridge University Press, Cambridge, 2015
work page 2015
-
[4]
P.J. Bickel, C.A.J. Klaassen, Y. Ritov, and J.A. Wellner. Efficient and Adaptive Esti- mation for Semiparametric Models . Johns Hopkins series in the mathematical sciences. Springer New York, 1998
work page 1998
-
[5]
M.J. van der Laan and J.M. Robins. Unified Methods for Censored Longitudinal Data and Causality . Springer Series in Statistics. Springer, 2003
work page 2003
-
[6]
Anastasios A. Tsiatis. Semiparametric Theory and Missing Data . Springer Series in Statistics. Springer, New York, NY, 2006
work page 2006
-
[7]
James Robins, Lingling Li, Eric Tchetgen, and Aad W. van der Vaart. Quadratic semiparametric Von Mises calculus. Metrika, 69(2-3):227–247, March 2009
work page 2009
-
[8]
Victor Chernozhukov, Whitney K. Newey, and Rahul Singh. Automatic Debiased Ma- chine Learning of Causal and Structural Effects. Econometrica, 90(3):967–1027, 2022. eprint: https://onlinelibrary.wiley.com/doi/pdf/10.3982/ECTA18515
Show all 37 references
-
[9]
van der Laan and Daniel Rubin
Mark J. van der Laan and Daniel Rubin. Targeted Maximum Likelihood Learning. The International Journal of Biostatistics , 2(1), December 2006. Publisher: De Gruyter
2006
-
[10]
Williams, Kara E
Herbert Susmann, Nicholas T. Williams, Kara E. Rudolph, and Iv´ an D ´ ıaz. Longitudinal Generalizations of the Average Treatment Effect on the Treated for Multi-valued and Continuous Treatments, October 2024. arXiv:2405.06135
2024 arXiv
-
[11]
Newey, Victor Quintas-Martinez, and Vasilis Syrgka- nis
Victor Chernozhukov, Whitney K. Newey, Victor Quintas-Martinez, and Vasilis Syrgka- nis. Automatic Debiased Machine Learning via Riesz Regression, March 2024. arXiv:2104.14737 [econ, math, stat]
2024 arXiv
-
[12]
Newey, Victor Quintas-Martinez, and Vasilis Syrgka- nis
Victor Chernozhukov, Whitney K. Newey, Victor Quintas-Martinez, and Vasilis Syrgka- nis. RieszNet and ForestRiesz: Automatic Debiased Machine Learning with Neural Nets and Random Forests, June 2022. arXiv:2110.03031 [cs, econ, stat]. 17
2022 arXiv
-
[13]
Hirshberg, and Jos´ e R
Eli Ben-Michael, Avi Feller, David A. Hirshberg, and Jos´ e R. Zubizarreta. The Balanc- ing Act in Causal Inference, October 2021. arXiv:2110.14831 [stat]
2021 arXiv
-
[14]
Entropy Balancing for Causal Effects: A Multivariate Reweighting Method to Produce Balanced Samples in Observational Studies
Jens Hainmueller. Entropy Balancing for Causal Effects: A Multivariate Reweighting Method to Produce Balanced Samples in Observational Studies. Political Analysis , 20(1):25–46, January 2012
2012
-
[15]
Graham, Cristine Campos De Xavier Pinto, and Daniel Egel
Bryan S. Graham, Cristine Campos De Xavier Pinto, and Daniel Egel. Inverse Probabil- ity Tilting for Moment Condition Models with Missing Data. The Review of Economic Studies, 79(3):1053–1079, 2012. Publisher: [Oxford University Press, Review of Eco- nomic Studies, Ltd.]
2012
-
[16]
Zubizarreta
Jos´ e R. Zubizarreta. Stable Weights that Balance Covariates for Estima- tion With Incomplete Outcome Data. Journal of the American Statistical Association, 110(511):910–922, July 2015. Publisher: ASA Website eprint: https://doi.org/10.1080/01621459.2015.1023805
2015
-
[17]
Zubizarreta
Ambarish Chattopadhyay and Jose R. Zubizarreta. On the implied weights of linear regression for causal inference, July 2022. arXiv:2104.06581 [stat]
2022 arXiv
-
[18]
Friedman
Jerome H. Friedman. Greedy Function Approximation: A Gradient Boosting Machine. The Annals of Statistics , 29(5):1189–1232, 2001. Publisher: Institute of Mathematical Statistics
2001
-
[19]
An empirical comparison of supervised learning algorithms
Rich Caruana and Alexandru Niculescu-Mizil. An empirical comparison of supervised learning algorithms. In Proceedings of the 23rd international conference on Machine learning, ICML ’06, pages 161–168, New York, NY, USA, June 2006. Association for Computing Machinery
2006
-
[20]
XGBoost: A Scalable Tree Boosting System
Tianqi Chen and Carlos Guestrin. XGBoost: A Scalable Tree Boosting System. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Dis- covery and Data Mining , KDD ’16, pages 785–794, New York, NY, USA, August 2016. Association for Computing Machinery
2016
-
[21]
LightGBM: A Highly Efficient Gradient Boosting Decision Tree
Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. LightGBM: A Highly Efficient Gradient Boosting Decision Tree. In Advances in Neural Information Processing Systems , volume 30. Curran Associates, Inc., 2017
2017
-
[22]
Tabular data: Deep learning is not all you need
Ravid Shwartz-Ziv and Amitai Armon. Tabular data: Deep learning is not all you need. Information Fusion, 81:84–90, May 2022
2022
-
[23]
When Do Neural Nets Outperform Boosted Trees on Tabular Data?, July 2024
Duncan McElfresh, Sujay Khandagale, Jonathan Valverde, Vishak Prasad C, Ben- jamin Feuer, Chinmay Hegde, Ganesh Ramakrishnan, Micah Goldblum, and Colin White. When Do Neural Nets Outperform Boosted Trees on Tabular Data?, July 2024. arXiv:2305.02997. 18
2024 arXiv
-
[24]
Deep Neural Networks and Tabular Data: A Survey
Vadim Borisov, Tobias Leemann, Kathrin Seßler, Johannes Haug, Martin Pawelczyk, and Gjergji Kasneci. Deep Neural Networks and Tabular Data: A Survey. IEEE Transactions on Neural Networks and Learning Systems , 35(6):7499–7519, June 2024. Conference Name: IEEE Transactions on N...
2024
-
[25]
Friedman
Jerome H. Friedman. Stochastic gradient boosting. Computational Statistics & Data Analysis, 38(4):367–378, February 2002
2002
-
[26]
Population Intervention Causal Effects Based on Stochastic Interventions
Iv´ an D ´ ıaz and Mark van der Laan. Population Intervention Causal Effects Based on Stochastic Interventions. Biometrics, 68(2):541–549, 2012. eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1541-0420.2011.01685.x
2012
-
[27]
Hoffman, and Edward J
Iv´ an D ´ ıaz, Nicholas Williams, Katherine L. Hoffman, and Edward J. Schenck. Non- parametric Causal Effects Based on Longitudinal Modified Treatment Policies. Journal of the American Statistical Association , 118(542):846–857, April 2023. Publisher: ASA Website eprint: http...
2023
-
[28]
Asymptotic Theory for Cross-validated Tar- geted Maximum Likelihood Estimation
Wenjing Zheng and Mark van der Laan. Asymptotic Theory for Cross-validated Tar- geted Maximum Likelihood Estimation. U.C. Berkeley Division of Biostatistics Working Paper Series, November 2010
2010
-
[29]
Double/debiased machine learning for treatment and structural parameters
Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21(1):C1–C68, Febru- ary 2018
2018
-
[30]
Kernel Ridge Riesz Representers: Generalization, Mis-specification, and the Counterfactual Effective Dimension, July 2024
Rahul Singh. Kernel Ridge Riesz Representers: Generalization, Mis-specification, and the Counterfactual Effective Dimension, July 2024. arXiv:2102.11076 [stat]
2024 arXiv
-
[31]
Smoothing noisy data with spline functions
Peter Craven and Grace Wahba. Smoothing noisy data with spline functions. Nu- merische Mathematik , 31(4):377–403, December 1978
1978
-
[32]
Tibshirani
Stefan Wager, Wenfei Du, Jonathan Taylor, and Robert J. Tibshirani. High-dimensional regression adjustments in randomized experiments.Proceedings of the National Academy of Sciences, 113(45):12673–12678, November 2016. Publisher: Proceedings of the Na- tional Academy of Sciences
2016
-
[33]
Boosting with early stopping: Convergence and consistency
Tong Zhang and Bin Yu. Boosting with early stopping: Convergence and consistency. The Annals of Statistics , 33(4):1538–1579, August 2005. Publisher: Institute of Math- ematical Statistics
2005
-
[34]
Lassoed Tree Boosting, December
Alejandro Schuler, Yi Li, and Mark van der Laan. Lassoed Tree Boosting, December
-
[35]
van der Laan
Alejandro Schuler and Mark J. van der Laan. Introduction to Modern Causal Inference
-
[36]
1q. Therefore, to derive the EIF, we must take this dependency into consideration (e.g. using the delta method on the inverse probability parameter 1 PpA“1q and the “partial
Alan E. Hubbard, Nicholas P. Jewell, and Mark J. van der Laan. Direct Effects and Effect Among the Treated. In Targeted Learning: Causal Inference for Observational and Experimental Data , pages 133–143. Springer New York, New York, NY, 2011. 19 Appendices A Efficient Estimati...
2011
-
[2023]
arXiv:2205.10697 [stat]
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.