REVIEW 3 major objections 5 minor 39 references
On Volume Minimization in Conformal Regression
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read A finite-sample bound shows conformal intervals shrink when the base predictor minimizes the (1−α)-quantile of absolute error.
desk verdict A genuine finite-sample excess-volume bound for split conformal regression, with the main soft spot being an uninstantiated uniform deviation assumption that may make the learning term vacuous for the neural-network experiments. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the $(1-\alpha)$-QAE problem: minimize the $(1-\alpha)$-quantile of the absolute residual $|Y-f(X)|$ over the function class $\mathcal{F}$, because that quantile is exactly the half-width of the optimal constant-size oracle interval. To make the argument quantitative, the paper treats the learning step as an empirical minimum-volume-set estimator and applies the Dvoretzky–Kiefer–Wolfowitz inequality to control empirical quantile deviations. Two assumptions carry the rates: a Hölder continuity condition on the quantile function of the residuals (Assumption 1), which converts probability differences into length differences, and a uniform high-probability bound $\varphi(\mathcal{F},\delta,n)$ on the sup over $f\in\mathcal{F}$ and $t\ge 0$ of $|P(|Y-f(X)|\le t) - $ the empirical counterpart $|$ (Assumption 2), which measures the complexity of $\mathcal{F}$. For optimization, the empirical quantile is smoothed with a differentiable approximation $\Gamma_\varepsilon$ so that gradients can be computed.
What would settle it
Measure the realized excess volume of EffOrt on a heavy-tailed asymmetric noise model with a neural-network class $\mathcal{F}$, and compare it with Theorem 1 using an empirically estimated $\varphi(\mathcal{F},\delta,n_\ell)$. If the gap between EffOrt and the oracle interval does not shrink as $n_\ell$ grows, or if the uniform deviation $\varphi$ cannot be bounded even for moderate $n_\ell$, the theorem's learning-rate term is not the right description.
Extended reading notes
Core claim
With a fixed base predictor $f$, the optimal constant-width interval is $[f(x)-t_*, f(x)+t_*]$ where $t_*$ is the $(1-\alpha)$-quantile of $S=|Y-f(X)|$, and the split conformal calibration step estimates that quantile from a calibration set, giving an excess length of order $O(n_c^{-\gamma})$ under a Hölder condition. The paper's central claim is that when $f$ is also learned, the whole procedure is a minimum-volume-set problem, and the correct objective is to minimize $Q(1-\alpha; |Y-f(X)|)$ over $f$ in the class $\mathcal{F}$. EffOrt approximately solves that problem with a gradient-based smooth quantile solver. Theorem 1 states that, with high probability, the interval length of EffOrt is at most the oracle length plus $2L(1/n_c + \sqrt{\log(2/\delta)/(2n_c)})^\gamma + 4L\varphi(\mathcal{F},\delta,n_\ell)^\gamma$, where the first terms come from calibration and the last from learning. This is the first finite-sample excess-volume bound that explicitly separates the two split steps and shows the learning term, governed by the complexity of $\mathcal{F}$, dominates the rate.
Load-bearing premise
The results assume a uniform high-probability bound $\varphi(\mathcal{F},\delta,n)$ on how well any function in the class $\mathcal{F}$ approximates the true probability that $|Y-f(X)|\le t$, simultaneously for all thresholds $t$; if the function class is complex and the learning sample is small, this bound may be large or unverified, and the main excess-volume guarantee becomes vacuous.
Editorial extensions
If this is right
- Split conformal calibration is asymptotically volume-optimal for a fixed base predictor, with rate $O(n_c^{-\gamma})$ under Hölder regularity.
- The learning objective for short intervals is the $(1-\alpha)$-quantile of absolute residuals, not squared error; MSE can be suboptimal under asymmetry and heavy tails.
- Excess volume decomposes additively into a calibration term ($n_c$) and a learning term ($\varphi(\mathcal{F},\delta,n_\ell)^\gamma$), so more learning data than calibration data is preferable.
- EffOrt provides a practical gradient-based solver via a smooth quantile approximation, and Ad-EffOrt extends the approach to covariate-adaptive interval widths.
- Marginal coverage at level $1-\alpha$ is preserved almost surely after the calibration step.
Reading between the lines
- The additive decomposition suggests an optimal data-split ratio could be derived by balancing $1/n_c$ against $\varphi(\mathcal{F},\delta,n_\ell)^\gamma$, a calculation the paper does not perform.
- The QAE objective may be useful beyond conformal prediction, for example as a robust regression loss that targets a prespecified quantile rather than the mean, especially under asymmetric noise.
- For adaptive intervals, a similar separation between learning and calibration errors could be obtained via Assumption 4; verifying that assumption for neural-network classes would be a natural next step.
- The bound implies that controlling the capacity of $\mathcal{F}$ (for instance through regularization) directly reduces prediction interval length, connecting interval efficiency to classical generalization bounds.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies volume (interval length) optimality in split conformal regression. It first observes that, for a fixed base predictor, the calibration step of split conformal prediction solves an empirical version of a minimum-volume-set problem, and it derives finite-sample excess-volume bounds for the calibrated interval (Proposition 1 and Corollary 1). It then argues that the learning step should minimize the (1-alpha)-quantile of absolute error rather than the mean squared error, and introduces EffOrt, a split conformal method that does this. Theorem 1 gives a coverage guarantee and an excess-volume bound that separates a calibration error term depending on n_c from a learning error term involving a uniform deviation phi(F,delta,n_l) (Assumption 2). The paper also introduces Ad-EffOrt for covariate-adaptive interval widths, with additional theoretical results in Appendix B.3, and reports synthetic and real-data experiments in which EffOrt and Ad-EffOrt are compared with standard split conformal, locally weighted conformal, and conformalized quantile regression.
Significance. If Theorem 1 is correct, the paper provides a genuinely finite-sample analysis of interval volume in conformal regression, a topic that is usually treated only empirically or asymptotically. The decomposition of excess volume into calibration and learning terms is a useful conceptual contribution, and the proposal to replace least-squares fitting with quantile-of-absolute-error fitting is well motivated and clearly explained. The paper also ships code and its experimental section compares several heavy-tailed and asymmetric settings, which is a strength. The central derivation is not circular: it uses DKW and Scott-Nowak style MVS theory as external tools. The main risks are that Assumption 2 is not instantiated for the function classes used in the experiments, and that Assumption 1 is stated per f rather than uniformly, while the proof of Theorem 1 uses common constants for the data-dependent predictor f_hat and the oracle f*. These issues are fixable but affect the main theorem as written.
major comments (3)
- [Section 3.4, Theorem 1 and Assumption 2] The learning-error term in Eq. (13) is entirely controlled by Assumption 2, which requires a uniform high-probability bound on sup_{t>=0, f in F} |P(|Y-f(X)|<=t) - empirical CDF|. This is substantially stronger than pointwise DKW because it must hold simultaneously over all thresholds and all functions in F. The paper gives a finite-class bound (Proposition 2) and generic VC/Rademacher bounds (Appendix B.2), but it never instantiates phi(F,delta,n_l) for the function classes used in the experiments. In particular, for the one-hidden-layer ReLU networks used in Appendix D.1, the VC dimension of the induced class can be large relative to n_l=1000, so the condition phi(F,delta,n_l)<=r may fail and Eq. (13) becomes uninformative. Since Theorem 1 is stated for a general F and the separation between calibration and learning errors is a central claim, the authors should either instantiate phi for the linear class of Section 5.1 and for the neural-network class of Appendix D.1, or explicitly restate Theorem 1 only for classes for which such a phi is shown to satisfy the required smallness condition, and verify that condition in the experiments.
- [Section 3.4 and Appendix A.2, Assumption 1] Assumption 1 is stated as: for every f in F and every alpha, there exist r, gamma, L such that the quantile function of |Y-f(X)| is locally (gamma,L)-Holder. This allows the constants L and gamma to depend on f. In the proof of Theorem 1 (Appendix A.2, Step 2), the same L and gamma are applied to both Q(.;|Y-f_hat(X)|) and Q(.;|Y-f*(X)|), and the statement of Eq. (13) uses a single pair L,gamma. Since f_hat is data-dependent and F is an arbitrary class, the per-f statement of the assumption does not provide the common constants needed by the proof. The assumption should be strengthened to require uniform constants over F (or the theorem should explicitly define L and gamma as suprema/infima over F and state the additional conditions needed for those to be finite and attained). As written, the proof of the central excess-volume bound has a gap.
- [Appendix B.3, Lemma 2 and Theorem 2] There are two inconsistencies in the adaptive-size theory. First, Lemma 2 states in Eq. (23) a bound against the oracle lambda(C^{1-alpha}_{f*,s*,t*}), but in the proof the bad event Theta_lambda is defined with lambda(C^{1-alpha+phi}_{f*,s*,t*}), and after the first implication the proof switches to s*_{1-alpha+phi}. These objects do not match, so the proof as written does not establish the stated lemma. Second, the proof of Theorem 2 applies Assumption 1 on an interval whose lower endpoint is 1-alpha-2phi(F,S,delta,n_l), which requires 2phi(F,S,delta,n_l)<=r, whereas the theorem only assumes phi(F,S,delta,n_l)<=r. Both issues appear local and fixable, but they must be corrected before the appendix results provide the 'strong theoretical grounding' that the paper claims for Ad-EffOrt.
minor comments (5)
- [Section 3.1 and Theorem 1] Proposition 1 assumes that (n_c+1)(1-alpha) is not an integer, but Theorem 1, which invokes Proposition 1, does not restate this condition. Please add the condition or explain why the atomless assumption makes the integer case harmless.
- [Appendix B.1, Proposition 3] Proposition 3 uses sqrt(log(1/delta)/(2n_c)) in the quantile argument, whereas Proposition 1 uses sqrt(log(2/delta)/(2n_c)). Please align the constants or explain the different confidence accounting.
- [Section 3.3.1 and Algorithm 1] The gradient formula in Eq. (27) and Algorithm 1 divide by sum_i B_i, which can be zero if no training loss falls within epsilon of the current quantile. Please state the safeguard used in practice or note the degeneracy explicitly.
- [Section 5.1 and Appendix D.1] The empirical QAE objective is nonconvex, but the paper does not specify the initialization theta_1 used by the gradient descent. Since the reported interval lengths may depend on this choice, please provide the initialization for the linear and neural-network experiments.
- [Introduction and Section 3.1] The abstract and introduction refer to 'the shortest oracle prediction interval' without the qualifier that the oracle is restricted to the interval class C^{const}_F (and C^{adap}_{F,S} for Ad-EffOrt). Please make this restriction explicit to avoid overclaiming.
Circularity Check
No significant circularity: the main EffOrt excess-volume bound is derived from DKW, explicit assumptions, and external MVS theory, not from fitted inputs or definitional equivalences.
full rationale
After walking the derivation chain, I find no circularity. Theorem 1 is obtained by (i) applying the DKW inequality to the residual CDF to control the calibration quantile (Proposition 1, Appendix A.1), (ii) using Assumption 2 as a uniform deviation bound that is instantiated for finite classes by a union bound plus DKW and for infinite classes by Rademacher/VC bounds (Proposition 2, Appendix B.2), and (iii) adapting Scott and Nowak's minimum-volume-set theorem to compare the empirical QAE solution with the oracle interval. None of these steps is a fitted constant renamed as a prediction, and the (1-alpha)-QAE objective is an explicit equivalence with interval-length minimization over the class C_const^F, not a hidden definition of the target. The proof of the secondary Theorem 2 cites Proposition 24 of Humbert et al. (2024) for an upper-coverage inequality; although that citation is author-overlapping, it is a supporting lemma for the adaptive-size extension and the central EffOrt bound does not reduce to it. The absence of a concrete phi for the neural-network class used in the appendix is a limitation of Assumption 2, but a limitation is not a circular reduction.
Assumptions & free parameters
free parameters (4)
- smoothing parameter ε =
0.1
- gradient descent iterations n_iter =
1000
- step-size schedule =
{(1/t)^0.6}_{t=1}^{n_iter}
- random forest max depth =
5
assumptions (6)
- domain assumption Assumption 1: the quantile function Q(·;|Y−f(X)|) is locally (γ,L)-Hölder on [1−α−r, 1−α+r] for every f∈F.
- domain assumption Assumption 2: a uniform deviation bound φ(F,δ,n) exists for the coverage function over all f∈F and t≥0.
- domain assumption Atomless distribution of Y.
- domain assumption Exchangeability or i.i.d. of learning and calibration samples.
- domain assumption Assumption 3: expected size of nested sets grows linearly in t (a,b constants).
- domain assumption Assumption 4: uniform deviation bounds for the adaptive problem (ϕ(F,S,δ,n) and ψ(S,δ,n)).
Cite this review
Pith. "Pith review of On Volume Minimization in Conformal Regression." pith.science (2026). https://pith.science/paper/OJCDFQ2R
@misc{pith2026250209985,
author = {Pith},
title = {Pith review of: On Volume Minimization in Conformal Regression},
year = {2026},
howpublished = {\url{https://pith.science/paper/OJCDFQ2R}},
note = {Machine review of arXiv:2502.09985}
}
read the original abstract
We study the question of volume optimality in split conformal regression, a topic still poorly understood in comparison to coverage control. Using the fact that the calibration step can be seen as an empirical volume minimization problem, we first derive a finite-sample upper-bound on the excess volume loss of the interval returned by the classical split method. This important quantity measures the difference in length between the interval obtained with the split method and the shortest oracle prediction interval. Then, we introduce EffOrt, a methodology that modifies the learning step so that the base prediction function is selected in order to minimize the length of the returned intervals. In particular, our theoretical analysis of the excess volume loss of the prediction sets produced by EffOrt reveals the links between the learning and calibration steps, and notably the impact of the choice of the function class of the base predictor. We also introduce Ad-EffOrt, an extension of the previous method, which produces intervals whose size adapts to the value of the covariate. Finally, we evaluate the empirical performance and the robustness of our methodologies.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Angelopoulos, A. N. and Bates, S. (2023). Conformal prediction: A gentle introduction. Foundations and Trends in Machine Learning , 16(4):494--591
work page 2023
-
[2]
Bai, Y., Mei, S., Wang, H., Zhou, Y., and Xiong, C. (2022). Efficient and differentiable conformal prediction with general function classes. In International Conference on Learning Representations
work page 2022
-
[3]
Biau, G. and Patra, B. (2011). Sequential quantile prediction of time series. IEEE Transactions on Information Theory , 57(3):1664--1674
work page 2011
-
[4]
Chernozhukov, V., W \"u thrich, K., and Zhu, Y. (2021). Distributional conformal prediction. Proceedings of the National Academy of Sciences , 118(48):e2107794118
2021
-
[5]
E., Wachter, A., and Zavala, V
Curtis, F. E., Wachter, A., and Zavala, V. M. (2018). A sequential algorithm for solving nonlinear optimization problems with chance constraints. SIAM Journal on Optimization , 28(1):930--958
work page 2018
-
[6]
Dvoretzky, A., Kiefer, J., and Wolfowitz, J. (1956). Asymptotic minimax character of the sample distribution function and of the classical multinomial estimator. The Annals of Mathematical Statistics , pages 642--669
1956
-
[7]
Fontana, M., Zeni, G., and Vantini, S. (2023). Conformal prediction: a unified review of theory and new challenges. Bernoulli , 29(1):1--23
2023
-
[8]
Gupta, C., Kuchibhotla, A. K., and Ramdas, A. (2022). Nested conformal prediction and quantile out-of-bag ensemble methods. Pattern Recognition , 127:108496
work page 2022
Show all 39 references
-
[9]
and Rubinfeld, D
Harrison Jr, D. and Rubinfeld, D. L. (1978). Hedonic housing prices and the demand for clean air. Journal of environmental economics and management , 5(1):81--102
1978
-
[10]
Howard, S. R. and Ramdas, A. (2022). Sequential estimation of quantiles with applications to a/b testing and best-arm identification. Bernoulli , 28(3):1704--1728
2022
-
[11]
Huber, P. J. (1964). Robust Estimation of a Location Parameter . The Annals of Mathematical Statistics , 35(1):73 -- 101
1964
-
[12]
Humbert, P., Le Bars, B., Bellet, A., and Arlot, S. (2024). Marginal and training-conditional guarantees in one-shot federated conformal prediction. arXiv preprint arXiv:2405.12567
2024 arXiv
-
[13]
Izbicki, R., Shimizu, G., and Stern, R. B. (2022). Cd-split and hpd-split: Efficient conformal regions in high dimensions. Journal of Machine Learning Research , 23(87):1--32
2022
-
[14]
Kiyani, S., Pappas, G., and Hassani, H. (2024). Length optimization in conformal prediction. arXiv preprint arXiv:2406.18814
2024 arXiv
-
[15]
and Bassett Jr, G
Koenker, R. and Bassett Jr, G. (1978). Regression quantiles. Econometrica: journal of the Econometric Society , pages 33--50
1978
-
[16]
and Hallock, K
Koenker, R. and Hallock, K. F. (2001). Quantile regression. Journal of economic perspectives , 15(4):143--156
2001
-
[17]
J., and Wasserman, L
Lei, J., G’Sell, M., Rinaldo, A., Tibshirani, R. J., and Wasserman, L. (2018). Distribution-free predictive inference for regression. Journal of the American Statistical Association , 113(523):1094--1111
2018
-
[18]
Lei, J., Robins, J., and Wasserman, L. (2013). Distribution-free prediction sets. Journal of the American Statistical Association , 108(501):278--287
2013
-
[19]
and Wasserman, L
Lei, J. and Wasserman, L. (2014). Distribution-free prediction bands for non-parametric regression. Journal of the Royal Statistical Society Series B: Statistical Methodology , 76(1):71--96
2014
-
[20]
Liang, R., Zhu, W., and Barber, R. F. (2024). Conformal prediction after efficiency-oriented model selection. arXiv preprint arXiv:2408.07066
2024 arXiv
-
[21]
and Larson, J
Luo, F. and Larson, J. (2022). An empirical quantile estimation approach to nonlinear optimization problems with chance constraints. arXiv preprint arXiv:2211.00675
2022 arXiv
-
[22]
Manokhin, V. (2022). Awesome conformal prediction
2022
-
[23]
Massart, P. (1990). The tight constant in the D voretzky- K iefer- W olfowitz inequality. The Annals of Probability , pages 1269--1283
1990
-
[24]
Mohri, M. (2018). Foundations of machine learning
2018
-
[25]
and Moguerza, J
Munoz, A. and Moguerza, J. M. (2006). Estimation of high-density regions using one-class neighbor machines. IEEE Transactions on Pattern Analysis and Machine Intelligence , 28(3):476--480
2006
-
[26]
Nash, W., Sellers, T., Talbot, S., Cawthorn, A., and Ford, W. (1994). Abalone . UCI Machine Learning Repository. DOI : https://doi.org/10.24432/C55C7W
1994 doi
-
[27]
Papadopoulos, H., Gammerman, A., and Vovk, V. (2008). Normalized nonconformity measures for regression conformal prediction. In Proceedings of the IASTED International Conference on Artificial Intelligence and Applications (AIA 2008) , pages 64--69
2008
-
[28]
Papadopoulos, H., Proedrou, K., Vovk, V., and Gammerman, A. (2002). Inductive confidence machines for regression. In European Conference on Machine Learning , pages 345--356. Springer
2002
-
[29]
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). Scikit-learn: Machine learning in P ython. Jou...
2011
-
[30]
R., and Waachter, A
Pena-Ordieres, A., Luedtke, J. R., and Waachter, A. (2020). Solving chance-constrained problems via a smooth sample-based nonlinear approximation. SIAM Journal on Optimization , 30(3):2221--2250
2020
-
[31]
and Yao, Q
Polonik, W. and Yao, Q. (2000). Conditional minimum volume predictive regions for stochastic processes. Journal of the American Statistical Association , 95(450):509--519
2000
-
[32]
Romano, Y., Patterson, E., and Candes, E. (2019). Conformalized quantile regression. Advances in neural information processing systems , 32
2019
-
[33]
C., Shawe-Taylor, J., Smola, A
Sch \"o lkopf, B., Platt, J. C., Shawe-Taylor, J., Smola, A. J., and Williamson, R. C. (2001). Estimating the support of a high-dimensional distribution. Neural computation , 13(7):1443--1471
2001
-
[34]
and Nowak, R
Scott, C. and Nowak, R. (2005). Learning minimum volume sets. In Weiss, Y., Sch\" o lkopf, B., and Platt, J., editors, Advances in Neural Information Processing Systems , volume 18. MIT Press
2005
-
[35]
T., Doucet, A., et al
Stutz, D., Cemgil, A. T., Doucet, A., et al. (2022). Learning optimal conformal classifiers. ICLR
2022
-
[36]
Vovk, V. (2012). Conditional validity of inductive conformal predictors. In Asian conference on machine learning , pages 475--490. PMLR
2012
-
[37]
Vovk, V., Gammerman, A., and Shafer, G. (2005). Algorithmic learning in a random world . Springer Science & Business Media
2005
-
[38]
and Kuchibhotla, A
Yang, Y. and Kuchibhotla, A. K. (2024). Selection and aggregation of conformal prediction sets. Journal of the American Statistical Association , pages 1--13
2024
-
[39]
Yeh, I.-C. (1998). Modeling of strength of high-performance concrete using artificial neural networks. Cement and Concrete research , 28(12):1797--1808
1998
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.