REVIEW 2 major objections 4 minor 1 cited by
Self-supervised Conformal Prediction for Uncertainty Quantification in Imaging Problems
T0 review · 2 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read The paper proposes a conformal prediction method for imaging that calibrates uncertainty from noisy measurements alone, using Stein's unbiased risk estimator in place of ground-truth scores.
desk verdict SURE-based self-calibration for conformal prediction is a genuinely useful idea with encouraging experiments, but the paper's central quantile-agreement claim is false without extra conditions, so the coverage guarantee is unproven. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the SURE estimate of the measurement-space reconstruction error. For the score $s(x,y)=\|Ax - A\hat{x}(y)\|_2^2/m$, the corresponding SURE value is $SURE(y)=\|y - A\hat{x}(y)\|_2^2/m - \sigma^2 + (2\sigma^2/m)\operatorname{div}(A\hat{x}(y))$, computed from the measurement alone; the divergence term is approximated by Hutchinson's stochastic trace estimator using automatic differentiation. Calibration then proceeds exactly as split conformal prediction, but with $S_i=SURE(y_i)$ in place of $s(x_i,y_i)$, and the final prediction set is the sublevel set of the same score below the SURE-calibrated quantile. The full-rank assumption on $A$ keeps this sublevel set bounded and the method applicable to any differentiable estimator.
What would settle it
A concrete check: set $A=I_m$, fix $\sigma$, and use the do-nothing estimator $\hat{x}(y)=y$. Then $SURE(y)=\sigma^2$ is constant, while the true scores $\|x-y\|_2^2/m$ follow a scaled chi-squared distribution with mean $\sigma^2$. For confidence levels whose true score quantile differs from $\sigma^2$, the SURE-calibrated quantile gives coverage that deviates from $1-\alpha$ by an exactly computable amount, showing where unbiasedness alone is insufficient and how large a gap remains in more realistic cases.
Extended reading notes
Core claim
The paper claims that in the linear Gaussian model $Y|X=x \sim \mathcal{N}(Ax,\sigma^2 I_m)$ with full-rank $A$, the quantity $SURE(y) = \|y - A\hat{x}(y)\|_2^2/m - \sigma^2 + (2\sigma^2/m)\operatorname{div}(A\hat{x}(y))$ is an unbiased estimate of the score's conditional expectation, and that when $m$ is large this estimate is accurate enough that the empirical $\lceil M(1-\alpha)\rceil/M$-quantile of $\{SURE(y_i)\}$ can serve as a conformal calibration threshold. The prediction set is $\{x \in \mathbb{R}^n: \|Ax - A\hat{x}(y)\|_2^2/m \le \hat{Q}_\alpha\}$, with full rank of $A$ keeping the set bounded. If correct, the method transfers conformal prediction's near-exact marginal coverage guarantee to settings where only noisy measurements are observed, without any ground-truth calibration data.
Load-bearing premise
The argument depends on the empirical distribution of the SURE values sitting close to the distribution of the true scores, so that a quantile of one is a valid conformal threshold for the other; SURE's unbiasedness for the mean does not by itself control the quantiles.
Editorial extensions
If this is right
- Any differentiable image-restoration estimator for a linear Gaussian inverse problem with full-rank $A$ can be equipped with conformal prediction sets using only exchangeable noisy measurements, with no paired ground-truth images needed for calibration.
- Because calibration happens in the measurement domain on data of the same type encountered at deployment, the method sidesteps the calibration-to-deployment distribution shift that supervised conformal prediction can suffer.
- The same SURE principle can be combined with self-supervised training objectives, so an entire uncertainty-quantified restoration pipeline can be built from measurements alone, as demonstrated by the denoising experiment.
- In the large-$m$ regime where SURE's variance is small, the coverage gap between self-supervised and supervised conformal prediction should vanish; the paper reports coverage curves for denoising and deblurring that are close to the supervised baseline across confidence levels.
Reading between the lines
- A direct way to stress-test the method beyond the paper's experiments would be to bootstrap the SURE calibration sample and measure the spread of $\hat{Q}_\alpha$; when that spread is large relative to the score quantile's intrinsic scale, coverage will degrade, giving practitioners a cheap diagnostic.
- The same measurement-only calibration idea should transfer to other noise models with unbiased risk estimators, such as Poisson or Poisson-Gaussian noise; the paper lists this as future work but does not claim a result.
- The full-rank requirement on $A$ is what keeps prediction sets bounded; for rank-deficient operators one could project scores onto the measurement range or add equivariance constraints, an extension the paper mentions as a perspective rather than a demonstrated method.
- An immediate testable refinement of the paper's evaluation would be to compare expected set volumes as well as coverage, since a coverage guarantee alone does not indicate how informative the resulting prediction regions are.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a self-supervised conformal prediction method for image restoration. The key idea is to replace the usual calibration scores s(x_i, y_i) = ||A x_i - A xhat(y_i)||^2/m, which require ground-truth images, with SURE estimates computed from noisy measurements alone. The authors argue that because SURE is unbiased and accurate in high dimensions, the quantiles of the SURE estimates will be close to the quantiles of the true scores, so the resulting conformal prediction sets nearly maintain the desired coverage. The method is evaluated on image denoising with a SURE-trained DRUNet and on non-blind deblurring with the Polyblur estimator, reporting empirical coverage close to the supervised conformal baseline on the DIV2K dataset.
Significance. If the central claim were correct, the method would provide a valuable practical tool: conformal calibration without any ground-truth data for a broad class of linear Gaussian inverse problems. The experimental setup is sensible, the empirical comparison with supervised conformal prediction is informative, and the use of SURE with a Hutchinson trace approximation is computationally attractive. However, the paper's theoretical foundation is the load-bearing weakness: the assertion that quantiles of SURE estimates approximate quantiles of true scores is not merely unproven; it is false in general, as demonstrated by a simple estimator satisfying all of the paper's assumptions. Because the abstract and Section 3 promise 'remarkably accurate' and 'nearly exact' coverage, the unsupported and actually false claim undermines the paper's main contribution.
major comments (2)
- [Section 3, after Eq. (5)] The central claim that 'the conformal calibration quantiles obtained from SURE will be in close agreement with the true quantiles of s(X,Y)' is false as stated. SURE(Y) is an unbiased estimator of the conditional mean risk E[s(X,Y)|X=x], not of the realized score s(X,Y). Replacing each realized score by an estimate of its conditional mean can shift the upper quantile of the calibration distribution downward whenever the conditional variance of s(X,Y)|X is positive. This is not a mere technicality: take A = I_m and xhat(y) = y, which satisfies the stated assumptions (full-rank A, differentiable estimator). Then Eq. (5) gives SURE(y) = sigma^2 exactly, while the true scores are s(x,y) = ||x - y||^2/m, which under the Gaussian noise model follow (sigma^2/m) chi^2_m. With M calibration samples, Algorithm 1 returns the threshold sigma^2, and for a new test point the coverage is P(chi^2_m <= m), which is approximately 0.5 for large m, far below 0.9 for alpha = 0.1. Thus the method does not achieve the advertised coverage even in the infinite-calibration-sample limit. To make the central claim valid, the authors would need a condition on the estimator (e.g., small conditional variance of s(X,Y)|X) or a quantitative bound on the quantile error; no such condition or bound is provided.
- [Section 3, Algorithm 1 and surrounding text] The paper does not provide any finite-sample or asymptotic theorem that would justify the coverage property of the proposed self-supervised calibration. The only formal guarantee in the paper, Eq. (2), applies to split conformal prediction with the true scores {s(X_i,Y_i)}; Algorithm 1 instead uses {SURE(Y_i)}, and the finite-sample correction used for supervised conformal prediction is not shown to carry over. Moreover, the text states that each prediction set C(y_i) uses a leave-one-out quantile with the i-th SURE estimate removed, but Algorithm 1 line 4 computes a single quantile over all M estimates without any leave-one-out step. The pseudocode should be reconciled with the text, and a rigorous statement of what the algorithm actually guarantees (if anything) must be supplied.
minor comments (4)
- [Section 3, Eq. (8)] Equation (8) appears to have a typo: the Hutchinson approximation is written as (1/K) n^T J n^T h(y), which does not have consistent dimensions; it should likely be (1/K) n^T J h(y) n or the vector form should be made explicit.
- [Section 4, Figures 2-4] The figures report empirical coverage and histograms, but there are no error bars or standard deviations over multiple test folds; given the small test size (200 images), error bars would help assess the variability of the reported coverage values.
- [Section 3, Eq. (3)] The display in Eq. (3) uses both a norm notation with subscript Sigma(y) and an explicit quadratic form; please use one consistent notation to avoid confusion.
- [Algorithm 1] The quantile level in Algorithm 1 is written as top ceil((M+1)(1-alpha))/M, while the text describes a leave-one-out quantile over M-1 values; please clarify which finite-sample correction is actually used in the experiments.
Circularity Check
No significant circularity: SURE is an externally grounded unbiased risk estimator, the conformal quantile is computed from calibration measurements, and coverage is evaluated on held-out test data.
full rationale
The paper's derivation chain is not circular. The non-conformity score s(x,y) is defined from the forward operator and the estimator (Eq. 4), and SURE in Eq. (5) is Stein's externally established unbiased risk estimator, not a parameter fitted to the quantity being predicted. The conformal threshold is computed from SURE values on a calibration set and the reported coverage is evaluated on separate measurement-truth test pairs; no ground-truth score or coverage value is used to construct the threshold. The central coverage claim rests on the informal premise that quantiles of SURE values will be close to quantiles of true scores, which is an unproved (and in some cases false) concentration/quantile assertion, but that is a correctness or validity gap rather than a circular reduction: the method does not define the target in terms of its own output, and it does not rename a fitted value as a prediction. Self-citations in the manuscript (e.g., equivariant bootstrapping, uncertainty literature) are contextual and not load-bearing for the proposed SURE calibration step. Therefore the appropriate circularity finding is no significant circularity, with score 0.
Assumptions & free parameters
free parameters (1)
- Number K of Hutchinson samples
assumptions (5)
- domain assumption Observations follow a Gaussian noise model (Y|X=x*) ~ N(Ax*, sigma^2 I_m) with known sigma^2.
- domain assumption The forward operator A is full-rank but possibly ill-conditioned.
- domain assumption The estimator xhat(y) is differentiable almost everywhere so the divergence term and Hutchinson's trace estimator apply.
- domain assumption Calibration measurements are exchangeable with the test measurement.
- ad hoc to paper The quantile of SURE estimates is close to the quantile of true scores, so SURE estimation noise does not materially bias the calibrated threshold.
Cite this review
Pith. "Pith review of Self-supervised Conformal Prediction for Uncertainty Quantification in Imaging Problems." pith.science (2026). https://pith.science/paper/3N3XH5YW
@misc{pith2026250205127,
author = {Pith},
title = {Pith review of: Self-supervised Conformal Prediction for Uncertainty Quantification in Imaging Problems},
year = {2026},
howpublished = {\url{https://pith.science/paper/3N3XH5YW}},
note = {Machine review of arXiv:2502.05127}
}
read the original abstract
Most image restoration problems are ill-conditioned or ill-posed and hence involve significant uncertainty. Quantifying this uncertainty is crucial for reliably interpreting experimental results, particularly when reconstructed images inform critical decisions and science. However, most existing image restoration methods either fail to quantify uncertainty or provide estimates that are highly inaccurate. Conformal prediction has recently emerged as a flexible framework to equip any estimator with uncertainty quantification capabilities that, by construction, have nearly exact marginal coverage. To achieve this, conformal prediction relies on abundant ground truth data for calibration. However, in image restoration problems, reliable ground truth data is often expensive or not possible to acquire. Also, reliance on ground truth data can introduce large biases in situations of distribution shift between calibration and deployment. This paper seeks to develop a more robust approach to conformal prediction for image restoration problems by proposing a self-supervised conformal prediction method that leverages Stein's Unbiased Risk Estimator (SURE) to self-calibrate itself directly from the observed noisy measurements, bypassing the need for ground truth. The method is suitable for any linear imaging inverse problem that is ill-conditioned, and it is especially powerful when used with modern self-supervised image restoration techniques that can also be trained directly from measurement data. The proposed approach is demonstrated through numerical experiments on image denoising and deblurring, where it delivers results that are remarkably accurate and comparable to those obtained by supervised conformal prediction with ground truth data.
Figures
Figures from the paper (1 more)
Forward citations
Cited by 1 Pith paper
-
Hypothesis Testing in Imaging Inverse Problems
Semantic hypotheses about reconstructed images are tested using CLIP embeddings and e-values, with calibrated Type I error control and higher power than zero-shot CLIP classification.
Reference graph
Works this paper leans on
-
[1]
J. Kaipio and E. Somersalo,Statistical and computational inverse problems . Germany: Springer, 2004
work page 2004
-
[2]
Efficient bayesian computation by proximal markov chain monte carlo: When langevin meets moreau,
A. Durmus, E. Moulines, and M. Pereyra, “Efficient bayesian computation by proximal markov chain monte carlo: When langevin meets moreau,” SIAM Journal on Imaging Sciences , vol. 11, no. 1, pp. 473–506, 2018
work page 2018
-
[3]
Maximum-a-posteriori estimation with bayesian confidence re- gions,
M. Pereyra, “Maximum-a-posteriori estimation with bayesian confidence re- gions,” SIAM Journal on Imaging Sciences,vol.10,no.1,pp.285–302,2017
work page 2017
-
[4]
Bayesian imaging using plug & play priors: when langevin meets tweedie,
R. Laumont, V. D. Bortoli, A. Almansa, J. Delon, A. Durmus, and M. Pereyra, “Bayesian imaging using plug & play priors: when langevin meets tweedie,” SIAM Journal on Imaging Sciences , vol. 15, no. 2, pp. 701– 737, 2022
work page 2022
-
[5]
Bayesian imaging with data- driven priors encoded by neural networks,
M. Holden, M. Pereyra, and K. C. Zygalakis, “Bayesian imaging with data- driven priors encoded by neural networks,”SIAM Journal on Imaging Sci- ences, vol. 15, no. 2, pp. 892–924, 2022
work page 2022
-
[6]
Do bayesian imaging methods report trustworthy probabilities?
D. Y. W. Thong, C. K. Mbakam, and M. Pereyra, “Do bayesian imaging methods report trustworthy probabilities?” 2024. [Online]. Available: https://arxiv.org/abs/2405.08179
arXiv 2024
-
[7]
Equivariant bootstrapping for uncertainty quantification in imaging inverse problems,
J. Tachella and M. Pereyra, “Equivariant bootstrapping for uncertainty quantification in imaging inverse problems,”Proceedings of Machine Learn- ing Research, vol. 238, pp. 4141–4149, 2024
work page 2024
-
[8]
M. Cherif, T. I. Liaudat, J. Kern, C. Kervazo, and J. Bobin, “Uncertainty quantification for fast reconstruction methods using augmented equivariant bootstrap: Application to radio interferometry,” 2024. [Online]. Available: https://arxiv.org/abs/2410.23178
arXiv 2024
Show all 21 references
-
[9]
A gentle introduction to conformal prediction and distribution-free uncertainty quantification,
A. N. Angelopoulos and S. Bates, “A gentle introduction to conformal prediction and distribution-free uncertainty quantification,”arXiv preprint arXiv:2107.07511, 2021
2021 arXiv
-
[10]
Posterior-variance– based error quantification for inverse problems in imaging,
D. Narnhofer, A. Habring, M. Holler, and T. Pock, “Posterior-variance– based error quantification for inverse problems in imaging,”SIAM Journal on Imaging Sciences , vol. 17, no. 1, pp. 301–333, 2024
2024
-
[11]
Conformal uncertainty sets for robust optimiza- tion,
C. Johnstone and B. Cox, “Conformal uncertainty sets for robust optimiza- tion,” in Conformal and Probabilistic Prediction and Applications . PMLR, 2021, pp. 72–90
2021
-
[12]
Estimation of the mean of a multivariate normal distribution,
C. M. Stein, “Estimation of the mean of a multivariate normal distribution,” The annals of Statistics , pp. 1135–1151, 1981
1981
-
[13]
Second-order stein: Sure for sure and other applications in high-dimensional inference,
P. C. Bellec and C.-H. Zhang, “Second-order stein: Sure for sure and other applications in high-dimensional inference,”The Annals of Statistics , vol. 49, no. 4, pp. 1864–1903, 2021
1903
-
[14]
Unbiased risk estimates for singular value thresholding and spectral estimators,
E. J. Candès, C. A. Sing-Long, and J. D. Trzasko, “Unbiased risk estimates for singular value thresholding and spectral estimators,”IEEE Transactions on Signal Processing, vol. 61, no. 19, pp. 4643–4657, 2013. 12 J.M. Everink, B. Tamo Amougou and M. Pereyra
2013
-
[15]
Degrees of freedom in lasso problems,
R. J. Tibshirani and J. Taylor, “Degrees of freedom in lasso problems,”The Annals of Statistics , vol. 40, no. 2, pp. 1198–1232, 2012
2012
-
[16]
Monte-Carlo SURE: A black-box op- timization of regularization parameters for general denoising algorithms,
S. Ramani, T. Blu, and M. Unser, “Monte-Carlo SURE: A black-box op- timization of regularization parameters for general denoising algorithms,” IEEE Transactions on image processing, vol. 17, no. 9, pp. 1540–1554, 2008
2008
-
[17]
On divergence ap- proximations for unsupervised training of deep denoisers based on stein’s unbiased risk estimator,
S. Soltanayev, R. Giryes, S. Y. Chun, and Y. C. Eldar, “On divergence ap- proximations for unsupervised training of deep denoisers based on stein’s unbiased risk estimator,” inICASSP 2020 - 2020 IEEE International Con- ference on Acoustics, Speech and Signal Processing (ICASSP...
2020
-
[18]
Ntire 2017 challenge on single image super- resolution: Dataset and study,
E. Agustsson and R. Timofte, “Ntire 2017 challenge on single image super- resolution: Dataset and study,” inThe IEEE Conference on Computer Vi- sion and Pattern Recognition (CVPR) Workshops , July 2017
2017
-
[19]
Plug- and-play imagerestorationwith deepdenoiserprior,
K. Zhang, Y. Li, W. Zuo, L. Zhang, L. Van Gool, and R. Timofte, “Plug- and-play imagerestorationwith deepdenoiserprior,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 44, no. 10, pp. 6360–6376, 2021
2021
-
[20]
Unsure: Unknown noise level stein’s unbiased risk estimator,
J. Tachella, M. Davies, and L. Jacques, “Unsure: Unknown noise level stein’s unbiased risk estimator,”arXiv preprint arXiv:2409.01985 , 2024
2024 arXiv
-
[21]
Poly- blur: Removing mild blur by polynomial reblurring,
M. Delbracio, I. Garcia-Dorado, S. Choi, D. Kelly, and P. Milanfar, “Poly- blur: Removing mild blur by polynomial reblurring,”IEEE Transactions on Computational Imaging, vol. 7, pp. 837–848, 2021
2021
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.