REVIEW 5 major objections 5 minor 31 references
Conformal coronary calcification volume estimation with conditional coverage via histogram clustering
T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Cluster-based conformal prediction yields calibrated calcium-volume intervals without retraining, with better triage than conventional conformal intervals.
desk verdict A clean, honest application of clustered conformal prediction to calcium scoring; the coverage guarantee is undermined by the patch-to-full-image transfer that the authors themselves flag. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the histogram of the network's output probability map, clipped to voxels with probability above 0.2, which summarizes how much of the calcium signal sits in uncertain probability bands without being dominated by background voxels. A constrained K-means algorithm groups calibration histograms into clusters of similar predicted volume and uncertainty, and each cluster receives its own conformal quantile $q_c$ computed from the score function $S_i=\max(L_i-V_t^i,\;V_t^i-H_i)$, where $L_i$ and $H_i$ are pre-calibrated low and high volume bounds and $V_t^i$ is the true volume. At inference, a new scan's histogram is assigned to the nearest cluster center by Euclidean distance, and the cluster's correction is applied to form $[L_i-q_c,\;H_i+q_c]$. This turns one global correction into several conditional corrections, which is what delivers conditional coverage.
What would settle it
Take the calibrated ensemble and run it on a new unselected CT cohort from a different scanner family or protocol; if full-volume empirical coverage falls well below 0.85, for instance below 0.75, or if low-volume clusters show markedly worse coverage than high-volume clusters, the patch-learned cluster quantiles do not transfer and the exchangeability caveat is the cause.
Extended reading notes
Core claim
The central claim is that cluster-specific conformal corrections, conditioned on the histogram of the predicted probability map, provide statistically calibrated predictive intervals for coronary calcium volume and preserve more useful clinical information than a single global conformal correction. After tuning on augmented patches, the method was applied to deterministic, MC-dropout, and deep-ensemble 3D U-Net segmentations. The calibrated ensemble reached 0.83 empirical coverage against a target of 0.85, with a CAr-CEr of 54.1%, versus 0.82 coverage and 27.2% CAr-CEr for conventional conformal prediction. The authors interpret this as evidence that conditioning on histogram clusters makes the intervals more informative for triaging patients into Agatston-derived risk categories without sacrificing coverage.
Load-bearing premise
The reliability of the intervals rests on the calibration patches and the full test volumes being exchangeable, and the paper itself flags that moving from 16x256x256 augmented patches to whole-volume images may violate this assumption.
Editorial extensions
If this is right
- Calibration can be attached to an already trained segmentation network, so the method changes only post-processing and not the model itself.
- With the proposed calibration, the ensemble model's full-volume test coverage reached 0.83 against a 0.85 target, matching conventional conformal prediction while improving CAr-CEr from 27.2% to 54.1%.
- The same cluster-based calibration recipe worked for deterministic, MC-dropout, and ensemble predictions, suggesting the benefit does not depend on a particular uncertainty mechanism.
- In the triage read, the calibrated ensemble would have sent roughly 70% fewer cases to review while keeping risk-category errors near 2.4%.
- Cluster-conditional intervals could be recomputed for any target coverage without retraining, as long as the calibration set remains representative.
Reading between the lines
- The paper leaves implicit that the same histogram-conditioning idea could attach calibrated intervals to any segmentation-based volume estimate, such as brain lesion load or liver fat fraction, wherever background voxels dominate.
- The patch-to-full-image transfer is the main stress point; a natural extension is re-estimating or reweighting cluster quantiles on a small set of full-volume calibration scans before clinical deployment.
- Cluster assignment drift could serve as a cheap online monitoring signal: if incoming probability-map histograms stop falling near calibration cluster centers, that is an early warning that coverage may no longer hold.
- Because the conditioning is on the segmentation histogram rather than patient covariates, the method may indirectly absorb some scanner or protocol shifts, but the paper does not test that claim.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a cluster-based conditional conformal prediction (CP) method for estimating coronary calcium volume from CT scans. The method uses the histogram of the predicted probability map to assign each case to one of several clusters, and applies a cluster-specific conformal correction factor to the predictive interval. The authors compare this approach with conventional CP and with no calibration for three segmentation models (deterministic, MC-dropout, ensemble) on a private dataset of 295 CT scans, split into 142/72/81 for training/calibration/test. They report similar empirical coverage to conventional CP (0.83 vs 0.82 on the test set) and a higher triage metric CAr-CEr (54.1% vs 27.2%). The main claim is that the method yields calibrated coverage without retraining and improves confident risk-category assignment.
Significance. The paper addresses a clinically relevant problem: incidental coronary calcium detection and quantification, where over-reporting can cause anxiety and burden on the medical system. The proposed method is simple, requires no model retraining, and the idea of using probability histograms as a conditioning variable for conformal prediction is interesting and potentially useful in medical imaging. If the reported results are robust, the method could reduce radiologist workload by triaging confident cases. The paper also honestly discusses the limitation that moving from patches to full images violates the exchangeability assumption underlying conformal prediction. However, the evaluation is limited to a single private dataset with a small test set (81 patients), no error bars are provided, and there is an arithmetic inconsistency in the reported ensemble counts. These issues currently prevent a strong recommendation.
major comments (5)
- [Section 2.3.2] The histogram clustering step does not specify whether the histograms are normalized. Calibration is performed on 16x256x256 patches, while inference is performed on full-volume heart crops. Even after clipping to probability > 0.2, the absolute bin counts scale with the number of slices and the heart region size. The Euclidean distance between a full-image histogram and the patch-derived cluster centers will therefore be dominated by the total count rather than by the shape of the distribution, making the cluster assignment at inference questionable. The authors should either normalize the histograms (e.g., by the total number of voxels above 0.2) or otherwise demonstrate scale invariance. Without this, the conditional coverage claim is not well supported.
- [Section 4 / Abstract] The abstract claims 'calibrated coverage,' but the conformal guarantee rests on exchangeability between calibration and test data. The authors explicitly state in Section 4 that moving from patches to full images 'might mean that the exchangeability hypothesis is no longer respected.' Consequently, the reported test coverage (0.83) is an empirical observation on a single test set, not a guaranteed coverage. The paper should temper the calibration claim or provide additional evidence, such as comparing cluster assignments on paired patch and full-image predictions, or using a weighted conformal approach to account for the shift. A bootstrap confidence interval for the coverage would also help quantify the uncertainty.
- [Section 3.2 / Figure 4] The text reports '54 confident accurate, 15 uncertain accurate, 2 confident erroneous and 10 uncertain erroneous' for the ensemble cluster-based method. These counts sum to 81, but they imply CAr-CEr = 54/(54+15) - 2/(2+10) = 0.616, whereas Table 3 reports 54.1% for the cluster-based ensemble. Please correct the counts or the reported metric. This inconsistency affects the headline triage result and must be resolved.
- [Table 3] The test-set metrics are point estimates without error bars or significance tests. With only 81 patients and very few confident errors (e.g., 2 in the ensemble cluster-based result), the reported difference in CAr-CEr between conventional CP (27.2%) and the proposed method (54.1%) could be within sampling variability. Please provide confidence intervals (e.g., via bootstrapping) and, ideally, the full contingency tables for each model and calibration method.
- [Section 2.4.2 and 3.1] The target coverage (85%) and the number of clusters (3) are chosen on validation data with 5 random splits. While this is a reasonable model selection strategy, it means the empirical coverage on the test set is not a pure evaluation of the conformal guarantee at the nominal 85% level, because alpha was selected based on validation performance. The authors should clarify this and report test-set performance for a range of alpha values or use a nested splitting protocol.
minor comments (5)
- [References] The citation numbering is inconsistent. In Section 2.2.2, the composed Dice and weighted focal loss is cited as [14], but the focal loss paper is [15]; MCDropout is cited as [15] but should be [16]; ensembling is cited as [16] but should be [17]. In Section 2.3.2, constrained K-means is cited as [18], but the constrained K-means reference is [20]. Please fix the numbering throughout.
- [Section 2.3.2] The histogram clipping threshold of 0.2 is described as 'arbitrary.' Please discuss the sensitivity of the method to this threshold and to the number of histogram bins, which are not specified.
- [Section 2.4.1] The interpretation of CAr-CEr is slightly imprecise. If CAr-CEr = 0, the confidence labels are independent of correctness; negative values mean errors are more likely to be confident than accurates. Consider rephrasing.
- [Section 3.2] The sentence 'reviewing 70% less cases led to only 2.4% errors in risk category predictions' is unclear. Please specify the denominator: 2 errors out of the 56 confident cases is 3.6%, while 2 out of 81 is 2.5%. Clarify the calculation.
- [Figure 4] The caption describes blue/red colors, but the figure is not visible in the text; ensure the final figure is legible and the colors are distinguishable.
Circularity Check
No significant circularity: conformal calibration is an independent held-out procedure and the reported coverage/triage metrics are empirical measurements, not re-impositions of the fitted values.
full rationale
The paper's central claim is that cluster-based conformal prediction produces calibrated coverage and improved triage metrics relative to conventional conformal prediction. The calibration step (Section 2.3) computes a conformal correction q (or cluster-specific q_c) from the scores S_i = max(L_i - Vt_i, Vt_i - H_i) on a dedicated calibration set. The test-set coverage and CAr-CEr values (Section 3.2) are then measured on a separate held-out set of 81 patients. This is the standard conformal workflow: the quantile is fitted to calibration scores, and the test evaluation is an independent empirical result. No equation in the paper reduces the prediction to its inputs by construction, and no fitted parameter is renamed as a prediction. The only self-citation of note is reference [12], which describes the retrospective dataset; this is data provenance, not a load-bearing theoretical premise. The authors explicitly acknowledge in Section 4 that moving from patches to full images 'might mean that the exchangeability hypothesis is no longer respected.' That is an honest generalizability caveat about the statistical guarantee, not a circular step: the reported 0.83 test coverage is an observed quantity, and the paper does not claim the formal guarantee carries over. Hyperparameters such as target coverage (85%) and number of clusters (3) were selected on validation splits, then a final calibration was run on the full calibration set before testing; this selection is standard tuning and does not make the test results equivalent to the training objective. The clustering itself uses histograms of network output probabilities, but the cluster assignments only select which precomputed q_c to apply; they do not encode the true volumes or the test risk-category labels. Overall, the derivation chain is self-contained: the method is a conditional conformal prediction variant with an externally evaluated empirical outcome, and no circular dependence on the claimed result was found.
Assumptions & free parameters
free parameters (6)
- Target coverage (1-α) =
0.85
- Number of clusters (N_clusters) =
3
- Histogram clipping threshold =
>0.2
- Volume thresholds (λ) =
0.5, 0.7, 0.3
- Focal loss weight w =
4450
- Dropout rate =
0.1
assumptions (5)
- domain assumption Calibration and test data are exchangeable
- domain assumption Volume-to-Agaston scaling factor 3.13 gives valid risk categories
- domain assumption Clinical annotations are the ground truth
- ad hoc to paper Histogram of probability maps carries information about volume and uncertainty that is stable across patch and full-image scales
- standard math Constrained K-means with equal-size clusters yields valid per-cluster conformal calibration
Cite this review
Pith. "Pith review of Conformal coronary calcification volume estimation with conditional coverage via histogram clustering." pith.science (2026). https://pith.science/paper/GJASQZOW
@misc{pith2026250604030,
author = {Pith},
title = {Pith review of: Conformal coronary calcification volume estimation with conditional coverage via histogram clustering},
year = {2026},
howpublished = {\url{https://pith.science/paper/GJASQZOW}},
note = {Machine review of arXiv:2506.04030}
}
read the original abstract
Incidental detection and quantification of coronary calcium in CT scans could lead to the early introduction of lifesaving clinical interventions. However, over-reporting could negatively affect patient wellbeing and unnecessarily burden the medical system. Therefore, careful considerations should be taken when automatically reporting coronary calcium scores. A cluster-based conditional conformal prediction framework is proposed to provide score intervals with calibrated coverage from trained segmentation networks without retraining. The proposed method was tuned and used to calibrate predictive intervals for 3D UNet models (deterministic, MCDropout and deep ensemble) reaching similar coverage with better triage metrics compared to conventional conformal prediction. Meaningful predictive intervals of calcium scores could help triage patients according to the confidence of their risk category prediction.
Figures
Reference graph
Works this paper leans on
-
[1]
INTRODUCTION Coronary heart disease (CHD) is the prime cause of death worldwide. According to the World Health Organization, in 2019, 32% of all deaths were due to CHD[1]. Coronary artery calcification (CAC) is a highly prognostic biomarker for adverse cardiovascular events[2]. Depending on the risk due to calcium burden, treatments can range from simple ...
work page 2019
-
[2]
METHODS 2.1. Overview Fig1. Inference framework for cluster -based conformal coronary calcium prediction. A) Inference is performed (with an averaging step for ensemble and MC -Dropout methods). B) Pre- inference, clusters and corresponding q c corrections are determined from a calibration dataset. C) Conformal intervals of volume are calculated based on ...
-
[3]
Patch Based Calibration Table 1
RESULTS 3.1. Patch Based Calibration Table 1. Target coverage experiment results averaged over 𝑁𝑟𝑢𝑛𝑠=5 runs. Target coverage of 0.85 was selected (in bold) Target coverage experiment results are presented in Table 1. Mean Coverage Error was 1. 6% and 3. 3% for conventional and clustered approaches , respectively . However, the clustered predictions had hi...
-
[4]
It enabled targeted coverage of calcium volume with improved triage metrics
DISCUSSION The proposed cluster-based calibration was compared to the cCP approach for a range of models (Deterministic, MC - Dropout or Ensemble). It enabled targeted coverage of calcium volume with improved triage metrics. With the proposed calibrated ensemble method, reviewing 70% less cases led to only 2.4% errors in risk category predictions. An esti...
-
[5]
CONCLUSION In conclusion, the proposed method enabled creation of meaningful predictive intervals with high CAr-CEr rates and coverage outperforming the conventional approach . It required no model re-training using histograms of the output probabilities as a condition for conformal prediction . The method could help reduce the burden on radiologists give...
-
[6]
H, S.D) are employees of Canon Medical Research Europe
ACKNOWLEDGMENTS Six of the authors ( O.J, S.M, K.G, S.S.M, J. H, S.D) are employees of Canon Medical Research Europe
-
[7]
COMPLIANCE WITH ETHICAL STANDARDS The study was sponsored by NHS Golden Jubilee, UK and approved by Yorkshire and The Humber – Leeds East Research Ethics Committee (REC Ref 21/YH/0217
-
[8]
World Health Organization. World Health Organization. Cardiovascular diseases (CVDs) . WHO Website 2019. https://www.who.int/news-room/fact-sheets/detail/cardiovascular- diseases-(cvds) (accessed January 6, 2025)
work page 2019
Show all 31 references
-
[9]
The coronary calcium score is a more accurate predictor of significant coronary stenosis than conventional risk factors in symptomatic patients: Euro -CCAD study
Nicoll R, Wiklund U, Zhao Y, Diederichsen A, Mickley H, Ovrehus K, et al. The coronary calcium score is a more accurate predictor of significant coronary stenosis than conventional risk factors in symptomatic patients: Euro -CCAD study. Int J Cardiol 2016;207:13–9. https://doi...
2016 doi
-
[10]
Coronary Artery Calcification and its Progression: What Does it Really Mean? JACC Cardiovasc Imaging 2018;11:127 –42
Mori H, Torii S, Kutyna M, Sakamoto A, Finn A V., Virmani R. Coronary Artery Calcification and its Progression: What Does it Really Mean? JACC Cardiovasc Imaging 2018;11:127 –42. https://doi.org/10.1016/J.JCMG.2017.10.012
2018 doi
-
[11]
Viewing the Value of Coronary Artery Calcium Testing From Different Perspectives
Polonsky TS, Greenland P. Viewing the Value of Coronary Artery Calcium Testing From Different Perspectives. JAMA Cardiol 2018;3:908 –10. https://doi.org/10.1001/JAMACARDIO.2018.2199
2018
-
[12]
Quantification of coronary artery calcium using ultrafast computed tomography
Agatston AS, Janowitz WR, Hildner FJ, Zusmer NR, Viamonte M, Detrano R. Quantification of coronary artery calcium using ultrafast computed tomography. J Am Coll Cardiol 1990;15:827–32. https://doi.org/10.1016/0735-1097(90)90282-T
1990 doi
-
[13]
Hecht HS, Cronin P, Blaha MJ, Budoff MJ, Kazerooni EA, Narula J, et al. 2016 SCCT/STR guidelines for coronary artery calcium scoring of noncontrast noncardiac chest CT scans: A report of the Society of Cardiovascular Computed Tomography and Society of Thoracic Radiology. J Car...
2016 doi
-
[14]
A method for coronary artery calcium scoring using contrast -enhanced computed tomography
Otton JM, Lønborg JT, Boshell D, Feneley M, Hayen A, Sammel N, et al. A method for coronary artery calcium scoring using contrast -enhanced computed tomography. J Cardiovasc Comput Tomogr 2012;6:37 –44. https://doi.org/10.1016/j.jcct.2011.11.004
2012 doi
-
[15]
Deep Learning -Based Automated Quantification of Coronary Artery Calcification for Contrast-Enhanced Coronary Computed Tomographic Angiography
Lee JO, Park EA, Park D, Lee W. Deep Learning -Based Automated Quantification of Coronary Artery Calcification for Contrast-Enhanced Coronary Computed Tomographic Angiography. Journal of Cardiovascular Development and Disease 2023, Vol 10, Page 143 2023 ;10:143. https://doi.or...
2023 doi
- [16]
-
[17]
Transduction with Confidence and Credibility
Saunders C, Gammerman A, Vovk V. Transduction with Confidence and Credibility. Sixteenth International Joint Conference on Artificial Intelligence (IJCAI ’99) 1999:722–6
1999
- [18]
-
[19]
Deep learning-based detection of coronary artery calcification in non -contrast and contrast -enhanced CT scans
Mohammadi S, Goatman K, Dahdouh S, Jaubert O, Voisey J, Good R. Deep learning-based detection of coronary artery calcification in non -contrast and contrast -enhanced CT scans. ResearchSquare 2024. https://doi.org/10.21203/RS.3.RS - 4281908/V1
2024 doi
-
[20]
TotalSegmentator: Robust Segmentation of 104 Anatomic Structures in CT Images
Wasserthal J, Breit HC, Meyer MT, Pradella M, Hinck D, Sauter AW, et al. TotalSegmentator: Robust Segmentation of 104 Anatomic Structures in CT Images. Radiol Artif Intell 2023;5. https://doi.org/10.1148/ryai.230024
2023 doi
- [21]
-
[22]
Focal Loss for Dense Object Detection
Lin T-Y, Goyal P, Girshick R, He K, Dollár P. Focal Loss for Dense Object Detection. Proceedings of the IEEE International Conference on Computer Vision 2017:2980–8
2017
-
[23]
Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
Gal Y, Ghahramani Z. Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. 33rd International Conference on Machine Learning, ICML 2016 2015;3:1651–60
2016
-
[24]
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
Lakshminarayanan B, Pritzel A, Blundell C. Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles. Adv Neural Inf Process Syst 2016;2017-December:6403–14
2016
-
[25]
Automatic brain segmentation in preterm infants with post -hemorrhagic hydrocephalus using 3D Bayesian U -Net
Largent A, De Asis-Cruz J, Kapse K, Barnett SD, Murnick J, Basu S, et al. Automatic brain segmentation in preterm infants with post -hemorrhagic hydrocephalus using 3D Bayesian U -Net. Hum Brain Mapp 2022;43:1895 –916. https://doi.org/10.1002/HBM.25762
2022 doi
-
[26]
Conformalized Quantile Regression
Romano Y, Patterson E, Candès EJ. Conformalized Quantile Regression. Adv Neural Inf Process Syst 2019;32
2019
-
[27]
Constrained K - Means Clustering
Bradley PS, Bennett KP, Demiriz A. Constrained K - Means Clustering. Microsoft Research, Redmond 2000:1 –8. https://www.microsoft.com/en-us/research/wp- content/uploads/2016/02/tr-2000-65.pdf
2000
-
[28]
Improving conformalized quantile regression through cluster -based feature relevance
Sousa M, Tomé AM, Moreira J. Improving conformalized quantile regression through cluster -based feature relevance. Expert Syst Appl 2024;238:122322. https://doi.org/10.1016/J.ESWA.2023.122322
2024
-
[29]
Greenland P, Bonow RO, Brundage BH, Budoff MJ, Eisenberg MJ, Grundy SM, et al. ACCF/AHA 2007 Clinical Expert Consensus Document on Coronary Artery Calcium Scoring By Computed Tomography in Global Cardiovascular Risk Assessment and in Evaluation of Pat ients With Chest. J Am Co...
2007 doi
-
[30]
Coronary Calcium Characteristics as Predictors of Major Adverse Cardiac Events in Symptomatic Patients: Insights From the CORE320 Multinational Study
Lo-Kioeng-Shioe MS, Vavere AL, Arbab -Zadeh A, Schuijf JD, Rochitte CE, Chen MY, et al. Coronary Calcium Characteristics as Predictors of Major Adverse Cardiac Events in Symptomatic Patients: Insights From the CORE320 Multinational Study. J Am Heart As soc 2019;8. https://doi....
2019 doi
-
[31]
A rosetta stone for coronary calcium risk stratification: Agatston, volume, and mass scores in 11,490 individuals
Rumberger JA, Kaufman L. A rosetta stone for coronary calcium risk stratification: Agatston, volume, and mass scores in 11,490 individuals. American Journal of Roentgenology 2003;181:743–8. https://doi.org/10.2214/AJR.181.3.1810743
2003 doi
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.