REVIEW 3 major objections 5 minor 19 references
Less Annotation, More Interpretation: Prior-Guided Concept Bottleneck Models for Interpretable Cancer Imaging Diagnosis
T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read A prior-guided hybrid concept bottleneck model improves mean concept AUC in the 0–20% annotation regime across three cancer imaging tasks while keeping diagnostic AUC close to black-box baselines.
desk verdict Solid empirical study of a prior-guided hybrid CBM, but the 'less annotation' headline is conditional on aggregate priors computed from full training labels, which the standard baseline does not receive. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the prior-guided hybrid CBM, a concept bottleneck model whose concept head $g_\theta(x)$ maps images to concept probabilities $\hat{c}=\sigma(z)$, followed by a linear diagnosis head. Three mechanisms carry the argument: (1) a distribution-matching loss $L_{\text{dist}}$ that compares the mean predicted concept vector of unannotated patients in a minibatch to a class-conditional prevalence prior $\pi_y$, giving weak supervision from aggregate label proportions; (2) initialization of the diagnosis head weight for concept $k$ as $\logit\{P(y=1\mid c_k=1)\}-\logit\{P(y=1\mid c_k=0)\}$, with bias from the malignancy base rate; and (3) an optional L2 anchor $L_{\text{reg}}$ keeping the head near that initialization. The load-bearing mechanism is the stabilized concept-to-diagnosis mapping: with a plausible start, gradient updates teach image-to-concept mappings while the diagnosis head stays close to the prior, which the drift analysis shows does not happen with random initialization.
What would settle it
Compute the data-derived priors from only the annotated 10% of training patients (instead of the full training cohort) and rerun the low-annotation experiments; if the mean concept AUC advantage over the standard CBM shrinks to near zero or reverses, the paper's central benefit is attributable to full-cohort aggregate labels rather than to the prior-guided mechanism itself.
Extended reading notes
Core claim
The central discovery is that a hybrid supervision scheme—mixing instance-level concept labels on a few patients with class-conditional concept distribution matching on the rest, plus a prior-initialized diagnosis head—lets concept bottleneck models learn fine-grained radiological concepts annotation-efficiently. The paper reports consistent mean concept ROC-AUC gains in the 0–20% annotation regime relative to a matched standard CBM, with the largest effect coming from prior initialization of the concept-to-diagnosis head rather than from the distribution loss alone. Ablations show that the prior-initialized head drifts almost not at all (relative L2 drift under 0.5%) when no concept labels are available, while randomly initialized heads drift by more than 85%. The paper also shows that zero-shot vision-language models are insufficient for fine-grained tumor-level concept recognition, scoring around 0.50 AUC.
Load-bearing premise
The method's benefit depends on having reliable class-conditional concept prevalence estimates (the expected fraction of each radiological concept in benign versus malignant cases), and in the experiments these are computed from the full training set's concept labels even when instance-level supervision is only 0–20%; if those aggregate statistics are not available without near-full annotation, the claimed reduction in annotation burden weakens.
Editorial extensions
If this is right
- A clinically meaningful concept bottleneck for cancer imaging can be trained with as little as 0–20% patient-level concept annotations, with concept AUC close to what fully annotated models give in the tested tasks.
- The diagnosis head initialized from clinical priors is robust: at 0% annotation its weights drift by less than 0.5% relative, while randomly initialized heads drift by more than 85%, explaining why prior initialization matters most when labels are absent.
- Zero-shot vision-language models do not yet provide reliable fine-grained tumor-level concept predictions, so annotation-free concept bottleneck models are not a viable alternative in this setting.
- Correcting concept predictions at test time improves diagnosis AUC by up to 0.139 on lung nodules, indicating that selective clinician review of concept outputs could become a practical workflow.
- Data-derived class-conditional priors outperform manual BI-RADS-style priors for concept supervision, showing that cohort-calibrated aggregate statistics are the stronger prior source.
Reading between the lines
- If the prior-initialization effect generalizes, prevalence priors drawn from historical radiology reports or clinical databases could cold-start interpretable models for new cancer types where only outcome labels exist, without any new concept annotation.
- The distribution-matching loss is learning from label proportions; a natural extension is estimating $\pi_y$ iteratively from the unannotated pool (e.g., via expectation-maximization), which would remove the need for full-cohort concept statistics and directly test how much aggregate information the method truly requires.
- The oracle concept-correction gains suggest a prospective clinician-in-the-loop study: have radiologists correct only high-impact concept predictions, then measure real diagnostic AUC improvement over a no-correction baseline; the paper's oracle numbers bound the possible benefit.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a prior-guided hybrid concept bottleneck model (PriorHybrid-CBM) that combines limited instance-level concept annotations with class-conditional distribution matching on unannotated patients and prior initialization of the concept-to-diagnosis head. The method is evaluated on CBIS-DDSM mammographic masses and calcifications and LIDC-IDRI pulmonary nodules across concept annotation fractions from 0% to 100%. The central empirical claim is that in the clinically relevant 0-20% annotation regime, the hybrid CBM consistently improves mean concept ROC-AUC over a matched standard CBM while maintaining diagnostic AUC close to black-box models, with reported 10%-annotation improvements from 0.619 to 0.741 (masses), 0.650 to 0.787 (calcifications), and 0.597 to 0.642 (nodules). The paper also reports that zero-shot vision-language models are insufficient for fine-grained tumor-level concept prediction.
Significance. If the reported gains hold under a properly matched information comparison, the method would be a meaningful step toward reducing concept-annotation costs for interpretable medical imaging. The empirical work has notable strengths: percentile bootstrap confidence intervals, paired significance tests, patient-level splitting, a component ablation, and publicly released code. The paper also provides a clear negative result for zero-shot VLM transfer. However, the central 'less annotation' claim currently rests on a comparison in which the hybrid model receives aggregate concept-label statistics from the full training cohort, while the standard CBM baseline does not. This limits the validity of the headline conclusion as stated.
major comments (3)
- [Section 4.6, Table 1] The principal concern raised in the review process is confirmed by the manuscript text. Section 3.2 states that 'data-derived aggregate priors were computed separately from the training patients of each fold' and that annotation fractions 'applied only to labels used in the instance-level concept loss.' As a result, at every annotation fraction, including 0%, the hybrid model receives class-conditional concept prevalence vectors pi_y (Eq. 3) and P(y|c_k) (Eq. 4) estimated from the concept labels of the entire training cohort, whereas the matched standard CBM receives none of this aggregate label information. The reported concept-AUC gains in the 0-20% regime, including the 10% figures in the abstract, therefore conflate the proposed mechanism with an information advantage. The authors should either re-run the experiments with priors estimated only from the annotated subset (or from a separate external source) to provide a truly matched comparison, or re-frame the central claim as conditional on the availability of full-training-set aggregate priors. As it stands, the 'less annotation' claim is not supported by the current experimental design.
- [Section 4.4, Section 4.6] The ablation titled 'Random init.' removes prior initialization but retains the distribution-matching loss (lambda_d) and the anchoring loss (lambda_r). Since the distribution-matching targets are data-derived priors computed from the full training cohort, this ablation does not isolate the effect of prior initialization from the use of full-cohort aggregate label information. The observation in Section 4.4 that random-init diagnosis heads drift far from the data-prior direction is made at 0% concept annotation, where the standard CBM has no concept supervision at all; the comparison is therefore not information-matched. The conclusion that 'prior initialization is the main component' needs to be re-stated with the caveat that this component is evaluated in the presence of full-training-set prior statistics.
- [Supplementary Fig. S1, Section 4.6] The paper's own results show that data-derived priors generally outperform manual priors, which are label-free. For example, in the 0-20% regime, mean concept AUC improves from 0.681 to 0.716 for masses and from 0.726 to 0.757 for calcifications when moving from manual to data-derived priors. This implies that the practical annotation-reduction benefit depends on having access to cohort-level label statistics, which may themselves require substantial annotation effort. The limitation section (4.6) acknowledges this, but the abstract and conclusion state the annotation reduction as established rather than conditional on such statistics. The framing should be corrected to match the evidence.
minor comments (5)
- [Introduction] Typo: 'Several approached have been proposed' should be 'Several approaches have been proposed.'
- [Section 2.2] Typo: 'specific dataset' is written as 'specifc dataset.'
- [Section 3.1] The nodule crop size is written as '483' without a superscript; it should be 4^3 (64) or the intended notation should be clarified.
- [Table captions] The table captions contain 'T able' rather than 'Table'; please fix the formatting.
- [Section 3.2] The selection of lambda_d is performed using validation concept AUC over the 0-20% regime. This is a second, smaller use of concept labels beyond the training objective and should be mentioned when summarizing the label budget of the method.
Circularity Check
No significant circularity; the empirical comparison is self-contained and evaluated on held-out test data.
full rationale
No circularity found. The paper's central claim is an empirical comparison of concept AUC on held-out test sets, and no derivation is claimed that would reduce to its inputs. The prior vectors pi_y in Eq. (3) and P(y|c_k) in Eq. (4) are inputs estimated from the training folds; they are not fitted to test labels, and the held-out test evaluation is external. The only caveat is that data-derived priors are computed from all training patients' concept labels, which means the hybrid model receives aggregate label information that the standard CBM does not; this weakens the 'reduced annotation burden' framing but is not circularity. The paper explicitly discloses this in Section 4.6, stating that 'any reduction in clinical workload is conditional on a suitable prior being available.' Self-citations [10,11] are supporting context, not load-bearing. Therefore the derivation chain is self-contained with no circular step.
Assumptions & free parameters
free parameters (3)
- λ_d (distribution matching weight) =
0.7 (mass), 0.3 (calcification), 1.0 (nodule)
- λ_r (anchor regularization weight) =
0.05
- λ_c (concept loss weight) =
not reported (assumed 1)
assumptions (4)
- domain assumption Concept labels are reliable and the predefined concepts capture clinically relevant attributes.
- domain assumption Class-conditional priors π_y can be estimated accurately from training data and generalize to the test distribution.
- domain assumption A linear diagnosis head on concept probabilities is sufficient to represent the concept-to-diagnosis mapping.
- standard math Standard optimization and loss functions (BCE with logits, L2 regularization) behave as expected.
Cite this review
Pith. "Pith review of Less Annotation, More Interpretation: Prior-Guided Concept Bottleneck Models for Interpretable Cancer Imaging Diagnosis." pith.science (2026). https://pith.science/paper/TBC6S7EX
@misc{pith2026260813148,
author = {Pith},
title = {Pith review of: Less Annotation, More Interpretation: Prior-Guided Concept Bottleneck Models for Interpretable Cancer Imaging Diagnosis},
year = {2026},
howpublished = {\url{https://pith.science/paper/TBC6S7EX}},
note = {Machine review of arXiv:2608.13148}
}
read the original abstract
Concept bottleneck models (CBMs) can improve the transparency of cancer image diagnostic prediction by expressing predictions through radiological concepts. However, their dependence on instance-level concept annotations limits practical applicability. We propose a prior-guided hybrid CBM that integrates limited concept annotations, class-conditional concept distribution matching on unannotated patients, and prior initialization of the concept-to-diagnosis head. We evaluate the method on CBIS-DDSM mammographic masses and calcifications and LIDC-IDRI pulmonary nodules across 0-100% concept annotation. In the clinically relevant 0-20% annotation regime, the hybrid CBM consistently improves mean concept AUC over a matched standard CBM, while maintaining diagnostic performance close to black-box models. At 10% annotation specifically, concept AUC increases from 0.619 to 0.741 for masses, from 0.650 to 0.787 for calcifications, and from 0.597 to 0.642 for pulmonary nodules. Ablation experiments identify prior initialization as the main component contributing to improved concept detection, likely by stabilizing the concept-to-diagnosis head. Zero-shot VLMs remain insufficient for reliable fine-grained tumor-level concept prediction. These findings suggest that structured priors can substantially reduce the annotation burden of interpretable cancer imaging models.
Figures
Reference graph
Works this paper leans on
-
[1]
Nature Medicine25, 954–961 (2019)
Ardila, D., Kiraly, A.P., Bharadwaj, S., Choi, B., Reicher, J.J., Peng, L., Tse, D., Etemadi, M., Ye, W., Corrado, G., Naidich, D.P., Shetty, S.: End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography. Nature Medicine25, 954–961 (2019). https://doi.org/10.1038/ s41591-019-0447-x
work page 2019
-
[2]
Medical Physics38(2), 915–931 (2011)
Armato, S.G., et al.: The lung image database consortium (LIDC) and image database resource initiative (IDRI): A completed reference database of lung nodules on CT scans. Medical Physics38(2), 915–931 (2011)
work page 2011
-
[3]
arXiv preprint arXiv:1904.00625 (2019)
Chen, S., Ma, K., Zheng, Y.: Med3D: Transfer learning for 3d medical image analysis. arXiv preprint arXiv:1904.00625 (2019)
arXiv 2019
-
[4]
In: Medical Image Computing and Computer Assisted Intervention
Ghosh, S., Poynton, C.B., Visweswaran, S., Batmanghelich, K.: Mammo-CLIP: A vision language foundation model to enhance data efficiency and robustness in mammography. In: Medical Image Computing and Computer Assisted Intervention. pp. 632–642 (2024)
work page 2024
-
[5]
Nature Biomedical Engineering (2025).https://doi.org/10.1038/s41551-025-01599-y
Hamamci, I.E., Er, S., Wang, C., et al.: Developing generalist foundation mod- els from a multimodal dataset for 3d computed tomography. Nature Biomedical Engineering (2025).https://doi.org/10.1038/s41551-025-01599-y
-
[6]
arXiv preprint arXiv:2406.18992 (2024)
Hu, L., Huang, T., Xie, H., Gong, X., Ren, C., Hu, Z., Yu, L., Ma, P., Wang, D.: Semi-supervised concept bottleneck models. arXiv preprint arXiv:2406.18992 (2024)
arXiv 2024
-
[7]
In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 4700–4708 (2017).https://doi.org/10.1109/ CVPR.2017.243
work page 2017
-
[8]
In: Proceedings of the 37th International Conference on Machine Learning
Koh, P.W., Nguyen, T., Tang, Y.S., Mussmann, S., Pierson, E., Kim, B., Liang, P.: Concept bottleneck models. In: Proceedings of the 37th International Conference on Machine Learning. pp. 5338–5348 (2020)
work page 2020
Show all 19 references
-
[9]
Scientific Data4, 170177 (2017) Prior-Guided CBMs for Cancer Imaging 11
Lee, R.S., Gimenez, F., Hoogi, A., Miyake, K.K., Gorovoy, M., Rubin, D.L.: A curated mammography data set for use in computer-aided detection and diagnosis research. Scientific Data4, 170177 (2017) Prior-Guided CBMs for Cancer Imaging 11
2017
-
[10]
In: 23rd IEEE International Symposium on Biomedical Imaging, ISBI 2026, London, United Kingdom, April 8–11, 2026
Ma, B., Madzia-Madzou, D.K., Kraaijveld, R.C.J., Ouyang, J.: Ranking XAI methods for head and neck cancer outcome prediction. In: 23rd IEEE International Symposium on Biomedical Imaging, ISBI 2026, London, United Kingdom, April 8–11, 2026. pp. 1–4. IEEE (2026). https://doi.org...
2026
-
[11]
Ma, B., Madzia-Madzou, D.K., Ouyang, J., Gilhuijs, K.G.A.: Can vision-language models enable more efficient concept-based learning with less supervision for in- terpretable lung nodule diagnosis? In: MIDL 2026 Short Papers (2026),https: //openreview.net/forum?id=OyOHg986R5, poster
2026
-
[12]
Nature577, 89–94 (2020).https://doi.org/10.1038/s41586-019-1799-6
McKinney, S.M., Sieniek, M., Godbole, V., Godwin, J., Antropova, N., Ashrafian, H., Back, T., Chesus, M., Corrado, G.S., Darzi, A., Etemadi, M., Garcia-Vicente, F., et al.: International evaluation of an AI system for breast cancer screening. Nature577, 89–94 (2020).https://do...
2020 doi
-
[13]
arXiv preprint arXiv:2511.01131 (2025)
Nahiduzzaman, M., Korevaar, S., Bab-Hadiashar, A., Tennakoon, R.: Weakly supervised concept learning with class-level priors for interpretable medical diagnosis. arXiv preprint arXiv:2511.01131 (2025)
2025
-
[14]
In: International Conference on Learning Representations (2023)
Oikarinen, T., Das, S., Nguyen, L.M., Weng, T.W.: Label-free concept bottleneck models. In: International Conference on Learning Representations (2023)
2023
-
[15]
Computers in Biology and Medicine198, 111145 (2025)
Patrício, C., Rio-Torto, I., Cardoso, J.S., Teixeira, L.F., Neves, J.C.: CBVLM: Training-free explainable concept-based large vision language models for medical image classification. Computers in Biology and Medicine198, 111145 (2025). https://doi.org/10.1016/j.compbiomed.2025.111145
2025
-
[16]
In: Proceedings of the 26th International Conference on Machine Learning
Quadrianto, N., Smola, A.J., Caetano, T.S., Le, Q.V.: Estimating labels from label proportions. In: Proceedings of the 26th International Conference on Machine Learning. pp. 776–783 (2009)
2009
-
[17]
In: Proceedings of the 38th International Conference on Machine Learning
Radford, A., et al.: Learning transferable visual models from natural language supervision. In: Proceedings of the 38th International Conference on Machine Learning. pp. 8748–8763 (2021)
2021
-
[18]
In: Proceedings of the IEEE International Conference on Computer Vision
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad- CAM: Visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 618–626 (2017)
2017
-
[19]
In: Proceedings of the 34th International Conference on Machine Learning
Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks. In: Proceedings of the 34th International Conference on Machine Learning. pp. 3319–3328 (2017) 12 B. Ma and K. Gilhuijs A Complete Hybrid-CBM Prior and Distribution Sweep 0 5 10 20 50 75 100 0.5 0.6 ...
2017
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.