REVIEW 3 major objections 6 minor 28 references
Fairness and Robustness of CLIP-Based Models for Chest X-rays
T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read CLIP-based chest X-ray models appear to lean on chest drains as a shortcut for pneumothorax, while their embeddings leak sensitive attributes that PCA misses.
desk verdict Useful comparative evaluation of six CLIP-based CXR models; the shortcut claim is plausible but needs pseudo-label validation and scoping. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The argument runs through two instruments. The first is a two-group pneumothorax comparison: with and without chest drains, where drain labels come from NEATX annotations and a DenseNet detector trained with that paper's hyperparameters to label non-pneumothorax images; the difference in adjusted AUPRC between the groups is the evidence of shortcut learning. The second is an embedding audit: PCA visualisation of image and text embeddings alongside linear probes, k-NN, and MLP classifiers trained to predict sensitive attributes, plus per-dimension centroid differences borrowed from modality-gap analysis, which together show that absent visual separation does not mean absent encoding. The adjusted AUPRC metric is what makes the imbalanced pneumothorax comparison meaningful.
What would settle it
Recompute the pneumothorax with-drain versus without-drain AUC and adjusted AUPRC on a test set where every image's chest-drain status is confirmed by expert annotation; if the performance gap disappears, reverses, or falls within noise for most models, the claimed reliance on chest drains as a spurious correlation is not supported.
Extended reading notes
Core claim
On its own terms, the paper establishes that CLIP-based chest X-ray models inherit the same hidden-stratification failure seen in convolutional networks: for pneumothorax classification, five of the six models obtain clearly higher adjusted AUPRC on images with chest drains than without (improvements ranging from +0.09 to +0.30), with CXR-CLIP the partial exception, and the paper interprets the overall pattern as reliance on spurious chest-drain correlations. It further reports that simple supervised probes (linear, k-nearest neighbours, and a one-hidden-layer MLP) classify patient sex, race, and age from image embeddings at levels above random, even though PCA projections show no visible subgroup structure; race is the hardest attribute but is still decoded by the MLP. The same analysis shows that the image-text modality gap is concentrated in a few dimensions, while differences between subgroup centroids are smaller and spread across many dimensions, and that all models are miscalibrated and overconfident.
Load-bearing premise
The shortcut conclusion assumes the automatically generated chest drain labels for non-pneumothorax images are accurate; the paper trains its DenseNet detector using NEATX hyperparameters but reports no accuracy for it, so if those labels are wrong the with-drain versus without-drain comparison collapses.
Editorial extensions
If this is right
- Pneumothorax screening with these models will underperform exactly in the emergency cases that matter most: pneumothorax without a visible chest drain, so deployment should not proceed on benchmark AUC alone.
- Accuracy and AUC rankings of CLIP-based chest X-ray models overstate clinical utility; calibration and subgroup-stratified metrics must be reported alongside them.
- A PCA plot showing no demographic separation is not evidence that a model is fair; embedding classification probes should become part of model auditing.
- The age-related performance gaps, especially unreliable estimates for small subgroups like Asian patients and 18-25 year olds, call for more diverse test cohorts before conclusions about sex and race fairness are trusted.
- Because all models are miscalibrated and overconfident, their probability outputs should not be used as confidence measures in clinical decision support.
Reading between the lines
- The same drain/no-drain protocol could be applied to other line- and tube-associated findings, such as central venous catheters or pacemakers, to test whether shortcut reliance generalises beyond pneumothorax.
- Because the non-pneumothorax drain labels are machine-generated, part of the reported gap could reflect detector errors rather than model shortcut behaviour; an expert-labelled subset would separate these explanations.
- If sensitive attributes are decodable by simple probes, then embedding-level debiasing, for example adversarial removal of attribute information, becomes a testable intervention for these models, though the paper does not propose one.
- Extending this evaluation to generative multimodal models, rather than contrastive CLIP-style encoders, would show whether the same fairness and shortcut patterns hold for models that produce reports and not just embeddings.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper empirically evaluates six CLIP-based vision-language models on chest X-ray classification using MIMIC-CXR, NIH-CXR14, and NEATX. It assesses zero-shot classification performance, fairness across age, sex, and race subgroups, robustness to shortcut learning via chest drains in pneumothorax classification, calibration, and whether sensitive attributes are recoverable from the models' embeddings. The authors report age-related performance gaps, broadly equitable sex and race results, lower adjusted AUPRC on images without chest drains for most models, miscalibration, and successful classification of sensitive attributes from embeddings despite no visible PCA separation.
Significance. If the central shortcut finding is upheld after validation of the chest-drain labels, the paper provides a timely extension of previously documented CNN shortcut behavior to several CLIP-based medical models, and the embedding analysis adds a concrete caveat about PCA-based fairness checks. The release of code and the use of multiple public datasets are strengths, and the paper appropriately notes that embedding classification does not by itself prove shortcut use. The study is empirical and reproducible in design, though its headline conclusions currently rest on claims that need tightening.
major comments (3)
- [Abstract; §3.3; §4] The central claim that 'all models exhibit lower performance on images without chest drains' is contradicted by the paper's own results: §3.3 states 'all models except CXR-CLIP obtain better adjusted AUPRC on images with chest drains,' and Fig. 4b plots this. The Discussion repeats the 'all models' wording. Since the universal shortcut is the headline finding, the claim must be scoped to the five models that show the effect, and the abstract and Discussion revised accordingly.
- [§2.3] The shortcut analysis in §3.3 depends on automatically generated chest-drain labels for non-pneumothorax images from a DenseNet trained on NEATX, but the paper reports no accuracy, precision, recall, or any validation for this detector on non-pneumothorax images. NEATX annotations cover only pneumothorax-positive samples, so the detector is applied under a domain shift; if its errors correlate with image content such as other tubes or lines, the with-drain versus without-drain AUPRC gap could be inflated or spurious. Please report the detector's performance on a held-out set or via manual review of a random sample, and ideally show that the main comparison is robust to label noise, for example by recomputing the adjusted AUPRC gap under a range of assumed label-error rates.
- [§3.1] The claim of 'gaps across patient ages' and 'more equitable results' for sex and race is based on visual inspection of Fig. 1-style barplots with bootstrap confidence intervals. No statistical test, effect size, or quantitative disparity metric is reported, and overlapping confidence intervals make visual comparison unreliable. Since age fairness is one of the paper's two central empirical findings, add a quantitative subgroup-disparity measure, such as gap in AUC or AUPRCadj with bootstrap CIs, or a paired significance test, across the six conditions.
minor comments (6)
- [§2.3] The report section is referred to as 'FINDING' in some places and 'FINDINGS' in others; please harmonize the terminology.
- [§2.1] The adjusted AUPRC formula is not numbered, and the notation 'AUPRCad j' appears to have a spacing artifact; please number the equation and clean up the notation.
- [§3.1] The statement that the 18-25 year old subgroup has limited positive samples 'leading to more extreme values and confidence intervals' should be supported by reporting the subgroup sample sizes or by moving this point to the limitations section.
- [Fig. 4] The with-drain versus without-drain comparisons and the calibration curves do not show confidence intervals or error bars; adding them would strengthen the quantitative comparison.
- [§2.3] The paper should clarify whether all MIMIC-CXR analyses use the 8,950-image subset with FINDINGS sections, and discuss whether this subset is representative of the full test split.
- [§2.3] There is a typo in the dataset download description: 'Version3downloadedfrom' lacks spaces.
Circularity Check
Empirical evaluation of pretrained CLIP models; no derivation reduces to fitted inputs or load-bearing self-citations.
full rationale
This paper is an experimental audit of six pretrained CLIP-based chest X-ray models; it contains no mathematical derivation whose outputs are equivalent to its inputs. The shortcut analysis (Section 3.3) partitions pneumothorax images by chest-drain presence using a DenseNet trained on NEATX with hyperparameters from Damgaard et al. [6] to 'automatically generate the labels for non-pneumothorax samples' (Section 2.3). This detector is an external auxiliary tool, not a parameter fitted to the models' performance gaps, so the with-drain versus without-drain comparison is not forced by construction. The central empirical finding of better adjusted AUPRC on images with chest drains is measured against external benchmarks, and the paper explicitly scopes its conclusions to the evaluated datasets and models in the Discussion. The one relevant self-citation, [23] on lung masking, is motivational context rather than load-bearing evidence for the drain-shortcut result, which is instead aligned with independent prior work [11,18]. The embedding analysis is equally self-contained: supervised probes are trained on patient-separated splits and compared to PCA visualizations, with no fitted parameter being renamed as a prediction. Two non-circular weaknesses are nonetheless worth noting: the abstract's 'all models exhibit lower performance on images without chest drains' is internally contradicted by Section 3.3's 'all models except CXR-CLIP obtain better adjusted AUPRC on images with chest drains,' and Section 2.3 does not report the accuracy of the DenseNet pseudo-labeler, whose errors could bias the drain/no-drain partition; these are correctness and validation concerns, not circularity. Accordingly, no circular step is present.
Assumptions & free parameters
free parameters (1)
- Auxiliary chest drain detector (DenseNet) =
trained model on NEATX, weights not shipped
assumptions (4)
- domain assumption The zero-shot softmax protocol using 'Chest {CLASS}' and 'Chest No Findings' templates is a valid measure of discriminative performance for all six models.
- domain assumption The MIMIC-CXR test subset with FINDINGS sections is representative of the full test set for fairness evaluation.
- domain assumption Automatically generated chest drain labels from a DenseNet trained on NEATX are accurate for non-pneumothorax samples.
- domain assumption Auxiliary classifiers (linear probe, k-NN, MLP) trained on embeddings provide evidence of algorithmic encoding of sensitive attributes.
Cite this review
Pith. "Pith review of Fairness and Robustness of CLIP-Based Models for Chest X-rays." pith.science (2026). https://pith.science/paper/ZTGMMR32
@misc{pith2026250721291,
author = {Pith},
title = {Pith review of: Fairness and Robustness of CLIP-Based Models for Chest X-rays},
year = {2026},
howpublished = {\url{https://pith.science/paper/ZTGMMR32}},
note = {Machine review of arXiv:2507.21291}
}
read the original abstract
Motivated by the strong performance of CLIP-based models in natural image-text domains, recent efforts have adapted these architectures to medical tasks, particularly in radiology, where large paired datasets of images and reports, such as chest X-rays, are available. While these models have shown encouraging results in terms of accuracy and discriminative performance, their fairness and robustness in the different clinical tasks remain largely underexplored. In this study, we extensively evaluate six widely used CLIP-based models on chest X-ray classification using three publicly available datasets: MIMIC-CXR, NIH-CXR14, and NEATX. We assess the models fairness across six conditions and patient subgroups based on age, sex, and race. Additionally, we assess the robustness to shortcut learning by evaluating performance on pneumothorax cases with and without chest drains. Our results indicate performance gaps between patients of different ages, but more equitable results for the other attributes. Moreover, all models exhibit lower performance on images without chest drains, suggesting reliance on spurious correlations. We further complement the performance analysis with a study of the embeddings generated by the models. While the sensitive attributes could be classified from the embeddings, we do not see such patterns using PCA, showing the limitations of these visualisation techniques when assessing models. Our code is available at https://github.com/TheoSourget/clip_cxr_fairness
Figures
Reference graph
Works this paper leans on
-
[23]
Journal of Imaging Informatics in Medicine pp
Sourget, T., Hestbek-Møller, M., Jiménez-Sánchez, A., Junchi Xu, J., Cheplygina, V.: Mask of truth: model sensitivity to unexpected regions of medical images. Journal of Imaging Informatics in Medicine pp. 1–18 (2025) Fairness and Robustness of CLIP-Based Models for Chest X-rays 11
work page 2025
-
[1]
Bahre, G.H., Hamidi, H., Calimeri, F., Sellergren, A., Celi, L.A., Seyyed-Kalantari, L.: Fairness of ai models in vector embedded chest x-ray representations. In: Ad- vancements In Medical Foundation Models: Explainability, Robustness, Security, and Beyond (2024)
work page 2024
-
[2]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Bannur, S., Hyland, S., Liu, Q., Perez-Garcia, F., Ilse, M., Castro, D.C., Boecking, B., Sharma, H., Bouzid, K., Thieme, A., et al.: Learning to exploit temporal struc- ture for biomedical vision-language processing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 15016–15027 (2023)
2023
-
[3]
In: European conference on computer vision
Boecking, B., Usuyama, N., Bannur, S., Castro, D.C., Schwaighofer, A., Hyland, S., Wetscherek, M., Naumann, T., Nori, A., Alvarez-Valle, J., et al.: Making the most of text semantics to improve biomedical vision–language processing. In: European conference on computer vision. pp. 1–21. Springer (2022)
2022
-
[4]
Cheplygina, V., Cathrine, D., Eriksen, T.N., Jiménez-Sánchez, A.: Neatx: Non- expert annotations of tubes in x-rays (2025),https://doi.org/10.5281/zenodo. 14944064
doi:10.5281/zenodo 2025
-
[5]
arXiv preprint arXiv:2410.06542 (2024)
Codella, N.C., Jin, Y., Jain, S., Gu, Y., Lee, H.H., Abacha, A.B., Santamaria- Pang, A., Guyman, W., Sangani, N., Zhang, S., et al.: Medimageinsight: An open-source embedding model for general domain medical imaging. arXiv preprint arXiv:2410.06542 (2024)
arXiv 2024
-
[6]
Augmenting Chest X-ray Datasets with Non-Expert Annotations
Damgaard, C., Eriksen, T.N., Juodelyte, D., Cheplygina, V., Jiménez-Sánchez, A.: Augmenting chest x-ray datasets with non-expert annotations. arXiv preprint arXiv:2309.02244 (2023)
work page Pith review arXiv 2023
-
[7]
In: Medical Imaging with Deep Learning (2025), https://openreview.net/forum?id=RuqEg2XAWq
Fay, L., Delbrouck, J.B., Küstner, T., Yang, B., Codella, N.C., Lungren, M.P., Langlotz, C., Gatidis, S.: Beyond the prompt: Deploying medical foundation mod- els on diverse chest x-ray populations. In: Medical Imaging with Deep Learning (2025), https://openreview.net/forum?id=RuqEg2XAWq
work page 2025
Show all 28 references
-
[8]
The Lancet Digital Health 4(6), e406–e414 (2022) 10 T
Gichoya, J.W., Banerjee, I., Bhimireddy, A.R., Burns, J.L., Celi, L.A., Chen, L.C., Correa, R., Dullerud, N., Ghassemi, M., Huang, S.C., et al.: Ai recognition of patient race in medical imaging: a modelling study. The Lancet Digital Health 4(6), e406–e414 (2022) 10 T. Sourget et al
2022
-
[9]
EBioMedicine89 (2023)
Glocker, B., Jones, C., Bernhardt, M., Winzeck, S.: Algorithmic encoding of pro- tected characteristics in chest x-ray disease detection models. EBioMedicine89 (2023)
2023
-
[10]
Nature Reviews Cancer18(8), 500–510 (2018)
Hosny, A., Parmar, C., Quackenbush, J., Schwartz, L.H., Aerts, H.J.: Artificial intelligence in radiology. Nature Reviews Cancer18(8), 500–510 (2018)
2018
-
[11]
In: 2023 IEEE 20th In- ternational Symposium on Biomedical Imaging (ISBI)
Jiménez-Sánchez, A., Juodelyte, D., Chamberlain, B., Cheplygina, V.: Detecting shortcuts in medical images-a case study in chest x-rays. In: 2023 IEEE 20th In- ternational Symposium on Biomedical Imaging (ISBI). pp. 1–5. IEEE (2023)
2023
-
[12]
Scientific data 6(1), 317 (2019)
Johnson, A.E., Pollard, T.J., Berkowitz, S.J., Greenbaum, N.R., Lungren, M.P., Deng, C.y., Mark, R.G., Horng, S.: Mimic-cxr, a de-identified publicly available database of chest radiographs with free-text reports. Scientific data 6(1), 317 (2019)
2019
-
[13]
PhysioNet (2024)
Johnson, A.E., Pollard, T.J., Mark, R.G., Berkowitz, S.J., Horng, S.: Mimic- cxr database (version 2.1.0). PhysioNet (2024). https://doi.org/10.13026/ 4jqj-jw95
2024
-
[14]
Proceedings of the National Academy of Sciences117(23), 12592–12594 (2020)
Larrazabal, A.J., Nieto, N., Peterson, V., Milone, D.H., Ferrante, E.: Gender im- balance in medical imaging datasets produces biased classifiers for computer-aided diagnosis. Proceedings of the National Academy of Sciences117(23), 12592–12594 (2020)
2020
-
[15]
Ad- vances in Neural Information Processing Systems35, 17612–17625 (2022)
Liang, V.W., Zhang, Y., Kwon, Y., Yeung, S., Zou, J.Y.: Mind the gap: Under- standing the modality gap in multi-modal contrastive representation learning. Ad- vances in Neural Information Processing Systems35, 17612–17625 (2022)
2022
-
[16]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Luo, Y., Shi, M., Khan, M.O., Afzal, M.M., Huang, H., Yuan, S., Tian, Y., Song, L., Kouhana, A., Elze, T., et al.: Fairclip: Harnessing fairness in vision-language learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 12289–12301 (2024)
2024
-
[17]
European Radiology34(12), 7895–7903 (2024)
Mosquera, C., Ferrer, L., Milone, D.H., Luna, D., Ferrante, E.: Class imbalance on medical image classification: towards better evaluation practices for discrimination and calibration performance. European Radiology34(12), 7895–7903 (2024)
2024
-
[18]
In: Proceedings of the ACM conference on health, inference, and learning
Oakden-Rayner, L., Dunnmon, J., Carneiro, G., Ré, C.: Hidden stratification causes clinically meaningful failures in machine learning for medical imaging. In: Proceedings of the ACM conference on health, inference, and learning. pp. 151–159 (2020)
2020
-
[19]
Research Square pp
Restrepo, D., Wu, C., Vásquez-Venegas, C., Nakayama, L.F., Celi, L.A., López, D.M.: Df-dm: A foundational process model for multimodal data fusion in the artificial intelligence era. Research Square pp. rs–3 (2024)
2024
-
[20]
In: The Thirteenth International Conference on Learning Representations (2025), https://openreview.net/forum?id=uAFHCZRmXk
Schrodi, S., Hoffmann, D.T., Argus, M., Fischer, V., Brox, T.: Two effects, one trigger: On the modality gap, object bias, and information imbalance in contrastive vision-language models. In: The Thirteenth International Conference on Learning Representations (2025), https://o...
2025
-
[21]
Nature medicine27(12), 2176–2182 (2021)
Seyyed-Kalantari, L., Zhang, H., McDermott, M.B., Chen, I.Y., Ghassemi, M.: Underdiagnosisbiasofartificialintelligencealgorithmsappliedtochestradiographs in under-served patient populations. Nature medicine27(12), 2176–2182 (2021)
2021
-
[22]
JMIR medical informatics7(3), e10010 (2019)
Shen, J., Zhang, C.J., Jiang, B., Chen, J., Song, J., Liu, Z., He, Z., Wong, S.Y., Fang, P.H., Ming, W.K., et al.: Artificial intelligence versus clinicians in dis- ease diagnosis: systematic review. JMIR medical informatics7(3), e10010 (2019). https://doi.org/10.2196/10010
2019 doi
-
[24]
Nature biomedical engineering6(12), 1399–1406 (2022)
Tiu, E., Talius, E., Patel, P., Langlotz, C.P., Ng, A.Y., Rajpurkar, P.: Expert-level detection of pathologies from unannotated chest x-ray images via self-supervised learning. Nature biomedical engineering6(12), 1399–1406 (2022)
2022
-
[25]
In: Computer Vision and Pattern Recognition
Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.M.: Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised clas- sification and localization of common thorax diseases. In: Computer Vision and Pattern Recognition. pp. 2097–2106 (2017)
2017
-
[26]
In: Proceedings of the Conference on Empirical Methods in Natural Language Processing
Wang, Z., Wu, Z., Agarwal, D., Sun, J.: Medclip: Contrastive learning from un- paired medical images and text. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Conference on Empirical Methods in Natural Language Processing. vol. 2022, p. 3...
2022
-
[27]
Science Advances11(13), eadq0305 (2025)
Yang, Y., Liu, Y., Liu, X., Gulhane, A., Mastrodicasa, D., Wu, W., Wang, E.J., Sahani, D., Patel, S.: Demographic bias of expert-level vision-language foundation models in medical imaging. Science Advances11(13), eadq0305 (2025)
2025
-
[28]
In: International Conference on Medical Image Computing and Computer-Assisted Intervention
You, K., Gu, J., Ham, J., Park, B., Kim, J., Hong, E.K., Baek, W., Roh, B.: Cxr- clip: Toward large scale chest x-ray language-image pre-training. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 101–111. Springer (2023)
2023
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.