Pith. sign in

REVIEW 4 major objections 6 minor 21 references

Comparison of ConvNeXt and Vision-Language Models for Breast Density Assessment in Screening Mammography

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-tuned CNN beats vision-language model on breast density.

desk verdict Plausible small benchmark, but the patient-level split is unconfirmed and the comparison is fine-tuning vs linear probing—so the headline gap is provisional. read the letter →

arxiv 2506.13964 v1 pith:7WUFX77B submitted 2025-06-16 eess.IV cs.LG

Yusdivia Molina-Román , David Gómez-Ortiz , Ernestina Menasalvas-Ruiz , José Gerardo Tamez-Peña , Alejandro Santos-Díaz This is my paper · ORCID
classification eess.IVcs.LG
keywords breastdensityclassificationBI-RADSmammographyConvNeXtBioMedCLIPvision-languagemodelslinearprobingfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether a vision-language model pretrained on biomedical image-caption pairs can match a conventional convolutional network for the specialized task of mammographic breast density classification. On a dataset of 1,160 screening exams organized into four BI-RADS density classes, it compares BioMedCLIP under zero-shot and linear-probe settings with a ConvNeXt model fine-tuned end-to-end. The reported result is that fine-tuned ConvNeXt reaches 0.73 accuracy and 0.78 F1, above BioMedCLIP's 0.64/0.63 linear probe and 0.47/0.31 zero-shot scores. A sympathetic reading of the paper's conclusion is that for this task, end-to-end adaptation of a vision model matters more than large-scale multimodal pretraining, and that VLM text prompts need domain-specific detail before they help.

What carries the argument

The comparison rests on two pretrained models evaluated under one controlled protocol. BioMedCLIP is a contrastive vision-language model pretrained on about 15 million biomedical image-caption pairs; here its image encoder is used either with no training (zero-shot textual prompts) or as a frozen feature extractor feeding a linear classifier. ConvNeXt-Base is a modernized convolutional network pretrained on ImageNet; here it is fine-tuned end-to-end with its final layer replaced by a four-class head. The argumentative work is done by holding the dataset, the reported 85/15 split for the two trained experiments, the optimizer settings, and the metrics fixed while varying only the adaptation strategy, so the measured gap is attributed to whether the model adapts its features to mammographic density patterns or only reuses frozen embeddings.

What would settle it

Rerun Experiments 2 and 3 with the split made per case rather than per image; if images from the same exam appear on both sides of the current split, or if the ConvNeXt margin shrinks under a per-case split, the comparison does not measure generalization to new patients.

Watch

Extended reading notes

Core claim

The paper's central claim is that, on a balanced four-class BI-RADS density task built from real screening mammograms, a fine-tuned ConvNeXt-Base outperforms BioMedCLIP in both accuracy and F1-score. The numbers supporting the claim are 0.73 accuracy and 0.78 F1 for ConvNeXt fine-tuning, 0.64 and 0.63 for a linear probe trained on frozen BioMedCLIP embeddings, and 0.47 and 0.31 for zero-shot BioMedCLIP. In the paper's own framing, this shows that CNN-based models with end-to-end fine-tuning provide stronger performance for specialized medical imaging than frozen VLM embeddings with a linear head, and it attributes part of the VLM shortfall to textual descriptors that are too coarse to separate adjacent density categories.

Load-bearing premise

The result assumes the 85/15 split separated patients, but the paper never states that the four images from one screening exam all stayed in the same partition, so reported accuracy and F1 could be inflated by patient overlap.

Editorial extensions

If this is right

  • For similar screening mammography settings, a fine-tuned CNN should be the baseline to beat before introducing a VLM, because end-to-end feature adaptation gave the best reported numbers.
  • Zero-shot classification by BioMedCLIP is too weak for breast density work; any clinical use of the model would need at least a trained classifier on its embeddings.
  • VLM improvements should focus on richer textual descriptions or domain-specific fine-tuning, since the linear probe's main failure was separating adjacent BI-RADS categories that the frozen embeddings did not distinguish.
  • The confusion patterns shared by both models, especially around heterogeneously dense tissue, indicate that the remaining errors reflect genuine radiological ambiguity rather than a single architecture's flaw.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper compares only a linear probe for BioMedCLIP, not fine-tuning the VLM itself, so its conclusion is best read as 'frozen-embedding VLMs underperform fine-tuned CNNs here' rather than 'VLMs cannot match CNNs on medical imaging.'
  • Because the split is described by image counts and never stated to be per patient, the reported gap could shrink under a strict per-case split; that is a testable concern, not a claim the paper makes.
  • A direct test would be to run the same two models on a public mammography density benchmark with external labels; the private one-hospital dataset leaves open how well the comparison transfers.
  • The 0.47 zero-shot accuracy with 0.31 F1 suggests heavy class imbalance in predictions; per-class recall numbers would show whether the zero-shot model simply collapses toward majority categories.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper compares a vision-language model (BioMedCLIP) and a CNN (ConvNeXt) for BI-RADS breast density classification on a private screening mammography dataset. BioMedCLIP is evaluated in zero-shot and linear-probing modes, while ConvNeXt is fully fine-tuned. Reported results are: zero-shot accuracy 0.47 / F1 0.31; BioMedCLIP linear probe accuracy 0.64 / F1 0.63; ConvNeXt fine-tuned accuracy 0.73 / F1 0.78. The authors conclude that fine-tuned ConvNeXt consistently outperforms BioMedCLIP and that CNN-based models with end-to-end fine-tuning provide stronger performance for specialized medical imaging.

Significance. If the results are robust, the paper provides a useful empirical data point for the medical imaging community, particularly in comparing general-purpose VLMs with domain-tuned CNNs on a clinically relevant, private dataset with paired radiology reports. The use of a real clinical dataset and the inclusion of three learning scenarios are strengths. However, the significance is limited by several experimental design gaps: the absence of patient-level splitting, lack of uncertainty quantification, and an uncontrolled comparison between a frozen-encoder linear probe and a fully fine-tuned model. These issues mean the central claim, as stated, is not firmly established by the presented evidence.

major comments (4)
  1. [Section III.C.2 and III.C.3] The dataset split is not described as patient-exclusive. Each case has four images (MLO/CC for both breasts), and the manuscript does not state that all images from the same patient or exam were kept in the same split. A random image-level split can place images from the same patient in both the training and test sets, which can inflate accuracy and F1 through case-specific memorization rather than density learning. This is load-bearing because the paper's central conclusion about ConvNeXt's superiority over BioMedCLIP depends on generalization to unseen patients. The authors must specify whether the split was performed at the case/patient level, and if not, repeat the experiments with a patient-level split and report the results.
  2. [Table I and Section IV] No confidence intervals, standard deviations, or multiple random seeds are reported for any experiment. With a balanced dataset of roughly 450 images per class, the observed gaps (accuracy 0.73 vs 0.64; F1 0.78 vs 0.63) could be within run-to-run variability. The authors should repeat each experiment at least 5 times with different random seeds and report mean ± standard deviation or bootstrap confidence intervals, and ideally apply a paired significance test such as McNemar's test for the classification outcomes.
  3. [Sections III.B and IV] The comparison is not architecture-controlled: BioMedCLIP is evaluated only with a frozen encoder plus a linear head, while ConvNeXt is fully fine-tuned end-to-end. The observed performance gap therefore conflates model architecture with training procedure. The Abstract's conclusion that "CNN-based models with end-to-end fine-tuning provide stronger performance" is an overgeneralization; a fairer comparison would also fine-tune BioMedCLIP (or its image encoder) or evaluate ConvNeXt with a linear probe on frozen features. Without this control, the central claim is not established by the presented experiments.
  4. [Section III.A] The exact post-downsampling dataset size is not reported. The text states "approximately 450 images per breast density class," but the exact number per class, the total number of images used for the split, and whether downsampling was performed before or after the patient-level split are not given. This prevents the reader from computing the test set size and from assessing the effective sample size and statistical power of the comparisons.
minor comments (6)
  1. [Section IV.C] The text refers to "validation accuracy" and "validation F1" when describing results obtained on the held-out 15% test set; the terminology should be consistent with "test" accuracy and F1 to avoid confusion.
  2. [Section III.C.1] The four textual prompts used for zero-shot classification are not listed; providing the exact prompts is necessary for reproducibility.
  3. [Section II] ConvNeXt is described as "a refined version of ResNet-50"; this is imprecise. ConvNeXt is a modernized ResNet-style architecture with transformer-inspired design choices, and the description should be more accurate.
  4. [Sections II and V] Reference [13] is cited to justify linear probing as the evaluation setup used in the original BioMedCLIP benchmark experiments, but [13] is a paper on microscopic hematological image classification, not the BioMedCLIP original; the citation appears inappropriate.
  5. [Section IV.C] The phrase "Characterized by scattered areas of pattern density" is awkward and inconsistent with the class name "Scattered areas of fibroglandular density" used elsewhere in the paper.
  6. [Section V] There is a typo: "Aditionally" should be "Additionally."

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported comparison is an empirical benchmark whose inputs are external to the conclusions drawn.

full rationale

The paper's central comparison (Table I: BioMedCLIP zero-shot 0.47/0.31, BioMedCLIP linear probe 0.64/0.63, ConvNeXt fine-tune 0.73/0.78) is an empirical benchmark, not a derivation from first principles. The fitted parameters are standard supervised-training weights; the reported accuracy and F1 are evaluated on a held-out 15% test set (Sections III.C.2 and III.C.3), and no reported quantity is defined in terms of another reported quantity such that a 'prediction' equals its input by construction. The choice of BioMedCLIP linear probing over fine-tuning (Section III.B.1) is justified by dataset size, compute, and alignment with the original BioMedCLIP benchmark [13], an external citation; this is a methodological choice, not a circular argument. The preprocessing steps citing [15] and [12] describe prior methodology and do not contain the target comparison. The only notable methodological concern is that the 85/15 split is not stated to be patient-exclusive despite each case containing four images (Section III.A), which could inflate scores via same-patient leakage; however, this is a data-splitting validity risk, not circularity, because the measured quantities are not forced to equal the training inputs by construction. There is no load-bearing self-citation chain, no fitted parameter renamed as a prediction, and no uniqueness theorem imported from the authors' prior work.

Assumptions & free parameters 2 free parameters · 2 assumptions · 0 invented entities

The paper introduces no new theoretical entities or parameters. The central claim depends on two empirical assumptions: the correctness of the extracted ground-truth labels and the patient-independence of the data split. The two free parameters listed are manual choices that affect the reported performance numbers.

free parameters (2)
  • Downsampled class size = Approximately 450 images per density class
    Random downsampling was applied to balance the four BI-RADS classes; the exact final number of images is not reported, and the imbalance-correction choice affects the learned distribution and the reported metrics.
  • Train/test split ratio = 85% train, 15% test
    The split ratio is manually chosen and is not justified, and the paper does not state whether the split respects patient identity.
assumptions (2)
  • domain assumption Labels extracted from radiology reports via regular expressions after translation are correct ground truth for breast density.
    Section III.A describes regex extraction and standardization into four BI-RADS categories. Transcription or translation errors would corrupt the labels used to train and evaluate all models.
  • ad hoc to paper The 85/15 split does not mix images from the same patient across training and test sets.
    The split is described as a simple random split with no mention of patient grouping. Since each case contains four images (two views per breast), leakage across views would inflate performance.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Comparison of ConvNeXt and Vision-Language Models for Breast Density Assessment in Screening Mammography." pith.science (2026). https://pith.science/paper/7WUFX77B

@misc{pith2026250613964,
  author       = {Pith},
  title        = {Pith review of: Comparison of ConvNeXt and Vision-Language Models for Breast Density Assessment in Screening Mammography},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7WUFX77B}},
  note         = {Machine review of arXiv:2506.13964}
}
read the original abstract

Mammographic breast density classification is essential for cancer risk assessment but remains challenging due to subjective interpretation and inter-observer variability. This study compares multimodal and CNN-based methods for automated classification using the BI-RADS system, evaluating BioMedCLIP and ConvNeXt across three learning scenarios: zero-shot classification, linear probing with textual descriptions, and fine-tuning with numerical labels. Results show that zero-shot classification achieved modest performance, while the fine-tuned ConvNeXt model outperformed the BioMedCLIP linear probe. Although linear probing demonstrated potential with pretrained embeddings, it was less effective than full fine-tuning. These findings suggest that despite the promise of multimodal learning, CNN-based models with end-to-end fine-tuning provide stronger performance for specialized medical imaging. The study underscores the need for more detailed textual representations and domain-specific adaptations in future radiology applications.

Figures

Figures reproduced from arXiv: 2506.13964 by the authors.

Figure 1
Figure 1. Outline of the processes applied to dataset. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Outline of the methodology used. This study conducts a comparative analysis of two state-of￾the-art models for breast density classification: BioMedCLIP and ConvNeXt. The goal is to evaluate their performance under consistent experimental conditions using a balanced dataset. 1) BioMedCLIP: Vision-Language Model: In this study, BioMedCLIP is evaluated under two learning scenarios: • Zero-shot learning: Classification… view at source ↗
Figure 4
Figure 4. Confusion matrix for ConvNeXt fine-tune. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (1 more)
Figure 3
Figure 3. Figure 3: Confusion matrix for BioMedCLIP with linear probing. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 15 canonical work pages

  1. [1]

    Machine Learning-Based Approaches for Breast Density Estima- tion from Mammograms: A Comprehensive Review

    Khaldoon Alhusari and Salam Dhou. “Machine Learning-Based Approaches for Breast Density Estima- tion from Mammograms: A Comprehensive Review”. In:Journal of Imaging11 (2 2025), p. 38.DOI: 10 . 3390/jimaging11020038

    work page 2025

  2. [2]

    American College of Radiol- ogy, 2013.URL: https : / / edge

    American College of Radiology.ACR BI-RADS® At- las — Mammography. American College of Radiol- ogy, 2013.URL: https : / / edge . sitecorecloud . io / americancoldf5f - acrorgf92a - productioncb02 - 3650 / media/ACR/Files/RADS/BI- RADS/Mammography- Reporting.pdf

    work page 2013

  3. [3]

    Automatic breast density classifi- cation using neural network

    Dooman Arefan et al. “Automatic breast density classifi- cation using neural network”. In:Journal of Instrumen- tation10.12 (Dec. 2015), T12002.DOI: 10.1088/1748- 0221/10/12/T12002.URL: https://dx.doi.org/10.1088/ 1748-0221/10/12/T12002

    doi:10.1088/1748- 2015

  4. [4]

    PubMedCLIP: How Much Does CLIP Benefit Visual Question Answering in the Medical Domain?

    Sedigheh Eslami, Christoph Meinel, and Gerard de Melo. “PubMedCLIP: How Much Does CLIP Benefit Visual Question Answering in the Medical Domain?” In:Findings of the Association for Computational Lin- guistics: EACL 2023. Association for Computational Linguistics, May 2023, pp. 1181–1193.DOI: 10.18653/ v1/2023.findings-eacl.88

    work page 2023

  5. [5]

    Shantanu Ghosh et al.Mammo-CLIP: A Vision Lan- guage Foundation Model to Enhance Data Efficiency and Robustness in Mammography. 2024. arXiv: 2405. 12255[eess.IV]

    work page 2024

  6. [6]

    Automatic Quantification of Breast Density from Mammography Using Deep Learning

    Kenichi Inoue et al. “Automatic Quantification of Breast Density from Mammography Using Deep Learning”. In: Nihon H ¯oshasen Gijutsu Gakkai zasshi77 (10 2021), pp. 1165–1172.DOI: 10.6009/JJRT.2021 JSRT 77.10. 1165

    work page doi:10.6009/jjrt.2021 2021

  7. [7]

    Automated breast density assessment for full-field digital mammography and digital breast to- mosynthesis

    Shu Jiang et al. “Automated breast density assessment for full-field digital mammography and digital breast to- mosynthesis”. In:Cancer Prevention Research(2024). DOI: 10.1158/1940-6207.capr-24-0338

    work page doi:10.1158/1940-6207.capr-24-0338 2024

  8. [8]

    Jiaxiang Liu et al.KPL: Training-Free Medical Knowl- edge Mining of Vision-Language Models. 2025. arXiv: 2501.11231[cs.CV]

    work page Pith review arXiv 2025

Show all 21 references
  1. [9]

    Zhuang Liu et al.A ConvNet for the 2020s. 2022. arXiv: 2201.03545[cs.CV]

    2022 arXiv

  2. [10]

    Breast Density and Parenchymal Patterns as Markers of Breast Cancer Risk: A Meta-analysis

    Valerie McCormack and Isabel dos Santos Silva. “Breast Density and Parenchymal Patterns as Markers of Breast Cancer Risk: A Meta-analysis”. In:Cancer Epidemiology, Biomarkers & Prevention15.6 (June 2006), pp. 1159–1169.ISSN: 1055-9965.DOI: 10.1158/ 1055-9965.EPI-06-0034

    2006

  3. [11]

    Enhancing di- agnostic accuracy in breast cancer: integrating novel machine learning approaches with enhanced image pre- processing for improved mammography analysis

    Mohsen Mehrabi and Nafise Salek. “Enhancing di- agnostic accuracy in breast cancer: integrating novel machine learning approaches with enhanced image pre- processing for improved mammography analysis”. In: Polish Journal of Radiology89 (2025), pp. 573–583. DOI: 10.5114/pjr/195523

    2025 doi

  4. [12]

    Automated Radi- ology Report Generation Using Radiomics and Natural Language Processing Techniques

    Beatriz Alejandra Bosques Palomo. “Automated Radi- ology Report Generation Using Radiomics and Natural Language Processing Techniques”. Master of Science in Computer Science. Instituto Tecnol ´ogico y de Estu- dios Superiores de Monterrey, Campus Monterrey, May 2024

    2024

  5. [13]

    Microscopic Hematological Image Classifica- tion with Captions Using Few-Shot Learning in Data- Scarce Environments

    Tanviben Patel, Hoda El-Sayed, and Md Kamruzzaman Sarker. “Microscopic Hematological Image Classifica- tion with Captions Using Few-Shot Learning in Data- Scarce Environments”. In:2024 IEEE International Conference on Internet of Things and Intelligence Sys- tems (IoTaIS). 202...

    2024

  6. [14]

    Alec Radford et al.Learning Transferable Visual Mod- els From Natural Language Supervision. 2021. arXiv: 2103.00020[cs.CV]

    2021 arXiv

  7. [15]

    Named Entity Recognition in Mammography Radiology Reports us- ing a Multilingual Transfer Learning Approach

    Esteban Ricardo Salazar Cabrera et al. “Named Entity Recognition in Mammography Radiology Reports us- ing a Multilingual Transfer Learning Approach”. In: 2024 IEEE 37th International Symposium on Computer- Based Medical Systems (CBMS). IEEE Computer So- ciety, June 2024, pp. 2...

    2024

  8. [16]

    Daan Schouten et al.Navigating the landscape of mul- timodal AI in medicine: a scoping review on technical challenges and clinical applications. 2024. arXiv: 2411. 03782[cs.AI].URL: https : / / arxiv. org / abs / 2411 . 03782

    2024

  9. [17]

    Transformers in medical imag- ing: A survey

    Fahad Shamshad et al. “Transformers in medical imag- ing: A survey”. In:Medical Image Analysis88 (2023), p. 102802.ISSN: 1361-8415.DOI: https : / / doi . org / 10 . 1016 / j . media . 2023 . 102802.URL: https : / / www . sciencedirect . com / science / article / pii / S1361841...

    2023

  10. [18]

    BioCLIP: A Vision Foundation Model for the Tree of Life

    Samuel Stevens et al. “BioCLIP: A Vision Foundation Model for the Tree of Life”. In:2024 IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion (CVPR). 2024, pp. 19412–19424.DOI: 10.1109/ CVPR52733.2024.01836

    2024

  11. [19]

    ConvNext: A Contemporary Archi- tecture for Convolutional Neural Networks for Image Classification

    Agastya Todi et al. “ConvNext: A Contemporary Archi- tecture for Convolutional Neural Networks for Image Classification”. In:2023 3rd International Conference on Innovative Sustainable Computational Technologies (CISCT). 2023, pp. 1–6.DOI: 10.1109/CISCT57197. 2023.10351320

    2023

  12. [20]

    An Image Classification Method Based on Self-attention Con- vNeXt

    Zujian Yang, Zhao Qiu, and HuiJuan Xie. “An Image Classification Method Based on Self-attention Con- vNeXt”. In:Proceedings of the 12th International Conference on Computer Engineering and Networks. Springer Nature Singapore, 2022, pp. 657–666.ISBN: 978-981-19-6901-0

    2022

  13. [21]

    Sheng Zhang et al.BiomedCLIP: a multimodal biomed- ical foundation model pretrained from fifteen million scientific image-text pairs. 2025. arXiv: 2303 . 00915 [cs.CV]

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.