Distribution-Based Masked Medical Vision-Language Model Using Structured Reports

Lu Yang; Ruichi Zhang; Shreyank N Gowda; Xiao Gu; Ying Weng

REVIEW 5 major objections 5 minor 2 cited by

Distribution-Based Masked Medical Vision-Language Model Using Structured Reports

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that representing chest X-ray image patches and text tokens as Gaussian distributions and training with a distribution-based masked objective, on structured LLM-generated reports, produces state-of-the-art representations…

desk verdict Useful incremental method for chest X-ray VLM pre-training, but the undisclosed source of disease names in the structured reports could leak labels into pre-training and make the SOTA claims unfair. read the letter →

arxiv 2507.21794 v1 pith:X6A2GPVW submitted 2025-07-29 cs.CV

Shreyank N Gowda , Ruichi Zhang , Xiao Gu , Ying Weng , Lu Yang This is my paper · ORCID

classification cs.CV

keywords medicalvision-languagepre-trainingchestX-rayuncertaintymodelingmaskedimage-languageGaussiandistributionsstructuredreportszero-shotclassificationsegmentation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that uncertainty-aware pre-training, in which image patches and text tokens are represented as Gaussian distributions rather than fixed points, gives chest X-ray vision-language models better downstream performance than deterministic pre-training. The authors combine this distribution-based masked modeling objective with structured reports generated by a large language model, where each report opens with a disease definition and an 'appearance' section before listing observations and verdicts. Across semi-supervised, fully supervised, zero-shot, grading, and segmentation benchmarks, they report consistent gains over prior medical vision-language methods. If correct, the work suggests that explicitly modeling ambiguity in both modalities is a transferable improvement for medical image-text pre-training.

What carries the argument

The load-bearing device is the distribution-based masked modeling objective: each token or patch is mapped to a Gaussian $N(\mu_i, \sigma_i^2)$, masked elements are predicted as a distribution $N(\hat\mu_i, \hat\sigma_i^2)$, and the loss $L_{\text{D-MLM}}$ is the KL divergence between predicted and target distributions over all masked text tokens and image patches. A secondary alignment loss minimizes Wasserstein distance between paired image-text distributions. The other key component is the structured report template produced by an LLM: a disease definition, a radiographic 'appearance' section, then observations and verdicts. The paper uses this text to guide masking toward diagnostically relevant regions and to anchor both modalities in clinical semantics.

What would settle it

Regenerate the structured reports using disease labels extracted automatically from the original free-text radiology reports rather than from ground-truth annotations, retrain D-MLM on MIMIC-CXR, and re-run the zero-shot and fine-tuned benchmarks; if the margin over prior baselines shrinks or disappears, the extra supervision in the LLM text, not the distribution-based objective, is the active ingredient.

Watch

Extended reading notes

Core claim

The central claim is that a pre-training method called D-MLM, which encodes image patches and text tokens as multivariate Gaussians and trains by predicting masked elements through KL divergence, captures the inter- and intra-modal uncertainty that deterministic embeddings miss, and that this is what drives the reported improvements. The paper argues that the structured reports add clinically grounded context that further stabilizes alignment, and that the combination yields state-of-the-art results on chest X-ray classification, zero-shot classification, edema severity grading, and disease segmentation. The authors state this in their conclusion as the ability to learn meaningful associations between visual and textual information while accounting for the natural ambiguity and variability of clinical data.

Load-bearing premise

The load-bearing premise is that the structured-report pre-training text is a fair substitute for the original reports; the paper does not disclose where the disease name used in the LLM prompts comes from, so if it is the ground-truth label of the image, the model receives explicit disease supervision that label-agnostic baselines do not.

Editorial extensions

If this is right

Pre-trained chest X-ray representations transfer to 1%, 10%, and 100% label regimes, with the largest margins where labels are scarce.
Zero-shot classification works not only on diseases seen during pre-training but also on COVID-19, which is absent from the pre-training corpus.
Distribution-based features can be fine-tuned for segmentation as well as classification, suggesting the uncertainty modeling does not collapse spatial information.
The structured-report template is itself a reusable ingredient: replacing noisy free text with definition, appearance, observations, and verdicts improves downstream results in the paper's ablations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A fair decomposition of the gain would require ablating the source of the disease name used to prompt the LLM; if that name comes from ground-truth labels, part of the advantage over label-agnostic baselines may be supervision leakage rather than uncertainty modeling.
The per-token variance parameters produced by the Gaussian encoders could double as a cheap uncertainty signal for flagging out-of-distribution or ambiguous X-rays, but the paper does not evaluate them that way.
The same masked-distribution recipe should transfer to other medical modalities such as CT or pathology whenever paired structured text can be generated, but the paper only demonstrates chest X-ray evidence.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Shreyank and colleagues propose a straightforward next step on their own M&M model: represent image and text features as Gaussians, do masked reconstruction with KL divergence, and replace the original radiology reports with LLM-generated structured reports. The combination is new in chest X-ray pre-training, and the paper delivers consistent, if modest, improvements across a wide set of benchmarks: semi-supervised classification, zero-shot, segmentation, and edema grading. The ablations are honest—the alignment loss is not essential, and the structured reports are what move the needle. Putting the generated texts on GitHub is helpful.

The main problem is one load-bearing disclosure gap. The prompts in Section 2.1 are built around '[disease name]', but the paper never states where that name comes from. Section 3.1 says MIMIC-CXR v2 is 'annotated with 14 common chest conditions, which we leverage for pre-training our model.' That reads like the ground-truth disease labels are used to generate the structured report for each image. If that's the case, then every pre-training text explicitly contains the class label, which baselines like M&M, ConVIRT, GLoRIA, and PRIOR do not receive. The zero-shot results in Tables 2 and 3 would then be partially supervised, and the adaptive masking guided by the 'Appearance' section would also be label-guided. This can fully account for the reported 1-2 point gains over unsupervised baselines. The GitHub link stores the generated texts but not the assignment rule, so the ambiguity is unresolved.

There are two smaller gaps. The adaptive masking mechanism is never actually specified—what does 'identified from the Appearance section' mean operationally? And the zero-shot evaluation protocol is not described: no information on the text prompts used per class or the exact metric computation. Tables also lack error bars, even though the text mentions standard deviations for Table 1, so the margins could be noise.

These are repair problems, not fatal contradictions. The core idea is coherent and the experiments are broad enough to justify referee time. But the authors need to disclose the disease-name source and, if labels are used, re-benchmark against label-aware baselines or show a label-free protocol. Without that, the SOTA claim is not trustworthy. I'd send it to review with a major-revision recommendation, and I would not cite the results until the leakage question is settled. If you run a reading group, this is a good example of why pre-training comparisons need careful protocol disclosure.

Referee Report

5 major / 5 minor

Summary. This paper proposes D-MLM, a medical vision-language pre-training method that represents image and text features as multivariate Gaussian distributions and trains with a distribution-based masked image-language modeling objective together with an optional Wasserstein alignment loss. The text inputs are LLM-generated structured reports that contain a disease definition, an appearance section, and observations/verdicts. The authors evaluate on chest X-ray datasets and report state-of-the-art results in semi-/fully supervised classification, zero-shot classification, edema severity grading, and segmentation.

Significance. If the reported results hold, D-MLM would offer a strong pre-trained representation for chest X-ray analysis, with consistent gains over several strong baselines across multiple tasks and datasets. The paper provides the generated structured report text at a public link, which helps reproduce the text-input side. The ablation showing that the alignment loss is not essential is a useful negative result and reduces concern that the main gains reduce to that loss term. However, the significance is currently tempered by missing disclosure of the source of the disease names used in prompt generation, an unspecified adaptive masking mechanism, and an undescribed zero-shot evaluation protocol.

major comments (5)

[§2.1 and §3.1] The paper never states where the disease name in the prompts 'Define [disease name]' and 'distinguishing radiographic signs of [disease name]' comes from. Section 3.1 says MIMIC-CXR v2 is 'annotated with 14 common chest conditions, which we leverage for pre-training our model.' If the disease name is taken from these ground-truth annotations, then every structured report explicitly contains the class label, and the adaptive masking guided by the Appearance section is also label-guided. This would make the zero-shot comparisons in Tables 2 and 3, and the classification gains in Table 1, partially attributable to label leakage rather than to the proposed uncertainty modeling. The manuscript must disclose the source of the disease name, and either provide a label-free generation protocol or re-run the experiments without label-derived text.
[§2.2 and §3.5] The adaptive masking mechanism is not specified. Section 2.2 says image masking is focused on 'diagnostically relevant regions identified from the Appearance section of reports,' and Section 3.5 mentions an adaptive ratio for images based on the paired text, but no procedure, equation, or algorithm maps the Appearance text to image patches. Without this specification, Table 7's ablation cannot be reproduced, and it is impossible to determine whether the masking gains come from the proposed mechanism or from label guidance. Provide a concrete description of how the text is converted to mask locations and how the mask ratio is computed.
[§3.2 and §3.5] The zero-shot classification protocol is not described. The paper says only 'Following MedKLIP [33]' and reports AUC/F1/ACC for RSNA, SIIM-ACR, NIH, and COVID-19. It does not state the text prompt templates used for each class, whether the same prompts are applied to all baseline methods, or whether any classification head or thresholding is used. This information is essential for interpreting Tables 2 and 3 and for assessing the fairness of the comparison.
[§3.2 and Tables 1–5] Table 1 states that results are averages with standard deviations over five runs 'as provided by PRIOR,' but no standard deviations, confidence intervals, or significance tests are reported in any table. Without error bars, the reported improvements of roughly 0.5–2.3% over M&M may not be statistically meaningful. The authors should report the standard deviations they claim to have, or explicitly state that they are omitted and provide significance tests for the main comparisons.
[§3.6] The ablation does not isolate the contribution of the distribution-based representation. D-MLM differs from M&M in the Gaussian representation, the structured reports, and the adaptive masking. Table 7 includes 'No Masking' and other masking baselines, but none uses the same Gaussian representation with everything else held fixed. The claim that modeling features as probabilistic distributions is responsible for the gains is therefore not directly supported. Add an ablation that keeps the report format and masking strategy fixed and toggles only the distribution-based representation.

minor comments (5)

[§3.2/§3.3] Table 3 (COVID-19 zero-shot) is discussed in Section 3.2 but the table appears in Section 3.3; the layout makes the referents of Tables 3 and 4 confusing and should be adjusted.
[§2.2, Eq. (1)] Equation (1) writes each representation as N(µ_i, σ_i^2) but does not state whether σ_i is a scalar, diagonal vector, or full covariance matrix, nor how the encoder produces these parameters. Specify the parameterization and the reparameterization used during training.
[§3.5] The text says masking ratios are 'dynamically adjusted: an adaptive ratio for images based on the paired text, and a 30% ratio for text,' but Section 2.2 says text tokens are masked at a fixed 30%. Clarify whether the text mask ratio is fixed or also adaptive.
[§3.6] The statement 'even without alignment loss, our model outperforms competitors' is not accompanied by a table, figure, or numerical result. Provide the corresponding ablation numbers so the claim can be verified.
[General] The GitHub link contains the generated structured report text but no code or trained checkpoints. Since the adaptive masking and Gaussian parameterization are not fully specified in the paper, a code release would substantially improve reproducibility.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the D-MLM objective is self-supervised and the alignment loss is shown to be non-essential; only minor self-citation to M&M, and the undisclosed disease-name source is a leakage risk rather than a derivation circularity.

full rationale

The paper's derivation chain is self-contained. Equation (3) defines a masked reconstruction loss using KL divergence between the predicted Gaussian and the model's own encoded Gaussian for the unmasked token or patch, so the objective is a standard self-supervised autoencoding target rather than a fitted downstream label. Equation (4)'s alignment loss is explicitly shown to be non-essential: the paper states 'even without it we outperform existing approaches,' and no parameter is fitted to a subset of the evaluation data and then reported as a prediction. Downstream results are reported on held-out benchmarks (RSNA, SIIM, NIH, CheXpert, COVIDx, Edema Severity). The only self-citation is to M&M [13], the authors' own prior work, which is used for the 'Observations and Verdicts' text component and as a baseline; this is not load-bearing for the distribution-based modeling or the adaptive masking, which are the paper's actual contributions and are compared against M&M rather than assumed from it. A legitimate threat to validity, though not a circularity of the derivation, is that Section 2.1 generates reports from an unspecified '[disease name]' while Section 3.1 says MIMIC-CXR annotations are 'leveraged for pre-training'; if the disease name is the ground-truth label, the text leaks class labels into pre-training and makes the zero-shot comparisons to label-agnostic baselines unfair. That is a data-leakage or correctness concern, not a case of the derivation assuming its own conclusion.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical entities or novel mathematical objects. Its assumptions are modeling and data-generation choices: the validity of LLM reports, the mapping from text to image regions, the source of disease names, and the Gaussian representation of features. These are pragmatic assumptions rather than new postulates.

free parameters (2)

text mask ratio = 0.3
Section 2.2 sets text masking to 30%, and Section 3.6 states it was fixed based on experimental analysis.
loss weight lambda = 0.2
Section 3.6: 'We use a fixed masking ratio of 0.3 and a lambda of 0.2 based on experimental analysis.' This weights the D-MLM loss vs the alignment loss.

assumptions (4)

ad hoc to paper LLM-generated structured reports are clinically accurate and consistent with the paired images.
Section 2.1 assumes GPT-4 outputs for disease definitions and radiographic appearance match the actual image content. No validation of report correctness is provided.
ad hoc to paper The Appearance text can be used to identify diagnostically relevant image regions for masking.
Section 2.2 relies on this to implement adaptive masking, but no mapping from text to image patches is specified.
domain assumption Per-image disease names are available without leaking supervised label information.
Section 2.1 requires '[disease name]' to prompt the LLM. If the disease name comes from ground-truth labels, the pre-training is partially supervised, which is not disclosed.
domain assumption Gaussian distribution parameterization of transformer features is a valid representation of uncertainty.
Section 2.2 assumes each token and patch can be modeled as a Gaussian with mean and variance, with no justification or calibration of the uncertainty estimates.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Distribution-Based Masked Medical Vision-Language Model Using Structured Reports." pith.science (2026). https://pith.science/paper/X6A2GPVW

@misc{pith2026250721794,
  author       = {Pith},
  title        = {Pith review of: Distribution-Based Masked Medical Vision-Language Model Using Structured Reports},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/X6A2GPVW}},
  note         = {Machine review of arXiv:2507.21794}
}

read the original abstract

Medical image-language pre-training aims to align medical images with clinically relevant text to improve model performance on various downstream tasks. However, existing models often struggle with the variability and ambiguity inherent in medical data, limiting their ability to capture nuanced clinical information and uncertainty. This work introduces an uncertainty-aware medical image-text pre-training model that enhances generalization capabilities in medical image analysis. Building on previous methods and focusing on Chest X-Rays, our approach utilizes structured text reports generated by a large language model (LLM) to augment image data with clinically relevant context. These reports begin with a definition of the disease, followed by the `appearance' section to highlight critical regions of interest, and finally `observations' and `verdicts' that ground model predictions in clinical semantics. By modeling both inter- and intra-modal uncertainty, our framework captures the inherent ambiguity in medical images and text, yielding improved representations and performance on downstream tasks. Our model demonstrates significant advances in medical image-text pre-training, obtaining state-of-the-art performance on multiple downstream tasks.

Figures

Figures reproduced from arXiv: 2507.21794 by the authors.

**Figure 1.** Overview of our D-MLM framework for medical image-text alignment. (a) Structured Report Generation: An LLM creates standardized reports with disease definitions, appearance guidance, observations, and verdicts. (b) Distribution-Based Representation: Image and text features are encoded as probabilistic distributions with means and variances. (c) D-MLM: The model masks and predicts tokens and patches using distribut… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Prototype-Enhanced Confidence Modeling for Cross-Modal Medical Image-Report Retrieval
cs.CV 2025-08 conditional novelty 5.0 of 10

PECM combines multi-level prototypes with dual-stream confidence weighting and reports up to 10.17% retrieval gains on radiology datasets, including zero-shot transfer.
Quality Versus Sparsity in Image Recovery by Dictionary Learning Using Iterative Shrinkage
cs.CV 2025-08 reject novelty 3.0 of 10

A dictionary learning paper whose abstract claims sparsity does not hurt recovery quality, but whose full text is an unrelated medical retrieval manuscript.

Reference graph

Works this paper leans on

35 extracted references · 19 canonical work pages · cited by 2 Pith papers

[33]

Proceedings of the IEEE/CVF International Conference on Computer Vision (2023)

Wu, C., Zhang, X., Zhang, Y., Wang, Y., Xie, W.: Medklip: Medical knowledge en- hanced language-image pre-training. Proceedings of the IEEE/CVF International Conference on Computer Vision (2023)

work page 2023
[1]

arXiv preprint arXiv:2303.08774 (2023)

Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.: Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

arXiv 2023
[2]

arXiv preprint arXiv:1904.03323 (2019)

Alsentzer, E., Murphy, J.R., Boag, W., Weng, W.H., Jin, D., Naumann, T., McDermott, M.: Publicly available clinical bert embeddings. arXiv preprint arXiv:1904.03323 (2019)

arXiv 2019
[3]

In: International Conference on Medi- cal Image Computing and Computer-Assisted Intervention

Baumgartner, C.F., Tezcan, K.C., Chaitanya, K., Hötker, A.M., Muehlematter, U.J., Schawkat, K., Becker, A.S., Donati, O., Konukoglu, E.: Phiseg: Capturing uncertainty in medical image segmentation. In: International Conference on Medi- cal Image Computing and Computer-Assisted Intervention. pp. 119–127. Springer (2019)

2019
[4]

In: European Conference on Computer Vision

Boecking, B., Usuyama, N., Bannur, S., Castro, D.C., Schwaighofer, A., Hyland, S., Wetscherek, M., Naumann, T., Nori, A., Alvarez-Valle, J., et al.: Making the most of text semantics to improve biomedical vision–language processing. In: European Conference on Computer Vision. pp. 1–21. Springer (2022)

work page 2022
[5]

In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part II 23

Chauhan, G., Liao, R., Wells, W., Andreas, J., Wang, X., Berkowitz, S., Horng, S., Szolovits,P.,Golland,P.:Jointmodelingofchestradiographsandradiologyreports for pulmonary edema assessment. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part II 23. pp. ...

work page 2020
[6]

In: International Conference on Machine Learning

Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for con- trastive learning of visual representations. In: International Conference on Machine Learning. pp. 1597–1607. PMLR (2020)

work page 2020
[7]

In: Proceedings of the IEEE/CVF International Conference on Computer Vision

Cheng, P., Lin, L., Lyu, J., Huang, Y., Luo, W., Tang, X.: Prior: Prototype rep- resentation joint learning from medical images and reports. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 21361–21371 (2023)

work page 2023

Show all 35 references

[8]

In: IEEE Conference on Computer Vision and Pattern Recognition

Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large- scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition. pp. 248–255. IEEE (2009)

2009
[9]

arXiv preprint arXiv:1810.04805 (2018) 10 S

Devlin,J.,Chang,M.W.,Lee,K.,Toutanova,K.:Bert:Pre-trainingofdeepbidirec- tional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018) 10 S. N. Gowda et al

2018 arXiv
[10]

arXiv preprint arXiv:2010.11929 (2020)

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

2020 arXiv
[11]

Medical Image Analysis 68, 101855 (2021)

Ghesu, F.C., Georgescu, B., Mansoor, A., Yoo, Y., Gibson, E., Vishwanath, R., Balachandran, A., Balter, J.M., Cao, Y., Singh, R., et al.: Quantifying and leverag- ing predictive uncertainty for medical image assessment. Medical Image Analysis 68, 101855 (2021)

2021
[12]

In: European Conference on Computer Vision

Gowda, S.N., Clifton, D.A.: Cc-sam: Sam with cross-feature attention and context for ultrasound image segmentation. In: European Conference on Computer Vision. pp. 108–124. Springer (2024)

2024
[13]

In: International Con- ference on Medical Image Computing and Computer-Assisted Intervention

Gowda, S.N., Clifton, D.A.: Masks and manuscripts: Advancing medical pre- training with end-to-end masking and narrative structuring. In: International Con- ference on Medical Image Computing and Computer-Assisted Intervention. pp. 426–436. Springer (2024)

2024
[14]

Gowda, S.N., Yuan, C.: Colornet: Investigating the importance of color spaces for imageclassification.In:Asianconferenceoncomputervision.pp.581–596.Springer (2018)

2018
[15]

He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick, R.: Masked autoencoders are scalablevisionlearners.In:ProceedingsoftheIEEE/CVFConferenceonComputer Vision and Pattern Recognition. pp. 16000–16009 (2022)

2022
[16]

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 9729–9738 (2020)

2020
[17]

He,K.,Zhang,X.,Ren,S.,Sun,J.:Deepresiduallearningforimagerecognition.In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (June 2016)

2016
[18]

In: European Conference on Computer Vision

He, X., Yang, Y., Jiang, X., Luo, X., Hu, H., Zhao, S., Li, D., Yang, Y., Qiu, L.: Unified medical image pre-training in language-guided common semantic space. In: European Conference on Computer Vision. pp. 123–139. Springer (2025)

2025
[19]

In: Proceedings of the IEEE/CVF International Conference on Computer Vision

Huang, S.C., Shen, L., Lungren, M.P., Yeung, S.: Gloria: A multimodal global-local representation learning framework for label-efficient medical image recognition. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 3942–3951 (2021)

2021
[20]

In: Proceedings of the AAAI Conference on Artificial Intelligence

Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., et al.: Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In: Proceedings of the AAAI Conference on Artificial In...

2019
[21]

Scientific Data 6(1), 317 (2019)

Johnson, A.E., Pollard, T.J., Berkowitz, S.J., Greenbaum, N.R., Lungren, M.P., Deng, C.y., Mark, R.G., Horng, S.: Mimic-cxr, a de-identified publicly available database of chest radiographs with free-text reports. Scientific Data 6(1), 317 (2019)

2019
[22]

In: The Eleventh International Conference on Learning Representations (2022)

Kwon, G., Cai, Z., Ravichandran, A., Bas, E., Bhotika, R., Soatto, S.: Masked vision and language modeling for multi-modal representation learning. In: The Eleventh International Conference on Learning Representations (2022)

2022
[23]

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

Li, Z., Yang, L.T., Ren, B., Nie, X., Gao, Z., Tan, C., Li, S.Z.: Mlip: Enhanc- ing medical visual representation with divergence encoder and knowledge-guided contrastive learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11704...

2024
[24]

IEEE Transactions on Medical Imaging (2024)

Liu, C., Cheng, S., Shi, M., Shah, A., Bai, W., Arcucci, R.: Imitate: Clinical prior guided hierarchical vision-language pre-training. IEEE Transactions on Medical Imaging (2024)

2024
[25]

for imaging informatics in medicine, S.: Siim-acr pneumothorax segmentation (2019), https://www.kaggle.com/ c/siim-acr-pneumothorax-segmentation

2019
[26]

In: European Conference on Computer Vision

Müller, P., Kaissis, G., Zou, C., Rueckert, D.: Joint learning of localized represen- tations from medical images and reports. In: European Conference on Computer Vision. pp. 685–701. Springer (2022)

2022
[27]

arXiv preprint arXiv:1807.03748 (2018)

Oord, A.v.d., Li, Y., Vinyals, O.: Representation learning with contrastive predic- tive coding. arXiv preprint arXiv:1807.03748 (2018)

2018 arXiv
[28]

Frontiers in Medicine9, 861680 (2022)

Pavlova, M., Terhljan, N., Chung, A.G., Zhao, A., Surana, S., Aboutalebi, H., Gunraj, H., Sabri, A., Alaref, A., Wong, A.: Covid-net cxr-2: An enhanced deep convolutional neural network design for detection of covid-19 cases from chest x-ray images. Frontiers in Medicine9, 861...

2022
[29]

In: International Conference on Machine Learning

Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International Conference on Machine Learning. pp. 8748–8763. PMLR (2021)

2021
[30]

Radiology: Artificial Intelligence1(1), e180041 (2019)

Shih, G., Wu, C.C., Halabi, S.S., Kohli, M.D., Prevedello, L.M., Cook, T.S., Sharma, A., Amorosa, J.K., Arteaga, V., Galperin-Aizenberg, M., et al.: Aug- menting the national institutes of health chest radiograph dataset with expert annotations of possible pneumonia. Radiology...

2019
[31]

In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.M.: Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classi- fication and localization of common thorax diseases. In: Proceedings of the IEEE Conference on Computer Vision and Patte...

2017
[32]

Journal of Visual Communication and Image Representation78, 103145 (2021)

Wen,Y.,Chen,L.,Deng,Y.,Zhou,C.:Rethinkingpre-trainingonmedicalimaging. Journal of Visual Communication and Image Representation78, 103145 (2021)

2021
[34]

In: International conference on artificial neural networks

Yang, S., Fevens, T.: Uncertainty quantification and estimation in medical image classification. In: International conference on artificial neural networks. pp. 671–
[35]

In: Machine Learning for Healthcare Conference

Zhang, Y., Jiang, H., Miura, Y., Manning, C.D., Langlotz, C.P.: Contrastive learn- ing of medical visual representations from paired images and text. In: Machine Learning for Healthcare Conference. pp. 2–25. PMLR (2022)

2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [33]

Proceedings of the IEEE/CVF International Conference on Computer Vision (2023)

Wu, C., Zhang, X., Zhang, Y., Wang, Y., Xie, W.: Medklip: Medical knowledge en- hanced language-image pre-training. Proceedings of the IEEE/CVF International Conference on Computer Vision (2023)

work page 2023

[2] [1]

arXiv preprint arXiv:2303.08774 (2023)

Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.: Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

arXiv 2023

[3] [2]

arXiv preprint arXiv:1904.03323 (2019)

Alsentzer, E., Murphy, J.R., Boag, W., Weng, W.H., Jin, D., Naumann, T., McDermott, M.: Publicly available clinical bert embeddings. arXiv preprint arXiv:1904.03323 (2019)

arXiv 2019

[4] [3]

In: International Conference on Medi- cal Image Computing and Computer-Assisted Intervention

Baumgartner, C.F., Tezcan, K.C., Chaitanya, K., Hötker, A.M., Muehlematter, U.J., Schawkat, K., Becker, A.S., Donati, O., Konukoglu, E.: Phiseg: Capturing uncertainty in medical image segmentation. In: International Conference on Medi- cal Image Computing and Computer-Assisted Intervention. pp. 119–127. Springer (2019)

2019

[5] [4]

In: European Conference on Computer Vision

Boecking, B., Usuyama, N., Bannur, S., Castro, D.C., Schwaighofer, A., Hyland, S., Wetscherek, M., Naumann, T., Nori, A., Alvarez-Valle, J., et al.: Making the most of text semantics to improve biomedical vision–language processing. In: European Conference on Computer Vision. pp. 1–21. Springer (2022)

work page 2022

[6] [5]

In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part II 23

Chauhan, G., Liao, R., Wells, W., Andreas, J., Wang, X., Berkowitz, S., Horng, S., Szolovits,P.,Golland,P.:Jointmodelingofchestradiographsandradiologyreports for pulmonary edema assessment. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part II 23. pp. ...

work page 2020

[7] [6]

In: International Conference on Machine Learning

Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for con- trastive learning of visual representations. In: International Conference on Machine Learning. pp. 1597–1607. PMLR (2020)

work page 2020

[8] [7]

In: Proceedings of the IEEE/CVF International Conference on Computer Vision

Cheng, P., Lin, L., Lyu, J., Huang, Y., Luo, W., Tang, X.: Prior: Prototype rep- resentation joint learning from medical images and reports. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 21361–21371 (2023)

work page 2023