Pith. sign in

REVIEW 3 major objections 7 minor 40 references

CorBenchX: Large-Scale Chest X-Ray Error Dataset and Vision-Language Model Benchmark for Report Error Correction

T0 review · 3 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read CorBenchX builds a 26,326-report synthetic error benchmark, shows top VLMs still miss roughly half of chest X-ray report errors, and finds staged reinforcement learning sharply improves detection on an open model.

desk verdict CorBenchX is a genuinely useful benchmark for chest X-ray report error correction, but the MSRL gains are uninterpretable without a stated train/test split. read the letter →

arxiv 2505.12057 v1 pith:A56Q3SNG submitted 2025-05-17 cs.AI

Jing Zou , Qingqiu Li , Chenyu Lian , Lihao Liu , Xiaohan Yan , Shujun Wang , Jing Qin This is my paper · ORCID
classification cs.AI
keywords CorBenchXradiologyreporterrordetectionchestX-rayvision-languagemodelsyntheticinjectionmulti-stepreinforcementlearningGRPObenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

CorBenchX aims to give the field a common, large-scale yardstick for two related tasks: finding mistakes in chest X-ray reports and rewriting them correctly. It builds 26,326 corrupted reports by injecting clinically common errors into MIMIC-CXR text with a reasoning model, labels each with error type and description, then benchmarks nine vision-language models. Under zero-shot prompting, the best model, o4-mini, detects the correct error type only about half the time, and all tested systems remain below what clinical use would require. The paper then argues that a multi-step reinforcement-learning schedule, which rewards format, error-type accuracy, and lexical similarity at separate stages, sharply improves detection on QwenVL2.5-7B while giving a smaller gain in correction. The value of the package is a shared testbed plus evidence that staged reward design can push open models closer to closed-source performance.

What carries the argument

Two mechanisms carry the argument. First, the dataset pipeline: DeepSeek-R1 prompts inject exactly one or two-to-three errors of five types (omission, insertion, spelling error, side confusion, other) into MIMIC-CXR reports, with a three-stage human-and-script quality control process that turns raw reports into paired corrupted/original examples with labels. Second, multi-step reinforcement learning (MSRL): the model is trained with group-relative policy optimization (GRPO), a PPO variant that normalizes advantages within sampled groups, over a three-step trajectory of error identification, error description, and error correction, with a per-step reward combining format compliance, accuracy, and BLEU similarity. The trajectory decomposition is what lets the model be rewarded for intermediate reasoning rather than only for the final rewrite.

What would settle it

Take a random sample of the source reports CorBenchX labels clean, have two radiologists independently mark any genuine error, and measure agreement. If a nontrivial fraction (say more than 2%) of 'clean' reports contain a real mistake, the reported precision and recall figures are partly counting agreement with synthetic labels rather than true error detection. A complementary check is to run the trained models on real resident-draft reports with expert-annotated errors and compare detection accuracy against the synthetic benchmark numbers.

Watch

Extended reading notes

Core claim

The central discovery is that a deliberately constructed corpus of 26,326 synthetic chest X-ray error reports exposes a clear capability gap: the best closed model, o4-mini, reaches only 50.6% average recall on single-error type identification and BLEU 0.853 on correction, while open models trail further. Applying the paper's multi-step reinforcement learning to QwenVL2.5-7B raises single-error detection precision by 38.3% and recall by 30.5% over the zero-shot baseline, and improves single-error correction by 5.2%, with larger relative gains on multi-error correction. The paper interprets these numbers as showing both that synthetic error injection at scale is a workable evaluation substrate and that sequential supervision of identification, description, and correction is more effective than single-step reinforcement learning.

Load-bearing premise

The load-bearing premise is that the 26,326 MIMIC-CXR reports sampled as 'clean' are actually error-free; the paper does not verify this, so a single-error report could secretly contain two errors and the ground-truth labels would be unreliable.

Editorial extensions

If this is right

  • CorBenchX becomes a reusable public benchmark: any future error-detection or correction model can be scored on the same 26,326 cases, making cross-paper comparisons meaningful.
  • Staged reinforcement learning is the key gain: single-step RL underperforms MSRL by 13.3%, so decomposing the task into identify, describe, correct is itself worth more than the choice of RL algorithm.
  • Current VLM detection is the bottleneck: even the best model's 50.6% detection recall means roughly half of errors go unclassified, so report-level correction scores overstate practical reliability.
  • After MSRL, an open-source model (QwenVL2.5-7B+MSRL) beats the best closed-source model on all six report-level correction metrics in the single-error setting.
  • Multi-error correction remains harder than single-error correction even after MSRL, so scaling to reports with several interacting mistakes is the next evident challenge.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the errors are LLM-injected, high scores may partly reflect learning the statistical fingerprints of DeepSeek-R1's perturbations rather than general clinical reasoning; testing on natural errors from real resident-draft reports with expert annotations would separate these.
  • The 'clean report' assumption means the measured gains could be either conservative or inflated depending on how many source reports already contain genuine mistakes; rerunning the benchmark with independently verified-clean reports would tighten the numbers.
  • The staged-reward design suggests an ordering principle that may transfer to other medical text tasks: reward correct intermediate outputs before expecting reliable final corrections, for example in discharge summaries or CT reports with identifiable error types.
  • The 38.3% detection gain versus 5.2% correction gain implies detection and correction are not equally amenable to the same reward scheme, so future work should target correction-specific rewards such as span-level edit metrics rather than BLEU.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. CorBenchX is a benchmark and error-dataset paper for chest X-ray report error detection and correction. The authors sample 26,326 reports from MIMIC-CXR, inject synthetic errors of five types via DeepSeek-R1 prompting, and pair each corrupted report with its original text, error type, and error description. They then evaluate nine vision-language models under zero-shot prompting on both detection and correction, using lexical, semantic, and clinically oriented metrics. They additionally propose MSRL, a three-stage GRPO-based reinforcement learning method that sequentially supervises error identification, error description, and error correction, and they report substantial gains over the zero-shot baselines, including 38.3% higher precision and 30.5% higher recall on single-error detection and 5.2% improvement on single-error correction with QwenVL2.5-7B.

Significance. If the reported MSRL gains survive held-out evaluation, the paper would provide both a large-scale resource and a useful method contribution. The dataset scale (26,326 pairs), the inclusion of multi-error cases, the broad VLM coverage, and the multi-metric evaluation are genuine strengths. The paper also ships an ablation against single-step RL and evaluates on clinical metrics such as CheXbertF1 and RadGraphF1, which are not directly part of the RL reward and therefore partially mitigate circularity concerns. However, the current manuscript does not establish that the evaluation set is disjoint from the MSRL training data, and it assumes source-report cleanliness without validation; both are load-bearing for the central performance claims.

major comments (3)
  1. [Section 4 and Section 5.2, Figures 4-5, Tables 2-3] The manuscript never specifies a train/test split for the MSRL experiments. Section 4 trains the policy on rewards computed from ground-truth error types, descriptions, and corrected reports, and Section 5.2 reports gains over zero-shot baselines. If the same 26,326 reports are used for both training and evaluation, the reported 38.3% precision gain, 30.5% recall gain on single-error detection, and the 5.2% correction gain are in-sample results and do not demonstrate generalization. Please state the exact number of training and evaluation reports, describe how the split was created (ideally ensuring no report or its corrupted variant appears in both sets), and report held-out results for all MSRL comparisons.
  2. [Section 3, Dataset Source and Sampling and Quality Control Pipeline] The paper states that it 'randomly sample[s] 26,326 clean reports' from MIMIC-CXR but provides no evidence that the source reports are error-free. Since the ground truth is defined by injecting exactly one or N errors into a 'clean' report, any pre-existing error in a source report makes the labels unreliable (e.g., a 'single-error' sample could actually contain two errors). Please quantify source-report cleanliness, for example by expert review of a random sample with inter-annotator agreement, and describe the human review in the QC pipeline in sufficient detail (number of annotators, qualifications, disagreements, and how many reports actually passed each stage).
  3. [Section 5.2 and Tables 2-3] The headline comparisons are presented as point estimates without confidence intervals or significance tests. For example, o4-mini (BLEU 0.853) and Claude 3.7 sonnet (BLEU 0.852) are ranked differently, but the difference is 0.001 and may be within noise. Please add bootstrap confidence intervals or appropriate statistical tests for the main detection and correction comparisons, especially for the MSRL gains, so readers can assess whether the reported improvements are reliable.
minor comments (7)
  1. [Section 6 (Conclusion)] The word 'yieding' should be 'yielding'.
  2. [Section 5.2] The phrase 'which has been approved in [31, 40]' should read 'which has been demonstrated in [31, 40]' or 'shown in [31, 40]'.
  3. [Table 4] The rows are labeled only 'RL' and 'MSRL' without model sizes; please clarify which rows correspond to QwenVL2.5-3B and which to QwenVL2.5-7B.
  4. [Figure 1 caption] The caption uses 'MsRL' while the text uses 'MSRL'; please standardize the capitalization.
  5. [Section 5.1] The sentence-level evaluation is not precisely defined; please state how sentences are extracted and aligned for the sentence-level metrics.
  6. [Abstract and Section 5.2] The term 'clinical-level accuracy' is vague; please specify the clinical threshold or reference standard used for comparison.
  7. [Section 5.1 and Appendix] The prompt templates and hyperparameters are said to be in the Appendix, but the Appendix is not present in this version; please ensure it is included in the final submission.

Circularity Check

1 steps flagged · score 6.0 of 10

MSRL's headline gains are in-sample: the RL rewards equal the evaluation metrics and no train/test split is stated.

  1. fitted input called prediction [Section 4.1, Eqs. (3)-(4); Section 5.1 Evaluation Metrics; Sections 3 and 5.2 (no train/test split).]
    "Racc = 1(Errpred =Errgt), where Errpred denotes the model's predicted error type, and Errgt refers to the ground truth. ... Rbleu =BLEU (Despred,Desgt) ... Error correction in report level: assessed with two word level metrics: BLEU [26] and ROUGE [25] ... The dataset serves as a comprehensive benchmark for developing and evaluating radiology report error detection and correction systems."

    The RL rewards used to train MSRL are exactly the functions later reported as benchmark scores. Step 1's accuracy reward is a match against the dataset's ground-truth error type, which is the same match counted as detection precision/recall; Step 2 and Step 3's BLEU reward is the same BLEU metric used in Table 2 for correction quality. The paper never states a train/test split: Section 3 creates one pool of 26,326 reports, and Section 5 evaluates MSRL on the same CorBenchX resource used for RL training. As written, the 38.3% precision gain, 30.5% recall gain, and 5.2% BLEU-correction gain are the optimized training objectives measured on the training distribution, not independent predictions.

full rationale

CorBenchX is primarily an empirical dataset-and-benchmark paper, so the VLM rankings, dataset statistics, and zero-shot evaluations are not circular derivations. The only serious circularity is the coupling between MSRL's rewards and the headline evaluation metrics. Eq. (3) defines the detection reward as equality with the ground-truth error type, and Eq. (4) defines the description/correction reward as BLEU against ground truth; the paper then reports detection precision/recall and BLEU correction scores against those same ground-truth labels. Because no held-out split is described anywhere in Sections 3-5, the MSRL improvement claims are, as written, in-sample reward values rather than evidence of generalization. The clinical metrics (CheXbertF1, RadGraphF1) are not part of the reward and provide partial independent evidence, which keeps the circularity from being total. There is no load-bearing self-citation chain or imported uniqueness theorem, so the score is 6 rather than higher.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central claims rest on several domain assumptions about the cleanliness of source reports and the validity of LLM-generated synthetic errors; none are quantitatively validated in the preprint.

assumptions (5)
  • domain assumption MIMIC-CXR sampled reports are clean and error-free.
    Section 3 states clean reports are sampled but provides no verification; if false, ground-truth error counts and labels are corrupted.
  • domain assumption DeepSeek-R1 prompt injection produces clinically plausible errors with accurate labels.
    Section 3 relies on LLM generation plus human review, but no inter-annotator agreement or error-rate statistics are reported.
  • domain assumption The five error categories cover the clinically common error types.
    Table 1 lists the categories, but the paper itself notes the exclusion of other error types such as temporal comparison errors.
  • domain assumption Zero-shot prompting is a fair and comparable setup across models.
    Section 5 uses a single prompt template without ablating prompt sensitivity across models.
  • domain assumption Standard text similarity metrics reflect clinical correction quality.
    Section 5 uses BLEU, ROUGE, BERTScore, SembScore, CheXbertF1, and RadGraphF1, which approximate but do not directly measure clinical correctness.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CorBenchX: Large-Scale Chest X-Ray Error Dataset and Vision-Language Model Benchmark for Report Error Correction." pith.science (2026). https://pith.science/paper/A56Q3SNG

@misc{pith2026250512057,
  author       = {Pith},
  title        = {Pith review of: CorBenchX: Large-Scale Chest X-Ray Error Dataset and Vision-Language Model Benchmark for Report Error Correction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/A56Q3SNG}},
  note         = {Machine review of arXiv:2505.12057}
}
read the original abstract

AI-driven models have shown great promise in detecting errors in radiology reports, yet the field lacks a unified benchmark for rigorous evaluation of error detection and further correction. To address this gap, we introduce CorBenchX, a comprehensive suite for automated error detection and correction in chest X-ray reports, designed to advance AI-assisted quality control in clinical practice. We first synthesize a large-scale dataset of 26,326 chest X-ray error reports by injecting clinically common errors via prompting DeepSeek-R1, with each corrupted report paired with its original text, error type, and human-readable description. Leveraging this dataset, we benchmark both open- and closed-source vision-language models,(e.g., InternVL, Qwen-VL, GPT-4o, o4-mini, and Claude-3.7) for error detection and correction under zero-shot prompting. Among these models, o4-mini achieves the best performance, with 50.6 % detection accuracy and correction scores of BLEU 0.853, ROUGE 0.924, BERTScore 0.981, SembScore 0.865, and CheXbertF1 0.954, remaining below clinical-level accuracy, highlighting the challenge of precise report correction. To advance the state of the art, we propose a multi-step reinforcement learning (MSRL) framework that optimizes a multi-objective reward combining format compliance, error-type accuracy, and BLEU similarity. We apply MSRL to QwenVL2.5-7B, the top open-source model in our benchmark, achieving an improvement of 38.3% in single-error detection precision and 5.2% in single-error correction over the zero-shot baseline.

Figures

Figures reproduced from arXiv: 2505.12057 by the authors.

Figure 1
Figure 1. Overview of CorBenchX. (a): Error report dataset construction pipeline and dataset statistics. (b): Benchmark results across nine vision–language models for error detection and correction. (c): Illustration of our proposed multi-step reinforcement learning (MsRL) method and its performance improvements over the baseline. Given these persistent challenges, there is increasing interest in leveraging Large language Mod… view at source ↗
Figure 2
Figure 2. Example of a chest X-ray, paired original radiology report, and the corresponding error [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Illustration of our multi-step reinforcement-learning framework: the model sequentially [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Precision and recall for single-error detection across various VLMs and models enhanced [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Precision and recall for multi-error detection across various VLMs and models enhanced [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 19 canonical work pages

  1. [1]

    The growing nationwide radiologist shortage: current opportunities and ongoing challenges for international medical graduate radiologists

    Sohrab Afshari Mirak, Sree Harsha Tirumani, Nikhil Ramaiya, and Inas Mohamed. The growing nationwide radiologist shortage: current opportunities and ongoing challenges for international medical graduate radiologists. Radiology, 314(3):e232625, 2025

    work page 2025

  2. [2]

    Large-scale validation of the feasibility of gpt-4 as a proofreading tool for head ct reports

    Songsoo Kim, Donghyun Kim, Hyun Joo Shin, Seung Hyun Lee, Yeseul Kang, Sejin Jeong, Jaewoong Kim, Miran Han, Seong-Joon Lee, Joonho Kim, et al. Large-scale validation of the feasibility of gpt-4 as a proofreading tool for head ct reports. Radiology, 314(1):e240701, 2025

    work page 2025

  3. [3]

    Potential of gpt-4 for detecting errors in radiology reports: implications for reporting accuracy

    Roman Johannes Gertz, Thomas Dratsch, Alexander Christian Bunck, Simon Lennartz, Andra- Iza Iuga, Martin Gunnar Hellmich, Thorsten Persigehl, Lenhard Pennig, Carsten Herbert Gietzen, Philipp Fervers, et al. Potential of gpt-4 for detecting errors in radiology reports: implications for reporting accuracy. Radiology, 311(1):e232714, 2024

    work page 2024

  4. [4]

    Diagnostic error and bias in the department of radiology: a pictorial essay

    Li Zhang, Xin Wen, Jian-Wei Li, Xu Jiang, Xian-Feng Yang, and Meng Li. Diagnostic error and bias in the department of radiology: a pictorial essay. Insights into Imaging, 14(1):163, 2023

    work page 2023

  5. [5]

    Errors in radiology: A standard review

    Filippo Pesapane, Giulia Gnocchi, Cettina Quarrella, Adriana Sorce, Luca Nicosia, Luciano Mariano, Anna Carla Bozzini, Irene Marinucci, Francesca Priolo, Francesca Abbate, et al. Errors in radiology: A standard review. Journal of Clinical Medicine, 13(15):4306, 2024

    work page 2024

  6. [6]

    Work overload and diagnostic errors in radiology

    Ömer Kasalak, Haider Alnahwi, Romy Toxopeus, Jan P Pennings, Derya Yakar, and Thomas C Kwee. Work overload and diagnostic errors in radiology. European Journal of Radiology, 167: 111032, 2023

    work page 2023

  7. [7]

    Chexagent: Towards a foundation model for chest x-ray interpretation

    Zhihong Chen, Maya Varma, Jean-Benoit Delbrouck, Magdalini Paschali, Louis Blankemeier, Dave Van Veen, Jeya Maria Jose Valanarasu, Alaa Youssef, Joseph Paul Cohen, Eduardo Pontes Reis, et al. Chexagent: Towards a foundation model for chest x-ray interpretation. arXiv preprint arXiv:2401.12208, 2024

    arXiv 2024

  8. [8]

    LLM-CXR: instruction-finetuned llm for cxr image understanding and generation

    Suhyeon Lee, Won Jun Kim, Jinho Chang, and Jong Chul Ye. LLM-CXR: instruction-finetuned llm for cxr image understanding and generation. arXiv preprint arXiv:2305.11490, 2023

    arXiv 2023

Show all 40 references
  1. [9]

    Cross-modal causal intervention for medical report generation

    Weixing Chen, Yang Liu, Ce Wang, Jiarui Zhu, Shen Zhao, Guanbin Li, Cheng-Lin Liu, and Liang Lin. Cross-modal causal intervention for medical report generation. arXiv preprint arXiv:2303.09117, 2023

    2023 arXiv

  2. [10]

    Collaboration between clinicians and vision–language models in radiology report generation.Nature Medicine, 31(2):599–608, 2025

    Ryutaro Tanno, David GT Barrett, Andrew Sellergren, Sumedh Ghaisas, Sumanth Dathathri, Abigail See, Johannes Welbl, Charles Lau, Tao Tu, Shekoofeh Azizi, et al. Collaboration between clinicians and vision–language models in radiology report generation.Nature Medicine, 31(2):59...

    2025

  3. [11]

    Dacg: Dual attention and context guidance model for radiology report generation

    Wangyu Lang, Zhi Liu, and Yijia Zhang. Dacg: Dual attention and context guidance model for radiology report generation. Medical Image Analysis, 99:103377, 2025

    2025

  4. [12]

    Enhancing llms for impression generation in radiology reports through a multi-agent system

    Fang Zeng, Zhiliang Lyu, Quanzheng Li, and Xiang Li. Enhancing llms for impression generation in radiology reports through a multi-agent system. arXiv preprint arXiv:2412.06828, 2024

    2024 arXiv

  5. [13]

    Large language models for error detection in radiology reports: a comparative analysis between closed-source and privacy-compliant open-source models

    Babak Salam, Claire Stüwe, Sebastian Nowak, Alois M Sprinkart, Maike Theis, Dmitrij Kravchenko, Narine Mesropyan, Tatjana Dell, Christoph Endler, Claus C Pieper, et al. Large language models for error detection in radiology reports: a comparative analysis between closed-source...

    2025

  6. [14]

    The use of large language models in detecting chinese ultrasound report errors

    Yuqi Yan, Kai Wang, Bojian Feng, Jincao Yao, Tian Jiang, Zhiyan Jin, Yin Zheng, Yahan Zhou, Chen Chen, Lin Sui, et al. The use of large language models in detecting chinese ultrasound report errors. npj Digital Medicine, 8(1):66, 2025

    2025

  7. [15]

    Radiology report expert evaluation (rexval) dataset, 2023

    Feiyang Yu, Mark Endo, Rayan Krishnan, Ian Pan, Andy Tsai, Eduardo Pontes Reis, EKU Fonseca, Henrique Lee, Zahra Shakeri, Andrew Ng, et al. Radiology report expert evaluation (rexval) dataset, 2023. 10

    2023

  8. [16]

    Radiology report generation models evaluation dataset for chest x-rays (RadEvalX)

    Amos Rubin Calamida, Farhad Nooralahzadeh, Morteza Rohanian, Mizuho Nishio, Koji Fujimoto, and Michael Krauthammer. Radiology report generation models evaluation dataset for chest x-rays (RadEvalX)

  9. [17]

    RRED: a radiology report error detector based on deep learning framework

    Dabin Min, Kaeun Kim, Jong Hyuk Lee, Yisak Kim, and Chang Min Park. RRED: a radiology report error detector based on deep learning framework. In Proceedings of the 4th Clinical Natural Language Processing Workshop, pages 41–52, 2022

    2022

  10. [18]

    ReXErr: Synthesizing clinically meaningful errors in diagnostic radiology reports

    Vishwanatha M Rao, Serena Zhang, Julian N Acosta, Subathra Adithan, and Pranav Rajpurkar. ReXErr: Synthesizing clinically meaningful errors in diagnostic radiology reports. In Biocom- puting 2025: Proceedings of the Pacific Symposium, pages 70–81. World Scientific, 2024

    2025

  11. [19]

    Mimic-cxr, a de-identified publicly available database of chest radiographs with free-text reports

    Alistair EW Johnson, Tom J Pollard, Seth J Berkowitz, Nathaniel R Greenbaum, Matthew P Lungren, Chih-ying Deng, Roger G Mark, and Steven Horng. Mimic-cxr, a de-identified publicly available database of chest radiographs with free-text reports. Scientific data, 6(1):317, 2019

    2019

  12. [20]

    Chest x- ray report generation through fine-grained label learning

    Tanveer Syeda-Mahmood, Ken CL Wong, Yaniv Gur, Joy T Wu, Ashutosh Jadhav, Satyananda Kashyap, Alexandros Karargyris, Anup Pillai, Arjun Sharma, Ali Bin Syed, et al. Chest x- ray report generation through fine-grained label learning. In Medical Image Computing and Computer Assi...

    2020

  13. [21]

    Automated radiographic re- port generation purely on transformer: A multicriteria supervised approach

    Zhanyu Wang, Hongwei Han, Lei Wang, Xiu Li, and Luping Zhou. Automated radiographic re- port generation purely on transformer: A multicriteria supervised approach. IEEE Transactions on Medical Imaging, 41(10):2803–2813, 2022

    2022

  14. [22]

    Writing by memorizing: Hierar- chical retrieval-based medical report generation

    Xingyi Yang, Muchao Ye, Quanzeng You, and Fenglong Ma. Writing by memorizing: Hierar- chical retrieval-based medical report generation. arXiv preprint arXiv:2106.06471, 2021

    2021 arXiv

  15. [23]

    Retrieval- based chest x-ray report generation using a pre-trained contrastive language-image model

    Mark Endo, Rayan Krishnan, Viswesh Krishna, Andrew Y Ng, and Pranav Rajpurkar. Retrieval- based chest x-ray report generation using a pre-trained contrastive language-image model. In Machine Learning for Health, pages 209–219. PMLR, 2021

    2021

  16. [24]

    ReXrank: A public leaderboard for ai-powered radiology report generation

    Xiaoman Zhang, Hong-Yu Zhou, Xiaoli Yang, Oishi Banerjee, Julián N Acosta, Josh Miller, Ouwen Huang, and Pranav Rajpurkar. ReXrank: A public leaderboard for ai-powered radiology report generation. arXiv preprint arXiv:2411.15122, 2024

    2024 arXiv

  17. [25]

    Rouge: A package for automatic evaluation of summaries

    Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. InText summarization branches out, pages 74–81, 2004

    2004

  18. [26]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318, 2002

    2002

  19. [27]

    Chexbert: combining automatic labelers and expert annotations for accurate radiology report labeling using bert

    Akshay Smit, Saahil Jain, Pranav Rajpurkar, Anuj Pareek, Andrew Y Ng, and Matthew P Lungren. Chexbert: combining automatic labelers and expert annotations for accurate radiology report labeling using bert. arXiv preprint arXiv:2004.09167, 2020

    2004 arXiv

  20. [28]

    Evaluating progress in automatic chest x-ray radiology report generation

    Feiyang Yu, Mark Endo, Rayan Krishnan, Ian Pan, Andy Tsai, Eduardo Pontes Reis, Eduardo Kaiser Ururahy Nunes Fonseca, Henrique Min Ho Lee, Zahra Shakeri Hossein Abad, Andrew Y Ng, et al. Evaluating progress in automatic chest x-ray radiology report generation. Patterns, 4 (9), 2023

    2023

  21. [29]

    Green: Generative radiology report evaluation and error notation

    Sophie Ostmeier, Justin Xu, Zhihong Chen, Maya Varma, Louis Blankemeier, Christian Bluethgen, Arne Edward Michalson, Michael Moseley, Curtis Langlotz, Akshay S Chaud- hari, et al. Green: Generative radiology report evaluation and error notation. arXiv preprint arXiv:2405.03595, 2024

    2024 arXiv

  22. [30]

    Generative large language models trained for detecting errors in radiology reports

    Cong Sun, Kurt Teichman, Yiliang Zhou, Brian Critelli, David Nauheim, Graham Keir, Xindi Wang, Judy Zhong, Adam E Flanders, George Shih, et al. Generative large language models trained for detecting errors in radiology reports. arXiv preprint arXiv:2504.04336, 2025. 11

    2025 arXiv

  23. [31]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  24. [32]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  25. [33]

    Medvlm-r1: Incentivizing medical reasoning capability of vision-language models (vlms) via reinforcement learning

    Jiazhen Pan, Che Liu, Junde Wu, Fenglin Liu, Jiayuan Zhu, Hongwei Bran Li, Chen Chen, Cheng Ouyang, and Daniel Rueckert. Medvlm-r1: Incentivizing medical reasoning capability of vision-language models (vlms) via reinforcement learning. arXiv preprint arXiv:2502.19634, 2025

    2025 arXiv

  26. [34]

    Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Yuchen Duan, Hao Tian, Weijie Su, Jie Shao, et al. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479, 2025

    2025 arXiv

  27. [35]

    Baichuan-omni-1.5 technical report

    Yadong Li, Jun Liu, Tao Zhang, Song Chen, Tianpeng Li, Zehuan Li, Lijun Liu, Lingfeng Ming, Guosheng Dong, Da Pan, et al. Baichuan-omni-1.5 technical report. arXiv preprint arXiv:2501.15368, 2025

    2025

  28. [36]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025

    2025 arXiv

  29. [37]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  30. [38]

    Introducing openai o3 and o4-mini

    OpenAI. Introducing openai o3 and o4-mini. https://openai.com/index/ introducing-o3-and-o4-mini/ , April 2025

    2025

  31. [39]

    Bertscore: Evaluating text generation with bert

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675, 2019

    1904 arXiv

  32. [40]

    X-reasoner: Towards generalizable reasoning across modalities and domains

    Qianchu Liu, Sheng Zhang, Guanghui Qin, Timothy Ossowski, Yu Gu, Ying Jin, Sid Kiblawi, Sam Preston, Mu Wei, Paul V ozila, et al. X-reasoner: Towards generalizable reasoning across modalities and domains. arXiv preprint arXiv:2505.03981, 2025. 12

    2025 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.