Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

Adapting Biomedical Abstracts into Plain language using Large Language Models

T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single GPT-4 prompt with distilled guidelines ranked first for simplicity in a shared task adapting medical abstracts to plain language.

desk verdict A useful shared-task system report: a distilled one-shot GPT-4 prompt wins PLABA simplicity, but the win is not shown to be statistically robust. read the letter →

arxiv 2501.15700 v1 pith:OMVSKTUU submitted 2025-01-26 cs.CL

Haritha Gangavarapu , Giridhar Kaushik Ramachandran , Kevin Lybarger , Meliha Yetisgen , Özlem Uzuner This is my paper · ORCID
classification cs.CL
keywords plainlanguageadaptationbiomedicalabstractslargemodelsin-contextlearningGPT-4textsimplificationhealthliteracyhumanevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Health information in biomedical abstracts is too technical for many readers, and this paper asks whether large language models can bridge that gap automatically. The authors built several systems for a shared plain-language adaptation task and found that their best system was not a heavily fine-tuned one but a GPT-4 model prompted with a distilled version of the adaptation guidelines plus one worked example. In the external human evaluation, this one-shot prompt tied for the top average simplicity score and placed third on average accuracy among seven participating teams. The result matters because it suggests that carefully encoding expert guidelines into a prompt can achieve strong simplification without expensive fine-tuning.

What carries the argument

The load-bearing mechanism is the one-shot in-context learning prompt given to GPT-4: it contains a distilled version of the official annotation guidelines for plain-language adaptation plus one example sentence pair chosen to illustrate most of those rules. The guidelines tell the model to split long sentences, expand abbreviations, explain expert terms in parentheses or clauses at first mention, omit statistical details such as p-values, and drop or retain sentences as needed. The single example demonstrates how those rules apply, letting the model infer the transformation style without weight updates. The paper contrasts this with fine-tuned open models (T5, LLaMa-2, GPT-3.5), which scored better on automatic metrics but often repeated abstract sentences with minimal simplification.

What would settle it

Re-score the top system alongside its closest competitors with a new set of expert annotators, or bootstrap confidence intervals from the per-sentence evaluation scores; if the simplicity ranking flips or the intervals overlap substantially, the claim of ranking first is not supported.

Watch

Extended reading notes

Core claim

The paper's central claim is that an 'annotation-guideline-instructed GPT-4-based one-shot in-context learning model' produces plain-language adaptations that human evaluators judged simpler than those of the other six teams, while remaining reasonably faithful and complete. In the task's external evaluation, this system scored 91.28 on the average simplicity axis, matching the top score among all rank-one submissions and exceeding the 88.86 median; on the average accuracy axis it scored 92.52, ranking third. The authors attribute the result to the prompt, which carried a condensed version of the annotation guidelines and a single training example, rather than to any fine-tuning of the model.

Load-bearing premise

The ranking rests on the assumption that the official external human evaluation is a valid, unbiased measure of adaptation quality, and that small differences like 91.28 versus 88.86 on the simplicity average are real rather than annotator noise.

Editorial extensions

If this is right

  • A guideline-distilled one-shot prompt can outperform fine-tuned open models on human-judged simplicity for plain-language adaptation.
  • Automatic metrics such as BLEU, ROUGE, and SARI may reward conservative repetition, so system ranking should include human evaluation.
  • The same prompting recipe could be applied to other consumer health texts, such as clinical trial summaries or discharge instructions.
  • Because the model needs no task-specific fine-tuning, the approach is cheap to deploy and update as new biomedical abstracts appear.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported margins are small: a 2.42-point simplicity gap over the median and ties at the top, with no inter-annotator agreement or confidence intervals reported, so the ranking may not replicate under a different panel of annotators.
  • The prompt is tuned to a proprietary model whose behavior can change across versions; testing whether open-weight models with the same guidelines-plus-example prompt reproduce the result would say whether the method generalizes.
  • The 'distilled guidelines plus one example' pattern is a general template for any rubric-driven rewriting task; one could test it on other domains such as legal, technical, or financial text.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper describes the authors' submissions to the TAC-2023 PLABA shared task on adapting biomedical abstracts into plain language. The authors fine-tune T5-Large, LLaMa2 (13B and 70B chat), and GPT-3.5, and also use in-context learning with GPT-4, comparing systems on automatic metrics and a small internal human evaluation. They report that their GPT-4-based one-shot in-context learning system, prompted with a distilled version of the PLABA annotation guideline and one training example, ranked first in the official external human evaluation on the average simplicity axis and third on the average accuracy axis.

Significance. If the reported ranking is robust, the paper provides a practically relevant finding: a simple one-shot GPT-4 prompt with annotation guidelines can outperform more expensive fine-tuning approaches on plain-language adaptation of biomedical text. The use of the official external human evaluation is a strength, as is the observation that high BLEU/ROUGE scores can reflect superficial repetition rather than genuine simplification. However, the paper does not provide uncertainty quantification for the external evaluation, and the exact prompt and example selection are not specified, which limits the strength of the headline conclusions. The work is a solid shared-task system description but needs additional evidence to support its central claims at the journal level.

major comments (3)
  1. [§4.2, Table 4] The claim of ranking first in the average simplicity measure rests on an external human evaluation of 430 sentences, but the paper reports no inter-annotator agreement, confidence intervals, or per-item score distributions. The reported average of 91.28 is only 2.42 points above the median of 88.86 on a scale derived from ternary (-1/0/1) ratings; roughly 20 sentence-level one-step rating changes could account for this gap, so the first-place ranking is not established as robust. Please add uncertainty quantification (e.g., bootstrap confidence intervals if per-sentence scores are available) or explicitly state that the observed difference may be within annotation noise.
  2. [§3.4.1] The internal human evaluation used to select the GPT-4 submission is the only evidence that GPT-4 outperformed the fine-tuned systems in human judgment, but it is based on approximately 100 sentences per model, evaluated by the lead authors with no inter-annotator agreement, no detailed rating criteria, and only a preference ranking. This is a small and potentially biased sample; the paper should either provide a more rigorous evaluation protocol or temper the claim that GPT-4 was the best-performing of the authors' own systems.
  3. [§3.2, Figure 2] The top-performing GPT-4 system uses a 'distilled version of the PLABA annotation guideline' with one training example, but the full prompt and the method for selecting that example are not provided. Since the paper's central conclusion attributes the result to this specific annotation-guideline-instructed one-shot prompt, the omission prevents replication and independent verification. Please include the exact prompt and describe the example-selection procedure.
minor comments (6)
  1. [Abstract] The abstract says the authors 'leveraged the best open-source Large Language Models,' but GPT-4 is proprietary; this should be corrected to avoid a factual error.
  2. [§4.1, Table 3] The column headers in Table 3 are garbled, so the reader cannot tell which system corresponds to each score or which score is the rank-1 submission in the external SARI comparison; please use a clearly labeled table.
  3. [§3.4.2] The transformation from the -1/0/1 rating scale to the 1-100 scale is not specified; please clarify the mapping.
  4. [§4.1] The observation that T5 and LLaMa2 'tend to frequently repeat the abstract sentence as is' is qualitative; including example outputs or a quantitative measure (e.g., percentage of unchanged sentences) would strengthen the point.
  5. [§5] Calling the systems 'state-of-the-art' is an overclaim given the third-place accuracy ranking; consider using 'competitive' or 'well-performing' instead.
  6. [Throughout] There are several typos and formatting issues (e.g., 'Simplicty axis' in Table 4, the garbled author name 'Y etisgen', and 'V olume' in the reference list); these should be fixed in a revision.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the ranking claim rests on an external official evaluation of systems selected by internal human judgments made before the external scores were known.

full rationale

No circularity identified. The paper's central claim is that its GPT-4-based one-shot in-context learning system ranked first on the average simplicity axis and third on the average accuracy axis in the PLABA external manual evaluation. The load-bearing step is model selection: the authors state that 'we chose the results from our GPT-4 model as rank 1 based on our internal human evaluation' and then report the external organizer-provided scores in Table 4. The internal test set is used only to select which submitted run to designate as top-ranked; the external test set and external human annotations are independent of that selection. There is no fitted parameter later renamed as a prediction, and no quantity is defined in terms of the outcome it is supposed to predict. The prompt includes a distilled version of the PLABA annotation guidelines and one training example, but those are external task resources, not the evaluation scores. The paper contains no derivation chain whose equations reduce to their own inputs, and no load-bearing self-citation: the cited dataset and guideline documents come from the track organizers and prior PLABA work, not from the present authors' own unpublished results. The absence of inter-annotator agreement statistics, confidence intervals, and per-item scores is a legitimate robustness concern about whether the 2.42-point simplicity-axis lead is meaningful, but that is an uncertainty/validity limitation, not a circularity. Accordingly, the circularity score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

No mathematical derivation is attempted; the paper is an empirical system comparison. The claims rest on the PLABA dataset and its official evaluation, on the assumption that the one-shot guideline prompt is representative, and on the assumption that the external test set was not used during development. No new scientific entities are introduced.

free parameters (3)
  • LoRA rank = 16
    Chosen for 4-bit parameter-efficient fine-tuning of LLaMa-2 chat models; affects LLaMa-2 system performance in the comparison but not the top GPT-4 system.
  • Number of in-context examples for GPT-4 = 1
    The final GPT-4 prompt uses one training example with distilled PLABA guidelines; the choice of one example is an experimental design decision that the top system's performance depends on.
  • Internal human evaluation sample = 5 consumer questions, 2 abstracts each, about 100 sentences per model
    The lead authors chose this small sample to rank systems internally; the same sample was used to pick the GPT-4 submission for external evaluation.
assumptions (3)
  • domain assumption The PLABA dataset's gold-standard adaptations and the organizers' external evaluation are reliable measures of plain language quality.
    The central ranking claims depend on the correctness and comparability of organizer-provided manual evaluation scores, described in Section 3.4.2 and Table 4.
  • domain assumption A single training example, together with a distilled version of the PLABA annotation guidelines, sufficiently illustrates the adaptation task for GPT-4 one-shot in-context learning.
    This is the core prompt design assumption behind the top system, stated in Section 3.2 and summarized in Figure 2.
  • domain assumption The GPT-4 system was not optimized against the external test set.
    Interpretation of the external rank as an unbiased result requires that external scores were only used for final evaluation, not for prompt tuning; the paper states it predicted on the external set after internal selection in Section 3.3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Adapting Biomedical Abstracts into Plain language using Large Language Models." pith.science (2026). https://pith.science/paper/OMVSKTUU

@misc{pith2026250115700,
  author       = {Pith},
  title        = {Pith review of: Adapting Biomedical Abstracts into Plain language using Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OMVSKTUU}},
  note         = {Machine review of arXiv:2501.15700}
}
read the original abstract

A vast amount of medical knowledge is available for public use through online health forums, and question-answering platforms on social media. The majority of the population in the United States doesn't have the right amount of health literacy to make the best use of that information. Health literacy means the ability to obtain and comprehend the basic health information to make appropriate health decisions. To build the bridge between this gap, organizations advocate adapting this medical knowledge into plain language. Building robust systems to automate the adaptations helps both medical and non-medical professionals best leverage the available information online. The goal of the Plain Language Adaptation of Biomedical Abstracts (PLABA) track is to adapt the biomedical abstracts in English language extracted from PubMed based on the questions asked in MedlinePlus for the general public using plain language at the sentence level. As part of this track, we leveraged the best open-source Large Language Models suitable and fine-tuned for dialog use cases. We compare and present the results for all of our systems and our ranking among the other participants' submissions. Our top performing GPT-4 based model ranked first in the avg. simplicity measure and 3rd on the avg. accuracy measure.

Figures

Figures reproduced from arXiv: 2501.15700 by the authors.

Figure 1
Figure 1. elements of the Prompt structure for Fine-tuning LLaMa-2 models In-context learning(ICL) using GPT-J We adapt GPT-3.531, 32 and GPT-433 for PLABA using in-context learning and fine-tuning. We experimented with a variety of in-context learning strategies for GPT-4, ranging from short prompt instructions to detailed guidelines. Our final GPT-4-based model included a detailed distilled version of the PLABA annotation g… view at source ↗
Figure 2
Figure 2. Prompt structure for the ICL using GPT-4 and Fine-tuning GPT-3.5 3.3 Experimental setup We used the PLABA dataset containing 750 abstracts and 921 adaptations for training and fine-tuning our models. 171 abstracts are adapted more than once, in order to create abstract-adaptation sentence pairs for training, we repeat the sentences in the abstract as many times as the number of corresponding adaptations creating a t… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Lessons from the TREC Plain Language Adaptation of Biomedical Abstracts (PLABA) track

    cs.CL 2025-07 conditional novelty 6.0 of 10

    Across two TREC shared-task years, top LLM systems matched human writers on factual accuracy and completeness but not on simplicity or brevity, while common automatic metrics correlated poorly with manual judgments.

Reference graph

Works this paper leans on

39 extracted references · 34 canonical work pages · cited by 1 Pith paper

  1. [1]

    Plain Language Adaptation of Biomedical Abstracts - A track of TAc 2023; 2023

    track organizers P. Plain Language Adaptation of Biomedical Abstracts - A track of TAc 2023; 2023. https://bionlp.nlm.nih.gov/plaba2023/. Available from: https://bionlp.nlm.nih. gov/plaba2023/

    work page 2023

  2. [2]

    A Dataset for Plain Language Adaptation of Biomedical Abstracts

    Attal K, Ondov B, Demner-Fushman D. A Dataset for Plain Language Adaptation of Biomedical Abstracts. Scientific Data. 2022

    work page 2022

  3. [3]

    Low Health Literacy and Health Outcomes: An Updated Systematic Review

    Berkman ND, Sheridan SL, Donahue KE, Halpern DJ, Crotty K. Low Health Literacy and Health Outcomes: An Updated Systematic Review. Annals of Internal Medicine. 2011;155(2):97-

    work page 2011

  4. [4]

    Determinants of Poor Treatment Adherence among Patients with Type 2 Diabetes and Limited Health Literacy : A Scoping Review

    Pourhabibi N, Mohebbi B, Sadeghi R, Shakibazadeh E, Sanjari M, Tol A, et al. Determinants of Poor Treatment Adherence among Patients with Type 2 Diabetes and Limited Health Literacy : A Scoping Review. Journal of diabetes research. 2022

    work page 2022

  5. [5]

    Available from: https://www.cdc.gov/ healthliteracy/developmaterials/plainlanguage.html

    CDC Plain Language Materials & Resources; 2023. Available from: https://www.cdc.gov/ healthliteracy/developmaterials/plainlanguage.html

    work page 2023

  6. [6]

    Available from: https://www.nih.gov/ institutes-nih/nih-office-director/office-communications-public-liaison/ clear-communication/plain-language/

    Plain Language at NIH; 2023. Available from: https://www.nih.gov/ institutes-nih/nih-office-director/office-communications-public-liaison/ clear-communication/plain-language/

    work page 2023

  7. [7]

    Evaluating breast cancer websites targeting Arabic speakers: empirical inves- tigation of popularity, availability, accessibility, readability, and quality

    Jasem Z, AlMeraj Z, Alhuwail D. Evaluating breast cancer websites targeting Arabic speakers: empirical inves- tigation of popularity, availability, accessibility, readability, and quality. BMC medical informatics and decision making. 2022:126

    work page 2022

  8. [8]

    Readability of Trauma-Related Patient Education Materials From the American Academy of Orthopaedic Surgeons

    Eltorai AE, P Thomas N, Yang H, Daniels AH, Born CT. Readability of Trauma-Related Patient Education Materials From the American Academy of Orthopaedic Surgeons. Trauma monthly. 2016

    work page 2016

Show all 39 references
  1. [9]

    Readability of Patient-oriented Online Dermatology Resources

    Tulbert B, Snyder C, Brodell R. Readability of Patient-oriented Online Dermatology Resources. J Clin Aesthet Dermatol. 2011:27-33. Available from: https://pubmed.ncbi.nlm.nih.gov/21464884/

    2011

  2. [10]

    Readability Assessment of Online Patient Education Material on Congestive Heart Failure

    Kher A, Johnson S, Griffith R. Readability Assessment of Online Patient Education Material on Congestive Heart Failure. Advances in preventive medicine. 2017

    2017

  3. [11]

    Readability of patient education materials available at the point of care

    Stossel L, Segar N, Gliatto P, Fallar R, Karani R. Readability of patient education materials available at the point of care. Journal of General Internal Medicine. 2012

    2012

  4. [12]

    The Role of Health Literacy in Diabetes Knowledge, Self-Care, and Glycemic Control: a Meta-analysis

    Marciano L, Camerini AL, Schulz PJ. The Role of Health Literacy in Diabetes Knowledge, Self-Care, and Glycemic Control: a Meta-analysis. Journal of general internal medicine. 2019:1007–1017

    2019

  5. [13]

    Misperceptions of medical understanding in low-literacy patients: implications for cancer prevention; 2006

    Rogers ES, Wallace LS, Weiss BD. Misperceptions of medical understanding in low-literacy patients: implications for cancer prevention; 2006

    2006

  6. [14]

    Where are biomedical research plain-language summaries? Health science reports

    FitzGibbon H, King K, Piano C, Wilk C, Gaskarth M. Where are biomedical research plain-language summaries? Health science reports. 2020

    2020

  7. [15]

    Plain Language Summaries of Clinical Trial Results: A Preliminary Study to Assess Availability of Easy-to-Understand Summaries and Approaches to Improving Public Engagement

    Penlington M, Silverman H, Vasudevan A, Pavithran P. Plain Language Summaries of Clinical Trial Results: A Preliminary Study to Assess Availability of Easy-to-Understand Summaries and Approaches to Improving Public Engagement. Pharmaceutical medicine. 2020:401–406

    2020

  8. [16]

    Simple and Effective Text Simplification Using Semantic and Neural Methods

    Sulem E, Abend O, Rappoport A. Simple and Effective Text Simplification Using Semantic and Neural Methods. In: Gurevych I, Miyao Y , editors. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Papers). Melbourne, Australia:...

    2018

  9. [17]

    Towards Effective Sentence Simplification for Au- tomatic Processing of Biomedical Text

    Jonnalagadda S, Tari L, Hakenberg J, Baral C, Gonzalez G. Towards Effective Sentence Simplification for Au- tomatic Processing of Biomedical Text. In: Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Co...

    2009

  10. [18]

    Paragraph-level Simplification of Medical Texts

    Devaraj A, Marshall IJ, Wallace BC, Li JJ. Paragraph-level Simplification of Medical Texts. CoRR. 2021;abs/2104.05767. Available from: https://arxiv.org/abs/2104.05767

    2021 arXiv

  11. [19]

    CompLex — A New Corpus for Lexical Complexity Prediction from Likert Scale Data

    Shardlow M, Cooper M, Zampieri M. CompLex — A New Corpus for Lexical Complexity Prediction from Likert Scale Data. In: Proceedings of the 1st Workshop on Tools and Resources to Empower People with REAding DIfficulties (READI). Marseille, France: European Language Resources Ass...

    2020

  12. [20]

    Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer; 2023

    Raffel C, Shazeer N, Roberts A, et al.. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer; 2023

    2023

  13. [21]

    PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization

    Zhang J, Zhao Y , Saleh M, Liu PJ. PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization. ArXiv. 2019;abs/1912.08777. Available from: https://api.semanticscholar.org/ CorpusID:209405420

    2019 arXiv

  14. [22]

    BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

    Lewis M, Liu Y , Goyal N, et al. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Online: Association for Computationa...

    2020

  15. [23]

    Multitask Prompted Training Enables Zero-Shot Task Generalization

    Sanh V , Webson A, Raffel C, et al. Multitask Prompted Training Enables Zero-Shot Task Generalization. In: International Conference on Learning Representations; 2022. Available from: https://openreview.net/ forum?id=9Vrb9D0WI4

    2022

  16. [24]

    Finetuned Language Models Are Zero-Shot Learn- ers

    Wei J, Bosma M, Zhao V , Guu K, Yu AW, Lester B, et al. Finetuned Language Models Are Zero-Shot Learn- ers. ArXiv. 2021;abs/2109.01652. Available from: https://api.semanticscholar.org/CorpusID: 237416585

    2021 arXiv

  17. [25]

    Calibrate Before Use: Improving Few-shot Performance of Language Models

    Zhao Z, Wallace E, Feng S, Klein D, Singh S. Calibrate Before Use: Improving Few-shot Performance of Language Models. In: Meila M, Zhang T, editors. Proceedings of the 38th International Conference on Machine Learning. vol. 139 of Proceedings of Machine Learning Research. PMLR...

    2021

  18. [26]

    Embracing Large Language Models for Medical Applications: Opportunities and Challenges.; 2023

    M K, K M. Embracing Large Language Models for Medical Applications: Opportunities and Challenges.; 2023

    2023

  19. [27]

    Large Language Models Encode Clinical Knowledge; 2022

    Singhal K, Azizi S, Tu T, et al.. Large Language Models Encode Clinical Knowledge; 2022

    2022

  20. [28]

    QLoRA: Efficient Finetuning of Quantized LLMs; 2023

    Dettmers T, Pagnoni A, Holtzman A, Zettlemoyer L. QLoRA: Efficient Finetuning of Quantized LLMs; 2023

    2023

  21. [29]

    Llama 2: Open Foundation and Fine-Tuned Chat Models; 2023

    Touvron H, Martin L, Stone K, et al.. Llama 2: Open Foundation and Fine-Tuned Chat Models; 2023

    2023

  22. [30]

    Instruction Tuned Models are Quick Learners; 2023

    Gupta H, Sawant SA, Mishra S, et al.. Instruction Tuned Models are Quick Learners; 2023. Available from: https://arxiv.org/abs/2306.05539

    2023 arXiv

  23. [31]

    A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Mod- els

    Ye J, Chen X, Xu N, et al. A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Mod- els. ArXiv. 2023;abs/2303.10420. Available from: https://api.semanticscholar.org/CorpusID: 257632113

    2023 arXiv

  24. [32]

    Language Models are Few-Shot Learners

    Brown TB, Mann B, Ryder N, et al. Language Models are Few-Shot Learners. In: Larochelle H, Ranzato M, Hadsell R, Balcan MF, Lin H, editors. Advances in Neural Information Processing Systems. vol. 33. Curran Asso- ciates, Inc.; 2020. p. 1877-901. Available from: https://proceed...

    2020

  25. [33]

    GPT-4 Technical Report; 2024

    OpenAI, Achiam J, Adler S, et al.. GPT-4 Technical Report; 2024. Available from: https://arxiv.org/ abs/2303.08774

    2024 arXiv

  26. [34]

    PLABA data annotation guidelines; 2023

    track organizers P. PLABA data annotation guidelines; 2023. Available from: https://bionlp.nlm.nih. gov/plaba2023/annotation_guidelines.pdf

    2023

  27. [35]

    ROUGE: A Package for Automatic Evaluation of Summaries

    Lin CY . ROUGE: A Package for Automatic Evaluation of Summaries. In: Text Summarization Branches Out. Barcelona, Spain: Association for Computational Linguistics; 2004. p. 74-81. Available from: https: //aclanthology.org/W04-1013

    2004

  28. [36]

    Bleu: a Method for Automatic Evaluation of Machine Translation

    Papineni K, Roukos S, Ward T, Zhu WJ. Bleu: a Method for Automatic Evaluation of Machine Translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. Philadelphia, Pennsylvania, USA: Association for Computational Linguistics; 2002. ...

    2002

  29. [37]

    Document-Level Text Simplification: Dataset, Criteria and Baseline

    Sun R, Jin H, Wan X. Document-Level Text Simplification: Dataset, Criteria and Baseline. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Online and Punta Cana, Do- minican Republic: Association for Computational Linguistics; 2021. p....

    2021

  30. [38]

    PLABA evaluation notes; 2023

    track organizers P. PLABA evaluation notes; 2023. Available from: https://bionlp.nlm.nih.gov/ plaba2023/#evaluation

    2023

  31. [107]

    Available from: https://www.acpjournals.org/doi/abs/10.7326/ 0003-4819-155-2-201107190-00005

    PMID: 21768583. Available from: https://www.acpjournals.org/doi/abs/10.7326/ 0003-4819-155-2-201107190-00005

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.