The use of large language models to enhance cancer clinical trial educational materials

Aman Varshney; Brian Anthony; Claire Novack; Danielle S. Bitterman; David Kozono; Dmitriy Dligach; Elad Sharon; Erik Duhaime; Howard Isenstein; Jack Gallifant

REVIEW 3 major objections 4 minor 1 references

The use of large language models to enhance cancer clinical trial educational materials

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that GPT-4 can turn cancer trial consent forms into readable patient summaries and multiple-choice questions with high crowd agreement, but that hallucinations require human review before use.

desk verdict A useful, honest proof-of-concept whose headline '97.91% accuracy' is really human–model agreement, not correctness; worth reviewing but the metric needs reframing and external validation. read the letter →

arxiv 2412.01955 v2 pith:X6WJ5UVR submitted 2024-12-02 cs.CL cs.AI

Mingye Gao , Aman Varshney , Shan Chen , Vikram Goddla , Jack Gallifant , Patrick Doyle , Claire Novack , Maeve Dillon-Martin

show 10 more authors

Teresia Perkins Xinrong Correia Erik Duhaime Howard Isenstein Elad Sharon Lisa Soleymani Lehmann David Kozono Brian Anthony Dmitriy Dligach Danielle S. Bitterman

This is my paper · ORCID

classification cs.CLcs.AI

keywords largelanguagemodelsclinicaltrialsinformedconsentpatienteducationtextsummarizationmultiple-choicequestionscrowdsourcedevaluationhallucination

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tests whether a large language model can do the work of translating informed consent forms into educational materials without fine-tuning. It claims GPT-4 can generate plain-language summaries that clinicians rate as readable and comprehensive, and that patient surveys suggest improve understanding (85% of 13 participants). It also claims GPT-4 can generate multiple-choice question-answer pairs from consent forms, with 97.91% of 1,335 pairs receiving a majority of crowdsourced readers agreeing with GPT-4's answer. The authors stress that hallucinations do occur, most often when the consent form lacks the requested content, so human oversight is still required. If true, this means trial education materials can be drafted at scale with minimal engineering, as long as a person reviews them.

What carries the argument

The method is two prompt designs. For summaries, a sequential approach first extracts the sections of the consent form matching the basic elements of informed consent (purpose, duration, procedures, risks, benefits, alternatives, confidentiality, and related items), then summarizes only that extracted text; this is compared with direct summarization of the full consent form. For questions, one-shot in-context learning feeds an oncologist-written exemplar question-answer pair for each consent element to GPT-4 along with the target consent form, generating multiple-choice question-answer pairs. Evaluation of the questions uses crowdsourced readers who answer the stems from the consent form, with accuracy defined as the percentage of questions where the majority reader answer matches GPT-4's assigned answer.

What would settle it

Have a clinician verify the ground-truth answer for every one of the 1,335 generated questions against the source consent forms, then recompute accuracy from clinician-verified labels; if the verified accuracy on the majority-agree questions is materially lower than 97.91%, the headline accuracy claim fails. A cheaper check is to take the 78 hardest questions and count how many of the 24 labeled as human error are actually cases where readers chose a different but textually defensible answer, which would show the metric is undercounting genuine ambiguity.

Watch

Extended reading notes

Core claim

The central discovery is that out-of-the-box GPT-4, with simple prompts and no fine-tuning, can repurpose existing informed consent forms into two patient-facing resources: short summaries organized around the standard elements of informed consent, and multiple-choice question-answer pairs testing those elements. On clinician evaluation of 11 consent forms, both direct and sequential summarization approaches produced acceptable-quality summaries, with fewer inaccuracies and hallucinations in the sequential version. Crowdsourced evaluation of 1,335 generated question-answer pairs found majority agreement with GPT-4's assigned answers in 97.91% of cases, with average reader agreement of 86.87%. Manual review of the 78 hardest questions found errors split across human error, missing information in the consent form, GPT-4 errors, ambiguous definitions, and non-English forms; the authors acknowledge inaccuracies and hallucinations in both resource types, most often when the source consent form did not contain the requested topic.

Load-bearing premise

The high accuracy figure counts a question as correct when most crowdsourced readers chose the same answer GPT-4 assigned, so the claim rests on the assumption that reader agreement with the model equals factual correctness.

Editorial extensions

If this is right

With minimal prompt engineering, GPT-4 can produce consent-form-derived summaries and quizzes that patients find readable and useful, lowering the barrier to generating trial-specific education materials.
Accurate multiple-choice questions can be generated at scale (1,335 from 91 consent forms), enabling trial-specific self-assessment of understanding.
Errors and hallucinations occur mainly when the consent form does not actually contain the prompted topic, so human review by an oncologist remains necessary for any patient-facing use.
A multi-LLM cross-check showed other models tended to disagree on the same problematic questions, suggesting automated pre-screening of generated materials is feasible.
The methods appear to generalize across trial types (observational, Phase I, I/II, and III), suggesting applicability beyond cancer trials.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the accuracy metric is taken at face value, the 97.91% figure likely overstates performance on genuinely hard questions, since the paper's own manual review of the 78 hardest questions found 17 GPT-4 errors; a clinician-verified ground-truth evaluation would give a more realistic accuracy estimate.
The same extract-then-summarize recipe could be applied to other dense regulatory documents, such as protocol amendments or device instructions, where hallucinations arise from missing source content.
A testable extension would randomize patients to receive the GPT-4 summary before or after consent discussions and measure comprehension with a validated consent-quality questionnaire rather than self-report.
Because errors concentrated in contact information, expected duration, and alternatives, prompt designers could add targeted verification steps for those elements.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper is a careful, straightforward test of whether GPT-4 can turn informed consent forms (ICFs) into plain-language trial summaries and multiple-choice questions (MCQAs). It does what a good proof-of-concept should: clear methods, explicit definitions, and a genuine attempt to surface failure modes. The sequential summarization prompt (extract key elements first, then summarize) is a practical idea, and the decision to have an oncologist manually review summaries before patient evaluation matches how the tool would actually be used. The authors also deserve credit for reporting hallucinations and for breaking down errors into human, model, missing-information, and ambiguity categories. The soft spot is real, and the stress-test note gets it right. The 97.91% figure in the abstract is not accuracy in any ordinary sense. Table 2 defines it as the percentage of MCQAs where the majority of crowdsourced readers matched the GPT-4 assigned answer. That is agreement with the generator, not correctness against the ICF or any external gold standard. The paper's own quality-assurance set (78 hard questions) found 17 cases where the GPT-4 question or answer was wrong, so the metric can clearly mask defects. The manual analysis is a good faith effort, but because that subset is selected for disagreement, it cannot quantify the overall error rate. The abstract's wording should be tempered, and the metric should be renamed something like 'majority agreement with the model-assigned answer.' The patient survey is too small (n=13) and non-diverse; the authors acknowledge this. A smaller point: the summaries patients saw were edited by an oncologist, so the survey measures the pipeline including human review, not raw GPT-4 output. That is not a fatal flaw, but it should be stated more prominently. The paper is otherwise sound. The claims are mostly hedged, the limitations section is honest, and the multi-agent LLM cross-check is a reasonable exploratory step. The core idea — that LLMs can draft trial education materials with human oversight — is plausible and potentially useful. Who should read this: people building patient-facing trial materials, and anyone evaluating LLM output in clinical settings. It is a good cautionary example for the evaluation-metrics conversation. I would send this to peer review. It deserves a serious referee to push on the metric definition, request an independent gold standard for a sample of MCQAs, and ask for a more careful abstract. The work is not ready to support strong accuracy claims, but it is a solid, honest baseline that the field can build on.

Referee Report

3 major / 4 minor

Summary. The paper proposes using GPT-4 to automatically generate two types of patient-facing educational materials from cancer clinical trial informed consent forms: plain-language summaries and multiple-choice question-answer pairs (MCQAs). The summaries are evaluated by four clinicians on 11 trials and by 13 patients in a single-institution survey; the MCQAs are generated for 91 ICFs (1,335 items) and evaluated by crowdsourced readers, with a manual error analysis of 78 selected high-difficulty/low-agreement items. The authors conclude that GPT-4 can draft readable and comprehensive summaries and accurate MCQAs with minimal engineering, but emphasize that hallucinations require human oversight.

Significance. If the claims were properly supported, this would be a useful proof-of-concept for scalable generation of clinical trial education materials from existing consent documents. The study has notable strengths: it uses real ICFs from ClinicalTrials.gov, publishes detailed prompts in the appendix, generates a large MCQA set, includes a transparent failure-mode analysis, and explicitly recommends a human-in-the-loop deployment model. However, the primary MCQA 'accuracy' metric is defined as agreement with GPT-4's own answer, which is not an independent measure of correctness, and the patient survey is small and non-diverse. The significance of the work is therefore real but the central claims are currently stronger than the evidence supports.

major comments (3)

[Multiple choice question-answer pair evaluation, Table 2, Results] The primary accuracy metric is defined as 'Percentage of MCQAs where the majority answer matches the GPT4 answer' (Table 2). This measures agreement with the generator, not correctness against the source ICF or an independent gold standard. The headline 97.91% is therefore not evidence of 'high accuracy' as claimed in the abstract and discussion. The quality-assurance analysis of 78 selected high-difficulty/low-agreement MCQAs found 17 GPT-4-generated errors and 18 cases of missing ICF information (Table 5); because this set was selected for disagreement, it cannot quantify the overall error rate, but it demonstrates that the agreement metric can mask genuine errors. To support the accuracy claim, the authors should either (a) obtain expert gold-standard labels on a random sample of MCQAs and report accuracy against those labels, or (b) revise the abstract, results, and discussion to describe 'high majority agreement with crowdsourced readers' and present the error analysis as qualitative failure-mode evidence only.
[Patient summary evaluation, Tables 3 and 4, Abstract] The patient survey that supports the abstract claim that summaries 'may improve patients' understanding and interest in clinical trials' has 13 participants, of whom 12 identify as White and 10 as male, and a substantial portion did not complete the non-BROADBAND trial summaries (Tables 3 and 4). With missing responses and a homogeneous sample, the survey cannot support the generalized wording in the abstract. The Discussion does list small sample size and lack of diversity as limitations, but the abstract and the first Discussion sentence present the finding as more robust than the data allow. Please temper these statements (e.g., 'was perceived as readable and helpful in a small pilot sample') and signal the pilot nature of the survey more explicitly.
[Summary Evaluation (Methods), Patient summary evaluation] The patient survey used summaries that 'underwent manual review and editing by an oncologist' before being shown to participants. The survey therefore measures the edited artifact, not the raw GPT-4 output. The abstract's statement that 'GPT4-generated summaries were both readable and comprehensive' should be qualified to indicate that the summaries were expert-reviewed and edited before the patient evaluation. Without this qualification, readers may infer that unedited out-of-the-box GPT-4 output was patient-facing, which is not what was tested.

minor comments (4)

[Throughout] Many spaces between words are missing (e.g., 'Theuseoflargelanguagemodels...', 'GPT4-generated' vs 'GPT-4'); please run a cleanup pass to improve readability.
[Table 1] In the row 'Any additional costs', option 'C' appears twice and the 'None of the above' option is labeled 'C' instead of 'D'; this appears to be a typographical error.
[Appendix, Tables 6] The appendix contains two tables numbered 'Table 6' (the MCQA statistics table and the NIH Common Rule mapping table); please renumber the second one.
[References] Reference [20] reads as a placeholder title ('A Study to Evaluate the Safety and Efficacy of [Study Intervention/Drug Name, if applicable]') rather than a specific trial; please verify and complete this citation.

Circularity Check

1 steps flagged · score 6.0 of 10

The headline 'accuracy' of the GPT-4-generated MCQAs (97.91%) is defined as the percentage of MCQAs where the crowd's majority answer matches GPT-4's own assigned answer, so the abstract's 'high accuracy' claim reduces by construction to agreement with the generator rather than correctness against the ICF.

self definitional [Table 2 (metric definitions); Results, 'Multiple choice question-answer pair evaluation'; Abstract.]
"Table 2: "Accuracy Percentage of MCQAs where the majority answer matches the GPT4 answer." Results: "The majority answer agreed with the GPT4 answer in 1307/1335 (97.91%) of MCQAs." Abstract: "The multiple-choice questions demonstrated high accuracy and agreement with crowdsourced annotators.""

The headline quantitative claim is reported as 'high accuracy' and quantified as 1307/1335 (97.91%), but Table 2 defines Accuracy as the percentage of MCQAs where the crowd's majority answer matches GPT-4's assigned answer. GPT-4's assigned answer is the very system output under evaluation, not an independent gold standard extracted from the ICF. Consequently, the 'accuracy' figure reduces by construction to a statement that a majority of crowdsourced readers selected the generator's own label: it measures model-rater agreement, not correctness. A defective question with a wrong-but-plausible GPT-4 answer still scores as 'accurate' whenever a majority of readers chooses that label, so the metric cannot by itself detect the errors the paper later acknowledges.

full rationale

The summary-generation arm is self-contained: clinician ratings of readability, inaccuracies, biases, and hallucinations, and the patient self-report survey, are independent human judgments and are not quantities derived from GPT-4's own outputs. No fitted parameter is renamed as a prediction in that arm, and no load-bearing self-citation appears; the cited prior work by a co-author (ref. 58, on the need for human oversight of medical AI) is contextual, not evidential for the claims. The circularity is confined to the MCQA evaluation. Table 2 defines Accuracy as the percentage of MCQAs where the crowd's majority answer matches GPT-4's assigned answer, and Difficulty is likewise defined as the fraction of readers disagreeing with the GPT-4 answer; both key metrics are therefore anchored to the generator's label. The results section then reports 1307/1335 (97.91%) under this definition, and the abstract converts this into 'high accuracy and agreement with crowdsourced annotators.' Because 'accuracy' is defined relative to the generator's own answer, the central quantitative claim reduces by construction to 'readers usually agreed with GPT-4,' and a wrong-but-plausible GPT-4 answer still counts as accurate if a majority of readers selects it. The paper is partially mitigating in that it transparently defines the metric, selects 78 high-disagreement MCQAs for manual review (finding 17 GPT-4 errors, 18 missing-information cases, and 13 ambiguous definitions), and uses four external LLMs to cross-check the QA set. However, that QA set is deliberately biased toward disagreement and cannot validate the headline 97.91% value; the main accuracy figure itself is never checked against an independent gold standard taken from the source ICFs. Overall, this is a partial, definitional circularity of the paper's central quantitative claim, not total circularity, because the qualitative findings, the summary evaluations, the manual error analysis, and the recommendation for human oversight all rest on genuinely external human judgment.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The central claim rests primarily on domain assumptions about content frameworks and evaluation proxies rather than numerical free parameters. The most consequential assumption is that agreement with GPT-4's own answer indicates correctness, which is an ad hoc choice that introduces circularity into the reported accuracy.

assumptions (3)

domain assumption The basic elements of informed consent from Subpart A of the Revised Common Rule are the correct content framework for trial summaries and MCQA topics.
Used to design the prompts for summary generation (Appendix A Figure 3 and Table 2) and the 15 oncologist-written in-context MCQA examples (Table 1).
domain assumption Crowdsourced readers answering MCQA questions using only the ICF text provides a valid measure of question quality.
The evaluation assumes that if a majority of lay readers can select GPT-4's assigned answer from the ICF, the question is good; this treats reader agreement as a proxy for correctness (Table 2).
ad hoc to paper GPT-4's generated answers are treated as the gold standard in the primary accuracy metric.
The 'Accuracy' metric in Table 2 is the fraction of MCQAs where the majority answer matches the GPT4 answer, not an externally verified answer; this makes the headline accuracy partly self-referential.

how reviews work

0 comments

Cite this review

Pith. "Pith review of The use of large language models to enhance cancer clinical trial educational materials." pith.science (2026). https://pith.science/paper/X6WJ5UVR

@misc{pith2026241201955,
  author       = {Pith},
  title        = {Pith review of: The use of large language models to enhance cancer clinical trial educational materials},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/X6WJ5UVR}},
  note         = {Machine review of arXiv:2412.01955}
}

read the original abstract

Cancer clinical trials often face challenges in recruitment and engagement due to a lack of participant-facing informational and educational resources. This study investigated the potential of Large Language Models (LLMs), specifically GPT4, in generating patient-friendly educational content from clinical trial informed consent forms. Using data from ClinicalTrials.gov, we employed zero-shot learning for creating trial summaries and one-shot learning for developing multiple-choice questions, evaluating their effectiveness through patient surveys and crowdsourced annotation. Results showed that GPT4-generated summaries were both readable and comprehensive, and may improve patients' understanding and interest in clinical trials. The multiple-choice questions demonstrated high accuracy and agreement with crowdsourced annotators. For both resource types, hallucinations were identified that require ongoing human oversight. The findings demonstrate the potential of LLMs "out-of-the-box" to support the generation of clinical trial education materials with minimal trial-specific engineering, but implementation with a human-in-the-loop is still needed to avoid misinformation risks.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

1 extracted references

[1]

out-of-the-box

Theuseoflargelanguagemodelstoenhancecancerclinical trial educational materials MingyeGao,MS 1* ;AmanVarshney,MS 2* ;ShanChen,MS 3,4 ;VikramGoddla 3,4 ,JackGallifant,MBBS 3,4 ;PatrickDoyle,MS 4 ;ClaireNovack,BS 4 ;MaeveDillon-Martin,BS 4 ;TeresiaPerkins,BS 4 ;XinrongCorreia 5 ;ErikDuhaimePhD 5 ;HowardIsenstein,MA 6 ,EladSharon,MD 7 ;LisaSoleymani Lehmann,M...

arXiv 2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

out-of-the-box

Theuseoflargelanguagemodelstoenhancecancerclinical trial educational materials MingyeGao,MS 1* ;AmanVarshney,MS 2* ;ShanChen,MS 3,4 ;VikramGoddla 3,4 ,JackGallifant,MBBS 3,4 ;PatrickDoyle,MS 4 ;ClaireNovack,BS 4 ;MaeveDillon-Martin,BS 4 ;TeresiaPerkins,BS 4 ;XinrongCorreia 5 ;ErikDuhaimePhD 5 ;HowardIsenstein,MA 6 ,EladSharon,MD 7 ;LisaSoleymani Lehmann,M...

arXiv 2023