Pith. sign in

REVIEW 5 major objections 5 minor 48 references

Smart Trial: Evaluating the Use of Large Language Models for Recruiting Clinical Trial Participants via Social Media

T0 review · 5 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that current large language models, despite their text understanding strength, cannot reliably screen social media users for clinical trial eligibility: they default to 'Unknown' when answers need complex, multi-hop inferen

desk verdict Useful new dataset, honest qualitative finding, but the fine-tuned comparison is compromised by a likely user-leakage split and needs to be re-run. read the letter →

arxiv 2509.10584 v1 pith:7QRG63RM submitted 2025-09-12 cs.CY cs.AIcs.CL

Xiaofan Zhou , Zisu Wang , Janice Krieger , Mohan Zalake , Lu Cheng This is my paper · ORCID
classification cs.CYcs.AIcs.CL
keywords clinicaltrialrecruitmentlargelanguagemodelssocialmediahealthdataeligibilitycriteriascreeningTRIALQAdatasetnaturalinferencemulti-hopreasoningReddit
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether large language models can spot potential clinical trial participants from what people post on social media. To test this, it builds TRIALQA, a dataset of Reddit posts from colon cancer and prostate cancer communities, annotated against real trial eligibility criteria and interest signals. Benchmarking seven LLMs under six prompting and fine-tuning strategies, the paper finds that no LLM consistently beats a small fine-tuned NLI model, and models often answer 'Unknown' precisely when eligibility requires connecting implicit clues. The take-away is that automated trial screening from social media is promising but not yet reliable, because the hardest part—multi-hop reasoning over self-reported health posts—remains unsolved.

What carries the argument

TRIALQA, a dataset of 502 colon cancer subreddit users (806 posts) and 799 prostate cancer subreddit users (1,555 posts), each annotated against eight real clinical-trial eligibility criteria plus six interest-reason indicators for the colon cancer set, using a True/False/Unknown scheme. The evaluation machinery is a benchmark of seven open LLMs (three 70B-class, four 7-8B) under six strategies—direct prompting, in-context learning, self-consistency, chain-of-thought, entry-level fine-tuning, and user-level fine-tuning—with RoBERTa-large-MNLI as a non-LLM baseline.

What would settle it

Re-annotate a random sample of 100 users per subreddit with independent clinical annotators and compute inter-annotator agreement (kappa) on the three-class labels; if kappa is below about 0.4, the accuracy differences between models cannot be interpreted as task difficulty. Alternatively, run a probe where each criterion is restated as an explicit sentence (e.g., 'I am 45 years old') and check whether LLMs answer 'True' there—if they still default to 'Unknown', the failure is more basic than multi-hop reasoning.

Watch

Extended reading notes

Core claim

The central discovery is that LLM-driven eligibility assessment from social media posts fails mainly at implicit, multi-hop reasoning. Across both TRIALQA datasets, models score highest on criteria dominated by the 'Unknown' label and show low recall on 'True' labels, indicating a default to 'Unknown' when evidence is not stated explicitly. A fine-tuned RoBERTa NLI baseline matches or outperforms most LLMs on the eligibility task under direct prompting and in-context learning, and error analysis surfaces three recurring failure modes: correct reasoning yielding wrong conclusions, incorrect reasoning, and confusion between 'No' and 'Unknown'.

Load-bearing premise

The ENTIRE benchmark treats the annotators' True/False/Unknown labels as reliable ground truth, yet the paper reports no inter-annotator agreement; if those labels are noisy or 'Unknown' is applied inconsistently, every model comparison inherits that noise.

Editorial extensions

If this is right

  • Automated pre-screening of social media users for trial eligibility is not yet dependable with current LLMs; a small, cheap NLI model is competitive or better, so cost-effective deployment should consider non-LLM baselines first.
  • The systematic default to 'Unknown' means these tools would miss real eligible participants whose posts imply but do not state eligibility, so recruitment workflows would need human review for the 'Unknown' bucket.
  • Fine-tuning, especially entry-level, clearly improves over prompting, showing the user posts do contain extractable signals—the bottleneck is inference strategy, not information absence.
  • Larger models gain more from in-context learning and chain-of-thought, but smaller fine-tuned models can match them, suggesting that the path to better screening is better reasoning training, not merely larger models.
  • The named error types—wrong conclusions from correct reasoning, incorrect reasoning, and No/Unknown confusion—give concrete targets for future LLM reasoning research.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If this result generalizes beyond Reddit and these two cancer types, LLM-assisted recruitment should be framed as a triage tool that sorts posts into 'clearly eligible', 'clearly ineligible', and 'needs information or human check', rather than as an end-to-end eligibility oracle.
  • A natural extension is to measure performance separately on one-hop criteria (age, recent antibiotics) versus multi-hop criteria (intestinal bowel disease history), predicting that the gap between LLMs and fine-tuned baselines widens with the number of inferential steps.
  • One testable improvement: train or prompt models with an explicit instruction to say 'Unknown' only when the information is truly absent, and to provide a confidence or evidence quote otherwise; this could be evaluated by whether recall on implicit 'True' rises without precision dropping.
  • The absence of inter-annotator agreement statistics suggests a concrete next step: re-annotate a sample with independent raters to quantify label reliability; if 'Unknown' is applied inconsistently, the benchmark's difficulty claims would need recalibration.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces TRIALQA, a Reddit-based dataset for evaluating LLMs on clinical trial participant recruitment. It contains 502 colon-cancer users / 806 posts and 799 prostate-cancer users / 1,555 posts, annotated per user for eight eligibility criteria (and, for colon cancer, six expressed-interest reasons). The authors benchmark seven open-source LLMs under direct prompting, in-context learning, self-consistency, chain-of-thought, and two LoRA fine-tuning settings, and compare them with a RoBERTa NLI baseline. The central claim is that current LLMs perform modestly, frequently default to the 'Unknown' class, and are not consistently better than the smaller, cheaper RoBERTa baseline.

Significance. If the qualitative conclusion holds, this is a useful, cautionary result for an important application domain: using social-media self-reports to identify clinical-trial candidates. The paper's strengths are its use of real-world eligibility criteria, the breadth of LLMs and inference strategies, the explicit error-example analysis, and the honest acknowledgment of dataset limitations. The dataset could be a valuable benchmark, though its non-release currently limits community use. However, the quantitative evidence supporting the parity claim is compromised by unresolved data-splitting, annotation-reliability, and statistical-uncertainty issues, so the benchmark comparisons need substantial reinforcement before the paper's claims can be accepted as stated.

major comments (5)
  1. [Section 3.2, Table 3] The Entry-level FT setting treats each post as a sample and says only 'We split training data and test data by 4:1.' But Sections 2.1 and 2.3 show that labels are assigned per user after concatenating all posts from each user. If the 4:1 split is post-level, the same user can appear in both training and test sets, allowing a fine-tuned model to memorize user-specific phrasing and making test labels non-independent. This could substantially inflate the Table 3 results, which are the sole basis for the claim that fine-tuned RoBERTa is at parity with or better than LLMs. Please state whether the split is user-level; if not, rerun with a user-level split and report results across multiple seeds.
  2. [Section 2.3] The annotation protocol uses three-class labels (True/False/Unknown) and a reconciliation procedure, but no inter-annotator agreement statistics are reported. Since the benchmark's ground truth is the reconciled labels and the 'Unknown' category is central to the main qualitative finding, uncontrolled label noise or inconsistent application of 'Unknown' can directly affect all accuracy and F1 numbers. Report agreement (e.g., Cohen's or Fleiss' kappa) per dataset and per criterion, and ideally per-label agreement.
  3. [Tables 2 and 3] All results are reported from a single train/test split with no error bars, confidence intervals, or repeated runs. Many of the headline comparisons are within 0.01–0.02, which is almost certainly within sampling noise (e.g., DeepSeek-Q7B 0.70 vs RoBERTa 0.69 on colon criteria; Llama3.1-8B 0.69 vs RoBERTa 0.68 on prostate). The word 'consistently' in the abstract is therefore not supported. Add majority-class baselines and either bootstrap confidence intervals or multiple seeded runs, especially for the fine-tuned comparisons.
  4. [Table 5] The row for Qwen2.5-72B in the prostate-cancer ICL table (c1=0.27, c2=0.54, c3=0.83, c4=0.24, c5=0.38, c6=0.96, c7=0.49, c8=0.45) is identical to the corresponding row in the colon-cancer table (Table 4). This appears to be a copy-paste error. Because RQ2 draws per-criterion conclusions from these tables, the prostate results need to be verified and corrected.
  5. [Section 3.5, Table 7] The DeepSeek-Q7B row under Direct Prompting on prostate cancer reports False-class precision 0.56, recall 0.00, F1 0.00. A non-zero precision with zero recall is impossible unless there are no false predictions but false samples are all predicted correctly? Actually, with zero true positives, precision is undefined or zero, not 0.56. This suggests an error in the reported values or in the evaluation code. Please check all rows for similar inconsistencies.
minor comments (5)
  1. [Conclusion] The dataset is referred to as 'TRAIQA' (typo for 'TRIALQA').
  2. [Sections 1 and 3] The Introduction says '5 different prompting and reasoning methods' while Section 3 says 'six distinct strategies' and Section 3.2 indeed lists six (direct, ICL, self-consistency, CoT, user-level FT, entry-level FT). Clarify the count.
  3. [Section 3.2] The self-consistency method is described only as 'the most consistent output.' Report the number of sampled outputs, temperature, and how consistency was determined.
  4. [Section 5] The paper states that ethical constraints prevent dataset release. This limits reproducibility; even if the dataset cannot be shared, releasing annotation guidelines and evaluation code would strengthen the work.
  5. [Tables 4 and 5] The per-criterion tables report only accuracy, which is misleading for the heavily imbalanced classes discussed in Section 3.4. Adding class-wise recall or balanced accuracy would better support the 'default to Unknown' narrative.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: TRIALQA is an empirical benchmark evaluated against independent human labels; no prediction reduces to its construction.

full rationale

This paper contains no derivation that is equivalent to its inputs. The benchmark labels in Section 2.3 are produced by human annotators from Reddit posts and eligibility criteria, independently of the models being evaluated. The central claims—that LLMs often default to Unknown and that RoBERTa is at parity or better on the criteria tasks—are read directly from Tables 2-8, not entailed by the annotation scheme. The 'Unknown' class is defined as 'relevant information was not mentioned' and the models are prompted to answer True/False/Unknown; a model could, in principle, produce any distribution, so high Unknown recall is an empirical finding rather than a construction artifact. There are no fitted parameters that are then relabeled as predictions, no uniqueness theorem imported from the authors' prior work, and no ansatz smuggled in via citation. The only self-citation is a related-work reference [26] on virtual-human recruitment, which is not load-bearing. The acknowledged limitations (two U.S. trials, Reddit-only, no dataset release) are scope constraints. The lack of inter-annotator agreement statistics and the ambiguous post-level vs user-level split are validity/reliability concerns, not circularity. Therefore no circular step is present.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces a dataset, not a new theoretical entity. The central claim rests on domain assumptions about annotation quality, the sufficiency of social media context, and the representativeness of two trials. No free parameters are fitted to the outcome; model weights come from pretraining and LoRA fine-tuning, whose hyperparameters are not reported.

assumptions (3)
  • domain assumption Annotator labels are treated as reliable ground truth for whether a user meets an eligibility criterion, despite no inter-annotator agreement statistics.
    Section 2.3 describes three annotators and reconciliation, but no agreement metrics are reported. All model evaluations assume these labels are correct.
  • domain assumption A user's concatenated Reddit post history contains sufficient information to infer trial eligibility, with 'Unknown' reserved for genuinely missing information.
    Section 2.1 reconstructs user post history as the input context; the validity of the True/False/Unknown task depends on this being a meaningful basis for judgment.
  • domain assumption The two selected clinical trials (Metabiomics Colon Cancer Study and Prostate Cancer Genetic Risk Study) are representative sources of typical eligibility criteria for benchmarking.
    Section 2.2 adopts criteria from only these two trials, yet the conclusions generalize to the broader task of CT recruitment.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Smart Trial: Evaluating the Use of Large Language Models for Recruiting Clinical Trial Participants via Social Media." pith.science (2026). https://pith.science/paper/7QRG63RM

@misc{pith2026250910584,
  author       = {Pith},
  title        = {Pith review of: Smart Trial: Evaluating the Use of Large Language Models for Recruiting Clinical Trial Participants via Social Media},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7QRG63RM}},
  note         = {Machine review of arXiv:2509.10584}
}
read the original abstract

Clinical trials (CT) are essential for advancing medical research and treatment, yet efficiently recruiting eligible participants -- each of whom must meet complex eligibility criteria -- remains a significant challenge. Traditional recruitment approaches, such as advertisements or electronic health record screening within hospitals, are often time-consuming and geographically constrained. This work addresses the recruitment challenge by leveraging the vast amount of health-related information individuals share on social media platforms. With the emergence of powerful large language models (LLMs) capable of sophisticated text understanding, we pose the central research question: Can LLM-driven tools facilitate CT recruitment by identifying potential participants through their engagement on social media? To investigate this question, we introduce TRIALQA, a novel dataset comprising two social media collections from the subreddits on colon cancer and prostate cancer. Using eligibility criteria from public real-world CTs, experienced annotators are hired to annotate TRIALQA to indicate (1) whether a social media user meets a given eligibility criterion and (2) the user's stated reasons for interest in participating in CT. We benchmark seven widely used LLMs on these two prediction tasks, employing six distinct training and inference strategies. Our extensive experiments reveal that, while LLMs show considerable promise, they still face challenges in performing the complex, multi-hop reasoning needed to accurately assess eligibility criteria.

Figures

Figures reproduced from arXiv: 2509.10584 by the authors.

Figure 2
Figure 2. Word cloud of prostate cancer posts we need to annotate each post with the corresponding labels for every criterion and interest reason. Below, we present the creation of our datasets, TRIALQA, designed specifically for participant re￾cruitment based on the eligibility criteria of the Metabiomics Colon Cancer Clinical Research Study2 and the Prostate Cancer Screening for People at Genetic Risk for Aggressive Disease… view at source ↗
Figure 3
Figure 3. Distribution of colon cancer CT criteria annotation. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 13 linked inside Pith

  1. [1]

    Llama 3 model card

    AI@Meta. Llama 3 model card. 2024. URL https://github.com/meta-llama/llama3/ blob/main/MODEL_CARD.md

    2024

  2. [2]

    Idofew: intermediate training using dual-clustering in language models for few labels text classification

    Abdullah Alsuhaibani, Hamad Zogan, Imran Razzak, Shoaib Jameel, and Guan- dong Xu. Idofew: intermediate training using dual-clustering in language models for few labels text classification. InProceedings of the 17th ACM International Conference on Web Search and Data Mining, pages 18–27, 2024

    2024

  3. [3]

    Introducing the next generation of claude, March 2024

    Anthropic. Introducing the next generation of claude, March 2024. URL https: //www.anthropic.com/news/claude-3-family. Accessed: 2025-08-09

    2024

  4. [4]

    System card: Claude opus 4 & claude sonnet 4, May 2025

    Anthropic. System card: Claude opus 4 & claude sonnet 4, May 2025. URL https: //www-cdn.anthropic.com/4263b940cabb546aa0e3283f35b686f4f3b2ff47.pdf. Ac- cessed: 2025-08-09

    2025

  5. [5]

    Artificial intelligence tool for opti- mizing eligibility screening for clinical trials in a large community cancer center

    J Thaddeus Beck, Melissa Rammage, Gretchen P Jackson, Anita M Preininger, Irene Dankwa-Mullan, M Christopher Roebuck, Adam Torres, Helen Holtzen, Sadie E Coverdill, M Paul Williamson, et al. Artificial intelligence tool for opti- mizing eligibility screening for clinical trials in a large community cancer center. JCO clinical cancer informatics, 4:50–59, 2020

    2020

  6. [6]

    Language models are few-shot learners.Advances in neural infor- mation processing systems, 33:1877–1901, 2020

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners.Advances in neural infor- mation processing systems, 33:1877–1901, 2020

    1901

  7. [7]

    Improving the efficiency of clinical trial recruitment using an ensemble machine learning to assist with eligibility screening.ACR open rheumatology, 3(9):593–600, 2021

    Tianrun Cai, Fiona Cai, Kumar P Dahal, Gabrielle Cremone, Ethan Lam, Charlotte Golnik, Thany Seyok, Chuan Hong, Tianxi Cai, and Katherine P Liao. Improving the efficiency of clinical trial recruitment using an ensemble machine learning to assist with eligibility screening.ACR open rheumatology, 3(9):593–600, 2021

    2021

  8. [8]

    Denise Calaprice-Whitty, Karim Galil, Wael Salloum, Ashkon Zariv, and Bernal Jimenez. Improving clinical trial participant prescreening with artificial intelli- gence (ai): a comparison of the results of ai-assisted vs standard methods in 3 oncology trials.Therapeutic innovation & regulatory science, 54(1):69–74, 2020

    2020

Show all 48 references
  1. [9]

    Visual field endpoints for neuroprotective trials: a case for ai-driven patient enrichment.American journal of ophthalmology, 243:118–124, 2022

    Andrew Chen, Giovanni Montesano, Randy Lu, Cecilia S Lee, David P Crabb, and Aaron Y Lee. Visual field endpoints for neuroprotective trials: a case for ai-driven patient enrichment.American journal of ophthalmology, 243:118–124, 2022

    2022

  2. [10]

    TRACT: Regression-aware fine-tuning meets chain-of-thought reasoning for LLM-as-a-judge

    Cheng-Han Chiang, Hung-yi Lee, and Michal Lukasik. TRACT: Regression-aware fine-tuning meets chain-of-thought reasoning for LLM-as-a-judge. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Proceedings of the 63rd Annual Meeting of the As...

    2025

  3. [11]

    Clinical trial delays: America ˘2019s patient recruit- ment dilemma

    Clinical Trials Arena. Clinical trial delays: America ˘2019s patient recruit- ment dilemma. https://www.clinicaltrialsarena.com/features/featureclinical-trial- patient-recruitment/, July 2012. Accessed: 2025-08-09

    2012

  4. [12]

    The role of social media in enhancing clinical trial recruitment: scoping review.Journal of medical Internet research, 22(10):e22810, 2020

    Ida Darmawan, Caitlin Bakker, Tabetha A Brockman, Christi A Patten, and Milton Eder. The role of social media in enhancing clinical trial recruitment: scoping review.Journal of medical Internet research, 22(10):e22810, 2020

    2020

  5. [13]

    Deepseek-r1: Incentivizing reasoning capability in llms via rein- forcement learning, 2025

    DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in llms via rein- forcement learning, 2025. URL https://arxiv.org/abs/2501.12948

    2025 arXiv

  6. [14]

    Selecting sites and investigators.Applied clinical trials, 11(3): 56–66, 2002

    Janos Demeter. Selecting sites and investigators.Applied clinical trials, 11(3): 56–66, 2002

    2002

  7. [15]

    Machine learning predictive models can improve efficacy of clinical trials for alzheimer’s disease.Journal of Alzheimer’s Disease, 74(1):55–63, 2020

    Ali Ezzati, Richard B Lipton, and Alzheimer’s Disease Neuroimaging Initiative. Machine learning predictive models can improve efficacy of clinical trials for alzheimer’s disease.Journal of Alzheimer’s Disease, 74(1):55–63, 2020

    2020

  8. [16]

    What do people really know and think about clinical trials? a comparison of rural and urban communities in the south.Journal of Community Health, 38(4):642–651, 2013

    Daniela B Friedman, Caroline D Bergeron, Caroline Foster, Andrea Tanner, and Sei-Hill Kim. What do people really know and think about clinical trials? a comparison of rural and urban communities in the south.Journal of Community Health, 38(4):642–651, 2013

    2013

  9. [17]

    Real-time virtual human for promoting clinical trial education and recruitment

    Rashi Ghosh, Andrew Maxim, Christopher You, and Benjamin Lok. Real-time virtual human for promoting clinical trial education and recruitment. In2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pages 1184–1185. IEEE, 2024

    2024

  10. [18]

    Optimizing clinical trials recruitment via deep learning.Journal of the American Medical Informatics Association, 26(11):1195–1202, 2019

    Jelena Gligorijevic, Djordje Gligorijevic, Martin Pavlovski, Elizabeth Milkovits, Lucas Glass, Kevin Grier, Praveen Vankireddy, and Zoran Obradovic. Optimizing clinical trials recruitment via deep learning.Journal of the American Medical Informatics Association, 26(11):1195–1202, 2019

    2019

  11. [19]

    The llama 3 herd of models, 2024

    Aaron Grattafiori et al. The llama 3 herd of models, 2024. URL https://arxiv.org/ abs/2407.21783

    2024 arXiv

  12. [20]

    Cohortgpt: An enhanced gpt for participant recruitment in clinical study.arXiv preprint arXiv:2307.11346, 2023

    Zihan Guan, Zihao Wu, Zhengliang Liu, Dufan Wu, Hui Ren, Quanzheng Li, Xiang Li, and Ninghao Liu. Cohortgpt: An enhanced gpt for participant recruitment in clinical study.arXiv preprint arXiv:2307.11346, 2023

    2023 arXiv

  13. [21]

    Strategies addressing barriers to clinical trial enrollment of underrepresented populations: a systematic review

    Caren Heller, Joyce E Balls-Berry, Jill Dumbauld Nery, Patricia J Erwin, Dawn Littleton, Mimi Kim, and Winston P Kuo. Strategies addressing barriers to clinical trial enrollment of underrepresented populations: a systematic review. Contemporary clinical trials, 39(2):169–182, 2014

    2014

  14. [22]

    Lora: Low-rank adaptation of large language models.ICLR, 1(2):3, 2022

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models.ICLR, 1(2):3, 2022

    2022

  15. [23]

    Enhancing large language models in coding through multi-perspective self-consistency

    Baizhou Huang, Shuai Lu, Xiaojun Wan, and Nan Duan. Enhancing large language models in coding through multi-perspective self-consistency. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Proceedings of the 62nd Annual Meeting of the Association for Computational Lingu...

    2024 doi

  16. [24]

    Jiang et al

    Albert Q. Jiang et al. Mistral 7b, 2023. URL https://arxiv.org/abs/2310.06825

    2023 arXiv

  17. [25]

    Robust training for conversational question answering models with reinforced reformulation generation

    Magdalena Kaiser, Rishiraj Saha Roy, and Gerhard Weikum. Robust training for conversational question answering models with reinforced reformulation generation. InProceedings of the 17th ACM International Conference on Web Search and Data Mining, pages 322–331, 2024

    2024

  18. [26]

    Tailoring recruitment communication using virtual human technology to increase participation of older minority adults in clinical trials.Alzheimer’s & Dementia, 18:e064306, 2022

    Janice Krieger and Stephen Anton. Tailoring recruitment communication using virtual human technology to increase participation of older minority adults in clinical trials.Alzheimer’s & Dementia, 18:e064306, 2022

    2022

  19. [27]

    Retrieval-augmented generation for knowledge-intensive nlp tasks.Advances in neural information processing systems, 33:9459–9474, 2020

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks.Advances in neural information processing systems,...

    2020

  20. [28]

    Roberta: A robustly optimized bert pretraining approach.arXiv preprint arXiv:1907.11692, 2019

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach.arXiv preprint arXiv:1907.11692, 2019

    1907 arXiv

  21. [29]

    Xiaoran Lu, Chen Yang, Lu Liang, Guanyu Hu, Ziyi Zhong, and Zihao Jiang. Artificial intelligence for optimizing recruitment and retention in clinical trials: a scoping review.Journal of the American Medical Informatics Association, 31(11): 2749–2759, 2024

    2024

  22. [30]

    Riccardo Miotto and Chunhua Weng. Case-based reasoning using electronic health records efficiently identifies eligible patients for clinical trials.Journal of the American Medical Informatics Association, 22(e1):e141–e150, 2015

    2015

  23. [31]

    Untapping the power of indirect relationships in entity summarization

    Atefeh Moradan, Mohammad Sorkhpar, Atsushi Miyauchi, Davide Mottin, and Ira Assent. Untapping the power of indirect relationships in entity summarization. InProceedings of the Eighteenth ACM International Conference on Web Search and Data Mining, pages 820–828, 2025

    2025

  24. [32]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    OpenAI. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023. URL https://arxiv.org/abs/2303.08774

    2023 arXiv

  25. [33]

    A deep reinforced model for abstractive summarization.arXiv preprint arXiv:1705.04304, 2017

    Romain Paulus, Caiming Xiong, and Richard Socher. A deep reinforced model for abstractive summarization.arXiv preprint arXiv:1705.04304, 2017

    2017 arXiv

  26. [34]

    Language models as knowledge bases? arXiv preprint arXiv:1909.01066, 2019

    Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H Miller, and Sebastian Riedel. Language models as knowledge bases? arXiv preprint arXiv:1909.01066, 2019

    1909 arXiv

  27. [35]

    Open domain question answering using early fusion of knowledge bases and text.arXiv preprint arXiv:1809.00782, 2018

    Haitian Sun, Bhuwan Dhingra, Manzil Zaheer, Kathryn Mazaitis, Ruslan Salakhut- dinov, and William W Cohen. Open domain question answering using early fusion of knowledge bases and text.arXiv preprint arXiv:1809.00782, 2018

    2018 arXiv

  28. [36]

    Qwen2.5: A party of foundation models, September 2024

    Qwen Team. Qwen2.5: A party of foundation models, September 2024. URL https://qwenlm.github.io/blog/qwen2.5/

    2024

  29. [37]

    The use of social media in recruit- ment for medical research studies: a scoping review.Journal of medical Internet research, 18(11):e286, 2016

    Jane Topolovec-Vranic and Karthik Natarajan. The use of social media in recruit- ment for medical research studies: a scoping review.Journal of medical Internet research, 18(11):e286, 2016

    2016

  30. [38]

    Llama: Open and efficient foundation language models.CoRR, abs/2302.13971,

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lam- ple. Llama: Open and efficient foundation langua...

    arXiv

  31. [39]

    Self-consistency improves chain of thought reasoning in language models.arXiv preprint arXiv:2203.11171, 2022

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models.arXiv preprint arXiv:2203.11171, 2022. Smart Trial: Evaluating the Use of Large Language Models...

    2022 arXiv

  32. [40]

    Finetuned language models are zero-shot learners.arXiv preprint arXiv:2109.01652, 2021

    Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. Finetuned language models are zero-shot learners.arXiv preprint arXiv:2109.01652, 2021

    2021 arXiv

  33. [41]

    Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems, 35: 24824–24837, 2022

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems, 35: 24824–24837, 2022

    2022

  34. [42]

    Lighter and better: Towards flexible context adaptation for retrieval augmented generation

    Chenyuan Wu, Ninglu Shao, Zheng Liu, Shitao Xiao, Chaozhuo Li, Chen Zhang, Senzhang Wang, and Defu Lian. Lighter and better: Towards flexible context adaptation for retrieval augmented generation. InProceedings of the Eighteenth ACM International Conference on Web Search and D...

    2025

  35. [43]

    Antonio Yaghy, Aaron Y Lee, Pearse A Keane, Tiarnan DL Keenan, Luisa SM Mendonca, Cecilia S Lee, Anne Marie Cairns, Joseph Caroll, Hao Chen, Julie Clark, et al. Artificial intelligence-based strategies to identify patient populations and advance analysis in age-related macular...

    2022

  36. [44]

    Qwen2 technical report.arXiv preprint arXiv:2407.10671, 2024

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Cheng- peng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jin Xu, Jingren Zhou, Jinze Bai, Jinzhe...

    2024 arXiv

  37. [45]

    Gap: a grammar and position-aware framework for efficient recognition of multi-line mathematical formulas

    Zhe Yang, Qi Liu, Kai Zhang, Shiwei Tong, and Enhong Chen. Gap: a grammar and position-aware framework for efficient recognition of multi-line mathematical formulas. InProceedings of the 17th ACM International Conference on Web Search and Data Mining, pages 901–910, 2024

    2024

  38. [46]

    Explainable ctr prediction via llm reasoning

    Xiaohan Yu, Li Zhang, and Chong Chen. Explainable ctr prediction via llm reasoning. InProceedings of the Eighteenth ACM International Conference on Web Search and Data Mining, pages 707–716, 2025

    2025

  39. [47]

    Importance of clinical trials and contributions to contemporary medicine: commentary.Annals of Medicine, 57(1):2451190, 2025

    Haofuzi Zhang and Xiaofan Jiang. Importance of clinical trials and contributions to contemporary medicine: commentary.Annals of Medicine, 57(1):2451190, 2025. Received 20 February 2007; revised 12 March 2009; accepted 5 June 2009

    2025

  40. [2023]

    URL http://dblp.uni-trier.de/db/journals/corr/corr2302.html#abs-2302- 13971

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.