Pith. sign in

REVIEW 4 major objections 6 minor 49 references

Dr.Copilot: A Multi-Agent Prompt Optimized Assistant for Improving Patient-Doctor Communication in Romanian

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A three-agent LLM assistant for doctors lifts positive patient reviews by 70%.

desk verdict Real system, real deployment, weak causal evidence—the 70.22% headline shouldn't be taken at face value. read the letter →

arxiv 2507.11299 v2 pith:3I7O4JOS submitted 2025-07-15 cs.CL

Andrei Niculae , Adrian Cosma , Cosmin Dumitrache , Emilian Rǎdoi This is my paper
classification cs.CL
keywords telemedicinedoctor-patientcommunicationmulti-agentlanguagemodelsRomanianlow-resourceNLPpromptoptimizationpatientsatisfactionresponsequalityfeedback
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a language-model assistant can improve doctor-patient communication in a low-resource language by critiquing only the presentation of written replies, not their medical content. The system scores each reply on 17 interpretable quality axes, generates targeted rewriting suggestions, and lets the doctor decide whether to use them. It is built from open-weight models with about 100 annotated examples, so it runs on-premise and keeps patient data private. In a live deployment with 41 Romanian doctors, replies revised with the assistant's suggestions received positive patient reviews 40.82% of the time versus 23.98% for unrevised replies, a 70.22% relative increase, and measured quality scores improved about 51% relative to baseline versus a 37% estimate from offline simulation. If the result holds, it gives a practical, low-cost route to better telemedicine experiences for underrepresented languages.

What carries the argument

The carrying mechanism is the three-agent feedback loop built on 17 interpretable presentation-quality axes. A Scorer Agent assigns scores along these axes (Likert or binary), a Recommender Agent generates concrete, actionable suggestions for the weakest axes, and a Reconciliation Agent—used during evaluation, not deployment—simulates a doctor applying those suggestions so the revised reply can be re-scored; the live interface surfaces suggestions for only the top three axes ranked by their correlation with patient reviews. The loop is parameterized not by fine-tuning but by automatic prompt optimization on roughly 100 annotated examples, which makes it cheap and privacy-preserving for a proprietary, low-resource Romanian dataset.

What would settle it

Stratify the 212 live interactions by doctor and by question difficulty and compare the like-to-response ratio of the 49 revised responses with unrevised responses from the same doctors on comparably difficult questions; if the 70% advantage vanishes or reverses within this matched comparison, the attribution of the gain to the assistant collapses.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that Dr.Copilot—a multi-agent system with a Scorer Agent, a Recommender Agent, and a Reconciliation Agent—can measurably improve text-based doctor-patient communication in Romanian. The Scorer Agent evaluates replies along 17 hand-defined axes such as empathy, number of problems addressed, grammatical correctness, and platform-appropriate behavior; the Recommender Agent turns low scores into ranked, specific suggestions that are surfaced to the doctor in the live interface; the Reconciliation Agent is used offline to simulate a doctor applying suggestions and re-scores the result. The paper's central empirical claims are that prompt-optimized open-weight models match expert agreement on the scoring task; that the offline reconciliation procedure estimates a 37% relative quality improvement; and that in a live deployment, doctors who incorporated suggestions achieved a 51% relative quality improvement and a 70.22% increase in the like-to-response ratio compared with responses where suggestions were not incorporated. The system intentionally never judges medical correctness and never talks to the patient directly.

Load-bearing premise

The live satisfaction comparison assumes that which responses were revised is unrelated to doctor skill, response quality, or case difficulty, so the 40.82% versus 23.98% review-rate gap can be credited to the assistant rather than to the doctors who chose to incorporate its suggestions.

Editorial extensions

If this is right

  • Romanian text telemedicine can be meaningfully improved with open-weight models and about 100 labeled examples, without waiting for a dedicated medical Romanian LLM.
  • Because doctors keep full control of the final text, the assistant can be deployed in settings where liability and trust concerns rule out autonomous AI advice.
  • The offline reconciliation procedure, despite underestimating the live gain, gives a usable lower-bound predictor of deployment impact.
  • The 17-axis feedback scheme gives doctors actionable guidance on exactly what to change, and the live data rank which axes matter most for patient reviews.
  • If similar gains occur elsewhere, presentation-focused copilots become a viable first step for low-resource languages before clinical-accuracy systems are mature.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the live numbers are observational, not experimental; a randomized assignment of suggestions (or an encouragement design) would be needed to separate the assistant's causal effect from selection of which doctors and cases got revised responses.
  • Editorial inference: because the system scores only presentation, it could in principle reward a fluent but medically incomplete reply; a testable extension is to add a separate medical-content checklist (e.g., mention of contraindications) and see whether satisfaction changes.
  • Editorial inference: the 17 axes were chosen in advance and may not capture everything patients value; mining the review text itself to discover new, data-driven axes could extend the system and is a natural follow-up.
  • Editorial inference: the 51% vs 37% gap suggests doctors apply suggestions more effectively than the simulated reconciliation agent; studying which suggestion types doctors actually applied (the appendix lists counts per type) could refine the offline predictor.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents Dr.Copilot, a multi-agent LLM system for Romanian telemedicine that scores doctor responses along 17 interpretable quality axes, generates presentation-focused recommendations, and lets doctors decide whether to incorporate them. The agent prompts are optimized with DSPy using 100 annotated question-response pairs, and the system is deployed on-premise with open-weight Gemma-family models. Offline experiments compare scorer models and prompt optimizers, a self-evaluation procedure uses a Reconciliation Agent to apply recommendations and re-score the revised response, and a live deployment with 41 doctors (212 requests, 49 revised responses) is reported. The headline claims are a 70.22% increase in the like-to-response ratio for revised responses and a 51% relative improvement in quality metrics compared to the 37% estimated by the self-evaluation procedure.

Significance. If the causal claims held, this would be a valuable demonstration of a privacy-preserving, deployable LLM assistant in a low-resource medical setting, with one of the few live deployment reports for Romanian medical NLP. The paper has concrete strengths: the code is public, the models are open-weight and run on-premise, the 17-axis feedback scheme is interpretable, and the deployment includes real doctor-patient interactions. The central evidence, however, is currently observational and partly circular: the live satisfaction comparison is a self-selected comparison without randomization or controls, and the quality-improvement numbers come from the same Scorer Agent that was optimized on the same metric definitions. These issues undermine the paper's main quantitative claims until addressed.

major comments (4)
  1. [Section 4.1, Figure 6] The 70.22% increase in the like-to-response ratio is not causally identified. Doctors chose whether to incorporate Dr.Copilot's suggestions, and the comparison of 49 revised against 163 unrevised responses includes no randomization, no baseline adjustment, no doctor-level fixed effects, and no statistical test. Revision likelihood is likely correlated with doctor diligence, time pressure, case difficulty, initial response quality, and response length, any of which could explain the difference without a true system effect. The Limitations section acknowledges the small number of doctors but does not address this selection mechanism. Please report confidence intervals, cluster standard errors by doctor, and ideally a matched or propensity-adjusted comparison, or explicitly reframe the finding as an association.
  2. [Sections 3.2 and 4 (Figures 5 and 6)] The live quality improvement (51%) and the offline self-evaluation estimate (37%) are both produced by the same MedGemma-27B Scorer Agent, whose prompts were optimized via DSPy-SIMBA on the same metric definitions. The scorer was validated only on original responses (80 examples), not on revised responses or on responses that adopt the system's recommended phrasing. The score improvement may therefore reflect the scorer's preference for recommendation-consistent language rather than an independent quality gain. I ask for human evaluation of a sample of revised responses, or scoring by an independent judge model that was not optimized on these metrics, before accepting the 51% improvement claim.
  3. [Section 3.1, Table 1] Inter-annotator agreement after the second annotation round is not reported. Table 1 shows first-round Cohen's kappas as low as 0.12 ("Only recommends visit") and 0.24 ("Explanation next steps", "Cannot help online"), and the text states the metrics were reviewed and clarified to ensure full agreement, but no second-round kappa or percentage-agreement numbers are given. Without these values, the quality of the gold labels used to optimize and evaluate the Scorer Agent is not established. Please report the second-round agreement statistics.
  4. [Section 4.1] No significance tests or effect-size uncertainty are reported for any of the deployment results. With only 49 revised responses, the observed 40.82% versus 23.98% difference is highly variable, and the claim of a "measurable" improvement requires at least a confidence interval or a stated hypothesis test. Please add these.
minor comments (6)
  1. [Abstract] The phrase "comprises of" should be "comprises".
  2. [Appendix A.1] The metric list defines 16 metrics, but the paper claims 17 axes and Tables 1 and 2 include a "Clarifications" metric that is not formally defined in the metric list. Please add the missing definition.
  3. [Section 3.2] The agent is referred to inconsistently as both "Scoring Agent" and "Scorer Agent" throughout the text and figures; please unify the terminology.
  4. [Figure 10 caption] There is a typo in the caption: "Romanian on the leftand the translation" should read "Romanian on the left and the translation".
  5. [Section 3.1 and Figure 3] Figure 3 reports correlations between quality measures and user reviews, but no correlation coefficients, sample sizes, or significance values are given in the text or figure. Please add at least the numeric values.
  6. [Section 4.1] The like-to-response ratio is used as the patient-satisfaction measure, but its exact definition (e.g., whether the denominator is responses, patients, or reviews, and how multiple reviews per response are handled) is not fully specified. Please clarify.

Circularity Check

1 steps flagged · score 4.0 of 10

Partial circularity: the offline 37% estimate and the live 51% 'quality improvement' are both measured by the system's own Scorer Agent; the headline patient-satisfaction result is external but observationally confounded.

  1. fitted input called prediction [Section 3.2 (Self-Evaluation Procedure) and Section 4.1 / Figure 6]
    "we propose a 'Self-Evaluation Procedure': we create a Reconciliation Agent which is tasked to incorporate the recommendations into the original response, and the revised response is re-scored using the Scorer Agent. ... In Figure 6, we show score improvements across metrics when suggestions are incorporated by doctors in the live environment."

    The 37% offline improvement is computed by re-scoring the Reconciliation Agent's revised responses with the same Scorer Agent that produced the scores consumed by the Recommender. The 51% live improvement is then presented as the 'real score improvement by doctors', but it is measured with the same Scorer Agent; no independent human rating of revised responses is reported. Thus the quality-improvement 'prediction' is the system's own fitted instrument grading responses that were rewritten to satisfy that instrument. The Scorer's validation against 80 human-annotated original responses (Figure 4) gives the scores some external anchor, so this is not a pure tautology, but the improvement estimate is partially self-confirming.

full rationale

The paper's central patient-satisfaction claim (40.82% vs 23.98% like-to-response ratio, a 70.22% increase) is an external, observed outcome and is not circular by construction; its main threat is selection bias because doctors chose when to incorporate suggestions and no controls are reported. The circularity concern is confined to the response-quality improvement claims. Section 3.2 explicitly defines a 'Self-Evaluation Procedure' in which a Reconciliation Agent applies the Recommender's suggestions and the revised response is re-scored by the same Scorer Agent that generated the input scores. Section 4.1 reports a 51% 'real score improvement by doctors' over the 37% estimate, but Figure 6 and the surrounding text indicate this live improvement is also computed with the same Scorer Agent, not by human raters or an external benchmark. Because the Scorer was validated against 80 held-out human annotations, the scores are not pure definitions, but the validation was on original responses only, and the recommendations targeted the same scorer's criteria. The independent patient-review outcome keeps the overall circularity moderate rather than total, so a score of 4 reflects partial self-confirmation rather than a fully forced derivation.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters are fitted in a mathematical sense; the system's DSPy prompts are optimized on 20 training examples, but the fitted prompts are not released. The main implicit assumptions are the validity of the scoring axes and the scorer as a quality measure.

assumptions (4)
  • domain assumption The 17 quality measures define presentation quality and are causally linked to patient satisfaction.
    Section 3.1 and Figure 3 rely on correlations from 100 annotated pairs to rank metrics, but the measures themselves are chosen by the authors and the correlation is not established as causal.
  • domain assumption The Scorer Agent's automatically optimized scores are valid proxies for human-judged quality.
    Validation in Figure 4 is against 80 held-out annotations for selected metrics, with mixed agreement; for many binary metrics the annotators' first-round kappa was below 0.5.
  • domain assumption The Reconciliation Agent's revised responses faithfully simulate how doctors apply recommendations.
    Section 3.2 uses this to estimate improvement (37%), but doctors in deployment may use suggestions differently; the paper later says doctors actually get higher scores (51%).
  • domain assumption Patients cannot judge medical accuracy and therefore base satisfaction on presentation quality.
    Used in the introduction to motivate the system; supported only by citations (Martin et al., 2005; Reis et al., 2024), not by data in this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dr.Copilot: A Multi-Agent Prompt Optimized Assistant for Improving Patient-Doctor Communication in Romanian." pith.science (2026). https://pith.science/paper/3I7O4JOS

@misc{pith2026250711299,
  author       = {Pith},
  title        = {Pith review of: Dr.Copilot: A Multi-Agent Prompt Optimized Assistant for Improving Patient-Doctor Communication in Romanian},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3I7O4JOS}},
  note         = {Machine review of arXiv:2507.11299}
}
read the original abstract

Text-based telemedicine has become increasingly common, yet the quality of medical advice in doctor-patient interactions is often judged more on how advice is communicated rather than its clinical accuracy. To address this, we introduce Dr. Copilot , a multi-agent large language model (LLM) system that supports Romanian-speaking doctors by evaluating and enhancing the presentation quality of their written responses. Rather than assessing medical correctness, Dr. Copilot provides feedback along 17 interpretable axes. The system comprises of three LLM agents with prompts automatically optimized via DSPy. Designed with low-resource Romanian data and deployed using open-weight models, it delivers real-time specific feedback to doctors within a telemedicine platform. Empirical evaluations and live deployment with 41 doctors show measurable improvements in user reviews and response quality, marking one of the first real-world deployments of LLMs in Romanian medical settings.

Figures

Figures reproduced from arXiv: 2507.11299 by the authors.

Figure 1
Figure 1. Dr.Copilot evaluates a doctor’s response to a [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overall diagram of Dr.Copilot. A patient-doctor interaction is scored and recommendations are generated [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Correlation between each quality measure for [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Comparison between base models and prompt-optimized variants across selected quality measures for the [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Evaluation of the Recommender Agent, by incorporating recommendations through a Reconciliator Agent in the original responses and re-scoring [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Score improvement across selected metrics by [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 8
Figure 8. Figure 8: The distribution of across classes for the bi [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: The distribution of values for Likert-style [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: Example of suggestions (in Romanian on the [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

49 extracted references · 25 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Daniel Adiwardana, Minh-Thang Luong, David R So, Jamie Hall, Noah Fiedel, and 1 others. 2020. Towards a human-like open-domain chatbot . arXiv preprint arXiv:2001.09977

    arXiv 2020

  4. [4]

    Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion. 2024. Jailbreaking leading safety-aligned llms with simple adaptive attacks . arXiv preprint arXiv:2404.02151

    arXiv 2024

  5. [5]

    Rahul K Arora, Jason Wei, Rebecca Soskin Hicks, Preston Bowman, Joaquin Qui \ n onero-Candela, and 1 others. 2025. Healthbench: Evaluating large language models towards improved human health . arXiv preprint arXiv:2505.08775

    arXiv 2025

  6. [6]

    Kellen Cheng, Anna Lisa Gentile, Pengyuan Li, Chad DeLuca, and Guang-Jie Ren. 2024. Don’t be my Doctor! Recognizing Healthcare Advice in Large Language Models . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 970--980

    work page 2024

  7. [7]

    Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and 1 others. 2017. https://proceedings.neurips.cc/paper_files/paper/2017/file/d5e2c0adad503c91f91df240d0cd4e49-Paper.pdf Deep Reinforcement Learning from Human Preferences . In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc

    work page 2017

  8. [8]

    Jacob Cohen. 1960. A coefficient of agreement for nominal scales . Educational and psychological measurement, 20(1):37--46

    work page 1960

Show all 49 references
  1. [9]

    Gemini Team . 2025. https://arxiv.org/abs/2312.11805 Gemini: A Family of Highly Capable Multimodal Models . Preprint, arXiv:2312.11805

    2025 arXiv

  2. [10]

    Gemma Team , Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, and 1 others. 2025. Gemma 3 technical report . arXiv preprint arXiv:2503.19786

    2025 arXiv

  3. [11]

    Google. 2025. MedGemma Hugging Face . https://huggingface.co/collections/google/medgemma-release-680aade845f90bec6a3f60c4. Accessed: 2025-07-4

    2025

  4. [12]

    Karthik Gopalakrishnan, Behnam Hedayatnia, Qinlang Chen, Anna Gottardi, Sanjeev Kwatra, and 1 others. 2019. https://doi.org/10.21437/Interspeech.2019-3079 Topical-Chat: Towards Knowledge-Grounded Open-Domain Conversations . In Proc. Interspeech 2019, pages 1891--1895

    2019 doi

  5. [13]

    Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, and 1 others. 2024. A survey on llm-as-a-judge. arXiv preprint arXiv:2411.15594

    2024 arXiv

  6. [14]

    Fenfei Guo, Angeliki Metallinou, Chandra Khatri, Anirudh Raju, Anu Venkatesh, and 1 others. 2018. Topic-based evaluation for conversational bots . arXiv preprint arXiv:1801.03622

    2018 arXiv

  7. [15]

    Tianyu Han, Lisa C Adams, Jens-Michalis Papaioannou, Paul Grundmann, Tom Oberhauser, and 1 others. 2023. MedAlpaca--an open-source collection of medical conversational AI models and training data . arXiv preprint arXiv:2304.08247

    2023 arXiv

  8. [16]

    Kizilcec, Dominic DiFranzo, Zhila Aghajari, Hannah Mieczkowski, and 1 others

    Jess Hohenstein, Rene F. Kizilcec, Dominic DiFranzo, Zhila Aghajari, Hannah Mieczkowski, and 1 others. 2023. https://doi.org/10.1038/s41598-023-30938-9 Artificial intelligence in communication impacts language and social relationships . Scientific Reports, 13(1):5487

    2023 doi

  9. [17]

    Kaiyu Huang, Fengran Mo, Xinyu Zhang, Hongliang Li, You Li, and 1 others. 2024. A survey on large language models with multilingualism: Recent advances and new frontiers . arXiv preprint arXiv:2405.10936

    2024 arXiv

  10. [18]

    Tim Isbister, Fredrik Carlsson, and Magnus Sahlgren. 2021. Should we stop training more monolingual models, and simply use machine translation instead? arXiv preprint arXiv:2104.10441

    2021 arXiv

  11. [19]

    Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, and 1 others. 2024. DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines

    2024

  12. [20]

    Dr LLM, what do I have?

    Wojciech Kusa, Edoardo Mosca, and Aldo Lipani. 2023. “Dr LLM, what do I have?”: The Impact of User Beliefs and Prompt Formulation on Health Diagnoses . In Proceedings of the Third Workshop on NLP for Medical Conversations, pages 13--19

    2023

  13. [21]

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, and 1 others. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention . In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles

    2023

  14. [22]

    Yunxiang Li, Zihan Li, Kai Zhang, Ruilong Dan, Steve Jiang, and 1 others. 2023. Chatdoctor: A medical chat model fine-tuned on a large language model meta-ai (llama) using medical domain knowledge . Cureus, 15(6)

    2023

  15. [23]

    Rensis Likert. 1932. A technique for the measurement of attitudes. Archives of psychology

    1932

  16. [24]

    Fenglin Liu, Jinge Wu, Hongjian Zhou, Xiao Gu, Soheila Molaei, and 1 others. 2025. RiskAgent: Autonomous Medical AI Copilot for Generalist Risk Prediction . arXiv preprint arXiv:2503.03802

    2025

  17. [25]

    Leslie R Martin, Summer L Williams, Kelly B Haskard, and M Robin DiMatteo. 2005. The challenge of patient adherence . Therapeutics and clinical risk management, 1(3):189--199

    2005

  18. [26]

    Mihai Masala, Denis Ilie-Ablachim, Alexandru Dima, Dragos Georgian Corlatescu, Miruna-Andreea Zavelca, Ovio Olaru, Simina-Maria Terian, Andrei Terian, Marius Leordeanu, Horia Velicu, Marius Popescu, Mihai Dascalu, and Traian Rebedea. 2024. https://doi.org/10.18653/v1/2024.find...

    2024 doi

  19. [27]

    Mihai Masala, Stefan Ruseti, and Mihai Dascalu. 2020. https://doi.org/10.18653/v1/2020.coling-main.581 R o BERT -- A R omanian BERT Model . In Proceedings of the 28th International Conference on Computational Linguistics, pages 6626--6637, Barcelona, Spain (Online). Internatio...

    2020 doi

  20. [28]

    Shikib Mehri and Maxine Eskenazi. 2020. https://doi.org/10.18653/v1/2020.sigdial-1.28 Unsupervised Evaluation of Interactive Dialog with D ialo GPT . In Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 225--235, 1st virtual ...

    2020 doi

  21. [29]

    Subhabrata Mukherjee, Paul Gamble, Markel Sanz Ausin, Neel Kant, Kriti Aggarwal, and 1 others. 2024. Polaris: A safety-focused llm constellation architecture for healthcare . arXiv preprint arXiv:2403.13313

    2024 arXiv

  22. [30]

    Hellina Hailu Nigatu, Atnafu Lambebo Tonja, Benjamin Rosman, Thamar Solorio, and Monojit Choudhury. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.983 The Zeno ' s Paradox of `Low-Resource' Languages . In Proceedings of the 2024 Conference on Empirical Methods in Natural La...

    2024 doi

  23. [31]

    OpenAI. 2024. https://arxiv.org/abs/2303.08774 GPT-4 Technical Report . Preprint, arXiv:2303.08774

    2024 arXiv

  24. [32]

    Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.225 Red teaming language models with language models . In Proceedings of the 2022 Conference on...

    2022 doi

  25. [33]

    Zachary J Peters, Jessica Lendon, Christine Caffrey, Kelly L Myrick, Mohsin Mahar, and 1 others. 2024. Telemedicine Use During the COVID-19 Pandemic by Office-based Physicians and Long-term Care Providers . National Health Statistics Reports, (210):10--15620

    2024

  26. [34]

    Traian Rebedea, Razvan Dinu, Makesh Narsimhan Sreedhar, Christopher Parisien, and Jonathan Cohen. 2023. https://doi.org/10.18653/v1/2023.emnlp-demo.40 N e M o Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails . In Proceedings of the 2023 ...

    2023 doi

  27. [35]

    Moritz Reis, Florian Reis, and Wilfried Kunde. 2024. Influence of believed AI involvement on the perception of digital medical advice . Nature Medicine, pages 1--3

    2024

  28. [36]

    Zhiyao Ren, Yibing Zhan, Baosheng Yu, Liang Ding, and Dacheng Tao. 2024. Healthcare copilot: Eliciting the power of general llms for medical consultation . arXiv preprint arXiv:2402.13408

    2024 arXiv

  29. [37]

    Joan Sargeant, Karen Mann, Douglas Sinclair, Cees Van der Vleuten, and Job Metsemakers. 2008. Understanding the influence of emotions and reflection upon multi-source feedback acceptance and use . Advances in Health Sciences Education, 13:275--288

    2008

  30. [38]

    Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, and 1 others. 2023. Towards understanding sycophancy in language models . arXiv preprint arXiv:2310.13548

    2023 arXiv

  31. [39]

    Jamin Shin, Peng Xu, Andrea Madotto, and Pascale Fung. 2020. Generating empathetic responses by looking ahead the user’s sentiment . In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7989--7993. IEEE

    2020

  32. [40]

    Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, and 1 others. 2023. Large language models encode clinical knowledge . Nature, 620(7972):172--180

    2023

  33. [41]

    Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, and 1 others. 2025. Toward expert-level medical question answering with large language models . Nature Medicine, pages 1--8

    2025

  34. [42]

    Richard L Street Jr, Gregory Makoul, Neeraj K Arora, and Ronald M Epstein. 2009. How does communication heal? Pathways linking clinician--patient communication to health outcomes . Patient education and counseling, 74(3):295--301

    2009

  35. [43]

    Brandon T Willard and R \'e mi Louf. 2023. Efficient guided generation for large language models . arXiv preprint arXiv:2307.09702

    2023 arXiv

  36. [44]

    Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, and 1 others. 2024. How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge ai safety by humanizing llms . In Proceedings of the 62nd Annual Meeting of the Association for Computational Ling...

    2024

  37. [45]

    Chen Zhang, Luis Fernando D'Haro, Yiming Chen, Malu Zhang, and Haizhou Li. 2024. https://doi.org/10.1609/aaai.v38i17.29923 A comprehensive analysis of the effectiveness of large language models as automatic dialogue evaluators . In Proceedings of the Thirty-Eighth AAAI Confere...

    2024 doi

  38. [46]

    Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2020. https://doi.org/10.18653/v1/2020.acl-demos.30 DIALOGPT : Large-scale generative pre-training for conversational response generation . In Proceeding...

    2020 doi

  39. [47]

    Xuejiao Zhao, Siyan Liu, Su-Yin Yang, and Chunyan Miao. 2025 a . A Smart Multimodal Healthcare Copilot with Powerful LLM Reasoning . arXiv preprint arXiv:2506.02470

    2025 arXiv

  40. [48]

    Xuejiao Zhao, Siyan Liu, Su-Yin Yang, and Chunyan Miao. 2025 b . MedRAG: Enhancing Retrieval-augmented Generation with Knowledge Graph-Elicited Reasoning for Healthcare Copilot . In Proceedings of the ACM on Web Conference 2025, pages 4442--4457

    2025

  41. [49]

    Hongjian Zhou, Fenglin Liu, Boyang Gu, Xinyu Zou, Jinfa Huang, Jinge Wu, Yiru Li, Sam S Chen, Peilin Zhou, Junling Liu, and 1 others. 2023. A survey of large language models in medicine: Progress, application, and challenge. arXiv preprint arXiv:2311.05112

    2023 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.