Improving Heart-Focused Medical Question Answering in LLMs via Variance-Aware Rubric Rewards with GRPO

Arash Ahmadi; Charles Nicholson; David Ebert; Mike Banad; Parisa Masnadi Khiabani; Sarah Sharif

REVIEW 4 major objections 7 minor 47 references

Improving Heart-Focused Medical Question Answering in LLMs via Variance-Aware Rubric Rewards with GRPO

T0 review · 4 major / 7 minor · reviewed 2026-07-12 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Continuous variance-aware rubric rewards lift a 14B model to near-frontier accuracy on heart medical QA under GRPO.

desk verdict Solid applied GRPO recipe for heart QA with real held-out gains and code; the SFT-only control is missing, so the reward-design claim is only partly isolated. read the letter →

arxiv 2606.05174 v1 pith:6HV7V3EK submitted 2026-04-17 cs.CL cs.AI

Arash Ahmadi , Parisa Masnadi Khiabani , Sarah Sharif , Charles Nicholson , David Ebert , Mike Banad This is my paper · ORCID

classification cs.CLcs.AI

keywords GroupRelativePolicyOptimizationrubric-basedrewardsmedicalquestionansweringheart-focusedclinicalreasoningvariance-awarerewardshapingLLM-as-judgeparameter-efficientfine-tuning

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

General-purpose language models are hard to deploy in real clinical settings because of privacy, cost, and edge constraints, so smaller models need better post-training. This paper shows that Group Relative Policy Optimization works for heart-focused medical question answering when the reward is built from multi-criterion clinical rubrics rather than a single overall score. The key move is to turn each criterion-level pass/fail judgment into a continuous, variance-aware reward that preserves partial credit, penalizes unsafe content, and scales with how hard the rubric is. On a held-out heart subset of HealthBench, the best such reward raises a 14B base model from 0.362 to 0.502 accuracy and from 0.532 to 0.668 F1, nearly matching a much larger 120B system while running on a single workstation GPU. The same design substantially outperforms the original fixed-weight and holistic Likert aggregations from the Rubrics-as-Rewards framework. A sympathetic reader cares because the result suggests that careful reward shaping, not just model size, can close much of the gap between local medical assistants and frontier models on multi-criteria clinical tasks.

What carries the argument

Variance-Aware Reward Framework: continuous functions (Complexity-aware power transform with logarithmic rubric-size bonus, and Hybrid base-plus-perfection bonus) that map signed criterion pass/fail outcomes into a scalar reward with non-zero within-group variance, partial credit, negative-criteria penalties, and modest complexity awareness.

What would settle it

A blinded physician re-grading of the same 500 heart HealthBench items that shows the GRPO models do not actually satisfy more rubric criteria than the base model, or a head-to-head where the same continuous rewards are driven by human criterion labels and the accuracy lift disappears.

Watch

Extended reading notes

Core claim

The paper establishes that continuous analytical rewards derived from criterion-level rubric outcomes, when used inside GRPO, give richer and more stable optimization signals than either weighted binary aggregation or a single overall Likert score, and that this variance-aware design produces large gains on held-out heart-related medical question answering for a 14B model.

Load-bearing premise

The binary yes/no decisions of a large language-model judge on each clinical criterion are accurate and unbiased enough that optimizing against them produces real gains on physician-derived labels.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The useful takeaway is practical, not foundational: continuous Hybrid/Complexity rewards under GRPO lift a LoRA-quantized Qwen3-14B from 0.362 Acc / 0.532 F1 to 0.502 / 0.668 on a held-out heart HealthBench slice (n=500), nearly matching GPT-OSS-120B, and clearly beat the original RaR Explicit/Implicit aggregations on the same pipeline.

What is actually new is the reward engineering for GRPO’s within-group normalization—power transform, negative-ratio penalty, log complexity bonus, and a hybrid perfection jump—plus a clean heart-filter + MedGemma reasoning-trace pipeline and released code/data. Equations 12–16 are explicit; they report CIs, McNemar tests, training curves, multi-model baselines, and ablations that put RaR-style aggregation well below their continuous variants. That is real engineering evidence for people who care about local medical assistants under hardware constraints.

The soft spot that matters is attribution, not fabrication. Every GRPO row sits on an SFT warm-start that teaches the structured tags required for criterion judging; there is no SFT-only (or format-only) HealthBench row. So the jump from Base to GRPO-COMPLEXITY mixes format learning, synthetic traces, and the variance-aware transforms. The RaR ablations share that SFT base, so they isolate aggregation style better than they isolate “SFT vs RL.” Hand-set constants (α=1.2, β=0.2, λ=0.5, B/P) and an unvalidated GPT-OSS-120B criterion judge are secondary but real; final metrics use physician-derived labels, so the result is not circular by construction, but judge agreement on these heart rubrics is not shown.

Math and citations look fine—DeepSeekMath/GRPO, RaR, HealthBench, MedGemma are used appropriately. This is for people doing medical LLM post-training and privacy-preserving deployment, not for someone hunting a new RL theorem. I would send it to peer review; ask for an SFT-only control and a small physician agreement check on the judge. Worth engaging if you work in this lane.

Referee Report

4 major / 7 minor

Summary. The paper post-trains Qwen3-14B with LoRA under Group Relative Policy Optimization for heart-focused medical QA, using RaR-Medicine rubrics filtered to cardiac themes and criterion-level LLM judging (GPT-OSS-120B). It proposes a Variance-Aware Reward Framework that replaces RaR Explicit (weighted binary sum) and Implicit (holistic Likert) aggregation with continuous rewards: a Complexity-aware power transform with logarithmic rubric-size bonus (Eqs. 12–15) and a Hybrid partial-credit plus perfection-bonus form (Eq. 16). After an SFT format warm-start, GRPO is run with G=6 samples per prompt. On a held-out heart-related HealthBench subset (n=500, seed 42), COMPLEXITY reaches 0.502 accuracy / 0.668 F1 versus base 0.362 / 0.532, near GPT-OSS-120B (0.508 / 0.674), with McNemar tests and RaR ablations showing large gains over RaR-Explicit/Implicit (+38.7% vs +9.4%/+13.8% relative accuracy).

Significance. If the attribution holds, the work offers a practical, hardware-constrained recipe for rubric-aligned RL on multi-criteria clinical QA where exact verifiers are unavailable: continuous partial-credit rewards that preserve within-group variance for GRPO, plus a reproducible heart-filtered pipeline with code and data released. Closing much of the gap from a 14B local model to a 120B-class baseline on HealthBench heart items is of real deployment interest for privacy-preserving medical assistants. Strengths include held-out evaluation with CIs and McNemar tests, explicit RaR ablations, training dynamics, multi-model baselines, and open artifacts. The distinctive methodological claim—that continuous variance-aware shaping beats RaR aggregation under GRPO—is the main contribution beyond applying GRPO to medical dialogue.

major comments (4)

§1.2 Tables 2–3 and Abstract attribute the full Base→COMPLEXITY jump (+0.140 Acc, +0.137 F1) to variance-aware GRPO rewards, but §3.3.1 and Discussion state SFT is required for structured tags and that reported gains reflect the end-to-end pipeline. There is no SFT-only (or SFT + fixed-format decoding) row on the same n=500 HealthBench split. RaR-Explicit/Implicit ablations share the SFT warm-start and therefore isolate reward shaping from each other, not SFT from GRPO. Without an SFT-only control, the headline Base comparison conflates format/imitation learning (including MedGemma synthetic traces) with continuous reward design. Please add SFT-only metrics on the identical evaluation split and restate claims so that (i) Base→pipeline and (ii) reward-form comparisons are cleanly separated.
§3.4.2–3.4.5 and Discussion rest the optimization target on binary criterion decisions from GPT-OSS-120B, justified mainly by prior RaR results and Chatbot Arena Elo, not by agreement with physicians on the heart rubrics used here. Final metrics use physician-derived HealthBench labels, which avoids pure train–eval circularity, but residual judge–label correlation and systematic bias remain unquantified. A load-bearing premise of the central claim is that optimizing these judge decisions yields genuine clinical-rubric gains. At minimum, report judge–physician (or dual-judge) agreement on a stratified sample of criteria/completions, or a sensitivity analysis swapping judge models; otherwise qualify claims as “judge-aligned rubric satisfaction” rather than clinical correctness.
§3.6 and Table 2 report Accuracy/Precision/Recall/F1 against “physician-derived binary labels,” while HealthBench and the training pipeline are multi-criteria rubric systems (Fig. 2, §3.4). How binary labels are obtained from multi-criterion rubrics (overall pass threshold, majority criteria, primary criterion, etc.) is not specified. This definition is load-bearing for interpreting 0.502 accuracy and for comparing to GPT-OSS-120B/Kimi-K2. Please define the label construction explicitly and, if feasible, also report mean rubric score / criterion-level satisfaction rates so results are not solely dependent on an underspecified binarization.
§3.4.4 treats α=1.2, β=0.2, λ=0.5, B=15, P=5, nmax=25 as “theory-informed design constants” without sensitivity or ablation. The paper’s distinctive claim is that the continuous Complexity/Hybrid forms (Eqs. 14–16) drive the large gap over RaR aggregation. If performance is brittle to these free parameters, that claim is overstated. A small grid or leave-one-knob study (e.g., α∈{1.0,1.2,1.5}, β∈{0,0.2,0.5}, with/without perfection bonus) on a development slice would substantially strengthen the reward-design contribution.

minor comments (7)

Fig. 1 caption mentions Hybrid and Complexity formulas; the figure body also shows R_comp / R_hybrid fragments that do not fully match Eqs. 14–16—align notation.
§3.2.2 uses <start working out>/<end working out> while Fig. 1 and elsewhere use <reasoning>…</reasoning>; standardize the output schema throughout.
Table 1 uses w for points while §3.1 uses wk; keep a single weight symbol.
Supplementary figures are heavily referenced for dataset stats and radar charts; ensure the main text is self-contained for the primary accuracy/F1 claims.
Discussion cites AlphaGo as [45] after earlier [30]; check reference numbering consistency for Silver et al.
Clarify whether HealthBench multi-turn structure is collapsed to single-turn prompts for evaluation, and how that interacts with heart filtering.
Report wall-clock and approximate judge-call counts more precisely; 26 h/run is useful but cost of criterion-level judging is central to reproducibility claims.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: held-out HealthBench metrics are independent of the training reward construction; gains are empirical, not definitional.

full rationale

The paper’s load-bearing claim is an empirical comparison: SFT+GRPO with continuous variance-aware rewards (Complexity/Hybrid) raises Qwen3-14B accuracy/F1 on a held-out heart-related HealthBench subset (n=500, physician-derived labels) relative to the base model and to RaR Explicit/Implicit aggregations. Training rewards are built from criterion-level LLM-judge binary decisions on RaR-Medicine rubrics (Eqs. 8–16), then optimized with GRPO group-normalized advantages (Eqs. 4–7). Evaluation uses a different corpus (HealthBench) and physician-derived binary labels, not the training judge scores. Hyperparameters (α, β, λ, B, P, n_max) are stated as fixed design constants, not fitted to HealthBench. There is no self-definitional loop (reward ≠ reported metric by construction), no fitted-input-called-prediction, no load-bearing self-citation uniqueness theorem, and no renaming of a known result as a derivation. Residual concerns (LLM-judge proxy quality; missing SFT-only control) are experimental-validity issues, not circularity under the stated criteria. The derivation chain is self-contained against an external benchmark.

Assumptions & free parameters 6 free parameters · 5 assumptions · 2 invented entities

The central claim rests on standard RL/LLM machinery plus several hand-set reward constants and domain assumptions about judge reliability and heart filtering. No new physical entities are postulated; the invented pieces are reward functional forms and the filtered training pipeline. Free parameters are design constants, not data-fitted coefficients, but the claim still depends on their particular values.

free parameters (6)

reward exponent α = 1.2
Set to 1.2 in the Complexity reward to mildly sharpen high scores; hand-chosen, not learned.
complexity coefficient β = 0.2
Log-rubric-size bonus strength in Eq. 12/14; hand-chosen modest value.
negative penalty λ = 0.5
Scales normalized negative ratio before nonlinear transform; design constant.
base scale rbase / Hybrid B and P = rbase=20, B=15, P=5
rbase=20; Hybrid splits into continuous base B=15 and perfection bonus P=5 (and 4.5=0.3B negative cap).
nmax rubric normalization = 25
Maximum rubric count used in log complexity term; set to 25 from training data scale.
GRPO group size G and LoRA rank r = G=6, r=16
Sampling and adapter capacity choices that affect optimization dynamics and capacity.

assumptions (5)

domain assumption GRPO group-normalized advantages require non-degenerate within-group reward variance to produce learning signal.
Stated as design principle in §3.3.2 and §3.4.1; standard for GRPO but load-bearing for continuous rewards.
domain assumption An LLM judge can independently and reliably mark each rubric criterion present/absent for medical completions.
§3.4.2 and Discussion; justified by RaR and Arena Elo, not by new physician agreement on these heart rubrics.
domain assumption Heart-related filtering (MedGemma classifier / keywords) yields a coherent cardiac QA training distribution without severe label noise.
§1.1.1 data curation; errors here would mis-scope both training and evaluation.
ad hoc to paper SFT is only a format warm-start; reported gains can be attributed primarily to rubric-aligned GRPO in the end-to-end pipeline.
Discussion §2; main tables do not fully isolate SFT-only vs GRPO on the same metrics.
standard math Autoregressive policy factorization and clipped importance-ratio GRPO objective are valid for this generation setting.
Eqs. 1–7; standard RLHF/GRPO machinery.

invented entities (2)

Variance-Aware Reward Framework (Complexity-aware and Hybrid continuous rewards)
purpose: Map multi-criterion binary rubric outcomes into continuous scalar rewards with partial credit, negative penalties, and optional complexity bonus for GRPO.
Core methodological object of the paper; defined analytically in §3.4.3–3.4.5 rather than as a physical entity.
Heart-filtered RaR-Medicine + synthetic MedGemma reasoning traces pipeline
purpose: Create SFT/GRPO splits specialized to cardiac questions with structured reasoning tags.
Data-construction contribution enabling the experiments; not independently validated as a clinical corpus standard.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Improving Heart-Focused Medical Question Answering in LLMs via Variance-Aware Rubric Rewards with GRPO." pith.science (2026). https://pith.science/paper/6HV7V3EK

@misc{pith2026260605174,
  author       = {Pith},
  title        = {Pith review of: Improving Heart-Focused Medical Question Answering in LLMs via Variance-Aware Rubric Rewards with GRPO},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6HV7V3EK}},
  note         = {Machine review of arXiv:2606.05174}
}

read the original abstract

Large Language Models (LLMs) have shown strong promise in healthcare applications. Yet deploying general-purpose models in real-world settings remains difficult due to data privacy constraints, inference costs, and limited suitability for edge or on-device use. These challenges motivate the development of smaller, more efficient models that require robust post-training strategies to ensure reliable medical reasoning. In this work, we investigate Group Relative Policy Optimization (GRPO) for post-training LLMs on heart-focused medical question answering with rubric-based supervision derived from RaR-Medicine. We propose a Variance-Aware Reward Framework that extends the Explicit Aggregation and Implicit Aggregation strategies of Rubrics as Rewards by replacing weighted binary criterion aggregation and single overall Likert-style scoring with continuous analytical reward functions derived from criterion-level rubric outcomes. This formulation provides richer optimization signals for feedback that is sparse, multi-criteria, and difficult to verify automatically, and enables more stable on-policy reinforcement learning. On a held-out heart-related subset of HealthBench, our best GRPO variant improves accuracy from 0.362 to 0.502 and F1 from 0.532 to 0.668 relative to the Qwen3-14B base model, while remaining competitive with GPT-OSS-120B (0.508 accuracy, 0.674 F1). Our findings show that carefully designed rubric-based rewards provide a practical strategy for improving heart-focused medical question answering in LLMs, with potential to extend to other rubric-based tasks.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 10 linked inside Pith

[1]

Topol, E. J. High-performance medicine: the convergence of human and artificial intelligence. Nature Medicine25, 44–56 (2019)

2019
[2]

& Albadawy, M

Khalifa, M. & Albadawy, M. Artificial intelligence for clinical prediction: exploring key domains and essential functions.Computer Methods and Programs in Biomedicine Update5, 100148 (2024)

2024
[3]

Nature(2017)

Esteva, A.et al.Dermatologist-level classification of skin cancer with deep neural networks. Nature(2017)

2017
[4]

Gulshan, V.et al.Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs.JAMA(2016). 24

2016
[5]

M.et al.International evaluation of an ai system for breast cancer screening

McKinney, S. M.et al.International evaluation of an ai system for breast cancer screening. Nature(2020)

2020
[6]

& Ranganath, R

Huang, K., Altosaar, J. & Ranganath, R. Clinicalbert: Modeling clinical notes and predicting hospital readmission.arXiv preprint arXiv:1904.05342(2019)

arXiv 1904
[7]

Jerfy, A.et al.The growing impact of natural language processing in public health and healthcare: A narrative review.Frontiers in Public Health(2024)

2024
[8]

Eguia, H.et al.Clinical decision support and natural language processing in health care: A systematic review.Journal of Medical Internet Research(2024)

2024

Show all 47 references

[9]

Liu, L.et al.Using natural language processing to extract information from clinical text for populating clinical registries: a review.Journal of the American Medical Informatics Association(2025)

2025
[10]

J.et al.Large language models in medicine.Nature Medicine29, 1930–1940 (2023)

Thirunavukarasu, A. J.et al.Large language models in medicine.Nature Medicine29, 1930–1940 (2023)

1930
[11]

Naghavi, M.et al.Global burden of 292 causes of death in 204 countries and territories and 660 subnational locations, 1990–2023: a systematic analysis for the global burden of disease study 2023.The Lancet406, 1811–1872 (2025)

1990
[13]

Li, T.et al.Cancergpt for few shot drug pair synergy prediction using large pretrained language models.NPJ Digital Medicine7, 40 (2024)

2024
[14]

Weber, S.et al.Using a fine-tuned large language model for symptom-based depression evaluation.npj Digital Medicine8, 598 (2025)

2025
[15]

Su, X.et al.Large language models driven neural architecture search for universal and lightweight disease diagnosis on histopathology slide images.npj Digital Medicine8, 682 (2025)

2025
[16]

Lopez, I.et al.Clinical entity augmented retrieval for clinical information extraction, npj digital medicine 8 (2025)

2025
[17]

Wada, A.et al.Retrieval-augmented generation elevates local llm quality in radiology contrast media consultation.NPJ Digital Medicine8, 395 (2025)

2025
[18]

Hao, Y.et al.Personalizing prostate cancer education for patients using an ehr-integrated llm agent.NPJ Digital Medicine8, 770 (2025)

2025
[19]

Zheng, H.et al.Kt-llm: an evidence-grounded and sequence text framework for auditable kidney transplant modeling.npj Digital Medicine(2026). 25

2026
[20]

& Bertsimas, D

Petridis, P., Margaritis, G., Stoumpou, V. & Bertsimas, D. Holistic ai in medicine; improved performance and explainability.npj Digital Medicine(2026)

2026
[21]

S., Barto, A

Sutton, R. S., Barto, A. G.et al. Reinforcement learning: An introduction, vol. 1 (MIT press Cambridge, 1998)

1998
[22]

Watkins, C. J. & Dayan, P. Q-learning.Machine learning8, 279–292 (1992)

1992
[23]

Wang, Y.-H., Li, T.-H. S. & Lin, C.-J. Backward q-learning: The combination of sarsa algorithm and q-learning.Engineering Applications of Artificial Intelligence26, 2184–2193 (2013)

2013
[24]

Mnih, V.et al.Playing atari with deep reinforcement learning.arXiv preprint arXiv:1312.5602 (2013)

2013 arXiv
[25]

& Silver, D

Van Hasselt, H., Guez, A. & Silver, D. Deep reinforcement learning with double q-learning. In Proceedings of the AAAI conference on artificial intelligence, vol. 30 (2016)

2016
[26]

Halat, S., Ebadzadeh, M. M. & Amani, K. Modified double-dqn: addressing stability. In2024 11th International Symposium on Telecommunications (IST), 697–702 (IEEE, 2024)

2024
[27]

& Kautz, J

Babaeizadeh, M., Frosio, I., Tyree, S., Clemons, J. & Kautz, J. Reinforcement learning through asynchronous advantage actor-critic on a gpu.arXiv preprint arXiv:1611.06256(2016)

2016 arXiv
[28]

& Moritz, P

Schulman, J., Levine, S., Abbeel, P., Jordan, M. & Moritz, P. Trust region policy optimization. InInternational conference on machine learning, 1889–1897 (PMLR, 2015)

2015
[29]

& Klimov, O

Schulman, J., Wolski, F., Dhariwal, P., Radford, A. & Klimov, O. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347(2017)

2017 arXiv
[30]

Silver, D.et al.Mastering the game of go without human knowledge.nature550, 354–359 (2017)

2017
[31]

Jumper, J.et al.Highly accurate protein structure prediction with alphafold.Nature596, 583–589 (2021)

2021
[32]

Shao, Z.et al.Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300(2024)

2024 arXiv
[33]

Guo, D.et al.Deepseek-r1 incentivizes reasoning in llms through reinforcement learning.Nature 645, 633–638 (2025)

2025
[34]

& Gabbrielli, M

Pennino, F., Raimondi, B., Rondelli, M., Gurioli, A. & Gabbrielli, M. From reasoning to code: Grpo optimization for underrepresented languages.arXiv preprint arXiv:2506.11027(2025)

2025 arXiv
[35]

& Zhou, J

Wang, J., Meng, F. & Zhou, J. Deeptrans: Deep reasoning translation via reinforcement learning.Transactions of the Association for Computational Linguistics14, 47–63 (2026). 26

2026
[36]

InInternational Conference on Medical Image Computing and Computer-Assisted Intervention, 337–347 (Springer, 2025)

Pan, J.et al.Medvlm-r1: Incentivizing medical reasoning capability of vision-language models (vlms) via reinforcement learning. InInternational Conference on Medical Image Computing and Computer-Assisted Intervention, 337–347 (Springer, 2025)

2025
[37]

& Ngo, C

Pham, T.-H. & Ngo, C. Rarl: Improving medical vlm reasoning and generalization with reinforce- ment learning and lora under data and hardware constraints.arXiv preprint arXiv:2506.06600 (2025)

2025 arXiv
[38]

Wei, J.et al.Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems35, 24824–24837 (2022)

2022
[39]

Zheng, L.et al.Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in neural information processing systems36, 46595–46623 (2023)

2023
[40]

Gunjal, A.et al.Rubrics as rewards: Reinforcement learning beyond verifiable domains.arXiv preprint arXiv:2507.17746(2025)

2025 arXiv
[41]

Giuffr` e, M.et al.Expert of experts verification and alignment (eval) framework for large language models safety in gastroenterology.NPJ Digital Medicine8, 242 (2025)

2025
[42]

Croxford, E.et al.Evaluating clinical ai summaries with large language models as judges.npj Digital Medicine8, 640 (2025)

2025
[43]

Sellergren, A.et al.Medgemma technical report.arXiv preprint arXiv:2507.05201(2025)

2025 arXiv
[44]

K.et al.Healthbench: Evaluating large language models towards improved human health.arXiv preprint arXiv:2505.08775(2025)

Arora, R. K.et al.Healthbench: Evaluating large language models towards improved human health.arXiv preprint arXiv:2505.08775(2025)

2025 arXiv
[45]

R., Beck, A

Granter, S. R., Beck, A. H. & Papke Jr, D. J. Alphago, deep learning, and the future of the human microscopist.Archives of pathology & laboratory medicine141, 619–621 (2017)

2017
[46]

InForty-first International Conference on Machine Learning(2024)

Chiang, W.-L.et al.Chatbot arena: An open platform for evaluating llms by human preference. InForty-first International Conference on Machine Learning(2024)

2024
[47]

& Leibler, R

Kullback, S. & Leibler, R. A. On information and sufficiency.The annals of mathematical statistics22, 79–86 (1951)

1951
[48]

Moon, S.et al.A latency processing unit: A latency-optimized and highly scalable processor for large language model inference.IEEE Micro44, 17–33 (2024). 27

2024

Pith tools

Reviewed July 12, 2026 · model on record in the stance chip above.

[1] [1]

Topol, E. J. High-performance medicine: the convergence of human and artificial intelligence. Nature Medicine25, 44–56 (2019)

2019

[2] [2]

& Albadawy, M

Khalifa, M. & Albadawy, M. Artificial intelligence for clinical prediction: exploring key domains and essential functions.Computer Methods and Programs in Biomedicine Update5, 100148 (2024)

2024

[3] [3]

Nature(2017)

Esteva, A.et al.Dermatologist-level classification of skin cancer with deep neural networks. Nature(2017)

2017

[4] [4]

Gulshan, V.et al.Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs.JAMA(2016). 24

2016

[5] [5]

M.et al.International evaluation of an ai system for breast cancer screening

McKinney, S. M.et al.International evaluation of an ai system for breast cancer screening. Nature(2020)

2020

[6] [6]

& Ranganath, R

Huang, K., Altosaar, J. & Ranganath, R. Clinicalbert: Modeling clinical notes and predicting hospital readmission.arXiv preprint arXiv:1904.05342(2019)

arXiv 1904

[7] [7]

Jerfy, A.et al.The growing impact of natural language processing in public health and healthcare: A narrative review.Frontiers in Public Health(2024)

2024

[8] [8]

Eguia, H.et al.Clinical decision support and natural language processing in health care: A systematic review.Journal of Medical Internet Research(2024)

2024