Pith. sign in

REVIEW 4 major objections 4 minor 56 references

HRIPBench: Benchmarking LLMs in Harm Reduction Information Provision to Support People Who Use Drugs

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Current state-of-the-art LLMs cannot reliably deliver accurate, safe harm-reduction information to people who use drugs, and some answers carry severe safety risks—even when the models are given retrieved domain knowledge.

desk verdict HRIPBench is a genuinely first benchmark for harm reduction information provision, but its GPT-4o-mini-generated ground truth is unvalidated and one of its own examples misreads the label — the qualitative finding likely holds, the specific numbers don't yet. read the letter →

arxiv 2507.21815 v1 pith:AVNRQ6CX submitted 2025-07-29 cs.CL cs.CY

Kaixuan Wang , Chenxin Diao , Jason T. Jacques , Zhongliang Guo , Shuai Zhao This is my paper
classification cs.CLcs.CY
keywords largelanguagemodelsharmreductionbenchmarkpeoplewhousedrugssafetyevaluationretrieval-augmentedgenerationpolysubstancehealthinformation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

HRIPBench asks whether today's large language models can be trusted to answer practical safety questions from people who use drugs: whether an activity is safe, what dose or timing values are correct, and how risky mixing two substances is. To test this, the authors built HRIP-Basic, a dataset of 2,160 question-answer-evidence pairs drawn from four harm-reduction sources, and scored 11 models under two conditions: asking directly and asking with retrieved domain knowledge supplied. The paper's central finding is that even the strongest models fall short, with accuracy below 60% on quantitative questions in many settings, and with some answers that would put a person at risk, such as under-calling a dangerous drug combination. The authors conclude that LLMs are not yet fit for autonomous use in harm-reduction information provision, and that deploying them without careful constraints could cause negative health outcomes.

What carries the argument

The engine of the evaluation is HRIP-Basic: 2,160 question-answer-evidence triples built by using GPT-4o-mini to extract questions and verbatim source evidence from four harm-reduction information sources, with polysubstance-risk items generated by a rule-based script from a structured interaction chart. Each triple is one of three task types: safety boundary check (yes/no), quantitative value (number with unit), and polysubstance-use risk level. The benchmark then compares two conditions per model: an Instruction scheme, where the model answers from its own knowledge following a constrained prompt, and a retrieval-augmented generation (RAG) scheme, where the model receives the top-3 retrieved chunks from the same sources via mixed dense and sparse retrieval fused by Reciprocal Rank Fusion. Scoring separates the directive from its explanation, using response rate, classification metrics, and tolerance bands of 0–50% for numbers, plus directional risk categories of underestimation, severe underestimation, and overestimation.

What would settle it

Have two independent addiction-medicine clinicians manually score a random sample of the 2,160 ground-truth pairs against the stated sources and compare disagreements with the benchmark's labels; a material disagreement rate would show the safety conclusions rest on generated labels rather than clinical consensus.

Watch

Extended reading notes

Core claim

The paper's claim is that current state-of-the-art LLMs, evaluated on HRIPBench, cannot reliably provide accurate harm-reduction information to people who use drugs and sometimes produce answers that are actively unsafe. On the 1,638 safety-boundary checks, no model exceeded 90% accuracy in the instruction-only condition, and several fell into the 59–88% range; adding retrieved knowledge lifted most models but left the best at 95%. On 372 quantitative questions, accuracy stayed below roughly 60% even with generous error tolerances, with concrete examples of overdose-relevant doses being overstated. On 150 polysubstance-risk questions, models typically overestimated risk, which the authors read as overcautious and unhelpful, but some severely underestimated it, as when mixing opioids with ketamine was called “low risk”; retrieval removed the worst underestimates while increasing plain underestimates. The paper therefore concludes that general-purpose LLMs are currently insufficient for this high-stakes use, and that their deployment in harm-reduction contexts should be cautiously constrained.

Load-bearing premise

The load-bearing premise is that the 2,160 question-answer pairs generated by GPT-4o-mini from four harm-reduction sources are a trustworthy gold standard, since no human expert validation or inter-annotator agreement is reported.

Editorial extensions

If this is right

  • Autonomous use of current LLMs as harm-reduction advisers is unsafe; any real deployment would need to restrict the model to verified content and keep a human in the loop.
  • Retrieval helps but is not a cure-all: giving models relevant domain passages improves safety-boundary accuracy and removes the most dangerous risk underestimates, yet quantitative answers remain unreliable enough to cause harm.
  • A model that refuses to answer can be as problematic as one that answers wrongly, since refusals block access to life-relevant information; response rate is therefore a safety metric, not just a usability one.
  • Evaluators of health LLMs should track the direction of errors, not only accuracy, because overestimating risk and underestimating risk have opposite public-health consequences.
  • Model scale and medical specialization do not guarantee safe harm-reduction answers; in the tested Qwen3 family, larger size was associated with lower accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the benchmark's gold standard is generated by an LLM from a small set of English-language harm-reduction sources, so a replication that adds clinician or peer validation of the 2,160 pairs would test whether the safety failures are failures against clinical consensus or only against these sources' wording.
  • Beyond the paper: because the underlying sources are largely UK-focused, the benchmark's difficulty ranking of models could shift when applied to other jurisdictions, languages, or community-specific harm-reduction knowledge, and no current claim covers those settings.
  • Beyond the paper: the finding that retrieval eliminates severe underestimates but increases mild ones suggests a testable design lever for future systems—different chunk sizes, reranking weights, or answer-time warnings might trade off false reassurance against overcautious refusal.
  • Beyond the paper: the paper reports harms in qualitative examples, such as a 50 mg overstatement of a ketamine dose; a quantitative risk-severity weighting of each error type would let stakeholders compare models by expected number of harmful outcomes rather than by raw accuracy.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The manuscript introduces HRIPBench, a benchmark for evaluating large language models (LLMs) in providing harm reduction information to people who use drugs (PWUD). The dataset HRIP-Basic contains 2,160 question-answer-evidence pairs covering three tasks: safety boundary checks (1,638), quantitative questions (372), and polysubstance use risks (150). Ground truth is constructed automatically: GPT-4o-mini extracts QA pairs from four harm reduction sources via structured prompts, and TripSit's interaction chart is parsed by a rule-based script. The authors evaluate 11 open-source, closed-source, and medical-domain LLMs under an instruction-only scheme and a retrieval-augmented generation (RAG) scheme, reporting response rates, answer accuracy (including tolerance bands for numeric answers), and directional risk misclassification. Across models, results show low numeric accuracy, mixed safety-boundary accuracy, and a tendency to overestimate polysubstance risk, with occasional severe underestimation. The authors conclude that current state-of-the-art LLMs are insufficiently accurate and can pose safety risks to PWUD in high-stakes cases.

Significance. If the dataset is valid, this is a timely and socially important benchmark filling a real gap: existing medical QA benchmarks omit the practical, non-clinical, judgment-heavy information needs of PWUD. The three-task design, the inclusion of a RAG comparison, the response-rate construct, and the asymmetric risk metrics distinguish under- and overestimation are valuable contributions. The paper evaluates a broad set of 11 models, which strengthens the qualitative pattern. However, the central result hinges on the validity of the automatically generated ground truth and the authority of the underlying sources. The manuscript currently provides no human validation, inter-annotator agreement, or expert adjudication of the labels, and one illustrative example is internally inconsistent with its own ground truth. With a demonstrated label audit and appropriate qualification of the claims, this would be a significant contribution to both NLP benchmarking and public health applications.

major comments (4)
  1. [Section 3.2] The entire benchmark rests on the 2,160 ground-truth pairs produced by GPT-4o-mini with no reported human validation, inter-annotator agreement, or expert adjudication. Because the headline conclusion is that models make safety-critical errors, silent extraction errors in the labels directly contaminate every accuracy and risk figure. The authors should either (a) conduct and report an expert audit of a representative sample (ideally all 150 polysubstance labels and a stratified sample of the other two tasks) and re-run the evaluation with corrected labels, or (b) clearly reframe the benchmark as measuring alignment with the four selected sources rather than absolute accuracy or safety.
  2. [Observation 3 (Section 4.2)] The ketamine example is internally inconsistent: the ground truth is stated as 'more than 150 mg', which is a lower bound, so o3-mini's '200 mg' is consistent with it. The text claims this is a '50 mg higher' overestimate, which only makes sense if 'more than 150 mg' is treated as a point estimate of 150 mg. This contradicts both the source expression and the tolerance-based evaluation logic described in Appendix A.2. This example should be corrected or replaced, and the regex-based tolerance evaluation should be checked for how it handles non-point numeric ground truths such as lower bounds and ranges.
  3. [Table 12 and Section 3.2] All 150 polysubstance risk labels come from TripSit, which the authors themselves describe as a community-driven harm reduction project. The 'Dangerous' label used in Observation 4 to demonstrate severe underestimation is therefore unadjudicated. The authors should report expert or clinical validation of the interaction chart labels, or at minimum a sensitivity analysis comparing a subset of the combinations against a second authoritative source. Without this, the 'severe safety risk' conclusions in the abstract and conclusion measure alignment with one community wiki rather than clinical consensus.
  4. [Sections 3.2 and 4.1] GPT-4o-mini generated the ground-truth labels and is also one of the evaluated models. Although this does not invalidate comparisons among the other ten models, it creates a possible inflation for GPT-4o-mini's scores and is a correctness risk: the dataset may implicitly favor the stylistic patterns of the label-generating model. The authors should acknowledge this dual role and report the main results with GPT-4o-mini excluded, or otherwise demonstrate that its performance is not systematically advantaged.
minor comments (4)
  1. [Table 3 caption] The caption says 'Values represent the accuracy of queries that received expected responses', but Section 3.5 defines this metric as response rate. Please correct the caption to match the terminology.
  2. [Appendix A.1] The semantic chunking thresholds (cosine similarity greater than 0.8, maximum chunk size of 350 words) and the RAG chunk size (250 tokens with 10% overlap) are introduced without justification or sensitivity analysis. A brief rationale or a small ablation would strengthen the reproducibility.
  3. [Section 3.4 (RAG scheme)] The RRF fusion score uses a constant k, but its value is never specified. Please report the constant (and any retrieval hyperparameters) so the experiments are fully reproducible.
  4. [Section 4.2, Observation 1] The statement that response rates are 'lower when providing numerical values' should be qualified: several models, including GPT-4.1 and GPT-4o-mini, have response rates above 98% on quantitative questions, whereas the dramatic drops are specific to certain models (e.g., Gemma-3-27B). Consider reporting per-model details rather than a general trend.

Circularity Check

1 steps flagged · score 2.0 of 10

No central circularity: the benchmark's ground truth is anchored to external harm-reduction sources. One mild self-reference exists: GPT-4o-mini both generated the gold-standard QA pairs and is one of the evaluated models, so its own score partly measures agreement with its own extraction distribution.

  1. self definitional [Section 3.2 (Building HRIP-Basic) and Section 4.1 (Experiment Setup, LLMs list)]
    "For categories of safety boundary check and quantitative questions, GPT-4o-mini ... is employed as a controlled information extraction tool ... GPT-4o-mini is instructed to extract data from each knowledge chunk ... we used OpenAI models (GPT-4o-mini, GPT-4.1, o3-mini, and o4-mini)."

    The same model that constructs the gold-standard answers for two of the three task types is later scored against those answers. For GPT-4o-mini, 'accuracy' reduces partly to consistency between the model's test-time output and the model's earlier dataset-construction output under similar format instructions ('Yes/No, followed by a brief explanation'). Because the extraction was constrained to verbatim source evidence, this is not a full reduction to a fitted parameter, but it is a mild self-definitional loop: the reference label for GPT-4o-mini is a GPT-4o-mini production, and no human validation or inter-annotator check is reported to break the loop.

full rationale

Apart from the GPT-4o-mini dual role, the evaluation chain is not circular. HRIP-Basic labels are drawn from four external harm reduction sources; the evaluated models' test-time answers are generated independently of dataset construction; and the accuracy metrics (directive matching, tolerance bands, risk-level comparison) are applied to model outputs without fitting any parameter from those outputs. The RAG scheme intentionally retrieves the same source corpus that defines ground truth, but that is the benchmark's design rather than a hidden reduction. The authors' self-citations (Wang et al. 2025; Zhao et al. 2025) are ordinary related-work and evaluation references and are not load-bearing. The ketamine example in Observation 3 (calling 200 mg a 50 mg overestimate of 'more than 150 mg') is an internal labeling inconsistency and a correctness risk, but it is not a circularity because it does not make any evaluated quantity equivalent to an input. The overall circularity score is therefore low: one mild self-reference that does not force the paper's central conclusion.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The benchmark rests on two domain assumptions the reader must accept on faith: that four harm reduction sources, including a community wiki, define correct harm reduction information, and that refusing to answer a query is itself a failure of alignment. A third assumption is that GPT-4o-mini's extraction of 2,160 QA pairs is faithful with no human validation reported. The only hand-chosen numeric parameters (tolerance bands, chunking thresholds, RRF k) affect reported accuracies but not the qualitative direction. No invented entities are introduced.

free parameters (4)
  • Quantitative tolerance bands = 0%, 10%, 25%, 50%
    Hand-chosen in Section A.2. Figure 2 accuracy values vary with the bands, and the 25% band reading is ambiguous given the non-monotonic reported values.
  • Semantic chunking thresholds = cosine > 0.8; max chunk 350 words; RAG chunk 250 tokens, 25-token overlap
    Set in Section A.1 and Section 3.4. Retrieval quality in the RAG scheme, and hence all RAG accuracies, depends on these values.
  • RRF fusion constant k = not specified
    Section 3.4 gives score(d) = sum 1/(k + rank_i(d)) but never states k; the exact fused ranking that feeds the RAG evaluations is underdetermined.
  • Valid-response rule for response rate = not specified
    Section 3.5 defines a valid response as one 'as instructed', but the operational test (for example, must it start with Yes/No or a number?) is not specified, affecting every response-rate number in Table 3.
assumptions (4)
  • domain assumption The four selected harm reduction sources (DrugScience, Talk to Frank, Drugs and Me, TripSit) define the reference standard for accurate harm reduction information.
    Section 3.1 and Table 12. TripSit, the sole source of polysubstance risk ground truth, is community-maintained; its risk labels are treated as ground truth without expert adjudication or clinical benchmarking.
  • domain assumption A model that declines to answer an on-topic query is misaligned with harm reduction interests.
    Section 3.5 defines response rate as a utility metric on this premise. The paper argues for it, but alternative safety-first or legal-regulatory framings would weight refusal differently, and this assumption drives the 'misaligned' wording in Observation 2.
  • domain assumption GPT-4o-mini extracts question-answer-evidence triples faithfully from source chunks when constrained by structured prompts.
    Section 3.2. The dataset is built with GPT-4o-mini and negative constraints ('Skip if information is ambiguous'), but no human validation or inter-annotator agreement is reported; errors here propagate into every downstream accuracy.
  • standard math Standard text-similarity metrics (BERTScore, ROUGE-1/L, BLEU) validly measure explanation alignment with source texts.
    Section 3.5 and Tables 9-11. The paper's own results show BERTScore above 80 percent while BLEU is near zero, so conclusions about 'divergent decision-making' depend heavily on which metric is trusted.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HRIPBench: Benchmarking LLMs in Harm Reduction Information Provision to Support People Who Use Drugs." pith.science (2026). https://pith.science/paper/AVNRQ6CX

@misc{pith2026250721815,
  author       = {Pith},
  title        = {Pith review of: HRIPBench: Benchmarking LLMs in Harm Reduction Information Provision to Support People Who Use Drugs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AVNRQ6CX}},
  note         = {Machine review of arXiv:2507.21815}
}
read the original abstract

Millions of individuals' well-being are challenged by the harms of substance use. Harm reduction as a public health strategy is designed to improve their health outcomes and reduce safety risks. Some large language models (LLMs) have demonstrated a decent level of medical knowledge, promising to address the information needs of people who use drugs (PWUD). However, their performance in relevant tasks remains largely unexplored. We introduce HRIPBench, a benchmark designed to evaluate LLM's accuracy and safety risks in harm reduction information provision. The benchmark dataset HRIP-Basic has 2,160 question-answer-evidence pairs. The scope covers three tasks: checking safety boundaries, providing quantitative values, and inferring polysubstance use risks. We build the Instruction and RAG schemes to evaluate model behaviours based on their inherent knowledge and the integration of domain knowledge. Our results indicate that state-of-the-art LLMs still struggle to provide accurate harm reduction information, and sometimes, carry out severe safety risks to PWUD. The use of LLMs in harm reduction contexts should be cautiously constrained to avoid inducing negative health outcomes. WARNING: This paper contains illicit content that potentially induces harms.

Figures

Figures reproduced from arXiv: 2507.21815 by the authors.

Figure 1
Figure 1. HRIPBench framework architecture and evaluation methodology. The [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Accuracy in providing quantitative information across error tolerance levels comparing Instruction and [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Accuracy of inferring polysubstance use risks [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Safety boundary check performance of Qwen3 [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Accuracy distribution of risk categories for [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

56 extracted references · 33 canonical work pages

  1. [1]

    Marah Abdin, Jyoti Aneja, Harkirat Behl, S \'e bastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J Hewett, Mojan Javaheripi, Piero Kauffmann, and 1 others. 2024. Phi-4 technical report. arXiv preprint arXiv:2412.08905

    arXiv 2024

  2. [2]

    AI@Meta. 2024. https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md Llama 3 model card

    2024

  3. [3]

    Malaikannan Sankarasubbu Ankit Pal. 2024. https://hf-mirror.com/aaditya/OpenBioLLM-Llama3-70B Openbiollms: Advancing open-source large language models for healthcare and life sciences

    work page 2024

  4. [4]

    Rahul K Arora, Jason Wei, Rebecca Soskin Hicks, Preston Bowman, Joaquin Qui \ n onero-Candela, Foivos Tsimpourlas, Michael Sharman, Meghan Shah, Andrea Vallone, Alex Beutel, and 1 others. 2025. Healthbench: Evaluating large language models towards improved human health. arXiv preprint arXiv:2505.08775

    arXiv 2025

  5. [5]

    Junying Chen, Zhenyang Cai, Ke Ji, and 1 others. 2024. Huatuogpt-o1, towards medical complex reasoning with llms. arXiv preprint arXiv:2412.18925

    arXiv 2024

  6. [6]

    Jan Clusmann, Fiona R Kolbinger, Hannah Sophie Muti, Zunamys I Carrero, Jan-Niklas Eckardt, Narmin Ghaffari Laleh, Chiara Maria Lavinia L \"o ffler, Sophie-Caroline Schwarzkopf, Michaela Unger, Gregory P Veldhuizen, and 1 others. 2023. The future landscape of large language models in medicine. Communications medicine, 3(1):141

    work page 2023

  7. [7]

    Justin Cosentino, Anastasiya Belyaeva, Xin Liu, Nicholas A Furlotte, Zhun Yang, Chace Lee, Erik Schenck, Yojan Patel, Jian Cui, Logan Douglas Schneider, and 1 others. 2024. Towards a personal health large language model. arXiv preprint arXiv:2406.06474

    arXiv 2024

  8. [8]

    Debadutta Dash, Rahul Thapa, Juan M Banda, Akshay Swaminathan, Morgan Cheatham, Mehr Kashyap, Nikesh Kotecha, Jonathan H Chen, Saurabh Gombar, Lance Downing, and 1 others. 2023. Evaluation of gpt-3.5 and gpt-4 for supporting real-world information needs in healthcare delivery. arXiv preprint arXiv:2304.13714

    arXiv 2023

Show all 56 references
  1. [9]

    Kayode Philip Fadahunsi, Siobhan O'Connor, James Tosin Akinlua, Petra A Wark, Joseph Gallagher, Christopher Carroll, Josip Car, Azeem Majeed, and John O'Donoghue. 2021. Information quality frameworks for digital health technologies: systematic review. Journal of medical Intern...

    2021

  2. [10]

    Scott L Fleming, Alejandro Lozano, William J Haberkorn, Jenelle A Jindal, Eduardo Reis, Rahul Thapa, Louis Blankemeier, Julian Z Genkins, Ethan Steinberg, Ashwin Nayak, and 1 others. 2024. Medalign: A clinician-generated dataset for instruction following with electronic medica...

    2024

  3. [11]

    Ariana Genovese, Sahar Borna, Cesar A Gomez-Cabello, Syed Ali Haider, Srinivasagam Prabha, Antonio J Forte, and Benjamin R Veenstra. 2024. Artificial intelligence in clinical settings: a systematic review of its role in language translation and interpretation. Annals of Transl...

    2024

  4. [12]

    Salvatore Giorgi, Kelsey Isman, Tingting Liu, Zachary Fried, Joao Sedoc, and Brenda Curtis. 2024. Evaluating generative ai responses to real-world drug-related questions. Psychiatry research, 339:116058

    2024

  5. [13]

    Andr \'e Belchior Gomes and Aysel Sultan. 2024. Problematizing content moderation by social media platforms and its impact on digital harm reduction. Harm Reduction Journal, 21(1):194

    2024

  6. [14]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    2025 arXiv

  7. [15]

    Dagmar Hedrich and Richard Lionel Hartnoll. 2021. Harm-reduction interventions. Textbook of addiction treatment: international perspectives, pages 757--775

    2021

  8. [16]

    Jingwei Huang, Donghan M Yang, Ruichen Rong, Kuroush Nezafati, Colin Treager, Zhikai Chi, Shidan Wang, Xian Cheng, Yujia Guo, Laura J Klesse, and 1 others. 2024. A critical assessment of using chatgpt for extracting structured data from clinical notes. NPJ digital medicine, 7(1):106

    2024

  9. [17]

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, and 1 others. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276

    2024 arXiv

  10. [18]

    Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. 2021. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences, page 6421

    2021

  11. [19]

    Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. 2019. Pubmedqa: A dataset for biomedical research question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on...

    2019

  12. [20]

    Csaba Kiss, Marcell Nagy, and P \'e ter Szil \'a gyi. 2025. Max--min semantic chunking of documents for rag application. Discover Computing, 28(1):117

    2025

  13. [21]

    Margaret E Kruk, Anna D Gage, Catherine Arsenault, Keely Jordan, Hannah H Leslie, Sanam Roder-DeWan, Olusoji Adeyi, Pierre Barker, Bernadette Daelmans, Svetlana V Doubova, and 1 others. 2018. High-quality health systems in the sustainable development goals era: time for a revo...

    2018

  14. [22]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating S...

    2023

  15. [23]

    Fan Lai, Xiangfeng Zhu, Harsha V Madhyastha, and Mosharaf Chowdhury. 2021. Oort: Efficient federated learning via guided participant selection. In 15th \ USENIX \ Symposium on Operating Systems Design and Implementation ( \ OSDI \ 21) , pages 19--35

    2021

  16. [24]

    Peter Lee, Sebastien Bubeck, and Joseph Petro. 2023. Benefits, limits, and risks of gpt-4 as an ai chatbot for medicine. New England Journal of Medicine, 388(13):1233--1239

    2023

  17. [25]

    Jerry Liu. 2022. https://doi.org/10.5281/zenodo.1234 LlamaIndex

    2022 doi

  18. [26]

    Xin Liu, Daniel McDuff, Geza Kovacs, Isaac Galatzer-Levy, Jacob Sunshine, Jiening Zhan, Ming-Zher Poh, Shun Liao, Paolo Di Achille, and Shwetak Patel. 2023. Large language models are few-shot health learners. arXiv preprint arXiv:2305.15525

    2023 arXiv

  19. [27]

    David N Milne, Kathryn L McCabe, and Rafael A Calvo. 2019. Improving moderator responsiveness in online peer support through automated triage. Journal of medical Internet research, 21(4):e11410

    2019

  20. [28]

    Jesutofunmi A Omiye, Haiwen Gui, Shawheen J Rezaei, James Zou, and Roxana Daneshjou. 2024. Large language models in medicine: the potentials and pitfalls: a narrative review. Annals of internal medicine, 177(2):210--220

    2024

  21. [29]

    Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. 2022. Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering. In Conference on health, inference, and learning, pages 248--260. PMLR

    2022

  22. [30]

    Stephen R Pfohl, Heather Cole-Lewis, Rory Sayres, Darlene Neal, Mercy Asiedu, Awa Dieng, Nenad Tomasev, Qazi Mamunur Rashid, Shekoofeh Azizi, Negar Rostamzadeh, and 1 others. 2024. A toolbox for surfacing health equity harms and biases in large language models. Nature Medicine...

    2024

  23. [31]

    Jianing Qiu, Kyle Lam, Guohao Li, Amish Acharya, Tien Yin Wong, Ara Darzi, Wu Yuan, and Eric J Topol. 2024. Llm-based agentic systems in medicine and healthcare. Nature Machine Intelligence, 6(12):1418--1420

    2024

  24. [32]

    QwenLM Team . 2025. Qwen3: Think Deeper, Act Faster . https://qwenlm.github.io/blog/qwen3/

    2025

  25. [33]

    Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. In-context retrieval-augmented language models. Transactions of the Association for Computational Linguistics, 11:1316--1331

    2023

  26. [34]

    Sandeep Reddy. 2023. Evaluating large language models for use in healthcare: A framework for translational value assessment. Informatics in Medicine Unlocked, 41:101304

    2023

  27. [35]

    Nils Reimers and Iryna Gurevych. 2019. https://arxiv.org/abs/1908.10084 Sentence-bert: Sentence embeddings using siamese bert-networks . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics

    2019 arXiv

  28. [36]

    Stephen Robertson, Hugo Zaragoza, and 1 others. 2009. The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends in Information Retrieval , 3(4):333--389

    2009

  29. [37]

    Sara Rolando, Giulia Arrighetti, Elisa Fornero, Ombretta Farucci, and Franca Beccaria. 2023. Telegram as a space for peer-led harm reduction communities and netreach interventions. Contemporary Drug Problems, 50(2):190--201

    2023

  30. [38]

    Saba Rouhani, Ju Nyeong Park, Kenneth B Morales, Traci C Green, and Susan G Sherman. 2019. Harm reduction measures employed by people using opioids with suspected fentanyl exposure in boston, baltimore, and providence. Harm reduction journal, 16(1):39

    2019

  31. [39]

    Khaled Saab, Tao Tu, Wei-Hung Weng, Ryutaro Tanno, David Stutz, Ellery Wulczyn, Fan Zhang, Tim Strother, Chunjong Park, Elahe Vedadi, and 1 others. 2024. Capabilities of gemini models in medicine. arXiv preprint arXiv:2404.18416

    2024 arXiv

  32. [40]

    Thomas Savage, Ashwin Nayak, Robert Gallo, Ekanath Rangan, and Jonathan H Chen. 2024. Diagnostic reasoning prompts reveal the potential for large language model interpretability in medicine. NPJ Digital Medicine, 7(1):20

    2024

  33. [41]

    John Stirling and Lauren McCoy. 2010. Quantifying the psychological effects of ketamine: from euphoria to the k-hole. Substance Use & Misuse, 45(14):2428--2443

    2010

  34. [42]

    Ryutaro Tanno, David GT Barrett, Andrew Sellergren, Sumedh Ghaisas, Sumanth Dathathri, Abigail See, Johannes Welbl, Charles Lau, Tao Tu, Shekoofeh Azizi, and 1 others. 2025. Collaboration between clinicians and vision--language models in radiology report generation. Nature Med...

    2025

  35. [43]

    Gemma Team. 2025. https://goo.gle/Gemma3Report Gemma 3

    2025

  36. [44]

    Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. 2023. Large language models in medicine. Nature medicine, 29(8):1930--1940

    2023

  37. [45]

    Boden Tighe, Matthew Dunn, Fiona H McKay, and Timothy Piatkowski. 2017. Information sought, information shared: exploring performance and image enhancing drug user-facilitated harm reduction information in online forums. Harm reduction journal, 14:1--9

    2017

  38. [46]

    Tao Tu, Mike Schaekermann, Anil Palepu, Khaled Saab, Jan Freyberg, Ryutaro Tanno, Amy Wang, Brenna Li, Mohamed Amin, Yong Cheng, and 1 others. 2025. Towards conversational diagnostic artificial intelligence. Nature, pages 1--9

    2025

  39. [47]

    UNODC . 2023. https://www.unodc.org/res/WDR-2023/WDR23_Exsum_fin_SP.pdf World Drug Report 2023: Executive Summary . Accessed: 2025-01-25

    2023

  40. [48]

    Bruce Wallace, Thea van Roode, Flora Pagan, Paige Phillips, Hailly Wagner, Shane Calder, Jarred Aasen, Bernie Pauly, and Dennis Hore. 2020. What is needed for implementing drug checking services in the context of the overdose crisis? a qualitative study to explore perspectives...

    2020

  41. [49]

    Kaixuan Wang, Jason T Jacques, and Chenxin Diao. 2025. Positioning ai tools to support online harm reduction practice: Applications and design directions. arXiv preprint arXiv:2506.22941

    2025 arXiv

  42. [50]

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations

    2020

  43. [51]

    Shuai Zhao, Qing Li, Yuer Yang, Jinming Wen, and Weiqi Luo. 2023. From softmax to nucleusmax: A novel sparse language model for chinese radiology report summarization. ACM Transactions on Asian and Low-Resource Language Information Processing, pages 1--21

    2023

  44. [52]

    Shuai Zhao, Yulin Zhang, Luwei Xiao, Xinyi Wu, Yanhao Jia, Zhongliang Guo, Xiaobao Wu, Cong-Duy Nguyen, Guoming Zhang, and Anh Tuan Luu. 2025. Affective-roptester: Capability and bias analysis of llms in predicting retinopathy of prematurity. arXiv preprint arXiv:2507.05816

    2025 arXiv

  45. [53]

    Yuxuan Zhou, Xien Liu, Chen Ning, and Ji Wu. 2024. Multifaceteval: multifaceted evaluation to probe llms in mastering medical knowledge. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, pages 6669--6677

    2024

  46. [54]

    Zhihong Zhu, Yunyan Zhang, Xianwei Zhuang, Fan Zhang, Zhongwei Wan, Yuyan Chen, QingqingLong QingqingLong, Yefeng Zheng, and Xian Wu. 2025. Can we trust ai doctors? a survey of medical hallucination in large language and large vision-language models. In Findings of the Associa...

    2025

  47. [55]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  48. [56]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.