{"id":"bdc39ac9-ba82-4e27-9000-fbc95afb8215","arxiv_id":"2507.21815","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"State-of-the-art LLMs are frequently inaccurate, and sometimes dangerous, when answering harm reduction questions about drug use, even when given retrieved source material.","lead":"This paper introduces a benchmark of 2,160 questions about drug safety and harm reduction, and tests 11 large language models on it. It finds that even the best models give wrong or risky advice often enough that they should not be trusted to counsel people who use drugs.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Ground-truth QA pairs are generated by GPT-4o-mini without human validation; the central 'LLMs are unsafe' claim inherits every extraction error, and one cited example already misreads its own ground truth.","rationale":"I agree with the reader's weakest assumption: the unvalidated, LLM-generated ground truth is the epistemic foundation of every accuracy and safety-risk conclusion. The ketamine example is concrete evidence that label errors exist, and Figure 2's non-monotonic tolerance curves further suggest the quantitative evaluation pipeline has defects. These issues do not appear to overturn the qualitative finding that LLMs underperform across 11 models, so the paper remains a valuable conditional contribution rather than a reject. The appropriate action is to keep the CONDITIONAL verdict and require the proposed human-validation audit before the quantitative claims are cited as settled.","tokens_in":19392,"tokens_out":3478,"duration_ms":42831,"concrete_test":"Release 200 randomly stratified QA pairs (including all 150 polysubstance pairs) to two independent harm-reduction experts, blind to the original labels, with instructions to verify each answer against the cited verbatim context and, for polysubstance risks, against TripSit's chart. Compute label error rate and Cohen's kappa. Separately, re-evaluate the ketamine example by checking whether 200 mg falls within the ground-truth range; if the range is 'more than 150 mg' with no upper bound, the example must be removed or reclassified. If expert error rate exceeds about 5% or kappa is below 0.8, recompute all reported accuracies after correcting labels; if the qualitative finding (most models below 90% on safety boundary and poor quantitative accuracy) survives, the CONDITIONAL verdict stands; if not, the central claim weakens.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The benchmark's central claim—that SOTA LLMs provide inaccurate and sometimes severely unsafe harm reduction information—rests entirely on HRIP-Basic's 2,160 ground-truth answer labels. Section 3.2 states these labels were produced by GPT-4o-mini using structured prompts and a rule-based script over TripSit's community wiki, with no human validation, inter-annotator agreement, or expert adjudication reported. Appendix Table 12 lists TripSit as 'community-driven'; its interaction chart supplies all 150 polysubstance risk labels. If GPT-4o-mini extracted values incorrectly, or if sources deviate from clinical consensus, every accuracy figure and 'severe safety risk' example measures alignment with those sources rather than correctness. The risk is not hypothetical: Observation 3's ketamine example says ground truth is 'more than 150 mg' and calls o3-mini's '200 mg' a 50 mg overestimate—yet 200 mg satisfies 'more than 150 mg', so the example misstates its own ground truth under the benchmark's stated tolerance logic. This is exactly the kind of label error that silent LLM extraction can inject at scale. Without an audit of the labels, the paper's headline conclusions are conditional on an unverified gold standard.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript introduces HRIPBench, a benchmark for evaluating large language models (LLMs) in providing harm reduction information to people who use drugs (PWUD). The dataset HRIP-Basic contains 2,160 question-answer-evidence pairs covering three tasks: safety boundary checks (1,638), quantitative questions (372), and polysubstance use risks (150). Ground truth is constructed automatically: GPT-4o-mini extracts QA pairs from four harm reduction sources via structured prompts, and TripSit's interaction chart is parsed by a rule-based script. The authors evaluate 11 open-source, closed-source, and medical-domain LLMs under an instruction-only scheme and a retrieval-augmented generation (RAG) scheme, reporting response rates, answer accuracy (including tolerance bands for numeric answers), and directional risk misclassification. Across models, results show low numeric accuracy, mixed safety-boundary accuracy, and a tendency to overestimate polysubstance risk, with occasional severe underestimation. The authors conclude that current state-of-the-art LLMs are insufficiently accurate and can pose safety risks to PWUD in high-stakes cases.","tokens_in":19519,"tokens_out":5266,"duration_ms":66153,"significance":"If the dataset is valid, this is a timely and socially important benchmark filling a real gap: existing medical QA benchmarks omit the practical, non-clinical, judgment-heavy information needs of PWUD. The three-task design, the inclusion of a RAG comparison, the response-rate construct, and the asymmetric risk metrics distinguish under- and overestimation are valuable contributions. The paper evaluates a broad set of 11 models, which strengthens the qualitative pattern. However, the central result hinges on the validity of the automatically generated ground truth and the authority of the underlying sources. The manuscript currently provides no human validation, inter-annotator agreement, or expert adjudication of the labels, and one illustrative example is internally inconsistent with its own ground truth. With a demonstrated label audit and appropriate qualification of the claims, this would be a significant contribution to both NLP benchmarking and public health applications.","major_comments":[{"comment":"The entire benchmark rests on the 2,160 ground-truth pairs produced by GPT-4o-mini with no reported human validation, inter-annotator agreement, or expert adjudication. Because the headline conclusion is that models make safety-critical errors, silent extraction errors in the labels directly contaminate every accuracy and risk figure. The authors should either (a) conduct and report an expert audit of a representative sample (ideally all 150 polysubstance labels and a stratified sample of the other two tasks) and re-run the evaluation with corrected labels, or (b) clearly reframe the benchmark as measuring alignment with the four selected sources rather than absolute accuracy or safety.","section":"Section 3.2"},{"comment":"The ketamine example is internally inconsistent: the ground truth is stated as 'more than 150 mg', which is a lower bound, so o3-mini's '200 mg' is consistent with it. The text claims this is a '50 mg higher' overestimate, which only makes sense if 'more than 150 mg' is treated as a point estimate of 150 mg. This contradicts both the source expression and the tolerance-based evaluation logic described in Appendix A.2. This example should be corrected or replaced, and the regex-based tolerance evaluation should be checked for how it handles non-point numeric ground truths such as lower bounds and ranges.","section":"Observation 3 (Section 4.2)"},{"comment":"All 150 polysubstance risk labels come from TripSit, which the authors themselves describe as a community-driven harm reduction project. The 'Dangerous' label used in Observation 4 to demonstrate severe underestimation is therefore unadjudicated. The authors should report expert or clinical validation of the interaction chart labels, or at minimum a sensitivity analysis comparing a subset of the combinations against a second authoritative source. Without this, the 'severe safety risk' conclusions in the abstract and conclusion measure alignment with one community wiki rather than clinical consensus.","section":"Table 12 and Section 3.2"},{"comment":"GPT-4o-mini generated the ground-truth labels and is also one of the evaluated models. Although this does not invalidate comparisons among the other ten models, it creates a possible inflation for GPT-4o-mini's scores and is a correctness risk: the dataset may implicitly favor the stylistic patterns of the label-generating model. The authors should acknowledge this dual role and report the main results with GPT-4o-mini excluded, or otherwise demonstrate that its performance is not systematically advantaged.","section":"Sections 3.2 and 4.1"}],"minor_comments":[{"comment":"The caption says 'Values represent the accuracy of queries that received expected responses', but Section 3.5 defines this metric as response rate. Please correct the caption to match the terminology.","section":"Table 3 caption"},{"comment":"The semantic chunking thresholds (cosine similarity greater than 0.8, maximum chunk size of 350 words) and the RAG chunk size (250 tokens with 10% overlap) are introduced without justification or sensitivity analysis. A brief rationale or a small ablation would strengthen the reproducibility.","section":"Appendix A.1"},{"comment":"The RRF fusion score uses a constant k, but its value is never specified. Please report the constant (and any retrieval hyperparameters) so the experiments are fully reproducible.","section":"Section 3.4 (RAG scheme)"},{"comment":"The statement that response rates are 'lower when providing numerical values' should be qualified: several models, including GPT-4.1 and GPT-4o-mini, have response rates above 98% on quantitative questions, whereas the dramatic drops are specific to certain models (e.g., Gemma-3-27B). Consider reporting per-model details rather than a general trend.","section":"Section 4.2, Observation 1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within scope for a CS/NLP benchmarking venue and addresses a socially important problem. The main barrier is the absence of human validation of the gold standard; the internally inconsistent ketamine example strengthens the concern that automatic extraction errors may be systematic. If the authors add an expert audit and re-run the evaluation, I would be supportive of publication. I also recommend that the editor consult a reviewer with domain expertise in harm reduction or addiction medicine to adjudicate the authority of the selected sources, particularly TripSit."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one thing to know about HRIPBench: it is a genuine first for a high-stakes, underserved domain, but the ground truth it stands on has not been validated, and one of its own examples is misread. The benchmark and the qualitative finding are worth engaging with; the specific numbers are conditional until the labels are audited.\n\nWhat's new: this is the first benchmark aimed at harm reduction information for PWUD, and the design has real thought behind it. The three-task split (safety boundaries, quantitative values, polysubstance risk) maps onto actual information needs. The QA pairs come with verbatim source context, which makes fact-checking possible. The asymmetric risk metric — distinguishing underestimation from overestimation — is a good idea, as is the refusal/response-rate measure, because a model that clams up isn't helping anyone. The Instruction-vs-RAG comparison is reasonable and the evaluation across 11 models gives a consistent picture: LLMs underperform on these tasks, and RAG helps in many cases but not enough.\n\nNow the soft spots. The load-bearing assumption is that the 2,160 QA pairs are correct. They were generated by GPT-4o-mini with no reported human validation or inter-annotator agreement, and the polysubstance labels come from a community-maintained wiki. That's a real risk. The stress-test catches a concrete symptom: in Observation 3 the paper claims o3-mini's 200 mg ketamine answer is 50 mg above the ground truth of 'more than 150 mg' — but 200 mg is within that range. If the exemplar misreads its own ground truth, the unvalidated labels are a legitimate worry. Figure 2 also shows non-monotonic accuracy across tolerance bands, which shouldn't happen for cumulative tolerances, and Table 3's caption calls response rates 'accuracy.' These are fixable, but they lower confidence in the quantitative claims.\n\nThat said, the overall direction — that SOTA LLMs are not reliable providers of harm reduction information — likely survives an audit, since the pattern is consistent across models and tasks. The paper just needs to earn its headline numbers.\n\nThis is for anyone working on LLM evaluation in public health or building patient-facing chatbots. It deserves a serious referee, but the revision should require a label-validation report and a fix of the internal errors.\n\nRecommendation: send to peer review with a clear request for major revisions.","headline":"HRIPBench is a genuinely first benchmark for harm reduction information provision, but its GPT-4o-mini-generated ground truth is unvalidated and one of its own examples misreads the label — the qualitative finding likely holds, the specific numbers don't yet.","tokens_in":20165,"tokens_out":3863,"would_cite":true,"duration_ms":40550,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Current state-of-the-art LLMs cannot reliably deliver accurate, safe harm-reduction information to people who use drugs, and some answers carry severe safety risks—even when the models are given retrieved domain knowledge.","keywords":["large language models","harm reduction","benchmark","people who use drugs","safety evaluation","retrieval-augmented generation","polysubstance use","health information"],"falsifier":"Have two independent addiction-medicine clinicians manually score a random sample of the 2,160 ground-truth pairs against the stated sources and compare disagreements with the benchmark's labels; a material disagreement rate would show the safety conclusions rest on generated labels rather than clinical consensus.","tokens_in":19074,"feed_emoji":"⚠️","tokens_out":7423,"duration_ms":77795,"temperature":0.7,"pith_summary":"HRIPBench asks whether today's large language models can be trusted to answer practical safety questions from people who use drugs: whether an activity is safe, what dose or timing values are correct, and how risky mixing two substances is. To test this, the authors built HRIP-Basic, a dataset of 2,160 question-answer-evidence pairs drawn from four harm-reduction sources, and scored 11 models under two conditions: asking directly and asking with retrieved domain knowledge supplied. The paper's central finding is that even the strongest models fall short, with accuracy below 60% on quantitative questions in many settings, and with some answers that would put a person at risk, such as under-calling a dangerous drug combination. The authors conclude that LLMs are not yet fit for autonomous use in harm-reduction information provision, and that deploying them without careful constraints could cause negative health outcomes.","feed_headline":"Leading LLMs fail a 2,160-question harm-reduction safety test","feed_subtitle":"Top models gave wrong doses, censored answers, and even under-called dangerous drug combinations in three benchmark tasks.","key_machinery":"The engine of the evaluation is HRIP-Basic: 2,160 question-answer-evidence triples built by using GPT-4o-mini to extract questions and verbatim source evidence from four harm-reduction information sources, with polysubstance-risk items generated by a rule-based script from a structured interaction chart. Each triple is one of three task types: safety boundary check (yes/no), quantitative value (number with unit), and polysubstance-use risk level. The benchmark then compares two conditions per model: an Instruction scheme, where the model answers from its own knowledge following a constrained prompt, and a retrieval-augmented generation (RAG) scheme, where the model receives the top-3 retrieved chunks from the same sources via mixed dense and sparse retrieval fused by Reciprocal Rank Fusion. Scoring separates the directive from its explanation, using response rate, classification metrics, and tolerance bands of 0–50% for numbers, plus directional risk categories of underestimation, severe underestimation, and overestimation.","core_discovery":"The paper's claim is that current state-of-the-art LLMs, evaluated on HRIPBench, cannot reliably provide accurate harm-reduction information to people who use drugs and sometimes produce answers that are actively unsafe. On the 1,638 safety-boundary checks, no model exceeded 90% accuracy in the instruction-only condition, and several fell into the 59–88% range; adding retrieved knowledge lifted most models but left the best at 95%. On 372 quantitative questions, accuracy stayed below roughly 60% even with generous error tolerances, with concrete examples of overdose-relevant doses being overstated. On 150 polysubstance-risk questions, models typically overestimated risk, which the authors read as overcautious and unhelpful, but some severely underestimated it, as when mixing opioids with ketamine was called “low risk”; retrieval removed the worst underestimates while increasing plain underestimates. The paper therefore concludes that general-purpose LLMs are currently insufficient for this high-stakes use, and that their deployment in harm-reduction contexts should be cautiously constrained.","pith_inferences":["Beyond the paper: the benchmark's gold standard is generated by an LLM from a small set of English-language harm-reduction sources, so a replication that adds clinician or peer validation of the 2,160 pairs would test whether the safety failures are failures against clinical consensus or only against these sources' wording.","Beyond the paper: because the underlying sources are largely UK-focused, the benchmark's difficulty ranking of models could shift when applied to other jurisdictions, languages, or community-specific harm-reduction knowledge, and no current claim covers those settings.","Beyond the paper: the finding that retrieval eliminates severe underestimates but increases mild ones suggests a testable design lever for future systems—different chunk sizes, reranking weights, or answer-time warnings might trade off false reassurance against overcautious refusal.","Beyond the paper: the paper reports harms in qualitative examples, such as a 50 mg overstatement of a ketamine dose; a quantitative risk-severity weighting of each error type would let stakeholders compare models by expected number of harmful outcomes rather than by raw accuracy."],"forward_implications":["Autonomous use of current LLMs as harm-reduction advisers is unsafe; any real deployment would need to restrict the model to verified content and keep a human in the loop.","Retrieval helps but is not a cure-all: giving models relevant domain passages improves safety-boundary accuracy and removes the most dangerous risk underestimates, yet quantitative answers remain unreliable enough to cause harm.","A model that refuses to answer can be as problematic as one that answers wrongly, since refusals block access to life-relevant information; response rate is therefore a safety metric, not just a usability one.","Evaluators of health LLMs should track the direction of errors, not only accuracy, because overestimating risk and underestimating risk have opposite public-health consequences.","Model scale and medical specialization do not guarantee safe harm-reduction answers; in the tested Qwen3 family, larger size was associated with lower accuracy."],"supporting_citations":[{"why":"Supplies the GPT-4o-mini model used as the controlled information-extraction tool that generated the question-answer-evidence triplets.","marker":"Hurst et al., 2024"},{"why":"Identifies the basic information needs of people who use drugs, which define the three benchmark tasks.","marker":"Wallace et al., 2020"},{"why":"Provides the harm-reduction framework and taxonomy that shape the extraction prompts and ground-truth structure.","marker":"Hedrich and Hartnoll, 2021"},{"why":"Motivates the concern that moderation and prohibitionist stances can censor needed harm-reduction information, shaping the response-rate metric.","marker":"Gomes and Sultan, 2024"},{"why":"Offers prior evidence that LLM answers to real drug-related questions can be unsafe, motivating the safety-risk analysis.","marker":"Giorgi et al., 2024"},{"why":"Provides the semantic chunking strategy used to prepare source texts so that safety-critical information is not fragmented.","marker":"Kiss et al., 2025"},{"why":"Defines the BM25 sparse retriever used as one component of the mixed retrieval pipeline in the RAG scheme.","marker":"Robertson et al., 2009"}],"fun_headline_variants":["LLMs unsafe on drug-harm questions, 2,160-item benchmark shows","Top AI models flunk 2,160-question harm-reduction exam","LLMs fail safety check on 2,160 drug-harm questions","LLMs flub doses and combos in 2,160-item harm-reduction test","Study: LLMs can't reliably answer harm-reduction queries"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the 2,160 question-answer pairs generated by GPT-4o-mini from four harm-reduction sources are a trustworthy gold standard, since no human expert validation or inter-annotator agreement is reported.","fun_headline_variants_meta":{"raw":{"variants":["LLMs unsafe on drug-harm questions, 2,160-item benchmark shows","Top AI models flunk 2,160-question harm-reduction exam","LLMs fail safety check on 2,160 drug-harm questions","LLMs flub doses and combos in 2,160-item harm-reduction test","Study: LLMs can't reliably answer harm-reduction queries"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000681,"raw_usage":{"total_tokens":3106,"prompt_tokens":971,"completion_tokens":2135,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":587,"completion_tokens_details":{"reasoning_tokens":2036}},"tokens_in":587,"tokens_out":2135,"duration_ms":19408,"temperature":1.0,"reasoning_tokens":2036,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T12:21:45.562437+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have two independent addiction-medicine clinicians manually score a random sample of the 2,160 ground-truth pairs against the stated sources and compare disagreements with the benchmark's labels; a material disagreement rate would show the safety conclusions rest on generated labels rather than clinical consensus.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Identifies the basic information needs of people who use drugs, which define the three benchmark tasks."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the harm-reduction framework and taxonomy that shape the extraction prompts and ground-truth structure."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Motivates the concern that moderation and prohibitionist stances can censor needed harm-reduction information, shaping the response-rate metric."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Offers prior evidence that LLM answers to real drug-related questions can be unsafe, motivating the safety-risk analysis."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the semantic chunking strategy used to prepare source texts so that safety-critical information is not fragmented."}],"review_version":1}