{"id":"980ece72-7949-42b0-87e8-acb77b94c1bd","arxiv_id":"2501.03266","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Ethical refusals in LLM responses sharply reduce user win rates in Chatbot Arena compared to technical refusals and normal answers, though detailed refusals and clearly harmful prompts reduce the penalty.","lead":"This study analyzed nearly 50,000 Chatbot Arena votes and found that large language models lose user preference when they refuse prompts, especially when the refusal is for ethical or safety reasons. The results suggest a design tension between safety alignment and user satisfaction, with implications for how chatbots explain and calibrate refusals.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unvalidated refusal classifier on unfiltered data: single-annotator labels and no per-class metrics leave the 8% vs 16% ethical/technical win-rate gap vulnerable to misclassification bias.","rationale":"The reader's weakest assumption and my analysis converge on the refusal classifier. The paper's core claims—the 8% vs 16% win rates and the 37 pp vs 21 pp OLS coefficients—are only as credible as the automated distinction between ethical and technical refusals. The reported evidence is insufficient: one human annotator, an LLM second annotator whose disagreements were adjudicated by the same human, a single weighted-F1 number, and no evaluation on organic unfiltered prompts. Since ethical refusals are a rare class in the full data, even small per-class error rates can distort the estimated penalty substantially. This is not an internal inconsistency, but it is a clear correctness risk. The paper does have independent support in the form of a publicly released classifier and dataset, which makes the proposed validation test feasible. The within-prompt paired comparisons (e.g., Figure 2's 4% win rate for ethical refusals against normal responses) mitigate prompt-level confounds, but they do not mitigate classification error. Therefore the appropriate outcome remains CONDITIONAL: the claim is plausible but needs the per-class, unfiltered-data validation I describe. I agree with the reader that this is the weakest assumption and that no verdict change beyond the reader's CONDITIONAL is warranted.","tokens_in":17933,"tokens_out":8678,"duration_ms":86516,"concrete_test":"Stratified validation of the classifier on unfiltered data: draw a random sample of ~600 responses from the full 99,876-response dataset, stratified by the classifier's predicted class (or by refusal-indicating phrase presence and absence), and have at least two independent annotators, blind to the model's predictions, code each response using the paper's five-category taxonomy. Compute per-class precision, recall, and F1 for Refusal Ethical and Refusal Technical. Then recompute Table 4 win rates and re-estimate Table 5 coefficients using (a) the ground-truth labels on the annotated sample, and (b) an inverse-precision reweighting of the full-data predictions.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central estimate—that ethical refusals win at only 8% versus 16% for technical refusals (Section 4.1) and carry a 37 pp OLS penalty (Table 5)—rests entirely on the RoBERTa classifier. The classifier is trained on 3,500 responses labeled by a single author (Section 3.2), with GPT-4-turbo as a second annotator whose disagreements were adjudicated by the same author. Table 3 reports only weighted F1 (0.88) and accuracy on a test set drawn from the same phrase-pre-filtered distribution used for training. No per-class precision/recall is given, and the classifier is never validated on unfiltered, organic Chatbot Arena responses, where ethical refusals are rare (2.66% of the full data; Table 4). Because the test set is 40.6% Normal and 17.0% Refusal Ethical, a 0.88 weighted F1 can coexist with substantially lower per-class F1 on the minority refusal categories. If the classifier systematically confuses ethical and technical refusals—categories that are differentiated by motivation, not surface form—the headline 8% versus 16% gap and the 16 pp adjusted difference (Table 5: -0.375 vs -0.214) could be partly or wholly an artifact. The author-led single annotation and self-adjudicated disagreement check also leave no independent ground truth. This is the load-bearing link between raw data and every downstream claim in RQ1/RQ2.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper analyzes nearly 50,000 single-turn Chatbot Arena comparisons to measure how LLM response refusals affect user satisfaction. It introduces a fine-tuned RoBERTa classifier that labels responses as Normal, Refusal Ethical, Refusal Technical, or corresponding Disclaimer categories, based on a hand-labeled dataset of 3,500 responses. The main empirical claim is that ethical refusals receive sharply lower win rates than technical refusals and normal responses (8% vs. 16% vs. 36% unconditional; a 37.5 percentage point adjusted win penalty in Table 5). The paper also tests moderators: refusal phrasing (response length and semantic similarity to the prompt) and prompt sensitivity (OpenAI Moderation API flags). The authors report consistent evidence that refusals are penalized, ethical refusals more so than technical refusals, and that the penalty is softened by more aligned and longer refusals and by clearly sensitive prompts.","tokens_in":18221,"tokens_out":6856,"duration_ms":71741,"significance":"If the central result holds, the paper makes a valuable empirical contribution: it moves refusal research from scripted safety benchmarks to organic, large-scale user preference data, and it is one of the few studies to separately estimate user reactions to ethical versus technical refusals. The public release of the refusal classifier and annotation dataset is a concrete asset. The paper also engages seriously with theory from human-computer interaction, politeness, and expectation violation research. However, the load-bearing empirical link is the classifier's ability to distinguish ethical from technical refusals at scale, and that link is not yet established with adequate validation. The headline 8% vs. 16% gap and the adjusted 37 pp vs. 21 pp penalty difference both inherit any bias in the refusal classifier, so the significance of the contribution depends on the classifier concerns being resolved.","major_comments":[{"comment":"The classifier evaluation is insufficient to support the central claim. The test set is drawn from the same phrase-pre-filtered distribution used for training, and Table 3 reports only weighted F1 and accuracy (0.88). Per-class precision and recall are not reported. Because the full-data prevalence is 2.66% for Refusal Ethical and 3.21% for Refusal Technical (Table 4), while the training/test set has 40.6% Normal and 17.0% Refusal Ethical, a weighted F1 of 0.88 is compatible with materially lower performance on the refusal classes. More importantly, the ethical/technical distinction is based on motivation rather than surface form, so differential misclassification between these two categories could bias the 8% vs. 16% win-rate gap and the Table 5 coefficients. Please provide the full confusion matrix, per-class precision/recall/F1, and validate the classifier on an unfiltered random sample of Chatbot Arena responses with manual labels, reporting the same per-class metrics on that sample.","section":"§3.2–3.3, Table 3"},{"comment":"The ground-truth annotation rests on a single author. The paper reports that GPT-4-turbo served as a second annotator and that the author adjudicated the 50 examined disagreements, but this does not provide independent human ground truth. The reported Krippendorff's alpha (0.668) and Cohen's kappa (0.67) measure agreement with an LLM, not with an independent human rater. Given that the key contrast is ethical versus technical motivation, the current reliability evidence does not rule out systematic labeling bias. At minimum, the paper should present dual independent human annotations on a random subset—especially cases near the Ethical/Technical boundary—along with disagreement statistics and a transparent adjudication procedure.","section":"§3.2"},{"comment":"There is a numerical inconsistency in the reported Tie coefficients for Refusal Ethical. Table 5 reports 0.005*** with standard error 0.014, which cannot be statistically significant at the conventional level, while Appendix Table AT1 reports 0.055*** with standard error 0.014 for what should be the same focal-model coefficient. The significance stars, coefficients, and standard errors need to be reconciled and corrected across the main text and appendix.","section":"§4.1, Table 5 vs. Appendix Table AT1"}],"minor_comments":[{"comment":"There is a typo in the introduction: \"Antropic\" should be \"Anthropic.\"","section":"§1"},{"comment":"The table caption and text are ambiguous about the F1 metric: the text says \"weighted F1\" in the training description but Table 3 labels it only as \"F1.\" Please state clearly that it is a weighted average and also report per-class scores.","section":"§3.3, Table 3"},{"comment":"The sentence \"In 49 out of 50 cases, the human annotation was confirmed as correct\" should specify that the confirmation was made by the same author who created the labels, so that readers can judge the independence of the validation.","section":"§3.2"},{"comment":"The note \"Controls Opp. Model represent controls for all listed variables for the opponent model\" is imprecise: it should say \"for the opponent response\" and clarify which variables are included (response category, similarity, length, and model fixed effects).","section":"§4.1, Table 5 note"},{"comment":"The manuscript contains two \"citation omitted for anonymity\" placeholders. These should be resolved before publication; if anonymity is required during review, the editor should ensure the final version replaces them.","section":"§3.2 and §5.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is potentially a good fit for the journal and the empirical setting is well chosen. My recommendation rests on the classifier validation gap: the headline ethical-versus-technical refusal contrast depends entirely on a classifier whose per-class performance on the real distribution is not shown. This is fixable with confusion matrices and an unfiltered validation sample, so I do not recommend rejection, but the current version should not be accepted without that evidence. The single-annotator design is a second, related concern; if the authors can provide an independent human-label subset, that would substantially strengthen the paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: this paper is worth engaging—it's the first large-scale Arena-based evidence I know of that separates ethical from technical refusals and shows users penalize the former much more heavily. The reported numbers are big: ethical refusals win 8% of matchups vs 16% for technical, and the OLS penalty is 37 points. The authors also check plausible confounds, release the classifier and labeled data, and frame the hypotheses with relevant HCI theory. That's a solid package.\n\nThe soft spot is the measurement of the key variable. The classifier is trained on 3,500 hand-labeled responses, but the labels come from a single author, with GPT-4-turbo as the only second annotator and the author adjudicating disagreements. There's no independent ground truth, so we're relying on one person's interpretation of what counts as 'ethical' vs 'technical.' The test-set performance is reported only as weighted F1 and accuracy on a pre-filtered distribution that is 40% normal and roughly 17% ethical refusals. Per-class precision/recall aren't given. Since these classes are differentiated by motivation, not surface form, and since ethical refusals are rare in the full data (2.66%), I can't tell whether the 8% vs 16% gap is real or amplified by misclassification. A confusion matrix, per-class F1, and a hand-checked sample of unfiltered Arena responses would go a long way.\n\nThere's also an unexplained unit-of-analysis choice. Table 5 lists 49,938 observations—the number of pairwise comparisons—but each comparison has two responses. How was the focal response chosen? Using one row per pair without explaining the selection rule could bias the estimates, and the robust standard errors aren't clustered by prompt or pair, which matters if both responses are used. This needs to be clarified, not just because it's sloppy but because it changes what the regressions are actually estimating.\n\nNone of these are fatal. The direction of the finding is plausible, the effect is much larger than what annotation noise would typically produce, and the theoretical motivation is reasonable. The limitations section is honest about several issues, though it doesn't address the classifier validation gap.\n\nI'd send this to peer review. The question matters, the data is appropriate, and a good referee can push the authors to tighten the validation and the econometrics. As it stands, I'd treat the headline as a well-motivated hypothesis rather than a measured fact.","headline":"A genuinely new Arena-based result on refusal penalties, but the classifier needs stronger validation before the headline numbers can be trusted.","tokens_in":18715,"tokens_out":4911,"would_cite":false,"duration_ms":49864,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper establishes a refusal penalty in user preferences: ethical refusals win only 8% of Chatbot Arena comparisons, about half the rate of technical refusals, and the penalty is smaller when refusals are longer, more aligned with the…","keywords":["LLM content moderation","response refusals","user satisfaction","Chatbot Arena","ethical alignment","pairwise preference","RoBERTa classifier","prompt sensitivity"],"falsifier":"Have several independent human raters label a random sample of responses the classifier calls Refusal Ethical and Refusal Technical, then re-estimate the win-rate regressions using consensus labels; if a material share of cases changes category, the 8%-versus-16% win-rate gap and the 37-point coefficient would shrink or disappear.","tokens_in":17733,"feed_emoji":"🤖","tokens_out":7621,"duration_ms":70182,"temperature":0.7,"pith_summary":"Using nearly 50,000 single-turn pairwise comparisons from Chatbot Arena, the paper asks whether users punish LLMs for refusing to answer, and whether the reason matters. It builds a five-category classifier that separates ethical refusals from technical refusals and from disclaimers that still attempt the task. The central finding is a large refusal penalty: ethical refusals win only 8% of comparisons, technical refusals 16%, and normal responses 36%, with a 37-percentage-point regression coefficient for ethical refusals. The penalty shrinks when refusals are longer, more aligned with the prompt, or issued in response to content flagged as clearly unsafe. The paper interprets this as evidence that safety-aligned moderation can conflict with user satisfaction, and that context and phrasing can reduce that conflict.","feed_headline":"When models refuse on ethics, their win rate drops to 8%","feed_subtitle":"In 50k Chatbot Arena matchups, ethical refusals win 8% vs 16% for technical ones; phrasing and prompt risk soften the blow.","key_machinery":"The load-bearing instrument is a fine-tuned RoBERTa-Large classifier trained on 3,500 hand-labeled responses, which sorts each model response into five categories: Normal, Refusal Ethical, Refusal Technical, Disclaimer Ethical, and Disclaimer Technical. The classifier is applied to all responses in the Chatbot Arena dataset, converting free text into per-response labels, and user win/loss/tie outcomes are then regressed on those labels with controls for the focal and opponent model. Two auxiliary measurements carry the moderation analyses: cosine similarity between prompt and response embeddings as a proxy for conversational alignment, and flags from the OpenAI Moderation API as a proxy for prompt sensitivity.","core_discovery":"The paper's central claim is that users systematically prefer normal responses to refusals, and that ethical refusals are judged substantially more harshly than technical ones. In the raw data, Refusal Ethical responses win 8% of comparisons, Refusal Technical 16%, and Normal responses 36%; in OLS regressions controlling for model and opponent, an ethical refusal lowers win probability by 37 percentage points while a technical refusal lowers it by 21. The penalty is not fixed: among ethical refusals, a one-standard-deviation increase in response length raises win rate by 9 percentage points, higher semantic similarity to the prompt raises it by 1.6, and refusals to prompts flagged as unsafe by a moderation API roughly double the win/loss ratio compared with unflagged prompts. The paper reads these results as support for the idea that refusal type, refusal phrasing, and prompt sensitivity all shape user satisfaction.","pith_inferences":["Inference: If the revealed-preference pattern holds, a provider optimizing user engagement should reserve hard ethical refusals for genuinely harmful or illegal requests, since the data suggest this could recover a large share of the 37-point penalty without weakening safety.","Inference: Because Chatbot Arena voters skew toward AI enthusiasts, the size of the penalty may differ in general-population or domain-specific settings; that is an extrapolation beyond the paper's evidence.","Inference: The positive effect of length and similarity on ethical refusals suggests refusal wording is a controllable design variable: refusal templates could be generated or fine-tuned to be longer and more contextually anchored while keeping the decision to refuse unchanged.","Inference: The disclaimers result raises a testable design hypothesis the paper does not pursue: converting hard ethical refusals into ethical disclaimers that flag concerns but still give a qualified answer may trade a small satisfaction cost for a much smaller one."],"forward_implications":["Ethical refusals are the most disliked response type in the dataset, about twice as costly in win rate as technical refusals and far costlier than a normal answer.","Refusals can be made less costly without removing the refusal itself: longer, context-specific refusals and refusals to clearly unsafe prompts receive better user ratings.","Most ethical refusals in the data occur on prompts not flagged as unsafe, and those are the refusals users penalize most, pointing to miscalibrated moderation as a key driver of the penalty.","Disclaimers that state a limitation but still attempt the task are penalized much less than outright refusals, suggesting that partial engagement protects user satisfaction."],"supporting_citations":[{"why":"Supplies the Chatbot Arena pairwise-comparison dataset of about 50,000 single-turn matchups that all win-rate analyses are built on.","marker":"Chiang et al., 2024"},{"why":"Provides the RoBERTa-Large architecture that is fine-tuned into the refusal classifier.","marker":"Liu et al., 2019"},{"why":"Moderation API flags prompts as unsafe or not unsafe, operationalizing prompt sensitivity for the H4 analysis.","marker":"OpenAI, 2024b"},{"why":"SORRY-Bench is the existing refusal benchmark the paper extends by separating ethical from technical refusals in organic prompts.","marker":"Xie et al., 2024"},{"why":"Expectancy-violations theory grounds the prediction that ethical refusals violate relational expectations more than technical limits do.","marker":"Burgoon, 1993"},{"why":"CASA framework supports the claim that users treat LLMs as social partners, making moralizing refusals feel more personal and jarring.","marker":"Nass & Moon, 2000"},{"why":"Supports the opacity mechanism by which unexplained or judgment-like restrictions trigger user discomfort.","marker":"Eslami et al., 2019"},{"why":"Documents a length bias in LLM evaluation that motivates testing response length as a moderator of refusal penalties.","marker":"Huang et al., 2024"}],"fun_headline_variants":["Ethical refusals win just 8% of battles, vs 16% for technical","Users penalize ethical AI refusals: 8% vs 16% technical","Refusing for ethics drops LLM win rate to 8%","Safety refusals cost LLMs: 8% win rate, but phrasing helps"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire headline gap rests on the classifier's ability to tell ethical refusals from technical ones, and because the training labels came mostly from one author with only moderate agreement against a model-based second annotator, a systematic labeling error would directly distort the 37-point coefficient.","fun_headline_variants_meta":{"raw":{"variants":["Ethical refusals win just 8% of battles, vs 16% for technical","Users penalize ethical AI refusals: 8% vs 16% technical","Refusing for ethics drops LLM win rate to 8%","Safety refusals cost LLMs: 8% win rate, but phrasing helps"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000558,"raw_usage":{"total_tokens":2669,"prompt_tokens":973,"completion_tokens":1696,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":589,"completion_tokens_details":{"reasoning_tokens":1607}},"tokens_in":589,"tokens_out":1696,"duration_ms":12545,"temperature":1.0,"reasoning_tokens":1607,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:14:20.406210+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have several independent human raters label a random sample of responses the classifier calls Refusal Ethical and Refusal Technical, then re-estimate the win-rate regressions using consensus labels; if a material share of cases changes category, the 8%-versus-16% win-rate gap and the 37-point coefficient would shrink or disappear.","supporting_citations":[{"cited_title":"The dataset consists of 57,477 conversation pairs, with each pair comprising responses from two distinct models to a shared user prompt","cited_arxiv_id":null,"evidence_quote":"Supplies the Chatbot Arena pairwise-comparison dataset of about 50,000 single-turn matchups that all win-rate analyses are built on."},{"cited_title":"I’m sorry, I can’t help with that","cited_arxiv_id":null,"evidence_quote":"Provides the RoBERTa-Large architecture that is fine-tuned into the refusal classifier."},{"cited_title":"refusal penalty","cited_arxiv_id":null,"evidence_quote":"SORRY-Bench is the existing refusal benchmark the paper extends by separating ethical from technical refusals in organic prompts."}],"review_version":1}