REVIEW 23 cited by
SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Evaluating aligned large language models' (LLMs) ability to recognize and reject unsafe user requests is crucial for safe, policy-compliant deployments. Existing evaluation efforts, however, face three limitations that we address with SORRY-Bench, our proposed benchmark. First, existing methods often use coarse-grained taxonomies of unsafe topics, and are over-representing some fine-grained topics. For example, among the ten existing datasets that we evaluated, tests for refusals of self-harm instructions are over 3x less represented than tests for fraudulent activities. SORRY-Bench improves on this by using a fine-grained taxonomy of 44 potentially unsafe topics, and 440 class-balanced unsafe instructions, compiled through human-in-the-loop methods. Second, linguistic characteristics and formatting of prompts are often overlooked, like different languages, dialects, and more -- which are only implicitly considered in many evaluations. We supplement SORRY-Bench with 20 diverse linguistic augmentations to systematically examine these effects. Third, existing evaluations rely on large LLMs (e.g., GPT-4) for evaluation, which can be computationally expensive. We investigate design choices for creating a fast, accurate automated safety evaluator. By collecting 7K+ human annotations and conducting a meta-evaluation of diverse LLM-as-a-judge designs, we show that fine-tuned 7B LLMs can achieve accuracy comparable to GPT-4 scale LLMs, with lower computational cost. Putting these together, we evaluate over 50 proprietary and open-weight LLMs on SORRY-Bench, analyzing their distinctive safety refusal behaviors. We hope our effort provides a building block for systematic evaluations of LLMs' safety refusal capabilities, in a balanced, granular, and efficient manner. Benchmark demo, data, code, and models are available through https://sorry-bench.github.io.
Forward citations
Cited by 23 Pith papers
-
IatroBench: Pre-Registered Evidence of Iatrogenic Harm from AI Safety Measures
Frontier models systematically withhold clinically necessary guidance from laypeople that they provide to physicians on identical facts, and LLM judges fail to detect that omission harm.
-
The bitter lesson of misuse detection
A new benchmark for LLM supervision systems finds generalist models repurposed as harm classifiers outperform specialized commercial guardrails.
-
AISPA: User-Centric System Prompt Auditing for Large Language Model Applications
A human-in-the-loop audit of system prompts from 88 commercial AI products finds protective instructions nearly universal yet incomplete, with ~40% of products containing at least one user-harmful directive.
-
Item Response Theory for AI Safety
Using item response theory on 192 models and eight safety benchmarks, this paper finds three latent safety factors, cuts evaluation cost by 97-99% with adaptive item selection, and detects naive sandbagging and API mo...
-
Architectural Backdoors in Vision-Language Model Supply Chains via Representation Steering
Trigger-gated additive steering vectors embedded in VLM architecture definitions create dormant backdoors that work across VQA, text-to-image, retrieval, and brand/safety biasing without data poisoning.
-
BioTIER: A Refusal Benchmark for Targeted Biological Risk Mitigation
BioTIER, a 542-prompt benchmark with three risk tiers, shows frontier AI models differ by 90 percentage points in refusing dangerous biological queries, with top refusers over-refusing benign topics at the boundary.
-
Not All Refusals Are Equal: How Safety Alignment Fails Cybersecurity at Scale
Domain-specific abliteration removes cybersecurity refusal on Kimi K2 (100%→7%) while preserving most other safety refusals; susceptibility depends mainly on safety-training method and architecture, not size.
-
Chain-of-Models: Cross-Model Auditing for Bias-Robust LLM Judges
Per-bias selection of a cross-family LLM auditor lifts biased-judgment accuracy from 0.805/0.824 baselines to 0.884.
-
MetaBreak: Jailbreaking Online LLM Services via Special Token Manipulation
Special tokens that structure LLM conversations can be injected and swapped for lookalike words to bypass both built-in safety and external content filters.
-
YouthSafe: A Youth-Centric Safety Benchmark and Safeguard Model for Large Language Models
Introduces YAIR, a youth-GenAI risk benchmark, and YouthSafe, a fine-tuned classifier with AUPRC 0.94 on it, though it compares a trained model to untrained baselines.
-
Does More Inference-Time Compute Really Help Robustness?
With exposed reasoning chains, increasing inference-time compute consistently decreases measured robustness across 12 open-source reasoning models, while hidden chains show improvements.
-
`For Argument's Sake, Show Me How to Harm Myself!': Jailbreaking LLMs in Suicide and Self-Harm Contexts
Academic-framing prompts bypass safety filters in most tested LLMs, turning prior self-harm and suicide intent into detailed actionable instructions.
-
DABstep: Data Agent Benchmark for Multi-step Reasoning
DABstep releases 450+ real-world financial analytics tasks requiring iterative code and documentation reasoning; the best baseline solves 14.55% of hard tasks.
-
How Jailbreak Attacks Inform Safety Alignment: A Defender-Centric, Shapley-Based Evaluation of Jailbreak Contributions
Attacks that break LLMs best are not the ones that improve safety most; a Shapley- and greedy-based framework that selects attack subsets by downstream defender utility outperforms attacker-centric and attribution-onl...
-
Fence: Specialized SLM Guardrails for LLM Applications
Small language models trained on GAN-style adversarial synthetic data outperform prompted large models on use-case-specific guardrails for off-topic and prompt-injection detection.
-
MEDIC-AD: Towards Medical Vision-Language Model's Clinical Intelligence
MEDIC-AD adds anomaly-aware and difference tokens to a medical VLM, claiming SOTA lesion detection, temporal tracking, and visual grounding; the zero-shot claim is undermined by likely train/test overlap.
-
$C$-$\Delta\Theta$: Circuit-Restricted Weight Arithmetic for Selective Refusal
Selective refusal can be improved by editing only the small weight circuit found by EAP-IG, yielding offline checkpoints with low over-refusal.
-
R1-ACT: Efficient Reasoning Model Safety Alignment by Activating Safety Knowledge
Adding an explicit 'is this harmful?' step to the reasoning chain, trained on just 1,000 examples, substantially reduces harmful responses from reasoning models while roughly preserving benchmark reasoning performance.
-
Beyond Jailbreaks: Revealing Stealthier and Broader LLM Security Risks Stemming from Alignment Failures
JailFlipBench and JailFlip attacks show that leading LLMs can be made to answer benign-looking questions with plausible but factually wrong and dangerous responses.
-
Advancing Jailbreak Strategies: A Hybrid Approach to Exploiting LLM Vulnerabilities and Bypassing Modern Defenses
GCG+PAIR and GCG+WordGame hybrids show mixed attack-success improvements, but methodological flaws, including a questionable GCG loss term and a pre-generated baseline, undermine the paper's main claims.
-
Fine-Tuning Lowers Safety and Disrupts Evaluation Consistency
Fine-tuning small LLMs on benign data raises harmfulness scores, but those scores vary widely across random seeds, temperatures, and repeated runs, making single-run safety comparisons unreliable.
-
The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs
A structured survey of LLM safety evaluation that proposes a why/what/where/how taxonomy and catalogs metrics, datasets, benchmarks, evaluators, and frameworks.
-
Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges
A broad survey of LLM alignment that catalogs objectives, benchmarks, SFT/RLHF/DPO methods, and safety challenges, without contributing new experimental or theoretical results.
Discussion (0). Sign in to comment.