REVIEW 6 cited by
Safe Inputs but Unsafe Output: Benchmarking Cross-modality Safety Alignment of Large Vision-Language Model
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
As Artificial General Intelligence (AGI) becomes increasingly integrated into various facets of human life, ensuring the safety and ethical alignment of such systems is paramount. Previous studies primarily focus on single-modality threats, which may not suffice given the integrated and complex nature of cross-modality interactions. We introduce a novel safety alignment challenge called Safe Inputs but Unsafe Output (SIUO) to evaluate cross-modality safety alignment. Specifically, it considers cases where single modalities are safe independently but could potentially lead to unsafe or unethical outputs when combined. To empirically investigate this problem, we developed the SIUO, a cross-modality benchmark encompassing 9 critical safety domains, such as self-harm, illegal activities, and privacy violations. Our findings reveal substantial safety vulnerabilities in both closed- and open-source LVLMs, such as GPT-4V and LLaVA, underscoring the inadequacy of current models to reliably interpret and respond to complex, real-world scenarios.
Forward citations
Cited by 6 Pith papers
-
MDIT-Bench: Evaluating the Dual-Implicit Toxicity in Large Multimodal Models
The authors release a large multimodal benchmark showing that current LMMs struggle to detect toxicity that emerges only from combining image and text, and that many-shot toxic demonstrations further reduce their accuracy.
-
SafeWork-R1: Coevolving Safety and Intelligence under the AI-45$^{\circ}$ Law
SafeWork-R1 shows that a staged RL pipeline with safety, value, and knowledge verifiers can improve both safety and general reasoning scores over a base multimodal model.
-
Bridging the Gap in Vision Language Models in Identifying Unsafe Concepts Across Modalities
Vision-language models consistently recognize unsafe content better from text than from images, and a simplified reinforcement learning fine-tune narrows that gap.
-
ShieldVLM: Safeguarding the Multimodal Implicit Toxicity via Deliberative Reasoning with LVLMs
ShieldVLM detects multimodal implicit toxicity through deliberate cross-modal reasoning, outperforming existing moderation APIs and models on the new MMIT benchmark.
-
Seeing the Threat: Vulnerabilities in Vision-Language Models to Adversarial Attack
A two-stage evaluation framework and token-projection analysis show that LVLMs encode harmful semantic cues from images even without OCR, while remaining vulnerable to cross-modal attacks.
-
A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluations
A survey of LVLM safety that adds a lifecycle taxonomy and new benchmark results showing Janus-Pro-7B has weaker safety than several open-source LVLMs.
Discussion (0). Continue with ORCID to comment.