Pith. sign in

REVIEW 5 major objections 6 minor 71 references

Bridging the Gap in Vision Language Models in Identifying Unsafe Concepts Across Modalities

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that the eight tested vision-language models consistently flag unsafe concepts when they appear as text but often call images of the same concepts safe, and that a simplified PPO fine-tune using a RoBERTa classifier…

desk verdict A useful new fine-grained unsafe-concept benchmark and a plausible cross-modal evaluation, plus a promising but not yet fully validated classifier-reward PPO recipe. read the letter →

arxiv 2507.11155 v1 pith:QITTRFOI submitted 2025-07-15 cs.CR cs.AI

Yiting Qu , Michael Backes , Yang Zhang This is my paper
classification cs.CRcs.AI
keywords vision-languagemodelmodalitygapunsafeconceptRLHFPPOcontentmoderationsafetyalignmentimagedataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Vision-language models (VLMs) are supposed to judge whether images are safe, and this paper argues that the open-source ones have a consistent blind spot: they can name an unsafe concept when asked what an image shows, but when asked whether that same image is safe to view or generate, they often answer "yes," while flagging the identical concept described in text. To establish this, the authors build UnsafeConcepts, a dataset of 75 fine-grained unsafe concepts and 1,567 expert-annotated images, and measure eight VLMs on perception (recognizing the concept) and alignment (judging it unsafe). They then introduce a simplified reinforcement-learning fine-tune—PPO with reward scores from a RoBERTa response classifier plus a length bonus—that raises the unsafe-image alignment of one open-source VLM, LLaVA-7B, from 0.510 to 0.884 while keeping general capability scores nearly unchanged, and that outperforms supervised fine-tuning and direct preference optimization. If correct, the paper shows that a wide class of deployed VLMs under-report visual harm and that this can be patched without human preference data or a separate reward model.

What carries the argument

The load-bearing mechanism is the perception/alignment decomposition of safety judgment plus a simplified proximal policy optimization (PPO) loop. Perception is scored with single-choice questions in which the VLM must pick the correct unsafe concept among distractors; alignment is scored by asking five general-context questions ("safe to view on social media?", "safe to generate with AI?") for images versus for the same concept described in text. The proposed fine-tune replaces the standard RLHF reward model with the RoBERTa-based response classifier: the reward is the negative cross-entropy between the classifier's safe/unsafe prediction and the ground-truth label, plus a length bonus that penalizes one-word answers, while an entropy bonus and a KL-divergence term to the original policy keep training stable. Because the same classifier serves as the evaluation metric and the reward signal, the optimization directly targets the measured modality gap, and it allows the method to skip supervised-fine-tuning initialization and human preference collection entirely.

What would settle it

Take the PPO-tuned LLaVA-7B, present every image from the UnsafeConcepts test set that it labels unsafe, and ask it in free text to name the specific unsafe concept, with human annotators scoring whether the name is correct; if the model says "unsafe" but cannot name the concept, the alignment gain is a refusal reflex, whereas accurate naming would confirm that the visual concept is genuinely recognized.

Watch

Extended reading notes

Core claim

The paper's central claim is that there is a consistent modality gap across the eight tested VLMs: textual unsafe concepts are almost always judged unsafe (alignment scores 0.81–0.95), whereas visual depictions of the same concepts are judged unsafe far less often, with overall alignment scores as low as 0.14 for one open-source model even though perception accuracy (0.90–0.97 for most models) shows the concept is recognized. The second claim is that this gap can be substantially bridged by a simplified RLHF procedure that uses the same RoBERTa classifier that measured alignment as the PPO reward signal, with a length bonus to discourage terse refusals and KL and entropy terms to protect general abilities. On LLaVA-7B the method raises across-the-board alignment accuracy from 0.736 to 0.903 and the unsafe-split accuracy from 0.510 to 0.884, while the general-capability score drops only from 0.708 to 0.687; the same procedure also generalizes to two out-of-domain safety datasets and produces more informative responses than SFT or DPO.

Load-bearing premise

The load-bearing premise is that the RoBERTa response classifier—trained on only 480 alignment annotations and 240 perception annotations—is an unbiased measure of whether a VLM response actually identifies an image as unsafe; if that classifier is biased, the reported modality gap and the PPO improvement could be artifacts of the measurement instrument.

Editorial extensions

If this is right

  • Safety evaluations that only probe text inputs will overstate a VLM's real-world behavior; image-based safety tests are the binding constraint.
  • The simplified recipe—classifier-as-reward, no SFT stage, no learned reward model—is claimed to be sufficient for this safety-alignment task, which lowers the cost of aligning VLMs to a safety standard.
  • Because PPO preserves general capabilities better than SFT or DPO, it offers a calibration path for already-deployed models rather than a full retraining.
  • The concept-level UnsafeConcepts labels give future safety work a finer-grained benchmark than binary safe/unsafe labels for measuring both recognition and judgment.
  • The gap is largest in general contexts and shrinks when explicit unsafe cues are given, so content-moderation systems that lack context face the hardest case.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable implication the paper leaves implicit is that the root cause is data scarcity: unsafe images are rare and often filtered from pretraining corpora, so the language backbone already knows the concepts while the vision side never maps them; if so, adding a few thousand unsafe image-caption pairs during pretraining should narrow the gap more cheaply than RLHF.
  • Because the reward signal and the evaluation metric are the same classifier, part of the measured PPO gain could reflect the model learning to satisfy that classifier—for example, reflexive refusal—rather than genuine concept recognition; an audit that asks the tuned model to justify each "unsafe" answer and verifies the specific concept is named would separate these.
  • The perception/alignment decomposition is general: the same two-question design could be applied to non-safety value judgments such as misinformation, deception, or political bias, where a similar visual blind spot may exist.
  • The method's transfer to two external safety datasets suggests the PPO objective generalizes across taxonomies; a natural next probe is whether it also closes the gap for video frames or typographic/adversarial images of unsafe concepts.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. This paper compiles UnsafeConcepts, a dataset of 75 unsafe concepts with 1,567 expert-annotated images, and uses it to evaluate eight VLMs along two axes: perception (multiple-choice recognition of the depicted concept) and alignment (whether the model judges the content unsafe in general contexts). The authors report that most open-source VLMs perceive unsafe concepts accurately yet often fail to label visual versions as unsafe, while performing much better on textual descriptions, yielding a consistent modality gap. To close the gap, they fine-tune LLaVA-7B with a simplified PPO-based RLHF procedure in which a RoBERTa response classifier supplies the reward, and they report that this improves alignment accuracy and response quality over SFT and DPO with only a small loss in general capabilities. The paper also reports generalization on SMID and NSFW and a small human evaluation.

Significance. The UnsafeConcepts dataset is a useful contribution: the taxonomy is grounded in existing content policies, the annotation process includes Fleiss' kappa, and the perception/alignment decomposition is a clean way to separate visual recognition from ethical judgment. If the modality-gap finding survives an independent measurement, it would be an important result for content-moderation and VLM safety. The proposed classifier-reward PPO is also interesting because it removes the need for preference data, and the paper deserves credit for reporting external-dataset generalization and a human evaluation. However, the headline conclusions currently rest on a single classification instrument used both as the training reward and as the evaluation metric, and on a learning-rate selection protocol that is not specified; those issues must be resolved before the empirical claims can be taken at face value.

major comments (5)
  1. [§4.2, §5.3, Table 3] The central claim that PPO 'effectively enhances VLM alignment on images' is supported by an Alignment Score computed with the same RoBERTa classifier that supplies the PPO reward. The classifier was fine-tuned on 480 alignment annotations (60 per VLM, Section 4.2), a set that includes responses from the target LLaVA-7B; optimizing the policy against this classifier and then reporting accuracy with the same classifier means that the gains in Table 3 can reflect exploitation of classifier artifacts (e.g., longer refusals, particular phrasings) rather than genuine recognition of unsafe concepts. I would need an evaluation that is independent of the reward instrument: for example, human rescoring of the full Alignment-Test (or of a sufficiently large random subset), or a held-out classifier that was never used during training. Without this, the central comparison in Table 3 is not decisive.
  2. [§5.4, Table 3] The manuscript states that the learning rate is chosen as 'the learning rate that yields the best performance for each method' but does not state which split is used for this selection. If any trial evaluated Alignment-Test, then the reported PPO advantage over SFT and DPO is optimistically biased by selection on the test set. Please specify the validation protocol (e.g., a held-out split of the alignment training set) and report the scores of the non-selected trials, or at least the selected learning rates for each method on a clearly separate validation set.
  3. [§4.2, §5.5] The response-processing rule treats refusal responses such as 'Sorry, I cannot help' as 'Unsafe' and gives them full credit in the Alignment Score. Because PPO is rewarded by this classifier, the model can increase its reported alignment accuracy by producing generic refusals rather than by identifying the depicted concept; the length bonus and 1-SelfBLEU are too coarse to prevent this, since a longer refusal also receives the length bonus. The human evaluation in Section 5.4 covers only 100 responses and is not used to re-score the full Alignment-Test. Please report the distribution of response types (concept-identifying, generic refusal, safe/unsafe judgment) before and after fine-tuning, and re-score a representative sample of the test responses by human annotators.
  4. [§4.2, Figure 5] The claimed modality gap is also measured by the same RoBERTa classifier, and the paper does not validate that the classifier is modality-invariant. If the classifier is more likely to map visual-domain responses to 'Safe' than textual-domain responses (or vice versa), the gap in Figure 5 could be an artifact of the measurement instrument. Please provide a per-modality validation of the classifier against human judgments, and/or report the modality gap computed directly from human-annotated responses for a subsample.
  5. [§5.4, Table 3] The SFT and DPO baselines are trained on fixed template responses, which yields near-zero response diversity (1−SelfBLEU of 0.076 and 0.111 in Table 3). The claim that PPO 'outperforms' these baselines is therefore partly built into the baseline design; a template-based SFT is not representative of the SFT stage used in standard RLHF pipelines. I would like to see either a stronger SFT/DPO baseline trained on diverse, high-quality responses (even if generated by a strong VLM) or a clear statement that the comparison is only against template-based variants.
minor comments (6)
  1. [§4.1, Figures 11 and 12] The perception classifier is trained on 240 responses and the alignment classifier on 480 responses, each with a 6:4 split; please report per-class sample sizes and confidence intervals for the accuracy and F1 values shown in Figures 11 and 12, since the test sets are very small.
  2. [§5.3, Eq. (4)] The notation P(x) log P(y) is not defined as a cross-entropy over the classifier's class distribution; please clarify the exact computation of the reward term.
  3. [§5.4] 1−SelfBLEU is described as a response quality score and called 'informativeness' in places, but SelfBLEU measures repetitiveness, not informativeness; please relabel this as response diversity and rely on the human evaluation for informativeness claims.
  4. [§4.1] The generation setting 'top_k to 0.9' is likely top_p for nucleus sampling; please correct or clarify.
  5. [Related Work] Reference [23] appears twice in the related-work section; additionally, the sentence in Section 2.1 saying 'eight VLMs from six VLM families' lists six model names rather than eight checkpoints.
  6. [Table 3] It would help to report the number of training steps and the variance across seeds for the alignment and general-capability scores, since PPO training is stochastic and the current table reports point estimates only.

Circularity Check

2 steps flagged · score 6.0 of 10

The reported PPO alignment gains are measured with the same RoBERTa classifier that supplies the training reward, so the headline increases in Tables 3 and 5 track reward optimization by construction.

  1. self definitional [Section 5.3 (Evaluation phase and Eq. (4)); Section 5.4 (Evaluation Metrics); Section 4.2 (Response Processing)]
    "To design the reward score, we utilize the same RoBERTa classifier (see alignment measurement in Section 4.2). Specifically, we calculate the cross-entropy loss between the predicted classes and the ground-truth labels, and design the reward score based on the negative loss."

    The metric behind the paper's central claim is computed by the very classifier that defines the training reward. Section 4.2's Alignment Score is the percentage of images for which the RoBERTa classifier (trained on 480 alignment annotations, 60 per VLM, including the target LLaVA-7B) labels the response Unsafe; Section 5.3's reward is the negative cross-entropy of that same classifier against the same ground-truth labels; Section 5.4 then reports 'the same alignment accuracy.' Maximizing Eq. (4) is therefore, by construction, maximizing the quantity reported in Tables 3 and 5.

  2. self definitional [Section 4.2 (Response Processing); Section 5.3 (Evaluation)]
    "For refusal responses such as "Sorry, I cannot help with that," we categorize them into the "Unsafe" class, as the inputs have triggered the VLM's safeguards due to the harmful nature."

    The target category 'Unsafe' — the sole object of both reward and metric — is defined to include responses that never identify the concept. A policy that answers 'Sorry, I cannot help' receives full Alignment Score credit, so the claim that PPO 'enhances VLM alignment on images' is partly a claim about producing the classifier's preferred refusal pattern rather than recognizing the specific unsafe concept. The paper itself concedes the hazard ('using a single-dimensional reward score can often lead to reward hacking') and grafts on a hand-tuned length bonus γ·ℓ(y) to compensate, acknowledging that the raw classifier reward is not the desired behavior; the corrective bonus is an additional parameter, not an independent measurement of concept recognition.

full rationale

Most of the paper is not circular. UnsafeConcepts is built from UnsafeBench images with concept-level human annotation (Fleiss kappa 0.682); the perception/alignment measurements on eight VLMs are observations; and the modality-gap finding, while inheriting the RoBERTa classifier's validity, is not fitted to force a predetermined outcome. The circularity is concentrated in the claim that the simplified PPO 'effectively enhances VLM alignment on images.' The training reward (Eq. (4)) is the negative cross-entropy of the Section 4.2 RoBERTa classifier against ground-truth labels, and the reported alignment accuracy (Tables 3 and 5) is the same classifier's Safe/Unsafe/N-A verdict on the same prompts — the evaluation metric is the training objective renamed, and any policy that optimizes R(x,y) will by construction score higher on it. The refusal-counts-as-Unsafe rule tightens the loop, and the paper's own length bonus is an admission that the raw signal is hackable. Independent content is real but narrow: the 100-response human evaluation (PPO Soundness 4.659 vs SFT 5.000; Informativeness 4.682 vs 1.978) and the kappa-annotated dataset. The external SMID/NSFW results still use the same classifier, so they extend the closed loop to new images rather than break it. A compounding hygiene concern is that Section 5.4 selects the learning rate that 'yields the best performance for each method' without stating whether selection used the test split. Because one central claim reduces by construction to its reward function while genuine independent checks anchor part of the contribution, the score is 6 rather than 8.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claims rest mostly on the annotated dataset and the classifier-based reward, both of which are constructed in this paper. No new physical or conceptual entities are postulated.

free parameters (6)
  • RoBERTa classifier parameters = trained on 240 perception and 480 alignment annotated responses
    Used as both the reward model and the evaluation metric; its parameters encode the ethical judgment and directly shape the central results.
  • Length bonus coefficient gamma = 4
    Set from ablation (Appendix B); controls response length reward, affects 1-SelfBLEU.
  • Entropy bonus coefficient lambda = 0.02
    Set from ablation; higher values hurt alignment and general scores.
  • KL coefficient beta = 0.1
    Set from ablation; balances alignment and general capability retention.
  • Learning rates = 3e-5 for SFT and PPO, 2e-6 for DPO
    Chosen as best-performing among 1e-6 to 1e-4 without a stated validation split.
  • LoRA rank, batch size, epochs = rank 128, batch 32, epochs 4
    Training hyperparameters held constant across methods.
assumptions (4)
  • domain assumption The 75 unsafe concepts and their annotations represent a valid unified human ethical standard for general safety contexts.
    The paper uses UnsafeConcepts as 'a proxy to represent the human-defined ethical standard' (Section 3) and acknowledges in Limitations that a single standard fails to capture context nuance.
  • domain assumption The RoBERTa classifier maps open-ended VLM responses to Safe, Unsafe, and N/A classes accurately enough to serve as both reward and evaluation.
    Trained on 480 alignment and 240 perception annotated responses (Sections 4.1 and 4.2), with reported test accuracy near 1.0; no external validation of the classifier is provided.
  • domain assumption The five general-context prompts capture contexts in which these concepts are typically considered unsafe.
    Prompts were designed by the authors and supplemented by OpenAI o1 (Section 4.2); the safe and unsafe cue case study shows context dependence.
  • domain assumption Reward hacking is sufficiently mitigated by the length bonus and KL penalty.
    The paper relies on these terms to prevent degenerate 'No' responses (Section 5.3); no direct evidence that the final policy is not hacking the classifier is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Bridging the Gap in Vision Language Models in Identifying Unsafe Concepts Across Modalities." pith.science (2026). https://pith.science/paper/QITTRFOI

@misc{pith2026250711155,
  author       = {Pith},
  title        = {Pith review of: Bridging the Gap in Vision Language Models in Identifying Unsafe Concepts Across Modalities},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QITTRFOI}},
  note         = {Machine review of arXiv:2507.11155}
}
read the original abstract

Vision-language models (VLMs) are increasingly applied to identify unsafe or inappropriate images due to their internal ethical standards and powerful reasoning abilities. However, it is still unclear whether they can recognize various unsafe concepts when presented in different modalities, such as text and images. To address this, we first compile the UnsafeConcepts dataset, featuring 75 unsafe concepts, i.e., ``Swastika,'' ``Sexual Harassment,'' and ``Assaults,'' along with associated 1.5K images. We then conduct a systematic evaluation of VLMs' perception (concept recognition) and alignment (ethical reasoning) capabilities. We assess eight popular VLMs and find that, although most VLMs accurately perceive unsafe concepts, they sometimes mistakenly classify these concepts as safe. We also identify a consistent modality gap among open-source VLMs in distinguishing between visual and textual unsafe concepts. To bridge this gap, we introduce a simplified reinforcement learning (RL)-based approach using proximal policy optimization (PPO) to strengthen the ability to identify unsafe concepts from images. Our approach uses reward scores based directly on VLM responses, bypassing the need for collecting human-annotated preference data to train a new reward model. Experimental results show that our approach effectively enhances VLM alignment on images while preserving general capabilities. It outperforms baselines such as supervised fine-tuning (SFT) and direct preference optimization (DPO). We hope our dataset, evaluation findings, and proposed alignment solution contribute to the community's efforts in advancing safe VLMs.

Figures

Figures reproduced from arXiv: 2507.11155 by the authors.

Figure 1
Figure 1. An example of a modality gap where the unsafe con [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Construction of the UnsafeConcepts dataset. We use the Harassment category as an example. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Examples of unsafe images in the UnsafeConcepts [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Perception Scores of eight VLMs in recognizing the [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Alignment Scores of eight VLMs in identifying visual and textual unsafe concepts. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: A misaligned example between visual and textual un [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Impact of entropy bonus on performance for align [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Impact of KL coefficient on performance for align [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 10
Figure 10. Figure 10: An example of the single-choice questions in percep [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]
Figure 9
Figure 9. Figure 9: Impact of length bonus on performance for alignment [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 12
Figure 12. Figure 12: Performance of the RoBERTa classifier on the test [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

71 extracted references · 36 canonical work pages

  1. [1]

    https://huggingface.co/deepseek-ai/ 13 DeepSeek-R1

    DeepSeek-R1. https://huggingface.co/deepseek-ai/ 13 DeepSeek-R1. 2

    work page

  2. [2]

    https://openai.com/index/gpt-4-1/

    GPT-4.1. https://openai.com/index/gpt-4-1/. 8

    work page

  3. [3]

    https://openai.com/research/gpt-4v- system-card

    GPT-4V. https://openai.com/research/gpt-4v- system-card. 3, 17

    work page

  4. [4]

    https://huggingface.co/datasets/laion/ laion2B-en

    LAION-2B. https://huggingface.co/datasets/laion/ laion2B-en. 7, 17

    work page

  5. [5]

    https: //academictorrents.com/details/ 1cda9427784a6b77809f657e772814dc766b69f5

    NudeNet Classifier Dataset v1. https: //academictorrents.com/details/ 1cda9427784a6b77809f657e772814dc766b69f5. 11, 12

    work page

  6. [6]

    https://web.archive.org/web/ 20220406151527/https://labs.openai.com/policies/ content-policy

    OpenAI Content Policy. https://web.archive.org/web/ 20220406151527/https://labs.openai.com/policies/ content-policy. 3

    work page

  7. [7]

    https://openai.com/o1/

    OpenAI o1. https://openai.com/o1/. 6

    work page

  8. [8]

    https://osf.io/2rqad/

    SMID Dataset. https://osf.io/2rqad/. 11

    work page

Show all 71 references
  1. [9]

    Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Ka- davath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nel- son Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tris-...

    2022 arXiv

  2. [10]

    Designing Neural Network Architectures using Rein- forcement Learning

    Bowen Baker, Otkrist Gupta, Nikhil Naik, and Ramesh Raskar. Designing Neural Network Architectures using Rein- forcement Learning. In International Conference on Learning Representations (ICLR), 2017. 2, 3

    2017

  3. [11]

    Image Safeguarding: Reasoning with Conditional Vision Language Model and Obfuscating Unsafe Content Counterfactually

    Mazal Bethany, Brandon Wherry, Nishant Vishwamitra, and Peyman Najafirad. Image Safeguarding: Reasoning with Conditional Vision Language Model and Obfuscating Unsafe Content Counterfactually. In AAAI Conference on Artificial Intelligence (AAAI), pages 774–782. AAAI, 2024. 1, 12

    2024

  4. [12]

    InternLM2 Technical Re- port

    Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, Xiaoyi Dong, Haodong Duan, Qi Fan, Zhaoye Fei, Yang Gao, Jiaye Ge, Chenya Gu, Yuzhe Gu, Tao Gui, Aijia Guo, Qipeng Guo, Conghui He, Yingfan Hu, Ting Huang, Tao Jiang,...

    2024 arXiv

  5. [13]

    ShareGPT4V: Improving Large Multi-Modal Models with Better Captions

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. ShareGPT4V: Improving Large Multi-Modal Models with Better Captions. CoRR abs/2311.12793, 2023. 17

    2023 arXiv

  6. [14]

    Christiano, Jan Leike, Tom B

    Paul F. Christiano, Jan Leike, Tom B. Brown, Miljan Mar- tic, Shane Legg, and Dario Amodei. Deep Reinforcement Learning from Human Preferences. In Annual Conference on Neural Information Processing Systems (NIPS), pages 4299–

  7. [15]

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven C. H. Hoi. InstructBLIP: Towards General- purpose Vision-Language Models with Instruction Tuning. In Annual Conference on Neural Information Processing S...

    2023

  8. [16]

    ImageNet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 248–255. IEEE, 2009. 10

    2009

  9. [17]

    ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Time

    Yi Ding, Bolian Li, and Ruqi Zhang. ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Time. CoRR abs/2410.06625, 2024. 12

    2024 arXiv

  10. [18]

    InternLM-XComposer2: Mastering Free- form Text-Image Composition and Comprehension in Vision- Language Large Model

    Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Bin Wang, Linke Ouyang, Xilin Wei, Songyang Zhang, Haodong Duan, Maosong Cao, Wenwei Zhang, Yining Li, Hang Yan, Yang Gao, Xinyue Zhang, Wei Li, Jingwen Li, Kai Chen, Conghui He, Xingcheng Zhang, Yu Qiao, Dahua Lin, and Jiaqi Wa...

    2024 arXiv

  11. [19]

    Heller, Stephen Pfohl, Deepak Ramachandran, Pe- ter Shaw, and Jonathan Berant

    Jacob Eisenstein, Chirag Nagpal, Alekh Agarwal, Ahmad Beirami, Alex D’Amour, Dj Dvijotham, Adam Fisch, Kather- ine A. Heller, Stephen Pfohl, Deepak Ramachandran, Pe- ter Shaw, and Jonathan Berant. Helping or Herding? Re- ward Model Ensembles Mitigate but do not Eliminate Rewar...

    2023 arXiv

  12. [20]

    Fleiss’ kappa statistic without paradoxes

    Rosa Falotico and Piero Quatto. Fleiss’ kappa statistic without paradoxes. Quality & Quantity, 2015. 4

    2015

  13. [21]

    Measuring Nominal Scale Agreement Among Many Raters

    Joseph L Fleiss. Measuring Nominal Scale Agreement Among Many Raters. Psychological Bulletin, 1971. 4

    1971

  14. [22]

    MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models

    Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Meng- dan Zhang, Xu Lin, Zhenyu Qiu, Wei Lin, Jinrui Yang, Xi- awu Zheng, Ke Li, Xing Sun, and Rongrong Ji. MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models. CoRR abs/2306.13394, 2023. 5, 10

    2023 arXiv

  15. [23]

    Fig- Step: Jailbreaking Large Vision-language Models via Typo- graphic Visual Prompts

    Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tian- shuo Cong, Anyu Wang, Sisi Duan, and Xiaoyun Wang. Fig- Step: Jailbreaking Large Vision-language Models via Typo- graphic Visual Prompts. CoRR abs/2311.05608, 2023. 1, 12

    2023 arXiv

  16. [24]

    Kwok, and Yu Zhang

    Yunhao Gou, Kai Chen, Zhili Liu, Lanqing Hong, Hang Xu, Zhenguo Li, Dit-Yan Yeung, James T. Kwok, and Yu Zhang. Eyes Closed, Safety on: Protecting Multimodal LLMs via Image-to-Text Transformation. In European Conference on Computer Vision (ECCV), pages 388–404. Springer, 2024. 12

    2024

  17. [25]

    Moderating Illicit Online Image Promo- tion for Unsafe User-Generated Content Games Using Large Vision-Language Models

    Keyan Guo, Ayush Utkarsh, Wenbo Ding, Isabelle On- dracek, Ziming Zhao, Guo Freeman, Nishant Vishwamitra, and Hongxin Hu. Moderating Illicit Online Image Promo- tion for Unsafe User-Generated Content Games Using Large Vision-Language Models. In USENIX Security Symposium (USENI...

    2024

  18. [26]

    LLavaGuard: VLM-based Safeguards for Vision Dataset Curation and Safety Assess- ment

    Lukas Helff, Felix Friedrich, Manuel Brack, Kristian Kerst- ing, and Patrick Schramowski. LLavaGuard: VLM-based Safeguards for Vision Dataset Curation and Safety Assess- ment. CoRR abs/2406.05113, 2024. 1, 3, 12

    2024 arXiv

  19. [27]

    Glass, Akash Srivastava, and Pulkit Agrawal

    Zhang-Wei Hong, Idan Shenfeld, Tsun-Hsuan Wang, Yung- Sung Chuang, Aldo Pareja, James R. Glass, Akash Srivastava, and Pulkit Agrawal. Curiosity-driven Red-teaming for Large Language Models. In International Conference on Learning Representations (ICLR), 2024. 10 14

    2024

  20. [28]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations (ICLR), 2022. 11

    2022

  21. [29]

    Deep Reinforcement Learning for Di- alogue Generation

    Jiwei Li, Will Monroe, Alan Ritter, Dan Jurafsky, Michel Gal- ley, and Jianfeng Gao. Deep Reinforcement Learning for Di- alogue Generation. In Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages 1192–1202. ACL, 2016. 2, 3

    2016

  22. [30]

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven C. H. Hoi. BLIP: Bootstrapping Language-Image Pre-training for Uni- fied Vision-Language Understanding and Generation. CoRR abs/2201.12086, 2022. 16

    2022 arXiv

  23. [31]

    Silkie: Preference Distillation for Large Visual Lan- guage Models

    Lei Li, Zhihui Xie, Mukai Li, Shunian Chen, Peiyi Wang, Liang Chen, Yazheng Yang, Benyou Wang, and Lingpeng Kong. Silkie: Preference Distillation for Large Visual Lan- guage Models. CoRR abs/2312.10665, 2023. 12

    2023 arXiv

  24. [32]

    Red Teaming Visual Language Models

    Mukai Li, Lei Li, Yuwei Yin, Masood Ahmed, Zhenguang Liu, and Qi Liu. Red Teaming Visual Language Models. In Annual Meeting of the Association for Computational Lin- guistics (ACL), pages 3326–3342. ACL, 2024. 12

    2024

  25. [33]

    GOAT-Bench: Safety Insights to Large Multi- modal Models through Meme-Based Social Abuse

    Hongzhan Lin, Ziyang Luo, Bo Wang, Ruichao Yang, and Jing Ma. GOAT-Bench: Safety Insights to Large Multi- modal Models through Meme-Based Social Abuse. CoRR abs/2401.01523, 2024. 12

    2024 arXiv

  26. [34]

    Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C

    Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO: Common Objects in Context. In European Conference on Computer Vision (ECCV), pages 740–755. Springer, 2014. 17

    2014

  27. [35]

    Improved Baselines with Visual Instruction Tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved Baselines with Visual Instruction Tuning. CoRR abs/2310.03744, 2023. 9

    2023 arXiv

  28. [36]

    Visual Instruction Tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual Instruction Tuning. In Annual Conference on Neural Information Processing Systems (NeurIPS). NeurIPS, 2023. 1, 3, 9, 10, 16

    2023

  29. [37]

    Unraveling and Mitigating Safety Alignment Degradation of Vision-Language Models

    Qin Liu, Chao Shang, Ling Liu, Nikolaos Pappas, Jie Ma, Neha Anna John, Srikanth Doss, Lluís Màrquez, Miguel Ballesteros, and Yassine Benajiba. Unraveling and Mitigating Safety Alignment Degradation of Vision-Language Models. CoRR abs/2410.09047, 2024. 12

    arXiv

  30. [38]

    MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models

    Xin Liu, Yichen Zhu, Jindong Gu, Yunshi Lan, Chao Yang, and Yu Qiao. MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models. In Eu- ropean Conference on Computer Vision (ECCV), pages 386–

  31. [39]

    RoBERTa: A Robustly Opti- mized BERT Pretraining Approach

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettle- moyer, and Veselin Stoyanov. RoBERTa: A Robustly Opti- mized BERT Pretraining Approach. CoRR abs/1907.11692,

    1907 arXiv

  32. [40]

    Safety Alignment for Vision Language Models

    Zhendong Liu, Yuanbi Nie, Yingshui Tan, Xiangyu Yue, Qiushi Cui, Chongjun Wang, Xiaoyong Zhu, and Bo Zheng. Safety Alignment for Vision Language Models. CoRR abs/2405.13581, 2024. 12

    2024 arXiv

  33. [41]

    From Meme to Threat: On the Hateful Meme Understanding and Induced Hateful Content Generation in Open-Source Vision Language Mod- els

    Yihan Ma, Xinyue Shen, Yiting Qu, Ning Yu, Michael Backes, Savvas Zannettou, and Yang Zhang. From Meme to Threat: On the Hateful Meme Understanding and Induced Hateful Content Generation in Open-Source Vision Language Mod- els. In USENIX Security Symposium (USENIX Security) . ...

    2025

  34. [42]

    OpenAI. GPT-4o. https://openai.com/index/hello- gpt-4o/. 1, 10

  35. [43]

    GPT-4 Technical Report

    OpenAI. GPT-4 Technical Report. CoRR abs/2303.08774 ,

    arXiv

  36. [44]

    Efros, and Trevor Darrell

    Deepak Pathak, Pulkit Agrawal, Alexei A. Efros, and Trevor Darrell. Curiosity-driven Exploration by Self-supervised Pre- diction. In International Conference on Machine Learning (ICML), pages 2778–2787. PMLR, 2017. 10

    2017

  37. [45]

    Visual Adversarial Examples Jail- break Large Language Models

    Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Mengdi Wang, and Prateek Mittal. Visual Adversarial Examples Jail- break Large Language Models. CoRR abs/2306.13213, 2023. 12

    2023 arXiv

  38. [46]

    On the Evolution of (Hateful) Memes by Means of Multimodal Contrastive Learn- ing

    Yiting Qu, Xinlei He, Shannon Pierson, Michael Backes, Yang Zhang, and Savvas Zannettou. On the Evolution of (Hateful) Memes by Means of Multimodal Contrastive Learn- ing. In IEEE Symposium on Security and Privacy (S&P) . IEEE, 2023. 12

    2023

  39. [47]

    Unsafe Diffusion: On the Gen- eration of Unsafe Images and Hateful Memes From Text-To- Image Models

    Yiting Qu, Xinyue Shen, Xinlei He, Michael Backes, Savvas Zannettou, and Yang Zhang. Unsafe Diffusion: On the Gen- eration of Unsafe Images and Hateful Memes From Text-To- Image Models. In ACM SIGSAC Conference on Computer and Communications Security (CCS). ACM, 2023. 3, 6, 12

    2023

  40. [48]

    UnsafeBench: Benchmarking Image Safety Classifiers on Real-World and AI-Generated Im- ages

    Yiting Qu, Xinyue Shen, Yixin Wu, Michael Backes, Savvas Zannettou, and Yang Zhang. UnsafeBench: Benchmarking Image Safety Classifiers on Real-World and AI-Generated Im- ages. CoRR abs/2405.03486, 2024. 1, 3, 11, 12

    2024 arXiv

  41. [49]

    Learning Transferable Visual Models From Natural Language Supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning Transferable Visual Models From Natural Language Supervision. In International Con...

    2021

  42. [50]

    Manning, Stefano Ermon, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. Direct Prefer- ence Optimization: Your Language Model is Secretly a Re- ward Model. In International Conference on Learning Repre- sentations (ICLR). ICLR, 2023. 2, 3, 9

    2023

  43. [51]

    Zero shot VLMs for hate meme detection: Are we there yet? CoRR abs/2402.12198, 2024

    Naquee Rizwan, Paramananda Bhaskar, Mithun Das, Swad- hin Satyaprakash Majhi, Punyajoy Saha, and Animesh Mukherjee. Zero shot VLMs for hate meme detection: Are we there yet? CoRR abs/2402.12198, 2024. 1, 3, 12

    2024 arXiv

  44. [52]

    Safe Latent Diffusion: Mitigating In- appropriate Degeneration in Diffusion Models

    Patrick Schramowski, Manuel Brack, Björn Deiseroth, and Kristian Kersting. Safe Latent Diffusion: Mitigating In- appropriate Degeneration in Diffusion Models. CoRR abs/2211.05105, 2022. 3

    2022 arXiv

  45. [53]

    Proximal Policy Optimization Algo- rithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Rad- ford, and Oleg Klimov. Proximal Policy Optimization Algo- rithms. CoRR abs/1707.06347, 2017. 2, 3, 9, 12

    2017 arXiv

  46. [54]

    HateBench: Benchmarking Hate Speech Detectors on LLM-Generated Content and Hate Cam- paigns

    Xinyue Shen, Yixin Wu, Yiting Qu, Michael Backes, Savvas Zannettou, and Yang Zhang. HateBench: Benchmarking Hate Speech Detectors on LLM-Generated Content and Hate Cam- paigns. In USENIX Security Symposium (USENIX Security) . USENIX, 2025. 1

    2025

  47. [55]

    Assessment of Multimodal Large Language Mod- els in Alignment with Human Values

    Zhelun Shi, Zhipin Wang, Hongxing Fan, Zaibin Zhang, Li- jun Li, Yongting Zhang, Zhenfei Yin, Lu Sheng, Yu Qiao, and 15 Jing Shao. Assessment of Multimodal Large Language Mod- els in Alignment with Human Values. CoRR abs/2403.17830,

    arXiv

  48. [56]

    Joar Skalse, Nikolaus H. R. Howe, Dmitrii Krasheninnikov, and David Krueger. Defining and Characterizing Reward Gaming. In Annual Conference on Neural Information Pro- cessing Systems (NeurIPS). NeurIPS, 2022. 9

    2022

  49. [57]

    Align- ing Large Multimodal Models with Factually Augmented RLHF

    Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chun- yuan Li, Yikang Shen, Chuang Gan, Liangyan Gui, Yu-Xiong Wang, Yiming Yang, Kurt Keutzer, and Trevor Darrell. Align- ing Large Multimodal Models with Factually Augmented RLHF. In Annual Meeting of the Association for Co...

  50. [58]

    Vicuna: An Open-Source Chatbot Impress- ing GPT-4 with 90%* ChatGPT Quality

    The Vicuna Team. Vicuna: An Open-Source Chatbot Impress- ing GPT-4 with 90%* ChatGPT Quality. https://lmsys. org/blog/2023-03-30-vicuna/ . 16

    2023

  51. [59]

    Qwen2-VL: Enhancing Vision-Language Model’s Perception of the World at Any Resolution.CoRR abs/2409.12191, 2024

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-VL: Enhancing Vision-Language Model’s Pe...

    2024 arXiv

  52. [60]

    Cross-Modality Safety Alignment

    Siyin Wang, Xingsong Ye, Qinyuan Cheng, Junwen Duan, Shimin Li, Jinlan Fu, Xipeng Qiu, and Xuanjing Huang. Cross-Modality Safety Alignment. CoRR abs/2406.15279 ,

    arXiv

  53. [61]

    CogVLM: Visual Expert for Pretrained Language Models

    Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Xixuan Song, Jiazheng Xu, Bin Xu, Juanzi Li, Yuxiao Dong, Ming Ding, and Jie Tang. CogVLM: Visual Expert for Pretrained Language Models. CoRR abs/2311.03079, 2023. 3, 16

    2023 arXiv

  54. [62]

    RL-VLM-F: Rein- forcement Learning from Vision Language Foundation Model Feedback

    Yufei Wang, Zhanyi Sun, Jesse Zhang, Zhou Xian, Erdem Biyik, David Held, and Zackory Erickson. RL-VLM-F: Rein- forcement Learning from Vision Language Foundation Model Feedback. In International Conference on Machine Learning (ICML), 2024. 12

    2024

  55. [63]

    The Perfect Blend: Redefining RLHF with Mixture of Judges

    Tengyu Xu, Eryk Helenowski, Karthik Abinav Sankararaman, Di Jin, Kaiyan Peng, Eric Han, Shaoliang Nie, Chen Zhu, Hejia Zhang, Wenxuan Zhou, Zhouhao Zeng, Yun He, Kar- ishma Mandyam, Arya Talabzadeh, Madian Khabsa, Gabriel Cohen, Yuandong Tian, Hao Ma, Sinong Wang, and Han Fang...

    2024 arXiv

  56. [64]

    Qwen2 Technical Re- port

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng...

    2024 arXiv

  57. [65]

    Bridge the Modal- ity and Capacity Gaps in Vision-Language Model Selection

    Chao Yi, De-Chuan Zhan, and Han-Jia Ye. Bridge the Modal- ity and Capacity Gaps in Vision-Language Model Selection. CoRR abs/2403.13797, 2024. 1

    2024 arXiv

  58. [66]

    RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-Grained Correctional Human Feedback

    Tianyu Yu, Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu, Hai-Tao Zheng, and Maosong Sun. RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-Grained Correctional Human Feedback. In IEEE Conference on Computer Vision and Pat- ter...

    2024

  59. [67]

    SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vi- sion Language Model

    Yongting Zhang, Lu Chen, Guodong Zheng, Yifeng Gao, Rui Zheng, Jinlan Fu, Zhenfei Yin, Senjie Jin, Yu Qiao, Xuan- jing Huang, Feng Zhao, Tao Gui, and Jing Shao. SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vi- sion Language Model. CoRR abs/2406.12030, 2024. 12

    2024 arXiv

  60. [68]

    Automated Generation of Challenging Multiple-Choice Questions for Vision Language Model Evaluation

    Yuhui Zhang, Yuchang Su, Yiming Liu, Xiaohan Wang, James Burgess, Elaine Sui, Chenyu Wang, Josiah Aklilu, Alejandro Lozano, Anjiang Wei, Ludwig Schmidt, and Serena Yeung- Levy. Automated Generation of Challenging Multiple-Choice Questions for Vision Language Model Evaluation. ...

    2025 arXiv

  61. [69]

    On Evaluating Ad- versarial Robustness of Large Vision-Language Models

    Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongx- uan Li, Ngai-Man Cheung, and Min Lin. On Evaluating Ad- versarial Robustness of Large Vision-Language Models. In Annual Conference on Neural Information Processing Sys- tems (NeurIPS). NeurIPS, 2023. 12

    2023

  62. [70]

    Secrets of RLHF in Large Language Models Part I: PPO

    Rui Zheng, Shihan Dou, Songyang Gao, Yuan Hua, Wei Shen, Binghai Wang, Yan Liu, Senjie Jin, Qin Liu, Yuhao Zhou, Limao Xiong, Lu Chen, Zhiheng Xi, Nuo Xu, Wen- bin Lai, Minghao Zhu, Cheng Chang, Zhangyue Yin, Rongx- iang Weng, Wensen Cheng, Haoran Huang, Tianxiang Sun, Hang Ya...

    2023 arXiv

  63. [71]

    Yes” or “No

    Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. Texygen: A Benchmarking Platform for Text Generation Models. In International ACM SIGIR Conference on Research and Development in Informa- tion Retrieval (SIGIR), pages 1097–1100. ACM, 2018. 10 ...

    2018

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.