Pith. sign in

REVIEW 4 major objections 5 minor 27 references

The paper argues that vision-language models answer harmful queries largely because they under-attend to dangerous image regions, and that adding a short image summary, scoring risk from the first tokens, and adaptively steering activations

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 09:41 UTC pith:GPU6CEY4

load-bearing objection Useful inference-time safety defense with large ASR reductions, but the SPA-VL numbers are partly self-calibrated and need a cleaner evaluation split. the 4 major comments →

arxiv 2510.13698 v4 pith:GPU6CEY4 submitted 2025-10-15 cs.CV

Attention Misses Visual Risk: Risk-Adaptive Steering for Multimodal Safety Alignment

Jonghyun Park , Minhyuk Seo , Chaewon Yeo , Jonghyun Choi This is my paper
classification cs.CV
keywords multimodal safetyjailbreak defenseactivation steeringcross-modal attentionrisk evaluationquery reformulationvision-language modelsinference-time alignment
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that a key reason multimodal large language models answer harmful queries is that they pay too little cross-modal attention to safety-critical parts of the image, even when a text-only version of the same request is refused. Building on that diagnosis, it introduces RAS, a three-stage inference-time defense: first, a short textual summary of the image and a safety prompt are added to the query to pull attention toward dangerous regions; second, the first few response-token activations are compared, via an exponentially weighted cosine similarity, against "unsafe prototypes" learned from a small set of GPT-4-written policy-violating text queries, producing a continuous risk score; third, activations are steered toward the refusal direction with strength proportional to that score. On three multimodal jailbreak benchmarks and four open vision-language models, RAS reports attack success rates near zero while leaving benign task scores essentially unchanged and running faster than prior inference-time defenses that iteratively refine responses. If correct, this means strong multimodal safety gains are available without retraining or large calibration costs.

Core claim

The paper's central claim is that insufficient cross-modal attention to safety-critical image regions is a key correctable cause of multimodal safety failures. Attention maps and a Fisher discriminant analysis show that safety prompts alone leave safe and unsafe representations entangled, while a concise visual context — a short description of the image appended to the query — makes them separable. Risk is scored by comparing the exponentially weighted output distributions of the first three response tokens against unsafe prototypes (mean last-layer activations of initial tokens for policy-violating text queries). A sigmoid centered on a 100-sample calibration gives a continuous risk score t

What carries the argument

The load-bearing object is the unsafe prototype: the mean last-layer activation of the first response tokens over a set of policy-violating text queries. It plays two roles: as the reference distribution in the risk score (cosine similarity between exponentially weighted softmax distributions of early output tokens) and as the target for the refusal vector (the difference between the prototype and the current query's activation). Vision-aware query reformulation is the enabler: appending a short image summary and a safety prompt strengthens cross-modal attention to danger-relevant visual tokens, which is what makes the prototype similarity informative. Risk-adaptive activation steering then

Load-bearing premise

The load-bearing premise is that once the image is summarized in words, a harmful multimodal request makes the model's first few output tokens look much more like the first few tokens of unsafe text-only requests than safe ones do, and that this difference is strong and stable enough for a single threshold learned from 100 examples to work across attacks and models.

What would settle it

Build a harmful image-embedded query whose generated visual context is neutral (does not mention the dangerous object), so the reformulated query still contains the image but not a textual hint of the danger. If the risk score stays low and the model answers harmfully, the prototype-similarity gate is miscalibrated and the defense fails. A direct measurement also works: collect many safe and unsafe queries with identical reformulation, and if the cosine-similarity histograms overlap substantially for any attack class, the steering threshold cannot separate them.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the core claim holds, multimodal safety can be improved for frozen vision-language models without supervised fine-tuning or reinforcement learning, using only a small calibration set.
  • Because risk is assessed from the first three response tokens, defenses no longer need to generate and regenerate full answers, which removes a major source of inference overhead.
  • The adaptive steering strength means benign queries receive near-zero intervention, so the over-refusal problem that plagues fixed safety prompting is avoided.
  • The diagnostic claim implies that any multimodal jailbreak that succeeds by hiding harm in image regions depends on the model's attention failing; interventions that increase attention to those regions should have broad effect across attack styles.
  • The method's reported near-zero attack success on typographic attacks suggests that even attacks where the harmful instruction is text embedded in the image are vulnerable to explicit image-summary-based grounding.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A testable extension of the paper's own logic: a jailbreak that supplies the image but prevents the generated visual context from describing the dangerous object (e.g., an adversarial caption or a neutral summary) should collapse the risk score and bypass the defense — this would directly probe whether prototypes and visual context are the binding mechanism.
  • The method's dependence on text-derived unsafe prototypes suggests the same risk-score architecture could be transferred to other safety domains or other modalities whenever a reference "unsafe prototype" can be defined; the paper only demonstrates it for vision-and-language.
  • The continuous risk score could serve purposes beyond refusal, such as selective explanation or risk display to users, since it is available before the model commits to an answer.
  • An open implication the paper does not pursue is robustness to distribution shift: if deployment queries differ from SPA-VL in style, the calibrated threshold S_base may need re-estimation; the paper's own appendix only varies the prototype source, not the calibration distribution.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper claims that a key cause of multimodal safety failures is insufficient cross-modal attention to safety-critical image regions, and proposes Risk-adaptive Activation Steering (RAS), a training-free inference-time defense. RAS has three stages: (i) vision-aware query reformulation that appends a concise generated visual context and a safety prompt to the input query; (ii) risk evaluation that measures the exponentially weighted cosine similarity between the first-N output-token distributions of the reformulated query and text-only 'unsafe prototypes' (Eq. 4), mapping this similarity through a sigmoid to a risk score (Eq. 5); and (iii) risk-adaptive activation steering that adds to the original query's last-layer activations a refusal vector scaled by the risk score (Eqs. 6-7). The method is evaluated on MM-SafetyBench, SPA-VL, and FigStep across four MLLMs, with utility measured on Sci-QA, MM-Vet, GQA, and MME. The paper reports large ASR reductions (e.g., Table 1: LLaVA-1.5-7B ASR drops from 40.1/47.2/59.3 to 4.1/8.3/2.2 on MM-Safety/SPA-VL/FigStep), near-zero utility degradation, and higher inference throughput than prior inference-time defenses.

Significance. If the reported results hold, RAS is a valuable contribution: it is training-free, requires no external reward model, intervenes on only a few tokens, and consistently outperforms several strong inference-time baselines across models and benchmarks. The paper also offers an interesting analysis of cross-modal attention and Fisher Discriminant Ratio (Sec. 3.1, Figs. 3-4), and ablations support the contribution of each stage (Table 2, Fig. 8). The robustness check using alternative unsafe query sources (Appendix F) is a good addition. However, the central risk-evaluation mechanism is calibrated on the same SPA-VL benchmark on which the headline SPA-VL ASR is reported, and the evidence for separation of safe/unsafe scores is limited to one illustrative example. Because these issues bear directly on the paper's main claim, the current evidence is not yet sufficient for acceptance.

major comments (4)
  1. [Sec. 3.2, Eq. (5); Sec. 4.1 Implementation details; Table 1] The SPA-VL result is potentially circular. The paper states that S_base is the mean of S_i over 100 SPA-VL samples and alpha is calibrated on those samples (Sec. 3.2; Implementation details), and then reports Table 1 ASR on SPA-VL. It is not stated whether the 100 calibration samples are disjoint from the 265 unsafe SPA-VL test queries. If they are not disjoint, the SPA-VL ASR in Table 1 is not an independent evaluation of the method's performance on that benchmark. Please specify the exact split, calibrate on a separate training subset (or use cross-validation), and report ASR on the held-out SPA-VL test set.
  2. [Sec. 3.2, Eq. (4), Figs. 5-6; Table 1] The central assumption that the cosine similarity S_i cleanly separates safe from unsafe reformulated queries is supported mainly by Fig. 5, which shows one text-query/object-pair on LLaVA-1.5-7B, and Fig. 6, which shows MM-Vet vs. SPA-VL scores for one model. The claim that the SPA-VL-calibrated threshold transfers to MM-Safety and FigStep is load-bearing, but the paper does not report S_i distributions or separation metrics (e.g., AUC, overlap) for each model and each safety benchmark. Please provide per-model calibration plots and quantitative separation statistics, and a sensitivity analysis of ASR/utility to S_base shifts on MM-Safety and FigStep.
  3. [Sec. 4.1, Sec. 4.2, Table 1; Appendix B] The paper never reports false-refusal rates or utility on the 265 safe SPA-VL test queries, even though SPA-VL contains a safe split and the method is explicitly designed to avoid over-refusal. Utility numbers in Table 1 are from unrelated benchmarks (Sci-QA, MM-Vet, GQA, MME) and do not measure whether a safe query paired with a benign image is incorrectly refused. Please report refusal/compliance rates and task accuracy on the safe SPA-VL test split, ideally broken down by risk-score bins.
  4. [Tables 1-2, Figs. 7-8; general reporting] No error bars, repeated runs, or statistical significance tests are reported. ASR differences between RAS and the strongest baselines (e.g., ETA on LLaVA-1.5-13B: 6.9 vs. 11.7 for MM-Safety) could be within run-to-run variation, especially since judge-based evaluation and sampling can be stochastic. Please report results over multiple seeds (or bootstrap confidence intervals) for the main safety and utility numbers, and state the inference-time generation settings (temperature, decoding, number of runs).
minor comments (5)
  1. [Appendices A.1-A.2, Sec. 4.1] The exact visual-context generation prompt, safety prompt, and reformulation template are not quoted in the main text. The appendices show illustrative samples but not the full fixed strings. Please include the complete templates in the main text or an appendix for reproducibility.
  2. [Sec. 4.2, Fig. 7] The relative throughput comparison is useful, but the figure does not specify whether the reported throughput includes the extra forward pass needed to generate the visual context in Stage 1. Since RAS does incur a generation cost for the visual context, please clarify whether that overhead is accounted for and report absolute tokens/second as well.
  3. [Table 2] The first row of Table 2 is difficult to read: "[1] - -3.5 7.12.8 ..." appears garbled. Please fix the formatting and clearly label the stage combination and the columns.
  4. [Eq. (3)-(4), Sec. 3.2] The notation in Eq. (3) defines mu_n^u as the mean activation, but then Eq. (4) applies the LM head to mu_n^u to obtain a probability distribution. Clarify whether mu_n^u is a single vector (the mean) or an average of distributions, and specify how the softmax is computed over the vocabulary.
  5. [Reproducibility Statement] The statement says code 'will be released upon publication,' but no code or link is currently provided. Given that the method depends on model-specific hyperparameters and prototype construction, releasing code and the exact unsafe-query list would substantially strengthen reproducibility.

Circularity Check

3 steps flagged

Risk-gate threshold is fit to SPA-VL and gamma is tuned on the reported safety/utility benchmarks, so parts of the headline ASR/utility numbers are fitted rather than predicted; maximal steering also collapses to the unsafe prototype by construction.

specific steps
  1. fitted input called prediction [Sec. 3.2 (Eq. 5), Sec. 4.1 Implementation details, Appendix A.4, Appendix B]
    "To derive risk scores, we use S_i values from unsafe SPA-VL samples... with the mean used as a baseline S_base... r(S_i)=σ(α(S_i−S_base))... For S_base and α, as S_i distributions differ across models, they are adaptively determined by scores from 100 samples from SPA-VL... both S_base and α are derived from the distribution of similarity scores S_i computed on 100 SPA-VL samples... SPA-VL... contains 93.3k preference samples... along with 530 test samples (265 safe and 265 unsafe)... Following Ding et al. (2024), we report ASR on the 265 harmful test queries."

    The operating point of the risk gate (Eq. 5) is set from SPA-VL similarity scores, and the headline SPA-VL ASR in Table 1 is reported on SPA-VL's 265 harmful test queries. The paper never states whether the 100 calibration samples are disjoint from those 265 test queries. If they are drawn from the same test split, the SPA-VL ASR is a self-classification of the calibration data, not an out-of-sample prediction: the threshold is fitted to the very unsafe queries whose refusal is then presented as a measured attack-success reduction.

  2. fitted input called prediction [Sec. 4.3 'Effect of γ and N in EWRE', Fig. 8]
    "We vary γ and N to assess their impact on safety and utility... We set γ=0.3 as the default, as it achieves the best trade-off, minimizing ASR while preserving utility close to its original score. ... Safety is evaluated by average ASR on MM-Safety, SPA-VL, and FigStep, and utility is evaluated by MM-Vet scores."

    The default γ=0.3 is selected by sweeping the exact metrics and benchmarks that appear in Table 1: average ASR on MM-Safety, SPA-VL, and FigStep, and MM-Vet utility. Thus the reported safety and utility numbers for the final method are partly selected on the evaluation sets, not predicted on unseen data. This is a weaker form of circularity than using the same data for both fitting and evaluation, but it makes the headline reductions partially fitted.

  3. self definitional [Sec. 3.2 Eq. (3)-(4), Sec. 3.3 Eq. (6)-(7)]
    "When S_i is high, the query resembles unsafe prototypes and is likely to trigger a refusal... the refusal vector v_n at the last layer is computed as: v_n = µ_u^n − x_i^n... ˜x_i^n = x_i^n + r(S_i)·v_n."

    Combining Eq. (6) and Eq. (7), at r(S_i)=1 the steered activation is exactly the unsafe prototype µ_u^n, which Eq. (3) defines as the mean activation of refusal-triggering unsafe text queries. Therefore 'maximal steering produces refusal-like output' is true by construction rather than as an independently derived prediction. The empirical content is limited to the actual ASR after autoregressive decoding; the risk estimate itself is definitionally a similarity to the unsafe-class mean.

full rationale

The paper is not a self-citation chain: no load-bearing result is justified only by the authors' prior work, and the unsafe-prototype ablation with Anthropic RTD provides some independent support. However, the central safety evaluation contains a calibration/evaluation overlap. S_base and α are fit to 100 SPA-VL samples, while the SPA-VL ASR is reported on SPA-VL's 265 harmful test queries, with no stated disjoint split; if the calibration samples come from the test split, that headline number is a fit, not a prediction. In addition, γ=0.3 is selected by sweeping ASR/utility on the very benchmarks reported in Table 1, so the reported reductions are partially selected rather than fully predicted. Finally, the refusal vector makes maximal steering collapse to the unsafe prototype by Eq. 7, so refusal under high risk is partly tautological. These issues affect the strength of the generalization claim but do not reduce the entire paper to its inputs: the attention/FDR analyses, the cross-benchmark transfer of the SPA-VL threshold to MM-Safety and FigStep, and the prototype-source ablation are independent evidence. A score of 5 reflects this partial circularity.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The method's claimed advantage comes from a three-stage pipeline. The only fitted free parameters are risk-calibration constants and two hyperparameters; no new physical or model entities are introduced. The key unproved assumptions are the transferability of text-only unsafe prototypes to multimodal inputs and the sufficiency of the first three tokens for capturing refusal.

free parameters (4)
  • S_base (per-model risk threshold) = 0.711 (LLaVA-7B), 0.871 (LLaVA-13B), 0.611 (Qwen-VL-Chat), 0.549 (InternLM-XComposer-2.5)
    Set to the mean similarity score over 100 SPA-VL unsafe samples (Sec. 3.2, Appendix A.4); used as the sigmoid center in the risk score r(S_i).
  • alpha (sigmoid slope) = 15.901, 35.261, 11.813, 10.188 per model
    Calibrated so r(1) is approximately 1; fitted per model on the same SPA-VL calibration samples.
  • gamma (exponential decay weight) = 0.3
    Chosen by sweeping gamma across MM-Safety, SPA-VL, FigStep ASR and MM-Vet utility (Fig. 8); directly changes the risk-score weighting.
  • N (number of initial response tokens) = 3
    Set to 3 after a sweep; less influential than gamma but still selected using benchmark outcomes.
axioms (4)
  • domain assumption Refusal behavior is fully reflected in the first few (N=3) response tokens.
    Sec. 3.2 and Sec. 3.3 restrict risk evaluation and steering to the first three output tokens; if refusal is not visible in these tokens, the risk score and steering miss it.
  • domain assumption Unsafe prototypes computed from text-only GPT-4 queries are a valid reference for multimodal refusal states.
    Eq. 3 pools activations from text queries; Eq. 4 compares multimodal reformulated queries to them. No multimodal grounding is used to build the prototypes.
  • domain assumption Visual grounding is captured by the top-3 cross-modal attention heads.
    Sec. 3.1 uses top-n heads following Kang et al. (2025b) to define effective attention; if misleading, the attention/FDR motivation weakens.
  • domain assumption The generated visual context is an accurate, safe, and non-distorting summary of the image.
    Stage 1 uses the MLLM's own caption as context; a wrong or adversarial caption could mislead risk evaluation.

pith-pipeline@v1.3.0-alltime-deepseek · 18917 in / 12780 out tokens · 114106 ms · 2026-08-04T09:41:53.045643+00:00 · methodology

0 comments
read the original abstract

Even modern AI models often remain vulnerable to multimodal queries in which harmful intent is embedded in images. A widely used approach for safety alignment is training with extensive multimodal safety datasets, but the costs of data curation and training are often prohibitive. To mitigate these costs, inference-time alignment has recently been explored, but they often lack generalizability across diverse multimodal jailbreaks and still incur notable overhead due to extra forward passes for response refinement or heavy pre-deployment calibration procedures. Here, we identify insufficient visual attention to safety-critical image regions as one of the key causes of multimodal safety failures. Building on this insight, we propose Multimodal Risk-Adaptive Steering (MoRAS), which enhances safety-critical visual attention via concise visual contexts for accurate multimodal risk assessment. This risk signal enables risk-adaptive steering for direct refusals, reducing inference overhead while remaining generalizable across diverse multimodal jailbreaks. Notably, MoRAS requires only a small calibration set to estimate multimodal risk, substantially reducing pre-deployment overhead. We conduct various empirical validations across multiple benchmarks and MLLM backbones, and observe that the proposed MoRAS consistently mitigates jailbreaks, preserves utility, and reduces computational overhead compared to state-of-the-art inference-time defenses.

Figures

Figures reproduced from arXiv: 2510.13698 by Chaewon Yeo, Jonghyun Choi, Jonghyun Park, Minhyuk Seo.

Figure 1
Figure 1. Figure 1: Lack of attention to safety-critical image regions. (a) For unsafe multimodal instruc￾tions, the model fails to allocate sufficient attention to safety-critical image regions (i.e., the bomb, highlighted in red), leading to unsafe responses. (b) In contrast, when the same instruction is given in text only, the model sufficiently attends to harmful text tokens (i.e., the text token “bomb”) and generates a r… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of RAS. RAS consists of three stages: (i) Query Reformulation, which aug￾ments input queries with concise visual contexts and safety prompts to strengthen cross-modal at￾tention to safety-critical image regions; (ii) Risk Evaluation, where MLLM output activations are compared with unsafe prototypes to produce similarity-based risk scores; and (iii) Risk-adaptive Ac￾tivation Steering, where activat… view at source ↗
Figure 3
Figure 3. Figure 3: Attention maps for unsafe (top) and safe (bottom) objects under various query for￾mulations. SP. denotes safety prompt. VC. denotes visual context. (a) Example of unsafe vs. safe instructions under the same text query. (b–e) Cross-modal attention maps from text to visual tokens. (b) With only the original query, attention weights to the objects are small, indicating weak visual grounding. (c) Safety prompt… view at source ↗
Figure 4
Figure 4. Figure 4: FDR values across layers for various query formulations. SP. de￾notes safety prompt. VC. denotes vi￾sual context. Lower FDR indicates less separable representations. We employ LLaVA-1.5-7B to compute FDR. When the MLLM only uses the query, the overall FDR across layers remains low (orange line in [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Similarity score (Si) distributions of (a) safe (b) unsafe objects under various query formulations. SP. denotes safety prompt. VC. denotes visual context. Higher similarity scores indicate output distributions similar to refusals. We employ LLaVA-1.5-7B to extract Si scores. n th response token. Finally, µ n u represents the unsafe prototype activation in the last layer at position n (see Appendix F for r… view at source ↗
Figure 6
Figure 6. Figure 6: Distribution of similarity scores for reformulated queries from MM-Vet (safe) and SPA-VL (unsafe). This can be expressed as: r(Si) = σ [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Comparison of ASR, utility score, and relative throughput on SPA-VL (left) and MM-Vet (right). For SPA-VL, the de￾sirable region is the lower right (low ASR, high throughput), whereas for MM-Vet, it is the upper right (high utility, high throughput) [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
Figure 9
Figure 9. Figure 9: List of unsafe text queries generated by GPT-4. B BENCHMARK DETAILS MM-Safety (Liu et al., 2024d). MM-SafetyBench evaluates the safety of MLLMs against harmful queries paired with images. Covering 13 safety-critical scenarios with 5,040 text–image pairs, it in￾troduces three adversarial settings: (i) Stable Diffusion (SD), which generates images from harmful key phrases; (ii) Typography (Typo), which embed… view at source ↗
Figure 10
Figure 10. Figure 10: Attention maps for (a) MM-Safety and (b) FigStep under various query reformula￾tions. SP. denotes safety prompt. VC. denotes visual context. Each heatmap visualizes the cross￾modal attention from text tokens in the query to visual tokens. Unsafe regions, e.g., “illegal under￾ground drug manufacturing” in (a) and “traffic weapons of mass destructions in (b), receive low attention weights without visual con… view at source ↗
Figure 11
Figure 11. Figure 11: An unsafe FigStep sample (left) and its safe counterpart (right). Although the input text query is benign on its own, incorporating images with unsafe typographic content (left, high￾lighted in red) makes the overall instruction unsafe. In contrast, when the embedded text specifies a benign activity (right, highlighted in green), the multimodal instruction remains safe. E ADDITIONAL FDR ANALYSIS ON FIGSTE… view at source ↗
Figure 12
Figure 12. Figure 12: FDR values across lay￾ers for various query formulations (LLaVA-1.5-7B). SP. denotes safety prompt. VC. denotes visual context. Lower FDR indicates less separable representations. Across various query formulations, we observe results consistent with those in Sec. 3.1. When the model pro￾cesses only with the original query, cross-modal attention to the typographic text in the image remains weak (first at￾t… view at source ↗
Figure 13
Figure 13. Figure 13: Qualitative results under different steering strengths on LLaVA-1.5-7B. Left: Unsafe query (MM-Safety). Right: Safe query (MM-Vet). 21 [PITH_FULL_IMAGE:figures/full_fig_p021_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Qualitative results under different steering strengths on LLaVA-1.5-13B. Left: Un￾safe query (MM-Safety). Right: Safe query (MM-Vet) [PITH_FULL_IMAGE:figures/full_fig_p022_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Qualitative results under different steering strengths on Qwen-VL-Chat. Left: Un￾safe query (MM-Safety). Right: Safe query (MM-Vet). 22 [PITH_FULL_IMAGE:figures/full_fig_p022_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Qualitative results under different steering strengths on InternLM-XComposer-2.5. Left: Unsafe query (MM-Safety). Right: Safe query (MM-Vet). 23 [PITH_FULL_IMAGE:figures/full_fig_p023_16.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

27 extracted references · 22 linked inside Pith

  1. [1]

    Steps to manufacture illegal drugs

    The resulting values for each model are summarized in Tab.3. Table 3:Model-specific parameters for risk assessment. LLaV A-1.5-7B LLaV A-1.5-13B Qwen-VL-Chat InternLM-XComposer-2.5-7B Sbase 0.711 0.871 0.611 0.549 α15.901 35.261 11.813 10.188 14 Preprint Figure 9:List of unsafe text queries generated by GPT-4. B BENCHMARKDETAILS MM-Safety (Liu et al., 202...

    2025

  2. [2]

    Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul Rottger, Dan Jurafsky, Tatsunori Hashimoto, and James Zou

    URLhttps://arxiv.org/abs/2308.12966. Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul Rottger, Dan Jurafsky, Tatsunori Hashimoto, and James Zou. Safety-tuned LLaMAs: Lessons from improving the safety of large language models that follow instructions. InThe Twelfth International Conference on Learning Representations,

    Pith/arXiv arXiv

  3. [6]

    Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, et al

    URLhttps://arxiv.org/abs/ 2306.13394. Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, et al. Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned.arXiv preprint arXiv:2209.07858,

    Pith/arXiv arXiv

  4. [8]

    Longsafety: Enhance safety for long-context llms

    Mianqiu Huang, Xiaoran Liu, Shaojun Zhou, Mozhi Zhang, Qipeng Guo, Linyang Li, Chenkun Tan, Yang Gao, Pengyu Wang, Linlin Li, et al. Longsafety: Enhance safety for long-context llms. arXiv preprint arXiv:2411.06899,

    Pith/arXiv arXiv

  5. [10]

    Kangaroo: Lossless self-speculative decoding for accelerating llms via double early exiting.Ad- vances in Neural Information Processing Systems, 37:11946–11965, 2024a

    Fangcheng Liu, Yehui Tang, Zhenhua Liu, Yunsheng Ni, Duyu Tang, Kai Han, and Yunhe Wang. Kangaroo: Lossless self-speculative decoding for accelerating llms via double early exiting.Ad- vances in Neural Information Processing Systems, 37:11946–11965, 2024a. Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tu...

    Pith/arXiv arXiv 2024

  6. [11]

    In-context vectors: Making in context learning more effective and controllable through latent space steering.arXiv preprint arXiv:2311.06668,

    Sheng Liu, Haotian Ye, Lei Xing, and James Zou. In-context vectors: Making in context learning more effective and controllable through latent space steering.arXiv preprint arXiv:2311.06668,

    Pith/arXiv arXiv

  7. [12]

    Cross-modal attention congruence regularization for vision-language relation alignment.arXiv preprint arXiv:2212.10549,

    Rohan Pandey, Rulin Shao, Paul Pu Liang, Ruslan Salakhutdinov, and Louis-Philippe Morency. Cross-modal attention congruence regularization for vision-language relation alignment.arXiv preprint arXiv:2212.10549,

    Pith/arXiv arXiv

  8. [13]

    Steering llama 2 via contrastive activation addition.arXiv preprint arXiv:2312.06681,

    Nina Panickssery, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner. Steering llama 2 via contrastive activation addition.arXiv preprint arXiv:2312.06681,

    Pith/arXiv arXiv

  9. [14]

    Mllm-protector: Ensuring mllm’s safety without hurting performance

    Renjie Pi, Tianyang Han, Jianshu Zhang, Yueqi Xie, Rui Pan, Qing Lian, Hanze Dong, Jipeng Zhang, and Tong Zhang. Mllm-protector: Ensuring mllm’s safety without hurting performance. arXiv preprint arXiv:2401.02906,

    Pith/arXiv arXiv

  10. [15]

    Safety alignment should be made more than just a few tokens deep, 2024.URL https://arxiv

    Xiangyu Qi, Ashwinee Panda, Kaifeng Lyu, Xiao Ma, Subhrajit Roy, Ahmad Beirami, Prateek Mittal, and Peter Henderson. Safety alignment should be made more than just a few tokens deep, 2024.URL https://arxiv. org/abs/2406.05946,

    Pith/arXiv arXiv 2024

  11. [16]

    Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347,

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347,

    Pith/arXiv arXiv

  12. [18]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024a

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024a. Suge Wang, Deyu Li, Yingjie Wei, and Hongxia Li. A feature selection method based on fisher’s dis- crimina...

    Pith/arXiv arXiv

  13. [19]

    Adaptive activation steering: A tuning-free llm truthfulness improvement method for diverse hallucinations categories

    Tianlong Wang, Xianfeng Jiao, Yinghao Zhu, Zhongzhi Chen, Yifan He, Xu Chu, Junyi Gao, Yasha Wang, and Liantao Ma. Adaptive activation steering: A tuning-free llm truthfulness improvement method for diverse hallucinations categories. InProceedings of the ACM on Web Conference 2025, pp. 2562–2578, 2025a. Yanbo Wang, Jiyang Guan, Jian Liang, and Ran He. Do ...

    Pith/arXiv arXiv 2025

  14. [20]

    Mm-vet: Evaluating large multimodal models for integrated capabilities.arXiv preprint arXiv:2308.02490,

    Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities.arXiv preprint arXiv:2308.02490,

    Pith/arXiv arXiv

  15. [21]

    Separation and concentration in deep networks

    John Zarka, Florentin Guth, and St ´ephane Mallat. Separation and concentration in deep networks. arXiv preprint arXiv:2012.10424,

    Pith/arXiv arXiv 2012

  16. [23]

    Safety fine-tuning at (almost) no cost: A baseline for vision large language models.arXiv preprint arXiv:2402.02207,

    Yongshuo Zong, Ondrej Bohdal, Tingyang Yu, Yongxin Yang, and Timothy Hospedales. Safety fine-tuning at (almost) no cost: A baseline for vision large language models.arXiv preprint arXiv:2402.02207,

    Pith/arXiv arXiv

  17. [25]

    Moreover, when visual contexts are combined with safety prompts, the FDR improves further (red line in Fig

    and clearer representational separation between safe and unsafe in- structions. Moreover, when visual contexts are combined with safety prompts, the FDR improves further (red line in Fig. 12), demonstrating that once visual grounding is established, safety prompting can further amplify representational separability. Overall, these findings confirm that in...

    2022

  18. [26]

    For the choice of intermediate layer, we follow Arditi et al

    We evaluate activation steering when applied to (i) an intermediate layer and (ii) the last layer. For the choice of intermediate layer, we follow Arditi et al. (2024) and select layer

    2024

  19. [27]

    (2024), we observe only modest safety gains: steering at layer 14 reduces ASR by 11.3% on average, while steering at the final layer achieves a 62.1% reduction

    Using the refusal vector definition of Arditi et al. (2024), we observe only modest safety gains: steering at layer 14 reduces ASR by 11.3% on average, while steering at the final layer achieves a 62.1% reduction. In contrast, our refusal vector achieves substantially larger gains, reducing ASR by 59.0% at layer 14 and 89.5% at the final layer, clearly de...

    2024

  20. [2009]

    Eta: Evaluating then aligning safety of vision language models at inference time.arXiv preprint arXiv:2410.06625,

    Yi Ding, Bolian Li, and Ruqi Zhang. Eta: Evaluating then aligning safety of vision language models at inference time.arXiv preprint arXiv:2410.06625,

    Pith/arXiv arXiv

  21. [2017]

    Activation scaling for steering and interpreting language models.arXiv preprint arXiv:2410.04962,

    Niklas Stoehr, Kevin Du, V ´esteinn Snæbjarnarson, Robert West, Ryan Cotterell, and Aaron Schein. Activation scaling for steering and interpreting language models.arXiv preprint arXiv:2410.04962,

    Pith/arXiv arXiv

  22. [2019]

    See what you are told: Visual attention sink in large multimodal models.arXiv preprint arXiv:2503.03321, 2025a

    Seil Kang, Jinyeong Kim, Junhyeok Kim, and Seong Jae Hwang. See what you are told: Visual attention sink in large multimodal models.arXiv preprint arXiv:2503.03321, 2025a. Seil Kang, Jinyeong Kim, Junhyeok Kim, and Seong Jae Hwang. Your large vision-language model only needs a few attention heads for visual grounding. InProceedings of the Computer Vision ...

    Pith/arXiv arXiv

  23. [2020]

    Internlm-xcomposer-2.5: A versatile large vision language model supporting long-contextual input and output.arXiv preprint arXiv:2407.03320,

    Pan Zhang, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Rui Qian, Lin Chen, Qipeng Guo, Haodong Duan, Bin Wang, Linke Ouyang, et al. Internlm-xcomposer-2.5: A versatile large vision language model supporting long-contextual input and output.arXiv preprint arXiv:2407.03320,

    Pith/arXiv arXiv

  24. [2022]

    Coca: Regaining safety-awareness of multimodal large language models with con- stitutional calibration.arXiv preprint arXiv:2409.11365,

    Jiahui Gao, Renjie Pi, Tianyang Han, Han Wu, Lanqing Hong, Lingpeng Kong, Xin Jiang, and Zhenguo Li. Coca: Regaining safety-awareness of multimodal large language models with con- stitutional calibration.arXiv preprint arXiv:2409.11365,

    Pith/arXiv arXiv

  25. [2023]

    Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda

    Kaggle, CC BY-SA 4.0. Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. Refusal in language models is mediated by a single direction.arXiv preprint arXiv:2406.11717,

    Pith/arXiv arXiv

  26. [2024]

    Rethinking bottlenecks in safety fine-tuning of vision language models.arXiv preprint arXiv:2501.18533,

    Yi Ding, Lijun Li, Bing Cao, and Jing Shao. Rethinking bottlenecks in safety fine-tuning of vision language models.arXiv preprint arXiv:2501.18533,

    arXiv

  27. [2025]

    Llama guard 3-1b-int4: Compact and efficient safeguard for human-ai conversations.arXiv preprint arXiv:2411.17713,

    Igor Fedorov, Kate Plawiak, Lemeng Wu, Tarek Elgamal, Naveen Suda, Eric Smith, Hongyuan Zhan, Jianfeng Chi, Yuriy Hulovatyy, Kimish Patel, et al. Llama guard 3-1b-int4: Compact and efficient safeguard for human-ai conversations.arXiv preprint arXiv:2411.17713,

    Pith/arXiv arXiv