Pith. sign in

REVIEW 3 major objections 5 minor 43 references

Whose Refusal Is It? The Unmeasured Contribution of Black-Box Multimodal Guardrails

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A defended pipeline's reported safety benefit is a sum of two refusal producers, and the guardrail's share can range from 0% to 99% depending on evaluation choices no results table reports.

desk verdict A guarded pipeline's reported safety benefit is a two-producer sum; this paper separates it cleanly and shows protocol choice alone moves the guardrail's share from 0% to 99%—the silent-default root cause needs artifact release, but the core result is solid. read the letter →

arxiv 2608.08641 v1 pith:PBCXBK5V submitted 2026-08-09 cs.CR

Haoyu Zhang , Xiao Luo , Haowen Xu , Yi Feng , Shibo Zheng , Mohammad Zandsalimy , Shanu Sushmita This is my paper · ORCID
classification cs.CR
keywords guardrailevaluationrefusalattributionmultimodalsafetyprotocolchannelcoverageencodedjailbreaksblack-boxdefensesattacksuccessrate
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is about a measurement the field is not making. A defended pipeline contains two components that can refuse—the guardrail and the target model's own alignment—and every reported attack-success number is a sum over both. The authors show the guardrail's share of that sum is not a property of the guardrail: with a text guard and two open-weight targets, the share runs from 0% when the harmful payload is rendered as pixels, through 41–45% when the guard reads the encoded prompt the attacker actually sent, to about 99% when the harness fills the guard's internal read with the unencoded request behind the attack. All three settings would be described identically in a results table. The split is recoverable at no extra cost because a guard block replaces the model's response, so the two refusal counts are disjoint.

What carries the argument

The key machinery is the refusal decomposition: for a gate defense, guard blocks and model refusals are disjoint by construction because a block substitutes a fixed canned string for the model's response, so the guard's share is exactly $B/(B+M)$, where $B$ is the exact-match guard-block count and $M$ is the model's own refusal count among passed prompts. That identity makes the split recoverable from any already-collected campaign at no extra cost. The rest of the argument rides on a second variable, the defense's internal read: the string an evaluation harness places in each of the defense's slots (the encoded prompt as sent versus the unencoded request behind it), and the position of that slot in the defense's computation—decision, caption, or answer regeneration. Comparing the deployable protocol (read equals the encoded prompt) with the granted protocol (read equals the unencoded request) produces the measured inflation, and the within-defense isolation shows that the read conditioning the generated answer, not the read making the harm verdict, carries the effect.

What would settle it

Instrument the reference implementation of a caption-mediated guardrail to log which string actually populates each internal read slot, then run the same attack corpus under the deployable protocol; if the guard's measured benefit remains large when the read provably contains only the attacker's encoded prompt, the central protocol-artifact claim is refuted. Alternatively, if the released code is found anywhere to distinguish the attacker-sent string from the benchmark behavior string, the silent-default narrative fails.

Watch

Extended reading notes

Core claim

The central discovery is that a guardrail's reported safety benefit is a sum over two refusal producers, and the guardrail's own share is decided by two variables no evaluation records: which channel carries the payload, and what text the evaluation harness places in the defense's internal read. Measured on one text guard across two open-weight targets with 100 prompts per cell, the share moves from 0% (payload rendered as pixels; guard blocks 0/100 and the model produces every refusal) through 41–45% (guard reads the encoded prompt as sent; guard blocks 28–30 against 34–44 model refusals) to about 99% (harness fills the read with the unencoded request; guard blocks 98, model falls to about 1), with no change in how a results table would describe the pipeline. Because a guard block replaces the model's response, guard blocks and model refusals are disjoint and separately countable at no extra collection cost. Granting the unencoded read inflates measured benefit by 24–47 percentage points for a gate, 12–36 for a caption-mediated re-check, and about zero for a majority-vote smoother; isolating the grant inside one defense shows the effect is carried by the stage that regenerates the answer, not the stage that decides harm. The inflated setting is not an evaluator's careless choice: the reference implementation builds every stage from a single prompt field that cannot distinguish what the attacker sent from what the benchmark records, so faithful porting supplies it silently, in the direction that flatters the defense; the authors' own previously published figures are among those revised.

Load-bearing premise

The claim that the flattering protocol is the silent default rather than an evaluator's choice rests on the authors' root-cause inspection of one released implementation, ECSO, showing that all three of its stages read one undifferentiated prompt field; the artifacts of that inspection are not released in this preprint, so an independent reader cannot yet audit that the two strings are never distinguished.

Editorial extensions

If this is right

  • Every attack-success number reported for a defended pipeline is unreadable unless it states which channel carried the payload and which protocol filled the defense's internal read.
  • A guardrail can be the minority producer of the safety credited to it: in the honest deployable setting measured here, the guard produced 41–45% of the refusals while the target model supplied the rest.
  • Text-only guards are blind, not inaccurate, in the image channel: across 300 inputs spanning harmful, hard-benign, and ordinary benign traffic, their image-channel decision never changes, so no amount of tuning recovers a signal that never arrives.
  • Evaluation protocols that hand the defense the unencoded request can manufacture tens of points of apparent safety: 24–47 percentage points for a gate and 12–36 for a caption-mediated re-check, with the ordering reproduced in an independent replicate.
  • A defense whose read only selects among already-generated responses is protocol-robust, while a defense whose read conditions the final generated answer is protocol-sensitive.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the authors leave implicit is that comparing guardrails across papers without a standardized audit of each defense's internal read will tend to rank evaluations rather than defenses; a shared harness that logs which string fills each slot would settle that comparison.
  • The finding that the answer-regeneration stage carries the grant inflation suggests a testable extension: hold the defense architecture fixed and vary only whether a stage rewrites the final response or merely selects among candidate responses, predicting that only the rewriting stage is protocol-sensitive.
  • The 0% end implies that a channel-routed panel—the best-calibrated text guard on the text channel and a multimodal guard on the image channel—could be evaluated end-to-end; the paper recommends it but does not build it, and the binding constraint on that deployment would be detector recall for encoded inputs.
  • Because the paper revises its own previously published granted-protocol figures, an editorial inference is that other published guardrail evaluations using encoded attacks may carry similar inflation; re-scoring stored responses under the deployable protocol would reveal how widespread the artifact is.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper measures how much of a defended pipeline's reported safety is actually produced by the guardrail rather than by the target model's own refusal behavior. On a text guard over two open-weight targets, the guard's measured share of refusals ranges from 0% (payload rendered as pixels, guard blocks 0/100), through 41–45% (guard reads the encoded prompt as sent), to about 99% (harness fills the guard's internal read with the unencoded request), while a results table would describe all three settings identically. The authors further show that granting the unencoded read inflates measured benefit by 24–47 pp for a guard gate, 12–36 pp for a caption-mediated re-check, and approximately 0 pp for a majority-vote smoother; that the inflation is carried by the answer-regenerating stage rather than the harm-verdict stage; and that the reference ECSO implementation reportedly builds all three stages from a single prompt field, so faithful porting silently supplies the granting protocol. The paper also reports channel-coverage experiments for gate and transform defenses, benign-traffic discrimination controls, a detector-gated deployment variant, adaptive-attack checks, and a correction of the authors' own previously published granted-protocol numbers.

Significance. If the results hold, this is a substantial methodological contribution to multimodal safety evaluation. The core decomposition is elegant and, for gate defenses, exact: guard blocks and model refusals are disjoint and separately countable, so the guardrail's share of credited safety can be recovered from any already-collected campaign. The empirical work is unusually careful: block counts are exact string matches rather than similarity heuristics; paired tests use exact McNemar with family-wise Holm–Bonferroni correction; the central protocol grid was re-collected end-to-end in a designed 32-cell replicate; the reported ordering reproduces across campaigns; multi-judge re-scoring preserves the direction and significance of every paired claim; and the authors explicitly falsify their own registered prediction that the harm-verdict stage would carry the grant effect. The paper also publicly corrects its own prior reported figures, which is a notable scientific practice.

major comments (3)
  1. [§4.6 and Limitations (Artifact release)] The root-cause claim that the granted protocol is "what faithful porting supplies, silently" is load-bearing for the conclusion that the flattering setting is the default of released code rather than an evaluator's avoidable choice. Currently this rests on the authors' inspection of one ECSO implementation, and the artifacts promised in the Limitations are not released with the preprint, so an independent reader cannot verify that TELL, CAP, and SAFE all read one undifferentiated prompt field, that the field is populated with the benchmark's behavior string, and that no other code path preserves the attacker-sent encoded string. The Appendix A slot-level specification shows the substitution the protocol makes, but it does not show the actual code bindings. I ask the authors to either release the audited implementation together with the audit script (or an independent port inspection) before publishing the silent-default assertion, or to scope the claim to "the implementation we inspected supplies it silently" and mark the generalization as a hypothesis. The measured protocol gap and the 0%/41–45%/99% share range do not depend on this point and would remain.
  2. [§4.8, bounds of stage attribution] The claim that the answer-regenerating stage (SAFE) carries the grant inflation is supported only in the two code_attack cells where the full grant effect is large; on formal_logic no stage moves significantly, and the paper itself states that no stage attribution is possible where the full effect is small. One cell (pixtral/code) shows SAFE recovering 113% of the full effect, which the authors report without clipping. This is honest, but the concluding summary in the abstract and conclusion states the mechanism as a general result: "the stage that regenerates the answer carries the effect." I recommend that the summary-level wording be qualified to "where the effect is large enough to attribute," matching the bounds stated in §4.8.
  3. [§4.9–§4.10, granted-protocol cross-model grid] Table 10 and Figure 1 contain granted-protocol ECSO values that are explicitly labeled as measurements of the artifact rather than as attainable defense efficacy, and §4.6 re-collects the deployable arm showing most of the effect does not survive. This is a deliberate and mostly well-handled frame. However, the 'amplification' language in §4.10 still invites an efficacy reading, and the section title "Amplification: ECSO with the decoy" does not itself carry the protocol caveat. Given that the paper's central lesson is that protocol choice manufactures benefit, the caveat should appear in the section heading or in the first sentence of every paragraph reporting these magnitudes, not only in the table captions and the initial caveat paragraph.
minor comments (5)
  1. [§4.8] The paper contains an unresolved cross-reference "§??" in the paragraph beginning "The cross-family pattern remains an ordering"; this should be fixed to the section that explains why cross-defense orderings cannot identify read position.
  2. [Figure 1 caption] The caption still uses the term "oracle protocol (unattainable)" while the main text and tables have replaced "oracle" with "granted"; the terminology should be made uniform.
  3. [§4.8 / protocol grid] The paper refers to a "registered prediction" and says the protocol grid's readout was fixed in advance, but no registration identifier or repository link is provided; if a registration exists, it should be cited so readers can verify the pre-commitment.
  4. [References] One reference entry contains a dated provenance annotation ("Venue confirmed 2026-08-08...") that belongs in a reviewer communication rather than in a published reference list; it should be removed or placed in a footnote.
  5. [§4.3 and Table 4] The claim that the text guards are "blind, not inaccurate" is well supported by the 0/300 constant on the image arm, but the wording "spanning the full harm range" should make explicit that the harmful arm used the placeholder text channel; otherwise a reader may infer the image-channel input varied in text content across the 300 inputs.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical audit whose headline numbers are measured, not fitted, and whose only self-citations are non-load-bearing inputs or contextual references.

full rationale

The paper's central claims are measured decompositions rather than derived conclusions: guard blocks are counted by exact string match against a canned refusal string, model refusals are judged from stored responses, and the guard's share is the ratio of two disjoint counts (Table 1). No parameter is fitted to a subset of the data and then reported as a prediction; the paper explicitly reports that its one registered prediction (that the harm-verdict stage would carry the grant inflation) was empirically overturned in Section 4.8. The protocol-contrast numbers (Tables 6-8) are direct comparisons of two well-defined conditions, and the ordering across defense families is reproduced in an independent replicate campaign. The only self-citations are the formal-logic encoder used as one of several attack encodings and a concurrent-work reference in the Discussion; neither is used to justify the paper's measured effects, and the central findings also replicate with externally developed encodings (code-completion, set-theory) and external defenses (ECSO, SAGE). The paper's correction of its own prior granted-protocol figures is evidence against circularity rather than for it. The root-cause claim about the reference implementation building all stages from a single prompt field is an implementation inspection, not a definitional reduction, and although the artifacts are not yet released, that is an auditability limitation, not a circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters are fitted: every number is a measured rate from a fixed cell, with n=100 per cell as a sample size rather than a fitted constant. The paper's claims rely instead on benchmark validity, judge validity, and the structural property of gate defenses that a block prevents a model call; those are listed as axioms. No new entities are introduced.

assumptions (3)
  • domain assumption Exact string match against the guard's canned block string identifies guard blocks, and blocked prompts never query the model, making guard blocks and model refusals disjoint.
    The entire decomposition of §4.1 depends on this implementation property; the paper verifies canned strings in stored responses but cannot rule out undisclosed internal fallback paths for every guard.
  • domain assumption HarmBench and JailbreakBench paired classifiers, with the chosen LLM judges, are valid operationalizations of harmful and refusal outcomes for the comparisons made.
    All ASR and refusal numbers inherit this benchmark validity; the paper shows paired significance survives four judges but absolute levels are judge-relative.
  • standard math Paired exact McNemar tests with Holm-Bonferroni correction within four pre-fixed families are the appropriate statistical apparatus for prompt-paired binary outcomes.
    Standard methodology; the paper reports uncorrected p-values alongside corrected status, so re-analysis with other corrections is possible.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Whose Refusal Is It? The Unmeasured Contribution of Black-Box Multimodal Guardrails." pith.science (2026). https://pith.science/paper/PBCXBK5V

@misc{pith2026260808641,
  author       = {Pith},
  title        = {Pith review of: Whose Refusal Is It? The Unmeasured Contribution of Black-Box Multimodal Guardrails},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PBCXBK5V}},
  note         = {Machine review of arXiv:2608.08641}
}
read the original abstract

A black-box guardrail is evaluated as though the safety number it earns were its own. It is not. A defended pipeline holds two components that can refuse (the guardrail, and the target model out of its own alignment), and every reported metric is a sum over both. We show that the guardrail's actual share of the safety credited to it runs from none of it to essentially all of it, decided by two variables no evaluation records: which channel carries the payload, and what text the harness places in the defense's internal read. The split is recoverable at no extra cost, because a guard block replaces the model's response and the two counts are therefore disjoint. On a text guard across two open-weight targets: with the payload rendered as pixels the guard blocks nothing and the model produces every refusal the system makes; reading the encoded prompt the attacker actually sent, the guard produces a minority of the refusals attributed to it; reading the unencoded request behind the attack, it blocks almost everything and the model falls silent. The blindness is not inaccuracy: the same guards block no benign image inputs either, so their image-channel decision is a constant. Granting the unencoded request inflates measured benefit substantially for a guard gate, less for a caption-mediated re-check, and not at all for a majority-vote smoother; the ordering reproduces in an independent replicate. Isolating the grant within one defense shows it does not improve detection: the harm-verdict stage contributes nothing, while the stage that regenerates the answer carries the effect. Nor is the inflated setting careless; the reference implementation builds every stage from a single prompt field that cannot distinguish what the attacker sent from what the benchmark records, so faithful porting supplies it silently. Previously published figures of our own are among those revised.

Figures

Figures reproduced from arXiv: 2608.08641 by the authors.

Figure 1
Figure 1. The same cells, both protocols. ECSO text →decoy ASR change (pp; more negative = larger ap￾parent safety benefit) for the seven (model, attack) cells re￾collected under both protocols. Grey: the granted protocol, in which the defense’s internal read is filled with the unen￾coded request — a quantity no deployed defender facing an encoded attack possesses. Blue: the deployable protocol, in which the defense reads onl… view at source ↗
Figure 2
Figure 2. Three input variants. text: encoded text on text channel, no image. ir_plain: placeholder text + typographic image of encoded text (content lives in the image only; this is the relocated variant, used for the gate arms of §4.2 — [PITH_FULL_IMAGE:figures/full_fig_p023_2.png] view at source ↗
Figure 3
Figure 3. Decoy images. (a) “mountain” is the text-pseudo￾image used throughout [PITH_FULL_IMAGE:figures/full_fig_p024_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 31 canonical work pages

  1. [1]

    Aerni, M.; Zhang, J.; and Tram \`e r, F. 2024. Evaluations of Machine Learning Privacy Defenses are Misleading. In Proceedings of the 2024 ACM SIGSAC Conference on Computer and Communications Security (CCS), 1271--1284

    work page 2024

  2. [2]

    Athalye, A.; Carlini, N.; and Wagner, D. 2018. Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018

    2018

  3. [3]

    Azulay, A.; Dubi \'n ski, J.; Li, Z.; Mittal, A.; and Gandelsman, Y. 2026. Jailbreaking Vision-Language Models Through the Visual Modality. In Forty-third International Conference on Machine Learning

    work page 2026

  4. [4]

    Bethany, E.; Bethany, M.; Flores, J. A. N.; Jha, S. K.; and Najafirad, P. 2024. Jailbreaking Large Language Models with Symbolic Mathematics. arXiv:2409.11445

    arXiv 2024

  5. [5]

    Carlini, N.; Athalye, A.; Papernot, N.; Brendel, W.; Rauber, J.; Tsipras, D.; Goodfellow, I.; Madry, A.; and Kurakin, A. 2019. On Evaluating Adversarial Robustness. arXiv:1902.06705

    arXiv 2019

  6. [6]

    J.; Tram\` e r, F.; Hassani, H.; and Wong, E

    Chao, P.; Debenedetti, E.; Robey, A.; Andriushchenko, M.; Croce, F.; Sehwag, V.; Dobriban, E.; Flammarion, N.; Pappas, G. J.; Tram\` e r, F.; Hassani, H.; and Wong, E. 2024. JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models. In Advances in Neural Information Processing Systems, volume 37, 55005--55029

    work page 2024

  7. [7]

    Chen, Y.; Xu, Z.; Jiang, Y.; Zheng, T.; and Hsieh, C.-J. 2026 a . Text is All You Need for Vision-Language Model Jailbreaking. arXiv:2602.00420

    work page arXiv 2026

  8. [8]

    Chen, Z.; Li, C.; Li, C.; Zhang, X.; Zhang, L.; and He, Y. 2026 b . Jailbreaking LLMs & VLMs: Mechanisms, Evaluation, and Unified Defense. arXiv:2601.03594

    work page arXiv 2026

Show all 43 references
  1. [9]

    D.; Upasani, K.; and Pasupuleti, M

    Chi, J.; Karn, U.; Zhan, H.; Smith, E.; Rando, J.; Zhang, Y.; Plawiak, K.; Coudert, Z. D.; Upasani, K.; and Pasupuleti, M. 2024. Llama Guard 3 Vision: Safeguarding Human-AI Image Understanding Conversations. arXiv:2411.10414

    2024 arXiv

  2. [10]

    Croce, F.; and Hein, M. 2020. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In Proceedings of the 37th International Conference on Machine Learning, ICML'20. JMLR.org

    2020

  3. [11]

    Ding, P.; Kuang, J.; Wang, Z.; Cao, X.; Cai, X.; Chen, J.; and Huang, S. 2025. Why Not Act on What You Know? Unleashing Safety Potential of LLM s via Self-Aware Guard Enhancement. In Findings of the Association for Computational Linguistics: ACL 2025, 6279--6299

    2025

  4. [12]

    S.; Chakraborty, S.; Singh, V.; Guan, T.; Wang, M.; Beirami, A.; Huang, F.; Velasquez, A.; Manocha, D.; and Bedi, A

    Ghosal, S. S.; Chakraborty, S.; Singh, V.; Guan, T.; Wang, M.; Beirami, A.; Huang, F.; Velasquez, A.; Manocha, D.; and Bedi, A. S. 2025. Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignment. In Proceedings of the IEEE/CVF Conference on C...

    2025

  5. [13]

    Gong, Y.; Ran, D.; Liu, J.; Wang, C.; Cong, T.; Wang, A.; Duan, S.; and Wang, X. 2025. FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts. Proceedings of the AAAI Conference on Artificial Intelligence, 39(22): 23951--23959

    2025

  6. [14]

    T.; and Zhang, Y

    Gou, Y.; Chen, K.; Liu, Z.; Hong, L.; Xu, H.; Li, Z.; Yeung, D.-Y.; Kwok, J. T.; and Zhang, Y. 2024. Eyes Closed, Safety on: Protecting Multimodal LLMs via Image-to-Text Transformation. In Computer Vision -- ECCV 2024, 388--404. Springer-Verlag

    2024

  7. [15]

    Hu, X.; Liu, D.; Li, H.; Huang, X.; and Shao, J. 2025. VLSB ench: Unveiling Visual Leakage in Multimodal Safety. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 8285--8316. Association for Computational Linguistics

    2025

  8. [16]

    Huang, G.; Peng, Q.; Xu, G.; Huang, Y.; Lu, Y.; and Shen, Y. 2026. LLaVAShield: Safeguarding Multimodal Multi-Turn Dialogues in Vision-Language Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2026

  9. [17]

    Jia, X.; Liao, J.; Guo, Q.; Ma, T.; Qin, S.; Duan, R.; Li, T.; Huang, Y.; Zeng, Z.; Wu, D.; Li, Y.; Ren, W.; Cao, X.; and Liu, Y. 2025. OmniSafeBench-MM: A Unified Benchmark and Toolbox for Multimodal Jailbreak Attack-Defense Evaluation. arXiv:2512.06589

    2025

  10. [18]

    Li, J.; and Zhan, K. 2026. Safe Responses Matter: Output-Aware Safety Guardrail Mitigate Over-Refusal in MLLMs. In European Conference on Computer Vision (ECCV)

    2026

  11. [19]

    X.; and Wen, J.-R

    Li, Y.; Guo, H.; Zhou, K.; Zhao, W. X.; and Wen, J.-R. 2025. Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models. In Computer Vision -- ECCV 2024, 174--189. Springer Nature Switzerland

    2025

  12. [20]

    Li, Y.; Jiang, H.; and Wei, Z. 2025. D e TAM : Defending LLM s Against Jailbreak Attacks via Targeted Attention Modification. In Findings of the Association for Computational Linguistics: ACL 2025, 11781--11797

    2025

  13. [21]

    Lin, S.; Suri, A.; Oprea, A.; and Tan, C. 2026. Toward Principled LLM Safety Testing: Solving the Jailbreak Oracle Problem. In Proceedings of Machine Learning and Systems (MLSys)

    2026

  14. [22]

    Mazeika, M.; Phan, L.; Yin, X.; Zou, A.; Wang, Z.; Mu, N.; Sakhaee, E.; Li, N.; Basart, S.; Li, B.; Forsyth, D.; and Hendrycks, D. 2024. HarmBench: a standardized evaluation framework for automated red teaming and robust refusal. In Proceedings of the 41st International Confer...

    2024

  15. [23]

    V.; Hayes, J.; Ilie, M.; Pluto, J.; Song, S.; Chaudhari, H.; Shumailov, I.; Thakurta, A.; Xiao, K

    Nasr, M.; Carlini, N.; Sitawarin, C.; Schulhoff, S. V.; Hayes, J.; Ilie, M.; Pluto, J.; Song, S.; Chaudhari, H.; Shumailov, I.; Thakurta, A.; Xiao, K. Y.; Terzis, A.; and Tram\`er, F. 2026. The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbr...

    2026 arXiv

  16. [24]

    Oh, S.; Jin, Y.; Sharma, M.; Kim, D.; Ma, E.; Verma, G.; and Kumar, S. 2024. UniGuard: Towards Universal Safety Guardrails for Jailbreak Attacks on Multimodal Large Language Models. arXiv:2411.01703

    2024 arXiv

  17. [25]

    Ren, Q.; Gao, C.; Shao, J.; Yan, J.; Tan, X.; Lam, W.; and Ma, L. 2024. C ode A ttack: Revealing Safety Generalization Challenges of Large Language Models via Code Completion. In Findings of the Association for Computational Linguistics: ACL 2024, 11437--11452

    2024

  18. [26]

    Shayegani, E.; Dong, Y.; and Abu-Ghazaleh, N. 2023. Jailbreak in Pieces: Compositional Adversarial Attacks on Multi-Modal Language Models. ICLR 2024 Spotlight per the authors' repository; venue not stated on the arXiv record, arXiv:2307.14539

    2023 arXiv

  19. [27]

    Shi, S.; Yin, R.; Xu, N.; Chen, J.; Zhou, C.; Du, T.; Fu, Z.; Wang, J.; Wang, Z.; and Ji, S. 2026. The Boy Who Cried Wolf: Adversarial Misclassification of Safe Inputs as Unsafe in Multimodal Guardrails. In Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery a...

    2026

  20. [28]

    Tramer, F.; Carlini, N.; Brendel, W.; and Madry, A. 2020. On Adaptive Attacks to Adversarial Example Defenses. In Larochelle, H.; Ranzato, M.; Hadsell, R.; Balcan, M.; and Lin, H., eds., Advances in Neural Information Processing Systems, volume 33, 1633--1645. Curran Associates, Inc

    2020

  21. [29]

    Verma, S.; Hines, K.; Bilmes, J.; Siska, C.; Zettlemoyer, L.; Gonen, H.; and Singh, C. 2025. MULTIGUARD : An Efficient Approach for AI Safety Moderation Across Languages and Modalities. In Christodoulopoulos, C.; Chakraborty, T.; Rose, C.; and Peng, V., eds., Proceedings of th...

    2025

  22. [30]

    Wang, H.; Wang, G.; and Zhang, H. 2025. Steering Away from Harm: An Adaptive Approach to Defending Vision Language Model Against Jailbreaks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 29947--29957

    2025

  23. [31]

    Wang, Z.; Wang, H.; Tian, C.; and Jin, Y. 2025. Implicit Jailbreak Attacks via Cross-Modal Information Concealment on Vision-Language Models. arXiv:2505.16446

    2025 arXiv

  24. [32]

    Wei, Z.; Li, Q.; Ruan, J.; Qin, Z.; Wen, L.; Liu, D.; and Shen, W. 2026. Understanding and Defending VLM Jailbreaks via Jailbreak-Related Representation Shift. arXiv:2603.17372

    2026

  25. [33]

    Xu, P.; Pan, M.; Chen, Z.; Yang, S.; Xiao, C.; and Li, B. 2026. SafeVision: Efficient Image Guardrail with Robust Policy Adherence and Explainability. In International Conference on Learning Representations (ICLR)

    2026

  26. [34]

    Ying, Z.; Liu, A.; Liu, X.; and Tao, D. 2024. Unveiling the Safety of GPT-4o: An Empirical Study using Jailbreak Attacks. arXiv:2406.06302

    2024 arXiv

  27. [35]

    Zhang, H.; Wang, Z.; Zheng, S.; Xiao, Z.; Guan, X.; Zandsalimy, M.; and Sushmita, S. 2026 a . Recover, Decode, Reguard: Guard-Agnostic Defense Amplification against Encoded VLM Jailbreaks. arXiv:2607.26574

    2026 arXiv

  28. [36]

    Zhang, H.; Zandsalimy, M.; and Sushmita, S. 2026. Exposing LLM Safety Gaps Through Mathematical Encoding:New Attacks and Systematic Analysis. arXiv:2605.03441

    2026 arXiv

  29. [37]

    Zhang, X.; Zhang, C.; Li, T.; Huang, Y.; Jia, X.; Hu, M.; Zhang, J.; Liu, Y.; Ma, S.; and Shen, C. 2025 a . JailGuard: A Universal Detection Framework for Prompt-based Attacks on LLM Systems. ACM Trans. Softw. Eng. Methodol., 35(1)

    2025

  30. [38]

    Zhang, Y.; Li, J.; Cai, L.; and Li, G. 2026 b . DAVSP : Safety Alignment for Large Vision-Language Models via Deep Aligned Visual Safety Prompt. In Proceedings of the AAAI Conference on Artificial Intelligence. ArXiv:2506.09353

    2026

  31. [39]

    Zhang, Z.; Sun, Z.; Zhang, Z.; Guo, J.; and He, X. 2025 b . FC-Attack: Jailbreaking Multimodal Large Language Models via Auto-Generated Flowcharts. In Findings of the Association for Computational Linguistics: EMNLP 2025, 9299--9316

    2025

  32. [40]

    Zhao, Y.; Zheng, X.; Li, Y.; and Ma, X. 2026. Defense-to-attack: Bypassing weak defenses enables stronger jailbreaks in Vision-Language Models. Pattern Recognition, 179: 113805

    2026

  33. [41]

    S.; and Guo, Q

    Zhou, Q.; Wang, D.; Li, T.; Lin, Y.; Liu, Y.; Dong, J. S.; and Guo, Q. 2025. Defending LVLMs Against Vision Attacks through Partial-Perception Supervision. In Proceedings of the 42nd International Conference on Machine Learning (ICML)

    2025

  34. [42]

    Zong, Y.; Bohdal, O.; Yu, T.; Yang, Y.; and Hospedales, T. 2024. Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models. In Proceedings of the 41st International Conference on Machine Learning (ICML). ArXiv:2402.02207

    2024 arXiv

  35. [43]

    Zou, X.; Kang, J.; Kesidis, G.; and Lin, L. 2026. Understanding and Rectifying Safety Perception Distortion in VLM s. In The Thirty-ninth Annual Conference on Neural Information Processing Systems

    2026

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.