Pith. sign in

REVIEW 3 major objections 6 minor 73 references

GhostPrompt: Cross-Image Adversarial Prompt for Vision-Language Models

T0 review · 3 major / 6 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read GhostPrompt claims that one adversarial text suffix, trained once, can steer vision-language models to attacker-chosen outputs on many unseen images.

desk verdict Genuinely new cross-image prompt attack with a sensible min-max formulation, but the headline ASR gains rest on undisclosed evaluation details that need to be fixed before the numbers can be trusted. read the letter →

arxiv 2607.19683 v1 pith:PAIUGJU2 submitted 2026-07-22 cs.CR

Li Zeng , Zeyu Ye , Meng Xie , Hangtao Zhang , Xianlong Wang , Yanchun Li , Zhetao Li This is my paper
classification cs.CR
keywords vision-languagemodelsadversarialattackpromptcross-imagetransferabilitymin-maxoptimizationcontinuousrelaxationsuccessrateinjection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that text-based adversarial prompts for vision-language models do not have to be re-optimized for each image. It introduces GhostPrompt, a single adversarial suffix that is trained once and then, when appended to a benign prompt template, steers a VLM to produce an attacker-chosen response on many unseen images. The key move is to treat the image as a variable during optimization: an inner loop synthesizes the 'worst-case' image for the current suffix, and an outer loop updates the suffix so that the target response still wins under that hard visual condition. The authors report that this joint optimization raises attack success rates by over 30% relative to prior text-based attacks on four open VLMs while cutting training time by roughly 70%. If correct, this makes prompt-reuse attacks practical in settings where users share or download prompt templates.

What carries the argument

The load-bearing mechanism is an alternating min-max optimization over two modalities. In the inner (image maximization) stage, a small perturbation is added to a clean training image via projected gradient ascent to maximize the negative log-likelihood of the target response, producing a 'worst-case' visual condition for the current suffix. In the outer (text minimization) stage, the suffix is represented as a continuous token distribution matrix — each row a soft distribution over the vocabulary — sampled through a continuous relaxation of discrete token choices so gradients flow end-to-end. Three losses are combined: a guided adversarial loss (negative log-likelihood of the target), a sem

What would settle it

Re-run the attack with a fixed, pre-registered split in which the 20 training images come from a hold-out set disjoint from all test images, count success only on exact full-target-string matches, and compute ASR per image rather than per prompt-image pair; if the reported >30% relative improvement over baselines collapses or reverses, the cross-image transferability claim does not hold.

Watch

Extended reading notes

Core claim

Central claim: a suffix optimized once in continuous token space, relaxed into soft one-hot vectors by a differentiable sampling trick and trained against adversarially perturbed 'worst-case' images, encodes image-invariant adversarial features. When appended to a prompt template, this discrete suffix steers four open VLMs toward attacker-chosen outputs on unseen MS-COCO and ImageNet images. Prior text attacks hold the image fixed, entangling the suffix with one visual input; GhostPrompt solves a min-max problem where an inner step constructs hard visual conditions and an outer step updates the suffix to stay effective under them, collapsing the model's internal states into an image-agnostic

Load-bearing premise

The load-bearing premise is that the reported attack success rates measure true cross-image generalization: the paper does not state the exact split between the roughly 20 training images and the MS-COCO/ImageNet test subsets, nor the precise string-matching rule for success, so a favorable selection or matching could inflate the gains.

Editorial extensions

If this is right

  • If GhostPrompt works as reported, a single poisoned prompt template can be distributed through prompt-sharing platforms and hijack interactions on many different user images without per-image optimization.
  • Perplexity filtering and tokenization defenses would be largely ineffective against the optimized suffix, while adversarial training that does reduce the attack also degrades clean-task accuracy.
  • The attack success holds across both MS-COCO and ImageNet subsets and across four different VLM architectures, suggesting the vulnerability is not tied to one dataset or model.
  • The roughly 70% reduction in training time would lower the cost of generating transferable adversarial prompts, making large-scale prompt poisoning more feasible.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper does not test: whether the same min-max recipe transfers to video or multi-image inputs, where the 'worst-case' visual condition could be a temporal perturbation rather than a single-image one.
  • The semantic alignment loss is described as helping the suffix evade safety alignment; a reader could test this directly by comparing trigger success on aligned versus unaligned VLM checkpoints with and without that loss.
  • If the t-SNE interpretation is right — the suffix collapses the model's first-token distribution into one cluster — a defense could look for abnormally low variance in internal states across images, a direction the paper does not discuss.
  • The paper's black-box results are limited to open models; if the optimization were run on a stronger teacher, cross-architecture transfer might extend to commercial APIs, though the paper reports low success there and lists it as a limitation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. GhostPrompt proposes a text-only adversarial suffix that, once optimized, can be appended to a prompt template and steer vision-language models (MiniGPT-4, BLIP-2, InstructBLIP, LLaVA-v1.5-7b) to attacker-chosen outputs across unseen images. The method uses a min-max objective (Eq. 3) over a continuous token distribution (Gumbel-Softmax) and a projected-gradient inner loop that synthesizes 'worst-case' images, plus losses for target likelihood, text coherence, and semantic alignment. Experiments report >30% relative ASR gains over baselines on MS-COCO/ImageNet subsets, ~70% training-time reduction, black-box transfer, and some defense resistance. The manuscript also states limitations: discrete-text ASR below continuous image attacks, limited cross-model transfer, and poor performance on commercial VLMs.

Significance. The paper's contribution is a new attack setting (cross-image transferable adversarial prompts) and a technically plausible min-max framework; the alternating optimization with Gumbel-Softmax and semantic-alignment regularizers is a reasonable approach. Strengths: code is promised, experiments cover four VLMs and many baselines, ablations isolate loss components, and defense evaluation (including adversarial training) goes beyond ASR-only reporting. If the empirical claims hold, this is a useful and timely result for the VLM security community. However, the current manuscript does not disclose enough evaluation-protocol detail to establish the central claim; the reported gains may be inflated by test-set tuning or loose matching. I therefore treat the significance as conditional on the missing protocol being supplied and the numbers rechecked.

major comments (3)
  1. [Sec. 5.1 / Table 1 / Sec. 8 / Fig. 7] The central claim — a single prompt transferring to unseen images with >30% relative ASR gain — requires a clean separation between the 20 training images and the MS-COCO/ImageNet test subsets. The paper never states this split. Moreover, Fig. 7 uses ASR on these same benchmarks to choose 20 as the default, so the test benchmarks are used for hyperparameter selection. Finally, 'matches the attacker-specified target response' is undefined: Fig. 2 shows responses longer than the target 'Yes', so exact-match, prefix-match, and containment give very different ASRs. Please report the split, the matching rule, and preferably a separate validation set for choosing the number of training images.
  2. [Algorithm 1 / Sec. 4.4 / Eq. (3)] Algorithm 1 calls the inner update 'Projected Gradient Ascent' but line 17 has no projection or clipping; the perturbation δ_v is unbounded. With an unbounded inner maximizer, the 'worst-case image' can drift arbitrarily far from the clean image, so the trained prompt can learn to ignore the visual stream entirely rather than to transfer across realistic images. This is not the 'subtle perturbations' threat model in the abstract. Also, Algorithm 1 initializes a single x'_v and never loops over multiple images, which is inconsistent with Eq. (3)'s expectation over D and with Sec. 8's '20 training images'. Please add the projection step (budget epsilon), and show how multiple training images enter the alternating loop.
  3. [Sec. 5.1 / Eq. (7)] Evaluation-critical hyperparameters are not reported: lambda_align, lambda_tc, mu, alpha1, alpha2, K, N, tau, suffix length L, and the composition of I_benign/I_harmful in Eq. (9). Without these, the experiments cannot be reproduced or compared. Since ASR is sensitive to suffix length and to the alignment-loss weight, please include a full configuration table and the exact test-subset sizes.
minor comments (6)
  1. [Sec. 8 / Table 3] The text says incorporating the semantic alignment loss improves ASR by 'an average of 7%', but Table 3 implies average relative improvements of about 11–14% (MiniGPT-4: 0.620 vs 0.558; BLIP-2: 0.597 vs 0.541; InstructBLIP: 0.512 vs 0.447). Also, the statement that removing L_tc leads to a 'slight increase' is not true for MiniGPT-4 (0.616 vs 0.620).
  2. [Abstract] Typo: 'avalable' should be 'available'.
  3. [Sec. 5.1] The sizes of the MS-COCO and ImageNet subsets are not given. Reporting only 'a subset' is insufficient to assess statistical power or to compare with prior work.
  4. [Fig. 5(c)] The cross-model transfer ASRs are very low (roughly 0.13–0.23). The text says GhostPrompt outperforms 'on the majority of benchmarks (2 out of 3)', but the figure is hard to read; please add numeric labels or a table.
  5. [Sec. 4.3] Gumbel-Softmax temperature τ appears in Eq. (4) but no annealing schedule is described. Since τ strongly affects the soft-to-hard transition, reporting the schedule would improve reproducibility.
  6. [Table 2] The PF column reports PPL values, but the caption only says 'ASR'. Please clarify which columns are ASR and which are PPL.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: GhostPrompt's min-max optimization and held-out ASR evaluation do not reduce to their own inputs; the flagged concerns are evaluation-validity risks, not derivational circularity.

full rationale

The paper's central claim is empirical: an adversarial suffix optimized on a small set of images via the min-max objective in Eq. (3) transfers to held-out images. The optimization minimizes L_text = L_ga + λ_align L_align + λ_tc L_tc, where L_ga maximizes the likelihood of the target response on training images. This is a standard training objective, not a fitted parameter that is then relabeled as a prediction; the reported ASR is measured on test images and can in principle be low. The method does not define cross-image transferability in terms of the training loss, nor does it invoke a uniqueness theorem or a self-citation to force its conclusions. While the paper self-cites extensively in the references, those citations are contextual and are not load-bearing for the core attack claim; the key techniques (Gumbel-Softmax, projected gradient ascent, GCG, etc.) are external, standard methods. The 'worst-case image' mechanism is empirically validated by comparison with a data-augmentation baseline, not asserted by construction. The disclosed limitations in Sec. 9 further acknowledge that perfect ASR, black-box transfer across very different architectures, and commercial VLMs remain unachieved, which is inconsistent with a result that is purely true by definition. The remaining concerns about undisclosed train/test split details, the exact matching rule for ASR, and the missing projection step in Algorithm 1 are important for reproducibility and could affect the validity or generalizability of the numeric claims, but they do not constitute circular reasoning: the reported numbers are not equivalent to the paper's inputs by construction. Therefore, this paper has no significant circularity.

Assumptions & free parameters 9 free parameters · 6 assumptions · 0 invented entities

The method itself is an attack optimizer, so the fitted suffix tokens are expected. The ledger lists the hyperparameters and datasets that are load-bearing but not specified; the most significant is the undisclosed training/test image split, since the cross-image claim depends on it. The unbounded image perturbation and unstated instruction sets are secondary but material gaps.

free parameters (9)
  • λ_align = not reported
    Weight of Semantic Alignment Loss in Eq. 7; attack performance depends on it.
  • λ_tc = not reported
    Weight of Text Coherence Loss in Eq. 7.
  • μ = not reported
    Balance factor between L_cls and L_far in Eq. 11.
  • α1, α2 (image and prompt step sizes) = not reported
    Step sizes in Algorithm 1; convergence and final ASR depend on them.
  • K, N (iteration count and update interval) = not reported
    Control the alternating optimization schedule in Algorithm 1.
  • suffix length L = not reported (swept 8–64 in Fig. 5a)
    Trade-off between attack effectiveness and detector evasion.
  • Gumbel-Softmax temperature τ = not reported
    Controls the hardness of the continuous relaxation in Eq. 4.
  • number of training images = 20
    Default set in Sec. 8 based on ASR saturation ablation on the same benchmarks used for evaluation.
  • benign/harmful instruction sets for Lalign = unspecified
    The centroids c+ and c- in Eq. 9 are computed from hand-selected instruction sets that are not described, so the semantic alignment behavior is not externally auditable.
assumptions (6)
  • domain assumption Attacker has white-box access to the target VLM's gradients and embeddings
    Section 3.1 assumes white-box access to optimize the suffix; the entire method depends on this.
  • domain assumption Gumbel-Softmax continuous relaxation is a faithful proxy for discrete argmax token selection
    Eqs. 4-5 and the argmax extraction in Algorithm 1 assume training signal transfers to discrete tokens at test time.
  • ad hoc to paper The image perturbation update can be treated as unbounded PGA with no explicit projection
    Algorithm 1 lines 13-18 update x'_v by sign gradient without an epsilon-ball projection, although Section 4.4 refers to PGD-style perturbation.
  • domain assumption A suffix that raises the likelihood of the target prefix will cause natural autoregressive completion of the rest
    Section 3.1 asserts that next-token prediction continues the target response; the ASR metric relies on this.
  • domain assumption Benign/harmful instruction centroid directions in text encoder space correspond to safety alignment behavior
    Eq. 9's L_align assumes Rep(I) captures harmfulness and closeness to c+ improves stealth.
  • domain assumption Test images are representative and disjoint from the 20 training images
    Implicit in Sec. 5.2; the cross-image transferability claim collapses if the split is not clean, and the paper never states it.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GhostPrompt: Cross-Image Adversarial Prompt for Vision-Language Models." pith.science (2026). https://pith.science/paper/PAIUGJU2

@misc{pith2026260719683,
  author       = {Pith},
  title        = {Pith review of: GhostPrompt: Cross-Image Adversarial Prompt for Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PAIUGJU2}},
  note         = {Machine review of arXiv:2607.19683}
}
read the original abstract

Vision-Language Models (VLMs) are known to be vulnerable to adversarial attacks, where subtle perturbations to images or texts induce erroneous outputs. However, most text-based attacks are adapted from language-model-centric methods, in which the visual input is fixed during optimization, resulting in adversarial prompts that are tied to specific images and thus limiting their attack effectiveness. To this end, we first introduce a new research perspective: cross-image transferability for adversarial prompts. We then propose GhostPrompt, an adversarial prompt that is optimized once and reused to steer VLM outputs toward attacker-specified responses across diverse images. GhostPrompt employs a joint optimization that distills image-invariant adversarial features into the prompt by "worst-case" generation. Specifically, it alternates between constructing hard visual conditions for the current prompt and updating the prompt to remain effective under these conditions. Extensive experiments on prevalent VLMs verify that \ourmethod achieves an improvement of over 30% in attack success rates compared to state-of-the-art (SoTA) baselines, while reducing computation time by ~70%. Our code is avalable at https://github.com/Ye-ze-yu/GhostPrompt.

Figures

Figures reproduced from arXiv: 2607.19683 by the authors.

Figure 1
Figure 1. A classic case of our GhostPrompt attack. A user gets prompt templates from a network platform to elicit high-quality, task-specific reviews from the VLM ( [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Example adversarial prompts in a realistic usage [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The pipeline of GhostPrompt. Both the image perturbation 𝛿𝑣 and the token distribution matrix 𝜃 are learnable. In each iteration, 𝛿𝑣 is updated by maximizing the L𝑣𝑖𝑠𝑢𝑎𝑙 to find the worst-case visual embedding, while 𝜃 is updated by minimizing the L𝑡𝑒𝑥𝑡 to craft a cross-image adversarial prompt. The two perturbations are optimized with opposing objectives. However, directly optimizing over the full visual distributi… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Average training times to generate a complete ad [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: (a) Detection rate under different suffix lengths, [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Experimental validation of GhostPrompt. (a) ASR for GhostPrompt versus a random-image baseline over op￾timization iterations. (b) t-SNE visualization of internal re￾sponse states under GhostPrompt and the prompt template. to a recovery in ASR. Moreover, the PF leads to…
Figure 7
Figure 7. Figure 7: ASR comparison between GhostPrompt and Multi-I when varying the numbers of the training image. We evaluate both methods on three VLMs using 1, 5, 10, 20, 50 and 100 training images. 9 Conclusion, Limitations, and Future Work In this paper, we present GhostPrompt, a met…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

73 extracted references · 20 linked inside Pith

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report.arXiv preprint arXiv:2303.08774 (2023)

    arXiv 2023

  2. [2]

    Gabriel Alon and Michael Kamfonas. 2023. Detecting language model attacks with perplexity.arXiv preprint arXiv:2308.14132(2023)

    arXiv 2023

  3. [3]

    Tom B Brown, Dandelion Mané, Aurko Roy, Martín Abadi, and Justin Gilmer

  4. [4]

    Hwan Chang, Yonghyun Jun, and Hwanhee Lee. 2025. Chatinject: Abusing chat templates for prompt injection in llm agents.arXiv preprint arXiv:2509.22830 (2025)

    arXiv 2025

  5. [5]

    Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. 2023. Jailbreaking Black Box Large Language Models in Twenty Queries.arXiv preprint arXiv:2310.08419(2023)

    arXiv 2023

  6. [6]

    Jiawei Chen, Simin Huang, Jiawei Du, Shuaihang Chen, Yu Tian, Mingjie Wei, Chao Yu, and Zhaoxia Yin. 2026. Tex3D: Objects as attack surfaces via adversarial 3D textures for vision-language-action models.arXiv preprint arXiv:2604.01618 (2026)

    arXiv 2026

  7. [7]

    Jiawei Chen, Yang Yang, Chao Yu, Yu Tian, Zhi Cao, Xue Yang, Linghao Li, Hang Su, and Zhaoxia Yin. 2026. Red teaming large reasoning models. InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 22559–22591

    2026

  8. [8]

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Albert Li, Pascale Fung, and Steven C. H. Hoi. 2023. In- structBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning. InProceedings of the 37th International Conference on Neural Information Processing Systems (NeurIPS’23), Vol. 36

    2023

Show all 73 references
  1. [9]

    Lang Gao, Jiahui Geng, Xiangliang Zhang, Preslav Nakov, and Xiuying Chen

  2. [10]

    Goodfellow, Jonathon Shlens, and Christian Szegedy

    Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and Harnessing Adversarial Examples. InProceedings of the 3rd International Conference on Learning Representations (ICLR’15). 1–11

    2015

  3. [11]

    Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh

  4. [12]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. InProceedings of the 12th International Conference on Learning Representations (ICLR’22)

    2022

  5. [13]

    Hanxun Huang, Sarah Monazam Erfani, Yige Li, Xingjun Ma, and James Bailey

  6. [14]

    InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17)

    Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). 6904–6913

  7. [15]

    Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchen- bauer, Ping-yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, and Tom Goldstein. 2024. Baseline Defenses for Adversarial Attacks Against Aligned Language Models. InProceedings of the 12th Inter...

    2024

  8. [16]

    Eric Jang, Shixiang Gu, and Ben Poole. 2017. Categorical reparameterization with gumbel-softmax. InProceedings of the 5th International Conference on Learning Representations (ICLR’17)

    2017

  9. [17]

    Chengxin Li, Yujie Gu, Pengpeng Qiao, Shengli Pan, Kouichi Sakurai, and Zhetao Li. 2026. Privacy-Preserving Yet Vulnerable: Data Poisoning Attacks Against Differential Privacy Sparse Mobile Crowdsensing System.IEEE Transactions on Mobile Computing(2026)

    2026

  10. [18]

    Drew A Hudson and Christopher D Manning. 2019. GQA: A new dataset for real- world visual reasoning and compositional question answering. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR’19). 6700–6709

    2019

  11. [19]

    Minghui Li, Hangtao Zhang, Yanjun Zhang, Li Zeng, Chao Chen, Qiyun Shao, Wei Wan, Shengshan Hu, and Leo Yu Zhang. 2025. Fine-grained poisoning framework against federated learning.IEEE Transactions on Dependable and Secure Computing(2025)

    2025

  12. [20]

    Shen Li, Liuyi Yao, Lan Zhang, and Yaliang Li. 2024. Safety layers in aligned large language models: The key to llm security.arXiv preprint arXiv:2408.17003(2024)

    2024 arXiv

  13. [21]

    Zeyi Liao and Huan Sun. 2024. Amplegcg: Learning a universal and transferable generative model of adversarial suffixes for jailbreaking both open and closed llms.arXiv preprint arXiv:2404.07921(2024)

    2024 arXiv

  14. [22]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. BLIP-2: Boot- strapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models. InProceedings of the 40th International Conference on Machine Learning (ICML’23). 12888–12900

    2023

  15. [23]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual Instruc- tion Tuning.arXiv preprint arXiv:2304.08485(2023)

    2023 arXiv

  16. [24]

    Hongfu Liu, Yuxi Xie, Ye Wang, and Michael Shieh. 2024. Advancing Adversarial Suffix Transfer Learning on Aligned Large Language Models. InProceedings of the 29th Conference on Empirical Methods in Natural Language Processing (EMNLP’24). 7213–7224

    2024

  17. [25]

    Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2024. AutoDAN: Generat- ing Stealthy Jailbreak Prompts on Aligned Large Language Models. InProceedings of the 12th International Conference on Learning Representations (ICLR’24)

    2024

  18. [26]

    Lawrence Zitnick

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014. Microsoft COCO: Common Objects in Context. InProceedings of the 13th European Conference on Computer Vision (ECCV’14). 740–755

    2014

  19. [27]

    Dong Lu, Zhiqiang Wang, Teng Wang, Weili Guan, Hongchang Gao, and Feng Zheng. 2023. Set-level Guidance Attack: Boosting Adversarial Transferability of Vision-Language Pre-training Models. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV’23). 102–111

    2023

  20. [28]

    Haochen Luo, Jindong Gu, Fengyuan Liu, and Philip Torr. 2024. An Image Is Worth 1000 Lies: Transferability of Adversarial Images across Prompts on Vision- Language Models. InProceedings of the 12th International Conference on Learning Representations (ICLR’24). 1–15

    2024

  21. [29]

    Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. Towards deep learning models resistant to adversarial attacks. InProceedings of the 6th International Conference on Learning Representations (ICLR’18)

    2018

  22. [30]

    Saiqin Long, Chongxi Rao, Haolin Liu, Yunjie Chen, Qingyong Deng, Jing Shang, and Zhetao Li. 2025. Fault-Tolerant Aware Task Offloading Based on Rein- forcement Learning in Mobile Edge Computing.IEEE Transactions on Mobile Computing(2025)

    2025

  23. [31]

    Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. 2017. Universal adversarial perturbations. InProceedings of the 2017 MM ’2026, November 10–14, 2026, Rio de Janeiro, Brazil Li Zeng, Zeyu Ye, Meng Xie, Hangtao Zhang, Xianlong Wang, Yanchun Li, an...

    2017

  24. [32]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, et al. 2022. Training Language Models to Follow Instructions with Human Feedback. InProceedings of the 36th International Conference on Neural I...

    2022

  25. [33]

    Wenbo Pan, Zhichao Liu, Xianlong Wang, Haining Yu, and Xiaohua Jia. 2026. Towards Long-Horizon Interpretability: Efficient and Faithful Multi-Token Attri- bution for Reasoning LLMs.arXiv preprint arXiv:2602.01914(2026)

    2026 arXiv

  26. [34]

    Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, and Amin Karbasi. 2023. Tree of Attacks: Jailbreaking Black-Box LLMs Automatically.arXiv preprint arXiv:2312.02119(2023)

    2023 arXiv

  27. [35]

    Learn Prompting. 2024. Instruction Defense. https://learnprompting.org/docs/ prompt_hacking/defensive_measures/instruction

    2024

  28. [36]

    ProtectAI.com. 2024. Fine-Tuned DeBERTa-v3-base for Prompt Injection Detec- tion. https://huggingface.co/ProtectAI/deberta-v3-base-prompt-injection-v2

    2024

  29. [37]

    Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, and Prateek Mittal. 2024. Visual Adversarial Examples Jailbreak Aligned Large Language Models. InProceedings of the 38th AAAI Conference on Artificial Intelli- gence (AAAI’24), Vol. 38. 21527–21536

    2024

  30. [38]

    PromptBase. 2025. PromptBase | Prompt Marketplace: Midjourney, ChatGPT, Sora, FLUX & more. https://promptbase.com/ Accessed: 2025-07-15

    2025

  31. [39]

    Bernstein, Alexander C

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael S. Bernstein, Alexander C. Berg, and Li Fei-Fei. 2015. ImageNet Large Scale Visual Recognition Challenge.International Journal of Computer Vis...

    2015

  32. [40]

    Zedian Shao, Hongbin Liu, Yuepeng Hu, and Neil Zhenqiang Gong. 2024. Re- fusing safe prompts for multi-modal large language models.arXiv preprint arXiv:2407.09050(2024)

    2024 arXiv

  33. [41]

    Yufei Song, Ziqi Zhou, Minghui Li, Xianlong Wang, Hangtao Zhang, Menghao Deng, Wei Wan, Shengshan Hu, and Leo Yu Zhang. 2025. Pb-uap: Hybride universal adversarial attack for image segmentation. InICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal P...

    2025

  34. [42]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. InProceedings ...

    2021

  35. [43]

    Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2014. Intriguing properties of neural networks. InProceedings of the 2nd International Conference on Learning Representations (ICLR’14)

    2014

  36. [44]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yas- mine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhos- ale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288(2023)

    2023 arXiv

  37. [45]

    Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research9 (2008), 2579–2605

    2008

  38. [46]

    Yufei Song, Ziqi Zhou, Qi Lu, Hangtao Zhang, Yifan Hu, Lulu Xue, Shengshan Hu, Minghui Li, and Leo Yu Zhang. 2026. Segtrans: Transferable adversarial examples for segmentation models.IEEE Transactions on Multimedia(2026)

    2026

  39. [47]

    Ruofan Wang, Xingjun Ma, Hanxu Zhou, Chuanjun Ji, Guangnan Ye, and Yu-Gang Jiang. 2024. White-box Multimodal Jailbreaks Against Large Vision-Language Models. InProceedings of the 32nd ACM International Conference on Multimedia (MM’24). Association for Computing Machinery, 6920–6928

    2024

  40. [48]

    Xianlong Wang, Minghui Li, Wei Liu, Hangtao Zhang, Shengshan Hu, Yechao Zhang, Ziqi Zhou, and Hai Jin. 2024. Unlearnable 3d point clouds: Class-wise transformation is all you need.Advances in Neural Information Processing Systems 37 (2024), 99404–99432

    2024

  41. [49]

    Xianlong Wang, Hewen Pan, Hangtao Zhang, Minghui Li, Shengshan Hu, Ziqi Zhou, Lulu Xue, Aishan Liu, Yunpeng Jiang, Leo Yu Zhang, et al. 2024. Trojan- robot: Physical-world backdoor attacks against vlm-based robotic manipulation. arXiv preprint arXiv:2411.11683(2024)

    2024

  42. [50]

    Hetvi Waghela, Jaydip Sen, and Sneha Rakshit. 2024. Enhancing adversarial text attacks on bert models with projected gradient descent.arXiv preprint arXiv:2407.21073(2024)

    2024 arXiv

  43. [51]

    Yichen Wang, Yuxuan Chou, Ziqi Zhou, Hangtao Zhang, Wei Wan, Shengshan Hu, and Minghui Li. 2025. Breaking barriers in physical-world adversarial examples: Improving robustness and transferability via robust feature. InProceedings of the AAAI Conference on Artificial Intelligen...

    2025

  44. [52]

    Yichen Wang, Hangtao Zhang, Hewen Pan, Ziqi Zhou, Xianlong Wang, Peijin Guo, Lulu Xue, Shengshan Hu, Minghui Li, and Leo Yu Zhang. 2026. Advedm: Fine-grained adversarial attack against vlm-based embodied agents.Advances in Neural Information Processing Systems38 (2026), 136551–136575

    2026

  45. [53]

    Xikang Yang, Xuehai Tang, Fuqing Zhu, Jizhong Han, and Songlin Hu. 2024. Enhancing Cross-Prompt Transferability in Vision-Language Models through Contextual Injection of Target Tokens.arXiv preprint arXiv:2406.13294(2024)

    2024 arXiv

  46. [54]

    Xianlong Wang, Hangtao Zhang, Wenbo Pan, Ziqi Zhou, Changsong Jiang, Li Zeng, and Xiaohua Jia. 2026. Dual-branch Robust Unlearnable Examples.arXiv preprint arXiv:2605.01718(2026)

    2026 arXiv

  47. [55]

    Zonghao Ying, Aishan Liu, Tianyuan Zhang, Zhengmin Yu, Siyuan Liang, Xiang- long Liu, and Dacheng Tao. 2024. Jailbreak Vision Language Models via Bi-Modal Adversarial Prompt.arXiv preprint arXiv:2406.04031(2024)

    2024 arXiv

  48. [56]

    Li Zeng, Xiaojun Mo, Meng Xie, Hangtao Zhang, Yixiang Liu, Yezhuo Peng, and Yanchun Li. 2025. PSFD: Proactive Spatial-Frequency Defense against Mali- cious Exemplar-Guided Image Editing. In2025 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 1–6

    2025

  49. [57]

    Qiusi Zhan, Richard Fang, Henil Shalin Panchal, and Daniel Kang. 2025. Adap- tive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents. InProceedings of the 2025 Conference of the North American Chapter of the Association for Computational Linguistics...

    2025

  50. [58]

    Zeming Yao, Hangtao Zhang, Yicheng Guo, Xin Tian, Wei Peng, Yi Zou, Leo Yu Zhang, and Chao Chen. 2024. Reverse backdoor distillation: Towards online backdoor attack detection for deep neural network models.IEEE Transactions on Dependable and Secure Computing21, 6 (2024), 5098–5111

    2024

  51. [59]

    Hangtao Zhang, Yichen Wang, Shihui Yan, Chenyu Zhu, Ziqi Zhou, Linshan Hou, Shengshan Hu, Minghui Li, Yanjun Zhang, and Leo Yu Zhang. 2025. Test-time backdoor detection for object detection models. InProceedings of the Computer Vision and Pattern Recognition Conference. 24377–24386

    2025

  52. [60]

    Hangtao Zhang, Zeming Yao, Leo Yu Zhang, Shengshan Hu, Chao Chen, Alan Liew, and Zhetao Li. 2023. Denial-of-service or fine-grained control: Towards flexi- ble model poisoning attacks on federated learning.arXiv preprint arXiv:2304.10783 (2023)

    2023 arXiv

  53. [61]

    Hangtao Zhang, Yucheng Zhao, Sishun Liu, Ziqi Zhou, Zeyu Ye, Wei Wan, Minghui Li, Shengshan Hu, Yanjun Zhang, Yi Liu, and Leo Yu Zhang. 2026. Defending Jailbreak Attacks on Large Language Models via Manifold Trajectory Kinetics. In35th USENIX Security Symposium (USENIX Security 26)

    2026

  54. [62]

    Hangtao Zhang, Shengshan Hu, Yichen Wang, Leo Yu Zhang, Ziqi Zhou, Xianlong Wang, Yanjun Zhang, and Chao Chen. 2024. Detector collapse: Physical-world backdooring object detection to catastrophic overload or blindness in autonomous driving.arXiv preprint arXiv:2404.11357(2024)

    2024 arXiv

  55. [63]

    Jiaming Zhang, Junhong Ye, Xingjun Ma, Yige Li, Yunfan Yang, Yunhao Chen, Jitao Sang, and Dit-Yan Yeung. 2025. Anyattack: Towards Large-scale Self-supervised Adversarial Attacks on Vision-language Models. InProceedings of the 2025 Com- puter Vision and Pattern Recognition Conf...

    2025

  56. [64]

    Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan LI, Ngai-Man (Man) Cheung, and Min Lin. 2023. On Evaluating Adversarial Robustness of Large Vision-Language Models. InProceedings of the 37th Conference on Neural Infor- mation Processing Systems (NeurIPS’23), Vol. 36. 5...

    2023

  57. [65]

    Ziqi Zhou, Menghao Deng, Yufei Song, Hangtao Zhang, Wei Wan, Shengshan Hu, Minghui Li, Leo Yu Zhang, and Dezhong Yao. 2025. Darkhash: A data-free backdoor attack against deep hashing.IEEE Transactions on Information Forensics and Security(2025)

    2025

  58. [66]

    Hangtao Zhang, Chenyu Zhu, Xianlong Wang, Ziqi Zhou, Changgan Yin, Minghui Li, Lulu Xue, Yichen Wang, Shengshan Hu, Aishan Liu, et al. 2025. BadRobot: Jailbreaking Embodied LLM Agents in the Physical World. InProceedings of the Thirteenth International Conference on Learning R...

    2025

  59. [67]

    Ziqi Zhou, Yufei Song, Minghui Li, Shengshan Hu, Xianlong Wang, Leo Yu Zhang, Dezhong Yao, and Hai Jin. 2024. Darksam: Fooling segment anything model to segment nothing.Advances in Neural Information Processing Systems37 (2024), 49859–49880

    2024

  60. [68]

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2024. MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models. InProceedings of the 12th International Conference on Learning Representations (ICLR’24). 1–17

    2024

  61. [69]

    Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models.arXiv preprint arXiv:2307.15043(2023)

    2023 arXiv

  62. [70]

    Ziqi Zhou, Shengshan Hu, Minghui Li, Hangtao Zhang, Yechao Zhang, and Hai Jin. 2023. Advclip: Downstream-agnostic adversarial examples in multimodal contrastive learning. InProceedings of the 31st ACM International Conference on Multimedia. 6311–6320

    2023

  63. [2017]

    Adversarial patch.arXiv preprint arXiv:1712.09665(2017)

    2017 arXiv

  64. [2024]

    Shaping the safety boundaries: Understanding and defending against jailbreaks in large language models.arXiv preprint arXiv:2412.17034(2024)

    2024 arXiv

  65. [2025]

    InProceedings of the 42nd International Conference on Machine Learning (ICML’25)

    X-Transfer Attacks: Towards Super Transferable Adversarial Attacks on CLIP. InProceedings of the 42nd International Conference on Machine Learning (ICML’25). Poster presentation

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.