Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

Black-Box Adversarial Attack on Vision Language Models for Autonomous Driving

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a transfer-based black-box attack, Cascading Adversarial Disruption (CAD), misleads vision-language driving models with no gradient or query access, cutting real-world route completion by 61.11% and causing a…

desk verdict Genuinely new black-box attack pipeline for AD VLMs with a solid dataset, but the transferability claim outruns the evidence: no victim with a non-CLIP visual encoder is tested. read the letter →

arxiv 2501.13563 v1 pith:KHNQY7PC submitted 2025-01-23 cs.CV cs.AI

Lu Wang , Tianyuan Zhang , Yang Qu , Siyuan Liang , Yuwei Chen , Aishan Liu , Xianglong Liu , Dacheng Tao This is my paper · ORCID
classification cs.CVcs.AI
keywords black-boxadversarialattackvision-languagemodelsautonomousdrivingtransfer-basedCLIPembeddingalignmentpatchesreasoningchainCADAdataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to establish that autonomous-driving systems built on vision-language models (VLMs) can be hijacked blindly: no knowledge of the victim model's weights, gradients, or even its outputs is required. It introduces Cascading Adversarial Disruption (CAD), which uses a surrogate image-text model, CLIP, to align perturbed driving images with deceptive textual descriptions of a wrong perception-prediction-planning chain, and to flip the model's high-level assessment of whether the scene is safe. On open-loop and closed-loop benchmarks spanning Dolphins, DriveLM, LMDrive, and general VLMs including GPT-4o, CAD is reported to outperform existing attack baselines by 13.43% on average. In physical tests on two robot vehicles, adversarial patches printed and attached to a stop sign and an obstacle vehicle reduce route completion from 72.22% to 11.11% and make one vehicle crash into the obstacle. If these results are right, a practical vulnerability class follows: a camera-visible patch alone can cause VLM-driven vehicles to fail.

What carries the argument

The load-bearing mechanism is a three-part loss optimized in the embedding space of CLIP, a pre-trained contrastive model that maps images and text into a shared space, rather than in the victim VLM itself. The decision-chain loss $L_l$ minimizes the cosine similarity between the visual embedding of the perturbed image and the text embedding of deceptive sentences generated backward along the perception-prediction-plan chain by an auxiliary VLM. The risky-scene loss $L_h$ computes an image-text matching probability over the descriptors 'a safe driving scenario' and 'an unsafe driving scenario' and uses a mask that flips the class that was less probable on clean images, forcing the scene-level safety assessment to reverse. A third term $L_d$ minimizes the cosine similarity between clean and adversarial image embeddings, and the full objective is $L_{\mathrm{adv}}=\alpha L_l+\beta L_h+\gamma L_d$ with $\alpha=0.75$, $\beta=0.05$, and $\gamma=0.75$, optimized by momentum-based gradient descent under an $\ell_\infty$ norm budget of $0.1$. Because CLIP's alignment is what defines 'deceptive' and 'risky', the victim's own reasoning is never queried; the attack depends on CLIP-to-victim transferability.

What would settle it

Test CAD against a driving VLM whose visual encoder was trained without any CLIP-style contrastive alignment and whose text decoder was never exposed to CLIP embeddings, using the same $\ell_\infty=0.1$ budget on the DriveLM benchmark; if the score drop shrinks to the no-attack level, the transferability premise is false. A cheaper physical check is to reuse the printed patch that made the LIMO robot crash on the same robot with a different VLM backbone: if the collision rate returns to the clean rate, the real-world result is pipeline-specific rather than a general vulnerability of VLM driving.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that black-box attacks on driving VLMs work best when the perturbation is crafted to corrupt two different levels of reasoning at once. The decision-chain component works backward from a desired planning error to produce a coherent deceptive text for the perception, prediction, and planning stages, then pulls the adversarial image's CLIP embedding toward that text; the risky-scene component uses an image-text matching head over the descriptors 'a safe driving scenario' and 'an unsafe driving scenario' and flips the less probable class. These two objectives, plus a semantic-discrepancy term that pushes clean and adversarial visual embeddings apart, are optimized jointly under a projected $\ell_\infty$ budget of $0.1$. The paper reports that this is the first black-box attack designed specifically for VLM-based autonomous driving, and that it achieves state-of-the-art effectiveness: an average score reduction of 19.60% across driving-specific VLMs and general VLMs in open-loop digital tests, an 18.87% drop in closed-loop CARLA driving score, and a fall in real-vehicle route-completion success from 72.22% to 11.11%, including a direct collision into an obstacle vehicle carrying the adversarial patch.

Load-bearing premise

The load-bearing premise is that a pattern that shifts the meaning of an image inside CLIP's shared image-text space will also shift the meaning inside any victim driving VLM, even though the attacker has never seen the victim's architecture, gradients, parameters, or outputs.

Editorial extensions

If this is right

  • If CAD is correct, a camera-only VLM driving system can be degraded by an attacker who has no model access at all: a printed patch on a sign or obstacle reduced the paper's real-vehicle route-completion success rate from 72.22% to 11.11%.
  • Because the attack transfers to Dolphins, DriveLM, LMDrive, InstructBlip, LLaVA, MiniGPTv4, and GPT-4o, robustness evaluation of AD VLMs should include surrogate-based transfer attacks, not only white-box and query-based ones.
  • The defense experiments show that image denoising gives the strongest partial mitigation of the tested options, while some input transformations can make performance worse, so practical deployers should prefer denoising and output filtering over generic image preprocessing.
  • The released CADA dataset, with 18,808 adversarial visual-question-answer pairs at four severity levels, provides a common benchmark for measuring how AD VLMs degrade under increasing perturbation budgets and patch sizes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the attack optimizes in CLIP space, it should transfer to any future driving VLM that keeps a CLIP-like contrastive encoder; by the same token, replacing the visual encoder with a non-CLIP-aligned architecture may neutralize this attack class without adversarial training.
  • The risky-scene induction loss is effectively a probe of whether a driving VLM's safety judgment is grounded in scene semantics or in shallow descriptor statistics; it could be reused to distinguish models that truly reason about safety from models that match text templates.
  • The same recipe—generate deceptive text, align images to it in embedding space, and flip a safety descriptor—could be adapted to other safety-critical black-box VLM deployments such as remote inspection, assistive navigation, or medical image interpretation, where query access is also unavailable and a single misreasoning can be costly.
  • The textual-enhancement defense the paper tests suggests a cheap and immediately deployable hardening: adding safety-constraint prompts to the instruction text. Since adversarial training is not yet covered, fine-tuning on the released CADA examples is a natural next test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This manuscript proposes Cascading Adversarial Disruption (CAD), a black-box transfer-based attack on vision-language models for autonomous driving. The attack optimizes an ℓ∞-bounded perturbation in the embedding space of a CLIP surrogate to (i) align adversarial images with deceptive texts describing a broken perception-prediction-planning chain (Decision Chain Disruption, Eq. 7), (ii) invert the image-text matching probability between safe and unsafe scene descriptors (Risky Scene Induction, Eq. 10), and (iii) maximize semantic divergence from the clean image (Eq. 11). The three objectives are combined with weights α, β, and γ (Eq. 12). The paper reports open-loop results on Dolphins, DriveLM, and four general VLMs, closed-loop results on LMDrive in CARLA, and real-world road-following tests on JetBot and LIMO vehicles. It also introduces the CADA dataset of 18,808 adversarial visual-question-answer pairs.

Significance. If the claims hold, CAD would be the first black-box adversarial attack specifically designed for AD VLMs, and the real-world route-completion drop (61.11%) would demonstrate a serious physical-world vulnerability. The paper is clearly written and the attack design is coherent: all optimization objectives are defined against external surrogate components, so the method does not require gradients or queries to the victim. The authors also provide ablations, countermeasure evaluations, and a dataset, which are useful contributions. However, the evidence does not yet support the full breadth of the claims: the transferability argument relies on experiments that never test a genuinely non-CLIP victim encoder, and the headline open-loop results are obtained with hyperparameters tuned on the same Dolphins benchmark used for evaluation. These points require additional experiments rather than mere rewording.

major comments (4)
  1. [§3.5, §4.1–4.3, Table 5] The paper's black-box claim is explicitly founded on transferability ('our attacks mainly rely on transferability', §3.5), and the loss functions in Eqs. (7), (10), and (11) are all computed in CLIP's embedding space. Yet the only surrogate-model ablation (Table 5) holds the victim fixed at Dolphins, whose OpenFlamingo backbone uses a CLIP visual encoder. No experiment attacks a target whose visual encoder is outside the CLIP family, so the claim that CAD transfers to 'arbitrary victim VLM' is unsubstantiated. I recommend adding one or more targets with non-CLIP encoders (e.g., a BLIP-2-based InstructBLIP variant or a pure BLIP encoder) and/or a surrogate that is not CLIP, and reporting the resulting success rates; without this, the central black-box claim is not established beyond same-family transfer.
  2. [§5.1, §5.4, Tables 1–2] The loss weights α=0.75, β=0.05, γ=0.75 are selected via the ablation study in Sec. 5.4, which uses the Dolphins model on the Dolphins Benchmark. The same benchmark then appears in the headline open-loop results (Table 1a and Table 2). This constitutes selection on the evaluation set and can inflate the reported attack effectiveness. The authors should either fix the weights a priori, or choose them on a separate validation set (e.g., DriveLM) and then evaluate on Dolphins and the general VLMs, or at minimum show that the results are stable across a range of weight values.
  3. [§5.3, Table 4] The real-world experiments are based on 36 total runs, i.e., three repetitions per route per vehicle, and Table 4 reports only success counts without confidence intervals or statistical tests. The claim that 'route completion rate drops by 61.11%' is computed from 13/18 successes to 2/18; a 95% exact binomial confidence interval for the attack condition spans roughly 1.4% to 35%, which weakens the practical-applicability claim. Please provide more repetitions, per-route statistics, and ideally confidence intervals or a significance test. Also specify whether the adversarial patches were optimized for each carrier image and whether lighting or camera viewpoint vary between runs.
  4. [Abstract, §5.2] The abstract and introduction state that CAD 'significantly outperforming existing methods (+13.43% on average)', but the calculation behind this number is not defined in the paper. From Tables 1 and 2, the average relative improvement over the best baseline per model is about 7.6%, while the average relative drop against the mean of all baselines is about 14.4%; neither yields 13.43%. Please specify the baseline set and the averaging procedure used to obtain the headline 13.43% figure.
minor comments (5)
  1. [Figure 7] The y-axis labels and legend in the ablation figure are corrupted with unicode escape sequences (e.g., '/uni00000013/...'), making the figure unreadable; please regenerate the figure with proper fonts.
  2. [Table 3 and Figure 3] The caption and surrounding text contain typos: 'LMDirve' should be 'LMDrive', and the duplicate line '(b)LMDirve Model after attack... 4 79' appears to be a LaTeX artifact; please clean up.
  3. [Section 6] The heading 'Image Denoising (I.T)' should be 'Image Denoising (I.D)' to match the category label in Table 6.
  4. [References] The references list contains duplicates (e.g., [17] and [18] are the same paper) and incomplete entries (e.g., [49] is cited as 'ChatGPT, 2023' in the bibliography but elsewhere as 'GPT-4o'); please standardize citations.
  5. [Section 5.2] The phrase 'LLaV A' contains an erroneous space; it should be 'LLaVA'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the attack is an external surrogate-space optimization evaluated on independent victim models.

full rationale

The paper's derivation chain is self-contained. The perturbation objective in Eqs. (7), (10), and (11) is defined entirely in the embedding space of a fixed public surrogate (CLIP) and in text generated by ChatGPT; it never uses the victim model's gradients, logits, or outputs, so the attack results are not equivalent to the optimization objective by construction. Transfer to victim VLMs is measured on separate benchmarks (Dolphins, DriveLM, LangAuto/CARLA, and general VLMs), and success is reported as degradation of the victims' own scores, not of the surrogate's matching loss. The only potentially self-referential elements are (i) the choice of hyperparameters alpha, beta, and gamma from ablations on Dolphins and their reuse in the Dolphins column, and (ii) GPT-4o serving both to generate deceptive texts and to score some responses. Neither makes the central claim reduce to its inputs: the hyperparameters are fixed and transfer to other target models and benchmarks where they were not selected, and the generated text is an input to the attack while the score reflects victim-model behavior. The cited prior AD VLM attack (ADvLM [74]) is by overlapping authors but is used only as a comparison baseline and related work, not as load-bearing external evidence, and no uniqueness theorem or pre-adopted ansatz is imported to force the design. The transferability breadth is thinner than the headline claim (Table 5 only varies the surrogate against the CLIP-based Dolphins target), but that bears on correctness risk, not circularity. The stated limitation about validating against commercial AD systems also concerns breadth, not circularity. Overall, no prediction or first-principles result collapses into its own input by definition.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim depends on a small set of domain assumptions about CLIP transferability and the sufficiency of CLIP and ChatGPT as stand-ins for driving reasoning, plus three loss weights chosen by ablation. No new physical or theoretical entities are introduced; the CADA dataset is a resource, not an invented entity.

free parameters (4)
  • loss weights alpha, beta, gamma = 0.75, 0.05, 0.75
    Tuned via controlled-variable ablation on the Dolphins benchmark in Sec. 5.4, the same benchmark used for the headline results in Table 1. The values are not derived from first principles and may overfit the evaluation target.
  • perturbation budget epsilon = 0.1 (l-infinity)
    Hand-selected attack budget with ablation in Sec. 5.4; attack strength depends strongly on this choice.
  • number of iterations N = 160
    Hand-selected number of PGD-style updates; Fig. 8b shows a non-monotonic effect of this parameter.
  • real-world patch size = 12% of original image size
    Chosen for the physical experiments in Sec. 5.3; not varied or justified independently.
assumptions (4)
  • domain assumption CLIP is a valid surrogate for the visual-semantic space of target AD VLMs.
    Sec. 4.1 and 4.2 use CLIP features to optimize adversarial perturbations; the paper ablates the choice of alignment model but does not test whether CLIP surrogate failures limit attack transfer.
  • domain assumption Adversarial examples transfer across models without queries or gradients.
    Sec. 3.5 states 'our attacks mainly rely on transferability'; all experimental results depend on this premise.
  • domain assumption ChatGPT generates causally coherent deceptive reasoning chains representative of driving errors.
    Eq. (5) uses ChatGPT with a Reversal Reasoning Template to construct deceptive texts; the paper does not validate that the generated chains match actual perception-prediction-plan failures in target models.
  • domain assumption Image-text matching with safe and unsafe descriptors captures safety-relevant context for autonomous driving.
    Sec. 4.2 defines scene safety via two descriptor classes and uses CLIP matching scores as the optimization target, without independent evidence that this matches AD safety semantics.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Black-Box Adversarial Attack on Vision Language Models for Autonomous Driving." pith.science (2026). https://pith.science/paper/KHNQY7PC

@misc{pith2026250113563,
  author       = {Pith},
  title        = {Pith review of: Black-Box Adversarial Attack on Vision Language Models for Autonomous Driving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KHNQY7PC}},
  note         = {Machine review of arXiv:2501.13563}
}
read the original abstract

Vision-language models (VLMs) have significantly advanced autonomous driving (AD) by enhancing reasoning capabilities; however, these models remain highly susceptible to adversarial attacks. While existing research has explored white-box attacks to some extent, the more practical and challenging black-box scenarios remain largely underexplored due to their inherent difficulty. In this paper, we take the first step toward designing black-box adversarial attacks specifically targeting VLMs in AD. We identify two key challenges for achieving effective black-box attacks in this context: the effectiveness across driving reasoning chains in AD systems and the dynamic nature of driving scenarios. To address this, we propose Cascading Adversarial Disruption (CAD). It first introduces Decision Chain Disruption, which targets low-level reasoning breakdown by generating and injecting deceptive semantics, ensuring the perturbations remain effective across the entire decision-making chain. Building on this, we present Risky Scene Induction, which addresses dynamic adaptation by leveraging a surrogate VLM to understand and construct high-level risky scenarios that are likely to result in critical errors in the current driving contexts. Extensive experiments conducted on multiple AD VLMs and benchmarks demonstrate that CAD achieves state-of-the-art attack effectiveness, significantly outperforming existing methods (+13.43% on average). Moreover, we validate its practical applicability through real-world attacks on AD vehicles powered by VLMs, where the route completion rate drops by 61.11% and the vehicle crashes directly into the obstacle vehicle with adversarial patches. Finally, we release CADA dataset, comprising 18,808 adversarial visual-question-answer pairs, to facilitate further evaluation and research in this critical domain. Our codes and dataset will be available after paper's acceptance.

Figures

Figures reproduced from arXiv: 2501.13563 by the authors.

Figure 1
Figure 1. Illustration of our CAD black-box attack. The added visual perturbations cause the model to misinterpret a red traffic light as green. This misjudgment leads to incorrect driving actions, ultimately resulting in a collision. tasks such as image captioning, question answering, and multimodal reasoning, leading to their rapid development and widespread applications across various domains, includ￾ing autonomous driving… view at source ↗
Figure 2
Figure 2. Attack Framework. Our approach introduces Decision Chain Disruption, which targets low-level reasoning breakdown [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 4
Figure 4. Overview of the VLM-driven pipeline. The system [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figures from the paper (5 more)
Figure 3
Figure 3. Figure 3: Experimental results of LMDrive in the closed-loop [PITH_FULL_IMAGE:figures/full_fig_p008_3.png]
Figure 6
Figure 6. Figure 6: Consecutive frames showing Jetbot and LIMO [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Ablation studies on decision chain disruption (re [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: Ablation results on perturbation budgets and num [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Examples of our CADA Dataset, which correspond to scene-level and object-level perturbations, respectively. or where data collection is prohibitively expensive. In light of this, we propose a safety evaluation dataset specifically for AD VLMs, aimed at fostering the de…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. High-Entropy Tokens as Multimodal Failure Points in Vision-Language Models

    cs.CV 2025-12 unverdicted novelty 6.0 of 10

    High-entropy tokens act as concentrated multimodal failure points in VLMs, enabling sparse Entropy-Guided Attacks that achieve 93-95% success and 30-38% harmful rates with cross-model transfer.

  2. Benchmarking the Robustness of Autonomous Driving to Environmental Illusions: A Lane Perception Perspective

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Environmental illusions cause 5-7% accuracy drops in lane detection models and can trigger collisions in closed-loop simulation, with a proposed defense (MIDA) recovering ~4% robustness.

Reference graph

Works this paper leans on

78 extracted references · 45 canonical work pages · cited by 2 Pith papers

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    Flamingo: a visual language model for few-shot learn- ing

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, An- toine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learn- ing. NeurIPS, 35:23716–23736, 2022

    work page 2022

  3. [3]

    Vlmo: Unified vision- language pre-training with mixture-of-modality-experts

    Hangbo Bao, Wenhui Wang, Li Dong, Qiang Liu, Owais Khan Mohammed, Kriti Aggarwal, Subhojit Som, Songhao Piao, and Furu Wei. Vlmo: Unified vision- language pre-training with mixture-of-modality-experts. NeurIPS, 35:32897–32912, 2022

    work page 2022

  4. [4]

    Towards evaluating the robustness of neural networks

    Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In S&P, pages 39–57. Ieee, 2017. 13

    work page 2017

  5. [5]

    Mp3: A unified model to map, perceive, predict and plan

    Sergio Casas, Abbas Sadat, and Raquel Urtasun. Mp3: A unified model to map, perceive, predict and plan. In CVPR, pages 14403–14412, 2021

    work page 2021

  6. [6]

    Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models

    Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM workshop on artificial intelligence and security , pages 15–26, 2017

    work page 2017

  7. [7]

    Adversarial attack on attackers: Post-process to mitigate black-box score- based query attacks

    Sizhe Chen, Zhehao Huang, Qinghua Tao, Yingwen Wu, Cihang Xie, and Xiaolin Huang. Adversarial attack on attackers: Post-process to mitigate black-box score- based query attacks. NeurIPS, 35:14929–14943, 2022

    work page 2022

  8. [8]

    Detection as regression: Certified object detection with median smoothing

    Ping-yeh Chiang, Michael Curry, Ahmed Abdelkader, Aounon Kumar, John Dickerson, and Tom Goldstein. Detection as regression: Certified object detection with median smoothing. NeurIPS, 33:1275–1286, 2020

    work page 2020

Show all 78 references
  1. [9]

    Instructblip: Towards general-purpose vision-language models with instruc- tion tuning

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards general-purpose vision-language models with instruc- tion tuning. arXiv preprint arXiv:2305.06500, 2023

    2023 arXiv

  2. [10]

    Deep learning-based autonomous driving systems: A survey of attacks and defenses

    Yao Deng, Tiehua Zhang, Guannan Lou, Xi Zheng, Jiong Jin, and Qing-Long Han. Deep learning-based autonomous driving systems: A survey of attacks and defenses. TII, 17(12):7897–7912, 2021

    2021

  3. [11]

    Driverless vehicle security: Challenges and future research opportunities

    Gonzalo De La Torre, Paul Rad, and Kim-Kwang Ray- mond Choo. Driverless vehicle security: Challenges and future research opportunities. FGCS, 108:1092–1111, 2020

    2020

  4. [12]

    Boosting adversar- ial attacks with momentum

    Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversar- ial attacks with momentum. In CVPR, pages 9185–9193, 2018

    2018

  5. [13]

    Carla: An open urban driving simulator

    Alexey Dosovitskiy, German Ros, Felipe Codevilla, An- tonio Lopez, and Vladlen Koltun. Carla: An open urban driving simulator. In CoRL, pages 1–16. PMLR, 2017

    2017

  6. [14]

    Boosting transferability in vision-language attacks via diversification along the intersection region of adversarial trajectory

    Sensen Gao, Xiaojun Jia, Xuhong Ren, Ivor Tsang, and Qing Guo. Boosting transferability in vision-language attacks via diversification along the intersection region of adversarial trajectory. In ECCV, pages 442–460. Springer, 2025

    2025

  7. [15]

    Explaining and harnessing adversarial exam- ples

    Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial exam- ples. arXiv preprint arXiv:1412.6572, 2014

    2014 arXiv

  8. [16]

    Countering adversarial im- ages using input transformations

    Chuan Guo, Mayank Rana, Moustapha Cisse, and Lau- rens Van Der Maaten. Countering adversarial im- ages using input transformations. arXiv preprint arXiv:1711.00117, 2017

    2017 arXiv

  9. [17]

    A comprehensive evaluation frame- work for deep model robustness

    Jun Guo, Wei Bao, Jiakai Wang, Yuqing Ma, Xinghai Gao, Gang Xiao, Aishan Liu, Jian Dong, Xianglong Liu, and Wenjun Wu. A comprehensive evaluation frame- work for deep model robustness. PR, 137:109308, 2023

    2023

  10. [18]

    A comprehensive evaluation frame- work for deep model robustness

    Jun Guo, Wei Bao, Jiakai Wang, Yuqing Ma, Xinghai Gao, Gang Xiao, Aishan Liu, Jian Dong, Xianglong Liu, and Wenjun Wu. A comprehensive evaluation frame- work for deep model robustness. PR, 2023

    2023

  11. [19]

    St-p3: End-to-end vision- based autonomous driving via spatial-temporal feature learning

    Shengchao Hu, Li Chen, Penghao Wu, Hongyang Li, Junchi Yan, and Dacheng Tao. St-p3: End-to-end vision- based autonomous driving via spatial-temporal feature learning. In ECCV, pages 533–549. Springer, 2022

    2022

  12. [20]

    Planning-oriented autonomous driving

    Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, et al. Planning-oriented autonomous driving. In CVPR, pages 17853–17862, 2023

    2023

  13. [21]

    Senna: Bridging large vision-language models and end-to-end autonomous driving

    Bo Jiang, Shaoyu Chen, Bencheng Liao, Xingyu Zhang, Wei Yin, Qian Zhang, Chang Huang, Wenyu Liu, and Xinggang Wang. Senna: Bridging large vision-language models and end-to-end autonomous driving. arXiv preprint arXiv:2410.22313, 2024

    arXiv

  14. [22]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In ICML, pages 19730–19742. PMLR, 2023

    2023

  15. [23]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In ICML, pages 12888–12900. PMLR, 2022

    2022

  16. [24]

    Align before fuse: Vision and language representation learning with momentum distillation.NeurIPS, 34:9694– 9705, 2021

    Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. Align before fuse: Vision and language representation learning with momentum distillation.NeurIPS, 34:9694– 9705, 2021

    2021

  17. [25]

    Poisoned forgery face: Towards backdoor attacks on face forgery detec- tion

    Jiawei Liang, Siyuan Liang, Aishan Liu, Xiaojun Jia, Junhao Kuang, and Xiaochun Cao. Poisoned forgery face: Towards backdoor attacks on face forgery detec- tion. arXiv preprint arXiv:2402.11473, 2024

    2024 arXiv

  18. [26]

    A large- scale multiple-objective method for black-box attack against object detection

    Siyuan Liang, Longkang Li, Yanbo Fan, Xiaojun Jia, Jingzhi Li, Baoyuan Wu, and Xiaochun Cao. A large- scale multiple-objective method for black-box attack against object detection. In European Conference on Computer Vision, 2022. 14

    2022

  19. [27]

    Gener- ate more imperceptible adversarial examples for object detection

    Siyuan Liang, Xingxing Wei, and Xiaochun Cao. Gener- ate more imperceptible adversarial examples for object detection. In ICML 2021 Workshop on Adversarial Machine Learning, 2021

    2021

  20. [28]

    Efficient adversarial attacks for visual object track- ing

    Siyuan Liang, Xingxing Wei, Siyuan Yao, and Xiaochun Cao. Efficient adversarial attacks for visual object track- ing. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceed- ings, Part XXVI 16, 2020

    2020

  21. [29]

    Parallel rectangle flip attack: A query-based black-box attack against object detection

    Siyuan Liang, Baoyuan Wu, Yanbo Fan, Xingxing Wei, and Xiaochun Cao. Parallel rectangle flip attack: A query-based black-box attack against object detection. arXiv preprint arXiv:2201.08970, 2022

    2022 arXiv

  22. [30]

    Badclip: Dual- embedding guided backdoor attack on multimodal con- trastive learning

    Siyuan Liang, Mingli Zhu, Aishan Liu, Baoyuan Wu, Xiaochun Cao, and Ee-Chien Chang. Badclip: Dual- embedding guided backdoor attack on multimodal con- trastive learning. arXiv preprint arXiv:2311.12075 , 2023

    2023 arXiv

  23. [31]

    {X-Adv}: Physical adversarial object attacks against x-ray prohibited item detection

    Aishan Liu, Jun Guo, Jiakai Wang, Siyuan Liang, Ren- shuai Tao, Wenbo Zhou, Cong Liu, Xianglong Liu, and Dacheng Tao. {X-Adv}: Physical adversarial object attacks against x-ray prohibited item detection. In 32nd USENIX Security Symposium (USENIX Security 23) , 2023

    2023

  24. [32]

    Spatiotemporal attacks for embodied agents

    Aishan Liu, Tairan Huang, Xianglong Liu, Yitao Xu, Yuqing Ma, Xinyun Chen, Stephen J Maybank, and Dacheng Tao. Spatiotemporal attacks for embodied agents. In ECCV, 2020

    2020

  25. [33]

    Perceptual- sensitive gan for generating adversarial patches

    Aishan Liu, Xianglong Liu, Jiaxin Fan, Yuqing Ma, An- lan Zhang, Huiyuan Xie, and Dacheng Tao. Perceptual- sensitive gan for generating adversarial patches. In AAAI, 2019

    2019

  26. [34]

    Training robust deep neural networks via adversarial noise propagation.IEEE TIP, 2021

    Aishan Liu, Xianglong Liu, Hang Yu, Chongzhi Zhang, Qiang Liu, and Dacheng Tao. Training robust deep neural networks via adversarial noise propagation.IEEE TIP, 2021

    2021

  27. [35]

    Towards defending multiple lp-norm bounded adversarial pertur- bations via gated batch normalization

    Aishan Liu, Shiyu Tang, Xinyun Chen, Lei Huang, Hao- tong Qin, Xianglong Liu, and Dacheng Tao. Towards defending multiple lp-norm bounded adversarial pertur- bations via gated batch normalization. IJCV, 2023

    2023

  28. [36]

    Exploring the relationship between architecture and adversarially robust generalization

    Aishan Liu, Shiyu Tang, Siyuan Liang, Ruihao Gong, Boxi Wu, Xianglong Liu, and Dacheng Tao. Exploring the relationship between architecture and adversarially robust generalization. In CVPR, 2023

    2023

  29. [37]

    Bias-based universal adversarial patch attack for automatic check-out

    Aishan Liu, Jiakai Wang, Xianglong Liu, Bowen Cao, Chongzhi Zhang, and Hang Yu. Bias-based universal adversarial patch attack for automatic check-out. In ECCV, 2020

    2020

  30. [38]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. NeurIPS, 36, 2024

    2024

  31. [39]

    Improving adversarial transferability by stable diffusion

    Jiayang Liu, Siyu Zhu, Siyuan Liang, Jie Zhang, Han Fang, Weiming Zhang, and Ee-Chien Chang. Improving adversarial transferability by stable diffusion. arXiv preprint arXiv:2311.11017, 2023

    2023 arXiv

  32. [40]

    Harnessing perceptual adversarial patches for crowd counting

    Shunchang Liu, Jiakai Wang, Aishan Liu, Yingwei Li, Yijie Gao, Xianglong Liu, and Dacheng Tao. Harnessing perceptual adversarial patches for crowd counting. In ACM CCS, 2022

    2022

  33. [41]

    Set-level guidance attack: Boosting adversarial transferability of vision- language pre-training models

    Dong Lu, Zhiqiang Wang, Teng Wang, Weili Guan, Hongchang Gao, and Feng Zheng. Set-level guidance attack: Boosting adversarial transferability of vision- language pre-training models. In ICCV, pages 102–111, 2023

    2023

  34. [42]

    Dolphins: Multimodal language model for driving

    Yingzi Ma, Yulong Cao, Jiachen Sun, Marco Pavone, and Chaowei Xiao. Dolphins: Multimodal language model for driving. arXiv preprint arXiv:2312.00438, 2023

    2023 arXiv

  35. [43]

    Towards deep learning models resistant to adversarial attacks

    Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083, 2017

    2017 arXiv

  36. [44]

    Gpt-driver: Learning to drive with gpt

    Jiageng Mao, Yuxi Qian, Junjie Ye, Hang Zhao, and Yue Wang. Gpt-driver: Learning to drive with gpt. arXiv preprint arXiv:2310.01415, 2023

    2023 arXiv

  37. [45]

    Lingoqa: Video question an- swering for autonomous driving

    Ana-Maria Marcu, Long Chen, Jan Hünermann, Al- ice Karnsund, Benoit Hanotte, Prajwal Chidananda, Saurabh Nair, Vijay Badrinarayanan, Alex Kendall, Jamie Shotton, et al. Lingoqa: Video question an- swering for autonomous driving. arXiv preprint arXiv:2312.14115, 2023

    2023 arXiv

  38. [46]

    A self-supervised approach for adversarial robustness

    Muzammal Naseer, Salman Khan, Munawar Hayat, Fa- had Shahbaz Khan, and Fatih Porikli. A self-supervised approach for adversarial robustness. In CVPR, pages 262–271, 2020

    2020

  39. [47]

    Reason2drive: Towards interpretable and chain-based reasoning for autonomous driving

    Ming Nie, Renyuan Peng, Chunwei Wang, Xinyue Cai, Jianhua Han, Hang Xu, and Li Zhang. Reason2drive: Towards interpretable and chain-based reasoning for autonomous driving. arXiv preprint arXiv:2312.03661, 2023

    2023 arXiv

  40. [48]

    Jetbot, 2021

    Nvidia. Jetbot, 2021. https://github.com/NVIDIA-AI- IOT/jetbot

    2021

  41. [49]

    Chatgpt, 2023

    OpenAI. Chatgpt, 2023. https://chat.openai.com

    2023

  42. [50]

    Adaptive adversarial videos on 15 roadside billboards: Dynamically modifying trajectories of autonomous vehicles

    Naman Patel, Prashanth Krishnamurthy, Siddharth Garg, and Farshad Khorrami. Adaptive adversarial videos on 15 roadside billboards: Dynamically modifying trajectories of autonomous vehicles. In IROS, pages 5916–5921. IEEE, 2019

    2019

  43. [51]

    Learning transferable visual models from natural lan- guage supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sas- try, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural lan- guage supervision. In ICML, pages 8748–8763. PMLR, 2021

    2021

  44. [52]

    Limo, 2021

    Agilex Robotics. Limo, 2021. https://github.com/agilexrobotics

    2021

  45. [53]

    Dirty road can attack: Security of deep learning based automated lane center- ing under {Physical-World} attack

    Takami Sato, Junjie Shen, Ningfei Wang, Yunhan Jia, Xue Lin, and Qi Alfred Chen. Dirty road can attack: Security of deep learning based automated lane center- ing under {Physical-World} attack. In 30th USENIX security symposium (USENIX Security 21), pages 3309– 3326, 2021

    2021

  46. [54]

    Lm- drive: Closed-loop end-to-end driving with large lan- guage models

    Hao Shao, Yuxuan Hu, Letian Wang, Guanglu Song, Steven L Waslander, Yu Liu, and Hongsheng Li. Lm- drive: Closed-loop end-to-end driving with large lan- guage models. In CVPR, 2024

    2024

  47. [55]

    Drivelm: Driving with graph visual question answering

    Chonghao Sima, Katrin Renz, Kashyap Chitta, Li Chen, Hanxue Zhang, Chengen Xie, Ping Luo, Andreas Geiger, and Hongyang Li. Drivelm: Driving with graph visual question answering. arXiv preprint arXiv:2312.14150, 2023

    2023 arXiv

  48. [56]

    Intriguing properties of neural networks

    C Szegedy. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013

    2013 arXiv

  49. [57]

    Robustart: Benchmarking ro- bustness on architecture design and training techniques

    Shiyu Tang, Ruihao Gong, Yan Wang, Aishan Liu, Jiakai Wang, Xinyun Chen, Fengwei Yu, Xianglong Liu, Dawn Song, Alan Yuille, et al. Robustart: Benchmarking ro- bustness on architecture design and training techniques. ArXiv, 2021

    2021

  50. [58]

    Carla autonomous driving leaderboard.,

    CARLA Team. Carla autonomous driving leaderboard.,

  51. [59]

    Drivevlm: The convergence of autonomous driving and large vision-language models

    Xiaoyu Tian, Junru Gu, Bailin Li, Yicheng Liu, Yang Wang, Zhiyong Zhao, Kun Zhan, Peng Jia, Xianpeng Lang, and Hang Zhao. Drivevlm: The convergence of autonomous driving and large vision-language models. arXiv preprint arXiv:2402.12289, 2024

    2024 arXiv

  52. [60]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Bap- tiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  53. [61]

    Transferable multimodal attack on vision-language pre- training models

    Haodi Wang, Kai Dong, Zhilei Zhu, Haotong Qin, Ais- han Liu, Xiaolin Fang, Jiakai Wang, and Xianglong Liu. Transferable multimodal attack on vision-language pre- training models. In S&P, 2024

    2024

  54. [62]

    Dual attention suppres- sion attack: Generate adversarial camouflage in physical world

    Jiakai Wang, Aishan Liu, Zixin Yin, Shunchang Liu, Shiyu Tang, and Xianglong Liu. Dual attention suppres- sion attack: Generate adversarial camouflage in physical world. In CVPR, 2021

    2021

  55. [63]

    Diversifying the high-level features for better adversarial transferability

    Zhiyuan Wang, Zeliang Zhang, Siyuan Liang, and Xi- aosen Wang. Diversifying the high-level features for better adversarial transferability. arXiv preprint arXiv:2304.10136, 2023

    2023 arXiv

  56. [64]

    Transferable adversarial attacks for image and video object detection

    Xingxing Wei, Siyuan Liang, Ning Chen, and Xi- aochun Cao. Transferable adversarial attacks for image and video object detection. arXiv preprint arXiv:1811.12641, 2018

    2018 arXiv

  57. [65]

    Feature squeezing: Detecting adversarial examples in deep neural networks

    W Xu. Feature squeezing: Detecting adversarial examples in deep neural networks. arXiv preprint arXiv:1704.01155, 2017

    2017 arXiv

  58. [66]

    Drivegpt4: Interpretable end-to-end autonomous driving via large language model

    Zhenhua Xu, Yujia Zhang, Enze Xie, Zhen Zhao, Yong Guo, Kwan-Yee K Wong, Zhenguo Li, and Hengshuang Zhao. Drivegpt4: Interpretable end-to-end autonomous driving via large language model. RAL, 2024

    2024

  59. [67]

    Vlattack: Multimodal adversarial attacks on vision-language tasks via pre-trained models

    Ziyi Yin, Muchao Ye, Tianrong Zhang, Tianyu Du, Jin- guo Zhu, Han Liu, Jinghui Chen, Ting Wang, and Feng- long Ma. Vlattack: Multimodal adversarial attacks on vision-language tasks via pre-trained models. NeurIPS, 36, 2024

    2024

  60. [68]

    Coca: Con- trastive captioners are image-text foundation models

    Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Seyedhosseini, and Yonghui Wu. Coca: Con- trastive captioners are image-text foundation models. arXiv preprint arXiv:2205.01917, 2022

    2022 arXiv

  61. [69]

    Interpreting and improving adversarial robustness of deep neural net- works with neuron sensitivity

    Chongzhi Zhang, Aishan Liu, Xianglong Liu, Yitao Xu, Hang Yu, Yuqing Ma, and Tianlin Li. Interpreting and improving adversarial robustness of deep neural net- works with neuron sensitivity. IEEE TIP, 2021

    2021

  62. [70]

    Anyattack: Towards large-scale self-supervised generation of tar- geted adversarial examples for vision-language models

    Jiaming Zhang, Junhong Ye, Xingjun Ma, Yige Li, Yun- fan Yang, Jitao Sang, and Dit-Yan Yeung. Anyattack: Towards large-scale self-supervised generation of tar- geted adversarial examples for vision-language models. arXiv preprint arXiv:2410.05346, 2024

    arXiv

  63. [71]

    Towards adver- sarial attack on vision-language pre-training models

    Jiaming Zhang, Qi Yi, and Jitao Sang. Towards adver- sarial attack on vision-language pre-training models. In ACM MM, 2022

    2022

  64. [72]

    Module-wise adaptive adversarial training for end-to-end autonomous driving

    Tianyuan Zhang, Lu Wang, Jiaqi Kang, Xinwei Zhang, Siyuan Liang, Yuwei Chen, Aishan Liu, and Xiang- long Liu. Module-wise adaptive adversarial training for end-to-end autonomous driving. arXiv preprint arXiv:2409.07321, 2024. 16

    2024 arXiv

  65. [73]

    Lanevil: Benchmarking the robustness of lane detection to environmental illusions

    Tianyuan Zhang, Lu Wang, Hainan Li, Yisong Xiao, Siyuan Liang, Aishan Liu, Xianglong Liu, and Dacheng Tao. Lanevil: Benchmarking the robustness of lane detection to environmental illusions. InACM MM, pages 5403–5412, 2024

    2024

  66. [74]

    Visual adversar- ial attack on vision-language models for autonomous driving

    Tianyuan Zhang, Lu Wang, Xinwei Zhang, Yitong Zhang, Boyi Jia, Siyuan Liang, Shengshan Hu, Qiang Fu, Aishan Liu, and Xianglong Liu. Visual adversar- ial attack on vision-language models for autonomous driving. arXiv preprint arXiv:2411.18275, 2024

    2024 arXiv

  67. [75]

    On evaluating adversarial robustness of large vision- language models

    Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Man Cheung, and Min Lin. On evaluating adversarial robustness of large vision- language models. NeurIPS, 2024

    2024

  68. [76]

    Advclip: Downstream- agnostic adversarial examples in multimodal contrastive learning

    Ziqi Zhou, Shengshan Hu, Minghui Li, Hangtao Zhang, Yechao Zhang, and Hai Jin. Advclip: Downstream- agnostic adversarial examples in multimodal contrastive learning. In ACM MM, 2023

    2023

  69. [77]

    Minigpt-4: Enhancing vision- language understanding with advanced large language models

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing vision- language understanding with advanced large language models. arXiv preprint arXiv:2304.10592, 2023. 17

    2023 arXiv

  70. [2021]

    https://leaderboard.carla.org/

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.